Fast and reliable bundle adjustment is essential in many applications such as mobile vision, augmented reality, and robotics. Two recent ideas to reduce the associated computational cost are structure-less SFM (structure from motion) and incremental smoothing. The former formulates the cost function in terms of multi-view constraints instead of re-projection error, thereby eliminating the 3D structure from the optimization. The latter was developed in the SLAM (simultaneous localization and mapping) community and allows one to perform efficient incremental optimization, adaptively identifying the variables that need to be recomputed at each step.
In this paper we combine these two key ideas into a computationally efficient bundle adjustment method, and additionally introduce the use of three-view constraints to remedy commonly encountered degenerate camera motions. We formulate the problem in terms of a factor graph, and incrementally update a directed junction tree which keeps track of the current best solution. Typically, only a small fraction of the camera poses are recalculated in each optimization step, leading to a significant computational gain. If desired, all or some of the observed 3D points can be reconstructed based on the optimized camera poses. To deal with degenerate motions, we use both two and three-view constraints between camera poses, which allows us to maintain a consistent scale during straight-line trajectories. We validate our approach using synthetic and real-imagery datasets and compare it to standard bundle adjustment, in terms of performance, robustness and computational cost.