Motion and Structure from Image Sequences

1. Introduction.- 1.1 Formulation of the Problem.- 1.2 Overview of Recent Progress.- 1.3 Bibliographical Notes.- 2. Image Matching.- 2.1 Approaches to Establishing Correspondences.- 2.2 An Approach to Image Matching.- 2.2.1 Image Attributes.- 2.2.2 Relationships of the Attributes.- 2.2.3 Intra-Regional Smoothness and Occlusion.- 2.2.4 Multi-Resolution Multi-Grid Structure.- 2.2.5 Limitations.- 2.3 Algorithm.- 2.3.1 Locally Rigid Motion.- 2.3.2 3-D Motion and Image Plane Motion.- 2.3.3 Motion Insensitive Image Attributes.- 2.3.4 Edgeness and Cornerness.- 2.3.5 Smoothness.- 2.3.6 Minimizing Residuals.- 2.3.7 Recursive Blurring.- 2.3.8 Preprocessing and Normalization.- 2.3.9 Outline of the Matching Algorithm.- 2.4 Motion and Structure Computation.- 2.5 Refinements.- 2.5.1 Bottom-Up and Top-Down.- 2.5.2 Refinement Based on Rigidity.- 2.6 Examples.- 2.7 Summary.- 2.8 Bibliographical Notes.- 3. Two-View Analysis.- 3.1 Some Basic Issues.- 3.2 An Algorithm.- 3.2.1 Problem Statement.- 3.2.2 Algorithm.- 3.2.3 Justification of the Algorithm.- 3.3 Error Estimation.- 3.3.1 Algorithm and Perturbation.- 3.3.2 Perturbation of Eigenvalues and Eigenvectors.- 3.3.3 Error Estimation for the Algorithm.- 3.4 Error Analysis.- 3.4.1 Structure of the Scene.- 3.4.2 Motion Parameters.- 3.4.3 System Parameters.- 3.5 Performance.- 3.5.1 Performance.- 3.5.2 Error Estimation.- 3.5.3 Error versus Motion and System Parameters.- 3.5.4 Real Images.- 3.6 Summary.- 3.7 Bibliographical Notes.- Appendix 3.A Perturbation of Eigenspace.- Appendix 3.B Quaternions.- Appendix 3.C Alternative Fitting.- 4. Optimization.- 4.1 Motivations.- 4.2 Stability of Linear Algorithms.- 4.2.1 The Linear Algorithms.- 4.2.2 The Epipolar Constraint.- 4.2.3 Using the Constraint in Matrix E.- 4.2.4 A Type of Motion.- 4.2.5 Beyond the Epipolar Constraint.- 4.3 Maximum Likelihood Estimation.- 4.3.1 Gaussian Distribution.- 4.3.2 Uncertainty Polyhedron Distribution.- 4.4 A Two-Step Approach and Computation.- 4.5 Minimum Variance Estimation.- 4.6 Error Estimation and Error Bounds.- 4.6.1 Error Estimation.- 4.6.2 Error Bounds.- 4.7 Batch and Sequential Methods.- 4.7.1 Linear Systems.- 4.7.2 Nonlinear Systems.- 4.7.3 Recursive-Batch Solution.- 4.8 Numerical Examples.- 4.8.1 Whether a Good Initial Guess Is Necessary.- 4.8.2 Epipolar Improvement versus Optimization.- 4.8.3 Sequential versus Batch Solutions.- 4.8.4 Uncertainty Polyhedron Model.- 4.8.5 Error Estimation and Error Bound.- 4.8.6 Inherent Limitation of Optical Flow.- 4.8.7 Real World Images.- 4.9 Summary.- 4.10 Bibliographical Notes.- Appendix 4.A Decomposability.- Appendix 4.B Weights.- Appendix 4.C Triangulation with Noise.- Appendix 4.D Matrix Derivatives.- Appendix 4.E Cramér-Rao Bound.- 5. Planar Scenes.- 5.1 Planar Scene as a Degenerate Case.- 5.2 Motion from a Plane.- 5.2.1 Problem Statement.- 5.2.2 Algorithm Overview.- 5.2.3 Intermediate Parameter Matrix F.- 5.2.4 General Case.- 5.2.5 Two Solutions Are Both Valid.- 5.2.6 Special Cases.- 5.3 Inherent Uniqueness.- 5.3.1 Decomposability.- 5.3.2 Rank Condition Is Algorithm-Independent.- 5.3.3 The Fundamental Theorem.- 5.3.4 Plane-Perceivable Surfaces.- 5.3.5 Optimality.- 5.3.6 Three-View Problem.- 5.4 Examples.- 5.4.1 Simulations.- 5.4.2 Real Images.- 5.5 Conclusions.- 5.6 Bibliographical Notes.- Appendix 5.A Conditions on the Rank of A.- Appendix 5.B Two Solutions from F.- Appendix 5.C Plane-Perceivable Surfaces.- Appendix 5.D Condition on Consistent Normals.- Appendix 5.E Two-View Algorithm.- Appendix 5.F Error Estimation.- 6. From Line Correspondences.- 6.1 Lines as Features.- 6.2 Solution and Algorithm.- 6.2.1 Why Two Views Are Not Sufficient.- 6.2.2 From Three Views.- 6.2.3 Two Important Equations.- 6.2.4 A Geometrical View.- 6.2.5 Intermediate Parameters.- 6.2.6 Motion from Intermediate Parameters.- 6.2.7 Structure and Sign of Translation Vectors.- 6.2.8 In the Presence of Noise.- 6.2.9 Algorithm.- 6.3 Degeneracy.- 6.4 Optimization.- 6.4.1 Lines from Pixels.- 6.4.2 Optimal Solution.- 6.4.3 How Accurate It Can Be.- 6.5 Simulations.- 6.5.1 Setup.- 6.5.2 Linear Algorithm.- 6.5.3 Algorithm with Optimization.- 6.5.4 Comparison with the Bound.- 6.5.5 Comparison with the Point-Based Algorithm.- 6.6 Conclusions and Discussions.- 6.7 Bibliographical Notes.- Appendix 6.A Ranks.- Appendix 6.B Unique Consistent Assignment.- Appendix 6.C Degeneracy.- Appendix 6.D Distinct Locations Are Necessary.- Appendix 6.E Alternative Degeneracy Condition.- 7. Stereo.- 7.1 Stereo Camera Systems.- 7.2 Stereo Triangulation.- 7.3 Closed-Form Solution.- 7.3.1 A Matrix-Weighted Objective Function.- 7.3.2 Unweighted and Scalar-Weighted Versions.- 7.3.3 Matrix-Weighted Least-Squares Solution.- 7.3.4 Uniqueness.- 7.4 Iterative Optimal Solution.- 7.4.1 A Basic Objective Function.- 7.4.2 Optimization Using Space Decomposition.- 7.4.3 Estimating Errors.- 7.5 Outliers and Robust Estimators.- 7.5.1 Some Basic Concepts of Robust Statistics.- 7.5.2 A Robust Method for Motion Estimation.- 7.5.3 Using Closed-Form Solution.- 7.6 Examples.- 7.6.1 Simulations.- 7.6.2 Experiments with a Real Stereo Setup.- 7.7 Without Stereo Correspondences.- 7.7.1 Motivations.- 7.7.2 Motions Seen from Two Cameras.- 7.7.3 Determining Complete Motion and Structure.- 7.7.4 Computational Aspect of the Degeneracy.- 7.8 Long Image Sequences.- 7.8.1 Local and Global Coordinate Systems.- 7.8.2 A Recursive-Batch Approach.- 7.8.3 Recursive-Batch Updating.- 7.8.4 Local and Global Representations.- 7.8.5 Numerical Examples.- 7.9 Conclusions.- 7.10 Bibliographical Notes.- Appendix 7.A MWCC Theorem.- Appendix 7.B Least-Squares Matrix Fitting.- 8. Motion Modeling and Prediction.- 8.1 Coherence of Motion.- 8.2 The LCAM Model.- 8.2.1 Motion of a Rigid Body in 3-D.- 8.2.2 Motion of Rotation Center.- 8.2.3 Solutions of the Coefficient Equation.- 8.2.4 Continuous and Discrete Motions.- 8.3 Estimation and Prediction.- 8.4 Monocular Vision.- 8.5 Optimization.- 8.6 Experimental Examples.- 8.6.1 Computer Simulation.- 8.6.2 Experiment with Real Images.- 8.7 Summary.- 8.8 Bibliographical Notes.- Appendix 8.A Solution of Coefficient Equations.- Appendix 8.B Rotation without Precession.- Appendix 8.C Singularity of the Matrix.- References.