A Few Steps Towards 3D Active Vision

1. From 2D to 3D Active Vision.- 1.1 The Concept of Active Vision.- 1.1.1 Can Reactive Vision Be "Better" Than Passive Vision?.- 1.1.2 A Blink at the State of the Art in Active Vision.- 1.2 A Short Review of Existing Active Visual Systems.- 1.2.1 Active Visual Sensors.- 1.2.2 Control for Active Vision.- 1.2.3 Auto-Calibration of an Active Visual System.- 1.2.4 Perception of 3D Parameters Using Active Vision.- 1.3 Architecture of an Active Visual System.- 1.3.1 The Three Main Functions of an Active Visual System.- 1.3.2 Basic Requirements of a Visual System.- 1.4 2D Versus 3D Vision in an Active Visual System.- 1.4.1 Active Vision and 2D Visual Servoing.- 1.4.2 Introduction of 3D Vision in Visual Loops.- 1.4.3 Basic Modules for a 3D Active Visual System.- 1.5 Gaze Control in 3D Active Visual Systems.- 2. 3D Active Vision on a Robotic Head.- 2.1 A One-to-One 3D Gaze Controller.- 2.1.1 Technical Data on the Robotic Head.- 2.1.2 Computing the Inverse Kinematic for 3D Fixation.- 2.2 Active Observation of a 3D Visual Target.- 2.2.1 A ID Model of the Eye-Neck Tracking.- 2.2.2 A Linearized Adaptive ID Model of the Eye-Neck Tracking.- 2.2.3 Statistical Filtering of the Linearized Model.- 2.2.4 Controlling the Neck and Eye Positions.- 2.2.5 Automatic Tuning from the System Residual Error.- 2.2.6 Estimating VR.- 2.2.7 Estimating ?.- 2.2.8 Estimating ?.- 2.2.9 Simulation of the Combined Behavior.- 2.3 Detection of Visual Targets for 3D Visual Perception.- 2.4 Computing the 3D Parameters of a Target.- 2.4.1 A Unique Framework to Integrate Depth from Vergence, Motion and Zoom.- 2.4.2 Using Second-Order Focus Variations to Compute Depth.- 2.4.3 Multi-Model Concurrency in 3D Tracking.- 2.4.4 Considering Further 3D Information.- 2.5 Experimental Results.- 2.5.1 Head Intrinsic Calibration.- 2.5.2 Looking at a 3D Point.- 2.5.3 Where to Look Next Experiment.- 2.5.4 Reconstruction of a Coarse 3D Map.- 2.5.5 Tracking of a 3D Target: Simulation Experiments.- 2.5.6 Tracking of a 3D Target: Real Object Experiments.- 2.5.7 Conclusion.- 3. Auto-Calibration of a Robotic Head.- 3.1 Introduction.- 3.2 Reviewing the Problem of Visual Sensor Calibration.- 3.3 Equations for the Tracking of a Stationary Point.- 3.4 Recovering the Parameters of the Trajectory.- 3.4.1 An Initial Estimate of the Coefficients.- 3.4.2 Minimizing the Nonlinear Criterion.- 3.5 Computing Calibration Parameters.- 3.5.1 Equations for the Intrinsic Calibration Parameters.- 3.5.2 Extrinsic Parameters Computation.- 3.5.3 Calibration Algorithm.- 3.6 Experimental Results.- 3.6.1 How Stable are Calibration Parameters When Zooming?.- 3.6.2 Experiment 1: Parameter Estimation with Synthetic Data.- 3.6.3 Experiment 2: Trajectory Identification Using Real Data.- 3.6.4 Experiment 3: Parameters Estimation Using Real Data.- 3.7 Conclusion.- 3.8 Comparison with the Case of Known Translations.- 3.9 Application to the Case of a Binocular Head.- 3.10 Instantaneous Equations for Calibration.- 3.10.1 Reviewing the Definition of the Fundamental Matrix.- 3.10.2 Characterizing the Essential Matrix for Fixed Axis Rotations.- 3.10.3 Calibration Using the Fundamental Matrix.- 3.10.4 Discussion.- 4. Inertial Cues in an Active Visual System.- 4.1 Introduction.- 4.2 The Use of Inertial Forces in a Robotic System.- 4.2.1 Origins of Inertial Forces on a Robot.- 4.2.2 Distinction with Inertial Navigation on Vehicles.- 4.2.3 Available Inertial Sensors.- 4.2.4 Comparison with the Human Vestibular System.- 4.3 Auto-Calibration of Inertial Sensors.- 4.3.1 Presentation.- 4.3.2 Sensor Models.- 4.3.3 Accelerometers Intrinsic Calibration.- 4.3.4 Static Evaluation of Calibration Parameters.- 4.3.5 Experimental Procedure for Accelerometers Calibration.- 4.3.6 Gyrometers Intrinsic Calibration.- 4.3.7 Principle of the Dynamic Calibration.- 4.3.8 Computing the Rotation from g.- 4.3.9 Performing a Rotation in the Vertical Plane.- 4.3.10 Experimental Procedure for Calibration.- 4.3.11 Experimental Results for Calibration.- 4.4 Separation Between Gravity and Linear Acceleration.- 4.4.1 Using Special Assumptions on the Environment.- 4.4.2 Method 1: Estimating ?(O) in an Absolute Frame of Reference.- 4.4.3 Method 2: Estimating ?(O) Using the Jerk.- 4.5 Integration of Angular Position.- 4.5.1 A Rational Representation of 3D Rotations.- 4.5.2 Relation Between ? and the Angular Position.- 4.5.3 Cooperation of the Inertial and Visual Systems.- 4.5.4 Application of Inertial Information in a Visual System.- 4.5.5 Computation of the Scale Factor.- 4.6 Computing Self-Motion with a Vertical Estimate.- 4.6.1 A Representation of Self-Motion Using Vertical Cues.- 4.6.2 The "Gyro-Rotation" and the Vertical Rectification.- 4.6.3 Stabilization Using 2D Translation and Rotation.- 4.6.4 Implications on the Structure from Motion Paradigm.- 4.6.5 Estimation of the 3D Rotation.- 4.6.6 Implication on Token Matching.- 4.6.7 Experimental Results.- 4.7 Conclusion.- 5. Retinal Motion as a Cue for Active Vision.- 5.1 Definition and Notation.- 5.1.1 Calibration: The Camera Model.- 5.1.2 Motion: Discrete Rigid Displacements.- 5.1.3 Structure: Rigid Planar Structures.- 5.2 Using Collineations to Analyse the Retinal Motion.- 5.2.1 Definition.- 5.2.2 Properties.- 5.3 Estimation of Retinal Motion from Correspondences.- 5.3.1 Estimation of a H-Matrix from Point Correspondences.- 5.3.2 Performing Tests on the Estimate.- 5.3.3 Considering Correspondences Between Non-punctual Primitives.- 5.4 Implementation and Experimental Results.- 5.5 Conclusion.- 6. Uncalibrated Motion of Points and Lines.- 6.1 Introduction.- 6.2 Representations of the Retinal Motion for Points.- 6.2.1 Considering the Euclidean Parameters of the Scene.- 6.2.2 From Euclidean to Affine Parameters.- 6.2.3 Prom Affine to a Projective Parameterization.- 6.2.4 Conclusion: Choosing a Composite Representation.- 6.3 Representations of the Retinal Motion for Points and Lines.- 6.3.1 The Retinal Motion for Lines.- 6.3.2 Motion of Lines in Three Views.- 6.3.3 Conclusion on Lines and Points Motion.- 6.4 Implementation and Experimental Results.- 6.4.1 Implementation of the Motion Module.- 6.4.2 Experimental Results.- 6.5 Conclusion.- 6.6 Application to the Planar Case.- 7. Conclusion.- References.