Engineering UAS Applications: Sensor Fusion, Machine Vision and Mission Management

Chapter 1. Introduction and state of the art

1.1 Introduction and types of UAVs
1.2 Main technologies used in UAS
1.2.1 Navigation
1.2.2 Communications
1.2.3 Machine vision
1.2.4 Coordination. Swarms of UAVs and applications
1.2.5 Simulation
1.3 Summary and structure of this book
References



Chapter 2: Components of UAS

2.1 Introduction
2.2 Flight controller
2.2.1 Logic components
2.2.1.1 Control
2.2.1.2 Navigation
2.2.1.3 Guidance
2.2.2 Physical components
2.3 Communications
2.3.1 Radio communication technologies
2.3.2 Communication protocols
2.3.3 UAV messaging protocols
2.4 Payload
2.4.1 Payload types
2.4.1.1 Image sensors
2.4.1.2 Wave sensors
2.4.1.3 Internet of Things (IoT)
2.4.1.4 Actuators
2.4.2 Payload positioning
2.4.2.1 Front sensor-FPV
2.4.2.2 Bottom sensor
2.4.2.3 Mounted fixed sensor
2.4.2.4 360? sensor
2.5 Mission management units
2.5.1 Ground station
2.5.2 Companion computer
2.5.3 Control APIs
2.6 Obstacle avoidance use case
2.6.1 Phase 1. Assembly of the physical components
2.6.2 Phase 2. Setting up the software
2.6.3 Phase 3. Mission design from the ground station
2.6.4 Phase 4. Mission start
2.6.5 Phase 5. Obstacle detection and avoidance
References



Chapter 3: Simulation of UAS

3.1 Introduction
3.2 From development to reality
3.2.1 Simulation software
3.2.2 Software-in-the-loop
3.2.3 Hardware-in-the-loop
3.2.4 External Hardware-in-the-loop
3.2.5 Simulation in Hardware
3.2.6 Vehicle in the loop
3.3 UAS simulators
3.3.1 Current simulators
3.3.2 jMAVsim
3.3.3 JBSim
3.3.4 FlightGoggles
3.3.5 Gazebo
3.3.5.1 Gazebo Ignition
3.3.6 FlightGear
3.3.7 MATLAB UAV toolbox
3.3.7.1 RflySim
3.3.8 AirSim
3.3.8.1 Aerial Autonomy: Project AirSim
3.3.8.2 Unreal Engine addons
3.3.9 Simulators comparison
3.4 AirSim simulation examples
3.4.1 Framework required programs
3.4.2 Simulation environment
3.4.3 AirSim settings
3.4.4 SimpleFlight simulation
3.4.5 Mission 1. Using SimpleFlight SITL
3.4.5.1 Python mission
3.4.5.2 Data gathering
3.4.5.3 Multiple drones
3.4.5.4 Manual control using joystick or USB controller
3.4.6 PX4 simulation
3.4.7 Mission 2. Using PX4 SITL
3.4.7.1 QGroundControl mission
3.4.7.2 MavSDK mission
3.4.7.3 Multiple PX4 SITL
3.4.8 Mission 3. Using PX4 HITL
3.4.8.1 Manual control using receiver and transmitter
3.4.9 Flight analysis
References



Chapter 4: Navigation systems on UAS

4.1 Introduction
4.2 Reference Frame Systems
4.2.1 Global frames (WGS84 and ECEF) and local frame at tangent point ENU and NED
4.2.2 Geodetic to ECEF transformation
4.2.3 ECEF to geodetic transformation
4.2.4 ECEF to local Cartesian (ENU and NED) transformation
4.2.5 Local Cartesian (ENU or NED) to ECEF transformation
4.3 Attitude mathematical concepts
4.3.1 Attitude representation
4.3.1.1 Direction cosine matrix (DCM)
4.3.1.2 Euler angles
4.3.1.3 Quaternions
4.3.2 Attitude Kinematics
4.3.2.1 DCM Kinematics
4.3.2.2 Euler Angles
4.3.2.3 Quaternions
4.4 Fusion of the INS and GNSS
4.4.1 State estimation
4.4.2 INS State vector
4.4.3 GNSS State vector
4.4.4 Fusion of the INS and GNSS
4.5 Application: Search for the best navigation parameters
4.5.1 Fusion quality metrics
4.5.1.1 Based on ground truth
4.5.1.2 Without ground truth
4.5.2 PX4 navigation system
4.5.3 Search best EKF parameters
4.5.3.1 Design test mission and flight plan
4.5.3.2 Analysis of results
References



Chapter 5: Machine vision systems of UAS

5.1 Introduction
5.2 Computer vision system
5.2.1 Pinhole camera
5.2.2 Camera calibration
5.2.3 AirSim camera calibration
5.3 Image stabilization
5.3.1 Mechanical stabilization
5.3.1.1 Example of mechanical stabilization in AirSim
5.3.2 Computational stabilization
5.3.2.1 Digital image motion
5.3.2.2 Example of computational correction
5.4 Object detection
5.4.1 Problems of object detection
5.4.2 How to evaluate object detection?
5.4.3 Object detection example
5.5 Visual object tracking
5.5.1 Visual object tracking algorithms
5.5.2 Drone VOT system in AirSim simulator
5.5.2.1 Evaluation
References



Chapter 6: Illustrative examples of UAS

6.1 Introduction
6.2 Sense and avoid illustrative example
6.2.1 Simulation architecture
6.2.2 Software modules
6.2.3 Detection System
6.2.4 Occupancy map
6.2.5 Avoidance strategy
6.2.6 Evaluation
6.2.7 Conclusions
6.3 Decentralized drone swarm illustrative example
6.3.1 Simulation architecture
6.3.2 Software modules
6.3.3 Swarm generator
6.3.4 Communication loop
6.3.5 Mission vote system
6.3.6 Formation generation avoiding collisions
6.3.7 Swarm formation to target
6.3.8 Mission movement
6.3.9 Swarm obstacle avoidance
6.3.10 Mission recalculation
6.3.11 Evaluation
6.3.12 Conclusions
6.4 Terrain reconstruction illustrative example
6.4.1 Simulation architecture
6.4.2 Software modules
6.4.3 Multi-UAV path planning
6.4.4 UAV Mission
6.4.5 Real-time orthoimagery generation
6.4.6 Evaluation
6.4.7 Conclusions
References



Chapter VII: Conclusions and future works