Computer Vision and Imaging in Intelligent Transportation Systems

List of Contributors xiii Preface xvii Acknowledgments xxi About the Companion Website xxiii 1 Introduction 1 Raja Bala and Robert P. Loce 1.1 Law Enforcement and Security 1 1.2 Efficiency 4 1.3 Driver Safety and Comfort 5 1.4 A Computer Vision Framework for Transportation Applications 7 1.4.1 Image and Video Capture 8 1.4.2 Data Preprocessing 8 1.4.3 Feature Extraction 9 1.4.4 Inference Engine 10 1.4.5 Data Presentation and Feedback 11 Part I Imaging from the Roadway Infrastructure 15 2 Automated License Plate Recognition 17 Aaron Burry and Vladimir Kozitsky 2.1 Introduction 17 2.2 Core ALPR Technologies 18 2.2.1 License Plate Localization 19 2.2.2 Character Segmentation 24 2.2.3 Character Recognition 28 2.2.4 State Identification 38 3 Vehicle Classification 47 Shashank Deshpande, Wiktor Muron and Yang Cai 3.1 Introduction 47 3.2 Overview of the Algorithms 48 3.3 Existing AVC Methods 48 3.4 LiDAR Imaging-Based 49 3.4.1 LiDAR Sensors 49 3.4.2 Fusion of LiDAR and Vision Sensors 50 3.5 Thermal Imaging?-Based 53 3.5.1 Thermal Signatures 53 3.5.2 Intensity Shape?-Based 56 3.6 Shape?- and Profile?-Based 58 3.6.1 Silhouette Measurements 60 3.6.2 Edge?-Based Classification 65 3.6.3 Histogram of Oriented Gradients 67 3.6.4 Haar Features 68 3.6.5 Principal Component Analysis 69 3.7 Intrinsic Proportion Model 72 3.8 3D Model?-Based Classification 74 3.9 SIFT?-Based Classification 74 3.10 Summary 75 4 Detection of Passenger Compartment Violations 81 Orhan Bulan, Beilei Xu, Robert P. Loce and Peter Paul 4.1 Introduction 81 4.2 Sensing within the Passenger Compartment 82 4.2.1 Seat Belt Usage Detection 82 4.2.2 Cell Phone Usage Detection 83 4.2.3 Occupancy Detection 83 4.3 Roadside Imaging 84 4.3.1 Image Acquisition Setup 84 4.3.2 Image Classification Methods 85 4.3.3 Detection?-Based Methods 94 5 Detection of Moving Violations 101 Wencheng Wu, Orhan Bulan, Edgar A. Bernal and Robert P. Loce 5.1 Introduction 101 5.2 Detection of Speed Violations 101 5.2.1 Speed Estimation from Monocular Cameras 102 5.2.2 Speed Estimation from Stereo Cameras 108 5.2.3 Discussion 115 5.3 Stop Violations 115 5.3.1 Red Light Cameras 115 5.4 Other Violations 125 5.4.1 Wrong?-Way Driver Detection 125 5.4.2 Crossing Solid Lines 126 6 Traffic Flow Analysis 131 Rodrigo Fernandez, Muhammad Haroon Yousaf, Timothy J. Ellis, Zezhi Chen and Sergio A. Velastin 6.1 What is Traffic Flow Analysis? 131 6.1.1 Traffic Conflicts and Traffic Analysis 131 6.1.2 Time Observation 132 6.1.3 Space Observation 133 6.1.4 The Fundamental Equation 133 6.1.5 The Fundamental Diagram 133 6.1.6 Measuring Traffic Variables 134 6.1.7 Road Counts 135 6.1.8 Junction Counts 135 6.1.9 Passenger Counts 136 6.1.10 Pedestrian Counts 136 6.1.11 Speed Measurement 136 6.2 The Use of Video Analysis in Intelligent Transportation Systems 137 6.2.1 Introduction 137 6.2.2 General Framework for Traffic Flow Analysis 137 6.2.3 Application Domains 143 6.3 Measuring Traffic Flow from Roadside CCTV Video 144 6.3.1 Video Analysis Framework 144 6.3.2 Vehicle Detection 146 6.3.3 Background Model 146 6.3.4 Counting Vehicles 149 6.3.5 Tracking 150 6.3.6 Camera Calibration 150 6.3.7 Feature Extraction and Vehicle Classification 152 6.3.8 Lane Detection 153 6.3.9 Results 155 6.4 Some Challenges 156 7 Intersection Monitoring Using Computer Vision Techniques for Capacity, Delay, and Safety Analysis 163 Brendan Tran Morris and Mohammad Shokrolah Shirazi 7.1 Vision?-Based Intersection Analysis: Capacity, Delay, and Safety 163 7.1.1 Intersection Monitoring 163 7.1.2 Computer Vision Application 164 7.2 System Overview 165 7.2.1 Tracking Road Users 166 7.2.2 Camera Calibration 169 7.3 Count Analysis 171 7.3.1 Vehicular Counts 171 7.3.2 Nonvehicular Counts 173 7.4 Queue Length Estimation 173 7.4.1 Detection?-Based Methods 174 7.4.2 Tracking?-Based Methods 175 7.5 Safety Analysis 177 7.5.1 Behaviors 178 7.5.2 Accidents 182 7.5.3 Conflicts 185 7.6 Challenging Problems and Perspectives 187 7.6.1 Robust Detection and Tracking 187 7.6.2 Validity of Prediction Models for Conflict and Collisions 188 7.6.3 Cooperating Sensing Modalities 189 7.6.4 Networked Traffic Monitoring Systems 189 7.7 Conclusion 189 8 Video?-Based Parking Management 195 Oliver Sidla and Yuriy Lipetski 8.1 Introduction 195 8.2 Overview of Parking Sensors 197 8.3 Introduction to Vehicle Occupancy Detection Methods 200 8.4 Monocular Vehicle Detection 200 8.4.1 Advantages of Simple 2D Vehicle Detection 200 8.4.2 Background Model-Based Approaches 200 8.4.3 Vehicle Detection Using Local Feature Descriptors 202 8.4.4 Appearance?-Based Vehicle Detection 203 8.4.5 Histograms of Oriented Gradients 204 8.4.6 LBP Features and LBP Histograms 207 8.4.7 Combining Detectors into Cascades and Complex Descriptors 208 8.4.8 Case Study: Parking Space Monitoring Using a Combined Feature Detector 208 8.4.9 Detection Using Artificial Neural Networks 211 8.5 Introduction to Vehicle Detection with 3D Methods 213 8.6 Stereo Vision Methods 215 8.6.1 Introduction to Stereo Methods 215 8.6.2 Limits on the Accuracy of Stereo Reconstruction 216 8.6.3 Computing the Stereo Correspondence 217 8.6.4 Simple Stereo for Volume Occupation Measurement 218 8.6.5 A Practical System for Parking Space Monitoring Using a Stereo System 218 8.6.6 Detection Methods Using Sparse 3D Reconstruction 220 9 Video Anomaly Detection 227 Raja Bala and Vishal Monga 9.1 Introduction 227 9.2 Event Encoding 228 9.2.1 Trajectory Descriptors 229 9.2.2 Spatiotemporal Descriptors 231 9.3 Anomaly Detection Models 233 9.3.1 Classification Methods 233 9.3.2 Hidden Markov Models 234 9.3.3 Contextual Methods 234 9.4 Sparse Representation Methods for Robust Video Anomaly Detection 236 9.4.1 Structured Anomaly Detection 237 9.4.2 Unstructured Video Anomaly Detection 243 9.4.3 Experimental Setup and Results 245 9.5 Conclusion and Future Research 253 Part II Imaging from and within the Vehicle 257 10 Pedestrian Detection 259 Shashank Deshpande and Yang Cai 10.1 Introduction 259 10.2 Overview of the Algorithms 259 10.3 Thermal Imaging 260 10.4 Background Subtraction Methods 261 10.4.1 Frame Subtraction 261 10.4.2 Approximate Median 262 10.4.3 Gaussian Mixture Model 263 10.5 Polar Coordinate Profile 263 10.6 Image?-Based Features 265 10.6.1 Histogram of Oriented Gradients 265 10.6.2 Deformable Parts Model 266 10.6.3 LiDAR and Camera Fusion-Based Detection 266 10.7 LiDAR Features 268 10.7.1 Preprocessing Module 268 10.7.2 Feature Extraction Module 268 10.7.3 Fusion Module 268 10.7.4 LIPD Dataset 270 10.7.5 Overview of the Algorithm 270 10.7.6 LiDAR Module 272 10.7.7 Vision Module 275 10.7.8 Results and Discussion 276 10.7.8.1 LiDAR Module 276 10.7.8.2 Vision Module 276 10.8 Summary 280 11 Lane Detection and Tracking Problems in Lane Departure Warning Systems 283 Gianni Cario, Alessandro Casavola and Marco Lupia 11.1 Introduction 283 11.2 LD: Algorithms for a Single Frame 285 11.2.1 Image Preprocessing 285 11.2.2 Edge Extraction 287 11.2.3 Stripe Identification 291 11.2.4 Line Fitting 294 11.3 LT Algorithms 297 11.3.1 Recursive Filters on Subsequent N frames 298 11.3.2 Kalman Filter 298 11.4 Implementation of an LD and LT Algorithm 299 11.4.1 Simulations 300 11.4.2 Test Driving Scenario 300 11.4.3 Driving Scenario: Lane Departures at Increasing Longitudinal Speed 300 11.4.4 The Proposed Algorithm 302 11.4.5 Conclusions 303 12 Vision?-Based Integrated Techniques for Collision Avoidance Systems 305 Ravi Satzoda and Mohan Trivedi 12.1 Introduction 305 12.2 Related Work 307 12.3 Context Definition for Integrated Approach 307 12.4 ELVIS: Proposed Integrated Approach 308 12.4.1 Vehicle Detection Using Lane Information 309 12.4.2 Improving Lane Detection using On?-Road Vehicle Information 312 12.5 Performance Evaluation 313 12.5.1 Vehicle Detection in ELVIS 313 12.5.2 Lane Detection in ELVIS 316 12.6 Concluding Remarks 319 13 Driver Monitoring 321 Raja Bala and Edgar A. Bernal 13.1 Introduction 321 13.2 Video Acquisition 322 13.3 Face Detection and Alignment 323 13.4 Eye Detection and Analysis 325 13.5 Head Pose and Gaze Estimation 326 13.5.1 Head Pose Estimation 326 13.5.2 Gaze Estimation 328 13.6 Facial Expression Analysis 332 13.7 Multimodal Sensing and Fusion 334 13.8 Conclusions and Future Directions 336 14 Traffic Sign Detection and Recognition 343 Hasan Fleyeh 14.1 Introduction 343 14.2 Traffic Signs 344 14.2.1 The European Road and Traffic Signs 344 14.2.2 The American Road and Traffic Signs 347 14.3 Traffic Sign Recognition 347 14.4 Traffic Sign Recognition Applications 348 14.5 Potential Challenges 349 14.6 Traffic Sign Recognition System Design 349 14.6.1 Traffic Signs Datasets 352 14.6.2 Colour Segmentation 354 14.6.3 Traffic Sign's Rim Analysis 359 14.6.4 Pictogram Extraction 364 14.6.5 Pictogram Classification Using Features 365 14.7 Working Systems 369 15 Road Condition Monitoring 375 Matti Kutila, Pasi Pyykonen, Johan Casselgren and Patrik Jonsson 15.1 Introduction 375 15.2 Measurement Principles 376 15.3 Sensor Solutions 377 15.3.1 Camera?-Based Friction Estimation Systems 377 15.3.2 Pavement Sensors 379 15.3.3 Spectroscopy 380 15.3.4 Roadside Fog Sensing 382 15.3.5 In?-Vehicle Sensors 383 15.4 Classification and Sensor Fusion 386 15.5 Field Studies 390 15.6 Cooperative Road Weather Services 394 15.7 Discussion and Future Work 395 Index 399