Generative AI for Connected and Autonomous Vehicles
1st Edition
Will be published approx. on 15. December 2026
Book
Hardback
978-1-394-42668-3 (ISBN)
Description
Harness generative AI for connected and autonomous vehicle systems
As generative AI reshapes intelligent transportation, engineers and researchers need rigorous frameworks for deploying these technologies in safety-critical systems. This book offers a comprehensive treatment of the models, protocols, and adaptation techniques required to bring large language models, vision-language models, diffusion models, and agentic systems into modern CAV architectures. The book progresses from generative foundations through multimodal representation, language-based interaction, agentic coordination, communication protocols, model adaptation, and validation and safety. Mathematical, probabilistic, and reinforcement-learning prerequisites are consolidated into dedicated appendices, allowing the main chapters to focus on system design and CAV-specific applications. Written by an award-winning researcher with more than 100 patents across connected vehicles and AI, readers will also find:
Practical treatment of prompt engineering, retrieval-augmented generation, function calling, and tool use for grounding GenAI in vehicular context
Chapters on agent architectures, multi-agent systems, orchestration frameworks, and MCP and A2A protocols for collaborative CAV intelligence
Adaptation and deployment techniques including supervised fine-tuning, reinforcement learning fine-tuning, and knowledge distillation for resource-constrained edge platforms
Dedicated chapters on simulation platforms, evaluation methodologies, and ethical and safety considerations for deploying generative AI in autonomous vehicles
Automotive engineers, AI researchers, and graduate students at the intersection of machine learning and intelligent transportation will find this book indispensable. Readers from OEMs, mobility startups, or academic labs gain the foundations and practices needed to design, adapt, and validate generative AI in connected mobility.
As generative AI reshapes intelligent transportation, engineers and researchers need rigorous frameworks for deploying these technologies in safety-critical systems. This book offers a comprehensive treatment of the models, protocols, and adaptation techniques required to bring large language models, vision-language models, diffusion models, and agentic systems into modern CAV architectures. The book progresses from generative foundations through multimodal representation, language-based interaction, agentic coordination, communication protocols, model adaptation, and validation and safety. Mathematical, probabilistic, and reinforcement-learning prerequisites are consolidated into dedicated appendices, allowing the main chapters to focus on system design and CAV-specific applications. Written by an award-winning researcher with more than 100 patents across connected vehicles and AI, readers will also find:
Practical treatment of prompt engineering, retrieval-augmented generation, function calling, and tool use for grounding GenAI in vehicular context
Chapters on agent architectures, multi-agent systems, orchestration frameworks, and MCP and A2A protocols for collaborative CAV intelligence
Adaptation and deployment techniques including supervised fine-tuning, reinforcement learning fine-tuning, and knowledge distillation for resource-constrained edge platforms
Dedicated chapters on simulation platforms, evaluation methodologies, and ethical and safety considerations for deploying generative AI in autonomous vehicles
Automotive engineers, AI researchers, and graduate students at the intersection of machine learning and intelligent transportation will find this book indispensable. Readers from OEMs, mobility startups, or academic labs gain the foundations and practices needed to design, adapt, and validate generative AI in connected mobility.
More details
Series
Language
English
Place of publication
United States
Publishing group
John Wiley & Sons Inc
Target group
Professional and scholarly
ISBN-13
978-1-394-42668-3 (9781394426683)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Person
Kyungtae Han, PhD, is a Senior Principal Researcher at Toyota Motor North America and was previously a Senior Research Scientist at Intel Labs. With over 90 publications and more than 100 patents spanning generative AI, connected and automated vehicles, cyber-physical systems, and edge computing, his work has earned the SAE Vincent Bendix Award, IEEE Best Application Award, and Intel Achievement Award.
Content
Preface xxxv
Acknowledgments xxxix
How to Read This Book xli
Statement on the Use of Generative AI xlv
Companion Resources xlvii
Acronyms xlix
I Foundations of Generative Intelligence for CAVs 1
1 Introduction to Generative AI for CAV Systems 3
1.1 Introduction: The Promise of Generative AI in CAVs 4
1.2 The Paradigm Shift: Generative vs. Discriminative AI 5
1.3 Transformative Capabilities in CAVs 10
1.4 Generative Model Families 13
1.5 The Generative AI Ecosystem 21
1.6 Summary and Roadmap 29
2 Large Language Models: Architecture and Operation 39
2.1 Introduction 40
2.3 The Rise of Transformers in Language Modeling 47
2.4 The LLM Generation Pipeline 51
2.5 Stage 1: Tokenization and Embeddings 54
2.6 Stage 2: Positional Encoding 57
2.7 Stage 3: Attention and Contextualization 60
2.8 Stage 4: Refinement and Stacking 66
2.9 Stage 5: Output and Sampling 71
2.10 Model Limitations and Comparative Landscape 74
2.11 Summary 81
II Multimodal Representation and World Grounding 86
3 Diffusion and Flow Matching 87
3.1 Introduction: Why Diffusion for CAV Systems 88
3.2 Conceptual Foundations 90
3.3 Mathematical Foundations 92
3.4 Architectural Choices in Diffusion and Transport Models 101
3.5 Applications in CAV Systems 104
3.6 Limitations and Considerations 107
3.7 Summary 108
4 GANs, VAEs, and Autoregressive Models 111
4.1 Introduction 112
4.2 Generative Adversarial Networks 114
4.3 Variational Autoencoders 123
4.4 Autoregressive Models for Sequential Prediction 134
4.5 Comparative Analysis 142
4.6 Summary 143
5 Vision-Language Models for Scene Understanding 149
5.1 Introduction 150
5.2 Architectural Principles of VLMs 151
5.3 Stage I: Visual Feature Extraction 155
5.4 Stage II: Vision-Language Adaptation 160
5.5 Stage III: Integration into the Language Model 164
5.6 Training Methodologies for VLMs 169
5.7 Deployment in Connected and Autonomous Vehicles 178
5.8 Case Study: Semantic Filtering of V2X Messages 181
5.9 Summary 185
6 Video Large Language Models for Temporal Reasoning in Driving Environments 189
6.1 Introduction 190
6.2 Fundamental Design Principles 191
6.3 Stage I: Spatio-Temporal Encoding 194
6.4 Stage II: Spatio-Temporal Fusion 198
6.5 Stage III: Integration with Language Models 201
6.6 Token Compression Techniques 207
6.7 Representative Architectures 212
6.8 Applications in Connected and Autonomous Vehicles 215
6.9 Summary 217
III Language-Based Interaction, Memory, and Tool Use 220
7 Prompt Engineering 221
7.1 Introduction and Motivation 222
7.2 Foundations of Prompt Engineering 224
7.3 Design Principles for Effective Prompting 228
7.4 Prompt Engineering in Practice 234
7.5 Prompt Fragility and Safety Considerations in CAV Systems 244
7.6 Summary 245
8 Retrieval-Augmented Generation for Context Grounding 249
8.1 Why RAG Matters in Connected and Autonomous Vehicles 250
8.2 Architectural Foundations of RAG 254
8.3 RAG vs. Rule-Based Information Systems in CAVs 258
8.4 Mathematical Modeling of RAG 260
8.5 Practical RAG Implementation for CAV Systems 263
8.6 Real-Time Feasibility and System Constraints 267
8.7 Applied Use Cases of RAG in CAV Systems 269
8.8 Summary 272
9 Function Calling and Tool Use 275
9.1 Introduction: Why CAVs Need Function Calling 276
9.2 System Architecture of Function Calling 279
9.3 Theoretical Foundations of Function Calling 283
9.4 Function Calling in Practice 288
9.5 Function Schema Library and Design Principles 294
9.6 Summary 302
IV Agentic Intelligence and Coordination 306
10 GenAI-Enhanced Agent Architectures 307
10.1 Introduction 308
10.2 Classical Agent Foundations and Architecture 310
10.3 GenAI Enhancement Concepts 313
10.4 Mathematical Formalization 320
10.5 ReAct Framework 323
10.6 Individual Agent Applications in CAVs 326
10.7 Limitations and Challenges of GenAI-Enhanced Agents 328
10.8 Summary 332
11 GenAI-Enhanced Multi-Agent Systems 335
11.1 Introduction 336
11.2 Multi-Agent Systems Foundations and Architecture 337
11.3 GenAI Enhancement Concepts 342
11.4 Mathematical Formalization 348
11.5 GenAI-Enhanced Applications in CAVs 354
11.6 Implementation Considerations and Practical Challenges 356
11.7 Summary 361
12 Orchestration Frameworks for Agentic AI 365
12.1 Introduction 366
12.2 Graph-Based Orchestration with LangGraph 368
12.3 Dialogue-Based Orchestration with AutoGen 378
12.4 Framework Comparison and Selection 387
12.5 Summary 390
V Communication Protocols and Collective Intelligence 392
13 Communication Protocols for Agentic AI 393
13.1 Introduction 394
13.2 Protocol Classification Framework 396
13.3 Protocol Evolution and Development 402
13.4 Protocol Examples and CAV Applications 403
13.5 Summary 406
14 Model Context Protocol for Agentic Systems 409
14.1 Introduction 410
14.2 Context Provision in CAV Systems 411
14.3 MCP Architecture 415
14.4 Server Features 423
14.5 Client Features 428
14.6 Mathematical Foundations 432
14.7 Practical Implementation 435
14.8 MCP in CAV Systems 439
14.9 Summary 444
15 Agent-to-Agent Protocols for Collaborative Intelligence 447
15.1 Introduction 448
15.2 A2A Architecture 449
15.3 Mathematical Foundations for Agent Collaboration 458
15.4 Security and Trust 464
15.5 A2A in CAV Systems 466
15.6 Limitations and Challenges 470
15.7 Summary 473
16 Integrated CAV Agent Architectures with MCP and A2A 477
16.1 Introduction 478
16.2 Protocol Comparison and Decision Framework 479
16.3 System Architecture and Mathematical Framework 482
16.4 Implementation and Execution Patterns 486
16.5 Summary 495
VI Adaptation and Deployment 497
17 Supervised Fine-Tuning 499
17.1 The Necessity of Fine-Tuning in CAV Applications 500
17.2 From Foundation Models to Specialization 502
17.3 Architectural Perspective on Fine-Tuning 504
17.4 Mathematical Formulation of Fine-Tuning 507
17.5 Practical Fine-Tuning 509
17.6 Parameter-Efficient Fine-Tuning Methods 515
17.7 Use Cases and Deployment Modes 524
17.8 Summary 528
18 Reinforcement Learning Fine-Tuning 531
18.1 Introduction 532
18.2 Mathematical Foundation 536
18.3 Policy Optimization Methods in Detail 544
18.4 RLHF Methodology 559
18.5 Summary 562
19 Knowledge Distillation for Edge Deployment 567
19.1 Introduction: The Deployment Challenge 568
19.2 The Teacher-Student Paradigm 568
19.3 Core Knowledge Distillation Mechanisms 570
19.4 Advanced Distillation Techniques 577
19.5 Deployment Considerations for CAV Applications 583
19.6 Summary 585
VII Validation and Safety 588
20 Simulation Platforms for GenAI-Driven CAVs 589
20.1 Introduction: Why Simulation Is Indispensable for GenAI-CAVs 590
20.2 Simulation as a System-Level Substrate for GenAI-CAVs 592
20.3 Scenario Generation and Coverage Expansion 595
20.4 Simulation Platforms for GenAI-CAV Development 597
20.5 Scaling Simulation: From Single-Agent to Multi-Agent Systems 602
20.6 Simulation-Evaluation Boundary 605
20.7 Summary 606
21 Evaluation of Generative AI Systems in CAVs 609
21.1 Introduction: The Role of Evaluation in GenAI-Driven CAVs 610
21.2 Evaluation Dimensions for GenAI Systems in CAV Applications 611
21.3 Fundamental Evaluation Metrics and Methodologies 615
21.4 Evaluation of Vision-Language Models for CAV Applications 637
21.5 Evaluation of Retrieval-Augmented Generation Systems 641
21.6 Multi-Turn Dialogue and Conversational Agents 648
21.7 Summary 650
22 Ethical and Safety Considerations 655
22.1 Introduction: The Ethical Stakes of Generative AI in CAVs 656
22.2 Foundational Ethical Principles and Challenges 658
22.3 Case Studies: Ethical Challenges in Practice 676
22.4 Regulatory and Safety Frameworks 679
22.5 Ethical Engineering Methods 683
22.6 Broader Environmental and Societal Considerations 689
22.7 Summary 690
Bibliography 692
Acknowledgments xxxix
How to Read This Book xli
Statement on the Use of Generative AI xlv
Companion Resources xlvii
Acronyms xlix
I Foundations of Generative Intelligence for CAVs 1
1 Introduction to Generative AI for CAV Systems 3
1.1 Introduction: The Promise of Generative AI in CAVs 4
1.2 The Paradigm Shift: Generative vs. Discriminative AI 5
1.3 Transformative Capabilities in CAVs 10
1.4 Generative Model Families 13
1.5 The Generative AI Ecosystem 21
1.6 Summary and Roadmap 29
2 Large Language Models: Architecture and Operation 39
2.1 Introduction 40
2.3 The Rise of Transformers in Language Modeling 47
2.4 The LLM Generation Pipeline 51
2.5 Stage 1: Tokenization and Embeddings 54
2.6 Stage 2: Positional Encoding 57
2.7 Stage 3: Attention and Contextualization 60
2.8 Stage 4: Refinement and Stacking 66
2.9 Stage 5: Output and Sampling 71
2.10 Model Limitations and Comparative Landscape 74
2.11 Summary 81
II Multimodal Representation and World Grounding 86
3 Diffusion and Flow Matching 87
3.1 Introduction: Why Diffusion for CAV Systems 88
3.2 Conceptual Foundations 90
3.3 Mathematical Foundations 92
3.4 Architectural Choices in Diffusion and Transport Models 101
3.5 Applications in CAV Systems 104
3.6 Limitations and Considerations 107
3.7 Summary 108
4 GANs, VAEs, and Autoregressive Models 111
4.1 Introduction 112
4.2 Generative Adversarial Networks 114
4.3 Variational Autoencoders 123
4.4 Autoregressive Models for Sequential Prediction 134
4.5 Comparative Analysis 142
4.6 Summary 143
5 Vision-Language Models for Scene Understanding 149
5.1 Introduction 150
5.2 Architectural Principles of VLMs 151
5.3 Stage I: Visual Feature Extraction 155
5.4 Stage II: Vision-Language Adaptation 160
5.5 Stage III: Integration into the Language Model 164
5.6 Training Methodologies for VLMs 169
5.7 Deployment in Connected and Autonomous Vehicles 178
5.8 Case Study: Semantic Filtering of V2X Messages 181
5.9 Summary 185
6 Video Large Language Models for Temporal Reasoning in Driving Environments 189
6.1 Introduction 190
6.2 Fundamental Design Principles 191
6.3 Stage I: Spatio-Temporal Encoding 194
6.4 Stage II: Spatio-Temporal Fusion 198
6.5 Stage III: Integration with Language Models 201
6.6 Token Compression Techniques 207
6.7 Representative Architectures 212
6.8 Applications in Connected and Autonomous Vehicles 215
6.9 Summary 217
III Language-Based Interaction, Memory, and Tool Use 220
7 Prompt Engineering 221
7.1 Introduction and Motivation 222
7.2 Foundations of Prompt Engineering 224
7.3 Design Principles for Effective Prompting 228
7.4 Prompt Engineering in Practice 234
7.5 Prompt Fragility and Safety Considerations in CAV Systems 244
7.6 Summary 245
8 Retrieval-Augmented Generation for Context Grounding 249
8.1 Why RAG Matters in Connected and Autonomous Vehicles 250
8.2 Architectural Foundations of RAG 254
8.3 RAG vs. Rule-Based Information Systems in CAVs 258
8.4 Mathematical Modeling of RAG 260
8.5 Practical RAG Implementation for CAV Systems 263
8.6 Real-Time Feasibility and System Constraints 267
8.7 Applied Use Cases of RAG in CAV Systems 269
8.8 Summary 272
9 Function Calling and Tool Use 275
9.1 Introduction: Why CAVs Need Function Calling 276
9.2 System Architecture of Function Calling 279
9.3 Theoretical Foundations of Function Calling 283
9.4 Function Calling in Practice 288
9.5 Function Schema Library and Design Principles 294
9.6 Summary 302
IV Agentic Intelligence and Coordination 306
10 GenAI-Enhanced Agent Architectures 307
10.1 Introduction 308
10.2 Classical Agent Foundations and Architecture 310
10.3 GenAI Enhancement Concepts 313
10.4 Mathematical Formalization 320
10.5 ReAct Framework 323
10.6 Individual Agent Applications in CAVs 326
10.7 Limitations and Challenges of GenAI-Enhanced Agents 328
10.8 Summary 332
11 GenAI-Enhanced Multi-Agent Systems 335
11.1 Introduction 336
11.2 Multi-Agent Systems Foundations and Architecture 337
11.3 GenAI Enhancement Concepts 342
11.4 Mathematical Formalization 348
11.5 GenAI-Enhanced Applications in CAVs 354
11.6 Implementation Considerations and Practical Challenges 356
11.7 Summary 361
12 Orchestration Frameworks for Agentic AI 365
12.1 Introduction 366
12.2 Graph-Based Orchestration with LangGraph 368
12.3 Dialogue-Based Orchestration with AutoGen 378
12.4 Framework Comparison and Selection 387
12.5 Summary 390
V Communication Protocols and Collective Intelligence 392
13 Communication Protocols for Agentic AI 393
13.1 Introduction 394
13.2 Protocol Classification Framework 396
13.3 Protocol Evolution and Development 402
13.4 Protocol Examples and CAV Applications 403
13.5 Summary 406
14 Model Context Protocol for Agentic Systems 409
14.1 Introduction 410
14.2 Context Provision in CAV Systems 411
14.3 MCP Architecture 415
14.4 Server Features 423
14.5 Client Features 428
14.6 Mathematical Foundations 432
14.7 Practical Implementation 435
14.8 MCP in CAV Systems 439
14.9 Summary 444
15 Agent-to-Agent Protocols for Collaborative Intelligence 447
15.1 Introduction 448
15.2 A2A Architecture 449
15.3 Mathematical Foundations for Agent Collaboration 458
15.4 Security and Trust 464
15.5 A2A in CAV Systems 466
15.6 Limitations and Challenges 470
15.7 Summary 473
16 Integrated CAV Agent Architectures with MCP and A2A 477
16.1 Introduction 478
16.2 Protocol Comparison and Decision Framework 479
16.3 System Architecture and Mathematical Framework 482
16.4 Implementation and Execution Patterns 486
16.5 Summary 495
VI Adaptation and Deployment 497
17 Supervised Fine-Tuning 499
17.1 The Necessity of Fine-Tuning in CAV Applications 500
17.2 From Foundation Models to Specialization 502
17.3 Architectural Perspective on Fine-Tuning 504
17.4 Mathematical Formulation of Fine-Tuning 507
17.5 Practical Fine-Tuning 509
17.6 Parameter-Efficient Fine-Tuning Methods 515
17.7 Use Cases and Deployment Modes 524
17.8 Summary 528
18 Reinforcement Learning Fine-Tuning 531
18.1 Introduction 532
18.2 Mathematical Foundation 536
18.3 Policy Optimization Methods in Detail 544
18.4 RLHF Methodology 559
18.5 Summary 562
19 Knowledge Distillation for Edge Deployment 567
19.1 Introduction: The Deployment Challenge 568
19.2 The Teacher-Student Paradigm 568
19.3 Core Knowledge Distillation Mechanisms 570
19.4 Advanced Distillation Techniques 577
19.5 Deployment Considerations for CAV Applications 583
19.6 Summary 585
VII Validation and Safety 588
20 Simulation Platforms for GenAI-Driven CAVs 589
20.1 Introduction: Why Simulation Is Indispensable for GenAI-CAVs 590
20.2 Simulation as a System-Level Substrate for GenAI-CAVs 592
20.3 Scenario Generation and Coverage Expansion 595
20.4 Simulation Platforms for GenAI-CAV Development 597
20.5 Scaling Simulation: From Single-Agent to Multi-Agent Systems 602
20.6 Simulation-Evaluation Boundary 605
20.7 Summary 606
21 Evaluation of Generative AI Systems in CAVs 609
21.1 Introduction: The Role of Evaluation in GenAI-Driven CAVs 610
21.2 Evaluation Dimensions for GenAI Systems in CAV Applications 611
21.3 Fundamental Evaluation Metrics and Methodologies 615
21.4 Evaluation of Vision-Language Models for CAV Applications 637
21.5 Evaluation of Retrieval-Augmented Generation Systems 641
21.6 Multi-Turn Dialogue and Conversational Agents 648
21.7 Summary 650
22 Ethical and Safety Considerations 655
22.1 Introduction: The Ethical Stakes of Generative AI in CAVs 656
22.2 Foundational Ethical Principles and Challenges 658
22.3 Case Studies: Ethical Challenges in Practice 676
22.4 Regulatory and Safety Frameworks 679
22.5 Ethical Engineering Methods 683
22.6 Broader Environmental and Societal Considerations 689
22.7 Summary 690
Bibliography 692