Generative AI for Communications Systems
Fundamentals, Applications, and Prospects
1st Edition
Published on 12. January 2026
Book
Hardback
368 pages
978-1-394-29390-2 (ISBN)
Description
Comprehensive review of state-of-the-art research and development in Generative AI for future communications and networking
Generative AI for Communications Systems provides a systematic foundation of knowledge on Generative AI for communications and networking. This book discusses the great potential and challenges in applying Generative AI as promising solutions to future communications systems and enables and facilitates "Generative AI as a Service" by exploring novel communications, networking architectures, protocols, and research trends.
The book also includes information on:
Crucial challenges to solve in Generative AI, such as training data availability, computational complexity, generalization for various scenarios, robustness of noisy and incomplete data, and real-time adaptation in communications and networking systems
Cybersecurity concerns such as ethics and privacy in relation to Generative AI
Applications of Generative AI across various layers, including the PHY layer, MAC layer, Network layer, and Application layer
Communications and networking solutions to meet the computing and communications challenges and demands to train and deploy large-scale Generative AI models
Generative AI for Communications Systems is an excellent up-to-date resource on the subject for scholars and researchers in the fields of communications, artificial intelligence, machine learning, and network optimization as well as professionals working in the communications industry including engineers, network architects, and system designers.
Generative AI for Communications Systems provides a systematic foundation of knowledge on Generative AI for communications and networking. This book discusses the great potential and challenges in applying Generative AI as promising solutions to future communications systems and enables and facilitates "Generative AI as a Service" by exploring novel communications, networking architectures, protocols, and research trends.
The book also includes information on:
Crucial challenges to solve in Generative AI, such as training data availability, computational complexity, generalization for various scenarios, robustness of noisy and incomplete data, and real-time adaptation in communications and networking systems
Cybersecurity concerns such as ethics and privacy in relation to Generative AI
Applications of Generative AI across various layers, including the PHY layer, MAC layer, Network layer, and Application layer
Communications and networking solutions to meet the computing and communications challenges and demands to train and deploy large-scale Generative AI models
Generative AI for Communications Systems is an excellent up-to-date resource on the subject for scholars and researchers in the fields of communications, artificial intelligence, machine learning, and network optimization as well as professionals working in the communications industry including engineers, network architects, and system designers.
More details
Language
English
Place of publication
United States
Publishing group
John Wiley & Sons Inc
Target group
Professional and scholarly
Dimensions
Height: 158 mm
Width: 239 mm
Thickness: 27 mm
Weight
646 gr
ISBN-13
978-1-394-29390-2 (9781394293902)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions
Diep N. Nguyen | Dinh Thai Hoang | Octavia A. Dobre
Generative AI for Communications Systems
Fundamentals, Applications, and Prospects
E-Book
01/2026
1st Edition
Wiley
€127.99
Available for download
Diep N. Nguyen | Dinh Thai Hoang | Octavia A. Dobre
Generative AI for Communications Systems
Fundamentals, Applications, and Prospects
E-Book
01/2026
1st Edition
Wiley
€127.99
Available for download
Persons
Diep N. Nguyen is the Head of UTS 5G/6G Lab with the Faculty of Engineering and Information Technology at the University of Technology Sydney (UTS), Sydney, NSW, Australia.
Nam H. Chu is a Faculty Member with the Department of Telecommunications Engineering at the University of Transport and Communications, Hanoi, Vietnam. He is also with the University of Technology Sydney (UTS), Australia, and the Crown Institute of Higher Education, Australia.
Dinh Thai Hoang is a Faculty Member at the University of Technology Sydney (UTS), Australia.
Octavia A. Dobre is a Professor and Canada Research Chair Tier-1 at Memorial University, Canada.
Dusit Niyato is a President's Chair Professor in Computer Science and Engineering in the College of Computing and Data Science at Nanyang Technological University, Singapore.
Petar Popovski is currently a Professor with Aalborg University in Denmark where he heads the Section on Connectivity. He is also a Visiting Excellence Chair with the University of Bremen, Germany.
Nam H. Chu is a Faculty Member with the Department of Telecommunications Engineering at the University of Transport and Communications, Hanoi, Vietnam. He is also with the University of Technology Sydney (UTS), Australia, and the Crown Institute of Higher Education, Australia.
Dinh Thai Hoang is a Faculty Member at the University of Technology Sydney (UTS), Australia.
Octavia A. Dobre is a Professor and Canada Research Chair Tier-1 at Memorial University, Canada.
Dusit Niyato is a President's Chair Professor in Computer Science and Engineering in the College of Computing and Data Science at Nanyang Technological University, Singapore.
Petar Popovski is currently a Professor with Aalborg University in Denmark where he heads the Section on Connectivity. He is also a Visiting Excellence Chair with the University of Bremen, Germany.
Content
List of Contributors xiii
Preface xxiii
Acronyms xxix
1 Future AI-empowered Communications Systems 1
Nguyen Van Huynh, Thien Huynh-The, and Quoc-Viet Pham
1.1 Fundamental Background of Future Communications Systems 1
1.1.1 Overview of Future Communications Systems 1
1.1.2 Key Challenges and Research Trends 7
1.2 AI-powered Communication Enablers 10
1.2.1 Deep Learning-based Approaches 10
1.2.2 Reinforcement Learning-based Approaches 17
1.2.3 Federated/Distributed Learning-based Approaches 24
1.2.4 Existing Challenges 28
1.2.5 Potential of Generative AI 28
1.3 Conclusion 30
References 30
2 Generative AI Background and Its Potentials for Future Communications Systems 39
Asmaa Abdallah, Abdulkadir Celik, and Ahmed M. Eltawil
2.1 Introduction 39
2.2 A Taxonomy of Generative Models 40
2.2.1 Explicit Density Models 41
2.2.2 Implicit Density Models 41
2.2.3 Ways GenAI Complements Discriminative AI 42
2.3 Prominent Generative Models 42
2.3.1 Generative Adversarial Networks 42
2.3.2 Variational Autoencoders 44
2.3.3 Flow-based Generative Models 47
2.3.4 Diffusion-based Generative Models 49
2.3.5 The Trilemma of GMs 51
2.3.6 Generative Autoregressive Models 52
2.3.7 Generative Transformers and LLMs 54
2.3.8 Strategies to Address LLM Limitations 59
2.4 GenAI Applications to Canonical Problems in Communications Systems 63
2.4.1 Physical Layer Design 63
2.4.2 Network Resource Management 64
2.4.3 Network Traffic Analytics 65
2.4.4 Cross-layer Network Security 65
2.4.5 Localization and Positioning 66
2.5 Future Communication Frontiers for GMs 66
2.5.1 Semantic Communications 66
2.5.2 Integrated Sensing and Communications 67
2.5.3 Digital Twins 68
2.5.4 AI-generated Content for 6G Networks 69
2.5.5 MEC and EAI 69
2.5.6 Adversarial Machine Learning and Trustworthy AI 70
2.6 Regulation and Policy 71
2.7 Summary 71
References 72
3 Key Study Cases of Generative AI Applications to Communications Systems 79
Mehdi Letafati, Samad Ali, and Matti Latva-aho
3.1 Overview on the Roles of Generative AI in Communication Systems 79
3.1.1 Use Cases of Generative Adversarial Networks in Communications 79
3.1.2 Use cases of VAEs in Communications 81
3.1.3 Use-cases of Diffusion Models in Communications 82
3.2 Case Study: Diffusion Models in Wireless Communications 83
3.2.1 Working Mechanism of Diffusion Models 83
3.2.2 Case Study: Diffusion Models Applications for Data Reconstruction Enhancement in Communication Systems 89
3.3 Future Implications and Potential Impacts on Communication Systems 98
3.4 Chapter Summary 99
References 99
4 Generative AI at PHY Layer: Native AI or Trainable Radios 105
Eren Balevi
4.1 Wireless Communications Empowered with Generative Models 105
4.1.1 Motivations of GenAI at the PHY 105
4.1.2 Applications of GenAI at the PHY 106
4.2 Channel Modeling 110
4.2.1 Generative Channel Modeling 111
4.2.2 Site-specific Generative Models 115
4.3 Generative Channel Estimation 116
4.3.1 Narrowband Channel Estimation with Reduced Pilots 120
4.3.2 Wideband Channel Estimation with Reduced Pilots 122
4.4 Channel Compression 123
4.5 Beamforming 125
4.6 Summary 129
References 129
5 Generative AI at the MAC Layer 133
Kemal Davaslioglu, Ender Ayanoglu, and Yalin E. Sagduyu
5.1 Introduction 133
5.2 Generative Models 137
5.2.1 Variational Autoencoders 139
5.2.2 Generative Adversarial Networks 140
5.2.3 Diffusion Models 141
5.3 Spectrum Awareness Applications 142
5.3.1 Data Augmentation and Synthetic Data Generation 143
5.3.2 Signal Classification Applications - UAV Classification 147
5.3.3 Anomaly Detection in RF Spectrum 148
5.4 RF Spectrum Security Applications 149
5.4.1 Emitter Identification 149
5.4.2 Wireless Spoofing 151
5.4.3 Enhanced Jamming Attacks 152
5.5 Scheduling Applications 153
5.5.1 Traffic Prediction and Pattern Generation 153
5.5.2 Adaptive Scheduling Algorithms 154
5.5.3 Interference Patterns 154
5.5.4 Fairness and QoS 154
5.5.5 Millimeter-wave Networks 155
5.6 Open Problems and Future Research Directions 155
5.6.1 Reconfigurable Intelligent Surface (RIS)-assisted Networks 156
5.6.2 Spectrum Sharing in the Presence of Interference 157
5.6.3 Integrated Sensing and Communications (ISAC) 159
5.6.4 Link Scheduling in Large Networks 160
5.6.5 Enhancing Wireless MAC-layer Security 160
5.7 Concluding Remarks 162
References 162
6 Generative AI at Network Layer 169
Athanasios Karapantelakis, Pegah Alizadeh, Abdulrahman Alabbasi, Kaushik Dey, and Alexandros Nikou
6.1 Introduction 169
6.2 Network Layer in Mobile Networks 172
6.2.1 Radio Access Network 172
6.2.2 Core Network 175
6.3 Generative AI in the Network Layer 177
6.3.1 Introduction 177
6.3.2 Advantages of GenAI Models 177
6.3.3 Short-term Applications (GenAI for Network Layer) 180
6.3.4 Long-term Applications (Network Layer for GenAI) 184
6.4 Challenges and Opportunities for GenAI in the Network Layer 185
6.4.1 Challenges 185
6.4.2 Research Opportunities 186
6.5 Summary 187
References 187
7 Generative AI at Application Layer: Mobile AI-generated Content 191
Paria Mohammadzadeh Hesar, Amirhossein Mohammadi, and Hina Tabassum
7.1 Introduction to AIGC 191
7.1.1 General Overview 191
7.1.2 AIGC in the Application Layer 192
7.1.3 AIGC Product Lifecycle 194
7.2 Collaborative Network Infrastructure for Enabling GenAI Services 196
7.2.1 Enabling AIGC - Challenges 196
7.2.2 Infrastructure Components and Capabilities 198
7.2.3 Collaborative Edge-cloud Infrastructure 201
7.3 Network Resource Efficient GenAI Methods 203
7.3.1 Model Optimization Techniques 203
7.3.2 Service Optimization Methods 206
7.4 Security and Privacy at Application Layer 208
7.4.1 Security Threat Models and Privacy Risks 209
7.4.2 Ethical Considerations in AIGC services 211
7.4.3 Enabling Secure AIGC-as-a-Service 212
7.5 Use Cases of Mobile AIGC 213
7.5.1 AI-generated Content in Social Media 213
7.5.2 Immersive Streaming (AR/VR) 215
7.5.3 Personalized AI Services 219
7.6 Conclusion and Research Directions 222
7.7 Summary 223
References 224
8 Applications of GenAI on Wireless and Cybersecurity 239
Brian Kim, Yalin E. Sagduyu, Tugba Erpek, Yi Shi, and Sennur Ulukus
8.1 Introduction to GenAI in Wireless and Cybersecurity 239
8.2 Adversarial Machine Learning in Wireless Communications 242
8.2.1 Different Types of Attacks Against GenAI-driven Wireless Applications 243
8.2.2 Defense Against Adversarial Attacks for GenAI-driven Wireless Applications 245
8.3 GenAI for Wireless Security and Cybersecurity 245
8.3.1 GenAI for Wireless Security 245
8.3.2 GenAI for Cybersecurity 248
8.3.3 GenAI-driven Attacks Against Wireless and Cybersecurity Applications 249
8.4 Ethical Issues Related to GenAI for Wireless Communications and Cybersecurity 251
8.5 Summary 252
References 253
9 Challenges and Opportunities for Generative AI in Wireless Communications and Networking 261
Songyang Zhang and Zhi Ding
9.1 Introduction 261
9.2 Challenges of Applying Generative AI in Wireless Communications 262
9.2.1 Efficiency and Robustness 263
9.2.2 Cost and Complexity 267
9.2.3 Standardization, Regulation, and Policy 269
9.3 Adopting Generative AI in NextG Communications: Case Studies 270
9.3.1 Integration of Generative AI and Physical Communications Models 270
9.3.2 Trustworthy Generative AI for Distributed Wireless Communications 277
9.4 Summary 281
References 281
10 Future Research Directions 285
Nam H. Chu, Diep N. Nguyen, Dinh Thai Hoang, Octavia A. Dobre, Dusit Niyato, and Petar Popovski
10.1 Introduction 285
10.2 Emerging Foundational Research Frontiers 286
10.2.1 Dedicated GenAI Models for Communication Systems 287
10.2.2 Fusion of GenAI and Emerging Technologies 288
10.3 Enhancing Generative AI Models for Wireless Communication Systems 290
10.3.1 Model Optimization and Generalization 290
10.3.2 Energy Efficiency 291
10.3.3 Generative AI for Spectrum Management 292
10.3.4 AI-driven Network Management and Orchestration 293
10.3.5 Security and Privacy Concerns 295
10.4 Practical Case Studies 296
10.4.1 AI-powered Network Optimization by T-Mobile 296
10.4.2 DeepSig's Generative AI for Wireless Communications 297
10.5 Conclusion 297
References 298
Index 305
Preface xxiii
Acronyms xxix
1 Future AI-empowered Communications Systems 1
Nguyen Van Huynh, Thien Huynh-The, and Quoc-Viet Pham
1.1 Fundamental Background of Future Communications Systems 1
1.1.1 Overview of Future Communications Systems 1
1.1.2 Key Challenges and Research Trends 7
1.2 AI-powered Communication Enablers 10
1.2.1 Deep Learning-based Approaches 10
1.2.2 Reinforcement Learning-based Approaches 17
1.2.3 Federated/Distributed Learning-based Approaches 24
1.2.4 Existing Challenges 28
1.2.5 Potential of Generative AI 28
1.3 Conclusion 30
References 30
2 Generative AI Background and Its Potentials for Future Communications Systems 39
Asmaa Abdallah, Abdulkadir Celik, and Ahmed M. Eltawil
2.1 Introduction 39
2.2 A Taxonomy of Generative Models 40
2.2.1 Explicit Density Models 41
2.2.2 Implicit Density Models 41
2.2.3 Ways GenAI Complements Discriminative AI 42
2.3 Prominent Generative Models 42
2.3.1 Generative Adversarial Networks 42
2.3.2 Variational Autoencoders 44
2.3.3 Flow-based Generative Models 47
2.3.4 Diffusion-based Generative Models 49
2.3.5 The Trilemma of GMs 51
2.3.6 Generative Autoregressive Models 52
2.3.7 Generative Transformers and LLMs 54
2.3.8 Strategies to Address LLM Limitations 59
2.4 GenAI Applications to Canonical Problems in Communications Systems 63
2.4.1 Physical Layer Design 63
2.4.2 Network Resource Management 64
2.4.3 Network Traffic Analytics 65
2.4.4 Cross-layer Network Security 65
2.4.5 Localization and Positioning 66
2.5 Future Communication Frontiers for GMs 66
2.5.1 Semantic Communications 66
2.5.2 Integrated Sensing and Communications 67
2.5.3 Digital Twins 68
2.5.4 AI-generated Content for 6G Networks 69
2.5.5 MEC and EAI 69
2.5.6 Adversarial Machine Learning and Trustworthy AI 70
2.6 Regulation and Policy 71
2.7 Summary 71
References 72
3 Key Study Cases of Generative AI Applications to Communications Systems 79
Mehdi Letafati, Samad Ali, and Matti Latva-aho
3.1 Overview on the Roles of Generative AI in Communication Systems 79
3.1.1 Use Cases of Generative Adversarial Networks in Communications 79
3.1.2 Use cases of VAEs in Communications 81
3.1.3 Use-cases of Diffusion Models in Communications 82
3.2 Case Study: Diffusion Models in Wireless Communications 83
3.2.1 Working Mechanism of Diffusion Models 83
3.2.2 Case Study: Diffusion Models Applications for Data Reconstruction Enhancement in Communication Systems 89
3.3 Future Implications and Potential Impacts on Communication Systems 98
3.4 Chapter Summary 99
References 99
4 Generative AI at PHY Layer: Native AI or Trainable Radios 105
Eren Balevi
4.1 Wireless Communications Empowered with Generative Models 105
4.1.1 Motivations of GenAI at the PHY 105
4.1.2 Applications of GenAI at the PHY 106
4.2 Channel Modeling 110
4.2.1 Generative Channel Modeling 111
4.2.2 Site-specific Generative Models 115
4.3 Generative Channel Estimation 116
4.3.1 Narrowband Channel Estimation with Reduced Pilots 120
4.3.2 Wideband Channel Estimation with Reduced Pilots 122
4.4 Channel Compression 123
4.5 Beamforming 125
4.6 Summary 129
References 129
5 Generative AI at the MAC Layer 133
Kemal Davaslioglu, Ender Ayanoglu, and Yalin E. Sagduyu
5.1 Introduction 133
5.2 Generative Models 137
5.2.1 Variational Autoencoders 139
5.2.2 Generative Adversarial Networks 140
5.2.3 Diffusion Models 141
5.3 Spectrum Awareness Applications 142
5.3.1 Data Augmentation and Synthetic Data Generation 143
5.3.2 Signal Classification Applications - UAV Classification 147
5.3.3 Anomaly Detection in RF Spectrum 148
5.4 RF Spectrum Security Applications 149
5.4.1 Emitter Identification 149
5.4.2 Wireless Spoofing 151
5.4.3 Enhanced Jamming Attacks 152
5.5 Scheduling Applications 153
5.5.1 Traffic Prediction and Pattern Generation 153
5.5.2 Adaptive Scheduling Algorithms 154
5.5.3 Interference Patterns 154
5.5.4 Fairness and QoS 154
5.5.5 Millimeter-wave Networks 155
5.6 Open Problems and Future Research Directions 155
5.6.1 Reconfigurable Intelligent Surface (RIS)-assisted Networks 156
5.6.2 Spectrum Sharing in the Presence of Interference 157
5.6.3 Integrated Sensing and Communications (ISAC) 159
5.6.4 Link Scheduling in Large Networks 160
5.6.5 Enhancing Wireless MAC-layer Security 160
5.7 Concluding Remarks 162
References 162
6 Generative AI at Network Layer 169
Athanasios Karapantelakis, Pegah Alizadeh, Abdulrahman Alabbasi, Kaushik Dey, and Alexandros Nikou
6.1 Introduction 169
6.2 Network Layer in Mobile Networks 172
6.2.1 Radio Access Network 172
6.2.2 Core Network 175
6.3 Generative AI in the Network Layer 177
6.3.1 Introduction 177
6.3.2 Advantages of GenAI Models 177
6.3.3 Short-term Applications (GenAI for Network Layer) 180
6.3.4 Long-term Applications (Network Layer for GenAI) 184
6.4 Challenges and Opportunities for GenAI in the Network Layer 185
6.4.1 Challenges 185
6.4.2 Research Opportunities 186
6.5 Summary 187
References 187
7 Generative AI at Application Layer: Mobile AI-generated Content 191
Paria Mohammadzadeh Hesar, Amirhossein Mohammadi, and Hina Tabassum
7.1 Introduction to AIGC 191
7.1.1 General Overview 191
7.1.2 AIGC in the Application Layer 192
7.1.3 AIGC Product Lifecycle 194
7.2 Collaborative Network Infrastructure for Enabling GenAI Services 196
7.2.1 Enabling AIGC - Challenges 196
7.2.2 Infrastructure Components and Capabilities 198
7.2.3 Collaborative Edge-cloud Infrastructure 201
7.3 Network Resource Efficient GenAI Methods 203
7.3.1 Model Optimization Techniques 203
7.3.2 Service Optimization Methods 206
7.4 Security and Privacy at Application Layer 208
7.4.1 Security Threat Models and Privacy Risks 209
7.4.2 Ethical Considerations in AIGC services 211
7.4.3 Enabling Secure AIGC-as-a-Service 212
7.5 Use Cases of Mobile AIGC 213
7.5.1 AI-generated Content in Social Media 213
7.5.2 Immersive Streaming (AR/VR) 215
7.5.3 Personalized AI Services 219
7.6 Conclusion and Research Directions 222
7.7 Summary 223
References 224
8 Applications of GenAI on Wireless and Cybersecurity 239
Brian Kim, Yalin E. Sagduyu, Tugba Erpek, Yi Shi, and Sennur Ulukus
8.1 Introduction to GenAI in Wireless and Cybersecurity 239
8.2 Adversarial Machine Learning in Wireless Communications 242
8.2.1 Different Types of Attacks Against GenAI-driven Wireless Applications 243
8.2.2 Defense Against Adversarial Attacks for GenAI-driven Wireless Applications 245
8.3 GenAI for Wireless Security and Cybersecurity 245
8.3.1 GenAI for Wireless Security 245
8.3.2 GenAI for Cybersecurity 248
8.3.3 GenAI-driven Attacks Against Wireless and Cybersecurity Applications 249
8.4 Ethical Issues Related to GenAI for Wireless Communications and Cybersecurity 251
8.5 Summary 252
References 253
9 Challenges and Opportunities for Generative AI in Wireless Communications and Networking 261
Songyang Zhang and Zhi Ding
9.1 Introduction 261
9.2 Challenges of Applying Generative AI in Wireless Communications 262
9.2.1 Efficiency and Robustness 263
9.2.2 Cost and Complexity 267
9.2.3 Standardization, Regulation, and Policy 269
9.3 Adopting Generative AI in NextG Communications: Case Studies 270
9.3.1 Integration of Generative AI and Physical Communications Models 270
9.3.2 Trustworthy Generative AI for Distributed Wireless Communications 277
9.4 Summary 281
References 281
10 Future Research Directions 285
Nam H. Chu, Diep N. Nguyen, Dinh Thai Hoang, Octavia A. Dobre, Dusit Niyato, and Petar Popovski
10.1 Introduction 285
10.2 Emerging Foundational Research Frontiers 286
10.2.1 Dedicated GenAI Models for Communication Systems 287
10.2.2 Fusion of GenAI and Emerging Technologies 288
10.3 Enhancing Generative AI Models for Wireless Communication Systems 290
10.3.1 Model Optimization and Generalization 290
10.3.2 Energy Efficiency 291
10.3.3 Generative AI for Spectrum Management 292
10.3.4 AI-driven Network Management and Orchestration 293
10.3.5 Security and Privacy Concerns 295
10.4 Practical Case Studies 296
10.4.1 AI-powered Network Optimization by T-Mobile 296
10.4.2 DeepSig's Generative AI for Wireless Communications 297
10.5 Conclusion 297
References 298
Index 305