Federated Learning for Future Intelligent Wireless Networks
Description
Explore the concepts, algorithms, and applications underlying federated learning
In Federated Learning for Future Intelligent Wireless Networks, a team of distinguished researchers deliver a robust and insightful collection of resources covering the foundational concepts and algorithms powering federated learning, as well as explanations of how they can be used in wireless communication systems. The editors have included works that examine how communication resource provision affects federated learning performance, accuracy, convergence, scalability, and security and privacy.
Readers will explore a wide range of topics that show how federated learning algorithms, concepts, and design and optimization issues apply to wireless communications. Readers will also find:
* A thorough introduction to the fundamental concepts and algorithms of federated learning, including horizontal, vertical, and hybrid FL
* Comprehensive explorations of wireless communication network design and optimization for federated learning
* Practical discussions of novel federated learning algorithms and frameworks for future wireless networks
* Expansive case studies in edge intelligence, autonomous driving, IoT, MEC, blockchain, and content caching and distribution
Perfect for electrical and computer science engineers, researchers, professors, and postgraduate students with an interest in machine learning, Federated Learning for Future Intelligent Wireless Networks will also benefit regulators and institutional actors responsible for overseeing and making policy in the area of artificial intelligence.
More details
Other editions
Additional editions
Persons
Yao Sun, PhD, is a Lecturer with the University of Glasgow in the United Kingdom. He was a former Research Fellow at UESTC in Chengdu, China.
Chaoqun You is a Research Fellow at the Singapore University of Technology and Design. She was formerly an Academic Guest with the Department of Electronic Computer Engineering at the University of Toronto.
Gang Feng is a Professor at the University of Electronic Science and Technology of China. He was an Associate Professor at Nanyang Technological University.
Lei Zhang, PhD, is a Professor at the University of Glasgow, UK. He was formerly a Research Fellow at the 5G Innovation Centre at the University of Surrey.
Content
About the Editors xv
Preface xvii
1 Federated Learning with Unreliable Transmission in Mobile Edge Computing Systems 1
Chenyuan Feng, Daquan Feng, Zhongyuan Zhao, Howard H. Yang, and Tony Q. S. Quek
1.1 System Model 1
1.2 Problem Formulation 4
1.3 A Joint Optimization Algorithm 10
1.4 Simulation and Experiment Results 16
2 Federated Learning with non-IID data in Mobile Edge Computing Systems 23
Chenyuan Feng, Daquan Feng, Zhongyuan Zhao, Geyong Min, and Hancong Duan
2.1 System Model 23
2.2 Performance Analysis and Averaging Design 24
2.3 Data Sharing Scheme 30
2.4 Simulation Results 42
3 How Many Resources Are Needed to Support Wireless Edge Networks 49
Yi-Jing Liu, Gang Feng, Yao Sun, and Shuang Qin
3.1 Introduction 49
3.2 System Model 50
3.3 Wireless Bandwidth and Computing Resources Consumed for Supporting FL-EnabledWireless Edge Networks 54
3.4 The Relationship between FL Performance and Consumed Resources 59
3.5 Discussions of Three Cases 62
3.6 Numerical Results and Discussion 67
3.7 Conclusion 75
3.8 Proof of Corollary 3.2 76
3.9 Proof of Corollary 3.3 77
4 Device Association Based on Federated Deep Reinforcement Learning for Radio Access Network Slicing 85
Yi-Jing Liu, Gang Feng, Yao Sun, and Shuang Qin
4.1 Introduction 85
4.2 System Model 87
4.3 Problem Formulation 90
4.4 Hybrid Federated Deep Reinforcement Learning for Device Association 94
4.5 Numerical Results 103
4.6 Conclusion 109
5 Deep Federated Learning Based on Knowledge Distillation and Differential Privacy 113
Hui Lin, Feng Yu, and Xiaoding Wang
5.1 Introduction 113
5.2 RelatedWork 115
5.3 System Model 118
5.4 The Implementation Details of the Proposed Strategy 119
5.5 Performance Evaluation 120
5.6 Conclusions 122
6 Federated Learning-Based Beam Management in Dense Millimeter Wave Communication Systems 127
Qing Xue and Liu Yang
6.1 Introduction 127
6.2 System Model 130
6.3 Problem Formulation and Analysis 133
6.4 FL-Based Beam Management in UDmmN 135
6.6 Conclusions 150
7 Blockchain-Empowered Federated Learning Approach for An Intelligent and Reliable D2D Caching Scheme 155
Runze Cheng, Yao Sun, Yijing Liu, Le Xia, Daquan Feng, and Muhammad Imran
7.1 Introduction 155
7.2 RelatedWork 157
7.3 System Model 159
7.4 Problem Formulation and DRL-Based Model Training 160
7.5 Privacy-Preserved and Secure BDRFL Caching Scheme Design 165
7.6 Consensus Mechanism and Federated Learning Model Update 170
7.7 Simulation Results and Discussions 173
7.8 Conclusion 177
8 Heterogeneity-Aware Dynamic Scheduling for Federated Edge Learning 181
Kun Guo, Zihan Chen, Howard H. Yang, and Tony Q. S. Quek
8.1 Introduction 181
8.2 RelatedWorks 184
8.3 System Model for FEEL 185
8.4 Heterogeneity-Aware Dynamic Scheduling Problem Formulation 189
8.5 Dynamic Scheduling Algorithm Design and Analysis 192
8.6 Evaluation Results 197
8.7 Conclusions 208
8.A.1 Proof of Theorem 8.2 208
8.A.2 Proof of Theorem 8.3 209
9 Robust Federated Learning with Real-World Noisy Data 215
Jingyi Xu, Zihan Chen, Tony Q. S. Quek, and Kai Fong Ernest Chong
9.1 Introduction 215
9.2 RelatedWork 217
9.3 FedCorr 219
9.4 Experiments 226
9.5 Further Remarks 232
10 Analog Over-the-Air Federated Learning: Design and Analysis 239
Howard H. Yang, Zihan Chen, and Tony Q. S. Quek
10.1 Introduction 239
10.2 System Model 241
10.3 Analog Over-the-Air Model Training 242
10.4 Convergence Analysis 245
10.5 Numerical Results 250
10.6 Conclusion 253
11 Federated Edge Learning for Massive MIMO CSI Feedback 257
Shi Jin, Yiming Cui, and Jiajia Guo
11.1 Introduction 257
11.2 System Model 259
11.3 FEEL for DL-Based CSI Feedback 260
11.4 Simulation Results 264
11.5 Conclusion 268
12 User-Centric Decentralized Federated Learning for Autoencoder-Based CSI Feedback 273
Shi Jin, Jiajia Guo, Yan Lv, and Yiming Cui
12.1 Autoencoder-Based CSI Feedback 273
12.2 User-Centric Online Training for AE-Based CSI Feedback 275
12.3 Multiuser Online Training Using Decentralized Federated Learning 279
12.4 Numerical Results 283
12.5 Conclusion 287
Bibliography 287
Index 291
Preface
It has been considered one of the key missing components in the existing 5G network and is widely recognized as one of the most sought-after functions for next-generation 6G communication systems. Nowadays, there are more than 10 billion Internet-of-Things (IoT) equipment and 5 billion smartphones that are equipped with artificial intelligence (AI)-empowered computing modules such as AI chips and GPU. On the one hand, the user equipment (UE) can be potentially deployed as computing nodes to process certain emerging service tasks such as crowdsensing tasks and collaborative tasks, which paves the way for applying AI in edge networks. On the other hand, in the paradigm of machine learning (ML), the powerful computing capability on these UEs can decouple ML from acquiring, storing, and training data in data centers as conventional methods.
Federated learning (FL) has been widely acknowledged as one of the most essential enablers to bring network edge intelligence into reality, as it can enable collaborative training of ML models while enhancing individual user privacy and data security. Empowered by the growing computing capabilities of UEs, FL trains ML models locally on each device where the raw data never leaves the device. Specifically, FL uses an iterative approach that requires a number of global iterations to achieve a global model accuracy. In each global iteration, UEs take a number of local iterations up to a local model accuracy. As a result, the implementation of FL at edge networks can also decrease the costs of transmitting raw data, relieve the burden on backbone networks, reduce the latency for real-time decisions.
This book would explore recent advances in the theory and practice of FL, especially when it is applied to wireless communication systems. In detail, the book covers the following aspects:
- 1) principles and fundamentals of FL;
- 2) performance analysis of FL in wireless communication systems;
- 3) how future wireless networks (say 6G networks) enable FL as well as how FL frameworks/algorithms can be optimized when applying to wireless networks (6G);
- 4) FL applications to vertical industries and some typical communication scenarios.
Chapter 1 investigates the optimization design of FL in the edge network. First, an optimization problem is formulated to manage the trade-off between model accuracy and training cost. Second, a joint optimization algorithm is designed to optimize the model compression, sample selection, and user selection strategies, which can approach a stationary optimal solution in a computationally efficient way. Finally, the performance of the proposed optimization scheme is evaluated by numerical simulation and experiment results, which show that both the accuracy loss and the cost of FL in the edge network can be reduced significantly by employing the proposed algorithm.
Chapter 2 studies non-IID data model for FL, derives a theoretical upper bound, and redesigns the federated averaging scheme to reduce the weight difference. To further mitigate the impact of non-IID data, a data-sharing scheme is designed to jointly minimize the accuracy loss, the energy consumption, and latency with constrained resource of edge systems. Then a computation-efficient algorithm is proposed to approach the optimal solution and provide the experiment results to evaluate our proposed schemes.
Chapter 3 theoretically analyzes the performance and cost of running FL, which is imperative to deeply understand the relationship between FL performance and multiple-dimensional resources. In this chapter, we construct an analytical model to investigate the relationship between the FL model accuracy and consumed resources in FL-enabled wireless edge networks. Based on the analytical model, we explicitly quantify the model accuracy, computing resources, and communication resources. Numerical results validate the effectiveness of our theoretical modeling and analysis and demonstrate the trade-off between the communication and computing resources for achieving a certain model accuracy.
Chapter 4 proposes an efficient device association scheme for radio access network (RAN) slicing by exploiting a federated reinforcement learning framework, with the aim to improve network throughput, while guaranteeing user privacy and data security. Specially, we use deep reinforcement learning to train local models on UEs under a hybrid FL framework, where horizontally FL is employed for parameter aggregation on BS, while vertically FL is employed for access selection aggregation on the encrypted party. Numerical results show that our proposed scheme can achieve significant performance gains in terms of network throughput and communication efficiency in comparison with some known state-of-the-art solutions.
Chapter 5 proposes a deep FL algorithm that utilizes knowledge distillation and differential privacy to safeguard privacy during the data fusion process. Our approach involves adding Gaussian noise at different stages of knowledge distillation-based FL to ensure privacy protection. Our experimental results demonstrate that this strategy provides better privacy preservation while achieving high-precision IoT data fusion.
Chapter 6 presents a novel systematic beam control scheme to tackle the formulated beam management problem, which is difficult due to the nonconvex objective function. The double deep Q-network (DDQN) under a FL framework is employed to solve the above optimization problem, thereby fulfilling adaptive and intelligent beam management in mmwave networks. In the proposed beam management scheme based on federated learning (BMFL), the non-raw-data aggregation can theoretically protect user privacy while reducing handoff costs. Moreover, a data cleaning technique is used before the local model training, with the aim to further strengthen the privacy protection while improving the learning convergence speed. Simulation results demonstrate the performance gain of the proposed BMFL scheme.
Chapter 7 proposes a double-layer blockchain-based deep reinforcement federated learning (BDRFL) scheme to ensure privacy-preserved and caching-efficient D2D networks. In BDRFL, a double-layer blockchain is utilized to further enhance data security. Simulation results first verify the convergence of BDRFL-based algorithm and then demonstrate that the download latency of the BDRFL-based caching scheme can be significantly reduced under different types of attacks when compared with some existing caching policies.
Chapter 8 aims to design a dynamic scheduling policy to explore the spectrum flexibility for heterogeneous federated edge learning (FEEL) so as to facilitate the distributed intelligence in edge networks. This chapter proposes a heterogeneity-aware dynamic scheduling problem to minimize the global loss function, with consideration of straggler and limited device energy issues. By solving the formulated problem, we propose a dynamic scheduling algorithm (DISCO), to make an intelligent decision on the set and order of scheduled devices in each communication round. Theoretical analysis reveals that under certain conditions, learning performance and energy constraints can be guaranteed in the DISCO. Finally, we demonstrate the superiority of the DISCO through numerical and experimental results, respectively.
Chapter 9 discusses FedCorr, a general multistage framework to tackle heterogeneous label noise in FL, which does not make any assumptions on the noise models of local clients while still maintaining client data privacy. Both theoretical analysis and experiment results demonstrate the performance gain of this novel FL framework.
Chapter 10 provides a general overview of the analog over-the-air federated learning (AirFL) system. Specially, we illustrate the general system architecture and highlight the salient feature of AirFL that adopts analog transmissions for fast (but noisy) aggregation of intermediate parameters. Then, we establish a new convergence analysis framework that takes into account the effects of fading and interference noise. Our analysis unveils the impacts from the intrinsic properties of wireless transmissions on the convergence performance of AirFL. The theoretical findings are corroborated by extensive simulations.
Chapter 11 investigates a FEEL-based training framework to DL-based channel state information (CSI) feedback. In FEEL, each UE trains an autoencoder network locally and exchanges model parameters via the base station. Therefore, data privacy is better protected compared with centralized learning because the local CSI datasets are not required to be uploaded. Neural network parameter quantization is then introduced to the FEEL-based training framework to reduce communication overhead. The simulation results indicate that the proposed FEEL-based training framework can achieve comparable performance with centralized learning.
Chapter 12 proposes a user-centric online training strategy in which the UE can collect CSI samples in the stable area and adjust the pretrained encoder online to further improve CSI reconstruction accuracy. Moreover, the proposed online training framework is extended to the multiuser scenario to improve performance sequentially. The key idea is to adopt decentralized FL without BS participation to combine the sharing of channel knowledge among UEs, which is called crowd intelligence....
