Communication Networks and Service Management in the Era of Artificial Intelligence and Machine Learning

List of Contributors xv Preface xxi Acknowledgments xxv Acronyms xxvii Part I Introduction 1 1 Overview of Network and Service Management 3 Marco Mellia, Nur Zincir-Heywood, and Yixin Diao 1.1 Network and Service Management at Large 3 1.2 Data Collection and Monitoring Protocols 5 1.2.1 SNMP Protocol Family 5 1.2.2 Syslog Protocol 5 1.2.3 IP Flow Information eXport (IPFIX) 6 1.2.4 IP Performance Metrics (IPPM) 7 1.2.5 Routing Protocols and Monitoring Platforms 8 1.3 Network Configuration Protocol 9 1.3.1 Standard Configuration Protocols and Approaches 9 1.3.2 Proprietary Configuration Protocols 10 1.3.3 Integrated Platforms for Network Monitoring 10 1.4 Novel Solutions and Scenarios 12 1.4.1 Software-Defined Networking - SDN 12 1.4.2 Network Functions Virtualization -NFV 14 Bibliography 15 2 Overview of Artificial Intelligence and Machine Learning 19 Nur Zincir-Heywood, Marco Mellia, and Yixin Diao 2.1 Overview 19 2.2 Learning Algorithms 20 2.2.1 Supervised Learning 21 2.2.2 Unsupervised Learning 22 2.2.3 Reinforcement Learning 23 2.3 Learning for Network and Service Management 24 Bibliography 26 Part II Management Models and Frameworks 33 3 Managing Virtualized Networks and Services with Machine Learning 35 Raouf Boutaba, Nashid Shahriar, Mohammad A. Salahuddin, and Noura Limam 3.1 Introduction 35 3.2 Technology Overview 37 3.2.1 Virtualization of Network Functions 38 3.2.1.1 Resource Partitioning 38 3.2.1.2 Virtualized Network Functions 40 3.2.2 Link Virtualization 41 3.2.2.1 Physical Layer Partitioning 41 3.2.2.2 Virtualization at Higher Layers 42 3.2.3 Network Virtualization 42 3.2.4 Network Slicing 43 3.2.5 Management and Orchestration 44 3.3 State-of-the-Art 46 3.3.1 Network Virtualization 46 3.3.2 Network Functions Virtualization 49 3.3.2.1 Placement 49 3.3.2.2 Scaling 52 3.3.3 Network Slicing 55 3.3.3.1 Admission Control 55 3.3.3.2 Resource Allocation 56 3.4 Conclusion and Future Direction 59 3.4.1 Intelligent Monitoring 60 3.4.2 Seamless Operation and Maintenance 60 3.4.3 Dynamic Slice Orchestration 61 3.4.4 Automated Failure Management 61 3.4.5 Adaptation and Consolidation of Resources 61 3.4.6 Sensitivity to Heterogeneous Hardware 62 3.4.7 Securing Machine Learning 62 Bibliography 63 4 Self-Managed 5G Networks 69 Jorge Martín-Pérez, Lina Magoula, Kiril Antevski, Carlos Guimarães, Jorge Baranda, Carla Fabiana Chiasserini, Andrea Sgambelluri, Chrysa Papagianni, Andrés García-Saavedra, Ricardo Martínez, Francesco Paolucci, Sokratis Barmpounakis, Luca Valcarenghi, Claudio EttoreCasetti, Xi Li, Carlos J. Bernardos, Danny De Vleeschauwer, Koen De Schepper, Panagiotis Kontopoulos, Nikolaos Koursioumpas, Corrado Puligheddu, Josep Mangues-Bafalluy, and Engin Zeydan 4.1 Introduction 69 4.2 Technology Overview 73 4.2.1 RAN Virtualization and Management 73 4.2.2 Network Function Virtualization 75 4.2.3 Data Plane Programmability 76 4.2.4 Programmable Optical Switches 77 4.2.5 Network Data Management 78 4.3 5G Management State-of-the-Art 80 4.3.1 RAN resource management 80 4.3.1.1 Context-Based Clustering and Profiling for User and Network Devices 80 4.3.1.2 Q-Learning Based RAN Resource Allocation 81 4.3.1.3 vrAIn: AI-Assisted Resource Orchestration for Virtualized Radio Access Networks 81 4.3.2 Service Orchestration 83 4.3.3 Data Plane Slicing and Programmable Traffic Management 85 4.3.4 Wavelength Allocation 86 4.3.5 Federation 88 4.4 Conclusions and Future Directions 89 Bibliography 92 5 AI in 5G Networks: Challenges and Use Cases 101 Stanislav Lange, Susanna Schwarzmann, Marija Gaji¿c, Thomas Zinner, and Frank A. Kraemer 5.1 Introduction 101 5.2 Background 103 5.2.1 ML in the Networking Context 103 5.2.2 ML in Virtualized Networks 104 5.2.3 ML for QoE Assessment and Management 104 5.3 Case Studies 105 5.3.1 QoE Estimation and Management 106 5.3.1.1 Main Challenges 107 5.3.1.2 Methodology 108 5.3.1.3 Results and Guidelines 109 5.3.2 Proactive VNF Deployment 110 5.3.2.1 Problem Statement and Main Challenges 111 5.3.2.2 Methodology 112 5.3.2.3 Evaluation Results and Guidelines 113 5.3.3 Multi-service, Multi-domain Interconnect 115 5.4 Conclusions and Future Directions 117 Bibliography 118 6 Machine Learning for Resource Allocation in Mobile Broadband Networks 123 Sadeq B. Melhem, Arjun Kaushik, Hina Tabassum, and Uyen T. Nguyen 6.1 Introduction 123 6.2 ML in Wireless Networks 124 6.2.1 Supervised ML 124 6.2.1.1 Classification Techniques 125 6.2.1.2 Regression Techniques 125 6.2.2 Unsupervised ML 126 6.2.2.1 Clustering Techniques 126 6.2.2.2 Soft Clustering Techniques 127 6.2.3 Reinforcement Learning 127 6.2.4 Deep Learning 128 6.2.5 Summary 129 6.3 ML-Enabled Resource Allocation 129 6.3.1 Power Control 131 6.3.1.1 Overview 131 6.3.1.2 State-of-the-Art 131 6.3.1.3 Lessons Learnt 132 6.3.2 Scheduling 132 6.3.2.1 Overview 132 6.3.2.2 State-of-the-Art 132 6.3.2.3 Lessons Learnt 134 6.3.3 User Association 134 6.3.3.1 Overview 134 6.3.3.2 State-of-the-Art 136 6.3.3.3 Lessons Learnt 136 6.3.4 Spectrum Allocation 136 6.3.4.1 Overview 136 6.3.4.2 State-of-the-Art 138 6.3.4.3 Lessons Learnt 138 6.4 Conclusion and Future Directions 140 6.4.1 Transfer Learning 140 6.4.2 Imitation Learning 140 6.4.3 Federated-Edge Learning 141 6.4.4 Quantum Machine Learning 142 Bibliography 142 7 Reinforcement Learning for Service Function Chain Allocation in Fog Computing 147 José Santos, Tim Wauters, Bruno Volckaert, and Filip De Turck 7.1 Introduction 147 7.2 Technology Overview 148 7.2.1 Fog Computing (FC) 149 7.2.2 Resource Provisioning 149 7.2.3 Service Function Chaining (SFC) 150 7.2.4 Micro-service Architecture 150 7.2.5 Reinforcement Learning (RL) 151 7.3 State-of-the-Art 152 7.3.1 Resource Allocation for Fog Computing 152 7.3.2 ML Techniques for Resource Allocation 153 7.3.3 RL Methods for Resource Allocation 154 7.4 A RL Approach for SFC Allocation in Fog Computing 155 7.4.1 Problem Formulation 155 7.4.2 Observation Space 156 7.4.3 Action Space 157 7.4.4 Reward Function 158 7.4.5 Agent 161 7.5 Evaluation Setup 162 7.5.1 Fog-Cloud Infrastructure 162 7.5.2 Environment Implementation 162 7.5.3 Environment Configuration 164 7.6 Results 165 7.6.1 Static Scenario 165 7.6.2 Dynamic Scenario 167 7.7 Conclusion and Future Direction 169 Bibliography 170 Part III Management Functions and Applications 175 8 Designing Algorithms for Data-Driven Network Management and Control: State-of-the-Art and Challenges 177 Andreas Blenk, Patrick Kalmbach, Johannes Zerwas, and Stefan Schmid 8.1 Introduction 177 8.1.1 Contributions 179 8.1.2 Exemplary Network Use Case Study 179 8.2 Technology Overview 181 8.2.1 Data-Driven Network Optimization 181 8.2.2 Optimization Problems over Graphs 182 8.2.3 From Graphs to ML/AI Input 184 8.2.4 End-to-End Learning 187 8.3 Data-Driven Algorithm Design: State-of-the Art 188 8.3.1 Data-Driven Optimization in General 188 8.3.2 Data-Driven Network Optimization 190 8.3.3 Non-graph Related Problems 192 8.4 Future Direction 193 8.4.1 Data Production and Collection 193 8.4.2 ML and AI Advanced Algorithms for Network Management with Performance Guarantees 194 8.5 Summary 194 Acknowledgments 195 Bibliography 195 9 AI-Driven Performance Management in Data-Intensive Applications 199 Ahmad Alnafessah, Gabriele Russo Russo, Valeria Cardellini, Giuliano Casale, and Francesco Lo Presti 9.1 Introduction 199 9.2 Data-Processing Frameworks 200 9.2.1 Apache Storm 200 9.2.2 Hadoop MapReduce 201 9.2.3 Apache Spark 202 9.2.4 Apache Flink 202 9.3 State-of-the-Art 203 9.3.1 Optimal Configuration 203 9.3.1.1 Traditional Approaches 203 9.3.1.2 AI Approaches 204 9.3.1.3 Example: AI-Based Optimal Configuration 206 9.3.2 Performance Anomaly Detection 207 9.3.2.1 Traditional Approaches 208 9.3.2.2 AI Approaches 208 9.3.2.3 Example: ANNs-Based Anomaly Detection 210 9.3.3 Load Prediction 211 9.3.3.1 Traditional Approaches 212 9.3.3.2 AI Approaches 212 9.3.4 Scaling Techniques 213 9.3.4.1 Traditional Approaches 213 9.3.4.2 AI Approaches 214 9.3.5 Example: RL-Based Auto-scaling Policies 214 9.4 Conclusion and Future Direction 216 Bibliography 217 10 Datacenter Traffic Optimization with Deep Reinforcement Learning 223 Li Chen, Justinas Lingys, Kai Chen, and Xudong Liao 10.1 Introduction 223 10.2 Technology Overview 225 10.2.1 Deep Reinforcement Learning (DRL) 226 10.2.2 Applying ML to Networks 227 10.2.3 Traffic Optimization Approaches in Datacenter 229 10.2.4 Example: DRL for Flow Scheduling 230 10.2.4.1 Flow Scheduling Problem 230 10.2.4.2 DRL Formulation 230 10.2.4.3 DRL Algorithm 231 10.3 State-of-the-Art: AuTO Design 231 10.3.1 Problem Identified 231 10.3.2 Overview 232 10.3.3 Peripheral System 233 10.3.3.1 Enforcement Module 233 10.3.3.2 Monitoring Module 234 10.3.4 Central System 234 10.3.5 DRL Formulations and Solutions 235 10.3.5.1 Optimizing MLFQ Thresholds 235 10.3.5.2 Optimizing Long Flows 239 10.4 Implementation 239 10.4.1 Peripheral System 239 10.4.1.1 Monitoring Module (MM): 240 10.4.1.2 Enforcement Module (EM): 240 10.4.2 Central System 241 10.4.2.1 sRLA 241 10.4.2.2 lRLA 242 10.5 Experimental Results 242 10.5.1 Setting 243 10.5.2 Comparison Targets 244 10.5.3 Experiments 244 10.5.3.1 Homogeneous Traffic 244 10.5.3.2 Spatially Heterogeneous Traffic 245 10.5.3.3 Temporally and Spatially Heterogeneous Traffic 246 10.5.4 Deep Dive 247 10.5.4.1 Optimizing MLFQ Thresholds using DRL 247 10.5.4.2 Optimizing Long Flows using DRL 248 10.5.4.3 System Overhead 249 10.6 Conclusion and Future Directions 251 Bibliography 253 11 The New Abnormal: Network Anomalies in the AI Era 261 Francesca Soro, Thomas Favale, Danilo Giordano, Luca Vassio, Zied Ben Houidi, and Idilio Drago 11.1 Introduction 261 11.2 Definitions and Classic Approaches 262 11.2.1 Definitions 263 11.2.2 Anomaly Detection: A Taxonomy 263 11.2.3 Problem Characteristics 264 11.2.4 Classic Approaches 266 11.3 AI and Anomaly Detection 267 11.3.1 Methodology 267 11.3.2 Deep Neural Networks 268 11.3.3 Representation Learning 270 11.3.4 Autoencoders 271 11.3.5 Generative Adversarial Networks 272 11.3.6 Reinforcement Learning 274 11.3.7 Summary and Takeaways 275 11.4 Technology Overview 277 11.4.1 Production-Ready Tools 277 11.4.2 Research Alternatives 279 11.4.3 Summary and Takeaways 280 11.5 Conclusions and Future Directions 282 Bibliography 283 12 Automated Orchestration of Security Chains Driven by Process Learning 289 Nicolas Schnepf, Rémi Badonnel, Abdelkader Lahmadi, and Stephan Merz 12.1 Introduction 289 12.2 RelatedWork 290 12.2.1 Chains of Security Functions 291 12.2.2 Formal Verification of Networking Policies 292 12.3 Background 294 12.3.1 Flow-Based Detection of Attacks 294 12.3.2 Programming SDN Controllers 295 12.4 Orchestration of Security Chains 296 12.5 Learning Network Interactions 298 12.6 Synthesizing Security Chains 301 12.7 Verifying Correctness of Chains 306 12.7.1 Packet Routing 306 12.7.2 Shadowing Freedom and Consistency 306 12.8 Optimizing Security Chains 308 12.9 Performance Evaluation 311 12.9.1 Complexity of Security Chains 312 12.9.2 Response Times 313 12.9.3 Accuracy of Security Chains 313 12.9.4 Overhead Incurred by Deploying Security Chains 314 12.10 Conclusions 315 Bibliography 316 13 Architectures for Blockchain-IoT Integration 321 Sina Rafati Niya, Eryk Schiller, and Burkhard Stiller 13.1 Introduction 321 13.1.1 Blockchain Basics 323 13.1.2 Internet-of-Things (IoT) Basics 324 13.2 Blockchain-IoT Integration (BIoT) 325 13.2.1 BIoT Potentials 326 13.2.2 BIoT Use Cases 328 13.2.3 BIoT Challenges 329 13.2.3.1 Scalability 332 13.2.3.2 Security 333 13.2.3.3 Energy Efficiency 334 13.2.3.4 Manageability 335 13.3 BIoT Architectures 335 13.3.1 Cloud, Fog, and Edge-Based Architectures 337 13.3.2 Software-Defined Architectures 337 13.3.3 A Potential Standard BIoT Architecture 338 13.4 Summary and Considerations 341 Bibliography 342 Index 345