Reinforcement Learning for Cyber-Physical Systems
with Cybersecurity Case Studies
Published on 22. February 2019
256 pages
978-1-351-00661-3 (ISBN)
Description
Reinforcement Learning for Cyber-Physical Systems: with Cybersecurity Case Studies was inspired by recent developments in the fields of reinforcement learning (RL) and cyber-physical systems (CPSs). Rooted in behavioral psychology, RL is one of the primary strands of machine learning. Different from other machine learning algorithms, such as supervised learning and unsupervised learning, the key feature of RL is its unique learning paradigm, i.e., trial-and-error. Combined with the deep neural networks, deep RL become so powerful that many complicated systems can be automatically managed by AI agents at a superhuman level. On the other hand, CPSs are envisioned to revolutionize our society in the near future. Such examples include the emerging smart buildings, intelligent transportation, and electric grids.
However, the conventional hand-programming controller in CPSs could neither handle the increasing complexity of the system, nor automatically adapt itself to new situations that it has never encountered before. The problem of how to apply the existing deep RL algorithms, or develop new RL algorithms to enable the real-time adaptive CPSs, remains open. This book aims to establish a linkage between the two domains by systematically introducing RL foundations and algorithms, each supported by one or a few state-of-the-art CPS examples to help readers understand the intuition and usefulness of RL techniques.
Features
Introduces reinforcement learning, including advanced topics in RL
Applies reinforcement learning to cyber-physical systems and cybersecurity
Contains state-of-the-art examples and exercises in each chapter
Provides two cybersecurity case studies
Reinforcement Learning for Cyber-Physical Systems with Cybersecurity Case Studies is an ideal text for graduate students or junior/senior undergraduates in the fields of science, engineering, computer science, or applied mathematics. It would also prove useful to researchers and engineers interested in cybersecurity, RL, and CPS. The only background knowledge required to appreciate the book is a basic knowledge of calculus and probability theory.
However, the conventional hand-programming controller in CPSs could neither handle the increasing complexity of the system, nor automatically adapt itself to new situations that it has never encountered before. The problem of how to apply the existing deep RL algorithms, or develop new RL algorithms to enable the real-time adaptive CPSs, remains open. This book aims to establish a linkage between the two domains by systematically introducing RL foundations and algorithms, each supported by one or a few state-of-the-art CPS examples to help readers understand the intuition and usefulness of RL techniques.
Features
Introduces reinforcement learning, including advanced topics in RL
Applies reinforcement learning to cyber-physical systems and cybersecurity
Contains state-of-the-art examples and exercises in each chapter
Provides two cybersecurity case studies
Reinforcement Learning for Cyber-Physical Systems with Cybersecurity Case Studies is an ideal text for graduate students or junior/senior undergraduates in the fields of science, engineering, computer science, or applied mathematics. It would also prove useful to researchers and engineers interested in cybersecurity, RL, and CPS. The only background knowledge required to appreciate the book is a basic knowledge of calculus and probability theory.
More details
Language
English
Place of publication
London
United Kingdom
Publishing group
Taylor & Francis Ltd
Target group
Professional and scholarly
College/higher education
Illustrations
4 Tables, black and white; 64 Illustrations, black and white
File size
4,90 MB
ISBN-13
978-1-351-00661-3 (9781351006613)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions
Chong Li | Meikang Qiu
Reinforcement Learning for Cyber-Physical Systems
with Cybersecurity Case Studies
Book
09/2020
1st Edition
Chapman & Hall/CRC
€69.50
Shipment within 15-20 days
Chong Li | Meikang Qiu
Reinforcement Learning for Cyber-Physical Systems
with Cybersecurity Case Studies
Book
02/2019
1st Edition
CRC Press
€116.30
Shipment within 10-20 days
Persons
Chong Li is co-founder of Nakamoto \& Turing Labs Inc. He is Chief Architect and Head of Research at Canonchain Network. He is also an adjunct assistant professor at Columbia University. Dr. Li was a staff research engineer in the department of corporate R&D at Qualcomm Technologies. He received a B.E. in Electronic Engineering and Information Science from Harbin Institute of Technology and a Ph.D in Electrical and Computer Engineering from Iowa State University.
Dr. Li's research interests include information theory, machine learning, blockchain, networked control and communications, coding theory, PHY/MAC design for 5G technology and beyond. Dr. Li has published many technical papers in top-ranked journals, including Proceedings of the IEEE, IEEE Transactions on Information Theory, IEEE Communications Magazine, Automatica, etc. He has served as session chair and technical program committee for a number of international conferences. He has also served as reviewer for many prestigious journals and international conferences, including IEEE Transactions on Information Theory, IEEE Transactions on Wireless Communication, ISIT, CDC, ICC, WCNC, Globecom, etc. He holds 200+ international and U.S. patents (granted and pending) and received several academic awards including the MediaTek Inc. and Wu Ta You Scholar Award, the Rosenfeld International Scholarship and Iowa State Research Excellent Award. At Qualcomm, Dr. Li significantly contributed to the systems design and the standardization of several emerging key technologies, including LTE-D, LTE-controlled WiFi and 5G. At Columbia University, he has been instructing graduate-level courses, such as reinforcement learning, blockchain technology and convex optimization, and actively conducting research in the related field. Recently, Dr. Li has been driving the research and development of blockchain-based geo-distributed shared computing, and managing the patent-related business at Canonchain.
Meikang Qiu received the BE and ME degrees from Shanghai Jiao Tong University and received Ph.D. degree of Computer Science from University of Texas at Dallas. Currently, he is an Adjunct Professor at Columbia University and Associate Professor of Computer Science at Pace University. He is an IEEE Senior member and ACM Senior member. He is the Chair of IEEE Smart Computing Technical Committee. His research interests include cyber security, cloud computing, big data storage, hybrid memory, heterogeneous systems, embedded systems, operating systems, optimization, intelligent systems, sensor networks, etc.
Dr. Li's research interests include information theory, machine learning, blockchain, networked control and communications, coding theory, PHY/MAC design for 5G technology and beyond. Dr. Li has published many technical papers in top-ranked journals, including Proceedings of the IEEE, IEEE Transactions on Information Theory, IEEE Communications Magazine, Automatica, etc. He has served as session chair and technical program committee for a number of international conferences. He has also served as reviewer for many prestigious journals and international conferences, including IEEE Transactions on Information Theory, IEEE Transactions on Wireless Communication, ISIT, CDC, ICC, WCNC, Globecom, etc. He holds 200+ international and U.S. patents (granted and pending) and received several academic awards including the MediaTek Inc. and Wu Ta You Scholar Award, the Rosenfeld International Scholarship and Iowa State Research Excellent Award. At Qualcomm, Dr. Li significantly contributed to the systems design and the standardization of several emerging key technologies, including LTE-D, LTE-controlled WiFi and 5G. At Columbia University, he has been instructing graduate-level courses, such as reinforcement learning, blockchain technology and convex optimization, and actively conducting research in the related field. Recently, Dr. Li has been driving the research and development of blockchain-based geo-distributed shared computing, and managing the patent-related business at Canonchain.
Meikang Qiu received the BE and ME degrees from Shanghai Jiao Tong University and received Ph.D. degree of Computer Science from University of Texas at Dallas. Currently, he is an Adjunct Professor at Columbia University and Associate Professor of Computer Science at Pace University. He is an IEEE Senior member and ACM Senior member. He is the Chair of IEEE Smart Computing Technical Committee. His research interests include cyber security, cloud computing, big data storage, hybrid memory, heterogeneous systems, embedded systems, operating systems, optimization, intelligent systems, sensor networks, etc.
Content
Section I Introduction
Chapter 1 ? Overview of Reinforcement Learning
Chapter 2 ? Overview of CyberPhysical Systems and Cybersecurity
Section II Reinforcement Learning for Cyber-Physical Systems
Chapter 3 ? Reinforcement Learning Problems
Chapter 4 ? Modelbased Reinforcement Learning
Chapter 5 ? Modelfree Reinforcement Learning
Chapter 6 ? Deep Reinforcement Learning
Section III Case Studies
Chapter 7 ? Reinforcement Learning for Cybersecurity
Chapter 8 ? Case Study: Online CyberAttack Detection in Smart Grid
Chapter 9 ? Case Study: Defeat Maninthemiddle Attack
Chapter 1 ? Overview of Reinforcement Learning
Chapter 2 ? Overview of CyberPhysical Systems and Cybersecurity
Section II Reinforcement Learning for Cyber-Physical Systems
Chapter 3 ? Reinforcement Learning Problems
Chapter 4 ? Modelbased Reinforcement Learning
Chapter 5 ? Modelfree Reinforcement Learning
Chapter 6 ? Deep Reinforcement Learning
Section III Case Studies
Chapter 7 ? Reinforcement Learning for Cybersecurity
Chapter 8 ? Case Study: Online CyberAttack Detection in Smart Grid
Chapter 9 ? Case Study: Defeat Maninthemiddle Attack
