An Introduction to Self-adaptive Systems
Description
A concise and practical introduction to the foundations and engineering principles of self-adaptation
Though it has recently gained significant momentum, the topic of self-adaptation remains largely under-addressed in academic and technical literature. This book changes that. Using a systematic and holistic approach, An Introduction to Self-adaptive Systems: A Contemporary Software Engineering Perspective provides readers with an accessible set of basic principles, engineering foundations, and applications of self-adaptation in software-intensive systems.
It places self-adaptation in the context of techniques like uncertainty management, feedback control, online reasoning, and machine learning while acknowledging the growing consensus in the software engineering community that self-adaptation will be a crucial enabling feature in tackling the challenges of new, emerging, and future systems.
The author combines cutting-edge technical research with basic principles and real-world insights to create a practical and strategically effective guide to self-adaptation. He includes features such as:
- An analysis of the foundational engineering principles and applications of self-adaptation in different domains, including the Internet-of-Things, cloud computing, and cyber-physical systems
- End-of-chapter exercises at four different levels of complexity and difficulty
- An accompanying author-hosted website with slides, selected exercises and solutions, models, and code
Perfect for researchers, students, teachers, industry leaders, and practitioners in fields that directly or peripherally involve software engineering, as well as those in academia involved in a class on self-adaptivity, this book belongs on the shelves of anyone with an interest in the future of software and its engineering.
More details
Other editions
Additional editions
Person
DANNY WEYNS, PHD, is a Professor at Katholieke Universiteit (KU) Leuven, Department of Computer Science, Leuven, Belgium. He obtained his doctorate from KU Leuven. He focuses on software engineering of trustworthy self-adaptive systems, exploiting design models and verification techniques at runtime.
Content
Foreword xi
Acknowledgments xv
Acronyms xvii
Introduction xix
1 Basic Principles of Self-Adaptation and Conceptual Model 1
1.1 Principles of Self-Adaptation 2
1.2 Other Adaptation Approaches 4
1.3 Scope of Self-Adaptation 5
1.4 Conceptual Model of a Self-Adaptive System 5
1.4.1 Environment 5
1.4.2 Managed System 7
1.4.3 Adaptation Goals 8
1.4.4 Feedback Loop 8
1.4.5 Conceptual Model Applied 10
1.5 A Note on Model Abstractions 11
1.6 Summary 11
1.7 Exercises 12
1.8 Bibliographic Notes 14
2 Engineering Self-Adaptive Systems: A Short Tour in Seven Waves 17
2.1 Overview of the Waves 18
2.2 Contributions Enabled by the Waves 20
2.3 Waves Over Time with Selected Work 20
2.4 Summary 22
2.5 Bibliographic Notes 23
3 Internet-of-Things Application 25
3.1 Technical Description 25
3.2 Uncertainties 28
3.3 Quality Requirements and Adaptation Problem 29
3.4 Summary 29
3.5 Exercises 30
3.6 Bibliographic Notes 31
4 Wave I: Automating Tasks 33
4.1 Autonomic Computing 34
4.2 Utility Functions 35
4.3 Essential Maintenance Tasks for Automation 37
4.3.1 Self-Optimization 37
4.3.2 Self-Healing 38
4.3.3 Self-Protection 40
4.3.4 Self-Configuration 42
4.4 Primary Functions of Self-Adaptation 43
4.4.1 Knowledge 44
4.4.2 Monitor 46
4.4.3 Analyzer 47
4.4.4 Planner 49
4.4.5 Executor 51
4.5 Software Evolution and Self-Adaptation 52
4.5.1 Software Evolution Management 53
4.5.2 Self-Adaptation Management 54
4.5.3 Integrating Software Evolution and Self-Adaptation 55
4.6 Summary 56
4.7 Exercises 59
4.8 Bibliographic Notes 60
5 Wave II: Architecture-based Adaptation 63
5.1 Rationale for an Architectural Perspective 64
5.2 Three-Layer Model for Self-Adaptive Systems 66
5.2.1 Component Control 67
5.2.2 Change Management 67
5.2.3 Goal Management 68
5.2.4 Three-Layer Model Applied to DeltaIoT 68
5.2.5 Mapping Between the Three-Layer Model and the Conceptual Model for Self-Adaptation 70
5.3 Reasoning about Adaptation using an Architectural Model 70
5.3.1 Runtime Architecture of Architecture-based Adaptation 71
5.3.2 Architecture-based Adaptation of the Web-based Client-Server System 73
5.4 Comprehensive Reference Model for Self-Adaptation 75
5.4.1 Reflection Perspective on Self-Adaptation 76
5.4.2 MAPE-K Perspective on Self-Adaptation 78
5.4.3 Distribution Perspective on Self-Adaptation 79
5.5 Summary 83
5.6 Exercises 84
5.7 Bibliographic Notes 87
6 Wave III: Runtime Models 89
6.1 What is a Runtime Model? 90
6.2 Causality and Weak Causality 90
6.3 Motivations for Runtime Models 91
6.4 Dimensions of Runtime Models 92
6.4.1 Structural versus Behavioral 93
6.4.2 Declarative versus Procedural 94
6.4.3 Functional versus Qualitative 95
6.4.3.1 Functional Models 95
6.4.3.2 Quality Models 95
6.4.4 Formal versus Informal 98
6.5 Principal Strategies for Using Runtime Models 101
6.5.1 MAPE Components Share K Models 101
6.5.2 MAPE Components Exchange K Models 103
6.5.2.1 Runtime Models 103
6.5.2.2 Components of the Managing System 104
6.5.3 MAPE Models Share K Models 105
6.6 Summary 108
6.7 Exercises 109
6.8 Bibliographic Notes 114
7 Wave IV: Requirements-driven Adaptation 115
7.1 Relaxing Requirements for Self-Adaptation 116
7.1.1 Specification Language to Relax Requirements 116
7.1.1.1 Language Operators for Handling Uncertainty 116
7.1.1.2 Semantics of Language Primitives 118
7.1.2 Operationalization of Relaxed Requirements 118
7.1.2.1 Handing Uncertainty 118
7.1.2.2 Requirements Reflection and Mitigation Mechanisms 119
7.1.2.3 A Note on the Realization of Requirements Reflection 121
7.2 Meta-Requirements for Self-Adaptation 122
7.2.1 Awareness Requirements 123
7.2.2 Evolution Requirements 124
7.2.3 Operationalization of Meta-requirements 126
7.3 Functional Requirements of Feedback Loops 127
7.3.1 Design and Verify Feedback Loop Model 128
7.3.2 Deploy and Execute Verified Feedback Loop Model 130
7.4 Summary 131
7.5 Exercises 132
7.6 Bibliographic Notes 134
8 Wave V: Guarantees Under Uncertainties 137
8.1 Uncertainties in Self-Adaptive Systems 139
8.2 Taming Uncertainty with Formal Techniques 141
8.2.1 Analysis of Adaptation Options 141
8.2.2 Selection of Best Adaptation Option 143
8.3 Exhaustive Verification to Provide Guarantees for Adaptation Goals 144
8.4 Statistical Verification to Provide Guarantees for Adaptation Goals 149
8.5 Proactive Decision-Making using Probabilistic Model Checking 154
8.6 A Note on Verification and Validation 160
8.7 Integrated Process to Tame Uncertainty 160
8.7.1 Stage I: Implement and Verify the Managing System 161
8.7.2 Stage II: Deploy the Managing System 162
8.7.3 Stage III: Verify Adaptation Options, Decide, and Adapt 163
8.7.4 Stage IV: Evolve Adaptation Goals and Managing System 163
8.8 Summary 164
8.9 Exercises 165
8.10 Bibliographic Notes 168
9 Wave VI: Control-based Software Adaptation 171
9.1 A Brief Introduction to Control Theory 173
9.1.1 Controller Design 174
9.1.2 Control Properties 175
9.1.3 SISO and MIMO Control Systems 176
9.1.4 Adaptive Control 177
9.2 Automatic Construction of SISO Controllers 177
9.2.1 Phases of Controller Construction and Operation 178
9.2.2 Model Updates 179
9.2.3 Formal Guarantees 181
9.2.4 Example: Geo-Localization Service 183
9.3 Automatic Construction of MIMO Controllers 184
9.3.1 Phases of Controller Construction and Operation 184
9.3.2 Formal Guarantees 186
9.3.3 Example: Unmanned Underwater Vehicle 186
9.4 Model Predictive Control 189
9.4.1 Controller Construction and Operation 189
9.4.2 Formal Assessment 191
9.4.3 Example: Video Compression 192
9.5 A Note on Control Guarantees 194
9.6 Summary 194
9.7 Exercises 196
9.8 Bibliographic Notes 199
10 Wave VII: Learning from Experience 201
10.1 Keeping Runtime Models Up-to-Date Using Learning 203
10.1.1 Runtime Quality Model 204
10.1.2 Overview of Bayesian Approach 205
10.2 Reducing Large Adaptation Spaces Using Learning 208
10.2.1 Illustration of the Problem 208
10.2.2 Overview of the Learning Approach 210
10.3 Learning and Improving Scaling Rules of a Cloud Infrastructure 213
10.3.1 Overview of the Fuzzy Learning Approach 214
10.3.1.1 Fuzzy Logic Controller 214
10.3.1.2 Fuzzy Q-learning 217
10.3.1.3 Experiments 221
10.4 Summary 223
10.5 Exercises 225
10.6 Bibliographic Notes 226
11 Maturity of the Field and Open Challenges 227
11.1 Analysis of the Maturity of the Field 227
11.1.1 Basic Research 227
11.1.2 Concept Formulation 228
11.1.3 Development and Extension 229
11.1.4 Internal Enhancement and Exploration 229
11.1.5 External Enhancement and Exploration 230
11.1.6 Popularization 230
11.1.7 Conclusion 231
11.2 Open Challenges 231
11.2.1 Challenges Within the Current Waves 231
11.2.1.1 Evidence for the Value of Self-Adaptation 231
11.2.1.2 Decentralized Settings 232
11.2.1.3 Domain-Specific Modeling Languages 232
11.2.1.4 Changing Goals at Runtime 233
11.2.1.5 Complex Types of Uncertainties 233
11.2.1.6 Control Properties versus Quality Properties 234
11.2.1.7 Search-based Techniques 234
11.2.2 Challenges Beyond the Current Waves 235
11.2.2.1 Exploiting Artificial Intelligence 235
11.2.2.2 Dealing with Unanticipated Change 236
11.2.2.3 Trust and Humans in the Loop 236
11.2.2.4 Ethics for Self-Adaptive Systems 237
11.3 Epilogue 239
Bibliography 241
Index 263
1
Basic Principles of Self-Adaptation and Conceptual Model
Modern software-intensive systems1 are expected to operate under uncertain conditions, without interruption. Possible causes of uncertainties include changes in the operational environment, dynamics in the availability of resources, and variations of user goals. Traditionally, it is the task of system operators to deal with such uncertainties. However, such management tasks can be complex, error-prone, and expensive. The aim of self-adaptation is to let the system collect additional data about the uncertainties during operation in order to manage itself based on high-level goals. The system uses the additional data to resolve uncertainties and based on its goals re-configures or adjusts itself to satisfy the changing conditions.
Consider as an example a simple service-based health assistance system as shown in Figure 1.1. The system takes samples of vital parameters of patients; it also enables patients to invoke a panic button in case of an emergency. The parameters are analyzed by a medical service that may invoke additional services to take actions when needed; for instance, a drug service may need to notify a local pharmacy to deliver new medication to a patient. Each service type can be realized by one of multiple service instances provided by third-party service providers. These service instances are characterized by different quality properties, such as failure rate and cost. Typical examples of uncertainties in this system are the patterns that particular paths in the workflow are invoked by, which are based on the health conditions of the users and their behavior. Other uncertainties are the available service instances, their actual failure rates and the costs to use them. These parameters may change over time, for instance due to the changing workloads or unexpected network failures.
Figure 1.1 Architecture of a simple service-based health assistance system
Anticipating such uncertainties during system development, or letting system operators deal with them during operation, is often difficult, inefficient, or too costly. Moreover, since many software-intensive systems today need to be operational 24/7, the uncertainties necessarily need to be resolved at runtime when the missing knowledge becomes available. Self-adaptation is about how a system can mitigate such uncertainties autonomously or with minimum human intervention.
The basic idea of self-adaptation is to let the system collect new data (that was missing before deployment) during operation when it becomes available. The system uses the additional data to resolve uncertainties, to reason about itself, and based on its goals to reconfigure or adjust itself to maintain its quality requirements or, if necessary, to degrade gracefully.
In this chapter, we explain what a self-adaptive system is. We define two basic principles that determine the essential characteristics of self-adaptation. These principles allow us to define the boundaries of what we mean by a self-adaptive system in this book, and to contrast self-adaptation with other approaches that deal with changing conditions during operation. From the two principles, we derive a conceptual model of a self-adaptive system that defines the basic elements of such a system. The conceptual model provides a basic vocabulary for the remainder of this book.
LEARNING OUTCOMES
- To explain the basic principles of self-adaptation.
- To understand how self-adaptation relates to other adaptation approaches.
- To describe the conceptual model of a self-adaptive system.
- To explain and illustrate the basic concepts of a self-adaptive system.
- To apply the conceptual model to a concrete self-adaptive application.
1.1 Principles of Self-Adaptation
There is no general agreement on a definition of the notion of self-adaptation. However, there are two common interpretations of what constitutes a self-adaptive system.
The first interpretation considers a self-adaptive system as a system that is able to adjust its behavior in response to the perception of changes in the environment and the system itself. The self prefix indicates that the system decides autonomously (i.e. without or with minimal human intervention) how to adapt to accommodate changes in its context and environment. Furthermore, a prevalent aspect of this first interpretation is the presence of uncertainty in the environment or the domain in which the software is deployed. To deal with these uncertainties, the self-adaptive system performs tasks that are traditionally done by operators. Hence, the first interpretation takes the stance of the external observer and looks at a self-adaptive system as a black box. Self-adaptation is considered as an observable property of a system that enables it to handle changes in external conditions, availability of resources, workloads, demands, and failures and threats.
The second interpretation contrasts traditional "internal" mechanisms that enable a system to deal with unexpected or unwanted events, such as exceptions in programming languages and fault-tolerant protocols, with "external" mechanisms that are realized by means of a closed feedback loop that monitors and adapts the system behavior at runtime. This interpretation emphasizes a "disciplined split" between two distinct parts of a self-adaptive system: one part that deals with the domain concerns and another part that deals with the adaptation concerns. Domain concerns relate to the goals of the users for which the system is built; adaptation concerns relate to the system itself, i.e. the way the system realizes the user goals under changing conditions. The second interpretation takes the stance of the engineer of the system and looks at self-adaptation from the point of view how the system is conceived.
Hence, we introduce two complementary basic principles that determine what a self-adaptive system is:
- External principle: A self-adaptive system is a system that can handle changes and uncertainties in its environment, the system itself, and its goals autonomously (i.e. without or with minimal required human intervention).
- Internal principle: A self-adaptive system comprises two distinct parts: the first part interacts with the environment and is responsible for the domain concerns - i.e. the concerns of users for which the system is built; the second part consists of a feedback loop that interacts with the first part (and monitors its environment) and is responsible for the adaptation concerns - i.e. concerns about the domain concerns.
Let us illustrate how the two principles of self-adaptation apply to the service-based health assistance system. Self-adaptation would enable the system to deal with dynamics in the types of services that are invoked by the system as well as variations in the failure rates and costs of particular service instances. Such uncertainties may be hard to anticipate before the system is deployed (external principle). To that end, the service-based system could be enhanced with a feedback loop. This feedback loop tracks the paths of services that are invoked in the workflow, as well as the failure rates of service instances and the costs of invoking service instances that are provided by the service providers. Taking this data into account, the feedback loop adapts the selection of service instances by the workflow engine such that a set of adaptation concerns is achieved. For instance, services are selected that keep the average failure rate below a required threshold, while the cost of using the health assistance system is minimized (internal principle).
1.2 Other Adaptation Approaches
The ability of a software-intensive system to adapt at runtime in order to achieve its goals under changing conditions is not the exclusivity of self-adaptation, but can be realized in other ways.
The field of autonomous systems has a long tradition of studying systems that can change their behavior during operation in response to events that may not have been anticipated fully. A central idea of autonomous systems is to mimic human (or animal) behavior, which has been a source of inspiration for a very long time. The area of cybernetics founded by Norbert Wiener at MIT in the mid twentieth century led to the development of various types of machines that exposed seemingly "intelligent" behavior similar to biological systems. Wiener's work contributed to the foundations of various fields, including feedback control, automation, and robotics. The interest in autonomous systems has expanded significantly in recent years, with high-profile application domains such as autonomous vehicles. While these applications have extreme potential, their successes so far have also been accompanied by some dramatic failures, such as the accidents caused by first generation autonomous cars. The consequences of such failures demonstrate the real technical difficulties associated with realizing truly autonomous systems.
An important sub-field of autonomous systems is multi-agent systems, which studies the coordination of autonomous behavior of agents to solve problems that go beyond the capabilities of...
