Engineering AI Systems

Preface xiii
Acknowledgments xvii
About the Authors xix

Chapter 1: Introduction 1
1.1 What We Talk about When We Talk about Things: Terminology 2
1.2 Achieving System Qualities 4
1.3 Life-Cycle Processes 6
1.4 Software Architecture 10
1.5 AI Model Quality 13
1.6 Dealing with Uncertainty 19
1.7 Summary 20
1.8 Discussion Questions 21
1.9 For Further Reading 21

Chapter 2: Software Engineering Background 23
2.1 Distributed Computing 23
2.2 DevOps Background 35
2.3 MLOps Background 42
2.4 Summary 44
2.5 Discussion Questions 45
2.6 For Further Reading 45

Chapter 3: AI Background 47
3.1 Terminology 48
3.2 Selecting a Model 49
3.3 Preparing the Model for Training 65
3.4 Summary 69
3.5 Discussion Questions 69
3.6 For Further Reading 69

Chapter 4: Foundation Models 71
4.1 Foundation Models 71
4.2 Transformer Architecture 72
4.3 Alternatives in FM Architectures 74
4.4 Customizing FMs 75
4.5 Designing a System Using FMs 86
4.6 Maturity of FMs and Organizations 91
4.7 Challenges of FMs 93
4.8 Summary 94
4.9 Discussion Questions 94
4.10 For Further Reading 94

Chapter 5: AI Model Life Cycle 97
5.1 Developing the Model 97
5.2 Building the Model 108
5.3 Testing the Model 109
5.4 Release 114
5.5 Summary 114
5.6 Discussion Questions 115
5.7 For Further Reading 115

Chapter 6: System Life Cycle 117
6.1 Design 118
6.2 Developing Non-AI Modules 121
6.3 Build 122
6.4 Test 123
6.5 Release and Deploy 125
6.6 Operate, Monitor, and Analyze 135
6.7 Summary 140
6.8 Discussion Questions 141
6.9 For Further Reading 141

Chapter 7: Reliability 143
7.1 Fundamental Concepts 143
7.2 Preventing Faults 145
7.3 Detecting Faults 149
7.4 Recovering from Faults 152
7.5 Summary 154
7.6 Discussion Questions 154
7.7 For Further Reading 154

Chapter 8: Performance 155
8.1 Efficiency 155
8.2 Accuracy 164
8.3 Summary 173
8.4 Discussion Questions 173
8.5 For Further Reading 174

Chapter 9: Security 175
9.1 Fundamental Concepts 176
9.2 Approaches to Mitigating Security Concerns 180
9.3 Summary 188
9.4 Discussion Questions 189
9.5 For Further Reading 189

Chapter 10: Privacy and Fairness 191
10.1 Privacy in AI Systems 192
10.2 Fairness in AI Systems 193
10.3 Achieving Privacy 194
10.4 Achieving Fairness 197
10.5 Summary 201
10.6 Discussion Questions 201
10.7 For Further Reading 202

Chapter 11: Observability 203
11.1 Fundamental Concepts 203
11.2 Evolving from Monitorability to Observability 204
11.3 Approaches for Enhancing Observability 207
11.4 Summary 211
11.5 Discussion Questions 211
11.6 For Further Reading 212

Chapter 12: The Fraunhofer Case Study: Using a Pretrained Language Model for Tendering 213
12.1 The Problem Context 214
12.2 Case Study Description and Setup 217
12.3 Summary 232
12.4 Takeaways 233
12.5 Discussion Questions 233
12.6 For Further Reading 233

Chapter 13: The ARM Hub Case Study: Chatbots for Small and Medium-Size Australian Enterprises 235
13.1 Introduction 235
13.2 Our Approach 236
13.3 LLMs in SME Manufacturing 238
13.4 A RAG-Based Chatbot for SME Manufacturing 238
13.5 Architecture of the ARM Hub Chatbot 239
13.6 MLOps in ARM Hub 244
13.7 Ongoing Work 251
13.8 Summary 252
13.9 Takeaways 253
13.10 Discussion Questions 254
13.11 For Further Reading 254

Chapter 14: The Banking Case Study: Predicting Customer Churn in Banks 255
14.1 Customer Churn Prediction 256
14.2 Key Challenges in the Banking Sector 265
14.3 Summary 265
14.4 Takeaways 266
14.5 Discussion Questions 266
14.6 For Further Reading 267

Chapter 15: The Future of AI Engineering 269
15.1 The Shift to DevOps 2.0 270
15.2 AI's Implications for the Future 271
15.3 AIWare or AI-as-Software 276
15.4 Trust in AI and the Role of Human Engineers 279
15.5 Summary 280
15.6 Discussion Questions 281
15.7 For Further Reading 281

References 283
Index 289