Preface

The objective of this edited volume is to compile leading edge methods and examples of analytical approaches to systems and software testing from leading authorities in applied statistics, computer science, and software engineering. The book provides a collection of methods for practitioners and researchers interested in a general perspective on this topic that affects our daily lives, and will become even more critical in the future. Our original objective was to focus on analytic methods but, as the many co-authors show, a contextual landscape of modern engineering is required to appreciate and present the related statistical and probabilistic models used in this domain. We have therefore expanded the original scope and offered, in one comprehensive collection, a state of the art view of the topic.

Inevitably, testing and validation of advanced systems and software is comprised in part of general theory and methodology, and in part of application-specific techniques. The former are transportable to new domains. While the latter generally are not, we trust the reader will share our conviction that case study examples of successful applications provide a heuristic foundation for adaptation and extension to new problem contexts. This is yet another example of where statistical methods need to be integrated with expert knowledge to provide added value.

The structure of the book consists of four parts:

Part I: Testing Concepts and Methods (Chapters 1-6)

Part II: Statistical Models (Chapters 7-12)

Part III: Testing Infrastructures (Chapters 13-17)

Part IV: Testing Applications (Chapters 18-21)

It constitutes an advanced reference directed toward industrial and academic readers whose work in systems and software development approaches or surpasses existing frontiers of testing and validation procedures. Readers will typically hold degrees in statistics, applied mathematics, computer science, or software engineering.

The 21 chapters vary in length and scope. Some are more descriptive, some present Advanced mathematical formulations, some are more oriented towards system and software engineering. To inform the reader about the nature of the chapters, we provide an annotated list below with a brief description of each.

The book provides background, examples, and methods suitable for courses on system and software testing with both an engineering and an analytic focus. Practitioners will find the examples instructive and will be able to derive benchmarks and suggestions to build upon. Consultants will be able to derive a context for their work with clients and colleagues. Our additional goal with this book is to stimulate research at the theoretical and practical level. The testing of systems and software is an area requiring further developments. We hope that this work will contribute to such efforts.

The authors of the various chapters clearly deserve most of the credit. They were generous in sharing their experience and taking the time to write the chapters. We wish to thank them for their collaboration and patience in the various stages of writing this project. We also acknowledge the professional help of the Wiley team who provided support and guidance throughout the long journey that lead to this book.

To help the reader we provide next an annotated list of chapters giving a peak preview as to their content. The chapters are grouped in three parts but there is no specific sequence to them so that one can meander from topic to topic without following the numbered order.

Annotated List of Chapters

1 Recent Advances in Classifying Risk-Based Testing Approaches A taxonomy of risk-based testing aligned with risk considerations in all phases of a test process. It provides a framework to understand, categorize, assess, and compare risk-based testing approaches. 2 Improving Software Testing with Causal Modeling An introduction to causal modeling using Bayesian networks and how it can be used to improve software testing strategies and predict software reliability. 3 Optimal Software Testing across Version Releases Looks at models for bug detection in software with a regular schedule of version releases. Model inference is developed from a Bayesian perspective and applied to data on bug reports of Mozilla Firefox. 4 Incremental Verification and Coverage Analysis of Strongly Distributed Systems A model of both software and hardware hierarchical systems as logical structures using a finite state machine. Properties to be tested are expressed with extensions of first-order logic. 5 Combinatorial Testing: An Approach to Systems and Software Testing Based on Covering Arrays Combinatorial testing is an approach for sampling from the input test space. It exploits evidence that system failures are typically due to the interaction of just a few inputs. 6 Conceptual Aspects in Development and Teaching of System and Software Test Engineering A novel approach for improving the learning of system and software testing in academic and lifelong learning programs. It builds on Meaning Equivalence Reusable Learning Objects (MERLO) used in formative assessment of conceptual understanding. 7 Non-homogeneous Poisson Process Models for Software Reliability The non-homogeneous Poisson process is used as a model for the reliability of repairable hardware systems. The chapter discusses its applicability to software reliability. 8 Bayesian Graphical Models for High-Complexity Testing: Aspects of Implementation The Bayesian graphical models approach to software testing is presented. It is a method for the logical structuring of software testing where the focus is on high reliability final stage integration testing. 9 Models of Software Reliability Software reliability time domain and data domain models are discussed. Topics such as stopping time for fault detection, the Chernoff-Ray procedure, and capture-recapture methods are covered. 10 Improved Estimation of System Reliability with Application in Software Development Shrinkage ideas are implemented to obtain improvements in the estimation of software reliability functions. This helps decide if software can be deployed in the field or further debugging is required. 11 Decision Models for Software Testing Game- and decision-theoretic frameworks are used to develop optimal software testing strategies. Specifically, optimal choice of release time for software under different scenarios is provided. 12 Modeling and Simulations in Control Software Design Control systems are studied through modeling and simulations. These designs have a major impact on the quality of control applications and the reliability, sustainability, and further extension of the system. 13 A Temperature Monitoring Infrastructure and Process for Improving Data Center Energy Efficiency with Results for a High Performance Computing Data Center An approach to energy efficiency improvement is presented. It includes a temperature monitoring infrastructure and a process that enables changes to DC cooling systems, infrastructure, and equipment. 14 Agile Testing with User Data in Cloud and Edge Computing Environments A framework for mitigating the risks of user errors is presented. It incorporates usability considerations in the design, testing, deployment, and operation of dynamic collaborative systems. 15 Automated Software Testing This chapter presents model-based automatic software testing challenges and solutions. Building such models involves modeling language selection and operational profile development. 16 Dynamic Test Case Selection in Continuous Integration: Test Result Analysis Using the Eiffel Framework Dynamic test case selection determines which tests to execute at a given time, rather than from predefined static lists. The Eiffel framework for continuous integration and delivery is presented. 17 An Automated Regression Testing Framework for a Hadoop-Based Entity Resolution System A framework built to support a large-scale entity resolution application is presented. It addresses distributed processing environments such as Hadoop Map/Reduce and Spark with parallel processing. 18 Testing Defense Systems Defense systems consistently push the limits of scientific understanding. The chapter highlights, with examples, the core statistical methodologies that have proven useful in testing defense systems. 19 A Search-Based Approach to Geographical Data Generation for Testing Location-Based Services A framework to support automatic location-based services testing is presented from two aspects, geographical test data generation and query results. How to design and validate tests is...