Preface

This book will show you how to comprehensively test new software features, whether implemented on web pages, a desktop, or mobile applications. It covers everything from generating in-depth specifications to detailed black- and white-box testing, handling error cases and user experience considerations, to areas sometimes missed off test plans such as maintainability and operating under failure conditions.

The same bugs and defects crop up repeatedly in software development, so this book has a huge range of tests to run and scenarios to check to help you achieve great test coverage. They're illustrated by dozens of example issues I've encountered during my decades in the software industry. While it's never possible to guarantee a product is bug-free, the recommendations here give you the best chance of catching defects.

Who this book is for

Many different people can perform testing during a development cycle. In addition to dedicated testers, developers should always try their code, and product owners, support engineers, and technical writers can also provide more or less formal feedback.

This book is for anyone involved in software testing, regardless of their official job title. Everyone can use the suggestions here to increase test coverage and improve the quality of the features you release.

Testing has a lovely learning curve. You can start with manual testing on easy-to-use public interfaces that have been well polished and documented. From there, you can go deeper into the system, checking logs and metrics and using internal interfaces. Testers with the right skill set can improve code directly by performing code reviews and writing unit tests. This book describes all these approaches to improve your testing, whether you're a novice or an expert tester.

Is this book for Agile or waterfall projects? Both!

This book does not discuss different development models such as Agile or waterfall. Whichever you are using, you will need to design a comprehensive test plan to exercise new features and discover their defects.

However, there will be significant timing differences depending on the method you're using. Waterfall projects may extend over many months, with weeks allocated to the different areas of testing described here. In contrast, Agile projects rapidly cycle through the different types of testing, doing a little of each in cycle after cycle.

Despite their differences, both project types, and the many real-world hybrids used in practice, require a thorough test plan covering the many different aspects of system behavior. Whatever project type you work on, this book can help you write better test plans.

Is this book for manual or automated testing? Both!

This book has many proposals for what you should test. It does not cover how those tests should be run. The aim, after applying this book to your feature, is to produce a comprehensive test plan with steps to detect many defect types.

How you run that test plan is up to you. Some areas must be done manually - exploratory testing, for instance, involves a tester getting their hands on a feature to investigate how it behaves in an unstructured way. Only once you have learned how it works can you write the feature specification and do systematic testing, either manual or automated. User experience testing is also best started with a genuine user present. Conversely, load testing requires automation to reach the required load levels and cannot be done by hand.

In between much of the functional testing, security, error cases, and even some user experience testing can be automated. Where possible, that is a vital stage in the testing process to free up testers' time for future test designs. That is especially true when following a Continuous Integration/Continuous Deployment (CI/CD) pipeline, in which changes are pushed to live systems within minutes. There, you must automate tests and carefully choose which to run on each change.

This book does not discuss how to perform manual testing, automate tests, or what tools you should use. Those are subjects for other books in the Packt catalog. Here, we will describe what tests you should run and how you should design your test plan.

Types of testing

There are several levels of testing, but this book focuses on just one - system testing. This is the hardest part of testing because it covers the whole system working together, end to end. Because it is so difficult, it requires the thought and planning described here. That does not mean it is the only kind of testing. You should supplement all the system testing here with testing at other levels since they each have different strengths.

At the lowest level is unit testing, also known as component testing, where scripts exercise individual functions or modules. This form of testing is often performed by developers, as they know what to expect from their code and are best placed to write test harnesses to drive it. Unit tests can thoroughly test individual actions, but they can't find issues that arise when functions are used together.

To find those issues, you need integration testing. This approach involves running entire modules isolated from the rest of the system, such as user interfaces or databases. At this stage, the testing becomes much more complex. Because far more can go wrong, it allows you to find another class of issues. While both function A and function B may work well on their own, incorrect assumptions or implementations might mean they do not work together. Integration testing finds that class of bugs. It can be carried out by either developers or dedicated testers, although it requires a significant amount of automation and is unsuitable for manual testing. Using modules or clients independently of the rest of the system needs test harnesses to stand in for areas that are not being run.

Beyond integration testing is system testing, the focus of this book. Once the whole system is working together, with all modules integrated with the others, then the final class of bugs is revealed. These issues only occur when running the full system in a realistic environment, such as loading and problems between modules. On the plus side, system testing can find all the realistic bugs that will affect users. On the downside, it is difficult to translate from system behavior down to the individual functions that are failing, and systems can be complex to test fully. This book aims to give techniques and ideas to successfully navigate that challenge.

The naming convention used here follows the International Software Testing Qualification Board (ISTQB) syllabus, which refers to component, integration, and system testing. There is some overlap with the topics covered there, although this book takes a far more hands-on, practical approach rather than focusing on the theory. By selecting the most useful pieces of the ISTQB syllabus, and expanding their practical aspects, this book will show you how to thoroughly test software features.

Test process

There are many important topics that this book can't address. This book is only about writing excellent test plans to find a wide variety of issues in your application. It does not cover test teams or anything associated with test processes.

Test prioritization is not covered in detail here. The aim is to give you a large bank of ideas and test scenarios, from which you can decide their relative priority for your application. Judging risk and the likelihood and impact of issues will depend on your circumstances. It's up to you to apply these ideas.

This book also doesn't cover team organization. How many testers do you need in your company? How should they be arranged to work with the product owner and developers? How much time should they devote to the different forms of testing? It also doesn't cover managing communication, such as how bugs should be raised and their life cycle, but it will tell you how to find those bugs.

Finally, this book doesn't describe the release process, including how test results should be evaluated, internal usage and beta programs, and how to sign off and perform releases. Again, that depends on your product and industry. Even if you don't decide to run all the tests here, you can deliberately choose which ones you exclude so that you have the best possible idea of the risks you are taking.

The aim, by the end of this book, is that you should be able to quickly prepare a comprehensive test plan, rapidly covering many different areas of testing relevant to your product and its features. Often in software development, the factor limiting your speed is identifying what you should do, so by understanding the main types of testing and common failures, you can develop tests faster. That lets you find more issues earlier in the development cycle, save time, speed up projects, and give your customers a better user experience.

With that said, we can now see the areas this book covers in...