xUnit Test Patterns
Refactoring Test Code
Published on 31. May 2007
Book
Hardback
944 pages
978-0-13-149505-0 (ISBN)
Description
Software testing has received renewed attention with the widespread adoption of Extreme Programming and other agile methodologies. While testing does not directly improve the quality of software, the under-appreciated practice provides a timely and accurate measurement (a reality check) so that the reader knows whether any new action needs to be taken. Automated software testing is used to ensure that once the software works, it is not accidentally broken during subsequent software development or maintenance activities. This book describes patterns for writing automated tests using the XUnit family (e.g., JUnit and NUnit) of test automation frameworks. The author uses the proven practice of patterns to illuminate proven techniques. When properly applied, these patterns result in tests that are easier to write, easier to understand, more robust and repeatable, easier to maintain, and ultimately more cost-effective.
More details
Other editions
Additional editions
E-Book
05/2007
1st Edition
Addison Wesley
€74.99
Available for download
E-Book
05/2007
1st Edition
Addison Wesley
€74.99
Available for download
Person
Gerard Meszaros is Chief Scientist and Senior Consultant at ClearStream Consulting, a Calgary-based consultancy specializing in agile development. He has more than a decade of experience with automated unit testing frameworks and is a leading expert in test automation patterns, refactoring of software and tests, and design for testability.
Content
Visual Summary of the Pattern Language xvii Foreword xix Preface xxi Acknowledgments xxvi Introduction xxix Refactoring a Test xlv PART I: The Narratives 1 Chapter 1 A Brief Tour 3 About This Chapter 3
The Simplest Test Automation Strategy That Could Possibly Work 3
Development Process 4
Customer Tests 5
Unit Tests 6
Design for Testability 7
Test Organization 7
What's Next? 8
Chapter 2 Test Smells 9 About This Chapter 9
An Introduction to Test Smells 9
What's a Test Smell? 10
Kinds of Test Smells 10
What to Do about Smells? 11
A Catalog of Smells 12
The Project Smells 12
The Behavior Smells 13
The Code Smells 16
What's Next? 17
Chapter 3 Goals of Test Automation 19 About This Chapter 19
Why Test? 19
Economics of Test Automation 20
Goals of Test Automation 21
Tests Should Help Us Improve Quality 22
Tests Should Help Us Understand the SUT 23
Tests Should Reduce (and Not Introduce) Risk 23
Tests Should Be Easy to Run 25
Tests Should Be Easy to Write and Maintain 27
Tests Should Require Minimal Maintenance as
the System Evolves Around Them 29
What's Next? 29
Chapter 4 Philosophy of Test Automation 31 About This Chapter 31
Why Is Philosophy Important? 31
Some Philosophical Differences 32
Test First or Last? 32
Tests or Examples? 33
Test-by-Test or Test All-at-Once? 33
Outside-In or Inside-Out? 34
State or Behavior Verification? 36
Fixture Design Upfront or Test-by-Test? 36
When Philosophies Differ 37
My Philosophy 37
What's Next? 37
Chapter 5 Principles of Test Automation 39 About This Chapter 39
The Principles 39
What's Next? 48
Chapter 6 Test Automation Strategy 49 About This Chapter 49
What's Strategic? 49
Which Kinds of Tests Should We Automate? 50
Per-Functionality Tests 50
Cross-Functional Tests 52
Which Tools Do We Use to Automate Which Tests? 53
Test Automation Ways and Means 54
Introducing xUnit 56
The xUnit Sweet Spot 58
Which Test Fixture Strategy Do We Use? 58
What Is a Fixture? 59
Major Fixture Strategies 60
Transient Fresh Fixtures 61
Persistent Fresh Fixtures 62
Shared Fixture Strategies 63
How Do We Ensure Testability? 65
Test Last--At Your Peril 65
Design for Testability--Upfront 65
Test-Driven Testability 66
Control Points and Observation Points 66
Interaction Styles and Testability Patterns 67
Divide and Test 71
What's Next? 73
Chapter 7 xUnit Basics 75 About This Chapter 75
An Introduction to xUnit 75
Common Features 76
The Bare Minimum 76
Defining Tests 76
What's a Fixture? 78
Defining Suites of Tests 78
Running Tests 79
Test Results 79
Under the xUnit Covers 81
Test Commands 82
Test Suite Objects 82
xUnit in the Procedural World 82
What's Next? 83
Chapter 8 Transient Fixture Management 85 About This Chapter 85
Test Fixture Terminology 86
What Is a Fixture? 86
What Is a Fresh Fixture? 87
What Is a Transient Fresh Fixture? 87
Building Fresh Fixtures 88
In-line Fixture Setup 88
Delegated Fixture Setup 89
Implicit Fixture Setup 91
Hybrid Fixture Setup 93
Tearing Down Transient Fresh Fixtures 93
What's Next? 94
Chapter 9 Persistent Fixture Management 95 About This Chapter 95
Managing Persistent Fresh Fixtures 95
What Makes Fixtures Persistent? 95
Issues Caused by Persistent Fresh Fixtures 96
Tearing Down Persistent Fresh Fixtures 97
Avoiding the Need for Teardown 100
Dealing with Slow Tests 102
Managing Shared Fixtures 103
Accessing Shared Fixtures 103
Triggering Shared Fixture Construction 104
What's Next? 106
Chapter 10 Result Verification 107 About This Chapter 107
Making Tests Self-Checking 107
Verify State or Behavior? 108
State Verification 109
Using Built-in Assertions 110
Delta Assertions 111
External Result Verification 111
Verifying Behavior 112
Procedural Behavior Verification 113
Expected Behavior Specification 113
Reducing Test Code Duplication 114
Expected Objects 115
Custom Assertions 116
Outcome-Describing Verification Method 117
Parameterized and Data-Driven Tests 118
Avoiding Conditional Test Logic 119
Eliminating "if" Statements 120
Eliminating Loops 121
Other Techniques 121
Working Backward, Outside-In 121
Using Test-Driven Development to
Write Test Utility Methods 122
Where to Put Reusable Verification Logic? 122
What's Next? 123
Chapter 11 Using Test Doubles 125 About This Chapter 125
What Are Indirect Inputs and Outputs? 125
Why Do We Care about Indirect Inputs? 126
Why Do We Care about Indirect Outputs? 126
How Do We Control Indirect Inputs? 128
How Do We Verify Indirect Outputs? 130
Testing with Doubles 133
Types of Test Doubles 133
Providing the Test Double 140
Configuring the Test Double 141
Installing the Test Double 143
Other Uses of Test Doubles 148
Endoscopic Testing 149
Need-Driven Development 149
Speeding Up Fixture Setup 149
Speeding Up Test Execution 150
Other Considerations 150
What's Next? 151
Chapter 12 Organizing Our Tests 153 About This Chapter 153
Basic xUnit Mechanisms 153
Right-Sizing Test Methods 154
Test Methods and Testcase Classes 155
Testcase Class per Class 155
Testcase Class per Feature 156
Testcase Class per Fixture 156
Choosing a Test Method Organization Strategy 158
Test Naming Conventions 158
Organizing Test Suites 160
Running Groups of Tests 160
Running a Single Test 161
Test Code Reuse 162
Test Utility Method Locations 163
TestCase Inheritance and Reuse 163
Test File Organization 164
Built-in Self-Test 164
Test Packages 164
Test Dependencies 165
What's Next? 165
Chapter 13 Testing with Databases 167 About This Chapter 167
Testing with Databases 167
Why Test with Databases? 168
Issues with Databases 168
Testing without Databases 169
Testing the Database 171
Testing Stored Procedures 172
Testing the Data Access Layer 172
Ensuring Developer Independence 173
Testing with Databases (Again!) 173
What's Next? 174
Chapter 14 A Roadmap to Effective Test Automation 175 About This Chapter 175
Test Automation Difficulty 175
Roadmap to Highly Maintainable Automated Tests 176
Exercise the Happy Path Code 177
Verify Direct Outputs of the Happy Path 178
Verify Alternative Paths 178
Verify Indirect Output Behavior 179
Optimize Test Execution and Maintenance 180
What's Next? 181
PART II: The Test Smells 183 Chapter 15 Code Smells 185 Obscure Test 186
Conditional Test Logic 200
Hard-to-Test Code 209
Test Code Duplication 213
Test Logic in Production 217
Chapter 16 Behavior Smells 223 Assertion Roulette 224
Erratic Test 228
Fragile Test 239
Frequent Debugging 248
Manual Intervention 250
Slow Tests 253
Chapter 17 Project Smells 259 Buggy Tests 260
Developers Not Writing Tests 263
High Test Maintenance Cost 265
Production Bugs 268
PART III: The Patterns 275 Chapter 18 Test Strategy Patterns 277 Recorded Test 278
Scripted Test 285
Data-Driven Test 288
Test Automation Framework 298
Minimal Fixture 302
Standard Fixture 305
Fresh Fixture 311
Shared Fixture 317
Back Door Manipulation 327
Layer Test 337
Chapter 19 xUnit Basics Patterns 347 Test Method 348
Four-Phase Test 358
Assertion Method 362
Assertion Message 370
Testcase Class 373
Test Runner 377
Testcase Object 382
Test Suite Object 387
Test Discovery 393
Test Enumeration 399
Test Selection 403
Chapter 20 Fixture Setup Patterns 407 In-line Setup 408
Delegated Setup 411
Creation Method 415
Implicit Setup 424
Prebuilt Fixture 429
Lazy Setup 435
Suite Fixture Setup 441
Setup Decorator 447
Chained Tests 454
Chapter 21 Result Verification Patterns 461 State Verification 462
Behavior Verification 468
Custom Assertion 474
Delta Assertion 485
Guard Assertion 490
Unfinished Test Assertion 494
Chapter 22 Fixture Teardown Patterns 499 Garbage-Collected Teardown 500
Automated Teardown 503
In-line Teardown 509
Implicit Teardown 516
Chapter 23 Test Double Patterns 521 Test Double 522
Test Stub 529
Test Spy 538
Mock Object 544
Fake Object 551
Configurable Test Double 558
Hard-Coded Test Double 568
Test-Specific Subclass 579
Chapter 24 Test Organization Patterns 591 Named Test Suite 592
Test Utility Method 599
Parameterized Test 607
Testcase Class per Class 617
Testcase Class per Feature 624
Testcase Class per Fixture 631
Testcase Superclass 638
Test Helper 643
Chapter 25 Database Patterns 649 Database Sandbox 650
Stored Procedure Test 654
Table Truncation Teardown 661
Transaction Rollback Teardown 668
Chapter 26 Design-for-Testability Patterns 677 Dependency Injection 678
Dependency Lookup 686
Humble Object 695
Test Hook 709
Chapter 27 Value Patterns 713 Literal Value 714
Derived Value 718
Generated Value 723
Dummy Object 728
PART IV: Appendixes 733 Appendix A Test Refactorings 735 Appendix B xUnit Terminology 741 Appendix C xUnit Family Members 747 Appendix D Tools 753 Appendix E Goals and Principles 757 Appendix F Smells, Aliases, and Causes 761 Appendix G Patterns, Aliases, and Variations 767 Glossary 785 References 819 Index 835
The Simplest Test Automation Strategy That Could Possibly Work 3
Development Process 4
Customer Tests 5
Unit Tests 6
Design for Testability 7
Test Organization 7
What's Next? 8
Chapter 2 Test Smells 9 About This Chapter 9
An Introduction to Test Smells 9
What's a Test Smell? 10
Kinds of Test Smells 10
What to Do about Smells? 11
A Catalog of Smells 12
The Project Smells 12
The Behavior Smells 13
The Code Smells 16
What's Next? 17
Chapter 3 Goals of Test Automation 19 About This Chapter 19
Why Test? 19
Economics of Test Automation 20
Goals of Test Automation 21
Tests Should Help Us Improve Quality 22
Tests Should Help Us Understand the SUT 23
Tests Should Reduce (and Not Introduce) Risk 23
Tests Should Be Easy to Run 25
Tests Should Be Easy to Write and Maintain 27
Tests Should Require Minimal Maintenance as
the System Evolves Around Them 29
What's Next? 29
Chapter 4 Philosophy of Test Automation 31 About This Chapter 31
Why Is Philosophy Important? 31
Some Philosophical Differences 32
Test First or Last? 32
Tests or Examples? 33
Test-by-Test or Test All-at-Once? 33
Outside-In or Inside-Out? 34
State or Behavior Verification? 36
Fixture Design Upfront or Test-by-Test? 36
When Philosophies Differ 37
My Philosophy 37
What's Next? 37
Chapter 5 Principles of Test Automation 39 About This Chapter 39
The Principles 39
What's Next? 48
Chapter 6 Test Automation Strategy 49 About This Chapter 49
What's Strategic? 49
Which Kinds of Tests Should We Automate? 50
Per-Functionality Tests 50
Cross-Functional Tests 52
Which Tools Do We Use to Automate Which Tests? 53
Test Automation Ways and Means 54
Introducing xUnit 56
The xUnit Sweet Spot 58
Which Test Fixture Strategy Do We Use? 58
What Is a Fixture? 59
Major Fixture Strategies 60
Transient Fresh Fixtures 61
Persistent Fresh Fixtures 62
Shared Fixture Strategies 63
How Do We Ensure Testability? 65
Test Last--At Your Peril 65
Design for Testability--Upfront 65
Test-Driven Testability 66
Control Points and Observation Points 66
Interaction Styles and Testability Patterns 67
Divide and Test 71
What's Next? 73
Chapter 7 xUnit Basics 75 About This Chapter 75
An Introduction to xUnit 75
Common Features 76
The Bare Minimum 76
Defining Tests 76
What's a Fixture? 78
Defining Suites of Tests 78
Running Tests 79
Test Results 79
Under the xUnit Covers 81
Test Commands 82
Test Suite Objects 82
xUnit in the Procedural World 82
What's Next? 83
Chapter 8 Transient Fixture Management 85 About This Chapter 85
Test Fixture Terminology 86
What Is a Fixture? 86
What Is a Fresh Fixture? 87
What Is a Transient Fresh Fixture? 87
Building Fresh Fixtures 88
In-line Fixture Setup 88
Delegated Fixture Setup 89
Implicit Fixture Setup 91
Hybrid Fixture Setup 93
Tearing Down Transient Fresh Fixtures 93
What's Next? 94
Chapter 9 Persistent Fixture Management 95 About This Chapter 95
Managing Persistent Fresh Fixtures 95
What Makes Fixtures Persistent? 95
Issues Caused by Persistent Fresh Fixtures 96
Tearing Down Persistent Fresh Fixtures 97
Avoiding the Need for Teardown 100
Dealing with Slow Tests 102
Managing Shared Fixtures 103
Accessing Shared Fixtures 103
Triggering Shared Fixture Construction 104
What's Next? 106
Chapter 10 Result Verification 107 About This Chapter 107
Making Tests Self-Checking 107
Verify State or Behavior? 108
State Verification 109
Using Built-in Assertions 110
Delta Assertions 111
External Result Verification 111
Verifying Behavior 112
Procedural Behavior Verification 113
Expected Behavior Specification 113
Reducing Test Code Duplication 114
Expected Objects 115
Custom Assertions 116
Outcome-Describing Verification Method 117
Parameterized and Data-Driven Tests 118
Avoiding Conditional Test Logic 119
Eliminating "if" Statements 120
Eliminating Loops 121
Other Techniques 121
Working Backward, Outside-In 121
Using Test-Driven Development to
Write Test Utility Methods 122
Where to Put Reusable Verification Logic? 122
What's Next? 123
Chapter 11 Using Test Doubles 125 About This Chapter 125
What Are Indirect Inputs and Outputs? 125
Why Do We Care about Indirect Inputs? 126
Why Do We Care about Indirect Outputs? 126
How Do We Control Indirect Inputs? 128
How Do We Verify Indirect Outputs? 130
Testing with Doubles 133
Types of Test Doubles 133
Providing the Test Double 140
Configuring the Test Double 141
Installing the Test Double 143
Other Uses of Test Doubles 148
Endoscopic Testing 149
Need-Driven Development 149
Speeding Up Fixture Setup 149
Speeding Up Test Execution 150
Other Considerations 150
What's Next? 151
Chapter 12 Organizing Our Tests 153 About This Chapter 153
Basic xUnit Mechanisms 153
Right-Sizing Test Methods 154
Test Methods and Testcase Classes 155
Testcase Class per Class 155
Testcase Class per Feature 156
Testcase Class per Fixture 156
Choosing a Test Method Organization Strategy 158
Test Naming Conventions 158
Organizing Test Suites 160
Running Groups of Tests 160
Running a Single Test 161
Test Code Reuse 162
Test Utility Method Locations 163
TestCase Inheritance and Reuse 163
Test File Organization 164
Built-in Self-Test 164
Test Packages 164
Test Dependencies 165
What's Next? 165
Chapter 13 Testing with Databases 167 About This Chapter 167
Testing with Databases 167
Why Test with Databases? 168
Issues with Databases 168
Testing without Databases 169
Testing the Database 171
Testing Stored Procedures 172
Testing the Data Access Layer 172
Ensuring Developer Independence 173
Testing with Databases (Again!) 173
What's Next? 174
Chapter 14 A Roadmap to Effective Test Automation 175 About This Chapter 175
Test Automation Difficulty 175
Roadmap to Highly Maintainable Automated Tests 176
Exercise the Happy Path Code 177
Verify Direct Outputs of the Happy Path 178
Verify Alternative Paths 178
Verify Indirect Output Behavior 179
Optimize Test Execution and Maintenance 180
What's Next? 181
PART II: The Test Smells 183 Chapter 15 Code Smells 185 Obscure Test 186
Conditional Test Logic 200
Hard-to-Test Code 209
Test Code Duplication 213
Test Logic in Production 217
Chapter 16 Behavior Smells 223 Assertion Roulette 224
Erratic Test 228
Fragile Test 239
Frequent Debugging 248
Manual Intervention 250
Slow Tests 253
Chapter 17 Project Smells 259 Buggy Tests 260
Developers Not Writing Tests 263
High Test Maintenance Cost 265
Production Bugs 268
PART III: The Patterns 275 Chapter 18 Test Strategy Patterns 277 Recorded Test 278
Scripted Test 285
Data-Driven Test 288
Test Automation Framework 298
Minimal Fixture 302
Standard Fixture 305
Fresh Fixture 311
Shared Fixture 317
Back Door Manipulation 327
Layer Test 337
Chapter 19 xUnit Basics Patterns 347 Test Method 348
Four-Phase Test 358
Assertion Method 362
Assertion Message 370
Testcase Class 373
Test Runner 377
Testcase Object 382
Test Suite Object 387
Test Discovery 393
Test Enumeration 399
Test Selection 403
Chapter 20 Fixture Setup Patterns 407 In-line Setup 408
Delegated Setup 411
Creation Method 415
Implicit Setup 424
Prebuilt Fixture 429
Lazy Setup 435
Suite Fixture Setup 441
Setup Decorator 447
Chained Tests 454
Chapter 21 Result Verification Patterns 461 State Verification 462
Behavior Verification 468
Custom Assertion 474
Delta Assertion 485
Guard Assertion 490
Unfinished Test Assertion 494
Chapter 22 Fixture Teardown Patterns 499 Garbage-Collected Teardown 500
Automated Teardown 503
In-line Teardown 509
Implicit Teardown 516
Chapter 23 Test Double Patterns 521 Test Double 522
Test Stub 529
Test Spy 538
Mock Object 544
Fake Object 551
Configurable Test Double 558
Hard-Coded Test Double 568
Test-Specific Subclass 579
Chapter 24 Test Organization Patterns 591 Named Test Suite 592
Test Utility Method 599
Parameterized Test 607
Testcase Class per Class 617
Testcase Class per Feature 624
Testcase Class per Fixture 631
Testcase Superclass 638
Test Helper 643
Chapter 25 Database Patterns 649 Database Sandbox 650
Stored Procedure Test 654
Table Truncation Teardown 661
Transaction Rollback Teardown 668
Chapter 26 Design-for-Testability Patterns 677 Dependency Injection 678
Dependency Lookup 686
Humble Object 695
Test Hook 709
Chapter 27 Value Patterns 713 Literal Value 714
Derived Value 718
Generated Value 723
Dummy Object 728
PART IV: Appendixes 733 Appendix A Test Refactorings 735 Appendix B xUnit Terminology 741 Appendix C xUnit Family Members 747 Appendix D Tools 753 Appendix E Goals and Principles 757 Appendix F Smells, Aliases, and Causes 761 Appendix G Patterns, Aliases, and Variations 767 Glossary 785 References 819 Index 835