Trade-off Analytics
Description
More details
Other editions
Additional editions
Person
Gregory S. Parnell, Ph.D, is a Research Professor in the Department of Industrial Engineering at the University of Arkansas. He is also a senior principal with Innovative Decisions, Inc., a decision and risk analysis firm and has served as Chairman of the Board. Dr. Parnell has published more than 100 papers and book chapters and was lead editor of Decision Making for Systems Engineering and Management, Wiley Series in Systems Engineering (2nd Ed, Wiley 2011) and lead author of the Handbook of Decision Analysis (Wiley 2013). He is a fellow of INFORMS, INCOSE, MORS, and the Society for Decision Professionals.
Content
List of Contributors xix
About the Authors xxi
Foreword xxxi
Preface xxxiii
Acknowledgments xli
About the Companion Website xlv
1 Introduction to Trade-off Analysis 1
Gregory S. Parnell, Matthew Cilli, Azad M. Madni and Garry Roedler
1.1 Introduction 2
1.2 Trade-off Analyses Throughout the Life Cycle 3
1.3 Trade-off Analysis to Identify System Value 3
1.4 Trade-off Analysis to Identify System Uncertainties and Risks 6
1.5 Trade-off Analyses can Integrate Value and Risk Analysis 6
1.6 Trade-off Analysis in the Systems Engineering Decision Management Process 8
1.7 Trade-off Analysis Mistakes of Omission and Commission 9
1.7.1 Mistakes of Omission 12
1.7.2 Mistakes of Commission 15
1.7.3 Impacts of the Trade-Off Analysis Mistakes 18
1.8 Overview of the Book 20
1.8.1 Illustrative Examples and Techniques Used in the Book 24
1.9 Key Terms 24
1.10 Exercises 25
References 26
2 A Conceptual Framework and Mathematical Foundation for Trade-Off Analysis 29
Gregory S. Parnell, Azad M. Madni and Robert F. Bordley
2.1 Introduction 29
2.2 Trade-Off Analysis Terms 30
2.3 Influence Diagram of the Tradespace 31
2.3.1 Stakeholder Needs System Functions and Requirements 33
2.3.2 Objectives 33
2.3.3 System Alternatives 34
2.3.4 Uncertainty 36
2.3.5 Preferences and Evaluation of Alternatives 37
2.3.6 Resource Analysis 44
2.3.7 An Integrated Trade-Off Analyses 44
2.4 Tradespace Exploration 46
2.5 Summary 46
2.6 Key Words 47
2.7 Exercises 48
References 48
3 Quantifying Uncertainty 51
Robert F. Bordley
3.1 Sources of Uncertainty in Systems Engineering 51
3.2 The Rules of Probability and Human Intuition 52
3.3 Probability Distributions 56
3.3.1 Calculating Probabilities from Experiments 56
3.3.2 Calculating Complex Probabilities from Simpler Probabilities 58
3.3.3 Calculating Probabilities Using Parametric Distributions 59
3.3.4 Applications of Parametric Probability Distributions 62
3.4 Estimating Probabilities 66
3.4.1 Using Historical Data 66
3.4.2 Using Human Judgment 68
3.4.3 Biases in Judgment 70
3.5 Modeling Using Probability 72
3.5.1 Bayes Nets 72
3.5.2 Monte Carlo Simulation 75
3.5.3 Monte Carlo Simulation with Dependent Uncertainties 76
3.5.4 Monte Carlo Simulation with Partial Information on Output Values 77
3.5.5 Variations on Monte Carlo Simulation 78
3.5.6 Sensitivity Analysis 78
3.6 Summary 81
3.7 Key Terms 81
3.8 Exercises 83
References 86
4 Analyzing Resources 91
Edward A. Pohl, Simon R. Goerger and Kirk Michealson
4.1 Introduction 91
4.2 Resources 92
4.2.1 People 92
4.2.2 Facilities 95
4.2.3 Costs 95
4.2.4 Resource Space 99
4.3 Cost Analysis 99
4.3.1 Cost Estimation 102
4.3.2 Cost Estimation Techniques 108
4.3.3 Learning Curves 120
4.3.4 Net Present Value 125
4.3.5 Monte Carlo Simulation 130
4.3.6 Sensitivity Analysis 134
4.4 Affordability Analysis 135
4.4.1 Background 136
4.4.2 The Basics of Affordability Analysis Are Not Difficult 137
4.4.3 DoD Comparison of Cost Analysis and Affordability Analysis 138
4.4.4 Affordability Analysis Definitions 139
4.4.5 "Big A" Affordability Analysis Process Guide 141
4.5 Key Terms 147
4.6 Excercises 149
References 152
5 Understanding Decision Management 155
Matthew Cilli and Gregory S. Parnell
5.1 Introduction 155
5.2 Decision Process Context 156
5.3 Decision Process Activities 157
5.3.1 Frame Decision 159
5.3.2 Develop Objectives and Measures 163
5.3.3 Generate Creative Alternatives 171
5.3.4 Assess Alternatives via Deterministic Analysis 180
5.3.5 Synthesize Results 183
5.3.6 Develop Multidimensional Value Model 187
5.3.7 Identify Uncertainty and Conduct Probabilistic Analysis 190
5.3.8 Assess Impact of Uncertainty 192
5.3.9 Improve Alternatives 196
5.3.10 Communicating Trade-Offs 197
5.3.11 Present Recommendation and Implementation Plan 197
5.4 Summary 199
5.5 Key Terms 199
5.6 Exercises 200
References 201
6 Identifying Opportunities 203
Donna H. Rhodes and Simon R. Goerger
6.1 Introduction 203
6.2 Knowledge 205
6.2.1 Domain Knowledge 205
6.2.2 Technical Knowledge 205
6.2.3 Business Knowledge 205
6.2.4 Expert Knowledge 206
6.2.5 Stakeholder Knowledge 206
6.3 Decision Traps 207
6.4 Techniques 210
6.4.1 Interviews 210
6.4.2 Focus Groups 213
6.4.3 Surveys 215
6.5 Tools 219
6.5.1 Concept Map 219
6.5.2 System Boundary 220
6.5.3 Decision Hierarchy 220
6.5.4 Issues List 221
6.5.5 Vision Statement 221
6.5.6 Influence Diagram 222
6.5.7 Selecting Appropriate Tools and Techniques 223
6.6 Illustrative Examples 223
6.6.1 Commercial 223
6.6.2 Defense 226
6.7 Key Terms 228
6.8 Exercises 230
References 230
7 Identifying Objectives and Value Measures 233
Gregory S. Parnell and William D. Miller
7.1 Introduction 233
7.2 Value-Focused Thinking 234
7.2.1 Four Major VFT Ideas 235
7.2.2 Benefits of VFT 235
7.3 Shareholder and Stakeholder Value 236
7.3.1 Private Company Example 237
7.3.2 Government Agency Example 237
7.4 Challenges in Identifying Objectives 238
7.5 Identifying the Decision Objectives 239
7.5.1 Questions to Help Identify Decision Objectives 239
7.5.2 How to Get Answers to the Questions 240
7.6 The Financial or Cost Objective 241
7.6.1 Financial Objectives for Private Companies 241
7.6.2 Cost Objective for Public Organizations 242
7.7 Developing Value Measures 243
7.8 Structuring Multiple Objectives 243
7.8.1 Value Hierarchies 244
7.8.2 Techniques for Developing Value Hierarchies 245
7.8.3 Value Hierarchy Best Practices 247
7.8.4 Cautions about Cost and Risk Objectives 248
7.9 Illustrative Examples 248
7.9.1 Military Illustrative Example 248
7.9.2 Homeland Security Illustrative Example 250
7.10 Summary 250
7.11 Key Terms 252
7.12 Exercises 253
References 255
8 Developing and Evaluating Alternatives 257
C. Robert Kenley, Clifford Whitcomb and Gregory S. Parnell
8.1 Introduction 257
8.2 Overview of Decision-making Creativity and Teams 258
8.2.1 Approaches to Decision-Making 258
8.2.2 Cognitive Methods for Creating Alternatives 260
8.2.3 Key Concepts for Building and Operating Teams 260
8.3 Alternative Development Techniques 263
8.3.1 Structured Creativity Methods 263
8.3.2 Morphological Box 266
8.3.3 Pugh Method for Alternative Generation 270
8.3.4 TRIZ for Alternative Development 271
8.4 Assessment of Alternative Development Techniques 275
8.5 Alternative Evaluation Techniques 276
8.5.1 Decision-Theory-Based Approaches 276
8.5.2 Pugh Method for Alternative Evaluation 276
8.5.3 Axiomatic Approach to Design (AAD) 277
8.5.4 TRIZ for Alternative Evaluation 280
8.5.5 Design of Experiments (DOE) 280
8.5.6 Taguchi Approach 282
8.5.7 Quality Function Deployment (QFD) 283
8.5.8 Analytic Hierarchy Process AHP 287
8.6 Assessment of Alternative Evaluation Techniques 290
8.7 Key Terms 290
8.8 Exercises 290
References 293
9 An Integrated Model for Trade-Off Analysis 297
Alexander D. MacCalman, Gregory S. Parnell and Sam Savage
9.1 Introduction 297
9.2 Conceptual Design Example 298
9.3 Integrated Approach Influence Diagram 300
9.3.1 Decision Nodes 300
9.3.2 Uncertainty Nodes 303
9.3.3 Constant Node 310
9.3.4 Value Nodes 314
9.4 Other Types of Trade-Off Analysis 322
9.5 Simulation Tools 322
9.5.1 Monte Carlo Simulation Proprietary Add-Ins 324
9.5.2 The Discipline of Probability Management 324
9.5.3 SIPmathTM Tool in Native Excel 324
9.5.4 Model Building Steps 325
9.6 Summary 329
9.7 Key Terms 330
9.8 Exercises 331
References 335
10 Exploring Concept Trade-Offs 337
Azad M. Madni and Adam M. Ross
10.1 Introduction 337
10.1.1 Key Concepts Concept Trade-Offs and Concept Exploration 341
10.2 Defining the Concept Space and System Concept of Operations 345
10.3 Exploring the Concept Space 346
10.3.1 Storytelling-Enabled Tradespace Exploration 346
10.3.2 Decisions and Outcomes 347
10.3.3 Contingent Decision-Making 347
10.4 Trade-off Analysis Frameworks 348
10.5 Tradespace and System Design Life Cycle 349
10.6 From Point Trade-offs to Tradespace Exploration 351
10.7 Value-based Multiattribute Tradespace Analysis 351
10.7.1 Tradespace Exploration and Sensitivity Analysis 353
10.7.2 Tradespace Exploration and Uncertainty 354
10.7.3 Tradespace Exploration with Spiral Development 356
10.7.4 Tradespace Exploration in Relation to Optimization and Decision Theory 356
10.8 Illustrative Example 359
10.8.1 Step 1: Determine Key Decision-Makers 359
10.8.2 Step 2: Scope and Bound the Mission 360
10.8.3 Step 3: Elicit Attributes and Utilities (Preference Capture) 360
10.8.4 Step 4: Define Design Vector Elements (Concept Generation) 362
10.8.5 Step 5: Develop Model(s) (Evaluation) 362
10.8.6 Step 6: Generate the Tradespace (Computation) 364
10.8.7 Step 7: Explore the Tradespace (Analysis and Synthesis) 365
10.9 Conclusions 369
10.10 Key Terms 371
10.11 Exercises 372
References 372
11 Architecture Evaluation Framework 377
James N. Martin
11.1 Introduction 377
11.1.1 Architecture in the Decision Space 378
11.1.2 Architecture Evaluation 379
11.1.3 Architecture Views and Viewpoints 380
11.1.4 Stakeholders 382
11.1.5 Stakeholder Concerns 382
11.1.6 Architecture versus Design 383
11.1.7 On the Uses of Architecture 384
11.1.8 Standardizing on an Architecture Evaluation Strategy 384
11.2 Key Considerations in Evaluating Architectures 385
11.2.1 Plan-Driven Evaluation Effort 386
11.2.2 Objectives-Driven Evaluation 387
11.2.3 Assessment versus Analysis 387
11.3 Architecture Evaluation Elements 389
11.3.1 Architecture Evaluation Approach 389
11.3.2 Architecture Evaluation Objectives 390
11.3.3 Evaluation Approach Examples 391
11.3.4 Value Assessment Methods 391
11.3.5 Value Assessment Criteria 393
11.3.6 Architecture Analysis Methods 394
11.4 Steps in an Architecture Evaluation Process 396
11.5 Example Evaluation Taxonomy 398
11.5.1 Business Impact Factors 398
11.5.2 Mission Impact Factors 398
11.5.3 Architecture Attributes 399
11.6 Summary 400
11.7 Key Terms 400
11.8 Exercises 402
References 402
12 Exploring the Design Space 405
Clifford Whitcomb and Paul Beery
12.1 Introduction 405
12.2 Example 1: Liftboat 406
12.2.1 Liftboat Fractional Factorial Design of Experiments 406
12.2.2 Liftboat Design Trade-Off Space 409
12.2.3 Liftboat Uncertainty Analysis 411
12.2.4 Liftboat Example Summary 411
12.3 Example 2: Cruise Ship Design 411
12.3.1 Cruise Ship Taguchi Design of Experiments 411
12.3.2 Cruise Ship Design Trade-Off Space 412
12.3.3 Cruise Ship Example Summary 416
12.4 Example 3: NATO Naval Surface Combatant Ship 417
12.4.1 NATO Surface Combatant Ship Stakeholder Need 418
12.4.2 NATO Surface Combatant Ship Box-Behnken Design of Experiments 420
12.4.3 NATO Surface Combatant Ship Cost-Effectiveness Trade-Off 421
12.4.4 NATO Surface Combatant Ship Design Tradespace 421
12.4.5 NATO Surface Combatant Ship Design Trade-Off 422
12.4.6 NATO Surface Combatant Ship Trade-Off Summary 430
12.5 Key Terms 431
12.6 Exercises 433
References 435
13 Sustainment Related Models and Trade Studies 437
John E. MacCarthy and Andres Vargas
13.1 Introduction 437
13.2 Availability Modeling and Trade Studies 439
13.2.1 FMDS Background 439
13.2.2 FMDS Availability Trade Studies 449
13.2.3 Section Synopsis 453
13.3 Sustainment Life Cycle Cost Modeling and Trade Studies14 454
13.3.1 The Total System Life Cycle Model 454
13.3.2 The O&S Cost Model 456
13.3.3 Life Cycle Cost Trade Study 459
13.4 Optimization in Availability Trade Studies 464
13.4.1 Setting Up the Optimization Problem 464
13.4.2 Instantiating the Optimization Model 465
13.4.3 Discussion of the Optimization Model Results 468
13.4.4 Deterministic Sensitivity Analysis 469
13.5 Monte Carlo Modeling 471
13.5.1 Input Probability Distributions for the Monte Carlo Model 471
13.5.2 Monte Carlo Simulation Results 472
13.5.3 Stochastic Sensitivity Analysis 473
13.6 Chapter Summary 475
13.7 Key Terms 476
13.8 Exercises 478
References 482
14 Performing Programmatic Trade-Off Analyses 483
Gina Guillaume-Joseph and John E. MacCarthy
14.1 Introduction 483
14.2 System Acceptance Decisions and Trade Studies 485
14.2.1 Acceptance Decision Framework 486
14.2.2 Calculating the Confidence That a System Is "Good" 491
14.2.3 Acceptance Test Design and Trade Studies 493
14.2.4 A "Delay Fix and Test" Cost Model 499
14.2.5 The Integrated Decision Model 504
14.2.6 Conclusions 511
14.3 Product Cancelation Decision Trade Study 512
14.3.1 Introduction 512
14.3.2 Significance 513
14.3.3 Defining Failure 514
14.3.4 Developing the Predictive Model 519
14.3.5 Research Results 522
14.3.6 Model Implementation In Industry 528
14.3.7 Predictive Model Deployment in Industry 530
14.3.8 When the Decision Has Been Made to Cancel the System 536
14.3.9 Conclusion 537
14.4 Product Retirement Decision Trade Study 538
14.4.1 Introduction 538
14.4.2 Legacy HR Systems 539
14.4.3 The US NAVY Retirement and Decommission Program for Nuclear-Powered Vessels 544
14.4.4 Decision Analysis for Decommissioning Offshore Oil and Gas Platforms in California 551
14.4.5 System Retirement and Decommissioning Strategy 559
14.4.6 Conclusion 561
14.5 Key Terms 562
14.6 Exercises 564
References 566
15 Summary and Future Trends 571
Gregory S. Parnell and Simon R. Goerger
15.1 Introduction 571
15.2 Major Trade-Off Analysis Themes 572
15.2.1 Use Standard Systems Engineering Terminology 572
15.2.2 Avoid the Mistakes of Omission and Commission 572
15.2.3 Use a Decision Management Framework 572
15.2.4 Use Decision Analysis as the Mathematical Foundation 573
15.2.5 Explicitly Define the Decision Opportunity 573
15.2.6 Identify and Structure Decision Objectives and Measures 574
15.2.7 Identify Creative Doable Alternatives 574
15.2.8 Use the Most Appropriate Modeling and Simulation Technique for the Life Cycle Stage 575
15.2.9 Include Resource Analysis in the Trade-Off Analysis 575
15.2.10 Explicitly Consider Uncertainty 575
15.2.11 Identify the Cost Value Schedule and Risk Drivers 575
15.2.12 Provide an Integrated Framework for Cost Value and Risk Analyses 576
15.3 Future of Trade-Off Analysis 576
15.3.1 Education and Training of Systems Engineers 577
15.3.2 Systems Engineering Methodologies and Tools 577
15.3.3 Emergent Tradespace Factors 580
15.4 Summary 581
References 581
Index 583
