Trade-off Analytics
Description
More details
Other editions
Additional editions
Person
Gregory S. Parnell, Ph.D, is a Research Professor in the Department of Industrial Engineering at the University of Arkansas. He is also a senior principal with Innovative Decisions, Inc., a decision and risk analysis firm and has served as Chairman of the Board. Dr. Parnell has published more than 100 papers and book chapters and was lead editor of Decision Making for Systems Engineering and Management, Wiley Series in Systems Engineering (2nd Ed, Wiley 2011) and lead author of the Handbook of Decision Analysis (Wiley 2013). He is a fellow of INFORMS, INCOSE, MORS, and the Society for Decision Professionals.
Content
List of Contributors xix
About the Authors xxi
Foreword xxxi
Preface xxxiii
Acknowledgments xli
About the Companion Website xlv
1 Introduction to Trade-off Analysis 1
Gregory S. Parnell, Matthew Cilli, Azad M. Madni and Garry Roedler
1.1 Introduction 2
1.2 Trade-off Analyses Throughout the Life Cycle 3
1.3 Trade-off Analysis to Identify System Value 3
1.4 Trade-off Analysis to Identify System Uncertainties and Risks 6
1.5 Trade-off Analyses can Integrate Value and Risk Analysis 6
1.6 Trade-off Analysis in the Systems Engineering Decision Management Process 8
1.7 Trade-off Analysis Mistakes of Omission and Commission 9
1.7.1 Mistakes of Omission 12
1.7.2 Mistakes of Commission 15
1.7.3 Impacts of the Trade-Off Analysis Mistakes 18
1.8 Overview of the Book 20
1.8.1 Illustrative Examples and Techniques Used in the Book 24
1.9 Key Terms 24
1.10 Exercises 25
References 26
2 A Conceptual Framework and Mathematical Foundation for Trade-Off Analysis 29
Gregory S. Parnell, Azad M. Madni and Robert F. Bordley
2.1 Introduction 29
2.2 Trade-Off Analysis Terms 30
2.3 Influence Diagram of the Tradespace 31
2.3.1 Stakeholder Needs System Functions and Requirements 33
2.3.2 Objectives 33
2.3.3 System Alternatives 34
2.3.4 Uncertainty 36
2.3.5 Preferences and Evaluation of Alternatives 37
2.3.6 Resource Analysis 44
2.3.7 An Integrated Trade-Off Analyses 44
2.4 Tradespace Exploration 46
2.5 Summary 46
2.6 Key Words 47
2.7 Exercises 48
References 48
3 Quantifying Uncertainty 51
Robert F. Bordley
3.1 Sources of Uncertainty in Systems Engineering 51
3.2 The Rules of Probability and Human Intuition 52
3.3 Probability Distributions 56
3.3.1 Calculating Probabilities from Experiments 56
3.3.2 Calculating Complex Probabilities from Simpler Probabilities 58
3.3.3 Calculating Probabilities Using Parametric Distributions 59
3.3.4 Applications of Parametric Probability Distributions 62
3.4 Estimating Probabilities 66
3.4.1 Using Historical Data 66
3.4.2 Using Human Judgment 68
3.4.3 Biases in Judgment 70
3.5 Modeling Using Probability 72
3.5.1 Bayes Nets 72
3.5.2 Monte Carlo Simulation 75
3.5.3 Monte Carlo Simulation with Dependent Uncertainties 76
3.5.4 Monte Carlo Simulation with Partial Information on Output Values 77
3.5.5 Variations on Monte Carlo Simulation 78
3.5.6 Sensitivity Analysis 78
3.6 Summary 81
3.7 Key Terms 81
3.8 Exercises 83
References 86
4 Analyzing Resources 91
Edward A. Pohl, Simon R. Goerger and Kirk Michealson
4.1 Introduction 91
4.2 Resources 92
4.2.1 People 92
4.2.2 Facilities 95
4.2.3 Costs 95
4.2.4 Resource Space 99
4.3 Cost Analysis 99
4.3.1 Cost Estimation 102
4.3.2 Cost Estimation Techniques 108
4.3.3 Learning Curves 120
4.3.4 Net Present Value 125
4.3.5 Monte Carlo Simulation 130
4.3.6 Sensitivity Analysis 134
4.4 Affordability Analysis 135
4.4.1 Background 136
4.4.2 The Basics of Affordability Analysis Are Not Difficult 137
4.4.3 DoD Comparison of Cost Analysis and Affordability Analysis 138
4.4.4 Affordability Analysis Definitions 139
4.4.5 "Big A" Affordability Analysis Process Guide 141
4.5 Key Terms 147
4.6 Excercises 149
References 152
5 Understanding Decision Management 155
Matthew Cilli and Gregory S. Parnell
5.1 Introduction 155
5.2 Decision Process Context 156
5.3 Decision Process Activities 157
5.3.1 Frame Decision 159
5.3.2 Develop Objectives and Measures 163
5.3.3 Generate Creative Alternatives 171
5.3.4 Assess Alternatives via Deterministic Analysis 180
5.3.5 Synthesize Results 183
5.3.6 Develop Multidimensional Value Model 187
5.3.7 Identify Uncertainty and Conduct Probabilistic Analysis 190
5.3.8 Assess Impact of Uncertainty 192
5.3.9 Improve Alternatives 196
5.3.10 Communicating Trade-Offs 197
5.3.11 Present Recommendation and Implementation Plan 197
5.4 Summary 199
5.5 Key Terms 199
5.6 Exercises 200
References 201
6 Identifying Opportunities 203
Donna H. Rhodes and Simon R. Goerger
6.1 Introduction 203
6.2 Knowledge 205
6.2.1 Domain Knowledge 205
6.2.2 Technical Knowledge 205
6.2.3 Business Knowledge 205
6.2.4 Expert Knowledge 206
6.2.5 Stakeholder Knowledge 206
6.3 Decision Traps 207
6.4 Techniques 210
6.4.1 Interviews 210
6.4.2 Focus Groups 213
6.4.3 Surveys 215
6.5 Tools 219
6.5.1 Concept Map 219
6.5.2 System Boundary 220
6.5.3 Decision Hierarchy 220
6.5.4 Issues List 221
6.5.5 Vision Statement 221
6.5.6 Influence Diagram 222
6.5.7 Selecting Appropriate Tools and Techniques 223
6.6 Illustrative Examples 223
6.6.1 Commercial 223
6.6.2 Defense 226
6.7 Key Terms 228
6.8 Exercises 230
References 230
7 Identifying Objectives and Value Measures 233
Gregory S. Parnell and William D. Miller
7.1 Introduction 233
7.2 Value-Focused Thinking 234
7.2.1 Four Major VFT Ideas 235
7.2.2 Benefits of VFT 235
7.3 Shareholder and Stakeholder Value 236
7.3.1 Private Company Example 237
7.3.2 Government Agency Example 237
7.4 Challenges in Identifying Objectives 238
7.5 Identifying the Decision Objectives 239
7.5.1 Questions to Help Identify Decision Objectives 239
7.5.2 How to Get Answers to the Questions 240
7.6 The Financial or Cost Objective 241
7.6.1 Financial Objectives for Private Companies 241
7.6.2 Cost Objective for Public Organizations 242
7.7 Developing Value Measures 243
7.8 Structuring Multiple Objectives 243
7.8.1 Value Hierarchies 244
7.8.2 Techniques for Developing Value Hierarchies 245
7.8.3 Value Hierarchy Best Practices 247
7.8.4 Cautions about Cost and Risk Objectives 248
7.9 Illustrative Examples 248
7.9.1 Military Illustrative Example 248
7.9.2 Homeland Security Illustrative Example 250
7.10 Summary 250
7.11 Key Terms 252
7.12 Exercises 253
References 255
8 Developing and Evaluating Alternatives 257
C. Robert Kenley, Clifford Whitcomb and Gregory S. Parnell
8.1 Introduction 257
8.2 Overview of Decision-making Creativity and Teams 258
8.2.1 Approaches to Decision-Making 258
8.2.2 Cognitive Methods for Creating Alternatives 260
8.2.3 Key Concepts for Building and Operating Teams 260
8.3 Alternative Development Techniques 263
8.3.1 Structured Creativity Methods 263
8.3.2 Morphological Box 266
8.3.3 Pugh Method for Alternative Generation 270
8.3.4 TRIZ for Alternative Development 271
8.4 Assessment of Alternative Development Techniques 275
8.5 Alternative Evaluation Techniques 276
8.5.1 Decision-Theory-Based Approaches 276
8.5.2 Pugh Method for Alternative Evaluation 276
8.5.3 Axiomatic Approach to Design (AAD) 277
8.5.4 TRIZ for Alternative Evaluation 280
8.5.5 Design of Experiments (DOE) 280
8.5.6 Taguchi Approach 282
8.5.7 Quality Function Deployment (QFD) 283
8.5.8 Analytic Hierarchy Process AHP 287
8.6 Assessment of Alternative Evaluation Techniques 290
8.7 Key Terms 290
8.8 Exercises 290
References 293
9 An Integrated Model for Trade-Off Analysis 297
Alexander D. MacCalman, Gregory S. Parnell and Sam Savage
9.1 Introduction 297
9.2 Conceptual Design Example 298
9.3 Integrated Approach Influence Diagram 300
9.3.1 Decision Nodes 300
9.3.2 Uncertainty Nodes 303
9.3.3 Constant Node 310
9.3.4 Value Nodes 314
9.4 Other Types of Trade-Off Analysis 322
9.5 Simulation Tools 322
9.5.1 Monte Carlo Simulation Proprietary Add-Ins 324
9.5.2 The Discipline of Probability Management 324
9.5.3 SIPmathTM Tool in Native Excel 324
9.5.4 Model Building Steps 325
9.6 Summary 329
9.7 Key Terms 330
9.8 Exercises 331
References 335
10 Exploring Concept Trade-Offs 337
Azad M. Madni and Adam M. Ross
10.1 Introduction 337
10.1.1 Key Concepts Concept Trade-Offs and Concept Exploration 341
10.2 Defining the Concept Space and System Concept of Operations 345
10.3 Exploring the Concept Space 346
10.3.1 Storytelling-Enabled Tradespace Exploration 346
10.3.2 Decisions and Outcomes 347
10.3.3 Contingent Decision-Making 347
10.4 Trade-off Analysis Frameworks 348
10.5 Tradespace and System Design Life Cycle 349
10.6 From Point Trade-offs to Tradespace Exploration 351
10.7 Value-based Multiattribute Tradespace Analysis 351
10.7.1 Tradespace Exploration and Sensitivity Analysis 353
10.7.2 Tradespace Exploration and Uncertainty 354
10.7.3 Tradespace Exploration with Spiral Development 356
10.7.4 Tradespace Exploration in Relation to Optimization and Decision Theory 356
10.8 Illustrative Example 359
10.8.1 Step 1: Determine Key Decision-Makers 359
10.8.2 Step 2: Scope and Bound the Mission 360
10.8.3 Step 3: Elicit Attributes and Utilities (Preference Capture) 360
10.8.4 Step 4: Define Design Vector Elements (Concept Generation) 362
10.8.5 Step 5: Develop Model(s) (Evaluation) 362
10.8.6 Step 6: Generate the Tradespace (Computation) 364
10.8.7 Step 7: Explore the Tradespace (Analysis and Synthesis) 365
10.9 Conclusions 369
10.10 Key Terms 371
10.11 Exercises 372
References 372
11 Architecture Evaluation Framework 377
James N. Martin
11.1 Introduction 377
11.1.1 Architecture in the Decision Space 378
11.1.2 Architecture Evaluation 379
11.1.3 Architecture Views and Viewpoints 380
11.1.4 Stakeholders 382
11.1.5 Stakeholder Concerns 382
11.1.6 Architecture versus Design 383
11.1.7 On the Uses of Architecture 384
11.1.8 Standardizing on an Architecture Evaluation Strategy 384
11.2 Key Considerations in Evaluating Architectures 385
11.2.1 Plan-Driven Evaluation Effort 386
11.2.2 Objectives-Driven Evaluation 387
11.2.3 Assessment versus Analysis 387
11.3 Architecture Evaluation Elements 389
11.3.1 Architecture Evaluation Approach 389
11.3.2 Architecture Evaluation Objectives 390
11.3.3 Evaluation Approach Examples 391
11.3.4 Value Assessment Methods 391
11.3.5 Value Assessment Criteria 393
11.3.6 Architecture Analysis Methods 394
11.4 Steps in an Architecture Evaluation Process 396
11.5 Example Evaluation Taxonomy 398
11.5.1 Business Impact Factors 398
11.5.2 Mission Impact Factors 398
11.5.3 Architecture Attributes 399
11.6 Summary 400
11.7 Key Terms 400
11.8 Exercises 402
References 402
12 Exploring the Design Space 405
Clifford Whitcomb and Paul Beery
12.1 Introduction 405
12.2 Example 1: Liftboat 406
12.2.1 Liftboat Fractional Factorial Design of Experiments 406
12.2.2 Liftboat Design Trade-Off Space 409
12.2.3 Liftboat Uncertainty Analysis 411
12.2.4 Liftboat Example Summary 411
12.3 Example 2: Cruise Ship Design 411
12.3.1 Cruise Ship Taguchi Design of Experiments 411
12.3.2 Cruise Ship Design Trade-Off Space 412
12.3.3 Cruise Ship Example Summary 416
12.4 Example 3: NATO Naval Surface Combatant Ship 417
12.4.1 NATO Surface Combatant Ship Stakeholder Need 418
12.4.2 NATO Surface Combatant Ship Box-Behnken Design of Experiments 420
12.4.3 NATO Surface Combatant Ship Cost-Effectiveness Trade-Off 421
12.4.4 NATO Surface Combatant Ship Design Tradespace 421
12.4.5 NATO Surface Combatant Ship Design Trade-Off 422
12.4.6 NATO Surface Combatant Ship Trade-Off Summary 430
12.5 Key Terms 431
12.6 Exercises 433
References 435
13 Sustainment Related Models and Trade Studies 437
John E. MacCarthy and Andres Vargas
13.1 Introduction 437
13.2 Availability Modeling and Trade Studies 439
13.2.1 FMDS Background 439
13.2.2 FMDS Availability Trade Studies 449
13.2.3 Section Synopsis 453
13.3 Sustainment Life Cycle Cost Modeling and Trade Studies14 454
13.3.1 The Total System Life Cycle Model 454
13.3.2 The O&S Cost Model 456
13.3.3 Life Cycle Cost Trade Study 459
13.4 Optimization in Availability Trade Studies 464
13.4.1 Setting Up the Optimization Problem 464
13.4.2 Instantiating the Optimization Model 465
13.4.3 Discussion of the Optimization Model Results 468
13.4.4 Deterministic Sensitivity Analysis 469
13.5 Monte Carlo Modeling 471
13.5.1 Input Probability Distributions for the Monte Carlo Model 471
13.5.2 Monte Carlo Simulation Results 472
13.5.3 Stochastic Sensitivity Analysis 473
13.6 Chapter Summary 475
13.7 Key Terms 476
13.8 Exercises 478
References 482
14 Performing Programmatic Trade-Off Analyses 483
Gina Guillaume-Joseph and John E. MacCarthy
14.1 Introduction 483
14.2 System Acceptance Decisions and Trade Studies 485
14.2.1 Acceptance Decision Framework 486
14.2.2 Calculating the Confidence That a System Is "Good" 491
14.2.3 Acceptance Test Design and Trade Studies 493
14.2.4 A "Delay Fix and Test" Cost Model 499
14.2.5 The Integrated Decision Model 504
14.2.6 Conclusions 511
14.3 Product Cancelation Decision Trade Study 512
14.3.1 Introduction 512
14.3.2 Significance 513
14.3.3 Defining Failure 514
14.3.4 Developing the Predictive Model 519
14.3.5 Research Results 522
14.3.6 Model Implementation In Industry 528
14.3.7 Predictive Model Deployment in Industry 530
14.3.8 When the Decision Has Been Made to Cancel the System 536
14.3.9 Conclusion 537
14.4 Product Retirement Decision Trade Study 538
14.4.1 Introduction 538
14.4.2 Legacy HR Systems 539
14.4.3 The US NAVY Retirement and Decommission Program for Nuclear-Powered Vessels 544
14.4.4 Decision Analysis for Decommissioning Offshore Oil and Gas Platforms in California 551
14.4.5 System Retirement and Decommissioning Strategy 559
14.4.6 Conclusion 561
14.5 Key Terms 562
14.6 Exercises 564
References 566
15 Summary and Future Trends 571
Gregory S. Parnell and Simon R. Goerger
15.1 Introduction 571
15.2 Major Trade-Off Analysis Themes 572
15.2.1 Use Standard Systems Engineering Terminology 572
15.2.2 Avoid the Mistakes of Omission and Commission 572
15.2.3 Use a Decision Management Framework 572
15.2.4 Use Decision Analysis as the Mathematical Foundation 573
15.2.5 Explicitly Define the Decision Opportunity 573
15.2.6 Identify and Structure Decision Objectives and Measures 574
15.2.7 Identify Creative Doable Alternatives 574
15.2.8 Use the Most Appropriate Modeling and Simulation Technique for the Life Cycle Stage 575
15.2.9 Include Resource Analysis in the Trade-Off Analysis 575
15.2.10 Explicitly Consider Uncertainty 575
15.2.11 Identify the Cost Value Schedule and Risk Drivers 575
15.2.12 Provide an Integrated Framework for Cost Value and Risk Analyses 576
15.3 Future of Trade-Off Analysis 576
15.3.1 Education and Training of Systems Engineers 577
15.3.2 Systems Engineering Methodologies and Tools 577
15.3.3 Emergent Tradespace Factors 580
15.4 Summary 581
References 581
Index 583
About the Authors
Gregory S. Parnell is Director, M.S. in Operations Management and Research Professor in the Department of Industrial Engineering at the University of Arkansas. He teaches systems engineering, decision analysis, operations management, and IE design courses. He coedited Decision Making for Systems Engineering and Management, Wiley Series in Systems Engineering, 2nd Ed, Wiley & Sons Inc., 2011, and cowrote the Wiley & Sons Handbook of Decision Analysis, 2013. Dr Parnell has taught at West Point, the United Stated Air Force Academy, Virginia Commonwealth University, and the Air Force Institute of Technology. He is a fellow of the International Committee on Systems Engineering (INCOSE), the Institute for Operations Research & Management Science, Military Operations Research Society, the Society for Decision Professionals, and the Lean Systems Society. During his Air Force career, he served in a variety of R&D positions and operations research positions including at the Pentagon where he led two analysis divisions supporting senior Air Force leadership. He is a retired Colonel in the US Air Force. Dr Parnell received a B.S. in Aerospace Engineering from the University of Buffalo, an M.E. in Industrial & Systems Engineering from the University of Florida, an M.S. in Systems Management from the University of Southern California, and a Ph.D. in Engineering-Economic Systems from Stanford University.
Robert F. Bordley is an adjunct professor of decision analysis and systems engineering at the University of Michigan and a full-time consultant for Booz Allen Hamilton. Bob was formerly technical Fellow at General Motors and a Program Director at the National Science Foundation. His Ph.D., M.S., and MBA in Operations Research are from the University of California, Berkeley with an M.S. in Systems Science, B.S. in Physics, and B.A. in Public Policy from Michigan State University. He is an INCOSE-certified expert systems engineering professional (ESEP), an INFORMS-certified analytic professional (CAP), a professional statistician (PStat), and a certified Project Management Professional (PMP). Bob is a Fellow of the Institute for Operations Research and Management Sciences, a Fellow of the American Statistical Association, and a Fellow of the Society of Decision Professionals. Bob also received the 2004 Best Decision Analysis Publication Award. At the National Science Foundation, he served as Program Director for Decision, Risk and Management Sciences. As Technical Fellow at General Motors, he received GM's Chairman Award, President's Council Award, Research Award of Excellence, GM's Engineering Award of Excellence, and UAW-GM Quality Award. Bob led the mission analysis group in Project Trilby, which helped launch GM's vehicle systems engineering effort as well as its R&D portfolio management group. Bob was also a Technical Director on GM's corporate strategy staff and served as internal consultant to GM's marketing, product planning, and quality engineering staffs. At Booz Allen Hamilton, Bob supports requirements management and concept selection for the Army.
Matthew Cilli received his Ph.D. in Systems Engineering from Stevens Institute of Technology in Hoboken, NJ, and leads an analytics group at the US Army's Armament Research Development and Engineering Center (ARDEC) in Picatinny, NJ. His research interests are focused on improving strategic decision-making through the integrated application of holistic thinking and analytics. Prior to his current position, Dr Cilli accumulated over 20 years of experience developing proposals, securing resources, and leading effective technology development programs for the US Army. Dr Cilli graduated from Villanova University, Villanova, PA, with a Bachelor of Electrical Engineering and a Minor in Mathematics in May 1989. He is also a graduate of the Polytechnic University, Brooklyn, NY, with a Master of Science - Electrical Engineering received in January 1992 and in May 1998, graduated from the University of Pennsylvania, Wharton Business School, Philadelphia, PA, with a Masters of Technology Management.
Simon R. Goerger is the ERDC Director of the Institute for Systems Engineering Research (ISER) at the Information Technology Laboratory (ITL) of the Engineer Research and Development Center (ERDC) in Vicksburg, MS. He has been an Operations Research Analyst with the US Army Corps of Engineers since 2012. Prior to working for the Corps of Engineers, he was a Colonel in the US Army serving as the Director of the Department of Defense Readiness Reporting System (DRRS) Implementation Office (DIO). Simultaneously, he served as Senior Defense Readiness Analyst in the Office of the Undersecretary of Defense (Personnel and Readiness). Simon has served as an Assistant Professor and the Director of the Operations Research Center of Excellence in the Department of Systems Engineering at the United States Military Academy, West Point, NY, before deploying to serve as the Joint Multinational Networks Division Chief, Coalition Forces Land Combatant Command/US Army Central Command, Kuwait. He received his Bachelor of Science from the United States Military Academy, his Master of Science (M.S.) in National Security Strategy from the National War College, and his M.S. in Computer Science and Doctorate of Philosophy in Modeling and Simulations from the Naval Postgraduate School. He is a board member for the Military Operations Research Society. His research interests include decision analysis, systems modeling, tradespace analysis, and combat modeling and simulations.
Dr Gina Guillaume-Joseph is an Information Systems Engineer at The MITRE Corporation in McLean, Virginia. In her current role, she acts as a trusted advisor to senior leadership in Federal Agencies by partnering with them to design enhancements to their work systems. Dr Guillaume-Joseph's work has led to improvements that allow the systems and processes to operate more efficiently and effectively in fulfillment of specific functions. Her various roles have included project manager, software developer, test engineer, and quality assurance engineer within the private, government consulting, nonprofit, and telecommunications arenas. Dr Guillaume-Joseph is President of the INCOSE Washington Metro Area (WMA) Chapter. Dr Guillaume-Joseph has a strong record of success based on direct personal contributions. She leads and develops teams that are adaptive, flexible, and highly responsive in the exceptionally dynamic environment of Government support. Her accomplishments and successes are based on strong program performance, leadership discipline, a commitment to developing relevant, innovative and adaptive solutions, and a vigilant focus on best value solutions for her clients. Dr Guillaume-Joseph has advanced knowledge of software development lifecycle activities, such as agile, waterfall, iterative, incremental, and associated processes including planning, requirements management, design and development, testing, and deployment. Her strong communication skills make her adept at conveying specialized technical information to nontechnical audiences. Dr Guillaume-Joseph received her B.A. in Computer Science from Boston College and M.S. in Information Technology Systems from the University of Maryland. She obtained her Ph.D. in Systems Engineering from George Washington University with a topic focused on Predicting Software Project Failure Outcomes using Predictive Analytics and Modeling.
C. Robert Kenley is an Associate Professor of Engineering Practice in Purdue's School of Industrial Engineering in West Lafayette, IN. He teaches courses in systems engineering at Purdue and has over 30 years of experience in industry, academia, and government as a practitioner, consultant, and researcher in systems engineering. He has published papers on systems requirements, technology readiness assessment and forecasting, Bayes nets, applied meteorology, the impacts of nuclear power plants on employment, agent-based simulation, and model-based systems engineering. Professor Kenley holds a Bachelor of Science in Management from Massachusetts Institute of Technology (MIT), a Master of Science in Statistics from Purdue University, and a Doctor of Philosophy in Engineering-Economic Systems from Stanford University.
Azad M. Madni is a Professor of Astronautical Engineering and the Technical Director of the multidisciplinary Systems Architecting and Engineering (SAE) Program at the University of Southern California's Viterbi School of Engineering. He is also a Professor of USC's Keck School of Medicine and Rossier School of Education. Dr Madni is the founder and Chairman of Intelligent Systems Technology, Inc., a high-tech company specializing in modeling, simulation, and gaming technologies for education and training. His research has been sponsored by several prominent government agencies including DARPA, DHS S&T, MDA, DTRA, ONR, AFOSR, AFRL, ARI, RDECOM, NIST, DOE, and NASA, as well as major aerospace companies including Boeing, Northrop Grumman, and Raytheon. He is the Co-Editor-in-Chief of Engineered Resilient Systems: Challenges and Opportunities in the 21st Century, Procedia Computer Science, 2014. His recent awards include the 2011 INCOSE Pioneer Award and the 2014 Lifetime Achievement Award from INCOSE-LA. He is a Fellow of AAAS, AIAA, IEEE, INCOSE, SDPS, and IETE. He is the Strategic Advisor of the INCOSE Systems Engineering Journal. He received his B.S., M.S., and Ph.D. degrees from the University of California, Los Angeles. He is also a graduate of AEA/Stanford Executive Institute for senior executives.
Alexander D. MacCalman is an Army Special Forces Officer in the Operations...
