Agile Analytics
A Value-Driven Approach to Business Intelligence and Data Warehousing
Published on 4. August 2011
Book
Paperback/Softback
368 pages
978-0-321-50481-4 (ISBN)
Description
Using Agile methods, you can bring far greater innovation, value, and quality to any data warehousing (DW), business intelligence (BI), or analytics project. However, conventional Agile methods must be carefully adapted to address the unique characteristics of DW/BI projects. In Agile Analytics, Agile pioneer Ken Collier shows how to do just that.
Collier introduces platform-agnostic Agile solutions for integrating infrastructures consisting of diverse operational, legacy, and specialty systems that mix commercial and custom code. Using working examples, he shows how to manage analytics development teams with widely diverse skill sets and how to support enormous and fast-growing data volumes. Collier's techniques offer optimal value whether your projects involve "back-end" data management, "front-end" business analysis, or both.
Part I focuses on Agile project management techniques and delivery team coordination, introducing core practices that shape the way your Agile DW/BI project community can collaborate toward success
Part II presents technical methods for enabling continuous delivery of business value at production-quality levels, including evolving superior designs; test-driven DW development; version control; and project automation
Collier brings together proven solutions you can apply right now-whether you're an IT decision-maker, data warehouse professional, database administrator, business intelligence specialist, or database developer. With his help, you can mitigate project risk, improve business alignment, achieve better results-and have fun along the way.
Collier introduces platform-agnostic Agile solutions for integrating infrastructures consisting of diverse operational, legacy, and specialty systems that mix commercial and custom code. Using working examples, he shows how to manage analytics development teams with widely diverse skill sets and how to support enormous and fast-growing data volumes. Collier's techniques offer optimal value whether your projects involve "back-end" data management, "front-end" business analysis, or both.
Part I focuses on Agile project management techniques and delivery team coordination, introducing core practices that shape the way your Agile DW/BI project community can collaborate toward success
Part II presents technical methods for enabling continuous delivery of business value at production-quality levels, including evolving superior designs; test-driven DW development; version control; and project automation
Collier brings together proven solutions you can apply right now-whether you're an IT decision-maker, data warehouse professional, database administrator, business intelligence specialist, or database developer. With his help, you can mitigate project risk, improve business alignment, achieve better results-and have fun along the way.
Reviews / Votes
"This book does a great job of explaining why and how you would implement Agile Analytics in the real world. Ken has many lessons learned from actually implementing and refining this approach. Business Intelligence is definitely an area that can benefit from this type of discipline." -Dale Zinkgraf, Sr. Business Intelligence Architect"One remarkable aspect of Agile Analytics is the breadth of coverage-from product and backlog management to Agile project management techniques, from self-organizing teams to evolutionary design practices, from automated testing to build management and continuous integration. Even if you are not on an analytics project, Ken's treatment of this broad range of topics related to products with a substantial data-oriented flavor will be useful for and beyond the analytics community."
-Jim Highsmith, Executive Consultant, ThoughtWorks, Inc., and author of Agile Project Management
"Agile methods have transformed software development, and now it's time to transform the analytics space. Agile Analytics provides the knowledge needed to make the transformation to Agile methods in delivering your next analytics projects."
-Pramod Sadalage, coauthor of Refactoring Databases: Evolutionary Database Design
"This book captures the fundamental strategies for successful business intelligence/analytics projects for the coming decade. Ken Collier has raised the bar for analytics practitioners-are you up to the challenge?"
-Scott Ambler, Chief Methodologist for Agile and Lean, IBM Rational Founder, Agile Data Method
"A sweeping presentation of the fundamentals that will empower teams to deliver high-quality, high-value, working business intelligence systems far more quickly and cost effectively than traditional software development methods."
-Ralph Hughes, author of Agile Data Warehousing
More details
Person
Ken Collier has worked with Agile methods since 2003, and pioneered the integration of Agile methods with data warehousing, business intelligence, and analytics to create the Agile Analytics style. He continues to refine these ideas as technical lead and project manager on several Agile DW/BI project teams. Collier frequently trains DW/BI teams in Agile Analytics, and has been a keynote speaker on the subject at HEDW (Higher Education Data Warehouse) 2011 and multiple TDWI (The Data Warehousing Institute) World Conferences. He is founder and president of KWC Technologies, Inc., and a senior consultant in the Cutter Consortium's Agile Development and Business Intelligence practice areas.
Content
Foreword by Jim Highsmith xv
Foreword by Wayne Eckerson xvii
Preface xix
Acknowledgments xxxiii
About the Author xxxv
Part I: Agile Analytics: Management Methods 1
Chapter 1: Introducing Agile Analytics 3
Alpine-Style Systems Development 4
What Is Agile Analytics? 7
Data Warehousing Architectures and Skill Sets 13
Why Do We Need Agile Analytics? 16
Introducing FlixBuster Analytics 22
Wrap-Up 23
Chapter 2: Agile Project Management 25
What Is Agile Project Management? 26
Phased-Sequential DW/BI Development 30
Envision ? Explore Instead of Plan ? Do 32
Changing the Role of Project Management 35
Making Sense of Agile "Flavors" 36
Tenets of Agility 39
Wrap-Up 56
Chapter 3: Community, Customers, and Collaboration 59
What Are Agile Community and Collaboration? 60
The Agile Community 64
A Continuum of Trust 67
The Mechanics of Collaboration 69
Consumer Collaboration 73
Doer Collaboration 77
Planner Collaboration 78
Precursors to Agility 80
Wrap-Up 82
Chapter 4: User Stories for BI Systems 85
What Are User Stories? 86
User Stories versus Requirements 89
From Roles to Use Cases to User Stories 92
Decomposing Epics 99
What's the Smallest, Simplest Thing? 103
Story Prioritization and Backlog Management 107
Story-Point Estimating 111
Parking Lot Diagrams 117
Wrap-Up 119
Chapter 5: Self-Organizing Teams Boost Performance 121
What Is a Self-Organizing Team? 122
Self-Organization Requires Self-Discipline 127
Self-Organization Requires Shared Responsibility 128
Self-Organization Requires Team Working Agreements 130
Self-Organization Requires Honoring Commitments 132
Self-Organization Requires Glass-House Development 134
Self-Organizing Requires Corporate Alignment 136
Wrap-Up 137
Part II: Agile Analytics: Technical Methods 139
Chapter 6: Evolving Excellent Design 141
What Is Evolutionary Design? 144
How Much Up-Front Design? 148
Agile Modeling 149
Data Model Patterns 152
Managing Technical Debt 154
Refactoring 157
What Is Refactoring? 159
Deploying Warehouse Changes 167
Other Reasons to Take an Evolutionary Approach 171
Case Study: Adaptive Warehouse Architecture 174
Wrap-Up 189
Chapter 7: Test-Driven Data Warehouse Development 193
What Is Agile Analytics Testing? 194
Agile Testing Framework 197
BI Test Automation 201
Sandbox Development 211
Test-First BI Development 215
BI Testing Guidelines 220
Setup Time 221
Functional BI Testing 222
Wrap-Up 223
Chapter 8: Version Control for Data Warehousing 225
What Is Version Control? 226
The Repository 230
Working with Files 233
Organizing the Repository 240
Tagging and Branching 245
Choosing an Effective Tool 252
Wrap-Up 254
Chapter 9: Project Automation 257
What Is Project Automation? 258
Getting Started 261
Build Automation 262
Continuous Integration 274
Push-Button Releases 281
Wrap-Up 288
Chapter 10: Final Words 291
Focus on the Real Problem 291
Being Agile versus Doing Agile 293
Gnarly Problems 296
What about Emerging Technologies? 298
Adoption Strategies 299
Closing Thoughts . . . 306
References and Recommended Reading 309
Index 315
Foreword by Wayne Eckerson xvii
Preface xix
Acknowledgments xxxiii
About the Author xxxv
Part I: Agile Analytics: Management Methods 1
Chapter 1: Introducing Agile Analytics 3
Alpine-Style Systems Development 4
What Is Agile Analytics? 7
Data Warehousing Architectures and Skill Sets 13
Why Do We Need Agile Analytics? 16
Introducing FlixBuster Analytics 22
Wrap-Up 23
Chapter 2: Agile Project Management 25
What Is Agile Project Management? 26
Phased-Sequential DW/BI Development 30
Envision ? Explore Instead of Plan ? Do 32
Changing the Role of Project Management 35
Making Sense of Agile "Flavors" 36
Tenets of Agility 39
Wrap-Up 56
Chapter 3: Community, Customers, and Collaboration 59
What Are Agile Community and Collaboration? 60
The Agile Community 64
A Continuum of Trust 67
The Mechanics of Collaboration 69
Consumer Collaboration 73
Doer Collaboration 77
Planner Collaboration 78
Precursors to Agility 80
Wrap-Up 82
Chapter 4: User Stories for BI Systems 85
What Are User Stories? 86
User Stories versus Requirements 89
From Roles to Use Cases to User Stories 92
Decomposing Epics 99
What's the Smallest, Simplest Thing? 103
Story Prioritization and Backlog Management 107
Story-Point Estimating 111
Parking Lot Diagrams 117
Wrap-Up 119
Chapter 5: Self-Organizing Teams Boost Performance 121
What Is a Self-Organizing Team? 122
Self-Organization Requires Self-Discipline 127
Self-Organization Requires Shared Responsibility 128
Self-Organization Requires Team Working Agreements 130
Self-Organization Requires Honoring Commitments 132
Self-Organization Requires Glass-House Development 134
Self-Organizing Requires Corporate Alignment 136
Wrap-Up 137
Part II: Agile Analytics: Technical Methods 139
Chapter 6: Evolving Excellent Design 141
What Is Evolutionary Design? 144
How Much Up-Front Design? 148
Agile Modeling 149
Data Model Patterns 152
Managing Technical Debt 154
Refactoring 157
What Is Refactoring? 159
Deploying Warehouse Changes 167
Other Reasons to Take an Evolutionary Approach 171
Case Study: Adaptive Warehouse Architecture 174
Wrap-Up 189
Chapter 7: Test-Driven Data Warehouse Development 193
What Is Agile Analytics Testing? 194
Agile Testing Framework 197
BI Test Automation 201
Sandbox Development 211
Test-First BI Development 215
BI Testing Guidelines 220
Setup Time 221
Functional BI Testing 222
Wrap-Up 223
Chapter 8: Version Control for Data Warehousing 225
What Is Version Control? 226
The Repository 230
Working with Files 233
Organizing the Repository 240
Tagging and Branching 245
Choosing an Effective Tool 252
Wrap-Up 254
Chapter 9: Project Automation 257
What Is Project Automation? 258
Getting Started 261
Build Automation 262
Continuous Integration 274
Push-Button Releases 281
Wrap-Up 288
Chapter 10: Final Words 291
Focus on the Real Problem 291
Being Agile versus Doing Agile 293
Gnarly Problems 296
What about Emerging Technologies? 298
Adoption Strategies 299
Closing Thoughts . . . 306
References and Recommended Reading 309
Index 315