1
The Importance of Being Data Savvy

One of the central causes of the Global Financial Crisis of 2008 and the Great Recession that followed was bad data. The available information on key financial instruments at the heart of the crisis was faulty. Even so, investors - from individual homeowners to our most storied financial institutions - bet billions of dollars on that information. The root cause of one of the worst economic meltdowns in history was a combination of poor-quality information, lack of transparency on its origins and limitations, and wishful thinking (and in some cases outright fraud) on the part of the people and organizations generating and analyzing and sharing the data.

The result was bankruptcies, crashing markets, thousands of jobs lost, hundreds of billions of dollars in government bailouts, a disaster that took years to recover from, and lasting damage to the public's trust in financial and governmental institutions.1

This book is the story of the central role of data in the way citizens, consumers, companies, institutions, and governments perceive and act in the world - and how we can all improve our skills and interactions within that data ecosystem. Limitations in our ability to use data effectively, together with inaccurate data or data of poor quality, lead to widespread misunderstanding, uncertainty, and deception: problems in which we all play a part that undermine the common workings of the society.

Why This Matters

We'll get back to the details of the data failures that created the Global Financial Crisis shortly. But first, let me explain why I care about this and why you should, too.

I have the privilege of serving as president and chief executive officer of NORC, one of the largest independent research organizations in the world. NORC is an objective, nonpartisan, global research institute that conducts hundreds of millions of dollars in research every year for governments, nonprofits, and businesses in the United States and many other nations. My background includes 35 years of conducting research using a wide range of data at some of the world's leading research institutes, and in private consulting, and also serving as a senior health policy advisor at the US Department of Health and Human Services.

I am committed not just to promoting and supporting honest, unbiased, and transparent research but also to helping everyone understand how data are generated2 and how to use and interpret data to inform their most important decisions.

Today, many forces combine to create a vast sea of information of varying quality, leading to uncertainty across all aspects of society. These forces include the creation of flawed or biased data, a lack of transparency about data sources, and the distortion of data to manipulate and mislead people. This book provides a framework for understanding all forms of data and their limitations, and what I hope will become common expectations about appropriate use of data. The idea is to live in a fact-forward world in which we consistently advance facts as the basis for making critical decisions. While this may sound elusive, I believe that the promise of a fact-forward world is before us. To get there, all of us as individuals and as a society must prioritize the development of better data skills, in how we create, access, use, and share data. The broad development of these skills across these multiple dimensions is what I refer to throughout the book as becoming "data savvy."

While it has been more than 15 years since the Global Financial Crisis, the data challenges it reveals are just as relevant today as they were in the 2000s. Moreover, we are now sufficiently far removed from these events to be able to look back at them and assess what went wrong and why. Four data problems led directly to this crisis: failures of data integrity, failures of data transparency, failures of data neutrality, and failures of data literacy. These problems remain highly relevant today, which means that we continue to be very much at risk for additional global disasters based on information failure.

The Role of Bad Data in the 2008 Global Financial Crisis

It was easy to get a mortgage in the 2000s.3 Consumers with limited incomes, poor credit, or inadequate down payments could still qualify for low-documentation or no-documentation mortgages. Mortgage brokers who made money on loan volume assured borrowers that they had the economic means to take on excessive mortgage debt. And mortgage bankers, incentivized to originate loans, were willing to lend money to underqualified borrowers. Many of these loans were adjustable-rate mortgages with "teaser" interest rates that stayed low for the first two years but increased rapidly thereafter.

According to a paper by the economist Thomas Herndon, 70% of the eventual losses in the mortgage markets were caused by defaults on these low-documentation and no-documentation loans.4 But on their own, these loan defaults would never have brought down global financial markets.

At the center of the crisis was a stack of financial instruments known as CDOs and CDSs. Collateralized debt obligations (CDOs) were bonds based on hundreds or thousands of mortgages, while credit default swaps (CDSs) were insurance on the value of those bonds.

The task of accurately measuring the risk in these instruments fell to the independent bond-rating agencies: Standard & Poor's (S&P), Moody's, and Fitch. An agency like S&P might rate a CDO bond backed by the highest-quality homeowners and mortgages AA, indicating an investment grade bond with a very low risk of default, while a bond backed by lower-quality mortgages might be graded BBB - still investment grade, but with a higher risk of default. The ratings agencies also assigned the highest possible ratings to most of the CDSs, indicating perhaps a 1-in-1,000 risk that their buyers would ever need to pay off the insurance.

The allure of low-risk, high-reward investments is enormous. The investment-grade ratings on CDOs and CDSs encouraged financial institutions throughout Wall Street to buy billions of dollars of them.

As long as home values continued to increase, homeowners were able either to refinance with a new mortgage or to sell their highly mortgaged houses at a profit before their teaser rates expired. Financial firms profited from the bonds and derivatives based on those homes. This in turn further fueled home values and attracted still more questionable borrowers into low-documentation loans to cash in on appreciating prices.

That, of course, is what a bubble looks like. And in 2008 - slowly, and then catastrophically - everything collapsed.

Home buyers began to default on their loans - especially as those two-year low-interest lockup periods began to expire, and their payments ballooned. The CDOs based on those mortgages became worthless. This triggered billions of dollars in insurance payments for the owners of the CDSs. The largest blue-chip investment firms on Wall Street - including AIG, Lehman Brothers, Bear Stearns, and Merrill Lynch - found themselves with massive, completely unanticipated losses. The resulting implosion in financial markets froze monetary liquidity and led to the Great Recession. Despite a $700 billion government bailout for Wall Street, the recession put almost 9 million Americans out of work.

The global financial crisis was caused by the triple whammy of the risky loans, which were then bundled into CDOs and CDSs, and were then rated as low risk by the ratings agencies. Despite the excellent ratings for these investments, they were all built on adjustable-rate mortgages doomed to eventually tumble, creating a highly correlated set of risks that blindsided all the major financial institutions at once. Each step was riddled with limited or bad data. This was the central cause of the crisis.

The Financial Disaster Reveals the Four Types of Data Failure

Now let's ask a crucially important question: Why were the ratings agencies creating the faulty ratings that led to the global financial crisis, even though these agencies' key purpose is to accurately assess risk?

The answer to that question illuminates the four main types of data failure that threaten every part of our global society that depends on data and, as I'll show, that includes virtually everything that government, business, and consumers do. Consider the four failures that led to the overoptimistic bond ratings that brought on the crisis:

• A failure of data integrity.5 Data integrity means that data are based on solid information interpreted in statistically valid ways. But the ratings agencies were not actually assessing risks of default; they were instead looking at broad general characteristics of loan pools, such as the median credit scores of borrowers. Unfortunately, such measures could conceal vast numbers of risky mortgages. As one former Goldman Sachs bond trader explained to the author Michael Lewis, "The ratings agencies didn't really have their own CDO model."6
• A failure of data transparency. Anyone using data to make decisions must be able to understand where the data came from and how they were analyzed. But for the ratings agencies to maintain their proprietary advantage and keep the creators of...