Introduction: My Forays into Failure

All it took was a moment. I was 21. Till then, life was going exactly how I had dreamed of and trained hard for - becoming a professional soccer player. Then a snap, and I felt my knee give way. A bad tackle during a regular practice session had turned into my worst nightmare. It was a career-ending injury. My orthopedic surgeon assured me that I would walk again, but a professional career was not in the cards anymore. To this day, I am not completely over it. Like paper, once crumpled, it cannot be undone. Still functioning, but the scars remain for life.

People often ask me how I came up with the idea of Productive Failure, the notion that we could somehow intentionally design for failure and bootstrap it for deep learning. After all, many people have talked about the value of learning from failure. These accounts, however, are largely reactive; they talk about learning from failure after it happens. The idea of Productive Failure is to be proactive; that is, if failure is so powerful for learning, then we should not wait for it to happen. We should intentionally design for it for deep learning.

I like to believe that I came up with the idea in a moment of brilliance during my doctoral studies at Teachers College, Columbia University, in New York City. The truth, however, is that many things in my life led me to it. As Steve Jobs put it, "The dots connect looking back." My dots were my failures, big and small, which slowly but surely nudged me toward it. But all of it only became clearer in hindsight. The lived experience was anything but.

Soccer Dreams

My journey started a long time ago, in my teenage years. I was in the sixth grade, doing stuff teenagers do. Falling in love was perhaps the most significant and consequential. However, unlike the first crush most people have in school, my first love was soccer.

As a teenager growing up in India, all I ever wanted to be was a professional soccer player. My father bought us our first color television just in time for the 1986 World Cup in Mexico. Diego Maradona was the man of the moment, and everyone I knew wanted to be like Maradona. I did too. Even though I was only in the sixth grade, I went for my high school team trials. In a country where everyone aspires to be the next cricket star, I took to soccer, and quite effortlessly so. And before the end of that academic year, I was the vice-captain and playmaker of my high school soccer team.

When I look back at my soccer years, a simple philosophy stands out. Training in the northern Indian city of Chandigarh in the foothills of the Himachal Mountains, my coach, Mr. Thomas, used to drill into us that matches are often won or lost in the last five to ten minutes of the game. Everyone can play well when they are fresh, he would say, but it is those who can push when extremely exhausted who make the difference in the last passage of play. Therefore, true to that philosophy, he designed our training to focus on preparing us for those game-changing moments.

How? By taking us to the other side of failure.

The idea was to train until you fail, and then you push just a little bit more. Do your push-ups and pull-ups till your arms buckle, and then push some more. Do your endurance training till your body cannot endure anymore, and then push some more. Of course, all this was done in a way that was safe to avoid injuries. Pushing just beyond the limit was sufficient. All the time to take the body to its limit, and then push a little bit more.

Why? Because good things happened on the other side of failure: this is where players maximized their physical and mental strength, and learned how to work the mind and the body together for optimal growth and performance. And our coach was right: when you look at tough games that go right to the end, quite often it is on the other side of failure where these games are won or lost. Lesson learned. In retrospect, this was the first of several dots I was to connect on understanding failure.

Unfortunately, just as I had made it to the national youth team, my soccer career ended with the injury. It was Spring of 1995. I remember it vividly. In one moment, everything that I had trained for in my life till then was gone. I was a case study in depression and failure. As effortless and enjoyable as soccer was, everything else after was quite the opposite, effortful and exhausting. Nothing seemed to work or make sense. So, the only thing I could do was to try to push through my backup option: finishing up my Engineering Bachelor's.

Struggling Through My Engineering Studies

It was not until my final year of engineering school that I started to take my studies seriously, for until then my life was all about soccer. In the final year of the engineering bachelor's degree, all students had to do a thesis. One could not graduate without completing a thesis. And I was certainly not in the mood for failing again. I was determined to succeed, but my professor and thesis advisor had other ideas.

First, I had to choose a project. Because other than soccer, the only other thing I was good at was math, I chose a project that involved a lot of mathematical analysis. My professor gave me a challenge to solve a special case of a differential equation in fluid dynamics. I was happy. It was mathematics and required neither an experimental setup nor building stuff. It was math and me, simple.

I tried several methods to solve the problem, without making any major inroads. After a couple of months, I saw my professor and showed him all I had done. He was quite pleased, even though I had not been "successful." He suggested I try a new approach, explaining the gist of it. I went back to the drawing table, working on it for a month, and still was not able to solve the problem. I could show that the professor's approach could not lead to a solution, but I wasn't able to actually improve upon it to solve the problem. When I saw the professor again, he was once again quite pleased, and gave me yet another approach. And again, the same result. This went on for three to four months; I'd follow through on all the approaches and suggestions, show that they couldn't solve the problem, but a full solution remained elusive.

By the end of the summer, I was panicking because I was nowhere near the end of this process, and I needed to graduate by the end of the year or else my scholarship would run out. I met the professor again at the start of the semester, sharing with him my predicament and concerns. He looked at me, and said, "Manu, all the strategies you have tried, including the ones I suggested, are known not to work." I was angry, but I tried not to show it. Why had he made me go through eight months of trying things that were known not to work? My professor explained, "Now that you have understood what does not work, you understand the problem way better than anyone else. Now I will tell you one last strategy. The problem cannot be solved mathematically. It has to be solved computationally."

It gives me goosebumps to this day when I think about that meeting. He was indeed right. Much as I hated admitting it at the time, I did understand the problem better and developing a computational solution and running simulations turned out to be straightforward. I did that quickly, and within a couple of months or so, I had completed the project and was even given the highest distinction for it. This was the second of several dots.

Looking back, both my soccer training and final year engineering thesis were the first two dots. In both, I was intentionally and repeatedly taken to the other side of failure. If I was paying attention, I would have connected the dots. I did not, or perhaps could not. Far from connecting the dots, I was merely happy just being able to graduate on time. I barely made it through with second-class honors, knowing very well that engineering was not what I wanted to do. My heart was simply not in it. I suppose, against the backdrop of a soccer career, it was hard for anything to come close.

It was time for trying out some other options. With some luck, I joined a management consulting firm, but within a few months even that did not work out, and I quit. Dot number three. Then, I ventured into the start-up world of the late Nineties during the Dot Com boom. This dream too fizzled out when the Dot Com boom turned into Dot Com doom. Dot number four.

By now, all my friends were already well into highly successful careers as doctors, lawyers, management consultants, and bankers; all I had to show was a string of failures - dots in a pattern that I had yet to realize - that I had in just about everything I had tried. I was running out of options.

And then, an opportunity to teach came along. Left with no other options at the time, and with bills to pay and make ends meet, I was forced into my fourth option: teaching.

Into the Classroom

Because I was reasonably good at math, I decided to become a math teacher, teaching the subject to high school kids for five years. It started as a one-year contract position, and I took it to give me some breathing space to figure out what I wanted to do with my life. As it turned out, this singular decision led directly to the discoveries that made my academic career a success and prompted me to write this book. However, these discoveries did not come from my prowess and skill as a teacher; instead, and as you will see, they are the direct result of my failure to teach mathematics to my...