Introduction

With huge advancements in technology in the last 30 years, the ability to gain insights and actions from data hasn't changed much. In general, applications are still designed to perform predetermined functions or automate business processes, so their designers must plan for every usage scenario and code the logic accordingly. They don't adapt to changes in the data or learn from their experiences. Computers are faster and cheaper, but not much smarter. Of course, people are not much smarter than they were 30 years ago either. That is about to change, for humans and machines. A new generation of an information system is emerging that departs from the old model of computing as process automation to provide a collaborative platform for discovery. The first wave of these systems is already augmenting human cognition in a variety of fields. Acting as partners or collaborators for their human users, these systems may derive meaning from volumes of natural language text and generate and evaluate hypotheses in seconds based on analysis of more data than a person could absorb in a lifetime. That is the promise of cognitive computing.

Human Intelligence + Machine Intelligence

Traditional applications are good at automating well-defined processes. From inventory management to weather forecasting, when speed is the critical factor in success and the processes are known in advance, the traditional approach of defining requirements, coding the logic, and running an application is adequate. That approach fails, however, when we need to dynamically find and leverage obscure relationships between data elements, especially in areas in which the volume or complexity of the data increases rapidly. Change, uncertainty, and complexity are the enemies of traditional systems.

Cognitive computing-based on software and hardware that learns without reprogramming and automates cognitive tasks-presents an appealing new model or paradigm for application development. Instead of automating the way we already conduct business, we begin by thinking about how to augment the best of what the human brain can do with new application capabilities. We start with processes for ingesting data from inside and outside the enterprise, and add functions to identify and evaluate patterns and complex relationships in large and sometimes unstructured data sets, such as natural language text in journals, books, and social media, or images and sounds. The result is a system that can support human reasoning by evaluating data in context and presenting relevant findings along with the evidence that justifies the answers. This approach makes users more efficient-like a traditional application-but it also makes them more effective because parts of the reasoning and learning processes have been automated and assigned to a tireless, fast collaborator.

Like the fundamentals of traditional computing, the concepts behind smart machines are not new. Even before the emergence of digital computers, engineers and scientists speculated about the development of learning machines that could mimic human problem solving and communications skills. Although some of the concepts underlying the foundation technologies-including machine intelligence, computational linguistics, artificial intelligence, neural networks, and expert systems-have been used in conventional solutions for a decade or more, we have seen only the beginning. The new era of intelligent computing is driven by the confluence of a number of factors:

• The growth in the amount of data created by systems, intelligent devices, sensors, videos, and such
• The decrease in the price of computer storage and computing capabilities
• The increasing sophistication of technology that can analyze complex data as fast as it is produced
• The in-depth research from emerging companies across the globe that are investigating and challenging long-held beliefs about what the collaboration of humans and machines can achieve

Putting the Pieces Together

When you combine Big Data technology and the changing economics of computing with the need for business and industry to be smarter, you have the beginning of fundamental change. There are many names for this paradigm shift: machine learning, cognitive computing, artificial intelligence, knowledge management, and learning machines. But whatever you call it, this change is actually the integration of the best of human knowledge about the world with the awesome power of emerging computational systems to interpret massive amounts of a variety of types of data at an unprecedented rate of speed. But it is not enough to interpret or analyze data. Emerging solutions for cognitive computing must gather huge amounts of data about a specific topic, interact with subject matter experts, and learn the context and language of that subject. This new cognitive era is in its infancy, but we are writing this book because of the significant and immediate market potential for these systems. Cognitive computing is not magic. It is a practical approach to supporting human problem solving with learning machines that will change markets and industries.

The Book's Focus

This book takes a deep look at the elements of cognitive computing and how it is used to solve problems. It also looks at the human efforts involved in evolving a system that has enough context to interpret complex data and processes in areas such as healthcare, manufacturing, transportation, retail, and financial services. These systems are designed as collaboration between machines and humans. The book examines various projects designed to help make decision making more systematic. How do expertly trained and highly experienced professionals leverage data, prior knowledge, and associations to make informed decisions? Sometimes, these decisions are the right ones because of the depth of knowledge. Other times, however, the decisions are incorrect because the knowledgeable individuals also bring their assumptions and biases into decision making. Many organizations that are implementing their first cognitive systems are looking for techniques that leverage deep experience combined with mechanization of complex Big Data analytics. Although this industry is young, there is much that can be learned from these pioneering cognitive computing engagements.

Overview of the Book and Technology

The authors of this book, Judith Hurwitz, Marcia Kaufman, and Adrian Bowles are veterans of the computer industry. All of us are opinionated and independent industry analysts and consultants who take an integrated perspective on the relationship between different technologies and how they can transform businesses and industries. We have approached the writing of this book as a true collaboration. Each of us brings different experience from developing software to evaluating emerging technologies, to conducting in-depth research on important technology innovations.

Like many emerging technologies, cognitive computing is not easy. First, cognitive computing represents a new way of creating applications to support business and research goals. Second, it is a combination of many different technologies that have matured enough to become commercially viable. So, you may notice that most of the technologies detailed in the book have their roots in research and products that have been around for years or even decades. Some technologies or methods such as machine learning algorithms and natural language processing (NLP) have been seen in artificial intelligence applications for many decades. Other technologies such as advanced analytics have evolved and grown more sophisticated over time. Dramatic changes in deployment models such as cloud computing and distributed computing technology have provided the power and economies of scale to bring computing power to levels that were impossible only a decade ago.

This book doesn't attempt to replace the many excellent technical books on individual topics such as machine learning, NLP, advanced analytics, neural networks, Internet of Things, distributed computing and cloud computing. Actually, we think it is wise to use this book to give you an understanding of how the pieces fit together to then gain more depth by exploring each topic in detail.

How This Book Is Organized

This book covers the fundamentals and underlying technologies that are important to creating cognitive system. It also covers the business drivers for cognitive computing and some of the industries that are early adopters of cognitive computing. The final chapter in the book provides a look into the future.

• Chapter 1: "The Foundation of Cognitive Computing." This chapter provides perspective on the evolution to cognitive computing from artificial intelligence to machine learning.
• Chapter 2: "Design Principles for Cognitive Systems." This chapter provides you with an understanding of what the architecture of cognitive computing is and how the pieces fit together.
• Chapter 3: "Natural Language Processing in Support of a Cognitive System." This chapter explains how a cognitive system uses natural language processing techniques and how these techniques create understanding.
• Chapter 4: "The Relationship Between Big Data and Cognitive Computing." Big data is one of the pillars of a cognitive system. This chapter demonstrates the Big Data technologies and approaches that are fundamental to a cognitive system.
• Chapter 5: "Representing Knowledge in Taxonomies and...