CHAPTER 1
Navigating the Copilot Ecosystem

Information technology (IT), as an organizational function, has the primary purpose of enabling business counterparts to implement solutions that improve productivity at the task and business process levels. For the past decade, IT leaders have been under incredible pressure to deliver business value while also being charged with continuing to drive down costs. For every new wave of technological innovation, IT leaders must navigate the fine line between embracing the hype and delivering tangible ROI.

Generative AI (GenAI) has forced an almost "gold rush" mentality within the IT industry, with consultants and independent software developers (ISVs) alike working hard to bring forward the next wave of innovation. Microsoft has made significant investments in GenAI through its Azure AI Studio service, which enables organizations to safely and securely develop their own GenAI applications, as well as its Copilot brand of GenAI-as-a-Service offerings, which are being built-in to its entire portfolio of applications. Satya Nadella, Chairman and Chief Executive Officer (CEO) of Microsoft, has been quoted as saying:

We are the Copilot company. We believe in a future where there will be a Copilot for everyone and everything you do. Microsoft Copilot is that one experience that runs across all our surfaces, understanding your context on the Web, on your device. And when you're at work, bringing the right skills to you when you need them. Just like, say today you boot up an operating system to access applications or a browser to navigate to a Web site, you can invoke a Copilot to do all these activities, and more-to shop, to call, to analyze, to learn, to create. We want the compiler to be everywhere you are.

redmondmag.com/articles/2023/11/16/nadella-ignite-2023-keynote.aspx

This chapter will begin with a quick primer on GenAI and Microsoft's role in maturing this technology over the past few years. We will then step through Microsoft's Copilot brand of products to better understand its strategy of transforming how people work by introducing artificial intelligence (AI) into their workflow. Finally, we will end with a high-level overview of Copilot Studio and how it will enable citizen developers and IT professionals to drive even greater business value by combining a low-code application development platform with GenAI.

What Is GenAI?

If you are unfamiliar with the term, GenAI is a type of AI that focuses on creating content by analyzing and learning patterns from large datasets. Examples of content GenAI can create include text, images, code, and audio. Additionally, GenAI can analyze existing content and provide you with feedback based on questions you ask it. For example, you could write an email and then ask it for suggestions on how to rewrite it for tone, clarity, or brevity. It will then analyze both the content you provided and its dataset to provide you with recommendations in natural language, meaning text.

GenAI is on the same trajectory as other large disruptive technologies, such as the graphical user interface (GUI), the Internet, and the iPhone. Unlike other technologies, such as robotics process automation (RPA), monolithic enterprise resource planning platforms, or event-driven architectures, what makes GenAI appealing is how it blends creativity with computation using the most powerful interface that exists: language.

GenAI blends creativity with computational power, enabling people to draft compelling narratives, design complex visuals, write code, and even create business strategies in a fraction of the time it once took. The technology is intuitive and adaptable, making it accessible to a wide range of users-from seasoned professionals to those without technical expertise. Its ability to personalize interactions, learn from context, and continuously improve makes GenAI a powerful tool for enhancing productivity, boosting innovation, and driving meaningful engagement across industries.

How Does GenAI Work?

At a high level, GenAI works through a combination of machine learning and then through the development of neural networks, which are a type of AI modeled after the human brain. Machine learning is when you feed in large amounts of data into a computer application, and it begins to create patterns to organize the data. Neural networks are an architecture within AI that include a series of interconnected nodes organized into various layers. These nodes, often referred to as neurons, are computational units that process information by performing mathematical operations on inputs, applying a weight (to emphasize importance), and passing the result through an activation function to determine the output. Like humans, GenAI models are trained with enormous amounts of data. While humans are trained over decades, GenAI models are trained over months with large datasets and computing infrastructure.

Like the human brain, as information flows through these interconnected nodes within the neural networks, it is transformed-but by mathematical functions. What is often viewed as being "GenAI magic" is a combination of being trained on a very large amount of data and the organization of this data in a way that can identify patterns and structures. At the end of the day, GenAI does an amazing job of predicting your desired output to the question you have asked it because it has been trained on a tremendous amount of data.

GenAI Key Terms and Definitions

While this book isn't meant to be a deep dive into GenAI, there are some key terms that are helpful to understand:

• GenAI models: Models are designed to create new data from machine learning patterns. The output could be text, images, music, etc.
• Large language models (LLMs): An LLM is a type of GenAI model that has been specifically trained on human language and can be used to create new content or summarize existing content.
• Training data: Data is the key to GenAI, as it is what feeds the GenAI models. Training data typically consists of large datasets, which include text, images, audio, etc. For LLMs specifically, the training data is human language in the form of text.
• Pattern recognition: This is where the model analyzes all the data you have trained it on and begins to identify patterns and structures that will be leveraged for content creation or content summarization.
• Content creation: This is where the model applies what it has learned and attempts to generate new content that mimics the style and structure of the training data it was provided when it responds to a user prompt.

The Risk of Bias in GenAI

When people talk about being careful about bias with GenAI, it is because the models are only as good as the data they are trained on. To provide a practical example, if you were building a model on baseball statistics and only provided statistics of the Boston Red Sox defeating the New York Yankees, the model would be biased toward the Boston Red Sox being the more dominant team. If you asked it who will win an upcoming game between the two teams, it would more than likely propose the Boston Red Sox based on its data. However, when you include all the matchups, the model might be biased toward the New York Yankees, since historically they have won more of the games between the two teams.

When you apply this same concept to people, the consequences can be even more drastic. For example, it has been proven that some ethnicities are statistically more prone to certain diseases. Therefore, it is important to have a wide set of training data for AI applications that are meant for health care use cases. If you train data only on a particular ethnicity, you may miss out on some of the potential nuances. Combined with a potential over-reliance on AI, this could lead to a situation where a clinician might miss a diagnosis even though they have access to this incredibly powerful AI application. The consequences of being over-reliant on technology that is wrong can have dire consequences, thus the need to plan for a comprehensive responsible AI strategy to help minimize these risks.

The technology industry and many data scientists are especially careful to point out the potential risks of bias that can happen with AI. Therefore, the need to obtain or create diverse datasets reflective of the various types of people, ethnicities, cultures, etc. is important to ensure that AI is for everyone, not just a subset of people. This risk is even more problematic given that the user experience of interacting with GenAI is very conversational in nature. In addition, the confidence with which LLMs provide responses is rather convincing, even when those answers are incorrect answer. The popularization of engaging with LLMs through conversation was catapulted into the spotlight through OpenAI's ChatGPT.

OpenAI Brings GenAI to the World

While OpenAI did not invent GenAI, it was able to bring it to the mainstream with its launch of ChatGPT in November 2022. ChatGPT was able to amass a user base of over 100 million users in less than a year, far quicker than any other technology to date. ChatGPT rapidly surpassed the adoption of commercial AI assistants such as Amazon's Alexa, Google's Echo, and Microsoft's Cortana by providing what seems like infinite expertise. ChatGPT's user experience, both then and now, is simple, with a basic chat-based user interface that is forgiving...