1

Defining a Business-Ready Generative AI System

Implementing a generative AI system (GenAISys) in an organization doesn't stop at simply integrating a standalone model such as GPT, Grok, Llama, or Gemini via an API. While this is often a starting point, we often mistake it as the finish line. The rising demand for AI, as it expands across all domains, calls for the implementation of advanced AI systems that go beyond simply integrating a prebuilt model.

A business-ready GenAISys should provide ChatGPT-grade functionality in an organization, but also go well beyond it. Its capabilities and features must include natural language understanding (NLU), contextual awareness through memory retention across dialogues in a chat session, and agentic functions such as autonomous image, audio, and document analysis and generation. Think of a generative AI model as an entity with a wide range of functions, including AI agents as agentic co-workers.

We will begin the chapter by defining what a business-ready GenAISys is. From there, we'll focus on the central role of a generative AI model, such as GPT-4o, that can both orchestrate and execute tasks. Building on that, we will lay the groundwork for contextual awareness and memory retention, discussing four types of generative AI memory: memoryless, short-term, long-term, and multiple sessions. We will also define a new approach to retrieval-augmented generation (RAG) that introduces an additional dimension to data retrieval: instruction and agentic reasoning scenarios. Adding instructions stored in a vector store takes RAG to another level by retrieving instructions that we can add to a prompt. In parallel, we will examine a critical component of a GenAISys: human roles. We will see how, throughout its life cycle, an AI system requires human expertise. Additionally, we will define several levels of implementation to adapt the scope and scale of a GenAISys, not only to business requirements but also to available budgets and resources.

Finally, we'll illustrate how contextual awareness and memory retention can be implemented using OpenAI's LLM and multimodal API. A GenAISys cannot work without solid memory retention functionality-without memory, there's no context, and without context, there's no sustainable generation. Throughout this book, we will create modules for memoryless, short-term, long-term, and multisession types depending on the task at hand. By the end of this chapter, you will have acquired a clear conceptual framework for what makes an AI system business-ready and practical experience in building the first bricks of an AI controller.

In a nutshell, this chapter covers the following topics:

• Components of a business-ready GenAISys
• AI controllers and agentic functionality (model-agnostic)
• Hybrid human roles and collaboration with AI
• Business opportunities and scope
• Contextual awareness through memory retention

Let's begin by defining what a business-ready GenAISys is.

Components of a business-ready GenAISys

A business-ready GenAISys is a modular orchestrator that seamlessly integrates standard AI models with multifunctional frameworks to deliver hybrid intelligence. By combining generative AI with agentic functionality, RAG, machine learning (ML), web search, non-AI operations, and multiple-session memory systems, we are able to deliver scalable and adaptive solutions for diverse and complex tasks. Take ChatGPT, for example; people use the name "ChatGPT" interchangeably for the generative AI model as well as for the application itself. However, behind the chat interface, tools such as ChatGPT and Gemini are part of larger systems-online copilots-that are fully integrated and managed by intelligent AI controllers to provide a smooth user experience.

It was Tomczak (2024) who took us from thinking of generative AI models as a collective entity to considering complex GenAISys architectures. His paper uses the term "GenAISys" to describe these more complex platforms. Our approach in this book will be to expand the horizon of a GenAISys to include advanced AI controller functionality and human roles in a business-ready ecosystem. There is no single silver-bullet architecture for a GenAISys. However, in this section, we'll define the main components necessary to attain ChatGPT-level functionality. These include a generative AI model, memory retention functions, modular RAG, and multifunctional capabilities. How each component contributes to the GenAISys framework is illustrated in Figure 1.1:

Figure 1.1: GenAISys, the AI controller, and human roles

Let's now define the architecture of the AI controllers and human roles that make up a GenAISys.

AI controllers

At the heart of a business-ready GenAISys is an AI controller that activates custom ChatGPT-level features based on the context of the input. Unlike traditional pipelines with predetermined task sequences, the AI controller operates without a fixed order, dynamically adapting tasks-such as web search, image analysis, and text generation-based on the specific context of each input. This agentic context-driven approach enables the AI controller to orchestrate various components seamlessly, ensuring effective and coherent performance of the generative AI model.

A lot of work is required to achieve effective results with a custom ChatGPT-grade AI controller. However, the payoff is a new class of AI systems that can withstand real-world pressure and produce tangible business results. A solid AI controller ecosystem can support use cases across multiple domains: customer support automation, sales lead generation, production optimization (services and manufacturing), healthcare response support, supply chain optimization, and any other domain the market will take you! A GenAISys, thus, requires an AI controller to orchestrate multiple pipelines, such as contextual awareness to understand the intent of the prompt and memory retention to support continuity across sessions.

The GenAISys must also define human roles, which determine which functions and data can be accessed. Before we move on to human roles, however, let's first break down the key components that power the AI controller. As shown in Figure 1.1, the generative AI model, memory, modular RAG, and multifunctional capabilities each play vital roles in enabling flexible, context-driven orchestration. Let's explore how these elements work together to build a business-ready GenAISys. We will first define the role of the generative AI model.

Model-agnostic approach to generative AI

When we build a sustainable GenAISys, we need model interchangeability-the flexibility to swap out the underlying model as needed. A generative AI model should serve as a component within the system, not as the core that the system is built around. That way, if our model is deprecated or requires updating, or we simply find a better-performing one, we can simply replace it with another that better fits our project.

As such, the generative AI model can be OpenAI's GPT, Google's Gemini, Meta's Llama, xAI's Grok, or any Hugging Face model, as long as it supports the required tasks. Ideally, we should choose a multipurpose, multimodal model that encompasses text, vision, and reasoning abilities. Bommasani et al. (2021) provide a comprehensive analysis of such foundation models, whose scope reaches beyond LLMs.

A generative AI model has two main functions, as shown in Figure 1.2:

• Orchestrates by determining which tasks need to be triggered based on the input. This input can be a user prompt or a system request from another function in the pipeline. The orchestration function agent can trigger web search, document parsing, image generation, RAG, ML functions, non-AI functions, and any other function integrated into the GenAISys.
• Executes the tasks requested by the orchestration layer or executes a task directly based on the input. For example, a simple query such as requesting the capital of the US will not necessarily require complex functionality. However, a request for document analysis might require several functions (chunking, embedding, storing, and retrieving).

Figure 1.2: A generative AI model to orchestrate or execute tasks

Notice that Figure 1.2 has a unique feature. There are no arrows directing the input, orchestration, and execution components. Unlike traditional hardcoded linear pipelines, a flexible GenAISys has its components unordered. We build the components and then let automated scenarios selected by the orchestration function order the tasks dynamically.

This flexibility ensures the system's adaptability to a wide range of tasks. We will not be able to build a system that solves every task, but we can build one that satisfies a wide range of tasks within a company. Here are two example workflows that illustrate how a GenAISys can dynamically sequence tasks based on the roles involved:

• Human roles can be configured so that, in some cases, the user input executes a simple API call to provide a straightforward response, such as requesting the capital of a country. In this case, the...