Preface

OpenSearch is a "Swiss Army knife" that touches diverse use cases spanning application features, operations, and generative AI. If there's one unifying theme of the software, it is that it enables storing and retrieving data to support intelligent decision-making. It's a database, but it's a funny kind of database that emphasizes speed and volume processing over consistency. It's a logs store, but a funny kind of logs store that emphasizes aggregations and log-line search. It's a data source for generative AI, but it's a funny kind of data source that brings rich search to the retrieval of information for prompts. In all these cases, OpenSearch provides high-volume request processing and intelligent retrieval of data.

In this book, you'll learn in depth the capabilities of OpenSearch, how and when to apply them, and where you can get the most benefits. You'll also learn about Amazon OpenSearch Service, its managed clusters and serverless deployment options, and how to get the most out of your OpenSearch Service domain or OpenSearch Serverless collection.

We'll begin with introductory chapters that give you a history and overview of OpenSearch and show you how to deploy OpenSearch and how to use OpenSearch Service. We'll then dive deep into OpenSearch's core capabilities-indexing and querying data and building aggregations and visualizations. We'll cover OpenSearch's large collection of plugins that deliver additional features, such as Structured Query Language (SQL), alerting, and k-nearest neighbor search. We'll dive deep into application-building and delivering AI-powered applications with generative AI. We will then move on to operational topics, including migrations, security, monitoring, backups, and recovery. We will round out the book with a deep dive on scaling and performance optimization.

In writing this book, we wanted to distill our years of experience and thousands of hours of customer interaction for you. We wish you every success, and happy OpenSearching!

Who this book is for

This book is for developers, operators, and DevOps engineers who want to add or modernize search for their applications, and who want to monitor those applications for uptime and diagnose and remediate errors. Experience with Amazon Web Services, the Python programming language, Docker, and Kubernetes will be helpful but is not necessary.

What this book covers

Chapter 1, Overview of OpenSearch, covers OpenSearch's history, its core capabilities, and the main use cases for OpenSearch, with real-world examples. It also introduces the topic of operational efficiency.

Chapter 2, Installing and Configuring OpenSearch, gives an overview of OpenSearch distributed system basics. It guides you through deploying OpenSearch via tarball and Docker, and covers OpenSearch Dashboards and the basics of securing your cluster.

Chapter 3, Deployment Options: Amazon OpenSearch Service and Amazon OpenSearch Serverless, guides you through deploying and running OpenSearch in the Amazon Web Services cloud, using Amazon OpenSearch Service, and operational basics such as scaling, storage management, and security.

Chapter 4, Indexing Data, details how to create and maintain OpenSearch indexes, including creating indexes, index settings, setting a mapping, different mapping types, and mapping templates.

Chapter 5, Searching: Core APIs, explains query processing in OpenSearch, leaf queries, hit highlighting, search suggestions, and search templates.

Chapter 6, Advanced Searching, covers OpenSearch's query APIs in depth, as well as compound queries, geospatial queries, faceted search, query percolation, and query performance and profiling.

Chapter 7, Analyze and Visualize OpenSearch Data, dives into aggregations, OpenSearch Dashboards, dashboards and visualizations, working with time-series data such as logs, and the Observability plugin.

Chapter 8, Introduction to OpenSearch Plugins, covers the key OpenSearch plugins, including SQL, alerting, security analytics, k-nearest neighbor, and the Neural plugin. It then details how to install, manage, and build your own plugins for OpenSearch.

Chapter 9, OpenSearch in Action: Making Apps Awesome, moves from the theoretical to the abstract, integrating the topics covered to help you bring the power of OpenSearch to your application with faceted search, auto completions, and connecting to OpenSearch's APIs from your application. It brings everything together in a Streamlit application.

Chapter 10, OpenSearch Vectors and Generative AI, provides a theoretical foundation on dense vectors, sparse vectors, and the large language models that produce them. It goes into depth on exact and approximate k-nearest neighbor search, with the algorithms and engines OpenSearch provides, closing with a generative AI example.

Chapter 11, Migrate to OpenSearch, guides you through why, whether, and how to migrate from other search solutions, including planning for your migration, executing a proof of concept, deploying your target, and moving data and traffic with and without OpenSearch Migration Assistant. It closes with two examples of migrations.

Chapter 12, Security in OpenSearch, explains OpenSearch's security features and guides you in using them to best effect to secure your data and cluster.

Chapter 13, Monitoring, Backup, and Recovery, enters the world of operations to help you use Amazon OpenSearch Service managed clusters efficiently. It covers the metrics that the service generates, how to monitor them, and how best to respond to issues with troubleshooting and backups.

Chapter 14, Scaling and Performance Optimization, explains OpenSearch as a distributed system and walks through the core resources your cluster provides and how OpenSearch maps your workload onto those resources. It finishes with best practices to optimize your cluster infrastructure for maximum efficiency.

To get the most out of this book

Some of the code examples provided are in Python. A working knowledge of the language, and a working Python installation for your system, will allow you to you run those examples.

Some knowledge of distributed systems and other database systems will help you follow the discussion.

Knowledge of Amazon Web Services, Amazon Elastic Compute Cloud, and Docker will enable you to more easily deploy OpenSearch for the examples.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter/X handles. For example: "The _bulk API reduces overhead."

A block of code is set as follows:

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

Bold: Indicates a new term, an important word, or words that you see on the screen. For instance, words in menus or dialog boxes appear in the text like this. For example: "Select Dev Tools from the left navigation panel."

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book or have any general feedback, please email us at customercare@packt.com and mention the book's title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you reported this to us. Please visit http://www.packt.com/submit-errata, and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us...