Preface

Python is the most widely used language for natural language processing (NLP) thanks to its extensive tools and libraries for analyzing text and extracting computer-usable data. This book will take you through a range of techniques for text processing, from basics such as parsing parts of speech to complex topics such as topic modeling, text classification, and visualization.

Starting with an overview of NLP, the book presents recipes for dividing text into sentences, stemming and lemmatization, removing stopwords, and parts-of-speech tagging to help you to prepare your data. You will then learn about ways of extracting and representing grammatical information, such as dependency parsing and anaphora resolution; discover different ways of representing the semantics using bag of words, TF-IDF, word embeddings, and BERT; and develop skills for text classification using keywords, SVMs, LSTMs, and other techniques.

As you advance, you will also see how to extract information from text, implement unsupervised and supervised techniques for topic modeling, and perform topic modeling of short texts, such as tweets. Additionally, the book covers visualizations of text data.

Finally, this book introduces Transformer-based models and how to utilize them to perform another set of novel NLP tasks. These encoder-decoder-based models are deep neural-network-based models and have been trained on large text corpora. These models have performed or exceeded the state of the art on various NLP tasks. Especially novel are the decoder-based generative models, which have the capability to generate text based on the context provided to them. Some of these models have reasoning capabilities built into them. These models will take NLP into the next era and make it a part of mainstream technology applications.

By the end of this NLP book, you will have developed the skills to use a powerful set of tools for text processing.

Who this book is for

Data scientists, machine learning engineers, and developers familiar with basic programming and data science concepts can gain practical insights from this book. It serves as a primer to introduce the concepts of NLP and their practical applications.

The roles that are the target of this book are the following:

Data scientists: As a data scientist, you will gain an understanding of how to work with text. Intermediate knowledge of Python will help you to get the most out of this book. If you are already an NLP practitioner, this book will serve as a code reference when working on your projects.

Software engineers and architects: Developers who want to build capability in the domain of NLP will be introduced to all the fundamental and advanced uses of NLP in text processing. This will help you level up on your knowledge and build yourself up to develop solutions using NLP when required.

Product managers: Though this book contains code examples for the recipes, each of these recipes is accompanied by explanations of why certain steps are being performed and what the end output for those steps is. This makes it a useful resource for product managers who want to understand what is possible with a certain NLP recipe, which would enable them to envision novel solutions using it.

What this book covers

Chapter 1, Learning NLP Basics, introduces the very basics of NLP. The recipes in this chapter show the basic preprocessing steps that are required for further NLP work. We show how to tokenize text, or divide it into sentences and words; assign parts of speech to individual words; lemmatize them, or get their canonical forms; and remove stopwords.

Chapter 2, Playing with Grammar, shows how to get grammatical information from text. This information could be useful in determining relationships between different entities mentioned in the text. We start by showing how to determine whether a noun is singular or plural. We then show how to get a dependency parse that shows relationships between words in a sentence. Then, we demonstrate how to get noun chunks, or nouns with their dependent words, such as adjectives. After that, we look at parsing out the subjects and objects of a sentence. Finally, we show how to use a regular-expression-style matcher to extract grammatical phrases in a sentence.

Chapter 3, Representing Text - Capturing Semantics, looks at different ways of representing text for further processing in NLP models. Since computers cannot deal with words directly, we need to encode them in vector form. In order to demonstrate the effectiveness of different methods of encoding, we first create a simple classifier and then use it with different encoding methods. We look at the following encoding methods: bag-of-words, N-gram model, TF-IDF, word embeddings, BERT, and OpenAI embeddings. We also show how to train your own bag-of-words model and demonstrate how to create a simple retrieval-augmented generation (RAG) solution.

Chapter 4, Classifying Texts, shows various ways of carrying out text classification, one of the most common NLP tasks. First, we show how to preprocess the dataset in order to prepare it for classification. Then, we demonstrate different classifiers, including a rule-based classifier, an unsupervised classifier via K-means, training an SVM for classification, training a spaCy model for text classification, and, finally, using OpenAI GPT models to classify texts.

Chapter 5, Getting Started with Information Extraction, shows how to extract information from text, another very important NLP task. We start off with using regular expressions for simple information extraction. We then look at how to use the Levenshtein distance to handle misspellings. Then, we show how to extract characteristic keywords from different texts. We look at how to extract named entities using spaCy, and how to train your own custom spaCy NER model. Finally, we show how to fine-tune a BERT NER model.

Chapter 6, Topic Modeling, shows how to determine topics of text using various unsupervised methods, including LDA, community detection with BERT embeddings, K-means clustering, and BERTopic. Finally, we use contextualized topic models that work with multilingual models and inputs.

Chapter 7, Visualizing Text Data, focuses on using various tools to create informative visualizations of text data and processing. We create graphic representations of the dependency parse, parts of speech, and named entities. We also create a confusion matrix plot and word clouds. Finally, we use pyLDAvis and BERTopic to visualize topics in a text.

Chapter 8, Transformers and Their Applications, provides an introduction to Transformers. This chapter begins by demonstrating how to transform text into a format suitable for internal processing by a Transformer model. It then explores techniques for text classification using pre-trained Transformer models. Additionally, the chapter delves into text generation with Transformers, explaining how to tweak the generation parameters to produce coherent and natural-sounding text. Finally, it covers the application of Transformers in language translation.

Chapter 9, Natural Language Understanding, covers NLP techniques that help infer the information contained in a piece of text. This chapter begins with a discussion on question-answering in both open and closed domains, followed by methods for answering questions from document sources using extractive and abstractive approaches. Subsequent sections cover text summarization and sentence entailment. The chapter concludes with explainability techniques, which demonstrate how models make classification decisions and how different parts of the text contribute to the assigned class labels.

Chapter 10, Generative AI and Large Language Models, introduces open source Large Language Models (LLMs) such as Mistral and Llama, demonstrating how to use prompts to generate text based on simple human-defined requirements. It further explores techniques for generating Python code and SQL statements from natural language instructions. Finally, it presents methods for utilizing a sophisticated closed source LLM from OpenAI to orchestrate custom task agents. These agents collaborate to answer complex questions requiring web searches and basic arithmetic to arrive at an end solution.

To get the most out of this book

You will need an understanding of the Python programming language and how to manage and install packages for it. Knowledge of Jupyter Notebook would be useful, though it is not required. For package management, the knowledge of poetry package management is recommended, though you can make the examples work via pip too. For recipes to be able to use GPUs (if present) in the system, ensure that the latest GPU device drivers are installed along with the CUDA/cuDNN...