Quick Start Guide to Large Language Models
Strategies and Best Practices for Using ChatGPT and Other LLMs
Published on 20. October 2023
288 pages
978-0-13-819933-3 (ISBN)
Description
The Practical, Step-by-Step Guide to Using LLMs at Scale in Projects and Products
Large Language Models (LLMs) like ChatGPT are demonstrating breathtaking capabilities, but their size and complexity have deterred many practitioners from applying them. In Quick Start Guide to Large Language Models, pioneering data scientist and AI entrepreneur Sinan Ozdemir clears away those obstacles and provides a guide to working with, integrating, and deploying LLMs to solve practical problems.
Ozdemir brings together all you need to get started, even if you have no direct experience with LLMs: step-by-step instructions, best practices, real-world case studies, hands-on exercises, and more. Along the way, he shares insights into LLMs' inner workings to help you optimize model choice, data formats, parameters, and performance. You'll find even more resources on the companion website, including sample datasets and code for working with open- and closed-source LLMs such as those from OpenAI (GPT-4 and ChatGPT), Google (BERT, T5, and Bard), EleutherAI (GPT-J and GPT-Neo), Cohere (the Command family), and Meta (BART and the LLaMA family).
Learn key concepts: pre-training, transfer learning, fine-tuning, attention, embeddings, tokenization, and more
Use APIs and Python to fine-tune and customize LLMs for your requirements
Build a complete neural/semantic information retrieval system and attach to conversational LLMs for retrieval-augmented generation
Master advanced prompt engineering techniques like output structuring, chain-ofthought, and semantic few-shot prompting
Customize LLM embeddings to build a complete recommendation engine from scratch with user data
Construct and fine-tune multimodal Transformer architectures using opensource LLMs
Align LLMs using Reinforcement Learning from Human and AI Feedback (RLHF/RLAIF)
Deploy prompts and custom fine-tuned LLMs to the cloud with scalability and evaluation pipelines in mind
"By balancing the potential of both open- and closed-source models, Quick Start Guide to Large Language Models stands as a comprehensive guide to understanding and using LLMs, bridging the gap between theoretical concepts and practical application."
--Giada Pistilli, Principal Ethicist at HuggingFace
"A refreshing and inspiring resource. Jam-packed with practical guidance and clear explanations that leave you smarter about this incredible new field."
--Pete Huang, author of The Neuron
Register your book for convenient access to downloads, updates, and/or corrections as they become available. See inside book for details.
Large Language Models (LLMs) like ChatGPT are demonstrating breathtaking capabilities, but their size and complexity have deterred many practitioners from applying them. In Quick Start Guide to Large Language Models, pioneering data scientist and AI entrepreneur Sinan Ozdemir clears away those obstacles and provides a guide to working with, integrating, and deploying LLMs to solve practical problems.
Ozdemir brings together all you need to get started, even if you have no direct experience with LLMs: step-by-step instructions, best practices, real-world case studies, hands-on exercises, and more. Along the way, he shares insights into LLMs' inner workings to help you optimize model choice, data formats, parameters, and performance. You'll find even more resources on the companion website, including sample datasets and code for working with open- and closed-source LLMs such as those from OpenAI (GPT-4 and ChatGPT), Google (BERT, T5, and Bard), EleutherAI (GPT-J and GPT-Neo), Cohere (the Command family), and Meta (BART and the LLaMA family).
Learn key concepts: pre-training, transfer learning, fine-tuning, attention, embeddings, tokenization, and more
Use APIs and Python to fine-tune and customize LLMs for your requirements
Build a complete neural/semantic information retrieval system and attach to conversational LLMs for retrieval-augmented generation
Master advanced prompt engineering techniques like output structuring, chain-ofthought, and semantic few-shot prompting
Customize LLM embeddings to build a complete recommendation engine from scratch with user data
Construct and fine-tune multimodal Transformer architectures using opensource LLMs
Align LLMs using Reinforcement Learning from Human and AI Feedback (RLHF/RLAIF)
Deploy prompts and custom fine-tuned LLMs to the cloud with scalability and evaluation pipelines in mind
"By balancing the potential of both open- and closed-source models, Quick Start Guide to Large Language Models stands as a comprehensive guide to understanding and using LLMs, bridging the gap between theoretical concepts and practical application."
--Giada Pistilli, Principal Ethicist at HuggingFace
"A refreshing and inspiring resource. Jam-packed with practical guidance and clear explanations that leave you smarter about this incredible new field."
--Pete Huang, author of The Neuron
Register your book for convenient access to downloads, updates, and/or corrections as they become available. See inside book for details.
Reviews / Votes
"Ozdemir's book cuts through the noise to help readers understand where the LLM revolution has come from--and where it is going. Ozdemir breaks down complex topics into practical explanations and easy to follow code examples."--Shelia Gulati, former GM at Microsoft and current Managing Director of Tola Capital
"When it comes to building Large Language Models (LLMs), it can be a daunting task to find comprehensive resources that cover all the essential aspects. However, my search for such a resource recently came to an end when I discovered this book.
"One of the stand-out features of Sinan is his ability to present complex concepts in a straightforward manner. The author has done an outstanding job of breaking down intricate ideas and algorithms, ensuring that readers can grasp them without feeling overwhelmed. Each topic is carefully explained, building upon examples that serve as steppingstones for better understanding. This approach greatly enhances the learning experience, making even the most intricate aspects of LLM development accessible to readers of varying skill levels.
"Another strength of this book is the abundance of code resources. The inclusion of practical examples and code snippets is a game-changer for anyone who wants to experiment and apply the concepts they learn. These code resources provide readers with hands-on experience, allowing them to test and refine their understanding. This is an invaluable asset, as it fosters a deeper comprehension of the material and enables readers to truly engage with the content.
"In conclusion, this book is a rare find for anyone interested in building LLMs. Its exceptional quality of explanation, clear and concise writing style, abundant code resources, and comprehensive coverage of all essential aspects make it an indispensable resource. Whether you are a beginner or an experienced practitioner, this book will undoubtedly elevate your understanding and practical skills in LLM development. I highly recommend Quick Start Guide to Large Language Models to anyone looking to embark on the exciting journey of building LLM applications."
--Pedro Marcelino, Machine Learning Engineer, Co-Founder and CEO @overfit.study
More details
Language
English
Place of publication
Boston
United States
Publishing group
Pearson Education (US)
Target group
Professional and scholarly
File size
28,13 MB
ISBN-13
978-0-13-819933-3 (9780138199333)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions
Sinan Ozdemir
Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs
Strategies and Best Practices for Using ChatGPT and Other LLMs
Book
10/2023
1st Edition
Addison Wesley
€46.99
Available immediately
Person
Sinan Ozdemir is currently the founder and CTO of Shiba Technologies. Sinan is a former lecturer of Data Science at Johns Hopkins University and the author of multiple textbooks on data science and machine learning. Additionally, he is the founder of the recently acquired Kylie.ai, an enterprise-grade conversational AI platform with RPA capabilities. He holds a master's degree in Pure Mathematics from Johns Hopkins University and is based in San Francisco, CA.
Content
Foreword xv
Preface xvii
Acknowledgments xxi
About the Author xxiii
Part I: Introduction to Large Language Models 1
Chapter 1: Overview of Large Language Models 3
What Are Large Language Models? 4
Popular Modern LLMs 20
Domain-Specific LLMs 22
Applications of LLMs 23
Summary 29
Chapter 2: Semantic Search with LLMs 31
Introduction 31
The Task 32
Solution Overview 34
The Components 35
Putting It All Together 51
The Cost of Closed-Source Components 54
Summary 55
Chapter 3: First Steps with Prompt Engineering 57
Introduction 57
Prompt Engineering 57
Working with Prompts Across Models 65
Building a Q/A Bot with ChatGPT 69
Summary 74
Part II: Getting the Most Out of LLMs 75
Chapter 4: Optimizing LLMs with Customized Fine-Tuning 77
Introduction 77
Transfer Learning and Fine-Tuning: A Primer 78
A Look at the OpenAI Fine-Tuning API 82
Preparing Custom Examples with the OpenAI CLI 84
Setting Up the OpenAI CLI 87
Our First Fine-Tuned LLM 88
Case Study: Amazon Review Category Classification 93
Summary 95
Chapter 5: Advanced Prompt Engineering 97
Introduction 97
Prompt Injection Attacks 97
Input/Output Validation 99
Batch Prompting 103
Prompt Chaining 104
Chain-of-Thought Prompting 111
Revisiting Few-Shot Learning 113
Testing and Iterative Prompt Development 123
Summary 124
Chapter 6: Customizing Embeddings and Model Architectures 125
Introduction 125
Case Study: Building a Recommendation System 126
Summary 144
Part III: Advanced LLM Usage 145
Chapter 7: Moving Beyond Foundation Models 147
Introduction 147
Case Study: Visual Q/A 147
Case Study: Reinforcement Learning from Feedback 163
Summary 173
Chapter 8: Advanced Open-Source LLM Fine-Tuning 175
Introduction 175
Example: Anime Genre Multilabel Classification with BERT 176
Example: LaTeX Generation with GPT2 189
Sinan's Attempt at Wise Yet Engaging Responses: SAWYER 193
The Ever-Changing World of Fine-Tuning 206
Summary 207
Chapter 9: Moving LLMs into Production 209
Introduction 209
Deploying Closed-Source LLMs to Production 209
Deploying Open-Source LLMs to Production 210
Summary 225
Part IV: Appendices 227
Appendix A: LLM FAQs 229
Appendix B: LLM Glossary 233
Appendix C: LLM Application Archetypes 239
Index 243
Preface xvii
Acknowledgments xxi
About the Author xxiii
Part I: Introduction to Large Language Models 1
Chapter 1: Overview of Large Language Models 3
What Are Large Language Models? 4
Popular Modern LLMs 20
Domain-Specific LLMs 22
Applications of LLMs 23
Summary 29
Chapter 2: Semantic Search with LLMs 31
Introduction 31
The Task 32
Solution Overview 34
The Components 35
Putting It All Together 51
The Cost of Closed-Source Components 54
Summary 55
Chapter 3: First Steps with Prompt Engineering 57
Introduction 57
Prompt Engineering 57
Working with Prompts Across Models 65
Building a Q/A Bot with ChatGPT 69
Summary 74
Part II: Getting the Most Out of LLMs 75
Chapter 4: Optimizing LLMs with Customized Fine-Tuning 77
Introduction 77
Transfer Learning and Fine-Tuning: A Primer 78
A Look at the OpenAI Fine-Tuning API 82
Preparing Custom Examples with the OpenAI CLI 84
Setting Up the OpenAI CLI 87
Our First Fine-Tuned LLM 88
Case Study: Amazon Review Category Classification 93
Summary 95
Chapter 5: Advanced Prompt Engineering 97
Introduction 97
Prompt Injection Attacks 97
Input/Output Validation 99
Batch Prompting 103
Prompt Chaining 104
Chain-of-Thought Prompting 111
Revisiting Few-Shot Learning 113
Testing and Iterative Prompt Development 123
Summary 124
Chapter 6: Customizing Embeddings and Model Architectures 125
Introduction 125
Case Study: Building a Recommendation System 126
Summary 144
Part III: Advanced LLM Usage 145
Chapter 7: Moving Beyond Foundation Models 147
Introduction 147
Case Study: Visual Q/A 147
Case Study: Reinforcement Learning from Feedback 163
Summary 173
Chapter 8: Advanced Open-Source LLM Fine-Tuning 175
Introduction 175
Example: Anime Genre Multilabel Classification with BERT 176
Example: LaTeX Generation with GPT2 189
Sinan's Attempt at Wise Yet Engaging Responses: SAWYER 193
The Ever-Changing World of Fine-Tuning 206
Summary 207
Chapter 9: Moving LLMs into Production 209
Introduction 209
Deploying Closed-Source LLMs to Production 209
Deploying Open-Source LLMs to Production 210
Summary 225
Part IV: Appendices 227
Appendix A: LLM FAQs 229
Appendix B: LLM Glossary 233
Appendix C: LLM Application Archetypes 239
Index 243
