Large Language Models
An Introduction
1st Edition
Published on 2. October 2024
480 pages
978-1-5015-2058-7 (ISBN)
Description
This book begins with an overview of the Generative AI landscape, distinguishing it from conversational AI and shedding light on the roles of key players like DeepMind and OpenAI. It then reviews the intricacies of ChatGPT, GPT-4, and Gemini, examining their capabilities, strengths, and competitors. Readers will also gain insights into the BERT family of LLMs, including ALBERT, DistilBERT, and XLNet, and how these models have revolutionized natural language processing. Further, the book covers prompt engineering techniques, essential for optimizing the outputs of AI models, and addresses the challenges of working with LLMs, including the phenomenon of hallucinations and the nuances of fine-tuning these advanced models. Designed for software developers, AI researchers, and technology enthusiasts with a foundational understanding of AI, this book offers both theoretical insights and practical code examples in Python. Companion files with code, figures, and datasets are available for downloading from the publisher.
More details
Series
Language
English
Place of publication
New York
United States
Target group
Professional and scholarly
US School Grade: College Graduate Student
File size
30,41 MB
ISBN-13
978-1-5015-2058-7 (9781501520587)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions
Book
10/2024
1st Edition
De Gruyter
€49.95
Article not available at the moment
Person
Oswald Campesato (San Francisco, CA) specializes in Deep Learning, Python, Data Science, and
generative AI. He is the author/co-author of over forty books including Google Gemini for Python,
LLMs for Developers, and GPT-4 for Developers (all Mercury Learning).
generative AI. He is the author/co-author of over forty books including Google Gemini for Python,
LLMs for Developers, and GPT-4 for Developers (all Mercury Learning).
Content
- Half Title
- Title
- Copyright
- Dedication
- Contents
- Preface
- Chapter 1: The Generative AI Landscape
- What Is Generative AI?
- Key Features of Generative AI
- Popular Techniques in Generative AI
- What Makes Generative AI Different
- The Successes of Generative AI
- Generative AI and Art and Copyrights
- Text-to-Image Generation
- Stability AI/Stable Diffusion
- Imagen (Google)
- Make-a-Scene (Meta)
- GauGAN2 (NVIDIA)
- Conversational AI Versus Generative AI
- Primary Objective
- Applications
- Technologies Used
- Training and Interaction
- Evaluation
- Data Requirements
- Is DALL-E Part of Generative AI?
- Are ChatGPT-3 and GPT-4 Part of Generative AI?
- Generative AI Versus ML, DL, NLP, and RL
- Which Fields Benefit the Most from Generative AI?
- How Will the Enterprise Space Benefit from Generative AI?
- The Impact of Generative AI on Jobs
- What is Artificial General Intelligence (AGI)?
- When Will AGI Arrive?
- How to Prepare for AGI
- Will AGI Control the World?
- Should Humans Fear AGI?
- Beyond AGI
- AGI Versus Generative AI
- DeepMind
- DeepMind and Games
- Player of Games (PoG)
- OpenAI
- Cohere
- Hugging Face
- Hugging Face Libraries
- Hugging Face Model Hub
- AI21
- Anthropic
- What are LLMs?
- A Brief History of Modern LLMs
- Aspects of LLM Development
- LLM Size Versus Performance
- Emergent Abilities of LLMs
- Success Stories in Generative AI
- Real-World Use Cases for Generative AI
- Generating Text from GPT-2
- SORA (OpenAI)
- OpenSORA
- Summary
- Chapter 2: ChatGPT and GPT-4
- What Is ChatGPT?
- ChatGPT: GPT-3 "On Steroids"?
- ChatGPT: Google "Code Red"
- ChatGPT Versus Google Search
- ChatGPT Custom Instructions
- ChatGPT on Mobile Devices and Browsers
- ChatGPT and Prompts
- GPTBot
- ChatGPT Playground
- Plugins, Advanced Data Analytics, and CodeWhisperer
- Plugins
- Advanced Data Analytics
- Advanced Data Analytics Versus Claude 3
- CodeWhisperer
- Detecting Generated Text
- Concerns About ChatGPT
- Code Generation and Dangerous Topics
- ChatGPT Strengths and Weaknesses
- Sample Queries and Responses from ChatGPT
- Alternatives to ChatGPT
- Google Gemini
- Gemini Ultra Versus GPT-4
- YouChat
- Pi from Inflection
- What Is InstructGPT?
- VizGPT and Data Visualization
- What Is GPT-4?
- GPT-4 and Test-Taking Scores
- GPT-4 Parameters
- GPT-4 Fine Tuning
- What Is GPT-4o?
- ChatGPT and GPT-4 Competitors
- Google Gemini (Formerly Bard)
- CoPilot (OpenAI/Microsoft)
- Codex (OpenAI)
- Apple GPT
- PaLM 2
- Claude 3
- LlaMa 3
- When Is GPT-5 Available?
- Summary
- Chapter 3: LLMs and the BERT Family
- What Is the Purpose of LLMs?
- Model Size Versus Training Set Size
- Do LLMs Understand Language?
- Caveats Regarding LLMs
- What Are Foundation Models?
- Pitfalls of Working with LLMs
- What Is BERT?
- The BERT Family
- ALBERT
- BART
- BioBERT
- ClinicalBERT
- deBERTa (Surpassing Human Accuracy)
- DistilBERT
- Google Smith
- TinyBERT
- VideoBERT
- VisualBERT
- XLNet
- Disadvantages of XLNet
- How to Select a BERT-Based Model
- Working with RoBERTa
- Italian and Japanese Language Translation
- Multilingual Language Models
- Training Multilingual Language Models
- BERT-Based Multilingual Language Models
- Translation for 1,000 Languages
- MBERT
- Comparing BERT-Based Models
- Web-Based Tools for BERT
- exBERT
- BertViz
- CNNViz
- Topic Modeling with BERT
- What Is T5?
- Working with PaLM
- What Is Pathways?
- Summary
- Chapter 4: Prompt Engineering
- LLMs and Context Length
- What Is Prompt Engineering?
- Overview of Prompt Engineering
- The Importance of Prompt Engineering
- Designing Prompts
- Prompt Categories
- Prompts and Completions
- Guidelines for Effective Prompts
- Examples of Effective Prompts for ChatGPT
- Concrete Versus Subjective Words in Prompts
- Common Types of Prompts
- "Shot" Prompts
- Instruction Prompts
- Reverse Prompts
- System Prompts Versus Agent Prompts
- Prompt Templates
- Prompts for Different LLMs
- Prompt Optimization
- Poorly Worded Prompts
- Prompt Injections
- Chain of Thought (CoT) Prompts
- Self-Consistency and CoT
- Self-Consistency, CoT, and Unsupervised Datasets (Language Model Self-Improved)
- Tree of Thought (ToT) Prompts
- Ranking Prompt Techniques
- Recommended Prompt Techniques
- Advanced Prompt Techniques
- GPT-4 and Prompt Samples
- SVG (Scalable Vector Graphics)
- GPT-4 and Arithmetic Operations
- Algebra and Number Theory
- The Power of Prompts
- Language Translation with GPT-4
- Can GPT-4 Write Poetry?
- GPT-4 and Humor
- Question Answering with GPT-4
- Stock-Related Prompts for GPT-4
- Philosophical Prompts for GPT-4
- Mathematical Prompts for GPT-4
- Inference Parameters
- Temperature Inference Parameter
- Temperature and the softmax() Function
- GPT-4o and Inference Parameters
- GPT-4o and the Temperature Inference Parameter
- Repeated Text from GPT-2
- Summary
- Chapter 5: Working with LLMs
- Kaplan and Undertrained Models
- Mixture of Experts (MOE)
- Aspects of LLM Evaluation
- LLMs and Hallucinations
- ChatGPT
- Meta AI
- Claude 3
- Grok
- Perplexity
- Gemini
- Reducing LLM Hallucinations
- ChatGPT
- Cohere
- Claude 3
- Meta AI
- Limitations of LLMs
- Open-Source Versus Closed-Source LLMs
- Well-Known LLMs
- Recently Created LLMs
- The LLMs in This Chapter
- Claude 3 (Anthropic)
- What Is Cohere?
- The Cohere Playground
- What Is Command R+?
- What Are the Main Features of Command R+?
- Command R+ Versus the Cohere Playground
- Google Gemini
- Gemini Ultra Versus GPT-4
- What Is Grok?
- Llama 3
- What Is Meta AI?
- What Are SLMs?
- Recent SLMs
- What Is Phi-3?
- Install and Run Phi-3 on a MacBook
- Interact with Phi-3 from the Command Line
- What Is OpenELM?
- Python Code with OpenELM
- What Is Gemma?
- Downloading Gemma-2b from Kaggle
- Mixtral (Mistral)
- Introduction to AI Agents
- What Can AI Agents Do?
- LLMs Versus AI Agents
- AI Agents That are Not LLMs
- Are LLMs a Subset of AI Agents?
- GPT-4 Versus AI Agents
- Summary
- Chapter 6: LLMs and Fine-Tuning
- What Is Fine-Tuning?
- Python Code Sample for Fine-Tuning GPT-2
- Well-Known Fine-Tuning Techniques
- When Should Fine-Tuning Be Used?
- Fine-Tuning BERT for Sentiment Analysis
- Generating Fine-Tuning Datasets
- SFT, RLHF, and PEFT
- Quantized LLMs and Testing
- Fine-Tuning LLMs for Specific NLP Tasks
- Fine-Tuning LLMs for Sentiment Analysis
- Preparing a Labeled Dataset for Sentiment Analysis
- Preparing a Labeled Dataset for Text Classification
- LLM Agents
- What Is Few-Shot Learning?
- Few-Shot Learning and Prompts
- Fine-Tuning Versus Few-Shot Learning
- Fine-Tuning
- Few-Shot Learning
- Fine-Tuning LLMs
- LoRA, Quantization, and QLoRA
- Parameter-Efficient Fine-Tuning (PEFT)
- Step-by-Step Fine-Tuning
- Fine-Tuning Versus Prompt Engineering
- Massive Prompts Versus LLM Fine-Tuning
- Synthetic Data and Fine-Tuning
- Fine-Tuning Tips
- LLM Benchmarks
- What Is Catastrophic Forgetting?
- Fine-Tuning and Reinforcement Learning (Optional)
- Discrete Probability Distributions
- Gini Impurity
- Entropy
- Cross Entropy
- Kullback Leibler Divergence (KLD)
- RLHF
- TRPO and PPO
- DPO
- Summary
- Chapter 7: SVG and GPT-4
- Working with SVG
- Use Cases for SVG
- Accessibility and SVG
- Security Issues with SVG
- SVG Linear Gradients
- SVG Radial Gradients
- A Triangle with a Radial Gradient
- SVG 2D Shapes and Gradients
- A Bar Chart in SVG
- SVG Quadratic Bezier Curves
- SVG Cubic Bezier Curves
- SVG and 2D Transforms
- Animated SVG Cubic Bezier Curves
- Hover Effects
- Hover Animation Effects
- SVG Versus CSS3: A Comparison
- SVG Versus PNG: A Comparison
- SVG Filters
- SVG Blur Filter
- SVG Turbulence Filter
- SVG and CSS3 in HTML Web Pages
- SVG and JavaScript in HTML Web Pages
- Elliptic Arcs with a Radial Gradient
- An SVG Checkerboard Pattern
- An SVG Checkerboard Pattern with Filter Effects
- A Master-Detail HTML Web Page
- Summary
- Chapter 8: Miscellaneous Topics
- Common Biases in Generative AI
- Bias Mitigation in Generative AI
- Ethical Issues in Generative AI
- Safety Issues in Generative AI
- Multilingual Generative AI
- Privacy and Security Issues
- Sustainability Issues
- Human/AI Collaboration
- Generative AI and Governance
- Advanced Data Handling Techniques
- Interdisciplinary Applications of Generative AI
- Hybrid Models in Generative AI
- Deploying Models to Production
- Case Studies and Industry Insights
- Gen AI Integration with IoT and Edge Devices
- What Are Guardrails in AI?
- Vector Databases
- Hardware Requirements for AI Modeling
- LLMs and Mobile Devices
- Quantum Computing and AI
- Robotics and Generative AI
- Neuromorphic Computing
- Augmented Reality and Virtual Reality
- LLMs and Deception
- LLMs and Intentional Deception
- The Generative AI Process
- Generating Text with a Language Model
- Training an ML Model Versus a Generative AI Model
- Future Trends in Generative AI
- Summary
- Index
