
Build a Large Language Model (From Scratch)
Sebastian Raschka(Author)
Manning Publications (Publisher)
Published on 1. November 2024
Book
Hardback
368 pages
978-1-63343-716-6 (ISBN)
Description
Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up!
In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You'll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks.
Build a Large Language Model (from Scratch) teaches you how to:
Plan and code all the parts of an LLM
Prepare a dataset suitable for LLM training
Fine-tune LLMs for text classification and with your own data
Use human feedback to ensure your LLM follows instructions
Load pretrained weights into an LLM
Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you'll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant. About the technology Physicist Richard P. Feynman reportedly said, "I don't understand anything I can't build." Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning.
In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You'll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks.
Build a Large Language Model (from Scratch) teaches you how to:
Plan and code all the parts of an LLM
Prepare a dataset suitable for LLM training
Fine-tune LLMs for text classification and with your own data
Use human feedback to ensure your LLM follows instructions
Load pretrained weights into an LLM
Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you'll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant. About the technology Physicist Richard P. Feynman reportedly said, "I don't understand anything I can't build." Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning.
More details
Language
English
Place of publication
New York
United States
Target group
Professional and scholarly
Dimensions
Height: 232 mm
Width: 189 mm
Thickness: 25 mm
Weight
624 gr
ISBN-13
978-1-63343-716-6 (9781633437166)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions

Test Yourself on Sebastian Raschka's Build a Large Language Model (From Scratch)
300+ practice problems to cement your learning
E-Book
07/2025
Simon + Schuster LLC
€0.00
Available for download

Sebastian Raschka
Build a Large Language Model (From Scratch)
E-Book
10/2024
1st Edition
Simon + Schuster LLC
€49.44
Available for download
Person
Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software.
The technical editor on this book was David Caswell.
The technical editor on this book was David Caswell.