
Building Large Language Models from Scratch
Description
Alles über E-Books | Antworten auf Fragen rund um E-Books, Kopierschutz und Dateiformate finden Sie in unserem Info- & Hilfebereich.
More details
Other editions
Additional editions

Person
Dilyan Grigorov is a software developer with a passion for Python software development, generative deep learning & machine learning, data structures, and algorithms. He is an advocate for open source and the Python language itself. He has 16 years of industry experience programming in Python and has spent 5 of those years researching and testing Generative AI solutions. His passion for them stems from his background as an SEO specialist dealing with search engine algorithms daily. He enjoys engaging with the software community, often giving talks at local meetups and larger conferences. In his spare time, he enjoys reading books, hiking in the mountains, taking long walks, playing with his son, and playing the piano.
Content
Chapter 1: What is a Large Language Model? Getting Started with Libraries and Environment Setup For Building an LLM from Scratch.- Chapter 2: Foundational Concepts in LLM Development.- Chapter 3: Building a Tokenizer For Transformers Architecture Model.- Chapter 4: RMS Normalization and Model Configuration.- Chapter 5: Rotary Positional Embeddings: Integrating NTK and YaRN Scaling.- Chapter 6: Scaled Dot-Product Attention Core - Sliding Window and Grouped Query Attention - ?he Core Behind All Transformer Models.- Chapter 7: AttentionBlock with Rotary Embedding & GQA & Sliding Window & Sink Tokens.- Chapter 8: MultiLayer Perceptron Block with Mixture of Experts (MoE) and SwiGLU.- Chapter 9: Transformer Block & Full Transformer Model - It's Time To Put The Puzzle Together.- Chapter 10: Dataset Preparation, Model Training, TokenGenerator for Inference & Prompting - The BIG Moment.- Chapter 11: Advanced Training and CUDA Kernels.
System requirements
File format: PDF
Copy protection: Watermark-DRM (Digital Rights Management)
System requirements:
- Computer (Windows; MacOS X; Linux): Use the free software Adobe Reader, Adobe Digital Editions, or any other PDF viewer of your choice (see eBook Help).
- Tablet/Smartphone (Android; iOS): Install the free app Adobe Digital Editions or another reading app for eBooks, e.g., PocketBook (see eBook Help).
- E-reader: Bookeen, Kobo, Pocketbook, Sony, Tolino and many more (only limited: Kindle).
The file format PDF always displays a book page identically on any hardware. This makes PDF suitable for complex layouts such as those used in textbooks and reference books (images, tables, columns, footnotes). Unfortunately, on the small screens of e-readers or smartphones, PDFs are rather annoying, requiring too much scrolling.
This eBook uses Watermark-DRM, a „soft” copy protection. This means that there are no technical restrictions to prevent illegal distribution. However, there is a personalised watermark embedded in the eBook that can be used to identify the purchaser of the eBook in the event of misuse and to provide evidence for legal purposes.
For more information, see our eBook Help page.