Preface

This book, in its second edition now, has been explicitly written with a view to helping you learn Linux kernel development in a practical, hands-on fashion, along with the necessary theoretical background to give you a well-rounded view of this vast and interesting topic area. It deliberately focuses on kernel development via the powerful Loadable Kernel Module (LKM) framework; this is because the vast majority of real-world/industry kernel projects and products, which includes device driver development, are done in this manner.

The focus is kept on both working hands-on with, and understanding at a sufficiently deep level, the internals of the Linux OS. In this regard, we cover everything from building the Linux kernel from source to understanding and working with complex topics such as synchronization within the kernel.

To guide you on this exciting journey, we divide this book into three sections. The first section covers the basics - setting up an appropriate workspace for kernel development, building the modern kernel from source, and writing your first kernel module.

The next section, a key one, will help you understand essential kernel internals details; its coverage includes the Linux kernel architecture, the task structure, user - and kernel-mode stacks, and memory management. Memory management is a key and interesting topic - we devote three whole chapters to it (covering the internals to a sufficient extent, and importantly, how exactly to efficiently allocate and free kernel memory). The internal working and deeper details of CPU (task) scheduling on the Linux OS round off this section.

The last section of the book deals with the more advanced topic of kernel synchronization - a necessity for professional design and code on the Linux kernel. We devote two whole chapters to covering key topics here.

The book uses the kernel community's 6.1 Long Term Support (LTS) Linux kernel. It's a kernel that will be maintained (both bug and security fixes) from December 2022 right through to December 2026. Moreover, the CIP (Civil Infrastructure Project) has adopted 6.1 as an SLTS (Super LTS) release and plans to maintain it for 10 years, until August 2033! This is a key point, ensuring that this book's content remains current and valid for years to come!

We very much believe in a hands-on approach: some 40 kernel modules (besides several user apps and shell scripts, double that of the first edition!) in this book's GitHub repository make the learning come alive, making it fun, practical, interesting, and useful.

We highly recommend you also make use of this book's companion guide, Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization: Create user-kernel interfaces, work with peripheral I/O, and handle hardware interrupts. It's an excellent industry-aligned beginner's guide to writing misc character drivers, performing I/O on peripheral chip memory, and handling hardware interrupts. You can get this book for free along with your print copy; alternately, you can also find this eBook in the GitHub repository at https://github.com/PacktPublishing/Linux-Kernel-Programming/tree/master/Linux-Kernel-Programming-(Part-2).

We really hope you learn from and enjoy this book. Happy reading!

Who this book is for

This book is primarily for those of you beginning your journey in the vast arena of learning and understanding modern Linux kernel architecture and internals, Linux kernel module development and, to some extent, Linux device driver development. It's also very much targeted at those of you who have already been working on Linux modules and/or drivers, who wish to gain a much deeper, well-structured understanding of Linux kernel architecture, memory management, task scheduling, cgroups, and synchronization. This level of knowledge about the underlying OS, covered in a properly structured manner, will help you no end when you face difficult-to-debug real-world situations.

What's been added in the second edition?

A pretty huge amount of new material has been added into this, the Second Edition of the Linux Kernel Programming book. As well, being based on the very recent (as of this writing) 6.1 LTS release, its information and even code will remain industry-relevant for many, many years to come.

Here's a quick chapter-wise summarization of what's new in this second edition:

• Materials updated for the 6.1 LTS kernel, maintained until December 2026, and until August 2033 via the CLP (6.1 SLTS)!
• Updated, new, and working code for the 6.1 LTS kernel
• Several new info-rich sections added to most chapters, many new diagrams, and new code examples to help explain concepts better
• Chapter 1, Linux Kernel Programming - A Quick Introduction
  • Introduction to the book
• Chapter 2, Building the 6.x Linux Kernel from Source - Part 1
  • The new LTS kernel lifetime mandate
  • More details on the kernel's Kconfig+Kbuild system
  • Updated approaches on configuring the kernel
• Chapter 3, Building the 6.x Linux Kernel from Source - Part 2
  • More details on the initramfs (initrd) image
  • Cross-compiling the kernel on an x86_64 host to an AArch64 target
• Chapter 4, Writing Your First Kernel Module - Part 1
  • new-ish printk indexing feature covered
  • Powerful kernel dynamic debug feature introduced
  • Rate-limiting macros updated (deprecated ones not used)
• Chapter 5, Writing Your First Kernel Module - Part 2
  • A better, 'better' Makefile (v0.2)
• Chapter 6, Kernel Internals Essentials - Processes and Threads
  • New linked list demo module
• Chapter 7, Memory Management Internals - Essentials
  • New coverage on how address translation works (including diagrams)
• Chapter 8, Kernel Memory Allocation for Module Authors - Part 1
  • Coverage on using the "exact" page allocator API pair
  • FAQs regarding (slab) memory usage and their answers
  • The graphing demo (via gnuplot) is now automated and even saved to an image file, via a helper script
  • Finding internal fragmentation (wastage) within the kernel
• Chapter 9, Kernel Memory Allocation for Module Authors - Part 2
  • Extracting useful information regarding slab caches
  • A word on slab shrinkers
  • Better coverage on the OOM killer (and systemd-oomd) and how it's triggered; includes a flowchart depicting demand-paging and possible OOM killer invocation
  • Better coverage on kernel page reclaim, as well as the new MGLRU and DAMON technologies
• Chapter 10, The CPU Scheduler - Part 1
  • New coverage on CFS scheduling period and timeslice. Coverage on the thread_info structure as well
  • New: the preempt dynamic feature
  • Enhanced coverage on exactly how and when schedule() is invoked
• Chapter 11, The CPU Scheduler - Part 2
  • Much more depth in the powerful cgroups (v2) coverage plus an interesting script to let you explore its content
  • Leveraging the cgroups v2 CPU controller via both systemd and manually to perform CPU bandwidth allocation
  • A note on Google's ghOSt OS
• Chapter 12, Kernel Synchronization - Part 1
  • A new intro to the LKMM (Linux Kernel Memory Model)
  • More on locking plus deadlock avoidance guidelines
• Chapter 13, Kernel Synchronization - Part 2
  • Expanded coverage on CPU caching and cache effects
  • New coverage on the powerful lock-free RCU synchronization technology
• Online Chapter, Kernel Workspace Setup
  • Fixed errors in package names and versions
  • Ubuntu-based helper script that auto-installs all required packages

Most (if not all) earlier code errors, typos, and URLs are now fixed, based on prompt feedback, raising Issues/PRs on the book's GitHub repo, from you, our wonderful readers!

What this book covers

Chapter 1, Linux Kernel Programming - A Quick Introduction, briefs you about the exciting journey in the sections of the book, which cover everything from building the Linux kernel from source to understanding and...