GPU Programming Primer

This book is a practical, system-oriented introduction to GPU programming, written specifically for traditional software developers, solution architects, site reliability engineers (SREs), and AI/MLOps practitioners who want to understand-and confidently apply-GPU acceleration in real-world systems.

Rather than focusing only on isolated CUDA syntax or theoretical parallel computing concepts, this book teaches you how to think in parallel, how GPUs actually work under the hood, and how to design, optimize, and operate GPU-accelerated applications in production environments.

You will start with the fundamentals-GPU architecture, execution models, and memory hierarchies-before progressing to hands-on GPU programming concepts, performance tuning techniques, and common parallel programming patterns. Along the way, you'll learn how GPU code behaves differently from CPU code, how to avoid common pitfalls such as memory bottlenecks and warp divergence, and how to reason about performance, scalability, and reliability.

The book then expands beyond development into systems and operations, covering how GPU workloads fit into modern cloud-native and AI platforms. Topics such as multi-GPU execution, observability, cost awareness, and production considerations are explained in a way that aligns with the responsibilities of architects, SREs, and MLOps teams.