Databricks Data Intelligence Platform
Description
Unlock the full potential of the Databricks Data Intelligence Platform in this fully updated second edition-your definitive guide to building enterprise-ready Generative AI (GenAI) solutions, managing complex data workflows, and deploying intelligent applications at scale. Whether you're a data engineer, AI developer, data scientist, or technology leader, this book empowers you to transform your organization with cutting-edge AI capabilities.
This edition brings a sharp focus to the latest advancements in Artificial Intelligence within Databricks. Learn how to build bespoke LLMs tailored to your enterprise, streamline development with Lakeflow and Declarative Pipelines, and harness the power of Agent Bricks and serverless GPU-now fully integrated with Mosaic AI. You'll also discover how Lakebase and Databricks Apps enable developers to create secure, governed AI experiences directly within the data ecosystem managed by Unity Catalog.
Whether you're new to Databricks and seeking a comprehensive, accessible introduction, or an experienced professional aiming to master the platform's latest AI capabilities and scale GenAI initiatives, this book delivers the insights, tools, and best practices you need to lead in today's data-driven, AI-powered enterprise landscape.
This book is an Open access publication
What You Will Learn
- Know the core principles of Lakehouse architecture, now enhanced with Lakebase
- Take a deep dive into Mosaic AI, Agent Bricks, and serverless GPU for scalable AI development
- Understand best practices for building and deploying GenAI applications from data ingestion to model serving and user interface integration
- Use Lakeflow, Declarative Pipelines, and Agent Bricks to streamline data and AI workflows
- Build secure, governed AI apps using Databricks Apps and Unity Catalog
Who This Book Is For
Solution architects, data engineers, data scientists, and Databricks practitioners who want to build and deploy Generative AI solutions using the Databricks Data Intelligence Platform. It's also a strategic resource for senior executives seeking to articulate the business value of Databricks to customers and stakeholders. People who are new to the Databricks Platform and want comprehensive insights will find the book accessible.
More details
Other editions
Previous edition
Persons
Jason Yip is a Databricks Most Valued Professional (MVP) and Director of Data and AI at Tredence, a Databricks Elite Partner. He serves on Databricks advisory boards, including the Partner Product Advisory Board and the Solution Architect Champion Advisory Board, and advises Fortune 500 companies on data and Generative AI strategies.
Nikhil Gupta is a Solution Architect at Databricks with more than 18 years of experience in big data technologies. He helps customers across industries modernize data and AI platforms, with expertise spanning data warehousing, data lakes, and real-time data processing.
Marcin Wojtyczka is a Practice Lead Resident Solutions Architect at Databricks and the creator of the open-source DQX data quality framework. A former software engineer and data architect, he has spent years building large-scale data platforms and ML/AI systems. Today, he builds reusable libraries and frameworks which enhance and complement the Databricks platform powering the next generation of data intelligence. Beyond the code, Marcin is a frequent speaker, active open-source contributor, product builder, and expedition sailor.
Content
Chapter 1. Databricks Platform: From Lakehouse to Data Intelligence.- Chapter 2. Generative AI on Databricks: Foundations, Capabilities,and Use Cases.- Chapter 3. Composable Agents: From Prototype to Production.- Chapter 4. Agent Bricks Deep Dive: Retrieval, Reasoning, and Tooling Architectures.- Chapter 5. Agentic AI Design Patterns.- Chapter 6. Databricks as an Agentic Platform.- Chapter 7. Quality Tuning with Agent Bricks.- Chapter 8. Database Lakebases.- Chapter 9. Building Security-First, Production-Grade Databricks Apps.- Chapter 10. Serverless GPU: A Tunable Copilot.- Chapter 11. MLflow 3.0 and the GenAI Agents.- Chapter 12. Spark Structured Streaming and Real-Time Machine Learning.- Chapter 13. Lakeflow Connect: Data Ingestion in the Lakehouse.- Chapter 14. Open Data Governance with Unity Catalog.- Chapter 15. Delta Lake: A Deep Dive.- Chapter 16. Lakeflow: Declarative Pipelines.- Chapter 17. Data Warehousing with Databricks SQL (DBSQL).- Chapter 18. Databricks Pricing and Observability Using System Tables.- Chapter 19: From Ideation to Creation: When AI Meets Surgeon.