Hadoop: The Definitive Guide
1st Edition
Published on 14. July 2009
Book
Paperback/Softback
524 pages
978-0-596-52197-4 (ISBN)
Description
Hadoop: The Definitive Guide helps you harness the power of your data. Ideal for processing large datasets, the Apache Hadoop framework is an open source implementation of the MapReduce algorithm on which Google built its empire. This comprehensive resource demonstrates how to use Hadoop to build reliable, scalable, distributed systems: programmers will find details for analyzing large datasets, and administrators will learn how to set up and run Hadoop clusters.
Complete with case studies that illustrate how Hadoop solves specific problems, this book helps you:
* Use the Hadoop Distributed File System (HDFS) for storing large datasets, and run distributed computations over those datasets using MapReduce
* Become familiar with Hadoop's data and I/O building blocks for compression, data integrity, serialization, and persistence
* Discover common pitfalls and advanced features for writing real-world MapReduce programs
* Design, build, and administer a dedicated Hadoop cluster, or run Hadoop in the cloud
* Use Pig, a high-level query language for large-scale data processing
* Take advantage of HBase, Hadoop's database for structured and semi-structured data
* Learn ZooKeeper, a toolkit of coordination primitives for building distributed systems
If you have lots of data -- whether it's gigabytes or petabytes -- Hadoop is the perfect solution. Hadoop: The Definitive Guide is the most thorough book available on the subject.
"Now you have the opportunity to learn about Hadoop from a master-not only of the technology, but also of common sense and plain talk."
-- Doug Cutting, Hadoop Founder, Yahoo!
Complete with case studies that illustrate how Hadoop solves specific problems, this book helps you:
* Use the Hadoop Distributed File System (HDFS) for storing large datasets, and run distributed computations over those datasets using MapReduce
* Become familiar with Hadoop's data and I/O building blocks for compression, data integrity, serialization, and persistence
* Discover common pitfalls and advanced features for writing real-world MapReduce programs
* Design, build, and administer a dedicated Hadoop cluster, or run Hadoop in the cloud
* Use Pig, a high-level query language for large-scale data processing
* Take advantage of HBase, Hadoop's database for structured and semi-structured data
* Learn ZooKeeper, a toolkit of coordination primitives for building distributed systems
If you have lots of data -- whether it's gigabytes or petabytes -- Hadoop is the perfect solution. Hadoop: The Definitive Guide is the most thorough book available on the subject.
"Now you have the opportunity to learn about Hadoop from a master-not only of the technology, but also of common sense and plain talk."
-- Doug Cutting, Hadoop Founder, Yahoo!
More details
Other editions
New editions
Book
05/2015
4th Edition
O'Reilly
€60.50
Shipment within 10-20 days
Additional editions
E-Book
05/2009
1st Edition
O'Reilly
€35.49
Available for download
E-Book
05/2009
1st Edition
O'Reilly
€37.49
Available for download
Person
Tom White has been an Apache Hadoop committer since February 2007, and is a member of the Apache Software Foundation. He works for Cloudera, a company set up to offer Hadoop support and training. Previously he was as an independent Hadoop consultant, working with companies to set up, use, and extend Hadoop. He has written numerous articles for O'Reilly, java.net and IBM's developerWorks, and has spoken at several conferences, including at ApacheCon 2008 on Hadoop. Tom has a Bachelor's degree in Mathematics from the University of Cambridge and a Master's in Philosophy of Science from the University of Leeds, UK.
Content
Inhaltsverzeichnis
* Chapter 1 Meet Hadoop
* Data!
* Data Storage and Analysis
* Comparison with Other Systems
* A Brief History of Hadoop
* The Apache Hadoop Project
* Chapter 2 MapReduce
* A Weather Dataset
* Analyzing the Data with Unix Tools
* Analyzing the Data with Hadoop
* Scaling Out
* Hadoop Streaming
* Hadoop Pipes
* Chapter 3 The Hadoop Distributed Filesystem
* The Design of HDFS
* HDFS Concepts
* The Command-Line Interface
* Hadoop Filesystems
* The Java Interface
* Data Flow
* Parallel Copying with distcp
* Hadoop Archives
* Chapter 4 Hadoop I/O
* Data Integrity
* Compression
* Serialization
* File-Based Data Structures
* Chapter 5 Developing a MapReduce Application
* The Configuration API
* Configuring the Development Environment
* Writing a Unit Test
* Running Locally on Test Data
* Running on a Cluster
* Tuning a Job
* MapReduce Workflows
* Chapter 6 How MapReduce Works
* Anatomy of a MapReduce Job Run
* Failures
* Job Scheduling
* Shuffle and Sort
* Task Execution
* Chapter 7 MapReduce Types and Formats
* MapReduce Types
* Input Formats
* Output Formats
* Chapter 8 MapReduce Features
* Counters
* Sorting
* Joins
* Side Data Distribution
* MapReduce Library Classes
* Chapter 9 Setting Up a Hadoop Cluster
* Cluster Specification
* Cluster Setup and Installation
* SSH Configuration
* Hadoop Configuration
* Post Install
* Benchmarking a Hadoop Cluster
* Hadoop in the Cloud
* Chapter 10 Administering Hadoop
* HDFS
* Monitoring
* Maintenance
* Chapter 11 Pig
* Installing and Running Pig
* An Example
* Comparison with Databases
* Pig Latin
* User-Defined Functions
* Data Processing Operators
* Pig in Practice
* Chapter 12 HBase
* HBasics
* Concepts
* Installation
* Clients
* Example
* HBase Versus RDBMS
* Praxis
* Chapter 13 ZooKeeper
* Installing and Running ZooKeeper
* An Example
* The ZooKeeper Service
* Building Applications with ZooKeeper
* ZooKeeper in Production
* Chapter 14 Case Studies
* Hadoop Usage at Last.fm
* Hadoop and Hive at Facebook
* Nutch Search Engine
* Log Processing at Rackspace
* Cascading
* TeraByte Sort on Apache Hadoop
* Appendix Installing Apache Hadoop
* Prerequisites
* Installation
* Configuration
* Appendix Cloudera's Distribution for Hadoop
* Prerequisites
* Standalone Mode
* Pseudo-Distributed Mode
* Fully Distributed Mode
* Hadoop-Related Packages
* Appendix Preparing the NCDC Weather Data
* Colophon
* Chapter 1 Meet Hadoop
* Data!
* Data Storage and Analysis
* Comparison with Other Systems
* A Brief History of Hadoop
* The Apache Hadoop Project
* Chapter 2 MapReduce
* A Weather Dataset
* Analyzing the Data with Unix Tools
* Analyzing the Data with Hadoop
* Scaling Out
* Hadoop Streaming
* Hadoop Pipes
* Chapter 3 The Hadoop Distributed Filesystem
* The Design of HDFS
* HDFS Concepts
* The Command-Line Interface
* Hadoop Filesystems
* The Java Interface
* Data Flow
* Parallel Copying with distcp
* Hadoop Archives
* Chapter 4 Hadoop I/O
* Data Integrity
* Compression
* Serialization
* File-Based Data Structures
* Chapter 5 Developing a MapReduce Application
* The Configuration API
* Configuring the Development Environment
* Writing a Unit Test
* Running Locally on Test Data
* Running on a Cluster
* Tuning a Job
* MapReduce Workflows
* Chapter 6 How MapReduce Works
* Anatomy of a MapReduce Job Run
* Failures
* Job Scheduling
* Shuffle and Sort
* Task Execution
* Chapter 7 MapReduce Types and Formats
* MapReduce Types
* Input Formats
* Output Formats
* Chapter 8 MapReduce Features
* Counters
* Sorting
* Joins
* Side Data Distribution
* MapReduce Library Classes
* Chapter 9 Setting Up a Hadoop Cluster
* Cluster Specification
* Cluster Setup and Installation
* SSH Configuration
* Hadoop Configuration
* Post Install
* Benchmarking a Hadoop Cluster
* Hadoop in the Cloud
* Chapter 10 Administering Hadoop
* HDFS
* Monitoring
* Maintenance
* Chapter 11 Pig
* Installing and Running Pig
* An Example
* Comparison with Databases
* Pig Latin
* User-Defined Functions
* Data Processing Operators
* Pig in Practice
* Chapter 12 HBase
* HBasics
* Concepts
* Installation
* Clients
* Example
* HBase Versus RDBMS
* Praxis
* Chapter 13 ZooKeeper
* Installing and Running ZooKeeper
* An Example
* The ZooKeeper Service
* Building Applications with ZooKeeper
* ZooKeeper in Production
* Chapter 14 Case Studies
* Hadoop Usage at Last.fm
* Hadoop and Hive at Facebook
* Nutch Search Engine
* Log Processing at Rackspace
* Cascading
* TeraByte Sort on Apache Hadoop
* Appendix Installing Apache Hadoop
* Prerequisites
* Installation
* Configuration
* Appendix Cloudera's Distribution for Hadoop
* Prerequisites
* Standalone Mode
* Pseudo-Distributed Mode
* Fully Distributed Mode
* Hadoop-Related Packages
* Appendix Preparing the NCDC Weather Data
* Colophon