Beginning Hadoop
Understanding Hadoop Scalability and Performance of Clusters
1st Edition
Published on 8. April 2016
Book
Paperback/Softback
250 pages
978-1-4842-1354-4 (ISBN)
Description
There are many challenges in setting up and scaling distributed frameworks like hadoop.
Despite, Hadoop being an Open Source product and with so many good documentations and books, it is difficult for an individual or an enterprise to define various use cases or working models, that too with a clear understanding of its workings and tuning it for optimal performance.
Pro Hadoop Administration by Gurmukh Singh, a Hadoop specialist and an infrastructure architect, takes a deep dive into configuring Hadoop services and its integration with various tools or frameworks. The book covers the processes right from scratch to building a Hadoop cluster at the production level, with best practices and optimal performance.
You will learn:
* Use Cases and set of recipes for the Hadoop production environment.* From Compiling Hadoop to setting up Cluster with Highly available services.* It's integration with various tools like Sqoop, Flume, HBase, Hive and many more.* Performance tuning and Cluster Planning.* Hadoop security like Kerberos, Encryption and other aspects of security like OS and Network Level.
More details
Edition
1., st ed. 2016
Language
English
Place of publication
Berkeley
United States
Publishing group
Springer-Verlag Berlin and Heidelberg GmbH & Co. KG
Target group
Professional and scholarly
Popular/general
Illustrations
biography
Dimensions
Height: 25.4 cm
Width: 17.8 cm
ISBN-13
978-1-4842-1354-4 (9781484213544)
Schweitzer Classification
Person
Gurmukh has over 12 years of experience in Infrastructure design, scalability, performance tuning and distributed Computing. He recently, Co-Founded "Netxillon Technologies", which is into BigData Consultancy services and trainings. Prior to starting his venture, he worked with companies like Yahoo, HP, JP Morgan on various technologies like OpenVMS, Yahoo Web Analytics platform and many network and security appliances. His areas of expertise include Scalability and Performance Engineering, Databases, Optimising Hadoop Infrastructure, Proxy Appliances and Automation. In addition to this he mentors and trains engineers on latest technologies and market trends.
Content
Chapter 1: Introduction to Distributed Computing and Hadoop.Chapter Goal: Talk about the Distributed computing, challenges and some of the existing platforms in the market.
Sub -Topics
* Introduction to Distributed computing.* Introduction to Hadoop and its history* Current Hadoop distributions and its market.* Problem statement why Hadoop is needed and its use cases
Chapter 2: Hadoop as a PlatformChapter Goal: Install and configure Hadoop basic Services
Sub - Topics
* Hadoop Compilation.* Hadoop Installation and its various modes* Hadoop Daemons Configuration.* Basic Hadoop Configuration Parameters.Chapter 3: Hadoop Daemons and ServicesChapter Goal: Setup Hadoop Secondary namenode and its purpose.
Sub - Topics:
* Secondary NameNode Setup.* Namenode Metadata Concepts.3. Recovery from Secondary namenode
4. Failover to Secondary
Chapter 4: Concepts of redundancy and Data Access Chapter Goal: Understand how replication works and setup rack awareness
Sub - Topics:
* Configure Hadoop Clients* Multi-A record Clients* * 4. Disk Storage Concepts.
Chapter 4: Hadoop Administration TasksChapter Goal: Learn about day-to-day activities, which are performed by Hadoo
p Admins like Cluster balancing, disk space issues etc
* Hadoop Cluster balancing.* Cluster Membership.* Adding Disks to Data Nodes* NameNode Metadata Operations* Trash Space Configuration
Chapter 5: User Quota Management and SchedulersChapter Goal: Learn about User management and Space Quota etc
* User Management.* Space Quota Management.* Job Schedulers* Queue setup and management.* ACL's for Queues.
Chapter 6: Hadoop 2.x and YARN ConfigurationChapter Goal: Learn about Hadoop 2.x features and YARN framework.
* Introduction to Hadoop 2.x.* Hadoop 2.x features.* Introduction to YARN and its components.* Installation and Configuration of YARN.* Setup Job Queues.
Chapter 7: Making Services High AvailableChapter Goal: Learn about High Availability for Namenode and Resource Manager.
* Namenode HA using Shared Storage.* Namenode HA using QJM.* Resource Manager HA.
Chapter 8: Data Ingestion using HIVE, PIG, SQOOP, FLUMEChapter Goal: Learn about Hive, sqoop, and flume for data ingestion.
* Introduction to Data Ingestion.* Introduction to PIG and its installation.* Introduction to Hive and its installation.* Introduction to SQOOP and its installation.* Introduction to Flume and its installation.* Examples for Data Ingestion.Chapter 9: Database for the Hadoop Platform.Chapter Goal: Learn about HBase and its integration with other tools of Hadoop.
* Introduction to HBase.* HBase Installation.* HBase with Hive* Importing Data from HBase* Phoneix with Hbase
Chapter 10: Hadoop Security.Chapter Goal: Learn about securing Hadoop with Kerberos and other tools.
* Introduction to Kerberos.* Installation and Configuring Kerberos.* Hadoop with Kerberos.* Securing Hadoop at the OS level.
Chapter 11: Hadoop Cluster Planning and performance.Chapter Goal: Learn about Cluster planning and performance tuning and other tools.
* Hadoop Cluster Planning.* Map Reduce Phases.* Performance tuning.* Hadoop Benchmarking.
Chapter 12: Hadoop Advanced Features.Chapter Goal: Learn about Federation, NFS, webHDFS.
* Introduction to Hadoop Federation.* Setup Hadoop Federation.* Introduction to Snapshots and its configuration.* NFSv3 configuration for Hadoop.* WebHDFS for REST API calls.