DevOps for VMware Administrators
Published on 16. April 2015
Book
Paperback/Softback
384 pages
978-0-13-384647-8 (ISBN)
Description
DevOps for VMware (R) Administrators is the first book focused on using DevOps tools and practices with VMware technologies. The authors introduce high-value tools from third parties and VMware itself, and guide you through using them to improve the performance of all your virtualized systems and applications. You'll walk through automating and optimizing configuration management, provisioning, log management, continuous integration, and more.
The authors also offer step-by-step coverage of deploying and managing applications at scale with Docker containers and Google Kubernetes. They conclude with an up-to-the-minute discussion of VMware's newest DevOps initiatives, including VMware vRealize Automation and VMware
vRealize Code Stream.
Coverage includes
* Understanding the challenges that DevOps tools and practices can help VMware administrators to solve
* Using Vagrant to quickly deploy Dev and Test environments that match production system specifications
* Writing Chef "recipes" that streamline server configuration and maintenance
* Simplifying Unix/Linux configuration management and orchestration with Ansible
* Implementing Docker containers for faster and easier application management
* Automating provisioning across the full lifecycle with Razor
* Integrating Microsoft PowerShell Desired State Configuration (DSC) and VMware PowerCLI to automate key Windows Server and vSphere VM admin tasks
* Using Puppet to automate infrastructure provisioning, configuration, orchestration, and reporting
* Supercharging log management with ELK (Elasticsearch, Logstash, Kibana)
* Supporting DevOps source code management with Git and continuous integration practices with Jenkins
* Achieving continuous integration, delivery, and deployment with VMware's vRealize Code Stream
The authors also offer step-by-step coverage of deploying and managing applications at scale with Docker containers and Google Kubernetes. They conclude with an up-to-the-minute discussion of VMware's newest DevOps initiatives, including VMware vRealize Automation and VMware
vRealize Code Stream.
Coverage includes
* Understanding the challenges that DevOps tools and practices can help VMware administrators to solve
* Using Vagrant to quickly deploy Dev and Test environments that match production system specifications
* Writing Chef "recipes" that streamline server configuration and maintenance
* Simplifying Unix/Linux configuration management and orchestration with Ansible
* Implementing Docker containers for faster and easier application management
* Automating provisioning across the full lifecycle with Razor
* Integrating Microsoft PowerShell Desired State Configuration (DSC) and VMware PowerCLI to automate key Windows Server and vSphere VM admin tasks
* Using Puppet to automate infrastructure provisioning, configuration, orchestration, and reporting
* Supercharging log management with ELK (Elasticsearch, Logstash, Kibana)
* Supporting DevOps source code management with Git and continuous integration practices with Jenkins
* Achieving continuous integration, delivery, and deployment with VMware's vRealize Code Stream
More details
Other editions
Additional editions
Trevor Roberts | Josh Atwell | Egle Sigler
DevOps for VMware Administrators
E-Book
03/2015
1st Edition
VMWare Press
from
€30.00
Available for download
E-Book
03/2015
1st Edition
VMWare Press
€32.49
Available for download
Persons
Trevor Roberts, Jr. is a Senior Technical Marketing Manager for VMware. Trevor
has the CCIE Data Center certification, and he is a VMware Certified Advanced Professional
in the Data Center Design and Administration concentrations. In his spare time,
Trevor shares his insights on data center technologies at http://www.VMTrooper.
com, via the vBrownBag Professional OpenStack and Professional VMware podcasts,
and on Twitter (@VMTrooper). His contributions to the IT community have garnered
recognition by his designation as a VMware vExpert, Cisco Data Center Champion, and
EMC Elect.
Josh Atwell is a Cloud Architect for SolidFire, focusing on VMware and automation
solutions. Over the past 10+ years, he has worked very hard to allow little pieces of code
to do his work for him through various automation tools. Josh is a father of two boys with
wife Stephanie, and a daughter is on the way in early 2015. Based in the Raleigh, North
Carolina, area, he enjoys time with his family, golf, audiobooks, and trying new bourbons.
Josh has been highly active in the virtualization community, where he's been a leader of
technology-based user groups such as CIPTUG, VMUG, and UCS Users Group. Josh
has worked with others on preparing for their professional development pursuits through
the vBrownBag podcast and the Virtual Design Master competition. Josh is also a regular
public speaker and contributing author to the Mastering vSphere series. Never known for
lacking an opinion, he blogs at vtesseract.com and talks shop on Twitter as @Josh_Atwell.
Egle Sigler (@eglute, anystacker.com) is currently a Principal Architect at Rackspace. She
started her career as a software developer, and still has a soft spot for all the people that
write, test, and deploy code, because she had a chance to do all of those tasks. Egle dreams
about a day when writing, testing, and deploying code will be a seamless and easy process,
bug and frustration free for all. Egle believes that knowledge should be shared, and tries to
do so by writing this book, giving talks and workshops at conferences, and blogging.
Yvo van Doorn has more than a decade of system administration experience. The first
part of his career, he manually built out and configured bare-metal servers. At Classmates,
Yvo became a champion of configuration management and virtualization. Before joining
Chef, he learned firsthand the power of VMware's products when he moved a small Seattle
technology company's complete production stack over to its virtualization platform. He's
a strong believer in the culture change that comes with DevOps. When he isn't busy
spreading the gospel of Chef, he's probably enjoying a hoppy IPA, exploring the great
outdoors, or celebrating his Dutch heritage while eating a wheel of gouda and watching
Oranje lose the World Cup. Yvo lives with his wife and black lab in Seattle, Washington.
has the CCIE Data Center certification, and he is a VMware Certified Advanced Professional
in the Data Center Design and Administration concentrations. In his spare time,
Trevor shares his insights on data center technologies at http://www.VMTrooper.
com, via the vBrownBag Professional OpenStack and Professional VMware podcasts,
and on Twitter (@VMTrooper). His contributions to the IT community have garnered
recognition by his designation as a VMware vExpert, Cisco Data Center Champion, and
EMC Elect.
Josh Atwell is a Cloud Architect for SolidFire, focusing on VMware and automation
solutions. Over the past 10+ years, he has worked very hard to allow little pieces of code
to do his work for him through various automation tools. Josh is a father of two boys with
wife Stephanie, and a daughter is on the way in early 2015. Based in the Raleigh, North
Carolina, area, he enjoys time with his family, golf, audiobooks, and trying new bourbons.
Josh has been highly active in the virtualization community, where he's been a leader of
technology-based user groups such as CIPTUG, VMUG, and UCS Users Group. Josh
has worked with others on preparing for their professional development pursuits through
the vBrownBag podcast and the Virtual Design Master competition. Josh is also a regular
public speaker and contributing author to the Mastering vSphere series. Never known for
lacking an opinion, he blogs at vtesseract.com and talks shop on Twitter as @Josh_Atwell.
Egle Sigler (@eglute, anystacker.com) is currently a Principal Architect at Rackspace. She
started her career as a software developer, and still has a soft spot for all the people that
write, test, and deploy code, because she had a chance to do all of those tasks. Egle dreams
about a day when writing, testing, and deploying code will be a seamless and easy process,
bug and frustration free for all. Egle believes that knowledge should be shared, and tries to
do so by writing this book, giving talks and workshops at conferences, and blogging.
Yvo van Doorn has more than a decade of system administration experience. The first
part of his career, he manually built out and configured bare-metal servers. At Classmates,
Yvo became a champion of configuration management and virtualization. Before joining
Chef, he learned firsthand the power of VMware's products when he moved a small Seattle
technology company's complete production stack over to its virtualization platform. He's
a strong believer in the culture change that comes with DevOps. When he isn't busy
spreading the gospel of Chef, he's probably enjoying a hoppy IPA, exploring the great
outdoors, or celebrating his Dutch heritage while eating a wheel of gouda and watching
Oranje lose the World Cup. Yvo lives with his wife and black lab in Seattle, Washington.
Content
About the Authors xvii
About the Reviewers xviii
Acknowledgments xix
About the Contributing Author xx
Introduction xxii
About This Book xxiii
You the Reader xxiii
What This Book Covers xxiii
Part 1 Introduction to DevOps
1 Introduction to DevOps 1
An Overview of DevOps Principles 1
Implement Systems Thinking 3
Change the Approach to Team Interactions 3
Change the Approach to Infrastructure Deployment 5
Change the Approach to Software Development and Deployment 6
Collect and Respond to Useful Systems Feedback Often and Adjust Accordingly 7
Furthering Your DevOps Knowledge and Skills 7
Summary 8
References 8
2 DevOps Tools 9
Organizing for Success: Kanban 9
Server Deployment 13
Confi guration Management 14
Continuous Integration 14
Log Analysis 15
Summary 15
References 15
3 Setting Up a DevOps Confi guration Management Test Environment 17
Environment Provisioning with AutoLab 17
Environment Provisioning with Vagrant 18
Creating Images with Packer 23
Managing Source Code 24
Using Git 24
Summary 31
References 31
Part 2 Puppet
4 Introduction to Puppet 33
Puppet Architecture 33
Standalone Deployment 34
Master-Agent Deployment 34
Preparing Your Puppet Test Lab 37
Puppet Resources 38
Puppet Manifests 39
Conditional Expressions and Variables 43
Puppet Modules 46
Puppet Forge 48
Creating Your First Puppet Module 48
Puppet Module Initialization Manifest (init.pp) 50
Templates 51
Using a Puppet Module 54
Final Step: Version Control Commit 54
Summary 55
Reference 55
5 Systems Management Tasks with Puppet 57
Optimizing the Web Tier with Data Separation 58
Parameters Class (params.pp) 59
Hiera 63
Node Classifi cation 67
Application Tier 68
Database Tier 70
Implementing a Production Recommended Practice 70
Deploying the Application Environment 71
Summary 71
Reference 71
6 VMware vSphere Management with Puppet 73
Puppet's Cloud Provisioner for VMware vSphere 73
Preparing VM Templates 73
Preparing Puppet Master 74
9780133846478_ROBERTS.indb x 3/5/15 4:20 PM
Contents xi
VMware's Management Modules 77
Using the vmware/vcenter Module 77
Summary 83
References 83
Part 3 Chef
7 Introduction to Chef 85
What Is Chef? 85
Core Philosophies Behind Chef 86
Order of Recipe 86
Idempotence 86
API-Based Server 87
The Client Does All the Legwork 87
Test-Driven Infrastructure 87
Chef Terminology 87
Recipe 88
Cookbook 88
Attributes 88
Role 88
Run List 88
Resource 88
Environments 88
The Diff erence Between Hosted Chef and Chef Server 89
Hosted Chef 89
Chef Server 89
Introduction to ChefDK 90
What Is ChefDK? 90
Installing ChefDK 90
Using Knife 92
Creating Your First Hello World Chef Recipe 94
Summary 98
8 Systems Management Tasks with Chef 99
Signing Up for Hosted Chef 100
Setting Up Local Repo with the Starter Kit 102
Community Cookbooks 105
Setting Up System Management 105
Prep/Setup System Management Task 1: Managing Time 105
Prep/Setup System Management Task 2: Managing Root Password 108
9780133846478_ROBERTS.indb xi 3/5/15 4:20 PM
xii Contents
Confi guring Your Virtual Guests 109
Installing Chef Client 109
Systems Management Tasks 111
Running Chef Client 113
Managing the Root Password 115
Creating Two Environment Files 116
Uploading Environment Files to your Hosted Chef Organization 117
Assigning Each Server to an Environment 118
Modifying Each Server's Run List to Run the Managedroot Cookbook 119
Applying Your Changes to Your Nodes 120
Validating the Enforced Policy 120
Summary 122
References 123
9 VMware vSphere Management with Chef 125
Knife Plugins 126
Getting Started with knife-vsphere 128
Confi guring the knife.rb File 128
Validating the Confi guration 130
Putting It All Together 130
Chef Provisioning 134
Chef Provisioning Architecture 134
Getting Started with Chef Provisioning 135
Spinning Up Some Nodes 136
Summary 138
Part 4 Ansible
10 Introduction to Ansible 139
Ansible Architecture 139
Preparing your Ansible Test Lab 141
Ansible Groups 142
Ansible Ad Hoc Command Execution 142
The Ping Module 143
The Command Module 144
The User Module 144
The Setup Module 144
Ansible Playbooks 144
Conditional Expressions and Variables 146
Ansible Roles 151
Templates 154
Ansible Galaxy 156
Summary 157
References 157
11 Systems Management Tasks with Ansible 159
Web Server Deployment 159
The Application Tier 160
The Database Tier 162
Role Structure Optimization 164
VMware Resource Management 166
Summary 171
References 171
Part 5 PowerShell 4.0
12 Introduction to PowerShell Desired State Confi guration (DSC) 173
What Is PowerShell DSC? 174
PowerShell DSC Requirements 175
PowerShell DSC Components 175
Native Cmdlets 175
Managed Object Format File 176
Local Confi guration Manager 176
PowerShell DSC Confi gurations 178
PowerShell DSC Modes 180
Local Push Mode 181
Remote Push Mode 181
Pull Mode 182
PowerShell DSC Resources 184
Summary 186
References 187
13 Implementation Strategies with PowerShell DSC 189
Use Cases for PowerShell DSC in VMware Environments 189
Scripted Deployments of VMs with PowerCLI 190
Incorporating PowerShell DSC in VM Templates 192
Challenges Implementing PowerShell DSC Confi gurations to New VMs 193
PowerCLI Invoke-VMscript 193
PowerCLI Copy-VMGuestFile 195
General Lessons Learned 196
Future Use Cases for PowerShell DSC in VMware Environments 197
Summary 198
References 198
Part 6 Application Deployment with Containers
14 Introduction to Application Containers with Docker 199
What Is an Application? 199
Hidden Complexity 200
Dependency and Confi guration Confl icts 200
Linux Containers 200
Control Groups 201
Namespaces 201
Container Management 203
Using Docker 203
Installing Docker 203
Docker Daemon 204
Docker Client 204
Docker Index 205
Running a Docker Container 205
Listing Running Containers 206
Connecting to Running Containers 206
Building and Distributing Docker Containers 208
Dockerfi le 209
Docker Hub 210
Docker Versus Virtual Machines 211
Docker Versus Confi guration Management 211
Summary 212
References 212
15 Running Docker Containers at Scale 213
Container Orchestration 213
Kubernetes 214
Kubernetes Workfl ow 214
Kubernetes Deployment 215
CoreOS and Kubernetes Cluster Management Utilities 216
CoreOS Cluster Deployment 217
etcd Server Confi guration 222
Network Overlays with Flannel 223
Kubernetes Cluster Nodes 223
Kubernetes Service Deployment 225
Kubernetes Workload Deployment 226
Platform-as-a-Service with Docker 230
Summary 231
References 231
Part 7 DevOps Tool Chain
16 Server Provisioning Using Razor 233
How Razor Works 233
Using Razor 236
Razor Collections and Actions 238
Building Razor Collections 245
Using Razor APIs 257
Razor Components 258
Razor Server 258
Razor Microkernel 258
Razor Client 259
Setting Up Razor 259
PE Razor 259
Puppet Install 259
Install from Source 260
Manual Release Install 260
Other Services 260
Summary 263
References 263
17 Intro to the ELK: Elasticsearch, Logstash, Kibana 265
Elasticsearch Overview 265
Getting Started 266
Understanding the Index 267
Working with Data 267
Installing Plugins 271
Using Clients 274
Logstash Overview 275
Getting Started 276
Confi guring Input to Logstash 276
Applying Filters 278
Kibana Overview 280
Sharing and Saving 285
Custom Data Views 286
Summary 286
References 287
18 Continuous Integration with Jenkins 289
Continuous Integration Concepts 289
Continuous Integration or Continuous Deployment? 290
Test Automation 290
Jenkins Architecture 292
Jenkins Deployment 293
Jenkins Workfl ow 296
Jenkins Server Confi guration 296
Jenkins Build Job 298
Git Hooks 302
Your First Build 304
Quality Assurance Teams? 306
Acceptance Testing 306
Development Team 306
Build/Test Infrastructure 307
Summary 307
References 307
Part 8 VMware DevOps Practices
19 VMware vRealize Automation in DevOps Environments 309
Emergence of DevOps 309
Stable Agility 310
People, Process, and Conway's Law 311
vRealize Automation 312
vRealize Application Services 313
Puppet Integration 315
Code Stream 321
Summary 327
References 327
9780133846478, TOC, 3/9/15
About the Reviewers xviii
Acknowledgments xix
About the Contributing Author xx
Introduction xxii
About This Book xxiii
You the Reader xxiii
What This Book Covers xxiii
Part 1 Introduction to DevOps
1 Introduction to DevOps 1
An Overview of DevOps Principles 1
Implement Systems Thinking 3
Change the Approach to Team Interactions 3
Change the Approach to Infrastructure Deployment 5
Change the Approach to Software Development and Deployment 6
Collect and Respond to Useful Systems Feedback Often and Adjust Accordingly 7
Furthering Your DevOps Knowledge and Skills 7
Summary 8
References 8
2 DevOps Tools 9
Organizing for Success: Kanban 9
Server Deployment 13
Confi guration Management 14
Continuous Integration 14
Log Analysis 15
Summary 15
References 15
3 Setting Up a DevOps Confi guration Management Test Environment 17
Environment Provisioning with AutoLab 17
Environment Provisioning with Vagrant 18
Creating Images with Packer 23
Managing Source Code 24
Using Git 24
Summary 31
References 31
Part 2 Puppet
4 Introduction to Puppet 33
Puppet Architecture 33
Standalone Deployment 34
Master-Agent Deployment 34
Preparing Your Puppet Test Lab 37
Puppet Resources 38
Puppet Manifests 39
Conditional Expressions and Variables 43
Puppet Modules 46
Puppet Forge 48
Creating Your First Puppet Module 48
Puppet Module Initialization Manifest (init.pp) 50
Templates 51
Using a Puppet Module 54
Final Step: Version Control Commit 54
Summary 55
Reference 55
5 Systems Management Tasks with Puppet 57
Optimizing the Web Tier with Data Separation 58
Parameters Class (params.pp) 59
Hiera 63
Node Classifi cation 67
Application Tier 68
Database Tier 70
Implementing a Production Recommended Practice 70
Deploying the Application Environment 71
Summary 71
Reference 71
6 VMware vSphere Management with Puppet 73
Puppet's Cloud Provisioner for VMware vSphere 73
Preparing VM Templates 73
Preparing Puppet Master 74
9780133846478_ROBERTS.indb x 3/5/15 4:20 PM
Contents xi
VMware's Management Modules 77
Using the vmware/vcenter Module 77
Summary 83
References 83
Part 3 Chef
7 Introduction to Chef 85
What Is Chef? 85
Core Philosophies Behind Chef 86
Order of Recipe 86
Idempotence 86
API-Based Server 87
The Client Does All the Legwork 87
Test-Driven Infrastructure 87
Chef Terminology 87
Recipe 88
Cookbook 88
Attributes 88
Role 88
Run List 88
Resource 88
Environments 88
The Diff erence Between Hosted Chef and Chef Server 89
Hosted Chef 89
Chef Server 89
Introduction to ChefDK 90
What Is ChefDK? 90
Installing ChefDK 90
Using Knife 92
Creating Your First Hello World Chef Recipe 94
Summary 98
8 Systems Management Tasks with Chef 99
Signing Up for Hosted Chef 100
Setting Up Local Repo with the Starter Kit 102
Community Cookbooks 105
Setting Up System Management 105
Prep/Setup System Management Task 1: Managing Time 105
Prep/Setup System Management Task 2: Managing Root Password 108
9780133846478_ROBERTS.indb xi 3/5/15 4:20 PM
xii Contents
Confi guring Your Virtual Guests 109
Installing Chef Client 109
Systems Management Tasks 111
Running Chef Client 113
Managing the Root Password 115
Creating Two Environment Files 116
Uploading Environment Files to your Hosted Chef Organization 117
Assigning Each Server to an Environment 118
Modifying Each Server's Run List to Run the Managedroot Cookbook 119
Applying Your Changes to Your Nodes 120
Validating the Enforced Policy 120
Summary 122
References 123
9 VMware vSphere Management with Chef 125
Knife Plugins 126
Getting Started with knife-vsphere 128
Confi guring the knife.rb File 128
Validating the Confi guration 130
Putting It All Together 130
Chef Provisioning 134
Chef Provisioning Architecture 134
Getting Started with Chef Provisioning 135
Spinning Up Some Nodes 136
Summary 138
Part 4 Ansible
10 Introduction to Ansible 139
Ansible Architecture 139
Preparing your Ansible Test Lab 141
Ansible Groups 142
Ansible Ad Hoc Command Execution 142
The Ping Module 143
The Command Module 144
The User Module 144
The Setup Module 144
Ansible Playbooks 144
Conditional Expressions and Variables 146
Ansible Roles 151
Templates 154
Ansible Galaxy 156
Summary 157
References 157
11 Systems Management Tasks with Ansible 159
Web Server Deployment 159
The Application Tier 160
The Database Tier 162
Role Structure Optimization 164
VMware Resource Management 166
Summary 171
References 171
Part 5 PowerShell 4.0
12 Introduction to PowerShell Desired State Confi guration (DSC) 173
What Is PowerShell DSC? 174
PowerShell DSC Requirements 175
PowerShell DSC Components 175
Native Cmdlets 175
Managed Object Format File 176
Local Confi guration Manager 176
PowerShell DSC Confi gurations 178
PowerShell DSC Modes 180
Local Push Mode 181
Remote Push Mode 181
Pull Mode 182
PowerShell DSC Resources 184
Summary 186
References 187
13 Implementation Strategies with PowerShell DSC 189
Use Cases for PowerShell DSC in VMware Environments 189
Scripted Deployments of VMs with PowerCLI 190
Incorporating PowerShell DSC in VM Templates 192
Challenges Implementing PowerShell DSC Confi gurations to New VMs 193
PowerCLI Invoke-VMscript 193
PowerCLI Copy-VMGuestFile 195
General Lessons Learned 196
Future Use Cases for PowerShell DSC in VMware Environments 197
Summary 198
References 198
Part 6 Application Deployment with Containers
14 Introduction to Application Containers with Docker 199
What Is an Application? 199
Hidden Complexity 200
Dependency and Confi guration Confl icts 200
Linux Containers 200
Control Groups 201
Namespaces 201
Container Management 203
Using Docker 203
Installing Docker 203
Docker Daemon 204
Docker Client 204
Docker Index 205
Running a Docker Container 205
Listing Running Containers 206
Connecting to Running Containers 206
Building and Distributing Docker Containers 208
Dockerfi le 209
Docker Hub 210
Docker Versus Virtual Machines 211
Docker Versus Confi guration Management 211
Summary 212
References 212
15 Running Docker Containers at Scale 213
Container Orchestration 213
Kubernetes 214
Kubernetes Workfl ow 214
Kubernetes Deployment 215
CoreOS and Kubernetes Cluster Management Utilities 216
CoreOS Cluster Deployment 217
etcd Server Confi guration 222
Network Overlays with Flannel 223
Kubernetes Cluster Nodes 223
Kubernetes Service Deployment 225
Kubernetes Workload Deployment 226
Platform-as-a-Service with Docker 230
Summary 231
References 231
Part 7 DevOps Tool Chain
16 Server Provisioning Using Razor 233
How Razor Works 233
Using Razor 236
Razor Collections and Actions 238
Building Razor Collections 245
Using Razor APIs 257
Razor Components 258
Razor Server 258
Razor Microkernel 258
Razor Client 259
Setting Up Razor 259
PE Razor 259
Puppet Install 259
Install from Source 260
Manual Release Install 260
Other Services 260
Summary 263
References 263
17 Intro to the ELK: Elasticsearch, Logstash, Kibana 265
Elasticsearch Overview 265
Getting Started 266
Understanding the Index 267
Working with Data 267
Installing Plugins 271
Using Clients 274
Logstash Overview 275
Getting Started 276
Confi guring Input to Logstash 276
Applying Filters 278
Kibana Overview 280
Sharing and Saving 285
Custom Data Views 286
Summary 286
References 287
18 Continuous Integration with Jenkins 289
Continuous Integration Concepts 289
Continuous Integration or Continuous Deployment? 290
Test Automation 290
Jenkins Architecture 292
Jenkins Deployment 293
Jenkins Workfl ow 296
Jenkins Server Confi guration 296
Jenkins Build Job 298
Git Hooks 302
Your First Build 304
Quality Assurance Teams? 306
Acceptance Testing 306
Development Team 306
Build/Test Infrastructure 307
Summary 307
References 307
Part 8 VMware DevOps Practices
19 VMware vRealize Automation in DevOps Environments 309
Emergence of DevOps 309
Stable Agility 310
People, Process, and Conway's Law 311
vRealize Automation 312
vRealize Application Services 313
Puppet Integration 315
Code Stream 321
Summary 327
References 327
9780133846478, TOC, 3/9/15