Using TRILL, FabricPath, and VXLAN
Designing Massively Scalable Data Centers (MSDC) with Overlays
Published on 20. February 2014
Book
Paperback/Softback
368 pages
978-1-58714-393-9 (ISBN)
Description
Using TRILL, FabricPath, and VXLAN
Designing Massively Scalable Data Centers with Overlays
TRILL, FabricPath, and VXLAN overlays help you distribute data traffic far more effectively, dramatically improving utilization in even the largest data center networks. Using TRILL, FabricPath, and VXLAN is the first practical and comprehensive guide to planning and establishing these high-efficiency overlay networks. The authors begin by reviewing today's fast-growing data center requirements, and making a strong case for overlays in the Massive Scale Data Center (MSDC). Next, they introduce each leading technology option, including FabricPath, TRILL, LISP, VXLAN, NVGRE, OTV, and Shortest Path Bridging (SPB). They also present a chapter-length introduction to IS-IS, focusing on details relevant to the control of FabricPath and TRILL networks. Building on this foundation, they offer in-depth coverage of FabricPath: its advantages, architecture, forwarding, configuration, verification, and benefits in Layer-2 networks. Through examples, they explain TRILL's architecture, functionality, and forwarding behavior, focusing especially on data flow. They also fully address VXLAN as a solution for realizing IP-based data center fabrics, including multi-tenant cloud applications.
Using TRILL, FabricPath, and VXLAN provides detailed strategies and methodologies for FabricPath, TRILL, and VXLAN deployment and migration, as well as best practices for management and troubleshooting. It also presents three detailed implementation scenarios, each reflecting realistic data center challenges. In particular, the authors show how to integrate multiple overlay technologies into a single end-to-end solution that offers exceptional flexibility, agility, and availability.
Sanjay K. Hooda is principal engineer in Catalyst switching software engineering at Cisco. He has more than 15 years of network design and implementation experience in large enterprise environments, and has participated in IETF standards activities. His interests include wireless, multicast, TRILL, FabricPath, High Availability, ISSU, and IPv6. He is co-author of IPv6 for Enterprise Networks.
Shyam Kapadia, Technical Leader at Cisco's Data Center Group (DCG), was an integral part of the team that delivered the next-generation Catalyst 6500 Sup 2T (2 Terabyte) platform. Since then, he has focused on developing new solutions for data center environments. He holds a Ph.D. in computer science from USC, where his research encompassed wired, wireless, ad hoc, vehicular, and sensor networks.
Padmanabhan Krishnan has more than 12 years of experience in networking and telecommunications, including 7 at Cisco. His recent experience has included providing data path solutions for TRILL in the Catalyst 6500 Sup 2T Platform using FPGA, as well as design and development of platform core infrastructure and L2 features.
n Discover how overlays can address data center network problems ranging from scalability to rapid provisioning
n Examine popular data center overlay examples
n Learn about extensions to IS-IS for TRILL and FabricPath
n Use FabricPath, TRILL, and VXLAN to simplify configuration, improve performance and availability, optimize efficiency, and limit table size
n Learn about FabricPath control and data plane architecture details
n Review example FabricPath configurations on Cisco Nexus 7000/6000/5000 switches
n Understand TRILL concepts and architecture, including overlay header, control and data plane, and MAC address learning
n Learn about VXLAN architecture details and packet forwarding
n Review example VXLAN configurations on a Cisco Nexus 1000V distributed virtual switch
n Implement TRILL/FabricPath networks with VXLAN to virtualized servers in an intra-data center environment
n Connect multiple traditional data centers using an OTV overlay as a Layer 2 extension
n Use OTV overlays to connect sites running FabricPath, TRILL, or both
Designing Massively Scalable Data Centers with Overlays
TRILL, FabricPath, and VXLAN overlays help you distribute data traffic far more effectively, dramatically improving utilization in even the largest data center networks. Using TRILL, FabricPath, and VXLAN is the first practical and comprehensive guide to planning and establishing these high-efficiency overlay networks. The authors begin by reviewing today's fast-growing data center requirements, and making a strong case for overlays in the Massive Scale Data Center (MSDC). Next, they introduce each leading technology option, including FabricPath, TRILL, LISP, VXLAN, NVGRE, OTV, and Shortest Path Bridging (SPB). They also present a chapter-length introduction to IS-IS, focusing on details relevant to the control of FabricPath and TRILL networks. Building on this foundation, they offer in-depth coverage of FabricPath: its advantages, architecture, forwarding, configuration, verification, and benefits in Layer-2 networks. Through examples, they explain TRILL's architecture, functionality, and forwarding behavior, focusing especially on data flow. They also fully address VXLAN as a solution for realizing IP-based data center fabrics, including multi-tenant cloud applications.
Using TRILL, FabricPath, and VXLAN provides detailed strategies and methodologies for FabricPath, TRILL, and VXLAN deployment and migration, as well as best practices for management and troubleshooting. It also presents three detailed implementation scenarios, each reflecting realistic data center challenges. In particular, the authors show how to integrate multiple overlay technologies into a single end-to-end solution that offers exceptional flexibility, agility, and availability.
Sanjay K. Hooda is principal engineer in Catalyst switching software engineering at Cisco. He has more than 15 years of network design and implementation experience in large enterprise environments, and has participated in IETF standards activities. His interests include wireless, multicast, TRILL, FabricPath, High Availability, ISSU, and IPv6. He is co-author of IPv6 for Enterprise Networks.
Shyam Kapadia, Technical Leader at Cisco's Data Center Group (DCG), was an integral part of the team that delivered the next-generation Catalyst 6500 Sup 2T (2 Terabyte) platform. Since then, he has focused on developing new solutions for data center environments. He holds a Ph.D. in computer science from USC, where his research encompassed wired, wireless, ad hoc, vehicular, and sensor networks.
Padmanabhan Krishnan has more than 12 years of experience in networking and telecommunications, including 7 at Cisco. His recent experience has included providing data path solutions for TRILL in the Catalyst 6500 Sup 2T Platform using FPGA, as well as design and development of platform core infrastructure and L2 features.
n Discover how overlays can address data center network problems ranging from scalability to rapid provisioning
n Examine popular data center overlay examples
n Learn about extensions to IS-IS for TRILL and FabricPath
n Use FabricPath, TRILL, and VXLAN to simplify configuration, improve performance and availability, optimize efficiency, and limit table size
n Learn about FabricPath control and data plane architecture details
n Review example FabricPath configurations on Cisco Nexus 7000/6000/5000 switches
n Understand TRILL concepts and architecture, including overlay header, control and data plane, and MAC address learning
n Learn about VXLAN architecture details and packet forwarding
n Review example VXLAN configurations on a Cisco Nexus 1000V distributed virtual switch
n Implement TRILL/FabricPath networks with VXLAN to virtualized servers in an intra-data center environment
n Connect multiple traditional data centers using an OTV overlay as a Layer 2 extension
n Use OTV overlays to connect sites running FabricPath, TRILL, or both
More details
Persons
Sanjay Hooda, CCIE No. 11737, is currently a principal engineer at Cisco, where he works with embedded systems and helps define new product architectures. His current passion is to design the next-generation campus architecture, and he is focused on simplifying
the design and deployment of wired and wireless infrastructure. Over the last 17 years, Sanjay's experience spans various areas including high availability; messaging in large-scale distributed systems; Supervisory Control and Data Acquisition (SCADA); large-scale software projects; and enterprise campus and LAN,WAN, and data center network design.
Shyam Kapadia, Ph.D., is currently a technical leader in the Data Center Group at Cisco. He graduated from the University of Southern California with Ph.D. and master's degrees in computer science in 2006. His research interests broadly lie in the area of networking systems including wired, wireless, ad-hoc, vehicular, and sensor networks. He has co-authored several conference and journal publications in these areas including a book chapter in the relatively nascent area of intermittently connected wireless networks (http://anrg.usc.edu/~kapadia/publications.html). At Cisco, for the first few years, he was an integral part of the team that delivered the next-generation Catalyst 6500 Sup 2T platform. During the past few years, he has been intrinsically involved in developing solutions for data center environments with more than 25 submitted patents in this area. Over the past 12 years, Shyam has been the speakers chair for a premiere Open Source conference, Southern California Linux Exposition (SCALE), hosted in the Los Angeles area. In his spare time, he loves watching international movies and is passionate about sports like cricket, basketball, and American football.
Padmanabhan Krishnan is a software engineer in the Data Center Group at Cisco. He joined Cisco 7 years ago and has more than 12 years of experience in various areas of networking and telecommunication. He obtained his master's degree in computer science from the University of Missouri, Kansas City, and his bachelor's degree in engineering from Madras University, India. His research work for the master's degree included Diffserv, MPLS traffic engineering, and QOS routing/Connection Admission Control in ad-hoc wireless networks. Padmanabhan has worked in many overlay technologies in Cisco such as 802.1ah, TRILL, FabricPath, and VPLS. He was responsible for the design and development of the core infrastructure used by the forwarding drivers and many Layer 2 features in the next generation Catalyst 6500 Sup 2T Platform. Prior to joining Cisco, Padmanabhan worked in ATM signaling and DVB-RCS, an interactive on-demand multimedia satellite communication system specification.
the design and deployment of wired and wireless infrastructure. Over the last 17 years, Sanjay's experience spans various areas including high availability; messaging in large-scale distributed systems; Supervisory Control and Data Acquisition (SCADA); large-scale software projects; and enterprise campus and LAN,WAN, and data center network design.
Shyam Kapadia, Ph.D., is currently a technical leader in the Data Center Group at Cisco. He graduated from the University of Southern California with Ph.D. and master's degrees in computer science in 2006. His research interests broadly lie in the area of networking systems including wired, wireless, ad-hoc, vehicular, and sensor networks. He has co-authored several conference and journal publications in these areas including a book chapter in the relatively nascent area of intermittently connected wireless networks (http://anrg.usc.edu/~kapadia/publications.html). At Cisco, for the first few years, he was an integral part of the team that delivered the next-generation Catalyst 6500 Sup 2T platform. During the past few years, he has been intrinsically involved in developing solutions for data center environments with more than 25 submitted patents in this area. Over the past 12 years, Shyam has been the speakers chair for a premiere Open Source conference, Southern California Linux Exposition (SCALE), hosted in the Los Angeles area. In his spare time, he loves watching international movies and is passionate about sports like cricket, basketball, and American football.
Padmanabhan Krishnan is a software engineer in the Data Center Group at Cisco. He joined Cisco 7 years ago and has more than 12 years of experience in various areas of networking and telecommunication. He obtained his master's degree in computer science from the University of Missouri, Kansas City, and his bachelor's degree in engineering from Madras University, India. His research work for the master's degree included Diffserv, MPLS traffic engineering, and QOS routing/Connection Admission Control in ad-hoc wireless networks. Padmanabhan has worked in many overlay technologies in Cisco such as 802.1ah, TRILL, FabricPath, and VPLS. He was responsible for the design and development of the core infrastructure used by the forwarding drivers and many Layer 2 features in the next generation Catalyst 6500 Sup 2T Platform. Prior to joining Cisco, Padmanabhan worked in ATM signaling and DVB-RCS, an interactive on-demand multimedia satellite communication system specification.
Content
Chapter 1 Need for Overlays in Massive Scale Data Centers 1
Evolution of the Data Center 1
Changing Requirements of Data Centers 4
Data Center Architectures 6
CLOS 8
Fat-Tree 9
Single Fabric 9
Need for Overlays 10
Summary 15
References 15
Chapter 2 Introduction to Overlay Technologies 19
Overlay Technologies Overview 20
Cisco FabricPath 22
FabricPath Requirements 22
FabricPath Benefits 23
FabricPath Architecture 24
FabricPath Encapsulation 24
FabricPath Data Plane Operation 25
TRILL 26
TRILL Requirements 27
TRILL Frame Format 28
TRILL Data Plane Operation 28
Locator ID/Separator Protocol 30
LISP Frame Format 30
LISP Routing 30
VXLAN 32
VXLAN Frame Format 33
VXLAN Data Path Operation 34
NVGRE 35
NVGRE Frame Format 36
NVGRE Data Path Operation 36
Overlay Transport Virtualization 38
OTV Frame Format 39
OTV Operation 40
Provider Backbone Bridges (PBB) 41
Shortest Path Bridging 43
Shortest Path Bridging MAC 43
Shortest Path Bridging VID 45
Summary 47
References 47
Chapter 3 IS-IS 49
Introduction to IS-IS 49
Concepts 50
Neighbor Discovery 51
Topology Exchange 51
Flooding 51
Route Computation 52
Link State Protocol Scaling 52
Link State Protocol in a Local Area Network 53
IS-IS Architecture Details 55
TRILL and FabricPath Specific Changes in IS-IS 56
Overview of TRILL and FabricPath 57
IS-IS Frame Formats 58
Router Capability TLV 59
Multitopology-Aware Port Capability TLV 59
TRILL IS-IS Neighbor Discovery 59
TRILL HELLOs 60
P2P HELLOs 63
TRILL Neighbor TLV 64
Router Capability Sub-TLVs 64
Multitopology-Aware Port Capability Sub-TLVs 64
Area Address TLV 67
Protocols Supported TLV 67
TRILL and FabricPath Topology Exchange 67
Flooding 69
Nickname or SwitchID Resolution 70
Shortest Path Computation 71
Distribution Trees Computation 71
Pruning the Distribution Tree 74
ESADI 77
MAC Reachability TLV 78
Fine Grained Labeling 79
Pseudo Node 81
Multi Topology Routing 83
Summary 84
References 84
Additional Resources 84
Chapter 4 FabricPath 85
FabricPath Overview 86
FabricPath Architecture 87
Core and Edge 88
Addressing Concepts 89
VLANs 89
vPC+ 89
FabricPath Encapsulation 91
FabricPath Control Plane Protocols 93
IGMP Snooping in FabricPath Multicast Networks 96
FabricPath Dynamic Resource Allocation Protocol 97
Allocation of Resources by DRAP 97
FabricPath MAC Address Learning 98
Control Plane Learning 98
Data Plane Learning 98
FabricPath STP Interaction 102
Topology Change Notifications Forwarding 105
FabricPath Packet Forwarding 106
Broadcast: ARP Request 108
Unicast: ARP Reply 111
Unicast: Data 113
IP Multicast Forwarding 116
FabricPath Basic Configuration 119
FabricPath Benefits 121
Summary 122
References 122
Chapter 5 TRILL 123
Need for TRILL 124
Spanning Tree in Layer 2 Networks 124
Issues with Spanning Tree Protocol 126
Virtual Switching System 127
Giant Virtual Switch 128
Flat Network 129
Layer 3 Network 130
Concepts and Terminologies 130
RBridge 131
Appointed Forwarder 132
Addressing Concepts 133
TRILL Frame Format 134
TRILL Control Plane 136
Unicast 136
Multicast 137
Pruning 139
TRILL Data Plane 141
Unicast 141
Ingress RBridge Processing 141
Processing of TRILL Packets 143
Multidestination 143
Ingress Processing 144
Core and Egress Processing 146
Egress Processing 146
MAC Address Learning in TRILL-Based Networks 147
Dynamic Learning 147
Learning Through Control Protocols 147
Work in Progress 148
Multitopology Routing 148
Fine-Grained Labeling 149
Ingress RBridge 152
Core RBridge 152
Egress RBridge 152
Pseudo Node 152
Choosing a Pseudo Nickname 154
Multiple Distribution Trees per Member RBridge 156
Synchronization of MAC Addresses 158
Case Studies 159
Bidirectional Packet Flow 159
Traffic from H1 to H2 160
Traffic from H2 to H1 164
Packet Flow for Pseudo Node 167
Packet Originating from Host H1 170
Reverse Traffic from Host H2 to H1 172
Summary 174
References 174
Additional Resources 175
Chapter 6 VXLAN 177
VXLAN Overview 177
Advent of VXLAN 178
VXLAN Architecture 179
VXLAN Header Format 181
VXLAN Packet Forwarding 182
Broadcast: ARP Request 183
Unicast: ARP Reply 184
Unicast: Data 186
Unknown Unicast 187
VM Mobility Case 188
IPv6 Data Forwarding 190
NS Request and NA Response 191
VXLAN Gateway 192
Inter-VXLAN Communication 196
Layer 3 Multicast 198
Multicast-Less VXLAN 200
Floodless VXLAN Forwarding 203
VXLAN as a Network Overlay 205
Other VXLAN Considerations 207
VXLAN Basic Configuration 208
VXLAN Gateway Configuration 210
Summary 211
References 211
Chapter 7 FabricPath Deployment, Migration, and Troubleshooting 213
vPC 214
vPC Overview 214
vPC Terminology 215
vPC Benefits 216
vPC Deployment Scenarios 217
Double-Sided vPC 218
vPC Operations 219
vPC Traffic Flow 224
Cisco Fabric Services over Ethernet 225
vPC ARP Sync 225
vPC Peer Gateway 225
vPC Verification 227
vPC+ 231
vPC+ Overview 231
vPC+ Basics 232
vPC+ Basic Packet Flow 236
Active/Active HSRP Forwarding 238
FabricPath Topologies 241
Migration to FabricPath Network 242
Conversion from Classical Layer 2 to FabricPath Network 242
Conversion of vPC to vPC+ (Classical Ethernet to FabricPath) 244
Configuring vPC+ on Secondary Switch 246
Configuring vPC+ on Primary Switch 249
Conversion of Access Switch (Sw3) Connecting to Secondary (Sw2) to FabricPath 251
Converting Access Switch Sw3 Uplink Connecting to Sw1 to FabricPath 254
Monitoring and Troubleshooting in FabricPath Networks 257
Loopback Message 258
Path Trace Message 259
Multicast Trace Message 259
FabricPath OAM Configuration Model 261
Summary 270
References 270
Chapter 8 TRILL Deployment, Migration, and Troubleshooting 271
Introduction 271
TRILL Deployment 271
TRILL Between Access and Distribution 274
TRILL Core 274
Layer 2 Bridging Case 276
Layer 3 Routing Cases 277
Expanding the POD 285
TRILL Everywhere 286
Meshed Distribution with No Core 287
Link Aggregation or Pseudo-Node Deployments 287
CLOS Network Model 289
Migration Toward TRILL 290
TRILL and Spanning Tree 291
Appointed Forwarder Solution 292
Spanning Tree Solution 293
Bottom-up Migration Toward TRILL 296
Top-down Migration Toward TRILL 298
Monitoring and Troubleshooting in TRILL Networks 299
OAM Packet Format 300
Connectivity Verification 302
Path Tracing 303
TRILL Configuration Model 304
Summary 304
References 305
Chapter 9 Multi-Overlay Deployments 307
Overview 307
Case Study 1: TRILL or FabricPath Network with VXLAN to Virtualized Servers 309
Case Study 2: Data Center Interconnect Using OTV 315
Case Study 3: Interconnecting TRILL or FabricPath Data Centers Using OTV 321
Merging TRILL or FabricPath Networks 321
Independent TRILL or FabricPath Networks 323
Interconnection of TRILL and FabricPath Data Centers 325
Packet Flow 325
Summary 327
References 328
Index 329