In this month’s column, I discussed the use of Ethernet fabrics technology for providing connectivity between zEnterprise and the IBM DB2 Analytics Accelerator for z/OS. This article will take a more detailed look at Ethernet fabrics.
Data centers continue to grow as digital assets increase and more applications are deployed. Businesses expect agile application deployment—in minutes, not months—to keep their competitive edge as markets and competitors become global in scale. And data center resources such as rack space, power and cooling are growing more scarce and costly. When deployed together effectively, high-density, multicore servers, network, server and storage virtualization, and IT orchestration tools can be used to pool IT resources and implement cloud architectures. Moving toward cloud computing can reduce capital and operational expenditures by driving the consolidation of applications and improving resource utilization, and at the same time create an infrastructure that rapidly scales and responds to business needs.
Data center networks rely on Ethernet. Over the decades, Ethernet has evolved as new types of application architectures emerged. Today, data center networks carry traffic for a diverse set of applications, including client/ server, Web services, unified communications, virtual machines and storage—each with different traffic patterns and network service requirements. Applications are increasingly deployed within virtual machines hosted on server clusters. And Ethernet can be used to build shared storage pools, which place stringent demands on the network, including lossless packet delivery, deterministic latency and high bandwidth.
An Ethernet fabric network is a type of Ethernet that is aware of all its paths, nodes, requirements and resources. Ethernet fabrics are able to automatically manage themselves to scale up or down, depending on demand. They also eradicate the need for the challenging and comparatively less-efficient Spanning Tree Protocol (STP), and the redundancies it creates. Compared to classic hierarchical Ethernet architectures, Ethernet fabrics provide the higher levels of performance, utilization, availability and simplicity required to meet the business needs of data centers today and into the future. Ethernet fabric systems can also be incorporated with pre-existing networks.
This article reviews classic Ethernet architecture in light of new data center requirements, provides an overview of the differences between traditional Ethernet and Ethernet fabrics, and explains how Ethernet fabrics can be used to address emerging data center challenges.
The Classic Three-Tier Ethernet Network Architecture
In order to better understand the Ethernet fabric, we first need to review a classic Ethernet network. The dominant architecture today—and one that has been around more than a decade—is the conventional three-tier, or hierarchical, data center network architecture. This architecture includes the familiar LAN access, aggregation and core tiers. It dates back to the era when clients consumed applications running on dedicated physical servers and network traffic typically flowed from the client, through the data center network tiers, to the application, and back out. This traffic pattern is typically referred to as north-south. This environment tolerates oversubscription in the switching components because, on average, each server connection utilizes a relatively small portion of network bandwidth. To help ensure application availability, network resiliency is delivered through redundant switching components and network connections. Most data centers need more ports than are available in a single Ethernet switch so multiple switches are connected to form a network with increased connectivity. All these Ethernet switches are connected, forming a hierarchical, or “Ethernet tree” topology, as shown in Figure 1.
Limitations of Classic Ethernet Architecture
In a classic Ethernet network, the connections between switches, or Inter-Switch Links (ISLs, shown as solid blue lines in Figure 1) are not allowed to form a loop or frames aren’t delivered. STP prevents loops by creating a tree topology with only one active path between any two switches. (In Figure 1, inactive paths are shown as dotted lines.) This means that ISL bandwidth is limited to a single logical connection, since multiple connections between switches are prohibited. Enhancements to Ethernet tried to overcome this limitation. Link Aggregation Groups (LAGs) were defined so that multiple links between switches were treated as a single connection without forming loops. But a LAG must be manually configured on each port in the LAG and is not very flexible.