How Switched Fabric Became the Dominant Data Center Topology

Data center architecture dates back as early as the 1940s, where banks of huge computers were interconnected for distributed computing. Fast forward to today, and the topology used to connect equipment in a data center has advanced right alongside computer systems and networking systems. Today’s computing era, where workloads are increasingly deployed in the cloud or private data centers, is characterized by extreme efficiency in computing and data transfer.

To enable data transfer between assets in a data center environment, the networking topology is just as important as the equipment being connected. Today’s dominant topology is switched fabric, which provides a dynamic and scalable interconnect solution that has enabled some of the most advanced applications. But why has this networking topology risen as the dominant architecture in the modern data center? An understanding of these concepts can inform the topology systems engineers might implement in their connected products.

Key Drivers for the Dominance of Switched Fabric

Switched fabric has been a major enabler of higher throughput connections between assets in a data center compared to other network topologies. In other topologies (bus, ring, etc.), there is either one point of failure, long delays in data transmission, and/or overall lower throughput. In a broadcast network topology, data was transmitted to all clients on the network regardless of whether they were the intended recipients.

Switched fabric takes advantage of point-to-point links to provide the higher throughput required in modern networking. Interconnects shown in the image below can be of either fiber or copper media as both are standard in data center environments.

There are two factors leading to higher throughput between nodes in the above topology:

• Chipsets and physical transmission media that enable higher bandwidth channels
• Direct transmission of data along links between equipment

Today’s advanced packages and chipsets are enabling higher throughput through faster data rates across interconnects by implementing faster edge rates, clocking rates, and multi-level modulation. Today, PAM-4 has spread from fast serdes channels into multiple computing interfaces, most notably in PCIe Gen 6.0, which provides a doubling of data transfer rate without requiring a doubling of channel bandwidth. As passive copper and active get closer to their physical data transfer limits, bandwidth scaling on-chip, on-board, and between computing assets can continue through the use of modulation.

Is high throughput the main factor driving switched fabric and the networking architecture to support it? If we dig below the surface, we see several reasons why switched fabric has become dominant in data centers.

Scalability and efficiency: Switched fabric is inherently scalable, to the point that expansion of a network is a simple matter of adding additional switch nodes without sacrificing throughput. Switched fabric allows distribution of network traffic across select physical links rather than broadcasting across a single shared link.

Active copper and optical interconnects: Transceivers are available that support both fiber and copper for routing data between switches, servers, etc. Applications like HD video streaming, distributed data analytics, and AI models require these higher bandwidth interconnects.

Technological advancements: Switch designs, routing algorithms, and innovations in networking protocols have made it easier to implement and manage switched fabric topologies.

Reliability: Switched fabric topologies do not have single points of failure that can bring down an entire network. While a single node going offline may isolate some nodes, it does not prevent data transfer across the network due to multiple paths being available within the topology.

Point-to-Point Beyond the Data Center

In today’s telecom infrastructure and architecture, the dominant point-to-point connection method exists outside the data center, and probably in your own home. Modern wireless networks use a switched-fabric topology for data transmission between base stations and user equipment, where the base stations effectively act like switches with a backhaul connection. Thanks to the scalability of switched fabric networks, expect to see these types of systems proliferate throughout chip design, data center systems design, and larger area networks.

Any system that aims to implement switched fabric topologies must be qualified to operate at the highest current data rates and beyond. Make sure you qualify your most advanced designs using the complete set of system analysis tools from Cadence. Only Cadence offers a comprehensive set of circuit, IC, and PCB design tools for any application and any level of complexity. Cadence PCB design products also integrate with a multiphysics field solver for thermal analysis, including verification of thermally sensitive chip and package designs.

Subscribe to our newsletter for the latest updates. If you’re looking to learn more about how Cadence has the solution for you, talk to our team of experts.