Now that spring is here, I already find myself longing for warm, summer weather. I can’t help but think of the days I spent on the lakes water skiing. As I remember those days something keeps popping in my head—what if the tow rope manufacturers told me they could only provide me a seven-foot rope to pull me with. Would they expect the boat manufacturers to redesign their boats around the short ropes? Think about it—where would they put the motors?
It really is a silly thing to think about. Who would expect boat designers to completely rethink and redesign a boat based on what the rope makers want to do? Why wouldn’t they just demand a longer rope?
Well, there is a parallel in data center architecture design. As virtualization becomes the norm in data centers, 10G access
port speeds have become more prevalent. The hardware manufacturers have responded by producing 10G switches; however, the cabling that is being used to support the 10G ports has been direct attached cables. These direct attached cables are Twinax cables with a maximum distance of seven meters for the 10G connection. The result is that the access switch has to be placed within seven meters of the server. This has led to the proliferation of a Top-of-Rack (ToR) architecture in the data center.
There a couple of things concerning ToR, the first being port utilization and the additional cost to provide access ports in each server cabinet. Then add to that the cost of providing redundancy to every server. Traditionally, server access is supported by End-of-Row (EoR) or Middle-of-Row (MoR) switches. These switches are chassis based to provide redundant power supplies and management modules. Also, by being able to cable to the EoR or MoR, all servers can get access from a common place allowing for better switch port utilization. Secondly, the cost of the direct attached cable is extremely high compared to other options.
The big question—what is my other option? Most switch manufacturers have introduced 10GBase-T switches as part of
their offering because they allow for the use of standard unshielded twisted pair (UTP) cable and patch cords. Category 6 or Category 6a cabling can be used depending on the distance from access to server. By using a UTP cabling system, data center architects are given complete design freedom for access connections which allows them to design the best network to support their application needs not their cabling needs. For the most flexibility, Category 6a cable can be used to support 10G for up to 100 meters.
There are three points that resonate with this topic. First, if ToR was really the best way to design a network, then why do a majority of 1G-based networks utilize EoR and MoR? Secondly, perhaps one of the most basic ideas is that of backwards compatibility allowing the cable to be placed to support a smooth migration from 1G to 10G. The third thought that will allow us to move away from ToR is that 10GBase-T now fully supports Fiber Channel over Ethernet. For those looking to take advantage of the high speeds offered by Ethernet for their traditional Fiber Channel SANs, you can now have complete freedom in designing your data center.
At the end of the day if we can figure out how to extend the access links in complex data centers, we certainly should not even have to figure out how to make a tow rope more than seven feet long. That’s good news for boat manufacturers…
