10G Ethernet: More Than a Big Pipeby Johan De Gelas on November 24, 2010 2:34 PM EST
Cleaning Up the Cable Mess
Between 6 and 12 I/O cables on one server is quite a bit. You end up with a complicated NIC configuration as illustrated in this VMware/Intel white paper.
The white paper does not consider the storage I/O, but if you use two FC cards, you are adding another 14W (7W x 2) and two cables. So if we take such a heavy consolidated server as an example you end up with about:
- 10 I/O Cables (without KVM and Server management)
- 2 quad-port NICs x 5W + 2 FC cards x 7W = 24W
24W is not enormous, but in reality this is the best case. Dual socket servers typically need between 200 and 350W, quad socket servers about 250 to 500W in total. So the I/O power consumption is about 5—15% of the total power consumption.
The 10 I/O cables are a bigger problem. The more ports and cables, the higher the chances are that something gets badly configured and the harder it gets to troubleshoot. It does not take much imagination to see that this kind of cabling might waste a lot of sysadmin time and thus money.
The biggest problem is of course cost. Fibre Channel cabling is not cheap, but it is nothing compared to the huge cost of FC HBAs, FC switches, and SFPs. Cabling up to eight Ethernet cables per server is not cheap either: while the cable cost is negligible, someone has to do the pulling and plugging and that task isn't free.