More Cable Management

Date June 16, 2009

or "I typed a lot on serverfault, I wonder if I can get a blog entry out of it"

Cable management is one of those things that you might be able to read about, but you will never really get the hang of it until you go out and do it. And it takes practice. Good cable management takes a lot of planning, too. You don't get great results if you just throw together a bunch of cable on a rack and call it a day. You've got to plan your runs, order (or create) the right kind of cables and cable management hardware that you need, and it's got to be documented. Only after the documentation is done is the cable job complete (if it even is, then).

When someone asked about Rack Cable Management, I typed out a few of my thoughts, and then kept typing. I've basically pasted it below, because I thought that some of you all might be interested as well.

And just for the record, I've talked about cable management before. Heck, I even did a HOWTO on it a long time ago.

Label each cable
I have a brother P-Touch labeler that I use. Each cable gets a label on both ends. This is because if I unplug something from a switch, I want to know where to plug it back into, and vice versa on the server end.

There are two methods that you can use to label your cables with a generic labeler. You can run the label along the cable, so that it can be read easily, or you can wrap it around the cable so that it meets itself and looks like a tag. The former is easier to read, the latter is either harder to read or uses twice as much label since you type the word twice to make sure it's read. Long labels on mine get the "along the cable" treatment, and shorter ones get the tag.

You can also buy a specific cable labeler which provides plastic sleeves. I've never used it, so I can't offer any advice.

Color code your cables
I run each machine with bonded network cards. This means that I'm using both NICs in each server, and they go to different switches. I have a red switch and a blue switch. All of the eth0's go to red switch using red cables (and the cables are run to the right, and all eth1's go to the blue switch using blue cables (and the cables are run to the left). My network uplink cables are an off color, like yellow, so that they stand out.

In addition, my racks have redundant power. I've got a vertical PDU on each side. The power cables plugged into the right side all have a ring of electrical tape matching the color of the side, again, red for right, blue for left. This makes sure that I don't overload the circuit accidentally if things go to hell in a hurry.

Buy your cables
This may ruffle some feathers. Some people say you should cut cables exactly to length so that there is no excess. I say "I'm not perfect, and some of my crimp jobs may not last as long as molded ends", and I don't want to find out at 3 in the morning some day in the future. So I buy in bulk. When I'm first planning a rack build, I determine where, in relation to the switches, my equipment will be. Then I buy cables in groups based on that distance.

When the time comes for cable management, I work with bundles of cable, grouping them by physical proximity (which also groups them by length, since I planned this out beforehand). I use velcro zip ties to bind the cables together, and also to make larger groups out of smaller bundles. Don't use plastic zip ties on anything that you could see yourself replacing. Even if they re-open, the plastic will eventually wear down and not latch any more.

Keep power cables as far from ethernet cables as possible
Power cables, especially clumps of power cables, cause ElectroMagnetic Interference (EMI aka radio frequency interference (or RFI)) on any surrounding cables, including CAT-* cables (unless they're shielded, but if you're using STP cables in your rack, you're probably doing it wrong). Run your power cables away from the CAT5/6. And if you must bring them close, try to do it at right angles.

  • Don

    Excellent post. I'm going to try and post this reply to serverfault, but google's openID doesn't seem to play well with our proxies there. Anyhow, to add to what you've said :

    Buy your cables
    I used to be very adamant that everyone should crimp their own cables. I still believe that if you plan on calling yourself a sysadmin, you should be able quickly make a 568A, 568B, crossover and null ethernet cables should any occasion require it. In fact, you should have your crimping tool, RJ45 plugs, bulk cable, and continuity tester in your office. (This is an appropriate skill to pass on to your future PFY, knowing how to make cables has definitely helped me out more than one once).

    BUT

    These days, I realize that crimping your own cables isn't appropriate for every site. I'd still check out each cable before I run it though, even when I've bought cables I've found some that have had issues.

    Keep power cables as far from ethernet cables as possible

    This is dependent on your site, but the worst cabling disasters I've dealt with are when people slavishly run power cables and ethernet cables up and down the sides of the rack. It seems like a great idea in theory, but the practice is a giant knot of either power or ethernet inside the rack post.

    The best way I've found to combat this problem is to run ethernet down the center of the rack, rather than the sides.

    Patch Panels
    These are your friend. If you have a separate networking group and they don't use patch panels, it is because they don't know how to use them.

    Patch Panels -- The Bad
    Yes, they take up space in your rack, yes they limit the amount of available ports per rack. Putting them in is more work. I will also admit they are not appropriate for every site.

    Patch Panels -- The Good
    Tracing cable runs of 6 feet or less is exponentially easier than tracing cable runs of 12 feet or more. Adding in new equipment is easier, no more fishing cables through 15 different runs. This can limit the amount of crap that gets shoved into one rack (sometimes this is good).
    It is a fact of life that servers move frequently comparative to other datacenter equipment. All your old cable drops just add to mess you have to deal with. Since power cables are confined to the rack and can always be reused, this keeps the clutter buildup down.
    Most of the frequently cited bad issues with patch panels can be mitigated through planning appropriately.

    Raised Floors
    Can still be a good thing in some cases. NEVER put network drops under a raised floor unless you hate your coworkers and plan to leave soon. If you have a raised floor only use it to run power cables.

    Ladder racks
    Overhead cable runs are the only way to preserve your sanity or blood pressure. This is especially true if you don't use patch panels.

  • Matt

    Awesome post, Don!

    You make great points here. I'd like to add that ever large scale datacenter I've been to that really knew what they were doing had the cable ladders in place (some have multiple levels!) and kept their power under floor.

    Great stuff! I'll be looking for it on serverfault! Thanks for the comment!

  • Ian

    I wish I had the time to be neat and orderly with my cabling here. Unfortunately whenever I'm cabling anything, things start out well, but end up getting ugly in a hurry because I don't have enough time to devote on it. Although I just recently cleaned up the MDF in our main building and it's looking great compared to the waterfall of cables that it had been.

    Being able to rip out cables that don't even link to anything anymore makes me feel giddy.

  • Brad Gilbert

    The ServerFault page is where I first heard of this Blog.

  • Matt
  • chewyfruitloop

    i recently moved from zip ties to velcro ties
    it makes it much better to add and remove cables, its just a lot more expensive to do

    i think a dozen velcro is the same price as 100 zips

    still.... you don't have to go round snipping them when you need to add something or move the position a bit :)

  • Matt

    @Chewyfruitloop

    I agree, the initial cost is steep, but you're right, when it comes time to MAC (move add change) cables, you're SO glad you used velcro :-)

    Do you have a good source for cheap velcro yet? I've been keeping my eyes open, so if you see one, drop me a line!

  • AJ

    Nice post!

    I always try to keep things clean here....but over time they get messy....plus lack of anyone letting me spend money on anything so I have to "make do" with what I either make or find laying around.

  • http://www.neon-light.info Neon Light

    ehternet cables are still the ones that i use for my home networking applications *":