Flashback: HOWTO: Racks and Rackmounting
March 1, 2010
This is flashback week, and today I’m including a HOWTO that I originally wrote in June of 2008 called “HOWTO: Racks and Rackmounting”. I had decided that no one was focusing on the physical aspect of system administration, even though most of us still have to deal with it. So I put together this information in the hopes that it would be useful.
Since it was originally published, it’s been viewed over 2,000 times and is still in the top 10 blog entries, which means that still, not enough people are covering the physical infrastructure aspect of system administration.
Enjoy!
I’m going to start a special feature on Fridays. It’s going to be sharing the sorts of tips that systems admins need to know, but can’t learn in a book. There are so many things that you learn on the job, figure out on your own, or run across on the net which make you realize that you’ve been doing something wrong for years. Sometimes you learn about things that you might have had no clue about. For instance, I just found out that you can do snapshots with LVM
Anyway, this Friday, I’m going to be showing you what I know about server racks.
I started out on a network that had a bunch of tower machines on industrial shelves; the sort you pick up at Harbor Freight or Big Lots. When we moved to racks and rackmount servers, it was like a whole new world.
The first difference is form-factor. Tower servers are usually rated by the “tower” descriptive. Full tower, half tower, mid-tower. Rack Servers are sized according to ‘U’s, short for “Rack Unit”. It’s equivalent to 1 3/4 inches, so a 2U server is 3.5” tall. The standard width for rackmount servers is 19” across. Server racks vary in depth, between 23 and 36”, with deeper being more common.
To anchor the rack hardware (also known as rails) to the rack itself, a variety of methods have been implemented. There are two main types of rack. Round hole racks, seen at the left, require a special type of rack hardware. Much more common is square hole racks, which require the use of rack nuts. The rack nuts act as screw anchors to keep the hardware in place. Some server manufacturers have created specific rack hardware that fits most square hole racks, and don’t require the use of rack nuts. Dell’s “rapidrail” system is one with which I’m very familiar. Typically you get the option of which rail system you want when you purchase the system.Installing the rack nuts is made easier with a specialized tool. I call it the “rack tool”, but I’m sure there’s another name. The rack nut is place with the inside edge clip in place, through the hole. The tool is inserted through the hole, grabs the outside clip, and then you pull the hook towards you. This pulls the outside clip to the front of the hole, securing the nut in place.
A typical server will require eight nuts, usually at the top and bottom of each rack unit, on the right and left sides, front and back. Each rack unit consists of three square holes, and a rack nut is put in the top and bottom of both the right and the left sides. Several pieces of networking equipment have space for four screws, but I’ve found that they stay in place fine with two. I can’t really recommend it for other people, but if you’re low on rack nuts, it’s better than letting the switches just sit there (and it almost always seems like you have fewer rack nuts than you need once your rack starts growing). If you only use two screws to hold in your n
etworking equipment, make sure it’s the bottom two. The center of gravity of a rackmount switch is always behind the screws, so if the top screws hold it up, the bottom has a tendency to swing out, and that’s not good for your rack or your hardware.
While I’m on the subject of swtches, let me give you this piece of advice. Mount your switches in the rear of the rack. It seems obvious, but you have no idea how many people mount them on the front in the beginning because “it looks cooler” and then regrets it when they continually have to run cable through the rack to the front.
Once your rack starts to fill out, heat will become an issue. When you align your rack for your air conditioner, another bit of common sense that’s frequently ignored. Air goes into the servers through the front, and hot air leaves through the back. This means that when you cool your rack, you should point the AC towards the front of your rack, not the back.
It’s probably not a stranger to anyone who’s used a computer, but the cables seem to have a mind of their own, and nowhere is it more apparent than a reasonably full server rack. Many higher-end solutions provide built-in cable management features, such as in-cabinet runs for power cables or network cables, swing arms for cabling runs, and various places to put tie-downs.
There is no end-all-be-all advice to rack management, but there are some tips I can give you from my own experience.
Use Velcro for cabling that is likely to change in the next year. Permanent or semi-permanent cabling can deal with plastic zipties, as long as they aren’t pulled too tight, but anytime you see yourself having to clip zipties to get access to a cable, use Velcro. It’s far too easy to accidentally snip an Ethernet cable in addition to the ziptie.
Your rackmount servers will, in many cases, come with cable management arms. Ignore them. Melt them down or throw them away, but all they’ve ever done for me is block heat from escaping out the back.
Label everything. That includes both ends of the wires. Do this for all wires, even power cables (or especially power cables). Write down which servers are powered by which power sources.
If you have a lot of similar servers, label the back of the servers too. Pulling the wrong wire from the wrong server is not my idea of a good time.
Keep your rack tool in a convenient, conspicuous spot. I ran a zip tie through the side of the rack, and hang mine there.
(Some photos were courtesy of Ronnie Garciavia Flickr)
Posted in General
Email me
content rss
March 1st, 2010 at 3:21 pm
Hi Matt,
Always a relevant article!
Something I’ve always found useful, when using servers with redundant power supplies I designate a colour for each side of the rack, i.e. blue for pdu’s on the left and red for pdu’s on the right. Then a small coloured cable tie is put on each end of the power cable making it easy to see at a glance which side of the rack each power supply is connected to and ensuring that you dont have a server with both feeds from one pdu which would end badly (for the server uptime) if the pdu decides to end itself.
cheers
Sam
March 1st, 2010 at 3:58 pm
Sam: Thanks! And that’s a good idea. What I got in the habit of doing after this posted the first time was to use blue and red electrical tape. It’s cheap, so I just leave a pair in each of my racks so it’s always handy.
I also try to wrap both ends, so there is no confusion. They should definitely sell color coded power cables for servers, though
March 1st, 2010 at 3:59 pm
It all sounds good, except that I just can’t get behind the “ditch the cable management arms” bit.
Certainly, cable management arms do slightly impede airflow. I absolutely love being able to pull a server out of a cage w/o worrying about what’s going on wth the cabling behind, though.
Cable management arms are definitely less effective when they aren’t on every machine in a rack (since the lack of cable management arms makes it that much easier to snag cables attached to servers that don’t have cable management arms installed). An entire rack filled with servers that have cable management arms attached is a dream to work on, to me.
March 1st, 2010 at 4:06 pm
Thanks, Evan. I’m against cable management arms because I’ve never had a rack full of the same (or even similar) equipment. If I did, I’m certain that I’d like them a lot better.
Plus, at the price point I buy servers at, I can count the number of times I’ve needed to take the lid off a server while it was running on zero hands. The only hot swappable things that I deal with are power supplies and drives, and they come out the front and back.
That being said, I can certainly see why they’re useful. If I had a fan die, I would have to power down the machine. If I had equipment like this, I’d definitely use cable arms.
As it is, they’ve only ever gotten in my way.
March 14th, 2010 at 10:56 am
Ahh that does bring back some memories !!
How many times the edge of the rack bolts went under my thumb nail because the “rack tool” had been misplaced !
Stephane
March 14th, 2010 at 11:03 am
*shudder*
Ouch.
I feel your pain. I did learn that you can almost always get them out by putting sufficient pressure on the back part of the nut and moving it to one side, then pulling. The “arm” gives enough to slip the other side out. Usually.
Now, I just keep my pocketknife handy.
April 19th, 2010 at 3:13 am
24 port rack mount ethernet switchGreat article! I’d like to add that the PoE Switch is also and ideal model for distant networking!