August 23, 2010
I have been to the Ohio Linux Festival once, four years ago. I had a really great time, met interesting people, and made plans to come back the next year. Then I got married the next year on the same weekend that OLF was being held. As much fun as I had at OLF06, I couldn’t really choose it over my own wedding (though frankly, I’m surprised some of our guests picked us over the show). The next year, they had the nerve to schedule it on my anniversary. Jeez!
This year, though, it’s scheduled earlier in September (from the 10th to the 12th), which means I can go! Except that I can’t. I’ve had other stuff scheduled for that weekend for almost a year. Ugh!
On the other hand, you CAN go (and I’m jealous). The schedule looks great, and I’ll let you in on a little tip. It’s directly across the street from Barley’s, home to some of the best burgers and beer in the city. That alone might be worth the trip!
As for the Linux Fest itself, it’s free admission, but if you’re coming (and you are, right?), you should really consider some of the OLFU classes, which are available for a fee. OLFU is the Ohio LinuxFest University, and it’s a day of training put on by LOPSA, the League of Professional System Administrators. I’m a (too-often absentee) member of the committee that is responsible for the classes, and I want to tell you that I’m very excited to see some of the things we’ve got lined up.
The thing that I’m most thrilled about is a class called Datacenters: Planning, Expanding, and Migrating. Finally, a physical infrastructure class! Holy Cow! If I could make it, I would sign up for this class in a heartbeat. How many times have you needed to make changes to the infrastructure, and were told, “Sorry, we can’t have any downtime”. I’m doing a big migration soon myself, and I would love to be part of this class. This alone may be worth the trip.
A course that sounds intriguing is Black Magic: Linux Troubleshooting and System Administration. I’ve talked to the instructor, John Billings, on IRC, and he really knows his stuff. I’m hoping that there are some notes or slides from this class (or maybe you could write a guest blog / review of the class, and I could post it here).
There are a ton of other courses as well. Check out the course list and decide what you want to take. As always with these things, the hard part is narrowing it down.
So go to Linux Fest and have a good time for me. Make sure to bring back all kinds of stories and let me know how it goes. Oh, and LOPSA is looking for volunteers to help them man the booth there, so if you want to volunteer some time, comment on this entry (or drop me an email) and let me know. We appreciate any help we can get!
Posted in General 
1 Comment »
August 9, 2010
Greg Ferro at the Ethereal Mind blog has some great tips up today on filling a server rack. Definitely check it out. It’s great to see someone mentioning physical infrastructure!
If you’re interested in this stuff, way back in 2002, I wrote an entry on Racks and Rackmounting and a piece on Server Cable Management that you may enjoy.
Posted in General 1 Comment »
August 9, 2010
I’m working on automating some installs that are going to happen during the infrastructure upgrade, and I need to decide what I want to use for automation.
I have used Kickstart before, and it’s essentially a single file that contains instructions for the RedHat installer (although Debian is in on that action, too). The idea with Kickstart is that your “normal” installation (whether that be through DVD, USB key, PXE, or whatever) points to the kickstart file, and the installation proceeds according to those instructions.
Cobbler goes the extra steps and becomes the installation server, PXE/DHCP boot provider, etc etc, in addition to working with kickstart files. In fact, it can even do crazy kickstart templating. It certainly seems full featured, and I’ve heard people recommend it before.
One of the coolest things I’ve seen it be able to do is automate new virtual machines. As I understand it, you basically hit a button and a VM is created, powered on, and installed according to the kickstart templates. That’s slick.
Unfortunately, the best support for Koan (the Cobbler client) is on Qemu/KVM. The site mentions support for VMware Server, but that’s anathema. There doesn’t appear to be support for ESXi (certainly not 4.1, which was just released last month), but I was hoping for something more recent than a question on VMware Communities from 2007.
So I come to you. If you’ve got an ESXi infrastructure, do you automate rollouts? Am I just doing this wrong? I’m leaning toward manually spinning up machines and using Cobbler / Kickstart to perform the installs (maybe with customized boot media, in the case of just kickstart). What do you suggest?
Posted in General 23 Comments »
August 5, 2010
The reason that I started this blog was to share what little information I had, learn from others, and build a community of system administrators who were interested in improving themselves and their peers.
It was for identical reasons that I joined the League of Professional System Administrators. It is very important to me that I contribute, in some way, to the betterment of our profession, and I’ve tried to do just that in every effort on this blog, as a member of LOPSA, and in my interactions with sysadmins of all types.
The way that I’ve found that seems to work best for me in understanding other people, and having them understand me, is to put myself in their place, and consider the situation from their perspective. Doing this requires humility, because it supposes that my way may not be the best or only way. This is difficult, because I have an ego and admitting I may not be right requires swallowing my pride. But I do it, because to work together for mutual improvement, it’s necessary to maintain an open mind.
Not everyone makes this effort, however. There is, and probably always will be, a minority of people who are xenophobic. That is, they are afraid of things and ideas that are different from theirs. As I was explaining to someone on twitter earlier, this xenophobia manifests itself in a complete antipathy toward methods and opinions which differ from their own. You’ve seen it. We’ve all seen it. You see it whenever someone decides an idea is wrong because it belongs to someone else, and if you challenge that stance, the person attacks you.
Let me be crystal clear. This condition is harmful. It’s harmful to the administrator who holds it, and to all of the systems that they deal with. Someone blindly refusing to implement the right solution because they’re prejudiced and superstitious is like a parent who refuses to inoculate their child. The solution doesn’t get applied because of irrational fear and mistrust.
This xenophobia, if left unchecked, advances to elitism. Let me say it again. Elitism is really just an advanced stage of xenophobia. Not only is your solution right, your solution is the best. In fact, anyone who doesn’t use your solution is inferior, obviously, and deserves derision, or at best, sympathy. You’re using $X? Oh, I’m sorry…
Elitism is what happens when your opinions are not only held for a long period of time, but encouraged by the people around you. If there aren’t dissenting voices, then obviously you’re not doing anything wrong, right? These kinds of questions fade away eventually into the assumption of correctness. By default, I’m right unless proven wrong. Younger members of the community see elders take these airs, but they mistake it for competency, which eventually produces more elders who feel the same wrong sense of entitlement, the same biases, the same assumptions, and the same elitism.
It’s very fortunate that not all, or even most of the community isn’t like this. In fact, even the ones who are aren’t usually this bad. As with all humans, we’re not black or white, we’re shades of grey. Even better, because we’re humans, eventually we can change, improve ourselves, and get over these petty biases which hold us back and weaken our communities.
I urge you. Take this as a charge to evaluate yourself for your biases, your own little pockets of xenophobia. They’re there, trust me. I have them too. Examine them, and make yourself aware of them, and then when you recognize the urge to respond with them, just stop and critically evaluate your position. You may be right, but it’s possible you’re wrong too. None of us will probably ever get rid of them entirely, but each one we eradicate will stop us from making the wrong decision at one point or another, or stop us from needlessly tearing down relationships that we worked so hard to build.
Take a moment to dig in deep and think of a couple biases you have, and that you should get rid of. It takes courage to admit that you have them, but it’s worth it, and by sharing it, you are forced to admit it, which will make it easier to get rid of.
Posted in General 10 Comments »
August 4, 2010
Gestalt IT (that wacky bunch of network, storage, and virtualization experts who let me tag along to Boston a while back) have started the “Get Away to VMworld” contest! That’s right, Gestalt IT wants to send you to VMworld for free.
Thanks to their sponsors Xsigo and Symantec, one person will get airfare, hotel, and a conference pass at no charge to themselves! I know what you’re asking yourself…is there a catch?
Well, there’s sort of a catch. They’re not drawing a name out of a hat…the winner has to promise to pay it forward. From the Gestalt IT page:
Inspired by winner Greg Stuart‘s desire to contribute to the community, we’re not just going to pick a winner at random. We’re going to pick the person who presents the best case for themselves.
Entrants must explain how they plan to “pay it forward” if they get to go to VMworld. Will you start a blog? Write some tutorials? Contribute to a forum or online community? Present to your local VMUG? Get creative and spread the wealth of knowledge you get from the event!
So there you go. What good would you do if you could go to VMworld? Fill out the entry form and let Gestalt IT know! You could be going to VMworld for free!
Posted in General 1 Comment »
August 3, 2010
Starting today, I’m beginning the ramp up to essentially replacing my infrastructure. Because any large upgrade takes a toll on service availability, it’s got to be lightning-fast.
Here at Investor Analytics, we’ve placed a lot of emphasis on availability. In addition to redundant servers in each of the critical roles, we maintain redundant physical sites in a hot / warm-standby configuration. Our primary site is where production takes place. If you talk to any of our servers, you’re talking to the primary site. Our backup site contains things like secondary application, mail, web, and file servers. In the event of issues at the primary site, DNS entries get changed and our secondary site becomes active.
Both our primary and our secondary sites need renovated. After significant consideration, I’ve decided that I’m going to perform the upgrade in stages, starting with the secondary site. After that work is done and well-tested, a week will be scheduled during which we will move production to the secondary site while the primary site is rebuilt in an identical fashion.
The process itself will involve migrating from one rack to a pair of racks, expanding our storage array, adding additional servers, networking equipment, and converting nearly all our physical hosts into VMs running on ESXi (as well as using vCenter to control them, if management listens to reason).
I’m estimating that it will take my junior admin and myself three full days of work to migrate the machines into the new racks, reconfigure them all, and test (and that’s assuming there are no large unforeseen issues). I’ve got a list of subjects that need researched, a list of things that need ordered, a list of tasks that need to be accomplished during the upgrade, and I’m currently working on a list of things to test following the migration. There are a lot of things on the list…on all of the lists, actually.
So that’s what is going to keep me busy in the near future. I’m happy to answer any questions about the details or techniques. I’m also very open to suggestion, so comment below!
Posted in General 7 Comments »
Email me
content rss