Happy SysAdmin Appreciation Day 2014!

Date July 25, 2014

Well, well, well, look what Friday it is!

In 1999, SysAdmin Ted Kekatos decided that, like administrative professionals, teachers, and cabbage, system administrators needed a day to recognize them, to appreciate what they do, their culture, and their impact to the business, and so he created SysAdmin Appreciation Day. And we all reap the benefit of that! Or at least, we can treat ourselves to something nice and have a really great excuse!

Speaking of, there are several things I know of going on around the world today:

As always, there are a lot of videos released on the topic. Some of the best I've seen are here:

  • A karaoke power-ballad from Spiceworks:
  • A very funny piece on users copping to some bad behavior from Sophos:
  • A heartfelt thanks from ManageEngine:
  • Imagine what life would be like without SysAdmins, by SysAid:

Also, I feel like I should mention that one of my party sponsors, Goverlan, is also giving away their software today to the first thousand people who sign up for it. It's a $900 value, so if you do Active Directory management, you should probably check that out.

Sophos was giving away socks, but it was so popular that they ran out. Not before sending me the whole set, though!

I don't even remember signing up for that, but apparently I did, because they came to my work address. Awesome!

I'm sure there are other things going on, too. Why not comment below if you know of one?

All in all, have a great day and try to find some people to get together with and hang out. Relax, and take it easy. You deserve it.

Goverlan press release on the Boston SysAdmin Day party!

Date July 22, 2014

Thanks again to the folks over at Goverlan for supporting my SysAdmin Appreciation Day party here in Boston! They're so excited, they issued a Press Release on it!

Goverlan Press Release

Click to read more

I'm really looking forward to seeing everyone on Friday. We've got around 60 people registered, and I imagine we'll have more soon. Come have free food and drinks. Register now!

Online Ticketing for 2014 Boston SysAdmin Appreciation Day Party powered by Eventbrite

Slackware is 21? Wow I'm old!

Date July 18, 2014

I saw that Slackware Linux is now officially old enough to drink (in the US, anyway). That's pretty amazing!

Patrick Volkerding sent out the initial announcement that Slackware was being worked in way back in 1993.

I didn't start using it then. Heck, I didn't even have a computer then! My first machine was an IBM Aptiva 486 DX2/66. Basically, this:

A friend gave me a copy of the 1996 version of the InfoMagic Linux Developer's Resource, and Slackware (3) was the only distribution I could get working on my machine. At around the same time, another friend's dad found me a copy of Sam's Teach Yourself UNIX in 24 Hours. And that was the genesis of my learning Linux.

I ran Slackware continually, on my desktops and servers, until around 2006, when I needed to do more enterprise-y things at work, and switched to RedHat based systems because they could auth against Active Directory, but I still have a lot of fondness in my heart for Slack.

A while back, I wrote about how instrumental Slackware was to my early knowledge, and from the Hacker News thread, I can see I'm not alone.

If you've never run it before and you'd like a taste, it's easy to get Slack and install it in a VM. You should do this, especially if you've never installed a Linux in anything other than an X-Windowed environment. Today, if you install a desktop Linux, the entire process is graphical, but Slack was never like that. The installation itself used curses, but once you rebooted into your fully functioning machine, you were at a text prompt, and you were expected to configure everything you needed from scratch. Ah, the good old days ;-)

"Time Since Install" is the new "Uptime"

Date July 16, 2014

Welcome to the converged future where everything is peachy. We use configuration management to make our servers work exactly as we want, our infrastructures have change management procedures, test-driven design, and so on. Awesome!


We're well past the date where uptime (as in, the amount of hours or days a specific server instance) is a bragging right. We all know implicitly that the longer a server is running, and the more changes that we make to it, the less likely it is to start up and return to a known-good state. Right? Because that's a real thing. Uptime of several years isn't just insecure, it's a dumb idea for almost all server OSes - you just don't know what state the machine will be in WHEN it restarts (because eventually, it will restart). I think we can take that as read.

So that leads me to an observation that I had recently... I think that the concept of repeatability and reliability of services following a reboot can be extended to the time since installation. Clearly, configuration management is meant to allow for utterly replicable machines. You're defining exactly what you want the machine to do in code, and then you're applying that configuration from the same code. A leads to B, so you have control.

The other, uglier side of that coin, is that modern configuration management solutions are application engines, not enforcement engines. So, you can write your, say, puppet code, so that a change is applied. But what if you make a change outside of configuration management? An overly simple example might be using puppet to install Apache, then manually installing PHP though the command line.

OK, OK, OK. I know. No one is going to do that. That's stupid. I know. I said it was overly simple. But the truth is, unless you tell puppet specifically to enforce a resource (one way or another - if you don't want PHP installed, you can't just not define it...you must say "ensure => absent" or puppet doesn't care). That's what I mean by enforcement. The dictated configuration is not "this and only this".

So while you're not going to do something monumentally stupid like installing php manually, what about when you change your configuration management so that a resource that you WERE managing is no longer under management? Our environment here has over 300 resources JUST to install packages. Over time, with a lot of catalog compilations, as that number increases, it's not going to scale, and we're going to have to take some shortcuts.

When a resource that was managed becomes unmanaged, there is no enforcement mechanism in place to ensure that the previously managed <whatever> is taken care of, removed, or dealt with whatsoever. The question then becomes, how does that previous resource affect your remaining services?

If I have a bit of code that installs a package, and as a result, makes some other change (either installing a dependency, or creates or removes a file), and I build my infrastructure with that dependent, but unmanaged, resource existing, it's possible that I become dependent on it. If the resource that caused the dependency to happen is then unmanaged, chances are good that the dependent resource remains, and I never know that anything is different...until I attempt to run the code on a fresh install which never had the original resource. Well, that sucks, huh?

The fix for this is, of course, a testing environment that includes some sort of functional testing using Docker or Vagrant (or $whatever) to create a fresh environment each and every time. When you've got that going, the only sticking point becomes, "How good are your tests?"

In any event, I've recently been thinking about a sort of regular reinstallation cycle for servers, much like many places have a quarterly reboot cycle, where each quarter, they reboot a third of the machines to ensure that all of the machines will reboot if they need to in an emergency.

What do you think of my observations? Is there a reason to reinstall servers relatively regularly in production? Why or why not?

New SysAdmin Party Sponsor: GOVERLAN

Date July 15, 2014

I'm really happy to announce that my SysAdmin Appreciation Day party has another sponsor! This one is GOVERLAN.


GOVERLAN isn't something that I was familiar with until just the other day. It seems like a pretty cool piece of technology that ties into Active Directory to help manage your Windows infrastructure. That's pretty vague, I know, but if you like things like "Active Directory Integration" and like managing Windows resources, you should head over to their YouTube channel and watch some videos, which can explain it much better than I can.

Thanks, GOVERLAN, for supporting the SysAdmin Appreciation Day party here in Boston. We appreciate it!

I originally opened 70 tickets, but pulled back 20, since I was having trouble getting sponsors. With this recent news, I'm adding 25 more, so we're at 75 tickets total (of which a lot have been claimed, so grab yours today!


Other SysAdmin Day Parties 2014

Date July 14, 2014

Every year, there are SysAdmin Appreciation Day parties all over the world. I've been super busy, so I've only found out about a few this year. Here are the ones I know of (and please comment and let me know about the others so I can update this list):

Yep, that's all I know of! Send me more! Comment below and link me to them!

Ain't no party like a sysadmin party cause a sysadmin party achieves the maximum available uptime required.