Goverlan press release on the Boston SysAdmin Day party!

Date July 22, 2014

Thanks again to the folks over at Goverlan for supporting my SysAdmin Appreciation Day party here in Boston! They're so excited, they issued a Press Release on it!

Goverlan Press Release

Click to read more

I'm really looking forward to seeing everyone on Friday. We've got around 60 people registered, and I imagine we'll have more soon. Come have free food and drinks. Register now!

Online Ticketing for 2014 Boston SysAdmin Appreciation Day Party powered by Eventbrite

Slackware is 21? Wow I'm old!

Date July 18, 2014

I saw that Slackware Linux is now officially old enough to drink (in the US, anyway). That's pretty amazing!

Patrick Volkerding sent out the initial announcement that Slackware was being worked in way back in 1993.

I didn't start using it then. Heck, I didn't even have a computer then! My first machine was an IBM Aptiva 486 DX2/66. Basically, this:

A friend gave me a copy of the 1996 version of the InfoMagic Linux Developer's Resource, and Slackware (3) was the only distribution I could get working on my machine. At around the same time, another friend's dad found me a copy of Sam's Teach Yourself UNIX in 24 Hours. And that was the genesis of my learning Linux.

I ran Slackware continually, on my desktops and servers, until around 2006, when I needed to do more enterprise-y things at work, and switched to RedHat based systems because they could auth against Active Directory, but I still have a lot of fondness in my heart for Slack.

A while back, I wrote about how instrumental Slackware was to my early knowledge, and from the Hacker News thread, I can see I'm not alone.

If you've never run it before and you'd like a taste, it's easy to get Slack and install it in a VM. You should do this, especially if you've never installed a Linux in anything other than an X-Windowed environment. Today, if you install a desktop Linux, the entire process is graphical, but Slack was never like that. The installation itself used curses, but once you rebooted into your fully functioning machine, you were at a text prompt, and you were expected to configure everything you needed from scratch. Ah, the good old days ;-)

"Time Since Install" is the new "Uptime"

Date July 16, 2014

Welcome to the converged future where everything is peachy. We use configuration management to make our servers work exactly as we want, our infrastructures have change management procedures, test-driven design, and so on. Awesome!


We're well past the date where uptime (as in, the amount of hours or days a specific server instance) is a bragging right. We all know implicitly that the longer a server is running, and the more changes that we make to it, the less likely it is to start up and return to a known-good state. Right? Because that's a real thing. Uptime of several years isn't just insecure, it's a dumb idea for almost all server OSes - you just don't know what state the machine will be in WHEN it restarts (because eventually, it will restart). I think we can take that as read.

So that leads me to an observation that I had recently... I think that the concept of repeatability and reliability of services following a reboot can be extended to the time since installation. Clearly, configuration management is meant to allow for utterly replicable machines. You're defining exactly what you want the machine to do in code, and then you're applying that configuration from the same code. A leads to B, so you have control.

The other, uglier side of that coin, is that modern configuration management solutions are application engines, not enforcement engines. So, you can write your, say, puppet code, so that a change is applied. But what if you make a change outside of configuration management? An overly simple example might be using puppet to install Apache, then manually installing PHP though the command line.

OK, OK, OK. I know. No one is going to do that. That's stupid. I know. I said it was overly simple. But the truth is, unless you tell puppet specifically to enforce a resource (one way or another - if you don't want PHP installed, you can't just not define must say "ensure => absent" or puppet doesn't care). That's what I mean by enforcement. The dictated configuration is not "this and only this".

So while you're not going to do something monumentally stupid like installing php manually, what about when you change your configuration management so that a resource that you WERE managing is no longer under management? Our environment here has over 300 resources JUST to install packages. Over time, with a lot of catalog compilations, as that number increases, it's not going to scale, and we're going to have to take some shortcuts.

When a resource that was managed becomes unmanaged, there is no enforcement mechanism in place to ensure that the previously managed <whatever> is taken care of, removed, or dealt with whatsoever. The question then becomes, how does that previous resource affect your remaining services?

If I have a bit of code that installs a package, and as a result, makes some other change (either installing a dependency, or creates or removes a file), and I build my infrastructure with that dependent, but unmanaged, resource existing, it's possible that I become dependent on it. If the resource that caused the dependency to happen is then unmanaged, chances are good that the dependent resource remains, and I never know that anything is different...until I attempt to run the code on a fresh install which never had the original resource. Well, that sucks, huh?

The fix for this is, of course, a testing environment that includes some sort of functional testing using Docker or Vagrant (or $whatever) to create a fresh environment each and every time. When you've got that going, the only sticking point becomes, "How good are your tests?"

In any event, I've recently been thinking about a sort of regular reinstallation cycle for servers, much like many places have a quarterly reboot cycle, where each quarter, they reboot a third of the machines to ensure that all of the machines will reboot if they need to in an emergency.

What do you think of my observations? Is there a reason to reinstall servers relatively regularly in production? Why or why not?

New SysAdmin Party Sponsor: GOVERLAN

Date July 15, 2014

I'm really happy to announce that my SysAdmin Appreciation Day party has another sponsor! This one is GOVERLAN.


GOVERLAN isn't something that I was familiar with until just the other day. It seems like a pretty cool piece of technology that ties into Active Directory to help manage your Windows infrastructure. That's pretty vague, I know, but if you like things like "Active Directory Integration" and like managing Windows resources, you should head over to their YouTube channel and watch some videos, which can explain it much better than I can.

Thanks, GOVERLAN, for supporting the SysAdmin Appreciation Day party here in Boston. We appreciate it!

I originally opened 70 tickets, but pulled back 20, since I was having trouble getting sponsors. With this recent news, I'm adding 25 more, so we're at 75 tickets total (of which a lot have been claimed, so grab yours today!


Other SysAdmin Day Parties 2014

Date July 14, 2014

Every year, there are SysAdmin Appreciation Day parties all over the world. I've been super busy, so I've only found out about a few this year. Here are the ones I know of (and please comment and let me know about the others so I can update this list):

Yep, that's all I know of! Send me more! Comment below and link me to them!

Ain't no party like a sysadmin party cause a sysadmin party achieves the maximum available uptime required.

Two Sponsors For SysAdmin Appreciation Day Party!

Date July 11, 2014

I'm very happy to announce today that we've got two sponsors so far for the SysAdmin Appreciation Day party!

Peak Hosting sponsoring $1,000 worth of food and drinks!

Netsource Global is sponsoring $500 worth of food and drinks!

This party is going to be great!

I want to thank both Peak Hosting and Netsource Global for helping out so much. I appreciate it, and I know the party guests do, too.

The room minimum is $2,000, so we're still looking for sponsors (and last year, we ended up with so many sponsors, that the food bill went fully paid and we were giving away gift cards at the party)!

You definitely want to come take part.

Eventbrite - 2014 Boston SysAdmin Appreciation Day Party