Leaving the LOPSA Board

Date December 10, 2014

It’s with some amount of sorrow and trepidation that I begin this blog entry.

One of the things that I often need to be reminded of is my own limitations. I think we all can forget that we’ve got human limits and sometimes we take on more than we can deal with. I am a chronic “joiner”. I like people, I like to build communities and organizations, and I like to put forth effort to make things happen.

By itself, this is fine, but in the macro, I try to do too much - certainly more than I can accomplish. My work suffers across the board from my lack of attention in any one area. It’s like the old problem of task switching, but when the tasks are completely unrelated to each other, it’s like context switching my entire brain out, and when I do it too often, I lose because of how inefficient it is. Worse than that, the tasks suffer.

For a long time, I was able to not let that be a massive problem, because I worked hard to keep myself out of the “critical path”, so that when I was concentrating on task B, task A could comfortably wait. But that’s not the case anymore. The quality of my work has been suffering, and it’s to the point where not only is everything I’ve been doing mediocre, those organizations where I’m in the critical path have suffered, and I’m no longer willing to make other people suffer because of problem of taking on too much.

Effective today at noon, I’m resigning as a Director of LOPSA. This might be surprising given how much I wanted to actively work and lead the change that I believe the organization needs, and I can tell you that no one is sorrier than I am that I’m stepping down. This isn’t me “breaking up” with the organization. I still believe that the organization has a lot to offer and its community of IT Admins is a potent force capable of a lot of good. But I’m not going to serve as a sea anchor to slow it down just when it needs to be more agile.

I’m really fond of the "golf ball an hour” analogy, and I’m going to start spending my golf balls on my family, and improving my IT skills. I remember when I was a good sysadmin. I don’t feel like that anymore. It’s not impostor syndrome in this case. It’s that I haven’t spent the time honing my skills and keeping up. So I’m going to try to fix that. And maybe I’ll be able to get some blog entries written about what I learn along the way.

So anyway, I’m going back to being a community member rather than a community leader, and I’m fine with that. The other LOPSA Board members have been very supportive of my decision, and I thank them for that, and I thank my many friends who have done the same.

If you were one of the many people who voted for me in the LOPSA Board election when I ran, thank you. You can take heart in the fact that I believe I was able to make some significant changes in the 18 months I served, and I really think that the organization is more aware of what its possibilities are than it ever has been. I’m glad I had the chance to serve and contribute. Thank you for giving me that opportunity.

What is Orbit?

Date December 3, 2014


Flying is learning how to throw yourself at the ground and miss
-- Douglas Adams

Even if you’ve never followed the space program, or know anything about rockets, you probably know what “orbit” is. I mean, all the time, the news reports when a spacecraft “reaches orbit”, and the Space Shuttle was often known as the “orbiter”. So you know, orbit is “up there”, and you circle a planet or moon or something. But have you ever thought about what it takes to get into orbit?

Lets imagine you are me - oh, I don’t know, 10 years ago, before I ever started looking into the specifics of how space stuff works. “Self”, I might ask you, “How do you get to space?”

The answer is clearly to go up. I didn’t know it 10 years ago, but “space” is basically anywhere with an altitude greater than 100km. So if you go up 100km, you will be in space. Will you then be in orbit? No. As soon as you stop going up, you will come right back down. This is what’s called a “ballistic trajectory”, and that’s not how you get to orbit. It doesn’t actually matter how far up you go, you’ll either come back down, or you’ll go away forever*. That’s not how you get to orbit.

Instead of going up, what happens if we go over? If you throw a baseball, what happens? It falls to the ground. But the Earth is round, and if you could throw the ball a thousand miles, when it eventually hit the ground, you can check on a globe and see that it would have followed the curve of the Earth. And if you threw it 4,000 miles? It would curve even more.

Because the Earth’s gravity is always trying to pull the baseball down, the baseball always curves toward the center of the Earth (well, specifically the center of the Earth’s gravity well). But because the baseball has a sideways motion, the ground also constantly curves away as the ball moves. If you can throw it fast enough that the ground curves away just as quickly as the ball goes toward the center, then the ball will never hit the ground. Instead, it will go all the way around and around, thereby completing an orbit.

Fortunately, we live on a planet with an atmosphere, so there’s really no way that you could throw a baseball and get it into orbit - the air would slow it down too much. But we have rockets, and we can use rockets to deliver baseballs. So lets use a rocket to make a baseball achieve orbit.

We have two choices. We could go straight up like we tried before, except this time, because we have a powered rocket, we can turn sideways once we’re out of the atmosphere and apply thrust in the direction we want to go, or we can turn sideways much sooner, and fly diagonally through the atmosphere, picking up speed vertically and horizontally at the same time.

The first one will work if you’ve got a rocket with enough fuel on it, but you end up wasting an awful lot. Whenever you’re just thrusting straight upup, you are making no progress going sideways, and in order to get an orbital trajectory, where you can make a full circle without hitting the ground (or, importantly, without coming back into the thicker parts of the atmosphere), you have to be going very fast unless you’re going very, very far away from the planet.

Instead, it almost always ends up being a good idea to go diagonally, though the flight profile (that is, how much thrust you use, when, and what direction you’re going) depends on the orbit you’re trying to reach, the mass of your rocket, and its performance capabilities. This technique is known as a gravity turn, because it uses gravity to help turn the vehicle into the orbit you want it to achieve.

Most of the things we launch are launched “prograde”, that is, they are going in the same direction that the planet spins. On a map, that means they go from West to East. The reason for this is that the planet is turning, and you get a free boost if you go in the same direction as the planet.

There are also times where you might want a “retrograde” orbit, where the orbit will end up going East to West. This is useful if you want your space craft to cover a lot of ground in a little bit of time (because the spacecraft is orbiting against the rotation, it sees the entire ground path in less time than if it were prograde and traveling with the ground). This orbit is harder to achieve because you’re working against the rotational boost of the planet.

A third option is a polar orbit, where the spacecraft orbits north to south, or south to north. You don’t get a rotational boost with this orbit, though your inherent rotation does need to be corrected so that it doesn’t skew your orbit away from the pole. This orbit has the benefit of being able to observe 100% of the body’s surface. Because the vehicle is orbiting from pole to pole as the planetary body turns underneath it, all of the surface winds up under the ground path.

As an aside, a ground path is, literally, the part of the ground under the spacecraft. If you’ve ever seen a picture or movie of Mission Control, there’s usually a map with curved lines tracing the path that the craft will follow. Those curves don’t mean that the craft is in orbit bobbing and weaving back and forth. It means that the craft is orbiting at an angle, called the inclination.

Inclination is almost always measured from the equator of the planetary body, so if a ship is orbiting at a 0% inclination, it means it’s basically tracing the equator. If it has an inclination of 51.65 degrees like the International Space Station, then the orbit the ship is taking is slightly steeper than halfway between a flat orbit on the equator and a flat orbit on the poles.

On a globe, it looks something like this:

and if you trace the path that orbit follows over the ground, it looks like this:

Now, whenever you see a ground track like that in mission control, you’ll usually see a couple of them together. That’s because the planet is constantly turning, and by the time one orbit is done, what was underneath the ship has moved, and now something else is.

The last thing we should talk about is orbital altitudes - that is, the distance from the ground.

It’s important to realize that almost no (and, in fact, probably no) orbits are perfectly round, and a lot of them are very elliptical. Simplistically, an orbit looks like a conic section. That is, if you take a cone, and cut it, the edge of the cone you cut resembles a potential orbit. In reality, irregularities in the mass distribution of the body a ship is orbiting (and in the ship itself) means that the orbit won’t perfectly match the conic section, and that you won’t be able to describe an orbit using a perfect ellipse. But you can get a pretty good idea of how orbits work by thinking about them in terms of ellipses.

The two things that determine a ship’s orbit around any given body are its altitude and its orbital velocity. With a circular orbit, the ship maintains a constant orbital speed (that is, how fast it travels through space) and a constant altitude (that is, distance from the ground). Remember that since this is an orbit, you don’t need to thrust constantly to maintain the orbit. You’re falling down and sideways at the same speed - you just keep going. That’s what orbit is.

* - There are very small possibilities where you could launch straight up, get a gravity assist from something else, then come back and orbit Earth via free return trajectory. But that’s cheating.

NASA Social Live - I'll be there! Watch Live! #nasasocial

Date December 2, 2014

We are down to the brass tacks, now. I'm sitting in a British pub next door to Cape Canaveral, getting my schedule stuff squared away for the next two days, and things are gelling amazingly well.  I picked up my NASA Press badge today, and it's starting to get to the point where it feels "real".

So yeah, it's getting real.

Tomorrow, my schedule starts at 5am, when I get up to arrive at the Operations and Support Building II, where we'll do group introductions and then start touring Kennedy Space Center at 9am and Launch Complex 37, where Orion will launch from on Thursday.

NASA OSBII at KSC

At the time of this writing, the chance of launching on schedule stands at 60%. The launch window is 2 hours and 39 minutes. If something happens and the Thursday launch is scrubbed, the Friday window is apparently identical.

My schedule tomorrow is looking pretty intense. NASA Social is going to be broadcasting live on NASA TV, which means that you can watch, too. So far, the Wednesday schedule looks like this:

There are also "pop-in speakers" between 4pm and 5pm, whatever that means. I'm not sure. But it might be exciting!

Thursday, I have to show up in a parking lot at 5pm to catch a bus to the NASA Causeway where we'll watch the launch. I don't yet know how close we'll be, but according to Google Maps, the view should be pretty good:

 

Anyway, make sure to tune in to NASA TV tomorrow, Wednesday, December 3rd at 1pm. This link should be live, but if not, check out NASA TV.


Watch live streaming video from nasa at livestream.com

 

Introduction to Rocket Engines! #NASASocial

Date November 28, 2014

How do rocket engines work? Very well, usually. Thanks for asking.

Ba-dum Tiss.

Seriously though, it might not be a bad idea to talk about the basics of how rockets work so that other, more detailed information about spaceflight makes sense. I’m most definitely not an expert, but I’m happy to teach what I know, and I’m even happier to be taught, so if you know of something I’ve written that’s wrong, just shoot me a line.

The very basics:

Newton’s laws are pretty clear. An object at rest stays at rest, and an object in motion stays in motion (unless acted upon by an outside force). Here on Earth, we get kind of a biased intuition about those things, because

A) we are surrounded by air, and air is something, so objects are acted upon by it, and

B)We’re attached to a giant rock, which has gravity, and gravity is an outside force that we don’t ever see but take for granted. So that skews our perception of what “rest” and “motion” mean (even though they’re all relative anyway).

If you were in some kind of environment in microgravity, like the ISS, and you had managed to sneak aboard with one of those compressed-air t-shirt launching guns that cheerleaders use at football stadiums (like the one that killed poor Mrs Flanders on The Simpsons), and went outside on a space walk and you brought the launcher, you’d get in a lot of trouble. But before you got in trouble, you could have some fun. If you shot a t-shirt, you would get propelled backwards as the t-shirt got propelled forwards. That’s because the t-shirt pushed as hard on your gun (and you) as your gun did on the t-shirt. Equal and opposite reactions.

If you had a large amount of compressed air and t-shirts, you could drive yourself around the ISS (or anywhere else in microgravity, for that matter). Granted, it would be slow, because while the t-shirt is shot out with enough force to hurl it to the top of the stands, you have a lot more mass (and therefore more inertia) than the shirt, so you won’t go as fast.

Sadly, though, you don’t have infinite t-shirts or infinite compressed air. You only have what you can hold, so eventually you’ll run out. Alas, it’s probably for the best that you don’t have so many, because t-shirts and compressed air might be (virtually) weightless in microgravity, but they still have mass, just like you do, so they have inertia, just like you do. The more t-shirts and gas you carry (and thus the more mass you have), the less effect each launched t-shirt has on you. More t-shirts gives you more fuel to push with, but it also loads you with more mass, so you have to push more anyway to get the same result.

This relationship is called the “tyranny of the rocket”, and it’s a very real problem that rocket scientists have to deal with.

So we've figured out that you have a number of t-shirts and an amount of compressed air, and that you and your fuel together have a certain amount of mass. As it turns out, we can use this information to figure out how fast you can go, as long as we know how hard each t-shirt is pushed away from you. Like this:

Suppose each t-shirt has a kilogram of mass (these are really big t-shirts - 2.2lbs!). We know that when you shoot a shirt, it comes out of the barrel at 1 kilometer per minute, or 60 kilometers per hour. We have also checked, and the gun uses 100 grams (0.1kg) of air for each shot. You, your space suit, and the gun together have a mass of 100kg, so you should eat something when you’re done with this exercise, maybe.

When we start, you’ve got 10 shirts, and 1 kg of compressed air. You have enough air for 10 shots, and when you’re out of air, you’ll be out of shirts. This is a balanced fuel mixture. That’s good, because any extra of either is wasted mass, and wasted mass is wasted fuel.

Now, you shoot the first shirt. It goes flying out of your barrel at 1km/min, your mass goes down by 1.1kg (because of the t-shirt and the air), and you go flying backwards…but by how much? Lets find out.

If you started out with 100kg, and a shirt is 1kg, and the shirt moves at 60km/hour, you will have 1/100th the effect, so you’ll start moving backwards at 0.6km/hr, or 0.01km/m. That’s pretty pokey. But wait, you don’t have 100kg of mass anymore. You’ve got 98.9 because the shirt and gas have been expended, so really, you’re going at 0.606km/hr, just a bit faster than we thought. And when you fire again, you’ll only have a mass of 97.8kg, so you’ll be accelerated by 0.613km/hr. And since you were already going backwards at 0.606km/hr, you’re now headed backwards at 1.219km/hr. Pretty soon, we’ll be talking about some real speed!

This change in velocity that we’re talking about is called DeltaV (Delta for the greek letter Delta, is the symbol for change, and the V is for velociraptors, as far as I can tell. Someone else told me that it was velocity, but who are you going to trust, me or some person I don’t even remember?). The amount of DeltaV a rocket has is measured in speed, and its value is determined by the relationship of the mass of the rocket and the mass (and efficiency) of the fuel.

So, if we fire every t-shirt in our arsenal, and we’re going backwards at 6.39km/hr, then that’s the DeltaV for our “rocket” - 6.39km/hr.

Remember back to when I mentioned air? If you had done this exercise in an environment with an atmosphere, like inside the ISS, you would end up going slower than 6.39km/hr, because the shirt would still be pushing you, but you would also be pushing against the air. Since you’re not at all aerodynamically shaped (sorry, you’re just not), it’s hard to tell exactly how fast you’d be going, but being shaped smoothly and tapered like a rocket helps decrease atmospheric drag, and increases your maximum speed in the end.

How much does it increase it? That depends on a lot of things. How much air we were dealing with, for instance. If we had the equivalent of sea level air pressure, that’s a lot more air than if we had the equivalent of an airplane cabin, most of which are pressurized at 10,000ft or so). And even THAT’S a lot more pressure than if you were outside of the airplane cabin at the 40,000ft cruising altitude of a lot of jetliners.

Which, again, is more than you get at the Karman Line, the internationally-recognized beginnings of space. Even though it’s still “space”, it’s not a perfect vacuum. In fact, the atmosphere of the Earth stretches REALLY FAR OUT. Hundreds and thousands of kilometers, past the ISS, past the Hubble Space Telescope, and sometimes out past the Van Allen Radiation Belts.

Anyway, this “pushing away mass in order to go faster in the opposite direction” style-acceleration is called a mass-reaction engine. It’s what basically every rocket engine has used, ever since the Chinese invented rockets centuries ago. In fact, if you think about it, every single method we know of to change our velocity relies on one thing: pushing off of something else.

If you’re in a car, you’re not using the internal combustion engine’s exhaust to push you. Instead, you’re using the explosive force to turn a crankshaft attached to wheels, and the wheels move you forward by relying on friction and pushing off the ground in the direction you want to go. This is why, even if you had wings on your car, you wouldn’t be able to fly…the wheels would have no ground to push on.

Let’s switch to propeller-driven aircraft or a propeller-driven boat. You have an internal combustion engine that’s turning a screw (or, if you’re on a big nuclear vessel, you’ve got a nuclear reaction producing heat that’s boiling water, and the pressure from that steam is turning the screw), and the propeller is pushing off of water or air to move the craft forward. This is why airplanes (and boats) won’t move in space. There’s no medium for them to push against.

Consider also jet-driven boats and airplanes. Both engines take in the fluid (either air or water), and accelerate it (though a turbo pump or through burning fuel), and expel it out the back faster than it came in. By pushing it out faster than it came in, the vessel is pushed forward faster, in an equal and opposite reaction. We’re getting closer now to rocketry, but if you take a jet engine into space, there’s no air or water for it to accelerate, so it doesn’t work.

So since there’s no medium that we can (or at least, that we know how to) push off in the vacuum of space, we have to bring our own mass, and throw it away from us. In the previous example, we used t-shirts and compressed air, and we said that we could accelerate the t-shirt to 60km/hr. What if we could change that to 600km/hr? Because of the whole “equal and opposite” thing, we would also need to change the other side of the equation, and we would find ourselves moving at over 6km/hr from a single shirt!

It’s probably not possible for 100g of compressed air to do this, sadly, so this isn’t realistic, but what you can see from this relationship is that the faster you expel the mass in the back, the faster you move forward, at least when taking into account the mass of the fuel and so on.

Here’s a table of the more common fuel mixtures that are used in rocketry, along with the energy that they can produce. T-shirts and compressed air aren’t listed, clearly because they haven’t caught on yet. The table is from a Rocket and Space Technology page.

SELECTED ROCKETS AND THEIR PROPELLANTS
Rocket Stage Engines Propellant Specific Impulse
Atlas/Centaur (1962) 0
1
2
Rocketdyne YLR89-NA7 (x2)
Rocketdyne YLR105-NA7
P&W RL-10A-3-3 (x2)
LOX/RP-1
LOX/RP-1
LOX/LH2
259s sl / 292s vac
220s sl / 309s vac
444s vacuum
Titan II (1964) 1
2
Aerojet LR-87-AJ-5 (x2)
Aerojet LR-91-AJ-5
NTO/Aerozine 50
NTO/Aerozine 50
259s sl / 285s vac
312s vacuum
Saturn V (1967) 1
2
3
Rocketdyne F-1 (x5)
Rocketdyne J-2 (x5)
Rocketdyne J-2
LOX/RP-1
LOX/LH2
LOX/LH2
265s sl / 304s vac
424s vacuum
424s vacuum
Space Shuttle (1981) 0
1
OMS
RCS
Thiokol SRB (x2)
Rocketdyne SSME (x3)
Aerojet OMS (x2)
Kaiser Marquardt R-40 & R-1E
PBAN Solid
LOX/LH2
NTO/MMH
NTO/MMH
242s sl / 268s vac
363s sl / 453s vac
313s vacuum
280s vacuum
Delta II (1989) 0
1
2
Castor 4A (x9)
Rocketdyne RS-27
Aerojet AJ10-118K
HTPB Solid
LOX/RP-1
NTO/Aerozine 50
238s sl / 266s vac
264s sl / 295s vac
320s vacuum

In that table, LOX is Liquid Oxygen, NTO is (di)Nitrogen Tetroxide, RP-1 is basically high-quality kerosene, LH2 is Liquid Hydrogen, Aerozine 50 is a commercial formulation of hydrazine, and HTPB is a plastic.

One thing that you’ll notice is that there are two components to each fuel mixture: a fuel and an oxidizer. This is important. Here on Earth, in the environment we’re used to, where 21% of the air surrounding us is oxygen, even pure hydrogen will burn, but since burning is literally just fast oxidation. We can’t get hydrogen to do anything fun or explosive in a vacuum unless we provide the oxygen, and the most efficient way of doing that is Liquid Oxygen, or LOX, as it’s commonly called. Different fuels have different oxidizers that we use. For instance, sugar burns remarkably well when you mix it with saltpeter, something that many experimental model rocket builders take advantage of.

There are three main types of rocket, depending on the phase of the fuel and oxidizer. Solid rockets are those where the fuel and oxidizer are, well, solid. Truth in advertising. These rockets are often used in small hobby scale, or as boosters for larger rockets. The fuel is packed into a tube with a hole molded down the center. The bottom of the fuel is ignited, and the fuel/oxidizer mix burns on the open surface, and the gas is expelled out of the nozzle. Here’s a diagram that might help:

(from How Stuff Works)

Because the fuel burns on the exposed surface area, the thrust of the solid rocket is largely dependent on the amount of exposed surface. Because of this, specific thrust profiles can be created by making the initially exposed area large, then shrinking it higher higher in the tube.

This ability to vary the thrust on a solid rocket engine is important, because solid rockets are like a candle, not an oil lamp. You can’t turn them down after you light them. They either burn up, or they explode (and if you’re launching them attached to a manned rocket, you want to be able to jettison them THEN explode them on purpose, in case something goes very wrong).

Liquid fueled rockets are, you guessed it. Rockets which have a liquid fuel and oxidizer. The thrust of these rockets is able to be controlled on-the-fly, which means they’re somewhat safer, though necessarily more complex.

There also exists a hybrid engine, where one part of the fuel/oxidizer mix will be liquid, and the other one solid. The only manned rocket that has ever successfully used one of these was SpaceShipOne, built by Burt Rutan and Scaled Composites, and it won the X Prize for reusable manned space flight. It used a rubber or plastic fuel and injected liquid nitrous oxide (yep, laughing gas).

Burning a fuel by itself doesn’t necessarily make your rocket move. To get thrust, you have to direct the result of the burning (after fuel is consumed, it becomes mostly hot gas and moisture) out the back of the rocket engine.

As you can see, there is a combustion chamber, which is under extremely high pressure (because fuel is actively burning in it, which increases both temperature and pressure according to Boyle's Law), and the engine nozzle, which expels the gas product of the burning.

Nozzles come in various shapes and capabilities. The simplest, such as those used on solid rockets, are simple clay disks with a hole in the middle. The lips of the nozzle lining the hole are specially shaped to force the gas to go supersonic, according to the Venturi Effect. This is called a de Laval Nozzle:

Diagram of a de Laval nozzle, showing flow velocity (v) increasing in the direction of flow, with decreases in temperature (t) and pressure (p). The Mach number (M) increases from subsonic, to sonic at the throat, to supersonic.

In liquid fueled engines, there are often additional parts designed to inject fuel at the proper rate and others to cool the nozzle so that it retains shape and performance. Depending on the tank that the liquid fuel is kept in, there may also be an inert gas that is used to maintain internal tank pressure so that the rocket itself maintains structural integrity during flight.

I hope that this brief introduction to how engines actually work provided some new information, and that you have a better idea of how rockets “go”, as well as some of the concerns that engineers have when designing a launch vehicle. As always, please let me know if there’s something that I’ve missed, or something that I got flat-out wrong. I’m always interested in learning more, and I’m definitely not an expert.

Thanks for reading!

—Matt

#SysChat discussion tonight on Twitter, Webcast on Friday

Date November 18, 2014

Well, this is new and interesting.

SolarWinds approached me a little while ago and asked me if I'd be interested in taking part in a combination Twitter discussion and webcast, talking about system administration, monitoring, and so on. I said yes, but what I really meant was, "You've seen my twitter stream...I was probably going to be talking about that stuff anyway, so sure!"

The details are that the twitter discussion will be using the #syschat hashtag. The idea is for this to be an interactive thing, with other twittererrers...twittites....twitterati...whatever, with other folks taking part. So if you see me tweeting with that hashtag, don't be surprised, and join in!

Watching a hashtag on Twitter is tough with only twitter.com, and I find even HootSuite or Tweet deck slow for these kinds of things, even though I use Hootsuite all the time. Whenever I'm watching a realtime thing like this (or, during launch events on the #nasa tag), I will use TwitterFall, which has a realtime display that kind of rains tweets down it, and since you're signed in, you can reply and retweet and interact the same way that you would on anything else. Once you're signed in, on the left side, just add "#syschat" to the searches, and you'll immediately get all of the past tweets falling.

Tonight's event is at 8:30pm Eastern time, and it'll probably last an hour or an hour and a half. And honestly, it's not like I ever get OFF of the internet, so I'll probably be watching for messages for the next day or so.

The event will be co-hosted by SolarWinds Head Geek LawrenceGarvin.

The webcast will be happening on Friday, 12pm-1pm. It's going to be a presentation on the topic of "Evolution of SysAdmin" and "Holistic Monitoring for Servers and Applications". You need to sign up to watch live, but it's free.

I want to be very up front about this: SolarWinds IS paying me to do these things, but at no point have they ever even suggested that I add mentions of SolarWinds products, or asked me to even say nice things about them, or about the company in general. The talk isn't even going to be about SolarWinds - it's on the topic of system administration, and questions about the kinds of things we deal with. This isn't me advocating for SolarWinds, because I can't. I've never had an environment where it made sense to use their products.

But I do thank them for asking me to take part in this thing. I have to imagine that being paid to tweet about things I would have talked about anyway is a little like how comedians feel when they get paid to host a stand-up special. I just want to say something like, "and you thought all that time tweeting was a waste. Ha!"

Anyway, see you today at 8:30pm ET, and Friday at noon. Don't forget to sign up!

New Technologies to Study from #LISA14

Date November 14, 2014

Well, I'm just now ending my first LISA as part of the program committee. This has been a long, long process (over a year, actually!) and I'm exhausted, but really happy that we had an awesome team and made a great conference come together. We had the highest attendance at a LISA conference since 2007 (over 1,100 people attended), even though AWS ReInvent was scheduled for the same week.

I wanted to take a few moments to reflect on some of the most talked about technologies, to give me some things to work on in the next year.

I need to improve my metrics.
Last year, I dedicated myself to spending the next 12 months increasing my visibility into the infrastructure, and to getting graphite up and running and watching what happened throughout my organization. Well, I succeed, to some extent, but this conference was a reminder that I'm no where near done.

Starting with the Statistics for Ops and R for SysAdmins classes, then later in the week when Theo Schlossnagle presented The Math Behind Bad Behaviors, I began to understand that monitoring isn't just pretty graphs and packet counts. In order to actually take full advantage of metrics, you have to have the right metrics. I'm getting what amounts to derivatives rather than the underlying measurements of the movements of my disks, network ports, and so on. I'm going to stop getting the 10,000ft view stats and start getting the ground level, base information.

Hadoop. Build one.
I have been hearing the word Hadoop for the past few years, but my sentiment on the subject was, "I don't need MapReduce. I don't even know what I would use it for. Why would I install Hadoop?"

Well, it turns out that I'm kind of dumb. Yes, MapReduce is a part of Hadoop, but it's hardly the only part. In fact, there are a lot of solutions both included in Hadoop and tightly coupled to it, such as a distributed file system (HDFS), distributed databases (HBASE, HIVE), and really, a ton of other things.

One of the things that uses Hadoop is OpenTSDB, which is a time series database, similar to RRDtool or Whisper (from Graphite). It can write with millisecond precision, so it's useful for monitoring low-latency IO (or anything else low latency, for that matter). Anyway, I want to start using it rather than graphite (partially because of the discrete resolution, and partially because it's not a lossy roll-up like RRD or Whisper, where you store long-term metrics in a lower resolution than they were captured, which makes it practically impossible to do a lot of correlations that would be interesting to use, as I learn more about how to do that).

Once I have OpenTSDB running, I can work on getting a cool new tool running, called Bosun. Invented by my friend Kyle Brandt at Stack Exchange, Bosun is a bit like an IDE or maybe like a DSL for alerting. It runs on OpenTSDB on the back end, so there's definitely a requirement change to make this work (although they have a docker instance to make it easier to learn, I've been told that you really don't want to run that for your production instance because, in Kyle's words, "You'll have a bad time".

Cool Tools
I ALWAYS come back from LISA with a list of cool tools that I want to check out. I usually forget about them. Now, I'm just going to write them down so you can play, too.

  • FTrace
  • Ftrace is the Linux kernel internal tracer that was included in the Linux kernel in 2.6.27. Although Ftrace is named after the function tracer it also includes many more functionalities. But the function tracer is the part of Ftrace that makes it unique as you can trace almost any function in the kernel and with dynamic Ftrace, it has no overhead when not enabled.

    It's deeply integrated by Brendon Gregg's Perf Tools, which is something else I want to get proficient with. Check out more info here.

  • Vdbench
  • A disk IO benchmarking tool, now by Oracle.

  • OIP
  • Super-sweet real-time network traffic visualizer that I saw in action at the LISA Labs. They actually tracked down a compromised machine while I stood there watching the cool animation.

  • Docker and Toys
  • Docker is a great interface for dealing with Linux Containers, and it makes it easy to spin up lightweight application deployment and more. There's an entire ecosystem of stuff to go with it, too.

    • Flannel
    • An etcd backed overlay network for containers

    • Pipework
    • SDN for Linux containers

    • Project Atomic
    • Somewhere between a collection of tools and a framework for managing Docker containers.

    • OSTree
    • OSTree is a tool for managing bootable, immutable, versioned filesystem trees. It is not a package system; nor is it a tool for managing full disk images. Instead, it sits between those levels, offering a blend of the advantages (and disadvantages) of both.

    • Kubernetes
    • Clustered container management. You can apparently live-migrate containers between clustered machines. Cool, right?

  • iRODS
  • Open-source object data store. Abstracts data services from data storage to facilitate executing services across heterogeneous, distributed storage systems

  • OVirt
  • Web-based Linux KVM administrative interface. Think vSphere, but for Linux. Kind of.

So there you go. There are undoubtedly more things that I heard of but have already forgotten - sorry about that. This should keep you and I both busy for the immediate future, though.

Go, play, have a blast. I know I'm going to. But first, I'm going to get some sleep.