Category Archives: Conferences

Big Changes at USENIX LISA in the last 5-10 Years

We received an interesting email recently:

> Did the submissions process for LISA change
> in recent years? I recall going to submit a talk a couple years ago
> and being really put off by the requirements for talks to be
> accompanied by a long paper, and be completely original and not
> previously presented elsewhere. Now it seems more in line with other
> industry conferences.

Yes, LISA is very different than it was years ago. If you haven’t attended LISA in a while, you may not realize how different it is!

The conference used to be focused on papers with a few select “invited talks”. A few years ago, the conference changed its focus to be great talks. LISA still accepts “original research” papers, but they’re just one track in a much larger conference and have a separate review process. In fact, the conference now publishes both a Call for Participation and a separate Call for Research Papers and Posters.

If LISA is now “talk-centric”, what kind of talks does it look for? Quoting from the Call for Participation, “We invite industry leaders to propose topics that demonstrate the present and future state of IT operations. [Talks should] inspire and motivate attendees to take actions that will positively impact their business operations.” LISA looks for a diverse mix of speakers, not just gender diversity, but newcomers and experienced speakers alike. We have special help for first time speakers, including assistant with rehearsals and other forms of mentoring.

What about the papers that LISA does publish? The papers have different criteria than talks. They should “describe new techniques, tools, theories, and inventions, and present case histories that extend our understanding of system and network administration.” Starting in 2014, the papers have been evaluated by a separate sub-committee of people with academic and research backgrounds. This has had an interesting side-effect: the overall quality of the papers has improved and become more research/forward-looking.

Because LISA mixes industry talks and research papers, attendees get to hear about new ideas along before they become mainstream. Researchers benefit by having the opportunity to network and get feedback from actual practitioners of system administration. This gives LISA a special something you don’t find anywhere else.

Another thing that makes LISA better is the “open access” policy. Posters, papers, and presentations are available online at no charge. This gives your work wider visibility, opening up the potential to have greater impact on our industry. Not all conferences do this, not even all non-profit conferences do this.

Does that make you more interested in submitting a proposal?

We hope it does!

All proposal submissions are due by April 17, 2015.

Tom Limoncelli and Matt Simmons
(volunteer content-recruiters for LISA ‘15)

P.S. LISA has a new mission statement:
LISA is the premier conference for IT operations, where systems engineers, operations professionals, and academic researchers share real-world knowledge about designing, building, and maintaining the critical systems of our interconnected world.

New Technologies to Study from #LISA14

Well, I’m just now ending my first LISA as part of the program committee. This has been a long, long process (over a year, actually!) and I’m exhausted, but really happy that we had an awesome team and made a great conference come together. We had the highest attendance at a LISA conference since 2007 (over 1,100 people attended), even though AWS ReInvent was scheduled for the same week.

I wanted to take a few moments to reflect on some of the most talked about technologies, to give me some things to work on in the next year.

I need to improve my metrics.
Last year, I dedicated myself to spending the next 12 months increasing my visibility into the infrastructure, and to getting graphite up and running and watching what happened throughout my organization. Well, I succeed, to some extent, but this conference was a reminder that I’m no where near done.

Starting with the Statistics for Ops and R for SysAdmins classes, then later in the week when Theo Schlossnagle presented The Math Behind Bad Behaviors, I began to understand that monitoring isn’t just pretty graphs and packet counts. In order to actually take full advantage of metrics, you have to have the right metrics. I’m getting what amounts to derivatives rather than the underlying measurements of the movements of my disks, network ports, and so on. I’m going to stop getting the 10,000ft view stats and start getting the ground level, base information.

Hadoop. Build one.
I have been hearing the word Hadoop for the past few years, but my sentiment on the subject was, “I don’t need MapReduce. I don’t even know what I would use it for. Why would I install Hadoop?”

Well, it turns out that I’m kind of dumb. Yes, MapReduce is a part of Hadoop, but it’s hardly the only part. In fact, there are a lot of solutions both included in Hadoop and tightly coupled to it, such as a distributed file system (HDFS), distributed databases (HBASE, HIVE), and really, a ton of other things.

One of the things that uses Hadoop is OpenTSDB, which is a time series database, similar to RRDtool or Whisper (from Graphite). It can write with millisecond precision, so it’s useful for monitoring low-latency IO (or anything else low latency, for that matter). Anyway, I want to start using it rather than graphite (partially because of the discrete resolution, and partially because it’s not a lossy roll-up like RRD or Whisper, where you store long-term metrics in a lower resolution than they were captured, which makes it practically impossible to do a lot of correlations that would be interesting to use, as I learn more about how to do that).

Once I have OpenTSDB running, I can work on getting a cool new tool running, called Bosun. Invented by my friend Kyle Brandt at Stack Exchange, Bosun is a bit like an IDE or maybe like a DSL for alerting. It runs on OpenTSDB on the back end, so there’s definitely a requirement change to make this work (although they have a docker instance to make it easier to learn, I’ve been told that you really don’t want to run that for your production instance because, in Kyle’s words, “You’ll have a bad time”.

Cool Tools
I ALWAYS come back from LISA with a list of cool tools that I want to check out. I usually forget about them. Now, I’m just going to write them down so you can play, too.

  • FTrace
  • Ftrace is the Linux kernel internal tracer that was included in the Linux kernel in 2.6.27. Although Ftrace is named after the function tracer it also includes many more functionalities. But the function tracer is the part of Ftrace that makes it unique as you can trace almost any function in the kernel and with dynamic Ftrace, it has no overhead when not enabled.

    It’s deeply integrated by Brendon Gregg’s Perf Tools, which is something else I want to get proficient with. Check out more info here.

  • Vdbench
  • A disk IO benchmarking tool, now by Oracle.

  • OIP
  • Super-sweet real-time network traffic visualizer that I saw in action at the LISA Labs. They actually tracked down a compromised machine while I stood there watching the cool animation.

  • Docker and Toys
  • Docker is a great interface for dealing with Linux Containers, and it makes it easy to spin up lightweight application deployment and more. There’s an entire ecosystem of stuff to go with it, too.

    • Flannel
    • An etcd backed overlay network for containers

    • Pipework
    • SDN for Linux containers

    • Project Atomic
    • Somewhere between a collection of tools and a framework for managing Docker containers.

    • OSTree
    • OSTree is a tool for managing bootable, immutable, versioned filesystem trees. It is not a package system; nor is it a tool for managing full disk images. Instead, it sits between those levels, offering a blend of the advantages (and disadvantages) of both.

    • Kubernetes
    • Clustered container management. You can apparently live-migrate containers between clustered machines. Cool, right?

  • iRODS
  • Open-source object data store. Abstracts data services from data storage to facilitate executing services across heterogeneous, distributed storage systems

  • OVirt
  • Web-based Linux KVM administrative interface. Think vSphere, but for Linux. Kind of.

So there you go. There are undoubtedly more things that I heard of but have already forgotten – sorry about that. This should keep you and I both busy for the immediate future, though.

Go, play, have a blast. I know I’m going to. But first, I’m going to get some sleep.

Checking things over before LISA

Tomorrow is my last day in the office for a week and a half, so I’m going through the various things that I manage, making sure that stuff is going to be alright while I’m away. You see, next week, I’m at LISA’14 in Seattle, and Wednesday afternoon, I’m flying to Chicago.

“But Matt”, you rightly say, “Chicago isn’t Seattle. I’ve seen a map. And that’s not close enough to walk, even if you wanted to”. Well, you make a good point. That’s why I’m not going to be walking. I’m going to be doing something much cooler. Well, “cool” depending on who you are, I suppose. I suspect you’ll think it’s cool, though, which is why I’m telling you.

Here’s my cross-country chariot:

The AmTrak Empire Builder

Yep, I am going to be the absolute envy of my five-year-old self. I’ve wanted to do this ever since I was a kid reading my fold-out train-shaped AmTrak propaganda.

The Empire Builder is a 2,000+ mile journey that crosses the Mississippi River, winds through the northern Great Plains, the Montana Rocky Mountains, Glacier National Park, and countless other interesting sights. The trip takes 44 hours, so my wife and I have reserved a “Roomette“, which is french for “closet with two beds”. But still, we can sleep laying down, so that’s good.

Apparently, we will have electricity, but there’s no internet on the cross country trains, so from my perspective, that’s great. I want some down-time to spend reading and writing anyway. My wife is taking part in NaNoWriMo, so I suspect she’ll feel the same way. So if you hit me up on twitter, don’t expect a reply until Saturday.

So now, back to work and the shoring up of the moving bits. Battaning down my particular hatches is much better than it used to be, since I’m part of a team now, but still, I’d feel bad if I stuck one of my coworkers with a faulty <anything> and left on a train for a few days.

By the way, if you’re going to LISA, I’ll be getting there on Saturday evening. Make sure to say hello if you see me!

(Oh, as an aside. Last night, I was a guest on the excellent VU-PaaS podcast with Gurusimran Khalsa, Chris Wahl, and Josh Atwell. Watch their website for when that goes live. I’ll post something here, of course, but in the meantime, they’v got 20-some episodes that you can listen to if you’re interested in virtualization. It’s a good use of your time, because they’re funny guys, and smart.)