Coding Horrors? No kidding!
December 12, 2009
If you read my blog, you probably know of Serverfault.com. It’s run by a couple of people, namely Joel Spolsky of Joel on Software fame, and Jeff Atwood, from Coding Horror. Both are very well known programmers with well-established blogs, books, fame, fortune, and lots of knowledge.
It seems that Coding Horror had a bit of an issue the other day, to the tune of roughly 100% data loss. Ouch.
As it turns out, they weren’t running offsite backups, and when their web host lost the data, it was nonrecoverable. shudder
So pretty much as soon as I read that, the thought occurred to me…”you know, I really need to script an off-site transfer of that nightly backup job I run on the blog…” so yeah, I manually backed up the most recent data and I’m going to work on scheduling the backup.
I don’t have anywhere to put it right now, but I’ve been wanting to play with Amazon’s cloud, so maybe it’s time for me to get an S3 account. The prices aren’t bad for the little bit I’d be sending, and as odd as it sounds, I’ve grown really attached to my blog and my readers. I don’t want to lose my data OR you guys who read what I write.
I’m also open to any non-Amazon solutions, if you know of something easy. Daily backups are ~50MB, if that matters.
Oh, and as for Jeff and Joel, they think they’ll be able to get 100% of their text back from various internet archives, but they’re missing the images. That’s definitely better than nothing, but still, I don’t want that to be me, and I know you don’t want it to be you.
Learn from the mistakes of others. Do your backups!
Posted in General
Email me
content rss
December 12th, 2009 at 1:07 pm
Matt, If you are looking for a linux solution I did a semi evaluation not so long ago on my blog and ended up going with spideroak for my home offsite backups, their client can run headless on servers as well and I’m paying 7/month for them for 100GB of storage. But it sounds like if you only have 50MB, theres no reason you couldn’t sync the data to a windows box each night and then use one of the more popular online backup solutions such as mozy.
December 12th, 2009 at 1:26 pm
I quite like rsync.net. Their owner talks sense.
December 12th, 2009 at 1:41 pm
Matt, maybe before I give my ideas of the matter on how you could do a backup, you could shed some light on how you access your blog. Do you have SSH? Only FTP or? I guess you would be backing up the htdocs plus the database.
December 12th, 2009 at 1:54 pm
I have a VPS that I backup to S3 using a combination of AutoMySqlBackup and Duplicity.
Here’s a link to the duplicity script at ServerFault: http://serverfault.com/questions/31828/duplicity-and-amazon-s3-script
December 12th, 2009 at 2:17 pm
So Jeff Atwood loses his entire blog history because of… an incredibly elementary mistake?! Sure, nobody is perfect, but a mistake like this is pretty good advertising for why you need sysadmins.
December 12th, 2009 at 2:32 pm
Your kidding me, coding horror? They didn’t have backups? I’m going to go backup my sites.
December 12th, 2009 at 4:15 pm
If you already got a local backup solution in place, scripting something to upload it to S3 is dead simple and the prices are super reasonable.
1GB of transfer in is about $0.10 and is currently free until March 30th.
1GB of transfer out is $0.17
1GB of storage for the month is $0.15.
Your Amazon bill for your backup should be less than $0.50 / month if you upload 50MB / day and keep 30 days worth of old backups.
To get the stuff up there, look into s3sync.rb. It’s a command line s3 client that behaves a lot like rsync. Very simple to use and put into a cron job.
We’re all a little guilty of not taking care of our backup needs, but getting the backups off the machines they live on is one of the most important things in the chain.
I think testing the backups we make is the one that gets left out the most.
December 12th, 2009 at 6:30 pm
Thanks for the reminder to back up my site again.
December 13th, 2009 at 10:18 am
You may want to consider . It is a very open system, clearly documented, and secure.
December 14th, 2009 at 6:57 am
I’m in the process of implementing/testing a rsnapshot based backup system of some personal sites hosted on a colo server to a local machine. Rsnapshot uses rsync under the hood for efficient transfers, with storage use kept under control by using hard links for unchanged files.
December 14th, 2009 at 11:29 pm
Jeff Atwood has been made fun of before for not being a sysadmin. He was posting information about his new stackoverflow servers and his backup strategy before. The tune of the comments was basically RAID is not backup, and backing up locally is useless.
I agree. Jeff is a programmer, and although he is in IT, it is foolish to think that he should have a full grasp of backup best practices and procedures.
It seems too many people, including system administrators do not do a thorough analysis of backup and recovery procedures as well as risks.
Step 1: Write down everything that can go wrong…fire? theft? user error? virus?
Step 2: Find out ways to combat all situations listed above…if some cover 2, great.
Step 3: Figure out your budget and fit as many of those situations into it…hope for all, but if not, have some sort of ranking in hand.
Budget is what kills almost all backup scenarios. Sure, backup and recovery would be easy with a redudant server with RAID, a 100mbps pipe to an offsite location with similar hardware there, etc etc.
With my own work, my Boss has been cheap on the storage for backups. We have many terabytes of data, and a very poor connection, so offsite backups are limited to the priority info. The rest is taken home daily on lto tapes…but what happens if my place burns down the same time the office does?
I do, in a sick sense, love when this stuff happens, because it allows the rest of us to scare our bosses into actually increasing the budget and getting a working backup solution.