January 21, 2007

S3 backups

Since Amazon's S3 web service seems to be everywhere, I thought I'd chime in with my experiences.

I'm a big fan of the concept of backing up data to an internet-based server. You have the joy of letting someone else deal with the hardware coupled with the safety of off-site backups. As an added bonus, often the data is available over the web, making files handy to access at work, for example.

So, I've been on the hunt for the perfect solution to back my data up to the cloud. At first, I used Xdrive, back in the days when I was backing up a windows box. At the time, I actually had trouble with the service crashing my machine. I'm not sure if was the service or my box, so your milage may very on this one.

Next, I moved to Joyent's Strongspace, using rsync for the backup. I actually had this working really well on my windows box. But, I only had four or five gigs of space available, so I wasn't backing up any of our audio files, so that wasn't ideal. (More capacity was, of course, available, but it got fairly pricey.) On the plus side, they have a great web-based file browser. Then, I moved back to the Mac, and I wasn't able to get the system to work quite right. Something about the SSH/rsync/cron combo just wasn't working right, and I never managed to get the system working automatically. Then, Joyent changed their pricing, and the whole thing started to look much less appealing.

Thus, back to the topic of S3. After reading a bunch of info on the web, I tried out JungleDisk. This handy little app basically lets you access your S3 account in the same way you would access any local network disk. I've been using IMSafe to do the automatic backups. IMSafe seems pretty easy to use, if lacking somewhat in flexibility (ie, there doesn't seem to be a way to exclude directories from a backup). And, I'm not 100% sure I have everything working automatically yet. But, it looks like I'm able to do backups. And, so far, backing up to S3 seems to be much, much cheaper than the other options I've seen.

I just noticed that JungleDisk has added automatic backups to the list of features, so I'll be checking that out, too.

Update: Oh, yeah, I forgot to tell you the big problem with net-based backups. Doing incremental backups is fine and dandy, but doing the initial full backup takes forever!