Backups are something I don't do well. I hope to write a more thorough article about them at some point. For now, here are some tips.

Like most home users, I don't have access to a tape drive or jukebox for backups. I'm therefore looking for a solution that backs up to a local block device. I haven't ruled out a solution that backs up to DVD-Rs, but I haven't found a suitable one.

I first tried rsnapshot, which uses hard link trees to represent increments. I found that this resulted in enormous filesystem metadata, sufficient to cause the machine I was running it on (an embedded ARM system) to slow to a crawl. I'd recommend to avoid anything using this technique.

I more recently have been using rdiff-backup. This has appeared to work quite well for a number of years. However, it does not gracefully handle the backup volume being full, spewing python backtraces. This is compounded with some confusing use of temporary directies and not honouring $TMPDIR. This has been known about for years (1, 2). I've just independently discovered these problems for the second time. Aside from that, because it does not de-duplicate files, it is sensitive to large files being moved around in the source being backed up. It also cannot do a dry-run estimate of the disk space required for the next increment. So: time to move on.

Back in 2010 I discovered bup, an intriguing backup tool that used a git-style backing store. It's interesting enough that I packaged it and use it in a few situations, but I wouldn't rely on it for my main backups, for two reasons: one, it's not tried-and-tested enough yet (but that is rapidly being resolved), two: you can't get rid of old increments, so your backup volume will always increase in size over time, and never throw away old data. So, for now: move on.

Next on my list to try is Lars Wirzenius' obnam.


Comments

comment 1

I just went through this same process. I delved into bup hardcore, because I happened to be digging into git-annex. The deduplication really does work well, but I agree - its too greedy for general purpose backups.

Obnam looks great for that exact reason - it can purge old copies, but the author states its ready for evaluation, but not yet prime time.

I never tried rdiff-backup, but it looks great.

In the end, I opted for tar -czf and s3cmd. :-)

Oh I also use etckeeper and backup-manager for local backups.

openid [www.openid.albertlash.com]
comment 2
I find boxbackup pretty good. It sits in the background, slowly backing up stuff incrementally. The server needs to run boxbackup but otherwise doesn't need to be trusted, because everything is encrypted. It might want to (send)mail you in case something goes wrong, but it shouldn't be too hard to setup something that either notifies you locally or forwards the mail somewhere accessible. (And cron wouldn't be that different.)
Philipp Kern