My tape drive was on the fritz (again), so I had lapsed in my daily backup strategy which I was previously anal about. Some of this was because I was terribly tired of the inadequacies of tape systems. They were expensive (for a small time setup like mine), slow as hell, too small (disk capacity is growing much faster than backup capacity), and error prone. I was also jealous of my mac.
On my mac system, I was using Retrospect and a 2 gig Jaz setup. With two disks I could backup my entire system in under an hour. I could also restore in under 30 minutes. A FULL restore, wiped disk and all. It was too easy. It was so easy that whenever I had ANY disk oddities, I’d just format and restore. I couldn’t do this on my unix system and I was jealous.
Then it struck. Lightening? No. Tiredness and stupidity.
Despite rsyncing the entire mass over to my OSX box, I succeeded in deleting a gig of photos that I had taken w/ my nifty digicam on BOTH SYSTEMS. If I had a backup this would be a no-brainer. But I didn’t. So I devised the following backup system and it works pretty well.
Basic Overview
I had the following requirements:
- It must be cheap.
- It must be fast.
- It must be working by midnight.
- It must allow for full and partial restores.
- I want two or more weeks of data.
- It need not be 100% failsafe, lightning and fires are rare enough.
This translated to:
- A 60 gig IDE drive
- A kernel rebuild w/ ATA drivers (previously only had SCSI)
- A script called ‘backup’
- A script called ‘restore-here’
- A crontab to automate it all.
backup
Backup takes an incremental number (like dump) and makes a backup of every partition mounted that doesn’t get filtered out by a regex. This allows me to add new disk and they will automatically get added to the backup. It stores all of it’s files on the big disk.
restore-here
restore-here does just that. It automates a restore of the current directory into a subdirectory called ‘restore’. Thus, if I nuke all the photos out of ~/photos, I do the following:
cd ~/photos
restore-here
mv restore/* .
rmdir restore
Wouldn’t that have been nice to have?
Details
backup
Backup is a bash script that simply wraps dump and does the right thing. It first processes the command line and global variables. The only command line argument used is the backup level, which it verifies is between 0 and 3 inclusive. More than that didn’t seem necessary to me for a disk based system.
Using the following set of variable definitions, it picks all of the partitions that exist minus the ones I don’t want:
BAD="(backup|obj|ports|mp3-.*)$"
DIR=$(mount -t ufs | cut -f 3 -d | egrep -v $BAD)
This is a much better strategy than listing the partitions you want to backup, because it becomes more error-prone as time goes on. By listing all mounted UFS partitions, we have a very flexible system. It might be better to simply list all partitions so it is more cross-platform, but I don’t want to backup mounted CDs or floppies, so we’d have tune this.
Besides this and the actual backup, the only other thing it does is communicate with users. It uses wall to tell current users that they might want to log out of the system and writes a message to /var/run/nologin to prevent new users from logging in.
Finally, for each partition it is backing up, it gets the top level directory name (‘/usr/home’ becomes ‘home’) and uses that for part of the filename. If the partition is the root partition (e.g. ‘/’), it names it ‘root’. It uses the following naming convention:
/backup/$HOST/backup.$YEAR-w$WEEK.$PARTITION.$DUMPLEVEL
The dump command it uses is not sophisticated:
dump -${level}ua -f $file $dir
restore-here
restore-here is a simple bash script that figures out what backups to use (using ‘df .’ to find out the partition you are on) and restores the current directory into a subdirectory called ‘restore-tmp’. In order to avoid requiring any input from the user, it invokes restore via perl’s Expect module to it can confirm the user input instead. Once it restores from all backup levels (verifying that it doesn’t do any subsequent levels that are older than the previous–see Thursday in the crontab for an example), it moves the potentially deep contents of restore-tmp to a directory called restore and cleans up.
Requirements:
- bash
- perl w/ Expect.pm installed.
- a big disk
Compatibility
The scripts were written for FreeBSD and should work on any other UNIX system with minor tweaks at most.
Code
TODO: I will be releasing the scripts via Download soon.
Crontab
# Backups: the backup script uses monday as beginning of week, so we do 0 then 0 4 * * 1 /root/Bin/backup 0 0 4 * * 2 /root/Bin/backup 1 0 4 * * 3 /root/Bin/backup 2 0 4 * * 4 /root/Bin/backup 3 0 4 * * 5 /root/Bin/backup 1 0 4 * * 6 /root/Bin/backup 2 0 4 * * 7 /root/Bin/backup 3 0 4 * * 1 cd /backup/greed.zenspider.com/; find . -mtime +7 -exec rm -v {} ;