data backup strategy
ever stop to think of the probable result if your source drive were to fail during a backup that overwrites or appends the previous backup file? in one nutsack shriveling moment your original data and its prior safety copy have been corrupted or trashed — during the same process initiated with the intention of preventing data loss. it’s a weakness of many a backup plan.
my backup goal is to never rely on hauling my hard drive(s) in a panic to some data recovery guy who speaks only hexadecimal and cash. though generally a disorganized person, i rarely get behind the curve on data backups. has saved my lunch a couple of times. until recently, however, i was knowingly taking chances during the backup, using only one slot for the destination. could’ve bitten me badly — a risk shared by all who back up to single files and don’t allot at least double the space for the solid practice of keeping the latest backup intact even during the next backup.
say you have a laptop with two 80 GB drives — one primary and the other purely backup. since you keep ~15% of the primary drive free for defragging and other optimization, that leaves plenty of room on the other for both full and sizable differential uncompressed backups. then when the differential gets too big, you redo the full.
of course that leaves you dead in the water if someone steals your laptop (effective backup plans usually require at least a local backup and an offsite, to protect in case of theft, fire, or other loss), but local backups are a good start, and — logistics depending — can be copied to drives for offsite storage instead of doing another backup from the primary source. i’m talking about the potential for immediate data loss, while you’re backing up.
chances are damned slim for major failure of the source drive while overwriting a good backup file, but think of the potential consequences. if your data’s important, you should generally never overwrite your most recent unique backup file. that means, assuming no compression, a packed 80 GB drive will require roughly 160 GB for a single reliable backup plan. and if the offsite backup isn’t copied directly from a fresh backup, that method as well may require double the space. the same applies for differential backups — often the most critical, being purely recently modified files. with the low cost of hard drives, it’s a good time to reassess old backup plans.
large businesses have long used rotating backups. the last joint where i worked backed up everything at least nightly, to tapes that weren’t overwritten for a week or two. the database server backed up changes throughout the day, and that was on top of the usual server deal of being able to yank any single drive from an array, toss it out the window, and still go on ticking like nothing happened (with the ultimate coolness of slapping in a fresh replacement drive and letting it build up the redundancy automatically while still using the server normally).
but personal systems are usually far off that mark (often for good reason). wasn’t until recently that i’d even think of backing up my entire drive, including windows system state. my old method was based on a batch file that would PKZip critical files super tight then send them to multiple drives (one a RAID 1 setup inside a fire-resistant gun safe). when i had my main hard drive crash around 2000, everything i needed was zipped onto a 128 MB compact flash card kept with my pocket PC. now that i have tons of video and audio files, that won’t cut it.
my latest tactic is to set up on some hard drives folders that rarely or never change. that way i don’t whir drives repeatedly backing up the same crap. my backups flow from my laptop to an external drive, then from that to another external, and then one more, content depending. for each drive i’ve made A/B pairs of batch files for full and differential backups (using windows backup, which i think is a pretty good stock utility), stored in each backup destination folder. with the drive online, i check the backup dates and then use the correct batch file to overwrite the oldest copy. each backup location then has 8 files in regular practice:
- full batch file A
- full batch file B
- differential batch file A
- differential batch file B
- full backup A
- full backup B
- differential backup A
- differential backup B
yeah, that could all be programmed easily to avoid at least the manual bit of searching for the oldest backup file to overwrite, but since my external drives are often disconnected and my data use unpredictable, i prefer to monitor and control it manually.
blah blah blah. just remember that though you may back up data regularly, each backup puts you at risk of total practical failure if you’re not writing to a new location. this is a classic gotcha making the seemingly fastidious potentially worse off than the guy who only backs up once every six months. charles lindbergh had it right when he banked on a single-engine plane for his crossing of the atlantic, recognizing that multiple engines multiplied the risk of engine failure. he couldn’t take multiple planes, but with the drop in hard drive prices you can back up your data to rotating file slots.