Data backup - how boring!

[John McCully]

The time has come, the walrus said…to take data backup seriously. I shoot HD, photographs, write…and all this gets more or less backed up when I get around to it, mostly by burning DVD’s (time consuming) and to external hard drives (which are hugely affordable these days) but at any given time I probably have a single copy of data that I cannot afford to loose because of a drive failure. It did happen; I didn’t loose a lot of data but it was an (unnecessary) annoyance. It could happen again, it will happen again, and before it does I need to get ahead of the game.

I’m not totally stupid; mission expensive data like helicopter shoots gets backed up to another drive that day before the very first sip of froth-topped dark-brown nectar; well, sometimes with it. But I do need to get things tidy on a more formal basis, automated at that. Not to do so is asking for trouble, and given the cost of storage these days there simply is no excuse!

I noted the Videoguys recommendation regarding the G-Technology products. Seems like an elegant solution. I also note that a Seagate 1TB External Hard Drive can be had at my local Dick Smith store for $228.00 NZ dollars. Hardly elegant, but not a lot of money.

I’m more of an multi-media artist than a hard core professional videographer so money is limited.

So what do I do?

I should add that my computer is a 2-year-old Dell Precision M90 laptop, core 2 duo with an express card slot and of course many usb 2 ports.

I need a backup strategy that considers the current cost realities and my laziness when it comes to this subject. I prefer doing other things, and I'm quite undisciplined, so you know what happens.

Recommendations gratefully received.

Cheers

John

[Andrew Smith]

Western Digital "mirror edition" external hard drives. Good insurance against spindle death (a physical hard drive failure) as there are two physical drives in parallel in the unit.

Backing up can be dull, but it's not as exciting as having to re-do the work again.

Andrew

[Giroud Francois]

backup is not hot about "where" you do it, but more about "how" and "when".
Currently the best way is not doing backups, but have a workflow that embed the feature of having redundant data at different place.
you would need some shared storage on network and programs like 2nd copy, that would automatically duplicate every file you work on , in a different place.
For example, you can share a network drive from a PC that is always on as the data drive.
each other machine that needs to keep data on its own disk will have at least a copy there.
for other data you do not need to travel with, they just are on the shared drive.
If you are really paranoid, you can have that shared drive use some RAID drives, and have it automatically backuped to another drive using 2nd copy. This one will be setup to keep the last 3 versions of the files.
you also can setup a server that use the "shadow copy" feature of microsoft server and XP.
it is the same than 2nd copy feature, even easier to use.
you would have no action to take to make sure your files are safe.

[Jon Shohet]

Presonally I wouldn't bother with mirrored drives. Better to just buy two regular seperate drives and have two physical backups (stored in two different locations if you want to be even safer). Super fancy enclosures have little to no benefit IMO in terms of backup. FW will make very little difference over usb for backup, and is usally more expensive. eSata will speed up transfers and is a good idea IMO.
One thing I noticed is that prebuilt enclosers from WD, Seagate, Lacie and so on, tend to get quite hot during use. They are often plasticy POS, with little thought towards cooling. If you buy a regular internal HD and, sepertely, an encloser with a fan/aluminium case, it would keep the drive cooler, arguably prolonging it's health.
Possibly a better solution would be a raid/NAS multi-disk enclosure, but they are much more expensive - I have lately been toying with the idea of getting one, but I still feel I have more urgent expenses, and - like you - a limited budget, so I can't offer any advice on that front.

Most prebuilt drives come with automic backup software, which is fine, I guess, but very basic. I would recommend investing a small some and getting a more powerfull backup software. This is a great remedy for the laziness/tediousness problem. Once setup correctly, you get more or less automatic backups, which at the same time you know you can rely on
there are two general approaches for backup:
- disk cloning/sector by sector. "Acronis" is a great software. Pros are it's very powerfull but very easy and quick to use. Cons are backups are stored in propreity format and can only be restored on the computer the software's installed on.
- sync/backup software. I personally really like and use a program called "SyncBack Pro", but there are many other options. Pros are it's even more versatile, and backups are regular files or zip files which can be accessed on any computer. cons are it takes initially a bit more time to set up and fiddle with.

Finally, for super critical stuff, you can also consider an additional backup to online storage/ftp, or even a free email server.

cheers:)

[Andrew Smith]

Even on slashdot, we get in to all sorts of arguments over what is the best media and strategy for backup and archiving.

Being able to store in two physically different places is definitely a good thing.

Andrew

[Oren Arieli]

The elephant in the room is the march of technology. How long before SATA hard drives get replaced with the latest & greatest in liquid-optical, nano-spectacle, digi-quantum tech? (if anyone decides to use these terms, please see my lawyer first ;-)
Do you remember floppy disks? How long were these the standard? As new formats take over, older formats soon become obsolete and the hardware increasingly difficult to find. If I had one of my old floppy drives, I doubt I could retrieve the information (even with the proper hardware). Your operating system has also evolved during that time.
So for me, there are no perfect 10 year or 20 year solutions. Short term is all we can hope for at this point and you'll still have to migrate the data every few years.

I've gotta' hand it to the lowly printed word, which has survived hundreds of years as a data storage tool.

[Pete Bauer]

Oren, I totally agree. It would be silly for someone to buy, say, a home server and think it was the last investment in their data storage they were ever going to make.

However, I'm very happy to know that I can make as many EXACT copies of my archived source files as I want (for me, at LEAST 2 copies stored in physically different places) and that whenever the time comes to replace storage hardware, I can easily just copy to the latest, greatest medium. As long as I do my part in maintaining my archives, the probability of losing my source files is tiny.

At some long time in the future when a standard file format like AVI or M2T becomes long in the tooth, I, or one of my heirs, will do a one time file conversion that'll be visually or probably mathematically lossless. No need to worry about deterioration of the original, ancient regular 8mm film reels or the availability and functionality of a projector, or if that 30 year old camcorder will still work. This applies to industry such as TV networks, too. They certainly hang on to hardware and imaging standards/formats a lot longer than consumers for economic reasons, but eventually that which is worth saving will have to be digitized. Heck, the movie industry has even gone a step further and remastered old B&W movies in color...for economic reasons I would imagine! ;-)

Paper was the only game in town for centuries and isn't going to disappear anytime soon - despite its high susceptibility to loss. Important documents survive through great care or dumb luck, but most copies of most printed material more than a few years old are no longer with us. I'll bet my digitized home movies have a better likelihood of being around in 100 or 200 years than does something printed in 1809 or even 1909.

However...I have to admit I'm not concerned about whether in 2 millenia they are lost to history, as were the entire half million or more works of knowledge in the Great Library of Alexandria.
:-)

Edit: Oh, and to answer John's original post...my method is probably not the most high end but as I said above, I feel pretty comfortable that my data are at low risk. On my plain old wired gigabit network, I have a D-Link 343 device (it's a 4-drive "bring-your-own-HDD" device that, like most similar ones, runs on a lite version of Linux). I have a separate 2-HDD server for everyday file use on the network, so the D-Link really only gets accessed for archiving purposes, or to grab copies of files off it to do a project on my editing box. Whenever I add files, I do a backup to one of two portable drives. The other portable drive is off-site and they swap locations after each update. A more sophistocated option would be to do the backups over the 'net, but I just personally haven't felt the need or desire to set that up. I never erase an original tape (my days with tape are nearing an end, though), and I don't clear CF or SD cards until I have the files copied to two locations.

[Ron Evans]

I have a Quantum LTo3HH tape drive and use EMC Retrospect. 400G (native) tapes cost $25 and its what the data industry uses for serious backup. It will backup or restore as fast as my hard drives can run. I just keep one copy on a hard drive for quick access of current projects and bare drives in a dock for past projects( using up the many hard drives I had before getting the tape drive!!!). Compared to the cost of the cameras and computers this is not an expensive option even for me as a serious hobby. This also simplifies the computer configuration too, no need for complex RAIDs just what is needed to edit.

Ron Evans

[James J. Lee]

I've been using this Voyager HDD dock NewerTech® - Voyager Q “Quad Interface” FireWire® 800/400/USB 2.0/eSATA - SATA I/II Hard Drive Docking Solution to swap out 3.5" HDDs like their DVDs or tapes and saving to multiple drives. TB drives w/out enclosures are relatively cheap, and I can swap them out quickly. In some ways it's a low end/low tech solution, but it's been working for me.

[Giroud Francois]

ok , you will all list your favorite disks and backup programs, so we go for a long thread.
but you missed the point. You still are making backup, and your data are as safe as you will doing it (or not doing it, and we all know the the second possibility is the likely one).
so just arrange your every day life so you do not need to DO backup, not even think about doing it. Just use technology as it should be , transparent.
there are so many ways to use data and just make sure that nothing will be lost or damaged.

[Jon Shohet]

How do you store the bare drives when they are not in use? do you feel it is a safe solution for long term archiving, or do you use the bare drives just for backup/work and keep archived stuff on drives in regular enclosures?

cheers

[John McCully]

Thanks for the discussion, folks. Most helpful.

I did pop down to Dick Smith. Picked up a 1 terabyte Seagate external drive. $228.00 NZ (peanuts). The salesman tells me they are selling like hotcakes. I open the box. Very basic; just a hard drive encased in a close-fitting shell of metal (I think) and plastic. I look for the ventilation ports. They really don’t exist. How many times have I read that heat is the enemy of hard drives, cooling is a must and if the case doesn’t have a fan then forget it; it will fail, soon. Not only does this drive not have a fan but the ventilation openings are minimal, for pressure equalization only perhaps.

I plug it in. I’m not sure it’s working. I can’t hear it. I touch the plastic top, and yes; I feel a faint hum. I’ve got it set up on a shelf near a fan for when it gets hot, as I know it will.

I work away, moving data onto the drive, and after an hour I check and no, it is not hot yet. Another hour, not hot. All day, not hot. So how does it go as a frying pan. Great, if you like your eggs raw, and only ever so slightly warm.

How quickly can I move data on and off the drive? USB 2 quick, and for my workflow that’s fast enough. I played an mxf file (1920 x 1080) off the drive, and sweet, plenty fast enough.

How delightful; we have obviously entered the era of inexpensive fast whisper-quiet cool-as-a-cucumber hard drives that we can string around all over the place. With a USB hub and a decent backup software solution it’s plain sailing!

And you know just around the corner: lighting-fast solid state mega-terabyte storage not much bigger than a matchbox (remember matchboxes?) are waiting in the wings.

So now the questions is software. Thanks Jon, I shall look at SyncBack Pro. I really don’t mind paying for an excellent software solution now that hard drives have got to where they are. That’s now my focus. Again, recommendations gratefully received.

And Pete, I love your point that ‘I have to admit I'm not concerned about whether in 2 millennia they are lost to history, as were the entire half million or more works of knowledge in the Great Library of Alexandria’, and you smile. It is an interesting thought that if I make a decent job of backing up my data it will indeed contribute (works of knowledge!) more to some abstract notion of the advance of the human condition (as if that matters) than the lost works of the Great Library of Alexandria, I say smilingly with a tear falling down my cheek, though I’m open to debate on the matter. Existence trumps non-existence, might be a point of departure for such a discussion.

Maybe it really doesn’t matter. But for now I like what I’ve done, mostly. Having a copy available forever is an interesting thought, but more important for me here and tomorrow is a huge cool hard drive, dozens of them, with everything I’ve touched squirreled away and readily available.

Just checked again; the el-cheapo plugged-in Seagate 1 terabyte external is still barely warm to the touch.

Cheers

John

[Nicholas de Kock]

Get yourself a Drobo!
Short & sweet.

PS: Make sure your Seagate firmware is 7200.12 otherwise your drive will crash for no reason due to 7200.11 firmware issues. Between me & you stand alone hard drives are not great back-up devices. I lost another 1TB today, they die for no reason.