Your backup/storage setup

[Mike Meier]

My RAID system reported a missing drive earlier this week and I spent four days since copying all of my files to a stack of 50 or so DVD, and that did not include the 1 TB of video and a 300 GB drive is use to store old stuff I can't quite delete, archive of downloads, patches, etc. It's been a chore, but at least now I have nearly all of my non-video work in off-line media. Thank goodness for RAID 1 because my existing backup system had just failed me completely.

Yes, I did have a backup system in place. I was using two external 1 TB USB drives, plus three external 500 GB USB drives. About a week ago the 1 TB drives started disappearing from my system, then reappearing, and when they came back up all the backed up data were gone. This wasn't the first time, either, but it had only happened to one drive, making me thing it was defective, but now both drivers were doing it. Googling, it looks like this is happening to a fair number of people. It could be the drives, or the USB interface in the drives, or the USB controller in the computer. I have also run into another problem with external USB drives, "Delayed Write Errors". This might be due to the USB card or to other windows settings and seems to be an issue when copying large files. To top all this off, a week or so before all this one of my three 500 GB drives quit working. I didn't have any data on it at the time.

So, USB is no longer a backup option I can trust. I'll use them to transfer/transport data, but not much else.

I looked into Blu-Ray and I could deal with the prices of the drives, but the media, especially to 50 GB media that I would use, cost more than an 80 GB SATA drive! Maybe I should just buy a case of 80 GB drives. Each one could hold 2 to 8 of my video projects.

So now I'm thinking of populating an old external SCSI case with 4 SATA hot-swappable drives. Problem now is, I've read a lot about how hot-swapping actually works, but I'm not finding much that I can use. The articles I have seen discuss the general features of hot-swapping, but hardly any of the nuts and bolts. For example, do I need a special controlled for hot-swapping, or a special tray that will deal with any voltage spikes, do my drives have to be RAID to also be hot-swappable, do I need a special power supply, etc.

So that's my story.

What have other people done?

[Tripp Woelfel]

Wow. It certainly is Monday where you live.

Let's bread it down by the specific issues. Firstly, it's important to note that fault tolerance (RAID) and backups are two different issues. RAID gives you maximum data availability. Backup is data protection. It's clear that your processes have that in mind but be careful not to try in mix them as you work your way through this.

Figuring out why the drive disappeared from your RAID might be the most important consideration to your short-term productivity. Are you using "green" drives? Someone recently reported problems with a RAID using green drives. Replacing them with conventional drives resolved the problem. The concept of a green drive really works at cross purposes with fault tolerant RAID.

You might want to play with the "missing" RAID drive by trying to directly connecting it to your system to see if it works and perform some diagnostics on it. If it tests OK, suspect a cable issue for starters. RAIDs can be complex devices so the failure might not be the fault of the drive.

The USB drive issue is baffling. Although I haven't researched this much, it seems they have been pretty reliable overall. I had one disappear on my editing system last week so I moved it to another machine using the same power supply and USB cable and it still wasn't right. I replaced the USB cable and it's all been good since then. I also had some data corruption issues last year copying video files from one USB drive to another. Again, the problem was traced to dodgy cables. The moral to this story is that good, solid and robust cables are key.

If your USB drives have in fact failed, it could be a USB interface problem and the data is still on the disk. You could extract the drive from the enclosure and install it into your machine or another case, depending upon whether the HDA is SATA, PATA or EIDE.

Finally, it's hard to comment on hot swapping drives in your RAID without knowing a lot more about the manufacturer's operating instructions for the unit. A RAID specifically designed for fault tolerant RAID "levels" (1, 3, 5, 10, etc.) are usually designed for hot swapping drives. They wouldn't be much good for data availability if you had to take them down for drive replacement. A non-fault tolerant RAID (0) may not support hot swapping since a drive failure would be fatal anyway. Check the instructions for your unit for their recommendations. If that's not enough, their tech support would be your next, best stop.

I offer this advice from over a decade working in RAID, disk drives and storage software. I hope it helps.

[Jeff Harper]

I personally do not use raid any longer.

But I am a fanatic for backing up. I use all 1TB drives for storage and I plenty of them. Reviews on the new Seagate 1.5 TB have been very mixed at newegg, or I would have those as well. Seagate is going to hell in a handbasket, IMO. I used to use only thier SCSI drives, but their SATA drive quality is spotty in my experience.

I was fed up with most external solutions for many reasons. Cable, physcial space, etc. it was all too much. The external boxes seem fine, but with my 64 bit OS I couldn't be sure to find compatibility with the chips in those boxes and my adaptec e-sata controller card.

So I went with a large tower that holds lots of drives, and I still use a couple of external esata enclosures.

I download video, then immediately copy it to another drive. I then create project folders on both drives, and when saving my project work I save it twice, once to each drive. Extremely simple. Extremely effective.

I use fast drives and I find 12GB of video usually copies in about 20 minutes. I am particularly fond of the WD Blacks. Really nice drives.

I was a RAID fanatic until I got tired of it. It was just all too much. When a technology or tool gets in my way as much as it helps me, it's time to be done with it. I know there is nothing wrong with RAID, and many use it happily. So I am NOT knocking it. But my last stint was marred by the unreliable controllers built into my last two motherboards, and I simply couldn't see buying an expensive external controller, hence my current method of backup.

I have full copies of everything without having to deal with RAID, and I much prefer it.

I have seen, on occasion, several posts here and there by people who have also stopped using raid, and in the past I used to think to myself "they must not know how to use it", or they are idiots, etc. I now feel the same way they do. It is simply another layer of technology that someone like me (small, one-man shop doesn't need).

For some it may be the perfect solution. It is used by huge corporations on servers, etc., and has a hugely important place. But not for me.

[Brian Standing]

Sorry, man. All hard drives WILL eventually fail. In my (limited) experience, using a RAID sometimes hastens this failure rate. It also seems that the hard drives I have sitting around on a shelf fail more quickly than the ones that are continuously in use. Also, the 3.5" drives seem to be more robust than the laptop and smaller drives. For stuff I need immediate access to, I have a variety of non-RAID SATA drives installed in my desktop.

If you want permanent, archival storage of data, however, you've got to use a stable media. Hard disks just won't cut it. For all my digital masters, project files and batch capture files, I'm backing them up to LTO tape. I found a reconditioned Ultrium-2 LTO-2 internal tape drive and an internal SCSI card for $500. 200gb uncompressed storage per tape cartridge for $29 each. I use the free ntbackup utility that comes on my Windows XP install disks. I figure since LTO is an open-source standard, the hardware is abundant and there must be millions of copies of ntbackup around, the system is less likely to become obsolete. Data tape is proven as a long-term storage medium and shouldn't degrade like cheap DVDs do. Blu-Ray? The media is too expensive per GB, and who knows how long it will last.

I have also not yet gone tapeless, so I also keep all my camera-original HDV tapes in storage as well. This gives me some kind of redundancy in case one of the data tapes doesn't work. Hasn't happened yet. Knock on wood.

[Mike Meier]

Thanks for the feedback.

I have RAID for up time and to reduce the chances I'll lose data, plus I was also doing backups, even though these backups were to another set of spinning disks. And even though it took days I am glad I now have much on my work on DVDs, in a box, in the closet. Gosh, now I guess I should duplicate them and store a copy in another location. Creeping paranoia...

I do keep all of my original DV tapes, which has been suggested.

I had not heard that RAID affects the drive's lifespan, or that letting a drive sit on the shelf did the same. I know that not turning on a computer for years can cause problems, mostly due to degraded electrolytic capacitors. There are few, if any, of these on hard drives. Maybe the bearings go bad if the disks are not spinning?

I'll be going with hot-swappable eSata drives for now, in an external box, ala Wiebe Tech, but in a cheaper enclosure. I guess hard drives are now the new floppy disk. JOBD.

I did look into tape backup and the prices were much too high for my budget. A deal on a used one sounds good.

Too bad that the 50GB Blu-Ray media is so expensive.

Edit - I managed to get my system to see the missing RAID drive. After I opened the case and blew out the dust I saw that a cable was loose. Odd that this happened at the same time that Windows did an early morning automatic update. When I came in in the morning the computer had rebooted and a drive was missing. Coincidence? Yes, as it turned out. Anyway, I didn't lose any data, just 5 days. (Just 5 days!)

[Andy Mace]

I also believe in tape. just a quick warning about using ntbackup. I've had it refuse to restore "good" backup's. Seems that it is likely to fail when the SSID and other things like machine architecture don't match the original system. When that happens, it's very, very, very disappointing to say the least. There are vendors out there that will sell you software that will read ntbackup tapes but that's another $100 piece of software to invest in.

Stay away from ntbackup. I don't have an alternative to recommend as I use Linux as a server and archival system.

[Brian Standing]

I think the hot-swappable SATA drives idea is a good one. You may want to fire them up periodically to make sure they're in good working order. Since all hard drives have multiple rapidly moving parts, I'm sure your theory about the bearings failing on idle drives is a good one. I can't explain the apparent higher failure rate of drives in a RAID, but then again, it may have just been bad luck on my part.

LTO is now up to LTO-4, which, of course means that the older LTO-2 gear is cheaper. I've seen brand new LTO-2 drives on sale online for under $1000. That's still cheaper than, say, any kind of HD video deck or XDCAM drive, and WAY cheaper per gigabyte of media cost than Blu-Ray. Hunt around, like I did, and you may be able to find a reconditioned one even cheaper.

The LTO standard requires that all hardware be able to read tapes that are at least 2 generations old. This means that all brand new LTO-4 drives should be able to read LTO-2 tapes (but not necessarily LTO-1). Since the archival lifespan of LTO tapes is supposed to be 15-30 years, I figure by the time my used drive wears out, I'll be able to get by staying 1 or 2 generations behind the current standard.

Meanwhile the 200gb per tape works well for the projects I'm doing at the moment, and the drive seems pretty robust and fairly fast. I'm breathing a bit easier at night.

If you're still using standard-definition DV, you have a big advantage in that you are working with a lossless codec. If you print a master (or 2 or 3) to DV tape, you'll have a digitally perfect copy on a robust, archival medium at extremely low cost. You may also want to print another master with all titles, background music and voiceover turned off, in case you ever need to re-edit. This was how I archived all my standard-definition material, and I've never lost anything yet. It was only after I switched to HDV, with a lossy MPEG-2 codec, that I thought it worthwhile to invest in a tape backup system.

[Tripp Woelfel]

Originally Posted by Mike Meier

I have RAID for up time and to reduce the chances I'll lose data.

RAIDs are not protection against losing data. That's what backups are for. Fault tolerant RAIDs are for providing non-stop access to data.

If you have a single failure in a fault tolerant RAID, the data will still be on-line. If you lose a second drive in that RAID your data is gone, pfffffft, see ya. And it will be a big hit with most people using multiple .5TB drives or larger. You may think that I'm flogging the expired equine but this is a very important point.

BTW: There are RAID "levels" that can survive multiple drive failures but those are very dear.

[Tripp Woelfel]

Originally Posted by Brian Standing

In my (limited) experience, using a RAID sometimes hastens this failure rate. It also seems that the hard drives I have sitting around on a shelf fail more quickly than the ones that are continuously in use.

The first experience is probably, unfortunately, unique to you. A drive that's part of a RAID will not necessarily fail faster, or slower for that matter, than a non-RAID drive. The drive neither knows or cares whether it's part of a RAID or not. It's just a very fancy box that holds data.

RAIDs spread the workload more or less evenly across all of the drives in a RAID 1, 0, 5 or 10. (This is not true for RAID 3 and others with dedicated parity drives.) This means that if your RAID has drives from the same manufacturer that are of the same age, when one drive fails you can expect the others to follow suit. This doesn't mean that they will, but statistical data supports this. This is why users of industrial strength RAIDs (think EMC), will replace apparently good drives after a certain amount of hours.

There are two key reasons why drives stored on a shelf somewhere will suffer a premature death. First, electronic components like to be used. That disuse is bad is simply an axiom. The second reason is due to something called "stiction". When in use, a disk's heads float on an extremely thin cushion of air. Heads like this. When powered down, the head arm will move off the data area of the disk to something that is called the "landing zone". There is no air cushion so the heads can, and often do, come into contact with coating on the platters. Over time, and it could be as little as a few months, a head in contact with the platter's coating can bond to it. Not like Gorilla Glue but enough to play havoc with the head. If a small fragment of the coating sticks to the head and the head drags it across the data area of the platter you can have a head crash and you're dead. When you're dead, you stay dead.

I don't mean to be pedantic about this but all of this stuff is important to understand if you are going to put together an intelligent storage strategy. I wish I could remember the name of the company that Charlie McConathy had before it went out of business not too long ago but his people probably had the best overall handle on best practices with storage for video. They were competitors to several companies I worked for and we couldn't compete with their "cred" and longevity in the market.

[Allan Black]

Thanks Tripp, useful info; and I understand Firewire is for video files not USB.

I'm running one external WD 2TB Raid 1 drive at FW800 for master archival work and 2 WD 500Gb drives for music and fx etc. WinXP pro. Vegas.

I keep a small continuous fan going to keep the air moving around all drives. As heat kills drives, if a WD fan comes on, I up the small fan rate. If ever a 2nd stage WD fan started up, I'd power down that drive.

After 18months the only problem so far one of the 2TB drives had one corrupt file. When I fired up the rig WD RAID took total control and copied over the whole of the good drive. It took 22 hours!!! Just using Windows 'checking for errors' takes 6hrs.

The 2TB RAID rig now has 380GB (760 total) on it and I back that up to a third WB 500GB drive.

My question is, does anyone know which is quicker, reformat this third drive each time or just WD backup the latest files from the 2TB drive. Thanks.

Cheers.

[Tripp Woelfel]

Originally Posted by Allan Black

My question is, does anyone know which is quicker, reformat this third drive each time or just WD backup the latest files from the 2TB drive.

I'm not sure I understand what you want to do. If you want to backup the data from the RAID to the external drive you can just copy the data from the RAID to the drive using whatever backup tool you use. You should be able to do an incremental backup that would backup only the files changed since the last full backup. That should be quicker than reformatting.

I cannot figure out why you want to reformat the drive. Unless you do a full format, which would take an hour or so on a 500GB drive, there's no benefit to reformatting. It will just rewrite the directory structure and blow away your files. You can do the same thing by dragging all the files to trash.

[Allan Black]

Originally Posted by Tripp Woelfel

You should be able to do an incremental backup that would backup only the files changed since the last full backup. That should be quicker than reformatting.

Thanks, that was the question, which is quicker, incremental or a full reformat.

As the 2TB drive gets fuller with more archival material, incremental backup to the 500GB drive is taking longer and longer; versus just full reformatting that drive and backing up everything from scratch each time. There's nothing else on it.

Cheers.

[Jim Andrada]

Re stiction: I believe all disk drives in common use unload the heads (ie actually lift the heads off the surface) when the drive is powered down. Not 100% sure, but I think so. Earlier drives had a textured zone near the center of the platter where the heads were parked. On 2 1/2 inch drives using glass substrate, they actually hit the surface with a laser in a tight dot pattern - the laser slightly melted each dot and the surface tension of the molten glass caused it to form a tiny bump.

By parking the heads on the textured area, the surface lubricant used on the platters wouldn't flow over the head and bond it to the surface of the platter. I have a few of the glass platters lying around here somewhere - amazingly tough stuff. You could sling the little disks onto a concrete floor so hard they would bounce up almost to the ceiling without breaking. Hoya (same people who make lens filters) was probably the biggest maker of these disks. Asahi Flat Glass kept trying to make inroads into the business, but never really overtook Hoya. Not sure about the current market shares as I stopped paying attention to disks a few years back when Hitachi bought out the IBM disk business.

Stiction in the old 14" diameter mainframe disks wasn't much weaker than Gorilla Glue. I've seen heads literally pulled off the suspensions when the machine was started up and the actuator tried to move the heads. We actually at one point implemented firmware that vibrated the actuator for a few seconds at startup to break the heads loose from the platter. Sounded awful but it worked - usually!

Disk drives have (or at least had) shelf lives that are shorter than their operating life. All kinds of bad things happen to a disk drive if it's left sitting around too long. I wouldn't think of a shelf full of disk drives as a secure long-term archive. Tape is engineered to sit around for decades, disks aren't.

LTO Generation 5 is on track to announce around year end 09 with 1.6 TB per cartridge native capacity, and Generation 6 (3.2TB) should be coming along around the end of 2011. After that, what happens next is not quite clear, as the official LTO consortium roadmap only runs through Gen 6 at present.

I believe that Quantum has implemented some form of separate partition support on their LTO tape drives that should make it possible to mount a file system on the cartridge which ought to support drag and drop of files to tape. I believe they call their drives "MXF Aware" as they have some capability to support the MXF metadata structure for video. Not too familiar with it personally.

[Brian Standing]

Originally Posted by Andy Mace

I also believe in tape. just a quick warning about using ntbackup. I've had it refuse to restore "good" backup's. Seems that it is likely to fail when the SSID and other things like machine architecture don't match the original system.

I have some reservations about ntbackup, too. But it's hard to find stand-alone, non-networked, affordable Win32 backup programs that recognize tape drives.

Tell me this, though. When you had trouble with restoring "good" backups, were you restoring a system/applications, or were you just restoring data? I've heard other horror stories with ntbackup, but most of them seemed to involve restoring a whole computer, or an operating system and software, not just files.

I'm hoping if I limit this to just saving and restoring relatively inert video files back and forth to tape, I can avoid some of these issues. I don't really understand what machine architecture would have to do with straight data. Did you try the "restore to new location" option?

Am I deluding myself?