Very basic question: is a RAID setup the only option for backing up lots of HD footag

[Noam Osband]

I'm going to make the leap from buying enclosed external hard drives to creating a setup that in the long run will be cheaper and easier for saving the large amount of data I'm accumulating by working in HD.

I know nothing about this topic and I've been trying to teach myself what to do going forward. So, this might sound painfully obvious but help me out: is a RAID setup the only option here? Correct me if I'm wrong, but a RAID setup seems like, essentially, a game plan for backing stuff up and then I just need to decide what kind of RAID and what particular hardware to buy to implement this. Am I right here?

If anyone has links to any articles or threads on the internet that discuss this topic really well, let me know too.

[Craig Parkes]

A RAID array is not a back up. What I mean by this, is that a RAID setup does not provide a back up as such, it provides a level of redundancy.

Redundancy means that your data can survive a hardware failure. A backup tends to mean even in the event of physical destruction of that unit (like it getting wet, set on fire etc, OR hardware failure.) that there is another copy somewhere.

Beyond Backup, there is Archival - Archival means a copy that will last a LONG time.

The cheapest way to backup a large amount of data currently, is to get a SATA dock enclosure, that can connect via e-sata our some other fast connect such as Firewire 800/USB 3, and physically copy all the data onto two separate hard drives.

If you have a RAID 1 array, this is kind of the same thing, except that the hard drives are in the same place and constantly being mirrored. The good thing is that there is no delay in the mirroring, the bad thing is that if the unit is hit with a sledge hammer, lost, set on fire etc, chances are both drives and all data on both drives will still be gone.

If you have a large raid with say RAID 5 redundancy, all it really means is that you can survive 1 hard drive failure at any given time and still rebuild the raid by replacing one drive. However, there are some added gotchas. You might not be able to survive a hard drive failure AND a raid controller failure. Also, if two hard drive failures happened simultaneously, all the date would be lost. And again, it's not actually a back up - can't keep the data in two places.

I've done some maths myself and what I have found is, for those just wanting to backup or even archive data in the short term - you should simply buy two naked hard drives, durable static free cases to store them in, and use something like a VANTEC storage dock.

The advantage here is that you can get good quality drives of your own choosing, have a relatively fast interconnect (rather than buy cheap Western digital enclosures with only USB 2 etc.) and also save space for storage over having lots of powered enclosures.

Cost wise, if you buy two 1 TB drives for each terabyte of data you are looking to back up, it gives you physical and practical redundancy, and you should aim to migrate all the data on those two drives onto new drives after two years.

Chances of failure of both drives in a two year period is relatively slim, less likely than a RAID (other than RAID 1) failing in completely in the same period.

Now in saying that, while RAID doesn't give you a backup, it does give you a level of redundancy, and can also give other advantages (such as speed - my RAID 5 Array at work can do read/writes at 500MB/s, much faster than any single spinning drive can do on it's own.)

Basically Raids can save you versus hardware failure, backups save you versus physical disaster. Often it's a good idea to have both.

[Richard Davidson]

If you want to look at a simple array that you can store data on then you might have a look at unRaid. The 3 disk license is free and it provides a way to rebuild your data in case of a hardrive failure. It is not designed for speed or files moving in and out of the arrary but more for storage of video files that you need to access at some time. I have used it for several years and I am more than happy with it's ability to rebuild a failed drive with little issue.

As a precation I do have all of my bluray movies backed up on old hardrives stored in boxes. A way to recycle smaller drives and keep things that will be hard to rebuild safe. Right now I am running over 60TBs of data on my unRaid systems. I have had some hard drive failures and never lost any data to to drive issues, just my own mistakes.

[Brett Sherman]

RAID does not protect you from accidental deletion, file system failures (more common than media failures), lightning, fire or other natural disasters.

Two bare drives as mentioned by Craig do provide you with this protection. I use a nightly syncing between two drives for my backup. When the drive gets filled I move one to another site.

Also keep in mind that RAID 5 is dependent on particular hardware. If that hardware goes down you can not access the data.

[Kawika Ohumukini]

Good points. Just one more disaster add; flood. If there's a fire, the firemen will flood the place. Cheers

[Bruce Foreman]

I'm taking a very simple approach. External hard drives in pairs.

I don't leave them spinning continuously, power them up when I have something to write to them or when I need to retrieve something.

When I come back from a shoot, first thing I do is copy my acquired footage from the media cards to a separate project named folder on each hard drive, verify the files are "good" then I can return the media cards to the camera for reformatting.

Then I copy the needed files from one of the externals to the drive I will be editing on.

Right now I'm using 2TB drives in a pair for media storage (and edited project storage as well) and a single 2TB external drive for editing.

It's a start anyway, soon I plan to add a third drive to the pair but update it once a month or so and mainly keep it "off site".

[Sareesh Sudhakaran]

If you have to back up digital files, the best option is to backup files (in different formats if possible) on different media, stored in different locations, and on the internet in different servers for redundancy.

Formats change, hardware and software changes, and disk technology changes. There are too many variables involved, and a completely reliable and guaranteed methodology that will stick around for 50+ years hasn't been found. This is why you'll find it tough to get articles on the subject, because most experts don't agree with each other.

RAID is not a long term backup option. It's a process that defines a system (a way of connecting disks) - it's not the system itself (which is hard disks connected to each other).

[Richard Davidson]

Raid is mainly for fault tolerance and not backup. My point about unRaid is that you can lose a drive but not lose your data. If you lose 2 drives at the same time then you only lose the data on those 2 drives but not the other drives. With other forms of raid if you lose 2 drives then you have lost all of your data.

For a true backup plan I really like Craig's setup, very neat, very organized, cost is very reasonable, and bottom line is that it works. Now how you store your tapes can be another issue that can defeat the best of plans.

For files that you want to have access to but want some security I would use some form of raid. For true backup I would go with the tape option. Since I have 2 servers running 20 drives each and I have changed out those drives as the larger ones come onto the market I have a lots of smaller drives to fill up. Cheap for me since they are paid for and easy to store.