Archiving EX files

Ted OMalley · October 14th, 2008, 02:50 PM

Quote:

Originally Posted by Bill Ravens

Chuck...
I'm sorry if I sounded rude. It's just that backup and archiving processes are pretty different for a business that has massive amounts of data to backup and retrieve, than for a small, home owned business like mine. NAS is something that requires an investment in hardware/software infrastructure that can't be justified for a small business. It's just not cost effective. The cost of installing such a system for my business would never pay back the investment.

One of my complaints with this otherwise excellent forum is that there are many different types of users who read and post here. Clearly, I think, not all architectures scale with the size of the business. I would never presume to advise someone on your scale of business with my processes. I would think the reverse would likewise be true. I think it's obvious that manual storage and retrieval of even 32Gb flash cards would be cumbersome and inefficient for your needs.

I have observed that people jump into discussions with advice, criticisms, judgments, arguments when their input is not necessarily relevant to the discussion. Such is the case, IMHO for NAS RAID backup systems and onesy/twosy backup systems.

Hey, Bill -

I have a very small business, and prefer the NAS or RAID system. My little RAID is 500GB is cost about $400. However, going with larger drives (1.5TB) does not cost much more yet provides many more GB per dollar!

Another solution that I've used for years that I really like, is a SATA drive bay in my editing system. A SATA card ($50 to $75) and a SATA bay ($30) and you can hot swap any SATA hard drive, fill it up, and place it on a shelf. For critical data, I will write it to two different drives. Drives only run about $130 for 1TB - so it is extremely cheap as well. I know many people don't like hard drives for storage, but I only have a need to retain data for about five years. Since the drives are barely used, stored air conditioned space, and not subject to abuse, they seem to do well. I'm only on year 3 of this system, but it has worked well.

The best part is when I need to refer to or load up some old media. I slide the drive in my media bay, Windows recognizes it, and the content is directly editable and I can load and run projects right from it at SATA II speeds.

Bill Ravens · October 14th, 2008, 07:41 PM

hmmm....
yeah, I've got 8, count 'em, 8 SATA II ports for various asundry external drives each on two machines.. Each drive holds its own project, sometimes work on 3 projects in a day, SATA external bluray burner that moves between 3 workstations. Some external SATA's move between my PC's and a MAC workstation. Use MACDRIVE so I can read HFS+ drives off of those Macs.

That's all well and good but I hate taking up space on an external drive with native files. I like to transcode them to intermediates/proxies, then archive the originals. Best option for archiving the originals seems to be CF cards. I have a slew of old IDE drives for storage, but, theyre very s-l-o-w.

So, for any reasonably efficient NAS you need a 10Gbit switch, router and server, plus, fibre optic connections would be nice to not bog the system down, Nothing beats the speed of moving a project drive between machines ILO trying to feed a 30 Gb file over a network. Then you need RAID cards plus RAID enclosures, and make that RAID1...RAID0 is too dangerous for project files.

Do you get where I'm going with this? All very nice, but, $$$$$$$$$

Alex Raskin · October 14th, 2008, 09:07 PM

Archiving - for what purpose?

To just store the client data, why not use a large RAID1 (very reliable, even if the controller card goes down, you still have 2 identical copies.)

If you are talking data security as in "off-site" storage, then I guess rotating DLT tape backups would be the ticket - but I personally have never done it.

It looks to me that the chance of catastrophic data loss is remote if you have a redundant storage system (RAID1).

Seems like other cases involving theft/fire/equipment failure should be covered by business insurance?

Keith Moreau · October 14th, 2008, 10:54 PM

I have used a myriad of solutions for archiving. The problem is nowadays we have massive amounts of data to back up, it makes archiving on optical or data tape impractical (cause it's too slow and inaccessible).

Backing up to a hard disk is really the only current practical solution to the current need to archive massive amounts of video. I have removable caddy systems for IDE and now SATA, but I've felt that unless there is a way to monitor the drive for it's 'health' there really is no guarantee that the drive will lose data or become inoperable while on the shelf. It's just a mechanical device that can either demagnetize or just break.

Raid 1 is simple and fine, if one drive goes down you replace it without data loss.

Recently a company called Data Robotics has introduced something called a Drobo. It's basically a unit that accepts up to 4 SATA drives and connects to your computer via USB 2 or Firewire. I basically uses a RAID like scheme that allows 1 of the 4 drives to go bad without losing any data. If one of the drives does go bad, you can replace that drive and it will go on functioning. The capacity is basically equivalent to 3 of the 4 drives (ie 4 TB drives = 3 TB of storage capacity).

I like the unit, it's not cheap but if you add up the cost for similar solutions, like RAID 1, or RAID 5, or RAID 0+1, it's pretty competitive. The also have add-ons to turn the device into a NAS. (I have it hooked in via Firewire 800 into a NAS like solution of my own, an older Macbook Pro on Gigabit ethernet.)

It's also really simple to use, the software is smart and it's fairly compact and quiet. I have one now and I'm probably going to standardize on them in the future (I hope they stay in business.)

Paul Joy · October 15th, 2008, 04:51 AM

Personally, I prefer the safety in numbers option. Drobo looks like a good solution giving further protection as an external drive (array), but I would be worried about trusting it with the only copy of my files, even if it is spanning them across multiple disks.

I've worked as an IT admin in the past and have witnessed many nasty incidents of RAID devices losing all data. They do protect from hard drive failure, but if they fail themselves the results can be just as destructive.

Is it possible to pull a single drive from the drobo as an archive that could be used inside a computer at a later date, or is the data stored in a way which would be impossible to read without the use of another drobo?

I use a dedicated internal drive for storing raw EX data (BPAV's) and then use time machine on the mac to back up all of the files to an external USB drive, once it's full it goes into storage and is replaced by a bigger external drive that backs up the same files but gives room for more. This seems to be working out pretty well at the moment as the drives are getting bigger and cheaper all the time.

I like knowing that I have archives of my older files across multiple inactive drives and that my newer files are on 2 active drives all the time.

Time machine also has the added benefit of being able to cycle back through various versions of project files as they are being worked on.

Mike Chandler · October 15th, 2008, 08:33 AM

Quote:

Originally Posted by Paul Joy

Is it possible to pull a single drive from the drobo as an archive that could be used inside a computer at a later date, or is the data stored in a way which would be impossible to read without the use of another drobo?

I don't think you can, Paul. Just asked a guy who owns one that same question and he said you can't pull a drive out of the drobo for separate use, as it's spanning the info over multiple drives. When your drive(s) is full, you have to put in another, larger one.

I'm thinking that for archiving, I'll try a hybrid system: backup onto blu-ray discs plus a single hard drive that gets put on a shelf. Don't see the value of having 4 discs spinning a la drobo or raid, if the purpose is to archive dailies.

John Peterson · October 15th, 2008, 08:57 AM

......unless there is a way to monitor the drive for it's 'health' there really is no guarantee that the drive will lose data or become inoperable while on the shelf. It's just a mechanical device that can either demagnetize or just break.
===================

I have never heard of that. If the drive is used only for archiving data it will be far far below the MTBF for the drive. The possibility of drive failure while on the shelf seems extremely remote.

John

Ted OMalley · October 15th, 2008, 08:59 AM

Mike's correct.

When data is striped across multiple drives, they are a married set and must remain together. In your scenario, which is probably a RAID 5, fault tolerance is in effect due to shared information. Once one drive is removed, the remaining three still contain 100% of the data. However, you cannot read that drive by itself or move it into another existing RAID 5 and expect it to retain it's data. The benefit to RAID 5 over other RAIDs is that only one drive of the set is used for redundancy. If you used ten 100GB drives, you'd have 900GB of storage

However, with a RAID 1, a simple mirror, both drives contain the same data. They are written to and read from as a unit. When one fails, like with a RAID 5, you are left with all of your data intact. The benefit is that a drive can be removed and read by itself. The downside, you only see half of your actual storage capacity. In my opinon, considering all the things that can and do go wrong, a RAID 1 is slightly more fault tolerant.

Ted OMalley · October 15th, 2008, 09:02 AM

Quote:

Originally Posted by John Peterson

......unless there is a way to monitor the drive for it's 'health' there really is no guarantee that the drive will lose data or become inoperable while on the shelf. It's just a mechanical device that can either demagnetize or just break.
===================

I have never heard of that. If the drive is used only for archiving data it will be far far below the MTBF for the drive. The possibility of drive failure while on the shelf seems extremely remote.

John

John,

This has been my experience as well. Some of my drives, though small, are just shy of ten years old. Also, though I've had five or six fail over the years, I've never had one go bad on the shelf (and I usually have at least eight or ten on the shelf).

John Peterson · October 15th, 2008, 09:05 AM

Quote:

Originally Posted by Ted OMalley

Mike's correct.

When data is striped across multiple drives, they are a married set and must remain together. In your scenario, which is probably a RAID 5, fault tolerance is in effect due to shared information. Once one drive is removed, the remaining three still contain 100% of the data. However, you cannot read that drive by itself or move it into another existing RAID 5 and expect it to retain it's data. The benefit to RAID 5 over other RAIDs is that only one drive of the set is used for redundancy. If you used ten 100GB drives, you'd have 900GB of storage

However, with a RAID 1, a simple mirror, both drives contain the same data. They are written to and read from as a unit. When one fails, like with a RAID 5, you are left with all of your data intact. The benefit is that a drive can be removed and read by itself. The downside, you only see half of your actual storage capacity. In my opinon, considering all the things that can and do go wrong, a RAID 1 is slightly more fault tolerant.

Exactly.

That is why I never bothered with Raid 1 setups for improved speed over the years. It wasn't worth risking data loss in the event of one of the two drives failing.

John

Alex Raskin · October 15th, 2008, 09:49 AM

In 5 years storing video on hard drives, I've lost some data with all solutions except RAID1.

RAID5 seemed particularly shaky, despite the promise of savings.

Go simple. RAID1 = instant data backup onto 2 drives inside the array. You'll need exactly twice the capacity, so for 1TB storage you'll need two 1TB drives. (Does not seem like a deal breaker at current low HDD prices.) And then you're all set. Even if the RAID controller card (Drobo and WD Books come to mind, as well as internal PC cards) fails, you still have your data intact on at least one of the two RAID1 drives.

Adam Plywaczewski · October 15th, 2008, 10:20 AM

Just few months ago we have went for tapeless acquisition, moving from reaally old now Betacam SP - somehow we've managed to skip the DV revolution in terms of acuisition format, so our equipemnt did pay for itself numerous times. this should show that we are not willing to spend money without justification.

This presented us with exactly the same problem. And this is what we've come up with.

While LTO drives are still pretty expensive, we went for considerably cheaper DLT drives. We use Quantum DLT-V4 drive, and while it is slower and much lower capacity than what George Kroonder stated about the Ultrium drive, it still stores 160GB native, and life expectancy beats any dvd, or presumably bluray bluray being a new format, as against to the DLT technology which has been around for almost "ages"

We have it hooked up to the RAID5 server where we store original files from EX and other sensitive data.The drive has e-sata and usb connection, and any XP machine has a backup software to use it with. So it really is a no brainer, considering you handle your project files in a coherent manner. Gather all files in one folder, mark it for backup, put in the tape, and for next 30 years or so you should sleep well.

We charge client for archiving on the per-GB basis. And most our projects easily fit within the 160GB tape limit.

I know this is not a perfect solution, but maybe someone will get inspired as we were inspired by Bart Walczak - dvinfo user and a friend of ours :)

Bill Ravens · October 15th, 2008, 10:22 AM

It's a mystery why people gravitate to a product like Drobo. DIY RAID enclosures and arrays have been around for years. One of the best, IMHO, is available at Addonics Technologies - Making Data Storage Simple, uses SATAII (3Gbps) hard drives. But, for all the reasons cited above, RAID is not a good solution for long term archival.

Adam Plywaczewski · October 15th, 2008, 10:24 AM

Quote:

Originally Posted by Alex Raskin

In 5 years storing video on hard drives, I've lost some data with all solutions except RAID1.

RAID5 seemed particularly shaky, despite the promise of savings.
...

As to RAID5 vs. RAID 1 - we've built our raid 5 some time ago, and now, with the dropping prices of drives, we are considering changing to the simple raid 1, as it is just faster solution than raid 5 - especially when saving files.

Keith Moreau · October 15th, 2008, 11:18 AM

Here's my reasoning:

Shelf: A good solution with the advent of cheap, reliable drives, but with risks. When on the shelf, they are not monitored. Drives lose magnetism over time, resulting in data loss. Drives can also 'seize,' it's called stiction (I know, I worked for one of the early Mac external drive companies and there were a lot of failures due to various drive hardware problems). The Mean Time Between Failures (MTBF) is just that. It's an average of the lifetime of a drive over thousands of data points. In one case, a drive could simply break in 1 day, another case could be viable for 20 years.

Large Capacity Optical: When Blue-Ray disks come down in price, this may be the best solution. However, many people don't know that Optical media is not necessarily a permanent solution. Optical disk substrates are created with aluminum, which, unless perfectly enclosed by the polycarbonate around it, can corrode. If you try a CD you've burned 10 years ago, there is a good chance there are some unreadable areas. If that area is an important piece of video, it's lost forever. Higher quality media can help mitigate this with more inert materials and coatings (gold, etc) but are a lot more expensive.

However, in addition to whatever 'online' solution I have (below), I always have an 'on the shelf' backup of the data as well.

Raid 1 (mirrored drives): I have a number of RAID enclosures that are now configured as RAID 0 for speed. The enclosures are not that expensive, perhaps $150 for a low end solution like from OWC computing (plus the drives). I could also configure them as RAID 1 as a backup if I chose to. When you configure a RAID to be 'mirrored' you lose the speed benefits of RAID 0. Any good online RAID 1 or other redundant solution, needs to be powered and monitored in some way for drive 'health.' In the case of a drive failure, there are benefits of being able to use the good remaining drive or put it into another enclosure and utilize it immediately if the enclosure goes down.

Raid 5, 1+0, or Drobo:

Raid 5 is a solution similar to a Drobo, which allows for the failure of 1 of 4 drives without data loss. It usually requires expensive enclosures and usually associated monitoring software. The drives need to be matched and the same capacity. The speed is about that of the slowest drive in the array.

Raid 1+0 is a set of at least 4 drives which are mirrored and striped together. It has the speed benefits of RAID 0 with the safety of RAID 1. It's complex but it would suitable for your 'working' drive setup. I'm considering standardizing on this solution in my Mac Pro, but I'm still researching it to see if others have had good luck with it. It's quite a commitment. And the capacity is limited and not expandable and you'll still need an additional archival system.

Drobo: Very simple, easy to use (basically plug it in and it works) elegant software (it can be configured to email you status about the drive including failures) and is expandable by replacing the drives with higher capacity drives. It's not that fast. Maybe 30MB/second over Firewire 800. I don't it as my 'working' drive, though I could in a pinch. I would recommend the Drobo for the same reason I recommend Mac computers, it's simple and it just works. You don't have to worry about configuring it or having much knowledge about RAID, and it's relatively inexpensive for what you get. The fact is I have and could have setup any type of archival system, but I have settled on the Drobo for the reasons I've stated. Maybe it's a mistake, time will tell.

Whatever solution you use, I'd recommend a 'shelf' backup (optical, data tape, hard drives) but I'd also have an online solution as well. And I'd have a regimen to put the 'shelf' media online periodically to ensure it's not degraded, and if it has, back it up again.

This is all a pain, but it's the digital age we live it. We're producing content exponentially, and we have to figure out a reasonable way to safely store it.

October 14th, 2008, 10:54 PM	#34
Keith Moreau Major Player Join Date: Nov 2006 Location: Brisbane, California Posts: 530	Drobo I have used a myriad of solutions for archiving. The problem is nowadays we have massive amounts of data to back up, it makes archiving on optical or data tape impractical (cause it's too slow and inaccessible). Backing up to a hard disk is really the only current practical solution to the current need to archive massive amounts of video. I have removable caddy systems for IDE and now SATA, but I've felt that unless there is a way to monitor the drive for it's 'health' there really is no guarantee that the drive will lose data or become inoperable while on the shelf. It's just a mechanical device that can either demagnetize or just break. Raid 1 is simple and fine, if one drive goes down you replace it without data loss. Recently a company called Data Robotics has introduced something called a Drobo. It's basically a unit that accepts up to 4 SATA drives and connects to your computer via USB 2 or Firewire. I basically uses a RAID like scheme that allows 1 of the 4 drives to go bad without losing any data. If one of the drives does go bad, you can replace that drive and it will go on functioning. The capacity is basically equivalent to 3 of the 4 drives (ie 4 TB drives = 3 TB of storage capacity). I like the unit, it's not cheap but if you add up the cost for similar solutions, like RAID 1, or RAID 5, or RAID 0+1, it's pretty competitive. The also have add-ons to turn the device into a NAS. (I have it hooked in via Firewire 800 into a NAS like solution of my own, an older Macbook Pro on Gigabit ethernet.) It's also really simple to use, the software is smart and it's fairly compact and quiet. I have one now and I'm probably going to standardize on them in the future (I hope they stay in business.)

October 15th, 2008, 04:51 AM	#35
Paul Joy Major Player Join Date: Mar 2007 Location: Norfolk, UK Posts: 627	Personally, I prefer the safety in numbers option. Drobo looks like a good solution giving further protection as an external drive (array), but I would be worried about trusting it with the only copy of my files, even if it is spanning them across multiple disks. I've worked as an IT admin in the past and have witnessed many nasty incidents of RAID devices losing all data. They do protect from hard drive failure, but if they fail themselves the results can be just as destructive. Is it possible to pull a single drive from the drobo as an archive that could be used inside a computer at a later date, or is the data stored in a way which would be impossible to read without the use of another drobo? I use a dedicated internal drive for storing raw EX data (BPAV's) and then use time machine on the mac to back up all of the files to an external USB drive, once it's full it goes into storage and is replaced by a bigger external drive that backs up the same files but gives room for more. This seems to be working out pretty well at the moment as the drives are getting bigger and cheaper all the time. I like knowing that I have archives of my older files across multiple inactive drives and that my newer files are on 2 active drives all the time. Time machine also has the added benefit of being able to cycle back through various versions of project files as they are being worked on. __________________ Blog: www.pauljoy.com Company: www.videotrader.co.uk

October 15th, 2008, 08:59 AM	#38
Ted OMalley Major Player Join Date: May 2006 Location: Tampa, FL Posts: 596	Mike's correct. When data is striped across multiple drives, they are a married set and must remain together. In your scenario, which is probably a RAID 5, fault tolerance is in effect due to shared information. Once one drive is removed, the remaining three still contain 100% of the data. However, you cannot read that drive by itself or move it into another existing RAID 5 and expect it to retain it's data. The benefit to RAID 5 over other RAIDs is that only one drive of the set is used for redundancy. If you used ten 100GB drives, you'd have 900GB of storage However, with a RAID 1, a simple mirror, both drives contain the same data. They are written to and read from as a unit. When one fails, like with a RAID 5, you are left with all of your data intact. The benefit is that a drive can be removed and read by itself. The downside, you only see half of your actual storage capacity. In my opinon, considering all the things that can and do go wrong, a RAID 1 is slightly more fault tolerant. __________________ Sony EX3, Vegas 9.0 64bit, Windows 7(64), Core i7, 12GB, RAID1 & 0, HotSwap SATA, 30" LCD(2560x1600)-GTX285 & 24" LCD(1360x768)-7800GT

October 15th, 2008, 11:18 AM	#45
Keith Moreau Major Player Join Date: Nov 2006 Location: Brisbane, California Posts: 530	Drobo vs RAID 1 vs Drives on the Shelf vs Optical Here's my reasoning: Shelf: A good solution with the advent of cheap, reliable drives, but with risks. When on the shelf, they are not monitored. Drives lose magnetism over time, resulting in data loss. Drives can also 'seize,' it's called stiction (I know, I worked for one of the early Mac external drive companies and there were a lot of failures due to various drive hardware problems). The Mean Time Between Failures (MTBF) is just that. It's an average of the lifetime of a drive over thousands of data points. In one case, a drive could simply break in 1 day, another case could be viable for 20 years. Large Capacity Optical: When Blue-Ray disks come down in price, this may be the best solution. However, many people don't know that Optical media is not necessarily a permanent solution. Optical disk substrates are created with aluminum, which, unless perfectly enclosed by the polycarbonate around it, can corrode. If you try a CD you've burned 10 years ago, there is a good chance there are some unreadable areas. If that area is an important piece of video, it's lost forever. Higher quality media can help mitigate this with more inert materials and coatings (gold, etc) but are a lot more expensive. However, in addition to whatever 'online' solution I have (below), I always have an 'on the shelf' backup of the data as well. Raid 1 (mirrored drives): I have a number of RAID enclosures that are now configured as RAID 0 for speed. The enclosures are not that expensive, perhaps $150 for a low end solution like from OWC computing (plus the drives). I could also configure them as RAID 1 as a backup if I chose to. When you configure a RAID to be 'mirrored' you lose the speed benefits of RAID 0. Any good online RAID 1 or other redundant solution, needs to be powered and monitored in some way for drive 'health.' In the case of a drive failure, there are benefits of being able to use the good remaining drive or put it into another enclosure and utilize it immediately if the enclosure goes down. Raid 5, 1+0, or Drobo: Raid 5 is a solution similar to a Drobo, which allows for the failure of 1 of 4 drives without data loss. It usually requires expensive enclosures and usually associated monitoring software. The drives need to be matched and the same capacity. The speed is about that of the slowest drive in the array. Raid 1+0 is a set of at least 4 drives which are mirrored and striped together. It has the speed benefits of RAID 0 with the safety of RAID 1. It's complex but it would suitable for your 'working' drive setup. I'm considering standardizing on this solution in my Mac Pro, but I'm still researching it to see if others have had good luck with it. It's quite a commitment. And the capacity is limited and not expandable and you'll still need an additional archival system. Drobo: Very simple, easy to use (basically plug it in and it works) elegant software (it can be configured to email you status about the drive including failures) and is expandable by replacing the drives with higher capacity drives. It's not that fast. Maybe 30MB/second over Firewire 800. I don't it as my 'working' drive, though I could in a pinch. I would recommend the Drobo for the same reason I recommend Mac computers, it's simple and it just works. You don't have to worry about configuring it or having much knowledge about RAID, and it's relatively inexpensive for what you get. The fact is I have and could have setup any type of archival system, but I have settled on the Drobo for the reasons I've stated. Maybe it's a mistake, time will tell. Whatever solution you use, I'd recommend a 'shelf' backup (optical, data tape, hard drives) but I'd also have an online solution as well. And I'd have a regimen to put the 'shelf' media online periodically to ensure it's not degraded, and if it has, back it up again. This is all a pain, but it's the digital age we live it. We're producing content exponentially, and we have to figure out a reasonable way to safely store it.

October 14th, 2008, 07:41 PM	#32
Bill Ravens Join Date: Nov 2001 Location: santa fe, nm Posts: 3,264 Images: 10	hmmm.... yeah, I've got 8, count 'em, 8 SATA II ports for various asundry external drives each on two machines.. Each drive holds its own project, sometimes work on 3 projects in a day, SATA external bluray burner that moves between 3 workstations. Some external SATA's move between my PC's and a MAC workstation. Use MACDRIVE so I can read HFS+ drives off of those Macs. That's all well and good but I hate taking up space on an external drive with native files. I like to transcode them to intermediates/proxies, then archive the originals. Best option for archiving the originals seems to be CF cards. I have a slew of old IDE drives for storage, but, theyre very s-l-o-w. So, for any reasonably efficient NAS you need a 10Gbit switch, router and server, plus, fibre optic connections would be nice to not bog the system down, Nothing beats the speed of moving a project drive between machines ILO trying to feed a 30 Gb file over a network. Then you need RAID cards plus RAID enclosures, and make that RAID1...RAID0 is too dangerous for project files. Do you get where I'm going with this? All very nice, but, $$$$$$$$$

October 14th, 2008, 09:07 PM	#33
Alex Raskin Trustee Join Date: Sep 2003 Posts: 1,435	Archiving - for what purpose? To just store the client data, why not use a large RAID1 (very reliable, even if the controller card goes down, you still have 2 identical copies.) If you are talking data security as in "off-site" storage, then I guess rotating DLT tape backups would be the ticket - but I personally have never done it. It looks to me that the chance of catastrophic data loss is remote if you have a redundant storage system (RAID1). Seems like other cases involving theft/fire/equipment failure should be covered by business insurance?

October 15th, 2008, 08:57 AM	#37
John Peterson Major Player Join Date: Jun 2008 Location: Monroe, NY Posts: 703	......unless there is a way to monitor the drive for it's 'health' there really is no guarantee that the drive will lose data or become inoperable while on the shelf. It's just a mechanical device that can either demagnetize or just break. =================== I have never heard of that. If the drive is used only for archiving data it will be far far below the MTBF for the drive. The possibility of drive failure while on the shelf seems extremely remote. John

October 15th, 2008, 09:49 AM	#41
Alex Raskin Trustee Join Date: Sep 2003 Posts: 1,435	In 5 years storing video on hard drives, I've lost some data with all solutions except RAID1. RAID5 seemed particularly shaky, despite the promise of savings. Go simple. RAID1 = instant data backup onto 2 drives inside the array. You'll need exactly twice the capacity, so for 1TB storage you'll need two 1TB drives. (Does not seem like a deal breaker at current low HDD prices.) And then you're all set. Even if the RAID controller card (Drobo and WD Books come to mind, as well as internal PC cards) fails, you still have your data intact on at least one of the two RAID1 drives.

October 15th, 2008, 10:20 AM	#42
Adam Plywaczewski New Boot Join Date: Jun 2007 Location: Warsaw, Poland Posts: 14	Just few months ago we have went for tapeless acquisition, moving from reaally old now Betacam SP - somehow we've managed to skip the DV revolution in terms of acuisition format, so our equipemnt did pay for itself numerous times. this should show that we are not willing to spend money without justification. This presented us with exactly the same problem. And this is what we've come up with. While LTO drives are still pretty expensive, we went for considerably cheaper DLT drives. We use Quantum DLT-V4 drive, and while it is slower and much lower capacity than what George Kroonder stated about the Ultrium drive, it still stores 160GB native, and life expectancy beats any dvd, or presumably bluray bluray being a new format, as against to the DLT technology which has been around for almost "ages" We have it hooked up to the RAID5 server where we store original files from EX and other sensitive data.The drive has e-sata and usb connection, and any XP machine has a backup software to use it with. So it really is a no brainer, considering you handle your project files in a coherent manner. Gather all files in one folder, mark it for backup, put in the tape, and for next 30 years or so you should sleep well. We charge client for archiving on the per-GB basis. And most our projects easily fit within the 160GB tape limit. I know this is not a perfect solution, but maybe someone will get inspired as we were inspired by Bart Walczak - dvinfo user and a friend of ours :)

October 15th, 2008, 10:22 AM	#43
Bill Ravens Join Date: Nov 2001 Location: santa fe, nm Posts: 3,264 Images: 10	It's a mystery why people gravitate to a product like Drobo. DIY RAID enclosures and arrays have been around for years. One of the best, IMHO, is available at Addonics Technologies - Making Data Storage Simple, uses SATAII (3Gbps) hard drives. But, for all the reasons cited above, RAID is not a good solution for long term archival.