Raid 0+1, 1+0 or 5?

[Kenny Shem]

Deciding which raid system is good for HDV editing. Any comments? Which is the heavier process when editing, write or read? 5 seems to be the better one for read intensive usage.

[Giroud Francois]

hdv is not more demanding than DV that is itself pretty low on disk.
depending your needs, simple mirroring will allow you to put 2 huge and cheap hard disk (2x500gig) together and work knowing your data are safe.
You will need anyway to buy a third HDD as spare, since model are changing fast and you will need the same disk in case of problem.
if you are looking for a bit of performance, raid 10 (1+0) will allow you to strip disks for performance and keep a mirror for safety.
then you can choose smaller disk (4x250 gig) but price will be anyway higher than simple mirroring.
raid 5 is great if you use a large number of drives, because a raid 5 need to be on at least 3 disks, but is better on 5 disk. So it is more expensive than the 4 disk solution of raid 1+0
Keep in mind that high tech raid solution require special controller for efficiency (can be costly) , high number of disk (can be difficult to put all of them into the pc case and will draw a lot of power from your powersupply.

[Kenny Shem]

I think the only factor in deciding whether to build a 1+0 or 5 setup for me is to know whether read is more important, or write is more important in video editing. Or are they equally important?
Either setup, I will have to buy 4 HDD, good tall casing with good ventilation & fan and a good PSU.

[Adam Gold]

Based on what I've read -- and I'm certainly not an expert -- if you can load up your system with 4 x 1TB drives, you may not want to go RAID at all. When rendering, my understanding is that reading is more intensive, as multiple content streams (video, audio, graphics, etc) are being combined into one.

Having different types of content on different drives could help here as multiple streams could be read simultaneously. I suppose you'd get the same advantage with RAID 0, but then if one drive fails the whole thing is out of order.

If I had a system with four internal drives, I'd have (for example) my NLE on C, project on D, media on D and E, and output to F.

But I could be totally out to lunch on this. Lots of (sometimes conflicting) info and advice out there... Adobe, for example, has a page in their online help which recommends how to set up scratch disks.

[Giroud Francois]

you better will put your money into a cineform prospect HD license , lot of memory, big processor and keep two 500 gig in mirroring + one disk for scratch than build a complex machine full of disks and controller that will give only the apparence of performance.
if you really need something powerfull, the correct level is to go high end workstation/server hardware (for performance and reliability) that is pretty expensive.

[Kenny Shem]

Originally Posted by Adam Gold

Based on what I've read -- and I'm certainly not an expert -- if you can load up your system with 4 x 1TB drives, you may not want to go RAID at all. When rendering, my understanding is that reading is more intensive, as multiple content streams (video, audio, graphics, etc) are being combined into one.

Having different types of content on different drives could help here as multiple streams could be read simultaneously. I suppose you'd get the same advantage with RAID 0, but then if one drive fails the whole thing is out of order.

If I had a system with four internal drives, I'd have (for example) my NLE on C, project on D, media on D and E, and output to F.

But I could be totally out to lunch on this. Lots of (sometimes conflicting) info and advice out there... Adobe, for example, has a page in their online help which recommends how to set up scratch disks.

Purpose of raid is to increase the performance speed (both writing and reading) and mirroring for backup. Why do you not want to have raid when you have 4 X 1TB HDD?

[Adam Gold]

Well, again, as I said, I'm not an expert, and I'm only summarizing what I've read from those who are. But it seems to me that, RAID or not, you've still got the same number of read/write heads running simultaneously, so I don't know how much speed advantage a RAID config would get you -- I've not seen data comparing speed.

Mirroring for backup is a whole different issue, but then of course you kill your capacity, which is an issue for HDV, especially if you're running a (recommended) Cineform plug-in, which will ease your editing overhead at the expense of much larger file sizes -- which is why you need so much HDD space to begin with.

It would seem to me that by having different drives with different letters for each type of content, you can really specify what goes where to maximize efficiency, whereas with RAID I'm not sure you would really know where anything is going. My interpretation has always been that the more separate data streams, the more can come in simultaneously. I don't know how efficient it would be if you had, say, three huge drives RAIDed together with one drive letter designation.

But again, maybe I'm misunderstanding how this all works and there are surely better experts than myself on this...

[Harm Millaard]

Let's start with a single disk, the average SATA disk nowadays has a sustained transfer rate of around 75 MB/s. More than enough for OS and programs.
Also let's directly clear up the myths of external disks. eSATA's have the same transfer rate as internal SATA's. External fire wire disks manage around 30 MB/s and external USB2.0 disks manage around 20 MB/s. So in general external non-eSATA disks are only usable for backup purposes.

Raid 0 achieves approximately 90% increase in transfer rate for each disk in the array, thus a 2 disk array will be nearly 180% faster than a single disk, a 3 disk array nearly 270% faster than a single disk, and so on. There is some overhead, but it is minor.

Raid 0+1 or 1+0 achieves nearly the same performance increase for the raid 0 part, but the mirrored disks show no performance gains, instead they show some performance loss due to the overhead of mirroring. Thus a 4 disk array, with 2 striped disks (0) and 2 mirrored disks (1) will only be around 180% faster than a single disk. The benefit is the mirror, not the performance gain.

Raid 5 achieves approximately 85% increase in transfer rate for each disk in the array minus 1. Thus a 3 disk raid 5 array will be around 170% faster than a single disk, a 4 disk array around 255%, a 8 disk array around 595% faster than a single disk. The advantage of raid 5 over raid 0 is the redundancy. If one drive in the array fails, you still have all your data and the array can be rebuild. In a raid 0 if one drive fails all is lost.

For editing you need one disk of OS & programs, just a single disk is enough as long as you make a Ghost on an external disk or use slipstreamed DVD's to protect against disk failure. Then you need a disk for media and projects which could well be a multiple disk array in raid 5. In addition you need storage for page file, scratch disks, preview files, conformed audio etc. This could well be a raid 0 array.

IMO this could be a good set up:

C: OS & programs, single disk
D: Media & projects, 4 or more disks in raid 5
E: page file, scratch, previews, conformed audio etc. 2 or more disks in raid 0

Hope this helps somewhat.

[Michael Nistler]

Kenny,

FYI - I use the G-RAID (two 400GB drives striped as RAID 0) for rendering my video.
http://www.videoguys.com/graid.html

I find this renders faster then my internal SATA WD 7200 RPM drives via Firewire 800. See Premiere benchmark (I'm BridgeHands, position #22):
http://mysite.verizon.net/wgehrke/pp...hmark%203.html

Yes, I could get faster drives but my gut tells me these give me a good balance between performance and reliability. Besides, I'm still living in SD, but may make the conversion to HD someday...

Regards, Michael

[Kenny Shem]

Originally Posted by Michael Nistler

Kenny,

FYI - I use the G-RAID (two 400GB drives striped as RAID 0) for rendering my video.
http://www.videoguys.com/graid.html

I find this renders faster then my internal SATA WD 7200 RPM drives via Firewire 800. See Premiere benchmark (I'm BridgeHands, position #22):
http://mysite.verizon.net/wgehrke/pp...hmark%203.html

Yes, I could get faster drives but my gut tells me these give me a good balance between performance and reliability. Besides, I'm still living in SD, but may make the conversion to HD someday...

Regards, Michael

Michael,

how much speed difference do you find when you use raid 0 compared to your sata WD 7200 drive? Thanks.

Kenny

[Marcus Marchesseault]

"Raid 5 achieves approximately 85% increase in transfer rate for each disk in the array minus 1. Thus a 3 disk raid 5 array will be around 170% faster than a single disk, a 4 disk array around 255%, a 8 disk array around 595% faster than a single disk."

Don't forget that the RAID 5 controller must do the calculations to divide the data and create the redundancy. The speed of the RAID 5 controller is the determining factor in the system's performance.

Considering that modern drives perform at about 20 times the rate of HDV, there is no compelling reason to use them. They increase the cost and complexity of a system. In most cases it would be better to have multiple disks each devoted to a single task. The bottleneck in drive performance is not the transfer speed but rather the access time. Access time is determined by the speed of the heads and the rpm of the platters. A RAID doesn't help access time in any way. It is better to have more drives doing their own thing since that will alleviate the delays caused by access time.

Unless you are doing uncompressed HD, you will likely get no speed benefit from a RAID. If all you want is redundancy, a RAID 1 is all you need.

[Ron Evans]

I agree with Marcus, my current system has a 160G OS, 200G for Temp and project info, 2 x250G and a 500G for storage. Boot and Project info backed up to external drive. No RAID. I spread the files out so that in a multicam shoot different camera files are on different drives. I think that this system may offer better performance than a RAID for this multicam/track operation. A RAID with all cameras on one RAID would have to deliver all the track info. As stated, RAID 5 at 170% performance of single disc would not be as fast as three separate drives and offer less storage!!!. The RAID would provide easier management( load it all up on the one RAID !!!) but would have less performance than the three separate discs. For DV and HDV it really isn't needed. Intermediate formats that may increase files sizes by 4 times may start to need a RAID for performance but this won't work for multi track, in this format one would need a RAID for EACH track. In other words emulate a faster hard drive dedicated to the one file for performance. Even for Canopus HQ or Cineform a 7200rpm drive is fast enough as long as the load on it is only for one track/camera file. The oldest and slowest of my storage drives still manages close to 50MB/s half full. Still many times the needs of Canopus HQ or Cineform when by project configuration only one file is being read.

Ron Evans

[Michael Nistler]

Originally Posted by Kenny Shem

Michael,

how much speed difference do you find when you use raid 0 compared to your sata WD 7200 drive? Thanks.

Kenny

Hi Kenny,

The RAID 0 speed improvement wasn't dramatic but certainly noticeable. If the combined benchmark rendering was 100 seconds for the RAID 0, I recall it was about 115-120 seconds with the internal WD SATA drive. The person hosting the benchmark had a hard time believing an external drive could actually be faster, but using Firewire 800 with RAID 0 is a nice solution.


Of course, having an external device provides nice options. I can run from my camera firewire port directly into my internal laptop firewire port (running DV Rack), saving the AVI output directly to the GRAID via a PCMCIA firewire card - never any dropouts and I've got 800GB of instant video access when I reconnect the GRAID back to my editing station.

Good luck, Michael

[Thomas Smet]

Originally Posted by Michael Nistler

Hi Kenny,

The RAID 0 speed improvement wasn't dramatic but certainly noticeable. If the combined benchmark rendering was 100 seconds for the RAID 0, I recall it was about 115-120 seconds with the internal WD SATA drive. The person hosting the benchmark had a hard time believing an external drive could actually be faster, but using Firewire 800 with RAID 0 is a nice solution.


Of course, having an external device provides nice options. I can run from my camera firewire port directly into my internal laptop firewire port (running DV Rack), saving the AVI output directly to the GRAID via a PCMCIA firewire card - never any dropouts and I've got 800GB of instant video access when I reconnect the GRAID back to my editing station.

Good luck, Michael

thats because even though internal SATA is fast the drive itself becomes a bottleneck. When it is empty you can get around 75MB/S. The firewire 800 drive however is a raid-0 so internally it is getting much better speeds. Even though firewire 800 isn't perfect it is capable of up to 100MB/S transfer. So the cable connection speed is slower but the disks themselves are faster and in this case the faster drives equal more then the single internal drive speed. Your speeds seem about rifht to reflect this slight advantage. Now if only G-raid would add a E-SATA port to their 2 disk raid-0 products. You would see a little bit more speed but not much. I read somewhere once that they raid-0 the rives together so one drive uses the inside of the disk while the other drive uses the outside. It does this to balance out the speed so it is about even no matter how full it is. So add what a empty and full drive could do and divide that by 2 and that is about the sustained rate for a G-raid.