Does RAID Really Help Much?

[Peter Moretti]

I have a 3.0 P4 w/ 2 Gig of RAM. I'm trying to keep this bad boy running for awhile longer. I'm wondering if RAIDing will help in that regard.

My primary drive is a Western Digital 80 gig, IDE, 7,200 RPM.

My secondary drive is a Western Digital Raptor 350 gig, SATA, 10,000 RPM.

Windows XP and all programs are installed on the C: partition of the primary (slower) drive.

All data is kept on the secondary (faster) drive.

Should I add another Raptor drive and RAID it with the existing one?

Also, is the strategy of storing the prog files and OS on the slower drive and data files on the faster drive (or RAID array) optimal?

Thanks much for all your help. It's very appreciated!


-Peter

[Harm Millaard]

Originally Posted by Peter Moretti

My primary drive is a Western Digital 80 gig, IDE, 7,200 RPM.

My secondary drive is a Western Digital Raptor 350 gig, SATA, 10,000 RPM.

I assume there is a typo here, the Raptor is 150 gig maximum. Apart from that, it would be beneficial to get a third drive and possibly even a fourth drive. Since the Raptors demand a significant price premium over 7.200 RPM's, I would suggest the following:

1. OS/programs on the 80 gig IDE
2. two Raptors as Raid0 for media
3. the fourth disk for project and scratch. This could be a cheap 320 gig SATA drive for around $ 70.

It really depends on the kind of projects you do. If the majority is HDV and usually contain a lot of tracks with numerous color corrections and effects, raiding is necessary. If your projects are mostly straight cuts, raiding may not be necessary. In that case you would likely benefit the most from just a third drive for projects and scratch.

If you want some more thoughts on this matter, please describe your typical projects some more.

[Bill Ravens]

If you're editting DV or HDV, the max thruput is 25MB/sec, by specification. All modern hard drives, whether they are SATA or IDE, are capable of about 30MB/sec. Therefore, a RAID array is of no use to you, assuming you mean RAID 0. To make the situation even worse, RAID 0 arrays have approximately half the reliability of non-RAID. If you decide to go with RAID, be sure to back up all your data on a regular basis.

[Don Blish]

Originally Posted by Bill Ravens

If you're editting DV or HDV, the max thruput is 25MB/sec, by specification. All modern hard drives, whether they are SATA or IDE, are capable of about 30MB/sec. .

To expand a bit, remember in editing you are running two streams in a dissolve and perhaps more if there are other layers. Secondly, that both DV and HDV bandwidth is 25megabits = 4 megabytes/sec. A naked SATA 7200 drive will sustain about 35 megabytes/sec. So native mpeg/HDV editing makes it easy on drives and very hard on the CPU,

If you use the CineformAccessHD plugin or something like it, the intermediate file is about three times as big (making it more demanding on disc thruput, but easily sustainable) and vastly lowering CPU intensity to enable more real-time effects.

I use Cineform and have RAID0ed my work drives. I just try to make sure that long term data is elsewhere - on a RAID5 or an occasionally connected external eSATA disk. If your are just doing standard-def, there should be little need to RAID modern drives.

[Bill Ravens]

Don...

Thanx for backing me up. I chronically forget the difference between MB and Mb, as the definition of bits and Bytes are counterintuitive. I get the diff between Mega, Micro, Giga and Nano, but, this base 8 business doesn't work well with me.

[Marcus Marchesseault]

Hard drives have been at their maximum physical speed for about 10 years. All 7200rpm drives have about the same access speed and the sustained transfer increases with density. The drives are not improving their rotational or head-seek speeds anymore. A RAID has no effect on this bottleneck of hard drives. The transfer speed of hard drives is far higher than what is needed for DV/HDV but the same old access speed slows things down if there are a lot of different pieces of your project that need to all be read simultaneously.

The only thing that could help with efficiency would be to get all the different sources of your project on different drives and also have a seperate drive for output/rendering. For instance, have one drive for the system, one for video track A, another for video track B, a separate drive for still images and/or sound clips, and a final one for the rendering output. This would keep all the drives working on only one thing at a time which would reduce accesses and that is the bottleneck of all drives.

A RAID only helps with transfer speed and that is already quite high. A RAID 0 also increases your data risk, as already pointed out earlier in this thread.

Your CPU is probably the bottleneck and there isn't much you can do about that without changing motherboard and memory type at the same time.

[Shaun Walker]

What if you're editing in the less-compressed Apple Intermediate CODEC if in FCP/FCE?? Is a RAID still not needed?
FW800 external good enough?
FW400?

[Harm Millaard]

Originally Posted by Shaun Walker

FW800 external good enough?
FW400?

For external storage I would suggest only eSATA, not FW nor USB. Maximum bandwidth for FW 400 is 400, USB is 480, FW 800 is 800 and eSATA is 3000. However bandwidth is only part of the story. Sharing the bandwidth with other devices on the same connection reduces the bandwidth significantly.

As to the question "good enough" depends on what you find acceptable, I cannot answer that for you. What is acceptable to me may be unacceptable to you or the other way around.

[Bill Zens]

Originally Posted by Marcus Marchesseault

.... one for video track A, another for video track B ... .

I don't have a clue how rendering works, but doesn't it work on where the video actually resides, not what tack it occupies in the timeline? And if so, then the video for track B would have to be in a separate drive from that of track A?

Otherwise, I've always wondered about the most realistic HD setup and this helped clarify it. ...While I have a separate drive for OS/Programs and one for vid, I use my third HD for excess storage of Video, which really isn't all that necessary. It would be better spent in defining it as the render disk and moving all my video to the bigger 400 mb drive. At the current price of HDD's it could be easily accomplished to add two more drives to my system.
Certainly, at least one more drive that would be a focused audio/SFX/Graphics drive might be of interest.

[Ron Evans]

The CPU does the rendering using RAM. It reads it from the source disc, renders in RAM( and swap disc if there isn't enough RAM) and then writes to the disc specified for the rendered output. Hence for the lowest disc load one should have seperate discs for each source file, rendered file and swap file. If one only has one disc for OS ( including the OS swap file), source video files and the rendered file then the data is being writen back and forth to the same disc many times..... not good for efficiency or performance!!!!!! Having seperate discs for OS, temp files/rendered files and source files, limits the tasks on any hard drive to one video stream at a time. This is a trivial task for most modern hard drives. Hard drives are cheap. My set up uses OS drive, Temp drive( which also has the backup image for the boot/OS drive) and 3 video drives.

Ron Evans

[Marcus Marchesseault]

Originally Posted by Shaun Walker

What if you're editing in the less-compressed Apple Intermediate CODEC if in FCP/FCE?? Is a RAID still not needed?
FW800 external good enough?
FW400?

I don't use external drives, so I couldn't make a good suggestion. Technically, even the lowly FW400 is more than fast enough for less-compressed formats as they are still only going to be in the range of 15-20 MBps (megabytes per second). A modern hard drive can do this easily. For instance, my 320 SATA drive tests at an average of 67 MBps and at the slowest inner cylinders rates at 42 MBps which can definitely handle something like the Cineform codec. The only thing I would be concerned about would be the ACTUAL speed of the interface once everything is considered. 400Mbps is fast enough as long as there aren't other devices on the connection bothering the hard drive. If you already have external firewire, give that a try before spending any money.

"If one only has one disc for OS ( including the OS swap file), source video files and the rendered file then the data is being writen back and forth to the same disc many times..... not good for efficiency or performance!"

That is a good explanation of what is happening or going wrong if only one drive is used. One drive is certainly possible, but it's not ideal.

***************

Here is the simplest way I can explain the situation with modern hard drives.

Hard drives are now very good at doing one thing at a time. They have plenty of transfer speed to read or write even a very beefy video file format (except UNcompressed).

Hard drives have had NO improvement in random access speed in several years. This makes hard drives today just as horrible at doing more than one thing at a time. The heads just can't be moved any faster back and forth across the platter.

*********

To improve performance, try to keep your hard drives each working on only one thing at a time. A RAID can not make your drives do more operations simultaneously. In fact, it makes two drives do one thing at a time. We generally need the opposite. To get more hard drive operations simultaneously, it takes more individual drives. Only UNcompressed video, with it's amazing data flow, would need a RAID system to increase data throughput.

Let's look at the optimal drive setup for uncompressed video:

A single drive for system files.

A RAID for each separate timeline video track. For instance, one for track A, another for track B, and another for overlay tracks.

A single drive for still images and/or additional audio clips.

A RAID for output.

Four RAIDs and two individual drives is a lot of hardware! Now you know why we compress video.

Since we compress video, a setup of three or four individual drives are more than sufficient.

[Marcus Marchesseault]

I'm going to look into using ATA to flash adapters to possibly enable using flash drives mounted inside the computer to take over some of the tasks done by hard drives. A 4gig flash drive could easily hold misc. files like audio tracks and stills from any of my projects. Heck, an 8gig flash drive would be big enough to render out .M2T mpeg HDV files from Vegas. I'm thinking that the best use of a flash drive in an editing system would be for storing overlay clips and occasional B-roll footage so that the primary video drive wouldn't need to bounce around between clips. The main video would go on the hard drive.

[Marcus Marchesseault]

My foray into documentation regarding flash drives as hard drives leaves me with this opinion:

It's almost ready for general use, but really only practical for laptops.

I use less than 8Gig for my system and software installation with Vegas and WinXP installed. I could trim it down a bit or substitute some less-used utilities to open up more space for big plugins. Regardless, 8Gig flash is now under $100 and the adapter is about $35.

So why isn't it completely ready? The overall speed isn't quite there. At first, 40Meg/second sound workable, but that's only the read speed. It seems that write speeds are only a little over half the read. If the sustained read and write was at least 40Meg/second, I would use one of these devices as my boot drive. As it is, we probably need to wait another generation before flash drives will be useful in editing systems.

For laptops, the story would be different. I would immediately look into switching laptop drives to flash. With the right flash card, the SATA to flash adpater can boot windows. Laptop drives are pathetically slow, so a flash drive would speed things up a bit and save power. There are 16Gig compact flash available, but the 8Gig seem faster.