How much faster is editing with a RAID? at DVinfo.net

David Delaney · January 17th, 2009, 11:15 AM

I am in the process of researching RAID for my editing experience. I am going to see much of a difference? With drives so cheap, it seems like a no-brainer, but I wonder with new solid-state media getting larger and dropping in price - is that going to be the way to go anyway?

Jeff Harper · January 17th, 2009, 01:01 PM

As a former RAID 0 entusiast, I'd submit that you would do well to purchase a 300GB Velociraptor, or 150 depending on your needs and forego RAID.

Those drives are the fastest non-SAS drives available, and any improvement running them in RAID would be relatively insignificant, IMO. This has been my experience.

Use the Velocipraptor drive for the video files of whatever you are editing, and then move them off to your storage drive, etc afterwards.

If you have the extra money and want to run RAID anyway it sure wouldn't hurt.

I personally run a 150GB for my OS and use the 300GB size as my scratch drive. Previously I ran the older Raptors in RAID 0 and when I upgraded to the Velociraptors I ran two 300s in RAID 0 for awhile and saw no benefit. When I stopped using RAID I gained another drive to use for storage.

The Velociraptors are fast enough that I came to beleive the system overhead required to run the contoller overall hurt my system performance more than the RAID helped.

If you have money to burn I would look at an i7 PC with and install 10K drives before I would invest in RAID on an older unit. That is where you would find the most benefit.

Of course the Asus P6t MOBO has built in RAID, so if you went the DIY route you'd have it all. But keep in mind there are many who don't find the integrated controllers so great. I'm one of them. If you go RAID, I recommend a good ADAPTEC PCIe controller card and skip the integrated contoller.

The ASUS board mentioned also has SAS controller, which means you could purchase 15K drives, but I have no experience with SAS and cannot recommend, though they are reputed to be very good, and some say safer for your data than conventional SATA drives.

Perrone Ford · January 17th, 2009, 01:17 PM

A lot is going to depend on what you're editing, and what format you edit in. If you're editing miniDV or Mpeg2 files, you probably won't see that big a difference. If you've got 3-8 hours of uncompressed 1080p to push around including titles, transitions, surround sound, effects, etc., then yes, you're certainly going to see a difference.

The Velociraptors look interesting, but I work in an a SAS SAN environment. When you're used to having 100 320GB SAS drives, the Velociraptors fall a bit flat. :) That said, they'd be a lot more interesting if they weren't so small. A year from now, when they reach 1TB, they might be far more interesting.

My usual shoots are long conferences. 2-3 days of 5-8 hours per day. When I come home with that data in 1080p and have to dump it, then transcode, I need lots of room. and the 300GB drives just aren't going to cut it. For my 1-2 hour projects, they may be just the ticket.

David Delaney · January 17th, 2009, 01:27 PM

But shouldn't the main drive with the OS and the video file drive both be RAID to get the most benefit? Thanks for the post, that is very interesting. I don't have the money to burn and would rather upgrade my computer so forgoing the RAID might be just the ticket.

Perrone Ford · January 17th, 2009, 01:33 PM

Nah, the OS drive is pretty quiet compared to what's happening on the video drives. Just get a decent drive for that, and call it a day. The video drives are the business end of the system.

David Delaney · January 17th, 2009, 01:36 PM

So even though VEGAS resides on a regular drive, as long as the RENDER TO drive and the VIDEO FILES LOCATION is a RAID drive, I should see the increase in speed?

Ron Evans · January 17th, 2009, 09:06 PM

If you have OS on one drive and a RAID of two drives that you feed from and render to you will have slower performance than two separate drives that you feed from one and then render to the other. The speed increase for RAID is about 1.7 times a single drive however if you are reading from and writing to the same RAID drive it has to perform BOTH tasks at the same time. IT will likely have to use the Paging/Swap file on C drive to cache information to write back to itself and will have at least twice the seeks for read and write that single drives have. With enough RAM the single drives can read into memory get processed by CPU and read out to the other drive. Two RAID ( at least 4 drives) will be faster but at this point I expect the CPU will be the bottleneck!!!! For DV or HDV it makes very little difference using modern hard drives as they can sustain over 50MBps even 80% full and DV and HDV only need 3.5MBps. It is the random seek times that will cause more problems if the drive has to find many small clips etc from several tracks at the same time and is also fragmented. I still don't think its a problem and personally do not use RAID on my system but do have separate drives for OS, Temp storage and for video storage and always render to a drive that does not contain the source video. Uncompressed is different as this will need high throughput from the hard drives AND a fast CPU. I would look first at what you are going to do and if you really need a RAID at all. IT adds complication, RAID 0 has serious fault condition ( if one drive fails you loose everything) speed use is questionable for lots of applications but it does allow one to be sloppy about directory management--just save to one big drive.
Ron Evans

Devin Termini · January 18th, 2009, 12:52 PM

Quote:

Originally Posted by Jeff Harper

Of course the Asus P6t MOBO has built in RAID, so if you went the DIY route you'd have it all. But keep in mind there are many who don't find the integrated controllers so great. I'm one of them. If you go RAID, I recommend a good ADAPTEC PCIe controller card and skip the integrated contoller.

I don't mean to hijack the thread here. Could you provide an example of a controller card that improves over the integrated raid solutions?

I'm assuming most cards would be an improvement, I just need to be pointed in the right direction.

Jeff Harper · January 18th, 2009, 01:06 PM

I can't recommend a particular card, as I don't know anything about your PC. For RAID cards I have always used Adaptec. Just go to their site and look around. Newegg doesn't sell them by the way.

It will take a bit of research, which you must do on your own. Looking around the Adaptec site you will learn a bit about the whole deal which would benefit you and help you to know what to buy and how it works.

J.K. Ahn · January 19th, 2009, 02:38 AM

I use a SAS drive RAID 0 array running off an Adaptec card and it is very fast. As mentioned above, it's not as simple as just having RAID, you need to configure your system to take advantage of it.

Also, remember that a 3 drive RAID 0 array is faster than a 2 drive one but with a 50% increase in drive failure risk over a 2 drive one. A 4 drive array is faster still but with double the drive fail risk.

Jason Robinson · January 19th, 2009, 11:54 AM

Quote:

Originally Posted by Devin Termini

I don't mean to hijack the thread here. Could you provide an example of a controller card that improves over the integrated raid solutions?

I'm assuming most cards would be an improvement, I just need to be pointed in the right direction.

If you want fast & secure, go find a raid card that does hardware XOR for Raid 5 and put 3 drives in it. One drive worth of storage is eaten up just to store the parity bit (aka, how to reconstruct the data from a failed drive).

Reads will be very fist, writes, slower but still fast. The real down side is that a RAID card that can do hardware raid 5 will start at $300.

I've been using RAID based systems for 7 years. I built a dual Athlon MP 1.5GHz system waaay back when and bought a hardware RAID 0/1/10 card. That system is just as fast to render my projects as my P4 3Ghz HT system with RAID-0 and 5x the ram.

When I compare my P4 system with RAID (a single RAID for OS & data) to my Core 2 Duo system with out RAID (a single drive for OS & data), the Core 2 Duo renders much faster, but is terrible as a render host.

I have lots of other posts here on DVInfo detailing my RAID vs non-RAID systems and their performance depending on if network rendering is used or not. Hit up search to find them.

Hank Coffin · January 19th, 2009, 01:25 PM

Dollar-for-dollar RAID probably isn't worth it unless you are working in uncompressed HD, when it becomes a requirement. I spent more time and money than I bargained for upgrading my system to RAID. I got a nice ($$$) 3ware RAID card that didn't fit my original motherboard, so I upgraded that. Then I had a problem with one of the four Seagate drives I was using on my OS volume, which was RAID 5. I ended up installing WD drives as my OS volume.

RAID 0 works better with > 2 drives. Just 2 drives and you probably won't notice much of a difference.

RAID 5 is cool, fast and reliable, but requires an add-on card.

Integrated motherboard RAID solutions are probably fine, except they will tax the CPU a little more than a dedicated controller. I'm also using my Intel ICH10 RAID, and that seems fine, too.

Make sure the drive manufacturer recommends the drive for > 2 drive RAID configurations. Sure, you might end up with a system that works, but for a few extra bucks you'll have more confidence and can at least complain. Probably avoid the new Green drives for RAID.

Just some of my 2 cents...

Jon McGuffin · January 19th, 2009, 06:13 PM

Quote:

Originally Posted by Hank Coffin

Dollar-for-dollar RAID probably isn't worth it unless you are working in uncompressed HD, when it becomes a requirement. I spent more time and money than I bargained for upgrading my system to RAID. I got a nice ($$$) 3ware RAID card that didn't fit my original motherboard, so I upgraded that. Then I had a problem with one of the four Seagate drives I was using on my OS volume, which was RAID 5. I ended up installing WD drives as my OS volume.

RAID 0 works better with > 2 drives. Just 2 drives and you probably won't notice much of a difference.

RAID 5 is cool, fast and reliable, but requires an add-on card.

Integrated motherboard RAID solutions are probably fine, except they will tax the CPU a little more than a dedicated controller. I'm also using my Intel ICH10 RAID, and that seems fine, too.

Make sure the drive manufacturer recommends the drive for > 2 drive RAID configurations. Sure, you might end up with a system that works, but for a few extra bucks you'll have more confidence and can at least complain. Probably avoid the new Green drives for RAID.

Just some of my 2 cents...

Based on my experiences, for what it's worth, I fully endorse what Hank here just said. Let his hard earned lesson not fall on deaf ears (or eyes).

Jon

Edward Troxel · January 19th, 2009, 07:57 PM

Bottom line - wherever the bottleneck is is where you need to be looking. If your hard drives are not keeping up, a raid could help. Right now I'm converting some files - from and to the same external drive. Having a faster connection/drive would NOT help in any way. The red light is just blinking every now and then and the processor is pegged at 99%. Would a raid help, absolutely not.

Now, if the red light was on constantly and the processor was at 10%, would a raid help? Possibly but even then testing would be needed to confirm.

There's so many factors that relate to "speed" that it's hard to say that a raid system would absolutely help.

January 17th, 2009, 11:15 AM	#1
David Delaney Trustee Join Date: Feb 2006 Location: Niagara Ontario Canada Posts: 1,121	How much faster is editing with a RAID? I am in the process of researching RAID for my editing experience. I am going to see much of a difference? With drives so cheap, it seems like a no-brainer, but I wonder with new solid-state media getting larger and dropping in price - is that going to be the way to go anyway?

January 17th, 2009, 01:17 PM	#3
Perrone Ford Inner Circle Join Date: Nov 2006 Location: Tallahassee, FL Posts: 4,100	A lot is going to depend on what you're editing, and what format you edit in. If you're editing miniDV or Mpeg2 files, you probably won't see that big a difference. If you've got 3-8 hours of uncompressed 1080p to push around including titles, transitions, surround sound, effects, etc., then yes, you're certainly going to see a difference. The Velociraptors look interesting, but I work in an a SAS SAN environment. When you're used to having 100 320GB SAS drives, the Velociraptors fall a bit flat. :) That said, they'd be a lot more interesting if they weren't so small. A year from now, when they reach 1TB, they might be far more interesting. My usual shoots are long conferences. 2-3 days of 5-8 hours per day. When I come home with that data in 1080p and have to dump it, then transcode, I need lots of room. and the 300GB drives just aren't going to cut it. For my 1-2 hour projects, they may be just the ticket. __________________ DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.

January 17th, 2009, 01:33 PM	#5
Perrone Ford Inner Circle Join Date: Nov 2006 Location: Tallahassee, FL Posts: 4,100	Nah, the OS drive is pretty quiet compared to what's happening on the video drives. Just get a decent drive for that, and call it a day. The video drives are the business end of the system. __________________ DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.

January 19th, 2009, 07:57 PM	#14
Edward Troxel Sponsor: JET DV Join Date: Dec 2001 Location: Southern Illinois Posts: 7,953	Bottom line - wherever the bottleneck is is where you need to be looking. If your hard drives are not keeping up, a raid could help. Right now I'm converting some files - from and to the same external drive. Having a faster connection/drive would NOT help in any way. The red light is just blinking every now and then and the processor is pegged at 99%. Would a raid help, absolutely not. Now, if the red light was on constantly and the processor was at 10%, would a raid help? Possibly but even then testing would be needed to confirm. There's so many factors that relate to "speed" that it's hard to say that a raid system would absolutely help. __________________ Edward Troxel [SCVU] JETDV Scripts/Scripting Tutorials/Excalibur/Montage Magic/Newsletters

January 17th, 2009, 01:01 PM	#2
Jeff Harper Inner Circle Join Date: Jun 2005 Location: Cincinnati, OH Posts: 8,425	As a former RAID 0 entusiast, I'd submit that you would do well to purchase a 300GB Velociraptor, or 150 depending on your needs and forego RAID. Those drives are the fastest non-SAS drives available, and any improvement running them in RAID would be relatively insignificant, IMO. This has been my experience. Use the Velocipraptor drive for the video files of whatever you are editing, and then move them off to your storage drive, etc afterwards. If you have the extra money and want to run RAID anyway it sure wouldn't hurt. I personally run a 150GB for my OS and use the 300GB size as my scratch drive. Previously I ran the older Raptors in RAID 0 and when I upgraded to the Velociraptors I ran two 300s in RAID 0 for awhile and saw no benefit. When I stopped using RAID I gained another drive to use for storage. The Velociraptors are fast enough that I came to beleive the system overhead required to run the contoller overall hurt my system performance more than the RAID helped. If you have money to burn I would look at an i7 PC with and install 10K drives before I would invest in RAID on an older unit. That is where you would find the most benefit. Of course the Asus P6t MOBO has built in RAID, so if you went the DIY route you'd have it all. But keep in mind there are many who don't find the integrated controllers so great. I'm one of them. If you go RAID, I recommend a good ADAPTEC PCIe controller card and skip the integrated contoller. The ASUS board mentioned also has SAS controller, which means you could purchase 15K drives, but I have no experience with SAS and cannot recommend, though they are reputed to be very good, and some say safer for your data than conventional SATA drives.

January 17th, 2009, 01:27 PM	#4
David Delaney Trustee Join Date: Feb 2006 Location: Niagara Ontario Canada Posts: 1,121	But shouldn't the main drive with the OS and the video file drive both be RAID to get the most benefit? Thanks for the post, that is very interesting. I don't have the money to burn and would rather upgrade my computer so forgoing the RAID might be just the ticket.

January 17th, 2009, 01:36 PM	#6
David Delaney Trustee Join Date: Feb 2006 Location: Niagara Ontario Canada Posts: 1,121	So even though VEGAS resides on a regular drive, as long as the RENDER TO drive and the VIDEO FILES LOCATION is a RAID drive, I should see the increase in speed?

January 17th, 2009, 09:06 PM	#7
Ron Evans Inner Circle Join Date: Mar 2003 Location: Ottawa, Ontario, Canada Posts: 4,220	If you have OS on one drive and a RAID of two drives that you feed from and render to you will have slower performance than two separate drives that you feed from one and then render to the other. The speed increase for RAID is about 1.7 times a single drive however if you are reading from and writing to the same RAID drive it has to perform BOTH tasks at the same time. IT will likely have to use the Paging/Swap file on C drive to cache information to write back to itself and will have at least twice the seeks for read and write that single drives have. With enough RAM the single drives can read into memory get processed by CPU and read out to the other drive. Two RAID ( at least 4 drives) will be faster but at this point I expect the CPU will be the bottleneck!!!! For DV or HDV it makes very little difference using modern hard drives as they can sustain over 50MBps even 80% full and DV and HDV only need 3.5MBps. It is the random seek times that will cause more problems if the drive has to find many small clips etc from several tracks at the same time and is also fragmented. I still don't think its a problem and personally do not use RAID on my system but do have separate drives for OS, Temp storage and for video storage and always render to a drive that does not contain the source video. Uncompressed is different as this will need high throughput from the hard drives AND a fast CPU. I would look first at what you are going to do and if you really need a RAID at all. IT adds complication, RAID 0 has serious fault condition ( if one drive fails you loose everything) speed use is questionable for lots of applications but it does allow one to be sloppy about directory management--just save to one big drive. Ron Evans

January 18th, 2009, 01:06 PM	#9
Jeff Harper Inner Circle Join Date: Jun 2005 Location: Cincinnati, OH Posts: 8,425	I can't recommend a particular card, as I don't know anything about your PC. For RAID cards I have always used Adaptec. Just go to their site and look around. Newegg doesn't sell them by the way. It will take a bit of research, which you must do on your own. Looking around the Adaptec site you will learn a bit about the whole deal which would benefit you and help you to know what to buy and how it works.

January 19th, 2009, 02:38 AM	#10
J.K. Ahn New Boot Join Date: Dec 2008 Location: London, UK Posts: 7	I use a SAS drive RAID 0 array running off an Adaptec card and it is very fast. As mentioned above, it's not as simple as just having RAID, you need to configure your system to take advantage of it. Also, remember that a 3 drive RAID 0 array is faster than a 2 drive one but with a 50% increase in drive failure risk over a 2 drive one. A 4 drive array is faster still but with double the drive fail risk.

January 19th, 2009, 01:25 PM	#12
Hank Coffin Regular Crew Join Date: Aug 2008 Location: Sacramento, CA Posts: 25	Dollar-for-dollar RAID probably isn't worth it unless you are working in uncompressed HD, when it becomes a requirement. I spent more time and money than I bargained for upgrading my system to RAID. I got a nice ($$$) 3ware RAID card that didn't fit my original motherboard, so I upgraded that. Then I had a problem with one of the four Seagate drives I was using on my OS volume, which was RAID 5. I ended up installing WD drives as my OS volume. RAID 0 works better with > 2 drives. Just 2 drives and you probably won't notice much of a difference. RAID 5 is cool, fast and reliable, but requires an add-on card. Integrated motherboard RAID solutions are probably fine, except they will tax the CPU a little more than a dedicated controller. I'm also using my Intel ICH10 RAID, and that seems fine, too. Make sure the drive manufacturer recommends the drive for > 2 drive RAID configurations. Sure, you might end up with a system that works, but for a few extra bucks you'll have more confidence and can at least complain. Probably avoid the new Green drives for RAID. Just some of my 2 cents...