HDD speeds

[Ed Kukla]

I'm building an edit system for my EX-3. I have a macbookpro with fw800 & CS4.

I'm trying to determine what type of HDD's I need for editing/storage. A lot of what I will be doing, I want to keep long term.

I could buy a set of fast HDD's for the actual editing and a set of slower USB drives just for backup.
I could buy Raid1 drives for simultanious backup and editing.
Or?

The reason for posting here instead of in the edit forums is I want to account for the speed needed to handle EX HD footage. How fast is fast enough? The projects will be fairly simple. 2 to 3 minute videos with basic cuts, disolves, some CC, a few titles, music bed & VO.

I would like portability as I will be travelling. Of course, cost is always a concern.

[Mitchell Lewis]

This was a real eye opener when I first learned that:

Standard Def DV = 25mbs
XDCAM EX = 35mbs

So most any hard drive will work okay. Although when you start converting things to ProRes HQ or worse yet, 10-but uncompressed HD. Then you will surely need a fast hard drive (RAID, etc...)

We have a 5-drive RAID that is striped RAID-5 so if one drive goes bad, the unit switches seamlessly to RAID-0 and you can keep working while you find a replacement drive.

In addition we have a larger, inexpensive drive that we use for daily backup using Apple's built-in Time Machine.

Once the RAID fills up, we archive off to portable 500GB bus powered hard drives and store them on the self. We use EMC Retrospect Desktop Backup software to keep track of everything.

[Mike Chandler]

Originally Posted by Mitchell Lewis

We have a 5-drive RAID that is striped RAID-5 so if one drive goes bad, the unit switches seamlessly to RAID-0 and you can keep working while you find a replacement drive.

Mitchell, I'm trying to decide on whether or not to go with RAID 1+0 vs RAID 5. Did you settle on the RAID5 config after comparing with 0+1 or 1+0, and, if so, did you see any performance hit in editing?

Welcome to AnandTech.com [ Article: IDE RAID Comparison]

It is with the real-world Content Creation 2001 benchmark that we can really see the toll that hardware RAID takes on the system. Both hardware RAID cards ended up performing the same in this benchmark, which left them a full 35% behind the single IDE drive.

As we can see, RAID 5 should be strictly reserved in cases where data integrity is of the utmost importance. It does offer advantages over a RAID 1 solution, as the array makes use of more hard drive space and has extra capabilities that RAID 1 can not offer. But with the rather noticeable performance hit that RAID 5 incurs, this RAID type is best left for servers with critical data but not much need for speed...

RAID 5 solutions should be reserved only for those who need it. We saw that a RAID 5 array can really decrease the performance of a set of drives, enough so that no home user will likely opt for this solution. RAID 5 is good in situations where data integrity is extremely important as well as in situations where the CPU is constantly under extreme load and disk access occurs on a regular basis. In these situations, the cons of RAID 5 are offset by the advantages of a very reliable data structure as well as very low CPU usage.

[Alan Kennedy]

I edit native XDcam with no conversion in Adobe PP3 and here is what I'm doing.

1) 2 x 500Gb 7200 rpm Seagates striped as RAID 0
2) 4 x 1Tb 7200 rpm Seagates in a DROBO

I use RAID 0 vs 1 for speed and only have media on the RAID that is backed up elsewhere. I use the DROBO for all my nearline backups (Drobo is redundant storage). I've had the 500Gb drvies for a couple of years so I'll probably replace them soon. If I was building today I would go with 2 x 1Tb. The important thing is that I don't put anything on the RAID other than video. Everything else goes to my project drive (I also have a scratch drive).

Alan

PS: I should add I am this is my home system that is only used for personal projects.

[Giroud Francois]

bandwith of video is ridiculously small in regard of hard disk speed.
XDCAM-EX is 35Mbps or about 4Mbytes/sec. Recent hard disk have 80 Mbytes/sec, so 20 time faster. Consequence of these highly compressed data is the huge amount of calculation required to uncompress/recompress the pictures. So you better will look for a fast computer than for a fast hard disk. The Core2 duo E8600 is rated a 3.2 ghz but can run easily around 4Ghz. Faster memory (DDR2 1066 and over instead classic DD2 800) will also makes the difference since video editing means moving huge quantity of memory.
The i920 is also a big performer when overclocked.

[Ed Kukla]

OK, here's a good one...what is a scratch disc or scratch drive?

I'm still trying to figure out a HDD config. I will want to keep clips long term. I'm thinking of getting pairs of drives, a FW 800 and a matching sized usb drive. Probably around 320 to 500gb for each pair. Use the usb drive as backup to the FW800 drive that would get the footage and then get backed up by the usb drive. I'm pretty green on configuring editing. Where should I put my project folders? I'm using a new mbp17

[Alan Kennedy]

A scratch drive is used as storage for temporary files such as renders, audio conforms etc. You certainly don't need a seperate drive for this! I just had a spare handy.

My system was built two years (dual core 2.66) has this configuration.

Inside the box:
1 x 160 Gb system drive - OS, software etc.
1 x 320 Gb Projects/Data drive - this is where I put my PremierePro and Encore projects, graphics etc.
2 x 500 Gb as RAID 0 - video material only
1 x 320 Gb drive for temp/scratch or any other replacable data

Outside:
4 x 1Tb Drobo - nearline storage of all current and past projects

No matter how you configure your dirves, the most important thing is backing up your data - drives do fail. I sync my data drive to the Drobo every day as well as periodic DVD backups. My video material exists on the Drobo as well as on a stored removable drive (soon to be backed up on Blu Ray). I don't backup my scratch or system drives.

As was stated earlier, the XDcam stream is small compared to the bandwidth of modern SATA drives. I have the RAID because I deal with a lot of uncompressed video (graphics).

If I was building a system today, at a minimum I would have a system, data and video drive - but that is just my opinion.

Alan

[Ola Christoffersson]

Originally Posted by Giroud Francois

bandwith of video is ridiculously small in regard of hard disk speed.
XDCAM-EX is 35Mbps or about 4Mbytes/sec. Recent hard disk have 80 Mbytes/sec, so 20 time faster. Consequence of these highly compressed data is the huge amount of calculation required to uncompress/recompress the pictures. So you better will look for a fast computer than for a fast hard disk. The Core2 duo E8600 is rated a 3.2 ghz but can run easily around 4Ghz. Faster memory (DDR2 1066 and over instead classic DD2 800) will also makes the difference since video editing means moving huge quantity of memory.
The i920 is also a big performer when overclocked.

This is true BUT you should know that the Long gop nature of XDCAM EX requires the editing program to read a lot more data to display a specific frame than just the data of that frame. I have found that using fast drives and raids around 100 MB/s gives much better response when moving through clips in Avid than if I use an external USB-drive (30-35 MB/s). So - in short - all drives can play back XDCAM EX but you need really fast drives and a fast computer to get the same tactile respons as you get when editing non long gop media.

[Marc Myers]

One only needs one hard drive crash to understand the value of RAID 5. IMHO, as individual drives have become larger they have become the more unsound. I've two systems, both with RAID 5, and both have a hard drive crash in the past year. Both times the RAID array saved the project. Dropped in a new drive and off I went. RAID 0 is faster but as described above, most of us don't need the speed.
I do use a separate scratch disk and that is not RAID. It it fails I can always regenerate that media from the project.

[Mike Chandler]

Understood. But what about the advantages/disadvantages of raid 5 vs raid 1+0 (or 0+1).

Originally Posted by Marc Myers

RAID 0 is faster but as described above, most of us don't need the speed.

When editing, we all need the speed.