best RAIDed hard-drive set-up for EX-1?

[Malcolm Hamilton]

Hi there,
I've been asking questions about hard-drive set-ups for editing on the Avid forum, but now I think I should ask here, because I'm starting to get the sense that it's as much about WHAT I'll be editing (EX-1 35Mbps) as what I'll be editing it WITH (Avid Media Composer, on a Mac... now a MacBook Pro, by the way, but a year from now I hope to get a Mac Pro).
Until I got the new EX, I'd been fine with external fw drives, but now I think I should consider a RAID 5 set-up that's more dependable, and faster.
I've divided my options into two categories: software-RAID units, and hardware-RAID units. People have been advising me regarding the differences, but I need the help of people on this forum, because a lot of Avid editors edit in SD, where software-RAID might be all they need; the thing is, I don't know if it'll do for EX-1 footage.
Would people who edit with EX-1 footage kindly advise me?
Here are my options in the software-RAID units:
- the Fusion 500p enclosure by sonnet technology. About $600; add your own SATA drives (and in my case, with a MacBook Pro, add the Express Card):
http://store1.sonnettech.com/product...91811a668f4ad0
- the Firmtek 5PM... also very highly regarded by some Avid editors. From the one review I read, the Firmtek is quiet, compared to some other options. This might sound trivial, but I think it's an important issue.
http://www.amug.org/amug-web/html/am...es/firmtek/5pm
- the G-Tech G-Speed eS drive. Looks gorgeous, and has a good name.

Only option I've found for hardware-RAID:
- the about-to-be-released Caldigit HDOne:
http://www.caldigit.com/HDOne/
which promises 20Gbps, and can be RAIDED in various ways (including my preferred RAID-5)... sounds fantastic. But it is $2,300 for the 2.5 TB model, which is expensive.
Anyway, I'd love some advice from people who've been editing with EX1 footage.
Thanks, malcolm

[Mike Stevens]

As I shoot docs in the desert I have moved to be totally mobile if required. I have a Sager core 2 17" laptop with a SIIG eSATA dual RAID express card going to two Western Digital 1T MyBooks. THE DRIVES, WIRE AND EXPRESS CARD ABOUT $560 FROM B&H.

I carry a Xantec power-pack supported by a 12v deep cycle battery from Costco that charge from the truck. It works great and even in the deepest desert I run out of wine before I run out of power or storage space.

Mike

[Barry J. Anwender]

Originally Posted by Malcolm Hamilton

Here are my options in the software-RAID units:
- the Fusion 500p enclosure by sonnet technology. About $600; add your own SATA drives (and in my case, with a MacBook Pro, add the Express Card):
http://store1.sonnettech.com/product...91811a668f4ad0

Malcolm, I use Sonnet's Fusion 500P with my MacPro 3.2 Octo, Leopard 10.5.2 software RAID and FCP Studio 2.0. I've used Sonnet RAID products for the last 8 years, the Fusion 500P for the last 2 years. One thing you might want to be concerned about is noise levels and heat management of the RAID unit. I choose the Fusion 500P because it is quite and provides lots of air flow to keep the hard drives cool. Sonnet has excellent engineering, reliability and they provide top notch support.

The Fusion 500P has performed flawlessly for me, running continuously for weeks at a time, with uncompressed 10-bit 4:2:2 video all the way up to 1920x1080/60i, i.e. data rates on the order of 158 MB/s. The caution with SATA drives is that they reduce data transfer speed as they become full. However, the EX-1 video data rates are no where near the maximum capabilities of a Fusion 500P RAID.

By the way, I tried to respond to your PM over the weekend but my reply bounced back saying: Unknown address error 550-'No such recipient'

Regards!

[Jonathan Bland]

Good Post Malcom. I'm in the same boat with a Mac Powerbook and international travel while editing.

Been hearing good things about the Sonnet stuff but for me a big box full of hard drives that weighs 30lbs is....... going to get expensive and combersome during travel.

Anyone with a decent solution to:

Storage while shooting..... while cutting..... while being transplanted to various countries???

Barry:
Where in Canada did you get your Sonnet?

Malcolm I just finished your posts in the avid forum and needless to say I'm very interested to see where you go on this.

The new Firmtek SeriTek/5PM with the Samsung Spinpoint 1TB drives seems like a very good option...

[Sean Donnelly]

Personally I go with HW RAID whenever possible, especially with higher levels like 3,5, and 6. Software means that every time you access the disk the processor gets pulled away to encode/decode which negates a lot of the speed you get. CalDigit makes fantastic RAID systems, I'm actually looking at the same one for on-set capture with much higher end systems. The data rate on XDCAM EX at 35mbps is only 4.4 MBps, not a big strain on an array that can handle 20gb/s, or 2.5 GB/s. I currently use a Ciprico U320RX 10 disk RAID 30 (2x 5 disk RAID 3 striped into a RAID 0) over dual channel U320 SCSI and sustains over 400 MB/s according to the Black Magic speed test. With that I can work with uncompressed 16 bit 2k image sequences, which require 307MB/s for consistent playback. I can't speak to the noise and other physical attributes, but it would be able to keep up with just about anything you could need in the coming years.

-Sean

[Harm Millaard]

Originally Posted by Sean Donnelly

The data rate on XDCAM EX at 35mbps is only 4.4 MBps, not a big strain on an array that can handle 20GB/s.

Where did you get that figure? I mean of course the array that can handle 20 GB/s. It is quite a contrast to the effective transfer rate of only 400 MB/s. That is a factor 50 difference.

[Sean Donnelly]

OOPS! Should be 20 gb/s or 2.5 GB/s, I'll fix it now. Thanks for noticing that, I must have gotten carried away with the shift key.

-Sean

[Paul Cronin]

Dulce RAID is excellent and you can get all the help you need from a DVinfo.net sponsor http://www.zotzdigital.com/. They are dialed in and will get you into a great product for a great price.

I have a 4TB Dulce Raid with 8 500GB drives and it has been flawless now for 8 months of hard use.

[Malcolm Hamilton]

Originally Posted by Sean Donnelly

Personally I go with HW RAID whenever possible, especially with higher levels like 3,5, and 6. Software means that every time you access the disk the processor gets pulled away to encode/decode which negates a lot of the speed you get. CalDigit makes fantastic RAID systems, I'm actually looking at the same one for on-set capture with much higher end systems
-Sean

Hi Sean,
I agree that this Caldigit unit is fantastic, and I understand from these posts that hardware RAID is much much better, but the cost! The Caldigit HDOne starts off as quite an expense (enclosure + 2TB worth of drives is $2,300), and then you have to buy their hd modules when you want to expand, and a 1T Caldigit module is $699.

With all the other options, like the Firmtek and the Sonnet and the G-Tech (with all of these, the basic driveless enclosure is about $600), you can buy standard SATA drives... now maybe Caldigit is more selective about its choice of SATA drives, and maybe there's something better about them, I don't know. But the new Samsung Spinpoint 1 TB drive, for example (that runs cooler because its got just three platters and comes with a three-year guarantee, I think) is just $219 on Newegg.

So... I hear what you're saying, and I know there are very good arguments for hardware RAID, and if I had lots of money, I'd buy one right now. But because I'm short of cash, I'm wavering. I'll spend it if someone tells me they had a software-RAID system that they had to put aside once they started editing with XDCAM EX 35mbps footage... and that they are really happy to have gone to a hardware-RAID system, despite the cost.

On the other hand, if I'm told that people are happily using software-RAID systems, or I should say, if they're happily using such a system in RAID-5 configuration (Barry seems to have put in a vote for a software-RAID set-up, but Barry, is that RAID-5?), then I think I'll try to go with the cheaper set-up.

Thanks so much for all the observations and advice.
Cheers, Malcolm

[Barry J. Anwender]

Originally Posted by Jonathan Bland

Barry:
Where in Canada did you get your Sonnet?

Jonathan, I purchased from Sonnet's Web store with FedEX delivery. It arrives on my door step 2-3 days after the order is processed and assuming the items are in stock.

[Barry J. Anwender]

Originally Posted by Malcolm Hamilton

Barry, is that RAID-5?

Malcom, Apple's software RAID is either RAID-0 (i.e. stripped) to achieve those high performance data rates or RAID-1 (i.e. mirrored) for protecting against disk errors. You can also configure variations of this, say two sets of stripped drives concatenated in a mirror configuration to achieve a balance of speed and data integrity.

Here is the thing. If one needs to concatenate, say 10 drives as mentioned above or more than say 5 drives, then yes by all means you will need RAID-5. Also to achieve those high data transfer rates and high reliability at those rates you have two more considerations to make. One is that reputable RAID manufacturers insist that you use the Enterprise version (read more expensive model) of the hard drive which has specialized firmware to manage among other things high speed RAID command queing. Second the enterprise drives are designed for demanding environments, both in usage and in physical treatment. Hence, the reason for RAID manufactures supplying the drives with the case--at a higher cost.

You may also have observed that RAID manufacturers worth their salt, provide a dedicated PCI-e hardware card with specialized features to work with their RAID drive enclosure and not necessarily an e-SATA interface. Again the reason for this is to achieve those ultra high data transfer rates (i.e. say higher than 200 MB/s). Apple provides such a dedicated PCI-e RAID card for their MacPro (early 2008) to work with those expensive SAS drives or high performance SATA RAID's. You will find these configurations in studios with large editing suites and multiple computers accessing the RAID server(s).

I am looking forward to working with the EX-3 SDI 4:2:2 at 24p, my Fusion 500P and Blackmagic Multibridge Extreme SDI interface to FCP Studio 2.0. Based upon discussions with the Blackmagic and Sonnet support staff, this should be viable and a reliable configuration to suit my needs.

The bottom line for me is that EX-1/3 data rates at 35 MB/s do not warrant a hardware RAID and FCP will support enough video streams for my editing purposes without the high data rates from very large RAID servers. Then again, I am not familiar with AVID and so it's requirements may indeed warrant a more sophisticated RAID. Clearly, if you plan on transcoding to work with a lot of uncompressed video streams then that's another story.

At this time, folks are also waiting for a viable portable SDI converter before they can try to interface the EX-1/3 SDI port to their Laptop computers, but again they will be very expensive. There is a thread in these forums to get up to speed on the portable SDI arena.

[Jon Braeley]

Most of the brand name Raid set ups are out of my price range, until I get a very deep pocketed client (I am an independent doc maker). However, I did put together a 3Tb Raid system that works for $800!
I bought a generic 4 drive hot-swop eSata enclosure with a PCI-x card for $300 and I installed 4 x Seagate 7200 750Gig drives at $129 ea.
At first I realized my PCI-x card was Non-raid, and so the suppliers, then sent me a Raid5 card.
It is not superfast, but I overcame any problems by using ProRes HQ set up and so far after 3 weeks of heavy useage, it is working very well.

[Jonathan Bland]

Sorry if this sound nieve.....

This just occured to me:

Can I edit uncompressed full res from the EX1 on a new Macbook Pro with one of these external sata raid set-ups like Fibertek/ Sonnet OR will I need to compress it (Pro res?) to work with it.

[Jonathan Bland]

Thanks Barry.

Can you take a quick shot at my question about editing uncompressed?

[Sean Donnelly]

Jonathan, the short answer is no. There is no way to capture uncompressed HD-SDI into a Macbook pro, at least not a feasible one (maybe a magma box with a black magic decklink pro card...but I don't know if the interface can keep up). What you CAN do, and I did this last week, use an AJA io box to capture into prores. It is miles above XDCAM, and for most purposes all that you need. All is takes to keep up with that is a decent eSata 2 disk RAID-0, like a g-tech or silversata.