Fast, inexpensive array for HVX users at DVinfo.net

Robert Lane · December 6th, 2006, 10:09 AM

Whether you're a new or experienced HVX user one thing of concern to all (as noted by the multiple threads on the topic) is both storage of P2 clips and fast access either for grabbing files and especially for editing. Fast access always means an array of some sort, be it e-SATA, SCSI or fiber. But arrays regardless of which kind usually mean spending thousands of dollars both in array hardware and HDD's themselves. This is an alternative to the ultra-expensive arrays - and it works, very well (test result enclosed).

This setup comes with some very important caveats, so pay attention before you spend your money or ask tons of questions:

- This is an e-SATA array, meaning there is no on-board controller cache and the only cache available is what the HDD's themselves have. That translates into the fact that the array will start out very fast however, as it gets continually used throughout an edit or if multiple file transfers occur it will slow down, sometimes to half it's original speed depending on use. Speed is regained when the individual drive cache is flushed (usually during system reboot or over time if the drives are not used for about 10 minutes or so). Until there are e-SATA HBA's that have on-board cache there is no way around this performance characteristic.

- If you have never setup an array or don't understand how they work, don't start spending your money on this setup UNTIL you get some research done or training by someone who can walk you through setup and usage.

- This setup, while very cost-effective and inexpensive, is more complex in that it uses multiple enclosures each with it's own power supply. That means there are multiple points of failure since you're dealing with more than one independent power supply.

- This setup does not allow for "hot-swap" as the drives are in a fixed enclosure - more on that as the hardware is discussed. Here's the physical setup:

(1) Highpoint Technologies "RocketRAID" 2322 PCI-e HBA w/ (2) mini-SAS to e-SATA cables. (that's a total of 8 individual drive cables) $295 - Cables are separate at $45 each depending on where you get them. $385 total

(4) OWC Mercury Elite-AL Dual Bay SATA Enclosures (2 drives per enclosure, direct JBOD connection) $75 each / $300 total

That's an 8-drive e-SATA array for less than $800 (not including HDD's)!

How fast is it? Using the KONA System Test I ran it using the same codec I shoot in DVCPRO-HD 720p and got just under 280 mb/s READ with a 1gb file test. That's damned impressive for an array that was thousands less than a SCSI setup. This test was done with (8) WD5000YS drives in a RAID-0 configuration. AMUG and BARE-FEATS tested this card with faster numbers but they used a Mac TERMINAL application to do the test, I prefer to use the KONA test because it's specific to mimicking a video-edit environment, so I feel the numbers are more real-world and not hype.

The RocketRAID 2322 is the first e-SATA HBA to create the array in HARDWARE not software, so there's no OS overhead being used for the array management. It's also the first HBA to allow for 8-drive access externally and, (2) RR2322's can be connected in parallel for a total of 16 drives! (this functionality in MacOS requires Disk Utility to join them, in XP you just add a jumper cable and the cards do the work). The card will do RAID 0, 1, 5, 10, and parity on 5, 10.

The RR2322 does not currently support the newer 750gb drives, but that's probably going to change soon.

The OWC enclosures are unique; they do not have HOT-SWAP trays, the drives are fixed so you lose the ability to quickly replace a drive. They also have their own individual power supply, so that means cabling becomes a bit of a mess in this tested config: (8) e-SATA data cables and (4) power cables between (4) enclosures.

The OWC "black boxes" are whisper quiet; with all 4 running I barely heard the fans in the back.

This is not an ideal setup for everybody, this setup is geared to those who want or need a fast array for accessing OR storing their P2 files but don't have mega-bucks for a SCSI or fiber array. Here's a cost comparison (minus the cost of HDD's, HBA and cables)

(4) OWC "Black boxes" as described above: $300
(2) Sonnet Fusion 400 enclosures (8 drives total, hot-swap capable, e-SATA connection): $760
(2) Wiebetech SilverSATA IV (same as above): $900
(1) Enhance-Tech T8 SCSI 8-drive enclosure (SATA to SCSI backplane): $2200
(2) Fibrenetix Qubex Fibre 6-drives x2 enclosures (SATA to Fibre backplane): $8100

As you can see, as you go up the food-chain of faster connectivity the costs skyrocket, sharply. Add the cost of HDD's and the HBA and you'll see how cost-effective this e-SATA suggested setup is.

It should be noted however, that no e-SATA array regardless of enclosure or connection type is as fast nor as data-stable (constant data-frame-speed) as SCSI or fiber. But, for the added speed and stability you really have to cough up some serious buckage - something most indies just can't afford or justify.

Tom Chartrand · December 6th, 2006, 10:23 AM

Quote:

Originally Posted by Robert Lane

Whether you're a new or experienced HVX user one thing of concern to all (as noted by the multiple threads on the topic) is both storage of P2 clips and fast access either for grabbing files and especially for editing. Fast access always means an array of some sort, be it e-SATA, SCSI or fiber. But arrays regardless of which kind usually mean spending thousands of dollars both in array hardware and HDD's themselves. This is an alternative to the ultra-expensive arrays - and it works, very well (test result enclosed).

How does this compare with just putting together 3 or 4 drives in the new macpro and doing an internal raid for SD and DVCPRO-HD stuff?

Robert Lane · December 6th, 2006, 10:37 AM

First, you can't put 4 drives together in a RAID on the Mac Pro; one drive needs to be reserved for the OS so that limits you to 3 drives.

OWC did a test where they used 3 SEAGATE 750gb drives and got about 290mb/s READ and a little faster WRITE (might have the figures backwards). I don't own a MacPro (yet) and have not tested that config myself so I can't comment on actual usage or numbers.

The main drawbacks to that setup are that you can only get a maximum of 2250gb (about 1980gb formatted) using the 3 extra internal MacPro HDD's and, it's internal meaning they are always running when the system is turned on. With any external array if you don't need to use the array you can leave it powered off and save the added time and heat wear on the array drives. Personally, I would not setup an internal array and would instead use those drives for either backup storage and or, assigning FCP/Photoshop cache files to them to alleviate the contantly changing cache files to be written to the array.

I would also reserve use on one the of the internal HDD's for creating a bootable software repair partition, such as for Tech Tool Pro or Disk Warrior in case things go south.

My advice for anyone currently using a MacPro or considering it is to be aware of the 2GB RAM limitation. Currently there are third party devices such as HBA's which are causing kernel panics on MacPro's with more than 2GB of RAM installed. See BARE-FEATS about this issue and how things are shaking out with bug fixes.

Barry Kay · December 7th, 2006, 09:48 AM

Thanks for the info; I'm looking forward to upgrading Everything to HVX so when online window-browsing I saw this new RAID 500GB product from LaCie for only $399 (!). Their promo says:

" a compact 2-disk device, the LaCie Two Big eSATA & USB comes with the fastest interface - eSATA (SATA II 3Gbits) and Hi-Speed USB 2.0 for connecting to almost any computer."

Compared to prices in the past and prices you mention this would seem like a great deal, plus I've heard the fact that not being able to take your RAID to other computers can be a hassle so the USB connection is valuable.

It's also hot-swappable and can be found at: http://www.lacie.com/products/product.htm?pid=10847

David Saraceno · December 7th, 2006, 10:18 AM

Quote:

Originally Posted by Robert Lane

First, you can't put 4 drives together in a RAID on the Mac Pro; one drive needs to be reserved for the OS so that limits you to 3 drives.

Not really. I have the boot drive in the second optical bay.

The remaining four drives can be RAIDed.

Robert Lane · December 7th, 2006, 11:33 AM

[QUOTE=Barry Kay]Thanks for the info; I'm looking forward to upgrading Everything to HVX so when online window-browsing I saw this new RAID 500GB product from LaCie for only $399 (!). Their promo says:

" a compact 2-disk device, the LaCie Two Big eSATA & USB comes with the fastest interface - eSATA (SATA II 3Gbits) and Hi-Speed USB 2.0 for connecting to almost any computer."[QUOTE]

Barry,

This device is mainly for storage, not an array for editing as it won't have the throughput required for multiple-stream previews and fast renders. Any JBOD e-SATA array would need a minimum of 4 drives in RAID-0 (as tested with the KONA System Test) to sustain the required througput and, it also requires an HBA interface not USB or Firewire as that throughput is limited to about 90mb/s.

Jeff Kilgroe · December 7th, 2006, 12:11 PM

Quote:

Originally Posted by Robert Lane

First, you can't put 4 drives together in a RAID on the Mac Pro; one drive needs to be reserved for the OS so that limits you to 3 drives.

OWC did a test where they used 3 SEAGATE 750gb drives and got about 290mb/s READ and a little faster WRITE (might have the figures backwards). I don't own a MacPro (yet) and have not tested that config myself so I can't comment on actual usage or numbers.

I saw OWC's test... Result numbers are strange and I think they goofed somewhere. But the biggest problem with using internal drives in a RAID configuration on the PowerMac and Mac Pro is that it's software RAID. For striped arrays, it's not a big deal, but this can become an issue when the system is being run under heavy load and RAID software still needs to manage RAID3/5 or even a RAID1 mirror.

As for the Mac Pro, all 4 SATA drives can be used for a single array. There are two ATA channels up front which each support two devices. They're intended for optical drive use with the standard optical drive being shipped installed as the primary device on the first ATA interface. A boot drive can easily be installed on either of those two interfaces...

...Not much point in doing such a thing though - going back to the software RAID issue. And I think as BluRay / HD-DVD start to make a bigger impact over the coming months, people are goint to want both optical drive bays and primary ATA channels at their disposal. Personally, I think it's well worth it to buy an external enclosure and an eSATA interface card. The few $100 this costs opens up your performance and expandability options a whole lot more. Fiberchannel is even better and multiple systems can talk to a proper fiber RAID system via fiber channel hubs/switches, but they do get very expensive rather quickly. Although in a situation where you have multiple workstations in use (even as few as 5 or so) at the same time and want several TB of workspace, a fiberchannel based SAN begins to make a lot of sense.

Barry Kay · December 7th, 2006, 12:58 PM

Quote:

Originally Posted by Robert Lane

This device is mainly for storage, not an array for editing as it won't have the throughput required for multiple-stream previews and fast renders.
...Any JBOD ...KONA System Test)....HBA interface...

Aggwwwhh! I suspected there had to be a 'catch' about the LaCie since the price was so low compared to what I've previously heard about.

The ideas you (and others) keep presenting is that my 'simplistic hopes' of trying to avoid extra expense by going through my Firewire800 (for whatever) might be undo-able. Humm...I'll have to check out all the terms you've used above. Fortunately I've given myself a couple of months to get this all figured out prior to purchases.

So...uh...how about getting a Firestore, use that for shooting, then just connect that by Firewire800 to my G5??

Well...I'll read up more and sure do appreciate these forums.

Tom Chartrand · December 7th, 2006, 05:32 PM

Quote:

Originally Posted by David Saraceno

Not really. I have the boot drive in the second optical bay.

The remaining four drives can be RAIDed.

What do you have in the 4? 500's or 750's? Everything smooth?

Zsolt Gordos · December 7th, 2006, 05:54 PM

Hi,

I am not a RAID expert, so I need some clear answer, please...:)
I plan to get 2 Lacie 500Gb ext drives and a 4 port eSATA PCI card to connect them...I would avoid RAID for budget reasons.

Would this be fast enough for editing or not? Whats happening if its slow? (FCP hang? or?)

G5 dual 2Ghz, 6Gig RAM, latest FCP update.

Thanks

Robert Lane · December 7th, 2006, 06:00 PM

Quote:

Originally Posted by Zsolt Gordos

Hi,

I am not a RAID expert, so I need some clear answer, please...:)
I plan to get 2 Lacie 500Gb ext drives and a 4 port eSATA PCI card to connect them...I would avoid RAID for budget reasons.

Would this be fast enough for editing or not? Whats happening if its slow? (FCP hang? or?)

G5 dual 2Ghz, 6Gig RAM, latest FCP update.

Thanks

2 drives is not enough speed to do more than one stream of HD and will slow down rendering as well. If you're not going to RAID the 2 drives together then you probably won't get real-time previews either without stuttering or frame drops, you'll just have to test it and see.

Zsolt Gordos · December 7th, 2006, 06:33 PM

Another advantage of HDV..... Much less space and speed needed. :(

Robert Lane · December 7th, 2006, 06:40 PM

Quote:

Originally Posted by Zsolt Gordos

Another advantage of HDV..... Much less space and speed needed. :(

That's actually completely wrong, Zsolt; HDV requires a great deal more processing power to both render and playback from the timeline. It actually takes 1.8x to 2.4x times more computing power to process HDV clips than MXF, which is actualy one of the biggest advantages of the P2 workflow - less time waiting for renders and imports.

In some cases I've seen HDV footage take up to 5x times longer than same-length P2-MXF clips just to import into FCP on the same machine.

In fact, if someone were to use this inexpensive array setup for editing more than 2 streams of HDV clips in one timeline they would be very dissapointed with the performance; within 10 mintues of constant editing the array would slow down to less than 1/4 of it's original speed, thus requiring a SCSI array or at the very least a larger e-SATA array to keep up the througput-frame speed.

Dale Hildebrand · December 7th, 2006, 07:40 PM

I picked up a HIGHPOINT 2322 for my G5 Quad along with a raid enclosure. The problem I had is that the raid would never go into deep sleep, and sometimes when I did put my machine to sleep (after I installed the 2322) I would come down in the morning and the entire computer (fans) were whirling at about a million RPM's - not sure if it was doing that all night. Scared the hell out of me. Sent it back. Perhaps I just couldn't figure out how to set it up properly, but the wierd configuring through the internet browser was not my cup of tea. I prefer to set up the raid through more familiar software - perhaps even as simple as disk utilities.

Zsolt Gordos · December 7th, 2006, 07:49 PM

Quote:

Originally Posted by Dale Hildebrand

The problem I had is that the raid would never go into deep sleep, and sometimes when I did put my machine to sleep (after I installed the 2322) I would come down in the morning and the entire computer (fans) were whirling at about a million RPM's - not sure if it was doing that all night. Scared the hell out of me.

Interesting, my 30 Cinema Display does the same to my G5 ever since I hooked it up. The poor thing never sleeps, unless I switch it off...

December 6th, 2006, 10:09 AM	#1
Robert Lane Go Go Godzilla Join Date: Nov 2004 Location: Scottsdale, AZ USA Posts: 2,825 Images: 15	Fast, inexpensive array for HVX users Whether you're a new or experienced HVX user one thing of concern to all (as noted by the multiple threads on the topic) is both storage of P2 clips and fast access either for grabbing files and especially for editing. Fast access always means an array of some sort, be it e-SATA, SCSI or fiber. But arrays regardless of which kind usually mean spending thousands of dollars both in array hardware and HDD's themselves. This is an alternative to the ultra-expensive arrays - and it works, very well (test result enclosed). This setup comes with some very important caveats, so pay attention before you spend your money or ask tons of questions: - This is an e-SATA array, meaning there is no on-board controller cache and the only cache available is what the HDD's themselves have. That translates into the fact that the array will start out very fast however, as it gets continually used throughout an edit or if multiple file transfers occur it will slow down, sometimes to half it's original speed depending on use. Speed is regained when the individual drive cache is flushed (usually during system reboot or over time if the drives are not used for about 10 minutes or so). Until there are e-SATA HBA's that have on-board cache there is no way around this performance characteristic. - If you have never setup an array or don't understand how they work, don't start spending your money on this setup UNTIL you get some research done or training by someone who can walk you through setup and usage. - This setup, while very cost-effective and inexpensive, is more complex in that it uses multiple enclosures each with it's own power supply. That means there are multiple points of failure since you're dealing with more than one independent power supply. - This setup does not allow for "hot-swap" as the drives are in a fixed enclosure - more on that as the hardware is discussed. Here's the physical setup: (1) Highpoint Technologies "RocketRAID" 2322 PCI-e HBA w/ (2) mini-SAS to e-SATA cables. (that's a total of 8 individual drive cables) $295 - Cables are separate at $45 each depending on where you get them. $385 total (4) OWC Mercury Elite-AL Dual Bay SATA Enclosures (2 drives per enclosure, direct JBOD connection) $75 each / $300 total That's an 8-drive e-SATA array for less than $800 (not including HDD's)! How fast is it? Using the KONA System Test I ran it using the same codec I shoot in DVCPRO-HD 720p and got just under 280 mb/s READ with a 1gb file test. That's damned impressive for an array that was thousands less than a SCSI setup. This test was done with (8) WD5000YS drives in a RAID-0 configuration. AMUG and BARE-FEATS tested this card with faster numbers but they used a Mac TERMINAL application to do the test, I prefer to use the KONA test because it's specific to mimicking a video-edit environment, so I feel the numbers are more real-world and not hype. The RocketRAID 2322 is the first e-SATA HBA to create the array in HARDWARE not software, so there's no OS overhead being used for the array management. It's also the first HBA to allow for 8-drive access externally and, (2) RR2322's can be connected in parallel for a total of 16 drives! (this functionality in MacOS requires Disk Utility to join them, in XP you just add a jumper cable and the cards do the work). The card will do RAID 0, 1, 5, 10, and parity on 5, 10. The RR2322 does not currently support the newer 750gb drives, but that's probably going to change soon. The OWC enclosures are unique; they do not have HOT-SWAP trays, the drives are fixed so you lose the ability to quickly replace a drive. They also have their own individual power supply, so that means cabling becomes a bit of a mess in this tested config: (8) e-SATA data cables and (4) power cables between (4) enclosures. The OWC "black boxes" are whisper quiet; with all 4 running I barely heard the fans in the back. This is not an ideal setup for everybody, this setup is geared to those who want or need a fast array for accessing OR storing their P2 files but don't have mega-bucks for a SCSI or fiber array. Here's a cost comparison (minus the cost of HDD's, HBA and cables) (4) OWC "Black boxes" as described above: $300 (2) Sonnet Fusion 400 enclosures (8 drives total, hot-swap capable, e-SATA connection): $760 (2) Wiebetech SilverSATA IV (same as above): $900 (1) Enhance-Tech T8 SCSI 8-drive enclosure (SATA to SCSI backplane): $2200 (2) Fibrenetix Qubex Fibre 6-drives x2 enclosures (SATA to Fibre backplane): $8100 As you can see, as you go up the food-chain of faster connectivity the costs skyrocket, sharply. Add the cost of HDD's and the HBA and you'll see how cost-effective this e-SATA suggested setup is. It should be noted however, that no e-SATA array regardless of enclosure or connection type is as fast nor as data-stable (constant data-frame-speed) as SCSI or fiber. But, for the added speed and stability you really have to cough up some serious buckage - something most indies just can't afford or justify. Last edited by Robert Lane; November 22nd, 2007 at 09:05 PM.

December 7th, 2006, 11:33 AM	#6
Robert Lane Go Go Godzilla Join Date: Nov 2004 Location: Scottsdale, AZ USA Posts: 2,825 Images: 15	[QUOTE=Barry Kay]Thanks for the info; I'm looking forward to upgrading Everything to HVX so when online window-browsing I saw this new RAID 500GB product from LaCie for only $399 (!). Their promo says: " a compact 2-disk device, the LaCie Two Big eSATA & USB comes with the fastest interface - eSATA (SATA II 3Gbits) and Hi-Speed USB 2.0 for connecting to almost any computer."[QUOTE] Barry, This device is mainly for storage, not an array for editing as it won't have the throughput required for multiple-stream previews and fast renders. Any JBOD e-SATA array would need a minimum of 4 drives in RAID-0 (as tested with the KONA System Test) to sustain the required througput and, it also requires an HBA interface not USB or Firewire as that throughput is limited to about 90mb/s. Last edited by Robert Lane; December 7th, 2006 at 06:01 PM.

December 6th, 2006, 10:37 AM	#3
Robert Lane Go Go Godzilla Join Date: Nov 2004 Location: Scottsdale, AZ USA Posts: 2,825 Images: 15	First, you can't put 4 drives together in a RAID on the Mac Pro; one drive needs to be reserved for the OS so that limits you to 3 drives. OWC did a test where they used 3 SEAGATE 750gb drives and got about 290mb/s READ and a little faster WRITE (might have the figures backwards). I don't own a MacPro (yet) and have not tested that config myself so I can't comment on actual usage or numbers. The main drawbacks to that setup are that you can only get a maximum of 2250gb (about 1980gb formatted) using the 3 extra internal MacPro HDD's and, it's internal meaning they are always running when the system is turned on. With any external array if you don't need to use the array you can leave it powered off and save the added time and heat wear on the array drives. Personally, I would not setup an internal array and would instead use those drives for either backup storage and or, assigning FCP/Photoshop cache files to them to alleviate the contantly changing cache files to be written to the array. I would also reserve use on one the of the internal HDD's for creating a bootable software repair partition, such as for Tech Tool Pro or Disk Warrior in case things go south. My advice for anyone currently using a MacPro or considering it is to be aware of the 2GB RAM limitation. Currently there are third party devices such as HBA's which are causing kernel panics on MacPro's with more than 2GB of RAM installed. See BARE-FEATS about this issue and how things are shaking out with bug fixes.

December 7th, 2006, 09:48 AM	#4
Barry Kay Regular Crew Join Date: Nov 2006 Location: Rural, AZ Posts: 29	Thanks for the info; I'm looking forward to upgrading Everything to HVX so when online window-browsing I saw this new RAID 500GB product from LaCie for only $399 (!). Their promo says: " a compact 2-disk device, the LaCie Two Big eSATA & USB comes with the fastest interface - eSATA (SATA II 3Gbits) and Hi-Speed USB 2.0 for connecting to almost any computer." Compared to prices in the past and prices you mention this would seem like a great deal, plus I've heard the fact that not being able to take your RAID to other computers can be a hassle so the USB connection is valuable. It's also hot-swappable and can be found at: http://www.lacie.com/products/product.htm?pid=10847

December 7th, 2006, 05:54 PM	#10
Zsolt Gordos Major Player Join Date: Nov 2006 Location: Budapest, Hungary Posts: 414	Hi, I am not a RAID expert, so I need some clear answer, please...:) I plan to get 2 Lacie 500Gb ext drives and a 4 port eSATA PCI card to connect them...I would avoid RAID for budget reasons. Would this be fast enough for editing or not? Whats happening if its slow? (FCP hang? or?) G5 dual 2Ghz, 6Gig RAM, latest FCP update. Thanks

December 7th, 2006, 06:33 PM	#12
Zsolt Gordos Major Player Join Date: Nov 2006 Location: Budapest, Hungary Posts: 414	Another advantage of HDV..... Much less space and speed needed. :(

December 7th, 2006, 07:40 PM	#14
Dale Hildebrand New Boot Join Date: Mar 2005 Posts: 12	I picked up a HIGHPOINT 2322 for my G5 Quad along with a raid enclosure. The problem I had is that the raid would never go into deep sleep, and sometimes when I did put my machine to sleep (after I installed the 2322) I would come down in the morning and the entire computer (fans) were whirling at about a million RPM's - not sure if it was doing that all night. Scared the hell out of me. Sent it back. Perhaps I just couldn't figure out how to set it up properly, but the wierd configuring through the internet browser was not my cup of tea. I prefer to set up the raid through more familiar software - perhaps even as simple as disk utilities.