Hard Drive Solutions at DVinfo.net

Joe Gaetani · May 5th, 2009, 02:17 PM

Not sure if this is the right place to post, but...

With my current setup, my scratch disk configuration is two 1.5 TB drives setup in a RAID 1. My other 2 drive bays are the OS drive and the OS drive backup. Once my scratch disk is full, I will switch the two 1.5 TB drives out with new ones. However, I will still want to access the data on these drives. What is the best solution for housing one of these drives (and possibly up to 4 or 6 total)?

If this isn't clear, I can provide more info.
Thanks.

Nicholas de Kock · May 5th, 2009, 03:56 PM

Get a Drobo or Drobo Pro!
Data Robotics, Inc.

Joe Gaetani · May 5th, 2009, 04:01 PM

Quote:

Originally Posted by Nicholas de Kock

Get a Drobo or Drobo Pro!
Data Robotics, Inc.

Yeah, I've heard great things about Drobo. My concern is speed. With FW800 being the fastest connection, will it have the throughout for HD video? Primarily ProRes 422 but also DVC Pro HD and RED files?

Harrison Murchison · May 5th, 2009, 04:27 PM

Quote:

Originally Posted by Joe Gaetani

Yeah, I've heard great things about Drobo. My concern is speed. With FW800 being the fastest connection, will it have the throughout for HD video? Primarily ProRes 422 but also DVC Pro HD and RED files?

It should, via the iSCSI connection you should be looking at 70+ megabytes per second writes roughly.

Joe Gaetani · May 5th, 2009, 04:29 PM

Quote:

Originally Posted by Harrison Murchison

It should, via the iSCSI connection you should be looking at 70+ megabytes per second writes roughly.

I'm not too familiar with iSCSI. What type of connection do I need on my machine? Is this something that is easily added on via PCI slot?

Harrison Murchison · May 5th, 2009, 04:39 PM

Quote:

Originally Posted by Joe Gaetani

I'm not too familiar with iSCSI. What type of connection do I need on my machine? Is this something that is easily added on via PCI slot?

Just your standard Gigabit connection. Data Robotics includes a free iSCSI initiator which communications with the DroboPro.

iSCSI is simply the SCSI command set you've known for years wrapped up in a packetized IP frame for easy transfer over long distances.

Once setup the DroboPro would appear as one big contiguous file or you can set up smart volumes (up to 16 at 16TB per smart volume) that actually hold more data than you really have. This way you can plan for future growth and plan for larger hard drive sizes (like the inevitable 2, 3 and 4TB hard drives coming)

Joe Gaetani · May 5th, 2009, 04:43 PM

hmm very interesting. thanks for all the information!

how does the drobo compare to NAS?

Harm Millaard · May 5th, 2009, 05:00 PM

Nicholas,

Interesting to see your link to the DroboPro, but from their website it looks like a JBOD solution with iSCSI capability. Am I correct in that assumption? JBOD IMO is a performance degrader of the first rank and offers no data security like a raid 3/5/6/10/30/50/60 does. Where is the benefit in comparison to for instance a Thecus NAS?

Harrison Murchison · May 5th, 2009, 05:05 PM

Quote:

Originally Posted by Joe Gaetani

hmm very interesting. thanks for all the information!

how does the drobo compare to NAS?

It appears as locally attached storage. No RAID levels to configure yet you still get dual drive redundancy. You can use mismatched drive sizes so as drives increase in size you just pop them in and the Drobo BeyondRAID system adds the storage to the pool.

Disadvantages compared to a NAS:

NAS uses common filesharing protocol such as SMB/CIFS, NFS, AFP so multiple computers can access the storage. DroboPro does not offer sharing functionaly though the 4-bay Drobo offers a $180 DroboShare device for sharing (not compatible with DroboPro).

If you want to have multiple computers access DroboPro you'd use the computer connected to your DroboPro as a NAS gateway and this computer would handle the file protocol. A bit more complicated in that end.

Harrison Murchison · May 5th, 2009, 05:12 PM

Quote:

Originally Posted by Harm Millaard

Nicholas,

Interesting to see your link to the DroboPro, but from their website it looks like a JBOD solution with iSCSI capability. Am I correct in that assumption? JBOD IMO is a performance degrader of the first rank and offers no data security like a raid 3/5/6/10/30/50/60 does. Where is the benefit in comparison to for instance a Thecus NAS?

No Drobo does not use standarized RAID levels but it does have their own RAID like data redundancy called BeyondRAID.

With Dual Disk redundancy in the DroboPro it could be said that you're running a RAID-6 equivalent array.

Nicholas de Kock · May 5th, 2009, 07:06 PM

Quote:

Originally Posted by Harm Millaard

Nicholas,

Interesting to see your link to the DroboPro, but from their website it looks like a JBOD solution with iSCSI capability. Am I correct in that assumption? JBOD IMO is a performance degrader of the first rank and offers no data security like a raid 3/5/6/10/30/50/60 does. Where is the benefit in comparison to for instance a Thecus NAS?

Harm the Drobo is better than RAID, it's hassle free, you simply put your drives in and consider it safe unless an earthquake takes out your studio. I have a RAID 5 setup however as soon as I can afford a Drobo Pro I'm buying one. The other week I lost one of my RAID 5 drives, took 21 hours to rebuild! Lost an entire day of work. Drobo Pro will continue accessing files after you remove two drives.

I don't advice anyone buy a Drobo (Firewire) with the intention of editing HD footage off it, way to slow for that, the network sharing dock for the Drobo I've read is not so great, however the Drobo Pro will be able to handle higher speeds, personally I still won't edit from it, RAID 0 is the best setup for speed, the DroboPro should be your backup.

Where is the benefit in comparison to for instance a Thecus NAS?
Thecus is annoying to set-up and hard to expand. Drobo is fully automatic, you can pop drives in/out as you wish, the Drobo will do what it does, worry free, with no interruption to work, rebuilding while you work.

Harrison Murchison · May 5th, 2009, 07:29 PM

I agree with Nicholas.

The DroboPro isn't likely the drive you're going to edit from but if I was doing data protection I'd want the Drobo features for storing data that I don't want to lose.

RAID-5 has it's own quirks.

1. Since each drive has to write parity info your write speeds don't improve but your read speeds are good.

2. Losing a drive is a pain in the arse (PITA) but you can rebuild based on the parity data from the other drives but as you see from Nicholas' reports it's not fast.

3. Software RAID-5 sucks and cheapo hardware cards can be flaky.

If I had a Mac Pro or I was looking for a RAID array to edit on I'd probably going with a RAID-10 setup and then use the DroboPro for my secondary storage. The DroboPro isn't out yet so it may benchmark fast enough to be primary storage but who knows?

RAID-10 is a nested RAID system.

You take at a minimum two sets of mirrored drives and then you stripe those two mirrors together.

RAID 1 + RAID 0 = RAID 10

Advantages :

No parity writes so you now have 2x write performance and 4x read. Now since the benefit if a mirrored array is that your rebuilds can be done much quicker if you lose a drive it's going to be relatively quickly rebuilt from its mirror or at worst the mirror from across the stripe.

Disadvantage:

4 drive minimum and you'll only get to access half your total storage.

There's a battle between database engineers about what is better RAID-5 or RAID-10. I think RAID-10 has a sizable lead because it writes faster. Since we're capturing video we're also beholden to write performance.

David Tamés · May 5th, 2009, 08:59 PM

For my new documentary I've come up with the following scheme:

1. MASTER MEDIA on a RAID-1 device (decided to get a 1TB G-Tech G-Safe, which in turn is backed up on two identical single SATA drives off site for super safe storage)

2. EDITING MEDIA on a RAID-0 device (for performance, with SATA interface, I've not decided on the actual device yet since editing does not start until September).

I thought of the Drobo for a while, but since I've heard mixed reviews, and it's still new and I made my decision before seeing the Pro version in action, I thought I'd stick with a tried and true hardware RAID-1 solution for data archiving, and a RAID-0 solution for decent editing performance. I guess I'm really shy about "new" technology when it comes to archiving. Super reliable digital storage is still many years away, so RAID-1 and a backup of that is prudent way to go.

Why not RAID-10, 5, or 6? Budgetary. I figure (and if anyone thinks I'm wrong, tell me, please) that with RAID-1 as the ARCHIVE copy, and two single bare SATA drive copies of that, I'm pretty safe as far as archiving my media, especially if I check it periodically. This is a three year storage solution, not a long term archiving solution.

Nicholas de Kock · May 11th, 2009, 11:45 PM

Though I'd share this, I have a RAID5 workstation on a Gigabit LAN that I use to back-up all my data however I recently got a 5-port switcher and installed Windows 7 on my system ever since my network has been performing very well. I can edit HDV footage in Sony Vegas in real-time over the network, at this point I have footage mixed in on the network with footage on my local RAID0 drives and I can't tell the difference, I don't know if I'm editing on my local drive or over the LAN it's that fast. It should be entirely possible to edit footage on a DroboPro or use it as a render farm, likewise a RAID5 setup also works if you'd rather build yourself a DIY NAS and gain another workstation while you're at it.

Tripp Woelfel · May 12th, 2009, 04:58 AM

Back in the day when a 300MB 5.25" ESDI drive cost north of US$1000, there were very compelling arguments for trading off the complexity of RAID 3 or 5 against the high cost of disk drives. But now that 1TB drives are hovering near US$100, RAID 1 or 10 seem like the optimal choice for desktop users, at least to me. Of course everyone needs to factor their own priorities and preferences into the mix but for simplicity, it's hard to beat RAID 1 or RAID 10.

May 5th, 2009, 02:17 PM	#1
Joe Gaetani Regular Crew Join Date: Aug 2008 Location: Burlington, VT Posts: 33	Hard Drive Solutions Not sure if this is the right place to post, but... With my current setup, my scratch disk configuration is two 1.5 TB drives setup in a RAID 1. My other 2 drive bays are the OS drive and the OS drive backup. Once my scratch disk is full, I will switch the two 1.5 TB drives out with new ones. However, I will still want to access the data on these drives. What is the best solution for housing one of these drives (and possibly up to 4 or 6 total)? If this isn't clear, I can provide more info. Thanks.

May 5th, 2009, 08:59 PM	#13
David Tamés Kino-Eye Join Date: Jul 2005 Location: Boston, MA Posts: 457	For my new documentary I've come up with the following scheme: 1. MASTER MEDIA on a RAID-1 device (decided to get a 1TB G-Tech G-Safe, which in turn is backed up on two identical single SATA drives off site for super safe storage) 2. EDITING MEDIA on a RAID-0 device (for performance, with SATA interface, I've not decided on the actual device yet since editing does not start until September). I thought of the Drobo for a while, but since I've heard mixed reviews, and it's still new and I made my decision before seeing the Pro version in action, I thought I'd stick with a tried and true hardware RAID-1 solution for data archiving, and a RAID-0 solution for decent editing performance. I guess I'm really shy about "new" technology when it comes to archiving. Super reliable digital storage is still many years away, so RAID-1 and a backup of that is prudent way to go. Why not RAID-10, 5, or 6? Budgetary. I figure (and if anyone thinks I'm wrong, tell me, please) that with RAID-1 as the ARCHIVE copy, and two single bare SATA drive copies of that, I'm pretty safe as far as archiving my media, especially if I check it periodically. This is a three year storage solution, not a long term archiving solution. __________________ David Tames { blog: http://Kino-Eye.com twitter: @cinemakinoeye }

May 5th, 2009, 03:56 PM	#2
Nicholas de Kock Major Player Join Date: May 2007 Location: KLD, South Africa Posts: 983	Get a Drobo or Drobo Pro! Data Robotics, Inc.

May 5th, 2009, 04:43 PM	#7
Joe Gaetani Regular Crew Join Date: Aug 2008 Location: Burlington, VT Posts: 33	hmm very interesting. thanks for all the information! how does the drobo compare to NAS?

May 5th, 2009, 05:00 PM	#8
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	Nicholas, Interesting to see your link to the DroboPro, but from their website it looks like a JBOD solution with iSCSI capability. Am I correct in that assumption? JBOD IMO is a performance degrader of the first rank and offers no data security like a raid 3/5/6/10/30/50/60 does. Where is the benefit in comparison to for instance a Thecus NAS?

May 5th, 2009, 07:29 PM	#12
Harrison Murchison Major Player Join Date: Jan 2004 Location: Seattle Posts: 243	I agree with Nicholas. The DroboPro isn't likely the drive you're going to edit from but if I was doing data protection I'd want the Drobo features for storing data that I don't want to lose. RAID-5 has it's own quirks. 1. Since each drive has to write parity info your write speeds don't improve but your read speeds are good. 2. Losing a drive is a pain in the arse (PITA) but you can rebuild based on the parity data from the other drives but as you see from Nicholas' reports it's not fast. 3. Software RAID-5 sucks and cheapo hardware cards can be flaky. If I had a Mac Pro or I was looking for a RAID array to edit on I'd probably going with a RAID-10 setup and then use the DroboPro for my secondary storage. The DroboPro isn't out yet so it may benchmark fast enough to be primary storage but who knows? RAID-10 is a nested RAID system. You take at a minimum two sets of mirrored drives and then you stripe those two mirrors together. RAID 1 + RAID 0 = RAID 10 Advantages : No parity writes so you now have 2x write performance and 4x read. Now since the benefit if a mirrored array is that your rebuilds can be done much quicker if you lose a drive it's going to be relatively quickly rebuilt from its mirror or at worst the mirror from across the stripe. Disadvantage: 4 drive minimum and you'll only get to access half your total storage. There's a battle between database engineers about what is better RAID-5 or RAID-10. I think RAID-10 has a sizable lead because it writes faster. Since we're capturing video we're also beholden to write performance.

May 11th, 2009, 11:45 PM	#14
Nicholas de Kock Major Player Join Date: May 2007 Location: KLD, South Africa Posts: 983	Though I'd share this, I have a RAID5 workstation on a Gigabit LAN that I use to back-up all my data however I recently got a 5-port switcher and installed Windows 7 on my system ever since my network has been performing very well. I can edit HDV footage in Sony Vegas in real-time over the network, at this point I have footage mixed in on the network with footage on my local RAID0 drives and I can't tell the difference, I don't know if I'm editing on my local drive or over the LAN it's that fast. It should be entirely possible to edit footage on a DroboPro or use it as a render farm, likewise a RAID5 setup also works if you'd rather build yourself a DIY NAS and gain another workstation while you're at it.

May 12th, 2009, 04:58 AM	#15
Tripp Woelfel Trustee Join Date: Jan 2008 Location: North Conway, NH Posts: 1,745	Back in the day when a 300MB 5.25" ESDI drive cost north of US$1000, there were very compelling arguments for trading off the complexity of RAID 3 or 5 against the high cost of disk drives. But now that 1TB drives are hovering near US$100, RAID 1 or 10 seem like the optimal choice for desktop users, at least to me. Of course everyone needs to factor their own priorities and preferences into the mix but for simplicity, it's hard to beat RAID 1 or RAID 10.