eSATA RAID 0 or internal striped? at DVinfo.net

Mike Browning · June 11th, 2009, 02:21 PM

Preface:

I don't have a RAID with my home system, and haven't needed one previously. I commonly work with HDV as well as Avid's DNxHD in Media Composer, all on a Mac Pro Quad-Core. My internal disks consist of a Mac OSX boot, a Windows XP boot, a capture disk, and a scratch disk (none of which are striped together). Previously, I would just move a project over to my backup disks when done. This setup has worked fine for me, until now. Little strain, didn't need to throw more money than I needed at it, no problem.

My problem:

However, I am starting to deal with higher volumes of HD projects. Even working with the low-speed requirements of HDV and DNxHD, my system is showing a lot of strain. I'm having to consider a different storage solution. There are a few options I'm not quite clear on. One option is to simply stripe two internal disks together, which wouldn't cost me a penny extra. Another is to purchase a four-disk eSATA RAID 0. The answer may be a no-brainer to you but I need a little light shed on the subject. Keep in mind that I'm working with compressed HD formats, and I and my clients are very happy with the results. I know that I could get a huge increase with an 8-disk or higher RAID, but I don't want to spend the money on that, nor do I feel I have the need for the transfer speed there.

My question:

In terms of speed, is there a noticeable difference between two striped internal disks vs. a 4-disk eSATA RAID 0? I could easily stripe together two internal disks, but would that provide enough speed? I've done a lot of research and I've gotten mixed answers on that. Also, if you say spend the money and go for an eSATA RAID 0, what do you recommend? I see Lacie and G-technologies both have some appealing prices. And if you recommend a completely different solution, please let me know.

I welcome the DVinfo community's professional opinion.

Thanks,

Mike Browning

Harm Millaard · June 11th, 2009, 02:51 PM

Maybe this will help you:

Adobe Forums: Storage rules for an editing rig. Some...

Mike Browning · June 11th, 2009, 02:57 PM

@ Harm:

Rules 1-8 - check.

Rule 9 - care to decode?

I know I definitely need a RAID solution, instead of the unstriped single disks I've been using. My question was, simply put, whether I should spend the money to buy a 4-disk eSATA RAID or if I would see the same performance increase in simply striping two of my internal disks together? In other words, which would be more worthwhile, especially for multiple streams of compressed HD? Does anyone have experience with either of these solutions?

Thanks,

Mike Browning

Harm Millaard · June 11th, 2009, 03:21 PM

First, SATA and eSATA are equally fast as single disks.

When you stripe (aid0, there is no redundancy, hence the missing R) two disks you will get around 1.6x the performance of a single disk (there is always some overhead), with 4 disks striped, you will get around 3.2x the performance of a single disk. However, the chance of a disk failure causing you to lose all data on the (R)aid grows with the number of disks. With 2 disks, the chance is twice as big compared to a single disk, with 4 disks, it is four times as large. In general, nobody would even consider an Aid0 array larger than four disks. Four disks is pushing it, IMO. I know I have pushed my array to the limit with 12 disks in a raid30.

Internal or external? Depends on the raid chip and the available space. If you have the room, use internals. No need for an extra housing, external cables, extra fans or power supply. Usually the controllers in external cages like G-Tech or LaCie just stink. They are worthless. OK, this is in comparison to a good internal raid controller and of course you get what you pay for.

Often you can use the onboard raid chip, which usually gives you raid0, 1, 5 or 10 capability. That is preferable to using a Windows software raid. Separate disk cages may help you to increase internal storage capacity. I currently use 17 internal hard disks and 2 BR burners with room for an additonal disk or two.

What is it you want to have decoded about rule 9? I realise it may appear as geek tech talk, but a careful read may make things more clear.

Sorry, after my response I realized you are on a MAC, so disregard my remark about Windows. I just assumed you belong to the 90% of people with a PC, not the 10% on a MAC. It has no further implications for your choice, all my earlier remarks hold, also for MAC's.

Mike Browning · June 11th, 2009, 03:35 PM

Thanks Harm,

If I stayed internal, I'd only have two disks to stripe together. The potential 3.2x speed sounds very appealing compared to the 1.6x, and I store project files elsewhere, so I'm not worried about the potential of losing a disk.

Another question now. I've heard Mac Pro's only have software raid capabilities as well, at least at the stock configuration. Does this have any barring on an external raid? How much do you think it would affect my CPU load, if so?

And since you don't recommend the Lacie or the G-raid, what do you recommend for an external setup? I'd like to stay under $1500, and I don't need more than 3-4TB at the most. I never work on more than 1 or 2 projects at a time.

As for Rule 9 - the alphabet soup scared me off. Although, having worked for government, it's nothing new I suppose lol.

Harm Millaard · June 11th, 2009, 04:01 PM

Mike,

The alphabet soup as you called it relates to the various chips used on motherboards or raidcontrollers. IHCR is often found on Intel based motherboards (note some Intel based motherboards have IHC chips, missing the R, missing the Raid capability), Marvell is again a chip on the motherboard that can offer SAS capabilities. The IOP is typically a chip on a raidcontroller and the latest and fastest incarnation currently is the IOP348.

If you Google for these terms or look at the Intel website, you can read quite a lot about it, but it is rather technical.

I'm not familiar with MAC/Intel based motherboards, so I don't know if they have the IHCR or Marvell chips there. A look in the specs will reveal that.

For disks I would recommend to have a look at either Samsung Spinpoint F1's (cheap, silent and reliable) or the even better WD Caviar Black (not as cheap, slightly faster and just as reliable). I would advise against Seagates. I have had so much trouble with these lately with a failure rate of more than 70%. (I use a NAS with 7x 1.5 TB Seagates as well, in addition to my 16+ TB in my workstation and some storage on the servers)

With your budget and limited internal space, I would look at Newegg for an external housing for at least 4 but preferably 5 of more disks (maybe Addonics?), swappable of course, populate that with 4x 1 TB disks to start with and depending on the raid capabilities of your mobo, get a very good raid controller (Areca is my favorite). The latter is going to cost quite a bit, so if your mobo supports it, start with onboard raid. However, consider using the 4 disks not in aid0 but in raid5. This will give you an approximate 2.8 x performance increase, but will add the safety that if one disk fails, you will not lose any of your data. It does reduce your effective storage capacity to 3 TB instead of 4 when using a aid0.

Mike Browning · June 11th, 2009, 04:43 PM

Harm,

Thanks for shedding so much light on the situation. This will help me further in my decision making process.

From your last paragraph, I believe you are suggesting assembling my own (from a casing, separate drives, and controller) as opposed to buying an array on the market. Is there an advantage to this alternative? i.e., cost, performance, satisfaction, etc.?

Harm Millaard · June 11th, 2009, 05:08 PM

To someone who has never done this, it may seem dounting to build it yourself. But the advantage is that YOU can select the components, it will be cheaper, it will perform better and the immediate downside is, YOU need to provide the technical support to yourself. That about sums it up.

For me it is easy to talk like this, since I have built my own systems for a very long time AND I have a MCSE/MCP/MCSA/CCSP/etc... son to help me out if I run into problems (sorry for the soup again, MCSE = Microsoft Certified System Engineer, MCP = Microsoft Certified Professional, MCSA = Microsoft Certified System Administrator, CCSP = Cisco Certified Security Professional).

You may have a look at some other guides I wrote, among them about building your own PC and how to optimize Vista64. It may not be of immediate relevance to you with a MAC, but it may show you some considerations about building your own system and what choices you have. It will (in a link at the bottom of the PC building guide) show you my own system, that I recently built. It shows you how I set up my drives. It may be interesting.

Adobe Forums: How to get the best from a PC? Some...

June 11th, 2009, 02:21 PM	#1
Mike Browning Regular Crew Join Date: Apr 2008 Location: Golden Isles, GA Posts: 25	eSATA RAID 0 or internal striped? Preface: I don't have a RAID with my home system, and haven't needed one previously. I commonly work with HDV as well as Avid's DNxHD in Media Composer, all on a Mac Pro Quad-Core. My internal disks consist of a Mac OSX boot, a Windows XP boot, a capture disk, and a scratch disk (none of which are striped together). Previously, I would just move a project over to my backup disks when done. This setup has worked fine for me, until now. Little strain, didn't need to throw more money than I needed at it, no problem. My problem: However, I am starting to deal with higher volumes of HD projects. Even working with the low-speed requirements of HDV and DNxHD, my system is showing a lot of strain. I'm having to consider a different storage solution. There are a few options I'm not quite clear on. One option is to simply stripe two internal disks together, which wouldn't cost me a penny extra. Another is to purchase a four-disk eSATA RAID 0. The answer may be a no-brainer to you but I need a little light shed on the subject. Keep in mind that I'm working with compressed HD formats, and I and my clients are very happy with the results. I know that I could get a huge increase with an 8-disk or higher RAID, but I don't want to spend the money on that, nor do I feel I have the need for the transfer speed there. My question: In terms of speed, is there a noticeable difference between two striped internal disks vs. a 4-disk eSATA RAID 0? I could easily stripe together two internal disks, but would that provide enough speed? I've done a lot of research and I've gotten mixed answers on that. Also, if you say spend the money and go for an eSATA RAID 0, what do you recommend? I see Lacie and G-technologies both have some appealing prices. And if you recommend a completely different solution, please let me know. I welcome the DVinfo community's professional opinion. Thanks, Mike Browning

June 11th, 2009, 02:51 PM	#2
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	Maybe this will help you: Adobe Forums: Storage rules for an editing rig. Some...

June 11th, 2009, 02:57 PM	#3
Mike Browning Regular Crew Join Date: Apr 2008 Location: Golden Isles, GA Posts: 25	@ Harm: Rules 1-8 - check. Rule 9 - care to decode? I know I definitely need a RAID solution, instead of the unstriped single disks I've been using. My question was, simply put, whether I should spend the money to buy a 4-disk eSATA RAID or if I would see the same performance increase in simply striping two of my internal disks together? In other words, which would be more worthwhile, especially for multiple streams of compressed HD? Does anyone have experience with either of these solutions? Thanks, Mike Browning

June 11th, 2009, 03:21 PM	#4
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	First, SATA and eSATA are equally fast as single disks. When you stripe (aid0, there is no redundancy, hence the missing R) two disks you will get around 1.6x the performance of a single disk (there is always some overhead), with 4 disks striped, you will get around 3.2x the performance of a single disk. However, the chance of a disk failure causing you to lose all data on the (R)aid grows with the number of disks. With 2 disks, the chance is twice as big compared to a single disk, with 4 disks, it is four times as large. In general, nobody would even consider an Aid0 array larger than four disks. Four disks is pushing it, IMO. I know I have pushed my array to the limit with 12 disks in a raid30. Internal or external? Depends on the raid chip and the available space. If you have the room, use internals. No need for an extra housing, external cables, extra fans or power supply. Usually the controllers in external cages like G-Tech or LaCie just stink. They are worthless. OK, this is in comparison to a good internal raid controller and of course you get what you pay for. Often you can use the onboard raid chip, which usually gives you raid0, 1, 5 or 10 capability. That is preferable to using a Windows software raid. Separate disk cages may help you to increase internal storage capacity. I currently use 17 internal hard disks and 2 BR burners with room for an additonal disk or two. What is it you want to have decoded about rule 9? I realise it may appear as geek tech talk, but a careful read may make things more clear. Sorry, after my response I realized you are on a MAC, so disregard my remark about Windows. I just assumed you belong to the 90% of people with a PC, not the 10% on a MAC. It has no further implications for your choice, all my earlier remarks hold, also for MAC's.

June 11th, 2009, 03:35 PM	#5
Mike Browning Regular Crew Join Date: Apr 2008 Location: Golden Isles, GA Posts: 25	Thanks Harm, If I stayed internal, I'd only have two disks to stripe together. The potential 3.2x speed sounds very appealing compared to the 1.6x, and I store project files elsewhere, so I'm not worried about the potential of losing a disk. Another question now. I've heard Mac Pro's only have software raid capabilities as well, at least at the stock configuration. Does this have any barring on an external raid? How much do you think it would affect my CPU load, if so? And since you don't recommend the Lacie or the G-raid, what do you recommend for an external setup? I'd like to stay under $1500, and I don't need more than 3-4TB at the most. I never work on more than 1 or 2 projects at a time. As for Rule 9 - the alphabet soup scared me off. Although, having worked for government, it's nothing new I suppose lol.

June 11th, 2009, 04:01 PM	#6
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	Mike, The alphabet soup as you called it relates to the various chips used on motherboards or raidcontrollers. IHCR is often found on Intel based motherboards (note some Intel based motherboards have IHC chips, missing the R, missing the Raid capability), Marvell is again a chip on the motherboard that can offer SAS capabilities. The IOP is typically a chip on a raidcontroller and the latest and fastest incarnation currently is the IOP348. If you Google for these terms or look at the Intel website, you can read quite a lot about it, but it is rather technical. I'm not familiar with MAC/Intel based motherboards, so I don't know if they have the IHCR or Marvell chips there. A look in the specs will reveal that. For disks I would recommend to have a look at either Samsung Spinpoint F1's (cheap, silent and reliable) or the even better WD Caviar Black (not as cheap, slightly faster and just as reliable). I would advise against Seagates. I have had so much trouble with these lately with a failure rate of more than 70%. (I use a NAS with 7x 1.5 TB Seagates as well, in addition to my 16+ TB in my workstation and some storage on the servers) With your budget and limited internal space, I would look at Newegg for an external housing for at least 4 but preferably 5 of more disks (maybe Addonics?), swappable of course, populate that with 4x 1 TB disks to start with and depending on the raid capabilities of your mobo, get a very good raid controller (Areca is my favorite). The latter is going to cost quite a bit, so if your mobo supports it, start with onboard raid. However, consider using the 4 disks not in aid0 but in raid5. This will give you an approximate 2.8 x performance increase, but will add the safety that if one disk fails, you will not lose any of your data. It does reduce your effective storage capacity to 3 TB instead of 4 when using a aid0.

June 11th, 2009, 04:43 PM	#7
Mike Browning Regular Crew Join Date: Apr 2008 Location: Golden Isles, GA Posts: 25	Harm, Thanks for shedding so much light on the situation. This will help me further in my decision making process. From your last paragraph, I believe you are suggesting assembling my own (from a casing, separate drives, and controller) as opposed to buying an array on the market. Is there an advantage to this alternative? i.e., cost, performance, satisfaction, etc.?

June 11th, 2009, 05:08 PM	#8
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	To someone who has never done this, it may seem dounting to build it yourself. But the advantage is that YOU can select the components, it will be cheaper, it will perform better and the immediate downside is, YOU need to provide the technical support to yourself. That about sums it up. For me it is easy to talk like this, since I have built my own systems for a very long time AND I have a MCSE/MCP/MCSA/CCSP/etc... son to help me out if I run into problems (sorry for the soup again, MCSE = Microsoft Certified System Engineer, MCP = Microsoft Certified Professional, MCSA = Microsoft Certified System Administrator, CCSP = Cisco Certified Security Professional). You may have a look at some other guides I wrote, among them about building your own PC and how to optimize Vista64. It may not be of immediate relevance to you with a MAC, but it may show you some considerations about building your own system and what choices you have. It will (in a link at the bottom of the PC building guide) show you my own system, that I recently built. It shows you how I set up my drives. It may be interesting. Adobe Forums: How to get the best from a PC? Some...