RAID 1 mirror question at DVinfo.net

Scott Auerbach · April 21st, 2006, 07:40 PM

I've never dealt with RAIDs before, as I've been an Avid guy until now and Avid has never offered RAIDs. I'm pretty sure I'm understanding what I've read, but I want to run one idea past the technoliterati in our midst... If I were to shoot on P2, then transfer to a RAID 1 pair attached to my MacBook Pro, wouldn't I have two identical, independent drives--one for the client (assuming they want to edit/compress their own footage) and one for me? It certainly seems so.

Secondly, is there any advantage or disadvantage to doing it this way versus a straight transfer followed by a straight drive-to-drive copy? Would the hardware slowdown associated with spraying the data onto two drives make a RAID 1 slower to build than just copying?

Jeff Kilgroe · April 22nd, 2006, 09:45 AM

If you have a proper RAID controller, all the mirroring will be handled by the controller and there won't be any additional drain on system resources or performance hits. When you are done, you will have two identical drives full of data. If it's a RAID attached to your Macbook, then the ideal solution would be an external drive enclosure with some form of RAID controller onboard. You can do it without, and just use an enclosure with two SATA connectors and then handle the RAID via software on the Mac. This method would introduce a slight performance hit, but would probably be insignificant anyway and would be cheaper due to a cheaper enclosure.

The advantage of using a proper RAID-1 to create a mirrored drive, is that it all happens real-time as you work with the data. No duplicating later so there's a time savings. It also offers some redundancy and a bit of secruity as you work in case one of the drives fail.

Giroud Francois · April 22nd, 2006, 10:05 AM

raid controller are "aware" of the structure you choose, so building a mirror RAID will give you 2 copy of the same disk, but breaking the mirror will be seen as a problem by the raid controller, so you will need to add a new disk (same type) and rebuild the mirror with the raid utility.
The good disk will be then copied to the new disk. So basically there is no escape to the copy process , you just delay it.

Gary L Childress · April 22nd, 2006, 10:50 PM

Scott,
Avid has sold plenty of RAIDs over the years. Prior to the LVD 10,000 rmp drives they had alot of 2 drive pairs in use and Unity is a big RAID. But to the main point here, I think RAID 1 is a bad choice for video acquisition and here's why.

Like you I thought this might be a good idea, so I went and bought a RAID 1 system, the Weibetech SilverSATA II. Since I was using a Powerbook, I bought a unit that had the controller inside to take a load off the laptop. The unit operates with only one SATA cable, so you don't have to worry about pushing the limits of the cardbus trying to pass two streams of video through the card at the same time.

Here is my unfortunate journey down that path.

The unit I purchased had removable sleds that proved to be difficult for drive installation due to the tightness of the screws. I had ordered an extra pair of sleds so that if I was on a shoot and ran out of drive space I could easily swap in a new pair preloaded with drives.

I had such a difficult time installing the drives on the sleds that I wound up bending the rails, which caused the drives to have problems in the unit. In a sense it was fortunate that this happened, because the problems I had with the drives pointed out some very serious issues with using RAID 1 for acquisition. Basically, I had on my hands the very problem you would buy RAID 1 for in the first place, a failed drive.

The unit would run for a bit and then show errors on the second drive. Then it would start the rebuild routine. This happened several times so I decided to let it go through the process just to see how long it would take. I had two 300gig SATA II 300 drives in the pair. The drives only had a couple of short SD clips on them that I had digitized as a test.

Total rebuild time: nearly two hours, even though they only had a few megs of material actually on them.

I went through this a couple of times with the problem sleds and then installed the drives in my desktop Firmtek SATA enclosures to see if the lengthy rebuild times were just the SSII unit. I took one of the drives with data and paired it with an empty drive in RAID 1 and let it go through a rebuild. That computer is a dual 2 ghz G5. Total rebuild time was nearly two hours.

So how do you deal with that at a shoot. If you have a drive fail, you will have to choose whether to shut the shoot down for the rebuild. Or you could bring extra drives and sleds two swap in the unit in case of a failure. I you swapped in new drives, then you are in the precarious situation of having only one good copy of the original good disk and no ability to back it up manually without disrupting the shoot. Of course you could also carry an extra RAID 1 array.

If you are going to have a RAID on a shoot, then you are PLANNING AHEAD for a drive to fail. So you are forced to consider what the scenario would be if a drive DID FAIL. I think RAID 1 does not present a pretty picture in a failure situation. Also, if you have a drive go down, you would be in the position of not knowing whether it was the drive or the RAID unit itself that was the problem. Thus you would have little confidence that there would not be similar problems with the replacement drive pair.

It was a very expensive lesson for me. On the shoot I bought the RAID 1 for, I switched and took the Firmtek pair which was not in a RAID. I then hand copied clips to the second drive after each dump. That seemed much more secure for me. It worked very well.

I think the best solution for acquisition is a RAID 5 unit. They are more expensive but offer the best protection.

Despite my experience with the SSII, the manufacturer is well respected and makes a variety of very high quality drive products. I would have no qualms about buying their 5 drive portable RAID 5 enclosure. It's $1700 with no drives but for the security it's probably worth it.

Scott Auerbach · April 22nd, 2006, 11:34 PM

Quote:

Originally Posted by Gary L Childress

Scott,
Avid has sold plenty of RAIDs over the years. Prior to the LVD 10,000 rmp drives they had alot of 2 drive pairs in use and Unity is a big RAID.

If you are going to have a RAID on a shoot, then you are PLANNING AHEAD for a drive to fail.

Sorry...my bad. Brain cramp. Yeah, I know the AvidStripe is a RAID 0, though Avid doesn't call it a RAID. I tend to use RAID to mean some level of mirroring/redundancy, though I know that's not the case.

I was only considering it for situations where the client wants to own a drive loaded with media (and I'd keep my own copy). Sorry you had such a bleak experience. Ugh.

Quote:

Originally Posted by Giroud Francois

raid controller are "aware" of the structure you choose, so building a mirror RAID will give you 2 copy of the same disk, but breaking the mirror will be seen as a problem by the raid controller, so you will need to add a new disk (same type) and rebuild the mirror with the raid utility.
The good disk will be then copied to the new disk. So basically there is no escape to the copy process , you just delay it.

I knew the RAID mirror could be rebuilt automatically, but I didn't know it was ~inevitable~. Though I should've guessed that. You're absolutely right, it makes more sense to just copy the drive. Thanks for the post!

Leonard Levy · April 23rd, 2006, 12:21 AM

Gary, That's a great post.
All I can say is I'm glad it was you not me.
That pretty much kills thge idea of a Raid for acquisition in my mind. Nothing beats the school of hard knocks.

You should post this on DVXUser.com as well.

April 21st, 2006, 07:40 PM	#1
Scott Auerbach Major Player Join Date: Feb 2006 Location: Atlanta Posts: 291	RAID 1 mirror question I've never dealt with RAIDs before, as I've been an Avid guy until now and Avid has never offered RAIDs. I'm pretty sure I'm understanding what I've read, but I want to run one idea past the technoliterati in our midst... If I were to shoot on P2, then transfer to a RAID 1 pair attached to my MacBook Pro, wouldn't I have two identical, independent drives--one for the client (assuming they want to edit/compress their own footage) and one for me? It certainly seems so. Secondly, is there any advantage or disadvantage to doing it this way versus a straight transfer followed by a straight drive-to-drive copy? Would the hardware slowdown associated with spraying the data onto two drives make a RAID 1 slower to build than just copying? __________________ DP/Editor, Sputnik Pictures \| Atlanta HD video and RED digital cinema production

April 22nd, 2006, 09:45 AM	#2
Jeff Kilgroe Major Player Join Date: Jun 2003 Location: Golden, CO Posts: 681	If you have a proper RAID controller, all the mirroring will be handled by the controller and there won't be any additional drain on system resources or performance hits. When you are done, you will have two identical drives full of data. If it's a RAID attached to your Macbook, then the ideal solution would be an external drive enclosure with some form of RAID controller onboard. You can do it without, and just use an enclosure with two SATA connectors and then handle the RAID via software on the Mac. This method would introduce a slight performance hit, but would probably be insignificant anyway and would be cheaper due to a cheaper enclosure. The advantage of using a proper RAID-1 to create a mirrored drive, is that it all happens real-time as you work with the data. No duplicating later so there's a time savings. It also offers some redundancy and a bit of secruity as you work in case one of the drives fail. __________________ - Jeff Kilgroe - Applied Visual Technologies \| DarkScience - www.darkscience.com

April 22nd, 2006, 10:05 AM	#3
Giroud Francois Inner Circle Join Date: Feb 2004 Location: switzerland Posts: 2,133	raid controller are "aware" of the structure you choose, so building a mirror RAID will give you 2 copy of the same disk, but breaking the mirror will be seen as a problem by the raid controller, so you will need to add a new disk (same type) and rebuild the mirror with the raid utility. The good disk will be then copied to the new disk. So basically there is no escape to the copy process , you just delay it.

April 22nd, 2006, 10:50 PM	#4
Gary L Childress Regular Crew Join Date: Feb 2006 Location: Los Angeles Posts: 42	Scott, Avid has sold plenty of RAIDs over the years. Prior to the LVD 10,000 rmp drives they had alot of 2 drive pairs in use and Unity is a big RAID. But to the main point here, I think RAID 1 is a bad choice for video acquisition and here's why. Like you I thought this might be a good idea, so I went and bought a RAID 1 system, the Weibetech SilverSATA II. Since I was using a Powerbook, I bought a unit that had the controller inside to take a load off the laptop. The unit operates with only one SATA cable, so you don't have to worry about pushing the limits of the cardbus trying to pass two streams of video through the card at the same time. Here is my unfortunate journey down that path. The unit I purchased had removable sleds that proved to be difficult for drive installation due to the tightness of the screws. I had ordered an extra pair of sleds so that if I was on a shoot and ran out of drive space I could easily swap in a new pair preloaded with drives. I had such a difficult time installing the drives on the sleds that I wound up bending the rails, which caused the drives to have problems in the unit. In a sense it was fortunate that this happened, because the problems I had with the drives pointed out some very serious issues with using RAID 1 for acquisition. Basically, I had on my hands the very problem you would buy RAID 1 for in the first place, a failed drive. The unit would run for a bit and then show errors on the second drive. Then it would start the rebuild routine. This happened several times so I decided to let it go through the process just to see how long it would take. I had two 300gig SATA II 300 drives in the pair. The drives only had a couple of short SD clips on them that I had digitized as a test. Total rebuild time: nearly two hours, even though they only had a few megs of material actually on them. I went through this a couple of times with the problem sleds and then installed the drives in my desktop Firmtek SATA enclosures to see if the lengthy rebuild times were just the SSII unit. I took one of the drives with data and paired it with an empty drive in RAID 1 and let it go through a rebuild. That computer is a dual 2 ghz G5. Total rebuild time was nearly two hours. So how do you deal with that at a shoot. If you have a drive fail, you will have to choose whether to shut the shoot down for the rebuild. Or you could bring extra drives and sleds two swap in the unit in case of a failure. I you swapped in new drives, then you are in the precarious situation of having only one good copy of the original good disk and no ability to back it up manually without disrupting the shoot. Of course you could also carry an extra RAID 1 array. If you are going to have a RAID on a shoot, then you are PLANNING AHEAD for a drive to fail. So you are forced to consider what the scenario would be if a drive DID FAIL. I think RAID 1 does not present a pretty picture in a failure situation. Also, if you have a drive go down, you would be in the position of not knowing whether it was the drive or the RAID unit itself that was the problem. Thus you would have little confidence that there would not be similar problems with the replacement drive pair. It was a very expensive lesson for me. On the shoot I bought the RAID 1 for, I switched and took the Firmtek pair which was not in a RAID. I then hand copied clips to the second drive after each dump. That seemed much more secure for me. It worked very well. I think the best solution for acquisition is a RAID 5 unit. They are more expensive but offer the best protection. Despite my experience with the SSII, the manufacturer is well respected and makes a variety of very high quality drive products. I would have no qualms about buying their 5 drive portable RAID 5 enclosure. It's $1700 with no drives but for the security it's probably worth it.

April 23rd, 2006, 12:21 AM	#6
Leonard Levy Trustee Join Date: Feb 2006 Location: USA Posts: 1,684	Gary, That's a great post. All I can say is I'm glad it was you not me. That pretty much kills thge idea of a Raid for acquisition in my mind. Nothing beats the school of hard knocks. You should post this on DVXUser.com as well.