do i need a RAID config for my mac pro?

Al Wilson · January 20th, 2007, 03:05 PM

Quote:

Originally Posted by Chuck Spaulding

I have the Western Digital WD500KS SE16 Caviar Disks [$159]. They have 16MB of secondary cache [apposed to 8MB normally] and are generally fast and reliable. They are a bit noisy on start-up but quite down nicely after five to ten minutes.

I have that exact same drive and use if for backup. It vibrates a little bit but not as much as the WD RE 320 models. I did have a problem at first due to some harmonic resonance, but by moving the drive to bay 4 seemed to eliminate that issue. I bought the RE drives solely for RAID and they substantially increased my vibration/sound problems until I stuffed some padding in there. They still buzz now and then but for the most part they are okay. I'm more concerned now about whether to keep them for go with another brand for my RAID setup. Are you using the SE16 Caviars for RAID?

You found a good price for RAM. I've bought ram online before and didn't have trouble but I might have gotten lucky. The prices have been slowly coming down; I would like to see them come down another 10-20% before I buy. I did find some ram on Newegg that was decent, but it's not Apple certified like they advertise on Macsales. I'm reluctant to go with anything other than the best. Ram is the most important components of the total system and a person shouldn't skimp when they have already invested so much in everything else.

Thanks!

Chuck Spaulding · January 21st, 2007, 01:41 AM

Yes I'm using the Caviars for the RAID.

Like I said, I shopped around for a legitimate price and after informing my Apple reseller they matched the price. I opened the Mac up and inspected what they installed and it matched the factory RAM [except for the size].

I'm guessing everyone is getting the RAM from the same four or five sources, even Apple, so all the reseller might be loosing is the additional mark up. Since I purchased the system from them they didn't seem to mind.

Jeff Krepner · January 22nd, 2007, 01:38 PM

On the topic of Western Digital Drives, I see there is a SE and RE (raid) additions. According to the newegg site--the RE version is better for RAID:

"RAID-specific time limited error recovery - Improves error handling coordination with RAID adapters and prevents drive fallout caused by extended drive error-recovery processes. Reliability in high duty cycle environments - Utilizing enterprise-class manufacturing processes, these drives are designed with enhanced reliability in a 24x7, high duty cycle environment."

Has this been confirmed? In all of my years I've never seen a drive specified for RAID and non-RAID environments.

Any thoughts?

Al Wilson · January 22nd, 2007, 02:31 PM

Quote:

Originally Posted by Jeff Krepner

On the topic of Western Digital Drives, I see there is a SE and RE (raid) additions. According to the newegg site--the RE version is better for RAID:

"RAID-specific time limited error recovery - Improves error handling coordination with RAID adapters and prevents drive fallout caused by extended drive error-recovery processes. Reliability in high duty cycle environments - Utilizing enterprise-class manufacturing processes, these drives are designed with enhanced reliability in a 24x7, high duty cycle environment."

Has this been confirmed? In all of my years I've never seen a drive specified for RAID and non-RAID environments.

Any thoughts?

I'm using the RE 320's that I bought specifically for RAID0. They are good drives with the exception of vibration. However, I might have gotten a fluke because one of them vibrates more than the other. Seagate has RAID type drives too. They are the ES (Enterprise Server) drives and are supposed to be longer lasting that normal drives. You pay a bit more for these kind of drives and whether it's worth it or not, I'm not sure. I bought RE drives because of the decent reviews and it seemed like a good idea at the time. Keep in mind that these drives are designed to run in a server environment, and we are running them in a single user desktop environment (at least I am).

Jeff Krepner · January 22nd, 2007, 04:08 PM

Thanks Al. I currently capture to one Western Digital SE 16 drive and wanted to just buy two more of the SE drives to set up as a RAID. At the time I didn't know they made a RE RAID edition. I guess I'll buy three new "RE" drives if I want to stick with my original plan of stripping three drives.

Jeff

Douglas Hockly · March 12th, 2007, 06:07 PM

I have a very similar setup, am planning on doing a largeish (60-90 min) 720 HD project. I am a relative newbie to video editing (but have done lots of multimedia stuff)

- is it worth filling my last 2 hard drive bays with identical drives and RAIDing them?

- what sort of RAID is the best for performance (say I backup some other way)? I could make a third drive available.

- is the software RAID that Apple provides OK? Can you boot from a RAIDed set? How does it compare to getting a dedicated RAID card? (oh, the questions!)

- it seems like it'd be ideal to have 4 drives available, one for the OS, one for the scratch, one for reading the media from and one for writing it to. What matters/slows things down, and what doesn't?

- What should I use the RAID set for - media, scratch, OS...?

Thanks in advance,
Douglas

Chuck Spaulding · March 13th, 2007, 01:02 PM

Is that 720P uncompressed, DVCProHD or HDV?

First off, I always try and discourage anyone from editing HDV natively. [but that's a topic for another thread].

So having said that I'll assume its one of the other formats. A 90 minute 720P (HD) program is a lot of data. Most of this discussion has been about a small independent project where there's not a lot of concern about the "safety" of the video.

For this large of a project you should consider a higher RAID level that will give you some redundency. The trade off is that the higher the level of security the lower the performance.

So to make up for decreased performance you need to increase the number of spindles [drives]. There are some fairly inexpensive SATA RAID enclosures that have 8 drive bays, eight would be the smallest I would go. Many have [and I don't remember the latest buzz word for this] port multiplication where they can stream all eight channels of data across two SATA cables for example.

I have a fairly simple philosophy about setting up a file system. Everything to do with the FCP project itself, graphics, budgets, stills etc., are stored in a client/project folder in my FCP folder on my system disk. This is content that generally required some human effort to create and could not be easily or quickly replaced.

If I'm using RAID0 the media, audio, video and elements created by the computer [rendered] that would require little [if any] human intervention are stored on the RAID. If the RAID fails or there are corrupt files then generally I take the batch digitizing list, which is stored with my project files and re-capture. What determines the RAID level is my threshhold for pain in re-capturing. Generally if there's more than four or five hours of work to get back to where I was before the failure I go up to the next RAID level.

I have both RAID0 and 5. You'd think I'd just use the RAID5 for everything but I actually use the RAID0 more. So far I've only done one feature film in HD, which I used RAID5, but most of my work is short form HD and its on and off the system fairly quickly so I use RAID0 [G-Techs SATA RAID]. This also makes it much easier to share the drive. I don't want to cart an 8-drive RAID around to other artisans working with me on a project.

Hope this helps.

Douglas Hockly · March 13th, 2007, 09:24 PM

Thanks for your time Chuck, that is great info to consider. Yes the longest thing I've done is 40min SD so it will be significantly bigger, good to think about it in advance. I can see the logic in your system.

David McGiffert · March 14th, 2007, 11:31 AM

Chuck,
you said:
"First off, I always try and discourage anyone from editing HDV natively. [but that's a topic for another thread]."

I would be most interested in reading your recommendations of how
to edit HDV. I have read that doing basic cuts in HDV then transcoding to
DVDProHD for color-correction and transitions etc is one way to go.
But, with the proponderance of information on these threads, I can't
find where I originally read that...

David

Chuck Spaulding · March 16th, 2007, 02:05 AM

Hi David,

Basically what people are doing by using the workflow you describe is using the DVCProHD codec as a "digital Intermediate." If I wanted to use the DVCProHD codec I would convert the HDV to DVCProHD on capture and edit that.

However I am not a big fan of the DVCProHD codec. Cineform recently announced a public beta of ProspectHD to create a QuickTime version of their Cineform codec that works with FCP. I have used the Cineform codec on the PC and it is a much better codec than the DVCProHD codec. You can learn more about it at http://www.cineform.com.

There is also the Sheer Video codec which I've heard good things about but I have not been able to test it yet.

Basically these codecs transcode the HDV from an m2t stream to a frame based 8 or 10bit 4:2:2 stream using wavelet compression. When you look at the comparisons on the Cineform website they are comparing the Cineform image to an uncompressed HD image, not an HDV image. It is very impressive.

Obviously there's a penalty, the amount of data goes from 25Mb's to approximately 100Mb's which is comparable to the data rate of DVCProHD. So you will need to make sure you have sufficient I/O bandwidth. It appears that both of these codecs have much better image quality with a smaller data footprint compared to DVCProHD and arguably comparable to uncompressed.

I'm hoping to have the opportunity to test both of these codecs soon. There is a third option and that is to use the Apple Intermediate Codec (AIC). It works well with FCP but just like the other two codecs mentioned it significantly increases the data rate but it is a 4:2:0 8bit codec. I'm not sure why Apple would have created such a codec unless it was to enable people to edit HDV until FCP could edit HDV natively.

Editing HDV natively is a bad idea, it is 4:2:0 8bits and no matter what you do, it must render to calculate where the edit point is in relation to the I-frame in the GOP. This can take a really long time.

Editing HD(V) is relatively new to all NLE's and unlike DV, which was ubiquitous and everyone pretty much edited it the same way there is no “one-size-fits-all” work flow for HD. Where I’ve noticed most people having a difficult time editing HDV is when they try to edit it the same as DV. Yes from an interface perspective it may appear similar, but HD is more like film than it is like SD/DV and if you’re going to edit with different codecs then you’re also working in more of an IT environment than a video one. Which again is the topic for yet another thread.

March 13th, 2007, 09:24 PM	#23
Douglas Hockly New Boot Join Date: Mar 2007 Location: Australia Posts: 13	Thanks for your time Chuck, that is great info to consider. Yes the longest thing I've done is 40min SD so it will be significantly bigger, good to think about it in advance. I can see the logic in your system. Last edited by Douglas Hockly; March 14th, 2007 at 05:17 AM.

January 21st, 2007, 01:41 AM	#17
Chuck Spaulding Trustee Join Date: Nov 2004 Location: Thousand Oaks Posts: 1,104	Yes I'm using the Caviars for the RAID. Like I said, I shopped around for a legitimate price and after informing my Apple reseller they matched the price. I opened the Mac up and inspected what they installed and it matched the factory RAM [except for the size]. I'm guessing everyone is getting the RAM from the same four or five sources, even Apple, so all the reseller might be loosing is the additional mark up. Since I purchased the system from them they didn't seem to mind.

January 22nd, 2007, 01:38 PM	#18
Jeff Krepner Regular Crew Join Date: Sep 2005 Location: Timonium, MD Posts: 142	On the topic of Western Digital Drives, I see there is a SE and RE (raid) additions. According to the newegg site--the RE version is better for RAID: "RAID-specific time limited error recovery - Improves error handling coordination with RAID adapters and prevents drive fallout caused by extended drive error-recovery processes. Reliability in high duty cycle environments - Utilizing enterprise-class manufacturing processes, these drives are designed with enhanced reliability in a 24x7, high duty cycle environment." Has this been confirmed? In all of my years I've never seen a drive specified for RAID and non-RAID environments. Any thoughts?

January 22nd, 2007, 04:08 PM	#20
Jeff Krepner Regular Crew Join Date: Sep 2005 Location: Timonium, MD Posts: 142	Thanks Al. I currently capture to one Western Digital SE 16 drive and wanted to just buy two more of the SE drives to set up as a RAID. At the time I didn't know they made a RE RAID edition. I guess I'll buy three new "RE" drives if I want to stick with my original plan of stripping three drives. Jeff

March 12th, 2007, 06:07 PM	#21
Douglas Hockly New Boot Join Date: Mar 2007 Location: Australia Posts: 13	I have a very similar setup, am planning on doing a largeish (60-90 min) 720 HD project. I am a relative newbie to video editing (but have done lots of multimedia stuff) - is it worth filling my last 2 hard drive bays with identical drives and RAIDing them? - what sort of RAID is the best for performance (say I backup some other way)? I could make a third drive available. - is the software RAID that Apple provides OK? Can you boot from a RAIDed set? How does it compare to getting a dedicated RAID card? (oh, the questions!) - it seems like it'd be ideal to have 4 drives available, one for the OS, one for the scratch, one for reading the media from and one for writing it to. What matters/slows things down, and what doesn't? - What should I use the RAID set for - media, scratch, OS...? Thanks in advance, Douglas

March 13th, 2007, 01:02 PM	#22
Chuck Spaulding Trustee Join Date: Nov 2004 Location: Thousand Oaks Posts: 1,104	Is that 720P uncompressed, DVCProHD or HDV? First off, I always try and discourage anyone from editing HDV natively. [but that's a topic for another thread]. So having said that I'll assume its one of the other formats. A 90 minute 720P (HD) program is a lot of data. Most of this discussion has been about a small independent project where there's not a lot of concern about the "safety" of the video. For this large of a project you should consider a higher RAID level that will give you some redundency. The trade off is that the higher the level of security the lower the performance. So to make up for decreased performance you need to increase the number of spindles [drives]. There are some fairly inexpensive SATA RAID enclosures that have 8 drive bays, eight would be the smallest I would go. Many have [and I don't remember the latest buzz word for this] port multiplication where they can stream all eight channels of data across two SATA cables for example. I have a fairly simple philosophy about setting up a file system. Everything to do with the FCP project itself, graphics, budgets, stills etc., are stored in a client/project folder in my FCP folder on my system disk. This is content that generally required some human effort to create and could not be easily or quickly replaced. If I'm using RAID0 the media, audio, video and elements created by the computer [rendered] that would require little [if any] human intervention are stored on the RAID. If the RAID fails or there are corrupt files then generally I take the batch digitizing list, which is stored with my project files and re-capture. What determines the RAID level is my threshhold for pain in re-capturing. Generally if there's more than four or five hours of work to get back to where I was before the failure I go up to the next RAID level. I have both RAID0 and 5. You'd think I'd just use the RAID5 for everything but I actually use the RAID0 more. So far I've only done one feature film in HD, which I used RAID5, but most of my work is short form HD and its on and off the system fairly quickly so I use RAID0 [G-Techs SATA RAID]. This also makes it much easier to share the drive. I don't want to cart an 8-drive RAID around to other artisans working with me on a project. Hope this helps.

March 14th, 2007, 11:31 AM	#24
David McGiffert Regular Crew Join Date: Feb 2007 Location: Topanga, CA Posts: 139	Chuck, you said: "First off, I always try and discourage anyone from editing HDV natively. [but that's a topic for another thread]." I would be most interested in reading your recommendations of how to edit HDV. I have read that doing basic cuts in HDV then transcoding to DVDProHD for color-correction and transitions etc is one way to go. But, with the proponderance of information on these threads, I can't find where I originally read that... David

March 16th, 2007, 02:05 AM	#25
Chuck Spaulding Trustee Join Date: Nov 2004 Location: Thousand Oaks Posts: 1,104	Hi David, Basically what people are doing by using the workflow you describe is using the DVCProHD codec as a "digital Intermediate." If I wanted to use the DVCProHD codec I would convert the HDV to DVCProHD on capture and edit that. However I am not a big fan of the DVCProHD codec. Cineform recently announced a public beta of ProspectHD to create a QuickTime version of their Cineform codec that works with FCP. I have used the Cineform codec on the PC and it is a much better codec than the DVCProHD codec. You can learn more about it at http://www.cineform.com. There is also the Sheer Video codec which I've heard good things about but I have not been able to test it yet. Basically these codecs transcode the HDV from an m2t stream to a frame based 8 or 10bit 4:2:2 stream using wavelet compression. When you look at the comparisons on the Cineform website they are comparing the Cineform image to an uncompressed HD image, not an HDV image. It is very impressive. Obviously there's a penalty, the amount of data goes from 25Mb's to approximately 100Mb's which is comparable to the data rate of DVCProHD. So you will need to make sure you have sufficient I/O bandwidth. It appears that both of these codecs have much better image quality with a smaller data footprint compared to DVCProHD and arguably comparable to uncompressed. I'm hoping to have the opportunity to test both of these codecs soon. There is a third option and that is to use the Apple Intermediate Codec (AIC). It works well with FCP but just like the other two codecs mentioned it significantly increases the data rate but it is a 4:2:0 8bit codec. I'm not sure why Apple would have created such a codec unless it was to enable people to edit HDV until FCP could edit HDV natively. Editing HDV natively is a bad idea, it is 4:2:0 8bits and no matter what you do, it must render to calculate where the edit point is in relation to the I-frame in the GOP. This can take a really long time. Editing HD(V) is relatively new to all NLE's and unlike DV, which was ubiquitous and everyone pretty much edited it the same way there is no “one-size-fits-all” work flow for HD. Where I’ve noticed most people having a difficult time editing HDV is when they try to edit it the same as DV. Yes from an interface perspective it may appear similar, but HD is more like film than it is like SD/DV and if you’re going to edit with different codecs then you’re also working in more of an IT environment than a video one. Which again is the topic for yet another thread.