best RAIDed hard-drive set-up for EX-1?

[Jonathan Bland]

Very cool Barry and really appreciate your take on this.

I think your right about having some kind of tutorial.... I've been working on FCP3 which I bought in Bangkok years ago and only half learned in the jungles of Laos and India. Sure would have been faster to take a few courses. Now I'm heading out again and this time.... tapeless with this new heavy weight version of FCP!

"Second, based upon my experience with FCP ProRes(HQ) video sequences. Two drives stripped (i.e. RAID-0) together are fast enough to edit at least three streams without dropped frames. I usually edit in FCP with timeline viewing set to full quality i.e. RT editing disabled. So for your setup, use a two port e-SATA card with your MacBook Pro and strip two drives together. I'm confident it will work fine for you with ProRes(HQ) sequences."

Got it.... and agree. But why go with Prores when Daniel (above post) says it causes him and his system trouble? Something is missing here.

I will be shooting a doc over the next 3 years in multiple countries living out of rolling duffle bags. I need to shoot/ offload/ back-up/ edit. The film will be a feature around 90 min. Do you or anyone else envision an offload/ workflow/ edit storage solution. I'll probably end up with around 2.5 TB of material over the course of shooting and that's before backing up.

I don't mean to highjack this thread.... but I've got like no time left to nail this.... and once your in the developing world..... it's a long haul/ expensive to get your gear sorted if you didn't the first time around.

Thoughts?

[Barry J. Anwender]

Originally Posted by Jonathan Bland

Very cool Barry and really appreciate your take on this.

Got it.... and agree. But why go with Prores when Daniel (above post) says it causes him and his system trouble? Something is missing here.

Sounds like Daniel is up and running, go for it. Firewire 800 transfers top out around 50MB/s continuous which makes sense for two mirrored drives (i.e. RAID-1) but provides no benefits for two stripped drives (i.e. RAID-0) because a single STATA-II drive performs better than 50MB/s. If you have the newer/latest drive controllers in your drive array, than transfers may be higher but still will only peak to 80 MB/s dual-feed firewire theoretical." Bare Feats" provides the test reports on how real world firewire performs on your MacBook. Search a little farther and you will discover why dual-feed firewire ports are used.

From Daniel's post, apparently Firewire 800 is enough for his needs with native XDCAM-EX video. My workflow needs to be flexible and depending on the project/number of streams is more demanding than Firewire can provide, hence my choice to a eSTATA port controller and the Sonnet Fusion 500P with five bays. I will stripe from 2-5 drives to accommodate the performance needs of any given project and number of streams -- flexibility is key because SATA drive transfer speed decrease as the drive fills. I keep my working backups spread across the spare three bays in the MacPro. Off-line backups to Blu-ray 25/50 GB DVD.

ProRes has several merits over EXCAM Long GOP native to the EX. That discussion belongs in the FCP forums. Some basic training will sort out your questions or concerns on that score.

As you know the Final Cut Studio 2.0 comes with a 6+ GB DVD with video tutorials that provide a pretty good overview of the essential features in the suite. Time well spent. For the next level down, I would suggest Lynda.com's video tutorials entitled, HD Workflows for Final Cut Studio 2.0. It is very good and should provide enough knowledge, so that you can avoid the obvious got-ya's. Here you can discover the significance of ProRes(HQ) timelines. Then you can focus on refining your workflow to suit your particular needs. Again, this one is time well spent. Just a suggestion.

After that, there a least a dozen other worthy titles that go into depth and may or may not be useful to help you with this workflow. This is were I use my intuition to guide me and only dig into the details if I run into issues or need to hike up the learning curve to achieve something new. Like shooting -- editing is also a creative process, a non-linear process. Live, Learn and play to have fun, rather than frustration :-))

[Malcolm Hamilton]

Originally Posted by Sean Donnelly

Now your 5th bay has to be 4 times the size of the others to be able to back up a full array. RAID 5 isn't a foolproof backup system, but it's great to work from. Can I ask which place this was?

-Sean

You're right, Sean... that does sound crazy; the advice comes from a guy (he's with FirmTek) who sounds very nice, though; I suspect I might have been quoting him out of context, or misunderstanding what he was saying. Here's what he said:

"I find that using 4-bays as a striped RAID set (RAID 0) and the 5th bay for off-line backup disks works well"

He goes on to say, re RAID 5:

"You should know that RAID 5 may be recoverable in the case of a hard disk failure but it cannot protect you from directory errors or a user error. An off-line backup is required for any important data NO matter what RAID type you use. In addition, rebuilding a RAID 5 with large hard drives can take a day or more. Sometimes recovering with a new RAID 0 is actually faster with a backup than waiting for a RAID 5 to rebuild"

What do you think?

In the meantime, Jonathan, are you any closer to having the set-up you need? You now know you don't need (or want) to edit uncompressed. Since you'll be travelling, would a four or five-bay eSATA enclosure be too big? If that's the case, are you thinking 2-bay eSATA, or Firewire 800? By the way, where in Canada do you live? Barry's got the praries covered, I'm in Ottawa...
Cheers, Malcolm

[Daniel Weber]

Originally Posted by Jonathan Bland

"Nope. I edited it native. The ProRes codec takes up more bandwidth on the drives than native XDCAM HD files."

More bandwidth?

You mean more space?

What are we talking about?

Prores requires greater bus/ throughput speeds?

Man this could take forever if we don't use the right terms. Myself included ;)

Sorry to be confusing!! By bandwidth, I meant data rates. I was thinking that since you will be out in the field it would be hard to carry around an external RAID enclosure. If you can, then that would be the best option. For the kind of stuff that I shoot in the field, I rely on small bus powered firewire drives. I am only transferring files and doing rough cut editing though. I come back to the office and edit on a MacPro with a FC server.

I think that you can do your doc on a Macbook Pro with the XDCAM codec. Then when you get back home, do your final edit by converting your offline to ProRes for your final output.

By editing in XDCAM in the field, you will be using up less drive space my not having to convert the files. Remember everything that you shoot will have to be converted to quicktime files by Final Cut. This will double the amount of storage you will need.

[Jonathan Bland]

Hi Daniel,

I will have a base from which to work from (rented house/ flat) with a desk to work at. If I'm traveling (on the move) I'll have a hotel to come back to and ingest the cards and start cutting. Simple.

Question..... Why would I want to edit full 1920x1280 XDCAM footage (out in the field as you say) then bring it home to the West (to online) and compress it into Prores and loose quality in this compression? It does not make sense.

What if.... I ingested the EX1 footage,
Make an identical backup on another drive and store it,
Then go back to the 1st drive and convert the original XDCAM footage to Prores and then deleted the original XDcam footage so that I'm left with only the Prores stuff.
Then when I have a locked picture in Prores I could go back to the original Full res XDcam and do a match back.

Thoughts?

[Daniel Weber]

ProRes is a better codec to do your master in. It is a 4:2:2 signal which means that when you do any color correcting and FX work it will hold up better. I don't think that you need to convert to ProRes until your online, but someone with more experience than me may tell you otherwise.

I would think that editing in the native format for your offline would work better than converting to an online format before hand.

In my mind it would be better to convert to ProRes at the end instead of the beginning.

Daniel Weber

[Barry J. Anwender]

Originally Posted by Daniel Weber

ProRes is a better codec to do your master in. It is a 4:2:2 signal which means that when you do any color correcting and FX work it will hold up better.

I have to agree with you Daniel. Moreover, ProRes(HQ) is 10-bit and does not require time consuming conforming. Other beneifits include precise placement of transitions and handling of VFX.

[Jonathan Bland]

Right on Daniel.
Thanks :)
Good to have your eyes on this Barry. Cheers.