Ok - which MacPro/Raid system for HD Multicam editing?

[David Cleverly]

Ok, it has become apparent that we are going to have to bite the bullet and invest in a system capable of multi-stream HD video editing. It looks like that's where we are going to get a substantial amount of our future work.

Would someone be kind enough to inform what the best system for editing multi-stream HD video would be.

Of course I understand that will be a MacPro with a RAID system installed, but which MacPro (is it necessarily the most expensive - is the 12-Core necessarily the best option?) and what is a suggested Raid system?

Any guidance to get the ball rolling would be appreciated.

Thank you.

David

[Craig Parkes]

It depends a lot on your workflow, but in regards to final cut and editing on the timeline there is no significant advantage between a sing duel core system and a 12 core system because of it's lack of native multithreading and only being 32 bit capable even if full multithreading was on the cards it would be hard for it to address enough ram to make a difference.

So, for a Final Cut workflow your workflow is likely to something relatively low overhead on processor, but high bandwidth. E.g transcoding/ingesting to prores or even uncompressed. What you then need is a very high bandwidth raid, and the best supported graphics card for your system set up, and enough ram. More cores will helpful when transcoding in compressor, but the reality there is often that more cores are more useful spread across multiple workstations and then set to transcode through to the shared fast raid on the network, so again there really is little need for the top of the range Mac Pro, especially if you look at the second hand market where you can often pick up a good deal.

There are many RAID controllers from third parties and many enclosures that will be able to perform Up to your specs, a year ago for about 3K NZ I picked up an 8bay array enclosure that connected to a raid controller using mini SAS connectors and 8x1.5TB SATA drives to fill the enclosure and set it up as a raid 5. That gives me a 500Mb/s+ read/write speed depending on how full the drives are which should sustain about 20 steams of full hd prores with overhead. However I doubt any final cut system could manage that many streams of full prores playing back simultaneously because of decompression/processor overhead, but I am think you could get 5 or 6 going on a reasonably spec'd system.

Same system could probably manage 4 streams of uncompressed 8bit full hd at 50i/25p and you wouldn't have to worry much about processor limitations at all until you start to stack fx.

If final cut software wasn't such a bottle neck, then it'd be easy to spec up a system to handle most any regular workflow, but currently you will need to find the sweet spot of graphics card, raid array, ram and codec to handle your workflow based on the number of cameras you are multiple cam editing. Number of cores (unfortunately) just doesn't make a huge difference on the timeline in final cut, or as far as I am aware on most othe NLEs as more and more effects get handed off to the gpu.

[William Hohauser]

This might help you make some decisions about your system (even the iMac you have now):
http://images.apple.com/finalcutstud..._ProRes_WP.pdf

Something I forgot about was that you can change the real-time playback quality and open up more multicam streams for switching. They don't look great while running the timeline when switching but that's temporary.

While a 12 core system isn't going to help much within FCP, it can be a boon for Compressor tasks.

[David Cleverly]

Thanks guys,

Trying to make the imac do the job at hand, somewhere I read that converting all my XDCAM HD clips to ProRes would allow the iMac to handle more streams of video than by using the XDCAMHD clips, but as a trade-off, the ProRes clips would be bigger than the originals.

I discovered that, yes, while the clips were indeed about twice the size of the originals and the quality appeared just as good as XDCAM HD, they did NOT play back any better.

After reading WiIlliam's recommended Apple ProRes PDF, I am somewhat confused because of it mentioning ProRes clips should be a fraction of the size of full-res HD files. Have I done something wrong?

I was unsure of which format of ProRes to convert to, so I converted to ProRes 422 HQ. Should I have converted to a different format of ProRes to achieve the better playback?

Thanks again for your support, guys.

David

[William Hohauser]

XDCam is more compressed than ProRes. It utilizes a MPEG compression formula that operates in groups of frames sharing data where ProRes operates with each frame taking up it's own set of data. Wikipedia has some excellent pages describing these files if you want to get familiar with the materials you are working with. ProRes is smaller than uncompressed HD while offering quality images. Learn how to change the real-time playback quality settings in FCP and you might be able to do six camera editing with the iMac you have now.

[Craig Parkes]

Prores is much larger than XDCAM-HD - but much smaller than UNCOMPRESSED HD.

Almost no one uses uncompressed HD for editing (except, potentially, online editing on all HDCAM originated shows or finishing for feature films, where uncompressed HD/2K would be the norm in the form of DPX sequences.) but if you are using it, it's MUCH larger than prores

XDCAM-HD is much more processor intensive than PRORES, but requires less bandwidth.

Your iMac set up is hitting bottlenecks at both ends. Can't handle the processor requirements of decoding 6 streams of XDCAMHD, AND can't handle the data throughput necessary for 6 streams of prores, even if it could handle the processor overhead.

The combination of these two means that currently your only option is either a more powerful machine, or potentially different software (depending on your set up you may have more luck cutting the XDCAMHD footage with Premiere CS5, because it is much more recently coded software that can potentially take advantage of much more computing power on your iMac. You may still run into drive issues though trying to edit 6 streams of XDCAM HD because of seek speeds of your drives even if the total bandwidth is only about 30/Mb/s and your firewire 800 should be able to handle that, the attached drive could prove a bottleneck in terms of accessing all streams simultaneously when getting into serious cutting.

[David Cleverly]

So you are saying that - in my "iMac" world, ProRes will NOT help me. Correct?

Ok, now here is something weird:

I just discovered that (so far, anyway, that this works) the first of any edits in the multi-clip edit are the troublesome ones. Playback happens as it should; it is just not updated in the viewer and canvas as it should be with the first four or five switches.

In another way to explain the real problem, the time line plays, I select a new camera and BANG! The timeline stops for a second but the playhead resumes again automatically without the viewer and canvas playing along; they have both stopped. I have to hit the space bar to stop playback, then hit it to start playback again with the viewer and canvas again linked in playback. The same happens with the second, third and fourth switch. Then, if I hit the spacebar again to resume playback, the multi-clip edit works from there on in until I choose to stop the timeline. It's random. But whatever the problem, I can actually get to a point, after 4 or 5 tries, that the playback is not interrupted by a camera switch.

Still, if I stop the playback by choice, the very next few edits will cause the playback to stop again. But again, from there on in, each edit works without stopping the playback.In fact I can do switches one after the other quickly and there is no interference to the playback. It's almost as if the computer has to gain momentum to avoid the switch stopping playback.

All very weird.

[Craig Parkes]

That's very indicative of bandwidth limitations or processor issues on an over taxed system, I.e not having constant access to everything.

[David Cleverly]

So here is another scenario:

There is every chance this project will not have to be HD. If I discover this project does not have to be HD, would I benefit in my editing (bandwidth issues with 6 streams) by converting all the HD stuff to, say, DVCPRO50? That way we would have high-end SD format with images similar to Digi-BETA quality but less taxing on the system...

Cheers

[David Cleverly]

Sorry to go on about this, but I am learning so much and being confused at the same time.

For argument's sake, I decided to try my 15" i7 Macbook Pro with the Firewire Hard Drives that I had hooked up to the iMac, with all the different HD camera clips spread over them...

Here are the TImeline settings:

Unlimited RT
Playback Video Quality - Dynamic
Playback Frame Rate - Full

...and guess what? I can switch the 6 streams of HD from the very same FW800 hard drives from the iMac. Perfectly. No stopping at any time.

Any ideas why?

I thought the MacBook would be less able to handle this for some reason...

Or is it just the "physical size" thing that got me? :-)

Cheers.

[Craig Parkes]

Could be graphics card related then. Or could just be that one of the computers has pro in the title, the other doesn't. :)

[Kevin Kelly]

David,

I am following this topic with great interest as I just bought an iMac and looking for info on how to set up fast external storage.

Interesting about the MBP performance. What model and year iMac do you have?

Thanks,
KK

[David Cleverly]

Hi Kevin,

It is the iMac i7 27" previous to the current one.

I am having other problems now, too, like messages saying 42 hour export file times from FCP for a 90 minute program whereas my lesser model Core 2 Duo 3.06 27" iMac does it easily in 30 minutes.

I have had no other issues with this machine until recently. It has always edited HD video without issue...it was only when trying to do HD Multiclip editing is when the problems arose...

For what it is worth to this topic, I did a verify disk in Apple's Disk Utility and it tells me the main hard drive/Startup disk is corrupt and needs repairing.

Something is def wrong....maybe the file system on the HDD needs repairing to solve all these problems.

[Oliver Neubert]

You have certainly worked it out by now - but still:

Free up diskspace !
- try to keep at least 10% of your available diskspace free. This should increase performance.