Editing XDCAM EX, HDV and DVCAM

[John Wallace]

I've read quite a bit on these forums regarding mixing HDV and EX footage in FCP, but am a little confused.

Can someone tell me if I have this right?

The Footage Situation:
- 4 hours PAL DVCAM tape
- 45 hours HDV 1080i50 tape
- 70 hours EX3 source files (720p25 and 1080i50 already 'Logged and Transferred')
- Some off air SD archive on VHS
- Possibly some 16mm archive via PAL Beta/DVCAM

The Edit Situation
- Mac Pro 2x 2.66Ghz, 2GB
- FCP 6.06

The Output Situation
Right now I need to get stuck into the cuts but I want to finish the thing properly (colour and dub) with a view to film festival showings and a TX slot if all goes well.

So I need to make sure I can keep things in order so that the project can be moved around various facilities.

Here's my solution...
- Capture HDV as HDV
- Capture DVCAM as DVCAM
- Ingest EX3 as native XDCAM EX
- Edit in native XDCAM EX 1080i50 timeline with ProRes render turned on

Drive space isn't a major concern, but I'd rather avoid having to ProRes all the HDV right now as that would start to ache a bit.

Meantime, most of the HDV acquired footage is already sitting on a drive but downconverted to SD (old habits, loooong project). Can I use these as offline in the meantime or am I better to re-capture those tapes before I get started?

All help and suggestions gratefully appreciated!

Meanwhile, a very merry festive season to you all.

JW

[Gary Nattrass]

Hi John what is your delivery format?

It may be that to speed up everything in the edit is would make sense to put it all to pro res at ingest so that it is all in the same format.

Or it may make sense to load the Dvcam and HDV material as XDcam EX as that is what you have most content on at least then all your material will be in the same format for editing.

You don't have a huge processing machine with lots of RAM so keeping everything to the same format will make everything quicker to edit.

I don't know a lot about XDcam EX but I suspect that it will be the same as my P2 where final cut re-wraps the native files as pro res so loading your other media in a similar format may be better than working with multiple formats.

[William Hohauser]

Edit in the HD size the majority of your footage is in. This seems to be 1080i.

Your MacPro is not the fastest out there but it's certainly fast enough to edit this footage with. You might find that HDV footage will not play on a XDCam sequence without a render, I don't have experience with this set-up. However I have mixed XDCam and HDV in an edit and I converted everything to ProRes which worked fine although you do have to schedule time for the conversion.

Another way to edit without converting all the files is to edit in a ProRes sequence. Everything will play in preview mode (not full resolution) and will need a final render but you can mix codecs without the dreaded red line. And if you stop the playhead, the still will revert to full resolution.

Dump the SD HDV footage but first use it to find the footage you really need so you can reduce the ingest time.

[John Wallace]

Thanks for the swift replies guys!

Originally Posted by Gary Nattrass

Hi John what is your delivery format?

This needs to be kept flexible - DVD viewers (NTSC and PAL), film festivals and TX will all require different encodes (and physical formats)

Aiming for an HD quality master (notwithstanding BBC ruling on sub-50mbps sources not being 'HD')

William - thanks for that. I just tried a couple of things you suggested:

- Mixing some HDV and EX3 1080i footage on a ProRes timeline along with some 720p:
Everything comes out as 'Preview' (bright green line)

- Dumping HDV into XDCAM EX timeline
HDV comes out as 'Full' (olive green line) - no need to render to play back here either.

Unrendered playback quality on both is by no means distracting (only minor difference). I guess this will get worse the more I try to layer effects, but I usually do complex bits on a separate timeline and export the results as a flat, standalone clip.

Render times for 20s of HDV in XDCAM EX timeline are roughly the same as for 50/50 EX/HDV in the ProRes 1080i timeline. As I'm anticipating using mostly the EX3 footage, I'm inclined towards the XDCam timeline with ProRes render.

Unless someone thinks this could be stacking problems for the future?

For example, if the colour grade were to get complex, could I still use the HDV source timecodes to recapture from the tapes as ProRes 422?

I got into serious trouble once before trying to use an edit I started using HDV captured material to recapture and downconvert to SD (shudders at memory).

Finally, could there be a quality difference in rendering to ProRes 422 HQ at capture, or later once the cut is settled, as far as the HDV footage is concerned (aside from the enormous difference in storage space required)?

Incidentally William, my partner really doesn't like your suggestion of using the SD footage to find the stuff I really want to use. She had settled on the notion I'd only be feeding tapes today, not gawking at a monitor into the wee smalls.

Cheers for now.

JW

[Gary Nattrass]

I think your master then needs to be pro res 422 HQ 1920x1080i at 50i or 25p.

How you get there with the other footage is up to you but you need to be aware that some of it will need re scaling during rendering or you need to have everything in the same starting format.

As mentioned if XDcam 1080i is your main source then I think it would be good to convert everything to this format within the edit system as playing back one stream of mixed sources is one thing but when you start to add any captions or special effects or grading I think your system will then have to render each time you try something new.

Personally I work with panasonic P2 from my HPX cameras or AVCHD from the little canon HF11 but everything ends up as re-wrapped or transcoded pro res HQ or LT at 1920x1080i 25np.

I then master to pro res HQ at 1920x1080i 25p.

[William Hohauser]

Capturing as ProRes isn't going to change the quality of the HDV footage or change the quality of renders if you have it already set for ProRes renders within FCP. The rendering process does the same thing as if you captured it in ProRes. Grading in Color, I'm not sure as I've never used it.

Have fun capturing.

[Craig Parkes]

If you are doing colour correction, titles, effects, grading at all on the timeline better to work on a prores timeline, footage doesn't need to be prores.

If you then want to go to color though, the different formats could cause issues, and or sending render files to color could cause issues. What you can do is render out a self contained QuickTime sequence of the final edited sequence and export an EDL of a cuts only single timeline version of he edit (without any still graphics, just replace them with slugs) and load this in color as a cut list for the QuickTime. You won't get handles on the footage but it is a way to avoid a lot of the round tripping hassles with Color, you just treat as if you were sending a tape based master off to be color graded.

Also, just a friendly reminder to watch out with the DVCam as it's lower field first. On the hd timeline final cut is going to automatically put a filter on to reverse the field order, this can bit you if you are not monitoring on an interlaced monitor and you copy and paste filters between footage and end up accidentally switching fields somewhere. Good reason to label the different source clips with different colors so you can quickly see which footage should have that field order filter on it an which should not.

Also, additionally, you might run into issues with the 25p stuff on the interlaced timeline with certain effects not matching the other footage because it's not interlaced. There are a couple of work abounds here, but probably not that necessary.
If it was 720p50 footage however I'd consider upresing it into 1080i50 first to properly, ensuring you don't lose the temporal information. (unfortunately if you drop 720p50 on a 25i timeline final cut just throws away the extra frame and splits up the fields from a single frame)

[John Wallace]

Again - thanks both for taking the time to think a bit about this for me.

Another experiment:

Taking what's been said about having to re-render with transitions and filters added, I lobbed three layers of CC 3-Way onto some HDV footage in the XDCAM timeline, then added some lower third text, with a dissolve to ease it in.

Everything retains 'Full' RT playback except the bit of text under the dissolve which pops up as 'preview'.

I can live with this while I get the heavy lifting done - maybe my machine isn't quite as dowdy as you'd expect.

Agreed I'll have fun rescaling and getting the arc right on the older archive.

Grateful for assurance on timing of ProRes re-rendering as far as eventual quality is concerned and I'll take the tip on a ProRes 422 HQ output for master.

All the best to you both for 2011.

JW

[John Wallace]

Originally Posted by Craig Parkes

a friendly reminder to watch out with the DVCam as it's lower field first. On the hd timeline final cut is going to automatically put a filter on to reverse the field order

Ah deary me yes. I had forgotten about that. What jolly fun I had the first time I fell victim to that splendid wee eccentricity.

I do have some 720p25 to deal with (overcranked wildlife and action shots), but I'll file your wisdom on upressing 720p50 in a safe place.

Now I'm curious though, why is it better to use a ProRes timeline? Does the 4:2:2 colour space make a difference, even when the best source I have is 4:2:0?

Also, why should (does) the 'Preview' quality of raw HDV on the ProRes timeline look exactly the same as the 'Full' quality HDV on the XDCAM timeline?

My gut instinct is that, yes, it's surely always better to be dealing with only one file format on a single timeline.

However, your old-school 'nail it down the send it to the grade' method of working with Color makes sense and is kinda what I thought might be the best route for me, time and storage space wise. I can work with 3-way to nudge the colours to an acceptable state until I get to a final cut. I am not a colourist and would really want to get someone else to do this part, ditto the dub.

I'm still interested to know if it's possible to re-link a cut to ProRes re-conformed footage when it originally referenced HDV or XDCAM EX footage.

Anyone tried it? Or do I feel another experiment coming on?

JW

[Shaun Roemich]

Originally Posted by John Wallace

I'm still interested to know if it's possible to re-link a cut to ProRes re-conformed footage when it originally referenced HDV or XDCAM EX footage.

IF you are using Firewire to do an HDV to ProRes, you can't Log and Capture - the HDV to ProRes option is a Capture Now scenario.

Now, if you do an HD-SDI, HDMI or analog component capture through hardware, the in and out points may make it workable for recapture.

[Craig Parkes]

Originally Posted by John Wallace

Now I'm curious though, why is it better to use a ProRes timeline? Does the 4:2:2 colour space make a difference, even when the best source I have is 4:2:0?

It doesn't if you aren't doing baseline keys, or titles, or graphics in your timeline. But if you are - and these days, most people are, it does make a difference. It also makes a difference when you send it to the grade, because either you will need to full re render out to prores orsuffer from additional compression artifacting from HDV/XDCAM on any graphics/transitions/effects done on the timeline. If you are rendering while you are working, and the renders are in pro res, you will have less rendering to do at the end of the edit to spit something out you can be happy to send to a grade (unless of course you are actually going down to tape or XDCAM-HD as a master, in which case your hardware kind of takes care of that.)

Also, why should (does) the 'Preview' quality of raw HDV on the ProRes timeline look exactly the same as the 'Full' quality HDV on the XDCAM timeline?

Not sure about why they look identical - as a policy I like everything to be rendered fully before it goes to tape, or if you are going to Quicktime it has to be rendered anyway, so I don't really trust the quality of any 'preview' or 'full' footage. My guess is that one is doing a codec transform, (mpeg to prores) where both HDV and XDCAM are mpeg based so it has a way of fitting the lower bitrate HDV stream more nicely to the XDCAM stream - but that's a pretty wild guess.

My gut instinct is that, yes, it's surely always better to be dealing with only one file format on a single timeline.

However, your old-school 'nail it down the send it to the grade' method of working with Color makes sense and is kinda what I thought might be the best route for me, time and storage space wise. I can work with 3-way to nudge the colours to an acceptable state until I get to a final cut. I am not a colourist and would really want to get someone else to do this part, ditto the dub.

I'm still interested to know if it's possible to re-link a cut to ProRes re-conformed footage when it originally referenced HDV or XDCAM EX footage.

Anyone tried it? Or do I feel another experiment coming on?

I'm 90% sure you'd run into issues, in that as its only the render files in ProRes you would not be able to do a proper conform to Color. Maybe if you media managed the project first, but because the timeline/XML would still reference original footage and timecode then you would need some way to decompose and recompose only the render files... which sounds tricky.

It might be that Color does automatically handle this in some workflow (potentially on FCS3 - I haven't spent significant time with it as it wasn't a justifiable upgrade for my workplace) but my believe is that it will always look back towards source footage.

What I can say, is that the methodology I outlined above of working on a prores timeline then simply rendering a full quicktime and exporting an edl for the grade in Color (cutting HDV on a prores timeline, mixed with DV, still images, archive footage from BetaSP, DVD, Go Pro HD and all manner of other sources) worked well reliably for me on a 13 ep series that is screening on Discovery Asia and Discovery UK.

Not having handles caused some headaches in the odd instance where there were post grade fixes that had to be done, but the sheer disparity of original source footage meant that relinking in a traditional way right back to source simply wasn't an option.

[Simon Denny]

Cutting on a ProRes timeline is the best solution if grading in Apple Color. Or render out all footage to ProRes. I'm finding lately that I'm rendering out my XDCAM to ProRes on smaller edits which have more than three tracks with lots of graphics flying around.

[John Wallace]

All good stuff guys - how many more timezones can we add to this thread I wonder?!

And all much appreciated. I've been working between HDV and EX for about a year making shorties and tasters, but this edit is way fatter than anything I've attempted so far so you'll understand why I'm niggling to get things right and identify potential problems before I set out.

Thanks Shaun - you just provided a second 'oh yeah, forgot that' moment since I started this thread. I've proresed HDV before from my sony deck. Bit of a faff as my setup overruns its buffer after about twenty minutes. Another reason for me not to capture to ProRes from tapes.

Meantime, y'all have persuaded me my timeline really does need to be ProRes.

Originally Posted by Craig Parkes

Not having handles caused some headaches in the odd instance where there were post grade fixes that had to be done, but the sheer disparity of original source footage meant that relinking in a traditional way right back to source simply wasn't an option.

...so in effect you had to do what instead? Manually go back and dig out the source tapes/discs/files, recapture what's needed on the fly and transcode to the appropriate format before dropping it in to your nearly finished sequence and tweaking it until fixed?

'Craig's Way' still seems like the only sensible option for me, so I'm heartened to hear it's worked so well.

Why?

Because I just got the stopwatch out, did some transcodes and a few sums.

If I ProRes everything (from files, not tape or card), I'll need

- around 8TB of storage for Prores HQ
- 6TB for LT
- about nine and a half days processor time

Then there's the nightmare of batching up several squillion XDCAM clips.

None of which appeals.

Gary's solution of transcoding all my HDV to XDCAM before I get started would be much. much lighter on storage (around 700GB for the XDCAM converted HDV), but would still take around five and a half days of processor time to complete, and I'd still be stuck with multiple renders in edit on my effects-friendly ProRes timeline.

Happy New Year to those of you tuning in 'Down Under'

Lang May Yer Lum Reek!

JW

[Gary Nattrass]

All the very best John and it just goes to show how total workflow from shooting through to delivery has to be considered these days. You do right to make these decisions now as it will cut down future headaches.

I set up my company to solve these type of problems mainly for broadcasters but I am still amazed at how little thought tends to go into post workflow with the crews at the front end dictating a lot of what is done.

Good luck with the project and give me a shout if you need any extra resources.

[John Wallace]

Originally Posted by Gary Nattrass

still amazed at how little thought tends to go into post workflow with the crews at the front end dictating a lot of what is done.

Hahaha - I WAS THAT CREW! In this case at least. The project is a personal labour of love conducted over seven years with whatever equipment I've been able to rustle up at the time.

But you're right, of course. With modern cameras capable of churning out such a variety of formats, and especially now with tapeless workflows making operators responsible for those crucial first stages of data wrangling, there's never been a greater need for shooters to have a solid appreciation of what's likely to happen in post and beyond to what the final delivery platform will be.

Thanks for your encouragement and wisdom.

Here's wishing you and yours a prosperous New Year.

Cheers.

JW