EX1 XDCAM EX vs. ProRes footage - added noise during conversion?

[Ryan Mitchell]

Hi all --

Don't have time to post frame captures (have to get to bed - beat), but wanted to get this out to see if anyone had seen what I'm seeing here. If necessary I can post frames tomorrow to illustrate.

I'm taking some footage from my EX-1, shot in 1080p24, -3dB gain, outside in bright sun. I walk in front of the camera with a color chart, wearing a dark gray shirt - I'm in the shadow where I'm standing.

When I view the footage rewrapped from Sony's XDCAM Transfer app, it's in the XDCAM EX codec, and it looks fine. However, after re-encoding to ProRes 422 HQ - it adds noise to the image - quite a bit of noise. It's noticeable when the movie it playing more than in a single frame, but you can tell the difference especially if you look in the shadows. It was obvious enough for me to stop what I was doing and examine the issue. If I use the non-HQ ProRes 422 codec as the compressor, the shadowed areas on my shirt are REALLY blocky and obvious even in a still frame - almost like a JPEG compression type thing.

Now, I'm not a codec expert or anything, but I thought that converting the footage to the ProRes codec wouldn't actually make the footage look WORSE, given that the data rate is higher than XDCAM EX as well as having more color information (4:2:2 vs. 4:2:0). However, what I'm seeing confuses me quite a bit. I'd think I should be seeing footage that looks so close to the same that I'd REALLY have to look to see a difference.

Now, if I output to uncompressed 8-bit, which I did just as a test, it comes out fine (and about 5x the file size). This leads me to believe that re-encoding to ProRes 422 HQ is more destructive than I had thought and could take quality out of the original long-GOP XDCAM EX-based footage...

For those who will ask: I'm re-encoding using Compressor, one of the several built-in ProRes settings, and I've fiddled with a variety of the options in the "Encoder" tab and the "Frame Controls" tab, including making sure that I'm not doing anything at all to the footage. In the end, I'm not resizing, retiming, deinterlacing, or anything else to the footage short of re-encoding it. I'm just amazed that I'm seeing output in a "more realtime" codec that actually looks worse...

Anyone else seen anything like this?

[Alister Chapman]

I have noticed that prores can be noisy. When I was looking at differences between recording in camera using xdcam and recording both prores and uncompressed via hdsdi I found that the prores version had more noise than I was expecting. So much so that in my chroma key tests I could get a cleaner key using the ex compressed footage than the prores footage. Of course the uncompressed gave the cleanest key.

[Michael Frkovich]

Originally Posted by Alister Chapman

I have noticed that prores can be noisy. When I was looking at differences between recording in camera using xdcam and recording both prores and uncompressed via hdsdi I found that the prores version had more noise than I was expecting. So much so that in my chroma key tests I could get a cleaner key using the ex compressed footage than the prores footage. Of course the uncompressed gave the cleanest key.

Alister, are you talking about footage already recorded in camera and then taken in via sdi
through a capture card like a kona? Or are you talking about pulling in footage direct from
the camera to prorez?

[Ryan Mitchell]

Originally Posted by Alister Chapman

I have noticed that prores can be noisy. When I was looking at differences between recording in camera using xdcam and recording both prores and uncompressed via hdsdi I found that the prores version had more noise than I was expecting. So much so that in my chroma key tests I could get a cleaner key using the ex compressed footage than the prores footage. Of course the uncompressed gave the cleanest key.

Well, I guess it's good that I'm not doing something entirely wrong and that someone else is seeing the same thing, although I'd rather be wrong and corrected, with a perfect ProRes solution. I wonder if the new ProRes 4444 codec could be of any help here?

I guess the question I'd have for you, then, is what codec you ended up using for editing? In your opinion is ProRes not a viable option? XDCAM isn't my first choice for editing, especially since at some point you end up having to go to something like ProRes as an intermediate format for Color or After Effects, and using XDCAM might involve another recompression step - not good.

[Paul Frederick]

To see how much noise is being added try this test. In After Effects put the original XDCAM clip on a layer. Then add the Prores clip of the same file to another layer directly above it. Set that layers transfer mode to "Difference", only the pixels that are "changed" from the conversion will show up. Try this with different codecs!

Try stacking the same XDCAM clip and set the transfer mode of the one on top to "Difference", all you'll see is black.

You must make sure the clips are frame matched. If they are off even a frame, there will be a difference from pixel to pixel. But if you can frame match them (which is easy if you re-encode the exact clips without changing length), then only pixel to pixel differences will show, thus showing you just how much noise is added from the Prores conversion. This eliminates any noise in the original confusing things, because only the "difference" between the two will show. Hope this makes sense, it's hard to explain! But quick and easy to do.

[Michael Maier]

Originally Posted by Alister Chapman

I have noticed that prores can be noisy. When I was looking at differences between recording in camera using xdcam and recording both prores and uncompressed via hdsdi I found that the prores version had more noise than I was expecting. So much so that in my chroma key tests I could get a cleaner key using the ex compressed footage than the prores footage. Of course the uncompressed gave the cleanest key.

Alister, are you talking about footage captured straight to prores from the SDI or XDCAM converted to prores? I have never had issues with prores being noisy while captured from the SDI and your remark about XDCAM producing cleaner keys over SDI to prores is what makes me think you were talking about XDCAM converted to prores rather than SDI to prores.

[Kent Beeson]

I'm probably missing something but is what is being said is that it's better to edit on an Uncompressed 8-bit timeline in FCP than Pro Res HQ?

[Mark OConnell]

Ryan, have you tried saving from from QT Pro as a ProRes file, or just using Compressor? The problem might be with Compressor.

[Alister Chapman]

This is an old thread and I haven't re visited this since last year, but at the time I was finding both converted and SDI captured prores material to be noisier than native EX footage. Now I have a NanoFlash I tend to use that at 100Mb/s and it's very clean.

[Daniel Doherty]

The ProRes HQ footage from my Ki Pro is beautiful. There definitely isn't any added noise from that device.

[Alister Chapman]

Originally Posted by Daniel Doherty

The ProRes HQ footage from my Ki Pro is beautiful. There definitely isn't any added noise from that device.

Compared to what?

[Tim Kolb]

I think that it's 8 bit vs 10 bit that may be the visual difference here... It's been a tactic of high end post guys for some years to add noise to 8 bit footage to make it seem a bit deeper.

8 bit needs to do a fair bit of 'rounding' and what many of us see as being 'clean' footage is actually footage of objects that have more texture and variation than we have color precision to faithfully capture with 8 bits of color depth. Compression definitely accentuates

Compressed 10 bit footage can definitely seem more 'noisy' than 8... I've done some recording comparisons between 8 and 10 bit external devices fed via SDI...and I've seen what I believe you're seeing.

10 bit is still preferable for any compositing work...those minor tonal variations are a huge help in selling an effect in cases where using 8 bit footage would cause the grayscale to collapse and posterize...

My .02 anyway...

[Tom Roper]

Originally Posted by Ryan Mitchell

Now, if I output to uncompressed 8-bit, which I did just as a test, it comes out fine (and about 5x the file size). This leads me to believe that re-encoding to ProRes 422 HQ is more destructive than I had thought and could take quality out of the original long-GOP XDCAM EX-based footage...

Ryan is going from XDCAM to ProRes 422 HQ, and then taking the same XDCAM and going to uncompressed 8-bit, which looks better to him.

If I understand what Tim is saying, is that Ryan is being confused by what he sees because the clean 8 bit uncompressed is actually footage of objects that have more texture and variation than there is color precision to faithfully capture with 8 bits of color depth, accentuated by the extra compression step added by ProRes, yet others are saying they see the noise going from 10 bit SDI to ProRes as well. That's a little perplexing.

Paul Frederick mentioned a clear way to prove his theory...

Originally Posted by Paul Frederick

To see how much noise is being added try this test. In After Effects put the original XDCAM clip on a layer. Then add the Prores clip of the same file to another layer directly above it. Set that layers transfer mode to "Difference", only the pixels that are "changed" from the conversion will show up. Try this with different codecs!

Try stacking the same XDCAM clip and set the transfer mode of the one on top to "Difference", all you'll see is black.

[Daniel Doherty]

Originally Posted by Alister Chapman

Compared to what?

Compared to XDCAM EX recorded onto SxS.

[Alister Chapman]

One thing to consider is that the footage from an EX1 is border line with regard to getting any advantage from 10 bit recording as the noise level of the camera is greater than the 10 bit quantization level, so there is little to nothing to be gained by using 10 bit over 8 bit for capture with an EX1 (or any other camera with a 54db or worse noise figure).