Noise, Sampling, Codecs and Green Screen: A Potential Horror Story

I’ve done a lot of testing with the Sony F55, and I’ve shot a couple of 4K projects in XAVC, but until recently I hadn’t used this camera for green screen. Sony’s implementation of XAVC uses 4:2:2 sampling, which can cause a lot of problems for green screen compositing, so when I got a call to shoot a green screen project with an F55 I decided I had to do some testing. Here’s what happened…

I have a love/hate relationship with the F55. It has the potential to be an amazing camera, and it certainly offers a lot of bang for the buck. The new LC-709 Type A look profile, which makes the image look a lot like what I’d get out of an Alexa, produces stunning images and gorgeous flesh tones, and Sony raw is pretty awesome—especially at high speeds. Unfortunately the menu system is one of the most complicated in the world, full of subtle landmines and booby traps, and basic functionality that I can find in Alexa–such as internal 4:4:4 recording to ProRes and the ability to white balance in log mode–are lacking.

Lack of white balance is particularly troubling for shoots under mixed light: normally I’d gel my tungsten lights green to match background fluorescents in office buildings as I can then white balance to remove the green cast, but the F55 doesn’t white balance in Cine-EI mode. It offers three presets–5500, 4300 and 3200–but as of software version 3 there’s no way to remove green overall. (This appears to be the same in v4 based on a recently released manual.) This means that my clients have to watch a green image unless I can wrangle an expensive additional box on set to manipulate the monitor feed in real time, which doesn’t happen as often as I’d like.

I’ve shot a lot of projects in XAVC 4:2:2 and not had any problems, but they’ve all been live action and most have been WYSIWYG. My only SLog2/SLog3 projects have been shot in 4K or raw. Codecs get away with quite a lot when compressing an image as our eyes are easily deceived when it comes to color perception and sharpness, but green screen is another story: suddenly it’s all about the edges and how clean they are, and all a codec’s faults come to light.

In particular, color subsampling (4:2:2, etc.) deliberately throws away color information to reduce data transmission rates and storage sizes. We don’t normally notice this destruction, but green screen doesn’t forgive.

The only way to record 4:4:4 footage internally in the F55 is to use Sony’s SStP HD codec (which is the current incarnation of Sony’s SR format.) The data rate is roughly double that of ProRes 4444 HD (at around 400mb/sec.) but the spec states that it records 12-bit color as opposed to Arri’s ProRes 4444 implementation, which is 10-bit. My task was to test XAVC 422 against SStP to see which one gave me cleaner edges, and then send the tests to post and see if they agreed with my assessment (they’re the ones who have to deal with it, so they get the final call).

I didn’t have a green screen handy so I used the next best thing: the green patch on a DSC Labs Chroma Du Monde chart. I spent a couple of hours at San Francisco’s Videofax playing around with an F55 trying to find the best way to stress the codec in a measurable way, and in the end I simply shot closeups of the green patch in various orientations in both XAVC and SStP. I sent short clips off to post and waited for their response.

SStP won by an astounding margin. Not only did post mandate that I shoot the green screen using SStP, they asked that I shoot ALL of our live action in SLog3 on SStP.

I pulled these stills in Resolve and cranked contrast up to 1.5 to better reveal edge artifacts and noise. First, XAVC 422 HD:

I shot the chart at an angle as 4:2:2 sampling primarily reduces horizontal resolution but there are very few objects in the average frame that are perfectly vertical. Horizontal resolution affects vertical lines, and vertical resolution affects horizontal lines. In a way this makes sense: if you’re a very tiny electronic person running horizontally along a scan line looking for changes in contrast, the ones that you’ll notice most are the ones that cross your path. Most of the things that cross a horizontal path tend to be vertical in some way.

As you can see (and if you can’t see then please click the image to enlarge it to HD size) the area of contrast between the patches and the black lines appear to be problematic. Just before this test I asked if anyone on the Cinematography Mailing List had any experience shooting green screen in XAVC 422 and a very helpful cinematographer sent me some footage. The edges looked as if they were dancing, and this still shows me why: the area around the black line looks as if fabric is tearing, due to the codec responding to the heightened contrast between the green patch and the abrupt black line. This is partially due to the codec trying to compress an area of high contrast, and partially due to sensor noise making that area appear to be in motion. As a result this “fabric tearing” effect will cause a kind of shimmer as the codec tries to deal both with this area of contrast and the overlaid noise.

What’s really interesting is that there’s almost no noise in the evenly colored areas. They show some blockiness when viewed in motion, but the noise level looks fairly manageable. Apparently the restricted bandwidth and high compression of XAVC acts as a sort of noise suppression filter by throwing out detail that it can’t comfortably store.

This is a bit different in the 4K version:

One option I had considered was shooting the green screen shots in XAVC 4K and asking post reduce them to HD size before compositing. My theory was that oversampling would make possibly jagged or noisy edges tolerable. While the edges are a bit cleaner than in HD I still don’t think they’re great. What’s really interesting is to see that noise in the gray part of the image is now quite clear. I suspect this is partially due to the fact that the noise is “native 4K” so the camera’s processor doesn’t have to average it out when scaling the image for HD.

Let’s take a look at the SStP footage:

The “fabric tearing” effect is vastly reduced. The noise isn’t as sharp as in the 4K version, probably because scaling 4K noise down to HD resolution mushes it together a bit. The black line bordering the green patches is sharper and shows much less “tearing” than the other codecs do.

Now I see why post was so emphatic about only shooting in SStP. Looking too closely at XAVC under these conditions is a bit scary.

I chatted with an LA-based DP about his experiences shooting XAVC 422 with an F55 for a TV series and he said he and his colorist hate it for anything other than WYSIWYG baked-in looks. The combination of noise and compression artifacts makes both of them nervous, and at ISO 1250 (the F55’s “native” ISO) he says the combination of compression and sensor noise makes the image very difficult to grade from a log image.

Sony raw and SStP, on the other hand, are a dream to grade.

I have to say that I became quite concerned by the amount of noise I saw in these images. Most of what I’ve shot so far with the F55 has been 4K SLog and raw exteriors and some HD WYSIWYG interiors, so looking at XAVC HD in SLog3 was a bit of a revelation. It’s not enough to say a camera should be rated at one ISO under most conditions; factors such as how the footage will be used in conjunction with the compression scheme come into play. Rating this camera at 1250 works well when recording to SStP and 4K XAVC as the increased bandwidth and better compression mean that noise will be recorded in more detail, which means less loss of detail overall because the codec is throwing less away. XAVC 422 probably works a lot better at lower ISOs where the lower noise floor results in a cleaner signal that’s easier to compress.

I wanted to show a motion version of these tests but I had to put the footage through so many conversion and compression stages that it just wasn’t fair to Sony. It was impossible to discern the artifacts baked into the footage from the artifacts introduced through my post process. I can, however, show you a comparison of all three images in one frame:

An acquaintance, visual effects DP David Stump, ASC has a simple but very accurate quote: “It didn’t used to be rocket science. Now it is.” Film was simple in comparison to what we have to deal with now. We may think we know how the image looks on set, but there’s so much going on underneath the surface that the more you know about how codecs work the less you’ll trust your monitor. Codecs are designed to take advantages of weaknesses in the human visual system, and most of the time their tricks succeed wonderfully. One of the few occasions where they don’t, however, is green screen.

It seems every camera manufacturer has a product that boasts an ISO of 800 or better but I’ve not found any that, under close scrutiny, hold up to that claim—at least not to my taste. Under many circumstances high ISO imagery can look great, but this depends on how the footage is compressed, processed, graded, transmitted and viewed. I’ve realized that the ISO that works for one project may not work for another. For example, if I’m capturing either raw or log to a high bit rate 4:4:4 intra-frame codec I can rate the camera faster as noise will have less of a destructive effect on the picture. Recording log to a more highly compressed 4:2:2 codec may result in visible artifacts as the contrast curve is stretched by the grading process, so by necessity I’ll rate the camera slower to reduce noise and help the codec out as much as I can or just bake in a look. Sensor noise doesn’t really go away with a change in ISO, it just gets pushed deeper into the shadows where our eyes lose track of it and codecs work most of their magic.

My next goal is to find some way to quickly and objectively rate sensor noise when assessing a new camera. In theory I should test every combination of recording method and ISO rating whenever a camera is publicly released, but I’m a working cinematographer and I don’t have that much time. Rather, I’m going to look at the noise floors of several cameras on waveform monitors and try to come up with a measurement threshold that makes me feel comfortable and confident. Noise shows up as a fat line on a luma waveform and the thickness of that line may be an excellent way for me to assess the inherent noisiness of a sensor. We shall see.

More to come.

Disclosure: I have worked as a paid consultant to DSC Labs, maker of the chart I used in this test.

About the Author

Director of photography Art Adams knew he wanted to look through cameras for a living at the age of 12. After ten years in Hollywood working on feature films, TV series, commercials, music videos, visual effects and docs he returned to his native San Francisco Bay Area, where he currently shoots commercials and high-end corporate marketing and branding projects.

When Art isn’t shooting he consults on product design and marketing for a number of motion picture equipment manufacturers. His clients have included Sony, Arri, Canon, Tiffen, Schneider Optics, PRG, Cineo Lighting, Element Labs, Sound Devices and DSC Labs.

His writing has appeared in HD Video Pro, American Cinematographer, Australian Cinematographer, Camera Operator Magazine and ProVideo Coalition. He is a current member of the International Cinematographers Guild, and a past active member of the SOC and SMPTE. His website is at www.fearlesslooks.com. Find him on Twitter: @artadams.