Edge Enhancement Correction in Post (de-EE)

[Will Thompson]

I've been shooting for a documentary with an HD1 for the past few months and intend on taking it on another month long trip in the near fututre. I have been able to avoid some EE problems with this camera by simply avoiding high-contrast, high-motion shots; however, this is not always possible, and is usually never desirable. The reason we are shooting with an GR-HD1 instead of the JY-HD10 is a long story, but regardless...

Has anyone successfully treated ugly-looking edge enhancement in post (HD1 or not)? We intend to color-correct using a combination of After Effects (for problematic shots needing more color accuracy) and AspectHD (real-time color correction for the rest). And we have already purchased our copy of AspectHD (FYI, David Newman).

The possibilities to thwart EE include things such as motion blurs, gaussian blurs, soft focus filters, diffusion filters, etc., all provided by software. There are numerous options for advanced tweaking and gimmicky filtering in post, but the crux of the problem lies in removing the hard black, jaggy lines created by EE. So a better question before trying to remove EE might be: how does EE even work? It is presumably a DSP implementation on-chip that behaves similarly to how Photoshop and similar other filters might enhance edges. The EE (a.k.a. "Sharpen Edges") algorithm typically finds some (heuristic) way of masking areas where neighboring pixels have an above average contrast ratio (edge detection) and then sharpening the masked area, which is usually just a clever adjustment to contrast, resulting in an "edge-enhanced" image.

My question is not whether or not EE can be removed completely. It clearly cannot. EE is not a mathematically reversible function - it is simply a heuristic algorithm. My question is rather what can be provided in post that has the capablility of diminishing EE, but also best preserves the oh so very important detail that makes HD worthwhile. It is even suggestable that a procedure intended to diminish EE may produce some positive side-effects, such as richer colors, less chroma-noise, among others. It is my intention to find out the best balance of the positive and negative effects of filters and the problematic EE.

If I may continue to speculate, HDV samples in the 4:2:0 color space, and therefore contains (just like "real" HD cameras and HD satellite transmissions) much more luminance data than chromanance data. This type of sub-sampling to lower data rates we know is effective because more detail is percieved visually from luminance information than from an equal amount of chromanance information. My question then is where does the EE occur here? Presumably, it occurs after the 4:2:0 sample is taken, but before it is compressed to the MPEG2 Transfer Stream. So is EE simple, such that the same EE function is applied to all sub-samples? Or is it opportunistic, affecting one more than the other? Or maybe it is a lumenance-only function. Knowing this could be helpful in the selection of a more appropriate de-EE filter.

So if we take the more pessimistic assumption that the EE implementation in the HD1 is like the former, more nieve case, options for post are far less interesting. In this case, most of the perceived EE would be from the lumenance sample anyway, so any function to soften EE would equally distort all lumenance information, resulting in a stodgy, less-than-HD image. One step worse case would be the "lumenance-only EE" suggested earlier.

But maybe due to cost or other considerations, EE is not as nievely implemented as the aforementioned paragraph suggests. If this is the case, we might be able to make more adjustment to the chroma data with filtering tools and still reduce EE without the adverse affects of heavily blurring the lumenance data. While this would be helpful to me, it would be presumptious to assume there exists a less intuitive implementation of EE so it could be removed more elegantly in post-production - especially in the consumer model of the camcorder.

So, your thoughts and comments regarding this topic would be appreciated, as I prepare to move from production to post by the end of the summer.

Has anyone had experience using anti-aliasing tools, "film-look" filters, etc. in post? Because DV usually samples (I think) in 4:1:1, any similar EE or sharpening problems in DV would likely be similar in HDV. The main difference in this case, I believe, is the fact that we are dealing with MPEG-TS data now, a more lossy (and very quirky) compression format; however, this should just add to the current problem (already somewhat present in editing DV) of revealing compression artifacts with post color manipulation . By working with the CFHD codec (provided by CineForm), I hope this issue will be less severe than it would be by working directly with the TS files. And I'm even willing to go up to uncompressed 4:4:4 AVIs for some of the bad scenes if it can help.

Maybe "Soften Edges" will fix everything. Anyone?

Thanks,

Will Thompson

[David Newman]

I believe EE is performed on luma only and is very similar computationally to an unsharpen mask. Minor sharpening maybe reversible, but the problem in when the EE clips edges to black. Any data that was within those black pixels are now lost. I can't think of any simple filter to fix these edges.

[Will Thompson]

David,

Thanks for replying so quickly. I am aware that when color values are changed to black, the original color values cannot be retrieved. So that, I agree, is mostly a lost cause.

However, there appear to be more pathological cases for the HD1's EE, specifically anything high contrast that moves. Not only are hard black lines present, but the EE does not bother to anti-alias, so they are jaggy. These present some scenes that will make a videographer cringe to watch. For example, trees against a sky, a simple brick building, any reflection highlights on smooth surfaces, etc., all have EE that I could tolerate were it not for the jaggies. Because with the jaggies present, the motion video exhibits a "crawling" effect that I consider the most unfortunate side-effect of EE.

I don't know why anyone would ever implement such extreme EE even in a consumer model, but regardless, do you know of a way to anti-alias video to improve at least the cases I mentioned?

Thanks.

[Chris Gordon]

On high contrast outdoor shots, I've been using a soft contrast filter on my HD-1 that basically keeps the camera from adding the edge enhancement to the image. When I edit the video, I then adjust the contrast level back to normal since the soft-con filter makes blacks look gray. Viola! No edge enhancement.

[Will Thompson]

I never thought that could work, but I'll try anything - Thanks for the post. I'll put up pictures soon after running tests. If there are other worthwhile suggestions, I'll add them to the comparison as well. Has anyone else tried this?