AVI decoding quality

[K. Tessman]

I'm revisiting something I was looking into quite a while ago, and sort of gave up on temporarily. Now that I'm doing final-quality encoding for DVD, however, it's reared its head again.

(Note: I know this isn't a Cineform problem per se; I'm just hoping that someone in this forum has experience with what I'm seeing.)

Basically the thing is this. I've attached a couple of images, the same frame output from the same .avi file in both Virtual Dub and Media Player Classic.

The VDub image is what it's supposed to look like. That is, that's what it looks like when I sign off on the color-correction in After Effects, or play it back in Premiere. The MPC image is noticeably lower contrast and even lower-detailed. I've set MPC to (I think) use no overlay, so I'm under the impression the two should be identical.

Problem is, the MPC (or WMP, for that matter) image is much more reflective of what I get when I feed the CFHD .avi file into ProCoder to encode to MPEG-2 for DVD. I end up with a DVD that doesn't have the richness/contrast or detail I expect. So I'm arriving at the conclusion that whatever pipeline is being used to get the CFHD into ProCoder is much more the MPC/WMP/DirectShow(?) way as opposed to VDub's (more accurate) way.

Now, I've tried methodically with various playback settings for both CFHD decoding, the applications themselves, various filters, and the graphics card (an NVidia 8600), but still don't know what's going on.

Has anyone else seen this? Or understand what might be happening?

Thanks.

[David Newman]

Run a file through MPC or WMP and leave it open, then open an new MPC window and play a file there, this new window will not get an overlay and therefore be correct. This is a NVidia bug that they haven't fixed in years. Their YUV to RGB conversion math is wrong. ATI and Matrox typically don't have this issue. Some NVidia cards can be manually calibrated, most don't support adjustments needed for the overlay.

[K. Tessman]

The Nvidia bug was one of the first things I thought of, but I wasn't able to isolate it to overlay behaviour, as per the attached. These are two instances of MPC running (with the same settings as above). They're identical to the solo MPC frame from the original post. Just for the record, I've seen this same sort of thing for a long time, since previously when I was using an ATI Radeon.

Regardless -- and I don't know much about exactly where and how the card operates in the read-render-write path -- does it affect more than just display? That is, should the graphics card have an effect when exporting a frame from either application, MPC or VDub? Does the frame actually get routed through the graphics hardware somewhere during reading, decoding, and output to whatever internal buffer is used to write the JPEG (or, more critically, the MPEG-2)? (I'm sure you know much more about this than I do.)

I've spent a couple hours trying to figure out if it's possible to feed Procoder via VirtualDub's frameserver to see if that would overcome this issue, but it doesn't seem like that's something Procoder is readily set up to do.

[K. Tessman]

Okay, so a bit more investigation. Despite using what I thought I was assured by reading was a non-overlay mode in MPC, it's still affected by the Nvidia control panel's overlay settings. At least I think they're overlay settings. They don't call them overlay settings anymore apparently; they call them "video color" settings.

As you can see in the first attachment, the only change I made was to enable the settings and slide the hue from 50% to 0%.

You can see the results in the second attachment -- which might be a bit of a jumble, but basically there's a WMP window in the background, two small MPC windows, and a Procoder window on the bottom left, all of which are tinted heavily blue.

The VirtualDub window is the only one not affected.

Can they all be using overlay? I didn't think it worked that way.

Or is the Nvidia "video" panel not specifically changing overlay, but something else?

I've tried tweaking the Nvidia control panel sliders to get the affected image to resemble the unaffected one, but it's approximate and inadequate.

So at the moment I'm a little stumped as to how to get my lovely CFHD master encoded with Procoder to MPEG-2 looking the way it's supposed to.

[David Newman]

The NVidia card is not used in the conversion, your issue are in preview, any conversion issues is something else. If you convert to MPEG2, then load that into VirtualDub, which does the image look like? Most CineForm images and YUV encoded, all MPEG2 images are YUV encoded, yet your PC only displays RGB, so the error you are seeing is in the YUV to RGB remapping. In VirtualDub the CineForm codec is asked for RGB data, and the codec correctly delivers computer systems RGB, which is correct for displaying on the computer monitor. If the conversion in ProCoder used DirectShow, that is a good thing, as YUV exports can be used, however YUV has two common flavors 601 and 709. If ProCoder uses RGB, that has two flavors, computer systems RGB (common, used VirtualDub), studio RGB (more dynamic range, used in Vegas.) If ProCoder is actually encoding to an under-contrasty image, it might using studio-RGB when it is expecting cgRGB. If the colors are slightly wrong there is 601 and 709 issue. Your job to determine which it is, and change whatever setting in ProCoder to address it -- if ProCoder is in fact wrong -- again check your MPEG2 output in VirtualDub (there is a patched VirtualDub that supports MPEG2.)

[K. Tessman]

Thanks, David. I'll grab VirtualDub-MPEG2 and check the output in that. As well, I'll take another look at Procoder's input and output settings. I know it does have 601<-->709 conversion filters, if those are ultimately necessary.

I'm still puzzled, though, as to why frames saved from MPC exhibit the same color-shift as the (overlayed?) video, and what buffer/data it must be using to write the JPEG (leaving me to wonder, more importantly to my current goal, where Procoder is getting its buffer/data).