Chris Hurd indicated that it might not be that fruitful to speculate as to the causes of the XL1 image-quality problems. Nonetheless, my brain is not willing to turn off while Canon figures this out. My thinking over the XL1 problems of banding, posterization, and red-ghosting is that these are due, at least in part, to the DV compression scheme. While not a compression expert, I do know something about the subject, including information that was posted in this newsgroup. Please feel free to correct my technical errors as I set forth my reasoning. I'll discuss banding first.

DV compression, a type of JPEG or MJPEG, is based on the notion that the human eye is more sensitive to gradual transitions than to dramatic transitions. Thus, in an image with both gradual transitions and dramatic transitions, more compression will be applied to the regions with the dramatic transitions. Thus, when the image is reconstructed, the gradual transitions will be reproduced relatively faithfully, while the dramatic transitions will be reproduced less faithfully (but we will not be able to tell, in theory).

Even in the uncompressed digital data, the naturally continuous range of colors is quantized; gradients are represented in discrete steps. However, humans cannot generally perceive the steps if the representation is in 16-bit precision. If the compression is mild, the steps are either not perceptible or are barely perceptible. However, if the compression is severe, banding of continuous tone images will be apparent. A familiar example of the results of severe compression can be obtained by viewing a continuous tone image with a monitor in 8-bit (256 color) mode.

A note in passing: 8 bits are sufficient quantization for any component. Consider a black-and-white movie being played from tape through a TBC. You're looking at 8-bit quantization, and there are never any visible quantization artifacts.

DV compression is "fixed-rate". An alternative would be "fixed quality". In a fixed-quality compression scheme, images with lots of dramatic transitions would be compressed more than images with mostly gradual transitions. However, in a fixed-rate compression scheme such as DV, the compression is the same regardless of the nature of the image. Therefore, in an image with mostly gradual transitions, the compression artifacts will be more perceptible than they will be in an image with more dramatic transitions. If one were to compare an image with a gradient region surrounded by a lot of detail with an image with a gradient surrounded only by more gradient, there might be banding apparent in the latter image where they might be no apparent banding in the former.

The quantization of the DCT coefficients is only the half of it. After that, Huffman encoding and run-length encoding reduce redundancy in the transformed image. For a fixed-rate scheme, an image with lots of frequency-domain redundancy (gradients) will require *less* entropy reduction (quantizing), not more (because you're already getting lots of compression from the redundancy reduction).

In addition, we're talking about streams with very little motion, so the inter-frame compression will be very high, greatly reducing the need for very much entropy reduction at all! A steady shot of a gradient (or mostly-gradient) requires a *very* mild quantizing table, perhaps a little heavier if there are some details in the image.

If one looks at some of the images bearing digital artifacts, such as those posted at www.mediadesign.net, one can note that these images are dominated by gradual transitions. Therefore, given the fixed-rate compression of DV one would expect some perceptible DV artifacts.

The artifacts one would expect to see would be blockiness around details when there is motion in the image. The details and the motion rob the huffman encoding, runlength encoding, and interframe of their ability to compress losslessly, forcing a nastier quantization table.

The banding is possible, but only if the designers are careless with how they program the quantizing tables.

What DV artifacts can one expect to see? This depends on the compression algorithm or the codec. Banding is the easiest to anticipate. I know that JPEG is block based. I do not know to what extent DV compression also uses inter-block compression. However, block-based and inter-block compression might help explain the displacement of image elements or colors seen in posterization and red-ghosting. In the case of red-ghosting, I have seen the artifact abate when a detailed object enters the scene; this would seem support the hypothesis that the compression is the source of the artifact.

If these are DV artifacts, why do they just show up on the Canon XL1. Well, I can find red-ghosting on my VX-1000. I have also seen banding on my VX-1000. In the case of banding, the banding is less noticeable because of its inherent noise. Even in a still image, the noise has the effect of breaking up the band boundaries-implementing a spatial dithering that makes the bands less pronounced. In a moving image, the noise results in a time-averaging that also makes the bands harder to perceive. The bands are more apparent in the XL1 because of its overall quality and low-noise characteristics. It's a feature, not a bug.

No, it's a bug. Low noise should allow redundancy reduction to work better, reducing the need for entropy reduction. There is never any reason to touch the low-frequency elements of the quantizing table. Much less so for a less noisy image.

So if compression is the problem, what is the solution? Within the DV standard, I assume there is a choice of compression algorithms. It is possible that a more optimal compression algorithm can be found that is compatible with the DV format.

It's possible not to screw up the implementation. That would be the most straightforward solution.

If compression is the problem, the best solution is to use less compression. Thus, it would be interesting to compare DVCPro-50 and Digital-S images for DV. Note that DVCPro-50 equipment is switchable between DV compression and a milder compression. Thus, such equipment would be ideal for examining artifacts as a function of compression.

There's always that danged implementation variable. You never know whether you're seeing differences caused by the standard or the implementation. There's plenty of leeway for implementation in DV.

To get rid of banding in an XL1 video, one could add noise. The noise could be added during editing. I am not sure anyone would want to add noise before the image is captured, but Canon could add a variable noise source to the CCD output.

That's called dithering. Based on many years of experience with uncompressed digital TV (one can scarcely find a TV signal which has not at one time or other been digitized at 8 bits per component - certainly every taped program has, due to the TBC) dithering doesn't seem to be required when you have video at 8 bits per component. In a DV situation, there should never be a neccessity to quantize the low-frequency coefficients below 8 bits, thus no neccessity to dither.

I hope my previous conclusion is wrong. I hope the fix, as someone suggested, is just using a better voltage regulator. If the problem is the compression, then I hope it can be tweaked. It will be interesting to see what Canon comes up with.

I love this kind of speculation, but Sherlock Holmes would be quite rightly appalled by all this. We're speculating without facts, and that's pitall not to be trifled with. The problem is, you become fond of your speculation, and when the facts emerge you might be reluctant to deduce the truth.

Your post deserves a more thorough response than this. But as to your note in passing. In the parlance of computer monitors, "8-bit color" is 256 colors, not 8 bits per component, which would probably be 24-bit color or about 16 million colors. 256 colors cannot faithfully render a continuous tone image. Compare the ability of 256 colors and 65536 colors to depict a continuous tone image. Banding will be much more pronounced when the setting is 256 colors.

We're in agreement. 8 bits is too few for more than one component.

If you render a continuous-tone black-and-white image on the 8-bit computer display, it will not show banding (assuming the palette is mapped to grey scale!). Color on the 8-bit display - yuck, because they have to divide the 8 bits up between three components.Note that applications introduce all kinds of dithering options for when the display is set to 8-bit color. As you mentioned before, this is a means to randomize the quanitzing noise, making the coarse quantization less objectionable, hiding the banding.

I appreciate your considered reply. You clearly have a good grasp of DV compression. It seems you agree that the codec could be at fault, and offer some optimism that Canon could improve the codec to achieve better image quality.

If I understand you correctly, you indicated that Sherlock Homes would be appalled by my "speculating without facts." I never met Sherlock Holmes, but I disagree that my "speculation" is not based on facts. I examined footage from two different DV cameras shooting the same event. For each camera, I compared images of different types. During tests I examined the effects of lighting on the artifacts. In determining the source of the problem, I used all sorts of equipment. I used three different DV cameras for playback. I used both analog and digital paths for playback. I have actively sought and used data gathered by other participants of this newsgroup. I have a lot of data. I realize that I am not the world's leading expert in these matters. I would be perfectly happy to be wrong. I do not care whether the fix is a different codec, a better voltage regulator, or improved time-base correction. I do not maintain that I am right, only that I can articulate an interesting position that no one else has addressed in detail in this newsgroup. Right, maybe not, based on facts, yes, to the extent they are available to me.

In response to my citation of the limitations of 8-bit (256) color mode, you replied that 8-bits per component are sufficient quantization. I think you agree with what I wrote, but not with what you read. I wrote "8-bit (256) color", you apparent read "8-bits per component". 8-bits per component is 24-bit (16 million-) color mode. I agree that for most purpose, 24-bit color adequate to represent continuous tone images. In fact, 16-bit (16,656-) color is usually adequate. 8-bit (total for three color dimensions) is not satisfactory. Banding is generally visible on gradients.

I understand the role of Huffman encoding and run-length encoding in DV compression. Maybe not as well as you do. I am not sure why a gradient image has more redundancy that a more detailed image. As long as the gradient levels are represented by different values, the Huffman encoding and run-length encoding should not result in dramatic compression. Right?

Redundancy is in the eye of the algorithm, but it can be shown that very primative prediction algorithms can find tremendous redundancy in smooth gradients, whether in the spacial domain or the frequency domain. For DV, we're in the frequency domain.

Smooth gradients are dominated by low frequencies, therefore the spectrum of the image will be bunched up in the upper-right-hand corner of the DCT matrix. The histogram of transform coefficients will be very steep - variable-length coding *loves* steep histograms. In addition, there will be large expanses of zeros in the higher-frequency portions of the matrix - run length coding *loves* vast expanses of zeros.

Contrast that with a detailed image where coefficients are somewhat more evenly distributed throughout the matrix, and there are more high-frequency components.

The fact that we're talking about 8x8 blocks of the image makes the gradient easier still. If you consider the published XL1 banding pictures such as the helicopter, the Coit tower, the bronze birds, and the paper title scotch-taped to the wall, and look at an 8x8 block within the gradient area, the pixel values only vary by 2 or 3. That means an overwhelming majority of the block will be contained in the DC coefficient!

You can satisfy yourself as to the compressability of gradients versus details, and get a feel for the magnitude of the relationship, using any JPEG image processing program and some sample images. For a given quality setting, the smooth gradient will produce a much smaller output file than an image with lots of details. The intra-frame compression we're talking about is very much the same for JPEG and MPEG.

You suggest that gradient images have very little motion.

Oh, I'm sorry, I meant to suggest that our discussion was limited to motionless images. I thought we were talking about all the various published images showing the XL1 banding, many of which were locked-down shots of stationary objects. I wasn't aware that you had also considered moving gradients.

What have you noticed in the moving gradients? Does the banding seem to change when there is motion?

If the camera is zooming and panning there is motion. I am not convinced that any interframe compression of gradient images is more effective that interframe compression of images with more dramatic transitions. If, as you suggest, interframe compression is a factor, then compression artifacts should appear more in scenes with more movement and less in scenes with less movement. I will try to check for this next time I review my footage. Also, the posts are all still images. Maybe those having the source tapes could indicate whether or not the scenes were basically stationary. I am presuming your statement that DV uses interframe compression is correct. I know MPEG uses interframe compression; I am not knowledgeable about whether or to what extent DV uses interframe compression.

I'm not too clear on it, either. I know the DV implementations I've looked at are *capable* of using inter-frame compression, and do so when there is a significant amount of motion. Whether they switch to I-frame only when there is little motion, and whether they rely on intra-frame more heavily than the inter-frame, I don't know.

You indicate that the artifacts we would see due to excessive compression would be blockiness around details. I do not think that is what we are seeing. So maybe it is not the compression.

Perhaps, and perhaps we're seeing poorly-implemented compression.

Your point about the greater compressibility of clean images is well taken. Still, bands are less prominent when noise is present. My observation is that the VX-1000 also suffers from banding, but it is less visible due, in part, to dithering.

It's entirely possible. But I gotta keep asking myself, why would anyone coarsely quantize the very low frequencies in the first place?

You indicate that it should never be necessary to quantize below 8-bits per component in DV. I suppose you are still open to the fact that Canon's codec is quantizing below that level.

(Below 8-bits for the *low-frequency* components) Absolutely! My theory is that Canon is screwing up, loading the quantizing table with inappropriate values for the low frequencies.

Let me reiterate that I am not convinced that it is the codec that is the problem. I set forth a position in the hope that it would foster a productive dialog. We have some facts, but much relevant information is not available.

I'm still voting for the voltage regulator.

When you say "codec" for a DV camcorder, you're referring to a system which is distributed between hardware and software.

I vote for a software bug.

Closing remarks, fellows? Clif? Charlie?

I want to thank Charlie Hand again for his well-informed contributions to the possible role of Canon's DV codec in producing the DV artifacts. I may need to revise my original post on this issue.

My current opinion is that the red ghosting that plagues some of my videos is neither primarily a codec issue nor a Canon XL1 issue. The artifact also shows up on videos I took with a Sony VX-1000 and a Canon ES2000. My current hypothesis is that the artifact is optical, comparable to vertical smear. This difference is that the displacement is horizontal and only seems to apply to red.

Charlie asked whether I had observed the effect of motion on the artifacts. I know I have footage of red ghosting in motion, but now that I no longer entertain red ghosting as a DV artifact, that footage is irrelevant. I have some real footage with banding, I will check for motion (if I can find the footage again).

I still think the banding is most likely a codec problem. Do you agree that the posterization is visually distinct from the banding? It seems to me that the banding is the result of too coarse quantization. The posterization appears to involve displacement of image elements. How would the DV codec produce such displacements? Are the displacements on the order of a block? I would like to know more about inter-frame compression in DV. Does anyone know a web publication on DV compression? It seems I came across one once. It does not seem that it could be used much. After all, DVD achieves huge, I think 200:1, compression ratios using interframe compression. Also, in photo mode, if interframe compression were used properly, stills would need no intraframe compression. Are the stills uncompressed?

As Charlie pointed out, there are many parts to the DV codec. I do not know whether the DCT coefficients are software, firmware, or hardware (seems unlikely), but it still seems that the DCT coefficients are the culprit for the banding.

I'd like to have a final word about Sherlock Holmes. I think for troubleshooting, whether it's troubleshooting of an electronic circuit, or a software program, or a refrigerator, there can be no greater mentor than Sir Arthur Conan Doyle's famous detective. In the stories, Doyle repeatedly presents us with a collection of facts which seem at face value to be entirely comprehensive and cohesive. But when Watson articulates the obvious conclusion, Holmes chastizes him for theorizing without sufficient facts, much to Watson's and our chagrin. Invariably, it turns out there really were additional facts which when added to the sum of facts painted an entirely different picture.

Clif has looked a lot of XL1 footage, and I've only looked at a couple of stills. And yet no one we've heard from has examined the inner behavior of the camcorder with instruments while it is producing these artifacts. I think we must entertain the likelyhood that the real solution turns out to be something we never dreamed of!