Deinterlacing -- in cameras, in software, and monitors

[Steve Mullen]

While the V1 does not use a deinterlacer IN the camera as do cameras that derive "no motion artifact video" from interlace scanned CCDs -- during capture to an intermediate codec or when playing a 1080i timeline to an HDTV and always when viewing on a flat-screen monitor -- deinterlacing is used.

The Realta HQV 1080i motion-adaptive deinterlacerchip is considered the "best" hardware deinterlacer. Most deinterlacer/scalers convert 1080i video to 1080p using vertical interpolation to scale each individual 1080i field directly to a 1080p frame. The vertical resolution of each displayed field is thus limited to the 540-line vertical resolution of each field. Vertical interpolation acts as a filter, which reduces the vertical resolution and softens the image.

The HQV uses pixel-based motion-adaptive deinterlacing, which applies "interpolation" to image areas that are in motion, but merges (weaves) information from odd and even fields in static image areas. Merging field information slightly reduces 1080 line vertical resolution in static image areas.

With HQV there isn’t a dramatic change of image sharpness when an object is moving -- an annoying artifact, called resolution pumping or breathing, which occurs when the entire image resolution decreases with movement. Or, when moving "regions" lose vertical resolution. One way to reduce this artifact is to slightly further decrease resolution on still images so that breathing is not as noticeable.

The Realta algorithms provide higher resolution in the moving image areas, so there is no need to decrease the resolution of static areas. The HQV’s motion-adaptive deinterlacing virtually eliminates line twitter during vertical movement, and there were very few jaggies along diagonal edges.

The HQV chip implements the algorithms in the $60,000 Teranex deinterlacer. Obviously the chip is far cheaper, but it does illustrate why the typical deinterlacer/scaler used in cameras, in software, and in monitors don't do a very good job. (In the case of software, the problem is doing the computations in real-time. If you are willing to wait -- software deinterlacing of 50i/60i is better than what can be done in the camera.)

Bottom line, while the V1 doesn't use deinterlacing "in camera" -- once it records interlace -- the quality you see (and many if not all of the artifacts folks think is FROM the camera) is dependent on the deinterlacer used AFTER the V1 does it's job of recording.

[Bob Grant]

I still fail to see what this has to do with the V1, sure if you're shooting interlaced then yes, it's applicable, but no more so than to any other "I" footage.
Shooting in progressive an adaptive de-interlacer isn't needed as it's PsF, all that's needed is a field merge, if anything adaptive de-interlacing could potentially get it wrong and really mess up the image. Adaptive de-interlacing can be fooled by noise, I can't speak for the Teranex boxes but the manual for an older Laird unit that I had floating around made much of this problem and the need to correctly set the noise threshold. These boxes though do kind of assume broadcast quality input, I'd imagine low light footage from any of the HDV camcorders could really spin them out.

As far as I can work out very little if anything seems to support 25p or even 30p so there's no way to tell the display device that it's getting PsF so it knows to deal with it properly, you'd think at least the display devices could have this somewhere in the menu.

Within my NLE I've got three choice for de-interlacing:

1) Merge, bit nasty with motion on I but perfect with PsF
2) Interpolate, no problme with either I or P but kiss half your vertical res goodbye.
3) Smart (from a 3rd party) not that smart, certainly not as good as deriving motion vectors, merge those parts of the frame where the two fields show no separation and interpolate those that do. Worst case with PsF, you'll loose vertical res over the whole frame during pans but motion blur typically masks that anyway.

None of them should introduce anything artifact like. Bottom line is there's no way anything can introduce interlace artifacts into PsF footage, there's no temporal separation between the fields to create the problem in the first place. Everything pretty well down here is broadcast as 1080i or 720p and so far I'm not seeing any problems and nor are the broadcasters, regardless of how the camera derives 25p it all ends up in a 50i transmission.

What has far more potential to really mess up an image is the scalers used in non native 1080 displays. The Bravia V series are plain aweful with SD or 1080, 720 looks OK but anything else really makes you wonder. None of it looks like a de-interlacing problem though, as one old hand said it looks more like way too much aperture correction.

[Steve Mullen]

Originally Posted by Bob Grant

I still fail to see what this has to do with the V1, sure if you're shooting interlaced then yes, it's applicable, but no more so than to any other "I" footage.
Shooting in progressive an adaptive de-interlacer isn't needed as it's PsF, all that's needed is a field merge, if anything adaptive de-interlacing could potentially get it wrong and really mess up the image.

As far as I can work out very little if anything seems to support 25p or even 30p so there's no way to tell the display device that it's getting PsF so it knows to deal with it properly, you'd think at least the display devices could have this somewhere in the menu.

I think you answered your own question -- only the V1 knows it's PsF. To every other device it is 1080i50 video and will be treated as such. If a deinterlacer correlates each pair of fields -- it will find no difference -- and may "weave." But, if it doesn't have motion sensing logic -- it will likely simply bob.

Bottom-line, anything bad that can happen to 1080i50 can happen to 25p from the V1. So, it applies equally to I and P. And, if you don't think 1080i has problems -- just go to the local store and compare 720p and 1080i broadcasts.

----

In the USA, the issue is different. The deinterlacer must recognize 2-3 pulldown -- either by MPEG-2 flags (DVD players) or by cadence (HDTVs). This tells the deinterlacer how to deal with judder frames. The HVQ supposedly does this very well. But 80% of our HDTVs fail to correctly sense cadence. Which raises the question of what these 80% do when they try to deinterlace judder frames?

Bottom-line -- my support for 720p has always been based upon the fact one could ignore all these issues. Capture P, encode P, record P, decode P, and display P. As you point-out, the only worry was scaling 1280x720 to some other size.

Now that I'm using 1080i -- all the issues that I avoided by using 720p are present -- plus the fact that even the P mode video passes through an interlace path. (And, so will F mode whenever it is broadcast as 1080i.) Thankfully, since I didn't see any artifacts -- I could ignore it all. But, given the VARIED reports of strange artifacts from PAL land, I've gone back and re-visited these issues.

[Michael Phillips]

This may be a stupid question, but when we are able to deliver a HD or Bluray
disc via a progressive player, will the final product that is displayed on a plasma or LCD TV have any of these problems that we get from direct connection of the camera to a TV or monitor.
Mick.

[Steve Mullen]

Originally Posted by Michael Phillips

This may be a stupid question, but when we are able to deliver a HD or Bluray
disc via a progressive player, will the final product that is displayed on a plasma or LCD TV have any of these problems that we get from direct connection of the camera to a TV or monitor.
Mick.

Good question. After Googling for an hour I've found:

1) line-flicker can NOT happen with progressive -- which I've always believed.

2) line-twitter occurs during vertical movement as very thin lines and hard edges fall between scan lines -- which I've always believed.

3) line-twitter has to do with high-contrast edges WITH interlace video. This is new to me.

So, it seems like you shouldn't see #1 and #3, but might see #2. In fact, there are reports of line twitter with 1080p Bluray and 1080p HDTVs.

But, even if this is true -- it's only one of many artifacts that you'll see because 4:2:0 sampling is used. There is not going to be perfection from consumer level products. According to experts DVDs look horrible. Not to millions of people.

The amount of compression artifacts on DTV and HDTV are horrible, but they seem acceptable to 99% of the viewers.

The problem comes when equipment is being reviewed. Are the things you are seeing in the product or elsewhere in the chain. Or, are they inherent -- such as certain MPEG-2 issues.

In production, you need to know if what are seeing is in your media or in your monitor.

The production community is buzzing with questions about the use of "consumer" HDTVs as monitors. The fact you cannot buy but one $40,000 CRT HD monitor has pushed everyone into going this way.

Even worse, IMHO, is the use of flat-screen PC monitors as substitutes for HDTVs. Since these are designed 99% for progressive signals -- they typically have junk deinterlacers. Yet, folks buy them because they have no education in selecting HDTVs.

[Mikko Lopponen]

Originally Posted by Steve Mullen

I think you answered your own question -- only the V1 knows it's PsF. To every other device it is 1080i50 video and will be treated as such.

So? Every camera outputs an interlaced signal even if it's in reality progressive. That's the nature of analog video. Unless you take the signal out digitally.

I don't understand why you're promoting a certain chip though. If you run an interlaced signal through an interlaced monitor it will also display twitters and flickers anyway. That's the nature of interlacing. Ofcourse you can minimise them by applying all kinds of "smart" techniques, but that doesn't take away what REALLY is in the signal and what gets interpolated. Adaptive deinterlacers have their own problems as Bob said.

LCD's are anyway pretty poor for video work because of their lackluster blacks and the backlights that tend to lose vividness in colors. LED-backlight LCD's are better in that regard though.

[Bob Grant]

This is discussion has for me at least a rather bizarres twist to it.
Why do I want to shoot 25p?
Well I don't really, I'd much rather 60p but that's another story. Reason is because I shouldn't have to worry about what any de-interlacer will do to my images, Now I'm being told oh, no, because my frames are being split into fields all manner of much worse things could happen than if I shot 50i, huh?

Let's just back track here for a minute, those two fields in 25PsF have no temporal separation, end of story. They don't really need de-interlacing in they way it's mostly talked about. I can grab a full frame (thats both fields merged) and drop the image into PS and bingo perfect image.

So OK, there's no way maybe to tell the dumb display device the thing is PsF, well OK but what's the worst mistake it can make and what will that do to the image. Worst case I can see is it'll ditch half the vertical res. Shucks. Well readin some test results, the Canon cameras and the HVX 200 only manage 540 lines anyway so not much loss anyway. Maybe the V1 manages 800 lines but heck, worst case I should be seeing 540, the same as any of the other cams, even the F350, not too shabby at all. Yeah it'd be nice to keep those 800 lines of res but remember I said worst case. The good news should be no matter what no interlace combing, no line twitter, I'll glady forgo a bit of vertical res to be rid of those.

Conversely if I'd shot 50i my image is very much at the mercy of the display device. Unless it's a $50K HD monitor it's really going to have to de-interlace the image and all manner of errors can be made. In fact there's no way to get a perfect result, even with the most expensive boxes, they might do fine or not. 25PsF sounds like a way more attractive proposition to me, results should be more predicatable.

That's not to say that Steve doesn't make a good point regardless, of course the display device plays a part in what you see and you need to know how it's performing before damning anything. Same goes for any test instrument. One way to do this without spending serious dollars is to feed it a known good signal and compare how a certain camera looks by comparison.

Now Steve has made a point that comparing the A1 with the V1 mightn't be fair as the V1 has more vertical res, well it's not that hard to create a test video with full 1440x1080 res, create somthing in PS, drop it on the T/L and encode it, bingo. See how that looks, if it looks like crud on the monitor you know the monitor has a problem. If it looks pristine and the V1 footage looks like crud there's a reasonable chance something is wrong with the V1. This isn't rocket science, it's standard practice in any lab.

[Piotr Wozniacki]

Originally Posted by Bob Grant

Now Steve has made a point that comparing the A1 with the V1 mightn't be fair as the V1 has more vertical res, well it's not that hard to create a test video with full 1440x1080 res, create somthing in PS, drop it on the T/L and encode it, bingo. See how that looks, if it looks like crud on the monitor you know the monitor has a problem. If it looks pristine and the V1 footage looks like crud there's a reasonable chance something is wrong with the V1. This isn't rocket science, it's standard practice in any lab.

I would do it, but I don't know how to "create a test video with full 1440x1080 res" other than with the camera:). Do you? If it is trivial and I'm missing something, please prepare such a clip with a bright scene, containing contrasty lines (or even better, some rez chart that would be available to everybody for shooting), and upload it for us - then comparing it with the V1E 25p output will be a trivial task, and we would know everything: either the camera's vertical rezolution is too high for most display to handle properly, or...well, the V1E does have problems, after all!

[Tom Roper]

Originally Posted by Steve Mullen

The Realta HQV 1080i motion-adaptive deinterlacerchip is considered the "best" hardware deinterlacer. Most deinterlacer/scalers convert 1080i video to 1080p using vertical interpolation to scale each individual 1080i field directly to a 1080p frame. The vertical resolution of each displayed field is thus limited to the 540-line vertical resolution of each field. Vertical interpolation acts as a filter, which reduces the vertical resolution and softens the image.

The HQV uses pixel-based motion-adaptive deinterlacing, which applies "interpolation" to image areas that are in motion, but merges (weaves) information from odd and even fields in static image areas. Merging field information slightly reduces 1080 line vertical resolution in static image areas.

With HQV there isn’t a dramatic change of image sharpness when an object is moving -- an annoying artifact, called resolution pumping or breathing, which occurs when the entire image resolution decreases with movement. Or, when moving "regions" lose vertical resolution. One way to reduce this artifact is to slightly further decrease resolution on still images so that breathing is not as noticeable.

The Realta algorithms provide higher resolution in the moving image areas, so there is no need to decrease the resolution of static areas. The HQV’s motion-adaptive deinterlacing virtually eliminates line twitter during vertical movement, and there were very few jaggies along diagonal edges.

The HQV chip implements the algorithms in the $60,000 Teranex deinterlacer. Obviously the chip is far cheaper, but it does illustrate why the typical deinterlacer/scaler used in cameras, in software, and in monitors don't do a very good job. (In the case of software, the problem is doing the computations in real-time. If you are willing to wait -- software deinterlacing of 50i/60i is better than what can be done in the camera.)

The Realta HQV does not use motion adaptive per pixel deinterlacing, nor is that method the best. The Realta HQV uses a better method called direction vector where the lines are analyzed and shifted into alignment horizontally. You also don't have to spend $60,000 on a Teranex deinterlacer to get the Silicon Optix HQV chip. It comes standard in the $800 (street) Toshiba HD-XA2 HD-DVD player, the $5k Mitsubishi HC5000 1080p projector among others.

[Tom Roper]

Originally Posted by Bob Grant

So OK, there's no way maybe to tell the dumb display device the thing is PsF, well OK but what's the worst mistake it can make and what will that do to the image. Worst case I can see is it'll ditch half the vertical res. Shucks. Well readin some test results, the Canon cameras and the HVX 200 only manage 540 lines anyway so not much loss anyway. Maybe the V1 manages 800 lines but heck, worst case I should be seeing 540, the same as any of the other cams, even the F350, not too shabby at all. Yeah it'd be nice to keep those 800 lines of res but remember I said worst case. The good news should be no matter what no interlace combing, no line twitter, I'll glady forgo a bit of vertical res to be rid of those.

Conversely if I'd shot 50i my image is very much at the mercy of the display device. Unless it's a $50K HD monitor it's really going to have to de-interlace the image and all manner of errors can be made. In fact there's no way to get a perfect result, even with the most expensive boxes, they might do fine or not. 25PsF sounds like a way more attractive proposition to me, results should be more predicatable.

That's not to say that Steve doesn't make a good point regardless, of course the display device plays a part in what you see and you need to know how it's performing before damning anything. Same goes for any test instrument. One way to do this without spending serious dollars is to feed it a known good signal and compare how a certain camera looks by comparison.

Now Steve has made a point that comparing the A1 with the V1 mightn't be fair as the V1 has more vertical res, well it's not that hard to create a test video with full 1440x1080 res, create somthing in PS, drop it on the T/L and encode it, bingo. See how that looks, if it looks like crud on the monitor you know the monitor has a problem. If it looks pristine and the V1 footage looks like crud there's a reasonable chance something is wrong with the V1. This isn't rocket science, it's standard practice in any lab.

Adam Wilt is a trusted source but he doesn't quote any numbers on vertical resolution for the V1 or A1 that I have seen. What he states is "800 TVL" which is about as vague as it gets, and is probably the horizontal resolution only. I've been doing the resolution testing long enough to know that by changing the test conditions, I can produce a resolution number that is completely valid yet absolutely irrelevant for any comparison test except ones I would do side by side under the same condition. By choosing how to represent the data, I can truthfully state the exact same result as 850-975 lines using edge profile or MTF30 and no sharpening compensation, or state the exact same image as 650 lines using MTF50 corrected for in-cam sharpening. So believe what you want, because that's how these numbers get bandied about in a flotilla of misinformation.

What I can tell you with certainty, is that the A1 24F mode is way better than the 540 lines vertical using the most conservative ratings, and the FX7 is no where near the 800 lines vertical in the interlace mode.

These numbers are everything to spec measurbators, but are one of the worst indicators to make comparisons with due to the plethora of ways that anyone can produce a TVL number.

The eyes alone are easily fooled because applied sharpening changes what's apparent subjectively. Computer software can discriminate the small differences in resolution, but because the result can be stated in so many different ways, people just latch on to a number. The difference between 650 and 800 TVL is very slight, barely observable.

The eyes are the best judge of the image quality. Just don't use them to correlate a number.

[Thomas Smet]

I still say for us HDV shooters the best option when we produce HD-DVD and Blu-Ray disks is to down convert to 720p. 60i/50i can be converted to 60p/50p with different levels of quality used to create the missing pixels. Even simple bobbing looks pretty good when down scaled to 720p. 24p,245p,30p,24F,25F,30F will down convert very well to 720p and will give amazing results.

The reason for all of this of course is to avoid and complications of what HDTV your client will be using to watch the content. While there are many different levels of how well a HDTV deals with 1080i content,720p is pretty standard and will give the most consistant results. This goes with Steve's information on how different 1080 displays deal with 1080 video in different ways.

Yes on a perfect display with a perfect progressive playback device your 1080 video may look great but really how many of your clients are going to have this perfect combination? For me I would rather deviler content that I know should look consistantly good no matter how the content is being viewed.

Try down converting some 1080p/F footage to 720p and then back up to 1080p and you will see that there is really very little difference. It is actually only a tiny bit softer and most of that could be from the bicubic or whatever other formula is used for the scaling. I have tried this with samples from every single HDV camera and so far I have been very impressed with the results. Plus the fact that 720p 24p,25p,30p take up so much less space which means more video or higher quality due to less compression.

I feel as though 720p cameras are not perfect which is why I prefer to shoot 1080 but deliver as 720p. Having an oversampled image seems to make a much nicer natural looking image compared to shooting as 720p.

I realise I may get butchered for suggesting down converting 1080 to 720 but I suggest people at least look into it. Doing so means your progressive video will be watched as progressive video no matter what display is being used. This will also pretty much get rid of any aliasing and flicker problems you may have.

[Tony Tremble]

Originally Posted by Tom Roper

Adam Wilt is a trusted source but he doesn't quote any numbers on vertical resolution for the V1 or A1 that I have seen. What he states is "800 TVL" which is about as vague as it gets, and is probably the horizontal resolution only. I've been doing the resolution testing long enough to know that by changing the test conditions, I can produce a resolution number that is completely valid yet absolutely irrelevant for any comparison test except ones I would do side by side under the same condition. By choosing how to represent the data, I can truthfully state the exact same result as 850-975 lines using edge profile or MTF30 and no sharpening compensation, or state the exact same image as 650 lines using MTF50 corrected for in-cam sharpening. So believe what you want, because that's how these numbers get bandied about in a flotilla of misinformation.

What I can tell you with certainty, is that the A1 24F mode is way better than the 540 lines vertical using the most conservative ratings, and the FX7 is no where near the 800 lines vertical in the interlace mode.

These numbers are everything to spec measurbators, but are one of the worst indicators to make comparisons with due to the plethora of ways that anyone can produce a TVL number.

The eyes alone are easily fooled because applied sharpening changes what's apparent subjectively. Computer software can discriminate the small differences in resolution, but because the result can be stated in so many different ways, people just latch on to a number. The difference between 650 and 800 TVL is very slight, barely observable.

The eyes are the best judge of the image quality. Just don't use them to correlate a number.

Tom

I think that is "Post of the Day"

I am sure I read something that Adam Wilt re: V1U was that with the alignment of the chips and the amount of interpolation was that it might be difficult to ascertain an exact tvl figure for the V1U and could range from 540-800+. If I have misquoted him I apologise in advance. I am pretty certain it was from him.

The problems the V1E has in progressive mode are not to do with resolution nor are they _anything whatsoever_ to do with de-interlacing. Steve has skilfully moved the goal posts of the argument around so much that the underlying issue of the quality has not been addressed.

The real issue is the quality of progressive mode when dealing with fine detail.

FootyA.jpg is a progressive grab from a "fixed" V1E
FootyB.jpg is an interlaced grab for same "fixed" V1E.

Nobody has explained why there is such a disparity in image quality. The field in the mid ground is turned into a blocky mess in progressive mode while in interlaced the detail remains.

The appalling mess the V1E makes of progressive footage is not explained by anything Steve has posted.

I would be very interested in grabs from Piotr's V1E that has come off the factory floor in a supposedly fixed state. Piotr can you shoot some scenes with fine detail like an area of grass?

These images are proof positive that TVL numbers don't prove anything in terms of image quality.

TT

[Piotr Wozniacki]

I clearly see the difference you're talking about in these pictures, Tony - but I haven't noticed anything like it YET. I'm waiting for a better weather to take the V1E into the field in the neighbourghood and shoot some tests.

So far, the only difference in the quality I've managed to spot is definitely more noise in the progressive (especially dark, monotone areas - my little theory is that perhaps in previous firmware version some DNR was switching on in such areas and selectively "oil-painted" them; nothing like this happens now - just a noise, especially with the sharpness at the default 7).

The other thing it striked me since the first time I used the "fixed" V1E is line twitter in bright, contrasty, nearly horizontal edges - but I tend to think this is display-device dependent.

[Tony Tremble]

Originally Posted by Piotr Wozniacki

The other thing it striked me since the first time I used the "fixed" V1E is line twitter in bright, contrasty, nearly horizontal edges - but I tend to think this is display-device dependent.

What LCD panel are you using? Dell?
TT

[Piotr Wozniacki]

It's a Fujitsu-Siemens ScenicView P24-1W