Potential/rumored Chromakey issues with HDV!

[Thomas Smet]

Ok I did a some examples in the following image that show what is wrong with the chroma on the new SONY V1.

(this is not a post to slam the camera in any way. The camera itself has nothing to do with any of this. It is just the way mpeg2 is.)

SONY went with the 24p inside of a 60i video because it makes the tapes playback in any 1080i device and the footage can edit in any 1080i HDV NLE. It really is the easiest way of dealing with it and it works very well. For normal footage you should get very good results.

The one area where you have to watch out however is with chroma keying or shooting scenes with really bright colors.

While the SONY V1 is the first true 1080p camera and should give really high detail progressive scan images I feel as though the chroma is limited compared to other 24p cameras. For example the Canon HDV cameras may have a reduced progressive resolution due to the interlaced chips but in the end they have better chroma detail. This is very important in chroma key situations.

What it comes down to is that mpeg2 uses two different forms of 4:2:0 color. One for interlaced and another for progressive. The way the interlaced 4:2:0 works is that each field is encoded as a 1440x540 frame at 4:2:0 and then they are combined together to form the interlaced 1440x1080. This results in extra jagged edges when compared to progressive 4:2:0. The same exact thing is true for DVD's and PAL DV as well.

Even though the 24p is progressive it has to be encoded as 60i so current HDV gear can recognize the format. The reason why Canon 24F will not play is any other HDV equipment is because they chose to use an unorthodox form of mepg2 encoding where the video is encoded as progressive. Other cameras and decks have no idea what to do with this since it is different.

If your desire is to have a camera for chroma key work you might be better off with any of the other cameras out there. Panasonic uses 4:2:2 while JVC and Canon use the progressive form of 4:2:0 which will give much cleaner keys.

Of course if you plan on capturing live from the HDMI port into an Intensity card this doesn't matter since it will be uncompressed and 4:2:2.

Along with this test image below I have checked the files that have been posted by DSE as well as the video posted from IBC including a 50i and 25p sample. All samples show this interlaced form of mpeg2 chroma including the 25p sample.

Note: If you use Cineform the chroma does get better since the Cineform codec tries to upsample the chroma to 4:2:2.


Here is an image I made that shows what happens with the different forms of chroma sampling in mpeg2. The four sections were made from the same RGB source and encoded with the Main Concept Encoder.


Again this is not an attack against the SONY V1 just a little warning to those who plan on doing some chroma key work with the camera. I had hoped I would be wrong but so far from the samples I have seen this is what the SONY V1 is doing. Hopefully I can check out some more raw samples from the camera soon.

[Marvin Emms]

Your argument is a little beyond my current understanding of MPEG, so I can't comment on that.

"While the SONY V1 is the first true 1080p camera"

I'm not clear how you reached this assumption. It may require a redefining of 'true' for my book.

For chroma keying I can't help but think the 1Mpixel sensors are going to cause more of a problem. The camera itself is more a 2:2:2 signal at full resolution and overall should not perform much better than a good 4:4:4 720p camera.

[Mikko Lopponen]

Thomas, yes, but you can blur those fields together. With pal equipment it's very easy to blur two fields together. As both of these fields have the same information except for chroma, that chroma will blur into one progressive frame perfectly.

60i is a bit more complex case.

[Mikko Lopponen]

Originally Posted by Marvin Emms

For chroma keying I can't help but think the 1Mpixel sensors are going to cause more of a problem. The camera itself is more a 2:2:2 signal at full resolution and overall should not perform much better than a good 4:4:4 720p camera.

That really hasn't been a problem with the z1.

[Marvin Emms]

The Z1 is not clearvid and as far as I am aware uses pixel shift, so this is potentially superior overall for keying, neglecting interlace problems.

Just a theory at this point of course.

Edit,

Additionally, I don't quite understand,

"As both of these fields have the same information except for chroma"

The blurring would seem to reduce resolution considerably, I don't see why the fields only have different chroma.

[Thomas Smet]

Originally Posted by Mikko Lopponen

Thomas, yes, but you can blur those fields together. With pal equipment it's very easy to blur two fields together. As both of these fields have the same information except for chroma, that chroma will blur into one progressive frame perfectly.

60i is a bit more complex case.

Thats the whole point. While you can blur the chroma you can also blur the chroma on any of the other chroma formats and get a much better result. 4:2:0 progressive blurred slightly will give you more overall detail compared to 4:2:0 interlaced that is blurred. There are plugins and tools out there to try and rebuild the chroma but why not start with better chroma to begin with? To blur interlaced 4:2:0 you have to do a 2x horizontal and a 4x vertical blur where as with 4:2:0 you only need to do a 2x2 blur. There is also the problem of ringing when you blur the chroma too much. That is the black line around your keyed subject. You can get rid of it with a matte choke but then you loose a lot of the thin details in the scene such as hair or the frames of really thin eye glasses.

[Thomas Smet]

Originally Posted by Marvin Emms

"While the SONY V1 is the first true 1080p camera"

I'm not clear how you reached this assumption. It may require a redefining of 'true' for my book.

Sorry I meant the first 1080p HDV camera that actually uses a 1080p set of chips and doesn't try to build the 1080p from interlaced chips. This is the first HDV camera to actually pull off 1080p. The horizontal resolution may be in question but the vertical is pretty solid.

[Marvin Emms]

The vertical resolution is no more solidly defined than the horizontal. Its a question of how you number them.

There are certainly cameras that will do 1080p, so in that sense it isn't true either. This may be the "first prosumer level camcorder to record in 1440x1080psf24 from a true progressive sensor (deep breath)" but then things get really muddy and you start having to define prosumer.

[Thomas Smet]

That is why I said "HDV" and not HD. I know there are HD cameras that do 1080p such as the Cinealta. I am talking about HDV cameras which are all under the $10,000.00 price mark.

[Dylan Pank]

Thomas, do you have any idea if this issue will also be the case for the V1E at 25P? or for 30p? I'm still trying get to grips with the issues you outline here so I'm aware I may be asking a dumb question.

[Marvin Emms]

Sorry Thomas, I didn't read your correction properly.

[Thomas Smet]

Originally Posted by Dylan Pank

Thomas, do you have any idea if this issue will also be the case for the V1E at 25P? or for 30p? I'm still trying get to grips with the issues you outline here so I'm aware I may be asking a dumb question.

Yes it is. Everything on the SONY cameras has to sit in an interlaced video so it will work with older gear. As far as a camera, deck or NLE knows it is dealing with a 1080i video. There were a few video samples from IBC of the V1E of a guy juggling some stuff at the show. There was a shot at 50i and a shot at 25p. The 25p clearly showed the interlaced form of chroma. I had hoped it wouldn't since 25p and 30p do not have to deal with pulldown but no matter what on the SONY the video has to sit on the tape as an interlaced video. Those samples could have been recompressed in the wrong way before they were posted but I doubt it. While SONY could have changed the HDV specs to force a progressive 1080 chroma format I still do not think it would matter. By changing the chroma format to progressive that would mean SONY would have faced the same problem as Canon where the 24F/25F/30F tapes do not play in other HDV equipment. Tapes shot on the V1 would have not worked at all in any of the current SONY HDV decks or cameras and I'm sure that would have really upset a lot of people.

I need a few more raw samples to check this to make sure I am 100% correct but from what I know and what I have seen from current samples then yes 24p/25p/30p/50i/60i all use the same form of interlaced chroma sampling.

[Mark Fry]

So, to clarify, this is likely to be a problem if one is using progressive footage (24P, 25P or 30P) from the V1 (or FX7) for chroma-key work. Do you think it will also be a problem when chroma-keying with interlaced V1 footage (50i, 60i)?

[Don Donatello]

you have similar interlace artifacts when keying FILM transferred to tape .
the pull down always gets in the way when keying and you have to work harder getting a good key .. remove the pull down and the interlace artifacts go away ... convert the m2t file using a intermediate ( cineform) then use combustion/AE to remove pull down and key it ...

in general:
person in front of green screen being interviewed would work for HDV ..however a persons moving constantly moving/running/jumping in front of green screen - HDV interlace camera would not be on my list ..
i'd have to test the new sony 24p with green screen to determine if i would use it on shots with alot of motion ..
anybody try sony 30p green screen yet ?

[Douglas Spotted Eagle]

Originally Posted by Don Donatello

i'd have to test the new sony 24p with green screen to determine if i would use it on shots with alot of motion ..
anybody try sony 30p green screen yet ?

Yes, I have. With the V1. But there isn't much point in having a discussion about how good it is or isn't, as it's subjective.