abrupt highlights clipping

[Michael H. Stevens]

Originally Posted by Piotr Wozniacki

Michael,

I started one there, too - since yesterday Glenn hasn't responded yet; still hoping he will :)

Piotr (like Peter or Pietro; not Piort :))

Sorry Piotr. How does your name sound?

[Piotr Wozniacki]

It's OK :) Cannot find the phonetic transcription right now; with short 'i' anyway...

[Michael H. Stevens]

Piotr:

OK thanks for the hint on changing color space. I now DO SEE the the washing out when moving from Computer RGB to Studio RGB. However I don't see this as a problem as there is no reason to go to Studio RGB is there?

SECONDLY: How do you take those Vegas scopes screen shot?

THIRDLY: I researched your name: pronounced pē-ŌT, or in english sounds Pea.oh'.tr

[Alexander Ibrahim]

Originally Posted by Bill Ravens

You guys absolutely amaze me with your compulsive analyzing of things. For example, the statement that one should move from 256 levels to 1024 levels. Keep in mind that each channel carries 256 values. Therefore for all three channels(and ignoring alpha channel), we're talking 256x256x256=1.68 MILLION colors. OTOH, 1024 levels would be over 1 BILLION colors!

Right on brother!

There isn't a display screen in the world that can display that.

Oh... wait up now. You've gone too far.

Sony BVM-L230 is a 10 bit LCD panel, and a 12 bit display engine. It will show 10 bit video color and fully complies with ITU-R BT.709, EBU and SMPTE C. It can also approximate D-Cinema display space, but can not display all the colors in the DCDM gamut. (And it can switch between all those color spaces too.)

Lots of D-Cinema projectors will show all that - and then some since they typically show the full DCDM gamut and have approximately the same total gamut as film. (They are under gamut in some colors and over gamut in others.)

Even your average 8 bit display will show some benefit- even if you NEVER post your work. For example the only banding you'd ever see would be as a result of the display's limitations, not your video signal. Even fairly light post can mess up a sky gradient- and I'm sure you can think of other examples.

You may think that all this is irrelevant. After all those displays I mentioned are expensive. The Sony is ~$30,000 USD when fully configured. D-Cinema projectors are $90,000 and way up.

Take a look at the HDMI spec. Deepcolor wasn't added as a marketing ploy. LCD manufacturers plan on implementing 10 bit display panels. They thought the panels would be readily available by now (2008) in a fakey format using dynamic display engines and 8 bit panels- a low end version of the Cine-Tal dsplays.

(That isn't a fair comparison. The Cine-Tal displays are suited for delicate DI and grading, and add a whole boatload of other features)

I personally don't think you'll see one at Best Buy until 2012, but they are coming.

You should consider what the footage you shoot today will look like in that post environment and on those consumer displays if you expect your work to have lasting value.

10 bit is starting to make sense for producers today the way HD made sense while we were all still delivering SD (and many of us are STILL delivering SD with our HD cameras!) It also neatly solves some problems we see in 8-bit workflows- just like shooting 720p gave us improved apparent resolution on SD delivery.

Of course, when 10 bit delivery becomes the norm we will all be scrambling to work with 10 bit log systems. The film guys will be working in 16 bit log and using 32 bit linear all day. (Some of them already are!)

That's the business.

[Alexander Ibrahim]

Originally Posted by Bob Grant

There's no 7.5 IRE setup in the digital realm.

Strictly speaking no, which is why I was specific in mentioning NTSC.

If you plan on NTSC delivery then you had better be thinking about NTSC's limitations as you shoot and post.

The impact of using the extra range from superblack to superwhite on an mepg-2 encoder would be negligable, certainly nothing like 17%.

Impact in what sense exactly? Impact on encoding efficiency? Impact on the image quality? How about its impact on the tendency of the codec to clip regions?

The biggest issue for the XDCAM or any mpeg-2 encoder is noise and motion.

I agree, but that aspect isn't relevant to abrupt highlight clipping.

Recording 10bit or 10bit log certainly can gives you more wiggle room.

So... wait I thought you were disagreeing with me? That is exactly what I am saying. Now one way to use that extra "wiggle room" is to benefit exposure latitude. That will help with abrupt clipping... which is what we are supposed to be talking about right?

With the SI-2K I'm effectively under exposing compared to the Z1 or EX1

I am not clear at all what you mean.

Do you mean that you are actually exposing differently when looking at the same pre-lit scene? That wouldn't be surprising. Do you mean that you are setting lighting differently to achieve different exposure targets with the camera? Again unsurprising- the RAW workflow is different.

Are you talking about the fact that the SI-2K records a "RAW" sensor feed and that it looks flat by comparison to the pre corrected ready to view images you get from cameras like the EX1?

In any case I don't see how it impacts the choice between 8 and 10 bit recording on a camera like the EX1.

and then digging into the shadows. It's nice to delay those decisions to post for sure and with care you can get a much more pleasing image. I can just as easily if not more easily mangle it too.

I agree with the first part, and agree conditionally with the bit about mangling. As you gain experience its actually easier to manage the entire workflow with good results from these cameras.

Its not as forgiving as film- but its far more forgiving than standard video cameras. It can be hard to find the new "edge" of performance... its somewhere between film and traditional video.

In the end you're pretty much forced to deliver in 8 bits. You can make the hard decisions in a camera like the EX1 or you can spend a lot of money on more expensive cameras or recording systems and shift the decisions to post. Unless you start with a sensor with high dynamic range you might be spending a lot of money to record higher quality noise.

What I am specifically saying is that the EX1 has a high dynamic range and you get more information when you upgrade from the XDCAM codec to a 10 bit codec.

I haven't measured it carefully yet, so I am not sure you really get 10 bits worth of data- but I am pretty sure you get 9 bits worth... and maybe a touch more. Its enough to be worth the hassle and expense of 10 bit recording provided you can manage and afford it.

Where I specifically disagree with your philosophy is that I can't conceive why you'd desire to make the hard decisions in the camera on set, and then discard the data that might allow you to change your mind.

On set you have all sorts of other concerns, like lighting, camera motion and exposure and schedule and... a lot of stuff. Why should you want to add making your final coloring decisions on set too?

I can understand practical reasons why you may not have a choice- and expense is a huge one - but I can't understand wanting to throw the data away.

Especially since we have so much more flexibility in post if the data exists.

[Mike Ward]

As i understand it,no matter what sort of camera you point at the horizon you're never going to get a perfect exposure.ethier the sky is correct & the earth is underexposed or vise versa.
Although i do see this strange effect on the grabs,it's as if the branches have turned blue.

[Glenn Chan]

I'm going to have to ask the obvious question. If 110% is the point of digital clipping how does the camera record 150%?

The camera doesn't / can't. I think it's a conceptual thing. Or those values get clipped off at whatever the cutoff is... since it records into 8-bit Y'CbCr values, and Y' only goes up to 255 (235 is where legal white is).

-----------------------------

In regards to the original topic:

A- Vegas' limitations have been discussed on the SCS board. I won't add much about that here, other than don't trust the scopes or the video preview window.

B- In this particular case, you can recover some extra highlight detail by mapping the superwhites into legal range.
For a 32-bit project (the following numbers won't work for 8-bit), apply the Levels filter. Output start should be 0.907.

C- Most cameras have knee functions and algorithms that try to fix the deficiencies of a simple knee.

A simple knee would just apply something like RGB curves to the whole image... like you had taken the image into Photoshop and done curves there to compress the highlights. The problem with that is that highlights will de-saturate when you do this. So what the various cameras do is implement their (secret?) sauce to try to keep the hue and saturation looking right. In a way, the camera plays colorist with your footage.

In some situations, the knee algorithm is pretty inappropriate and will give you crappy results. I think that's the case here if you choose the HISAT setting (that's what it looks like anyways).

D- To figure out the "ideal" exposure... just do a test. That's the best way in my opinion. Your variables are:

What are you shooting. (e.g. some scenes will have a high dynamic range and be challenging.)
Iris/shutter/exposure. Bracket this (e.g. vary the iris or shutter).
Camera settings.
Color grading / color correction in post.

Play around with all this stuff and see what looks best. And from there, figure out how the camera's display, zebras, etc. etc. correspond to the ideal exposure. This way, in the field, you can quickly nail the desired exposure.

To keep you test simple, go outside on a sunny day and shoot something with lots of highlights and shadow.
Try different camera settings, and talk into the camera's microphone to say what the settings are (or write them down on a sheet of paper and possibly film that).
Vary the iris to get a range of different exposures. (Probably do this in increments of half-stops.)

Take all that stuff into post and play around with it.

[Michael H. Stevens]

Glen: What you say is right but I ask all the members reading here to understand that Piotr and me and a few others ARE NOT SAYING WE CAN'T SHOOT GREAT VIDEO - we can. We are trying to "understand" what this camera is doing in various demanding or special situations and some of us are trying to decide if our camera has a problem or not in time to exchange it if necessary.

[Bob Grant]

I pretty regularly have to deal with broken cameras and VCRs. The probability of what's being discussed here being due to something broken in a camera is very, very slim. Almost all the processing is digital, things digital in general tend to work or not or sometimes produce very dramatic errors. There's many things I'd be more worried about in terms of your camera being below standard.
Dead pixels and missaligned block or optics would be my primary concerns.

[Michael H. Stevens]

Originally Posted by Bob Grant

I pretty regularly have to deal with broken cameras and VCRs. The probability of what's being discussed here being due to something broken in a camera is very, very slim. Almost all the processing is digital, things digital in general tend to work or not or sometimes produce very dramatic errors. There's many things I'd be more worried about in terms of your camera being below standard.
Dead pixels and missaligned block or optics would be my primary concerns.

True Bob: I think I did wrong to emphasis the broken thing. 90% of this to me is an intellectual exercise in wanting to understand. I still am totally baffled as to why my images reach maximum exposure (IE no white detail going away and looking fantastic) when the EX1 histogram goes from 10 to 80 say and the Vegas histogram goes from -15 to 115! And maybe I should stop worrying and just go shoot some pictures but no one, not Glen or Bill or anyone seems to really have an understanding of this that have been able to explain.

[Piotr Wozniacki]

Originally Posted by Michael H. Stevens

And maybe I should stop worrying and just go shoot some pictures but no one, not Glen or Bill or anyone seems to really have an understanding of this that have been able to explain.

Exactly what I think, too. Even though Glenn (unlike Bill) very politely encourages to ask these questions as completely legitimate, it seems nobody has complete understanding of the relationship between the EX1 scopes vs Vegas scopes, especially in the context of Vegas 8bit vs 32bit processing, and/or colour space conversions - not to add the extra complication that the PAL video format introduces as opposed to NTSC format.

Having technical and scientific background (in completely different areas than videography), I hate being unsure and using the trial&error method rather than understanding the theory behind what I'm doing, but I've given up asking about this specific issue, and instead am going to use my own eyes plus my own experimentation with different combinations of settings - I'm sure that just like with my previous cameras, I'll master the EX1 sooner or later, as well.

Because you see, while I realize my English is not perfect (far from that), I still believe my questions have not only been legitimate, but also worded clear enough. If anybody knew the answers and was willing to help, he would. Just see how clear and in-depth Alexander's posts can be...

[Piotr Wozniacki]

Originally Posted by Michael H. Stevens

Piotr:

OK thanks for the hint on changing color space. I now DO SEE the the washing out when moving from Computer RGB to Studio RGB. However I don't see this as a problem as there is no reason to go to Studio RGB is there?

SECONDLY: How do you take those Vegas scopes screen shot?

THIRDLY: I researched your name: pronounced pē-ŌT, or in english sounds Pea.oh'.tr

Michael,

Sorry I didn't notice your post (especially its SECOND point) earlier. Taking the scopes screenshots is easy in Windows - just click on its window to focus, press Alt+PrtScn on your keyboard, and voila!...nothing happens :) But you have the window in your clipboard, which you can now paste into e.g. the Paint program (which you have in Windows Accesories). Than "Save it As" a .jpg file...

[Steven Thomas]

Don't knock these guy. They happen to be very intelligent with these subjects.
It's really down to owning the camera and determinig yourself what these tools can do for you.

It appears you can't assume that images are safe by the extreme ends of the histogram.

[Alexander Ibrahim]

Originally Posted by Piotr Wozniacki

... it seems nobody has complete understanding of the relationship between the EX1 scopes vs Vegas scopes, especially in the context of Vegas 8bit vs 32bit processing, and/or colour space conversions - not to add the extra complication that the PAL video format introduces as opposed to NTSC format.

I don't use Vegas, but Glen Chan said, "Vegas' limitations have been discussed on the SCS board. I won't add much about that here, other than don't trust the scopes or the video preview window."

That concerns me.

I don't know why that should be the case- or in what way they tend to fail. For me that would be a reason to stop using Vegas. If the developers can screw up something that straightforward then what else is wrong?

Regardless, it does make the case neatly for a real hardware waveform monitor and external video monitoring.

The cheapest option I know is getting an SDI monitor with a built in Waveform/Vectorscope like the Sony LMD series. I wasn't happy with the detail in the scope displays- but it is fairly accurate and reliable. It shouldn't be your first choice for scopes, but it will let you know straight away if something is fishy with any of your other scopes. (Software scopes... I'm looking at you!)

Having technical and scientific background (in completely different areas), I hate being unsure and using the trial&error method rather than understanding the theory behind what I'm doing, but I've given up asking about this specific issue, and instead am going to use my own eyes plus my own experimentation with different combinations of settings - I'm sure that just like with my previous cameras, I'll master the EX1 sooner or later, as well.

The main thing is that we need for an EX1 user to sit down with test charts, a good set of scopes, and a light meter and map out what the camera can do in some detail. Then we need to take that into the various bits of software and compare.

If anybody knew the answers and was willing to help, he would. Just see how clear and in-depth Alexander's posts can be...

Flattery will get you places! Thanks, I'm just glad its worth the read.

[Bill Ravens]

For whatever reason, and not a reason I understand, using the Computer RGB to Video RGB performs an excessive amount of attenuating of the hilites and shadows of a native .mxf file in V8. All you have to do is look at the histogram. You're right guys, the image looks, and is, washed out.

The solution is simple, ignore the LEVELS FX preset and make your own, limiting the video to 16-235, instead of 24-220, or whatever weird levels the preset is generating. This has been very successful for me, not just on the EX1, but, the JVC HD110 images. I don't know why everyone is so tied into the presets.

WHATEVER the camera does, if delivery is via NTSC, , regardless of the processing precision, 8 or 32 bit, then set it for RGB16-235. The Histogram is the easiest way to see the levels. Now, having said that, scenes are constantly changing in brightness and shadow value, not to mention that some scenes don't have pure white or pure black. So, some DISCRETION is advised. This requires some thought....one can't just push a button and make it perfect, que no?

Both the JVC HD110 and the EX1 capture data outside of the conventional bandwidth everyone is used to. One can force the camera to legacy limits, or, one can accept the new paradigm and work with it, given the tools available, e.g. Vegas8.I'd bet 1 dollar US to a box of dognuts, Vegas will evolve to resolve the current apparent incongruities.

Actually, I think Steve Mullen is right. Vegas is hamstrung because it works in the RGB color space. Times have moved on and Vegas hasn't. Get over it. As for his contention that conversion introduces chroma errors...he doesn't know what he's talking about. It's easy to test by importing the NTSC bars FX onto the vegas timeline and setting the precision to 8 bit. Look at the WFM. It's "studio RGB", i.e. REC601. Now, convert to 32 bit precision....the WFM is not right, 'cuz you're looking at a REC601 COLORBAR with REC709 WFM.. Now, perform a Levels FX on the pattern, converting it to "Computer RGB". Whaddaya get? Yep, REC709 chroma and luma. So, what's the big mystery?

If you really wanna feel good, do the same NTSC pattern test only do it with a captured colorbar from the camera.