abrupt highlights clipping

Steven Thomas · March 4th, 2008, 05:06 PM

Piotr, you brought this thread full circle. It was already brought up earlier in this thread that Adam saw that saturated highlights show more hue shift and harsh clipping than he'd like. Adam also mentioned he would explore it more later.
Piotr, I don't believe no one disagrees that it can look bad when it clips.
I believe most do not care since they are not having an issue. If it clips, we back off.

Here again is what Adam Wilt wrote in his review:

"I’ve found that the EX1’s knee does a fine job except when highlights are strongly colored. Saturated highlights show more hue shift and harsh clipping than I’d like. I’m exploring this further out of curiosity, but even if the knees were perfect I would still shoot with cine gammas, because I prefer the progressive compression to the look of a traditional knee"

Tom Roper · March 4th, 2008, 09:53 PM

Piotr,

You mention that the phenomenon is observable when auto-iris is on. Can it occur if you used auto-shutter, or just auto iris? In other words, is the effect related to highlight clipping at the upper end of exposure, or possibly diffraction, which could happen with a small aperture but would not happen from a fast shutter?

I hope my question is not confusing. One thing I'd like to understand, is if the problem could be avoided by using a fixed aperture with shutter priority, instead of the other way around.

Piotr Wozniacki · March 5th, 2008, 02:51 AM

Quote:

Originally Posted by Tom Roper

Piotr,

You mention that the phenomenon is observable when auto-iris is on. Can it occur if you used auto-shutter, or just auto iris? In other words, is the effect related to highlight clipping at the upper end of exposure, or possibly diffraction, which could happen with a small aperture but would not happen from a fast shutter?

I hope my question is not confusing. One thing I'd like to understand, is if the problem could be avoided by using a fixed aperture with shutter priority, instead of the other way around.

No Tom - it's no connected with aperture values (and possible diffraction). It can occur with any aperture, as long as you don't move away from the exposure range that it's occuring in - by either slightly opening aperture (to blow out completely), or closing it (to stop clipping completely). And of course, you can achieve the same with shutter speed, if keeping constant aperture is yur priority (though you cannot be as precise as with the iris ring).

PS. Since this thread has drifted off its main subject, here is a grab to remind what we're talking about. And what Adam wrote to me when he saw this picture (I've just got his permission to publish, so that more users can benefit):

Hi Piotr,

What you are seeing is normal for STD gammas on the EX1, and it is
exactly the thing I found peculiar. You are right in observing that
it doesn't happen on other cameras, but it is there on all EX1s I've
seen.

I hope to have more to say about it soon, but for now, my advice is
not to trust the STD knees to protect or roll off your highlights
smoothly. Instead, use CINE gammas where important parts of the scene
are bright, like the sky in your image, and switch to STD gammas only
when you have control of the lighting, as in indoor interviews, or
when you do not care if the highlights blow out sharply.

CINE4 is very close in linear scene response (before the knee or the
curve) to STD3, so you might set up two Picture Presets, one on STD3
and one on CINE4 but with all other parameters the same. Switch
between them, using CINE4 for contrasty exteriors and long shots, and
STD3 for close-ups of bright faces where CINE4's desaturation is less
pleasing.

Cheers,
Adam Wilt

Tom Roper · March 5th, 2008, 08:36 AM

Quote:

Originally Posted by Piotr Wozniacki

... here is a grab to remind what we're talking about.

I remember. I've read closely since day one. Thank you for diligently maintaining these observations. And now you have Adam Wilt on board.

But as for the photo, the shift to cyan behind the branches seems like it's cast over other parts of the image as well.

Although confirmed it's not normal behavior with other cams, somewhere else I've seen it before. I think my original JVC GR-HD1 may have had this tendency.

Anyway...you're on the right track.

Regards,
Tom

Piotr Wozniacki · March 5th, 2008, 09:15 AM

Tom,

You're right about the cast. To prove it doesn't matter, I have posted above another grab with totally different colours (more red than cyan), and with the sky area only in one corner (as opposite to the previous ones where the sky dominated the scene). Still the same behind the trees...

I am totally aware the white balance (or colour matrix) is not neutral in either of the grabs, but - being opposite to each other - it shows my point that you cannot avoid the abrupt clipping with bluish patches (unless carefully watching and adjusting your exposure just for that, which is not practical).

And one more thing that needs stressing: this phenomenon - while easily explainable using laws of physics - is in fact very annoying and ugly. In other cameras it's handled in a fashion more pleasing to the eye. Why it is different on the EX1 beats me; unfortunately - while a bit safer like Adam says - the Cine gammas do NOT completely get rid of it at all. In fact, I just did some more tests with the safest one, the Cine2 (said in the manual to safely compress the highlights) - and while indeed the picture never even touches 100 (the zebra at 100 never shows up), the trees against the overcast, grey sky still create the artefacts around them.

I'd like to kindly ask knowledgeable people like Bob, Bill or Alexander to stop preaching that this is normal, and instead post their own examples of the phenomenon. I can't understand how people can leave with this - the clipping algorithm implemented in the EX1 is flawed IMO. It just seems unable to clip precisely around the very edges of darker (unclipped) objects, which are instead surrounded by ugly patches of unclipped sky... If this is how "broad dynamic range" is supposed to manifest itself, than no thanks - I prefer narrower range as in the V1E, or any other prosumer camera for that matter.

Yes I know the EX1 is capable of producing stunning pictures - I have shot lots of them as well, but only in very favourable or controlled lighting conditions (outdoors, with foreground lit generously against dark blue sky, or the opposite extremum: indoors, with low but controlled lightning). But in such conditions, even the HC1 I used to have is able of creating breath-taking imagery - and we're talking CineAlta here!

Unless my camera is indeed faulty, which only some examples from other people could help me to determine.

Steven Thomas · March 5th, 2008, 11:28 AM

Again,
I bring up Adam said this in his ORIGINAL review. It was brought up earlier in this thread maybe three times! Why bash on others for your problems. It's not ours.

Even Adam himself said to stick with the Cine curves during those shots.

If I saw this over exposure condition happening when composing the shot, I would readjust for the shot just like adjusting everything else, focus, aperture, focal length...ect..

Piotr Wozniacki · March 5th, 2008, 12:27 PM

Quote:

Originally Posted by Steven Thomas

Why bash on others for your problems. It's not ours.

Steven, with the above you're bashing me. Also, if this is not your problem - why post here at all?

Quote:

Originally Posted by Steven Thomas

If I saw this over exposure condition happening when composing the shot, I would readjust for the shot just like adjusting everything else, focus, aperture, focal length...ect..

Bullshit. None of the above can be considered overexposed, if the main subject was the foreground. They may show other settings' flaws (I was experimenting with WB / colour matrices extrema while taking them, NOT looking for the "abrupt clipping"), but certainly they're not generally overexposed! A "narrower latitude" camera ((like the V1, grabs of which I posted earlier in this thread as the comparison) would simply blow-out the sky; the EX1 blows it out partially as well, while leaving nasty patches around the contrasty objects. If you can't see this obvious flaw, see the optician.

If you still don't follow, I'll repeat it for you, Steven: with certain completely legal exposure levels, required to bring out darker foreground, the EX1 has problem with the compression algorithm, leaving unclipped boarders of bluish tint around darker objects where it should be blowing out like elsewhere in the sky (further away from these objects). It does so with all gammas; the cine gammas are only safer in that with auto iris this doesn't happen. But use manual iris to bring out some underexposed areas, and you can see the same "abrupt clipping" it does with standard gammas, even with auto iris on.

Did you understand now? Good, as this was my last effort to politely address your taunts.

Steven Thomas · March 5th, 2008, 01:19 PM

Piotr,
Have you happened to notice that you are really the only one complaining about this?

You wrote:
"Bob, Bill or Alexander to stop preaching that this is normal, and instead post their own examples of the phenomenon."

Maybe you're right, I'll let them speak for themselves, but I'm certainly under the impression they are not having issues, therefore, not going on with this rant.

Bob Grant · March 5th, 2008, 04:02 PM

As I suspect I'm one of the mentioned parties all I can say is the request to post our own examples borders on the absurd.

I can make an aircraft fall out of the sky without exceeding any of the acceptable control settings but it's not something one does more than once. This is why aircraft and many complex pieces of technology have an "envelope" of operation. It's why simply memorising a flight manual doesn't qualify you for a pilots licence, it's why memorising a camera manual doesn't make you a cameraman. It is why experience is everything.
Fortunately with a camera mistakes aren't as bad as they are with aircraft, automobiles and guns. The same logic still applies though, make mistakes by going outside the envelope and learn from them, yes there's not a lot published about where the limits of the envelopes are for each camera but then again we don't have air safety investigators studying smouldering piles of twisted metal like we do with bad images. So we have to learn from our own experiences.
Last night I shot a performance with my EX1, without even downloading the clips I know a lot of it is going to be crud. The camera did nothing unpredicatable, I operated it as well as it could be, the camera (heck any camera) simply could not cope with the flickering lighting, the very video system itself could not cope. For the record ultimately this was my mistake. From experience I now know that I should never of even agreed to attempt the shoot. Will I make the same mistake again, probably, I'm a sucker when a friend asks me to do something. Will you find me here posting examples of the crud I shot with 'please explains', good grief no.
So what would a professional have done if asked to shoot this. He would have insisted on two trucks and a crew of 10 to light and shoot this. Now he might have gotten away with it without that, just as I might have too. But I didn't and that's the difference between me and a professional. I take the risks and sometimes fail spectacularly, professionals don't, and I and the professional bill the client accordingly.

Now I know what a certain someone is thinking. "I didn't have these problems with my other camera". Well for sure. Learn to fly a 747 and you'll likely never have to worry about compressor stall like you will in a F18. That's why pilots get licences that are "qualified" for certain aircraft. It also shows that things built to extract more performance are also riskier to use, they take more experience to learn the dangers of, they are not easier, they're harder to work with.

It seems to me that the underlying false assumption is that because this camera has 'more' of this and 'more' of that it'll be easier to use. Wrong.

It might also explain why camera manufactures haven't unleashed the beast in their cheaper cameras in the past.

Piotr Wozniacki · March 5th, 2008, 04:28 PM

Quote:

Originally Posted by Bob Grant

Last night I shot a performance with my EX1, without even downloading the clips I know a lot of it is going to be crud. The camera did nothing unpredicatable, I operated it as well as it could be, the camera (heck any camera) simply could not cope with the flickering lighting, the very video system itself could not cope.

Bob if your intention has been to address my problem by drawing a parallel then I'm afraid it failed, as I am talking about perfectly normal conditions, which "camera (heck any camera) simply should cope with" ;)

Quote:

Originally Posted by Bob Grant

It seems to me that the underlying false assumption is that because this camera has 'more' of this and 'more' of that it'll be easier to use. Wrong.

I'm NOT making such assumptions; to the contrary - as a technical and cotrol freak rather than a creative videographer, I'm excited with the level of complexity this camera presents. I like it difficullt!

But a common sense tells me that while it indeed can do more (not necessarily easier), it should be doing the easy and obvious stuff no worse than other, lower-end cameras.

If you like analogies, a 1001 HP Bugatti Veyron can perform the same way as an average, 95 HP family car, but not vice versa. This camera even has the green button; never tried it but I'm sure that other than with perfect lighting, it cannot produce as good pictures in the "idiot-proof" mode as HC1 can, due to the effect I'm talking about. Do you think it's just OK?

Benjamin Eckstein · March 5th, 2008, 05:52 PM

Maybe for the sake of everyone's ego, temper, etc. this thread should be closed? Chris? It seems all points have been made and there is just a lot of bashing.

Bob Grant · March 5th, 2008, 06:34 PM

Quote:

Originally Posted by Piotr Wozniacki

Snip...
Bob if your intention has been to address my problem by drawing a parallel then This camera even has the green button; never tried it but I'm sure that other than with perfect lighting, it cannot produce as good pictures in the "idiot-proof" mode as HC1 can, due to the effect I'm talking about. Do you think it's just OK?

Yes, I think it's OK.
And I think that's what many of us have been trying to tell you.
That's why I'm glad I can and still do still use a PD170.
Perhaps one day I'll truly master the EX1 and I will not need to use a 170 anymore.

Should I point out that the Veyron comes with a selection of keys?
You don't give the red key to your teenager children.

Nick Williams · March 6th, 2008, 01:21 PM

Hi,

It looks like this may be a discussion that I really have no business being involved in, but the title of the thread seemed appropriate for my question.

I shot some timelapse video this morning of the sunrise with an ex1 using some settings I downloaded from another thread here, I believe from Bill Ravens-

PP3: TC2 C1

To put it simply, I was curious to see what kind of gradient I would get as the sun came up. When I was playing back the footage on my macbook pro I noticed if I tilted the screen to an extreme degree, I saw 3 very distinct levels of color rising up from the mountains. But I really could only see that clearly by tilting my screen. I opened the file on my macpro and it definitely wasn't as prominent on this monitor, but I knew it was there.

My question is, is there something that can be adjusted to make the transition between these brightness levels maybe a little more gradual?

To show you guys I've adjusted the levels to make it easier to see what I'm referring to:

the adjusted movie clip here:
http://www.alienbedroom.com/timelaps...e_adjusted.mov

original movie clip:
http://www.alienbedroom.com/timelapse/timelapse.mov

any info on this would be appreciated! or I'll happily move to another thread too.

thanks
nick

Steven Thomas · March 6th, 2008, 01:26 PM

Nick,
Using time lapse, this has been brought up before. I'm not sure why there's banding, but it's been seen before in the time lapse footage. I image it has to do with bit rate, anyone?

Bob Grant · March 6th, 2008, 05:51 PM

Quote:

Originally Posted by Steven Thomas

Nick,
Using time lapse, this has been brought up before. I'm not sure why there's banding, but it's been seen before in the time lapse footage. I image it has to do with bit rate, anyone?

Probably more to do with bit depth. This is an ongoing problem even without a camera when you try to wrangle images created with large bit depths into video with lower bit depths. I've seen some really ugly results from gradual gradients created in Lightwave when the sequence was bought into a NLE. The camera is trying to wrange 14bits per channel RGB into 8 bits. Banding and colour shifts at the bottom and top end of the range are to be expected.

What I suspect could be happening is the camera has been designed as a 10 bit system for the HD SDI output. The easiest way to get from 10 bit to 8 bit is to simply discard the two LS bits. This is does leave some issues that can be solved by dithering however dithering is adding digital noise, the last thing you might want in a mpeg-2 recording system.

We do have a SD SDI CRT monitor, I haven't seen how the SD downconvert from the EX1 looks on that but someone who has tells me it looks fantastic. It might be interesting to compare the signals coming out the SDI port to what gets recorded.

A lot of us were sceptical that Sony were truly sending a full 10 bit signal out the SDI port. Well maybe we've got want we wanted and might to some extent be loosing out in other areas. If it really is 10bit HD SDI I can see an expensive little box on my wish list. I'm not buying any 16GB SxS cards as yet, rather put the money towards a 10bit 3rd party recorder.

March 4th, 2008, 05:06 PM	#256
Steven Thomas Trustee Join Date: Sep 2005 Location: Gilbert, AZ Posts: 1,896	Piotr, you brought this thread full circle. It was already brought up earlier in this thread that Adam saw that saturated highlights show more hue shift and harsh clipping than he'd like. Adam also mentioned he would explore it more later. Piotr, I don't believe no one disagrees that it can look bad when it clips. I believe most do not care since they are not having an issue. If it clips, we back off. Here again is what Adam Wilt wrote in his review: "I’ve found that the EX1’s knee does a fine job except when highlights are strongly colored. Saturated highlights show more hue shift and harsh clipping than I’d like. I’m exploring this further out of curiosity, but even if the knees were perfect I would still shoot with cine gammas, because I prefer the progressive compression to the look of a traditional knee" Last edited by Steven Thomas; March 4th, 2008 at 09:54 PM.

March 5th, 2008, 09:15 AM	#260
Piotr Wozniacki Inner Circle Join Date: Aug 2006 Location: Poland Posts: 4,086	Tom, You're right about the cast. To prove it doesn't matter, I have posted above another grab with totally different colours (more red than cyan), and with the sky area only in one corner (as opposite to the previous ones where the sky dominated the scene). Still the same behind the trees... I am totally aware the white balance (or colour matrix) is not neutral in either of the grabs, but - being opposite to each other - it shows my point that you cannot avoid the abrupt clipping with bluish patches (unless carefully watching and adjusting your exposure just for that, which is not practical). And one more thing that needs stressing: this phenomenon - while easily explainable using laws of physics - is in fact very annoying and ugly. In other cameras it's handled in a fashion more pleasing to the eye. Why it is different on the EX1 beats me; unfortunately - while a bit safer like Adam says - the Cine gammas do NOT completely get rid of it at all. In fact, I just did some more tests with the safest one, the Cine2 (said in the manual to safely compress the highlights) - and while indeed the picture never even touches 100 (the zebra at 100 never shows up), the trees against the overcast, grey sky still create the artefacts around them. I'd like to kindly ask knowledgeable people like Bob, Bill or Alexander to stop preaching that this is normal, and instead post their own examples of the phenomenon. I can't understand how people can leave with this - the clipping algorithm implemented in the EX1 is flawed IMO. It just seems unable to clip precisely around the very edges of darker (unclipped) objects, which are instead surrounded by ugly patches of unclipped sky... If this is how "broad dynamic range" is supposed to manifest itself, than no thanks - I prefer narrower range as in the V1E, or any other prosumer camera for that matter. Yes I know the EX1 is capable of producing stunning pictures - I have shot lots of them as well, but only in very favourable or controlled lighting conditions (outdoors, with foreground lit generously against dark blue sky, or the opposite extremum: indoors, with low but controlled lightning). But in such conditions, even the HC1 I used to have is able of creating breath-taking imagery - and we're talking CineAlta here! Unless my camera is indeed faulty, which only some examples from other people could help me to determine. __________________ Sony PXW-FS7 \| DaVinci Resolve Studio; Magix Vegas Pro; i7-5960X CPU; 64 GB RAM; 2x GTX 1080 8GB GPU; Decklink 4K Extreme 12G; 4x 3TB WD Black in RAID 0; 1TB M.2 NVMe cache drive Last edited by Piotr Wozniacki; March 5th, 2008 at 03:14 PM.

March 6th, 2008, 01:21 PM	#268
Nick Williams New Boot Join Date: Mar 2007 Location: Salt Lake City, Utah Posts: 19	Hi, It looks like this may be a discussion that I really have no business being involved in, but the title of the thread seemed appropriate for my question. I shot some timelapse video this morning of the sunrise with an ex1 using some settings I downloaded from another thread here, I believe from Bill Ravens- PP3: TC2 C1 To put it simply, I was curious to see what kind of gradient I would get as the sun came up. When I was playing back the footage on my macbook pro I noticed if I tilted the screen to an extreme degree, I saw 3 very distinct levels of color rising up from the mountains. But I really could only see that clearly by tilting my screen. I opened the file on my macpro and it definitely wasn't as prominent on this monitor, but I knew it was there. My question is, is there something that can be adjusted to make the transition between these brightness levels maybe a little more gradual? To show you guys I've adjusted the levels to make it easier to see what I'm referring to: the adjusted movie clip here: http://www.alienbedroom.com/timelaps...e_adjusted.mov original movie clip: http://www.alienbedroom.com/timelapse/timelapse.mov any info on this would be appreciated! or I'll happily move to another thread too. thanks nick Last edited by Nick Williams; May 28th, 2008 at 12:02 PM.

March 4th, 2008, 09:53 PM	#257
Tom Roper Trustee Join Date: Jun 2004 Location: Denver, Colorado Posts: 1,891	Piotr, You mention that the phenomenon is observable when auto-iris is on. Can it occur if you used auto-shutter, or just auto iris? In other words, is the effect related to highlight clipping at the upper end of exposure, or possibly diffraction, which could happen with a small aperture but would not happen from a fast shutter? I hope my question is not confusing. One thing I'd like to understand, is if the problem could be avoided by using a fixed aperture with shutter priority, instead of the other way around.

March 5th, 2008, 11:28 AM	#261
Steven Thomas Trustee Join Date: Sep 2005 Location: Gilbert, AZ Posts: 1,896	Again, I bring up Adam said this in his ORIGINAL review. It was brought up earlier in this thread maybe three times! Why bash on others for your problems. It's not ours. Even Adam himself said to stick with the Cine curves during those shots. If I saw this over exposure condition happening when composing the shot, I would readjust for the shot just like adjusting everything else, focus, aperture, focal length...ect..

March 5th, 2008, 01:19 PM	#263
Steven Thomas Trustee Join Date: Sep 2005 Location: Gilbert, AZ Posts: 1,896	Piotr, Have you happened to notice that you are really the only one complaining about this? You wrote: "Bob, Bill or Alexander to stop preaching that this is normal, and instead post their own examples of the phenomenon." Maybe you're right, I'll let them speak for themselves, but I'm certainly under the impression they are not having issues, therefore, not going on with this rant.

March 5th, 2008, 04:02 PM	#264
Bob Grant Trustee Join Date: Nov 2005 Location: Sydney Australia Posts: 1,570	As I suspect I'm one of the mentioned parties all I can say is the request to post our own examples borders on the absurd. I can make an aircraft fall out of the sky without exceeding any of the acceptable control settings but it's not something one does more than once. This is why aircraft and many complex pieces of technology have an "envelope" of operation. It's why simply memorising a flight manual doesn't qualify you for a pilots licence, it's why memorising a camera manual doesn't make you a cameraman. It is why experience is everything. Fortunately with a camera mistakes aren't as bad as they are with aircraft, automobiles and guns. The same logic still applies though, make mistakes by going outside the envelope and learn from them, yes there's not a lot published about where the limits of the envelopes are for each camera but then again we don't have air safety investigators studying smouldering piles of twisted metal like we do with bad images. So we have to learn from our own experiences. Last night I shot a performance with my EX1, without even downloading the clips I know a lot of it is going to be crud. The camera did nothing unpredicatable, I operated it as well as it could be, the camera (heck any camera) simply could not cope with the flickering lighting, the very video system itself could not cope. For the record ultimately this was my mistake. From experience I now know that I should never of even agreed to attempt the shoot. Will I make the same mistake again, probably, I'm a sucker when a friend asks me to do something. Will you find me here posting examples of the crud I shot with 'please explains', good grief no. So what would a professional have done if asked to shoot this. He would have insisted on two trucks and a crew of 10 to light and shoot this. Now he might have gotten away with it without that, just as I might have too. But I didn't and that's the difference between me and a professional. I take the risks and sometimes fail spectacularly, professionals don't, and I and the professional bill the client accordingly. Now I know what a certain someone is thinking. "I didn't have these problems with my other camera". Well for sure. Learn to fly a 747 and you'll likely never have to worry about compressor stall like you will in a F18. That's why pilots get licences that are "qualified" for certain aircraft. It also shows that things built to extract more performance are also riskier to use, they take more experience to learn the dangers of, they are not easier, they're harder to work with. It seems to me that the underlying false assumption is that because this camera has 'more' of this and 'more' of that it'll be easier to use. Wrong. It might also explain why camera manufactures haven't unleashed the beast in their cheaper cameras in the past.

March 5th, 2008, 05:52 PM	#266
Benjamin Eckstein Major Player Join Date: Jan 2007 Location: Boston, Massachusetts Posts: 380 Images: 1	Maybe for the sake of everyone's ego, temper, etc. this thread should be closed? Chris? It seems all points have been made and there is just a lot of bashing.

March 6th, 2008, 01:26 PM	#269
Steven Thomas Trustee Join Date: Sep 2005 Location: Gilbert, AZ Posts: 1,896	Nick, Using time lapse, this has been brought up before. I'm not sure why there's banding, but it's been seen before in the time lapse footage. I image it has to do with bit rate, anyone?