"Rolling Shutter" Skew Tests and Grabs

[Eric Pascarelli]

Here's the movie.

It's highly compressed, but the timing is correct and the frames are intact.

It's the version with a 1/60 shutter.

[Bob Diaz]

Eric,

Thanks for the quick answer.

I did some thinking and I believe that it's possible to test the rolling shutter speed without test equipment.

If the EX-1 is set to a 1/120 or 1/60 shutter speed and you shoot a wall that is lit with a florescent light (powered by 60 Hz AC) and no other lighting, the wall should look evenly lit.

Now set the shutter speed to 1/2000 of a second and the wall will appear to have 2 horizontal bars (light --> dark --> light --> dark). The bars should appear to move slowly upward and wrap around. It should take a bit more than 8 seconds for one bar to move up off the top of the screen and appear on the bottom of the screen.

If the rolling shutter is always 1/60 (1/59.95) of a second, you should see the effect at 60p, 30p, and 24p on any still image.


WHY THIS WORKS:

Florescent lights flash at twice the AC Power Frequency. In the case of 60 Hz (60 Cycles Per Second), the lights flash at 120 Hz. This is because the AC power starts at zero volts, rises to the peak +voltage, drops back down to zero volts and reverses direction, rises to the peak -voltage in the reverse direction, and drops down to zero volts again. The light output of the florescent light follows the voltage at that moment.

When the shutter speed on the EX-1 is 1/2000 of a second, the rolling shutter is 32 scan lines high at 1080 and 21 (22 ?) scan lines high at 720. This slit moves down the CMOS image sensor in 1/60 of a second. As the florescent light flashes, the rolling shutter is moving down, so the current brightness of the light is recorded to the frame.

There is a slight mismatch to the frequency of the scan of the camera 59.95 Hz and the AC power 60 Hz. Thus it takes about 8.3 seconds for the 1/2 cycle to shift through the image.


50 Hz POWER:

I noticed that in the video, "South Bank" by Phil Bloom, that at 1:22 to 1:29, that the area to the right of the image, lit by a florescent light, seemed to have an odd flicker.

I know that the footage was shot at 25p, but I don't know what shutter speed Phil selected. I would think that a shutter speed of 1/50 of a second or 1/100 of a second would hide the flicker, so maybe the shutter speed was something else.

To me, it seems likely to me that the rolling shutter speed is 1/60 of a second even in 50p and 25p mode.


I don't have an EX-1 to work with yet, so posting a copy of the movie is very helpful. Thanks...


Sincerely,

Bob Diaz

[Eric Pascarelli]

Bob,

That all sounds right to me (I'll trust your math).

I did not test any of the "PAL" frame rates but it makes sense that the EX1 would scan at its fastest speed in all modes to minimize skew as much as possible.

As I am sure you know, PAL is 25 or 50 fps for real and not the horrible decimals we are stuck with (29.97 or 59.94) so PAL framerates won't "beat" with European line frequencies.

[David Lorente]

Originally Posted by Bob Diaz

(...)
50 Hz POWER:

I noticed that in the video, "South Bank" by Phil Bloom, that at 1:22 to 1:29, that the area to the right of the image, lit by a florescent light, seemed to have an odd flicker.

I know that the footage was shot at 25p, but I don't know what shutter speed Phil selected. I would think that a shutter speed of 1/50 of a second or 1/100 of a second would hide the flicker, so maybe the shutter speed was something else.

To me, it seems likely to me that the rolling shutter speed is 1/60 of a second even in 50p and 25p mode.
(...)

I've also noticed this flickering on Phil's video. Keep in mind that the shot we're talking about is slow motion, so the frame rate of the camera was not 25 fps. I think that in this particular shot the camera was overcranked to 60 fps.

If we divide the 100 Hz power cycle by 60 fps, we have a 5 to 3 relation. If I'm not wrong (again), this means it should take 3 frames for the flickering bars to repeat the same pattern, so it is quite annoying :(

I remember I saw a similar effect when I shoot in the projector room of a theater. They let me get in to shoot the film projector at work. I did some close-ups of the film passing by the gate, just in front of the shutter, and there was a noticeable flickering because of the film 24 fps rate vs PAL 25 fps, but, being a CCD camera, it was the whole image who flickered, going light and dark, instead of having rolling bars. If I had shot it with the EX1, I would have set the frame rate to 24, and the footage would have looked just nice.

So I think we would see similar problems with any kind of video camera, if we shoot something with a flicker rate close, but not matched, to the frame rate of the camera.

[Paul Joy]

Originally Posted by David Lorente

So I think we would see similar problems with any kind of video camera, if we shoot something with a flicker rate close, but not matched, to the frame rate of the camera.

From my experience with the camera the flicker you mention only appears when the framerate is not equal to the AC Frequency. For instance if I overcrank to 25/60 I see horizontal banding, but at 25/50 there is none.


Does anyone remember those first leaked shots of the EX1 from the trade show? That effect was one of the first things we all noticed. Now that I can control it, I actually quite like the effect it has when it only happens on portions of the image such as a shop window. I used it at the end of this sequence...

http://www.vimeo.com/604159

regards

Paul.

[Eric Pascarelli]

If you narrow the shutter substantially you will see banding even if the frame rate matches the mains frequency. The bands won't move, but they will be there.

[David Lorente]

Of course, if you raise the shutter speed, those bars will appear. So... don't do it!

As I said before, I don't mean we should ignore the rolling shutter, of course it's going to cause problems under certain circumstances and it would be better if those CMOS sensors worked like the CCDs we are used to, but I just think there are workarounds to avoid these problems, and there are always things that can't be done with a particular technology, so nor CMOS neither CCD cameras are perfect. For example, with a photographic camera with curtains shutter, you can't use a flash gun at shutter speeds higher than the X sync speed, because you would get a partial exposure. A central shutter camera would solve this, but central shutters can't achieve speeds higher than 1/500th second. Nowadays, almost all pro SLRs feature curtain shutters, and only very specific brands (like Hasselblad) still produce central shutter cameras.

So, what we have to do is encourage Sony to raise the X sync speed of their CMOS sensors. This will also lead to higher overcranking speeds! I think a rolling shutter of 1/250 and a maximum overcrank of 125 fps would just be fine... :)

[Bob Diaz]

Originally Posted by David Lorente

I've also noticed this flickering on Phil's video. Keep in mind that the shot we're talking about is slow motion, so the frame rate of the camera was not 25 fps. I think that in this particular shot the camera was overcranked to 60 fps.

If we divide the 100 Hz power cycle by 60 fps, we have a 5 to 3 relation. If I'm not wrong (again), this means it should take 3 frames for the flickering bars to repeat the same pattern, so it is quite annoying :(

I remember I saw a similar effect when I shoot in the projector room of a theater. They let me get in to shoot the film projector at work. I did some close-ups of the film passing by the gate, just in front of the shutter, and there was a noticeable flickering because of the film 24 fps rate vs PAL 25 fps, but, being a CCD camera, it was the whole image who flickered, going light and dark, instead of having rolling bars. If I had shot it with the EX1, I would have set the frame rate to 24, and the footage would have looked just nice.

So I think we would see similar problems with any kind of video camera, if we shoot something with a flicker rate close, but not matched, to the frame rate of the camera.

(Bob hits his head when he realizes that Phil's footage with the flicker was over cranked.) Boy, I was so busy looking at the flicker, I didn't look at the speed of the people walking.

Even with CCD cameras, if you use a very short shutter speed and florescent lights, you are going to have a flicker problem. In the areas with 60 Hz power and 59.94 Hz frame (or field) rate the flicker will be worse that 50 Hz power and 50 Hz frame rate.

For 50 Hz areas, there will be a VERY SMALL difference between the power frequency and the camera frequency. The difference could be 0.001% off, but over a long enough time it might show up. This would need to be a shot where one side is lit with regular light and the other side is lit with a florescent light.


AC Powered LED lights, like Christmas lights are going to pose a problem too. Most LEDs only work when the current is flowing in the forward direction and do not light when we try to pass the current in the reverse direction. Thus, they flash at 60 Hz USA, Canada, ... (50 Hz in Europe ...).

Indoor AC powered LED lighting is possible, but because of the high cost, we don't see it. Some of our traffic lights (or as we call them, "Stop Lights") in our area have switched to LEDs.

http://auto.howstuffworks.com/question178.htm

http://www.ledtronics.com/markets/traffic_index.htm

This could cause a flicker problem if the shutter speed of the camera is higher than the power frequency.


As for the battery powered LED lights, for units that dim the light, I believe that the frequency they are driven at is in the kHz range (1,000 Hz or higher). However, I have not had a chance to test this.

If I'm correct, battery powered LED lights won't pose a problem.

Bob Diaz

[Steve Mullen]

Originally Posted by David Lorente

By scanning rate, I mean the time it takes for the CMOS sensors to scan one complete frame. This is the time difference between the moment the top line is read and the moment the bottom line is read, and it is what effectively causes the skewing effect we see on the image.

Eric has pointed out that the scanning rate is always the same no matter which shutter speed you use. I'll go further saying that this scanning rate is not linked to the video frame rate. This means the scanning rate will be the same no matter you set the camera at 30, 25 or 24 fps. So, in this test we saw that the scanning rate was double of the frame rate, that is, 1/60th of second at 30 fps, but if the scanning rate was the same 1/60th of second at 24 fps, the effect would be less noticeable.

When this technology evolve in the future, they surely will avoid the scanning shutter effects by increasing the scanning rate: if the complete image is scanned, say, at 1/300th of second, nobody will notice any skewing or the other weird effects... .

Very well said -- and I concur completely. Today I was updating the description of CMOS technology in my V1 book to include EXMOR technology. After a good deal of thought I think I understand EXMOR and how it differs from 3ClearVid. If you read what Sony writes, EXMOR uses a all digital output function based upon an A/D per column. Decreased noise is what Sony stresses is the benefit.

Fundamentally, EXMOR implements an on-chip digital output system that is time decoupled from the on-chip analog system. Chip read-out time (R) is decoupled from exposure time (T). Therefore, as exposure times increase longer than R, the rolling shutter artifact is reduced over an un-decoupled all-analog CMOS chip such as 3ClearVid.

The rolling shutter artifact is created at even moderate shutter speeds by most CMOS-based cameras because "scanning time" is equal to the exposure time. Thus, as CMOS rows are reset, from top-to-bottom, the row above the last reset row is read-out. The result is a one-row “slit” that travels down a “synchronous” CMOS chip. For example, at 24p where the shutter-speed has been set to 1/48th second, the “scan time” is 1/48th second and R (read-out time) is 1/96th second.

The key to reducing the rolling shutter artifact is “asynchronous” CMOS chips that decouple R from T. Ideally, R can then be reduced to a fraction of most T values as is the case with film cameras where the shutter opens and closes very rapidly. Bottom-line, the greater the ratio between a current T value and the chip’s R value, the better.

For example, if the time (R) required to read-out all rows is fixed at 1/120th second (0.00833)—a scan time of 1/60th second—then 24p operation with a shutter-time of 1/48th second (0.020833) will have a higher T/R ratio (2.5 verses 2.0) than would operation where R is 1/96th second (0.01042).

OK -- a ratio of 2.5 isn't that much better than 2.0, but it IS better which supports a claim that EXMOR "can" help reduce rolling shutter.

Note, the EX1 offers Generation 1 EXMOR. Imagine an EXMOR chip that provides four digital output ports just as 3ClearVid now offers four analog ports. Scan time will drop to 1/250th and the ratio will jump to 10.0. Even at 1/60th second we will see a major reduction of rolling shutter.

So rather than see the world as CMOS for consumers and CCD for pros, I think it makes more sense to see the EX1 providing the first iteration of CMOS chips for "pro cameras slightly under $10K." If I'm correct, second or third generation EXMOR chips will offer the benefits of both CCD and CMOS technology and replace CCDs on more expensive pro cameras.

Sony's ability to develop its own imaging chips following a long-range strategy is one of its major capabilities.

[Bob Diaz]

Funny how looking backward, helps me to look forward.

Many years back at the CES Show, I remember seeing the first demos of consumer color cameras. At that time, there were two technologies, CCD and MOS.

The sales people showing off the cameras didn't really understand the advantage of solid state image sensors over tubes; lower power, ability to stand greater shock, lighter weight, smaller size, higher reliability, and no high voltage needed.

The only thing that mattered to them was, it didn't do low light as well as tube cameras. A true statement at that time, but we all know that over time, this issue was solved. However, I recall many professional video people preferred tube cameras over solid state, because tube cameras had a "richer image".

Well, CCD was able to reach lower light levels better than MOS, so CCD took over and MOS seemed to come to a dead end. Funny how it came back from the dead as an improved CMOS version.


So now we are at a point where there are trade-offs between the two technologies, but I believe that Steve has hit upon a key point on future improvements. So a faster "shutter" will reduce the impact of the "rolling shutter" and we are VERY likely to see more CMOS improvements in the upcoming years.


This is why I like following technology, it's always getting better and you never know what improvements will come next.


Bob Diaz

[Ali Husain]

one thing i noticed(*) is that the motion blur on the ex1 has a strange triplicate-image look... as if someone took three exposures in quick succession then merged them together. pretty gross looking to me. nothing like the hvx or even the dvx which were all smooth and creamy. i think i'm probably the only one who's more annoyed by the ex1's motion blur than its rolling shutter distortion. :)

edit:
---
(*) in my limited use and non-scientific testing! i could have noticed a standard exposure issue with flickering lights or have misadjusted something.

[Eric Pascarelli]

Ali,

Do you have a still or clip you can upload?

I really don't think the EX1 does motion blur any differently from any other camera - it's all about the accumulation of photons through an open shutter under motion, so it would be hard to create a visibly different look (for the blur itself, assuming the exposure and other aspects of the image are similar) without really trying. I think side by side tests would prove this out.

Perhaps you saw some motion blur mixed with some ambient flickering light source (such as a distant fluorescent) that created the look of multiple exposures? Or perhaps it's an unusual codec related issue in the footage you are referring to?

Having scrutinized single frames of motion picture film under a magnifying glass as part of my profession for many years, I find the motion blur on the EX1 to be quite satisfactory (because it's similar to film).

I'm not so thrilled with the scanning shutter issues, however.

[Ali Husain]

i've had far more limited use of the ex1 than i've had with the dvx100 or hvx200. and i haven't yet compared them under the same conditions. there's certainly a chance that there were issues in the lighting, or that settings i used made a difference. i've had a few years experience examining stills from video, but my experience with film stills is only second hand. i'm thinking it would be wise to defer to your experience. strange though that the hvx/dvx cadence, in my memory, looks more, to use the precise scientific term :), organic. perhaps there is some duty cycle differences with the exposure?

[Steven Thomas]

I've stepped frame by frame with the EX1, it looks like typical motion blur.

If you are seeing something, please post an example original.

[Eric Pascarelli]

Originally Posted by Ali Husain

perhaps there is some duty cycle differences with the exposure?

The amount of blur per frame depends on frame rate vs. shutter angle, both of which are very adjustable on the HVX and the EX1. So, yes, if they were set differently their motion blur would be different.

Also, speculars and other clipping whites will have a harder edge to their blurs than things that are well within the exposure range. That's because, even under motion, they are bright enough to saturate the sensor from the moment it's exposed.