DSLR's and Rolling Shutter

[Peter Moretti]

I have read that the current crop of DSLR video capable cameras having significant rolling shutter issues.

While I certainly believe it, I am scratching me head. I've never seen rolling shutter in a still camera picture. Such distortion would be completely unacceptable for motion photography.

So why/how is it showing up in the video from these cameras? Isn't their video just a succession of still frames?

I'm sure I'll have a "light bulb" moment eventually on this issue, but I'm not there yet.

THANKS.

[Soeren Mueller]

Well... basically it just shows up because for 24 or more frames per second you have to "shovel the data away from the sensor" quite fast - with still frames you have more than enough time for that. Even on previous "top of the line" cams that could to short "burts" of 4 or 5 fps stills in high quality there was not this much of a timing issue.
Now with 24/25 or more frames per second this becomes an issue as you have to read out all of the data quite fast and leave enough time for the next frame to be exposed/"digitized"...

But I guess there are others on here (especially from the DIY-HD-Cam section ;o) that can explain the issue in more detail...

[Jon Fairhurst]

It's simple really.

Imagine somebody running across the screen from left to right. Fast. Usain Bolt fast.

You capture the top of the screen right away. His head is at the left. You capture the middle of the screen later. His body is in the middle. You capture the bottom of the screen later still. His golden Puma shoes are at the right of the screen.

Now imagine that this shutter rolls smoothly from top to bottom. The result is that poor Usain is leaning backwards as he runs forward.

You've just turned Usain Bolt into jello.

[Stephen van Vuuren]

Originally Posted by Peter Moretti

I'm sure I'll have a "light bulb" moment eventually on this issue, but I'm not there yet.

THANKS.

This is really a great question and gets to the heart of the issue with these cameras. You would be correct that rolling shutter would be a disaster for still photos. However, these cameras are not using the same process to record video as they do for shooting films. Instead of shooting a sequence of still JPEGs (which you are correct, would have no rolling shutter), they are recording the video via live view mode. Live view mode is electronics that send a sensor readout to the LCD panel on the camera and is not the same path as sensor path as the JPEG/RAW writing.

So, since these are CMOS sensors, they rely on existing video shutter technology for CMOS - i.e. a rolling shutter which has the artifacts. Evidently, global shutters for CMOS are not possible price-wise etc.

Now, in burst mode, as the camera is writing a sequence of stills, there is no need for the rolling shutter, as I understand it, so no artifacts.

Obviously, some parts of the path are the same as some camera effects are applied both to stills and video. But the lack of manual controls etc. points to a split somewhere - interesting to know exactly where.

Evidently, CCDs are the only sensor that allow this (see the Ikonoskope camera) right now. But I wonder if you could not make a CMOS camera today that would shoot a "burst mode" of JPEG stills at 720p and bypass the rolling shutter. Several DSLRs do this at 3 - 5 fps until your card is filled at full resolution already. So why not half resolution at 24fps - seems math-wise that a half-resolution JPEG is one-quarter of the data, so 6fps times 4 - 24fps.

Of course, they might think people would freak at the image sequences instead of single file but a simple utility could convert lossless to MJPEG avi.

We need some camera engineer types to chime in here though.

[Evan Donn]

when shooting stills DSLRs use a mechanical shutter much like a traditional film camera. This isn't practical when shooting video because of the speed required for 24+ shots per second (hence the limited still burst speeds) and because you'd really quickly burn through the shutter's duty cycle. So in video mode the mechanical shutter is locked open and an electronic "shutter" is used which scans & resets the sensor from top to bottom as described above resulting in the "roll".

[Stephen van Vuuren]

Originally Posted by Evan Donn

when shooting stills DSLRs use a mechanical shutter much like a traditional film camera. This isn't practical when shooting video because of the speed required for 24+ shots per second (hence the limited still burst speeds) and because you'd really quickly burn through the shutter's duty cycle. So in video mode the mechanical shutter is locked open and an electronic "shutter" is used which scans & resets the sensor from top to bottom as described above resulting in the "roll".

That makes sense on one level as mechanical shutter would be off in live view but I don't think the burst speed limit is shutter related - it's write time to storage (thus raw much slower than JPEG etc.). Considering some DSLR can fire off bursts of 100 shots at very fast shutter speeds, 24 shutter fires second at 1/48th shutter would tax 5D shutter.

Mechanical shutter life, though would be an issue, 150,000 shutter fires is only about 100 minutes at 24fps.

But the solution here is better electronic shutters anyway since it's video we are talking about, not film. I can't imagine using the mechanical shutter for DLSR video - that would be a film camera :)

They are using an electronic shutter for video to adjust exposure etc - it just needs to be improved and re-routed in the control path system so it has manual control.

[Peter Moretti]

Thank you SO MUCH for all your help.

BTW, IMHO, it seems that using a record buffer could solve this problem in camera.

If you know the rate of the offset due to rolling, a properly aligned frame could be reconstructed from data recorded previously and afterwards.

An electronic vertical test stripe could even be sent to the sensor to aid the alignment.

[Jon Fairhurst]

Originally Posted by Peter Moretti

If you know the rate of the offset due to rolling, a properly aligned frame could be reconstructed from data recorded previously and afterwards.

That would be quite a trick.

Consider if I throw a ball against a wall. One frame shows the ball going towards the wall. The next frame shows it in the same place, traveling away from the wall. The camera has no idea that the ball hit the wall between the frames.

Or consider that I throw a ball into a baseball glove. In one frame we see the ball. In the next frame we see a closed glove and no ball. It's occluded.

BTW, film has a rolling shutter, but it's very fast. RED claims that they can now get the read-reset times to be as fast or faster than film. So for RED, anyway, rolling shutter should be a non-factor.

I've never heard anybody say, "I would shoot on 35mm film, but rejected the idea because of the rolling shutter."

Note that 35mm movie film also uses a mechanical shutter, but it's a rotary mechanism, rather than the back and forth mechanism of a still cam. ...and the Mazda goes "mmmm."

[Peter Moretti]

Originally Posted by Jon Fairhurst

That would be quite a trick.

Consider if I throw a ball against a wall. One frame shows the ball going towards the wall. The next frame shows it in the same place, traveling away from the wall. The camera has no idea that the ball hit the wall between the frames.

Or consider that I throw a ball into a baseball glove. In one frame we see the ball. In the next frame we see a closed glove and no ball. It's occluded.
...

That's why you'd have buffer for data recorded after and before the frame. It would contain enough between frame data to reconstruct the frame properly.

For instance, my Sound Devices recorder has a before buffer of a few seconds. It's not part of the harddrive or compact flash. When it's active, I can record sound that ocurrs before I press record.

IMHO, I think it would be plausible for a camcorder to record handles (info before and after the actual frame) to a buffer. This info can then be used to reconstruct a proper frame. (Unless I'm entirely missing something.)

But using a sensor with a faster top to bottom scan is obviously a more direct approach.

[Chris Medico]

Originally Posted by Stephen van Vuuren

That makes sense on one level as mechanical shutter would be off in live view but I don't think the burst speed limit is shutter related - it's write time to storage (thus raw much slower than JPEG etc.). Considering some DSLR can fire off bursts of 100 shots at very fast shutter speeds, 24 shutter fires second at 1/48th shutter would tax 5D shutter.

The mechanical shutter mechanism can't do this in a DSLR. There is more moving than just a set of shutter vanes. You will either have a mirror or a prism that has to swing up out of the way for the exposure cycle due to the autofocus system being in the light path to the eyepiece and not for the imager. In between each exposure the mirror/prism swings down and focus is rechecked then it swings up before snaping another photo. My 40d does this 6.5 time a secon and I am amazed this is even possible without tearing something apart in a short time.

The shutter in a movie camera isn't like this at all and is why they can operate at much higher speeds and have a longer lifespan.

[Luiz Murillo]

And lets not forget that film cameras(Panavision/Arri/Aaton shit) go to preventive repair every time they get back from a rent, so, in some cases, it will film 3 hours and stay 3 hours on repair. And that would not be nice in a 'home camera'.

[Steve Phillipps]

Originally Posted by Luiz Murillo

And lets not forget that film cameras(Panavision/Arri/Aaton shit) go to preventive repair every time they get back from a rent, so, in some cases, it will film 3 hours and stay 3 hours on repair. And that would not be nice in a 'home camera'.

That's not neccessarily true for all film cameras. Most freelancers with Super 16 kits for example would probably have their kit serviced once a year, maybe not even that. They're just built really well and that's why an Arri Sr3 body and a couple of mags will set you back as much as the latest HD cameras (around £40,000 or so) even though it's basically just a box with a hole in it!
Another expensive piece of kit is the Phantom HD camera, which has a rolling shutter, and presumably a very very good and expensive one as it'll go at 1000fps. At £100,000 or so for the camera, presumably they can afford to spend a fair bit on getting a really quick rolling shutter in there - never seen any specs on the shutter though.
The RED camera also appears to have a pretty decent rolling shutter by all accounts. I suppose on a £3,000 camera you just have to accept a rather cheap shutter.
Steve

[Steve Mullen]

Originally Posted by Peter Moretti

I've never seen rolling shutter in a still camera picture. Such distortion would be completely unacceptable for motion photography.

So why/how is it showing up in the video from these cameras? Isn't their video just a succession of still frames?

Yes, video is a series of "frames" but the rolling shutter artifact is inherent IN the "frame." The amount of the artifact is a function of how fast a CMOS chip can be read-out.

The total time to read-out a frame is a function of the speed of each read-out port and the number of ports.

The speed of each port, is determined by how fast the part is. The faster a chip runs, the greater the power consumed and hence the more heat that is generated within the part. Therefore, a still camera taking a burst of stills creates a certain amount of heat. The chip can cool while the frames are written to memory from a buffer.

With video, there is no opportunity for the chip to cool. So the maximum clock-rate must be limited. (Unless it is the unique Sony EXMOR chip used in the Casio F1.) This limitation forces video to have a slower read-out than for stills.

In the case of the D90 -- the read-out is way-too slow hence the horrible rolling shutter artifact. Even with the too slow read-out, the D90 chip still builds-up heat -- hence the limitation on shooting time. (Generally Sony supplies chips to Nikon, but it seems not to have supplied their best. Or, as I suspect, Video was a last minute hack by Nikon when they learned of the Canon. So the chip wasn't chosen for continuous shooting.)

[Jon Fairhurst]

This classic photo tells it all:

http://static.photo.net/attachments/...r-34153284.jpg

The camera is moving with respect to the background; hence, the people lean to the left. But it is not moving as fast as the car; hence the wheel leans to the right.

The exposure time in this photo is actually fairly quick. See how the number "6" is not blurred.

Each "row" of the image was exposed quickly, but the top row was exposed long before the bottom.

[Kurth Bousman]

We can hope that the 2nd gen dslrs' will be better at video . We should have at least three more this year at least . A 720p Rebel coming possibly next week and a Nikon d400 with 1080p coming soon , and the already announced (but not dslr) panasonic G1 hd ,which the guess is will be 1080 avchd . The Rebel is rumored to have digicV probably to help remedy this very problem . Also I can't imagine that Canon isn't working a XLseries pro camera with a FF sensor and EOS mount . And I can't imagine that Casio isn't working on a Scarlet killer for 1/2 the price . The next year should put an end to the rolling shutter problem with faster read times off the sensor and faster processors . It's going to be very cool. It's great that the early adopters have warmed up the waters . Jean Cocteau said when making films is as cheap as drawing , than it will finally be an artform. We are getting closer and closer , and in the process making obsolete 35mm adapters , which any but the diehard find just too cumbersome.