"Rolling Shutter" Skew Tests and Grabs

Eric Pascarelli · January 14th, 2008, 08:43 PM

Yes, I will. Give me a day or two.

Charles Dasher · January 16th, 2008, 10:08 PM

I got my EX back from Sony and the re alignment seems to have solved the vignette issue. But having read a fair amount about the rolling shutter effects I am considering an attempt to return my camera.

I primarily film action sports such as kitesurfing, surfing etc. My fear is that capturing this type of action it will be difficult to avoid quick panning movements or high speed images flying through the frame.

I was shooting some paddle surfing this morning and in one shot I quickly panned from one rider to another. The building in the background skewed so far over it was truly alarming.

I did one test where I locked off a shot directly across the street to see how the cars looked as they drove through the frame. At 35 mph the skew was acceptable but often I am shooting faster objects coming past the camera.

There are many things I love about the EX1. Larger chips, DOF, image quality, lens control, low light capabilities and image control.

I am moving up from the PD 170 CCD world and have waited a long time to make the jump. I am concerned with what I am seeing. I was not interested in HVX 200 but I am uncertain of where to turn.

Any advice is greatly appreciated

Tim Polster · January 16th, 2008, 11:36 PM

This is alarming.

The EX looks like a great value except for this rolling shutter behavior.

The next option imho is the HPX-500 or the XDCAM 335/350.

But they will bring the price to the $20,000 range.

Makes one think, "maybe a skewed building here and there is not so bad"...

Paul Curtis · January 17th, 2008, 03:07 AM

>The building in the background skewed so far over it was truly alarming.

Rapid changes of direction will be worse, if you're just panning in one direction the skew is perhaps acceptable and the public won't notice but if the skew changes direction then it leaps out as something wrong. And we haven't really touched on vertical squash and stretch.

Im glad that we're hearing and seeing some real world examples. I think the same will happen to Red as well (perhaps to a lesser degree) and maybe SI (the sensor can do 60fps as well which is probably the scanning speed limit). It's a case of how fast can the system read the sensor. On the EX i would imagine that 720 will be the same because the camera would scan everything and downsample? Or maybe it scans different lines to get the 720 image in which case it could scan faster (but loose some image quality advantages).

I've mentioned elsewhere that im quite disappointed with sony and their marketing hype. I remember statements being made about substantionally reducing this effect that simply aren't true. Which is a shame because if you look at sonys sensor side of the business they do have great cmos technology working at high speeds. So they do have the sensors and the knowhow but somewhere between the design and implementation things don't work as advertised.

There is a reason why sony mandate that pro and broadcast gear is CCD and consumer is CMOS. So perhaps this limitation in the EX is being driven more by marketing than tech.

We have some pilot/test stuff to shoot in the not too distant future, i may hire one to see for myself and compare to the other XDCAMs rather than just buy (because everything else is so attractive about it and i can get two for the price of something else)

cheers
paul

David Lorente · January 17th, 2008, 09:20 AM

I've been reading a lot of things about the rolling shutter, some of them are valuable comments, scientific explanations, or well done tests like the one posted here by Eric. But there are also a lot of misinformation, people who fear this effect like if it was the devil, and the people who post comments having no idea at all of what they are talking about.

Well, I'll put my two cents here. From the tests of Eric, we can see that the scanning rate is double than the frame rate.

By scanning rate, I mean the time it takes for the CMOS sensors to scan one complete frame. This is the time difference between the moment the top line is read and the moment the bottom line is read, and it is what effectively causes the skewing effect we see on the image. Current CCD's don't show this effect because the whole image is stored in a memory when the integration time is complete, and then it is readed slowly, but every pixel of the image has started and has stopped storing light at the same time.

Eric has pointed out that the scanning rate is always the same no matter which shutter speed you use. I'll go further saying that this scanning rate is not neccesarily linked to the video frame rate you set on the camera. This means that the scanning rate can be the same no matter you set the camera at 30, 25 or 24 fps. Just think about the camera like a photographic camera, but taking several images by a second (that's where all started...). You have a clock that tells the camera when to capture a frame, or 'that pushes the trigger button'. This is the frame rate. So, in this test we saw that the scanning rate was double of the frame rate, that is, 1/60th of second at 30 fps, but if the scanning rate was the same 1/60th of second at 24 fps, the effect would be less noticeable. And this explains why the effect is less visible in the EX1 than in the Canon camera whose model number I can't remember and whose scanning rate was surely the same as the frame rate (1/30th of a second).

Notice that this is just what I have figured out from what I know about the technology, but it must be confirmed with new tests at the different frame rates. Tests that I will do as soon as my camera arrives, it had to be here this week, but it has been delayed and I have to way another two weeks! :(

When this technology evolve in the future, they surely will avoid the scanning shutter effects by increasing the scanning rate: if the complete image is scanned, say, at 1/300th of second, nobody will notice any skewing or the other weird effects... unless you use the camera to shot crash tests or other high speed things.

But I wan't to say that we also have to relativise this effect: in the great majority of the cases, we shot without shutter (I mean the standard shutter), so fast moving objects or camera movements will appear completely blurred like in the test frame in which the shutter was set to 1/60th. So... a test chart will show skewing, and we will notice it, or not, but in real life shots, I'm absolutely sure that people watching the footage will just be stunned with the high quality of the image, and nobody, except us, the fools with the cameras, will notice anything wrong with the images.

I don't mean it's not a problem at all: I see a problem when using fast shutter settings (slow shutter rates will blur the action) and very fast movements/moving objects. If you're going to shoot these kind of things, then the EX1 may not be the best camera for you... But for a lot of other kind of works, this camera is just perfect. It's just a matter of compromising: this camera achieves it's high performance thanks to it's new CMOS sensors, but that is just what causes its weakness, scanning shutter artifacts.

I'm becoming a pain in the neck, so I just wan't to say that I have seen this infamous effect in... my old Betacam camera! Yes, it's powered with Hyper HAD CCD's, and yes, what I'm saying will cause some discussion, but I will demonstrate that as soon as I can.

Brian Drysdale · January 17th, 2008, 09:35 AM

Quote:

Originally Posted by David Lorente

I'm becoming a pain in the neck, so I just wan't to say that I have seen this infamous effect in... my old Betacam camera! Yes, it's powered with Hyper HAD CCD's, and yes, what I'm saying will cause some discussion, but I will demonstrate that as soon as I can.

Yes, this has been around on the 2/3" CCD cameras; pan any Betacam too fast and you get the same skewing effect. Usually it's a near whip job before you start noticing, but it's there none the less.

Eric Pascarelli · January 17th, 2008, 12:29 PM

Thanks, David for that post.

I tested some other speeds, without using motion control, but just looking at two successive frames to determine the ratio of pan speed in pixels per frame versus skew amount in pixels.

1080 24p had a ratio of 2.66, implying a scan time of about 1/64 second

720 24p had a ratio of 2.25, implying a scan time of 1/54 second

720 60p had a ratio of 1.0, implying a scan time of 1/60 second

It seems like the EX1 keeps a (reasonably) consistent scan time, as David suggested rather than a proportional scan time. So the effect is less noticeable on a frame basis at 24p, but in real world situations I would guess the skew would be equally noticeable at all speeds.

I am not sure why the numbers are a bit funny - there may be a fixed component to the timing that I am not incorporating into the algebra that would make everything equal out.

Anyway, the test is easy to do and I'd love to see people confirm or refute these numbers.

Shoot a vertical target (which crosses top and bottom of frame) at 1/2000 shutter. Pan rapidly. Examine 2 adjacent frames (in Photoshop) and see how much the target has shifted in pixels. That's the pan rate in pixels per frame.

Now look at either frame alone and measure the the horizontal difference from top to bottom of the target. That's the skew amount.

Divide. That's the ratio.

Noah Yuan-Vogel · January 17th, 2008, 04:29 PM

thats interesting. so the readout rate may always be 60fps... it would make sense if the maximum readout of the sensor allows at most 1920x1080@60fps. all other framerates are probably achieved by blanking(?) and frame sizes achieved not at the sensor level, just in processing. I suppose unless it is an interlaced sensor, it would have to be able to do at least 1080@60 to come about 1080i video. this might mean the sensor is capable of shooting 60fps at 1080, maybe the DSP cant handle it? this is all speculation of course. at least it is good to know this. this should probably mean that the skew and wobble characteristics will be the same over time, when played back at the speed it was captured at. now for stuff high speed stuff thats getting slowed down, 1:1 readout ratio might be tough...

Mooho Bae · January 17th, 2008, 07:21 PM

Eric, Thank you so much again. Thank you David.

Eric, May I understand that the scan speed is very close to the RED in 24P mode?

Eric Pascarelli · January 17th, 2008, 08:17 PM

The RED, from only one measurment, seemed to do 1/72, which is a bit faster than the Ex1. But further tests are necessary. I need to check with Jim Arthurs and have him try a few speeds.

Brendan Pyatt · January 19th, 2008, 11:39 AM

Anyone used an EX1 in a helicopter? With or without a gyro?

David Lorente · January 20th, 2008, 09:22 AM

Quote:

Originally Posted by Brian Drysdale

Yes, this has been around on the 2/3" CCD cameras; pan any Betacam too fast and you get the same skewing effect. Usually it's a near whip job before you start noticing, but it's there none the less.

I'm sorry. I have to apologize, because I said a stupid thing. What I saw in my Betacam was an optical effect, probably due to the fact the viewfinder is a CRT. It can also be seen on a CRT monitor. The skew is an illusion produced because it's the CRT who generates the image as a "rolling shutter".

http://img405.imageshack.us/img405/8164/mov2000dg3.jpg

These are two consecutive fields of a weirdly fast pan, shot with a DXC-537AP dockable camera + PVV-1AP Betacam SP recorder. Shutter speed was set to 1/2000. As you can see, there's no skewing at all (but the standard Fujinon lens makes some pincushion distortion!).

Brian Drysdale · January 20th, 2008, 04:59 PM

[QUOTE=David Lorente;811241]I'm sorry. I have to apologize, because I said a stupid thing. What I saw in my Betacam was an optical effect, probably due to the fact the viewfinder is a CRT. It can also be seen on a CRT monitor. The skew is an illusion produced because it's the CRT who generates the image as a "rolling shutter".
QUOTE]

Well, you do have to bear in mind the overall system. It's only in recent years that there was any alternative to the CRT - LCDs and Plasmas can add their own little artifacts.

Bob Diaz · January 20th, 2008, 09:43 PM

Hi Eric,

I've been studying your screen grabs showing the skew and doing a bit of math based on the image. IF I've calculated correctly, in your video any point on the chart should appear for 0.6 seconds or in other words, it first appears on the right side of the screen and 0.6 seconds later it falls off the left side of the screen after 0.6 seconds (18 frames). So this would be a VERY fast pan.

I've counted an offset of 53 pixels and an angle of 2.8 degrees.

All my numbers assume that the rolling shutter is 1/60th of a second from top to bottom.

It it's not too much trouble, could you please look at your footage frame by frame to see if any point on the chart appears for only 18 frames. At least that would tell me if my calculations are correct or not.

There is another way to measure the speed of the rolling shutter, but it takes some lab equipment, which I can borrow from the college where I work.

Sincerely,

Bob Diaz

Eric Pascarelli · January 20th, 2008, 09:56 PM

Bob,

Yes, a given point on the chart takes about 18 frames to make it from the left to the right side at the speed I was panning on the motion control.

I'll try to post the footage. The forum was not letting me do so for some reason, which is what I only posted the stills.

I designed the pan so that each frame, the camera pans the width of one of the black or white bars.

Each bar is about 106 pixels wide - and 106x18 = 1908, or almost 1920.

January 16th, 2008, 10:08 PM	#17
Charles Dasher Regular Crew Join Date: May 2006 Location: Juno Beach FL Posts: 169	What to do! I got my EX back from Sony and the re alignment seems to have solved the vignette issue. But having read a fair amount about the rolling shutter effects I am considering an attempt to return my camera. I primarily film action sports such as kitesurfing, surfing etc. My fear is that capturing this type of action it will be difficult to avoid quick panning movements or high speed images flying through the frame. I was shooting some paddle surfing this morning and in one shot I quickly panned from one rider to another. The building in the background skewed so far over it was truly alarming. I did one test where I locked off a shot directly across the street to see how the cars looked as they drove through the frame. At 35 mph the skew was acceptable but often I am shooting faster objects coming past the camera. There are many things I love about the EX1. Larger chips, DOF, image quality, lens control, low light capabilities and image control. I am moving up from the PD 170 CCD world and have waited a long time to make the jump. I am concerned with what I am seeing. I was not interested in HVX 200 but I am uncertain of where to turn. Any advice is greatly appreciated

January 17th, 2008, 09:20 AM	#20
David Lorente Regular Crew Join Date: Dec 2007 Location: Cuenca (Spain) Posts: 92	I've been reading a lot of things about the rolling shutter, some of them are valuable comments, scientific explanations, or well done tests like the one posted here by Eric. But there are also a lot of misinformation, people who fear this effect like if it was the devil, and the people who post comments having no idea at all of what they are talking about. Well, I'll put my two cents here. From the tests of Eric, we can see that the scanning rate is double than the frame rate. By scanning rate, I mean the time it takes for the CMOS sensors to scan one complete frame. This is the time difference between the moment the top line is read and the moment the bottom line is read, and it is what effectively causes the skewing effect we see on the image. Current CCD's don't show this effect because the whole image is stored in a memory when the integration time is complete, and then it is readed slowly, but every pixel of the image has started and has stopped storing light at the same time. Eric has pointed out that the scanning rate is always the same no matter which shutter speed you use. I'll go further saying that this scanning rate is not neccesarily linked to the video frame rate you set on the camera. This means that the scanning rate can be the same no matter you set the camera at 30, 25 or 24 fps. Just think about the camera like a photographic camera, but taking several images by a second (that's where all started...). You have a clock that tells the camera when to capture a frame, or 'that pushes the trigger button'. This is the frame rate. So, in this test we saw that the scanning rate was double of the frame rate, that is, 1/60th of second at 30 fps, but if the scanning rate was the same 1/60th of second at 24 fps, the effect would be less noticeable. And this explains why the effect is less visible in the EX1 than in the Canon camera whose model number I can't remember and whose scanning rate was surely the same as the frame rate (1/30th of a second). Notice that this is just what I have figured out from what I know about the technology, but it must be confirmed with new tests at the different frame rates. Tests that I will do as soon as my camera arrives, it had to be here this week, but it has been delayed and I have to way another two weeks! :( When this technology evolve in the future, they surely will avoid the scanning shutter effects by increasing the scanning rate: if the complete image is scanned, say, at 1/300th of second, nobody will notice any skewing or the other weird effects... unless you use the camera to shot crash tests or other high speed things. But I wan't to say that we also have to relativise this effect: in the great majority of the cases, we shot without shutter (I mean the standard shutter), so fast moving objects or camera movements will appear completely blurred like in the test frame in which the shutter was set to 1/60th. So... a test chart will show skewing, and we will notice it, or not, but in real life shots, I'm absolutely sure that people watching the footage will just be stunned with the high quality of the image, and nobody, except us, the fools with the cameras, will notice anything wrong with the images. I don't mean it's not a problem at all: I see a problem when using fast shutter settings (slow shutter rates will blur the action) and very fast movements/moving objects. If you're going to shoot these kind of things, then the EX1 may not be the best camera for you... But for a lot of other kind of works, this camera is just perfect. It's just a matter of compromising: this camera achieves it's high performance thanks to it's new CMOS sensors, but that is just what causes its weakness, scanning shutter artifacts. I'm becoming a pain in the neck, so I just wan't to say that I have seen this infamous effect in... my old Betacam camera! Yes, it's powered with Hyper HAD CCD's, and yes, what I'm saying will cause some discussion, but I will demonstrate that as soon as I can. __________________ Wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles. Radio operates exactly the same way, but there is no cat.

January 17th, 2008, 12:29 PM	#22
Eric Pascarelli Major Player Join Date: Nov 2007 Location: Los Angeles, CA Posts: 498	Good News (and bad). Thanks, David for that post. I tested some other speeds, without using motion control, but just looking at two successive frames to determine the ratio of pan speed in pixels per frame versus skew amount in pixels. 1080 24p had a ratio of 2.66, implying a scan time of about 1/64 second 720 24p had a ratio of 2.25, implying a scan time of 1/54 second 720 60p had a ratio of 1.0, implying a scan time of 1/60 second It seems like the EX1 keeps a (reasonably) consistent scan time, as David suggested rather than a proportional scan time. So the effect is less noticeable on a frame basis at 24p, but in real world situations I would guess the skew would be equally noticeable at all speeds. I am not sure why the numbers are a bit funny - there may be a fixed component to the timing that I am not incorporating into the algebra that would make everything equal out. Anyway, the test is easy to do and I'd love to see people confirm or refute these numbers. Shoot a vertical target (which crosses top and bottom of frame) at 1/2000 shutter. Pan rapidly. Examine 2 adjacent frames (in Photoshop) and see how much the target has shifted in pixels. That's the pan rate in pixels per frame. Now look at either frame alone and measure the the horizontal difference from top to bottom of the target. That's the skew amount. Divide. That's the ratio.

January 14th, 2008, 08:43 PM	#16
Eric Pascarelli Major Player Join Date: Nov 2007 Location: Los Angeles, CA Posts: 498	Yes, I will. Give me a day or two.

January 16th, 2008, 11:36 PM	#18
Tim Polster Inner Circle Join Date: Dec 2004 Location: Arlington, TX Posts: 2,231	This is alarming. The EX looks like a great value except for this rolling shutter behavior. The next option imho is the HPX-500 or the XDCAM 335/350. But they will bring the price to the $20,000 range. Makes one think, "maybe a skewed building here and there is not so bad"...

January 17th, 2008, 03:07 AM	#19
Paul Curtis Major Player Join Date: Mar 2005 Location: canterbury Posts: 411	>The building in the background skewed so far over it was truly alarming. Rapid changes of direction will be worse, if you're just panning in one direction the skew is perhaps acceptable and the public won't notice but if the skew changes direction then it leaps out as something wrong. And we haven't really touched on vertical squash and stretch. Im glad that we're hearing and seeing some real world examples. I think the same will happen to Red as well (perhaps to a lesser degree) and maybe SI (the sensor can do 60fps as well which is probably the scanning speed limit). It's a case of how fast can the system read the sensor. On the EX i would imagine that 720 will be the same because the camera would scan everything and downsample? Or maybe it scans different lines to get the 720 image in which case it could scan faster (but loose some image quality advantages). I've mentioned elsewhere that im quite disappointed with sony and their marketing hype. I remember statements being made about substantionally reducing this effect that simply aren't true. Which is a shame because if you look at sonys sensor side of the business they do have great cmos technology working at high speeds. So they do have the sensors and the knowhow but somewhere between the design and implementation things don't work as advertised. There is a reason why sony mandate that pro and broadcast gear is CCD and consumer is CMOS. So perhaps this limitation in the EX is being driven more by marketing than tech. We have some pilot/test stuff to shoot in the not too distant future, i may hire one to see for myself and compare to the other XDCAMs rather than just buy (because everything else is so attractive about it and i can get two for the price of something else) cheers paul

January 17th, 2008, 04:29 PM	#23
Noah Yuan-Vogel Major Player Join Date: Dec 2004 Location: New York City Posts: 613	thats interesting. so the readout rate may always be 60fps... it would make sense if the maximum readout of the sensor allows at most 1920x1080@60fps. all other framerates are probably achieved by blanking(?) and frame sizes achieved not at the sensor level, just in processing. I suppose unless it is an interlaced sensor, it would have to be able to do at least 1080@60 to come about 1080i video. this might mean the sensor is capable of shooting 60fps at 1080, maybe the DSP cant handle it? this is all speculation of course. at least it is good to know this. this should probably mean that the skew and wobble characteristics will be the same over time, when played back at the speed it was captured at. now for stuff high speed stuff thats getting slowed down, 1:1 readout ratio might be tough...

January 17th, 2008, 07:21 PM	#24
Mooho Bae Regular Crew Join Date: Jan 2005 Location: Korea Posts: 52	Eric, Thank you so much again. Thank you David. Eric, May I understand that the scan speed is very close to the RED in 24P mode?

January 17th, 2008, 08:17 PM	#25
Eric Pascarelli Major Player Join Date: Nov 2007 Location: Los Angeles, CA Posts: 498	The RED, from only one measurment, seemed to do 1/72, which is a bit faster than the Ex1. But further tests are necessary. I need to check with Jim Arthurs and have him try a few speeds.

January 19th, 2008, 11:39 AM	#26
Brendan Pyatt Regular Crew Join Date: Sep 2005 Location: Southampton UK Posts: 165	Anyone used an EX1 in a helicopter? With or without a gyro?

January 20th, 2008, 04:59 PM	#28
Brian Drysdale Inner Circle Join Date: Feb 2006 Location: Belfast, UK Posts: 6,152	[QUOTE=David Lorente;811241]I'm sorry. I have to apologize, because I said a stupid thing. What I saw in my Betacam was an optical effect, probably due to the fact the viewfinder is a CRT. It can also be seen on a CRT monitor. The skew is an illusion produced because it's the CRT who generates the image as a "rolling shutter". QUOTE] Well, you do have to bear in mind the overall system. It's only in recent years that there was any alternative to the CRT - LCDs and Plasmas can add their own little artifacts.

January 20th, 2008, 09:43 PM	#29
Bob Diaz Regular Crew Join Date: Jan 2008 Location: LA, California Posts: 170	Hi Eric, I've been studying your screen grabs showing the skew and doing a bit of math based on the image. IF I've calculated correctly, in your video any point on the chart should appear for 0.6 seconds or in other words, it first appears on the right side of the screen and 0.6 seconds later it falls off the left side of the screen after 0.6 seconds (18 frames). So this would be a VERY fast pan. I've counted an offset of 53 pixels and an angle of 2.8 degrees. All my numbers assume that the rolling shutter is 1/60th of a second from top to bottom. It it's not too much trouble, could you please look at your footage frame by frame to see if any point on the chart appears for only 18 frames. At least that would tell me if my calculations are correct or not. There is another way to measure the speed of the rolling shutter, but it takes some lab equipment, which I can borrow from the college where I work. Sincerely, Bob Diaz

January 20th, 2008, 09:56 PM	#30
Eric Pascarelli Major Player Join Date: Nov 2007 Location: Los Angeles, CA Posts: 498	Bob, Yes, a given point on the chart takes about 18 frames to make it from the left to the right side at the speed I was panning on the motion control. I'll try to post the footage. The forum was not letting me do so for some reason, which is what I only posted the stills. I designed the pan so that each frame, the camera pans the width of one of the black or white bars. Each bar is about 106 pixels wide - and 106x18 = 1908, or almost 1920.