Rolling shutter and lightning footage
 

I have an important and legitimate concern about the rolling shutter issue that I still haven't seen a concrete answer about.

I am a storm chaser and many of my chaser colleagues are looking at the FX7 / HC7 / HC5 for their next camera. Lightning is one of the primary subjects that we shoot, and given the rolling shutter problems seen with early CMOS HD cameras, naturally we are concerned about the newer CMOS-based cameras exhibiting this same problem.

The reason this is important is that rolling shutter effectively kills all lightning video. The footage is unusable, with frames with lightning channels/flashes divided in half. Many chasers who bought the HC1 / 3 models were terribly disappointed with the cameras' total deficiency in capturing lightning.

For us, this is not an 'extraordinary circumstance' that we are putting the camera through just to 'expose its flaws'. Lightning is one of our main subjects, so I feel it is a legitimate question to pose. Has anyone tested the new CMOS based cameras with lightning, or even with a camera flash (which in theory should reproduce the same effect)?

We've done some serious digging and have not found an answer. Short of just buying one and finding a storm to shoot, there's no way to tell (and no retailer will take returns on a camera used on a storm shoot!).

Any info about this would be appreciated, as there are many storm chasers considering the FX7 / HC7 / HC5 but are reluctant because of the lightning/CMOS issue. A deficiency in the lightning capturing abilities of these cameras will be a deal-breaker. Not looking to stir up trouble or slam any particular camera model or manufacturer, just would like a simple 'yes' or 'no' on whether these cameras can do the job with storm footage.

rolling shutter
 

Without the benefits of tests I'd expect the progressive exposure of a rolling shutter would make it quite unsuited to your application. Should be easy to test with a high speed flash illuminated target in the dealer's premises.

I wish there was a simple yes or no answer. I am inclined to say "no, they don't work", but the FX7 appears to be four times faster with it's shutter than the other cameras. This is not confirmed, as far as I know, but the Sony literature seemed to indicate that the FX7/V1 have quadruple circuits to read the CMOS unlike other cameras. I'm assuming the other cameras are the consumer single-CMOS cameras. I'm guessing that you will not get the results you want, but try a test with the FX7 if you get a chance. I'm sure the single-CMOS cameras are still too slow.

Rolling shutter will likely still be an issue - tested with the HC7, and found about one out of 4 shots showed the effect with camera flash - usually only on the top or bottom 10-15% of the frame - on the HC1 I saw it split pretty evenly top and bottom. Not sure if the HC7 is better, but seemed like it might be.

Also another wedding videographer reported the rolling shutter with the V1 IIRC - it's an old post. He had sample stills showing the problem. That would mean the FX7 has the issue too.

In short, I don't think there's a solution to the problem - if we ever get any rain and thundershowers out here in Cali, I'll try shooting some to see what happens, but the way it's been, that may be a loooong time...

The V1 I have shows that same thing when there's a camera flash...only a portion of the picture is lit by the flash. For my work...weddings, corporate, etc..., this camera is amazing, but for your specific situation, I'd go for the Canon XH-A1.

I agree with Sergio. Based on how my V1 responds to camera flashes I suspect it won't be usable for your application.

Maybe the FX1 or Z1 would be what you need if you are a Sony fan. Both are CCD based.

No current CMOS based video camera will likely be usable for your application due to the limitations of the technology. This isn't a flaw, its a characteristic of how the imager is read and is the same regardless of brand. CMOS has given us some good things such as longer battery life and great exposure latitude but we have had to pay for this with higher low light noise and the rolling shutter effect. In my application the camera works beautifully and I'm thrilled to have it.

As video processors get faster this technology may become usable to you at some point in the future. Right now, No.

Chris

Thanks for the info everyone - greatly appreciated and saved a lot of people from buying the wrong camera. I have an FX1 and it does just fine with lightning, so that's what I'll stick with and recommend to others.

The next chance I get to shoot some video of lightning here with my V1 I'll be happy to do it and let you know the result.

Chris

Hi there,

I'm very close to buying an HC7 but I'm also worried about the "rolling shutter" issue. In my case, a rolling shutter is bad news for my projects because I do match-moving in post-production (which is where the software analyses video footage and calculates the exact movements of the camera so that CGI elements can be inserted).

There is a simple test to determine if the HC7 has a rolling shutter. If anyone has an HC7, please can I ask you to do a simple test:

Please can you pan rapidly past an object with vertical lines (e.g. a door frame or a lamp post). On play-back, do the vertical lines look at all bent?

Here's an example of what happens when you do fast pans on the HV20 (which has a slow rolling shutter):

http://www.ssontech.com/content/crooked.mov

Many thanks,
Jack

Hi Jack,
As far as I know every CMOS camera uses rolling shutter. I believe it's inherent to the technology.
Those bent vertical lines appear on the V1 (not so drastically as in your video, i think) but I read somewhere that the engineers used some technique to minimize that.
I suppose the HC7 won't be any better than the HV20.

I have a video of the bars of the walkway on a bridge flying by at high speeds at the beginning of this video:

http://stage6.divx.com/user/jackvanc...httime-(720p60)

I hope this would show the speed of the rolling shutter.

Jack, thanks loads for sharing that video with us - it looks great. Man, you've gotta love that 720/60p-on-the-web look. Eat dust, YouTube.

Anyway... from what I can tell, that video provides good evidence to support the hypothesis that the HC7 either has a "global shutter" (i.e. it reads every pixel on a field at the same timepoint) or it has a very swift rolling shutter. Maybe my eyes are deceiving me but I couldn't see any "rolling-shutter" distortions in the footage of the railings and support rods. Shooting railings from a moving car is an excellent test for rolling shutter effects because the horizontal portion of the railings gives us an accurate orientation reference (i.e. it should be horizontal). If the video of the vertical bars supporting the horizontal railing aren't perpendicular to the railing then we know that the camera is distorting the image. But, apart from some barrel distortion due to the wide lens, I couldn't see any geometric distortions. Can anybody else see any distortions in this video?

Thanks again for posting that video,
Jack

I second that - perhaps some distorsion can be found when examining frame-by-frame, but the vido looks great! What shutter speed was used in the first portion?

The whole video is at a shutter speed of 1/60.

Hi Jack,

Going a little off-topic... but how did you do the conversion from 1080/60i to 720/60p? Did you use AviSynth?

Thanks,
Jack

There's some bending when the cars pass. Now this clip is 60p. What happens when you convert that to 30p by deinterlacing or to 24? It will show the bending way worse. That's one reason why people don't see the effect as much and think that the hv20 has more of it than other cams.

BUT it also looks like the hv20 has a faster rolling shutter rate. But I'd have to try the cam myself to be sure. I've seen clips of hc3 where it looked like there was no rolling shutter only to find out it myself. It does look very promising.

The material is beatiful anyway. Great colors.

Hi Mikko - thanks loads for the reply.

I'm confused - why would converting 720/60p to 720/30p help to eccentuate the rolling-shutter distortions? Surely the conversion from 720/60p to 720/30p just throws away every other frame?

I'm pretty sure that Russ Anderson's now infamous "crooked lamp" footage was shot at 1080/60i (actually 59.94i) on the HV20:

http://www.ssontech.com/content/crooked.mov

(the properties for the .mov file state that it's 29.97fps and the video was clearly shot interlaced - just look at that combing!)

In other words: both Russ Anderson's "crooked lamp" footage and Jack Zhang's "Stanley Park Sunset and Nighttime (720p60)" were both shot at 1080/60i. The differences are that Russ's footage was shot with an HV20 whilst Jack's was shot with an HC7 and then converted to 720/60p in post. I can't see why the conversion from 1080/60i to 720/60p would affect the rolling shutter effect so the conclusion that I draw from this comparison is that the HC7 is less prone to rolling shutter artefacts than the HV20.

On topic with this thread, I was lucky enough to catch a bolt of lightning with my HC7 yesterday and the video is now up on Stage6 (heck, I even uploaded it to The Weather Network... (Canadian version of the Weather Channel))

Stage6 is having trouble right now so I'll give the link later.

Great work! What was the result? Did the video you captured display rolling-shutter artefacts?

The initial flash was not captured at full brightness... so I don't know what that would mean...

And to the conversion question: I used Vegas 7.0e with smart resample and interpolate fields to render the video portion in 720p60. then I muxed the audio in later with virtualdubmod.

OK, this is starting to really confuse me! I've been searching Stage6 for HV20 videos to try to find evidence for the camera's slow rolling shutter.

Here's a video of a BMW M5 demo shot on an HV20 which includes lots of very fast pans and some footage shot from a moving car:

http://stage6.divx.com/user/emjoyner...---Spartanburg

I can find any serious distortions in that video.

Urg. Maybe Canon and Sony could tell us the speed of their rolling shutters?! I'll write to them both now.

Jack

I film myself flying down 95 south almost every weekend going about 85mph. I have no problems with rolling shutter artifacts. Video comes out looking smooth every time. I actually mount the cam on my tripod and the tripod in my cupholder. Works great.

My setup: http://www.fortvir.net/gallery/v/P3U_videos/Ian-T/

Thanks for the reply, Ian.

What frame rate did you use on the HV20? 1080/60i?

Thanks,
Jack

It's because one frame stays longer in view. Remember also that with a 1/60 shutter the motion blur will blur a lot of the smaller rolling shutter artifacts away.

If you see the combing you are looking at it at 29.97, not 60 fps.

That's a badly encoded video. It has blended the fields here and there and something has gone wrong when doing 24p material. It's pretty hard to see anything in the moving parts because of double blurring of the fields. I could still see the telltale bending going on when the camera shook. When you start spotting that the upper part moves at a slightly different time than the lower part of the picture, it's kinda annoying. But the bad encoding masked movements anyway.

Here's an old clip from my hc1

http://hmcindie.pp.fi/rollingshutter/

A high shutter reveals the effect very well. Remember that the effect is always there. It's just hidden from view when there is motion blur. It's still possible to make action shots with the hc1, it's just another hurdle to remember about.

I used 24P.

Well...just like Jack....I can't find any serious distortions in that BMW footage. I know there are some artifacts...but all in all...that looks good to me especially with the conditions that cam was in.

Ok, I think I might've produced distortions in the HC7 this time around crossing the Lions Gate Bridge.

Stage6 is acting weird so I'll post a screenshot.

Thanks for the screenshot. Hmm... the vertical supports certainly aren't exactly perpendicular to the horizontal railing.... but the vertical supports are bending the wrong way! I'm pretty sure that all rolling-shutter mechanisms on CMOS sensors start reading at the top of the sensor (so the top row is taken first, then the second row, then the third row and so on...). The HV20 sensor certainly starts from the top. If the sensor starts reading from the top then you'd expect the top part of the supports to be further to the *right* of the image than the bottom part. But that's the opposite to what's shown in that screenshot.

[edit - see the post below]

Ho hum.

Both Canon and Sony have promised to call me this morning to let me know about their rolling shutters.

What ever happens, I have to order either an HC7 or an HV20 this afternoon. I think I'm going for the HC7. Even if the HC7 and HV20 do have the same speed rolling shutters, I really like the 240-fields-per-second-for-3-seconds feature of the HC7. I love smooth slow-mo. I am really gutted that the Sony doesn't have "true progressive scan" but I've done some tests with motion-compensated de-interlacers and I'm happy that good de-interlacing can produce an image that's almost as pretty as a true progressive-scan image.

Yes, but it was shot 60i (60i = 60 fields per second = 30 (interlaced) frames per second)

I'm not sure I agree. Ghosting is an fairly inevitable side-effect of resolution-maintaining de-interlacing (even motion-compensated de-interlacers like MVbob produce some ghosting). Even using a top-of-the-line hardware de-interlacer will produce some ghosting. As far as I'm aware, the only way to de-interlace without producing ghosting is to throw away one of the fields. So, in other words, I would propose that ghosting does not equal bad encoding. But, more to the point, even bad encoding shouldn't completely hide the effects of a rolling shutter. Bad encoding wont change the angle of the rolling-shutter distortions.

Thanks loads for that. That certainly does contain some nasty rolling-shutter artefacts! And, very interestingly, that clip seems to suggest that the HC1 starts reading from the *bottom* of the sensor (the verticals seem to lean *into* to pan). If we assume that the HC7 also reads from the bottom of the sensor then this explains Jack's screenshot above.

Hi,

Sony called me this morning. I was impressed that they called me back but they really didn't get what the issue was. After I spent 10 minutes trying to explain what the rolling shutter issue is, the customer services guy went off to speak with the "camera person". He tried to fob me off by trying to persuade me that the issue is that shooting interlaced produces comb-lines when displayed on a progressive screen like a computer screen. They really didn't understand that the issue is all to do with the read timings *within* each field. So, the bottom line is that Sony told me nothing of value.

Still... I've run out of research time so I'm going to dump £700 on an HC7 this afternoon.

Quick rant: I'm really glad that "modern" companies like Cineform and Red "get" that it's not acceptable to separate tech support from the engineers who design the products. Consumers are intelligent people who want detailed answers. Large companies like Sony and Canon should take a long, hard look at how these smaller, more customer-focussed companies are handling their customer support.

Jack

Jack,

Did you call Prime Support? You can always say you're considering the V1E, and would like to learn about the rolling shutter issue. You could find a much more knowledgeable person there.

One warning for you, it is not in HD, it's processed in SD before being printed to tape so you're best off recording to the DV format when doing smooth slow rec.

Anyways, congrats on the purchase. and I seriously recommend that you also get a Blackmagic Intensity card to capture uncompressed or to a high quality compressed format such as NEOHD or DNxHD on the PC or ProRes 422 on the Mac for post work.

Thanks for the reply. Hmmm... I'm not sure which part of Sony Support I called. It's too late now, unfortunately. I have to make the purchase now otherwise I wont have the camera in time for testing in time for a project I'm doing next week.

Oh, fair enough. Still - 240 fields per second (which hopefully I can convert to 240 frames per second using MVbob) is awesome, even if it is in SD. I imagine you need lots of light.

Yes, I've had my eye on NEO HD for a while now. I have a Blackmagic Decklink HD card (HD-SDI inputs, no HDMI). Unfortunately I've run out of PCI-E slots so I can't add an Inensity card... although I might look into HDMI-to-HDSDI converters.

If you're recording to HDV tape, is it still better to capture via the HDMI port rather than ingesting as .m2t and converting to NEO HD? Is the camera's HDV decompressor better than NEO HD's HDV codec? (sorry, this is getting a little off topic)

Then something is wrong with your settings or the pulldown is going wrong.
Deinterlacing should never produce ghosting.

The HC1 actually has a warning in the manual about crooked pictures with the cmos sensor. Does the hc7 have the same warning?

Pull-down isn't required for converting 60i to 30p or 50i to 25p (which is usually what you're doing when you make video for the web from interlaced footage). It's only required when doing things like 23.976 fps to 29.97 or when you're extracting 24p from 60i (which is what you'd want to do if you were editing 24p footage shot on an HV20). Luckily, I live in PAL land so I very rarely have to worry about pull-down because most of my projects are shot either 50i or 25p.

http://en.wikipedia.org/wiki/3:2_pul...ldown_patterns
http://www.zerocut.com/tech/pulldown.html

A very common form of de-interlacing is to blend the two fields to make a single frame. That will always produce ghosting on moving objects. More advanced forms of de-interlacing like motion-compensation attempt to interpolate the missing information but this often produces artefacts which looks a little like ghosting.

http://en.wikipedia.org/wiki/Deinterlacing

As far as I'm aware, the only way to de-interlace footage without producing any artefacts is to throw away one of the fields. But this, of course, cuts the amount of vertical detail by half. Which might not be a problem if you're shooting 1080i and distributing in SD. Maybe this is the sort of de-interlacing that you're using?

If you record onto tape, the footage has already been compressed so outputing it to the HDMI port wouldn't make a difference between the quality of the .m2t capture and the quality of the HDMI from tape capture.

The only way to record uncompressed is to hook the HDMI directly to the card or adaptor and start rolling by starting a live capture on your PC.

Cool - that's what I would have hoped. Just out of interest - have you done a side-by-side test of footage decompressed by the camera and ingested uncompressed over HDMI versus footage ingested as m2t-over-firewire and decompressed by your computer's HDV codec?

I have heard that the HV20's HDV decompression algorithm is very good quality - so good that it's sometimes best to ingest material recorded on HDV tape as uncompressed over HDMI because the camera's HDV codec is better than many software HDV codecs... which would make sense.... not all MPEG2 codecs are made equal and it would make sense if you got an increase in quality by using the same codec for capture and decompression (i.e. the camera's)

Actually I use a laptop, but the image on my camera in play/edit and the image when playing a captured .m2t in VLC is identical.

I don't know about the decoding algorithms, The HC7 might have extra data from the x.v.color but I haven't played a tape in an HV20 to test out for myself.

Hi Jack
You're in the UK so it's 200fields per second, not 240fields per second.

240fields/sec is the rate for SSR in NTSC-country models (120frames per sec) and 200fields/sec is the SSR rate for PAL-country models. (100frames per sec).

OK - thanks for the heads up. 200fields-per-sec (converted to 200fps with MVbob) is still very useful. It's higher than an SR3-high-speed can get (120fps) so I'm happy.

I have to say that I'm well un-impressed with the published specs on Sony's website. Not only does is say factually incorrect statements like "Image Device: Progressive" but it doesn't make any mention of the framerate for SSR or the read-speed of the rolling shutter.

i'm very glad companies like Cineform and Red are taking a much more transparent approach to customer services.