The Great Camera Shootout Part 2

[Brian Drysdale]

This part covers sensors and sensitivity.

The Great Camera Shootout 2011: SCCE ~ Episode Two | Zacuto USA

[Karl Eklund]

I usually like the shootouts, but this one confused me a lot regarding sensitivity. So I will repost a question I asked on a different forum.

I sort of didn't get the point of their "sensitivity test". I understand the part of checking signal to noise and giving numbers for that. But the samples were all at different ISO's, and not on the cameras native sweet spot, so why bother? It would have been nice with a test of each camera at i.e. 500, 800, 1600, 3200. Maybe I am just confused. It wasn't clear to me if they are trying to get all the cameras to match up in exposure by changing ISO since the aperture is locked, or if they are changing shutter speed, ND filters, or what not to get the same exposure. Once again, I am maybe just confused, shouldn't the same ISO setting, the same F-stop and shutter speed, give the same medium gray value? My thinking is that 1250 ISO on the Alexa will be 1 and a 1/4th of a stop faster than the 500 ISO film stock, and obviously then the film stock will be underexposed and grainier in comparison.

Also, I don't understand why they use the 500 ISO film stock (5219), instead of the 200 (5213), to compare certain things. They also ended up rating the 200 to be 320 ISO. I guess the 500 is closer in sensitivity to the digital cameras, but it will have more grain.

[David Heath]

It may sound churlish after all the work they've done, but the elephant not in the room is the FS100.

Yes, I appreciate it's not possible to include every camera, and I appreciate it may not have been available at the time the tests were done. But.... I think the significance of the FS100 is such that it would have been worth delaying the tests until it could have been included. It wouldn't have been very long, would it?

I suspect the FS100 wouldn't have been as good as an F3 - but the question is by how much? I suspect it would have been better than the DSLRs - but by how much? And how does it really stack up against the AF101 - it's real rival in terms of cost?

Surely I'm not the only person to feel frustrated?

[Charles Papert]

Think of it this way: there's always another camera just around the bend. Once something of this magnitude gets scheduled (way in advance), it's not practical to "stop the presses" to wait for an upcoming camera to appear (and hope it won't get delayed). The frustrating thing about organizing these types of shootouts is that they have such a short shelf life, as cameras emerge with greater and greater rapidity--the ASC mounted a massive one a couple of years ago, and while it was an interesting snapshot at the time, it was already outdated by the time the results were released, and is now all but irrelevant.

[John Vincent]

"Think of it this way: there's always another camera just around the bend."

Well, that seems only half true here - the FS100 had a release date (one which I think Sony actually delivered early on) and I believe there were prototypes out in the field as early as March - certainly the FS100 couldn't be considered "Vaporware."

No other major releases had been announced at the time - Canon was (and continues to be) as silent as the grave, RED didn't seem to care if the testers had their newest camera (the director even makes reference to this fact), while JVC and Nikon only recently announced that there will be announcements down the line...

That leaves the FS100 as the only large release video camera meeting the other requirements of the test. And the FS100 looks like it's going to be a major player in the indie/wedding/interview world. While I wouldn't expect it to do better then the F3, I would have like to have seen it up against it's big brother - as well as the Af100.

One other glaring omission was the GH2 - a camera which I'd think many more curious DPs are looking at then the Nikon for lots of reasons.

The test last year seemed a bit more organic; a bit more realistic. While I appreciate why they did the test they way they did, the seemed a bit too worried about people being upset above objectivity. I'd rather have a bit more of what all those DPs really thought about the cameras then all the pixel peeping.

[Brian Drysdale]

They seem to be running these particular tests every year, so perhaps the FS100 will get a chance against the RED Epic, the other missing camera next year.

RED tend not to be happy about these tests anyway, so no real surprise that they didn't rush to give them an Epic M.

Yes, the sensitivity test seemed rather confusing in its objectives. It seems to be the test that causes problems anyway, another comparison used the same ISO for every camera and that caused complaints as well.

[David Heath]

Originally Posted by John Vincent

"Think of it this way: there's always another camera just around the bend."

Well, that seems only half true here - the FS100 had a release date (one which I think Sony actually delivered early on) and I believe there were prototypes out in the field as early as March - certainly the FS100 couldn't be considered "Vaporware."

No other major releases had been announced at the time - Canon was ........

Yes John - that's what I was thinking of. Leaving the higher priced cameras aside, there were three contenders for the middle field all due to come out within a few months of each other - The AF100, the F3 and the FS100. And yes again, my memory is that the FS100 was hinted at when the F3 was announced, and formally announced itself not many weeks later.

It just seems like there have been a rapid series of twisty bends, with a significant new camera round each one, then quite a long straight stretch which we're in now! And for what ever reason, these tests were done just before the last bend....... Annoying.

[Jon Fairhurst]

Scarlet was announced in April, 2008. Glad they didn't wait for that camera. ;)

EPIC, the FS100, the GH2, and the CineStyle picture style would have all been great to see. Regardless, I enjoyed seeing the results of the cameras they tested. I'm looking forward to Episode 3 - as well as whatever they might produce in 2012.

[Thomas Smet]

I agree why no GH1 and GH2? It is my understanding the GH2 shares some of the same guts as the AF100 minus the native HD filtering of course. I understand that perhaps a hacked camera would sit outside of the legal scope of this test but even unhacked the GH2 is amazing and I would say hands down beats the pants off of Canon DSLR's.

[Brian Drysdale]

I'd suspect there are resource and time constraints on these tests, so I shouldn't expect every possible camera to be tested. If they run the tests next year, no doubt some cameras won't be tested and perhaps some mightn't be retested..

[Steve Kalle]

Originally Posted by Karl Eklund

I usually like the shootouts, but this one confused me a lot regarding sensitivity. So I will repost a question I asked on a different forum.

I sort of didn't get the point of their "sensitivity test". I understand the part of checking signal to noise and giving numbers for that. But the samples were all at different ISO's, and not on the cameras native sweet spot, so why bother? It would have been nice with a test of each camera at i.e. 500, 800, 1600, 3200. Maybe I am just confused. It wasn't clear to me if they are trying to get all the cameras to match up in exposure by changing ISO since the aperture is locked, or if they are changing shutter speed, ND filters, or what not to get the same exposure. Once again, I am maybe just confused, shouldn't the same ISO setting, the same F-stop and shutter speed, give the same medium gray value? My thinking is that 1250 ISO on the Alexa will be 1 and a 1/4th of a stop faster than the 500 ISO film stock, and obviously then the film stock will be underexposed and grainier in comparison.

Also, I don't understand why they use the 500 ISO film stock (5219), instead of the 200 (5213), to compare certain things. They also ended up rating the 200 to be 320 ISO. I guess the 500 is closer in sensitivity to the digital cameras, but it will have more grain.

ISO and F-stop are ratings and guides - they do not represent the actual physical light transmission. T-stop represents the actual amount of light being transmitted which is why Cine lenses use it rather than F-stop.

In one of the courses over at FXPHD.com, they spent an entire class discussing noise and exposure properties of the Red One, 5D & 7D. With an 18% grey card, the 5D's exposure was identical to the Red (actual image varied slightly) but the 7D's exposure was a third or half a stop different. In addition, both the 7D and 5D had more red in the captured image than the R1 with the 7D having more red than the 5D.

Over at DxOMark, they test each camera's ISO rating and show the actual ISO (ie, a camera ISO of 200 is actually only 180). All DSLRs' ISO ratings differ from the actual ISO being captured.

With digital sensors, the rated exposure almost always differs from the real exposure accept for the very high end cameras. This is the nature of electronics manufacturing and signal processing.

[Steve Kalle]

About the testing and the video: I would have set up tests that can be repeated any time of the year; so, they could have tested more cameras each year and have the same benchmark for each camera. A lot of money and time could have been saved now and in the future.

I find the editing and general theme to be annoying. There are several instances where one of the 2 main guys is talking and then it is switched to the other guy and then back and forth. It is very annoying when one guy starts a sentence/thought and another guy finishes it.

Other than that, I loved the testing. Also, I hope the AE artist was paid commensurately - he had a lot of work and did a great job.

[Jon Fairhurst]

I liked the editing - except for the credit/head shots. The heads move a bit. The title ("crew") was in a large open space and moved a lot. Each person's name was crowded and static. I find that my eye is drawn to the generic title ("crew"), which is the least important information on the screen. Next I look at the face. By the time I pull my eyes away to see the name, we cut to the next person.

It's a nice looking section, but is really poor at drawing the eye to the most important info. Swap the title and the name, and it would be ten times better.

As I say, I like the editing and the style, but this one section could be shown as a "what not to do" example in an editing/graphics/directing class. Just like in magic, film is all about directing (or mis-directing) the focus of the audience.

[Steve Kalle]

I recently heard from a little birdie that Zacuto used a prototype F3 during the FIRST test, and Sony forced them to redo all of the tests. Apparently, they knew that the prototype had a bug but still went ahead with the testing. Sony said, "oh no you didn't".

This little birdie said that someone with a name like mine concluded the Alexa #1 and......... the F3 as #2. Makes me want one even more.

[Sergio Perez]

After seeing the video, I actually was impressed by the Af100 performance, and somehow disappointed with the F3. However, I was impressed by the f35 with S-Log. I can only imagine the F3 performing similarly, and if that is so... then its unbeatable in the price range. What I mean is F3 without Slog is not worth it, but with SLOG, definitely.