Moronic Question About A Theoretical HD Sensor

[Scott Bruffey]

Okay, so if this question is in the wrong location, my apologies; this just seemed the best place to ask it.

So here we go. I'm a big fan of practical lighting, especially in low-level conditions. I shoot with an EX1 and I've tweaked a night setting that (with the use of Neat Video's noise reduction) lets me shoot in VERY low-light conditions with very little noise (in some cases, only by the light of a cigarette butt). As you can probably guess, while some of the shots may look okay after tweaking and grading, they never get close to what my eye saw while on location.

I understand why (it's that whole Brain Processing Different Light Values and Combining Them Into An Illusuory Whole Image thing we primates have going on). But I would think it'd be possible to develop a Smart sensor with a user-definable range setting that would be able to balance the differing light intensities to create a visual representation of the illusion our brain is telling us we're seeing (a more sophisticated version of the Knee function).

So am I delusional in thinking such a sensor is workable and would be of value? I'd love to be able to have my iris dialled open and have the light bulbs in the room not be glowing supernovas (while bright, they aren't that blown out to my eye). I understand that you'd be talking about a pretty complicated sensor with some elaborate feedback circuits monitoring the light input and dialing down the output in realtime across different areas of the chip (not something I'd want to try and program). But to my limited understanding such a thing seems possible (albeit complicated).

Is anyone working on developing such a sensor? And is my question so freakin' obvious that many are rolling their eyes in embarrassment? *grin*

[Bruce Watson]

Nope. Not gonna happen. The human visual system is not at all like a camera. And a camera can not work like a human visual system. The only thing they have in common is the use of a lens system.

The big difference is that the camera takes in a scene all at once -- parallel capture. The camera's iris is set to a level that gives a good average exposure. The sensitivity and dynamic range of the film/chip(s) determines the range of luminance that can be recorded above and below this fixed exposure. But the key thing is, all the pixels that make up the image are recorded at one time (don't get pedantic on me, I know what rolling shutter is in CMOS systems but for practical purposes of this discussion it doesn't matter), and the exposure is fixed across all those pixels. So is the focus of the lens. So during exposure, all the pixels see the same iris, and the same focus settings.

The human visual system scans the scene, using a complicated and flexible algorithm that we don't yet fully understand and probably never will since at least part of it depends on what you are interested in. And this, is a version of serial capture. You look first here, then there, then somewhere else, moving, stopping, moving. All this time your iris is moving too, or can be. And so is your focus. So your brain builds up an image over time, and it uses various mechanisms to compensate for exposure resulting in a huge dynamic range, and for focus resulting in huge DOF.

I'm just sayin' you're comparing apples to oranges.

[Scott Bruffey]

Thanks for the insightful response, Bruce. After reading a bunch of stuff on the differences between camera chips and human eyes, I came to the same conclusion you did, but to my limited understanding of research trends in the former, it would seem that a chip as I described MIGHT be possible someday, creating in real time what the human brain averages over several passes. I was hoping that someone was working on such a thing and someone would be able to point me towards more info on it.

Ah well, I can dream... :)

[Dave Blackhurst]

If I understand your question correctly, it's already "possible"...

Sony is already doing HDR photos - all that might be required is a fast enough "frame rate" that could capture two different exposures (presuming of course a sensor that could ramp up gain and still get a usable "image" to process), and fast enough processing to combine them on the fly. I think that would achieve the desired results...

[Sander Vreuls]

Wouldn't a really fine grid of ND like material over a high resolution sensor be appropriate? That way it captures the normal image, and captures the same image with ND4 or so applied for the highlights..

[Scott Bruffey]

Dave: Interesting idea and I'm wondering how difficult it would be to tweak a camera and sensor to try it. In other words, you'd set your two iris settings, one for dark and one for bright areas, then shoot at 48 FPS. In post you'd split the feed into two separate 24 FPS clips and average the two together ala HDR layering. I'm not that experienced with HDR layering myself so I don't know how well the idea would work but it sounds pretty interesting.

Sander: I think your approach is kinda neat also. If it were possible to have a sync'd ND filter similar to the active polarizers in a 3D projection set-up, that would achieve roughly the same effect (perhaps with less control unless you had control over how dense the ND was). This idea would probably be cheaper to implement because it could be a completely external add-on that need only sync to the shutter (or similar) like the old Nu-View 3D camcorder adapters used to do.

So far we've come up with three interesting ways of improving dynamic range that to my ignorant mind, seem approachable and feasible. When do we get our grant money!? :)

[Dave Blackhurst]

Well, it's not something for the "layman", you'd have to have access to the hardware and firmware at a pretty intimate level... I'm just spitballin' based on existing art and what would seem like relatively simple steps forward. You've caught the idea - two data streams of 24/30p frames, different processing algorithms, and an on the fly integration algorithm to merge the data streams into one final output with expanded dynamic range and quite possibly reduced noise.

BUT if it hasn't been tried yet or isn't being experimented on, it should be shortly... DVi being a "public forum", known to be monitored by the big camera makers... I wouldn't be surprised to see it announced shortly in some new camera model. Improving low light performance is one of those "holy grail" goals in camera design, and so IF the concept has merit, I'll bet some engineer in a lab somewhere will be trying it <wink>!

[Jack Zhang]

Post-processed HDR video's already been done with 2 Canon DSLRs:

HDR Video Demonstration Using Two Canon 5D mark II's on Vimeo

But doing it in real time I guess is the next thing that's key.

[David Aronson]

The HDRx on the Red Epics and Scarlets is really good.

[Scott Bruffey]

If the big camera guys DO think these approaches are interesting, I hope they shoot me an e-mail; I got a TON of these ideas! *grin*