Moving HDR images..? at DVinfo.net

Karel Bata · January 11th, 2009, 06:52 AM

Maybe I'm a bit late coming to this, but I've only just discovered HDR photography and wondered if anyone's considered it's use in moving images.

Here's a page showing some stills examples HDR Tutorial

Cool or what? At the moment you have to take bracketed exposures and then do some faffing in photoshop, but I reckon there'll be a PS plug-in soon and cameras that will do it in one take. Itching to try it myself.

Any ideas on how to do this with moving images?

I'm curious about what exactly is happening to the sensitometric curves, but I've not had time to sit down and study this properly yet.

Karel Bata · January 11th, 2009, 07:20 AM

Found some software Photomatix at HDR photo software & plugin - Tone Mapping, Exposure Blending & HDR Imaging for photography

"If you have ever photographed a high contrast scene, you know that even the best exposure will typically have blown out highlights and flat shadows. Photomatix offers two ways to solve this problem:
› Exposure Blending: Merge differently exposed photographs into one image with increased dynamic range.
› Tone Mapping: Reveal highlight and shadow details in an HDR image created from multiple exposures. The tone mapped image is ready for printing while showing the complete dynamic range captured."

Here's an amazing gellery Photomatix User Gallery - Luke Kaven These look like they were painted. Imagine if they moved...

John Wyatt · January 11th, 2009, 07:47 AM

Is it possible to take several images for each single frame quickly enough without the subject moving and therefore cause ghosting when you try to combine them for a single HDR video frame? Even with still photography, HDR is usually used for tripod landscapes to allow accurate line-up of the bracketed exposures -- how could this translate to video in which movement with actors is needed? It would be a wonderful way to improve dynamic range, but I just don't understand how it could be done with "motion pictures". Can someone explain it to me?

Biel Bestue · January 11th, 2009, 02:47 PM

OMRON Automotive Electronics Technology GmbH

but they don't have a HD version of it yet!

Giroud Francois · January 11th, 2009, 03:09 PM

basically any camera able to shoot at 60p could do a 30p low exposed picture interlaced with a 30p high exposed picture. Then each picture would be merged into an HDR picture.
for sure, in fast moving subject there would be some problem.
and you need to add a flip-flop device that switch the exposure mode between each picture.

Sebastian Pichelhofer · January 13th, 2009, 05:26 AM

Tonemapping is just a way to reduce the bitdepth of an image.

Though instead of just plain scaling or cropping the values into 8 bit range most algorithms do pretty fancy local exposure adaptions which results in the typical painting style that most tonemapped images have, especially when processed with "Details Enhancer" in Photomatix.

Theoretically every motion picture camera that outputs 10 or 12 bit video could be used to create files that then could be tonemapped.
Though there is no real workflow for this yet as tonemapping so far only aimed at still picture processing.

I did some tonemapped HDR Timelapse videos some time with a DSLR camera and multiple exposures.

There are a few of these kind of videos available on youtube. Search for "hdr timelapse".

I had recently seen some tests done by Elphel camera team members where the camera is able to vary exposure time with every recorded frame. So the camera could film 3 exposure levels at 75fps resulting in real time 25fps video when combined into HDR images. Maybe we will see results of this soon.

Ben Winter · January 13th, 2009, 09:29 AM

I am pursuing experimentation with two HDV cameras with their iris settings locked at two extremes, mounted in a beam-splitter mirror setup used for 3D applications with their positions set to achieve the same field-of-view. Provided they can be genlocked for frame-accurate sync, I have confidence the images can be combined for HDR video.

But first I must procure two HDV cameras with genlock capability, and a mirror beam-splitter :)

Nicky Campos · January 13th, 2009, 11:23 AM

Yup... There are infact several ways to achieve HDR video, each technique has its own flaws though.

What I dont understand is why NOBODY on planet earth has EVER done it???
If I am wrong please send a link!!!

(1) shoot higher than 8bits (10-12 bits) and then SUCCESSFULLY encode (aka-tonemapping) into 8bit stream!

(2a) shoot multiple exposures at the same time using beam-splitter

(2b) shoot multiple exposures off-set by time, eg. frame01 under-exposed... frame02 over-exposed... then combined BUT will have temporal motion differences... can be fixed using motion vectors!!!

(3) just shoot normal, convert to 16-bit TIFF, use tonemapping (photomatix) ... But will suffer from heavy grain - not really suitable unless one shoots in super-flat/ambient conditions.

Last important note - Even if one successfully achieves ANY of the above techniques, playing the clip at video rates will display heavy grainy temporal inconsistencies because of the STATIC nature of HDR/Tonemapping processing.

The only fix I can suggest is a temporal stabilizer like Revisions DENOISER plug-in, or any other TEMPORAL-BASED denoising plug-in like Foundrys Furnace denoiser. (please note that a normal denoiser plug-in WILL NOT work because it will not STABILIZE the noise from frame to frame!!!)

By this time the overall image quality will be considerably soft due to HEAVY denoising! ( depending on how much weird temporal grain effects u can live with??)

One would need an extremely high-quality picture to start with in the first place.
4K raw perhaps?... depending on your delivery format, could work for SD?

Biel Bestue · January 14th, 2009, 08:26 AM

I think it will come once the cmos or the ccd can achieve it with the single exposure, because any other variant would limit shutter speed or the problem of having two much cameras mounted in a single place, so I think it will come once the chips can achieve that much latitude, even the "light stage" aproach doesn't work because it shoots more photographys per second (so it limits shutter speed), it will be all about the chips, and I think hdrc may have achieved it but at lower resolution, so it won't take them much time to get it to HD

Karel Bata · January 15th, 2009, 03:54 AM

In theory many decent stills cameras that feature auto-bracketing (which are +1 & -1, or even +2 +1 -1 & -2 ! ) can be hacked to carry on taking pics and feeding the result to a hard drive. (And you could use proper lenses without a dof adapter!) But whether you can get 24fps I don't know. But I reckon someone out there must be trying.

In fact I would think any video camera that allows external control of it's functions while the camera is running would be a candidate. It could be exposing 1/48, 1/96, and 1/192 in quick succession 24 times each second. But that means your key exposure is the 1/96, losing you 2 stops over normal 24fps.

(Another approach could involve multiple passes of a scene with motion control. You need only have the actors in once and software would produce the relevant traveling matte from the image differences etc. That could work in a location where you're not allowed to rig any big lights, but you could wheel in a pre-rigged motion-control camera. A potential niche market there, I say.)

The next step of stripping the video sequence into three seperate layers out of frames A B C A B C A etc. looks straightforward, but the tone mapping doesn't look like something you can automate. Not only would you have to tweak each shot individually, but you'd probably need to make changes during the progress of a shot. And would the whole thing be a tad temperamental like, say, toonit, where small changes in an image cascade into larger effects? Here's a Toonit example Toonit Experimentl Notice the amount of distracting shimmer? An HDR image that did that would lose the sense of hyper-reality a lot of HDR stills have.

Quite promising is the idea that you don't need multiple exposures, but can do it with a high dynamic range image capturing device. (Hey, like film!) The following were done with single exposures on a DSLR
http://farm1.static.flickr.com/126/3...51d0c366_b.jpg
http://farm1.static.flickr.com/73/19...b41730c2_b.jpg
http://farm4.static.flickr.com/3143/...f7a40f85_b.jpg

With all this you enter into a whole debate about what looks 'real'. I notice also that many of the HDR stills have enormous depth of field. I wonder if that's an aesthetic judgment taking advantage of a locked off camera, or if the algorithms used later don't work so well with blur...

But I have a question here: can someone explain tonemapping in terms us camera folk can understand? A high dynamic range usually means a flat image lacking punch, so how exactly does tonemeapping give you the best of both worlds..?

Bob Hart · January 15th, 2009, 06:45 AM

A team over here was working on a product called the "smart filter". I have been out of touch for a good while now. Their "filter" that I saw was a pass-through device which selectively adjusted exposure by ND filtering at a pixel level on its filter screen.

The level of ND was established by an operator choice. Their early coarser pitch prototype apparently enabled outdoors exposure to be correct yet indoors subjects to be also correct requiring only colour temperature filtering on the window or gels on interior lighting.

Karel Bata · January 15th, 2009, 07:17 AM

Interesting concept.

I've been looking around a bit and found a few very nice HDR videos. They're all time lapse but they show some of the problems like noise, artefacting, and some notable ringing. I note that not many folk decide to repeat the experience, suggesting that post may be time consuming, or that it's simply not easy to get it right! Some of these work best full screen.

YouTube - HDR Time Lapse
YouTube - HDR timelapse
YouTube - HDR Evening Timelapse
YouTube - HDR Timelapse massive ringing - but you notice the sun is in shot!
HDR Sunflower time-lapse on Vimeo David Lynch would love this!
Tamarama HDR Timelapse on Vimeo in HD!
Point Judith Sunset on Vimeo HD too - very very nice this one. How many hours did he put into that?

Nicky Campos · January 15th, 2009, 09:50 AM

Nice... although time-lapse HDRs dont really count as video.

The ABC shutter alternating from frame to frame is a good idea - but how to actually implement it is difficult... How exactly?

A similar trick would be to use a variable ND filter that darkens as u turn it - attach some sort of motor to continuously turn it so that frame to frame is darker then brighter ABC, etc.

Single 12-bit DR exposures are no-brainer but are limited to that specific DR... limiting.

Shallow DOF doesn't work with HDRs because it... well just is weird looking really... HDR look is surreal and painterly/toony - Just doesn't suit shallow DOF.

PS Technik make an expensive and bulky 3d/stereo rig that can actually achieve inter-ocular distances so close - that two cameras can record the SAME image and therefore achieve multiple exposures!!!

Karel Bata · January 15th, 2009, 10:40 AM

"but how to actually implement it is difficult... How exactly?"

Ask a camera software hacker! Not my line really, but I'm sure it could be done. There's plenty of folk shooting stop-frame animation using a stills camera linked to a PC. So it would have to be auto-bracketing and locked into a continuous burst mode - effectively shooting at 72fps! Hmm...

Another way would be to use an old-fashioned 3 CCD prism with 3 HD chips gen-locked together generating 3 video streams running to a hard drive (or 3 hard drives?). The technology for all that is well established.

My 2p.

Nicky Campos · January 15th, 2009, 02:16 PM

Theoretically it can be done Im sure... just like theoretically its possible to get the Canon 5D Mk2 to give us our dream camera! Find us a camera software hacker then.

Auto bracketing is a still camera function - no still camera can run a 72FPS... fastest is 15P I think. For video cameras iris cannot be used cos DOF will change... leaving us with shutter or ND filters.

Although if a "camera software hacker" could just implement NDs on the 3 chips separately (eg. green - no ND, Red - ND1, Blue - ND2) and of-coarse treat each chip as a single bayer pattern (eg. recover full color from each chip using software like Juan from Reel-stream) then maybe it can be done?

January 11th, 2009, 06:52 AM	#1
Karel Bata Regular Crew Join Date: Oct 2004 Location: London, Antarctica Posts: 199	Moving HDR images..? Maybe I'm a bit late coming to this, but I've only just discovered HDR photography and wondered if anyone's considered it's use in moving images. Here's a page showing some stills examples HDR Tutorial Cool or what? At the moment you have to take bracketed exposures and then do some faffing in photoshop, but I reckon there'll be a PS plug-in soon and cameras that will do it in one take. Itching to try it myself. Any ideas on how to do this with moving images? I'm curious about what exactly is happening to the sensitometric curves, but I've not had time to sit down and study this properly yet. __________________ What are the Alternatives to YouTube?

January 13th, 2009, 09:29 AM	#7
Ben Winter Trustee Join Date: Jun 2005 Location: Hollywood, CA Posts: 1,675 Images: 1	I am pursuing experimentation with two HDV cameras with their iris settings locked at two extremes, mounted in a beam-splitter mirror setup used for 3D applications with their positions set to achieve the same field-of-view. Provided they can be genlocked for frame-accurate sync, I have confidence the images can be combined for HDR video. But first I must procure two HDV cameras with genlock capability, and a mirror beam-splitter :) __________________ BenWinter.com

January 15th, 2009, 03:54 AM	#10
Karel Bata Regular Crew Join Date: Oct 2004 Location: London, Antarctica Posts: 199	In theory many decent stills cameras that feature auto-bracketing (which are +1 & -1, or even +2 +1 -1 & -2 ! ) can be hacked to carry on taking pics and feeding the result to a hard drive. (And you could use proper lenses without a dof adapter!) But whether you can get 24fps I don't know. But I reckon someone out there must be trying. In fact I would think any video camera that allows external control of it's functions while the camera is running would be a candidate. It could be exposing 1/48, 1/96, and 1/192 in quick succession 24 times each second. But that means your key exposure is the 1/96, losing you 2 stops over normal 24fps. (Another approach could involve multiple passes of a scene with motion control. You need only have the actors in once and software would produce the relevant traveling matte from the image differences etc. That could work in a location where you're not allowed to rig any big lights, but you could wheel in a pre-rigged motion-control camera. A potential niche market there, I say.) The next step of stripping the video sequence into three seperate layers out of frames A B C A B C A etc. looks straightforward, but the tone mapping doesn't look like something you can automate. Not only would you have to tweak each shot individually, but you'd probably need to make changes during the progress of a shot. And would the whole thing be a tad temperamental like, say, toonit, where small changes in an image cascade into larger effects? Here's a Toonit example Toonit Experimentl Notice the amount of distracting shimmer? An HDR image that did that would lose the sense of hyper-reality a lot of HDR stills have. Quite promising is the idea that you don't need multiple exposures, but can do it with a high dynamic range image capturing device. (Hey, like film!) The following were done with single exposures on a DSLR http://farm1.static.flickr.com/126/3...51d0c366_b.jpg http://farm1.static.flickr.com/73/19...b41730c2_b.jpg http://farm4.static.flickr.com/3143/...f7a40f85_b.jpg With all this you enter into a whole debate about what looks 'real'. I notice also that many of the HDR stills have enormous depth of field. I wonder if that's an aesthetic judgment taking advantage of a locked off camera, or if the algorithms used later don't work so well with blur... But I have a question here: can someone explain tonemapping in terms us camera folk can understand? A high dynamic range usually means a flat image lacking punch, so how exactly does tonemeapping give you the best of both worlds..? __________________ What are the Alternatives to YouTube?

January 11th, 2009, 07:20 AM	#2
Karel Bata Regular Crew Join Date: Oct 2004 Location: London, Antarctica Posts: 199	Found some software Photomatix at HDR photo software & plugin - Tone Mapping, Exposure Blending & HDR Imaging for photography "If you have ever photographed a high contrast scene, you know that even the best exposure will typically have blown out highlights and flat shadows. Photomatix offers two ways to solve this problem: › Exposure Blending: Merge differently exposed photographs into one image with increased dynamic range. › Tone Mapping: Reveal highlight and shadow details in an HDR image created from multiple exposures. The tone mapped image is ready for printing while showing the complete dynamic range captured." Here's an amazing gellery Photomatix User Gallery - Luke Kaven These look like they were painted. Imagine if they moved...

January 11th, 2009, 07:47 AM	#3
John Wyatt Regular Crew Join Date: Jan 2006 Location: West Country, UK Posts: 141	Is it possible to take several images for each single frame quickly enough without the subject moving and therefore cause ghosting when you try to combine them for a single HDR video frame? Even with still photography, HDR is usually used for tripod landscapes to allow accurate line-up of the bracketed exposures -- how could this translate to video in which movement with actors is needed? It would be a wonderful way to improve dynamic range, but I just don't understand how it could be done with "motion pictures". Can someone explain it to me?

January 11th, 2009, 02:47 PM	#4
Biel Bestue Regular Crew Join Date: Jul 2005 Posts: 96	OMRON Automotive Electronics Technology GmbH but they don't have a HD version of it yet!

January 11th, 2009, 03:09 PM	#5
Giroud Francois Inner Circle Join Date: Feb 2004 Location: switzerland Posts: 2,133	basically any camera able to shoot at 60p could do a 30p low exposed picture interlaced with a 30p high exposed picture. Then each picture would be merged into an HDR picture. for sure, in fast moving subject there would be some problem. and you need to add a flip-flop device that switch the exposure mode between each picture.

January 13th, 2009, 05:26 AM	#6
Sebastian Pichelhofer Regular Crew Join Date: Jan 2007 Location: Vienna, Austria Posts: 112	Tonemapping is just a way to reduce the bitdepth of an image. Though instead of just plain scaling or cropping the values into 8 bit range most algorithms do pretty fancy local exposure adaptions which results in the typical painting style that most tonemapped images have, especially when processed with "Details Enhancer" in Photomatix. Theoretically every motion picture camera that outputs 10 or 12 bit video could be used to create files that then could be tonemapped. Though there is no real workflow for this yet as tonemapping so far only aimed at still picture processing. I did some tonemapped HDR Timelapse videos some time with a DSLR camera and multiple exposures. There are a few of these kind of videos available on youtube. Search for "hdr timelapse". I had recently seen some tests done by Elphel camera team members where the camera is able to vary exposure time with every recorded frame. So the camera could film 3 exposure levels at 75fps resulting in real time 25fps video when combined into HDR images. Maybe we will see results of this soon.

January 13th, 2009, 11:23 AM	#8
Nicky Campos Regular Crew Join Date: Jan 2008 Location: Johannesburg, South Africa Posts: 35	Yup... There are infact several ways to achieve HDR video, each technique has its own flaws though. What I dont understand is why NOBODY on planet earth has EVER done it??? If I am wrong please send a link!!! (1) shoot higher than 8bits (10-12 bits) and then SUCCESSFULLY encode (aka-tonemapping) into 8bit stream! (2a) shoot multiple exposures at the same time using beam-splitter (2b) shoot multiple exposures off-set by time, eg. frame01 under-exposed... frame02 over-exposed... then combined BUT will have temporal motion differences... can be fixed using motion vectors!!! (3) just shoot normal, convert to 16-bit TIFF, use tonemapping (photomatix) ... But will suffer from heavy grain - not really suitable unless one shoots in super-flat/ambient conditions. Last important note - Even if one successfully achieves ANY of the above techniques, playing the clip at video rates will display heavy grainy temporal inconsistencies because of the STATIC nature of HDR/Tonemapping processing. The only fix I can suggest is a temporal stabilizer like Revisions DENOISER plug-in, or any other TEMPORAL-BASED denoising plug-in like Foundrys Furnace denoiser. (please note that a normal denoiser plug-in WILL NOT work because it will not STABILIZE the noise from frame to frame!!!) By this time the overall image quality will be considerably soft due to HEAVY denoising! ( depending on how much weird temporal grain effects u can live with??) One would need an extremely high-quality picture to start with in the first place. 4K raw perhaps?... depending on your delivery format, could work for SD?

January 14th, 2009, 08:26 AM	#9
Biel Bestue Regular Crew Join Date: Jul 2005 Posts: 96	I think it will come once the cmos or the ccd can achieve it with the single exposure, because any other variant would limit shutter speed or the problem of having two much cameras mounted in a single place, so I think it will come once the chips can achieve that much latitude, even the "light stage" aproach doesn't work because it shoots more photographys per second (so it limits shutter speed), it will be all about the chips, and I think hdrc may have achieved it but at lower resolution, so it won't take them much time to get it to HD

January 15th, 2009, 06:45 AM	#11
Bob Hart Inner Circle Join Date: Dec 2003 Location: PERTH. W.A. AUSTRALIA. Posts: 4,477	A team over here was working on a product called the "smart filter". I have been out of touch for a good while now. Their "filter" that I saw was a pass-through device which selectively adjusted exposure by ND filtering at a pixel level on its filter screen. The level of ND was established by an operator choice. Their early coarser pitch prototype apparently enabled outdoors exposure to be correct yet indoors subjects to be also correct requiring only colour temperature filtering on the window or gels on interior lighting.

January 15th, 2009, 07:17 AM	#12
Karel Bata Regular Crew Join Date: Oct 2004 Location: London, Antarctica Posts: 199	Interesting concept. I've been looking around a bit and found a few very nice HDR videos. They're all time lapse but they show some of the problems like noise, artefacting, and some notable ringing. I note that not many folk decide to repeat the experience, suggesting that post may be time consuming, or that it's simply not easy to get it right! Some of these work best full screen. YouTube - HDR Time Lapse YouTube - HDR timelapse YouTube - HDR Evening Timelapse YouTube - HDR Timelapse massive ringing - but you notice the sun is in shot! HDR Sunflower time-lapse on Vimeo David Lynch would love this! Tamarama HDR Timelapse on Vimeo in HD! Point Judith Sunset on Vimeo HD too - very very nice this one. How many hours did he put into that?

January 15th, 2009, 09:50 AM	#13
Nicky Campos Regular Crew Join Date: Jan 2008 Location: Johannesburg, South Africa Posts: 35	Nice... although time-lapse HDRs dont really count as video. The ABC shutter alternating from frame to frame is a good idea - but how to actually implement it is difficult... How exactly? A similar trick would be to use a variable ND filter that darkens as u turn it - attach some sort of motor to continuously turn it so that frame to frame is darker then brighter ABC, etc. Single 12-bit DR exposures are no-brainer but are limited to that specific DR... limiting. Shallow DOF doesn't work with HDRs because it... well just is weird looking really... HDR look is surreal and painterly/toony - Just doesn't suit shallow DOF. PS Technik make an expensive and bulky 3d/stereo rig that can actually achieve inter-ocular distances so close - that two cameras can record the SAME image and therefore achieve multiple exposures!!!

January 15th, 2009, 10:40 AM	#14
Karel Bata Regular Crew Join Date: Oct 2004 Location: London, Antarctica Posts: 199	"but how to actually implement it is difficult... How exactly?" Ask a camera software hacker! Not my line really, but I'm sure it could be done. There's plenty of folk shooting stop-frame animation using a stills camera linked to a PC. So it would have to be auto-bracketing and locked into a continuous burst mode - effectively shooting at 72fps! Hmm... Another way would be to use an old-fashioned 3 CCD prism with 3 HD chips gen-locked together generating 3 video streams running to a hard drive (or 3 hard drives?). The technology for all that is well established. My 2p.

January 15th, 2009, 02:16 PM	#15
Nicky Campos Regular Crew Join Date: Jan 2008 Location: Johannesburg, South Africa Posts: 35	Theoretically it can be done Im sure... just like theoretically its possible to get the Canon 5D Mk2 to give us our dream camera! Find us a camera software hacker then. Auto bracketing is a still camera function - no still camera can run a 72FPS... fastest is 15P I think. For video cameras iris cannot be used cos DOF will change... leaving us with shutter or ND filters. Although if a "camera software hacker" could just implement NDs on the 3 chips separately (eg. green - no ND, Red - ND1, Blue - ND2) and of-coarse treat each chip as a single bayer pattern (eg. recover full color from each chip using software like Juan from Reel-stream) then maybe it can be done?