Is it possible to downsample RAW in-camera without demosaicing? at DVinfo.net

Jack Kelly · November 15th, 2008, 09:35 AM

Hello,

I'm not sure where is best to ask this question but I figured that the Cineform forum would be a good choice because Cineform clearly know an awful lot about bayer sensors!

Is it theoretically possible to downsample a RAW image without demosaicing? For example, if you had a 10megapixel bayer sensor then would it be possible to downsample in-camera to produce, say, a 5megapixel RAW bayer signal?

Or can downsampling only be accomplished *after* demosaicing and hence the pixel dimensions of a camera's RAW output always has to be exactly the same as the sensor's pixel dimensions?

The reason I ask is because this question has popped up a number of times in the recent discussions about RED's latest offerings and about the possibility of seeing a low-cost, high-performance movie mode in a dSLR some time soon. Lots of people want a single camera which performs well as both a stills camera and as a movie camera recording compressed RAW. The problem is that stills cameras have huge pixel dimensions (10-20Mpixels) and it seems a little uneconomical to try to capture such huge frame sizes for most movie applications, especially when recording to off-the-shelf flash media. But, on the other hand, it would be nice to record a RAW movie format. Hence the question: is it possible to downsample a bayer image without demosaicing?

Is the best compromise to capture the entire bayer image but use relatively aggressive wavelet compression to get the bitrate down?

Many thanks,
Jack

David Taylor · November 15th, 2008, 08:58 PM

Jack, it is theoretically possible to resample Bayer data, but practically it is not really possible. The reason is that each of the Bayer pixel locations, although identified by convention as R, G, or B sites, are not pure in their color. In other words, each cell site has color bleeds that must be extracted through a demosaic algorithm. Color bleeds exist because of impure color masks at each pixel location. The color bleeding is not a problem by the way, instead it is a characteristic. But it requires a DeBayer (demosaic) filter to extract color purities from the individual cell sites. But of course when you perform a demosaic you create R, G, and B values at each pixel location. If you do NOT perform a demosaic, and just perform a more traditional down-res you get color aliasing that is visually objectionable.

Jack Kelly · November 17th, 2008, 04:10 AM

Hi David!

Many thanks for the excellent reply! Very informative, many thanks.

May I ask one more quick question: is it possible to DeBayer & downsample in one step? What I'm thinking is this: If I understand correctly, Bayer sesors have pixels in groups of 4 colours (e.g. GRGB). Is there a "mode" of DeBayering which outputs a single pixel (with RGB values) for every group-of-4-pixels on the sensor? i.e. the effect would be that the algorithm would DeBayer AND downsample in one go. So, for example, if you had a Bayer sensor with 3840x2160 photosites ("8.3megapixels") then you'd end up with a 1920x1080 RGB image (with a quarter of the noise present in a single photosite) in a single computational step. Is that possible?

Although, the more I read about DeBayering, the clearer it is that DeBayering is not something you want to compromise on and that - in an ideal filmmaking workflow - it's best to have as much control over the DeBayering as possible (i.e. for most "film" applications, you want to capture RAW off the sensor and DeBayer later - exactly the idea behind CineformRAW, CinemaDNG, REDraw etc etc). Some blogs have blamed some of the D90's ugly artefacting (in movie mode) on the use of pixel binning but I wonder if it's more accurate to blame some of the D90's artefacts on inaccurate quick-and-dirty DeBayering (I'm just guessing, I know nothing about the internals of the D90)?

David Taylor · November 17th, 2008, 08:52 AM

I also don't know what the D90 uses for their demosaicing strategy, so I won't jump into that one.

Yes, we demosaic and resize automatically. In fact, because CineForm builds demosaic into the codec function, when you want to resample spatially, we always first preform a demosaic.

The choice for demosaic when using PPro are in the Playback Settings panel when Prospect 4K is installed.

If not using PPro then the SetActiveMetadata utility (installed with Neo 4K and Prospect 4K) allows the choice of demosaic when rendering in other applications (like AE).

Jack Kelly · November 18th, 2008, 05:00 AM

Great stuff, many thanks for the reply.

November 15th, 2008, 09:35 AM	#1
Jack Kelly Major Player Join Date: Oct 2004 Location: London, UK Posts: 243	Is it possible to downsample RAW in-camera without demosaicing? Hello, I'm not sure where is best to ask this question but I figured that the Cineform forum would be a good choice because Cineform clearly know an awful lot about bayer sensors! Is it theoretically possible to downsample a RAW image without demosaicing? For example, if you had a 10megapixel bayer sensor then would it be possible to downsample in-camera to produce, say, a 5megapixel RAW bayer signal? Or can downsampling only be accomplished after demosaicing and hence the pixel dimensions of a camera's RAW output always has to be exactly the same as the sensor's pixel dimensions? The reason I ask is because this question has popped up a number of times in the recent discussions about RED's latest offerings and about the possibility of seeing a low-cost, high-performance movie mode in a dSLR some time soon. Lots of people want a single camera which performs well as both a stills camera and as a movie camera recording compressed RAW. The problem is that stills cameras have huge pixel dimensions (10-20Mpixels) and it seems a little uneconomical to try to capture such huge frame sizes for most movie applications, especially when recording to off-the-shelf flash media. But, on the other hand, it would be nice to record a RAW movie format. Hence the question: is it possible to downsample a bayer image without demosaicing? Is the best compromise to capture the entire bayer image but use relatively aggressive wavelet compression to get the bitrate down? Many thanks, Jack __________________ Jack Kelly's filmmaking website AfterCarbon.org

November 17th, 2008, 04:10 AM	#3
Jack Kelly Major Player Join Date: Oct 2004 Location: London, UK Posts: 243	Hi David! Many thanks for the excellent reply! Very informative, many thanks. May I ask one more quick question: is it possible to DeBayer & downsample in one step? What I'm thinking is this: If I understand correctly, Bayer sesors have pixels in groups of 4 colours (e.g. GRGB). Is there a "mode" of DeBayering which outputs a single pixel (with RGB values) for every group-of-4-pixels on the sensor? i.e. the effect would be that the algorithm would DeBayer AND downsample in one go. So, for example, if you had a Bayer sensor with 3840x2160 photosites ("8.3megapixels") then you'd end up with a 1920x1080 RGB image (with a quarter of the noise present in a single photosite) in a single computational step. Is that possible? Although, the more I read about DeBayering, the clearer it is that DeBayering is not something you want to compromise on and that - in an ideal filmmaking workflow - it's best to have as much control over the DeBayering as possible (i.e. for most "film" applications, you want to capture RAW off the sensor and DeBayer later - exactly the idea behind CineformRAW, CinemaDNG, REDraw etc etc). Some blogs have blamed some of the D90's ugly artefacting (in movie mode) on the use of pixel binning but I wonder if it's more accurate to blame some of the D90's artefacts on inaccurate quick-and-dirty DeBayering (I'm just guessing, I know nothing about the internals of the D90)? __________________ Jack Kelly's filmmaking website AfterCarbon.org

November 18th, 2008, 05:00 AM	#5
Jack Kelly Major Player Join Date: Oct 2004 Location: London, UK Posts: 243	Great stuff, many thanks for the reply. __________________ Jack Kelly's filmmaking website AfterCarbon.org

November 15th, 2008, 08:58 PM	#2
David Taylor Major Player Join Date: Sep 2003 Location: Solana Beach, CA Posts: 853	Jack, it is theoretically possible to resample Bayer data, but practically it is not really possible. The reason is that each of the Bayer pixel locations, although identified by convention as R, G, or B sites, are not pure in their color. In other words, each cell site has color bleeds that must be extracted through a demosaic algorithm. Color bleeds exist because of impure color masks at each pixel location. The color bleeding is not a problem by the way, instead it is a characteristic. But it requires a DeBayer (demosaic) filter to extract color purities from the individual cell sites. But of course when you perform a demosaic you create R, G, and B values at each pixel location. If you do NOT perform a demosaic, and just perform a more traditional down-res you get color aliasing that is visually objectionable.

November 17th, 2008, 08:52 AM	#4
David Taylor Major Player Join Date: Sep 2003 Location: Solana Beach, CA Posts: 853	I also don't know what the D90 uses for their demosaicing strategy, so I won't jump into that one. Yes, we demosaic and resize automatically. In fact, because CineForm builds demosaic into the codec function, when you want to resample spatially, we always first preform a demosaic. The choice for demosaic when using PPro are in the Playback Settings panel when Prospect 4K is installed. If not using PPro then the SetActiveMetadata utility (installed with Neo 4K and Prospect 4K) allows the choice of demosaic when rendering in other applications (like AE).