Mirage Recorder

Wayne Morellini · May 27th, 2007, 06:01 AM

Luckily, I believe in helping people, even if they don't reply to my emails, so here goes. You may notice people are not responding here, but go to the forum view it will show you how many times your page has been viewed. It is likely there are an number of people in the background keeping an eye on your progress.

To add some useful information, OpenGL has been reworked and jelled together, with two new versions coming. Version 2.6 will have an reworked Open GL, and version 3.0 will add DirectX 10 like GPU features that can work under XP (unlike standard dx10). So, you can have compatibility under Mac OS, Linux, Windows XP and Vista (and hand helds).

If I have not already, I aim to add an link to your thread from my thread so that everybody still subscribed can come and have an look.

Now, over at the Elphel thread, Zolt is stopping his compressed Raw Bayer project for the camera, and offering it up to who ever would like to take over.

I would like to ask some questions about your software that others might be interested in. Is it commercial, for what machines, OS's and cameras do you eventually aim ti make it for? Have you considered doing it for the many HD video enabled web cams coming out?

Thanks in advance

Wayne.

Take Vos · May 27th, 2007, 06:48 AM

Hello Wayne,

Did I forget to reply to an email? I am sorry about that. I do remember an email that asked me to post my findings as I am progressing with the application, which as you see I am doing.

OpenGL is sometimes quite messy and it is easy to get OpenGL in a not so desirable state, or even crash the computer as there is little checking done in the OS for performance reasons. Anyway I am using something along the line of OpenGL 2, but I do not know all the extensions and arbs that have been renamed in OpenGL 2. It doesn't really matter anyway.

As for your other questions:

Yes, it will be commercial, I will sell the license key for Mirage Recorder (an extension on top of Boom Recorder). I do not yet know if I will sell the complete system including camera head, mount, viewfinder, etc. If I don't sell the complete system I will include information in the manual on how to acquire and use all these components.

Currently I am using a Mac Book Pro for development and as target machine. A Mac Pro can probably also be used. As I use Boom Recorder as the base of this project I will not be able to target to Windows or Linux yet, unless OpenStep makes strides to be as complete as Apple's APIs.

I am also writing the software for the Pike 210 for now, although I am keeping my options open for other IIDC cameras. For example the file formats include information about the color space and pixel format. In the future I will only consider cameras that can be used to get the raw sensor data.

Why is Zolt stopping with his project?

Cheers,
Take

Djee Smit · May 27th, 2007, 08:34 AM

Hey Take, nice to see you here, I met you some time back, (Jonathan 'Djee; Smit) when I was doing a internship at Pat's company. He mentioned recently that you are working on a camera. It all sounds interesting. Keep up the good work.

Take Vos · May 27th, 2007, 09:14 AM

Hello Jonathan,

It is indeed pretty exiting. I hope I will be able to let someone use the camera soon. With a bit of luck Patrick will find a project for it.

Cheers,
Take

Djee Smit · May 27th, 2007, 09:41 AM

If Pat doesn't have anything soon, I might have some things coming up. a videoclip and a short film. (might be interesting then) I will keep an eye on this thread for the development.

Wayne Morellini · May 28th, 2007, 08:51 AM

Quote:

Originally Posted by Take Vos

Hello Wayne,

Did I forget to reply to an email? I am sorry about that. I do remember an email that asked me to post my findings as I am progressing with the application, which as you see I am doing.

Nope, no replies here (did I even ask to keep us informed by email, there was at least another email) or other replies.

OpenGL 2.6 is supposed to be cleaned up to be less messy.

Commercial, and 1394, I wish you luck, pity Apple never released an HD web cam. An lot of people round here used USB and GigE cameras, because the Firewire cones were so horribly expensive. Automotive firewire HD cameras might be potentially an lot cheaper, and some have very high latitude.

Zolt, is ready to start filming, so has run out of time to get the project ready, and needs to spend the money on hiring an camera.

Take Vos · May 28th, 2007, 10:06 AM

Hi Wayne,

Well, I don't have my screenplay ready for filming yet, so I think I will have a working solution before I start.

Anyway I have a question for everyone, I am making my calibration software and one of the features is that it shows some statistics about the footage that is loaded. This includes the average intensity, the amount of temporal noise and the signal to noise ratio.

In audio I would use dB and dBFS (decibel full scale) to show these values, which is probably also used in video signal processing. But I guess the people who would use this are more comfortable with stops.

So if a sensor would clamp its values between 0.0 (black) and 1.0 (white), then I would say a value of 0.125 to be: -20 dBFS or -3 stops.

What would you say?

Cheers,
Take

Take Vos · June 9th, 2007, 09:13 AM

Hello everyone,

I've made a calibration application and the recording software can read this calibration data. One problem is that I do not seem to be able to remove the static noise from the raw image yet. I believe this has to do with the temperature of the sensor. As a typical CCD sensor rises in temperature by 5 kelvin, the black current noise is doubled.

The calibration data I took was made at room temperature, while the footage I was attempting to compensate was made with a hot camera. The Pike gets quite hot, to a temperature where you can no longer hold the camera by hand, so I guess the temperature at the sensor is at least 50 kelvin higher.

I will need to redo the sensor calibration when the sensor has been exercising for at least 30 minutes. I will also look into how to keep the temperature low and stable.

I will probably be using the "Adaptive homogeneity-directed interpolation" demosaic algorithm in the recording application and in the QuickTime component. As soon as I figure out all of the math, I am getting there.

As you know the resolution is 1800 * 750 * 14 bits, but I believe it will be possible to handle 2048 * 850 * 12 bits in the future.

Cheers,
Take

Take Vos · June 22nd, 2007, 03:34 AM

I have three new pictures from my camera.
These are screenshots and it was scaled to fit the resolution of my screen by the recording application.

It shows the new demosaic algorithm and uses the color conversion matrix specified by the camera. It also does black current removable and compensates for the sensitivity of each pixel (although these don't work correct yet).

http://www.vosgames.nl/products/Mira...der/AHDDA1.png
http://www.vosgames.nl/products/Mira...der/AHDDA2.png
http://www.vosgames.nl/products/Mira...der/AHDDA3.png

Cheers,
Take

Jose A. Garcia · June 22nd, 2007, 04:08 AM

Those images look great Take! Looking forward to seeing more of your work.

David Delaney · June 22nd, 2007, 06:01 AM

This might be old news, but I heard that Kodak has a new patterning system that is better than Bayer and leads to less artifacting in low light.
http://www.pcworld.com/article/id,13...1/article.html

Take Vos · June 22nd, 2007, 07:55 AM

Hello David,

Yes, I read it on slashdot, it is kind of interesting. But you would have to modify the demosaic algorithm to handle it. I do not yet have the qualifications to create a demosaic algorithm from scratch.

I have seen some interesting other sensors, whereby there are smaller pixels around the large pixels, the smaller pixels could handle high luminosity.

Cheers,
Take

Take Vos · July 2nd, 2007, 03:15 PM

Hello everyone,

Here is a new screenshot, it shows the R,G,B waveform of part of a GretagMacbeth.

http://www.vosgames.nl/products/Mira...r/waveform.png

It is a bit disappointing as it uses to much CPU time. I don't like it when the fans of my notebook turn on.

But the simpler color and greyscale versions are fast and don't use too much CPU/GPU time.

Take Vos · July 5th, 2007, 01:22 PM

Hello everyone,

Could you please look at this noisy picture and tell me what you think?
Is this normal for an uncalibrated raw sensor image?

I have tried eliminating the noise with a dark image and a flat field image, but it seems that this noise is not linear depended on the pixel intensities.

http://www.vosgames.nl/products/Mira...FixedNoise.png

Jose A. Garcia · July 5th, 2007, 01:35 PM

Well... I would say no. I'm using CMOS and I know that's a CCD but I can't believe picture is so different comparing both sensors.

May 27th, 2007, 08:34 AM	#18
Djee Smit Regular Crew Join Date: May 2005 Location: Breda, Netherlands Posts: 141	Hey Take, nice to see you here, I met you some time back, (Jonathan 'Djee; Smit) when I was doing a internship at Pat's company. He mentioned recently that you are working on a camera. It all sounds interesting. Keep up the good work. __________________ www.wanah.nl

May 27th, 2007, 09:41 AM	#20
Djee Smit Regular Crew Join Date: May 2005 Location: Breda, Netherlands Posts: 141	If Pat doesn't have anything soon, I might have some things coming up. a videoclip and a short film. (might be interesting then) I will keep an eye on this thread for the development. __________________ www.wanah.nl

June 22nd, 2007, 03:34 AM	#24
Take Vos Major Player Join Date: Mar 2007 Location: Amsterdam The Netherlands Posts: 200	I have three new pictures from my camera. These are screenshots and it was scaled to fit the resolution of my screen by the recording application. It shows the new demosaic algorithm and uses the color conversion matrix specified by the camera. It also does black current removable and compensates for the sensitivity of each pixel (although these don't work correct yet). http://www.vosgames.nl/products/Mira...der/AHDDA1.png http://www.vosgames.nl/products/Mira...der/AHDDA2.png http://www.vosgames.nl/products/Mira...der/AHDDA3.png Cheers, Take __________________ VOSGAMES, http://www.vosgames.nl/ developer of Boom Recorder and Mirage Recorder

June 22nd, 2007, 07:55 AM	#27
Take Vos Major Player Join Date: Mar 2007 Location: Amsterdam The Netherlands Posts: 200	Hello David, Yes, I read it on slashdot, it is kind of interesting. But you would have to modify the demosaic algorithm to handle it. I do not yet have the qualifications to create a demosaic algorithm from scratch. I have seen some interesting other sensors, whereby there are smaller pixels around the large pixels, the smaller pixels could handle high luminosity. Cheers, Take __________________ VOSGAMES, http://www.vosgames.nl/ developer of Boom Recorder and Mirage Recorder

July 2nd, 2007, 03:15 PM	#28
Take Vos Major Player Join Date: Mar 2007 Location: Amsterdam The Netherlands Posts: 200	Hello everyone, Here is a new screenshot, it shows the R,G,B waveform of part of a GretagMacbeth. http://www.vosgames.nl/products/Mira...r/waveform.png It is a bit disappointing as it uses to much CPU time. I don't like it when the fans of my notebook turn on. But the simpler color and greyscale versions are fast and don't use too much CPU/GPU time. __________________ VOSGAMES, http://www.vosgames.nl/ developer of Boom Recorder and Mirage Recorder

May 27th, 2007, 06:01 AM	#16
Wayne Morellini Inner Circle Join Date: May 2003 Location: Australia Posts: 2,762	Luckily, I believe in helping people, even if they don't reply to my emails, so here goes. You may notice people are not responding here, but go to the forum view it will show you how many times your page has been viewed. It is likely there are an number of people in the background keeping an eye on your progress. To add some useful information, OpenGL has been reworked and jelled together, with two new versions coming. Version 2.6 will have an reworked Open GL, and version 3.0 will add DirectX 10 like GPU features that can work under XP (unlike standard dx10). So, you can have compatibility under Mac OS, Linux, Windows XP and Vista (and hand helds). If I have not already, I aim to add an link to your thread from my thread so that everybody still subscribed can come and have an look. Now, over at the Elphel thread, Zolt is stopping his compressed Raw Bayer project for the camera, and offering it up to who ever would like to take over. I would like to ask some questions about your software that others might be interested in. Is it commercial, for what machines, OS's and cameras do you eventually aim ti make it for? Have you considered doing it for the many HD video enabled web cams coming out? Thanks in advance Wayne.

May 27th, 2007, 06:48 AM	#17
Take Vos Major Player Join Date: Mar 2007 Location: Amsterdam The Netherlands Posts: 200	Hello Wayne, Did I forget to reply to an email? I am sorry about that. I do remember an email that asked me to post my findings as I am progressing with the application, which as you see I am doing. OpenGL is sometimes quite messy and it is easy to get OpenGL in a not so desirable state, or even crash the computer as there is little checking done in the OS for performance reasons. Anyway I am using something along the line of OpenGL 2, but I do not know all the extensions and arbs that have been renamed in OpenGL 2. It doesn't really matter anyway. As for your other questions: Yes, it will be commercial, I will sell the license key for Mirage Recorder (an extension on top of Boom Recorder). I do not yet know if I will sell the complete system including camera head, mount, viewfinder, etc. If I don't sell the complete system I will include information in the manual on how to acquire and use all these components. Currently I am using a Mac Book Pro for development and as target machine. A Mac Pro can probably also be used. As I use Boom Recorder as the base of this project I will not be able to target to Windows or Linux yet, unless OpenStep makes strides to be as complete as Apple's APIs. I am also writing the software for the Pike 210 for now, although I am keeping my options open for other IIDC cameras. For example the file formats include information about the color space and pixel format. In the future I will only consider cameras that can be used to get the raw sensor data. Why is Zolt stopping with his project? Cheers, Take

May 27th, 2007, 09:14 AM	#19
Take Vos Major Player Join Date: Mar 2007 Location: Amsterdam The Netherlands Posts: 200	Hello Jonathan, It is indeed pretty exiting. I hope I will be able to let someone use the camera soon. With a bit of luck Patrick will find a project for it. Cheers, Take

May 28th, 2007, 10:06 AM	#22
Take Vos Major Player Join Date: Mar 2007 Location: Amsterdam The Netherlands Posts: 200	Hi Wayne, Well, I don't have my screenplay ready for filming yet, so I think I will have a working solution before I start. Anyway I have a question for everyone, I am making my calibration software and one of the features is that it shows some statistics about the footage that is loaded. This includes the average intensity, the amount of temporal noise and the signal to noise ratio. In audio I would use dB and dBFS (decibel full scale) to show these values, which is probably also used in video signal processing. But I guess the people who would use this are more comfortable with stops. So if a sensor would clamp its values between 0.0 (black) and 1.0 (white), then I would say a value of 0.125 to be: -20 dBFS or -3 stops. What would you say? Cheers, Take

June 9th, 2007, 09:13 AM	#23
Take Vos Major Player Join Date: Mar 2007 Location: Amsterdam The Netherlands Posts: 200	Hello everyone, I've made a calibration application and the recording software can read this calibration data. One problem is that I do not seem to be able to remove the static noise from the raw image yet. I believe this has to do with the temperature of the sensor. As a typical CCD sensor rises in temperature by 5 kelvin, the black current noise is doubled. The calibration data I took was made at room temperature, while the footage I was attempting to compensate was made with a hot camera. The Pike gets quite hot, to a temperature where you can no longer hold the camera by hand, so I guess the temperature at the sensor is at least 50 kelvin higher. I will need to redo the sensor calibration when the sensor has been exercising for at least 30 minutes. I will also look into how to keep the temperature low and stable. I will probably be using the "Adaptive homogeneity-directed interpolation" demosaic algorithm in the recording application and in the QuickTime component. As soon as I figure out all of the math, I am getting there. As you know the resolution is 1800 * 750 * 14 bits, but I believe it will be possible to handle 2048 * 850 * 12 bits in the future. Cheers, Take

June 22nd, 2007, 04:08 AM	#25
Jose A. Garcia Major Player Join Date: May 2007 Location: Sevilla (Spain) Posts: 439	Those images look great Take! Looking forward to seeing more of your work.

June 22nd, 2007, 06:01 AM	#26
David Delaney Trustee Join Date: Feb 2006 Location: Niagara Ontario Canada Posts: 1,121	This might be old news, but I heard that Kodak has a new patterning system that is better than Bayer and leads to less artifacting in low light. http://www.pcworld.com/article/id,13...1/article.html

July 5th, 2007, 01:22 PM	#29
Take Vos Major Player Join Date: Mar 2007 Location: Amsterdam The Netherlands Posts: 200	Mirage Recorder: Weird fixed noise issue Hello everyone, Could you please look at this noisy picture and tell me what you think? Is this normal for an uncalibrated raw sensor image? I have tried eliminating the noise with a dark image and a flat field image, but it seems that this noise is not linear depended on the pixel intensities. http://www.vosgames.nl/products/Mira...FixedNoise.png __________________ VOSGAMES, http://www.vosgames.nl/ developer of Boom Recorder and Mirage Recorder

July 5th, 2007, 01:35 PM	#30
Jose A. Garcia Major Player Join Date: May 2007 Location: Sevilla (Spain) Posts: 439	Well... I would say no. I'm using CMOS and I know that's a CCD but I can't believe picture is so different comparing both sensors.