Mirage Recorder

[Take Vos]

Hello everyone,

There was some imperfections in the conversion from float to half, which was causing a small bit of colour banding. The only colour banding I have now seem to be caused by the LCD display of my MacBook Pro.

For the RAW12, I will have to convert the 14 bits from the DAC to the 12 bits in the data stream. So I will be using the LUT of the camera to keep precision in low light, but also have the latitude for high lights.

The idea is to use a truncated version of 16 bit floats (half). I take the 14 bit integer and divide it by 16384.0 (normalising from 0.0 to 1.0). Then I divide that result by 256.0, which causes the 16 bit float to:
- starts with an exponent of zero and a mantissa that increments by 4.
- It ends with an exponent of 6 and increments the mantissa by 1/8.
- sadly it has 1024 (512 after shift) numbers left unused.
- The first 2048 (1024 after shift) numbers encoded exactly.

Now I shift the 16 bit float one to the right. I am only using 9 bits of mantissa and 3 bits of exponent and no sign, 12 bits in total.

By encoding it in the LUT like this, I only have to shift it left by one before passing it on to the graphics card.

[Take Vos]

That was smart in theory, not in practice.
It seems that numbers with an exponent of zero are ignored in the calculations on the graphics card.

So I had to think of an other encoding. I first thought of 12 bit log, but the problem with that it would displace the measured values to fit on the log scale.

So I thought of an encoding where the first 1024 numbers are exact, after which it would slowly loose accuracy, but still have a somewhat monotone increments. The following shows how to define a table to convert 14 bit data into 12 bit.

    for (value = 0; value < 16384; value++) {
        if (value < 512) {
            e = 0;
            m = (value - 0) / 1;
        } else if (value < 1024) {
            e = 1;
            m = (value - 512) / 1;
        } else if (value < 2048) {
            e = 2;
            m = (value - 1024) / 2;
        } else if (value < 3584) {
            e = 3;
            m = (value - 2048) / 3;
        } else if (value < 5632) {
            e = 4;
            m = (value - 3584) / 4;
        } else if (value < 8704) {
            e = 5;
            m = (value - 5632) / 6;
        } else if (value < 12288) {
            e = 6;
            m = (value - 8704) / 7;
        } else {
            e = 7;
            m = (value - 12288) / 8;
        }
        
        uint14_to_uint1412[value] = (e << 9) | m;
    }

[Take Vos]

Hello everyone,

I thought I would share an other picture, this time I made on during dusk, from the trees in the park. So the image is incredibly dark.
This is still a screen shot, so we still don't have the full 14 bits of latitude.

http://www.vosgames.nl/products/Mira...derExposed.png

I am now sending the picture as integers to the graphics card, so I am no longer first converting them to 16 bit float. This would allow for a little bit more accuracy, as the graphics card would internally convert this to a normalised 32-bit float.

[Solomon Chase]

If your going commercial with this, it needs to capture at 1920x1080. You'd probably have to go 10-bit log or 8-bit. 8-bit would work well in 16-bit stream (no padding)

10-bit log is what Silicon Imaging is using for their high end 1080p camera. The 10-bit bayer capture codec they are licensing (cineform RAW) is a very efficient visually lossless wavelet codec. Since it is currently being ported to OSX with full quicktime support, you might want to inquire about licensing. They are cool guys..

I "push" graded the under exposed image you posted. (attached) Looks like there still a bit of information there, even if its not 14-bit :)

[Solomon Chase]

BTW, what is the bottleneck that is stopping you from getting the full 1920x1080? Harddrive speed? Firewire Bandwidth? Sensor Readout Speed? CPU?

[Take Vos]

Solomon,

The end result will be uncompressed RAW 12 bit non-linear (I call this '1412') with a resolution of 1920 x 800 (2.40:1). And it will be direct editable in QuickTime applications like Final Cut Pro.

As for the codec, I think it is too late now, I have designed my own codec to read the calibration data and compensate the image during playback. I have done this, so that the calibration is not fixed, i.e. you can redo the calibration or change calibration method after you have shot the footage.

The bottleneck right now is the Firewire 800 speed, and then after this the hard disk speed (although one could always make stripe set). But I actually like the idea that you can use a single disk to record on.

I could also design a '1408' or 8 bit log encoding, this would allow for a 2048 x 1080 image easily, but I am not sure if you really want to use only 8 bits to encode the image.

As you have probably noticed during the pushing of the image, there is quite a bit of "bias", I think this is caused by reflections inside the camera.
The C-mount which doubles as back-focus is made of bare aluminum. The 35mm lens is probably projecting onto this bare aluminum which gets reflected onto the sensor. I guess I will need to paint it, or design one that is black (but that may be prohibitively expensive for single runs).

I think there is also quite a lot of noise in the image as you have seen. The noise is quite random, so it does give some information about the actually darkness when you view it. I think I will need to shill the camera, so that the noise will be reduced. With a bit of luck noise is reduced by half for every 6 kelvin drop in temperature. Finally one could use make a temporal noise reduction algorithm running as a filter inside Final Cut Pro.

Cheers,
Take

[Take Vos]

Hello everyone,

Friday I received the new firmware for the Pike, which allows it to record RAW12 bit (through a LUT).
Mirage Recorder now can receive 1920 x 850 @ 24 fps, 12 bit.

Next step is getting this format to disk.

Cheers,
Take

[Solomon Chase]

Great!
I think 1920 x 850 and 1707 x 960 (or 1728 x 972?) should be good starting formats.

Unfortunately, as you have noticed, sensor temperature relates very much to noise. The Viper cinema camera has the same issue, and uses cooling to get better results. (very good results, so even uncooled isn't too bad)

[Take Vos]

Solomon,

I will design a housing for the camera which will include peltier cooling, so that I can cool to camera to around 5 degrees C or something like that.

With a bit of luck that is enough.

Cheers,
Take

[Take Vos]

Hello everyone,

I've just designed a new demosaic algorithm that works at near real time on a MacBook Pro GPU for 1920 x 850 @ 24 fps images.

I hope the description of the algorithm is easier to read than the AHD paper, but I am not really used to writing these kind of papers. It is only two pages, so don't be to shy, I would love to hear if anyone has any ideas.

http://www.vosgames.nl/products/Mira...R_demosaic.pdf

Here is the GLSL (OpenGL Shader Language) reference implementation.

http://www.vosgames.nl/products/Mira...FR_demosaic.fs

Cheers,
Take

[Igor Babic]

I just want you to know that I have great admire to your work here, and I love to see that someone has knowlege and will to do it. I hope that many others will see it and buy your software to do wonderfull things. Keep up your good work!

[Take Vos]

Here is a new update. Yesterday I got the quicktime component working for the first time. This includes processing of the RAW frames with OpenGL.

I also received a reply from Apple development support and they told me that Final Cut Pro can handle 16 bits per component (gamma 1.8) if you write your own codec, currently the highest quality codecs go up to 10 bits.

To handle high speeds in Final Cut Pro my codec has to be used as the timeline codec as well, to how many bits should I record processed RGB frames? The full 16 bits, or would lower like 12 or 10 suffice. Or record RGB as 10-12-10 (nice 32 bit words).

[Take Vos]

So I've got it working inside FCP, as you see I just dropped the video from the Finder straight onto the timeline. Of course I first configured the timeline to have the animation codec with a resolution of 1920x850 @ 24fps.

Here is a simple screenshot:
http://www.vosgames.nl/products/MirageRecorder/FCP1.png

I will now have to make it possible to use this codec on the timeline so that the red bar above the footage hopefully disappears.

[John C. Butler]

This project looks very promising. Take, how much is the Pike camera you're using? Someone on another thread said it costs several thousand dollars.

[Take Vos]

John,

You are correct, the camera cost somewhere between 4000 and 6000 dollars, depending I guess what reseller you can use. Maybe I can become a reseller for this camera myself in some way.

But I will also expand on this camera by adding cooling, preview-screen and tripod mountings. Also needed is some electronics or firmware change to get a stable clock, I was thinking of basing the clock on a word clock of an USB audio interface or an ambient lockit box.

I may also include some changes to the optics, where the current camera has a C-mount with a mirrored finish, which should really be matt black. And maybe I want to optically create some black lines on the sensor for continues calibration purposes.

Lastly some kind of calibration set up is also needed, where there is a white light that has a very even surface, with a specific light output at a specific distance.