4:4:4 10bit single CMOS HD project

[Wayne Morellini]

As I understand it the human eye population has the potential for 2400 Dots Per Inch discernment (from a close veiwing distance) this only in a small scanning region of the central vision, and individual eys in the peopulation may have a lot less potential. There is a methord to train your eye to see better, I think the book, better eye sight withoiut glasses might cover tha one, by shifting the brains region of central vision to the true centre (apparently a lot of people concentrate on an area of the eye that is not true centre, and concentrate on a lower res side region, and by practicing recognition of details (improving the brains recognition). 720p works out around 150dpi, colour vision goes to around 1200 dpi, so here is a big difference between what we (or Imax, which looks a lot better than 2K) use and the true top end. I think the problem is grain size, the smaller the grain the slower it is, so previously 2K images were probably only avialble in high lit situations on regular (cheap) fiilm stocks. If you look at it the increase resolution of Imax over 2K has a lot to do with the fact that the film frame is ten times bigger and can fit in ten times more grains in the projection (but it fills more screen so the difference to us is probably 4 times, like normal theatre at 8K and 16K to us, but extra winde and high). Now even with my poor eyesight I can pick grain normal cinema and in dark scenes in Imax. So whats the max we should worry about, tricky, I would say between 1Mp 720p and 16Mp (max colour res for radiated display).

Newer technologies, like 4K projection, can have more optical quality issues (plus it depends on room lighting and screen used). Resolution is not the complete picture, so I imagine that it would be quite easy toput up a worked out 2K projector against a tobe optimised 4K one. If your wondering about the limits of a 4:3 15 inch monitor on the eyesight, it's 64Mp (actually why are we saying K it's millions of pixels in a frame) for Monochrome, you will notice that this is one quater what it should be, that is because gradiated displays effect vision and halve the resolution in each direction (16Million for colour). But you will notice diminishing returns for doubling resolution over 150dpi )where surounding pixels are starting to intergrate with general vision). This 4Mp one, was that Sony's new ribbon tech?

[Wayne Morellini]

Soeren, yes there was big differences when the Intel compilers came out, because most compilers were using a very limited part of the insruction set. I don't know if the Intel one sues all he instruction set properly, but I imagine that it can't compete with the carefull instruction setup and manipulation of a good programmer, that even coding in C way stops the compiler from being able to organise such alternative sequences of instrucions.

But on to the question of project size, above a certain size it becomes rapidly more difficult to program MC than high level languages, so optimising for the 90%performance regions is best, like you guys say. But programming in MC can eliminate many errors, and what happens when programming in MC makes code 10 times smaller. So project size is everything. Even though much that was said about doing capture completely in MC was figuative, in reality only for embedded custom systems where capture canbe much smaller, due to lack of major OS and simplified standardised hardware model ;) but still major project. For Windows PC, optimise crucial sections in MC, optimise rest in C, as Rob says, with obviouse results. I would like to do my OS in MC because complete Windows completing OS could fit in 1-10MB (data structures etc) but this will require team of programmers and lots of money to do in yeart time frame. I have new programming team stratergy worked out (computer science) to prodcue best quality with top MC programmers (that don't normally work in team mode) exciting stuff, there is a sepcific benefit, I could even patent the stratergy, but in reality I might have o do in C then transfer it to VOS code, which is a high level form of low level language, so easier than C.

[Wayne Morellini]

David you mentioned the cineform visually lossless codec for Bayer with 4:1 compression. I read your normal codec has a range of 6:1-10:1 for visually lossless, what is the range for the bayer codec, can we get to 10:1 reliably with multi generation on bayer? I have worked out that 4:! bayer 720p is close to a 50Mb/s stream which is close to 10:1 4:4:4 3 chip, which is very useful.

Obviously in camera compression we need true lossless. visually lossless and down to the quality of HDV2 50MB's stream for outside pro work (maybe equivalent to your codec at 10:1) for different jobs. Have you thought of licensing your codec to camera manufacturers (like Sumix, Drake, SI, Micron, Rockwell, Sony etc etc) in FPGA design (that canbe converted to high speed cheap custom silicon core reasonably easily, if anyboidy wants to mass market chips based on it)?

Thanks

Wayne.

[Rob Lohman]

I think we are all on the same page in regards to hand crafting
certain pieces of code in assembly. This is only done after some
profiling and after implementation with some good testing to
make sure it increases the throughput enough. But as Rob's
example shows it is clearly a win situation on such demanding
applications.

Compilers have gotten far, FAR better at optimizing code them-
selves, but it still has troubles in optimally using registeres and
memory in my opinion which is one of the major places to speed
code up.

But as I said, I think we are all on the same page in that regard!

[Wayne Morellini]

Don't forget intiutive leaps to accuratelly using odd little used instructions in interesting sequences that somehow abstractly speed up realtime performance, instead of the more obviouse compiler sequences ;)

[Rai Orz]

What is the best resulotion for movie and cinema?

I think ARRI, as one of the best (film) camera produces since beginning of cinema, know the answer. So lets look a little bit inside of they first digital cinema camera D20:

http://www.arri.de/news/newsletter/a...211103/d20.htm

In this newsletter are some details about the chip, pixels and data outputs. You can also read some things between the lines.

Thats the goal, why not?

[Rob Lohman]

There is also the Panavision Genesis beside the ARRI. I've done
a bit of information gathering:

Arri D20:
+ sensor: single 35mm 12 bit CMOS max 150 fps
+ sampling: standard bayer GR/BG
+ resolution: 3018 x 2200
+ framerates: 1 - 60 fps including 23.976 and 29.97 fps
+ shutter: mirror + electronic
+ mount: 54mm PL
+ internal bus: 10 Gb/s (gbit?)
+ power consumption: 54 W @ 24 fps (without viewfinder)

+ video mode:
- 2880 x 1620 sampling (16:9)
- 1920 x 1080 output (16:9)
- YUV 4:2:2 10 bit (single HD-SDI)
- RGB 4:4:4 10 bit (dual HD-SDI)
- Super 35 HDTV aperture size

+ film mode:
- 3018 x 2200 sampling (4:3)
- raw bayer output 12 bit
- up to ANSI Super 35 aperture

http://www.arri.de/news/newsletter/a...211103/d20.htm
http://www.arri.de/prod/cam/d_20/articles.htm
http://www.arri.de/prod/cam/d_20/tech_spec.htm


Panavision Genesis:
+ sensor: 35mm (probably 3 or foveon?)
+ sampling: full RGB
+ resolution: 12.4 mega pixel
+ framerates: 1 - 50 fps
+ 10 bit log output (1920 x 1080?)
+ 4:2:2 single HD-SDI out
+ 4:4:4 dual HD-SDI out

http://www.panavision.com/product_de...e=c0,c202,c203


Unfortunately there is almost no information available on the
technical specs of the Pana Genesis. Too bad. At least we know
that in film mode with the ARRI you are supposed to crop to your
favorite resolution. So we get:

16:9 => 3018 x 1698 (22.82% loss)
1.85 => 3018 x 1630 (25.91% loss)
2.35 => 3018 x 1284 (41.64% loss)

However, if they where to attach an anamorphic lens creating
a pixel aspect ratio of 1.78 or an output resolution of 3910 x 2200.

This is already 16:9 so no loss for that, for the others:

1.85 => 3910 x 2114 (03.91% loss)
2.35 => 3910 x 1664 (24.36% loss)


The ARRI article had an interesting discussion on de-bayering:

If you look at the pixels you will notice that each red pixel, for instance, is surrounded by four green and four blue pixels. Also, because there is an overlap in the color spectra of red, green and blue, the available red value is at least in part the result of light in another color. Based on the knowledge of what the colors and values of those neighbor-pixels are, and based on the knowledge of the overlap in the color spectra, it is now possible to work out (reconstruct) what the green and blue values for that red pixel should be.

This process is more accurate than the interpolation used to increase the size (i.e. pixel count) of an image. In interpolation, completely new pixels are "made up" based on what the neighboring pixels look like. In Bayer data reconstruction we already have pixels, we just don't know two of the three color values. Since we do know the colors and values of neighbor pixels and since there is a color spectrum overlap, we can reconstruct the missing information very accurately.

Please note that the actual color reconstruction is more complicated than the method described here. For instance, to determine a given color value for a given pixel we use more than just the eight neighboring pixels. Furthermore, it is also possible to improve the result by incorporating certain assumptions about real world images in the algorithms (e.g. colors coincide at edges, etc.). We have simplified the process in this description to aid in understanding.

[Wayne Morellini]

Your quote above won't turn up in the fourm quote reply function,must be a bug.

Recreate them accurately, sure! Lets see some spectral overlap=impurity (I've thought about this type of technique before), taking a punt at solving the impurity and selling it as an advantage is restoring missing colour, OK but impurity also=reduced accuracy on the origional colour. So how much impurity and how much origional colour you need, if impurity is low it only gives few bits of sccuracy, not 12bits, but enough to tell of a major swing and interpolate a more accurate replacement. But how much of the accurate primary colour for that pixel is left, 10, 8 bit. But if you look at what they say, they estimate from image principles in nature (like bayer does that chroma tends to follow), so good guessing based on approx.

Now I would also like to say, lets see them do that with SD resolution frame, as I say before the increase resolution over 1080, hides much (artifacts and miss-approximation). As I say before 720p is a territory where pixels start blurring in with each other, so unless they look for it, casual viewer may not mentally notice as much, even if picture appears to be of less quality than accurate 3 chip SHD picture to them. So the you can say the impressiveness of picture then becomes sublinimal (??) noticable but not enough to put finger on for most of audiance. After upscaling these malformations could be smoothed out, making the picture a little softer, but imperfections/details start to dissapear. Why I wanted to go to 3 chip 720p as a minium instead of SD, or 2160p in bayer.

The truth of these pixel resolutions of these sensors they are using might be a technological limitation and marketable advantage over film than real audience limtis. I think with three chip you can get more light and need less resolution because it is probably better to resolution upscale. I'm going to take a punt, get three SD pal chips and offset them on a prism to obtain a calculable HD image ;). How much worse would this be than one chip bayer?

Maybe so :)

[Obin Olson]

i hate to say it Rob but if your supporting the 1300 I really think it is a waste of your time...that chip just can't cut it for a professional camera...the smear is that bad I think..I would say keep going though because the 3300 is not much to change code-wise to make it work..just WAY more data :)

Rob have you thought about sending the 1300 back and getting a 3300rgb?

[Steve Nordhauser]

Obin in 1300:
You could be right. I defer to people who know the application on this one. Maybe broadcast?

Wayne on color overlap:
Go to any color camera or sensor datasheet and you should see a spectral response curve. This is the cutoff for the bandpass color filter array (CFA) that is put on the sensor for single chip Bayer color. As in audio filtering, the rising edges are not vertical, they slope. This causes an overlap at the R-G and G-B borders of the wavelengths - the color impurity.

You probably need this, now that I think about it. Otherwise, a green object would look the same green (just different intensities as the filter response changes) as it changed in wavelength until it crossed over a filter boundary. Hmm, so it is this overlap that allows you to have a continuous color response. You learn something new every day.

[Obin Olson]

I *should* have 8bit 1080p captures today...I will keep everyone posted

[Wayne Morellini]

Sorry Steve, I thought they were talking about something else. About the 1300, is there a way to get rid of the smear etc, by not adjusting the gain to high or something? Will there be a new sensor version?

[David Newman]

<<<-- Originally posted by Wayne Morellini : David you mentioned the cineform visually lossless codec for Bayer with 4:1 compression. I read your normal codec has a range of 6:1-10:1 for visually lossless, what is the range for the bayer codec, can we get to 10:1 reliably with multi generation on bayer? I have worked out that 4:1 bayer 720p is close to a 50Mb/s stream which is close to 10:1 4:4:4 3 chip, which is very useful.-->>>

In our tests, visually lossless Bayer is ranges between 4:1 and 6:1. For 1280x720@24p 10bit that is between 36Mb/s and 56Mb/s (data rates double for 1920x1080p24) which will allows you to record 2-3 hours on a 60MB laptop drive.

We recently put up a quality analysis for our codec here: http://www.cineform.com/technology/quality.htm

We are moving forward to productize the Bayer version of the CineForm codec. And yes we have thought about the various licensing opportunities. However, these developments have been delayed due to the boom in the HDV side of our business (the Sony FX1/Z1 is causing a lot of interest in our technology.) For HDV work our compression technology has been licensed by Adobe and by Sony (announced today). Those two companies have been keeping us very busy. :)

[Rob Lohman]

David: is 4:1 - 6:1 compared to the original data stream (ie, 16
bits per color sample) or the packed sample (ie, just the 10 or
12 bits)?

[David Newman]

Rob,

4:1 to 6:1 is compared to the raw signal that we compress which is currently 10bits per Bayer element. The compression ratio would seem higher if it were compared to unpacked 16bit (350Mb/s) or slighly higher for packed 12bit (260Mb/s).

As for 12bit vs 10bit issue, which I'm sure will come up; the compression does a 12bit->10bit conversion with a user controllable gamma curve (much a like a good HD camera does from 12/14 to 8bit.) The 10bit data provides an excellent signal of downstream color correction without banding.