4:4:4 10bit single CMOS HD project

[Ben Syverson]

Juan -- when you say 4.5:1 lossless in the green channel, do you mean 4.5:1 in terms of the original pixel values, or the final image channel?

I ask because it's relatively easy to create a codec that stores only the G where it appears on the sensor. That could be considered 2:1 compression. Then apply standard lossless compression to that (which ranges from 2:1 to 2.5:1) and you have 4-4.5:1. That must be what you're doing, because I don't believe true 4.5:1 lossless image compression is possible at this time (at least not at realtime).

It's worth pointing out that bayer images have a "compression" ratio of 3:1 -- they represent the entire RGB in one B&W image. Add standard RLE or Huffman lossless compression, and you should be able to get it to around 5:1 or 6:1.

That would be an incredibly easy codec to write.

[Joshua Starnes]

By the time you get to a theatrical release print, which is many generations removed from the negative, no one in the world will be able to sit in a cineplex and tell the difference between 2k and 4k...

Exactly.

the movie was shot with a viper 3 chip 1080p filmstream @12bit

Really? I had heard CineAlta.

I've seen the trailer on the big screen, and it looks great. Can't wait to see the final print. Wonder why Mann chose to shot HD?

[Ben Syverson]

Obin, I've come around on Cineon. The most commonly-used Cineon format is 10-bit, and you can use it to store linear values. I don't know of any NLE's that support Cineon, but it certainly works with AE, Shake, etc.

For working in an NLE, the workflow would ideally be something like this:

1. Capture to TIFFs/RAWs/Cineons/whatever.
2. Convert files to AVI or Quicktime movies that use a lossless 10bit codec.
3. Also convert the files to DV or some more easily editable format.
4. Edit the movie in DV.
5. When you're done, replace the DV footage with the lossless 10bit HD movies. Most NLEs will let you do this automatically (without manually replacing each clip).

However, if you're using a good enough lossless 10 bit codec, maybe this wouldn't be necessary. If we store the images in the codec in their B&W bayer state, and do lossless compression on them, we should be able to get at least 5:1. And that may be small + fast enough to edit in 10bit uncompressed HD in realtime, without requiring RAID drives.

For example, 720p at 10bit RGB and 24fps takes up 79.1MB/sec. Cut that down by 3:1 if we just store the bayer: 26.36MB/sec. Cut that down by 1.5:1 (lets be conservative -- we could get as much as 2:1), and you get 17.57MB/sec. Now, 17.5MB/sec is not that much of a strain on anyone's computer for read/write. If you have a fast (7200 or 10000rpm) drive, you could probably get two uncompressed 10bit HD streams going at once!

Man, maybe I should look at the Quicktime SDK. I wonder how difficult it is to write a codec...

[Jason Rodriguez]

I'd second Ben's reply here.

Cineon/DPX is a SMPTE standard file format, and it can store linear values, you don't have to have a 10-bit log file, but if you're trying to get 12-bit vaules into a 10-bit file, then you want to use logarithmic encoding to preserve the entire dynamic range of the image, and not clip off values in the least significant bits.

[Ben Syverson]

I'm looking at building at QuickTime codec component right now... It looks fairly complex, but I'll see if I can hack my way through it.

- ben

[Juan M. M. Fiebelkorn]

Sorry guys but I get lost.
Too many questions to answer!!!! :D

BEN

I mean 4.5:1 only for Green.Don't know really.Maybe something became lossy in the way, which is a rather common bug when you're working with transformations,etc.Time will tell.....

OBIN

My usual workflow is as follow in such cases:
I convert everything to 8 bit.
I edit everything.
I export an EDL or AAF.
I open this EDL or AAF someplace with support for 16 bit.
I apply CC.

Ready :D

DAVID

So how do Huffyuv, XVID, and other YUV planar image formats to work with VFW?
About 10 or 16 bit I know nothing at this moment.
We were just thinking about going the usual way and use 6 bits from the alpha cannel to store them in the RGB or YUV 4:4:4 version.(Remember I'm not the coding expert)
You are the specialist :D :D

What do you mean by 'interchangeable' ?

So, you say we must use Directshow.....



Nobody answered my question about MXF yet.....

BTW: Speed Razor says it support Cineon 10 bit data.

[David Newman]

<<<-- Originally posted by Juan M. M. Fiebelkorn :
So how do Huffyuv, XVID, and other YUV planar image formats to work with VFW?-->>>

VFW only supports RGB on decode, this I know for sure. Try decoding a HUFF using graphEdit and you will see the problem. However the encoder does support YUY2 input, I don't know what tricks if any they do for YUV support. It seems the only way to use HUFFYUV as a YUV decoder is create your own importer for a tools like AE and PPro.

VFW is an obsolete API due to it 8bit and RGB issues. DirectShow it much more flexible, and less obsolete.

<<<--What do you mean by 'interchangeable' ?-->>>

The problem with Bayer data is it look like 1280x720x8 or 16bit image to average encoder, yet it is not. So if you declare the Bayer data with a new FOURCC code, your encoder can know what to do. If codec agree on the FOURCC codes different codec can be exchanged as needed.

[Obin Olson]

hmmm...SpeedRazor eh? is that a standalone package or a whole computer system with lots of hardware?

hmm 10bit...this sounds good...do you think that quicktime is supported in Razor ??

[Juan M. M. Fiebelkorn]

AFAIK is a software package.
Try Google :)


http://www.in-sync.com/products/product.php?ID=3E9DD01F

Don't know the rest.

[Jason Rodriguez]

Has anybody checked out Ben's plugin for AE??

Other than the zippered artifacts on some of the edges (Ben, I've emailed you about this, you might want to adjust your algorithm a little bit), this plug-in is very, very nice!

One bad side-effect of the zippering on the edges is that it makes them crawl as you're panning across an image and the edges pass across the screen.

But other than the zippering (and there are other algorithms out there that don't "zipper" as much), this is a very, very good plug-in

Hey Obin,

I'm beginning to wonder,

Is the horizontal "smearing" artifact we're seeing a result of the CMOS chip or your cheap lenses?

Since they're coming from bright areas in the image, and they're a bit diffuse and even taper off as you go towards the edges of the image, I'm thinking they might not be a results of the CMOS chip, but some interesting flare artifacts from the c-mount lenses you've been using.

[David Newman]

I haven't tried Ben's filters directly but they appear to be linear interlopaters only. I have the origin Obin sequence that generated this image using Ben's filter shown here:

http://www.dv3productions.com/test_i...en_test_on.jpg

Using the original source, here is the best de-bayer I could come up with so far. It is heavy processing (slow at 3fps on my 2.5GHz P4.)

http://www.cineform.com/downloads/CineForm_debayer.jpg

I still have a problem on the left edge, but I believe this is due to the why I extracted the source data not the bayer filter (as the right edge is perfect.) Overall I believe the image is sharper with more detail.

This image is zoomed in 400% to show the key areas of improvement.

http://www.cineform.com/downloads/BayerCompare.jpg

P.S. The slight color variation between examples is due to the gamma correction applied not the de-bayer algorithm.

[Jason Rodriguez]

Hi David,

That's a very nice interpolation you have going on there.

BTW, does this have to be bundled with Cineform, or can we also get a seperate plug-in like Ben's? I really like the way that Ben's plug-in can integrate into AE or Combustion (actually right now Ben's plug-in is crashing AE, but works in Combustion), and then I can add more filters on top of that.

Also do you think there's the possibility of getting that Bayer de-mosaic plug-in for the Mac version of AE?

BTW, there seems to be some "zippered" edges on the "O", the "N", etc. on the words "Natinonal Geographic". Also I'm noticing some heavy banding. Was this a result of just preliminary experiments? Are "zippering" artifacts something we're going to have to live with on edges? For some reason I never see "zippering" on my Canon D60 shots, even on fine edges when you zoom in on them, and that has to go through bayer-demosaicing too.

[Ben Syverson]

Jason and David,

Yeah, the current implementation of linBayer is, as the name suggests, purely linear. I do this for two reasons: it's fast, and it's lossless. By lossless I mean that the original values are maintained, so if you wanted to, you could take the RGB image and recover 100% of the original RAW data.

I'm also developing a higher-quality algorithm, probably based on splines.

I'll keep you posted!

Jason, the reason you never see zippering artifacts on your Canon is probably that Canon is doing some very aggressive de-bayering. I'm guessing they're losing a lot more softness than us...

- ben

[David Newman]

Jason,

It certain could be broken out into a AE filter (PC yes, Mac unlikely unless we release code); right now it is a DirectShow filter. CineForm is still working ideas on how to productize this stuff.

There is a balance maintaining detail and smoothing out any possible aliasing, the area you point out would be likely be aliased even on a 3 chip camera. I choose the algorithm based various de-mosiacing papers then indicated that this was a good all senarios (different image types) approach. This uses median filtering on chroma differences -- it is the median filtering that makes it slow.

[Jason Rodriguez]

Why would the Mac version be unlikely unless you release code (do you mean open source it?)??

Or are you saying that right now you're basing off some libraries/frameworks that are not available on the Mac (Directshow), so unless you actually release this plug-in to the public, you're going to have to re-program it for the mac.

I see what you're saying about the aliasing. It'd be interesting to see it in motion.

Looks good overall though.