The latest community technical update is over at the technical discussion thread:

http://www.dvinfo.net/conf/showthread.php?s=&postid=221443&#post221443

Obin,
If you could tell us how the format looks like, we can for sure open it....
I have a conversion application working now.It opens a RAW file (I mean pure binary file nothing more) and outputs a sequence of Tiff files...
Could anybody here tell me how to manage hot pixels.I mean what is the usual way to remove them....


What a test image should look like.......................

http://www.outbackphoto.com/artofraw/raw_09/essay.html

Hot pixels are typically "mapped" out, i.e., they are linearly interpolated from the surrounding pixels to look like they belong in the same area. Even if they are on a line, typically you won't notice if you only have one pixel out of place (again, it's typically a linear average of the surrounding 8 pixels in RGB).

That's pretty cool Juan that you made your own RAW converter. Is there anywhere that we can see the converted output?

Also, I think Juan has touched on something interested here, our RAW converters should have some sort of methodology of identifying and removing hot pixels. Maybe not the first alpha/beta, but definitely the final release.

Good on you, Juan :)

Hey Obin,

Here's the deal: FCP does work with the new Sheer at RGB 10-bit, 1920x1080, but it's only recongnizing the file as an RGB 8-bit file. That's an FCP limitation, where YUV is recognized at high bit-depth, such as 10 or 12-bit (and it renders at 32-bit floating point), but RGB is limited to 8-bit for now (no 16-bit).

While Quicktimes are very convenient, I still think that Sheer video might not be the best idea, especially if you have to convert from RGB to YUV for 10-bit processing-there are a whole ton of problems that can go wrong in that conversion. Of course the call is yours, but I would still leave open the door to a 16-bit TIFF file for those of us who don't want to mess with the Sheer Video processing that could be compromising the ultimate image quality or bit-depth of the image.

Now if you don't add any filters in FCP, then you're fine. You can maintain the 10-bits since FCP is only making pointers to Quicktime files, it's never modifying the original source files. So if you edit in FCP, and then take that edited timeline to AE or Combustion, etc. (via automatic duck, XML, etc.), then you are maintaining the total 10-bits RGB. But assume that you're going to be able to do your final render/conform in FCP-if you do that, then you're going to be filtering at 8-bits, and your result will be in 8-bits. I think if you do log-encoding to the 10-bit RGB Sheer file, then you're in very good shape, and will be preserving the totality of the 12-bits from the Altasens. Also at 1920x1080 we were running at 64MB/s.

this sounds good Jason...so if FCP is only a pointer app if you don't apply effects how does it deal with stuff like fade-to-white etc? if it's a pointer then it will not render a single file an dyou can jsut open the project in AE and CC or Combustion? how can you get a FCP timeline into AE or Combustion?

Jason I would like to call you on the phone for a chat..ok with you? I need number ;)

Juan M

can I send you a RAW file to look at and test?

send me your email and i will send you a raw file to test with

morsa @t morsa DOT net Dot ar

Remember it just opens .bin and it only does the color interpolation...
No gamma, no other things up to now....
I need a hot pixel filter.....:(

I have .raw files

hat is wrong with 8 bits as a master format Jason? do all the CC in 10bit/12bit and then edit and compress at 8bit..ok?

.raw

If they are just binary (I mean no headers, no tails, just the crude line of pixels it would be ok....)
I can only work with 1920x1080 and 1280x720 at 16 bit per pixel so no packing supported yet..

There's nothing wrong with 8-bit as a presentation format, for a Master format there's the insinuation that you'll be archiving on this format.

BTW, again, Sheer is a 10-bit codec, it's just that you can't Master in FCP with it right now, you're gonna have to move over to AE, etc. Frankly, as a 10-bit format, it's not as good as the Blackmagic, etc., which can take 16-bit renders out of AE and filter them down to 10-bits, so when you're selecting them under the Quicktime filter tab, they say "trillions of colors" rather than "Millions of Colors".

Again, I'm not telling you what to do, but as a potential end-user, I would like to see you keep the option for uncompressed 16-bit linear TIFF's for those of us who want uncompromised image quality.

But as far as Sheer goes, yes, it does work with Sheer 10-bit stuff, but only in YUV, not RGB.

Yes we will have tiff files but I am looking at compression as a way to allow the editing of files on a standard computer with no special hard disk drives etc...that is the reason for SHeer and to keep 10bit at a datarate of only 10-15MB/sec with very little compression like Sheer is amazing I think..it's like CineForm but it's an open codec that anyone can install and edit with if it's used by a mac

what are the other options at the moment for editing?

Does anyone know how hard it is to go from RGB -> YUV?? could this be a converter that could take RGB make it YUV AND compress it with quicktime Sheer codec all at the same time? how bad would the quality hit be?

Obin, what happened with the images you would send me?

I could not get at them today as the LightWave animation machine was in use by our animator ;( I will do it asap

Does anyone know how hard it is to go from RGB -> YUV??

Isn't this conversion process handled by the Quicktime conversion engine? Or is that not enough quality, or not operating at high enough bit-depth?

not sure Jason you said something about it being a bad process to use..didn't you?

RGB to YUV isn't a big penalty (about processing power)Don't know if image quality will suffer too much...
The only problem I see is that you would need a YV12 colorspace capable codec to compress the Bayer more or less directly.
If not you would need to interpolate at least the R and B parts.
Are you going to convert the Bayer on camera?

no we capture the RAW pixel data and then de-bayer the image and save with quicktime SheerVideo 10bit....unless that is that we have to go from RGB -> YUV first in a new process?? will the quicktime take care of that or not?

If this is a Quicktime process then I am home free and can jstu input the data into the quicktime SDK and let it do the RGB--> YUV conversion...I am not sure if this is how it works..anyone?

ohh, right.Clear now.
If that is the case just ask your programmer to implement the YUV conversion (I don't trust on built-in conversions, sorry)and send it to the codec..

good news:

Obin,

Within a few days we will release SheerVideo v1.9.8. This public
preview of version 2 will contain all the engine enhancements of v2,
including the incorporation of our StickyColor technology for
interconversion between RGB[A] and Y'CbCr[A] pixel formats. Version 2
turns the six specific SheerVideo codecs into a set of hubs capable of
inputting and outputting any pixel format with the least possible
information loss. In other words, every one of the Sheer codecs:

Sheer RGB[A] 8b
Sheer RGB[A] 10b
Sheer Y'CbCr[A] 8bv 4:4:4[:4]
Sheer Y'CbCr[A] 10bv 4:4:4[:4]
Sheer Y'CbCr[A] 8bv 4:2:2[:4]
Sheer Y'CbCr[A] 10bv 4:2:2[:4]

will directly input and output all of the following pixel formats:

ARGB (RGB[A] 8b)
'b64a' (RGB[A] 10+b)
'v408', 'r408' (Y'CbCr[A] 8bv 4:4:4[:4])
'v410' (Y'CbCr[A] 8bv 4:4:4[:4])
'2vuy', 'yuvs' (Y'CbCr[A] 8bv 4:2:2[:4])
'v210', 'v216', 'Y216' (Y'CbCr[A] 10bv 4:2:2[:4])

I hope this information helps answer your questions.

- Andreas

SheerVideo

Well....
Nice ,isn't it? :)

<<<-- Originally posted by Obin Olson : would anyone care for a frame or 2 in the RAW format we have going? I would like to know the Rob project could transcode this RAW file yet... -->>>

The conversion application will not support this without work from
you guys Obin. As soon as the basics are done it will be an open
source project and you can add your own handler to read your
format. We can't just "magically" add stuff for you.

RGB to YUV conversion is extensively documented and reasonable straight forward to code. It would be worth picking up this book for the algorithms (and better understanding of the process) Video Demystified (http://www.video-demystified.com/). One warning RGB to YUV calculations are always performed on gamma corrected RGB -- don't forget this stage.

Hey Obin,

Yah, I forgot about that "sticky color" stuff that Andreas had talked about before. Supposedly it's seemless/lossless RGB->YUV conversion, because typically you are loosing something in the translation from RGB->YUV; the color spaces are not identical (AFAIK), unless using true floating-point operations, and even then I think there are ways to screw it up. I think the big problem is that you typically color-correct in RGB, so even if you have something in YUV, you're going to have to convert it back to RGB for good color-correction and then back to YUV for editing.

ok so then the workflow would be TIFF for CC in high bit and then save to sheer in yuv for editing? how does that sound?

The whole point is to keep it in a greater then 8bit depth at all times..wha would the point of YUV be anyway?

That sounds good.

Altough you might want to consider the option of just doing offline/online-I think that you might be creating more problems that it's worth by trying to edit with the online files when that's not really necessary (unless it's short-form project). I know for anything over 10 minutes, you're gonna want to try and do offline/online for the sake of sanity.

so what an option that allows a save to SD resolution and DV codec??

Sticky color is lossless and artifact free 10 bit RGB > 12(16) bit YUV > 10 bit RGB and 10 bit YUV > 12(16) bit RGB > 10 bit YUV.
http://www.bitjazz.com/bitjazz/press/2004.04.22.shtml

FCP has a very nice mediamanager that will batch process your Project into another format.
It can automatically convert all your uncompressed 10 bit HD files into SD DV (or any other format of choise) and convert your edited sequences as well. All in one single step.
Later you can simply relink to your HD files to make an online version.

Either that or you can use Sequence Publisher from IRIDAS for the conversion of the 16-bit TIFF files to a DV codec. The nice thing about that program is it allows custom burn-windows, etc., things that come in very handy for offline/online workflows.

If you're on the Mac, I can't reccomend Shake enough. Shake combind with some Perl, Python, or Tcl/Tk scripting is an unbeatable combination for file conversion, etc., although it's an expensive file converter at that!

BTW, Obin,

Just want to say again, if you take your 12-bit RGB linear image, and map that to a 10-bit Log colorspace, then encode to Sheer, you should be perfectly fine. In fact, you can then use Stickycolor to go back and forth between YUV and RGB, and like Anders pointed out, no loss. If you just use 10-bit linear though, you're loosing two bits, and in the case of the Altasens, those two bits are valuable, since you have greater than 11bits that are not contaminated by noise according to the Altasens spec sheet.

Can we gamma correct the bayer pattern itself, I mean before "converting" it to full colour?
I guess it would be more or less the same, but need some more info...

In theory that should not be too much of a problem Juan. However,
the problem might be in clipping and such. If your not very carefull
you might loose information that you can never recover.

I know it.
Linear to log table would have exactly the same effect on the Bayer mosaic or the color image.
I was asking if it makes a difference to demosaick the linear bayer or the log bayer.This is the exact question.
I know for example that it is not the same to saturate color on a linear image or a logarithmic one.Results look quite different.
So that's why I'm asking...

I know that's what Jeff Kreines is doing for the Kinetta-10 bit Log monochrome bayer. So it can't be that bad.

Of course what I'm suggesting isn't a Log bayer pattern, but for Obin to first convert the bayer, and then log encode that to 10-bit SheerVideo Quicktime. I'm really not sure what 10-bit log bayer might do to the image conversion-wise, but I can find out.

But at kinetta.com it says it records full color images,or not??
I'm getting confused.....

On the magazine itself, Jeff is recording a 10-bit Log file to increase frame-rates and recording time; basically maximizing throughput like we are trying. There is another Xilinix FPGA in the magazine that then reads out those files and does the bayer conversion to color in real-time. From there you record out via HD-SDI or there's also a SDI downconverter. But the RAW files themselves are 10-bit files ecoded to a logarithmic file in order to preserve space (also SDI is only 10-bit, and he wanted to get as much of the 12-bits that the Altasens records into the 10-bits of SDI and HD-SDI). He also has some nice things like view LUT's, etc., but basically it's recording 12-bits off the camera head and then encoding that to a logarithmic 10-bit file for storage and playback across HD-SDI.

That's why I'm thinking if Obin did the bayer conversion and then encoded that to 10-bits RGB using SheerVideo, he should be fine.

Okay, not sure if this makes a big difference, but I took on of the linear files that I had on-hand, and converted it to a logarithmic file, with the whitepoints at the extremes of the bit range (0 is black and 65536 is white), and a gamma of 1.7. I then applied Ben's LinBayer plug-in, and did color-correction after that. Everything seemed fine to me, and I have a nice-looking image. Not sure if I did this "right", but I did take it from a washed-out image and now have it as a nice, colorful image.

OK, thank you for your testing :)

It is mathematically the same to apply the gamma correction to the raw bayer first rather than after the demosaicing filter. We are taking the 12bit data and gamma correcting it to 10bit raw before compression. As compression in our approach had to be real-time, it was important to delay the computationally demosiacing filter for later. The gamma correction before compression allows the visually significant image to be preserved in 10bit data. The demosiacing is done under user control during decompression (playback.)

David, have you come up with another capture device? What are you describing here when you say "our"?

BTW, if everything is getting converted to Cineform's codec, is there way that it can then be ported over to the Mac and edited in FCP?

Also if you've been designing a capture app, can it capture to a variety of frame-rates?

Just curious since it seems like you're talking about another capture app.

Yes, CineForm has been quietly working on our capture and high bit-depth bayer-based editing solution. One day we do intend to support the Mac, yet today we can get better performance out of a PC (for our architecture -- not a PC vs Mac thing.)

We intend to support compressed capture up to 30Hz for 1080p and 70+Hz for 720p. This will be on a single CPU system over standard PCI, to a single drive (or any type.) Higher frame rates will be obtainable on multi-CPU systems.

Jason,

One more more thing to add. Our capture solution is intended to be an engine that sits under alternative (third party) capture interfaces. We hope this technology will be licensed to work with a range of imaging solutions (including camera based direct to disk compression.) We will also be building our own GUI for capture for standalone applications.

David,
Could you show us a couple of frames (after being compressed with your codec) to see its quality?
I've never been able to see anything compressed with a Cineform product...

David could this be an option for CIneLInk later on when you get it done?

www.dv3productions.com/uploads/test3.zip

it's a file that has been captured with cinelink and output to a 16bit tiff file..nothing fancy just to show we are making progress ;)

www.dv3productions.com/uploads/raw/Rec00012.raw
www.dv3productions.com/uploads/raw/Rec00013.raw
www.dv3productions.com/uploads/raw/Rec00014.raw
www.dv3productions.com/uploads/raw/Rec00015.raw

raw output from CineLink datacapture

Global gain: 10db

Shutter: 1/60

fps: 24

Camera MHZ: 60

Camera: SI 1300RGB

Just curious,

Is the cineform product going to be RGB 4:4:4 or YUV 4:2:2?

Juan,

The new compression for bayer images is new (although it is an extension of our 10bit codec.) No images to release yet from the new stuff.

Obin,

I don't know what "CineLink" is. If this is your tool then we have discussed this possibility.

Jason,

"Is it RGB 4:4:4 or YUV 4:2:2?" Neither. It is a direct Bayer compressor. Bayer images go in and Bayer images can come out -- however, under normal operations, the decompressor demosaicing filter will be used to output either RGB 4:4:4, YUV 4:4:4 or 4:2:2. The compression is not involved with this decoding option. All intended to be very flexible.

Just wanted to throw in a quick hello...
All the work you guys have put in is absolutely incredible.
I am trying to keep up with all the tech stuff, and certainly learning alot as I go. I got in late in this discussion, so I am reading from the begining.
I was up till 3 last night and made some good progress.
I just wanted to start off tonight be saying thanks for laying all this incredible groundwork.
I am eventially looking to undertake a similar project with similar goals. I will truely be standing upon the shoulders of giants...
Keep up the good work.

Thanks,
Erik

Bayer images go in and Bayer images can come out -- however, under normal operations, the decompressor demosaicing filter will be used to output either RGB 4:4:4, YUV 4:4:4 or 4:2:2. The compression is not involved with this decoding option. All intended to be very flexible.So it's decompressing a Bayer stream in RT? For a good bayer demosaic filter at 1080/24p, that seems like a lot of horsepower will be needed, or is there a seperate converter app that we're talking about here that's non-RT, and I'm just really misunderstanding things? Or is it simply an AVI file that can't play back RT, but at least we don't have file sequences flying around everywhere? BTW, I'm not expecting RT bayer conversion, for a good CFA demosaic filter, I really don't believe that's possible. Sure linear is fast, but it also seems to have a lot of problems that have to be circumvented by an anti-aliasing filter somewhere in the conversion process.

In other words, with your capture/convert applications, are you capturing a stream to an AVI file that has bayer information in a monochrome format (on the disk), but when you open up the AVI it shows color information because it's decompressing the bayer info?

Also, how much compression are you applying if you're able to get this on single hard-drive without an array? Can you compress Bayer information?

And are you only supporting one brand of frame-grabber (I'm assuming EPIX)?

This all sounds very interesting, and would definitely change the hard-drive specs that I have for my current camera design.

Juan I posted the raw files for you to play with....did you get a chance yet? I would like feed back from anyone that has converted the .raw files ....Rob ? Juan?

<<<-- Originally posted by Jason Rodriguez :--->>>

So it's decompressing a Bayer stream in RT?Some bayer filter yes, others no. By delaying this discussion until you need it will save a huge amount of work-flow time. Fast de-bayer previews are perfect through-out a lot of the editing time. We can dynamically switch de-bayer filters on decode, based on your needs. Rendering out an effect will use the best (and slowest) de-bayer filter, but as most previews are real-time in the CineForm products, rendering can be mostly keeped until output.Also, how much compression are you applying if you're able to get this on single hard-drive without an array? Can you compress Bayer information? We compressed gamma corrected bayer data to around 4:1. That means around 15MByes/s for a 12bit stream.And are you only supporting one brand of frame-grabber (I'm assuming EPIX)?
That is what we are working with now. I'm get lots of excellent help from the guys at Silicon Imaging, so we have discussed other frame capture opportunities. But we are still in the research and development stage.