4:4:4 10bit single CMOS HD project

[Rob Scott]

Originally posted by Jason Rodriguez : I see what you're saying, but then you have the problem of too much motion blur when shooing from 48-60fps. Typical film shutters work at half the frame rate. So for 48-60fps, we need a shutter that's working effectively at 1/96th to 1/120th of a second.

With this particular chip, I don't think we have any alternative, unless we use a mechanical shutter.

[Jason Rodriguez]

The Kinetta's not using a mechanical shutter, only a mechanical one. I saw it myself. And it's going to have a lot of nice frame-rate features, so that must be feasable with this chip in an electronic shutter mode.

On a second though, the kinetta's going to use an Altasens chip, maybe there's something different with that chip than with the Micron.

Another thing we need to realize is that this Micron chip was made for digital still cameras (at least that's the way it's listed on their website). You can get very long exposures with those cameras, how come again there's no problems with rolling shutter artifacts. Also be aware of the fact that consumer digicams don't have any mechanical shutters, just electronic.

[Obin Olson]

I am doing my best to lead this along... I should have been working today instead of this stuff but I was not ;) anyway here is an AMAZING codec

http://www.bitjazz.com/sheervideo/

that thing compressed the 1300 tiff files from 580megs down to 109megs with NO visible loss in quality whatsoever...amazing ...I was playing full on HD off of my firewire drive today with NO problem at all...15-16megs a sec as it ran the clip!!!! then I took the image into after effects and whacked it out to see if it would fall apart from the compression...not at all, it looked great! Much better compression then the Panasonic varicam has.

And they are in BETA with a 10bit version of the codec!!! yeeeee

<<<-- Originally posted by Jason Rodriguez : Obin,

Right now Bitjazz is nice, but it's only 8-bit. I also believe that's real-time ecoding for 8-bit SD video, not HD video.

On the rolling shutter topic again:
How slow or fast does the shutter have to be in order to get away from the artifacts. From the footage that I saw on the wmv file, there's apparent stretching artifacts at all. How bad would it be at 24fps, with a 1/24 shutter speed? Also couldn't we just blank for 1/48 of a second and then take the next frame on the next 1/24th of a second, so it's imitating a mechanical shutter at 180 degrees (like a film camera). I think this was mentioned before, but I'm not sure what happened in the response. I heard something about a firmware upgrade needed.

we can capture at 30fps and run the camera at 60fps and then when we wnat to capture 60fps for slo-mo we just keep all frames..no biggie also I can still shoot at 24fps with a higher mhz like 54mhz on the camera, this will allow for a nice image like in the wmv file...if you put the mhz down to say 27 the rolling shutter is VERY VERY bad as you pan the top of the iage lags way behind the bottom...I think you guys need to calm down about this..it's not that big of a deal and I know of atleast 2 ways to deal with it..no problem

[Steve Nordhauser]

Obin,
The truth is in the pudding, whatever that means. Since we know what kinds of shots will be the worst, shoot a few and post them. I'll try to get to it but I'm not impartial (Wanna buy a camera?) and I'm not a cinematographer, I'm an electrical engineer (anybody get the DeForrest Kelly flashback from Star Trek? boy am I tired).

I think rolling shutter is only an issue for medium frame rate. If I were to expose for 1/8 sec and read out in a 1/60th of a sec, the top to bottom blur would be minor compared to any motion blur.

For testing with the current software and hardware, Obin, if you can't get the frame rate you want, use a smaller window. The frame rate is the issue - must be 24fps to be a valid test.

Bitjazz: No miracle, it is not lossless. 5:1 is not lossless. They might have a patent worth searching to see what they do. (don't infringe, that is intellectual property). We do need to understand the losses. They could make their own artifacts under certain conditions we don't know now. Remember what I said about fully understanding any step that has a potential effect on the final image quality. Compress before recording is the holy grail. This could be good. Even if we do the R,G,B separately into a non-standard file and provide a tool for crunching them back together. And we want 12 bits. But 8 or 10 is a fine start.

[Joel Corkin]

Sorry Steve, I'm not really following this thread too closely, but the Bitjazz Sheer video codec certainly is lossless according to them and according to my own tests.

[Rob Lohman]

Obin: we will calm down when we see some footage. Besides
24 fps I would also like to see the same footage at 48 fps to
see how much it goes away. The most easy way is to have a car
going from left to right and right to left and have the camera
in a 90 degree angle. The car should be driving a real speed
(ie not 10 mph or something).

It is not a problem you can fix. At 48 or 60 fps the problem will
still be there, only lesser. So you might not see it, but it will be
there because it is inherent to a rolling shutter.

Can you please respond to my question list that I put up above?

Steve: it looks like (according to their claims) that the codec is
lossless. Or so they claim. We could easily verify this. It looks like
it is only out for the Mac platform at the moment though. Also
they claim to do a 2 - 2.3 compression themselves.

I think this adds up. Think about it that Obin's TIFF files are 16
bit!! with only 8 bit information in them. So if you divide 580 by
two you get 290. Divide that by 109 and you get a 2.6:1
compression ratio which is basically what they claim.

I'm wondering if we can do something more efficient since we
know we will be working with Bayer data and they don't. I mean
in storing the data to disk. We still need to do Bayer conversion
after this so it might then be a very interesting codec to use
for editing (solves a lot of our problems). The Dalsa L3
compression is pretty similar in rates, but probably far far far
more expensive.

There is a text on compression for such chips which can be
found at this location. I'm not an IEEE member though...

Everyone: they are looking for beta testers so this might be
interesting. I'm wondering if they are supporting all sorts of
resolutions as well.

Steve: I have been looking a bit into pre-recoding compression
before laying it out to disk. I'd much rather have a single harddisk
then two or more. Ofcourse it should be lossless and fast.

[Steve Nordhauser]

Rob on BitJazz:
On their web site, they claim 2.4:1 or so for YUV. They also say they can do RGB but don't give numbers. This is realistic for lossless. I was commenting on Obin's <<from 580megs down to 109megs with NO visible loss in quality>> . This is lossy. I'm sure that they are doing more than packing bytes. Using the basic tools (Huffman, RLE and some varients that are common) you can do 2:1 lossless. Very image dependent though. And *Very* noise dependent. Remember that they are talking images after the Bayer de-mosaic step which puts that into the real-time path. I'm thinking the best method is to create a red, blue and green separate buffer and RLE each one. That should be doable at any resolution in real-time and save the Bayer for some real crunching. We need to hear from someone deep into Bayer. I'll try to get to the library and get a copy of that article.

[Obin Olson]

Rob, I don't think it's the framerate but what mhz you shoot at...I have shot at 24fps with huge rolling shutter and then put the camera mhz up and it goes away at 24fps

Steve, what I got was VERY acceptable for production and I could edit from a firewire disk drive! I would be happy with that codec..for really important stuff you can use 16bit tiff files

it's such a pain in the ass to capture with Xcap I will wait and see if streampix fixes the dll file and then try and shoot more...I have seen enough that I don't have a need for more. I do understand that you guys want more ;) and will do my best to help

[Obin Olson]

1) do you have a RAW 10 bit pre-bayer file for us? This CAN be a lower resolution STILL file. No movie, no high res.
I will look on the hard drive today..problem is i never have them i have 10bit packed into 16bit tiff files

2) which lens(es) are you using with your camera now?
el cheapo CCTV lenses, 25mm and a 75mm - yes I am sure the image would be better with a higher res lens

3) could it be the stuttering in that movie is due to your harddisk not keeping up?
I have issues with keeping the camera shooting at 24fps with Xcap, it could be my hard disk sure

4) which camera settings did you use to record that movie? (gain / exposure)
that is 3 shots...they are all 8 bit. the truck was exposed for the hot background and then I pulled truck up from the shadows
the shot of me is exposed for my face and blows the crap out of the backgorund
the flowers? dunno

5) did you do any color correction or other conversion to the material?
flowers are raw colors, everything else is "fixed"

6) Which Bayer algorithm did you use?
dunno whatever Xcap uses

This are important questions to the people looking at also getting
a chip or not...

[Steve Nordhauser]

Obin, you are doing great. On the XCAP, try recording to memory since your test clips are small, then you should not stutter. Norpix should have your new .dll files today.

I'm a little not clear - you are able to record at a fixed frame rate independent of the pixel clock? That is news to me - I will have to check it out. I would suspect you occasionally get duplicated frames - I will talk to the Epix technical people about this.

XCAP uses a very basic Bayer as default. I'm pretty sure that you can change that under the 'colors' tab in the left hand camera control window. I think they support a number of algorithms (just saw this yesterday) in the latest version. One of the nice things about Epix and Norpix is that they never charge for upgrades - just download the latest version.

[Obin Olson]

not sure Steve, I can just type in 54mhz and then shoot at 24fps..don't ask I have no idea! :) but it seem to work..Maybe it's not working at all, it's hard to tell when you can't just type in 24fps on this crappy software..What I do is slide the frame period up and down till the "video" and "Display" read about 24fps seems like they vary from 23.97 -24.2 ...this is one big reason we need software that LOCKS in 24fps 30fps 48fps etc...it's got to be a constant framerate, no fluctuation

[Wayne Morellini]

The Sumix info is up at the Viper thread, and it is good, Altsens sensor to, but no 3 chip yet.

This is something I forgot to post last night which I just mentioned in the Viper thread, that maybe of some interest to Rob, and Rob, and Steve:

I have some cheap tech that could do the compression without FPGA, and be programmed in weeks. I have spotted it some time ago, it has an array of parralell risc processors with memory and fpu's, at extremely high perfomance and low power, but only just remembered it. That would move developement time of this part of the circuit up 10 times. Steve if you would like to pass it along? This is not the other tech I mentioned before that was mainly for PC's processing. I am such a dope not to remember this earlier, this would let even a low speed PC do all the capture, compression, and editing you like (as long as the software can take advantage of it).

www.clearspeed.com/news.php?page=pr&pr=2

http://www.clearspeed.com/technology...c7694795e722fb

www.clearspeed.com/products.php?page=si

www.clearspeed.com/products.php?page=faq

I have included all these links as it is hard to get an idea of the extent of it's capabilities from one page.

Summary:
- from 10's to thousands of Risc 32-bit processors with Floating Piont untis
- programmable in C (Rob could move code to it).
- can act as main processor, or coproccessor
- very low power at very high performance, or even more performance at higher power levels.
- 64 bit version coming.

With something like this, a cameralink comrpession board could have been within reach of the Robs.

[Jason Rodriguez]

We better watch out with recording at 24fps. For the guys in the audio department, that could end up being a REALLY bad idea. Right now, everything shot for television at 24p is really shot at 23.976 to keep compatability with 29.97 NTSC. If you shoot at 24p and then try to go back to NTSC, and your audio is at 24p, that could spell problems. These are just a couple things I've learned from shooting on the F900 where you have the choice between both 23.976 and 24p. The 24p option and NTSC always meant trouble.

BTW, normally with film, the audio is slowed down and synced with the video in the telecine, which is also actually slowing the film down also from 24fps to 23.976.

[Rob Scott]

Jason Rodriguez wrote:
Right now, everything shot for television at 24p is really shot at 23.976 to keep compatability with 29.97 NTSC.

I may not have mentioned it, but from the beginning I was planning to have a standard vs. NTSC timebase option.

So in the UI you would have the option of any frame rate -- 24, 25, 30, 48 -- even 27.5 I suppose -- and then separately you would select NTSC timebase. I think that would keep the UI a bit cleaner -- what do you think?

[Obin Olson]

hmm...streampix is working now, just captured 48fps 8bit! using a ide raid 0 2 drive setup now...gets 51megs/sec single SATA drive gets 40 or so....looks like we may have trouble with rolling shutter at higher framerates..not sure why but I can see it playing 48fps at 24fps..I will do a better test outside soon