4:4:4 10bit single CMOS HD project

[Rob Lohman]

Just after posting the previous blurb I figured it out. This is not
1920x1080 but the full chips resolution of 2048x1536!

2048 x 1536 = 3.145.728

So it is 2048 x 1536 for 10 frames. BUT, that still leaves the
pixel packing/encoding "problem"...

Steve: could you read this chip out at 2048 x 1152?

[Jason Rodriguez]

BTW Rob,

What algorithms are you still planning on using? Last I remember it was Variable Number of Gradients, Pattern Recognition, and I guess Linear would be a nice addition if you need something quick for preview/offline.

For anything being blown-up or put on a big screen though, I think we need a good heavy-duty algorithm like Variable Number of Gradients (which from the Stanford paper will interpolate fully both Red, Green, and Blue channels to the best of it's ability).

[Rob Lohman]

Jason: what algorithms in what regard? To do what? Are you
talking about de-bayering? Basically it doesn't matter. A de-bayer
algorithm is fairly quickly developed and integrated and we will
probably have quick and more complex ones.

Ben has made one, Rob S. and myself have made a near neighbour
and half resolution model (preview) etc.

The main focus is working with the camera and getting that code
to run as fast as possible and get everything stored to disk etc.
Rob S. and myself are also working on the convert/processing
application but that will just be a basic version when it hits the
GPL/opensource state and will be developed futher. Anyone can
join in on that to develop things like bayer algorithms and
whatnot.

My personal spear points is working out the digital-negative
format, seeing if we can incorporate at least some lossless
on the fly compression to lower bandwidth usage and getting
the data to disk. After that I'll turn to the post-processing
algorithms and other stuff if others haven't finished that yet.

[Wayne Morellini]

<<<-- Originally posted by Joonas Kiviharju :Another thing is the mounting of cameras. I found this interesting future thing: <http://www.four-thirds.org/en/index.html>
It could be the future sensor size, and the mounting system of choise in a couple of years.

Joonas Kiviharju -->>>

Joonas, thanks for posting this. It looked disappointing at first, but as I read further, about straightening oblique light rays (at what cost to light) and making the three primaries focus at the same distance not the three film layers (is that good for the Foveon x3), it looked promising, and further on it looked disappointing again. This looks like a cheapened excuse not to pay for, and manufacture, 35mm sensors, and to get us to upgrade our lens systems. How bogus can they get, and we (the consumer) will probably have to be dragged along with it.


They claim that the lens needs only be half as long to get the same image size and brightness, that is an indication of a sensor that is half the size getting half the light (but over half the area it is the same. But what of DOF, or what of convergent lines etc? Their graphics depict it as neatly nestled between different abilities (which means flat out compromise).

Now to get more confusing, they mention a number of formats. 4/3 (four thirds) what? 4/3rd inches (a inch and a quarter or 34mm) that would be good compared to 35mm film, nope, it is 18mm across thats nearly 19mm about 3/4 of an inch. No it's aspect ratio, 4 across and three down 18*13.5mm. But if they said 18mm format people would immediately tweak it is inferior to 35mm, and they should have developed a Digital-35mm format instead. Here I was under the naive notion that 2/3rd lens meant 2/3rd and inch, but is much smaller (anybody with a link to a good format guide, I need to brush up). One small step for us, one great leap for camera manufacturers profits getting us to upgrade our lenses.

Why don' t they do Digital 35mm and digital 18 (16 better) instead, and make the 35mm conform to one of the existing SLR lens mounts with lens refinements for digital use. With an adaptor to condense and straighten the D35MM rays and feed it to the D18mm, or a straight 35mm format adaptor, and adapt it for pro video use, and cinema camera use. Another words a twin standard across film and video. But of course that would flatten profits from pro video and cinematic lenses.

Anyway maurandering rant over (I just don't like slick marketing stuff that doesn't really help).

[Wayne Morellini]

This post is only for completness, as it does not suit some people.

I have been chatting to VIA about the ITX products. From what I can gather there is nothing really suitable for us until next year. So the Pent M ITX boards are still the best for now. But he has told me of a chipset that could be used as a development platform until then.

It has twin 8 bit video inputs, the chipset also mentions a 12-bit capture, but my source only mentioned 8-bit and DVI input but I don't know where ever they are HD or not. If it is HD then it is good for acquiring third party 8 bit footage to use in programs. It has 4 drive sata Raid, 8 Gigabyte memory support, gigabit ethernet support, and the processor maxes at 1.4GHZ (Though I expect more before the end of the year but I don't know where ever this board would take it, my comment, not their's). It's is a consumer HDTV type thing, so has all that support and a dsp for Mpeg4/2 decoding and deblocking, not compression but some sub funtions may help.

http://www.via.com.tw/en/c-series/cn400.jsp

This chipset has been mentioned before, and is not really up our alley for live lossless compression (unless the dsp's are reprogrammable and helpful).

[Jason Rodriguez]

BTW guys, that 5.25" board that I was looking at from Axiomtek with Penitum M (Dothan), PCI-X, and SATA is $489.

I was also looking around at powersources, and it seems like the best bet for the size/weight is the the Anton Bauer HyTRON 120 which can sustain 175W output for 120W/hrs at 9-16V (normal operations at 14.4V). So that should easily power this board with all the fix'ns :-)

[Wayne Morellini]

Good on you Jason, we need more of this. With the 51/4 inch computers, car computers, ISA card upgrade computers, and Mini/Nano ITX single boards we have a wide selection of chioces. That VIA chipset mentioned something about 2.5W power consumption as well, but I've closed the window now.

Thanks

Wayne.

[Jason Rodriguez]

Also I was doing some number crunching, and you guys will have to tell me if I'm wrong or right on this.

Basically with 12-bits linear, you can cram in a maximum of around 10 stops without banding. 9 stops to play it safe and keep the noise levels down depending on how noisy the chip is. Hopefully the Altasens will have hardly any noise, giving us around 10 stops of total dyanamic range (1000:1). This is based on 10 stops being around 1000:1 contrast ratio. When you split that up between 4096 levels, you get around 409 leves for the bottom 100% of the image, which is where the black and white cards lie (90% of the total image is the white card, with the extra 10 percent of perceptual brightness being superwhites, but those superwhite areas actually should occupy 90% of a digital linear image). At 409 levels, you're sort of playing it close with banding issues and noise, so if you increase that to 800 levels for the 100:1 contrast range, then you have removed a stop from the top, so what was at 500:1 is now at 1000:1. So you've dropped from covering 10 stops to covering 9 stops. If the bottom stop is too noisey, then you'll have to increase exposure some in the capture process, or set the black-point higher, so you are now at 8 stops.

With 10-bits, you only have around 100 levels for the 100:1 contrast range in a linear image. This will not do, so you have to double it twice to get to 400, and if you want 800 steps, you have to double that three times, so you've lost 3 stops. That now gives you 7 stops captured in a 10-bit linear file. Since the Micron 1300 has the streaking problem, you have to crush the blacks, so on the bottom you're losing another stop or two, which leaves us with the 5-6 (maximum) observed stops that we're currently seeing.

If anyone is looking for reference to what I'm talking about, read "Digital Compositing for Film and Video" on the Chapters concerning Log-Linear conversions and Log versus Linear files/encoding issues.

[Steve Nordhauser]

OK, I went to the clue shop and got about half a clue. I've put new images at: http://www.siliconimaging.com/Samples/SI3300/
The 10 frame sequence (12 bit mono, 1920x1080) is now 40MB, which makes sense. The color images are 12MB (2Mpix * 2 bytes per color * 3 color, I hope) and the monochrome (color camera in mono mode) is 4MB per pixel.

The only correction done on these (all done in the sensor, not postprocessed so I think of these as RAW) was to add blue gain to balance the Bayer response and adjusting the black level offset. I used a Canon zoom and a bizarre mix of tungsten, halogen and florescent lighting. I'm shooting a calendar so there is some limitation there (I don't have my Macbeth right now).

What is confusing me is that PSP7 still thinks that these are 8 bit images, but is also says the size is on 2MB on the 4MB file.

[Spencer Houck]

Hey everyone, this is my first time chiming in on this giant (and amazing) thread. For starters I guess I should explain that I consider myself quite "techy", but this stuff is way beyond me. So excuse me for any ignorance to the complexities involved, but has anyone figured out the amount of storage required to make these cameras an option in the field? I would assume that one would have a sort of cartridge system of hard drives...possibly in raid pairs?...that they would use interchangeably just as they do with tapes. At night the raw footage could then be transferred to a storage raid array and the recording HDDs reused the next day. Does that sound about right?

Also, I'm just starting to wrap my head around all the HD formats and the hardware/software needed for editing with them. Reading through the various trade magazines, Post, Film&Video, DV, etc. about the workflow of the film (i should say video) Collateral has been quite informative. Shot on 2 Vipers and a couple Sony F900s, Collateral used hard drive recording in some instances, but preffered using sony's SRW-5000 decks, which introduced compression to the Viper's 4:4:4 RGB. The result was still perceptually lossless and therefore was an acceptable compromise for the flexibility it allowed the DP and the art of the project. Now I saw the film, video.... whatever, in theaters last weekend projected on regular 35mm film (nowhere near had a DLP projection of it) and it looked fantastic.

So, I guess my question is where is the point of diminishing returns when it comes to quality/ease of use/affordability? When do our dreams of perfect 4:4:4 uncompressed 12bit become an "unreality" for people of our monitary/equipment situation? I read in these magazines about smoke, quantel, and avid systems prepped for uncompromised HD, as well as the hardware necessary for such a thing and its all a tad overwhelming...especially in cost. How do we plan to do all this on our standard PCs and Macs, and will we have the same output capabilities of those huge, way out of budget systems?

The reason I'm bringing all of this up is that myself and a large crew are gearing up to pitch a feature film to investors which we hope to shoot next summer. We figure the budget will allow video, but we hope to have the option of going to film if distribution is found, so miniDV is not our preffered format.

So, for a group of independent film making college students, (we're actually video-production majors, not film guys), what is the feasibility of shooting, capturing, storing, editing, and outputting these massive formats? The workflow is certainly a scary one, and I need to be able to pitch this as a viable, and doable option. I am the would be editor for the project and am trying to initiate as much learning/planning for the format choice early on in pre-pro so we are set up to go to whatever format our hearts desire (depending on our budget of course. ;) )

Sorry for that huge post, I hope I made some sound arguements/questions?

Oh yeah, and thanks goes to all of the DVinfo community...this is certainly a mind-blowing idea, and it seems to actually be working!! Keep it up guys.

Spencer Houck

EDIT: I should add that i'm a pc guy workin' with Adobe Premiere Pro currently, but am unhappy with its current stability. Do you think a Mac with FCP HD should be included in the budget to work with this media, or will I be ok with my PC and possibly a future version of Adobe's PPro? I don't have any experience working with Quicktime files in Adobe on my PC, is that all good to go?

[Jason Rodriguez]

Also forgot,

We'll probably loose 1/2 to a full stop from the 12-bit image trying to white-balance, so it looks like we'll be around 9 2/3 stops, same as the Viper before white-balance, so theoretically we should get a little better than the Viper dynamic-range wise (the Viper's S/N ratio is only 54db whereas the Altasens is supposedly 68db).

[Jason Rodriguez]

Well this isn't going to be like editing video-tape, but I don't think it'll be quite the beast that it looks like initially.

If you store your files as RAW files with unpadded bits, you actually have fairly small file-sizes. 1080/24p will only be 74.6MB/s at 12-bit, around half-that for 720/24p. So if you're going for 720p, then you're looking at around the same storage needs as uncompressed 10-bit 4:2:2 NTSC.

There will be a lengthy "processing" time, to convert the raw files to a 16-bit TIFF, but that's not any similar to developing film neg, so while this isn't a "shoot and here's your tape" workflow, it's not quite that bad either. Once in 16-bit TIFF, go out to whatever quicktime/avi format you want.

The only problems I see right now are some sort of house-keeping functionality you'll need to incorporate to keep track of your file sequences. But once you get to a Quicktime format (such as the 10-bit Uncompressed RGB codec from Blackmagic), you should be fine. Just have plenty of fast disks and lots of offline firewire/USB drives.

[Rob Scott]

Jason Rodriguez wrote:
Once in 16-bit TIFF, go out to whatever quicktime/avi format you want.

16-bit TIFF is only the first format that we'll be supporting directly. It was the simplest, for one thing :-)

Soon I hope we can directly output to any QuickTime codec you like. There's no reason to use hard-drive-space-sucking 16-bit TIFF intermediates when you don't want TIFF as your final format.

[Aaron Shaw]

Just curious - I have never quite understood how this was supposed to work. Clearly we will need to assemble the images into a movie file of some sort once captured. How do we go about doing this, especially if we use a raid system where these files may be sequentially written upon various drives?

[Rob Scott]

Aaron Shaw wrote:
Clearly we will need to assemble the images into a movie file of some sort once captured. How do we go about doing this, especially if we use a raid system where these files may be sequentially written upon various drives?

With a real RAID system, this isn't an issue. With a "simulated" RAID (like I'm supporting in my Capture app), each frame has a global frame number on it. The Convert app then reads all the files, notes the the frame numbers and reassembles the clip properly.