4:4:4 10bit single CMOS HD project

[Eric Gorski]

thanks obin for posting some footage. keep it coming.

[Rob LaPoint]

I was thinking about my idea of using one computer for preview purposes and one for capture. Obin I am sure that you guys will get your program optimized and preview during capture no problem, but would it give us more leeway to design computers with dual processor and then just have one box with a processor each for capture and preview (or is processing not the issue)?

[Obin Olson]

hmm..pixel clock is or was 60mhz and shutter I think was 1/120 sec or maybe it was 1/60 sec.. gain was about 3db

that is a really bad bayer filter just the basic linbayer type

One problem is that when you expose like I did and then BOOST the gamma in post the range of CC in Combuston is very small because of all the gamma boosting that has happened..this is why we need over 8bit for an image that can be pushed really hard for color "looks"

[Obin Olson]

not a bad idea ROb....so have 2 main boards one for capture and one for preview inside the case? split the CameraLink signal that is coming into the systems?

[Jason Rodriguez]

Why do you need two computers, one for digitizing and another for preview?

That is totally cumbersome, and very impractical IMHO.

Might as well just lug a big cart behind you with everything in it.

Of course we have no way of knowing whether the problem is performance or whatnot since I have absolutely no clue what Obin is running.

If he's got a Pentium II 400Mhz with 128MB of RAM and a couple PCI slots, then yes, there's going to be performance problems.

Pentium M 2.0Ghz with 1GB of RAM, PCI-X, Intel Extreme Graphics 2, etc., I'm not really sure there's going to be problems, especially since all we're doing is a data dump to hard-drive, not compression or anything else like that.

Frankly as of right now there's no way to tell what the problem might be because nobody's willing to list their configurations here on the board.

[Obin Olson]

sob sob jason ;)

Hey I have a 3.06ghz P 4 with 2 gig ram and 2 IDE disk drives one 7200rpm and one 5400rpm both set as capture disks running the Epix 32bit framegrabber card on standard PCI 32 slot and a ATI AGP card with 128megs ram on it

I will be capturing 8bit from the framegrabber later on today..I will post results when that is done

[Rob Lohman]

I think I feel the same way as Jason about it. It just should be
possible to do it all in one system (even with some basic lossless
compression) for at least 720p.

Obin: do you know if your programmer is working in just C(++)
or is he also doing assembly/MMX/SSE etc. (ie, handcoding for
the CPU)? We are for Obscuracam for some important routines.

I did some demo coding back in the days that 386 was a hot CPU
and a LOT of speed can be gained with carefull handcrafting of
processor instructions, algorithm optimization and whatnot.

Even in the case where full resolution preview might be not too
good to do you could think of the following:

1. display the preview at the maximum framerate the extra CPU cycles will support (so it might be lower than the real frame rate which is not too much of a problem for preview, Rob S. is doing this now for example)

2. do a full resolution de-bayer when the camera is not recording (so you can more easily set critical focus) and switch to half resolution (ie, you don't need to do a de-bayer) as soon as you hit the record button

3. do the preview during recording in black and white (no need to de-bayer)

4. check multiple ways to construct 8 bits from 10/12 bits (without loosing the ability to see the dynamic range), see which is fastest for preview

You can combine all of this together ofcourse to get some very
critical speed increases for the viewfinder system without getting
a camera that you can't work with and hopefully gain a bit of
cycles to do things like zebra striping / histograms (although these
could perhaps be a pre-recording check only function as well)

[Jason Rodriguez]

Hmm . . .

Thanks for the info Obin.

Seems like a pretty good system to me. Maybe the 5200RPM drive isn't enough for capture, but processor-wise, you should have plenty of juice for 12-bit 1920x1080, especially now with that ATI card.

So you're only able to get 8-bit? Or is that a problem with the program?

edit: Nevermind, 12-bit is too much for that framegrabber, in fact 10-bit is probably too much also since there's no framebuffer on the card, so each 10-bit frame actually takes up two bytes per pixel rather than one.

[Rob Lohman]

Isn't the 32 bit PCI bus the problem here Obin? You are getting
a massive amount of data in at 1920x1080 *and* need to shift
this data to the harddisks....

Ofcourse in the end it all boils down to programming efficiency
(which is the major component here), which we ofcourse cannot
see for your program Obin (I'm not saying your programmer isn't
doing a good job, but basic programming has nothing in common
with hand crafted speed optimizing for example).

[Jason Rodriguez]

check multiple ways to construct 8 bits from 10/12 bits (without loosing the ability to see the dynamic range), see which is fastest for preview

Wouldn't the easiest method be to chop off the two LSB? A LUT would be ideal, but I think that would take too much time for each image, especially if it's 1920x1080.

One other thing I'm wondering is if you could do a 1/4 preview, and then apply a LUT to just those pixels for display, so you're not trying to do a transform on 2 megapixels @ 25fps. I think that'd be too much for any system (my dual G5 can't do real-time HD effects like that either, so I definitely wouldn't expect that out of a small computer). So with the 1/4 preview, you now can see what's happening dynamic range-wise with the LUT, and that's only performing an operation on 960x540, which should easily be feasable, especially on a new Pentium M or Pentium 4.

[Jason Rodriguez]

Isn't the 32 bit PCI bus the problem here Obin?

That's why I'm also curious what the chipset is.

If he's using a good southbridge, there should be no problems, at least if the IDE is on-board. Hublink 1.5 from Intel runs at 266MB/s full duplex, so there shouldn't be any problems going back and forth from main memory, if that's what he's using (which I'm assuming he is). Also the IDE bus should be straight into the southbridge on a seperate channel from the PCI bus to prevent more bottlenecking there.

If it's a good southbridge (more than ICH4, like hance rapids, ICH5 or ICH6), then there shouldn't be any problems, especially if it's a southbridge that supports PCI-X (which Obin's doesn't), because then it has to have a guaranteed bandwidth of at least 500MB/s+.

[Steve Nordhauser]

I think everyone is hitting around the right area. You need to pay attention to the system architecture. It is even worth looking at the chip sets. Some of the more recent Intel chip sets have a southbridge (ICH5-R) with a two drive SATA RAID built in. Sure you save $100 on the controller, but more importantly, the disk data (assuming two drives is enough) never hits the PCI bus. You can ignore this in the 64 bit world but it is very important to a 32 bit system.

Rob L. is correct (a least from my experience) that a little bit of assembly can go a long way. In these cameras 90% of the CPU time is probably taken in <1% of the software. Assuming that you use DMA to move blocks of memory to drives, the tightest loops in your software need to be examined. The Bayer preview. The compression, if any. Any real-time video algorithm (white balance). Since that is where the CPU spends its time, a 5% algorithm savings (time, not size) is almost equal to 5% faster CPU time. You might need to understand the CPU caching system also. You can prototype all in a high level, but the optimization should include a long stare at the loops in the code.

[Rob Lohman]

Steve: exactly. It's funny you mention the points you do since
the packing, some parts of the preview are all in handwritten
assembly at this moment (I'm working on converting the
compression to assembly as well).

Wouldn't the easiest method be to chop off the two LSB? A LUT would be ideal, but I think that would take too much time for each image, especially if it's 1920x1080.

One other thing I'm wondering is if you could do a 1/4 preview, and then apply a LUT to just those pixels for display, so you're not trying to do a transform on 2 megapixels @ 25fps. I think that'd be too much for any system (my dual G5 can't do real-time HD effects like that either, so I definitely wouldn't expect that out of a small computer). So with the 1/4 preview, you now can see what's happening dynamic range-wise with the LUT, and that's only performing an operation on 960x540, which should easily be feasable, especially on a new Pentium M or Pentium 4.

No, you can't simply lob off the 8 bits per color, here's why:

10 0000 0011 (10 bit color = 515)

this would end up being:

0000 0011 (8 bit color = 3

I don't have to tell you this results in a massive color and/or
brightness change for the pixel. I did it during some testing with
Rob S.'s RAW files once this way and got all kind of junk in the
frame (which took me a while to figure out). This is one of those
places where you easily go wrong with an algorithm implementation.

It would be far better to chop the 8 bits and set the highest bit
on the resulting bits if there was a value in the just chopped 8
bits (is what I did to check the RAW images from Rob S.). However,
you loose the 2 bits (or 4 for 12 bits) extra latitude during preview
so things will look washed out which you don't want. It would
probably be better to shift the 16 bits either by 2 or 4 bits to the
right. Or even better yet use a LUT (lookup table) indeed.

I do believe Rob S. is using a LUT for preview in his current version
of Obscuracam. This preview is indeed in half resolution (vertical &
horizontal) to reduce processing time and not having to de-bayer.

So that fits in neatly into the points I made above. Although it might
be interesting to have a full color/B&W before recording or have
a full B&W preview.

In my 3.2 GHz pentium 4 cpu I can view 1080i in realtime with a
WMV HD codec I think. So I think much must be possible, it all
boils down to proper design, testing, monitoring and profiling to
see which can and need to be (hand) optimized.

[Steve Nordhauser]

Rob,
I'm hoping that was lop off the bottom two bits as in a shift right by 2 (or an integer divide by 4)

10 0000 0011 (10 bit color = 515) or 515/1024

this would end up being:

1000 0000 (8 bit color = 128) or 128/256

[Obin Olson]

thanks for the info guys..I am sending all this to my code writer for review..

We will look at things in detail once I get a board to him for testing