4:4:4 10bit single CMOS HD project

[Juan M. M. Fiebelkorn]

Guys:

I'm still waiting for a link to a RAW image sequence (zipped) to test a little compression algo I'm developing with a couple of fellows.If someone could be so kind.....

@Steve

So no mechanical shutter....I see.The rolling shutter artifact doesn't exist at 48 fps?

@Obin
50,000 , are you nuys!? :)
Here that would cost no more than 5,000.Maybe 10,000.

If anybody is interested, a german company manufactures a PCI card that uses ADV202 jpeg2000 compressor, realtime.In case nobody knows that chip accepts a RAW input.

I'm planning to develop a camera myself, but I need a little more data....

[Rob Scott]

Juan M. M. Fiebelkorn wrote:
I'm still waiting for a link to a RAW image sequence (zipped) to test a little compression algo I'm developing with a couple of fellows.

It will take me a few days, but I'll be glad to. I could send you one pretty quickly if you don't mind it being 16-bit padded ... actually, it's 10-bit data shifted so it appears as 12-bits and then padded out to 16 bits. It looks kind of like this: 0000xxxxxxxxxx00 where x = a real data bit. In addition, the data is written to the files in Intel-order, so the actual file I'm writing out has the LSB and MSB swapped. So I guess really each pixel looks like "xxxxxx00 0000xxxx" in the data file.

(The final data format will be 10-bit packed and/or RLE-compressed and Motorola-order.)

ADV202 jpeg2000

That sounds pretty interesting too. I wonder what it would take to embed a chip like that into one of Silicon Imaging's cameras and then stream out the data via Gigabit Ethernet? (Of course you'll still need to do the Bayer filtering, unless this JPEG2000 chip does that too.)

[Juan M. M. Fiebelkorn]

Well the chip costs $40.

What I need for my testing is a 8 bit or 16 bit image.In fact I need the Bayer RAW pattern image intact to see how well it performs and if my Lossy system gives the results I guess with this camera.

[Rob Scott]

Juan M. M. Fiebelkorn wrote:
What I need for my testing is a 8 bit or 16 bit image.In fact I need the Bayer RAW pattern image intact to see how well it performs and if my Lossy system gives the results I guess with this camera.

Yup, the raw Bayer data will be in the file -- it's 10 bits per pixel for the SI1300 camera.

In unrelated news ... I've drawn up a UI mockup of the "Capture" application. Everyone, please feel free to send me any feedback you may have, as long as it is polite :-) Thanks!

[David Newman]

Juan,
One of the difficulties with bayer compression is that it consists of high frequency data that is significant; compressor like the ADV202 (years ago I used the ADV601) will treat the high frequency data like noise and quantize it heavily. The result is significant cross-talk between chroma channels, distortion, and poor compression. If you separate the channels you will have better luck.

CineForm's investigations into native bayer compression have gone very well, but it will take time to commercialize. If or when we develop a commercial product for this market we certainly make announcements here.

[Obin Olson]

Steve does the 1300 give readout of what it's settings are? or can you only upload changes to the camera, not download them for readout?

so can we see a register that we set like GAIN? or is that feature not implemented?

[Juan M. M. Fiebelkorn]

I always said I would separate the chroma channels.Believe me, I know what I'm talking about :).
Anyway at first I will compress LOSSLESS.

[Rob Scott]

Obin Olson wrote:
...does the 1300 give readout of what it's settings are? or can you only upload changes to the camera, not download them for readout?

Looking at the SDK, I'm pretty sure you can read the register settings. (Though I haven't been able to do it yet.)

Juan -- I uploaded a raw file like I described earlier. You can download it here. The resolution is 1280x720 and the values are 10-bit data stored in 16-bit fields as I described earlier in this thread.

[Steve Nordhauser]

Obin: I don't think you can read the registers in the SI-1300. I think someone got lazy and didn't implement the low level read - I don't see a high level command corresponding to the "ly" load register. This is what happens when a hardware person designs things - you get write only registers.

The way around this is to keep an image of the registers in software. The entire array gets written with an init command and modified whenever registers are written.

[Wayne Morellini]

Hello Oblin

I've tried to download your latest two clips but they come up as not found. I got nearly 170mb of the original sheervideo clip loaded in the background over a week, before it failed and returned this error. By the way can I even veiw these in Windows XP, this thread is getting big. Rob, any movement on my suggestions for an initial updatable title/reference post (or even a web link to homepages of things like the Wiki it would make it easy for new users to catch up. I've seen many questions I myself, or somebody else, have personaly answered coming up.


<<<-- Originally posted by David Newman : Juan,
CineForm's investigations into native bayer compression have gone very well, but it will take time to commercialize. If or when we develop a commercial product for this market we certainly make announcements here. -->>>

What about in a high end edition for the new professional high end digital movie cameras like the Arri.


<<<-- Originally posted by Juan M. M. Fiebelkorn : Guys:
@Obin
50,000 , are you nuys!? :)
Here that would cost no more than 5,000.Maybe 10,000.

If anybody is interested, a german company manufactures a PCI card that uses ADV202 jpeg2000 compressor, realtime.In case nobody knows that chip accepts a RAW input.

I'm planning to develop a camera myself, but I need a little more data.... -->>>

I agree, even though it would be complex (Eplin's allready fills most of 300 gates and they are moving to 1000K gates) that does sound like standard engineers' pricing. Watch out in dealing with engineers, you have to specifiy in the contract that you, not the engineer, solely own all relevant rights to the design and any IP associated with it (the following sort of things have allready happened). Otherwise you can lend up $50K poorer, holding a disk with the design on it that you can't even use without licensing from the Engineer. I have been an member of the inventors association and this has happened, as Engineer's can produce a single product under contract, and automatically own all rights to it, including replication of the product, it's like the portrait photographers click once gold mine.

<<<-- Originally posted by Juan M. M. Fiebelkorn : Well the chip costs $40. -->>>

Would that be around $400 to the price Steve?


<<<-- Originally posted by Rob Lohman : The are some rumours of a new nVidia chipset:

- Socket 754/939 and 940 capable
- 1 - 8 CPU's
- 32 PCI express lanes. Possible configuration: 1x16 X PCI-E slot, one 1 x 8X for SLI card and four times one lane for peripheral devices
- two times Nvidia Gbit Ethernet
- nVidia RAID
- eight S-ATA drives
- six P-ATA drives
- 10 USB 2.0 ports
- But it will arrive towards the very end of this year

Sounds interesting! -->>>

Cool! Does that mean we can do Ultra HD to Raid ;). Steve, 2 GbE = 2Gb's = 1.6Gb's throughput (with your drivers) = 24fps 1080p 3 chip. I think this is great sort of stuff from your end, as MB's with 2 free GBE on board (plus many more through PCI-E) hooked up to GbE cameras, without need for a capture cards, should be very attractive.

I had an interesting idea for a color filter previously, instead of the second green in the bayer pattern you could replace it with a infrared filter, or ultraviolet, or have alternating infra red and ultra violet pixels. This would be great for security footage, or parts defect analysis (I wouldn't mind using it in pro video work myself as an effect).


I notice some talk of HD display stuff, I have soon interesting news.

http://www.tomshardware.com/hardnews...02_125338.html

They are talking about 40inch LCD TV's being between $1K-1.5K within the next 12 months. I'm not sure wether they mean Panels or rear projection units, but there has been two or so cost reducing revolutions in LCD panel design I have been waiting to come through, this maybe it. What it means for us, is those bright, sharp accurate $700 17inch 720p LCD TV's may come down a lot in future (notice that not much is expected this year).

I'm looking for TV's at the moment and have been over at the avscience forums. DLP 720p projectors much more expensive than LCD projectors which have worse image quality (good price Steve). The Sony Widescreen TV is around $2999AUS over here at the moment, so it will probably be much cheaper in US dollers, I think it wouldn't be anymore than 720p. Pulling a Jaun, I wonder if some parts canbe reprogrammed or changed to allow it to display smaller/more scan lines, (I'm sure some of these TV's are using components with accuracy way past the displays spec, and only some parts might need mods to enable it).


Now an interesting topic:

Is it possible to get a Normal TV with component to accept and display a hires signal. I remember, years ago, that you could get the Commodore Amiga monitor (TV RGB) to do around 800*600, and even higher than 1000 pixels horizontal, it would sync to the signal. Could I do a simular thing through component, possibly adjusting the internal sync knobs etc as well, and picture sizing (out of warrantee of course), possibly even getting a 720*1280 display. I noticed some TV's seem to have a much denser pixel matrix (and far better/sharper picture), but I would be surprised if it could achived true 720*1280 resolution, but it is a fun idea and probably just as good as projecting my monitor through a freznel lense system.


<<<-- Originally posted by Steve Nordhauser : Freya:
There are people with varying needs here, from 1280x720@8bits/24fps to 1920x1080@12 bits/30fps - very different solutions are required.

I'm sure that I left things out, but I think that is the charter of what is going on in Alternative Image Methods in about 3 or 4 threads. -->>>

Accurate enough Steve. I think the support between 720, 1080 and 2160p, 3chip or Bayer, 24fps, 30fps, 1000fps, and the many different cameras from different manufacturers, can be handled by mods to the same software but progressive hardware support of larger data streams will biol down to hardware advances. I'm interested Steve, you have 6.x Mpixels camera, when do you expect to get a 8Mpixel bayer at 24-25fps? An x-ITX board with PCI-E (desktop version) capture card would handle 8-MPixel. Just saw Spidey Two, and was not impressed by the grain (another plus for HD) and resolution (it looked to soft), I think SHD bayer would give it that Imax edge.


<<<-- Originally posted by Rob Scott : The biggest challenge for me so far is squeezing in the time to work on it! It's also time-consuming because I'm working with stuff I've never done before -- DirectDraw for example. -->>>

I know of a system called GAPI that apparently provides high speed interface to Windows direct draw, Mac etc platforms. It is optimised for speed and used in games (that normally need that sort of thing) and I think it is free for non-commercial use. It probaly will greatly maximise your speed (instead of learning the optimisations yourself), as it properly handles the graphic pipe line.

<<<-- Originally posted by Obin Olson : creative freedom!

I am going NUTS over how well Vegas works with this camera footage! EVEN 8 bit in Vegas is not bad at all for color work..I have no idea why but it looks really clean!

just simple VEGAS VIDEO..I am impressed. VERY impressed.

Now I think I am starting to see what the "Vegas Cult" is all about.

;0
-->>>

Another suggestion, is when you find a good solution in 8 bits, write to them and ask them if they can include 10-12bit support (16 bit would also be good in the future), we can pre-edit transcode from bayer to a normal 4:?:? anyway. Maybe some would support it. You mentioned Caldera, or something, I think they have camera capture in that enironment, maybe they would like to do a Windows/Mac version with 10 bit bayer dirac stuff?

Have fun, have we though of power supply, how to go from DC battery through the PC AC power supply?

Wayne.

[Juan M. M. Fiebelkorn]

Wayne, many micro-itx are coming with support for DC energy.
Check it! :)

http://www.mini-itx.com

[Rob Scott]

Wayne Morellini wrote:
... 10-12bit support (16 bit would also be good in the future)
... 10 bit bayer dirac stuff ...

For a lossy-but-visually-lossless codec -- as I hope Dirac would be -- it seems like it wouldn't really matter what exact bit depth you started with. Even if the current Dirac is 8-bit, we should be able to create a 16-bit version; then you store everything as 16-bit, whether you start with 10, 12, 14, or 16 bits. Does that make sense?

I think I mentioned this before, but I think a nearly ideal system would be a FPGA-based camera that can do a high-quality Bayer and Dirac (16-bit, as above) compression in real-time and stream out a highly compressed (but still visually lossless, 8:1 or more?) stream via a single Gigabit Ethernet cable.

Juan wrote: many micro-itx are coming with support for DC

Yeah, I noticed that many of them can be powered from a single 12V source. With a transformer to produce 5V, we should be able to power the camera, motherboard and hard drives from a single 12V power supply.

[Steve Nordhauser]

Prolific Wayne:
ADV202: This could be used in a high volume inflexible design and get where you want to go. It takes YUV in so you need to do the Bayer and RGB->YUV in an FPGA design to be fast. You need to select an algorithm for Bayer that meets your image quality needs. 1080i takes two of these chips. This is a *big* design project. It takes high volume or $50K price tags to justify the engineering costs. CPUs are getting faster and cheaper - my bet is a codec that can be moved to a fast DSP. A bit more expensive but more flexible and much lower development costs.

Formats: You are correct, there are dozens of odd applications - high frame rate and such that vary from the garden variety cameras. It would be nice if you could do tradeoffs. CMOS is nice for this. The Altasens should be able to do 720p at over 120fps, VGA at over 300fps. There is no cost associated with trading resolution for frame rate - the pixel rate stays the same. Increasing the pixel clock costs more at the sensor, communications and system. The 8mpix from PVS are crazy expensive right now. We are working on higher resolution cameras but I think the Altasens will be the highest res at 24+fps for awhile.

Markups:
Originally posted by Juan M. M. Fiebelkorn : Well the chip costs $40. -->>> Would that be around $400 to the price Steve?
Maybe for a company with higher overhead. Otherwise the Altasens camera would cost $7K just on the sensor alone. It depends how much engineering costs are, product volume, yada yada. If I put a $10 FPGA in and do a complex design, part of what you pay for is the IP I developed. If I put in an A/D converter. That is what you get. I will admit, sometimes there is a 'perceived value' that you pay for - first to market, no competition but I'm an engineer and only marginally buy into that. Fair pricing on a new product creates a barrier to competition - why compete when the price of the existing stuff isn't artifically high?

GigE and compression:
You could use two channels to move more data. Or lossless compression. I find 'visually lossless' the most intriguing. David Newman says it is possible on a reasonable machine at 1920x1080x30fps. This means with a 6:1 you can record to a single removable drive. Let this brew for awhile but it could be big.

[Rob Scott]

Steve "back to reality" Nordhauser wrote:
If I put a $10 FPGA in and do a complex design, part of what you pay for is the IP I developed.

Yes indeed, but what if many of the components you need are already available as open-source? Your costs could be reduced substantially.

BTW, how many gates in a FPGA chip do you think it would take to do ...

• a real-time 1920x1080 Bayer filter at 60 fps
• conversion to YUV
• a wavelet compression such as Dirac

I'd be curious if current low-cost FPGA's would handle it. I won't hold you to anything -- even a WAG (wild-a**ed guess) would be useful.

[Obin Olson]

Steve I jsut spoke with a broadcaster and they will NOT take any 720p stuff...do you have any news on the 1080p camera yet? also how can we get 1080i...seems this broadcaster liks 1080i the best..can you get it from the progressive scan cameras?

wow 120fps from the Altsense at 720?!!!?? that is AWESOME! what data rate would that be? I am guessing it would take a 64bit card to capture that?