4:4:4 10bit single CMOS HD project

[Wayne Morellini]

Cross post.

Hi guys heres the little secret I mentioned a while back, and was what I was intending to use on my original camera project last year. I was supposed to read up on it and it has been waiting so long I nearly forgot about it. The technology is Processor in Memory modules. The idea is to implement processing elements (or full arrays) inside memory, simular to clearspeed but on the internal bus of the memory modules instead, here greater speeds and wider busses are accessable, and unlike clearspeed, large chunks of memory are directly accessible, which makes it very good for what we want, and indeed it is earmarked for things like compression. I think I have found the article I read last year, I thought it was refering to making standard dram sticks for PC's using the technology for 50% more, but I am unsure now. Production was hoped to be "18 months" (august 5th 2002 article). The speed up for one was upto 25-40 times over workstation performance (potential for several hundred), and for another it was upto 1000 times (Active page) using arrays of FPGA processing elements). The idea is that if they produce pc memory modules with it, you pop it into your PC memory slot and program it and "hey presto" your 1GHz nano-itx board is capable of processing and compressing 8 mpixel SHD streams (maybe a little exageration) but you get this without (maybe) even needing a cooling fan on the main board, low powered, low cost. To make things even better (depending on what Windows API standards are now) I think Windows had an API that allowed DSP's functions in add-in cards to be transparently used in programs (simular to Direct X API calls) to accelerate them. I remember some international meeting they had for this tech, the web site for the meetings would be a good source of contacts. For somebody like Steve in SI a non PC version could be hooked up to an ARM processor to provide a simple to program alternative to FPGA design (not that the programming is as simple as C coding on clearspeed). Normally I would keep this quiet (to stop companies from interfering) until I had researched, approached and negotiated with suitable companies about the possibilities of even using samples, but because of my health this is just not going to happen quick enough. So, if somebody with technical knowledge would like to do this for us it would be most appreciated.

Here are some links.

The only three that seemed to be aimed at intergration into memory modules is Diva, FlexRam, and Activepage (The FPGA solution), but I don't know which one is the standard PC module.. I have only been able to skim the documents due to health, so I don't know exactly the details, and haven't even looked up all the websites for each competing version.

www.isi.edu/stories/31.html
http://www.wired.com/news/technology...,54294,00.html

http://216.239.57.104/search?q=cache...pages%22&hl=en
http://216.239.57.104/search?q=cache...pages%22&hl=en
http://216.239.57.104/search?q=cache...pages%22&hl=en

This site was useful finding academic papers:

citeseer.ist.psu.edu/kang99flexram.html
citeseer.ist.psu.edu/context/100858/181563

[Steve Nordhauser]

I have checked with the guru of all here on the 24/48fps issue. The final answer is that you can output every other frame in two ways. First is to only request every other frame from the frame grabber. This will give you a frame rate of 24fps, a readout rate of 48fps and a maximum exposure of 1/48th sec.

The second method that we discussed is adding a longer vertical blanking period. This can be done by putting the blanking value in register 6. For this method, we set the pixel clock to the 48fps number, set the blanking time to 720 rows. The exposure time can now be set to any row count from one to 1440 - so you can get 1/24th sec exposures.
The clock should be:
(1280+244) x (720+720) x 24 = 52.66944MHz
The clock generator can put out 52.663MHz using the command: lc254701

Doing this gives you a frame rate of 24th sec, readout rae of 48fps and a maximum exposure of 1/24th sec.

[Rob Scott]

Wayne Morellini wrote:
Processor in Memory modules ...

That sounds pretty interesting, Wayne. It would certainly be an excellent way to get real-time processing using an otherwise standard hardware platform like Nano-ITX.

The traditional FPGA holds a lot of promise too and seems easier to develop that I initially thought ... at least if you know someone who can develop a board and integrate it into product, like Silicon Imaging can.

Steve Nordhauser wrote:
I have checked with the guru of all here on the 24/48fps issue

Thanks Steve! Once I figure out how to set the registers, I'll definitely try those.

[Wayne Morellini]

That's why I origionally targeted it, great peformance, and no production board run (if the PC sticks actually exist). The hardware manufacture can get quiet expensive (not to mention potential patent issues). The FPGA guy I was talking to is looking at starting something shortly, but has not given me permission to indentify him. It's a pitty I couldn't get in contact with Chen Ting (I think the same Bayer filter guy), he does FPGA processors. Some of the people I was involved with had an interest in on chip parralel processing array (one guy wanted 2000-3000 processors on one wafer, that would scaled to 20K 2.4-10Bip procesors now days, I think) but nobody could get access to a memory foundary to do embedded memory versions. I think one server design was in the works. One guy was in negotiation with something that could be able to blast away the speed of everything I mentioned (and any super computer) like the core of the sun would, I suspect they were the same people that a freind over hear ran into locally. I've heard of lots of stuff, one thing in common to most is that nothing arrives anywhere near the time they say it would.

By the way I haven't heard back from you guys about my email.

[Obin Olson]

I had a quote of $50,000 to build a FPGA system for a camera

I am sure we can come up with somthing a little less then that

[Rob Scott]

Obin Olson wrote:
I had a quote of $50,000 to build a FPGA system for a camera

Yikes!

Actually, that's probably reasonable for commercial development. But between existing open-source cores and other open-source software out there ... yeah, we can probably do it cheaper. (But almost certainly not quicker :-)

Wayne wrote:
I haven't heard back from you guys about my email.

Sorry for the delay, Wayne. I just responded.

[Steve Nordhauser]

Obin,
Be sure to specify the IP that comes with the FPGA design. Bayer conversion (algorithm, bits), compression (same), data packing, etc. Be sure you know who owns the board design, IP and any distribution rights and licenses involved. Hardware interfaces, maximum clock rate, cost to manufacture or purchase more are all important.
Steve

[Jason Rodriguez]

Obin,

ran into some very weird problems with FCP and the Sheer video beta. I've emailed them back to see if this is a bug, and if not, maybe I'm doing something wrong. But anyways, as of right now, the Sheer files you've sent me and FCP aren't working together, but they do play back fine inside of Quicktime Pro.

JFYI.

P.S. I was playing around with the Sheer file that you gave me in AE though (it works fine there), and there seems to be some wierd horizontal banding to the noise signal in the dark areas. I am very impressed though with the highlight control, and even when you've clipped the highlights, with some gamma correction and curves you can get a very nice look to the highlights, definitely much better than another digital video I've seen next to the Viper.

[Obin Olson]

thanks for the feedback Jason...I think the same thing about this camera but it's hard to tell when I am so close to it everyday! I like hearing some feedback..what was that clip I sent you? (I forgot) :)

BTW you have to set AE for 16bpc did you do that? if not your working with 8bpc

[Jason Rodriguez]

The camera was slowly turning around you.

No, I wasn't in 16-bit mode, but that shouldn't matter with a noise signature. If there's horizontal noise patterns, then they'll show up 16-bit or 8-bit. I'll send you a JPEG of what I'm talking about.

[Eliot Mack]

I was reading a bit about the Sheer Video codecs--they look very promising. 2:1 compression in real time for $150 is pretty much a no brainer. I'm on Windows, so I can't use it (yet).

Obin, does 1280x720x24fps 10 bit Sheer video require a RAID on your system, or is it working from a single drive?

Thanks,

Eliot

[Jason Rodriguez]

1280x720x24fps 10 bit Sheer video require a RAID on your system

Not Raid required, it's playing back fine from my single SATA drive, although I am having problems as mentioned before with the FCP implementation of the beta 10-bit codec(s)

[Obin Olson]

No raid on PC

[Rob Lohman]

The are some rumours of a new nVidia chipset:

- Socket 754/939 and 940 capable
- 1 - 8 CPU's
- 32 PCI express lanes. Possible configuration: 1x16 X PCI-E slot, one 1 x 8X for SLI card and four times one lane for peripheral devices
- two times Nvidia Gbit Ethernet
- nVidia RAID
- eight S-ATA drives
- six P-ATA drives
- 10 USB 2.0 ports
- But it will arrive towards the very end of this year

Source: The Inquirer

Sounds interesting!

[Rob Scott]

Rob Lohman wrote:
...rumours of a new nVidia chipset

That sounds very nice. If someone could put that in a Nano-ITX board with a low-power mobile chip, that would be nearly perfect.

Another software update -- see the blog.