4:4:4 10bit single CMOS HD project

Rai Orz · November 11th, 2004, 02:53 PM

<<<-- Originally posted by Barend Onneweer : It's always good to dream (mechanical shutter, optical viewfinder etc.)
...
............. but with all these ideas about FPGA's and other solutions, there's nobody around here that can actually build a camera that way.-->>>

Those things i love so... ... Noboy? Will see...

<<<--On another note: an optical viewfinder needs a prism to split the beam of light in two beams. Thus only half the amount of light hits the CMOS.-->>>

Not at all. Think on mirrow-shutters. No light loss.
And you will also need some kind of (external) shutter for the altasens. Thats true.

Dont think i love optical viewfinder over all. No, i only say, its one thing that work.

Marin Tchergarov · November 11th, 2004, 03:41 PM

Last thing for today (no more spam :)) !

This document could be very interesting-there is a chapter inside "ATA Interface Dissection":

Quote:

Since I wanted to interface a hard drive directly to the Internet,
speed and buffer storage were of significant importance.

source: http://www.timothyjordan.com/portfol...sis/netapp.pdf

...and is this chip similar to "FPGA"? http://www.st.com/stonline/books/ascii/docs/10350.htm

Best Regards

Marin

Valeriu Campan · November 11th, 2004, 03:53 PM

<<<-- Originally posted by Ronald Biese : Hm
Vance has done this on his PAL cam
do google for Vance Cam -->>>

Vance has used an LCD shutter with a light loss of ~2+ stops. You can see the diagrams to sync the shutter with camera. It would be nice to drag him back into this forum.

Obin Olson · November 11th, 2004, 04:54 PM

HUGE BREAKTHROUGH TODAY!

we can now capture 8bit 24fps with no dropped frames AND it looks like 10bit 24fps with NO dropped frames on pci 32!

FULL 1080p!!!!!

more to come later!

Rai Orz · November 11th, 2004, 05:11 PM

@Marin: Yes, indeed, its what i said: The pdf show a simple example for "direct to disc" solutions. You need nothings between HDD and CPU. No controller, no extra parts, nothing.
But for HD Camera the MC RCM3200 in this shematics is to slow. It runs only at 44,2MHz. But you are on the way.

Wayne Morellini · November 11th, 2004, 07:37 PM

Cores
OK as I said before, you can go to Opencores (or other) ask people there if anybody would like to be involved. You can ask in the opensource community too, but mainly programmers. I'm sure people would love to do this just for he fun of it. You can also ask on FPGA newsgroups (many pro's) who should also be able to tell you of pro mailing lists and web forums stack full of bored eager FPGA'ers. Opencores may have half the circuit designs you need (where ever they all work perfectly I don't know.

Opensource
Whatever FPGA core that is created should be opensource so any manufacturer can use it but not own/control/pricing it. Steve has allready offered to use it. You will need to break the design down into theree parts, front end, so different sensor heads/manufacturers be used, processing middle end, and rear end so that different capture interfaces canbe used. With this stratergy new sensors and interfaces canbe supported, it also is not very difficult to do. lots of flexibility that stops us getting tied down.

Cost
As I say before it takes tiem and money, more than software PC should,

How easy
I understand some of the programming langauge for FPGA is easy, and people here should be able to do, but you need to learn what your doing and have knowledge for circuit board design and interfacing in a high speed environment (that introduces niose factors that make design tricky). But my "guess" is that you can mostly black box the FPGA in isolation from real hardware and get somebody else to do the interfacing, aslong as you are sure of what data and format is going to come in and what data is needed to come out. My guess is that it will take upto 6 months to learn all this. A few people here indicated they would do it and started a thread (look it up).

Manufacture
The problem you then face is manufacture, unless somebody else pickup the tab (like SI) low volume costs are much higher than high volume costs, so you might find board is as expensive as PC system until sales hit at least 1-10K. But I'm not being completely negative, I think it canbe done.

The www.untratechnology.com forth processor chips section has links o a number of projects. Some people in this level of the forth community tend to be very good programmers and like community initiatives also.

Cheaper more flexible alternative/hybrid
I would suggest that it maybe better to put a small processor in the core fort the control functions etc (or get a processor with a core, they are out there but I don't know if they have big enough core). If you go to ultratechnology forth ciips link above you will find probably thee smallest/fastest cores available (I'm talking much less than 16K gates). Once you go down this path and also use functions from open compression cores on the web you can make parrallel arrays of processors (preferably in sequence, as parrallel to external memory is way difficult for a FPGA or in Silicon). Now you can reprogram any time, any thign you want, just like a PC (except you willhave to get Forth programmers to do it, because it is queit a leap in logic from C or even assembler). Still maybe you can get some cheap embedded processor to do it (AMD MIPS or Arm) and mostly scap the FPGA (though there might be ARM/MIP with FPGA). Still low volume cost still comes into it, but I'm sure people will start doing Windows devices on 2 Ghz Arms sometime, or embedded controller, that could be bought bulk OEM (rather than consumer) discount, or just buy the design and get it network (contract manufacture out) manufactured like Nintendo manufactured. Still make you own and network manufacture it. Or just say to camera/sensor manufacturers "here, opensource", or how muchh for oem and we add the baord, or whatever, speculative.

Others
You will probably find that somebody is allready down this path, but you face the problem that A PC is extremely flexible and code modifiable compared to FPGA. Also what codec and software you can support is more flexible on a PC (as I said elsewhere) so you could even boot the PC cqamera up as an editor and add monitor and drive and pheriphals by VGA and One usb port on motherboard for portable feild work after shooting.

Wayne Morellini · November 11th, 2004, 08:14 PM

<<<-- Originally posted by Steve Nordhauser :
..
Wayne:
The Altasens can read out at up to 60fps for a full frame, letting you drop every other frame at 24 or 30fps. It is still rolling shutter.-->>>

Steve, I was just quoting what I read that the actual read out time is many times faster than he frame rate without dropping frames, which is how they deal with rollign shutter and many times superior to what canbe done with present rollign shutter cameras.

David,
Thanks for that

Rob
I forgot not only reducing gain, but also clock speed as that might has something to do with it. I veiw it as feed of oversatutared pixels (which also works on satuation inside clock periode, faster clock might make it more leaky, or less, depending on the design problerm causing it). Steve?

Mechanical shutter
The problem is that the rolling shutter chips tend to readout in parralel with acquiring the image, we have tried to reduce this by putting readback back in the blanking part of the frame and speeding up the clock (is that right?). But still you get the readout going accross to much of he frames acquisition time. So the mechanical shuter is going to have to be shut for a big periode of he frame greatly darkenignthe image. With Ibis global shutter, from what I understand, you also lose substantial light (but probably not anywhere near as much), but you start with less fill facor (= lower light) to start with.

Next chip please! If they replaced these chips with more Altsens like versions we would be doing well.

Optical veiwfinder VS Electronic
I too would like a more professional electronic solution, no light loss and an objective version of what the sensor can see (pluss I can replay it and decide if the scene needs another take and the old take needs to be kept or not).

About Vance and Jaun
I think Vance knows about this project, he has been over in my Homemade Camera design thread before, adn I have posted details on Jauns thread. Maybe Jaun might like to do a free (or probably commercial) FPGA/capture device for us. I'm not all for integrating methords, unless it reduces cost.

Obin Olson · November 11th, 2004, 10:15 PM

I hate this board...can't post anything long at all/....

BTW check this out: www.dv3productions.com/pub/Obin Test3.mov

that is HEAVY compression for the internet..the original looks REALLY good....VERY soft and "filmy"

Eric Gorski · November 11th, 2004, 10:41 PM

holy crap, that looks good...

how much did that setup cost you obin? i mean if you weren't getting the camera for free for testing it :)

btw.. i just realized something cool.. if you're shooting full letterbox 2.35:1 aspect ratio, the bending of the image you get when panning quickly with the rolling shutter would be less of a problem because there's less vertical image..

Obin Olson · November 11th, 2004, 11:04 PM

I need help.

WIth our 8bit captures right now I have to WAY under expose to get the range out of the camera I want. The bad thing is that when I shoot like this and bring the gamma up in post the images are falling apart really bad. Lots of banding artifacts. I am doing this in Combustion and Digital Fusion. Both have the same results. Is the nature of 8bit captures or can somthing be done about this that I don't know about?

Please if someone knows the answer to this let me know.

I am sure that 10/12bit will be much better..but 8bit is AWESOME if the contrast of the scene like the above post is not too high.

"bending" or rolling shutter will not be an issue at or above 67mhz..I have run that with the camera in 8bit mode and it is almost gone..as soon as I can get a 64bit FG card I can run at a very high mhz and kill the rolling shutter issue.

WOW 1080p!

Oh the above image has been resized to 1280x720 so that you guys on the list could see it on your monitors...unless you have a monitor that is above 1024x1280 this mov file should playback just fine..if you have a high res CRT I could encode a full-on 1080p file ;)

I should have 1080p 12bit files soon..I will need a faster set of disk drives but I will do some tests at 15-20fps to see how the banding is from an under exposed 12bit capture

CPU load is now at 20% in black and white and about 50-60% with color display...we killed the multi-threading of the save. That was the issue with 100% CPU load before

Eric the camera costs about $2500 and I have put 3 months of work into this system to get it working with my programmer who I have hired for this project.

Jason Rodriguez · November 11th, 2004, 11:40 PM

Yes Obin, what you're describing is the nature of 8-bit LINEAR images.

Your good video cameras, Varicam, F900, etc. may record 8-bit images, but they're receiving 10 and 12-bit images off their A/D converters before video processing begins. For our project here we're doing video processing inside of a post app like Combustion, but in order to get the same results that you'll get out of a high-end camera, you'll need 10 to 12-bits to prevent that banding, etc.

Hope this helps.

BTW, sweet images!! (and no more jerkies!)

Obin Olson · November 12th, 2004, 07:49 AM

thank you Jason..that's what I was thinking but not sure...so at 8bit you really can't shoot high contrast stuff at all...hmmm if you do you have to expose for lots of blowout to get an image that will work or shoot the same scene in 10/12bit and under expose a great deal for later gamma work in post

I am waiting for the "convert" part of cinelink to be updated with 16bit tiff output

The shot of my face was very low-contrast lighting and it worked well. Just expose what you see and it's fine

Ok I am looking at 94MB/sec datarate off the 3300rgb @ 12bit/24fps 1080p..David have you made any progress with the realtime CineForm Codec? now would be the time to start working with it in CineLink..let me know what your thoughts are about this..thanks

Aaron Shaw · November 12th, 2004, 10:37 AM

Obin,

How hard would it be to make your system fully contained and portable?

David Newman · November 12th, 2004, 10:39 AM

Obin,

Yes we've made lots of progress on the codec. But unfortunately limited time and and some technical issues with Camera Link have slowed down a ready to go solution. I would be concerned but the 94MB/s over standard PCI, I found it would drop frames occassionally due to bus saturation. Email me for followup on codec details.

Hayden Rivers · November 12th, 2004, 10:45 AM

[Vader]Impressive.[/Vader]

I'm such a geek. Very nice clip. Doesn't play very smoothly on my Athlon 1700+ with 32mb video card or my Athlon 3200+ with 8mb video card (don't ask). Still, it looks very nice. I'll have to check it out at school on the Dual G5's to get a real good idea, but congrats on your breakthrough.

November 11th, 2004, 07:37 PM	#2121
Wayne Morellini Inner Circle Join Date: May 2003 Location: Australia Posts: 2,762	FPGA Cores OK as I said before, you can go to Opencores (or other) ask people there if anybody would like to be involved. You can ask in the opensource community too, but mainly programmers. I'm sure people would love to do this just for he fun of it. You can also ask on FPGA newsgroups (many pro's) who should also be able to tell you of pro mailing lists and web forums stack full of bored eager FPGA'ers. Opencores may have half the circuit designs you need (where ever they all work perfectly I don't know. Opensource Whatever FPGA core that is created should be opensource so any manufacturer can use it but not own/control/pricing it. Steve has allready offered to use it. You will need to break the design down into theree parts, front end, so different sensor heads/manufacturers be used, processing middle end, and rear end so that different capture interfaces canbe used. With this stratergy new sensors and interfaces canbe supported, it also is not very difficult to do. lots of flexibility that stops us getting tied down. Cost As I say before it takes tiem and money, more than software PC should, How easy I understand some of the programming langauge for FPGA is easy, and people here should be able to do, but you need to learn what your doing and have knowledge for circuit board design and interfacing in a high speed environment (that introduces niose factors that make design tricky). But my "guess" is that you can mostly black box the FPGA in isolation from real hardware and get somebody else to do the interfacing, aslong as you are sure of what data and format is going to come in and what data is needed to come out. My guess is that it will take upto 6 months to learn all this. A few people here indicated they would do it and started a thread (look it up). Manufacture The problem you then face is manufacture, unless somebody else pickup the tab (like SI) low volume costs are much higher than high volume costs, so you might find board is as expensive as PC system until sales hit at least 1-10K. But I'm not being completely negative, I think it canbe done. The www.untratechnology.com forth processor chips section has links o a number of projects. Some people in this level of the forth community tend to be very good programmers and like community initiatives also. Cheaper more flexible alternative/hybrid I would suggest that it maybe better to put a small processor in the core fort the control functions etc (or get a processor with a core, they are out there but I don't know if they have big enough core). If you go to ultratechnology forth ciips link above you will find probably thee smallest/fastest cores available (I'm talking much less than 16K gates). Once you go down this path and also use functions from open compression cores on the web you can make parrallel arrays of processors (preferably in sequence, as parrallel to external memory is way difficult for a FPGA or in Silicon). Now you can reprogram any time, any thign you want, just like a PC (except you willhave to get Forth programmers to do it, because it is queit a leap in logic from C or even assembler). Still maybe you can get some cheap embedded processor to do it (AMD MIPS or Arm) and mostly scap the FPGA (though there might be ARM/MIP with FPGA). Still low volume cost still comes into it, but I'm sure people will start doing Windows devices on 2 Ghz Arms sometime, or embedded controller, that could be bought bulk OEM (rather than consumer) discount, or just buy the design and get it network (contract manufacture out) manufactured like Nintendo manufactured. Still make you own and network manufacture it. Or just say to camera/sensor manufacturers "here, opensource", or how muchh for oem and we add the baord, or whatever, speculative. Others You will probably find that somebody is allready down this path, but you face the problem that A PC is extremely flexible and code modifiable compared to FPGA. Also what codec and software you can support is more flexible on a PC (as I said elsewhere) so you could even boot the PC cqamera up as an editor and add monitor and drive and pheriphals by VGA and One usb port on motherboard for portable feild work after shooting.

November 11th, 2004, 08:14 PM	#2122
Wayne Morellini Inner Circle Join Date: May 2003 Location: Australia Posts: 2,762	Rolling shutter and mechanical shutter and others <<<-- Originally posted by Steve Nordhauser : .. Wayne: The Altasens can read out at up to 60fps for a full frame, letting you drop every other frame at 24 or 30fps. It is still rolling shutter.-->>> Steve, I was just quoting what I read that the actual read out time is many times faster than he frame rate without dropping frames, which is how they deal with rollign shutter and many times superior to what canbe done with present rollign shutter cameras. David, Thanks for that Rob I forgot not only reducing gain, but also clock speed as that might has something to do with it. I veiw it as feed of oversatutared pixels (which also works on satuation inside clock periode, faster clock might make it more leaky, or less, depending on the design problerm causing it). Steve? Mechanical shutter The problem is that the rolling shutter chips tend to readout in parralel with acquiring the image, we have tried to reduce this by putting readback back in the blanking part of the frame and speeding up the clock (is that right?). But still you get the readout going accross to much of he frames acquisition time. So the mechanical shuter is going to have to be shut for a big periode of he frame greatly darkenignthe image. With Ibis global shutter, from what I understand, you also lose substantial light (but probably not anywhere near as much), but you start with less fill facor (= lower light) to start with. Next chip please! If they replaced these chips with more Altsens like versions we would be doing well. Optical veiwfinder VS Electronic I too would like a more professional electronic solution, no light loss and an objective version of what the sensor can see (pluss I can replay it and decide if the scene needs another take and the old take needs to be kept or not). About Vance and Jaun I think Vance knows about this project, he has been over in my Homemade Camera design thread before, adn I have posted details on Jauns thread. Maybe Jaun might like to do a free (or probably commercial) FPGA/capture device for us. I'm not all for integrating methords, unless it reduces cost.

November 12th, 2004, 10:39 AM	#2129
David Newman CTO, CineForm Inc. Join Date: Jul 2003 Location: Cardiff-by-the-Sea, California Posts: 8,095	Obin, Yes we've made lots of progress on the codec. But unfortunately limited time and and some technical issues with Camera Link have slowed down a ready to go solution. I would be concerned but the 94MB/s over standard PCI, I found it would drop frames occassionally due to bus saturation. Email me for followup on codec details. __________________ David Newman -- web: www.gopro.com blog: cineform.blogspot.com -- twitter: twitter.com/David_Newman

November 11th, 2004, 02:53 PM	#2116
Rai Orz Regular Crew Join Date: Jun 2004 Location: Germany Posts: 136	<<<-- Originally posted by Barend Onneweer : It's always good to dream (mechanical shutter, optical viewfinder etc.) ... ............. but with all these ideas about FPGA's and other solutions, there's nobody around here that can actually build a camera that way.-->>> Those things i love so... ... Noboy? Will see... <<<--On another note: an optical viewfinder needs a prism to split the beam of light in two beams. Thus only half the amount of light hits the CMOS.-->>> Not at all. Think on mirrow-shutters. No light loss. And you will also need some kind of (external) shutter for the altasens. Thats true. Dont think i love optical viewfinder over all. No, i only say, its one thing that work.

November 11th, 2004, 03:53 PM	#2118
Valeriu Campan Major Player Join Date: Mar 2004 Location: Melbourne, Australia Posts: 223	<<<-- Originally posted by Ronald Biese : Hm Vance has done this on his PAL cam do google for Vance Cam -->>> Vance has used an LCD shutter with a light loss of ~2+ stops. You can see the diagrams to sync the shutter with camera. It would be nice to drag him back into this forum.

November 11th, 2004, 04:54 PM	#2119
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	HUGE BREAKTHROUGH TODAY! we can now capture 8bit 24fps with no dropped frames AND it looks like 10bit 24fps with NO dropped frames on pci 32! FULL 1080p!!!!! more to come later!

November 11th, 2004, 05:11 PM	#2120
Rai Orz Regular Crew Join Date: Jun 2004 Location: Germany Posts: 136	@Marin: Yes, indeed, its what i said: The pdf show a simple example for "direct to disc" solutions. You need nothings between HDD and CPU. No controller, no extra parts, nothing. But for HD Camera the MC RCM3200 in this shematics is to slow. It runs only at 44,2MHz. But you are on the way.

November 11th, 2004, 10:15 PM	#2123
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	I hate this board...can't post anything long at all/.... BTW check this out: www.dv3productions.com/pub/Obin Test3.mov that is HEAVY compression for the internet..the original looks REALLY good....VERY soft and "filmy"

November 11th, 2004, 10:41 PM	#2124
Eric Gorski Regular Crew Join Date: Jun 2004 Location: Western Oregon Posts: 138	holy crap, that looks good... how much did that setup cost you obin? i mean if you weren't getting the camera for free for testing it :) btw.. i just realized something cool.. if you're shooting full letterbox 2.35:1 aspect ratio, the bending of the image you get when panning quickly with the rolling shutter would be less of a problem because there's less vertical image..

November 11th, 2004, 11:04 PM	#2125
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	I need help. WIth our 8bit captures right now I have to WAY under expose to get the range out of the camera I want. The bad thing is that when I shoot like this and bring the gamma up in post the images are falling apart really bad. Lots of banding artifacts. I am doing this in Combustion and Digital Fusion. Both have the same results. Is the nature of 8bit captures or can somthing be done about this that I don't know about? Please if someone knows the answer to this let me know. I am sure that 10/12bit will be much better..but 8bit is AWESOME if the contrast of the scene like the above post is not too high. "bending" or rolling shutter will not be an issue at or above 67mhz..I have run that with the camera in 8bit mode and it is almost gone..as soon as I can get a 64bit FG card I can run at a very high mhz and kill the rolling shutter issue. WOW 1080p! Oh the above image has been resized to 1280x720 so that you guys on the list could see it on your monitors...unless you have a monitor that is above 1024x1280 this mov file should playback just fine..if you have a high res CRT I could encode a full-on 1080p file ;) I should have 1080p 12bit files soon..I will need a faster set of disk drives but I will do some tests at 15-20fps to see how the banding is from an under exposed 12bit capture CPU load is now at 20% in black and white and about 50-60% with color display...we killed the multi-threading of the save. That was the issue with 100% CPU load before Eric the camera costs about $2500 and I have put 3 months of work into this system to get it working with my programmer who I have hired for this project.

November 11th, 2004, 11:40 PM	#2126
Jason Rodriguez Trustee Join Date: Mar 2003 Location: Virginia Beach, VA Posts: 1,095	Yes Obin, what you're describing is the nature of 8-bit LINEAR images. Your good video cameras, Varicam, F900, etc. may record 8-bit images, but they're receiving 10 and 12-bit images off their A/D converters before video processing begins. For our project here we're doing video processing inside of a post app like Combustion, but in order to get the same results that you'll get out of a high-end camera, you'll need 10 to 12-bits to prevent that banding, etc. Hope this helps. BTW, sweet images!! (and no more jerkies!)

November 12th, 2004, 07:49 AM	#2127
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	thank you Jason..that's what I was thinking but not sure...so at 8bit you really can't shoot high contrast stuff at all...hmmm if you do you have to expose for lots of blowout to get an image that will work or shoot the same scene in 10/12bit and under expose a great deal for later gamma work in post I am waiting for the "convert" part of cinelink to be updated with 16bit tiff output The shot of my face was very low-contrast lighting and it worked well. Just expose what you see and it's fine Ok I am looking at 94MB/sec datarate off the 3300rgb @ 12bit/24fps 1080p..David have you made any progress with the realtime CineForm Codec? now would be the time to start working with it in CineLink..let me know what your thoughts are about this..thanks

November 12th, 2004, 10:37 AM	#2128
Aaron Shaw Major Player Join Date: Jul 2004 Posts: 587	Obin, How hard would it be to make your system fully contained and portable?

November 12th, 2004, 10:45 AM	#2130
Hayden Rivers Regular Crew Join Date: May 2004 Posts: 53	[Vader]Impressive.[/Vader] I'm such a geek. Very nice clip. Doesn't play very smoothly on my Athlon 1700+ with 32mb video card or my Athlon 3200+ with 8mb video card (don't ask). Still, it looks very nice. I'll have to check it out at school on the Dual G5's to get a real good idea, but congrats on your breakthrough.