4:4:4 10bit single CMOS HD project

Obin Olson · March 26th, 2005, 09:57 AM

ok..so we are doing well with the bitpacking(thanks Kyle) we have a solid 33fps 1080p 12bit at 88MB/sec..things seem to be on a good track again and we are looking at getting a working test version with display and save today..

Obin Olson · March 26th, 2005, 09:58 AM

...I am keeping my fingers crossed! our overhead for bitpacking and save is only 4-13% cpu..this is good! as we will have leftover CPu for other things...display is 60% cpu...

Obin Olson · March 26th, 2005, 09:59 PM

I got some updates but it's too late to test as I am at hom now..I will go in the morning and see what we have! my programmer says the save and display is working well on his system...now I gotta test it on the Intel mobile board/chip here....

Wayne Morellini · March 27th, 2005, 01:11 AM

<<<-- Originally posted by Obin Olson : ...I am keeping my fingers crossed! our overhead for bitpacking and save is only 4-13% cpu..this is good! as we will have leftover CPU for other things...display is 60% cpu... -->>>

I thought the display was being done by the GPU, if so then display should be close to 4% also. This leaves me to believe, that the onboard GPU you are using does not support all the GPU features you are using, and is emulating the unsupported instructions in software. My experience with Direct X games is that when you switch from the custom made software render to the Direct X only software render, you get great slow downs (I forget but at least half, maybe 4+ times). I think Directx is good when it addresses actually hardware, but poor when it has to emulate it. So maybe similar things are happening here.

The best bet is to have a profiling program that looks to see what features are available in hardware that can't be emulated quicker on the local processors (some hardware is that slow compared to high speed processor with speed left over going to waste). Have dynamic code that uses custom written hi-speed portions to emulate the missing hardware instead of direct X. This may also be undoable with the GPU programming system you have. You might have to re-arrange the code so most of the portions that are compatible are together and separate from most of the emulated bits. One thing I think is happening in direct x is that their dynamic code is not written for speed, and is getting context hits through the different abstraction layers they are using. So the methods used to jump between different code portions cane also greatly slow you down.

If you can get this down to a low percentage then you will have enough to run lossless compression or very quiet, low speed processor on it.

Happy Easter

Radek Svoboda · March 27th, 2005, 07:59 AM

Is the Altasens CMOS avilable already? If no, when will it become available?

Steve Nordhauser · March 28th, 2005, 07:34 AM

Radek,
The short answer is yes. Shipping now. In the SI-1920HD camera. The longer answer is that we are still trying to bridge the gap between a working piece of hardware and a cinematography tool. Please contact me off-list for specific sales information.
Steve

Noah Yuan-Vogel · March 30th, 2005, 09:59 AM

I've been following these homemade HD threads for months and have been trying to decide what my best option is. I've been thinking about the SI-3300 for a while, but I am unsure as to whether it is in my price range with whatever other expenses it will require. I'm someone who would otherwise be buying a prosumer DV or HD camera but cant stand the limitations of the color sampling and compression of DV and HDV formats.
Do cameras (SI-3300 and 1920) from Silicon Imaging have built in GigE in the unit or do they use a cameralink to GigE adapter (is that then added to the price)?
Also, it sounds like there is no tried and true method of capturing video... It would be nice if someone could just condense all the best (and cheapest) options for each required piece of hardware/software necessary for getting an HD image onto HDD (camera, grabber, software).
Sounds like streampix (also expensive) and Xcap have their problems... but is such software adequate for adjusting, previewing, and capturing uncompressed video streams to HDD. Is a special framegrabber required for GigE cameras? or is a standard Gigabit ethernet connection all you need. Also, It is still unclear to me if these cameras can do 24 or 48fps while maintaining 1/48th shutter...

Maybe I just need to look harder for these answers, but if anyone can help me, thanks. I trying to learn as much as i can about this.

Anyone know anything about the Epix "silicon video" packages that have micron cmos sensor cameras bundled with cable, grabber and software? 1280x1024 at 30p over GigE with all components needed for capture at only $995 seems like a great deal.

http://www.epixinc.com/products/sv9m001.htm
http://www.epixinc.com/products/sv9m001.htm

Obin Olson · March 30th, 2005, 07:26 PM

the HuGE issue is SOFTWARE...and that is my focus..and as of now after 6 months we are VERY close but not done....

from my programmer:

I've just complete a new suite of test on a new system and I am
still running into the same problem even though the technology and approach
is totally different. When calling the routine across threads it seems that
there is a pretty big delay involved that ends up being close to saving time
itself. I'm building a fourth test case right now to see if I can find a way
around it as I have still 3-4 different things to try.

can anyone on here HELP with this issue? it seems that call times across threads is our last problem to a working software...we are at the LIMIT of the CPU at this point and MUST increase the performance or we will not have a working system

Obin Olson · March 30th, 2005, 07:35 PM

I can tell you right now that the 1280 images are not going to be a clear as you may want...it's a single cmos camera not 3CMOS system...makes a BIG change in the resolution of your image( after all your looking at RGGB in the space of 4 pixels instead of 4 pixels that EACH show RGB) I would try and go with the 1080P if you can...Kyle on the board has some software that will record the raw images from a GigaBit camera - the 3300rgb...THe 3300rgb is a GREAT camera...if you can capture your data from it!!

Wayne Morellini · March 31st, 2005, 08:25 AM

<<<-- Originally posted by Obin Olson :
can anyone on here HELP with this issue? it seems that call times across threads is our last problem to a working software...we are at the LIMIT of the CPU at this point and MUST increase the performance or we will not have a working system -->>>

Hunt around for a real-time embedded system/kernel for Windows XP (used to be popular for previous versions of Windows). I do not know exactly what you are meaning, but I can guess. Yes, getting a routine to work on another thread should introduce a big latency problem. There is a large latency hit in subroutine calls in modern computers and memory systems (for other readers here, the further you get away from primary cache to storage the longer, of course). When you call these routines through windows/abstraction layers they add a major hit too. I imagine waking up another thread to do something also has an significant hit. Just calculate all the various hits and find the shortest path. Windows is not so good at these things, I don't know if you can really reduce it (apart from a embedded replacement kernel) apart from doing things differently. Also the programming technique you use to cause a thread to wait for something to happen, can waste lots of cycles, but also putting something to sleep and waking it up can take a lot, but there are ways to do it with less latency and less cycles. I am used to thinking of a lot of latency in terms of less than one cycle, all this PC stuff is so primitive. Rob.L should know how to do these things on a PC, it would be good to ask him.

Your CPU overhead should be a lot less then what it is at the moment for view, what is Kyle doing on his system?

Thanks for your continuing efforts.

Rob.S, when are you coming back with yours. He might have some ideas that could help?

Juan M. M. Fiebelkorn · March 31st, 2005, 03:28 PM

I got an easy and foolproof solution.
It is called : LINUX

Wayne Morellini · April 1st, 2005, 02:01 AM

Yes, I think Mac OSX as well, but how do we get this Windows version completed.

I have news that a dual core Mac Mini will be out mid year season, that would be a good basis for the Mac version .

Have a technical update on the technical thread with other new Macs, batteries, JVC HD lower compression camera, interesting stuff:

http://www.dvinfo.net/conf/showthrea...872#post294872

By the way, where is Ronald Biese, he was interested in Linux version.

Obin Olson · April 1st, 2005, 11:03 AM

Juan how hard to convert our software to LINUX? we are still looking at other solutions..I am going to overclock the board a bit and see if that will be enough power

Obin Olson · April 1st, 2005, 11:43 AM

so anyone have some answers to our questions? we are a bit stuck at this point... I understand Linux but that is a last resort as we have everything working but this last mile...

Kyle Granger · April 1st, 2005, 01:20 PM

Hi Obin,
OK, you seem to have a problem. Before jumping to another OS or the Mac or overclocking your board, it may be wise to first determine WHAT the problem is.

It SOUNDS like there is a thread interaction problem (waiting for mutex, or some signal, not sleeping?), but you should first exclude a problem with one thread. Wayne is right about problems with subroutine calls and Windows programming in general, but I believe the problem is PROBABLY much less exotic. Also, if you assume the problem exists in your code, you then have a chance to fix it. These things for me have always turned out to be bugs. And bugs can be fixed.

For what it is worth, my application is extremely multi-threaded, and I have not experienced these problems.

You said earlier that the problem had to do with "When calling the routine across threads", but I don't really know what "across threads" means.

1) Profile each of your threads individually with a canonical test case (1920x1080x24p, or some standard config file) I assume there are three main threads or tasks: Capture, Display, and Writing. Get a rough CPU usage for each. Display can just read the same buffer, 24 times a second. Same with writing.

If the CPU usage is reasonable (say, 3% for Capture, %17 for Display, and 24% for Writing), then there is some interaction problem.

2) Try thread interactions, doing just Capture and Display.
Does that work with the same CPU usage as Capture only PLUS Display only?

3) Try Capture + Writing.

4) Try Display + Writing.

5) Look at how the threads get fed. Are they properly sleeping? If they are waiting for a signal, can you test just by polling and sleeping 1-10 ms?

6) Is one thread running at too high a priority?

If you are using DirectX for the interface to the GPU, you will have to rewrite that for Linux.

March 28th, 2005, 07:34 AM	#2691
Steve Nordhauser Silicon Imaging, Inc. Join Date: May 2004 Location: Troy, NY USA Posts: 325	Radek, The short answer is yes. Shipping now. In the SI-1920HD camera. The longer answer is that we are still trying to bridge the gap between a working piece of hardware and a cinematography tool. Please contact me off-list for specific sales information. Steve __________________ Silicon Imaging, Inc. We see the Light! http://www.siliconimaging.com

March 26th, 2005, 09:57 AM	#2686
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	ok..so we are doing well with the bitpacking(thanks Kyle) we have a solid 33fps 1080p 12bit at 88MB/sec..things seem to be on a good track again and we are looking at getting a working test version with display and save today..

March 26th, 2005, 09:58 AM	#2687
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	...I am keeping my fingers crossed! our overhead for bitpacking and save is only 4-13% cpu..this is good! as we will have leftover CPu for other things...display is 60% cpu...

March 26th, 2005, 09:59 PM	#2688
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	I got some updates but it's too late to test as I am at hom now..I will go in the morning and see what we have! my programmer says the save and display is working well on his system...now I gotta test it on the Intel mobile board/chip here....

March 27th, 2005, 01:11 AM	#2689
Wayne Morellini Inner Circle Join Date: May 2003 Location: Australia Posts: 2,762	<<<-- Originally posted by Obin Olson : ...I am keeping my fingers crossed! our overhead for bitpacking and save is only 4-13% cpu..this is good! as we will have leftover CPU for other things...display is 60% cpu... -->>> I thought the display was being done by the GPU, if so then display should be close to 4% also. This leaves me to believe, that the onboard GPU you are using does not support all the GPU features you are using, and is emulating the unsupported instructions in software. My experience with Direct X games is that when you switch from the custom made software render to the Direct X only software render, you get great slow downs (I forget but at least half, maybe 4+ times). I think Directx is good when it addresses actually hardware, but poor when it has to emulate it. So maybe similar things are happening here. The best bet is to have a profiling program that looks to see what features are available in hardware that can't be emulated quicker on the local processors (some hardware is that slow compared to high speed processor with speed left over going to waste). Have dynamic code that uses custom written hi-speed portions to emulate the missing hardware instead of direct X. This may also be undoable with the GPU programming system you have. You might have to re-arrange the code so most of the portions that are compatible are together and separate from most of the emulated bits. One thing I think is happening in direct x is that their dynamic code is not written for speed, and is getting context hits through the different abstraction layers they are using. So the methods used to jump between different code portions cane also greatly slow you down. If you can get this down to a low percentage then you will have enough to run lossless compression or very quiet, low speed processor on it. Happy Easter

March 27th, 2005, 07:59 AM	#2690
Radek Svoboda Major Player Join Date: Mar 2005 Location: Prague, Czech Republic Posts: 500	Is the Altasens CMOS avilable already? If no, when will it become available?

March 30th, 2005, 09:59 AM	#2692
Noah Yuan-Vogel Major Player Join Date: Dec 2004 Location: New York City Posts: 613	I've been following these homemade HD threads for months and have been trying to decide what my best option is. I've been thinking about the SI-3300 for a while, but I am unsure as to whether it is in my price range with whatever other expenses it will require. I'm someone who would otherwise be buying a prosumer DV or HD camera but cant stand the limitations of the color sampling and compression of DV and HDV formats. Do cameras (SI-3300 and 1920) from Silicon Imaging have built in GigE in the unit or do they use a cameralink to GigE adapter (is that then added to the price)? Also, it sounds like there is no tried and true method of capturing video... It would be nice if someone could just condense all the best (and cheapest) options for each required piece of hardware/software necessary for getting an HD image onto HDD (camera, grabber, software). Sounds like streampix (also expensive) and Xcap have their problems... but is such software adequate for adjusting, previewing, and capturing uncompressed video streams to HDD. Is a special framegrabber required for GigE cameras? or is a standard Gigabit ethernet connection all you need. Also, It is still unclear to me if these cameras can do 24 or 48fps while maintaining 1/48th shutter... Maybe I just need to look harder for these answers, but if anyone can help me, thanks. I trying to learn as much as i can about this. Anyone know anything about the Epix "silicon video" packages that have micron cmos sensor cameras bundled with cable, grabber and software? 1280x1024 at 30p over GigE with all components needed for capture at only $995 seems like a great deal. http://www.epixinc.com/products/sv9m001.htm http://www.epixinc.com/products/sv9m001.htm

March 30th, 2005, 07:26 PM	#2693
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	the HuGE issue is SOFTWARE...and that is my focus..and as of now after 6 months we are VERY close but not done.... from my programmer: I've just complete a new suite of test on a new system and I am still running into the same problem even though the technology and approach is totally different. When calling the routine across threads it seems that there is a pretty big delay involved that ends up being close to saving time itself. I'm building a fourth test case right now to see if I can find a way around it as I have still 3-4 different things to try. can anyone on here HELP with this issue? it seems that call times across threads is our last problem to a working software...we are at the LIMIT of the CPU at this point and MUST increase the performance or we will not have a working system

March 30th, 2005, 07:35 PM	#2694
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	I can tell you right now that the 1280 images are not going to be a clear as you may want...it's a single cmos camera not 3CMOS system...makes a BIG change in the resolution of your image( after all your looking at RGGB in the space of 4 pixels instead of 4 pixels that EACH show RGB) I would try and go with the 1080P if you can...Kyle on the board has some software that will record the raw images from a GigaBit camera - the 3300rgb...THe 3300rgb is a GREAT camera...if you can capture your data from it!!

March 31st, 2005, 08:25 AM	#2695
Wayne Morellini Inner Circle Join Date: May 2003 Location: Australia Posts: 2,762	<<<-- Originally posted by Obin Olson : can anyone on here HELP with this issue? it seems that call times across threads is our last problem to a working software...we are at the LIMIT of the CPU at this point and MUST increase the performance or we will not have a working system -->>> Hunt around for a real-time embedded system/kernel for Windows XP (used to be popular for previous versions of Windows). I do not know exactly what you are meaning, but I can guess. Yes, getting a routine to work on another thread should introduce a big latency problem. There is a large latency hit in subroutine calls in modern computers and memory systems (for other readers here, the further you get away from primary cache to storage the longer, of course). When you call these routines through windows/abstraction layers they add a major hit too. I imagine waking up another thread to do something also has an significant hit. Just calculate all the various hits and find the shortest path. Windows is not so good at these things, I don't know if you can really reduce it (apart from a embedded replacement kernel) apart from doing things differently. Also the programming technique you use to cause a thread to wait for something to happen, can waste lots of cycles, but also putting something to sleep and waking it up can take a lot, but there are ways to do it with less latency and less cycles. I am used to thinking of a lot of latency in terms of less than one cycle, all this PC stuff is so primitive. Rob.L should know how to do these things on a PC, it would be good to ask him. Your CPU overhead should be a lot less then what it is at the moment for view, what is Kyle doing on his system? Thanks for your continuing efforts. Rob.S, when are you coming back with yours. He might have some ideas that could help?

March 31st, 2005, 03:28 PM	#2696
Juan M. M. Fiebelkorn Major Player Join Date: Jun 2004 Location: Buenos Aires , Argentina Posts: 444	I got an easy and foolproof solution. It is called : LINUX

April 1st, 2005, 02:01 AM	#2697
Wayne Morellini Inner Circle Join Date: May 2003 Location: Australia Posts: 2,762	Yes, I think Mac OSX as well, but how do we get this Windows version completed. I have news that a dual core Mac Mini will be out mid year season, that would be a good basis for the Mac version . Have a technical update on the technical thread with other new Macs, batteries, JVC HD lower compression camera, interesting stuff: http://www.dvinfo.net/conf/showthrea...872#post294872 By the way, where is Ronald Biese, he was interested in Linux version.

April 1st, 2005, 11:03 AM	#2698
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	Juan how hard to convert our software to LINUX? we are still looking at other solutions..I am going to overclock the board a bit and see if that will be enough power

April 1st, 2005, 11:43 AM	#2699
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	so anyone have some answers to our questions? we are a bit stuck at this point... I understand Linux but that is a last resort as we have everything working but this last mile...

April 1st, 2005, 01:20 PM	#2700
Kyle Granger Regular Crew Join Date: Feb 2005 Location: . Posts: 52	Hi Obin, OK, you seem to have a problem. Before jumping to another OS or the Mac or overclocking your board, it may be wise to first determine WHAT the problem is. It SOUNDS like there is a thread interaction problem (waiting for mutex, or some signal, not sleeping?), but you should first exclude a problem with one thread. Wayne is right about problems with subroutine calls and Windows programming in general, but I believe the problem is PROBABLY much less exotic. Also, if you assume the problem exists in your code, you then have a chance to fix it. These things for me have always turned out to be bugs. And bugs can be fixed. For what it is worth, my application is extremely multi-threaded, and I have not experienced these problems. You said earlier that the problem had to do with "When calling the routine across threads", but I don't really know what "across threads" means. 1) Profile each of your threads individually with a canonical test case (1920x1080x24p, or some standard config file) I assume there are three main threads or tasks: Capture, Display, and Writing. Get a rough CPU usage for each. Display can just read the same buffer, 24 times a second. Same with writing. If the CPU usage is reasonable (say, 3% for Capture, %17 for Display, and 24% for Writing), then there is some interaction problem. 2) Try thread interactions, doing just Capture and Display. Does that work with the same CPU usage as Capture only PLUS Display only? 3) Try Capture + Writing. 4) Try Display + Writing. 5) Look at how the threads get fed. Are they properly sleeping? If they are waiting for a signal, can you test just by polling and sleeping 1-10 ms? 6) Is one thread running at too high a priority? If you are using DirectX for the interface to the GPU, you will have to rewrite that for Linux.