My little DIY camera project

Noah Yuan-Vogel · September 13th, 2006, 01:24 AM

Sorry its been such a long time, I've been working (gotta make money to fund these camera projects and such). No progress really, havent gotten around to doing much more latey. Hopefully I will get back into it when i have more time.

as for 48fps, you can do the calculations. 1280x720 8bit is 921600 bytes/frame so you need 44236800Hz (44.2MHz) to pull that off. I cant recall being able to break an effective pixelrate of much more than 37MHz. The camera technically supports up to 48MHz but after some data overhead (minimum blanking, other reserved bytes, i think) I think youd be lucky to get more than 40MHz effective. Some of this limitation might be related to my USB controller which is not considered by Sumix to be recommended. They only like Intel USB and for good reason. My VIA USB isnt so bad, but it has its limits. To sum up, I wouldnt count on 720p@48 with this camera head, but fortunately vertical blanking ought to get you a real close equivalent. Also, in my experience, even if you can get 48fps in the camera, capturing it to HDD is pretty tough. I'd say use 480 vertical blanking and 24fps, it should reduce rolling shutter artifacting by 33%. (running 48fps would reduce artifacts by 50%, but im not sure its possible, so 33% isnt bad)

As for M72, I havent heard a ton about it, but there is probably a reason for that. I'd say if you just want 720 see if you can get an M71 in color (they used to have them, but looks like the website only lists mono). They seem to have some smear issues but im not sure smear is so bad, especially if you have control over what you are shooting (it reminds me of lens flare), and they have significantly better light sensitivity and probably lower noise.

Rob Scott · September 13th, 2006, 05:43 AM

Quote:

Originally Posted by Noah Yuan-Vogel

as for 48fps ... 1280x720 8bit is 921600 bytes/frame so you need 44236800Hz (44.2MHz) to pull that off. I cant recall being able to break an effective pixelrate of much more than 37MHz.

Yeah ... I'm starting to think that your idea of using binning to get 1024x576 is going to work out the best with this camera. At that resolution you can get better signal-to-noise, 10 bits, 48 fps (hopefully) and use the entire imaging area. Resampling up to 1280x720 would soften the image a little, but given the other positives it seems like a decent tradeoff.

Edit: Apparently the Sumix doesn't do binning, just decimation. So you'd get aliasing and the signal-to-noise ratio would be unchanged. Nuts. Still, it might be the way to go in order to reduce the rolling shutter issues even further.

Noah Yuan-Vogel · September 13th, 2006, 08:38 AM

maybe take a look at the sensor documentation. I'm pretty sure it supports binning, but I think Sumix's API only supports it through register programming. And then I think it requires some overhead in the camera, which drops the effective pixelrate. Yeah, it's hard to know exactly whats best. USB isnt such a bad interface, but 48MHz is a pretty limiting pixelrate when trying to do any kind of HD while also minimizing rolling shutter. Maybe mechanical shutter plus global reset commands is the only way this camera will become acceptable past SD. 1280x720 and my anamorphic 960x1080 with vertical blanking arent so bad, but the M73 isnt made for 720p and the remaining rolling shutter and lack of binning are still problematic. Not to mention questionable color reproduction. I have pulled out the camera a few times since I moved to NYC, but the lighting in my new room looks horrible even with the IR/UV cut filter. Somehow it's still getting a lot of IR.

Rob Scott · September 13th, 2006, 08:54 AM

Quote:

Originally Posted by Noah Yuan-Vogel

maybe take a look at the sensor documentation. I'm pretty sure it supports binning, but I think Sumix's API only supports it through register programming.

That's definitely worth looking at; do you have any information on the registers? Are those Sumix-specific or do they go directly to the chip?

Quote:

Maybe mechanical shutter plus global reset commands is the only way this camera will become acceptable past SD ... the remaining rolling shutter and lack of binning are still problematic. Not to mention questionable color reproduction.

It's certainly not going to be perfect, but for the price it might still be a very acceptable alternative. Thanks for the info!

Noah Yuan-Vogel · September 13th, 2006, 12:52 PM

The registers are built into the sensor and very well documented if you download the data sheet for the sensor from www.micron.com . The way you access these registers is a Sumix thing though. And once you start using registers, you have to be careful about what Sumix functions you call since half of them will reset the registers or screw up because they dont expect registers to be changed manually.

Rob Scott · September 13th, 2006, 02:48 PM

Quote:

Originally Posted by Noah Yuan-Vogel

The registers are built into the sensor and very well documented ...

So, with a little care it should be possible to configure it properly for binning. Definitely promising. Thanks for the info!

Edit: According to my off-the-cuff calculations*, at 1024x576x10bit, 48fps would require ~27Mhz bandwidth and ~34 MB/s, assuming that Sumix packs the 10-bit data. By boosting the clock speed higher, this should allow for reducing rolling shutter significantly.

*Rob Lohman, thanks for the spreadsheet you posted earlier.

Noah Yuan-Vogel · September 13th, 2006, 10:01 PM

10bit is pretty much out on this camera since sumix doesnt pack 10bit. Each 10bit pixel is stored in 2 bytes. It does, however, have full control over the LUT, so 8bit log works as long as you program it. Youd be lucky to get this camera to work well for SD in 10bit mode. I forgot about trying to use raw 10bit on this camera a while ago. Its no fun trying to work with the linear image anyway, its super high contrast with no correction. I dont know much about how the 10bit mode in the camera works except about it using 16bits and I dont trust it anyway. Im not even sure you get a 10bit image in 10bit mode... looks the same to me as the 8bit, even when pushing the lighting to get color stepping...

Rob Scott · September 14th, 2006, 06:57 AM

Quote:

Originally Posted by Noah Yuan-Vogel

10bit is pretty much out on this camera since sumix doesnt pack 10bit. Each 10bit pixel is stored in 2 bytes. It does, however, have full control over the LUT...

I was afraid of that. Well, with the LUT it should still be possible to get good images with a bit more care during shooting.

Brendon Whateley · September 17th, 2006, 04:56 PM

Quote:

Originally Posted by Noah Yuan-Vogel

As for my progress, I'm finally back in the US and getting back to work on the software. So far my tests are good, less cpu utilization with my faster computer i left here, looks like i may need to change my disk writing method again since ive been using a little test program to try different methods of writing data and it seems .NET filestream can get 52MBps on a linear write as long as i dont give it more than 256KB to write at a time (weird, huh? maybe a disk cluster size thing... it benchmarks only about 30MBps running one 1MB write operation and 52MBps with 4 separate calls to write 256KB).

This is what you would expect. Remember that the drive interface is faster than the physical data transfer to the platter. For sustained writes, you cannot exceed the physical data transfer speed. You need to look at the drive spec to see what that is. If you do anything other than sequential writes, then the performance will be impacted quite negatively, since seeks are very much slower. The effects you are seeing are due to the drive (and possibly OS buffering). To get a good idea of speed, you want to write a very large amount of data, say 100MB.

Multi-threaded writing can be good, if it is to decouple the data source from the writing. Not if you have multiple threads writing to the same disk. In the first case, you are trying to make sure the drive is 100% busy. The second moves the disk head around. which is bad. Also, for best results, the Data disk should not be the same as the OS.

Hope that helps!

Noah Yuan-Vogel · September 17th, 2006, 06:43 PM

I'm not sure how you mean this is what you would expect. You would expect 4000 256KB writes to be faster than 2000 512KB writes? Sorry if my explanation was confusing, but I thought I mentioned that I was dealing with 500-10000MB transfers, the 256KB and 512KB data just relates to the way those sustained transfers are handled in programming.

That particular drive is generally capable of ~60MBps sustained on outer cylinders and ~40MBps on inner cylinders at least based on my benchmarks and tomshardware.com, so this is why im glad I have been able to hit >52MBps in practice using the .net filestream. It is also partitioned to keep all data on the faster section of the platters, and it is of course a dedicated, empty disk. I usually check for fragments after a long write benchmark or video recording, and things tend to be contiguous so I doubt disk seeks are where the bottleneck is.

Yeah I am mostly just relying on multithreaded writes for the sake of not dropping frames when the system is waiting for a write to kick off. I also seem to have better performance with multithreaded writes but also much trouble keeping the writes executing in the right order.. It's too bad I havent had much luck using long buffered writes. I wonder if compressing the data might help reduce the bottlenecks im seeing (maybe when i get a faster system).

Brendon Whateley · September 20th, 2006, 03:43 PM

Sorry, I must have misunderstood the question! Since I've spent quite a bit of time designing and building high performance I/O systems, I can still say that surprising results are not that surprising. You often find that between the compiler, the language libraries, Operating system, controller and disk, you can usually find a lot of strange behaviors. You have to really dig into the OS and language structures to get a good handle on how they may behave when trying to push the performance to the max.

If you are getting dropped frames, do you know where they are being dropped? Typically you have an in memory queue structure between the data producer and the thread that is writing to the disk. Having one thread that ONLY grabs data off of a queue and then does a blocking write tends to be the easiest to keep things in the correct order and keep the disk I/O maximized. You can then put data on the queue as quick as you need, the queue will make sure you don't drop frames until you run out of memory...

Anmol Mishra · August 18th, 2007, 11:17 PM

Can anyone recommend a firewire 800 version of the Sumix M73 ? I'm trying to get a firewire interface for it. Thanks!

August 18th, 2007, 11:17 PM	#132
Anmol Mishra Major Player Join Date: Jun 2007 Location: Sydney Australia Posts: 401	Firewire 800 version of the Sumix M73 Can anyone recommend a firewire 800 version of the Sumix M73 ? I'm trying to get a firewire interface for it. Thanks!

September 13th, 2006, 01:24 AM	#121
Noah Yuan-Vogel Major Player Join Date: Dec 2004 Location: New York City Posts: 613	Sorry its been such a long time, I've been working (gotta make money to fund these camera projects and such). No progress really, havent gotten around to doing much more latey. Hopefully I will get back into it when i have more time. as for 48fps, you can do the calculations. 1280x720 8bit is 921600 bytes/frame so you need 44236800Hz (44.2MHz) to pull that off. I cant recall being able to break an effective pixelrate of much more than 37MHz. The camera technically supports up to 48MHz but after some data overhead (minimum blanking, other reserved bytes, i think) I think youd be lucky to get more than 40MHz effective. Some of this limitation might be related to my USB controller which is not considered by Sumix to be recommended. They only like Intel USB and for good reason. My VIA USB isnt so bad, but it has its limits. To sum up, I wouldnt count on 720p@48 with this camera head, but fortunately vertical blanking ought to get you a real close equivalent. Also, in my experience, even if you can get 48fps in the camera, capturing it to HDD is pretty tough. I'd say use 480 vertical blanking and 24fps, it should reduce rolling shutter artifacting by 33%. (running 48fps would reduce artifacts by 50%, but im not sure its possible, so 33% isnt bad) As for M72, I havent heard a ton about it, but there is probably a reason for that. I'd say if you just want 720 see if you can get an M71 in color (they used to have them, but looks like the website only lists mono). They seem to have some smear issues but im not sure smear is so bad, especially if you have control over what you are shooting (it reminds me of lens flare), and they have significantly better light sensitivity and probably lower noise.

September 13th, 2006, 08:38 AM	#123
Noah Yuan-Vogel Major Player Join Date: Dec 2004 Location: New York City Posts: 613	maybe take a look at the sensor documentation. I'm pretty sure it supports binning, but I think Sumix's API only supports it through register programming. And then I think it requires some overhead in the camera, which drops the effective pixelrate. Yeah, it's hard to know exactly whats best. USB isnt such a bad interface, but 48MHz is a pretty limiting pixelrate when trying to do any kind of HD while also minimizing rolling shutter. Maybe mechanical shutter plus global reset commands is the only way this camera will become acceptable past SD. 1280x720 and my anamorphic 960x1080 with vertical blanking arent so bad, but the M73 isnt made for 720p and the remaining rolling shutter and lack of binning are still problematic. Not to mention questionable color reproduction. I have pulled out the camera a few times since I moved to NYC, but the lighting in my new room looks horrible even with the IR/UV cut filter. Somehow it's still getting a lot of IR.

September 13th, 2006, 12:52 PM	#125
Noah Yuan-Vogel Major Player Join Date: Dec 2004 Location: New York City Posts: 613	The registers are built into the sensor and very well documented if you download the data sheet for the sensor from www.micron.com . The way you access these registers is a Sumix thing though. And once you start using registers, you have to be careful about what Sumix functions you call since half of them will reset the registers or screw up because they dont expect registers to be changed manually.

September 13th, 2006, 10:01 PM	#127
Noah Yuan-Vogel Major Player Join Date: Dec 2004 Location: New York City Posts: 613	10bit is pretty much out on this camera since sumix doesnt pack 10bit. Each 10bit pixel is stored in 2 bytes. It does, however, have full control over the LUT, so 8bit log works as long as you program it. Youd be lucky to get this camera to work well for SD in 10bit mode. I forgot about trying to use raw 10bit on this camera a while ago. Its no fun trying to work with the linear image anyway, its super high contrast with no correction. I dont know much about how the 10bit mode in the camera works except about it using 16bits and I dont trust it anyway. Im not even sure you get a 10bit image in 10bit mode... looks the same to me as the 8bit, even when pushing the lighting to get color stepping...

September 17th, 2006, 06:43 PM	#130
Noah Yuan-Vogel Major Player Join Date: Dec 2004 Location: New York City Posts: 613	I'm not sure how you mean this is what you would expect. You would expect 4000 256KB writes to be faster than 2000 512KB writes? Sorry if my explanation was confusing, but I thought I mentioned that I was dealing with 500-10000MB transfers, the 256KB and 512KB data just relates to the way those sustained transfers are handled in programming. That particular drive is generally capable of ~60MBps sustained on outer cylinders and ~40MBps on inner cylinders at least based on my benchmarks and tomshardware.com, so this is why im glad I have been able to hit >52MBps in practice using the .net filestream. It is also partitioned to keep all data on the faster section of the platters, and it is of course a dedicated, empty disk. I usually check for fragments after a long write benchmark or video recording, and things tend to be contiguous so I doubt disk seeks are where the bottleneck is. Yeah I am mostly just relying on multithreaded writes for the sake of not dropping frames when the system is waiting for a write to kick off. I also seem to have better performance with multithreaded writes but also much trouble keeping the writes executing in the right order.. It's too bad I havent had much luck using long buffered writes. I wonder if compressing the data might help reduce the bottlenecks im seeing (maybe when i get a faster system).

September 20th, 2006, 03:43 PM	#131
Brendon Whateley Regular Crew Join Date: Jul 2006 Location: Fremont, CA Posts: 59	Sorry, I must have misunderstood the question! Since I've spent quite a bit of time designing and building high performance I/O systems, I can still say that surprising results are not that surprising. You often find that between the compiler, the language libraries, Operating system, controller and disk, you can usually find a lot of strange behaviors. You have to really dig into the OS and language structures to get a good handle on how they may behave when trying to push the performance to the max. If you are getting dropped frames, do you know where they are being dropped? Typically you have an in memory queue structure between the data producer and the thread that is writing to the disk. Having one thread that ONLY grabs data off of a queue and then does a blocking write tends to be the easiest to keep things in the correct order and keep the disk I/O maximized. You can then put data on the queue as quick as you need, the queue will make sure you don't drop frames until you run out of memory...