Yeah that is definitely true. I have thought about that, and the sensor manual says that is supports binning (reading extra pixel quads and averaging them), but I have not gone that far into it and do not know what impact that would have on the pixelrate. I would imagine it is less aliasing than i have seen running at full 2x skipping (im pretty sure sumix's standard binning is just skipping).

Actually I have settled on a mobo and proc, i plan to go with the core duo and the ibase MB899F motherboard. Jason, any guess on whether that is compatible with the silicon imaging camera? The question is just the gigabit ethernet controller i suppose.

My suggestion if you want a mini-itx mobo that's ideal for the SI camera would be the Aopen i945GTt-VFA.

Now for our camera, if you can wait for Merom, I'd do that (overhead is always a good thing) . . . of course you could always just add a Merom later.

DDR2-667 is also must (compared to DDR2-533), and it's compatable with this board.

http://global.aopen.com.tw/products/mb/i945GTt-VFA.htm

Thanks for the suggestion, does that support HDTV component out, out of the box? I just liked that the other one i was looking at could support DVI, HDTV, VGA, TV and has full size ram modules (which may be more available and better value). Is the aopen board what SI is using for their setup?

The Aopen does support HD component analog output out-of-the-box.

I would highly recommend it.

Noah -- I've ordered an M73. Hoping to familiarise myself with the software having the camera connected to my dektop, later will buy a laptop. I'm looking around for an Angenieux 15mm lens, but have a Pentax K/C-mount adapter to use a 28mm SLR lens in the meantime to start some tests. I'll keep you posted on how I get on. I believe the software is capable of recording a still image sequence, and I would like to experiment with this at 25 pictures a second (at whatever frame size works within the various bandwidth limitations).
John.

Pity more motherboards do not have component in as well, apart from some of the VIA ones, just a few cents/dollars motherboard costs, not $1K. Pirates have really blown it for the rest of us, and played us right into the hands of manufacturers.

John - Wow good luck with your project, I'll be very interested to hear how things go for you. It sounds like you are just using their camera application? I would imagine the 25 still frames per second is a pretty big limitation since I dont think it will store those in RAM like with the RAM capture option, and even with lossless TIF (which they now support), youre looking at some pretty high data rates since BMP and tiff i believe are RGB so that triples the data right there, and the cpu requirements for real-time lapacian debayer seem pretty high especially for more than SD res. But i suppose a nice dual core cpu and a fast HDD might be all you need.

Wayne - Actually I've been surprised to see component HD out on quite a few motherboards especially with the new integrated intel and nvidia 6150 gpus. I'm not sure I get what you mean about pirates. Do you mean people pirating movies? I would guess it probably has more to do with demand.


As for my progress, I'm finally back in the US and getting back to work on the software. So far my tests are good, less cpu utilization with my faster computer i left here, looks like i may need to change my disk writing method again since ive been using a little test program to try different methods of writing data and it seems .NET filestream can get 52MBps on a linear write as long as i dont give it more than 256KB to write at a time (weird, huh? maybe a disk cluster size thing... it benchmarks only about 30MBps running one 1MB write operation and 52MBps with 4 separate calls to write 256KB). I think I will play with non-threaded writing more and hope that will give me good results with this new revelation. I wasn't too happy with my many-threaded disk writing implementation anyway, it was great at avoiding dropping frames on my slow system but when pushed too much no good at writing them in the right order and I dont think im programmer enough to fix it right now :P

I'm not sure I get what you mean about pirates. Do you mean people pirating movies?

I meant component in for recording from cameras. I wish they would all go to HDMI now, and HDMI in on the motherboard. 10.2Gb/s, HDMI 1.3 (upto 16bit colour and mega resolutions) would make an excellent camera head interface, it has the digital structure that could be made to handle head commands. I would like them to replace short haul network, USB/Firewire, hard drive and VGA with HDMI, it would make an excellent data interface (please note, the extra cost limitation for slow cheap devices can be over come by having the interface conform to lower data rate USB/2.0/go version (automated requiring little CPU interaction, like firewire) standard for these devices, and repackaging the data stream at the hub. What a beautiful world that would be, don't matter how many ports you have on your computer you can simply plug anything into anywhere and it will instantly work. It could even replace many forms of PCI, and even the external PCI express desktop bus (what happened with that).

Pity more motherboards do not have component in as well, apart from some of the VIA ones

Noah –- you’re right: single and sequence still image recording to HDD (using the supplied software) is a simultaneous conversion to RGB, and therefore very heavy on data rate. Since my desktop is faster than my expected laptop (along with a faster 7,200 rpm drive), I thought I’d experiment with this though. I mentioned to the Sumix support team that it would be highly useful to be able to record RAW sequence frames (to bring the data rate down to one third), and they said they would consider this possibility for a future upgrade. Until then, for me normal laptop operation would involve RAM-recording the Bayer video file. Is there any possibility for the user to change the still image format to Bayer? I’m always interested to hear about your extraordinary modifications –- any conclusions yet on the electronic anamorphic image quality?
John.

That is incredible, I thought there was always a save to bayer option. They should have included capture to disk with customisable interface (that allows control through the PC keyboard and any external controls connected to the PC) in the default package by now. It would be good to suggest this to them, as they have sold a number of cameras to video people that have not been useful because of the lack of these things.

Noah -- I made a few RAM recording tests on my desktop computer which usefully has 2 GB of RAM. I made an avi version of the smx file with SMXView, but this avi can not be imported into After Effects. The 16:9 frame size was 1280 x 720 at 25 fps. Is this a case of getting the Sumix codec into After Effects so that the Sumix avi will work? How would this be done? Obviously it is important to be able to import the avi footage into other software: any advice would be gratefully accepted. It's a shame Bayer tifs can't be recorded to RAM and converted to RGB (everything can read an RGB tif!).
Regards,
John.

Hmm I vaguely recall having some issue with the way sumix implements containging uncompressed RGB video in the AVI wrapper. I never went to far checking it out, but chances are you might be able to open the avi in virtualdub and just stream copy it into a new file maybe thatll fix whatever went wrong with it (maybe avi header is not done right or something). I guess i can test a few things out for you, see what the deal is with their avi sequences. I'll tell you if i find anything.

Many thanks Noah; greatly appreciated. I seem to remember a couple of years ago reading that Ben Syverson (who was then using a Sumix 150C camera) converted the Sumix Bayer file using his own software called linBayer, which was a 16-bit After Effects plug-in. Does anyone have a recent contact for Ben? Is linBayer still available anywhere?
John.

Noah -- I tried VirtualDub and this played the Sumix avi! For your information (makes no sense to me of course!) the two VirtualDub warnings given for the clip were: "* AVI: index not found or damaged -- reconstructing via file scan. * AVI: invalid chunk detected at 2764810056. Enable aggressive recovery mode." From VirtualDub I was able to make an avi copy which imported into After Effects without problem, plays in Windows Media Player, etc. Thanks for the tip.

One of the things I wanted to do in After Effects was export a few particular tif frames to check image quality, and I was able to do this directly from VirtualDub as well -- bmp or tga (targa) files are the still image choices: I chose targa, and in Photoshop I was able to look at the frames as planned. Many thanks.
John.

How would this be done? Obviously it is important to be able to import the avi footage into other software: any advice would be gratefully accepted. It's a shame Bayer tifs can't be recorded to RAM and converted to RGB (everything can read an RGB tif!).
Regards,
John.

John, you could also try reading through that Obin thread, these sorts of things were discussed there.

Wayne -- do you know what Ben Syverson is doing now? I was wondering about his linBayer plug-in. Google seems no good on it...
John.

Don't know, people come and go at a crazy rate. Look at his profile and send him message. If you can't send Chris a message and ask him to ask Ben to contact you.

Have you looked at that thread, he had a website with his software, there might be upto date contacts there. If it is no longer there, try google, and click on the cache option. If that is not there go to archive.org (thewayback engine) and search for the sites history there and see if you can get an email address. He's not to bad, he might like to look into this, he has a Sumix camera too, so he would probably be a good person to contact. Actually, I think, the last time I saw him, was on my technical thread.

If you are out of luck, drop a message in a thread he should still be subscribed to, or in other forums that he would still check.

Noah -- I'm writing up my initial experiences of the camera -- ok if I post it here, to keep the M73 stuff together (as Oscar is doing on Forrest's Elphel 333 thread) ? Since RAM recording to a laptop using the supplied camera application is different to your re-coded/self-build computer project, perhaps you'd prefer to keep this thread focused on that?
John.

You may post what you like, but it might be worthwhile to start your own thread since it might get confusing otherwise. But I certainly dont mind either way.
I probably wont be able to post too often since I now have a regular job and unfortunately dont have too much time to work on the camera (hopefully i will find time). When do have time I seem to end up getting distracted thinking about building cool mini custom computers to run the camera off of (which is obviously not a priority for the camera at the moment, since there is a lot of programming to be done), but I sort of need a new computer anyway. But now that I am done moving and settling into my new job/surroundings, I might be able to start messing with my M73 again.

Noah, any progress recently? I've recently started my project back up and I'm looking seriously at supporting the Sumix M72/M73 cameras, mainly because of the low cost compared to the SI cameras.

Have you been able to get anywhere near 48 fps? I'm hoping it might be possible to get 1280x720 @ 48 fps (8 or 10 bits) to cut down on the rolling shutter issues.

Also, do you know of any reasons to avoid the M72? For 1280x720 it looks like it might be a better choice. Just curious.

Sorry its been such a long time, I've been working (gotta make money to fund these camera projects and such). No progress really, havent gotten around to doing much more latey. Hopefully I will get back into it when i have more time.

as for 48fps, you can do the calculations. 1280x720 8bit is 921600 bytes/frame so you need 44236800Hz (44.2MHz) to pull that off. I cant recall being able to break an effective pixelrate of much more than 37MHz. The camera technically supports up to 48MHz but after some data overhead (minimum blanking, other reserved bytes, i think) I think youd be lucky to get more than 40MHz effective. Some of this limitation might be related to my USB controller which is not considered by Sumix to be recommended. They only like Intel USB and for good reason. My VIA USB isnt so bad, but it has its limits. To sum up, I wouldnt count on 720p@48 with this camera head, but fortunately vertical blanking ought to get you a real close equivalent. Also, in my experience, even if you can get 48fps in the camera, capturing it to HDD is pretty tough. I'd say use 480 vertical blanking and 24fps, it should reduce rolling shutter artifacting by 33%. (running 48fps would reduce artifacts by 50%, but im not sure its possible, so 33% isnt bad)

As for M72, I havent heard a ton about it, but there is probably a reason for that. I'd say if you just want 720 see if you can get an M71 in color (they used to have them, but looks like the website only lists mono). They seem to have some smear issues but im not sure smear is so bad, especially if you have control over what you are shooting (it reminds me of lens flare), and they have significantly better light sensitivity and probably lower noise.

as for 48fps ... 1280x720 8bit is 921600 bytes/frame so you need 44236800Hz (44.2MHz) to pull that off. I cant recall being able to break an effective pixelrate of much more than 37MHz.
Yeah ... I'm starting to think that your idea of using binning to get 1024x576 is going to work out the best with this camera. At that resolution you can get better signal-to-noise, 10 bits, 48 fps (hopefully) and use the entire imaging area. Resampling up to 1280x720 would soften the image a little, but given the other positives it seems like a decent tradeoff.

Edit: Apparently the Sumix doesn't do binning, just decimation. So you'd get aliasing and the signal-to-noise ratio would be unchanged. Nuts. Still, it might be the way to go in order to reduce the rolling shutter issues even further.

maybe take a look at the sensor documentation. I'm pretty sure it supports binning, but I think Sumix's API only supports it through register programming. And then I think it requires some overhead in the camera, which drops the effective pixelrate. Yeah, it's hard to know exactly whats best. USB isnt such a bad interface, but 48MHz is a pretty limiting pixelrate when trying to do any kind of HD while also minimizing rolling shutter. Maybe mechanical shutter plus global reset commands is the only way this camera will become acceptable past SD. 1280x720 and my anamorphic 960x1080 with vertical blanking arent so bad, but the M73 isnt made for 720p and the remaining rolling shutter and lack of binning are still problematic. Not to mention questionable color reproduction. I have pulled out the camera a few times since I moved to NYC, but the lighting in my new room looks horrible even with the IR/UV cut filter. Somehow it's still getting a lot of IR.

maybe take a look at the sensor documentation. I'm pretty sure it supports binning, but I think Sumix's API only supports it through register programming.
That's definitely worth looking at; do you have any information on the registers? Are those Sumix-specific or do they go directly to the chip?
Maybe mechanical shutter plus global reset commands is the only way this camera will become acceptable past SD ... the remaining rolling shutter and lack of binning are still problematic. Not to mention questionable color reproduction.
It's certainly not going to be perfect, but for the price it might still be a very acceptable alternative. Thanks for the info!

The registers are built into the sensor and very well documented if you download the data sheet for the sensor from www.micron.com . The way you access these registers is a Sumix thing though. And once you start using registers, you have to be careful about what Sumix functions you call since half of them will reset the registers or screw up because they dont expect registers to be changed manually.

The registers are built into the sensor and very well documented ...
So, with a little care it should be possible to configure it properly for binning. Definitely promising. Thanks for the info!

Edit: According to my off-the-cuff calculations*, at 1024x576x10bit, 48fps would require ~27Mhz bandwidth and ~34 MB/s, assuming that Sumix packs the 10-bit data. By boosting the clock speed higher, this should allow for reducing rolling shutter significantly.

*Rob Lohman, thanks for the spreadsheet you posted earlier.

10bit is pretty much out on this camera since sumix doesnt pack 10bit. Each 10bit pixel is stored in 2 bytes. It does, however, have full control over the LUT, so 8bit log works as long as you program it. Youd be lucky to get this camera to work well for SD in 10bit mode. I forgot about trying to use raw 10bit on this camera a while ago. Its no fun trying to work with the linear image anyway, its super high contrast with no correction. I dont know much about how the 10bit mode in the camera works except about it using 16bits and I dont trust it anyway. Im not even sure you get a 10bit image in 10bit mode... looks the same to me as the 8bit, even when pushing the lighting to get color stepping...

10bit is pretty much out on this camera since sumix doesnt pack 10bit. Each 10bit pixel is stored in 2 bytes. It does, however, have full control over the LUT...
I was afraid of that. Well, with the LUT it should still be possible to get good images with a bit more care during shooting.

As for my progress, I'm finally back in the US and getting back to work on the software. So far my tests are good, less cpu utilization with my faster computer i left here, looks like i may need to change my disk writing method again since ive been using a little test program to try different methods of writing data and it seems .NET filestream can get 52MBps on a linear write as long as i dont give it more than 256KB to write at a time (weird, huh? maybe a disk cluster size thing... it benchmarks only about 30MBps running one 1MB write operation and 52MBps with 4 separate calls to write 256KB).

This is what you would expect. Remember that the drive interface is faster than the physical data transfer to the platter. For sustained writes, you cannot exceed the physical data transfer speed. You need to look at the drive spec to see what that is. If you do anything other than sequential writes, then the performance will be impacted quite negatively, since seeks are very much slower. The effects you are seeing are due to the drive (and possibly OS buffering). To get a good idea of speed, you want to write a very large amount of data, say 100MB.

Multi-threaded writing can be good, if it is to decouple the data source from the writing. Not if you have multiple threads writing to the same disk. In the first case, you are trying to make sure the drive is 100% busy. The second moves the disk head around. which is bad. Also, for best results, the Data disk should not be the same as the OS.

Hope that helps!

I'm not sure how you mean this is what you would expect. You would expect 4000 256KB writes to be faster than 2000 512KB writes? Sorry if my explanation was confusing, but I thought I mentioned that I was dealing with 500-10000MB transfers, the 256KB and 512KB data just relates to the way those sustained transfers are handled in programming.

That particular drive is generally capable of ~60MBps sustained on outer cylinders and ~40MBps on inner cylinders at least based on my benchmarks and tomshardware.com, so this is why im glad I have been able to hit >52MBps in practice using the .net filestream. It is also partitioned to keep all data on the faster section of the platters, and it is of course a dedicated, empty disk. I usually check for fragments after a long write benchmark or video recording, and things tend to be contiguous so I doubt disk seeks are where the bottleneck is.

Yeah I am mostly just relying on multithreaded writes for the sake of not dropping frames when the system is waiting for a write to kick off. I also seem to have better performance with multithreaded writes but also much trouble keeping the writes executing in the right order.. It's too bad I havent had much luck using long buffered writes. I wonder if compressing the data might help reduce the bottlenecks im seeing (maybe when i get a faster system).

Sorry, I must have misunderstood the question! Since I've spent quite a bit of time designing and building high performance I/O systems, I can still say that surprising results are not that surprising. You often find that between the compiler, the language libraries, Operating system, controller and disk, you can usually find a lot of strange behaviors. You have to really dig into the OS and language structures to get a good handle on how they may behave when trying to push the performance to the max.

If you are getting dropped frames, do you know where they are being dropped? Typically you have an in memory queue structure between the data producer and the thread that is writing to the disk. Having one thread that ONLY grabs data off of a queue and then does a blocking write tends to be the easiest to keep things in the correct order and keep the disk I/O maximized. You can then put data on the queue as quick as you need, the queue will make sure you don't drop frames until you run out of memory...

Can anyone recommend a firewire 800 version of the Sumix M73 ? I'm trying to get a firewire interface for it. Thanks!