My little DIY camera project

[Wayne Morellini]

PC's are disjointed, that's the problem. They are like a bionic man made up different parts from different auto manufacturers slapped together, how does the blood flow smoothly around such a thing, that's my point. Very careful, accurate programming is needed to get around the problem, but the problem could be just impossible to get over. I wish you luck on increasing the frame rate.

BTW, a remote bell rings in my head about some sort of DMA system for USB devices, obviously rarely built in. USB-GO probably had it. So the question is, are there any chip-sets that have some sort of usable DMA support on USB?

You know, we could still get our USB//GigE/Machine vision camera systems, if we asked open sourced people on programming forums if they want to be involved. But I would wait to see what Red/Sumix does in the next couple of months first. Then again, a simple hard disk recording FPGA like you have, would also do the job.

[Wayne Morellini]

Noah,

I have just looked over the Sumix software page, and realised that they have versions for QNX, for the IBIS Fillfactory sensor based cameras. It is worth asking them if they intend to produce versions for the Micron sensors, like the one you have, as QNX is one of the great embedded real time operating systems. If there are windows related performance problems, it could possibly offer significant performance gains for specific hardware (or virtually none).

Thanks

Wayne.

[Noah Yuan-Vogel]

ok so i took someone or other's advice and went ahead and tried to benchmark my disk access code. I basically set up a mini application to write 10GB to HDD in 1MB chunks with a memorystream for buffer. in other words, 1MB chunks would be written a RAM buffer and then the ram buffer would be emptied to disk as fast as possible. This is basically what my recording program does. if i send it way too much data, the RAM fills much faster than it can empty and i get an out of ram error (i plan to clean this up but im still trying to decide how). anyway, i found that given no video stream to display or usb stream to deal with, my system could handle 32MBps (+-3%) and averaged around 40% cpu usage (20%-60% fluctuation). I was using the system.threading.thread.sleep() method to do the delays, and i think it was because of its poor precision that i couldnt get better results than that at a 10ms delay it couldnt handle the data stream and at 15ms-30ms or so it ended up at ~32MBps. if you do the math, 1MB every 30+ms (30ms delay plus some small processing time to write) should be pretty close to the 32MBps and is just a bit more data than 720p@30fps@8bpp. This makes me confident that even though i dont know anything about getting VB to do anything straight-DMA, the code is fairly efficient and it ought to be able to at least handle a 720p@24p data stream if i were to disable display and could do more with some more processing power. keep in mind that i watched cpu and ram usage while running these tests and even while running at 32MBps ram buffering never really used more than 64MB of my ~400MB RAM left over after some windows services and integrated video take some, so 32MB might not even really have been pushing the limit... any thoughts?

oh yeah, sorry im horrible at separating paragraphs

[Wayne Morellini]

What happens to the performance and CPU usage when you turn on USB and display on individually and together? The 20% to 60% cpu swing interesting, what is it caused by, burst writing, or some Windows inefficient servicing? If you buffered 256MB before starting to write to eliminate burst writing from gaps between writes, how would performance behave? What ever the case it is likely that twice the performance could be derived from the system, but getting that would require some fancy programming experience, so it maybe worth just upgrading instead.

You could look at those Windows XP performance tuning sites I posted that I in the Digital Cinema home made camera technical thread, and turn off all the unneeded ones, and adjust the rest, and see what happens.

Thanks

Wayne.

[Rob Lohman]

I can't imagine Visual Basic handling this vary graciously in the way it is all
implemented (under the covers). I would not be suprised if you see a high
drop in CPU time when switching to C with the native WIN32 API doing
overlapped and asynchronous I/O

[Noah Yuan-Vogel]

If my understanding of C or C++ were good enough, i would have started with it. I even tried C# which is an easier interface for the non-experienced and decided the learning curve was still not favorable for me. I admit some of those cpu usage stats might be off, since I ran them from the development environment so that was all open in the background. normally i have a separate windows installation with most background services and some hardware disabled to keep idle state cpu usage to near 0%. I suppose i could try it out in there, but its much easier to debug from my development installation and most of my benchmark program was quick and messy and in need of recompiling between tests.

I actually thought 40% cpu use in a 32MBps HDD transfer through visual basic on my fairly outdated system was pretty reasonable. If only i knew where to begin in making it work better. I'm almost afraid to get back into trying new buffering and recording options since it took me probably 20 tries to get one that worked as well as it does.

[Rob Lohman]

C# would not be much good either with such a system in my opinion. It takes
you too far away from the bare metal that you need.

The "problem" with VB in this case is that you can't tell Windows (as far as I
know) how it should do the I/O. You also can't use multiple threads to do the
buffer filling and disk writing.

I know you can call directly into the WIN32 API from VB, so that might be a
better way than to use the native VB functions, since you'll have more control.

Good luck! :)

[Noah Yuan-Vogel]

Yes, I think I can call win32 apis directly but do I am not familiar enough with it to know what methods to use to get it to do faster writes. Perhaps you could give me an idea about what to do?

What do you mean you cant use multiple threads to do buffer filling and disk writing? do you mean its not possible or its not a good way to do things? currently my main program, for each frame, puts the frame in memory and then calls a new thread to write that buffered frame. is there a better way? perhaps using win32 api?

[Wayne Morellini]

Originally Posted by Noah Yuan-Vogel

I actually thought 40% cpu use in a 32MBps HDD transfer through visual basic on my fairly outdated system was pretty reasonable. If only i knew where to begin in making it work better. I'm almost afraid to get back into trying new buffering and recording options since it took me probably 20 tries to get one that worked as well as it does.

Yeah, it is pretty good, but there are GB/s in lots of PC's and getting it to do the job with preview capture and write, would be great. Best bet is to write a start time and end time for each record session (even trying to write frame times to screen can hiccup things if done wrong) and try testing it on your other sterile system you have setup. It would be interesting to see what speed can be achieved on a clean system compared to the development environment. Those links from my thread might also show you new ways to configure windows, because even if you get the programming perfect the OS can still drag things around.


Rob, Noah is using the camera development API and it's engine, so visual basic should be only a small part of the performance picture.


Noah, apart from the previous suggestions I have made, if you have too many options for recording maybe you can program each combination of options into a benchmark program, and run the benchmark program to give you times in comparison to each. Still difficult, but a bit easier than individually trying each separately, also you get to keep the code and retry it when you change your system, or other things, so you can use it to see if the method your using is still the best.

Rob is probably a good person to talk to about programming stuff. Maybe you too should help each other, but then there's the other Rob, what's he upto.

[Rob Lohman]

The other Rob had life intervene from what I understand.

Unfortunately I simply do not have the time at the moment to help anyone
out in that regard. Sorry. However, if you (Noah or Wayne) are coming to
NAB we can sit down and do some talking there.

Noah: what VB methods are you using to do multiple threading? I'll see if I can
dig up some info on win32 api calling. But that won't happen before the end of
the week since I'll be out of town on business for the next couple of days.

[Wayne Morellini]

Would love too, but can't afford it (and buy new camera) would be great to check out the new Red camera, wouldn't mind getting one for doco.

I support Noah's project, but seriously, I've asked Sumix and you guys what happened to their cinema cameras they were developing, everything went silent with no confirmation of death or life of it. I personally have my own project ideas for a cheap pixel shift camera with much less bandwidth than bayer.

[Noah Yuan-Vogel]

Well NAB would be awesome, but I am not in the US at the moment and probably wont be for several months.

Regarding the variety of different ways i was writing to disk, i did do some benchmarking back when i was working on finding one that worked, and this method was by far the best. any single threaded way of writing would drop frames a lot since the camera API function that recieves frames locks up the program until the next frame is available, and then synchronous disk access functions also lock up the program until the write is done. Right now for each frame there is a memorystream buffer into which is written a frame (or set of frames) and then as soon as that returns, a delegate is called to create a new thread whose sole purpose is to write that memorystream buffer to disk. truthfully i was learning as i go, so i learned just enough about it to implement it. I believe i tried other types of threading, but this was a suggested method to do asynchronous writes, and other ways i tried to do threading ended up still taking too long to return to the main thread from calling the new thread.

In other news, I was borrowing my friend's dSLR and got the chance to compare it to my diy camera. I found that given the default LUT (which in my opinion seems to have too hard gamma) and with my UV/IR filter, my diy camera rated about iso 50 at gain 4.0, iso100 at 8.0, iso200 at 16.0 and 400 at 32.0. and there seems to be minimal noise up to about 32.0 so it is potentially useable at iso400. This was a very rough test, but it gives me a pretty good idea of the useable light sensitivity of the camera. the camera goes up to gain 128.0 (iso1600?) but that results in a lot of color and banding noise.

[Wayne Morellini]

Hmm, from the SN ratio of the sensor, there should always be noise there, just so small it is not noticeable. If you blow up on a really big high SN screen you should notice it at no gain.

I am vary disappointed with the ISO rating of the Micron. I know, from examining the Ibis data-sheet charts, there can be huge loss of sensitivity from the filter across the frequency range. In that camera, I think it does ISO400, but should get around ISO1200 in 3 chip configuration following the chart, which is close to what the Altasens would get (upto ISO1600 from what I heard rumoured). Having a look at the SN between the Micron and Ibis sensors there is a similar difference in their ratios (though the Sumix implementation does not have a good reputation for it's signal).

Pity about that locking thing. I remember what I forgot to mention about programming, that task switching etc, has traditionally been very inefficient under the popular OS, costing even thousands of cycles for a switch. Subroutines are another problem (compile in line) and I imagine starting threads as well. The OS I am designing is designed to do switching in 1 cycle. Most programmers are unaware of the extent to which this is happening a vast number of times a second. But, if the API locks you in, there might not be a way around it, apart from requesting the company that originally made the API to upgrade it so that it doesn't. We should just ask these companies to make video recording primed API and capture program changes. Sumix must have been working on such a thing for their Cinema cameras.

Thanks

Wayne.

[John Wyatt]

Noah -- I really admire what you're doing with this project. I can only give you my encouragement since I have no programming skills to offer advice. I'll soon be ordering an M73 to experiment with myself, but unlike you I'll be staying within the scope of the supplied software since I wouldn't know how to begin to modify it as you are doing. I would be interested to hear more about your decision to replace the supplied IR filtering. You said the difference in colour was so marked, it's difficult to understand why the camera isn't sold in this way to begin with! You say you continue to have problems with the filter mounting -- my own plan is to buy a Pentax K to C-mount adapter (from Edmunds Scientific) to use existing 35mm SLR prime lenses I have. I could then use commonly available filters and holders (Hoya, Cokin, etc) to screw a UV and IR filter on the front without problems.

Oscar -- I think you are interested in "conventional" RAM recording like me using the standard camera application. I have been in contact with the Sumix support team and have had many questions answered about this (a guy called Petrovich has been very helpful and always replies promply). As far as I currently understand it then, standard RAM recording with the M73 works as follows: the camera sends 10-bit RAW Bayer and the camera driver transforms this (using a conversion table) to 8-bit RAW Bayer, accumulating the footage in RAM as an smx file. Having started the recording, it either stops when you halt recording or self-stops because the RAM has filled up. Using the supplied SMXView utility, you can play the smx file held in RAM to decide if you want to delete it (flush the RAM) or keep it (save to HDD). If you want to save, the smx file is converted at this point (Bayer to RGB) to an avi file and saved to HDD at a location you specifiy. The resultant avi is 8-bit 4:4:4 uncompressed. I previously speculated (wrongly) that a utility might be needed to cleave up the RAM to allow recording, but this is not necessary: the Sumix software manages this and, for my target RAM capacity of 2 Gb, 1.5 Gb would be used for recording and 0.5 Gb would automatically be reserved for the OS. With my target video specification of 1280 x 720 x 25 fps, around 65-70 seconds can be recorded within 1.5 Gb of RAM. These figures are for 8-bit -- using the software as it is currently supplied 10-bit is only possible recording tif stills, not video to RAM. However, 8-bit 4:4:4 uncompressed video is such an improvement over DV or even HDV, along with the option of using good C-mount or 35mm SLR lenses, this is potentially an exciting camera specification. However, as Noah says (and he's using one, I haven't!), the software may not be intuitive to use.

Wayne -- I asked a couple of weeks ago about new cameras coming from Sumix and was told about the M8 series, which seem to mirror the M7 series but have slower fps specs than the M7's -- presumably aimed at the scientific/industrial sector? From this I conclude that the Altesans are not ready (though I didn't ask specifically), so the M73, which Noah is using, is still the best Sumix available for film making.

Regards,
John.

[Wayne Morellini]

Yes, saw those, forgot to mention them, you probably would have to ask directly about the Altasens and the other camera. They seem to be pretty silent about the Altasens and the other camera.