My little DIY camera project

[Noah Yuan-Vogel]

Ok bit of an update. I was playing around with my camera today and with my IR filter pain, I almost forgot how many good things there are about this camera. Anyway, I have interface screenshots and a sample video showing that video capture is pretty smooth direct to disk at 1024x576 24fps on my 1ghz intel. Also note the shallow dof of my super sharp 17mm f.95 lens (did i mention it was about $1 on ebay? oh yeah i did but i just like saying that again). Anyway I output this directly to divx avi from raw bayer using my conversion program which is going to be integrated directly into the playback interface of my camera program. My program read it out as something like ~23.7fps maybe with one weird frame or two (still need to iron out all the kinks in the way my memory buffers work i suppose). Also, I think that 23.7fps really means a few dropped frames rather than non-24fps performance, so i think the frames are still 1/24th sec just a couple might be missing.

Ok here is the link for the sample video:

http://rapidshare.de/files/11809919/...01_01.avi.html

If anyone is interested in giving me a little space to keep video samples, let me know, I might even be able to do some with less compression. Ive never used rapidshare before so let me know if there are any problems.

[Wayne Morellini]

You're doing your own capture program, congratulations, 1Ghz, sound like something somebody around here used to say. Anyway, Can you tell us more about this program, how it's programmed, what it is based on etc?

How does it perform under 720p resolution?

Noah, as you have such a good relationship with Sumix, could you ask them something for us. Previously they were supposed to be making
a camera for our needs. There was to be an Altasens based camera (very nice performer) and IBis5a based camera, with a buffer, compression engine and Gigabit Ethernet interface, that was to be released in 2004/2005. Can you ask them what happened to that? We just get big silence when we ask them.

One last thing, don't let people discourage you about programming your camera program, work professionally with expertise, and learn whatever you need to do the job properly ;)

God Bless.

[Oscar Spierenburg]

Well, the first impressions are pretty good hey? No rolling shutter effect so far either. What is exactly the problem with the filter? Does the front of the lens rotate when you focus?

[Noah Yuan-Vogel]

I certainly like the look and control I am getting from this camera. My filter problem is that the 62mm IR filter doesnt mount directly to the m40.5 and m35.5 lens threads. I can mount the filter in the matte box and fasten that in front of the lens using my rails, but then there is a lot of reflection from behind the lens. For those pictures and the video, I set the focus and then threw a black sock over the section between the lens and the filter to block out light (certainly not a lasting solution). Also I have tried using sumix's framerate function and also separately tried setting blanking myself, but the problem with 1024x576 at 2x2 subsampled is there isnt much to vertically blank and use to reduce rolling shutter, only like 192 lines of blanking... As for 720p, i experimented with that a lot before and its certainly possible just not with my little 1ghz shelton board, although i might be able to do it if i turned off display of the image to reduce cpu usage but then how would i see what im shooting. For now I am sticking to using as much of 1/2" chip as possible and doing 1024x576 for proof of concept and for development. Anything I do here will be easily scalable to 720p given a hardware upgrade which i forsee happening when i get back to the US in 4-5 months. Maybe then I'll go back to my original venice core amd cpu plan (or see what new technology i can use to my advantage).

Oh yeah about the program, its just a relatively simple vb.net app that controls sumix's camera API. It is intended as the single program necessary to do all of the things one might do with the camera and controlled entirely by a numeric keypad. There is a recording and playback interface which i posted screenshots of. the playback interface reads the raw bayer videos and can skip through them and works in conjunction with a simple commandline conversion app that uses the avifile api to convert directly from bayer to any avi format using the windows avi codec dialog and sumix's lapacian debayer function. for some reason the encoded video is coming out upside down for me but im sure thats an easy fix. It's interesting because my playback program reads the bayer data fine rightsideup.

Oh yeah and dont mind the crappy look of the interface, I had to resize it for my little 800x600 screen so it looks a little funky now, it was originally designed to shoot 16x9 or 2.35:1 images on a 4:3 screen so there wouldnt be overlap of the gui stuff on the displayed video. I'll probably wait to fix it again until i get a screen that can support more than 800x600. You may notice the display cuts off some of the video that was captured, thats because it displays 800x600 and the video is 1024x576 and sumix's video display function doesnt scale the video without scaling the stream (using subsampling).

[Noah Yuan-Vogel]

that link again for screenshots of the gui and captured stills is:
http://community.webshots.com/user/nyvz4

[Wayne Morellini]

Originally Posted by Noah Yuan-Vogel

Oh yeah about the program, its just a relatively simple vb.net app that controls Sumix's camera API. It is intended as the single program necessary to do all of the things one might do with the camera and controlled entirely by a numeric keypad. There is a recording

So basically you are running their capture engine in the background and are sending control commands from visual basic, close enough to what I meant. How long did it take you? I regretted not being on the ball with my programing skills (health) in 2004, because I figured it could be done in 3 days if you knew what you were doing in advance. I tried to tell a few people.

Wish you the best with it. By the way did you find out about the Sumix camera they were doing for us, or has somebody else got it now?

Well, time to take a break from these forums until the new HD camera information comes (the news forum has the Sony HC3 information) running out of ISP hours ;)

Thanks

Wayne.

[Oscar Spierenburg]

I was wondering why you (or someone else) choose the SMX-m73 and not the SMX-150C with global shutter. Is it much more expensive?

Any progress Noah? Any ideas on that capture to RAM post (some posts up the thread)

[Noah Yuan-Vogel]

My reason for choosing the M73 over the 150C isnt so much the price ($999 vs $800) as it is the characteristics of the chip itself. I think i went over some of this before, but basically the ibis5 is said to be pretty poor in terms of light and color performance. And I enjoy the flexibility of the M73 with more pixels although it can be more difficult to use the whole sensor area without subsampling.

As for capturing to ram, I have no problem capturing to ram, but of course it provides limited (my current system is limited to 512MB). My current capture system uses ram as a frame buffer, holding incoming frames (in groups that can be easily configured) in ram until they are written to HDD. In other words, you set X and then it captures X frames to RAM and begins writing them to HDD as soon as X frames are in RAM. This of course means that I could just set X to the size of available ram. Then I'd get smooth capture of frames at high rates until my ram is filled at which time itd start getting really jumpy.

As for progress, I am working on my program, started messing with LUTs to 8bit log data, and I am still looking into how I can securely mount my darn IR filter so I can start taking my camera out and shoot handheld.

[Solomon Chase]

Very interested in seeing how this develops. good work so far :)

keep us updated!

[Noah Yuan-Vogel]

So from recent tests, ive found that my system has no problem recieving uncompressed 720x540 24p video and saving it realtime etc up to about 34fps. Also, generally it drops no frames in 960x540 24p once the HDD has spun up and assuming no other programs are running (sometimes drops a frame if i run it directly out of Visual Studio, since having the development environment running while the camera program is running eats up cpu and ram). From what ive seen, 1024x576 24p might be a little too much data for my system, but thats what i get for trying to do all this with an cacheless 1ghz intel system. My hope is that cpu is the primary bottleneck and it will be very easy do 720p or more with twice the processing power. Perhaps for now i will continue with that assumption, and focus on testing for framerate consistency and audio sync, and drop my working resolution to 960x540.

In other news, im still looking for designs to mount my mattebox and filter to the lens in a way that accomodates multiple lens diameters, and allows for focus ring rotation but still isolates from light. I've started looking at pro matte box designs because im not totally sure what the accepted method is to deal with a rotating lens front while not being able to rotate the matte box but still getting a good seal between the two. Any suggestions would be awesome.

[Wayne Morellini]

Noah, what is your hard drive? Often hard-drives are too slow for HD uncompressed (sometimes less than 10 MB/s). We looked at a lot of Seagate rapture like drives (min 50 MB/s sustained real data right, but I would not even trust that not to drop something) every now and then). But now even cheaper lines are moving towards 33-50 MB/s.

These files that you are getting, are they made up of 8 bit pixels, or are they 8 bit pixel in 16 bit words?

I'm curious about these things, to determine the outer limit of a 1GHZ Intel system.


About your Matt Box system, there are many DIY projects on the web, and a few pages full of DIY video projects. Best to google mat box and DIY, otherwise start with goggling slr adaptors (or $14 steadycam) and follow the link lists the sites have to find other types of DIY projects. I once spent many hours doing that and came up with a number.

[Noah Yuan-Vogel]

I dont think at this point (and I hope I am right) that the harddrive is my bottleneck. The data stream I was working with at 1024x576@24p was about 14MBps, and my WD3200JB harddrive should be able to handle 30MBps no problem, given linear writes. I bought this particular harddrive because tomshardware.com listed it with a minimum write speed about 36MBps and averages transfers around 50-60MBps, second only to the 10000rpm raptor line in terms of ATA drives (which brings greater $$,sound, power, and storage issues). If there is a problem with my HDD bandwidth, it is more likely due to limitations of the vb.net filestream methods or of my implementation of them. I have tested the drive and it does 50+MBps no problem. I even partitioned off the drive to avoid using the inner 1/3 of the cylinders (the ones that might result in the only 35MBps). And remember this drive is completely dedicated to raw video writes and avi conversions, which never occur at the same time, so these should all be linear writes. If it were forced into random writes, i think the performance would suffer so much that i wouldnt even be able to do 10MBps with consistency. I think it is more likely that at 15MBps frame captures, the cpu cannot run through the code while writing fast enough to catch all the frames sent by the camera. Someone let me know if im wrong though, because id hate to spend $300 on a mobo/cpu upgrade only to find the problem can only be fixed with a RAID.

Ill take a look at DIY matte boxes, although im worried they wont be what I need since most people making DIY matteboxes are dealing with fixed lens diameter and fixed filter thread (i have a lens that doesnt even have filter threads), and servo zoom/focus where the lens front doesnt rotate when focused and doesnt move (lengthen) when zoomed.

[Keith Wakeham]

Fstream.h in c++ is really slow. I done tests with it about a year ago when i was following this stuff and before I started work with FPGA's and found it to be really slow and definetly not able to handle video well. What I found was that it would make out my processor and that would be the bottleneck.

But this is c++ not Visual basic, so it might be different for you. I would do some raw tests first to make sure that it is using dma to get the data to the drive.

[Wayne Morellini]

Originally Posted by Noah Yuan-Vogel

Someone let me know if im wrong though, because id hate to spend $300 on a mobo/cpu upgrade only to find the problem can only be fixed with a RAID.

Sounds like your on the right path, what Keith says is worth noting, the stuff the drivers, etc, that ship standard, often are not the fastest. It is best to hunt down custom stuff from the suppliers, or third party. But it might be a lot of fuss, about very little, it might be a hard limit as you suspect. Thanks anyway, this gives me a good idea of the hard limit, assuming a good Intel chipset.

Thanks

Wayne.

[Keith Wakeham]

Assuming the camera is dma (USB controller isn't, keep this in mind) and that your program is writing to the drives using dma, then your processor power should be very minimal. Probably less than 10 or 15% with a P4 chip or pentium-m chip. If for any reason things in the chain aren't DMA you have a big problem.

I'd say find out for sure that the data coming in is dma and that your drives are transfering in dma and that they are linking the dma together. It could be using dma into the proc and then the proc has to transfer all that data out to the hard drive with an entirely differnt dma channel.

Some terms I used may have been incorrect, i'm always more at the signal level anyway so its hard to figure out your problem exactly.