Image extraction from video

[Daniel Simpson]

Hello,
I’ve got a few questions that, to me, are quite confusing.

I’ll first describe what I am trying to do: I work in a soil testing laboratory and I’m running a test that involves dropping a cone into some soil. Traditionally, only the end-of-test depth of penetration is measured.

I’m trying to use video capture and processing technology to record the penetration and develop time versus displacement data from the video footage.

I have developed MatLAB script which extracts frames from video and tracks motion, but must work with image files. I have also used IrfanView to extract image frames from video.

I started this experiment using my Nikon D5100, which will record video at 25fps. This worked well- I was able to get good image quality and I actually got 25 images per second of video.

I wanted greater resolution in the time-displacement data, so I thought I needed a camera with a higher frame rate.

To keep budget low, I wanted to try to use USB cameras (like webcams). Image quality is not SUPER important. So far, I have tried two cameras, the Sony Playstation 3 EYE (supposedly records at 120fps at 640x480) and the Microsoft Lifecam Studio (supposedly records at 30fps at 720p- I’ve only been able to get about 8fps at 720p).

I have used Windows Media Encoder, Debut Video Capture Software, and a program called CL-Eye Test for the playstation eye to control the webcams and capture video using a variety of formats, resolutions, and specified frame rates.

The problem I am running into is this: The extracted number of frames does not correlate to the length of video and specified recording frame rate (i. e. If I records at 30fps for 5 seconds, I would expect 150 frames, but I’m getting something like, say, 110 frames). This problem occurs whether I use the MatLAB script or IrfanView to extract frames and with several file types I have tried (.avi and .wmv). The Nikon records to .mov. Is this a filetype issue? I’ve heard .wmv compresses frames, but I’m not sure what this means.

Can anyone give some insight as to what is going on? I would also appreciate alternative suggestions to what I am doing. Thanks in advance for your help.

[Seth Bloombaum]

Windows Media Encoder is obselete; which in this case might mean that it doesn't properly parse the incoming USB video stream. There are quite a few encoding settings and little help. It also requires a pretty fast computer that can crunch video fast. How fast? Depends on those settings and the source. But WME itself has significant load on the system.

The other two capture programs I've not heard of. There are a lot of semi-commercial products based on open-source code, and many of them are obselete, too.

Some are not, however. I recommend you look at VLC Media Player, which is updated often, and is open-source freeware. It has the capability to record an external source, you do have to go into the interface settings to enable the record button's placement on the toolbar.

Lots of settings there, too. Such is life with free software.

Shoot a clock with a sweep second hand for a minute and work out what's happening with your length mis-matches. Occasionally this can be due to audio cards that don't play nice with video capture; the solution is to turn off system sounds, make sure nothing else is using the sound card, work through some alternate sound card settings, etc.

I will say that your experience is normal; trying to get consumer level gear to do something with open-source software takes a while to benchmark and move through the various pitfalls systematically.

MediaInfo is a great little utility that will tell you exactly what it is that you've recorded, framerate, bitrate, resolution, codec, etc., very handy when benchmarking. Select one of the advanced views (tree) to see all the details.

[Daniel Simpson]

Seth,

Thanks for your feedback. I'll try VLC (it's my default playback program, I don't know why I didn't think of this!) and mess around with audio settings.

[Rick L. Allen]

I also assume you are recording to a 7200 rpm hard drive, RAID or equivalent. Sustained capture in HD requires very fast storage media. Make sure that you've eliminated any data bottlenecks in your capture scenario.

Also the Sony Playstation 3 EYE (at 120fps at 640x480) is still Standard Definition (SD). fps is unrelated to resolution.

Finally you just may need to get a real camera.

[Jeff Pulera]

Forget web cams which are low quality, check out something like the GoPro HERO3: Black Edition with 1080p-60, 720p-120, WVGA-240 which will give you high-frame rate at HD resolution, perfect for what you want to accomplish. Also, since recording takes place in the camera itself, you're not relying on the computer to keep up with the recording demands. You will get all your frames in the end.

Thanks