Interest in open-source capture software for Sumix M73

Wayne Morellini · November 3rd, 2006, 07:04 AM

Rob, for some time I have been meaning to ask you if you would like to make your app record raw from webcams. There are a umber of models out there that do video, and in times past SD models have been able to record raw with off the shelf software. I had approached a company about getting their software to do this, but I don't like the chances. But the number of companies doing application makes me believe that manufacturers are willing to listen to developers. I also was lining up a deal with a webcam parts manufacturer to become a developer for their webcam design, but the deal went nowhere, for the moment (maybe with an partner I might resurrect, but have other potential commitments that have come up in the meantime).

I don't mention any of this in the Elphel thread, because I don't want to discourage them, they have product that does descent job at a potential cheap price, compared to webcam + portable pc. I think Micron is a bit of a let down for now (they are picking up). It is not descent Ibis5a + external ADC and circuits implementation, or a Altasens, or something else that has over 52db S/N and descent well capacity for latitude. There is probably a few other possible good sensor options out there.

Rob Scott · November 3rd, 2006, 08:04 AM

Quote:

Originally Posted by John Wyatt

It's good to know you're out there, working it!

I'm enjoying myself! :-) Hopefully I'll have something for you to try out fairly soon.

Quote:

Originally Posted by Wayne Morellini

Rob, for some time I have been meaning to ask you if you would like to make your app record raw from webcams. There are a number of models out there that do video, and in times past SD models have been able to record raw with off the shelf software.

I would certainly be interested, if it would be useful for filmmaking. Do you know of any models that currently return raw Bayer data?

I did a couple of Google searches, and discovered that there are a set of Firewire cameras that use the IIDC / DCAM standard. I also found Coriander, a Linux GUI for controlling these cameras. This site lists a number of compatible cameras.

Edit: I found several open-source control libraries: 1394-based DC Control Library for Linux and DSVideoLib for Windows. The ARToolKit project apparently contains a DSVideoLib package as well.

Wayne Morellini · November 4th, 2006, 08:05 AM

Thanks for the extra links Rob. The old firewire camera chart contains information on a Foveon based machine sensor, I have just emailed the manufacturer to find out about it.

I am unsure which one this is, but there is one listed as 3 Layer:
http://www.net-gmbh.com/e/3LayersCMOS.htm

The list has 3Layer listed under Toshiba Teli though:
http://damien.douxchamps.net/ieee139...ras/search.php

SD Firewire webcams with RAW recording are listed, along with the usual machine vision cameras. The Apple Isight is one, but the cheapest Unibrain's and Irez are also there. I forget which cameras have RAW enabled, under $140 I think, maybe $80. I don't know of any cheap HD firewire cameras yet (maybe Apple will start) the HD cameras are USB, I have not investigated which has RAW ability (something you are likely going to have to be activated after market) because I am otherwise busy for sometime.

The most you could expect out of Raw Webcam is to make footage better then Sanyo HD1 or SR1, or HC1 if a good sensor is available. I had been investigating a possibility to make a camera perform as good as the Sanyo HD1 without Raw for less than a few hundred dollars.

The Astro cam groups even modify cameras, write software and replace sensors/components with better ones for better performance. But I could never get an email back from them, so I just gave up on them.

The problem with finding people to work with is that many people are content to work on a level of "disruptive co-operation" if anything at all.

The market is changing, as prices tumble and HDMI recording is a reality low priced, webcams go HD, video camera corporations are worried about video capabilities in Digital Cameras and Cell Phones (despite what they think, in denial, over at camcorderinfo, the rising ability of sensors will make video, even HD acceptable enough to a standard comparable to Sanyo or current AVCHD cameras, even the microns are effectively the things you would use in low to mid end Digital cameras, and Mobile phones). This is an end of market scenario, it would be best if the video corporations concentrated on much better quality HD for the low end and the next level beyond HD, before mobiles get there. For us, it means that machine vision can be much better quality than the low end, but not much cheaper etc. This limits how much we can practically achieve dollar for dollar over low end HD camera with HDMI recording. So webcams can give a low cost solution similar to the quality you could expect from low end cams with HDMI for much less.

I have other concerns on my platter, even if I become available again, doing it all myself is a low priority.

Now, I am not saying, as I always did, that there is not a market for a sub $5K camera, just that it has to be a lot better quality per dollar now because of the competition. The new Sony V1 with a HDMI uncompressed recording solution will likely deliver a picture so good, most of the time, that it will be good enough for most low end people.

Wayne Morellini · November 8th, 2006, 10:08 AM

Got email back, this is a Foveon based camera 1/3rd inch, Hitachi based product. It is not available and no more information is available yet. I suspect it won't be cheap, and if it really si restricted to 400Mbps it won't have much of a frame rate in HD, and that firewire spec basically restricts you to 8-bits in RGB, but it is a start.

Forgot to mention: If it is based on recent Foveon X3 design, it might have performance that greatly exceeds the Microns.

Rob Scott · November 8th, 2006, 10:33 AM

I received my Sumix M73 camera today, so I hope to start adding support soon. I have been slowed down a bit though -- we had a death in the family and my kids got food poisoning, at least one stomach bug and possibly strep. It's been a fun week so far.

Wayne Morellini · November 12th, 2006, 06:02 AM

Quote:

Originally Posted by Rob Scott

I would certainly be interested, if it would be useful for film making. Do you know of any models that currently return raw Bayer data?

Rob, it looks more certain now I will be busy on another project for an undetermined amount of time. Please feel free to investigate the webcam thing yourself.

Rob Scott · November 12th, 2006, 07:14 AM

Quote:

Originally Posted by Wayne Morellini

Rob, it looks more certain now I will be busy on another project for an undetermined amount of time..

Best wishes on the project, Wayne!

Rob Scott · November 13th, 2006, 12:55 PM

I have implemented basic (read: very basic) support for the Sumix SMX-M73 camera in ObscuraCap. It's far from perfect, naturally -- I have no idea how accurate the frame rate is, the preview colors are screwy, etc.

John Wyatt · December 10th, 2006, 03:54 PM

Rob -- I've been waiting for better weather to shoot some brighter-looking test clips to illustrate the little website I'm writing which is about my experiences with the Sumix. I do like the uncompressed images it makes. How you getting on with your M73? Anything new with your software?

Regards,
John.

Rob Scott · December 10th, 2006, 04:25 PM

Quote:

Originally Posted by John Wyatt

How you getting on with your M73? Anything new with your software?

I have had limited time recently to work on it, but I am able to capture images. the biggest problem right now is that I'm not able to open/close the connection properly all the time. Sometimes the system just won't recognize the camera and I have to reboot. Making progress, though.

Igor Babic · December 27th, 2006, 04:19 AM

Rob,
I am intrested in your project. I have looking at SMX73 for an indie film project where this camera will be attached to helmet for first person view.
Your capture program will be a great toll for this kind of projects.
Did anyone try to capture with SMX73 in VirtualDub or StremPix(they have freedownload trilal)?

Igor Babic · December 29th, 2006, 05:02 AM

VirtualDub has exelent Capture utility and is famous for precise audio sinc to video. It has lots of real time info during capture: File size, Frames captured, CPU usage, Data rate, Compression ratio, Frame dropped, VU meter, Histogram, realtime preview. I think is open sourced and it will be maybe easier to try to adopt this or expand it for use with SMX73. If you guys that have this gadget can just try to capture and see if it works. It will be a good info. ( its free, and has no installation, simple drag&drop) BTW: It works with my BMD HD extreme to BMD Mjpeg codec...

John Wyatt · December 29th, 2006, 11:06 AM

Igor -- I believe VirtualDub will only capture an RGB or YUV video source to hard drive (?) which would limit the frame size you can record. To get the best out of a camera like the M73 you would want to capture HD-sized frames, though in RGB or YUV mode large frames would exceed the datarate of USB 2.0, and the hard drive would not be fast enough to capture them at our preferred speeds (such as 24 or 25 fps). What's needed is a capture program which will solve these problems by taking in Bayer video (one-third the size/datarate of RGB), and to record it temporarily to RAM (avoid the hard drive speeds, particularly relevant with laptops: many are usually still 5,400 rpm speed instead of 7,200 rpm found on desktops). The Sumix cameras come supplied with software for RGB/HDD and Bayer/RAM recording, so you can use them straight away. Rob is working on his capture software (originally coded for an SI camera) which may provide more Bayer/RAM recording enhancements for the Sumix cameras. Perhaps large frame RGB/HDD recording is possible with compression, though personally I like the uncompressed look of the images...

Cesar Rubio · December 29th, 2006, 11:52 AM

John:

The Stream Pix record RAW video, wich I prefer for the reasons that you've mentioned, and also because you can correct the video in post for better tuning, that sometimes when you are recording no are able to do for time constraints, or for not having the adequate equipment to do so on the field.

You can try Stream Pix for free, if you like it...don't forget to e-mail me please...you might actually like it, and it offers FREE LIFETIME UPGRADES!

Thanks,
Cesar Rubio.

Igor Babic · December 30th, 2006, 10:05 AM

John, I also agree with you about uncompressed, but none of those capture apps capture audio ( as I can see ). Cesar, correct me if I am wrong, but I am willing to lose some video quality for good and in sinc audio. It is importnant for my projects.

November 13th, 2006, 12:55 PM	#23
Rob Scott Major Player Join Date: May 2004 Location: Knoxville, TN (USA) Posts: 358	Basic support for M73 I have implemented basic (read: very basic) support for the Sumix SMX-M73 camera in ObscuraCap. It's far from perfect, naturally -- I have no idea how accurate the frame rate is, the preview colors are screwy, etc.

December 29th, 2006, 11:52 AM	#29
Cesar Rubio Regular Crew Join Date: May 2006 Location: Guadalajara, Mexico Posts: 48	John: The Stream Pix record RAW video, wich I prefer for the reasons that you've mentioned, and also because you can correct the video in post for better tuning, that sometimes when you are recording no are able to do for time constraints, or for not having the adequate equipment to do so on the field. You can try Stream Pix for free, if you like it...don't forget to e-mail me please...you might actually like it, and it offers FREE LIFETIME UPGRADES! Thanks, Cesar Rubio. __________________ It's the long term experience of problem solving what makes you successful! & what we do in life, echoes in eternity. Last edited by Cesar Rubio; December 29th, 2006 at 07:06 PM.

November 3rd, 2006, 07:04 AM	#16
Wayne Morellini Inner Circle Join Date: May 2003 Location: Australia Posts: 2,762	Rob, for some time I have been meaning to ask you if you would like to make your app record raw from webcams. There are a umber of models out there that do video, and in times past SD models have been able to record raw with off the shelf software. I had approached a company about getting their software to do this, but I don't like the chances. But the number of companies doing application makes me believe that manufacturers are willing to listen to developers. I also was lining up a deal with a webcam parts manufacturer to become a developer for their webcam design, but the deal went nowhere, for the moment (maybe with an partner I might resurrect, but have other potential commitments that have come up in the meantime). I don't mention any of this in the Elphel thread, because I don't want to discourage them, they have product that does descent job at a potential cheap price, compared to webcam + portable pc. I think Micron is a bit of a let down for now (they are picking up). It is not descent Ibis5a + external ADC and circuits implementation, or a Altasens, or something else that has over 52db S/N and descent well capacity for latitude. There is probably a few other possible good sensor options out there.

November 4th, 2006, 08:05 AM	#18
Wayne Morellini Inner Circle Join Date: May 2003 Location: Australia Posts: 2,762	Thanks for the extra links Rob. The old firewire camera chart contains information on a Foveon based machine sensor, I have just emailed the manufacturer to find out about it. I am unsure which one this is, but there is one listed as 3 Layer: http://www.net-gmbh.com/e/3LayersCMOS.htm The list has 3Layer listed under Toshiba Teli though: http://damien.douxchamps.net/ieee139...ras/search.php SD Firewire webcams with RAW recording are listed, along with the usual machine vision cameras. The Apple Isight is one, but the cheapest Unibrain's and Irez are also there. I forget which cameras have RAW enabled, under $140 I think, maybe $80. I don't know of any cheap HD firewire cameras yet (maybe Apple will start) the HD cameras are USB, I have not investigated which has RAW ability (something you are likely going to have to be activated after market) because I am otherwise busy for sometime. The most you could expect out of Raw Webcam is to make footage better then Sanyo HD1 or SR1, or HC1 if a good sensor is available. I had been investigating a possibility to make a camera perform as good as the Sanyo HD1 without Raw for less than a few hundred dollars. The Astro cam groups even modify cameras, write software and replace sensors/components with better ones for better performance. But I could never get an email back from them, so I just gave up on them. The problem with finding people to work with is that many people are content to work on a level of "disruptive co-operation" if anything at all. The market is changing, as prices tumble and HDMI recording is a reality low priced, webcams go HD, video camera corporations are worried about video capabilities in Digital Cameras and Cell Phones (despite what they think, in denial, over at camcorderinfo, the rising ability of sensors will make video, even HD acceptable enough to a standard comparable to Sanyo or current AVCHD cameras, even the microns are effectively the things you would use in low to mid end Digital cameras, and Mobile phones). This is an end of market scenario, it would be best if the video corporations concentrated on much better quality HD for the low end and the next level beyond HD, before mobiles get there. For us, it means that machine vision can be much better quality than the low end, but not much cheaper etc. This limits how much we can practically achieve dollar for dollar over low end HD camera with HDMI recording. So webcams can give a low cost solution similar to the quality you could expect from low end cams with HDMI for much less. I have other concerns on my platter, even if I become available again, doing it all myself is a low priority. Now, I am not saying, as I always did, that there is not a market for a sub $5K camera, just that it has to be a lot better quality per dollar now because of the competition. The new Sony V1 with a HDMI uncompressed recording solution will likely deliver a picture so good, most of the time, that it will be good enough for most low end people.

November 8th, 2006, 10:08 AM	#19
Wayne Morellini Inner Circle Join Date: May 2003 Location: Australia Posts: 2,762	Got email back, this is a Foveon based camera 1/3rd inch, Hitachi based product. It is not available and no more information is available yet. I suspect it won't be cheap, and if it really si restricted to 400Mbps it won't have much of a frame rate in HD, and that firewire spec basically restricts you to 8-bits in RGB, but it is a start. Forgot to mention: If it is based on recent Foveon X3 design, it might have performance that greatly exceeds the Microns.

November 8th, 2006, 10:33 AM	#20
Rob Scott Major Player Join Date: May 2004 Location: Knoxville, TN (USA) Posts: 358	I received my Sumix M73 camera today, so I hope to start adding support soon. I have been slowed down a bit though -- we had a death in the family and my kids got food poisoning, at least one stomach bug and possibly strep. It's been a fun week so far.

December 10th, 2006, 03:54 PM	#24
John Wyatt Regular Crew Join Date: Jan 2006 Location: West Country, UK Posts: 141	Rob -- I've been waiting for better weather to shoot some brighter-looking test clips to illustrate the little website I'm writing which is about my experiences with the Sumix. I do like the uncompressed images it makes. How you getting on with your M73? Anything new with your software? Regards, John.

December 27th, 2006, 04:19 AM	#26
Igor Babic Regular Crew Join Date: Dec 2006 Location: Zagreb, Croatia Posts: 171	Rob, I am intrested in your project. I have looking at SMX73 for an indie film project where this camera will be attached to helmet for first person view. Your capture program will be a great toll for this kind of projects. Did anyone try to capture with SMX73 in VirtualDub or StremPix(they have freedownload trilal)?

December 29th, 2006, 05:02 AM	#27
Igor Babic Regular Crew Join Date: Dec 2006 Location: Zagreb, Croatia Posts: 171	VirtualDub has exelent Capture utility and is famous for precise audio sinc to video. It has lots of real time info during capture: File size, Frames captured, CPU usage, Data rate, Compression ratio, Frame dropped, VU meter, Histogram, realtime preview. I think is open sourced and it will be maybe easier to try to adopt this or expand it for use with SMX73. If you guys that have this gadget can just try to capture and see if it works. It will be a good info. ( its free, and has no installation, simple drag&drop) BTW: It works with my BMD HD extreme to BMD Mjpeg codec...

December 29th, 2006, 11:06 AM	#28
John Wyatt Regular Crew Join Date: Jan 2006 Location: West Country, UK Posts: 141	Igor -- I believe VirtualDub will only capture an RGB or YUV video source to hard drive (?) which would limit the frame size you can record. To get the best out of a camera like the M73 you would want to capture HD-sized frames, though in RGB or YUV mode large frames would exceed the datarate of USB 2.0, and the hard drive would not be fast enough to capture them at our preferred speeds (such as 24 or 25 fps). What's needed is a capture program which will solve these problems by taking in Bayer video (one-third the size/datarate of RGB), and to record it temporarily to RAM (avoid the hard drive speeds, particularly relevant with laptops: many are usually still 5,400 rpm speed instead of 7,200 rpm found on desktops). The Sumix cameras come supplied with software for RGB/HDD and Bayer/RAM recording, so you can use them straight away. Rob is working on his capture software (originally coded for an SI camera) which may provide more Bayer/RAM recording enhancements for the Sumix cameras. Perhaps large frame RGB/HDD recording is possible with compression, though personally I like the uncompressed look of the images...

December 30th, 2006, 10:05 AM	#30
Igor Babic Regular Crew Join Date: Dec 2006 Location: Zagreb, Croatia Posts: 171	John, I also agree with you about uncompressed, but none of those capture apps capture audio ( as I can see ). Cesar, correct me if I am wrong, but I am willing to lose some video quality for good and in sinc audio. It is importnant for my projects.