4:4:4 10bit single CMOS HD project

Ben Syverson · July 15th, 2004, 01:13 PM

Steve,

It's my best shot right now, unless you have something that can operate over USB2 or Firewire at 24fps with 1280x720 res. As far as I can tell, the Sumix is pretty much the only thing on the market that can meet those requirements. And it costs around $1000, software included. One thing -- why is the SI-1280F-U (heh) limited to 20fps at 1280x1024? That's only 25 megabytes/sec, while USB2 should be able to handle a good deal more than that... Can you do 24fps 1280x720 on it, at 21megs/sec?

I know exactly what you're talking about in terms of black levels and pixel offsets. And I know that this thing outputs 8bit -- that's fine by me. I basically just need to play with it to understand what's going on. If it's not good enough for production use, I'll still have an interesting camera for experimental shots, and high-speed capture.

CPU loading is no problem -- while capturing, I won't be touching the machine. My feeling is that while the processor will be getting a bit of a workout, it's really the harddrive that will be feeling the heat. Have you guys ever tested your USB2 cams at high speeds/datarates on laptops? Seems like something field scientists and sports guys would want as well as filmmakers...

Obin -- cool, sounds good. I'll whip up a PC version later today.

- ben

Steve Nordhauser · July 15th, 2004, 01:30 PM

Ben, OK, it sounds like you know what you are stepping in. One last thought - will you be using it in global shutter or rolling shutter mode? In global shutter mode, you integrate, then you read out. This means a very short integration time (low light sensitivity) if you want to keep the frame rate up. I've run ours at 24fps at the full 1280x1024 and had pretty stable images (rolling shutter or very short integration). We found that you need an ICH4,5,6 Intel host controller to get the data rates up. These are scarce on laptops. If you are using rolling shutter, you might want to consider a Micron-based camera.

Our single frame transfer rates are over 40MHz. This does not allow enough interframe communications time to control the sensor over USB, so we extend the vertical blanking time a bit since it is the overlap of communications and data that causes instability.

As long as you are not depending on this for production work - more as an experiment, go for it.

And for the sports people, we suggest gigabit ethernet for speed and the SI-640HF VGA 250fps camera. This has a much better global shutter - integrates while reading and has big pixels.

Eric Gorski · July 15th, 2004, 01:30 PM

<<<-- If you had 1GB of RAM free, shooting at 21MB/second, you could record to RAM for about 48 seconds. -->>>

could you really just record to ram?... if you could, it would be awesome ...

dell has a laptop for $999 that can hold 2gb or ram... that would be close to 2 minute takes... or what about a rack-mount server? they're really small and can hold up to 32gb! of ram.... that would be like a full 16mm mag?

check out these little rack mounts, they could go on someone's back no problem:

http://www1.us.dell.com/content/products/compare.aspx/rack_optimized?c=us&cs=04&l=en&s=bsd

Ben Syverson · July 15th, 2004, 01:57 PM

Steve,

I'm going to have to experiment with global vs rolling. I'd obviously rather go with global, but I'll have to see how much it impacts sensitivity.

Do you know if most of these software capturing apps allow direct capture to RAM? If so that may be the way to go -- even if it's not built-in, I could always create a RAM disk and record to that...

Thank you for the comment about I/O controllers. I checked it out, and it seems like at least two Sony lines, the v505 and the Z1 include the ICH4 (82801DB). Also, there seems to be a mobile variant ICH4-M, although I'm not sure how similar they are.

This info about the controller chips is making me lean towards the v505. You can get these pretty souped up.

- ben

Ben Syverson · July 15th, 2004, 02:26 PM

Okay, after a little research, it turns out that the ICH4-M is standard issue on most Centrino laptops. According to this page, "The desktop and mobile versions of the ICH4 are identical in terms of features, but the ICH4M supports deeper power saving states."

They're so common that even the tiny Sony U50 and U70 have ICH4-M's! Hmm... there are probably other bottlenecks (such as the 1ghz processor), but imagine strapping an HD camera to a U70 -- it would be smaller than my GL1! :)

Edit: The Dell Inspiron 600m also sports an ICH4 (but possibly not of the M variety).

Steve, one more thing: would a laptop based on the ADM Athlon 64 help at all with capturing? You can actually get a pretty good Athlon laptop for around $1000 -- although it's a massive 7.8 pounds. :)

- ben

Steve Nordhauser · July 15th, 2004, 03:06 PM

Ben,
That might be pretty camera dependent. I would talk to that *other* camera vendor you have been cavorting with. For us, it is the chipset, first, then processor speed.

Ben Syverson · July 15th, 2004, 05:29 PM

:) Fair enough, Steve. Thank you again for your input and thoughts. I'm looking forward to your Altasens-based camera. Any chance you'll throw an IEEE-1394b port on there? I hear it's a relatively easy interface to develop for, and it's got all the throughput you could use...

Juan M. M. Fiebelkorn · July 15th, 2004, 05:36 PM

Obin,
Here is all you need to get the best quality from your Bayer Camera.

http://www.insflug.org/raw/software/tools/dcraw.php3

Les Dit · July 15th, 2004, 05:38 PM

Ben, There is at least one laptop company that has dual drive raid built in.
I forget the name, but I'm sure you can find it !
It's a smaller company... like alienware... but not them

-Les

Ben Syverson · July 15th, 2004, 05:43 PM

Thanks Les! I'll google it. Although I don't think the hard drive will be a limiting factor, since I've seen real-world 3rd party benchmarks for the Hitachi coming in at 31MB/sec sustained. Hopefully it will be able to reliably handle 21 or 22 MB/sec...

Obin Olson · July 15th, 2004, 07:14 PM

Ben if your going to shell out money like I did for a total shot in the dark..look around I found an IBIs5 camera for around $600 - I think your $1,000 is a little high for that chip/camera

anyway I REAALLLLYY want to try your after effects filter! email it my way when you get a chance

Juan:
is that a good bayer utility? I have heard it's not that good of a quality..input?

Juan M. M. Fiebelkorn · July 15th, 2004, 07:31 PM

Really good!!! Have you looked at the image comparison against digital photo cameras internal demosaicking algos??
I don't know why people think it is of low quality.Maybe they use its preview option which uses a simple bilinear method.

I think the imput would need to be modified in order to accept a stream from your camera.

http://www.insflug.org/raw/analysis/...fvu/crops.php3

Ben Syverson · July 16th, 2004, 01:42 AM

Hey everyone,

I've released the first beta version of linBayer, my bayer filter de-mosaicing plug-in. It's available for PC and Mac, and should work in just about any AE host, such as FCP or Premiere (though I haven't tried). Of course, in After Effects, it works in 16bit.

You can use it to de-mosaic B&W images straight from the camera, or to further clean up RGB images that have been de-mosaiced poorly.

Just visit http://www.bensyverson.com/software/plugins/linbayer/!

- ben

Juan M. M. Fiebelkorn · July 16th, 2004, 02:51 AM

Ben,
I'm not trying to destroy you work or anything, but DCRAW gives far better results.Don't know about its speed.
An idea.Couldn't you port it to AE, Premiere,etc,etc??

Ben Syverson · July 16th, 2004, 03:19 AM

Juan -- I can't seem to get dcraw to work. It doesn't like Obin's files, and it doesn't like the Sumix sample files. It keeps saying "unsupported file format." If you have it working, post a 200% crop from my software and from dcraw. Until I see the same image interpreted by linBayer and their software, I can't evaluate what we could do differently.

I don't think there's any point to altering dcraw for our uses -- it's specifically designed for digital still cameras, so it supports a lot of bayer modes we'll never ever need, and deals with EXIF metadata, etc.

July 15th, 2004, 01:30 PM	#767
Steve Nordhauser Silicon Imaging, Inc. Join Date: May 2004 Location: Troy, NY USA Posts: 325	Ben, OK, it sounds like you know what you are stepping in. One last thought - will you be using it in global shutter or rolling shutter mode? In global shutter mode, you integrate, then you read out. This means a very short integration time (low light sensitivity) if you want to keep the frame rate up. I've run ours at 24fps at the full 1280x1024 and had pretty stable images (rolling shutter or very short integration). We found that you need an ICH4,5,6 Intel host controller to get the data rates up. These are scarce on laptops. If you are using rolling shutter, you might want to consider a Micron-based camera. Our single frame transfer rates are over 40MHz. This does not allow enough interframe communications time to control the sensor over USB, so we extend the vertical blanking time a bit since it is the overlap of communications and data that causes instability. As long as you are not depending on this for production work - more as an experiment, go for it. And for the sports people, we suggest gigabit ethernet for speed and the SI-640HF VGA 250fps camera. This has a much better global shutter - integrates while reading and has big pixels. __________________ Silicon Imaging, Inc. We see the Light! http://www.siliconimaging.com

July 15th, 2004, 03:06 PM	#771
Steve Nordhauser Silicon Imaging, Inc. Join Date: May 2004 Location: Troy, NY USA Posts: 325	Ben, That might be pretty camera dependent. I would talk to that other camera vendor you have been cavorting with. For us, it is the chipset, first, then processor speed. __________________ Silicon Imaging, Inc. We see the Light! http://www.siliconimaging.com

July 16th, 2004, 01:42 AM	#778
Ben Syverson Major Player Join Date: Mar 2004 Location: chicago Posts: 434	linBayer released! Hey everyone, I've released the first beta version of linBayer, my bayer filter de-mosaicing plug-in. It's available for PC and Mac, and should work in just about any AE host, such as FCP or Premiere (though I haven't tried). Of course, in After Effects, it works in 16bit. You can use it to de-mosaic B&W images straight from the camera, or to further clean up RGB images that have been de-mosaiced poorly. Just visit http://www.bensyverson.com/software/plugins/linbayer/! - ben

July 15th, 2004, 01:13 PM	#766
Ben Syverson Major Player Join Date: Mar 2004 Location: chicago Posts: 434	Steve, It's my best shot right now, unless you have something that can operate over USB2 or Firewire at 24fps with 1280x720 res. As far as I can tell, the Sumix is pretty much the only thing on the market that can meet those requirements. And it costs around $1000, software included. One thing -- why is the SI-1280F-U (heh) limited to 20fps at 1280x1024? That's only 25 megabytes/sec, while USB2 should be able to handle a good deal more than that... Can you do 24fps 1280x720 on it, at 21megs/sec? I know exactly what you're talking about in terms of black levels and pixel offsets. And I know that this thing outputs 8bit -- that's fine by me. I basically just need to play with it to understand what's going on. If it's not good enough for production use, I'll still have an interesting camera for experimental shots, and high-speed capture. CPU loading is no problem -- while capturing, I won't be touching the machine. My feeling is that while the processor will be getting a bit of a workout, it's really the harddrive that will be feeling the heat. Have you guys ever tested your USB2 cams at high speeds/datarates on laptops? Seems like something field scientists and sports guys would want as well as filmmakers... Obin -- cool, sounds good. I'll whip up a PC version later today. - ben

July 15th, 2004, 01:30 PM	#768
Eric Gorski Regular Crew Join Date: Jun 2004 Location: Western Oregon Posts: 138	<<<-- If you had 1GB of RAM free, shooting at 21MB/second, you could record to RAM for about 48 seconds. -->>> could you really just record to ram?... if you could, it would be awesome ... dell has a laptop for $999 that can hold 2gb or ram... that would be close to 2 minute takes... or what about a rack-mount server? they're really small and can hold up to 32gb! of ram.... that would be like a full 16mm mag? check out these little rack mounts, they could go on someone's back no problem: http://www1.us.dell.com/content/products/compare.aspx/rack_optimized?c=us&cs=04&l=en&s=bsd

July 15th, 2004, 01:57 PM	#769
Ben Syverson Major Player Join Date: Mar 2004 Location: chicago Posts: 434	Steve, I'm going to have to experiment with global vs rolling. I'd obviously rather go with global, but I'll have to see how much it impacts sensitivity. Do you know if most of these software capturing apps allow direct capture to RAM? If so that may be the way to go -- even if it's not built-in, I could always create a RAM disk and record to that... Thank you for the comment about I/O controllers. I checked it out, and it seems like at least two Sony lines, the v505 and the Z1 include the ICH4 (82801DB). Also, there seems to be a mobile variant ICH4-M, although I'm not sure how similar they are. This info about the controller chips is making me lean towards the v505. You can get these pretty souped up. - ben

July 15th, 2004, 02:26 PM	#770
Ben Syverson Major Player Join Date: Mar 2004 Location: chicago Posts: 434	Okay, after a little research, it turns out that the ICH4-M is standard issue on most Centrino laptops. According to this page, "The desktop and mobile versions of the ICH4 are identical in terms of features, but the ICH4M supports deeper power saving states." They're so common that even the tiny Sony U50 and U70 have ICH4-M's! Hmm... there are probably other bottlenecks (such as the 1ghz processor), but imagine strapping an HD camera to a U70 -- it would be smaller than my GL1! :) Edit: The Dell Inspiron 600m also sports an ICH4 (but possibly not of the M variety). Steve, one more thing: would a laptop based on the ADM Athlon 64 help at all with capturing? You can actually get a pretty good Athlon laptop for around $1000 -- although it's a massive 7.8 pounds. :) - ben

July 15th, 2004, 05:29 PM	#772
Ben Syverson Major Player Join Date: Mar 2004 Location: chicago Posts: 434	:) Fair enough, Steve. Thank you again for your input and thoughts. I'm looking forward to your Altasens-based camera. Any chance you'll throw an IEEE-1394b port on there? I hear it's a relatively easy interface to develop for, and it's got all the throughput you could use...

July 15th, 2004, 05:36 PM	#773
Juan M. M. Fiebelkorn Major Player Join Date: Jun 2004 Location: Buenos Aires , Argentina Posts: 444	Obin, Here is all you need to get the best quality from your Bayer Camera. http://www.insflug.org/raw/software/tools/dcraw.php3

July 15th, 2004, 05:38 PM	#774
Les Dit Major Player Join Date: Oct 2003 Location: Southern Cal-ee-for-Ni-ya Posts: 608	Ben, There is at least one laptop company that has dual drive raid built in. I forget the name, but I'm sure you can find it ! It's a smaller company... like alienware... but not them -Les

July 15th, 2004, 05:43 PM	#775
Ben Syverson Major Player Join Date: Mar 2004 Location: chicago Posts: 434	Thanks Les! I'll google it. Although I don't think the hard drive will be a limiting factor, since I've seen real-world 3rd party benchmarks for the Hitachi coming in at 31MB/sec sustained. Hopefully it will be able to reliably handle 21 or 22 MB/sec...

July 15th, 2004, 07:14 PM	#776
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	Ben if your going to shell out money like I did for a total shot in the dark..look around I found an IBIs5 camera for around $600 - I think your $1,000 is a little high for that chip/camera anyway I REAALLLLYY want to try your after effects filter! email it my way when you get a chance Juan: is that a good bayer utility? I have heard it's not that good of a quality..input?

July 15th, 2004, 07:31 PM	#777
Juan M. M. Fiebelkorn Major Player Join Date: Jun 2004 Location: Buenos Aires , Argentina Posts: 444	Really good!!! Have you looked at the image comparison against digital photo cameras internal demosaicking algos?? I don't know why people think it is of low quality.Maybe they use its preview option which uses a simple bilinear method. I think the imput would need to be modified in order to accept a stream from your camera. http://www.insflug.org/raw/analysis/...fvu/crops.php3

July 16th, 2004, 02:51 AM	#779
Juan M. M. Fiebelkorn Major Player Join Date: Jun 2004 Location: Buenos Aires , Argentina Posts: 444	Ben, I'm not trying to destroy you work or anything, but DCRAW gives far better results.Don't know about its speed. An idea.Couldn't you port it to AE, Premiere,etc,etc??

July 16th, 2004, 03:19 AM	#780
Ben Syverson Major Player Join Date: Mar 2004 Location: chicago Posts: 434	Juan -- I can't seem to get dcraw to work. It doesn't like Obin's files, and it doesn't like the Sumix sample files. It keeps saying "unsupported file format." If you have it working, post a 200% crop from my software and from dcraw. Until I see the same image interpreted by linBayer and their software, I can't evaluate what we could do differently. I don't think there's any point to altering dcraw for our uses -- it's specifically designed for digital still cameras, so it supports a lot of bayer modes we'll never ever need, and deals with EXIF metadata, etc.