Jason, what are your notes on my algorithm? The green channel has a little zippering left over, but if you get rid of it completely, you lose a lot of information. At 100%, it's almost impossible to see the zippering, and I think once it was projected or on a screen, you wouldn't be able to see it at all...

Ben
Be a little careful about the sample images, as they were probably taken at the exposure that would result in the best still, rather that 1/48 sec, or whatever you want to shoot video with.
Ask them what shutter speed they used. Maybe it was 1/200 sec to minimize noise?
-Les

I think you mean 1/2 second. (?) That's impossible -- the maximum exposure time is 36.44ms. I'm sure they had the subjects well-lit, but so will I. :)

Basically, I need a camera that will operate over USB2 or Firewire. This is it. And it's extremely inexpensive. I'll give it a shot and keep people updated on how it fares. It costs less than half of what my GL1 did, so even if it's really noisy, it'll be an interesting experiment. It's not like I'm going to be handing this thing to Bill Pope, ASC to shoot my movies. :) I do have some really sweet C-mount B&H - Angenieux primes, though, left over from 16mm days -- the 25mm is f0.95! Hottt.

My thinking is that if I window the camera readout to 1280x720 and drop the frame rate to 24fps, I may be able to reduce the datarate enough to record directly to a laptop with one of those 7200rpm 60gb drives. We shall see. Even at full res and full frame rate, it's only 38 megabytes a second (it's 8 bit).

I think it'll be interesting to have some variety on this forum -- both the SI-1300 and the SMX150...

Oh, I should mention for the sake of comparison, that 36.44 milliseconds (0.03644) is rather close to 1/48 of a second, which is 20.83 milliseconds (0.0208333...).

We shall see how adjustable the exposure time is.

Obin,
Good luck in developing that software. Wicked good stuff man. Obviously I'd have to first own a sensor and frame grabber to even be considered for beta testing. I'm gonna have to come to a decision and get one soon. Then I can freaking contribute.

by the way, the "zippering" that ben pointed out in those images of the thumb pins is exactly what I was trying to explain about the image from your camera once it was projected on a big screen. Of course, it seems that this will not be a problem anymore. You need to get on recompressing us a video with no image doubling and no zippering! Heeyah! (Whip cracks!)

Ben,
Awesome that you're biting the bullet and getting that sensor. Best of luck to you as well. It would really be something if you could get a Smooth 24fps over USB2. Even if you end up stepping down to a resolution just below 1280x720 to get solid capture performance, it may prove well worth it.
If any of those nice lenses you have are anamorphic, you could always consider doing a 960x720 window from the SMX-150C and stretching it back out to 1280x720 in post. Or shoot some cinemascope 1280x480 stuff!

I have adobe after effects and a PC, and since I definitely plan to get one of these cameras, it would be great if you could compile that filter for those of us poor souls who can't 'think different'.

Matthew, I'm definitely going to try some anamorphic stuff! I've got a 16:9 adapter and a 2.35 adapter that I hope to test out in 1280x1024 mode. It'll be very interesting to see how that comes out.

The only thing I'm wrestling with now is finding a laptop that can handle the data. I just ordered the 7200rpm Hitachi Travelstar 7K60 (http://www.hitachigst.com/hdd/support/7k60/7k60.htm) drive from Memorylabs.net (http://www.memorylabs.net/) for pretty cheap ($189). Now I have to figure out what laptop to put it into. :)

One idea I had was to put the Hitachi in a USB2 enclosure, and bring it and the camera (and the software) along with me to a computer store. There I could actually test out various models. However, this doesn't test the most crucial thing I need to know: the machine's real-world ATA throughput. Once the drive is inside, the performance will probably be very different than running off the USB2. I've seen benchmarks for the Hitachi clock in at around 31 MB/sec sustained, so hopefully it will be able to handle ~21 MB/sec. (1280 * 720 * 24 / 1024 / 1024 = 21.09 MB/sec)

Anybody have any ideas or suggestions on high-performance laptops that weigh 5 pounds or less? I've looked at Dell's Inspirion 600m (http://www1.us.dell.com/content/products/productdetails.aspx/inspn_600m) -- the specs look good, but as a die-hard Mac user, I don't know if I can bring myself to buy something so ugly. I've also looked at the Sony V505 series (http://www.sonystyle.com/is-bin/INTERSHOP.enfinity/eCS/Store/en/-/USD/SY_BrowseCatalog-Start;sid=rCg9Eaa_x1A9leZJsYs3GumwUh-h1irjxCY=?CategoryName=cpu_VAIONotebookComputers_V505Series&Dept=cpu_VAIONotebookComputers), but they're a little pricey.

I guess I should just bite the bullet -- even at $1700 for a laptop, I'm still looking at around $3000 for a portable uncompressed HD rig. Not bad if it works! :)

You know, another idea is to not sweat the hard drive throughput so much. If you had 1GB of RAM free, shooting at 21MB/second, you could record to RAM for about 48 seconds. It wouldn't be the most convenient thing in the world, but in all honesty, most of my shots don't last longer than 48 seconds...

Ok we need a good bayer filter..anyone have an a link to somthing good?

Gang, I spoke with a film-out house in NYC, they are saying that most of the work they do is 1080p filmout!! They could do 4k but MOST stuff is 1080p! so that means with the new Rockwell chip at a NATIVE 1080p we can shoot right to film no problem! this is kIckIN it! they also told me the 1280stuff will be very clear and work GREAT for a big theater projection. Filmout only needs 8bit ;) just a FYI for you guys!

Hi everyone this movie was shot @ 1080p
from a famous director .

It will be out august 6th.





http://www.collateral-themovie.com/

what camera?

- wait - SHOT? or you mean filmout @ 1080p

Obin, did you check out my results?

Wayne,
about creating camacases, lets email together next time. At the moment i need a working inside.

- wait - SHOT? or you mean filmout @ 1080p

It was shot at 1080p using a Sony F950 and recorded to HDCAM.

I'm not sure what the filmout was but I have heard they did all the color correction at 2K, it's possible they filmed out at 2K. Star Wars filmed out at 2K, so do the Pixar movies (in fact, Pixar claims to have the ability to film out at 4K, but says they don't bother because there is absolutely nothing to be gained by doing it).

By the time you get to a theatrical release print, which is many generations removed from the negative, no one in the world will be able to sit in a cineplex and tell the difference between 2k and 4k...

the movie was shot with a viper 3 chip 1080p filmstream @12bit
2.37 aspect ratio
It was outputted to 35mm film . I can't wait to see this on the big screen.

Ben:
I don't like to talk about competitor's products so I will talk about a very similar product that we sell. This would be an IBIS-5 based USB 2.0 camera - the SI-1280F-RGB-USB. First, research the IBIS-5 info on this site. Also, be sure you know what bit depth you are working with - 8, 10 or 12. I think the internal A/D might be 10 but it isn't very good - we use an external 12 bit for less smearing. We consider the IBIS-5 (even the new A sensor) to be unacceptable for anything other than high speed and machine vision. Also, although we run our USB cameras at 40-48MHz under certain conditions, it is fairly CPU loading sensitive so if you do much on the PC you can drop frames. Some OEMs go this fast but don't worry about some corrupted frames. Also, the IBIS-5 is not very sensitive - OK, it is an insensitive lout. That is because so much of the pixel is used for the global shutter transistors. There is also a fair amount of global shutter leakage (1.5%) when the electronic shutter is off - this can lead to smearing of bright objects.

The sensor also has quite a bit of single pixel and column offset. You can correct this by doing a dark field correction (also called offset correction) and some other post processing but this eats up your dynamic range. I you start with 256 levels and have to subtract a value of 60 to reach black, you have less than levels to the pixel.

Maybe Sumix has done something magical here, but I would suggest an evaluation period.

Ben I want to try you filter ASAP in after effects on our footage
can you write it for PC AfterEffects?

Steve,

It's my best shot right now, unless you have something that can operate over USB2 or Firewire at 24fps with 1280x720 res. As far as I can tell, the Sumix is pretty much the only thing on the market that can meet those requirements. And it costs around $1000, software included. One thing -- why is the SI-1280F-U (heh) limited to 20fps at 1280x1024? That's only 25 megabytes/sec, while USB2 should be able to handle a good deal more than that... Can you do 24fps 1280x720 on it, at 21megs/sec?

I know exactly what you're talking about in terms of black levels and pixel offsets. And I know that this thing outputs 8bit -- that's fine by me. I basically just need to play with it to understand what's going on. If it's not good enough for production use, I'll still have an interesting camera for experimental shots, and high-speed capture.

CPU loading is no problem -- while capturing, I won't be touching the machine. My feeling is that while the processor will be getting a bit of a workout, it's really the harddrive that will be feeling the heat. Have you guys ever tested your USB2 cams at high speeds/datarates on laptops? Seems like something field scientists and sports guys would want as well as filmmakers...

Obin -- cool, sounds good. I'll whip up a PC version later today.

- ben

Ben, OK, it sounds like you know what you are stepping in. One last thought - will you be using it in global shutter or rolling shutter mode? In global shutter mode, you integrate, then you read out. This means a very short integration time (low light sensitivity) if you want to keep the frame rate up. I've run ours at 24fps at the full 1280x1024 and had pretty stable images (rolling shutter or very short integration). We found that you need an ICH4,5,6 Intel host controller to get the data rates up. These are scarce on laptops. If you are using rolling shutter, you might want to consider a Micron-based camera.

Our single frame transfer rates are over 40MHz. This does not allow enough interframe communications time to control the sensor over USB, so we extend the vertical blanking time a bit since it is the overlap of communications and data that causes instability.

As long as you are not depending on this for production work - more as an experiment, go for it.

And for the sports people, we suggest gigabit ethernet for speed and the SI-640HF VGA 250fps camera. This has a much better global shutter - integrates while reading and has big pixels.

<<<-- If you had 1GB of RAM free, shooting at 21MB/second, you could record to RAM for about 48 seconds. -->>>

could you really just record to ram?... if you could, it would be awesome ...

dell has a laptop for $999 that can hold 2gb or ram... that would be close to 2 minute takes... or what about a rack-mount server? they're really small and can hold up to 32gb! of ram.... that would be like a full 16mm mag?

check out these little rack mounts, they could go on someone's back no problem:

http://www1.us.dell.com/content/products/compare.aspx/rack_optimized?c=us&cs=04&l=en&s=bsd

Steve,

I'm going to have to experiment with global vs rolling. I'd obviously rather go with global, but I'll have to see how much it impacts sensitivity.

Do you know if most of these software capturing apps allow direct capture to RAM? If so that may be the way to go -- even if it's not built-in, I could always create a RAM disk and record to that...

Thank you for the comment about I/O controllers. I checked it out, and it seems like at least two Sony lines, the v505 and the Z1 include the ICH4 (82801DB) (http://www.intel.com/design/chipsets/datashts/290744.htm). Also, there seems to be a mobile variant ICH4-M (http://www.intel.com/design/mobile/datashts/252337.htm), although I'm not sure how similar they are.

This info about the controller chips is making me lean towards the v505. You can get these pretty souped up.

- ben

Okay, after a little research, it turns out that the ICH4-M (http://www.intel.com/design/mobile/datashts/252337.htm) is standard issue on most Centrino laptops. According to this page (http://tech-report.com/reviews/2003q3/pentiumm-1.4ghz/index.x?pg=1), "The desktop and mobile versions of the ICH4 are identical in terms of features, but the ICH4M supports deeper power saving states."

They're so common that even the tiny Sony U50 and U70 (http://www.dynamism.com/u70/) have ICH4-M's! Hmm... there are probably other bottlenecks (such as the 1ghz processor), but imagine strapping an HD camera to a U70 -- it would be smaller than my GL1! :)

Edit: The Dell Inspiron 600m also sports an ICH4 (but possibly not of the M variety).

Steve, one more thing: would a laptop based on the ADM Athlon 64 help at all with capturing? You can actually get a pretty good Athlon laptop for around $1000 -- although it's a massive 7.8 pounds. :)

- ben

Ben,
That might be pretty camera dependent. I would talk to that *other* camera vendor you have been cavorting with. For us, it is the chipset, first, then processor speed.

:) Fair enough, Steve. Thank you again for your input and thoughts. I'm looking forward to your Altasens-based camera. Any chance you'll throw an IEEE-1394b port on there? I hear it's a relatively easy interface to develop for, and it's got all the throughput you could use...

Obin,
Here is all you need to get the best quality from your Bayer Camera.


http://www.insflug.org/raw/software/tools/dcraw.php3

Ben, There is at least one laptop company that has dual drive raid built in.
I forget the name, but I'm sure you can find it !
It's a smaller company... like alienware... but not them

-Les

Thanks Les! I'll google it. Although I don't think the hard drive will be a limiting factor, since I've seen real-world 3rd party benchmarks for the Hitachi coming in at 31MB/sec sustained. Hopefully it will be able to reliably handle 21 or 22 MB/sec...

Ben if your going to shell out money like I did for a total shot in the dark..look around I found an IBIs5 camera for around $600 - I think your $1,000 is a little high for that chip/camera

anyway I REAALLLLYY want to try your after effects filter! email it my way when you get a chance


Juan:
is that a good bayer utility? I have heard it's not that good of a quality..input?

Really good!!! Have you looked at the image comparison against digital photo cameras internal demosaicking algos??
I don't know why people think it is of low quality.Maybe they use its preview option which uses a simple bilinear method.

I think the imput would need to be modified in order to accept a stream from your camera.

http://www.insflug.org/raw/analysis/dcrawvsfvu/crops.php3

Hey everyone,

I've released the first beta version of linBayer, my bayer filter de-mosaicing plug-in. It's available for PC and Mac, and should work in just about any AE host, such as FCP or Premiere (though I haven't tried). Of course, in After Effects, it works in 16bit.

You can use it to de-mosaic B&W images straight from the camera, or to further clean up RGB images that have been de-mosaiced poorly.

Just visit http://www.bensyverson.com/software/plugins/linbayer/!

- ben

Ben,
I'm not trying to destroy you work or anything, but DCRAW gives far better results.Don't know about its speed.
An idea.Couldn't you port it to AE, Premiere,etc,etc??

Juan -- I can't seem to get dcraw to work. It doesn't like Obin's files, and it doesn't like the Sumix sample files. It keeps saying "unsupported file format." If you have it working, post a 200% crop from my software and from dcraw. Until I see the same image interpreted by linBayer and their software, I can't evaluate what we could do differently.

I don't think there's any point to altering dcraw for our uses -- it's specifically designed for digital still cameras, so it supports a lot of bayer modes we'll never ever need, and deals with EXIF metadata, etc.

Ok, as you like.I'm not trying to start a war or say your filter doesn't work :).
If you think it is not suitable for this material, it's O.K. for me.
I have a couple of problems now that won't let me do anything with the machine I'm using.
Could you give me .Tiff frame from a Bayer camera to work with?


Edit:
Oh well, lastly I'm not sure wich image format it opens, so I guess it should open tiff, as some cameras output this format.
Anyway don't worry about what I say, cause I have no interest at this moment about modifying it to accept anything.

DCRAW only works with RAW camera files. Those Canon "TIFF" files, are in fact raw files, not true TIFF files, the extension is quite misleading there.

My main gripe with DCRAW is the zippering effect on the edges (look at the watch-you have to mouse over the image to see the DCRAW output), and the color moiré that also occurs on highlights, etc. If you don't believe me about the results of DCRAW, just go over to www.dpreview.com, and do a search on their forums about DCRAW, and you'll see the same complaints (mainly color moiré is the biggie).

If you'll notice on the Canon output, there are no zippered edges, and yes, while the DCRAW dynamic range is better, the actual interpolation I don't believe is better than what Canon provides in their SDK.

BTW, here's an idea. Can you guys reverse engineer from the DCRAW software a canon .CRW file, so that with Rob's program, you can then use the Canon tools to interpolate the files (because it thinks it's from a Canon camera)?

I will be testing that plugin today and have a report later on ;)

I hope that it's good enough to use for the filmout we are going to do with this footage soon ;)

First of all, i'm sorry if anyone else has raised this already, so please forgive me if this has been covered already,

BUT......

On the problem of storage, i was thinking why not use solid state?

Whatya guys thinks?

here is a quick google brought up:

E- Disk(R) SAN 48 FLASH SSD

Up to 2.5 million IOPS
Up to 16GB/sec Sustained Read Rate
Up to 9GB/sec Sustained Write Rate
48U cabinet: 295GB - 16.5TB
96 x 2Gbit (200MB/s) host ports
Remote/Local Software Management

And other such products


http://www.bitmicro.com/products_enterprise_net_ssd.php

please reply soon, i would like your take on this

Anhar Miah wrote:
On the problem of storage, i was thinking why not use solid state?It certainly looks good, the main issue would be cost.

i'm hoping more searching may bring up cheaper/ budget options.

Like i said earlier that was a quick search, i'll look later, if anyone else knows of any cheap options please tell us about it!!


Questions:

(a)What compression schemes are you guys considering?
(b)Would the CPU be fast enough to do compression in real-time before writing to disk, if so generally how fast a CPU would be appropiate (3.0 GHz?)

thanks,

(again sorry if these have been asked before)

Yes a 3GHz P4 is fast enough. We have been experimented with direct 10/12bit bayer compression and achieved compression frame rate of around 60fps using 1280x720 sources using a single P4 with a fast memory bus (standard these days.) For 1920x1080 we expect the encoding rate to be around 30fps. These figures were gathered from a variant of the CineForm HD (CFHD) codec used within Aspect HD and Prospect HD. We are investigating the marketability of this type product.

Ben did a good job. Check the images out:

http://www.dv3productions.com/test_images/ben_test_on.jpg
http://www.dv3productions.com/test_images/ben_test_off.jpg

Juan -- no worries, I was just trying to understand what you were saying.

Jason: I looked at the dcraw source, and it looks like they've documented the bayer patterns and file formats of each camera's RAW file pretty clearly. It should be a trivial effort to generate CRW's from any image -- bayer mosaic or not. I guess my question is: why? Is the Canon interpolation amazing enough to go through the effort?

Obin: thanks man! If you guys want to source to integrate into your app, let me know (it's pretty basic).

- ben

Guys, I need a quick answer :D

What output format does the capture software output?
Can I get an AVI with the RAW video inside???
I'm trying to make a proof of concept codec with some guy, and we need to know what we are working with.

What do you think about using MXF as container?

our capture software will support output of avi tiff jpeg maybe cineon - we will use any of the compressors for avi it would be nice to have 10bit native if someone has any ideas?

I don't know about AVI, but in Quicktime, there are a few options:
Blackmagic 10-bit (http://www.blackmagic-design.com/)
Microcosm from Digital Anarchy (http://www.digitalanarchy.com/product_micro.html)
None16 from Digital Anarchy (http://www.digitalanarchy.com/micro/micro_none16.html)

Juan, why do you need the files in AVI format if you're designing a new codec?

Obin,

Well that's the idea ;)

Ben,
Because it is a VFW codec.Just that :D
AVI is still the easiest container to work with under Windows environment , although it has a little overhead.
Codecs and containers are different things.
After some more testings, we get realtime 4.5:1 lossless compression for Green channel for a 1920x1080 Bayer.
Nice, isn't it?

The idea is: the same codec would support RAW Bayer Lossles and Near-Lossles (A.K.A. not so lossy), RGB and YUV 4:4:4 formats.
This way everybody could go NLE with the same codec the camera uses.
Ah, it is only meant for 1920x1080.

The problem with VfW codec is they can only export 8bit data in RGB, no YUV support and no deeper color supported. You should consider doing a DirectShow codec as it allows you to define custom pixel formats. I used the FOURCC codes BYR1 and BYR2. BYR1 supports 8bit packed raw bayer data from DirectShow and BYR2 is 16bit packed (with 12bit precision). If you use these FOURCC codes our codecs would be interchangable.

David, we want to support your codec for capture..can you provide anything?

Juan I see NO solution for editing 10bit unless David and CineForm jump in soon...I think if we could capture in 10bit compressed and then transcode over at 8bit when the NLE does it's renders etc that would be fine. This way we can open the 10bit compressed files in after effects and combustion for color work save 8bit and edit.

The biggest problem at the moment is if we want 10bit we must save 16bit tiff files and they are HUGE..no need for this at all..we need a codec/format that is a 10bit lossless compressed MASTER. Later in the editing pipeline it can be 8bit...

Juan -- when you say 4.5:1 lossless in the green channel, do you mean 4.5:1 in terms of the original pixel values, or the final image channel?

I ask because it's relatively easy to create a codec that stores only the G where it appears on the sensor. That could be considered 2:1 compression. Then apply standard lossless compression to that (which ranges from 2:1 to 2.5:1) and you have 4-4.5:1. That must be what you're doing, because I don't believe true 4.5:1 lossless image compression is possible at this time (at least not at realtime).

It's worth pointing out that bayer images have a "compression" ratio of 3:1 -- they represent the entire RGB in one B&W image. Add standard RLE or Huffman lossless compression, and you should be able to get it to around 5:1 or 6:1.

That would be an incredibly easy codec to write.

By the time you get to a theatrical release print, which is many generations removed from the negative, no one in the world will be able to sit in a cineplex and tell the difference between 2k and 4k...

Exactly.

the movie was shot with a viper 3 chip 1080p filmstream @12bit

Really? I had heard CineAlta.

I've seen the trailer on the big screen, and it looks great. Can't wait to see the final print. Wonder why Mann chose to shot HD?

Obin, I've come around on Cineon. The most commonly-used Cineon format is 10-bit, and you can use it to store linear values. I don't know of any NLE's that support Cineon, but it certainly works with AE, Shake, etc.

For working in an NLE, the workflow would ideally be something like this:

1. Capture to TIFFs/RAWs/Cineons/whatever.
2. Convert files to AVI or Quicktime movies that use a lossless 10bit codec.
3. Also convert the files to DV or some more easily editable format.
4. Edit the movie in DV.
5. When you're done, replace the DV footage with the lossless 10bit HD movies. Most NLEs will let you do this automatically (without manually replacing each clip).

However, if you're using a good enough lossless 10 bit codec, maybe this wouldn't be necessary. If we store the images in the codec in their B&W bayer state, and do lossless compression on them, we should be able to get at least 5:1. And that may be small + fast enough to edit in 10bit uncompressed HD in realtime, without requiring RAID drives.

For example, 720p at 10bit RGB and 24fps takes up 79.1MB/sec. Cut that down by 3:1 if we just store the bayer: 26.36MB/sec. Cut that down by 1.5:1 (lets be conservative -- we could get as much as 2:1), and you get 17.57MB/sec. Now, 17.5MB/sec is not that much of a strain on anyone's computer for read/write. If you have a fast (7200 or 10000rpm) drive, you could probably get two uncompressed 10bit HD streams going at once!

Man, maybe I should look at the Quicktime SDK. I wonder how difficult it is to write a codec...

I'd second Ben's reply here.

Cineon/DPX is a SMPTE standard file format, and it can store linear values, you don't have to have a 10-bit log file, but if you're trying to get 12-bit vaules into a 10-bit file, then you want to use logarithmic encoding to preserve the entire dynamic range of the image, and not clip off values in the least significant bits.

I'm looking at building at QuickTime codec component right now... It looks fairly complex, but I'll see if I can hack my way through it.

- ben