I'm pretty sure that 23.976 or 23.977 will be good enough. But
then again, I ain't in NTSC land and we have rock solid frame
rates here at exact 25 fps for example.

Steve & Rob: we are all on the same line in regards to first
seperating the planes and doing compression on them seperate.

I've just written a TIFF to RAW Bayer conversion program so we
can now test with any 16 bit TIFF basically. This gives us some
freedom. The program is in Rob S's posession as well.

Wayne: I did my first programming in low-level assembler before
moving to C(++) and then on to the Windows platform. I'm pretty
sure I know exactly how a PC works internally and how Windows
works as well. I've written assembler boot-loaders and some
low-level Windows stuff. The only thing I ain't really good at is
anything with Unix/Linux on the PC. Oh well...

Steve: do you happen to know if the 16-bit bytes are coming
in BIG or LITTLE endian order? Or do we get the high byte first
or the low?

A question regarding future camera's. Will they be controllable
and connectable in a similar fashion? In other words minor
changes to the software?

ok streampix works well..I shot 3500 frames in one set today at 12bit 24fps..this very good...workflow from stream pix is hell and goes like this:

1 Open xcap and set the camera exposure frame period gain ROI and mhz for each and every shot we shoot, save fmt camera file from xcap
2 Open streampix and load fmt file so that the camera is ready for the shot
3 Make new file for sequence
4 Shoot
5 Open shot black and white sequence with streampix
6 Load Bayer converter filter and make sure files will output in the right place with the right Bayer RGGB
7 Save sequence to disk for a long long time as the black and white sequence is being converted to color sequence
8 Open HUGE color sequence in streampix and output tiff files or avi or jpeg and wait for a very long time
9 Open what you have just saved and compress with some codec like SheerVideo so you can edit and finish project

almost all steps above should be this:

1 Open stream pix and in streampix set camera for shot incl. shutter fps etc
2 Shoot to sequence file
3 After shooting open sequence file and de-Bayer the image to color and save with a codec like SHeerVideo
4 Edit final footage

what's up with this board??

oh ya, almost forgot it takes over 45min to deal with the above files to get them into a SheerVIdeo codec quicktime compressed file! this is silly and will never work for professional work unless we can make fewer steps to get a sheervideo file

and don't forget the 22min it takes to transfer all the tiff files to the videoserver over gigabit lan

Obin,
In terms of LAN speed, we have found that streaming over a gigE lan is limited to about 250Mbps from one machine to another using the Windows drivers. Even the raw data for 3500 frames is 1280x720x16x3500 = 51Gbps
At 250Mbps, that should take 206 seconds or 3.5 minutes. That is 16 bits per pixel.

For bigger shoots, if you don't need a raid or maybe only a two drive, use removables (slide into a chassis slot right on the IDE bus, not firewire) to move the data around. Just watch the heat - I don't think the removables cool as well as the fixed.

Is anyone looking into building a simple control system that would send the footage as is to some stacked drives?

I'm following these threads the best I can and keep reading about Obin's plans to build a small miniATX machine, but why not just have SATA drives in an enclosure that's within reasonable size/weight/power consumption specs ala the Vance Cam?

What sort of intercept system would have to be built to catch the stream and compress it on the fly?

What I'm getting at is -- I'd rather capture footage to disk without lugging a computer around with the cam. I want to just get the footage on site and then bring it back to a machine for post.

Is this possible? If so, can it be done with our resources?

Also, I mentioned elsewhere that it would be neat if custom "looks" could be generated as small files -- like BIOS flash files -- that we could all trade online as filmmakers. Based on what I know of the project as is, this isn't really feasible given that the camera is technically already built -- we're just figuring out innovative ways to harness its footage.

To my mind it would be amazing to be able to assemble an interface that would apply different effects to the footage with a way to prerserve and select among settings. These prefs would be output to a small file, and could be downloaded and installed to the cam. In this way you could change the looks of your footage as easily as you do ringtones on your cell phone. A community of open-source mod'ers could build visual "filter" toolkits this way.

Steve -- what kind of control is possible for the ambitiout enduser with regards to the image's lattitude and other characteristics? Is it possible that they could be changed on the fly and plugins could be built for them?

Thanks for all contributing,

- jim

ouch..it says it will take 58min to convert 16bit tif file's into the beta sheer codec....guess this could be because it's on the network drive and or it's dealing with a high bit depth.


Jim your thinking big, and thats good, but we have very basic problems at the moment like what to do with all the data this camera spits out...how to deal with high bit depth images and the list goes on and on and on...arggghh I am getting tired of it! I wish we had a magic button for this!


what the heck am I doing with 16bit tiff files anyway? the 1300 camera outputs 10bit that gets captured at 12bit and converted to 16bit tiff...what the HECK is that about ?? anyone? why does After Effects edit in 8 or 16bit ONLY what about 10 and 12bit?


maybe we should stay with simple 8bit files...everything works with them....argghh

Obin on transfer time:
Don't do any processing on video on a network drive - move it locally and then back.

Jim on a control system:
A couple of things to watch. First is cost - PC motherboards are pretty cheap. If you are grabbing camera link, you could embed your own interface - not too complicated. How will you do the viewfinder? Pretty easy with a PC chassis. What about a smaller footprint like PC/104+? I know there are frame grabbers out there for it. Not extendable to 64 bit, as far as I know. There are some other standards for SBCs (single board computers) like this EBX format:

http://www.orbitmicro.com/products/e...370/EM370B.htm

It has PC/104+ so you could attach a frame grabber. There are many - don't get hung up on my first google search. You might find one with a RAID controller. Motherboard and PCI grabber will always be cheaper.

Jim on software filters:
It should be easy to do some kind of plugins in version 1.1 or so of the software. Color balance is just a 3x3 transformation table. I think photoshop lets you build your own filters by editing the table. Hey, why not use the standard photoshop plugins or whatever video editing tool uses plugins.?

Obin,

Well you already know the problems of 8bit, that is what you are trying to avoid. :)

Regard the 10 or 12bit issue. There is little point for applications to support a native 10 or 12 bit workflows when computers work better on 16bit. Our codec is 10bit yet our workflow is 16bit, this is completely normal. 10 or 12 bit data is simply left shifted to be processed as 16bit. This is then reversed when exporting back to the codec. The extra interim precision is a good thing.

sorta like filling your gas tank 3/4 the way full? so the image file is not bigger then a 10bit EVEN though it's 16bit?
David I sent you 2 cd-roms with some raw data on them, HTH

So we need to have a 16bit compression codec to edit with and a way to edit it in a normal edit system like premiere pro. We could do our color work in 16bit and save the files for editing in 8bit and then master in 8bit onto dvd HDDVD mpeg-2 HD DVCPROHD etc?

here is the card to get for this system

http://www.highpoint-tech.com/USA/rr1820a.htm

upto 1200MB/sec with 8 SATA disk drives and the card is under $200!!

Obin and Steve -- thanks for the responses.

I know I might be getting ahead of the game a bit, Obin, but I'm trying to think ahead conceptually so we can best accomodate certain design ideas from the outset. I find this helps when trying to avoid potential future pitfalls.

That said, I've a friend who's one of the PHP codifiers (not just a coder -- he helps standardize new PHP commands). He's got a lot of coding knowledge and is doing render/compression stuff for 3d environments at the moment. I'm hoping to persuade him that this project might be worth his time.

I've also taken the liberty of setting up a quick-n-dirty site of still images from the footage you've provided, Obin. Here it is.

I hope you don't mind -- I'm just trying to get the word out and I find the images are worth many words :D

Also, it helps for people who don't have WM9HD installed.

The site will be updated with condensed info and links to footage, images, etc. as I have time and as the results here merit.

Thanks again for the work you're all doing!

- jim

16 bits 12 bits 10 bits
 

I've been dealing with the bits problem for about 10 years.

For now, just to save disk space, I recommend using the Cineon ( fido ) format, it puts three 10 bit rgb code values into 32 bits ( 4 bytes ), so you only waste 2 bits per pixel.
That's for rgb demosaiked frames.
You could pack the bits in a similar fashion for the raw bayer image, and decode it later. It's all in the name of saving disk space.

Like I said before, LZ or RLE won't help much with > 8 bits.
But if you have to, the LZ Tiffs would crush out the wasted empty bits to give similar files sizes, but with more cpu to get there.

-Les

Okay, let's all slowdown a bit. Obin: I understand your frustration,
but this is what a lot of people have predicted and we know
would happen.

That's why we are trying to design a system that works like a
camera and not something with 20 steps to go through. As I
explained earlier, this is NOT going to happen overnight. Although
Rob S. and myself are working on details and testing things, we
both have jobs and Rob S. is still waiting on his camera.

Development of software, testing algorithms, optimizing, testing
systems and generating a workable platform takes time. Lots of
time. Where you expecting to be up and running in a couple of
weeks? That's just unrealistic.

David: is 10 or 12 bits always left justified? Is there a reason for
it being left instead of right?

Jim: the main problems is bandwidth / datarate and the system
to put inbetween the camera and the harddisk (or a set of
harddisks). Two programmers (Rob Scott and myself) are already
on the project so, yes, a middle system is being worked out but
don't expect the "magic button" (as Obin calls it) next week or so.

Les: Rob and myself pretty much agreed on doing no Bayer
conversion in the "camera" section. That takes way too much
time and resources to do good and we don't have that (we
must process a very large data stream).

So we are going to write the RAW stream coming from the camera
to disk in at least a packed form. This means that 10 bits per pixel
turn into 40 bits/5 bytes per 4 pixels instead of 8 bytes.

Storing it in Bayer form (10 bit) gives us a data reduction (not a
true reduction since this is what is coming from the camera, but
a reduction in comparison to full demosaiked RGB data) of 3:1.
This reduces bandwidth 3 times or 77%. Then we at least store
it packed which saves us 37.5%.

So for every 24 bytes (not bits) of RGB data we only store 5
which is a 4.8:1 compression or a reduction of 79.2%. This is
without loosing any quality (except for the Bayer filter, but that
you can't get without on single chip system either except for
a Foveon chip)

Since the camera is sending us raw Bayer at 16 bits this gives
us a 37.5% reduction or a 1.6:1 compression.

Rob S. and myself (and everyone is invited, ofcourse) are looking
into ways to futher LOSSLESS compress this. Rob has already
seen some codecs. I'm a bit worried about speed and we should
keep this as fast as possible (speed over compression) to deal
with the bandwidth (see below).

At the moment we are not looking at any compliant file because
I doubt there would be one for Bayer format (except RAW camera
files. Canon has CRW I think).

After it is recorded the signal must be dealt with on a computer:

1) convert from Bayer to full RGB with the best (selectable) algortihm

2) do things like white balancing / color correction (can only be done with the full RGB data)

3) write it out to a standard format / codec (perhaps with smaller resolution proxies [both in resolution and bit depth] to ease the editing)

Now I hear a lot of people think I thought the camera would do
white balance and color presets and whatnot. Don't count on it.
We probably don't have the processing power. Plugins for the
second phase of the capture process should be no problem.

The problem with that is, is that the data must be converted into
full RGB first before we can do anything with it. This also increases
our storage three-fold. What we might do is do some lower
resolution stuff for the viewfinder / monitor.

Instead of full bayer you simply make 1 pixel out of every macro
block that contains 4 pixels (2 horizontal, 2 vertical). This gives
you a 50% reduction in either resolution and a very fast de-bayer
algorithm. So a 1280x720 image becomes 640x360 which is more
easily displayed on a standard monitor as well. We might have
time to do a bit of color-shifting (white balancing) on that and
it might be possible to save such presets with the stream so the
second stage on a full blown computer can take that as the
default setting.

So for now it will definitely be a two stage process. Capture and
store and then the second phase transform and transcode to
a different format.

The list of the most problematic things in this project are (in my
opinion):

1. working out a system to go inbetween the camera and storage (full PC, FPGA, embedded chips etc.)

2. working out a storage system that can keep up (or compression)

3. getting a viewfinder / monitor out

Now to give everyone a low down on some data rates (again):

1280 x 720 x 24 fps x 8 bit, full RGB = 63.28 MB/s (126,56 MB/s)
1280 x 720 x 24 fps x 10 bit, 16-bit full RGB = 126.56 MB/s (253.12 MB/s)

1280 x 720 x 24 fps x 8 bit, Bayer = 21.09 MB/s (42.19 MB/s)
1280 x 720 x 24 fps x 10 bit, 16-bit Bayer = 42.19 MB/s (84.38 MB/s)
1280 x 720 x 24 fps x 10 bit, packed Bayer = 26.37 MB/s (52.73 MB/s)

1280 x 720 x 24 fps x 12 bit, packed Bayer = 31.64 MB /s (63.28 MB/s)

The numbers between the () are for 48 fps. I've added 12 bit on
the last row to see where it might be headed. So the most optimal
size and very easy to implement (10 bit, packed Bayer) is the one
that will definitely be there. Whether or not we can support any
futher compression depends on a lot of things (licenses, speed,
resources etc.).

Steve: can you explain to me in a little sentence what PC/104 is?
Is it like a form of mini-PCI or PCMCIA?

Rob L: PC/104 plus. One sentence- easy:
http://www.pc104.org/technology/pc104_tech.html

I'll elaborate. PC/104+ is an small stackable card format designed for industrial uses that mimics PCI-32 in signals. Instead of card edge connections (we've all had to reseat boards in computers) it uses connectors like hard drives use and the boards screw together. Since it mimics the standard PCI, designing a new board is usually just a relayout. That means there are a lot of boards out there. Price is definitely higher that PCI (volume of sales are lower), but possible for the integrated camera idea (CPU + LCD controller + disk controller) but the CPUs tend to lean towards lower speed, lower power. EBX is worth watching (same site). It is a somewhat larger embedded format that uses PC/104 plus as an expansion board. If I were designing an embedded PC product, this is where I would be looking. It could also be a migration path for what is being developed without any software changes.

By the way, Rob L, nice laying out the project for everyone. I think you spelled out all the major first pass design decisions. A 640x360 viewfinder should be able to be done with a cheap, small LCD screen.

A non-programmer who would like to assist can look for a mini-pc box, like the shuttle with a low noise power supply (or the PS from another source). Someone posted this:
http://www.logisysus.com/

I was told that it is better if you have a separate OS drive from the video. So, ideally, the box should be able to handle a small drive (laptop?) for the OS and two 3.5" for a RAID. Maybe removable. Definitely gigE. For people doing larger work (as Obin dicovered), they may need to move it faster. Or we be patient - I think 10gigE will be dropping quickly in price.

Steve,
I am coming back to the CMOS chip saga. I found something that looks almost incredible. I would like to know your opinion or anybody else of course.
The Lupa 1300 from FillFactory, probably the same manufacturer that supplies the Sumix camera:
http://www.fillfactory.com/htm/produ...0/lupa1300.htm.
It's size is only 30% smaller than full 35mm frame, has 62db S/N ratio, is capable of 30fps. I think that for a start (1.3Mpixels Bayer), is not a bad choice.
Any thoughts?

looks like a good chip..Steve?

I have a good idea, lets make the mcroatx computer run in RAM so when you turn it on you don't have to wait for windows to load an load and load...yes??

Fill Factory, the company that brought us the much maligned IBIS-5. This is the direct competition to the Micron MT9M413 http://www.micron.com/products/imagi...s/MT9M413.html
Noise spec is somewhat better, but nothing like the Micron 1.3Mpix rolling shutter or the Altasens. Simultaneous expose and readout - that is very good. 16 analog taps - very bad. This would be a very expensive camera (not a cheap sensor either).

I guess I haven't heard a consensus that you are willing to put up with noise and high cost (especially for 720p) to gain a global shutter and large pixels? Would you pay $4-5K for this camera? The Altasens will be in the same ballpark with 1920x1080 60fps true 12 bit.

4-5k is tooooo much for a prototype. Altasens with those specs sounds more promising. What is the size of the sensor? Still 2/3 or larger? Don't forget that the big boys are looking at full frame 35mm (Dalsa, Panavision, Arri). Also many from this list are aiming for a similar outcome.
The look given by the combination of FOV and DOF are still an important issue for narative long form projects.

Rob L - Thanks for excellent summary! Do you mind if I adapt it for my wiki on the project?
It is possible to put a CompactFlash drive in a PC that looks like a standard IDE drive ... but that would only help if Windows can be installed in 1 GB :-)
The AltaSens is 2/3" I believe.

I was talking a RAM drive..can't you install windows into ram?

http://www.cenatek.com/product_rocketdrive.cfm

EDIT:
here we go:
http://www.bitmicro.com/products_edisk_25_ide.php

<<<-- Originally posted by Rob Scott :
It is possible to put a CompactFlash drive in a PC that looks like a standard IDE drive ... but that would only help if Windows can be installed in 1 GB :-)
The AltaSens is 2/3" I believe. -->>>

Is the actual target platform for the camera Linux? If this is the case it may be much easier to get it onto a CompactFlash drive. Not sure what would have to be stripped from Linux, though. I may check this out a bit as I'd like to avoid a camera boot cycle.

Eliot

I don't think so. The biggest problem is that RAM disappears when the power is off, so when you power on, the PC has to create the RAM drive from scratch. I suppose it's possible that there is a boot loader that will pull Windows off the hard drive and copy it onto a RAM drive first, but that doesn't seem much quicker than just booting.

Edit ...
I looked at the Cenatek device, and (unless I'm mistaken) it requires power all the time. When the power goes away, so does the contents of the RAM.
That's a long-range goal. I'm not sure if/when that will happen, because I don't have much experience with development under Linux.

There are a number of "embedded" distributions of Linux, some small enough to fit on a single floppy. Linux on a CompactFlash card is definitely doable.

That is a nasty workflow for sure, but at least it works.

The new software we're working on will implement the new streamlined workflow, but until it's tested and debugged you might need to use the StreamPix workflow. Erg .. that's the software that costs $1500 isn't it?

it's around that price, yes

HEy it DOES have realtime preview of framerate WHILE I capture..this is a very good thing! and it's at 1280x720 full res

Software update
 

I discovered last night that my 22 fps limit wasn't because of the hard drive or disk-writing thread. The frame generation code was not running at full speed for some reason. I am now getting 25-27 out to disk frames pretty consistently. See my updates here if you're interested:
http://www.obscuracam.com/wiki/wiki/...ional%20status

That's good! Despire the nasty workflow, it sounds like you should stick with that while we develop and test our new software. I am sure you'd prefer to save the $1500, but Rob Lohman is right -- this is not going to happen overnight.

<<<-- Originally posted by Rob Lohman :
>David: is 10 or 12 bits always left justified? Is there a reason for it being left instead of right? -->>>

The issue is the editing or compositing software. Pixels stored on disk can be packed anyway you like (compressed or uncompressed, YUV or RGB etc), yet once the application sees the data is most have the format and white level it is expecting. For most 16bit compositors the white level is 65535, for After Effects the white level is 32768 (not 32767 -- so still using 16bit but with lots of headroom.) So the data must be left shifted is it less than the target applications bit depth. Note: 10 bit data for After Effects only needs to be shift by 5.

new windows media clip coming soon, RAW no white balance no color work 8bit

Steve, again why do I need to set gain at 3db for the whites in the image to be true white not grey?

David: thanks... weird that this would be different for different
applications. That's gonna be an issue in the second phase of
the capture process. But the good news is that it is easy to do
anything in that stage/phase.

Steve: Thanks for the PC/104 note.

All:

Personally I was not planning on having the OS on a fixed drive.
I've opted (to Rob) to go with Windows XP Embedded. It is
designed to be small, fast, configurable and able to boot of a
read-only device. So this could fit into flash memory. I'm not sure
how large it is fully.

It allows you to select which component / drivers you want to
include etc. so it should have a pretty small foot/memory print.
I have it lying around here (due to my work) and was hoping to
take a look at it lateron. Our attention is first on getting it to work
before working out on how to make it all small and portable.

The only problem with WXP Embedded is that you can't buy it
in the store. I'm not sure if you can buy this from any company
or Microsoft self as a consumer. An OEM who builds these camera's
should be able to buy those licenses, though.

We'll see where this goes!

Rob: ofcourse you can use my list and other details we talked
about through e-mail on your wiki page. Do put my name next
to it so we know later who said what and why...

Obin:
I'm guessing a bit since the analog stages are internal to the sensor. It sounds like the output of the amplifier without any gain is not large enough to reach the full scale input range for the A/D. The are a bunch more registers internal to the Micron chip that are not documented in our manual - that is only for the basic information that most people will use. We do allow access to all the registers (5F-64 provide black level calibration). XCAP lets you write serial strings to the camera if you want to experiment sending commands. This is a very helpful tool for a programmer to manually try out register commands. They may have done this on purpose to allow the auto black calibration some headroom in shifting the black reference before saturation occurs at the high end.

before after gama work 12bit file 1/2 resolution/size from photoshop jpeg

http://www.dv3productions.com/test_i...itexposure.jpg

still encoding wmv HD!

Hey Obin,
Can you post two consecutive raw bayer pix for us to see the noise on? 10 bits?
Thanks a bunch!

it's going to take 45min to upload but the address is:

www.dv3productions.com/Video Clips/RAW-8-bit.wmv

this is UNTOUCHED footage no white balance no color work jsut converted from 12bit to 8bit and compressed

ouch! the Bayer filter in Streampix hurts my eyes! it looks like a VERY basic filter

guys what is this:

1 Connector for LVDS module (Optional)

the ITX board has that...is that fast? do cameralink cards work on that?

LVDS is the older camera interface, before cameralink.
RS422.
My Atmel 8M's use it.
Is that option expensive?

16bit tiff file

http://www.dv3productions.com/test_i...-24fps1835.tif

pre-Bayer filter black and white image:

http://www.dv3productions.com/test_images/pre-bayer.tif

Wow ... there is a lot of the "zipper effect" left over in that one. (But PaintShopPro is all I have installed on this laptop, so perhaps it's not displaying it right.)

<<<-- Originally posted by Rob Scott : Wow ... there is a lot of the "zipper effect" left over in that one. (But PaintShopPro is all I have installed on this laptop, so perhaps it's not displaying it right.) -->>>

Like I said Bayer plugin is bad in Streampix

<<<-- Originally posted by Rob Lohman :
Wayne: I did my first programming in low-level assembler before
moving to C(++) and then on to the Windows platform. I'm pretty
sure I know exactly how a PC works internally and how Windows
works as well. I've written assembler boot-loaders and some
low-level Windows stuff. The only thing I ain't really good at is
anything with Unix/Linux on the PC. Oh well...
-->>>

Well your two levels up on most other app programmers, most of which wouldn't realise much about these timming issues, let alone how to handle these hardware timming issues, as ussually only highend 3D games need this much performance. But I thought it might help and provide nice reading for everybody else. I'll delete it then, no use cluttering up the space.

Thanks

Wayne.

Thanks for the two frames. I might try the variable gradient software on it and post the color result. That seems to be the best demosaiking method right now.

Can you post two raw 10 bit black and whites to allow me to evaluate the noise component? That would be frame N and frame N+1 from a capture.

-Les




<<<-- Originally posted by Obin Olson : 16bit tiff file

http://www.dv3productions.com/test_i...-24fps1835.tif

pre-Bayer filter black and white image:

http://www.dv3productions.com/test_images/pre-bayer.tif -->>>

Another 720P sample video!
 

Just for 5hits and giggles, here is a few seconds of media9 compressed output originating from my JVC HD10. Of course it is hampered by the 8 bits of color depth, and it's non-manual modes, etc, but those who are used to looking at DV footage should take a look.
This camera is a single chip 1.1 Mega Pixel sensor. It records to a 19 megabit mpeg2 stream. About $4K for this camera.
The detail is good, look at the thread details in the backpack label,
they seem to have survived the media9 compression!
The file is about 13 meg, it's a 5 megabit data rate.

I shot this yesterday in a park.

http://s95439504.onlinehome.us/park.wmv

-Les

I must say I'm pretty disappointed with the rolling shutter
effect. It is quite noticable, especially lateron on the movie
when you pan pretty fast along the fence, Obin.

Obin: could you perhaps do some testing with smaller resolution
files?

Capture 640x480 at 8 bit and do this at 24, 48 and 60 fps
(Your computer should be able to handle this in the lower
resolution) and do some similar sweeps along side the fence.

This can show us how much the effect changes at higher
framerates.

Thanks for testing!

Rob it has nothing to do with fps it's all in the mhz of the camera sensor...if you shoot high enough mhz it goes away..well almost goes away