Les,
I been doing some reading on various demosaiking algorithms, and I'm am aware that any conditionals are a pain for SSE optimization. Still it is worth a go. I would welcome links to any demosaiking algorithms you favor.

Hi David,

I've sent you a demosaicing algorithm that I've found along with sample MATLAB code that should be very interesting if it can be implemented. Another thing to note is that along with any demosaicing algorithm, there's going to have to be some sort of "false color" filter that can remove any vestages of the dreaded orange-blue color arifacts that occur once Nyquist has been exceeded for any chip. A good algorithm can reduce this color aliasing to almost nill, but there's still going to be some aliasing once Nyquist is exceeded, and that's going to happen with any sampling system out there, no matter how good your algorithm is. So again, we're also going to have to look at some sort of "false color" or de-moire/color-aliasing filter that can remove and color in the blue-orange areas with their surroundings.

David,
I've found that this paper was good at looking at the different methods of demosaiking:
http://www-ise.stanford.edu/~tingchen/main.htm

The sample code is in matlab, I got it working and tried them out. It's slow, but good for looking at the results.
There are a couple of other methods out there, one from Kodak ( patented, of course ) that seems to deal more with defect management than the actual demosaik itself.
Another method is 'Pixel grouping', but does not work as well as variable gradients for natural scenes.

On another topic, does Prem Pro 1.5 allow *any* higher bitdepth image to be imported without extra plugins? I tried tiff , rgb, and tga, and it does not seem to like any non 8-bit image from the get go. I guess it's a plug in I need to start testing with this idea. I have Cineons to start with.

-Les

Les & Jason,

I'll look into the various algorithms to see what is both practical and good quality.

Les, Premiere Pro 1.x only includes an 8 bit engine. It allows for the replacement of its engine (importing, mixing and exporting can all be deeper than 8bit as we did with Prospect HD) which makes it very flexible, but not within the appropiate plug-ins.

David.

david I am happy to send you raw files - what do you need from me?

Obin or Steve ,
How many noise free bits are coming from that cam ?

( subtracting 2 static 10 or 12 bit images, what's the average pixel value of the resulting close to black image ?)

Are we talking 9 bits at least , for a 1/48 sec integration?

-Les

Obin: can you please tell us how you got that movie? What
settings did you used for the camera (gain, FPS etc.). To what
format did you capture and how did you transform that into a
WM9 file? Did you do any color correction or other post work to
the footage? Which Bayer algorithm did you use?

What lens are you using for this system?

I'm wondering about your framerate issue with that WM9
encoded movie. Could it perhaps be that your harddisk could
not keep up with the data? To what harddisk system did you
record? (see my list below as well)

Steve: so can we do a full frame shutter with the chip or does
it only have a rolling shutter? Personally (as you know) I'm more
fond of getting a full frame than getting changing lines.

Regarding expensive bayer algorithms. As inidicated before
we can do bayer on the PC after capture if we want. An idea
that popped up in my mind was this that might be interesting:

1) get the RAW footage from the camera and dump this to disk

2) resize the footage to standard SD (include letterbox) using
some fast algorithm, do bayer (if needed) and do a simple
color shift for a crude white-balance. Send this to viewfinder
and realtime compress this in a second file that is being written
to disk as well.

The second file is a simple SD file that can be more easily
compressed and used in an edit system. We could then perhaps
edit the movie as normal and output a EDL that we could use
to conform the "big" raw uncompressed file?

The only issue remaining then is color correction, effects and
futher processing.

Rob,
The shuttering method is intregal to the chip design so you have to find a full frame shutter chip. This is the one I've seen mentioned on this site before:
http://www.micron.com/products/imaging/products/MT9M413.html

It has 12 micron pixels, 10 bit A/Ds, 15mm horizontal size, made my Micron (probably pretty low noise) and can do 720p. OK, maybe 100x faster also.

This would be an expensive camera since the chip is large and it is 10 tap. In 10 bit mode there are 100 parallel digital lines to be dealt with. Which would you rather have, gobal shutter or 1080p for the same $$?

I'd rather have global shutter... I think. As said I'm pretty new to
this whole (rolling) shutter system.

Since you have a lot more knowledge and experience with this
then I do; let me ask you the following:

1) do you think the camera will be able to give good enough pictures with a rolling shutter?

2) what type of images will we see skewing the most? I assume something fast moving?

3) is increasing the framerate and then dropping frames the only way we can get rid of this skewing

I'm not saying rolling shutter is bad etc. I'm just trying to
understand it all and see what the cons and pros are, as you
understand.

The "issue" I'm seeing now is that if you have fast moving
objects we will get skewing. The only way to get rid of it is by
moving the framerate up and dropping frames. But that loses
our edge of variable framerate recording for slow-motion which
almost always involve higher speeds.

So then the faster recording is not of any real use except to get
rid of skewing. Am I right in this conclusion?

I think the picture Obin showed looked very beautiful and I'm
very impressed with your chip. So that's a good sign for any
sells as far as I'm concerned!

Could we program the chip in such a manor that it will capture
say 48 fps but only sends 24 to the computer to get rid of skewing
and use less bandwidth since we don't use it anyway?

Thanks for your continued support on this matter. It IS greatly
appreciated by me and everyone else!

Personally I'm not too interested in HD, yet. There is no HD here
in Holland (basically). So the only reason would be blowing up to
film which might happen if we make a good enough movie. But
that is not garantueed. In that regard I'm not too scared since
they blowed up 28 days later as well and it still looked "okay" to
me.

BUT, ofcourse that was shot with 3 CCD's instead of 1 CMOS with
bayer pattern which ofcourse lowers the actual pixels.

The most important thing for me is uncompressed bits and the
larger 10 bit range. Ofcourse I'm going to run such a chip as yours
definitely at 1280x720 (progressive) to get the highest resolution
possible. That is neat to have, but is for me an extra.

I could care less about 1080p, personally. That is so much data
and bandwidth that I simply do not want to have. 720p will
already be a challenge enough in all of these regards so I'm
focussing on that first. Ofcourse in a couple of years I might want
to upgrade my chip to a higher resolution, but we'll see then
when we get there.

Ofcourse this is strictly for myself. Others seem to really want
the 1080p option. That's fine. Different people, different wishes.

I have a feeling not everyone is aware on how much data we are
actually talking about. So a small list below. I've included
recording time for a 200 GB harddisk (204800 MB) at these rates.

720 x 480 x 24 fps x 3 bytes (8 bit) = 23.73 MB/s (8630 sec / 143 min)
1280 x 720 x 24 fps x 3 bytes (8 bit) = 63.28 MB/s (3236 sec / 53 min)
1280 x 720 x 24 fps x 4 bytes (10 bit compact) = 84.38 MB/s (2427 sec / 40 min)
1280 x 720 x 24 fps x 6 bytes (10 bit) = 126.56 MB/s (1618 sec / 26 min)
1920 x 1080 x 24 fps x 3 bytes (8 bit) = 142.38 MB/s (1438 sec / 23 min)
1920 x 1080 x 24 fps x 4 bytes (10 bit compact) = 161.46 MB/s (1268 sec / 21 min)
1920 x 1080 x 24 fps x 6 bytes (10 bit) = 284.77 MB/s (719 sec / 11 min)

720 x 480 x 48 fps x 3 bytes (8 bit) = 47.46 MB/s (4315 sec / 71 min)
1280 x 720 x 48 fps x 3 bytes (8 bit) = 124.56 MB/s (1618 sec / 26 min)
1280 x 720 x 48 fps x 4 bytes (10 bit compact) = 168.76 MB/s (1213 sec / 20 min)
1280 x 720 x 48 fps x 6 bytes (10 bit) = 253.12 MB/s (809 sec / 13 min)
1920 x 1080 x 48 fps x 3 bytes (8 bit) = 284.76 MB/s (719 sec / 11 min)
1920 x 1080 x 48 fps x 4 bytes (10 bit compact) = 322.92 MB/s (634 sec / 10 min)
1920 x 1080 x 48 fps x 6 bytes (10 bit) = 569.54 MB/s (359 sec / 5 min)

Now even with raid I'd like to see us get over 100 - 150 MB/s
write rates. In any form I'd say compression is probably needed.

Rob,
The numbers that you show are appropriate for: pre-post processed single chip (Bayer de-mosaicing done but no compression) or 3 chip cameras. For single chip cameras like Obin's SI-1300, you are recording 1280x720x24fps, 10 bit, unpacked. (16 bits of transfer and storage) so you get about 44MB/sec *average* transfer or about 75minutes on a single drive.

Maybe Obin could do some test sequences of fast panning on objects with large vertical lines (buildings?) to test the rolling shutter artifacts at 24 and 48fps. I really don't know how objectionable the skew really is - everyone talks about it but I've never seen images other than packing labels on conveyor belts.
Steve

I think I missed the point that bayer is ofcourse a single color
per pixel indeed. So these numbers are indeed after bayer
conversion. When go to 10 bits we are indeed going to 2 bytes
per pixel instead of 1 (for 8-bits) or the 3 I had.

This futher indicates we would want to do bayer conversion
offline probably. This will greatly reduce bandwidth indeed.

Here is the new list storing the data BEFORE bayer conversion
(the previous list was AFTER):

1280 x 720 x 24 fps x 1 byte (8 bit) = 21.09 MB/s (9710 sec / 161 min)
1280 x 720 x 24 fps x 2 bytes (10 bit) = 42.19 MB/s (4854 sec / 90 min)
1920 x 1080 x 24 fps x 1 byte (8 bit) = 47.46 MB/s (4315 sec / 71 min)
1920 x 1080 x 24 fps x 2 bytes (10 bit) = 94.92 MB/s (2157 sec / 35 min)

1280 x 720 x 48 fps x 1 byte (8 bit) = 42.18 MB/s (4854 sec / 90 min)
1280 x 720 x 48 fps x 2 bytes (10 bit) = 84.38 MB/s (2427 sec / 40 min)
1920 x 1080 x 48 fps x 1 byte (8 bit) = 94.92 MB/s (2157 sec / 35 min)
1920 x 1080 x 48 fps x 2 bytes (10 bit) = 189.84 MB/s (1078 sec / 17 min)

We could perhaps lower this futher by packing data into all the
bits. So you get 4 pixels at 40 bits instead of 64 bits which is
another reduction in data bandwidth of 25%. If we re-shuffle
color bits we might even use a very fast lossless compression
method like RLE or something a bit more complex.

I will see if I can work something out on regards for an efficient
high-speed lossless algorithm for bayer patterns.

A quick search turned up that Dalsa is also using such an
algorithm which futhers enhances the feeling that his might be
the best way to go:DALSA's implementation extends basic L3 to include near-lossless and visually lossless compression, as well as compression of raw Bayer pattern data (as opposed to 3-channel RGB).It looks like we cannot use it since they hold a license on it, or
so they claim (http://www.dalsa.com/dc/workflow/L3compression.asp)

This futher indicates we would want to do bayer conversion
offline probably. This will greatly reduce bandwidth indeed.Yes, that's the plan -- to store the raw pre-Bayer data to disk as quickly as possible. In my tests so far, I've been able to write 30 fps (1280 x 720 x 10 bits) with a single drive. (With nothing else going on.)

I don't have the camera SDK yet, but over the weekend I implemented a simple program to...
"Grab" a frame (generates random data to simulate this)
Write the frame to disk
Do a simple Bayer filter (nearest neighbor)
Slap the buffer up on the display (using SDL)

I'm doing a preview of 1 out of every 20 frames and I'm getting about 9 fps out to disk.

This sounds pretty bad -- and it is -- but I have done absolutely no optimization at all. The first thing I will do is split all these things into separate threads; that way, the "write to disk" thread can run at its lazy pace without slowing down capture or preview. I'll let you know how it goes.

Qt is a really nice C++ GUI frameworkOne of my next steps was to evaluate GTK and Qt (my two top choices for GUI frameworks), but this recommendation may push me over the edge into Qt. I'll evaluate them and let you know.

sync the frame grabs with incoming time code from an external time code generatorI will definitely keep that in mind for ... what, version 1.2? Not sure yet. :-)

I think the biggest processing and recording issues are a complete understanding of any step that is a potential loss of quality - like the Bayer filter.I have a basic understanding ofthe Bayer filter -- I've implemented a "nearest neighbor" filter and have tested it to make sure it works properly (it does -- though it's a pretty brain-dead filter). I have several documents on these filters and I'll be reading up on them.

My plan is to have several filters to choose from during the processing step (which happen offline). This part of the project will definitely be open source, so anyone can audit the quality of the filters and add new ones if they like.

Good to see you on the move as well Rob <g>

I've done a bit of digging and I must say I could not find that
much about lossless bayer compression except for the Dalsa
link above and some paper that's a bit over my head in terms
of math. I can develop algorithms on my own, but not write it
down in math.

A Fast Vector Quantization Engine for CFA Data Compression (http://www.dmi.unict.it/~battiato/download/NSIP_2003_VQ.pdf)

A lot of terms in regards to Dalsa's encoding and general
encoding came up:

Golomb coding (http://www.tutorgig.com/encyclopedia/getdefn.jsp?keywords=Golomb_coding)
Entropy coding (http://www.tutorgig.com/encyclopedia/getdefn.jsp?keywords=entropy_coding)
Wavelet compression (http://www.tutorgig.com/encyclopedia/getdefn.jsp?keywords=Wavelet_compression)

It seems Dalsa is futher with me on my remark to maybe use
lower resolution/quality proxies to do the editing:For editing purposes, the camera will be able to output a 2K proxy to be edited in a fashion that will re-define our concept of "offline editing" with the final EDL and associated metadata used to assemble the final master from the original 4K files.Rob: In what did you code that test set?

I'm looking for RAW bayer encoded 10 bit (stored as 16 bit files
please) file to do some algorithm testing. It sounds like you
already have a basic setup working. Can you send me a RAW
files or compile program to make them or source?

I'm assuming you did store the 10 bits inside 16 bits when
you tested writing? If so you where doing 53 MB/s writing. I
don't think I have any disk here that will sustain that.

How do you know you wrote it to disk in realtime?

Steve: from the looks of it the 1300 chip has
1280 x 1024 = 1.310.720 pixels, and thus:

655.360 green pixels
327.680 red pixels
327.680 blue pixels

Right?

Steve: do you happen to have any RAW bayer 10 bit files from the
1300 that we can use and actually test if we get a correct
picture? Also with compression etc.

p.s. I've been wrapping my head around Bayer today and it is
sinking in, so that's a good thing.

In what did you code that test set?I wrote the code in C. The "test set" was two things:

A "unit test" I wrote in C with arbitrary data values; I also coded the expected results, so this is self-checking.
An 8-bit bitmap file I created with patterns I knew should generate certain colors. The test code read the "red" channel of the bitmap, "upscaled" it to 10 bits (thus simulating a raw image); then processed it and generated an 8-bit preview on screen.
I don't actually have any raw files yet, I'm just simulating them.

Does that answer your question?

I'm assuming you did store the 10 bits inside 16 bits when
you tested writing?No, I'm writing the packed 10-bit raw data.

Steve i would take global shutter ANYDAY over 1080P...I have seen the rolling shutter artifact and it is not what any of us want at all...

Rob and Rob tell me what you need and as soon as I get to work today i will try and upload stuff for you

Originally posted by Obin Olson : Rob and Rob tell me what you need and as soon as I get to work today i will try and upload stuff for you
A raw (pre-Bayer) file would be helpful. That way, either of us Robs (heh) could work on a Bayer algorithm without requiring the camera system or SDK.

anyone know of a HD DVD player that will play full on HD from a mpeg4 or windows media 9 endoded HD file on a dvd disk? I need a way to show this stuff in full blown HD and this seems like it would be a good answer if someone makes a player


A strange thing happens with the 1300 camera, if you shoot with NO gain on at all the whites are all grey..i must use about 3db gain to get whites and hotspots to be white not grey..anyone have an idea why this would be?

Rob Scott are you going to get a 1300 and capture board? Rob I think we need to take a very close look at Cineform for the Aspect HD stuff...we may need to use this because I sure can't edit any of this stuff at raw resolution on a standard 3ghz pc or over our gigabit network from our video file server

some info on the video clip I uploaded:

Everything is 8 bit - the software can't support 10 bit with my system at the moment.

The truck shot was exposed for the background that's in the sun and the truck was pulled up from the blacks - this 8 bit...you can do MUCH more in 10bit

The shot of me was exposed for my face - thats why it is so blown out in the background

Flowers are pretty much raw with no color work at all just a white balance

Everything has 3-4db gain on it and was shot close to 24fps

Originally posted by Obin Olson : anyone know of a HD DVD player that will play full on HD from a mpeg4 or windows media 9 endoded HD file on a dvd disk? I need a way to show this stuff in full blown HD and this seems like it would be a good answer if someone makes a player
Product -
http://www.vinc.com/product.asp?ID=56&PID=21&SEQ=2

News articles -
http://www.cdfreaks.com/news/9852
http://news.designtechnica.com/article2925.html

Rob Scott don't forget ROI in your program so we can capture SD at 720x480 if we want to from this camera

The only DVD player I know of that supports HD:
http://www.digitalproducer.com/2004/01_jan/news/01_05/vinc016.htm

Obin: are you referring to your older clip? We really would like
something 10 bit. Can your system handle it if you drop the
resolution to something lower?

Can you also please respond to my questions about how you
obtained that movie and whether you processed it or not?

Originally posted by Obin Olson :
Rob Scott are you going to get a 1300 and capture board?Yes, am placing my order today. How long did it take to get yours?
I think we need to take a very close look at Cineform for the Aspect HD stuffAbsolutely. My intention is to support any codec installed on the user's machine, but it would be good to be able to test with Aspect HD and other codecs. I wonder if we can get them to donate a few codecs for testing purposes ...
don't forget ROI in your program so we can capture SD at 720x480 if we want to from this cameraWell, version 0.1 will probably do strictly 1280x720. I'll add features after I get the core stuff working reliably. Feel free to maintain a list of your requirements -- on your wiki page, perhaps.

BTW, don't forget about the Mac people and FCP, especially when there are a lot of people editing with FCP. I understand that your bayer de-mosaic program will probably be running on Windows or Linux x86 (btw, if you're using GTK, we can get that running on OSX under X Windows), but make sure that whatever files you end up writing to, that they can be used on a Mac too. As of right now, Aspect HD, etc. won't run on Mac or in FCP.

obin I agree with the bravo d3. It will play up to 1080p wm9
this is a link to a highly rated 1280x720p projector it has the mustang chip found in much higher priced projectors .
you would be capturing in 720p playing a dvd in 720p and projecting in 720p this would look incredible

http://www.hometheaterhifi.com/volume_10_3/benq-8700-projector-8-2003.html

Originally posted by Jason Rodriguez : BTW, don't forget about the Mac people and FCP, especially when there are a lot of people editing with FCP. Absolutely! The "Capture" program will start as Windows-only (and eventually Linux), primarily because the SDK is only available for Windows and Linux. It will have a bare-bones UI, just enough to get by.

The "Convert" program will be based on GTK or Qt, both of which are cross-platform. We're probably going with Qt, which runs natively on Mac OS X - http://www.trolltech.com/products/qt/mac.html

I have a cousin who works in the industry and uses Mac/FCP, and he has agreed to help test the system.

(Edit - I just realized that Qt is not available for GPL programs for Windows, only for Linux/X11 and Mac OS. For this reason I may have to go with GTK ...)

more good news for the indie filmaker.
this camera could go straight to theatre.

http://www.filmfestivals.com/cgi-bin/shownewsdc.pl?obj=ShowNews&CfgPath=ffs/filinfo&Cfg=news.cfg&news=general&text_id=23982

Rob do you have a link to the wiki page? I lost it, sorry!

I will post stuff I think you need to write into the software on that page as I think of stuff...glad your getting the camera !!! awesome!

Rob do you have a link to the wiki page? I lost it, sorry!No problem - http://www.obscuracam.com/wiki/wiki

Hi everyone this is a link to a homemade screen I made about a year ago. It's made from material you can buy at joann fabric
it duplicates screens costing thousands the materials are less than $100. It's perfect for displaying 1280x720 material .

http://www.btinternet.com/~paulw77/

Perhaps we prefer not to think about this..... but
Can someone evaluate the noise on the 1.3 camera?
Just subtract two images and see what happens.

How many stops does it *really* have? It's easy to find out!

HI!

I'm new here... very excited about what you guys are doing..

I was wondering if any of these imaging devices could capture in an even wider format? like full cinemascope or something... maybe 1645 x 700 or something.. anything 2.35:1. Without having to buy a $10,000 anamorphic lens that is :)

Eric Gorski :
I was wondering if any of these imaging devices could capture in an even wider format? like full cinemascope or something... maybe 1645 x 700 or something.. anything 2.35:1. Without having to buy a $10,000 anamorphic lens that is :)
Most of these chips are 4:3 (1.33:1) aspect ratio, but you can certainly get 2.35:1 by ignoring some of the vertical resolution. The Silicon Imaging (http://www.siliconimaging.com) unit we're working with right now is 1280x1024 and we're planning (for the most part) to use it in 1280x720 mode to match the HDTV standard.

To get more horizonal pixels you'd have to go with a larger resolution chip. Silicon Imaging also has a model with 2048 x 1536 -- which means you could get 2048 x 870 in a "Cinemascope" mode.

You'd only need an anamorphic adapter if you really didn't want to waste vertical space on the chip and if you didn't mind getting rectangular pixels.

i guess 1280 x 545 would still kicks ass.

keep up the good work!

Gee, I get busy for a day and you guys go prolific on me. Rob, I'm very pleased that you are doing this project. I've never been happy with the software costs of the systems we have since we try hard to keep the prices down. I bugged Alison to get your software out the door.

I would like to suggest that anyone who wants to help Rob on software send him a list of skills and appoximate time they can donate. That way he can coordinate the effort and not do it all himself. It doesn't have to be all programming. It can be "wrapping your head" around some aspect. Or, in Obin's case, making test images and sequences.

Rob L: your numbers look good now but remember, they are average and not peak across the bus. You are correct about the color ratios on all Bayers.

Also, we we start benchmarking disk systems, remember, disks slow down on the inner tracks.

Obin,

do you see the rolling shutter artifacts at 24fps with a 1/24th second shutter? I didn't see what I would call any motion artifacts in your footage (although it did seem to have an annoying studder, like a shutter that was too short), so I'm wondering if even at 1/24th of a second (so that you're not dropping frames) they are present or noticeable (the rolling shutter "slanting" effect).

Just a quick update. Last night I split the three parts of the code --
Frame generator (simulates incoming frames)
Writer (to disk)
Preview (quick-and-dirty Bayer filter)

... and now I'm consistently getting between 20-22 fps (I was getting 9 before). Again, I haven't done much in the way of optimization, I just shuffled things around and put them in separate threads so they don't keep each other waiting.

I am using buffered I/O right now; I'm going to switch to standard I/O and see if that makes a difference. I am also wondering if I'm getting a worst-case scenario on my hard drive speed -- they are almost full. I may need to buy a new one and see if I get better speed with an empty drive.

The preview is only getting 1-2 fps, but that code is badly written and I think I can get much better speed just by doing a bit more optimization.

I'm also using SDL for display of the preview. Using DirectX would probably be faster, but I had trouble linking with the libraries.

If I have time I'll put up a Wiki page with some further details about my progress.

BTW, should I start a new thread for this?

I think this thread is fine Rob, Jason i will try and get you an example of rolling shutter artifact today..you won't like it, it's ugly...we can avoid it if we set the camera mhz high

uggh...I posted what i have been doing with HD at dv.com and got a bunch of child-like responses. Guess I will stick to this site!


Rob I really know nothing about DirectX but from what I hear overlay is the way to go...if you can offload anything to a graphics card I would think that would be good because of the raw power that new graphics cards have

overlay is the way to go...if you can offload anything to a graphics card I would think that would be good because of the raw power that new graphics cards have Yeah, I'll certainly look into that again once I have made a bit more progress. Thanks!

this may be worth looking at:

http://www.pluginz.com/news/970

" BitJazz Launches Real-Time Nondestructive Video Codec "

Mike Merten

I agree with you fully, see my posts in the other threads on custom camera designs and the viper. The camera here is Obin private experiment, it's also being used as a trial run for the others camera comming (see other threads).

Thanks

Wayne.

Obin

I've had a look at your pictures, great, I love the colour, beuatifal. I'm impressed and would to see the results from a 1080 or SHD version oneday. I noticed all the artifacts that people were talking about, and that is why I think that Bayer will take away from the resolution and detail quality for resolution upscaling for the big screen (not to mention green screening as the bayer does not give a correct per pixel colour edge).

My latest Opera Web browser or Radeon 9000 driver, appears to be performing some sort of upscaling/deblocking, I can't see a square pixel even at 1000% magnifcation, and it looks good (except colour averaging of the bayer).

How about using a test chart to work out the real picture specs (latitude, low light and sensitivity), or side by side comparisons with the DVX 100 on the same images? Alltogther it leaves SD single chips for dead, and even the colour of 3 chip SD (I hate oversatuation), this is probably the sort of colour we will eventually see on single chip HD cameras in the future. Definetly good enough for cheap indie film, it looks almost natural (the latitude and falling away to black and blowout being the biggest variation). I think your way ahead of 28 days latter.

Obin, about NLE editing computer, we are close to the technology of being able to fit the NLE in the box with medium to high comrpression, maybe not this year (unless a low powered multiple processor server board is availble) but maybe next, so you can NLE from the camerra connected to a usb hub (for mouse, keyboard, external drives) and monitor. Your system is also the lightest camera in processing requirements, so you probably can do that right now. Your camera's USB2 connection, a nano-itx board (fastest) and a single laptop drive might be able to handle a pure packed bayer image. If you can get a faster cpu you might even be able to compress. People get caught up in the camera link thing (and I wonder why SI don't pack their data in the head, it is a simple circuit/proceadure) but a PCI cameralink board should have more than enough bandwidth for 1080. I wonder if it has something to do with the unreliability of USB2.0, or the fact that the data is read so quick (unbuffered) that it maxes out USB2.0? Steve?

<<<-- Originally posted by Steve Nordhauser : Although it is possible to adjust the vertical blanking time somewhat, what you proposed hasn't been attempted yet. It might take some firmware changes in the camera. What Obin was proposing is based on the rolling shutter artifact. Each line is sequentially read from top to bottom. That means that there is a temporal difference on any image of one frame time from the top line to the bottom. The faster you go, the less the difference. Obin wants to readout at twice the frame rate he needs and toss every other frame. This will get him the desired rate with no timing adustments but reduce the single frame time in half.

I will have to think about what Rob suggested though - extending vertical blanking for a full frame time.....hmmmm. -->>>

The problem with these scheemes is that you will need as much shutter time (every 24th of a second) as possible to accumulate light, and also to blend movement from one frame to another (to stop the jerky motion). I think I'am prepared to live with upto one frame difference between top and bottom (as motion looks disorienting anyway), and at higher frame rates the jerkness dissapears and the motion difference will be much smaller anyway. I think it's possible that the horizontal oreintation of film shutters might be to get rid of the rolling shutter artifact, as movement tends to be horizontal in one direction. Whereas on the normal Video we are stuck with up and down movemnt, which is great for an overhanging camera on a assembly line.

Steve, for rolling shutter, am I understanding it right? Can you read out much faster than the 24th second exposure time, this should reduce the difference between top and bottom to much less than a frame?

One question I would like to ask about these cameras, is it possible to adjust the shutter, and gain to maximise the latitude of these cameras? Most of the time we have plenty of light and want to maximise the latitude (and minimise the blow outs), and most components have an optimum setting that expands their responsiveness. We then could use manual aperature to adjust the amount of light, therefore adjusting the range to teh desired level. So will this work Steve?

Steve to justify yourself to your boss, SI and Sumix are at the forefront of supplying this new market. But if you want the sales it has to be simple enough (aswell as good and cheap enough) to offer true pro performance at prosumer HDV prices. Ideally that would have meant either complete software, capture (with simple effective external camera/capture controlls/interface), and lense adaptor, pc computer, and display options worked out (which then can be sold to buyers), plus bundeling with one or more NLE (like Prospect HD) would be good. On the more advanced level (using the freeware prgrammable gate array work of that Russian fella (or a modified server like multiple processor mini-ITX with cameralink interface) a complete backend box system with raid storage, that can be sold to owners of any manufacturers industrial box camera. But neither SI or Sumix has expressed any desire to do offer complete simple systems, and that scares a lot of potential buyers, because it hasn't been worked out. So my guess you might be able to sell 10-100 a month as it is now until we work it out (then maybe 100+), but many more people would by a complete worked out package, and that is what people are trying to do on these threads, to effectively use SI and Sumix cameras. But what you are doing is helping to establish a new market for your boss. I myself, look forward to seeing your new HD cameras, I hope there will be 10+ stop box cameras, operating at below 2 lux, rendering true 1080 effective resolution and even three chip 1080 for $1-$3K. As you can see from the discussions that the requirements here are different from machine vision, for beautifal pictures, range, sensitivity, resolution and accuracy.

<<<-- Originally posted by Steve Nordhauser : David,

The word I hear in this group is that they don't want to compromise on what I've been calling pre-post processing - the Bayer filter, RGB->YUV, compression steps. If this can really be done in real-time, all the way up the camera scale, you have an impressive product (or will when the cheap camera link cameras are supported) and I would suggest that it is not worth anyone here writing their own code. $1K to record fully processed and compressed data with a preview window, maybe with basic camera controls would be great for the people here. -->>>

But what about the $500 cameralink bundle packages with the Prospect NLE lite software packages (actually I would prefer to pay $200, but I don't think so ;)?

<<<-- Originally posted by Steve Nordhauser :
I think the biggest processing and recording issues are a complete understanding of any step that is a potential loss of quality - like the Bayer filter. -->>>

Yes, hate bayer filter, love bayer filter (it is much of a love/hate relationship for me so far, but offer me a 8mp bayer camera and I'll forgtet all about it ;)!

<<<-- Originally posted by David Newman :
through the trouble to design and build your our camera you don't want to compromise on the quality of it output (i.e. you want 10bit or better.) CineForm Prospect HD is priced the way it is as is designed for multi-stream real-time 1920x1080 compressed and uncompressed workflows over HD-SDI. i.e. expensive hardare and the software to manage it. Do you guys need that?
-->>>

David, we don't need that but some do. So you could offer two light versions of Prospect HD, one with HDSDI support high end support, and one for DV, HDV, HDV2, box cameras, and low end professional camera formats. I'm aiming on using pc interfaces, disks and component i/o to do my transfers.

<<<-- Originally posted by David Newman : >-- Originally posted by Obin Olson :

It is designed for HD online. It is equilevent (or better) than D5 (which is YUV 4:2:2 10bit.) D5 is the workhorse standard for HD masters.

HD> David why YUV 4:2:2??? whynot RGB 4:4:4 with 10/12/16bit?
..

YUV is a more natural compression format -- optimized for the human visional system. RGB compression is less efficient. 150Mb/s compressed RGB would look worse than 150Mb/s YUV 4:2:2 (this is even true for 4:4:4 YUV although to a lesser extent.) Basically is you want the benefits of compression, YUV is the way to go.

> the whole point that this camera can shoot images that you have more control in post is 4:4:4...maybe 4:2:2 would be ok if it was high bit depth...?? -->>>


Yes that's the name I've been trying to remember, D5, I think there is also a 4:4:4 storage format as well?

My take on how good 4:2:2 is:
Industry have claimed that reduced colour bandwidth signals are a good compromise to 4:4:4 for years but I've been able to see the obviouse difference, and my eyesight is not that great. The problem is that eyesight maxes out at 2400dpi in B&W and and 1200dpi in colour, and HD is nowhere near that (approx 75dpi and 150dpi, equivalent to 150dpi and 250+dpi because it is a emissive display rather than a reflective paper display). Even taking into account the reduced resolution of the eye as you go out from the center, the maxium colour resolution of 720p is so low you would probably have to be legally blind to be completely unable to pick the difference between 4:2:2 and 4:4:4. But from my own observations and estiimates the situation is helped because in a semi trance thought (or watching movies) people tend to slightly defocuss their vision and the mind starts integrating nearby pixels at above 150dpi (720p). So I take 720p 4:4:4 to be the mininium suitable resolution for theater (except for Imax, or if your sitting near the front)) that also offers suitable detail, colour and res quality for resolution upscaling to 1080+. That is probably why they upscale SD for cinema release (but you will notice that the image details are not 150dpi small or are blended too much).

Now with 4:4:4 the colour accuracy should help a lot in the accurate upscaling (because at 300dpi, upscaled 150dpi 4:2:2 colour detail canbe inaccurate and/or equivalent to 4:1:1 double hieght 300DPI, now go to 450dpi (around SHD 8mp) 6:1:1 triple hieght colour 450dpi. At 4:4:4 150dpi you should end up with 450dpi 3:1:1 triple height colour instead, but still 150dpi accurate colour areas and detail so the colours should be well blended, and match the corresponding detail edges because of the correspondance between detail and colour (in a non bayer system). This is also very important in effective special effects,as the boundaeries should be easily and acuratly bleneded.

It all seems trivial, I know, but that is the unfortunate nature of the movie business. I also have my own unique upscaling concept that I want to try one day on true 4:4:4 footage.

Now it maybe true that 4:2:2 offers better compression, but I imagine that is because we are throwing 1/3rd the information away that the eye does not really notice at these resolutions.

I think implementing 4:4:4, bayer, and higher bit depth side support in your software would be good. It would come in handy for regular customers too, as the industry is moving to single chip consumer cameras, three chip prosumer and professional cameras, and bayer movie cameras.

Thanks

David. I am still going through this thread before I get to your over post you told me about (spent 17 hours + on catching up on the three threads, so far)

<<<-- Originally posted by Rob Scott : Product -
http://www.vinc.com/product.asp?ID=56&PID=21&SEQ=2

News articles -
http://www.cdfreaks.com/news/9852
http://news.designtechnica.com/article2925.html -->>>

If you look up Sigma Design on the web (even www.via.com.tw for embedded eden itx formfactor DVD players) I think they are the ones that make the chip sets for these. They ussually list their DVD player manufacturers somewhere on their site, or mention them in their news sections.


<<<-- Originally posted by Mike Metken : Guys,
..
There is a Sony HDC-X300 1080p camera that costs $15K. Canon HD auto focus/manual lens for it is $7K. It outputs HD SDI. If you can make the storage box work with this camera, you've made a Sony F900 killer.
..
Mike -->>>

Is that the 1440*1080 chip version there using in the Sony HDV1, and Canon XLHD?


<<<-- Originally posted by Les Dit :
There are a couple of other methods out there, one from Kodak ( patented, of course ) that seems to deal more with defect management than the actual demosaik itself.
-Les -->>>

I think somebody said that the Kodak routine was the best one out there.


<<<-- Originally posted by Steve Nordhauser :Maybe Obin could do some test sequences of fast panning on objects with large vertical lines (buildings?) to test the rolling shutter artifacts at 24 and 48fps. I really don't know how objectionable the skew really is - everyone talks about it but I've never seen images other than packing labels on conveyor belts.
Steve -->>>

:)

Well guys your doing a good job, and answering many of the questions I had, thanks, hope it all goes well.

<<<-- Originally posted by Obin Olson : uggh...I posted what i have been doing with HD at dv.com and got a bunch of child-like responses. Guess I will stick to this site! -->>>

Don't let it faze you Obin, I have seen the same on other forums, they probably will sing a different tune when a worked out system is availble. You probably will be one of the most interesting people on the forums when you have it worked out and show off real work.

Have a look at these, they might help cheer you up.

www.hoojum.com/index.php

hoojum.com/gallery/slideshow.php?set_albumName=nanode&slide_full=1
www.mini-itx.com/news/nanode/

Seen that plugin article, at least we know who has the best lossless codec at the moment "BitJazz's", sheer is close behind.

Everyone: I've gone through the whole thread and merged almost
all posts that someone made back-2-back (except for your last
large ones above here Wayne). Some posts counts has come
down as a result. No information has been removed!

Please try to avoid posting several times in a row. Stick to one
post unless you have very large posts like Wayne above. Thanks!

Steve: how can my numbers be an average transfer rate? At
fixed framerates this should be a sustained (uncompressed)
transfer rate (one way, not two way).

Obin: Can you please answer the following that was still left
unanswered?

1) do you have a RAW 10 bit pre-bayer file for us? This CAN be a lower resolution STILL file. No movie, no high res.

2) which lens(es) are you using with your camera now?

3) could it be the stuttering in that movie is due to your harddisk not keeping up?

4) which camera settings did you use to record that movie? (gain / exposure)

5) did you do any color correction or other conversion to the material?

6) Which Bayer algorithm did you use?

This are important questions to the people looking at also getting
a chip or not...

Rob S: how do you determine how many FPS you can write to disk?

Sorry Rob, so late, and I was trying to address significant posts to people individually. I don't think I will have much need to post here in the future as people are allready asking the right questions, and most significant issues have allready been covered (and I think that will be the trend in the other threads to).

Thanks Rob, your effort is most appreciated.

Rob L wrote: how do you determine how many FPS you can write to disk?I generate a set of bogus frames (of the correct size), shove 'em out to disk and keep track of how long it takes to write out n frames.

Er ... well, that's how I'm doing it with the code I'm writing. Is that what you meant, or was it more general?

Wayne: no problem! You wrote a lot as well. Your input is much
appreciated.

Rob S: well, both general and in this case. I've always had a hard
time wrapping my head around framerates and things like CPU
usage etc.

I was figuring you have one thread filling a circular buffer and
one thread reading is. When the the filling thread has caught up
with the reading thread you cannot write the data fast enough
to disk. How to express this in numbers....

Ofcourse you can write 1 GB to disk and time how long it takes
and divide that as well indeed....

It's just that I've never really dived into things like that.

Rob Lohman wrote:
I was figuring you have one thread filling a circular buffer and one thread reading is. When the the filling thread has caught up
with the reading thread you cannot write the data fast enough to disk. How to express this in numbers....Yeah, that's exactly right. I don't really have a scientific way of doing it. I have a "control block" class that acts as a gateway to the circular buffer. The frame generator grabs a buffer and stuffs in a frame at a predefined rate (24 fps, 30 fps, etc.). If the "frame generator" stuffs in frames too fast for the "writer" to handle then the "control block" marks up a "missed frame" but continues on. The "generator" runs for a preset period of time and stops. At this point, I stop disk activity; then count up the total time elapsed and the number of frames actually written to disk.

(Edit - at the end I also can determine how many frames were skipped.)