A very interestning thread indeed.

I have a suggestion over your project. I think it would be interestning to consider deliver a way to someway compress of the data to reasonably lower rate with yet maintaing good quality, 4:4:4 and 12-bit color depth benefits. I think cutting the stream by 2 or even 3 times would do no harm to its quality. If recorded on a laptop, wouldn't be a good idea to write a codec that would employ the MPEG-2 I-Frame compression, or at least MJPEG compression with a reasonable ratio of eg. 2:1 or 3:1.


regards

I am keeping all of this in mind. The important thing right now is to get raw frames and see if the output is good enough that people are interested in me doing a portable mod. At the very least, the final design allows you to select different decimations (4:1:1, 4:2:2, etc). Compression is an option as well, thoiugh i haven't implemented that yet in my design.

Update:
I am letting a pro do the soldering this time, i dropped it off this morning with a friend of mine who does soldering for the EE department, it should be ready this afternoon. After that it should be a matter of hooking it up and capturing frames tonight.

Juan

Glad to hear it, Juan. If you need somewhere to host video or pics, just let me know and i can take care of it for you.

dude check this out:

http://www.photonics.com/

they should have a beam splitter that goes 3 ways for 3 cmos chips..I don't have time to look at the site but someone told me they should have what you want

<<<-- Originally posted by Juan P. Pertierra : If I am wrong, someone please correct me.
-->>>

All the next generation Digital Cinema cameras which are inherently designed to be blown up to theatre-size screens are going to be one-chippers: Arri D20, Dalsa Origin, Kinetta HD and the Aaton D-Minima.

Oh i don't doubt that...however matthew was sugggesting using a 1-chip design for the HD box camera we are talking about here, which, unless the new generation of chips in these cutting edge cameras are cheap, is really not an option. I was talking about currently available chips that we can purchase as consumers for a relatively low price.

The whole point of the rockwell HD chips is that they can be had for relatively low price, and they already include all the necessary circuitry which further lowers the price of implementation as well as design complexity.

exactly.

when will the frame grabs be ready?

I am hooking it all up right now. It is 30 wires, but it takes time because if i'm not careful the surface mount connector can come off again and delay everything.

However, I decided to upload a very partial image. This image contains ONLY 27% of the imformation of the complete raw frame. ALSO: it still quite a lot of noise that can be seen as dark dots in sharp edges about the image...it's not perfectly focused either because the LCD screen is disconnected, and I can't put my eye against the viewfinder.

Once again, this is only the 10 red bits, coming off one CCD only. I will try and upload more partial frames as i carefully hook up more wires.

The noise is due to the fact that I am forced to use long thin wires with my test setup, because of the location of my computer. I will try and get completely rid of it once i have all colors hooked up.

http://expert.cc.purdue.edu/~pertierr/dvxcap.tiff

Juan

Correction...i misplaced one of the wires and that image is actually 9-bit ...bit 7 was mistakedly hooked up to ground so that image contains actually even less information...also this bad hookup could be the source for the speckled noise....

Eating dinner...then going back to work. :)

Juan

Well its nice to see some physical results none the less. The noise seems sporadic but groups more densely at bright/dark junctures.

I opened it in photoshop to view at 400 percent zoom as I was just doing the same thing yesterday with the same chart shot with the same camera -- but without the aid of your device. Very remarkable difference without any doubt. I can see no compression artifacts in your frame grab at all and the noise I hope is just that "noise" not damaged pixels. This could be verified very easily though, just grab 2 separate frames and compare noise locations. If they're at different locations it's noise if the same it's bad pixels in the CCDs.

Keep it up and prove all us doubting thomas' wrong -- I'm serious.

-Rodger

wow...very cool man very cool!!! :) why is it black and white? looks like it is very clear indeed!!

black and white because it is before the color filter?

Since it's only from one CCD, it is interpreted as a monochrome image...you can interpret it as red only but there's no point without the info from the other two CCD's.

Hey, These images blow up very nicely to 720p :-)

Also it does appear that you're getting more dynamic range than the camera typically puts out after gamma correction, etc. Additionally, it should be even better when you get that extra bit that accidently got left off. I'm not sure if that extra bit accounts for the slight clipping in the highlights or not, but nevertheless, it's a really nice image.

BTW, you still might want to consider 10-bit DPX log files for encoding these things to get the full range of the 12-bit image into a 10-bit image file. After Effects can read them, so there shouldn't be any compatability issues.

Well, i think there has to be a single intermittent connection, because there seems to be a 'method to the madness' as far as the noise goes...note how on the grayscale bar on the left, the dots are concentrated in one scale...it seems one of the higher order bits is intermittent, and when it is used for a tone it defaults to a dark color since it is 0....

This is gonna drive me nuts now.

Juan

Juan. I must tell this remains me of a scene in this film - "Hunting for Red October" (or something) and this guy on the submarine trying to establish back communication with their headquaters. As we see we have here the same suspense:)

And now some suggestions.

I would suggest to prepare a special testboard to actually emphasise the issues that we are talking about. So, on one hand there will be a difference between 4:2:0 (4:1:1) and 4:4:4. I think the best way is to prepare a tesboard with some very thin COLOR lines (with sections graduating in their thickness). On this board you have some tonal graduation, and there are some lines with thickness graduation as well. I cannot see it in thsi monochrome still, but I suppose they are black.

If so, they are useless as the luminance is not degraded in any way - we will not see any difference between 411 and 444. So they should be red, green or whatever.

Next thing, this time I would like to ask.

Is 12-bit thing means:

1) the "tonal range" of CCD 12-bit and DV 8-bit output is THE SAME, just there is more "tonal steps" in this range - so the white point and black point are the same

2) the tonal range of 12-bit is somewhat wider than the DV 8-bit output, so we actually can obtain from 12-bit image some tonal information that is not visible in dv (for example some details in highlight and shadow areas beyond the white and black points of DV footage)


If the 2th ver is correct, I would have additional suggestion.


PS. Is TIFF 4:4:4 medium?

Forget 720 hi-def!!! That image looks good up-rezzed 1080!!!

When Juan succeeds with this, the image should look better than HDV due to progressive quality of the CCD's, improved color and lack of compression.

It will interesting to test it head to head with HDV cams since I agree, it should uprez to 720 and 1080 quite nicely.

Actually, let me clarify something...i guess we're talking about a 4:4:4 signal, because we are capturing RGB but i think the x:x:x notation is associated with YUV signals. an RGB signal per-se is '4:4:4' because it contains the full color information...the only difference between an RGB signal and a 4:4:4 YUV signal in terms of quality is that the YUV has less reduntant information...i'm stretching it here if this doesn't sound right someone please correct me.

Thus, i think TIFF, like BMP is an interleaved RGB (uncompressed in this case) format, so in essence it has full color information, aka 4:4:4. I think the only formats that handle YUV decimation are compressed video formats like DV, might be wrong.

This morning i found an easier way to do the entire wire hookup procedure, i went to the local electronics store and gots some headers and ribbon cable...this should make everything a snap. I'm going to (getting paid for)work for a few hours and then come back to finish the (fun to do)work.

Juan

<<<-- Originally posted by Milosz Krzyzaniak :
Is 12-bit thing means:

1) the "tonal range" of CCD 12-bit and DV 8-bit output is THE SAME, just there is more "tonal steps" in this range - so the white point and black point are the same-->>>

Yes, the white and black points are the same because white is when the CCD's get saturated. What greater bit depth allows is the ability to "move around" within that white and black area and still get a decent picture with enough bit depth. For example, in 8-bit, you've clipped to white, so you decide to underexpose a stop to get the highlights down. Now in post you'll have to to re-normalize the picture so that it looks properly exposed, but keeping the whites in, so you'll use a levels or some other curves filter to modify the gamma some-what. The only problem is that since you've underexposed, you no longer have enough information per pixel to get smooth gradations-so you get banding, etc., because instead of using the entire 8-bits to describe the image, your undexposure limited the scene description to, lets say, 6-bits. Not good. A 12-bit image on the other hand, even if you underexposed and didn't use the two top bits, still gives you 10-bits to work with. So you can then normalize the picture without any problems. Also because of the greater amount of tonal gradations possible in 12-bits, the point at which a scene will clip to white will be further up the "white" scale, due to the encoders ability to describe the information in the whites. In a scene that has a large area of clipped "white", there is actual detail in the real-world scene. With 12-bits, instead of the whole area going to white, only a small portion, or maybe none at all will actually clip to white-it'll be bright, but there'll be some information there because the system had the headroom to describe what was in those whites. So yes, the white and black points remain the same in an absolute sense, but the ability of the system to "see" what's in those very bright and dark areas will be greatly increased because it's not clipping due to a lack of bits to describe what is in the extreme portions of the image.

The important thing about 12-bit capture is that you have the file format appropriate for capture available to you. Trying to cram a 12-bit file into a 16-bit file format is overkill, because there's 4-bits that you aren't using, and when bandwidth is at a premium, that can be a problem. Also taking a 12-bit linear file and clipping off the top two bits to create a 10-bit linear file, or compressing it down to an 8-bit file also does damage. That is why Kodak and SMPTE created the 10-bit LOG file format, Cineon or DPX, that using a logarithmic curve description. It's able to pack the bits from a 12-bit file into the most efficient means possible, while maintaining the bandwidth-friendly aspect of 10-bit files. The logarithmic nature of the encoding places more bits at the bottom of the scale in the blacks, and less in the whites. This is due to the way the eyes see light logarithmically-if you light a match in a dark room you'll automatically see it-it will appear bright, but if you light that same match in a sun-lit room, you'll hardly see it. So for encoding these 12-bit RAW files, probably the most economical and standards-based file format that we could use would be 10-bit Log DPX files, and you would simply linearize them in After Effects using the cineon conversion utility, or with a LUT in combustion.

With all due respect, I totally disagree. Creating a 10-bit DPX/Cineon log file would require reprocessing the image to fit into the log color space (and probably correct for "white balance")-- which probably can't be done in real-time while capturing.

The ideal situation would be to capture to raw 12 bit files, and then develop a converter application which could export 16bit TIFFs, DPX/Cineon, OpenEXR or whatever you like...

- ben

Hi everyone, I admit to lurking and following this thread for a long time. First off, I just want to say how great it is that someone like Juan is out there on the boundaries, pushing the envelope. Thank you!!

Now my first question is, and yes I am purely a creative filmmaker not technical in the least so bear with me, what kind of set up would be necessary to use this mod on location?

I shot a shot a feature on the CineAlta last year and i'm intrigued with the idea of using this mod to shoot pickups ( as opposed to spending thousands more to rend the Cine, camera pretty much ate the bughet -though it was worth it).

How could this mod be practically applied in the field?


thanks so much,


es

>>How could this mod be practically applied in the field?

Eduardo:

I have a 'production' version of this mod designed, which would be implemented as a box that mounts on the bottom of the camera, and has a firewire 800 output. You can then hookup the firewire to a portable drive, or a computer to capture the raw frames. The camera is of course closed and operates completely as normal.

The mod does not require any soldering or permanent modification of the camera, but it does need to be opened to be installed.

Juan

Great! I was worried I' d have to lug a desktop with me!

What kind of portable drives could I use? Or what kind of a laptop setup would be fast enough?

Thanks for responding so promptly,



es

Well, since the firewire interface is fast enough to handle full quality, i will probably include an option to do filtering on the software driver end so if you don't have that fast a drive, you can still record 4:2:2 or 4:1:1 uncompressed. However, for the full 4:4:4 you need a fast drive that can handle sustained 40MB/s. I know if at least one external drive that can do this: the LaCie FW800 external big drives. The new versions are even faster. I have an old version, and in all benchmarks it can do 67MB/s sustained, so it is more than enough.

My laptop drive is also fast enough according to benchmarks, but it is kinda cutting it close, so i'm not sure how other OS operations slow everything down.

Juan

Oh so close!

All wires hooked up, captured data, now i'm writing a small proggie to unpack the 30 relevant bits(10bit RGB) packed in the 32bit chunks, into a 16-bit RGB file.....

Juan

awesome, so your almost to get color eh? god this rocks man! so glad your doing it!!

Hmmm..so then, I would need a lacie with fw800 interface and that's it? or would I still need the laptop to control the hard disk recording.... that's great that I could get 4:2:2 though, no sour grapes there...

you're almost there, man! woo hoo! rootin for ya!



es

Yes, assuming you have power nearby, you can hookup the firewire directly to the drive. I am implementing a FW host on the programmable chip that's in the box mounted under the camera, so it will directly control the drive like a PC would.

However, if you're using the camera on batteries, you might need some battery for the drive itself.

Juan

Wow. Can't wait to see your results. Great job so far Juan. Keep it up.

<<<-- Originally posted by Ben Syverson : With all due respect, I totally disagree. Creating a 10-bit DPX/Cineon log file would require reprocessing the image to fit into the log color space (and probably correct for "white balance")-- which probably can't be done in real-time while capturing.

The ideal situation would be to capture to raw 12 bit files, and then develop a converter application which could export 16bit TIFFs, DPX/Cineon, OpenEXR or whatever you like...

- ben -->>>


Then why is 10-bit log DPX the SMPTE standard for Digital Cinema? Also for digital intermediate, film scanning, etc.?

Yes, that would be ideal to just capture RAW files, but then you're going to have to create a non-linear curve to expand it into a 16-bit file. If you've ever looked at a 12-bit linear file directly dumped into a 16-bit file, it's going to look very, very dark-my Canon D60 does this, and you have to then apply a LUT or color profile to the image to get it to look right. That would be A LOT of unecessary rendering in post IMHO, when you could simply apply a LUT to a DPX or Cineon file and go on from there. Also the file sizes are a lot smaller too, which means less hard-drive muscle, and more space for your dollar. Again, I don't think the DPX file format is simply there for kicks. Also there is a huge metadata header in the DPX format, where you can embed timecode, userbits, etc. That may not be something to sneeze at when you've recorded 2 hours+ of image sequences and you're trying to make heads or tails of your footage.

<<<-- Originally posted by Juan P. Pertierra : Oh so close!

All wires hooked up, captured data, now i'm writing a small proggie to unpack the 30 relevant bits(10bit RGB) packed in the 32bit chunks, into a 16-bit RGB file.....

Juan -->>>

I thought the A/D converter was 12-bits-per-pixel. What happened to the other two bits?

<<Then why is 10-bit log DPX the SMPTE standard for Digital Cinema? Also for digital intermediate, film scanning, etc.?>>

You're missing the point entirely. I wasn't dissing DPX/Cineon in general -- sure, it's a great format, and it's there for a reason. The point is: why convert to DPX in realtime as you're recording? It doesn't make any freaking sense -- the easiest thing to do is dump the data coming in off the A/D directly into a raw file. Then you can write an application (as Juan is doing right now) to convert the data into any format. If you simply must have your image in Cineon format, you can convert to it.

Although I can't see a good reason to convert
this stuff to Cineon. The whole point of the Cineon format (and log colorspace) is to protect the dynamic range of motion picture film. The idea is that you trade off image data in the darks and brights for way more dynamic range. You can protect those overbrights and sub-blacks that film can record.

But we're dealing with a strictly linear digital readout of a CCD. It exists in 10 (or 12?)bit linear colorspace. Why on earth you would throw away that data to use a file format designed by Kodak in the early nineties for a now-defunct compositing system is totally beyond my comprehension. High end compositing isn't even done in log-space anymore -- it's done in linear floating point. You may get Cineon files for input, but you have to do a log->linear conversion to even work with them.

A much more logical approach is what Juan seems to be doing, which is outputing the data in 16bit linear RGB files. There are two ways to accomplish this: padding (which would mean all of your images would seem "dark" in 16bit space) and "expanding" the 10/12bit range to 16bits, so that 100% white at 10/12bits becomes 100% white in 16bit. That's clearly the best (but slower) option. You would *NOT* need any "non-linear curve" to convert to 16bit.

And I must, must take this opportunity to point out that there is absolutely NO speed advantage to working in 10 or 12 bit over 16bit. Because processors can't work with 10 or 12 bit values (there's no data type for "10 bit"), they'll get padded to 16bits anyway -- so the speed will be identical. This is beside the fact that many image processing operations occur internally in 32-bit floating point space anyway -- regardless of whether the destination is 8bits or 16bits.

And although 16bit files do take up more room than 10bit files on disk, these files are so huge that it kind of doesn't matter after a while. If you can handle 2.5gigs/minute, you can handle 4gigs/minute.

- ben

Jason:

A/D's are 12-bit, but my test capture card can only do 32-bits at one time, so i am doing 10-bit RGB.

Juan

To Jason Rodriguez:


Jason. I asked my question about "tonal ranges" because my friend owns Canon D300 and there are 2 modes in it:

* TIFF mode - produces standard 8-bit images
* RAW mode - which, as my friend told me, produces 10 or 12 bit raw files that are just created by READING OFF THE CCD DIRECTLY.

Now. He opened one of the RAW files in Photoshop. The image itself looked bad - it was TOTALLY overexposed, there were absolutly no details in the middle and highlites. And now, the magic, one move with a slider - and he was able to recover the picture with (almost) all of the details that were previously unvisilbe (were beyond the white point of TIFF image)! Something of course completly imposssible with any TIFF 8-bit image.


So, isn't the same apply here?

Are we potentially looking at the birth of a new recording format here?

<<<-- Originally posted by Juan P. Pertierra : Jason:

A/D's are 12-bit, but my test capture card can only do 32-bits at one time, so i am doing 10-bit RGB.

Juan -->>>

Is money a stumbling block? If you need capital to buy equipment, raw materials, etc., perhaps all of us who are following your work with such great anticipation could donate to the effort.

If 20-40 of us could pitch in $50 that would raise $1000-$2000 for you, would that help build a prototype?

We would assume some risk the final product would never actually materialize perhaps in return for an "at cost" unit if/when you are able to create working ones.

I believe home-grown camera's (one or two man companies) have started to spring up and a couple of these cams will be shown at nab. These are film guys who have wrapped a body around some chips and a lens.

Just an update,

A lot of prgress! I've got pretty good quality R, G, B images out of it, but i did run into a small snafoo....apparently because of the way the prism is setup, the green frames come out inverted, so i have to modify my proggie to isolate and invert the frame before building the RGb file...i also have some bits that don't seem to be what they should, but i'm figuring it all out....

Juan

Ok, so let me first explain what this is.

This is a section of a frame, and of a very uninteresting scene, basically what my cannibalized DVX sees from it's test location. I know it is a bad test image, but you have all been waiting so i decided to release it.

This image ONLY contains about 77% of the complete color information. This is because 6 bits i can't fit into my current test card, and two bits seem to be disconnected for some reason.

There is also a hefty amount of obvious 'noise', which i am almost completely sure is caused by the disconnected bits. The noise is clearly seen as harsh black/red/etc dots peppered around specific areas.

It is slightly off focus, because i simply didn't :) Surprised i can even tell what i'm looking at....once again, no LCD no viewfinder.

I put the R,G,B layers together in photoshop. I used the default layer settings in photoshop, so this might not be the correct way of putting them together. If someone is a photoshop guru and knows how to combine three grayscale images into a color image perfectly, let me know.

Finally, a clear green hue can be seen which i beleive is normal of just about every image i have seen captured straight from CCD's, including the Thomson Viper.

I will be working this week on capturing a complete video clip, and somehow move my test setup to the balcony so I can capture a more interesting and hopefully colorful image.

Also, be sure to see it in an actual paint program like Photoshop, because browsers and preview programs perform smoothing:

http://expert.cc.purdue.edu/~pertierr/DVXcolorcap.bmp

This is the same image, but after doing an 'auto levels' on each color layer independently.

http://expert.cc.purdue.edu/~pertierr/DVXcolorcap_AL.bmp

I think there is going to be remarkable freedom to manipulate the colors in every frame, pull detail from dark areas, etc. And this is only at 9 1/3 bits. :)

wow..very cool...i see what you mean about color depth with this "RAW" file.....this is VERY exciting! dv is such a HUGE limitation for a really GREAT camera and creative project shooting just because of the compression...I love the dvx100 and hate DV ;)....Juan you may find this kinda cool:

http://www.dv3productions.com/uploads/Lens-Slingers_follow-focus-.jpg

I just got done building that unit...I will be selling it shortly...

when it comes time to build your "box" for the bottom of the camera feel free to talk with me if you want about the design and manufacture of your unit..

Yowsers! (okay, sounds like Woody Allen, anyways)

There is a TON of latitude in those blacks!!! Dang!

BTW, I will agree that the best way to record these images is just save the RAW data-with a converter program convert them to whatever format you want later. That way you perserve the RAW data, and if better algorithms, noise suppression, etc. come out later, you can take advantage of those improvements without any of the potential loss that a specific file format might incur-even if it is 16bits-per-pixel.

I still don't think that Log encoding is as antiquated as you might believe. Thomson is doing it, Kinetta will be doing it, Dalsa is doing it-all the big boys are doing it. Even panasonic is psuedo-"doing it" with their F-REC mode in the Varicam's cinegamma, at least by the indications from a white paper released at this year's HPA tech retreat. But again, I will agree with you that the best place for any of those conversions is after the fact, after the RAW data has been recorded-then like you said, we can do whatever we want.

Oops! The images are actually 8-bit/channel! None of the programs I have can save TIFF or BMP in more than 8-bit color per RGB channel, so the file format is actually cutting precision down.

Anyone have any suggestions for standard graphics formats that support at least 12-bits or 16-bits per RGB channel i.e. it would be something like 36-bit color.

Juan

16 bit tiff should be fine!

- ben

afaik TIFF can only do up to 24-bit color(8-bit RGB). 16-bit tiff would be 4-bit RGB with a 4-bit alpha channel...

Check again... 16bit/pcc TIFF is an export option in Photoshop.

It's been a few months since I looked at the TIFF file format spec, but I think the TIFF file format itself supports *any* color depth. Of course, the specific TIFF implementations in most commercial software only support 8bit and under, since other color depths are so rare.

- ben

Try professional image standards, I don't know all their names (one of them is RLA). You can find them in After Effects, Combustion and so on. Most of them could be 10 or 12 bit.

On my Photoshop CS (and I'm pretty sure 7 and maybe 6) I've got 16-bit TIFF option.

Here's one of my 48-bit scans (it's raw corrected tunsten film shot in daylight) saved as a 16-bit TIFF. Note it is 32 MB in size.

http://www.sevensmilingsharks.com/cat.tif