Mike, isn't the HD10 a 30p camera? The DVX though standard def, is 24P. 24P is what filmmakers want. This mod would be useful to filmmakers. Resolution is not the be all and end all of a filmic look.

Joel,

About 200 screens at Landmark Theaters are going digital, in all major US market. There is some government grant or something to convert about 300 screens in Great Britain to digital, mainly to serve indie filmmakers. Even places as far as India are going digital. Sony is now introducing 4K cinema projector for $60K. The current theater projectors are 2K, optical or digital. The 2K projectors have about 2 MP (megapixels); the 4K projector has about 8 MP.

were always attempts to raise film speeds to 30 fps. Some 70 mm films were made in 30 fps with parallel cameras running and filming at 24 fps for dual 24 and 30 mm release. 30 fps failed because of the extra cost, nothing else. 24 fps being better is a myth. IMAX HD is 48 fps. The digital cinemas accept 30p digital prints. No optical print is needed. Currently there are about 100 digital screens in the US.

Mike

Ya,

I first saw 28 days later on a small tv set, and was blown away by how much it looked like film . . . but then I saw it on a large tv screen, and was blown away by just how much the look had changed. It didn't look like film at all to me then. I can only imagine what it would have looked like on a theater screen. I mean, the look worked for the film it was, but don't be too fooled if you haven't viewed it on at least a really big tv screen. What got that flick into national commercial theaters, was the fact that the director was already famous, and that it was a relatively good horror script, IMHO. Could never get away with that as a first time filmmaker with a drama.

I'm really stoked about your mod, Juan. If it works with great color separation, I think the camera you could turn the new Canon to come into just might be the end of the Hollywood grip on indie filmmakers.

And I LOVE you're considering Mac OS file system, becasue I personally am going to get a Mac soon (sick of unstabel pc) and need a great HD cam to go with it.

Hi Mike, Thanks for the info. I've seen and worked with 30P. I've seen and worked with 24P. I prefer 24P. The reasons are subjective no doubt and hard to describe, but that does not imply a myth. Is the fact that a drama at 60i looks like crap any indication that a higher frame rate does not necessarily mean better? It's not worth debating, but I'll give you my opinion anyway. Would higher frame rates be more "realistic"? Probably, but 24P (whatever the initial reasons for its adoption) seems to have hit a sweet spot in human visual perception that triggers us to a movie's "unreality" and its stylized difference from the crisp reality we know. Using an exaggerated example to illustrate, its like looking at a painting of someone instead of a photograph of them. It helps us to use our imagination a bit more, and it's a slight remove from reality that seems suited to fictive cinema.

Hello Juan, everyone, another sideliner come out to speak here. Please bear with me here:

This is my suggestion. I'm just as excited as everyone else for Juan's Mod (especially after playing with the DV grab, and the Raw grab from the green screen test he posted a little while ago) but I'm afraid that this thread is getting a little...convoluted. I'm pleading with everybody, to please not let posts and conversations get any farther away matters at hand. What we need is to stay focused on getting a fully functional prototype system that allows one to capture 24fps 12-bit Raw files out of the DVX and write them out to a portable firewire 800 hard drive...that's IT. If people are interested in pushing other cameras (XL-1/it's sequel in limbo, that terrible JVC 'HD1U/HD10U' camera, or any others), please don't fill up any more pages of space on this particular thread asking Juan to start working on other cameras when he hasn't even finished the DVX-100 mod yet. Why can't we move some of this extraneous stuff to a sister thread of some sort, (ex: "Alternative Applications of Juan's 4:4:4 DVX Mod")?

Please, I'm begging everyone let's not forget, this thread is about the DVX-100 and it's already running 54 pages long. I think it's fabulous that such a dream come true of a project has been undertaken, found support and been so successful thus far. I'm just begging everyone not to get it bogged down here on other cameras and drives that Juan has explicitedly said he isn't planning on working on here (the dueling file formats, ext2/ext3/Fat32/ect, ect is tedious but necessary at this point in development I suppose) IE: if he says he doesn't want to work with that equipment or peripheral right now, let's respect that and not keep asking him.

Other than that, I'm excited to see things coming along and yes Juan you can chalk me up for your list when the happy day comes and this thing is complete.

I need some help with this:
~~~~~~~~~~~~~~~~~
What exactly is a white balance adjustment? Initially i thought using the 'white level' in the 'levels' dialog in Photoshop, but that cuts off a ton of latitude, and that doesn't happen when I use the white level adjustment in the camera, so it's not the same procedure.

I'm trying to find the answer to this, but i'm sure somebody else here probably knows...

Also need help with..
~~~~~~~~~~~~
Verifying my R,G,B alignment procedure.
1.Open any psd R,G,B layered RAW from my site
2.Scale ONLY the red channel down by 1 pixel in both dimensions. Do this by using the Edit->Transform->Scale and resizing the RED(named "Background Copy") layer down by 1 pixel in both dimensions.
3.Select the move tool and use the keyboard cursors to 'nudge' the layers and attempt alignment.

Let me know if you can get a perfect alignment using this method. I've tried it on 2 images so far and it seems to work perfectly. I'd like to make sure.

Thanks!

Hi Juan,

aligning the channels the way you suggested seems to work fine. I created a photoshop action for it:

http://www.hot.ee/whiteroom/alignment.atn

the red channel seems just a tiny bit high, but generally it's acceptable. I used the resolution chart you posted a while back.

As far as white balance goes, Auto color in photoshop seems to do the trick (should be used after levels correction)

I was taught that the way to color balance an image was to look for areas that should be a neutral gray tone (you should be able to find one in any image... a wall, an eyeball, white fur, etc.) check each of the RGB values (it works best if you can sample a bunch of pixels rather than just one). If any of the three values is considerably more than the others, then that's the one you need to fix. If there's too much red, add more cyan... too much blue, add more yellow... too much green, add more magenta. When all three values are equal then the image should be correct. That's essentially what the camera does when you white balance... that's why you have to shoot a white piece of paper or white wall when you do it... or sometimes DPs will whitebalance through a blue gel if they want to get a yellowish tone or through an amber gel if they want a blueish look, etc. It gets tricky with images that are mostly rich color, but you eventually develop an eye for it. Anyway, there should be a color balance pallate in Photoshop somewhere. In the adjustments menu?

Also, I've been away from my computer for a few days but I did get a chance to play around with the image you posted a week ago. I've been meaning to post my results, but you got to it first... you're correct Juan. The Red channel seems to be getting scaled down by 1 pixel somewhere in the process of being squeezed down to the DV resolution. That's why the DV image is perfectly aligned. It's really strange... why do you think the camera needs to do that? Is it compensating for a natual error in the beam splitting or something... or the inner components? I understand the concept of comressing the 773 image to 730 to get a considerably sharper image (particularly that defocussed blue channel) but this oddly sized red channel has me stumped.

John

That's great that you are all getting the same results.

My best guess is that this is to correct for the geometry of the prism. The re-size of 1 pixel is so small that any small difference in the optics could cause that.

The auto levels-auto color approach seems to work...here is an HD version of one of the last captures...this one was originally a VERY dark image and it is amazing how much detail you can pull out:

http://expert.cc.purdue.edu/~pertierr/cap10_HD.tif

This one was aligned with the new method.

I get the anamorphic adapter this wednesday, and I can't wait! :)

Juan, I sent you via email a grab of a color chart from www.pixelmonger.com in case you can use it for white balance alignment of the rgb colors.

By the way, Juan, WOW! Just checked that clip of the ext. night shot you uploaded. REALLY NICE!!!!!! How many stops of lattitude are you getting on that would you say?

actually Juan, on the last cap you posted (cap10), the blue ch. alignment was off by 1px to both down & right on 1240x794 scale i.e. move up & left to get it right.

Thanks Helen...i will re-do it when i get home today.

Laurence, I'm not sure how many stops that is but i'm going to take some shots on film for comparison. Problem is, last time i tried i used ISO400 film because i thought that was closest to a CCD, but I couldn't get a low enough exposure, they all came out very washed out.

Will try again tonight....

Anamorphic countdown timer: 24 hours <g>

Here is another HD test. I took 35mm stills of this one using 100ISO film this time. I'll have them developed tomorrow.

Only did an auto-levels and an up-rez with base settings.

http://expert.cc.purdue.edu/~pertierr/cap11_HD.tif

At last! I just hooked up the exposure control, so I now have iris control during RAW capture.

If anyone has any suggestions of what tests can be done with this important aspect of control, let me know. I still have the scene from the previous post setup.

Juan

Juan,

Responding to your request for suggestions:

One thing that I noticed about most of the images that you have posted is that when I adjust the levels in photoshop, I notice that they have most of their picture information in the dark through mid range, and very little picture info in the highlights. I was starting to wonder if maybe the CCD just didn't go that high, but this last one, cap11 HD, seemed to have at least some info at the high end.

So, I am interested to see how the raw images deal with the upper end of brightness; how it transitions to pure white. Now that you can control exposure, we can finally see just how far you can push this thing.

Thanks Justin....

Here's what I had in mind...setup a similar scene to the last post, and take RAW and DV frames in small exposure interfvals.

I'll try and put together a collage of the DV and raw frames so we can see how much more you can push it.

Haven't gotten the anamorphic yet, any minute now.

Gonna get a gyro, then i'll start the tests.....

Juan

Here's a thumbnail of the images i'm getting with he anamorphic adapter. I did some color correction on this one and applied a log curve...

http://expert.cc.purdue.edu/~pertierr/cap12_pre.tif

the few dots you see are speckles. I've been trying to wrap my head around the myriad of aspect ratio changes involved, i think i got it close but it might not be perfect.

Juan

Juan how are you going to deal with 12bit files? what can edit a 12bit file? I know after effects can open 16bit files but 12? I am having the same issue with the 4:4:4 10bit camera

Put the 12-bit data in the upper 12 bits of 16 bits. Set the lower 4 bits to 0. It's the same thing. The data is 16-bits in size but 12-bits in precision.

Juan

so then you waste disk space ? or does the 0 mean nothing in the bottom=16bit file that is the size of a 12bit file?

At processing time, having 16-bit data actually helps because even though you start with 12-bit precision, all color corrections/FX are done in 16-bit precision.

If you are concerned about storage, then you need to PACK the data.

Computers works in 8-bit bytes, so if you want to use the maximum amount of space, you have to pack the 12-bit data across byte boundaries, which could require some basic error correction to make sure you don't loose data somewhere.

do you know a good fast codec for packing? I will need it BAD as a tiff file at HD is about 5 megs! that will never edit in a computer unless its a SMOKE or some fancy box..do you know anything about BitJazz the makers of the SheerVideo codec?

35mm vs DVX100 RAW test
~~~~~~~~~~~~~~~~~

35mm Fujifilm ISO100 image:
http://expert.cc.purdue.edu/~pertierr/35vsDVX_35.jpg

DVX100 image:
http://expert.cc.purdue.edu/~pertierr/35vsDVX_RAW.tif

These were taken yesterday, without the anamorphic adapter. The scene is exactly the same in both images, the only differences being the camera/medium, and the film camera was located higher and to the left since the DVX is not very easy to move right now.

Juan

Juan, great example! The DVX image looks excellent. Will the speckles in the RAW DVX image ever be gone completely? I've noticed those in previous images as well. I recall you saying that would be fixed with a different capacitor, I could be wrong though.

Yes, the speckles will be gone in the prototype. They are simply a product of the noisy experimental setup I am using right now.

I haven't had much time to find the exact size capacitor that will get rid of them completely, though they have been minimized...before there was a TON of them.

I am currently working on the DV/RAW latitude comparison....

Juan

Juan,

The DVX/35mm comparison looks really good.

The only suggestion I would make for future comparisons like these would be to be careful about the angles. Using such a hard light, the angle of reflectance is different (on the box and bottle) from the film image to the digital one. This throws off the comparison quite a bit, because the DVX placement gets a harder reflection, and therefore blows out sooner than it would from the position where the SLR is.

A more accurate comparison could be achieved by taking the picture with the film camera, then replacing the camera with the DVX, to take a picture from exactly the same spot. Then, the lighting would be identical. Maybe I am being picky here, but I have a feeling that if the lighting was more similar here, we would be even more blown away by the similarity between the two images.

Justin,

I know what you mean! :) I really tried to make the images as similar as possible but the two problems were:

1.Differences between the lenses and formats, the SLR had a wider field of view and no zoom, so i had to get much closer

2.The current experimental setup really doesn't let me move the DVX without making some large changes...

In a few weeks the prototype should be ready, which will allow me to make a comparison from the exact same position...

Juan

Gotcha. Makes perfect sense.

Latitude Comparison between 4:4:4 10-bit Uncompressed output and standard DV Output
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Ok, this took a while so i hope it proves helpful.

An important note: this set of images is not intended to demonstrate any aspect of resolution or color, just to give an idea of the difference in dynamic range.

I compressed the file HEAVILY to make it a fast download, so the colors and resolution where very affected. The original file is 57MB, so if you would like a nicer version let me know.

http://expert.cc.purdue.edu/~pertierr/LatitudeTest.jpg

Wow Juan!!!!!

Incredible!!!!!

Hey, I don't know if you're hip to Scott Billup's site, but it's www.pixelmonger.com

He's a very knowledgable guy, has built his own mods, etc. You might email him there with questions about how to do tests, etc. And give hiim details about what you're doing. He's very helpful and loves pulling along the fellow man.

You probably already talk to him, but in case you don't he might have some really useful insight.

Not sure if this will help or not but I found a codec for free that supports 16 bit per channel as well as an alpha channel making a 64 bit video codec. It works on mac and pc with just quicktime 5. Best of all it is free. They even have a lossless codec that can get 6:1 compression with no loss but that codec is $99.00. I know it isn't 12 bit per channel but it might be an easier way for people to manage files opposed to a series of stills. Besides right now the tiff files will need to be 16 bit anyways.


http://www.digitalanarchy.com/micro/micro_none16.html

just adding to my last post.


the $99.00 codec will also support an audio track which is something you cannot get with tiff files. I am starting to like this Microcosm codec. The only problem is that we may not be able to write to it since it seems to be a little slow. Juan maybe you can download the free codec or the demo and see if it would work for capture. Digital Anarchy states it wasn't meant as an I/O codec but then again we arent exactly doing I/O in the normal way. Even if it doesn't work for capture the Microcosm codec sure would be nice for storing footage with 3:1 to 10:1 lossless compression. I am sure you or one of us could always write a small utility to convert tiffs to this codec once capturing is done. Even 3:1 compression would save us a lot of drive space for our projects.

I just downloaded and checked out the lattitude test.
If it isn't too much trouble, I think it would be of more help to upload 2 frames that we can look at that are each properly exposed. 1 from the DV output, and 1 4:4:4 uncompressed.

From what I saw in the sample, the 4:4:4 does not seem to have any more lattitude than the DV frames. The DV seems to simply be brighter overall. I mentioned in an earlier post that I thought this was probably due to the DSP, or the DV encoding, adding a certain amount of nominal gain to the image. That is pretty common in cameras. Our '0db' settings generally don't mean there is no gain.

The 4:4:4 frames are definitely darker, but they don't seem to have any greater contrast range than the DV frames. In fact, if anything, they seem to have a lower contrast ratio. The shadows don't seem as dark, and the highlights are not as bright.

In fact, while the DV frame at about 9.6 or maybe 11 seemed to be a properly balanced and well exposed frame, none of the 4:4:4 frames seems to be properly exposed to me. Either the entire image seems too dark, or too bright and milky.

It seems that although you are dealing with more increments in the luminance due to the 12bit image (4096 vs. 256), the extremes of that range may be the same.

The ideal way to check this would be to shoot a clip chart in both DV mode, and 4:4:4 mode, each properly exposed. That way we could easily see the range in grayscale that the chips can handle.


-Luis

Luis,

I have to say I completely disagree.

First of all, whether the frames look dark or milky is irrelevant. They are completely not color corrected. I can color correct almost all the frames in the left side to look good, while most of the DV frames have entire sections of white.

How do you explain the fact that there are DV frames which have huge areas of clipped white while the raw image still captures tons detail in that area? Remember, we are not evaluating the raw image subjectively, but rather how much information is captured. No image straight out of a CCD is going to look 'good'. Have you seen the un-corrected output from the Viper?

Finally, just drive this point home. The left image IS exactly what the camera starts with, before it processes it and outputs the image on the right to DV tape.

How exactly do you conclude there is no more latitude if the camera starts with an image with diverse information all over it, and spits out an image that is 50% white?

Maybe the complete jpg is just too compressed...the original file is 56MB in photoshop, maybe I can upload that and you can get a better look.

Juan

Damn it Juan, you post too fast.
I was coming back to edit my original post.

I think the jpg is just too compressed to do any real analysis,
I started trying to compare each frame's histogram, but realized it wasn't going to tell me much. Ideally we need 2 seperate frames we can compare, each properly exposed.

Don't upload the full file for me, as I'm on a dial up and can't download something that big. I appreciate it though.


"First of all, whether the frames look dark or milky is irrelevant."

I completely agree. I was only pointing that out to talk about the range of luminance values in an image. It is not irrelevant to point out that an image is too dark if at the same time someone is pointing out the the highlights are not clipping. That was all I was saying.

"They are completely not color corrected. I can color correct almost all the frames in the left side to look good, while most of the DV frames have entire sections of white."

You are right about that, that was what I was coming back to edit my post for. I fiddled with the 4:4:4 frames for a minute, and saw I could get them looking just as good, if not better. Again, cant' tell much from that much compression though.


Do you have each of those files still saved seperately?
Perhaps you could upload the f9.6 DV frame and the f5.6 4:4:4 frame? Those seem to be somewhat close to eachother, as a good starting point.

"How do you explain the fact that there are DV frames which have huge areas of clipped white while the raw image still captures tons detail in that area?"

I thought it may have been due to the gain being added in the DV compression. Not only are teh highlights clipped, but the shadows are lighter as well. It seems the entire image was brightened, not just clipped at the highlights.

"Remember, we are not evaluating the raw image subjectively, but rather how much information is captured. No image straight out of a CCD is going to look 'good'. Have you seen the un-corrected output from the Viper?"

No I haven't, and perhaps you're right. I need to remember what it is I'm looking for, and perhaps I misunderstood the outcome of what I saw.

"The left image IS exactly what the camera starts with, before it processes it and outputs the image on the right to DV tape."

I understand that. I guess my only problem is that you seem to be comparing the DV and 4:4:4 images at the same fstop. The 4:4:4 frames are obviously darker overall, and I feel we should be comparing properly exposed frames.

Perhaps I just don't understand what to look for.
I do know that it would be great if you coudl upload those frames seperately and uncompressed (just 2 of them, not all of them).
Then maybe this will clear up for me.

In the meantime, great work.
Sorry if I confused the issue.

Keep it up.

-Luis

Just to better illustrate the point, this site has a 'raw' image from the viper, and several steps taken to make it more film-like:

http://freespace.virgin.net/shaw.clan/dpviper.html

I say 'raw' because anyone who knows how the viper system works, by that point it already did some corrections to get it white-balanced. The original viper image is greenish, just like the raw output from any CCD.

CCD's capture LINEAR data. Until someone invents a logarithmic CCD, you WILL get 'milky' images out of the CCD because that's what a linear response to light results in.

REMEMBER: Like i warned up in the comparison image, this is ONLY for latitude and dynamic range.

The RAW data is LINEAR uncorrected data from the CCD's, while the DV OUTPUT has all the brightness, contrast, sharpness, color and film-look corrections of the DVX!

Luis:
Yes, i do have the original images but I am going to try and make a slightly bigger JPG...not sure why it got compressed that much.

Juan

I just uploaded a much better JPG of the latitude test available at the same place as before:

http://expert.cc.purdue.edu/~pertierr/LatitudeTest.jpg

Juan,

I checked out the viper link you posted, good example.

I also downloaded the new latitude film you uploaded.
MUCH nicer without all that compression.

I now see what you are saying.
The 4:4:4 at f16 is catching the highlight off the bottle, but can still be pushed to reveal all that shadow detail. The same shot in DV mode (I mean if you underexposed that much) would just crush all that shadow detail into oblivion.

Hope I didn't create confusion for anyone with my posts.
Disregard my earlier posts.

-Luis

I was also thinking, if anyone wants...pick an exposure or two from the image, and I will color-correct the RAW images and post it in full size, together with the full size DV images.

Juan

Say guys, I copied this from another thread:

Not sure if this will help or not but I found a codec for free that supports 16 bit per channel as well as an alpha channel making a 64 bit video codec. It works on mac and pc with just quicktime 5. Best of all it is free. They even have a lossless codec that can get 6:1 compression with no loss but that codec is $99.00. I know it isn't 12 bit per channel but it might be an easier way for people to manage files opposed to a series of stills. Besides right now the tiff files will need to be 16 bit anyways.


http://www.digitalanarchy.com/micro/micro_none16.html

how did my post get into a different thread? For me it shows up right in this thread. I also posted a followup.

http://www.digitalanarchy.com/micro/micro_none16.html

is the latitude of the normal dvx100 5 stops or 6?

what is the lat of the uncompressed version?

35mm cinema film is 13 stops?

thank you,

jake r.

has this thread died?


Anyways if you are out there Juan I was curious as to how hard would it be to just get 8 bit uncompressed 4:4:4 off of the mod. I have been doing some tests and except for really fine color correction 16 or 12 bit video does seem a little over kill. Whenever I convert these sample frames down to 8 bit they still look amazing better than anything I have ever seen from a video camera. I did some keys at 16 and 8 bit and I can't tell the difference. Color correction did suffer a little bit but not much. The only time you would really need that much range for color is if you really screwed up shooting and you had crappy footage.

If this isn't possible then I guess I could just convert to 8 bit when everything is captured.

In my AE work, sometime moving to 16 bit really helps banding and gradations, other times, not at all. It really depends on the source material. I agree that 8-bit is easier, but if 12 or 16 bit is possible, I would still like to have the option to start their and move down.

Hi,

Nope, not dead, i've just been working non-stop, i guess it's a good time for an update.

I have the camera closed and completely assembled for the first time in over 6 months. The flex cable slides out the bottom perfectly. I have the hardware prototype all hooked up and i'm right now just working on the software configuration of the FPGA, which is what's left to be done. Once again, all the probing done to the camera was done with absolutely NO SOLDERING.

My plan once I get it done, is to get one or two units together and lend them out to people who can try them out and give me feedback on any suggestions or added features.

About the 8-bit video, that's easy. Hadling 12-bit is actually much harder because it doesn't align to word boundaries. My hardware allows the user to capture several different color precisions to save space.

I understand what you mean about the 8-bit video. However, the problem with DV footage is not really that it is 8-bit, but more like they compressed the 8-bit values to a smaller dynamic range, loosing latitude. With the XRS-1(random name for my device) you will still get the full latitude of the CCD, just decreased precision to 8-bits when set to that mode. At full quality it will give you uncompressed 16-bit RGB color files with 12-bit RGB precision.

I've been busy with this, but if anyone wants to see any more tests, just let me know what you want to see and i'll take them.

I still have all the ORIGINAL frames from the latitude test, so if you want I can color-correct some of the RAW ones at different exposures for comparison with the DV frames. Remember, all the RAW frames in the latitude test were not color corrected, so they don't even have white balance adjustments done.

Juan

Hey Juan,

If you need people to be beta testers for your mod... I would be more than happy to donate my DVX to the cause. :)

So, what is the latitude of the
uncompressed dvx-100?

Thank you,

Jake Richardson

I'm probably not the right person to answer the question in photographic terms, i'm more of an engineer. But from looking at the latitude test results, it seems that there is a 2 F-stop difference between the optimal scene exposure for DV and RAW.

Can anyone else look at the images and see if you agree with this? I can post the original frames if needed.