View Full Version : Data to Film with Cineform RAW and Prospect 2k


Sergio Sanchez
May 21st, 2007, 05:33 PM
Hi:

I`ve shoot some material with the SI2K camera, and im doing some tests for a feature film.
All the workflow has been based on the Cineform codec. Everything has gone well, except some early problems solved with the software upgrades from Cineform.

I`ve rendered the master of a short film im planning to output to film as well as the camera tests for the film.

I have a couple of questions about the codec:

1) How can I know if the AVI files rendered in Premiere or After Effects are Linear or Log. This is important because i`ve been having problems with the VFX guys, they say all the material im giving them, is linear I`ve been comparing the Cineon files i gave them with the original RAW`s and the image looks very different, color has a lot of saturation and the dinamic range is lost.

This is an important issue for us because we are replacing the face of an actor and the VFX guys dont have the Cineform License and we dont want to get lost in translation.

2) Whats the bitdepth of the Cineform Quicktime files. When I compare it with the original RAW or with the exported Cineform AVI Filmscan 444, the quicktime has more detail and contrast than the original.

Im going to finish in Combustion, because the color managment in AE is very tricky, and the Cineon files im exporting dont look as the CineformAVI`s i feel im loosing color.

What do you recommend me to do

Thank you

David Newman
May 21st, 2007, 06:42 PM
Firstly, your effects guys are welcome to NEO Player, which provides a free decoder license that works with CineForm RAW and CineForm 444. If they are Adobe/PC based this might be what they need. Although without a CineForm encoding license they would need to return uncompressed file to you.

1) When you are generating Cineon files you have several options, log or linear or your own black and white points. Note: remember there is a terminlogy issue that some mistaking call 2.2 video gamma, linear -- is it not. Note 2: avoid true linear for exports, it is great for VFX work, not great in a 10-bit file, log is best for dynamic range, even 2.2 gamma is better than linear.

If you want you exports for VFW work with your look active, you want to make sure you don't clip highlights, as the look file can generate an HDR image. As the SI-2K image have there own curve, and the 3DLUT maybe another you what to normallize this for your VFX exports. Place your AE composite in 32-bit mode, and the CineForm importer will convert the image to a try linear 32-bit float, this image may have white levels greater than 1.0. When you export to Cineon/DPX you want to preserve you dynamic range by using a log curve, while you can customize your own white & black points, for convience just select "Standard" from the Cineon file "Format Options..." The "Logarithmic Conversion" should be set to convert 32-bit linear to 10-bit Cineon log. At VFX end these exported files will look flat, with a grey level for black, you Cineon Convert should be applied to convert log but to linear for processing.

If you disable the camera's look, the RAW image will not clip highlights, in this senario you would be best to export the 10-bit data directly without any curves applied. Note: you can use the above method for a standard Cineon deliverable, there will be slightly more tonal information in the RAW 10-bit export. To do this you want to use the 16-bit mode in After Effects, this way the cameras curves are not reversed. Then export to Cineon with the preset set to "Full range", with sets the point at 0 and black at 1023 with no gamma or log curve applied. Will the Cineon file will be flagged as linear the data within is camera log. This is a robust way of more data between tools.

2) The bitdepth of CineForm 444 is 12 whether you use MOV or QT wrappers, however there should not be any visible difference in detail or contrast -- I just did a test and all is fine. Make sure you set 444 and trillions of colors, allow in you are in 32-bit linear mode, you will what to apply a curve, just as you would in the Cineon export, you can use the Cineon Convertor filter to apply a standard cineon curve to your CineForm 444 exports.

Salah Baker
May 21st, 2007, 06:55 PM
I hope this is going in the new manual :D

David Newman
May 21st, 2007, 09:10 PM
It take less time for be to write my part of software and respond to questions online, then for me to write a manual. The whole of log vs linear processing is a book, not a manual entry, or read the entire http://prolost.blogspot.com.

Sergio Sanchez
May 22nd, 2007, 04:41 PM
Thanks for the advice David, i`m rendering files, lets see how they look.

I was thinking, if there is any possibility of keeping the files in RAW format, for the final correction in its native format. I`ve heard from someone that you are working with the Assimilate Inc. crew to support the Cineform RAW codec natively on the system.

If one could export somehow the project edited on PPro into a Scratch and finish the DI in that system in RAW it would be great.

Is that possible? For a not so distant future?

David Newman
May 22nd, 2007, 05:05 PM
All possible but we are announce partners for CineForm RAW processing yet.

Sergio Sanchez
June 20th, 2007, 01:26 PM
Ok, i did that and this is what happened.

Im doing the data to film in a Baselight with direct feedback from a Barco 2k projector, i`ve exported in 16bit mode, with no look applied, and as you said the Cineons exported are flagged as linear. The problem i have is that the footage is reacting as linear too. The highlights are clipping and i dont have room in the midtones.
In fact i`ve delivered different exports cineform filmscan2 444, cineform quicktime 444, Cineon sequence, and Quicktime h264.

In therms of detail all the files looked great, the problem is that if i try to push the midtones say 2stops some horrible signal noise apears on the shadows and mid areas of the shot, I dont believe this is a codec issue, i think is a sensor issue. So i have no way to get more detail, what is worse is that the curve breaks apart, and looks more like a waveform due all the cliping in the signal.

The surprise was that in terms of latitude the quicktime showed a better range in the vectorscope, it had more dinamic range, the noise is there if i pushed up the midtones, but i had more space to work since i was like 2 stops up.

Right now i believe that all the workflow is planned on using Adobe tools in the entire post workflow, but what happens if someone wants to finish in a more sophisticated DI enviroment? I`ve been doing the testing on the color finesse, but the real thing will be done in a Scratch and the final adjusments in the baselight.

I believe that the need to render out the material is limiting the capacity of the footage. What i mean is that you`re giving me a negative to work but i can manipulate the material only in a filmprint, thats what i think is happening, because when i color corrected in After Effects i had much more room in the RAW than in the Cineon used in the baselight.

My question would be... How can i mantain the footage integrity, during the post workflow if im not finishing the DI process in Adobe tools?

Jason Rodriguez
June 20th, 2007, 02:23 PM
Hi,

Expect an online video tutorial on this topic to help clear up some of the confusion. There are a couple other more basic ones I would like to get through first, but this is definitely a must-do topic that I plan on covering.

You are not locked into using Adobe tools. I think the only limitation right now is that one requires Adobe tools in order to change the metadata settings, but that will be changing soon as well.

Thanks,

Jason

David Newman
June 20th, 2007, 03:33 PM
Thanks Jason.

Sergio,

Somehow this seems more complex than is actually is. If in doubt use 32-bit linear and export to TIFF 16-bit per channel, that will represerve all you tonal range if you have the metadata switch off. In your DI suite, white-balance the clip(s), then reapply the look. You can do the same with Cineon you just need to be careful with the controls -- never encode linear as linear in a 10-bit Cineon file as there is not enough precision. With AE in 32-bit mode (photometerically linear), export to Cineon with "standard" and logarithm mapping, this will reserve the dynamic range and easily handle by any DI system. This is very like working with Viper Filmstream data.

Jason -- Can you use the in-built speed grade to remap the look to linear? Current look file are typically SI-Log mapping 2.2 display gamma. It would be nice to remap the same look to linear to linear and linear to display gamma. For this would be no issue for SpeedGrade DI. What we need a simple LUT remapper for DI suite that use linear or cineon curves, etc.

Sergio Sanchez
June 20th, 2007, 03:55 PM
"- never encode linear as linear in a 10-bit Cineon file as there is not enough precision. "

This is what i dont get, how could i be encoding linear to linear data if it is supposed that the RAW files are logarithmic?

I always disable the lookS for me they serve for preview puposes only.

David Newman
June 20th, 2007, 04:28 PM
OK.

1) Camera has a custom log curve design optimium shadow detail through compression. See image Camera Log image.
2) Your DI suite doesn't know this curve.
3) You need to remap to a curve you DI suite knows.
4) With RAW metadata off in AE 32-bit mode, the output is photometerically linear, the SI curve has been removed for you. See image Converted to Linear.
5) Add the "Cineon Converter" filter in AE, and use the Linear to Log preset. The could also be done with the Cineon export, with the "standard" setting. One or the other not both.
6) Export to Cineon with the "Full Range". See image Remapped to Cineon Log.
7) In your DI, tell the software that you are using standard Cineon LOG 10-bit.

That is it.

Sergio Sanchez
June 20th, 2007, 05:21 PM
Thats what i did, but in my case the source RAW looks exacly the same as the Cineon exported files.

The same happened to the guys at Ollin the exported some material to cineon and it looked exactly the same as the source files.

David Newman
June 20th, 2007, 05:25 PM
You made an error in not using the 32-bit mode to linearize the source data. If you missed this step you will be carrying the camera curve into your DI. Try again making sure your images look like those above at each step.

Sergio Sanchez
June 20th, 2007, 06:29 PM
[QUOTE=Jason Rodriguez;699879]Hi,

Expect an online video tutorial on this topic to help clear up some of the confusion. There are a couple other more basic ones I would like to get through first, but this is definitely a must-do topic that I plan on covering.

Jason:
When do you think you`ll have the tutorial ready...Because Ari is sending almost every latino interested in the SI2K with us and they are only interested in watching the data to film. The interface and operability of the camera and workflows are interesting to them only after looking the results.
And right now im having little trouble with the results.

I`m going to try the exports again, and ask the guys at Ollin to see how they made the exports, i dont think they didnt know the precise steps to export to Cineon. they do some major DI and VFX works for the big guys here and in the US as well.

Jason Rodriguez
June 20th, 2007, 09:29 PM
Hi Sergio,

As David noted, you messed up the 32-bit export process . . . I've been though this step many times before, and I know that it does work correctly.

I understand your need for tutorial information, but I can only do these at a certain pace, and they are being done in addition to the other tasks on-hand to get the camera shipped. Additionally there are other pressing topics to be done as well. So please be patient, but I definitely understand the need and will be making tutorial information available as soon as possible.

In the meantime, please look over David's steps, as this is where the mess-up is happening. If you did the 32-bit approach that David described, you have most likely accidentally exported photometrically linear data, not Cineon log data, and then the film calibration curve in the Baselight is being applied to the linear data, which will of course make the footage very contrasty and massively compress the dynamic range of the perceived image.

Sergio Sanchez
June 20th, 2007, 10:55 PM
Jason and David

I`ve made some quick test and that is what i`ve got.

I guess that the post guys (and me) didnt export correctly the LOG files.

Thank you for uncovering the misteries of the cineform exports in AE.

Sergio Sanchez
June 21st, 2007, 01:23 AM
Here`s another one, this one was shot with the latest GigeAPP and later Cineform RAW, i believe is a more accurate example.

I made a fast color correction example of the exported Cineform in Fusion.

Today im having a film out preview, i think this will be very handy.

Thank you both.

Jason Rodriguez
June 21st, 2007, 06:31 AM
No Problem :)

These images now look correct to me.

Jason Rodriguez
June 21st, 2007, 06:39 AM
Jason -- Can you use the in-built speed grade to remap the look to linear? Current look file are typically SI-Log mapping 2.2 display gamma. It would be nice to remap the same look to linear to linear and linear to display gamma. For this would be no issue for SpeedGrade DI. What we need a simple LUT remapper for DI suite that use linear or cineon curves, etc.

Yes, this can easily be done. We actually have a couple different gamma curves that we map to from the initial Log->Linear->Matrix->gamma curve conversions that we do in the "Matrix&Lut" section of OnSet, and I touch on that in the tutorial video that I had posted on SpeedGrade OnSet.

Basically if one wanted to relinearize the look, simply take out the post-matrix gamma curve. So this could very easily be done in SpeedGrade DI, etc. since the .look files import directly into that app, and one can change the attributes of the post-matix gamma curve.

What I've also done is setup a Cineon curve option in the post-matrix gamma option, so instead of re-linearizing, one can also very quickly make the .look files Cineon-compliant as well. Since SpeedGrade DI can also apply a film calibration viewLUT, you can then grade in-context of the film-out, but not bake the film calibration LUT into the .look file.

Sergio Sanchez
June 21st, 2007, 10:02 PM
Thats great, it will be very helpfull to be able to apply a film print .look.
That gives you the idea of how your scene will feel in theaters when shooting.
It would be great to have the preset, because most of the times the guys operating the camera are used to film or tape adquisition. And the LUT`s and color spaces are more related to postproducers, and vfx guys.

Its a very good idea.

Thanks

Jason Rodriguez
June 22nd, 2007, 06:50 AM
The other nice thing is that we support a separate ViewLUT from RecordLUT in the camera, so you can still record a Cineon-compliant file, while watching a "film-print" on the monitor.

Sergio Sanchez
June 22nd, 2007, 03:31 PM
Thats great a really useful tool. In fact we were finding the way to create a film out print accurate for be displayed on the camera. But i think i`ll wait to use your preset.

In the last version of the GigApp i`ve downloaded, maybe a month ago, i noticed that you integrated the Iridas into the camera software, but the application wasnt runnin` yet.

When do you think you will have it ready and fully operational?

The new toolset looks really great

Eric Wu
June 26th, 2007, 03:50 AM
Hello Jason, David and Sergio,

I've take the .jpgs you showed above and tried David's steps in AE.
I found that all of them were done using the AE cineon converter's default setting of gamma 1.7.
But according to AE's documentation to convert from logarithmic to linear the gamma should be set to 1.0
Or was I always wrong about it? I guess I've never fully understand the cineon gamma setting.
Where does that 1.7 come from anyway? If it's for normal viewing then shouldn't it be 2.2?
And setting it to 2.2 the result is not even equivalent to applying a "level" effect with gamma 2.2!
I'm so confused. Please help me figure it out!

Eric Wu

Jason Rodriguez
June 26th, 2007, 07:34 AM
Eric, you are right, the gamma should be set at 1.0, not the 1.7 that After Effects sets the gamma to. This is because in the 32-bit float mode, we're actually now a photometrically linear file.

The confusion of course comes in the math that Kodak uses for their Cineon equation. The gamma correction for a linear source should be 1.0, but in their equation, they have a Display Gamma input that gets divided by a constant of 1.7:

Display Gamma / 1.7 = Final Gamma

After Effects' implementation creates confusion because those familiar with the Cineon equation think that the "gamma" that AE is giving them access to is the Display Gamma setting from the Kodak Cineon equation, where-as it's actually the Final Gamma, which is already divided by 1.7 (or something close to that). So for instance, if you thought you were adjusting the Display Gamma input (which you're not), you would set the Adobe gamma setting to 1.7 . . . but in actuality you're adjusting the Final Gamma, which would make setting 1.7 incorrect, you want a Final Gamma of 1.0.

So in essence, Sergio's files are too dark to be Cineon-compliant.

Eric Wu
June 26th, 2007, 09:08 AM
Thank you Jason. It's very informative.

Sergio Sanchez
July 4th, 2007, 01:36 PM
I`ve been doing some cineon exports. I`ve used only the render to cineon options. But i`ve noticed that the final renders looks very different when rendering to cineon than when applying the cineon converter filter.

My question is how can i render to dpx or cin when using the cineon converter filter?.

Because i`ve noticed that if you use the cineon converter and render to cineon/dpx AE combine both and image doesnt work.

I had to render to tif (or any image extension) but not to .cin/.dpx.

Eric Wu
July 6th, 2007, 10:53 AM
Hello Sergio,

If you already use the cineon converter effect on the footage(s), then when you render to cineon sequence you should choose the "Full Range" preset in the format options. If you use the "Standard" preset then it will do the linear to log conversion again and the result will be incorrect. I think that may be what went wrong with your result.

Sergio Sanchez
August 14th, 2007, 08:15 PM
This is a clip someone uploaded its in low quality, but you can get the picture.

http://youtube.com/watch?v=MVqImnb7x9Y