Noise comparison: 35/4:2:0 vs. 180/4:2:2

[Piotr Wozniacki]

Ron,

Assuming you're shooting with an EX1/R/3, yes - that's right.

However it all depends on the actual S/N of your source camera. For instance, I've just run a series of new experiments - and with PP OFF (yes - not just DETAIL, but also the matrix, etc at the factory settings), even with my EX1 as the source, the 100 Mbps nanoFlash L-GoP looks gorgeous!

Well - no wonder; just the GIGO formula at work... The goal now being how to possibly tune the L-GoP nanoFlash encoding so that it does NOT augment the noise.

Piotr

[Luben Izov]

Originally Posted by Piotr Wozniacki

..... For instance, I've just run a series of new experiments - and with PP OFF (yes - not just DETAIL, but also the matrix, etc at the factory settings), even with my EX1 as the source, the 100 Mbps nanoFlash L-GoP looks gorgeous!..............
Piotr

Dear Piotr,
I am really surprised that it took you so long to try that option of test! I remember suggesting that months ago just as others did.....
I am finally happy for you! Some how we all got involved in your ( as you said) paranoia! ;-)

I love my NF and it does deliver more then I've expected! I am so pleased, that I even consider buying the 3D single unlocked unit. That in combination with the one I have would open way more oportunities for a freelancer!
Now that we may see some price war ( I hope, but I doubt) maybe the right time to buy it... CD may provide some special discount for exsisting costumers!, you never know....
Good luck to you and Thank you for this Thread!! It was exhausting but very informative in any direction!
Cheers mate

[Rafael Amador]

Hi Ron,
50 Mbps may be the sweet spot for the SONY XDCAM 422 customers.
With XDCAM, SONY has targeted his old Betacam customers. And SONY knows about the needs in term of archiving of his customers.
50 Mbps is a trade quality-files size.
But we are not XDCAM 422 users and the file size is not our main concern, but the picture.
We expect much more from the Long GOPs.
rafael

[Dan Keaton]

Dear Ron,

For Broadcast work, our 4:2:2, 50 Mbps, CBR Sony XDCam 422 Long-GOP footage is widely used and accepted.

We say that the 4:2:2, 100 Mbps, CBR XDCam 422 Long-GOP footage is the sweet spot as while this option uses more space, the quality is visually better, without using too much space.

Our 280 Mbps I-Frame Only is the best quality of all of our options.

For those wanting I-Frame Only, such as for Avid users, I personally recommend 140 Mbps or above.
Avid also support our 50 Mbps 4:2:2 Long-GOP.

[Piotr Wozniacki]

Originally Posted by Luben Izov

Dear Piotr,
I am really surprised that it took you so long to try that option of test! I remember suggesting that months ago just as others did.....
I am finally happy for you! Some how we all got involved in your ( as you said) paranoia! ;-)

I love my NF and it does deliver more then I've expected! I am so pleased, that I even consider buying the 3D single unlocked unit. That in combination with the one I have would open way more oportunities for a freelancer!
Now that we may see some price war ( I hope, but I doubt) maybe the right time to buy it... CD may provide some special discount for exsisting costumers!, you never know....
Good luck to you and Thank you for this Thread!! It was exhausting but very informative in any direction!
Cheers mate

Dear Luben,

I'm also very happy for you having so many freelancer opportunities:)

Just for the records, though: I did test all possible PP combinations, including PP off, before - if I commented obout it today was just to illustrate my answer to Ron - that there's no single best setting of the nanoFlash alone. Everyone must consider it together with his/her acquiring device.

With EX1, using rich colors in combination with certain gamma curves tends to increase the noise, and - along the GiGo rule - it gets augmented by the nanoFlash... No doubt about it, and the need to fine tune the nanoFlash high bitrate L-GoP encoding so that the increased storage space is accompanied with real quality gains....

Cheers

Piotr

[Andrew Stone]

Originally Posted by Piotr Wozniacki

However it all depends on the actual S/N of your source camera. For instance, I've just run a series of new experiments - and with PP OFF (yes - not just DETAIL, but also the matrix, etc at the factory settings), even with my EX1 as the source, the 100 Mbps nanoFlash L-GoP looks gorgeous!

Yes, I would suggest to all the people out there getting your camera to do gyrations with picture profiles to invest in a DSC Chart (they aren't cheap), shoot the chart at the start of each scene or shoot you do and don't yank the color in the camera but do it in POST. Some of those picture profiles though issuing relatively good color are pulling on the color space in a severe manner and is probably resulting in noise exaggeration.

I felt this for a while abut the EX but am only now coming to a more solid conclusion on it.

Further to that bring out the lights so the noise floor is further reduced.

[Piotr Wozniacki]

Now that the AJA mini is recording 10 bit, the discussion on whether the nanoFlash only recording 8 is enough has been revived. Before, the consensus has been that with cameras like the EX series the last 2 bits would not add any useful information but noise...While this is most probably right, I'd like to ask both Convergent Design and fellow users more knowledgeable than myself for some input on the following issue:

As you know, after a long and exhaustive testing the EX1/nanoFlash combo I have finally sort of reluctantly settled with recording 100 Mbps max (for L-GoP) or 220 Mbps min (for I-Fo), and preprocess with Neat Video plugin (whose Vegas version is working great, the only caveat being it takes ages).

However what I have noticed is that after noise removal, some areas of the picture (the very same that suffer from noise the most - mid-bright, uniform color) tend to show another ugly shortcoming: the color banding!

The logical explanation that I have is that before cleaning, the noise acts as dithering the color information; it's "dirty" but it's not banding, even with just 8 bit sampling. After noise removal, the same 8 bit is not enough - hence the banding.

What I've been doing with clips of the greatest importance to me (in terms of the PQ) is apply the Neat Video de-noiser, and render out into a 10 Bit codec (like the Sony YUV) - the banding is gone!

However I still am at loss as to a complete workflow; after all the delivery format I'm using is 8 bit anyway, so the final render must be to an 8 bit codec. So far, it looks promising as after doing all the editing stuff required (CC, levels, etc), my Vegas renders of such 10 bit clips into - say - a 35 Mbps, 8 bit, 4:2:0 mpeg-2 or 20 Mbps AVC for Blu-Ray do not show the banding - i.e. it's not coming back once eliminated.

I'm open for any suggestion re: the optimal workflow, but for now, I'd like to revive the 8 vs. 10 bit discussion on the nanoFlash. I'm not sure of details, but I think I've read the nanoFlash simply truncates the last 2 bits of the HD-SDI stream, and that the alternative method of actually dithering full 10 bit information into 8 bit recording could be possible...

Dan - do you think it could help with color banding like this mentioned above? And (I know it's a log shot) - wouldn't actually recording the full 10 bit (at lower bitrates - I know of the bandwidth limits) help with both noise not being augmented, and - after removing the noise generated in camera head - prevent the banding completely?

[Rafael Amador]

"Now that the AJA mini is recording 10 bit, the discussion on whether the nanoFlash only recording 8 is enough has been revived. Before, the consensus has been that with cameras like the EX series the last 2 bits would not add any useful information but noise...While this is most probably right,".

Piotr,
That makes no sense at all, and to discuss the advantages of 10b over 8b is a waist of time.
The signal out of the SDI, with or without noise, is 10b.
Do you think could be any benefit on crunching it to 8b?
The ideal would be to record it to 10b Uncompress. Any other option will degrade the picture, and an 8b codec will degrade the picture more than an 10b codec. I'm talking of course about decent codecs.

About NeatVideo, you said:
"What I've been doing with clips of the greatest importance to me (in terms of the PQ) is apply the Neat Video de-noiser, and render out into a 10 Bit codec (like the Sony YUV) - the banding is gone! "
Sure, NeatVideo must be rendered in 10b. Internally works in RGB, and unless you render in Floating point, you will get all your Highlights clipped. That happens when you render in 8b RGB: No room for "SuperWhites".
In FC is necessary set "render all YUV in High Precision".

"However I still am at lost as to a complete workflow; after all the delivery format I'm using is 8 bit anyway, so the final render must be to an 8 bit format. So far, it looks promising as after doing all the editing stuff required (CC, levels, etc), my Vegas renders of such 10 bit clips into - say- a35 Mbps, 8 bit, 4:2:0 mpeg-2 or AVC for Blu-Ray do not show the banding - i.e. it's not coming back once eliminated".
Acquire as good as you can, and always render with the top quality you can, exporting to the codec that retains the most from the process done. Compress to the delivery format in the last step.
rafael

[Dan Keaton]

Dear Piotr,

In reference to dithering:

We fully implemented dithering, then we tested it in our lab.

Dithering did not help the quality of the image, and it interfered with the efficiency of the codec.

So, we decided not to include dithering in our production firmware releases.

You mentioned the 8-Bit versus 10-Bit debate.

Of course, we know that 10-Bit is desirable, if one has a very good camera, with a very clean 10-bit output. These cameras do tend to be the expensive cameras.

And one should know, that while HDMI 1.3 or above supports 10-bit, all HDMI cameras that we know of have 8-Bit HDMI output.

HD-SDI, of course, can support 10-Bit.

The proof is in the images. While a camera may technically put out a 10-Bit HD-SDI source, one should carefully examine the images to determine if the 10-Bit actually helps or not.

This is not meant to detract from a good clean 10-Bit camera. We use the Sony codec module which produces does an amazing job, and it is only 8-Bit.

My comments are meant to say that one typically needs an expensive camera to get a good clean 10-Bit output.

One way to test a camera to determine if the 10-Bit is clean, is to use an expensive 10-Bit professional HD-SDI monitor. Just setup the camera at static scene, or a color chart, then examine the image on the monitor. Nothing should be moving and no color changes should be present. Then vary the lighting.

(Of course there are other ways to test the quality of the cameras output0.



As one piece of the puzzle, I would like to discuss codec efficiency.

Adding the extra 2 bits requires a higher bit-rate just to compensate for the two extra bits.

Doing the math:

220 Mbps at 10-Bit is similar in performance to 176 Mbps in 8-Bit

145 Mbps in 10-Bit is similar in performace to 116 Mbps in 8-Bit

220 Mbps in 8-Bit requires 275 Mbps in 10-Bit

280 Mbps in 8-Bit requires 350 Mbps in 10-Bit


The above examples do not take into consideration any of the other differences between the codecs, just the difference when having to process the extra two bits or not.


My reading of the Apple ProRes white paper, is that ProRes is always 10-bit (or higher).
I carefully read the white paper and did not find any mention that they would use only 8-Bits for an 8-Bit source. Of course, this level of detail may be missing from the white paper.

http://images.apple.com/finalcutstud..._July_2009.pdf

[Rafael Amador]

Hi Dan,
I disagree very much when you say "My comments are meant to say that one typically needs an expensive camera to get a good clean 10-Bit output".
Also I disagree very much when you suggest a 10b monitor.
The difference between 8b - 10b, (as its happens with the difference 420 - 422) may not be very obvious on your screen, but will arise in postproduction.
Cheers,
rafael

[Dan Keaton]

Dear Rafael,

I am trying to understand your comments.

Do you know of a low cost camera that puts out a good clean 10-Bit signal over HD-SDI?

One where the quality of the 10-bit image is not affected by the noise level of the camera?

If so, which one or ones?

My comment about using a good 10-Bit monitor was meant to provide a reasonable way to evaluate a camera's output, without any recorder, or NLE affecting the test.

How would you determine the quality of a cameras' 10-Bit output, without resorting to expensive test gear?

I respect you opinions and I agree that the benefits of modern technology help in post and in multiple generations.

[Rafael Amador]

Dear Dan,
I was in a hurry when I wrote that and didn't explained my self.
What I wanted to say is that the possible noise in an 10b SDI is not an excuse to record it at 8b.
MY comment about the monitor, in this moment, I don't remember why i wrote it.
Some times I post too late.
Best,
rafael

[Cees van Kempen]

Originally Posted by Dan Keaton

Dear Ron,

For Broadcast work, our 4:2:2, 50 Mbps, CBR Sony XDCam 422 Long-GOP footage is widely used and accepted.

We say that the 4:2:2, 100 Mbps, CBR XDCam 422 Long-GOP footage is the sweet spot as while this option uses more space, the quality is visually better, without using too much space.

For a newby like me it is quite hard to decide now what setting to use. I am going on an expedition. Not data storage capcity, but power capacity will be our challange. When downloading to a nexto, 50Mbps footage is twice as fast as 100Mbps, so it saves battery capacity (of the nexto). In what shootings is 100 visually better? I will make a nature documentary, slow pans, no fast movements (maybe the birds wings) etc. Is 100Mbps also better in such circumstances or can I stick to 50 and thus be more efficient with storage and power consumption?

[Piotr Wozniacki]

Cees, which camera are you using?

If it's an EX1(R)//EX3, than my recommendations are:

- for L-GoP: 100 Mbps max if you're planning extensive post-processing, 50 Mbps if you're just cutting
- for I-Frame Only: as high as possible (i.e. 280 Mbps if your cards allow, otherwise 220 minimum).

The reason being that the EX series cameras are indeed noisy, and L-GoP at bitrates higher than 50 Mbps actually augment this noise (or strictly speaking, do not mask it enough, like the more compressed, i.e. lower bitrate, formats do).

This observation has been confirmed by Dan who has found time to watch my sample footage of L-GoP @50 vs. @180 Mbps, and admitted some picture areas of the latter are unacceptable straight from the nanoFlash. (Thanks, Dan !).

Of course, when footage at even 180 Mbps bitrate is subject to heavy grading and multiple re-compressions, it should hold up better than the 50 Mbps recording. Also, the final encoding to a delivery format (at a bitrate usually not exceeding 50 Mbps anyway), should result in the excessive noise being suppressed. I'm saying "should", because I never actually tried it in my workflow; I'm sticking to the rules outlined earlier in this post.

If you want to use nanoFlash files without postprocessing, however, either use 50 Mbps Long GoP, or at least 220 Mbps I-Frame Only.

Piotr

[Dan Keaton]

Dear Cees,

Our 50 Mbps 4:2:2 is exactly the same image quality as the Sony PDW-700 and PDW-F800 cameras, and is widely accepted as Broadcast Quality.

Here is my opinion:

Our 50 Mbps looks very good. It is intended to be viewed at normal frame rates.

If one wants "extra insurance", or ability to capture great images where there is an excessive amount of detail, or lots of movement in the image, or lots of camera movement, or when each and every image may be used as a still frame, then I would go with 100 Mbps or one of our other higher bit-rate options.

If I was making a movie, and power was not a consideration, as it is for you, then I would use higher bit-rates.

You have one of the challenges that makes the nanoFlash so desirable.

You need high quality, but you are limited by power. You can easily choose 50 Mbps and help solved the power problem.

My recommendation: Run some tests, check out the 50 Mbps footage.

Background:

The Next DI devices have a built-in Liithium Ion battery. After xx Gigabytes of copying CompactFlash cards the battery is going to be exhausted.

50 Mbps footage is one half the size of 100 Mbps footage, so one gets twice the footage on one battery charge.

Cees, are you aware that Nexto has small, external batteries for the Nexto?
You could get a few of these and solve your power problems. They only weigh a few ounces.

And they have an AA Battery pack option.

Also, if at all possible, do the copying of files when you have AC power.
(I realize that this may not be possible on your upcoming trip).