Noise comparison: 35/4:2:0 vs. 180/4:2:2

[Piotr Wozniacki]

Originally Posted by Steve Kalle

"Neat Video" works in Vegas 32 & 64 bit.

Thanks Steve; I read on the SCS forum that Neat Video is even slower than the free DNR from Mike Crash...

Which provokes me to one more comment on the whole matter (sorry guys - I really thought I already could just accept the facts, and settle down with the conclusions I mentioned in my previous posts):

It really bugs me that in order to take advantage of what is supposed to be the most important nanoFlash strength - the high data rates - without substantial side-effects, one ever needs to consider using additional software tool... And with quite a good camera like the EX1/3, too!

Noise removal in post is not only a lengthy process; it also is very susceptible to user errors, leading to not always optimal, and sometimes unpredictable results. Also, using a tool like this is impossible without some side effects, partially negating the very purpose of recording to nanoFlash in the first place!

Therefore, I'd be very grateful indeed to Dan (or anybody else from Convergent Design), if the following was addressed firmly, and after giving it some thought is deserves in my opinion:

1. Can you confirm my conclusions in this thread - i.e., is what I've described confirmed with your own testings? Dan - I'd really like to exclude hardware malfunction from the equation, and I still seem to be the only one bothered with the problem :) Alister Chapman, for instance, said that "he rarely sees a huge difference between noise in his 35Mb footage and NanoFlash footage" in this thread: http://www.dvinfo.net/forum/sony-xdc...ml#post1534295

If however the answer to point 1 is indeed "yes", then let me ask again:

2. Could you re-consider my original idea of adding some NR to the future NanoFlash firmware? I hear what you say, Dan, that once smoothed in hardware it couldn't be undone - but I don't agree it makes the idea invalid, as the NR could be a menu *option* for the conscious user, OFF by default (perhaps even returning to the OFF setting with each power-cycling).

A moderate noise reduction in hardware wouldn't take any more time than the recording itself, and could be much more consistent than anything software. Of course, the algorithm could be more or less comprehensive and intelligent, depending on the horsepower available...

What do you say?

[Peter Moretti]

Originally Posted by Piotr Wozniacki

OK, I've done some more tests today; quite different lighting (full sunshine) - so don't worry, I'm not going to scare you with my ugly pictures :)

However, I'd like to share with you some more conclusions on the subject (most of them may be obvious for some people, but I hope they'll be of some use to those considering buying nanoFlash, and got scared by my complaining about the noise).

So, I tested several NF bitrates again for noise - but this time, I also played with detail settings of my EX1: I tested both 50 and 180 Mbps (NF) against the EX1 with both detail off and on, the latter with crispenig at zero (default) and +35. I was hoping that at the highest bitrate, the nanoFlash would compensate for detail loss due to detail off/positive crispening, while not suffering from the (reduced) source noise...

Unfortunately, this is not happening. Even with detail off, or with detail on and crispening at 35 - at the highest long-GoP bitrate the noise is still augmented and really distracting when watching on my 50" plasma. With 50 Mbps, it's at the same level as on SxS, with color resolution enhanced and mosquito noise reduced.

So, I'll repeat: those using the EX-series cameras in run&gun (or ENG) fashion, should stick to the 50 Mbps data rate of their nanoFlashes. The bitrates of 100 and above are for those shooting in controlled environment (or those lucky ones having cameras with higher S/N). ...

What is the source of the noise that you are talking about when shooting in bright sunlight? The signal should be strong enough and have enough color to make internal camera noise negligible, no?

[Piotr Wozniacki]

What makes you think there is noise in bright sunshine?

The problem is increasing the noise generated in low-light situations, particularly on large areas of uniform color. Such areas may also be present in the scene even in bright sunshine - it's enough that a wall, for instance, happen to be in a deep shadow, and the exposure used is such that the other, bright areas are not blow out.

But of course bright areas of the picture, where the signal is strong enough, do NOT pose any problems with the 54 dB of the S/N ratio. If you understood otherwise, my apologies - English is not my native language :)

[Billy Steinberg]

This may be the strangest thread in the forum; my 2¢.

It seems to me that you're on the wrong path here. Noise reduction does NOT belong in the recorder. The recorder's job is to record and playback your pictures without changing their quality.

If I were pursuing your quest, I would be devoting more of my time learning out how to make my camera look better in less than ideal situations, via the camera settings available to you. You don't like the noise in the pictures in some situations? Adjust the camera settings to reduce the noise. Or get a less noisy camera. Or remove the noise in post. Once you throw away information, you can't get it back again. Don't crush your peds. Don't over expose your whites. Don't suppress your resolution/detail.

Philosophically, wouldn't you want to adjust your camera's picture as best you can when you're shooting, with care taken to throw away as little information as possible, while still maintaing the look you want? Then you want your recorder to record your pictures as identically to what the camera is outputting as possible. If you want to alter the picture your camera is making, you want to do it in post, where you can always go back to the camera master and try different things. If a company came out with a spectacular noise reducer next month, and it didn't reduce the resolution/sharpness of the image at all, but it removed 99% of the noise, would you rather your camera masters were sharp and noisy, or mushy, full of artifacts, but quieter?

Even the very best, most expensive cameras impose tradeoffs depending on the shooting conditions and what you're shooting. I'm not familiar with the EX series; the Sony cameras I use are part of the professional broadcast line. Their detail circuits have dozens of adjustments, as well as the best noise reduction circuits in the business. There are essentially unlimited gamma tables and adjustments. I don't expect an EX to have all these adjustments, but it must have some. Next time you're shooting a house in the woods with most of the picture in the shadows, take a minute or two to play with your crisp control and watch on a really good monitor. There is somewhere between minimum and maximum that will reduce the amount of noise without throwing away all the fine detail. If you can't critically evaluate in the field, then make a handful of recordings with different settings so you can evaluate later. Just make sure you use a nano (in 100Mb or higher). If you use the EX's built in recorder, it will throw away so much information that you won't be able to tell very much.

Billy

[Peter Moretti]

Originally Posted by Piotr Wozniacki

What makes you think there is noise in bright sunshine?

The problem is increasing the noise generated in low-light situations, particularly on large areas of uniform color. Such areas may also be present in the scene even in bright sunshine - it's enough that a wall, for instance, happen to be in a deep shadow, and the exposure used is such that the other, bright areas are not blow out.

But of course bright areas of the picture, where the signal is strong enough, do NOT pose any problems with the 54 dB of the S/N ratio. If you understood otherwise, my apologies - English is not my native language :)

What you say directly above makes perfect sense to me. But further up you talked about a new set of tests you did in bright sunlight and said the noise in those shots still presented a problem.

[Peter Moretti]

Originally Posted by Billy Steinberg

This may be the strangest thread in the forum; my 2¢.

...

Billy

Billy, you're fee to your own opinion of course, but this thread has proved very useful (to me anyway). It brings out some very interesting "issues" that come along with the obvious benefits of the nano. I never thought about how poorer codecs can actually mask noise.

Strange it may be, but this thread is also very interesting. We're all just exploring this very new tool and how it mates with different cameras.

BTW, I DO agree with you that NR should be saved for post. There is no way a device like the nano could do any type of sophisticated NR, so you'd be stuck w/ simple blur... which can be done in post as a realtime effect.

[Piotr Wozniacki]

Originally Posted by Peter Moretti

What you say directly above makes perfect sense to me. But further up you talked about a new set of tests you did in bright sunlight and said the noise in those shots still presented a problem.

Peter,

As I said - our communication problem arise from my poor English; sorry.

And to make it clear: yes, I did more tests in bright sunshine, but I was judging results on the same portion of the wooden barn, taking advantage of the fact that it was partially in the shadow (see addition to the unclear post you're referring to). So while the picture as a whole were bright and generally noise-free, this particular crop was buzzing with noise...

Thanks for pointing this confusion out, and creating an opportunity for me to explain :)

Piotr

Edit
To clarify even more, I'm posting similar blown-up crops of the shadowy wood areas; as usual the SxS is at the LHS, and the 180 Mbps nano - at the RHS. This time, crispening was set to +35, which softens the picture quite a lot, but still doesn't prevent nanoFlash from adding grain. Looking at the static crops doesn't give full idea of how bad the grain is - even though they've been blown-up again; on my 50" plasma the added (or should I say: not removed) grain is shimmering at full screen playback (lower 2 grabs), thus becoming even more noticeable.

[Piotr Wozniacki]

Originally Posted by Billy Steinberg

If I were pursuing your quest, I would be devoting more of my time learning out how to make my camera look better in less than ideal situations, via the camera settings available to you. You don't like the noise in the pictures in some situations? Adjust the camera settings to reduce the noise. Or get a less noisy camera. Or remove the noise in post. Once you throw away information, you can't get it back again. Don't crush your peds. Don't over expose your whites. Don't suppress your resolution/detail.

Dear Billy,

If you care to search my posts in the EX1 forum, you will find I know how to use my camera controls, and minimize noise.

Originally Posted by Billy Steinberg

I'm not familiar with the EX series; the Sony cameras I use are part of the professional broadcast line.

If so, you really shouldn't find my quest "strange" at all; I wrote:

"those using the EX-series cameras in run&gun (or ENG) fashion, should stick to the 50 Mbps data rate of their nanoFlashes. The bitrates of 100 and above are for those shooting in controlled environment (or those lucky ones having cameras with higher S/N)."

I guess you fall into the category of those lucky ones - but please let us mortals find the optimum ways of using our EX1/3 with the nanoFlash - this is what I'm trying to achieve here...

Cheers Billy,

Piotr

[Peter Moretti]

Piotr, just for kicks, what happens if you go up to 280 but don't blur the image using the Crispening setting? I'm wondering if some of the "noise" starts to resemble detail.

[Piotr Wozniacki]

Sorry mate - my CF cards won't do 280 Mbps :)

Cheers

Piotr

[Jeff Silverman]

I think what Billy points out though is that a well adjusted EX1/3 does not equal the quality and amount of processing and control available in a true broadcast camera (which can cost 10 times as much).

In this forum EX1/3 cameras are considered a great camera, in the broadcast world I would consider them great cameras compared to their extremely cheap cost. They have drawbacks and limitations. And the Nano seems to capture that.

Luckily the Nano also seems to enhance these cameras in many positive ways as well.

Jeff

[Piotr Wozniacki]

Originally Posted by Jeff Silverman

Luckily the Nano also seems to enhance these cameras in many positive ways as well.

Fully agree - it's amazing that the nano offers the flexibility (like the wide choice of bit rates available) that allow for optimum use with cameras of any class, and picture quality.

I'm just trying to find and confirm those optimum nanoFlash settings for the EX series cameras, with their limitations in mind.

[Billy Steinberg]

Originally Posted by Piotr Wozniacki

If you care to search my posts in the EX1 forum, you will find I know how to use my camera controls, and minimize noise. If so, you really shouldn't find my quest "strange" at all; I wrote: "those using the EX-series cameras in run&gun (or ENG) fashion, should stick to the 50 Mbps data rate of their nanoFlashes. The bitrates of 100 and above are for those shooting in controlled environment (or those lucky ones having cameras with higher S/N)."

I guess you fall into the category of those lucky ones - but please let us mortals find the optimum ways of using our EX1/3 with the nanoFlash - this is what I'm trying to achieve here...

Indeed I do consider myself one of the lucky ones, though I'm just as mortal as everyone else. I'm glad to hear that you're on top of wringing the most from your camera, no matter where it fits in the range of horrible to spectacular. All I was really trying to say was that degrading your pictures in the recorder strikes me as a bad idea, even if the degradation also has the side effect of masking one of the limitations of the camera. Perhaps "strange" wasn't the right word; I didn't mean to be (or appear) condescending.

With what I've seen from the stills you've uploaded, I do not agree that "those using the EX should stick to the 50Mb codec". I find the mushing of the fine detail and the presence of artifacts ("mosquito noise", etc) far more disturbing than the shadow noise. But maybe that's just me. Starving a codec causes more problems, in most cases, than it solves; and none of your examples take into account motion. Once you add motion to the equation, restricting yourself to 50Mb will add much more degradation. Sure there are times that using a low bit rate codec makes sense, but as a general rule, I think it's misguided.

In any case, I admire your quest to understand and to get the best from your equipment, I just don't think camera noise reduction belongs in the recorder, nor do I believe it's a good general rule to limit yourself to a 50Mb codec when you record to the nano with an EX. The EX camera section is far more capable than the recorder section, and as far as I can tell, one of the main reasons for the nano is to provide a better end result in exactly that situation (better camera section than recorder section).

Billy

[Billy Steinberg]

Originally Posted by Peter Moretti

Billy, you're fee to your own opinion of course, but this thread has proved very useful (to me anyway). It brings out some very interesting "issues" that come along with the obvious benefits of the nano. I never thought about how poorer codecs can actually mask noise.

Strange it may be, but this thread is also very interesting.

Perhaps my choice of the word "strange" was ill conceived. I was trying not to appear condescending, though I seem to have failed. :) What I found "strange" was not the content or intent of the discussion, but the direction it was taking (i.e., "build noise reduction into your recorder and use a low bitrate codec").

So I find the thread interesting as well. (I've not missed reading a single message here for almost a year, though I rarely post, unless I find something particularly interesting). Codecs do play a huge part in how the final picture looks, as well as how easy or difficult it is to edit. I find the Sony MPEG2 hardware codec used in the nano to be particularly spectacular, especially when setting it to 100Mb or higher.

Billy

[Piotr Wozniacki]

Originally Posted by Billy Steinberg

With what I've seen from the stills you've uploaded, I do not agree that "those using the EX should stick to the 50Mb codec". I find the mushing of the fine detail and the presence of artifacts ("mosquito noise", etc) far more disturbing than the shadow noise. Billy

Dear Billy,

First of all, I didn't find your comments "condescending" - no bad feelings at all, as we're all equally entitled to express our opinions here.

Please consider that my conclusion you quoted - "those using the EX should stick to the 50Mb codec" - has indeed been conditional, i.e. I recommended the above for those shooting with EX-series cameras in uncontrolled environment only (like run&gun of the ENG shooting). However, I also suggested that those who have time and means to provide adequate lighting, scene composition, and camera setting tweaks, will certainly benefit from 100 Mbps (or even higher) bitrate recording on their nanoFlashes.

Also, the mosquito noise etc may look more disturbing than the shimmering grain I'm investigating in the stills I posted, However - and this I also stressed many times - when watching actual video (i.e. moving pictures) on a big enough display device, it's the shimmering noise that is most noticeable, and hence - disturbing.