Noise comparison: 35/4:2:0 vs. 180/4:2:2

[Dan Keaton]

Dear Piotr,

I welcome others to comment on your situation.

I believe that the key will be to attempt to reduce the noise originating with the camera.

Could you please post your camera settings and what type of scene you are shooting.

What is the gain?

What is the Shutter Speed?

What is the Aperature (F-Stop)?

Are you using a Custom Preset or Profile?

Does this problem occur when you have adequate light, such as on a bright sunny day and the entire scene is bright?

[Piotr Wozniacki]

Dear Dan,

C'mon, please - you know better than ask me all those questions again. Therefore, let me only answer the last one:

- yes, the problem is also present when I have adequate light, such as on a bright sunny day and the entire scene is bright.

I will not post any screen grabs, as the grain is only visible on the moving picture.

[Mark Job]

Hi Piotr:
There was a similar problem of poor quality playback of clips when playing back the clips inside of the Flash XDR SSDR. I and Aaron complained about this and this problem was fixed with firmware release 1.5.55. This was a recorder based problem and NOT a camera HD-SDI stream output issue.

*Here's a way to find out what's going on: Remove the recorded clips on your CF card from the Nano Flash and import them into your NLE completer editing system. Look at the clips inside your editing system, or with VLC video player and see if the same noise discrepancies you are seeing from the Nano Flash in playback are also visible from inside your computer when playing back RAW CF Card footage.

I hope this helps.

[Piotr Wozniacki]

Unfortunately Mark the grain is still there - no matter whether I play the clips from the nano, inside my NLE, or with VLC.

It's similar to the noise - that I was discussing for the most part of this thread - in that it's shimmering, thus most visible in the moving video. Unlike the noise, the grain is almost "translucent", and present everywhere in the picture (the noise being random darkish pixels, only present in low-light/shaded areas).

Another fact that makes it similar to the noise in L-GoP clips is that when I render such a grainy I-Frame 140 Mbps clip to 35 Mbps 4:2:0 (or even 50Mbps 4:2:2) in Vegas, the grain is filtered out and the picture becomes as clean as that originally recorded with the EX1, or nanoFlash at 50 Mbps.

This again makes me feel the latter format (L-Gop 4:2:2 50 Mbps) is the only nanoFlash format where advantages over the native EX recording are more readily visible than the artifacts nanoFlash introduces - but that is certainly not worth the extra hassle of external recorder with the price tag like this. Especially now that budget cameras become available recording 4:2:2 50 Mbps natively.

Very, very disappointing indeed.

[Mark Job]

Hi Piotr:
OK. Understood. I must therefore, conclude the malfunction can only be the result of two possibilities.

A) Non-Spec HD-SDI camera output from EX-1 Sony.

B) Busted Nano Flash.

Let us proceed to a secondary procedure to determine the source of your malfunction: Can you get your hands on another Nano Flash ? Perhaps there is a rental house in Poland which may have the Nano Flash in its inventory. Try your setup with a secondary Nano recorder.

OR

Locate PC or MAC with an HD-SDI video capture card. Capture some test shots live out to the computer and check on broadcast HD monitor.

[Piotr Wozniacki]

Mark,

If you keep in mind what I said about 50 Mbps L-GoP (which is the only format on my nanoFlash that doesn't introduce any artifacts over the native SxS recording), it becomes apparent that nothing is wrong with my camera's HD-SDI output.

This is, unfortunately, how the nanoFlash encoder chips perform. When - after my long testing of noise in Long-GoP - some other people came up and confirmed my observations, Mike & Dan offered the conclusion that L-GoP is only better with data rates of up to 100-140 Mbps (even though before, they kept saying L-GoP is preferable to I-Frame only with speeds of up to 180 Mbps).

I wonder what they say now. Last time I uploaded samples onto CD's ftp for them to analyze and comment on my findings, which they never did. Since the new kind of artifact that I now find with I-Frame only is not visible in a still frame grab, I wonder if somebody would spare some time and assess them this time, if I uploaded a couple of files again?

[Kris Rutkowski]

Hi Piotr, i have recently purchased nanoflash for my EX3 but not tried it yet. It's still in a box. I'll be flying to Poland soon and if you happen to live fairly close to me (SE Poland) i don't mind meeting up with you to compare our NF.

[Dan Keaton]

Dear Piotr,

1. We have been extremely busy.

2. We posted the results of the objective Video Clarity system.

However, the results from the Video Clarity system apply to the specific type of video that we tested, which was a football game. This test uses very clean footage.

One can use this as a guideline to select a bit-rate/mode, but the results will not apply under certain conditions, such as a completely different type of footage, or sending noisy images to the nanoFlash.

The Video Clarity system is a wonderful system, and it provides a wide variety of professionally recorded images to use to perform the tests. We used only one of the test scenarios in our testing.

3. Your camera in 35 Mbps 4:2:0 mode will reduce the noise in your images as opposed to what comes over the the HD-SDI output.

4. The nanoFlash in 35 Mbps 4:2:0 mode will reduce the noise in your images in exactly the same way.

5. If you camera sends noisy images to the nanoFlash, the nanoFlash will not distinguish the noise from detail and do its best to reproduce both.

Thus, the nanoFlash should not be expected to reduce the noise in your images.

6. One can easily setup a camera, or pick a scene in which the images will be noisy.

7. The Sony EX1 series is a very nice camera but is very capable of producing noisy images depending on the scene, the lighting, and the camera setup.

The nanoFlash is an extemely versatile device in which one can pick and choose the File Type, Bit-Rate, I-Frame Only or Long-GOP.

8. We have in the past, and still do, recommend 100 Mbps Long-GOP as the best, general purpose, option taking file size and recording time into consideration.

9. There are conditions were we recommend other modes.

If you are shooing a movie, and file size is not a consideration, we recommend 280 or 220 Mbps.

If you are wanting the nanoFlash to reduce the noise in your image, you could choose 35 Mbps.

However, we highly recommend that you first take all reasonable efforts to reduce the noise in your images by using appropriate lighting and setting up your camera to reduce noise.

10. The best way to test if the nanoFlash is doing it's job,
the best way to test if your camera is putting out a clean image,
and the best way to test various nanoFlash modes,
is to test your camera and nanoFlash in A-B fashion using a high-quality professional monitor.

If one is expecting
1) Excessive Camera Motion,
2) Excessive Motion in the image,
3) Excessive Detail in the image, or
4) Excessive Noise in the image, then other modes may give better results.

9. We really want to help you get the most of your camera and your nanoFlash.

[Mark Job]

Originally Posted by Piotr Wozniacki

Mark,

If you keep in mind what I said about 50 Mbps L-GoP (which is the only format on my nanoFlash that doesn't introduce any artifacts over the native SxS recording), it becomes apparent that nothing is wrong with my camera's HD-SDI output.

.....Which leads me to consider strongly the possibility you have a busted Nano Flash my friend. Perhaps the encoder circuit in your Nano is on the way out, or some other internal component, such as, a transistor or a resistor is coloring the input signal slightly ? The fact that you wrote the noise is "translucent" makes me wonder about something electronic in your Nano going bad. You need to test another Nano to confirm and eliminate any further variable.

Originally Posted by Piotr Wozniacki

is, unfortunately, how the nanoFlash encoder chips perform.

...Well, this is how your Nano Flash chips are performing, and you could be right, only, I would have thought other Nano Flash users would also be reporting this kind of anomaly.

I do not get any of the kind of translucent noise issues with my Flash XDR. The higher I go in Long GOP or I-Frame (Intra) the way cleaner things look. Here's a thought - and again, I'm not trying to be dogmatic about this either, but would it be possible that Sony has a special super DSP chip inside it's in camera encoder which is ultra-tweaked to do away with a great deal of in camera noise and optimizes everything to go onto the SxS cards ? With my Canon XL H1, it has this great big fat DSP chip called a Digic II, which creates the most incredible 4:2:0 HDV MPEG 2 m2t files on HDV cassette I have ever seen (Even tops all of Sony's HDV camcorders). Canon states in its literature that this chip removes noise and optimizes the image for a cleaner MPEG 2 compression. What got my attention was how you said all is well with Long GOP 50 Mbps setting. - Hmmmm ! Sony's in camera encoders are optimized for two settings -

1. 35 Mbps 4:2:0
2. XDCAM HD 4:2:2 Long GOP 50 Mbps.

The higher up you go with the Nano Flash or Flash XDR in encoding data rates, then the more faithful the built in Sony XDCAM HD 4:2:2 encoder will be at reproducing the signal, and this *may* be your problem here.

Originally Posted by Piotr Wozniacki

- after my long testing of noise in Long-GoP - some other people came up and confirmed my observations, Mike & Dan offered the conclusion that L-GoP is only better with data rates of up to 100-140 Mbps (even though before, they kept saying L-GoP is preferable to I-Frame only with speeds of up to 180 Mbps).

...This is interesting. Personally, I think Long GOP is useless above 50 Mbps because the industry doesn't allow for it, however, I-Frame 220 or 280 Mbps in particular is utterly astounding and I-frame can be edited on any "A" list NLE. Long GOP 180 is wonderful, but you're limited to FCP to post it, or go ahead and shoot in Long GOP 180 and play it out in realtime into your NLE via HD-SDI from the box - Now that's a BIG winning way to go.

Originally Posted by Piotr Wozniacki

wonder what they say now. Last time I uploaded samples onto CD's ftp for them to analyze and comment on my findings, which they never did. Since the new kind of artifact that I now find with I-Frame only is not visible in a still frame grab, I wonder if somebody would spare some time and assess them this time, if I uploaded a couple of files again?

...Perhaps these products reach the very limits of MPEG 2 8 Bit Video technology ? Maybe your combination of camera and Nano Flash exposes this ? I'm not sure you don't have a busted Nano however.

[Luben Izov]

Hello Piotr and thank you for your efforts and posting your findings!
I've not seeing anything disturbing in my NF material, not that I've looked anything as thorough as you have, but, the fact and the matter is that all of my material has no problem passing the Broadcast requirements in Canada and in the USA.
To come in rescue to your problem, you got an offer from other NF owner in Poland and that could help you relize that your NF is good or bad. Until then, the only think that I could suggest to you is to rent other EX1 (borrow if you can) or simply get any other camera with HD-SDI out and look for the same problem. I know, different camera, different things, but, if the NF is faulty with your EX1 records it would be faulty with other camera too. It very well could be your EX1 or the HD-SDI on it. Doing that would give you the idea that is your camera or is really the NF.
Just as Dan have said, the NF would record and reproduce everything that is given to it - gain, shadows and so on...
Also Dan, asked about the PP settings. That is a very good question. If you are using PP setting made on your own not calibrated by a vector scope, some or all of you colours could be trowing out a lot of noise.
I am not an expert on this, but, I've seen it on my EX3 just by playing with PP settings and testing. If you have the time and consider that option, please go to a camera center/ rental/post facilities and calibrate your EX1 PP settings and that would clear a lot of your troubles, I hope , if your HD-SDI and NF are working properly.
Just my 2c.
Hope you find out what the problem is and start enjoying your camera with NF very soon!
Luben

[Piotr Wozniacki]

Guys,

Thank you very much for your advice. Of course I hope you realize I'm talking subtleties here - just like with the (dark area) noise, the shimmering fine detail ("translucent" grain) can only be seen on a fairly large display, like my 50" plasma.

Kris,

Could you please send me a PM? I tried emailing you, but for some reason I can't. It so happens I do live in the SE of Poland, so I'd be more than happy to meet and share experience.

[Gints Klimanis]

Piotr, I'm willing to look at your files again. I've concluded that my Sony 52" Bravia (2007 model) unfairly amplifies noise and macro-block edges with sharpening that can not be reduced with user controls. The clincher was using the Nanoflash to record MPEG LongGOP at 18 MBps.

Have you considered trying a card that can record the uncompressed video signal and experimenting with another software video compressor such as VirtualDub/ffmpeg for comparison ? It appears that you are pleased with the severe high frequency reduction of SxS 35 MBps LongGOP and less so with the compression between that and very high bitrates. We've talked about the LongGOP shimmering noise in which the I-Frame is signficantly better than the preceded or following P-frame, but I'm not sure that all of your comments relate to that.

[Gints Klimanis]

Originally Posted by Piotr Wozniacki

Since the new kind of artifact that I now find with I-Frame only is not visible in a still frame grab, I wonder if somebody would spare some time and assess them this time, if I uploaded a couple of files again?

Piotr, sure, I'll take a look at your files. Please include more action in your scenes so that it's easier to match the Nano frames with the SxS. The very wide views of your house shots make it difficult to compare detail, so some close-ups would be preferred.

[Kris Rutkowski]

Originally Posted by Piotr Wozniacki

Kris,

Could you please send me a PM? I tried emailing you, but for some reason I can't. It so happens I do live in the SE of Poland, so I'd be more than happy to meet and share experience.

email sent

[Alister Chapman]

The EX is not the quietest of cameras. The noise it produces is across the entire luma range it is not restricted to just shadow and dark areas. The reason it is less obvious in brighter part of the image is that noise has a fixed amplitude relative to the brightness, so a dark part of the image at say 100mV with 10mV of noise is being modulated by 10% while a 500mV (brighter) signal with 10mV of noise is modulated by only 2% so the noise is much less obvious visually, but the same amount of noise is still actually present.

The higher the bit rate that you record at the better any such subtle image differences will be recorded. A low bit rate codec will often ignore such very fine image changes as part of the data reduction process. I often observe more visible noise in my NanoFlash footage. This is perfectly normal. The 35Mb/s coded from the EX turns noise into blocky artefacts. This is most noticeable in dark foliage where the image can become a little "muddy" for want of a better word. Put the same footage through the NanoFlash and the fine, subtle detail is restored, along with the image noise. There is a good example of this in the sky frame grabs in the review I did of the NanoFlash last year. XDCAM-USER.com Convergent Design NanoFlash
If you consider that the grain in the NanoFlash sky is moving from frame to frame it is easy to see that this could react with the detail enhancement circuits of a consumer TV to create "shimmer" while the blocky artefacts from the 35Mb/s EX would not have the same effect.

Large screen consumer TV's are never a good way to judge the true quality of footage. They have all kinds of processing circuits adding who knows what to the image making it very difficult to make any kind of objective conclusion.