Julio,

The I-frame 220 Mbps is cleanest of all nf flavors I can test with my CF cards, AND cleaner than the native EX-1 codec. Of course, we're still talking noise here.

Gints,

Thanks for this. I never doubted in nanoFlash beating the EX-1 native codec as far as most compression features are considered (mosquito noise, macro-blocking, edge and color definition), in all its modes and data rates. It's only the noise on low-signal picture areas that can be worse that on SxS, and only with 100+ Mbps Lon-GoP.

Attached is the results of a quick test I did yesterday.
EX1, iris and lens hood closed, 18dB gain, ex factory camera settings. Waveform monitor built into Panasonic BT-LH1760 17" monitor.
L.H. screenshot is the HD-SDI direct from the camera. R.H side is the same recorded and played back over HD-SDI into the monitor. I also tried a normal image at 18dB gain and noted the same result.
Within the resolution of the waveform monitor no difference in the noise levels between the signal fed from the camera over HD-SDI and that recorded by the camera.

I would suggest that if evaluating artifacts such as mosquito noise then a HDTV should not be relied on. Real instruments are what is needed but unfortunately I do not have ready access to a hardware HD-SDI scope good enough to perform those kind of tests. Given the complexities of mpeg-2 compression even if I did I have no clue as to how to create a proper test scenario either.

Comments please, Bob :)

Because as Alister puts it:

"The very act of compressing a signal will reduce the appearance of noise as part of the compression process is to discard some picture information, so noise often gets "smoothed out".".

And I'm prone to agree with his statement, so more compression by EX1's own encoder -> less noise from SxS than from the nanoFlash at high bit rates (less compression).

Therefor I don't see the point in making tests while tweaking the camera settings.
Changing the PPs we are changing the system noise.
I would make a test with the Factory Default setting. I would compare the picture on the SxS with the one on the Nano and an SDI capture through a video card to Prores or so.
IMO if its happens that the SxS shows less noise I can only think that the in camera MPEG-2 processor introduces some kind of de-noising prior to compression.
I'm not concern about noise (I de-noise everything) but in this thread there are many interesting observations statements that I would like to check.
rafael

Dear Dan,

Could Convergent Design please answer the above question of mine authoritatively? I'll reword it to make it simple:

- is it normal in your testing, that at high bitrates like 180 Mbps, Long-GoP nanoFlash files tend to contain noise higher than the camera's own encoder (like the EX1/3 35 Mbps, 4:2:0)?


Also, why do you think it only happens with Long-GoP format (I-Frame only files at up to 220 Mbps do not show noise increase)?

Thanks,

Piotr

Dear Piotr,

In my opinion, the proper test is to connect a professional HD-SDI monitor to your camera's live output and check for noise.

While you are checking for noise, then you can record using the nanoFlash at whatever bit rate you wish.

Then, playback, from the nanoFlash the same footage.

I have done this test hundreds of times, using a $5,000 Sony LMD-2460wHD monitor, at trade shows. I have tried all types of scenes, and a wide variety of bit-rates/options, using the nanoFlash.

All of this testing was done for hundreds of industry experts. Most said, "Wow, that is really clean".

Others had the same opinion but asked for further tests. One wanted to see deep shadows, so I focused on a very dark area of the ceiling at NAB in 2009. The result was the same: Very Clean!

One expert spent 30 minutes checking the images. He asked to see just the blue channel; it was clean also.

Priotr, you can run another test:

Set up the nanoFlash for 35 Mbps (1920 x 1080) and compare the footage produced via the nanoFlash and your camera.


Please note that high compression / low bit-rate codecs, such as the 35 Mbps codec will hide/remove some detail (and noise). Higher bit-rate codecs will preserve more of this detail and noise. This is perfectly normal.

The proper test, in my opinion, is the examine the live versus recorded images on a very high quality monitor.

It is not suprising to me at all that most any camera can be adjusted/misadjusted so that noise may be visible in the extreme shadows at 600% magnification.

Dear Dan,

Thanks for your answer. What you are saying is describing what things are supposed to be with the nanoFlash, unfortunately it's not how they actually are with my unit.

I know I should be using a proper, calibrated, professional HD-SDI monitor, which I haven't got unfortunately. Also, perhaps my previous examples of enlarged crops exaggerated things. Therefore, I'm posting two screen-grabs now that have NOT been enlarged. Please look at the back-lit surface of the house wall, and tell me where the "WOW" factor is...

The left hand one is from the EX1 (35Mbps, 4:2:0). The right hand one is from the nanoFlash at the mere 100 Mbps, 4:2:2, Long-GoP (previous examples of enlarged crops used the rather extreme 180 Mbps). I don't think I need a professional display to see the difference, Dan. While enlarged crops clearly show less macroblocking and better color resolution of the nanoFlash files, one doesn't need to enlarge the nanoFlash Long-Gop clips to see they are more noisy!

So, again - is what I have posted now normal? Because frankly, you're contradicting yourself in your answer, speaking first about how "much cleaner" the nanoFlash is, only to later admit that more noise is "normal"...

Thanks,

Piotr

PS. Disclaimer:

Please note I'm by no means bashing, ranting, or splitting hairs. I just want to understand things better, make sure my hardware is not malfunctioning, and find the best settings to use.

Dear Priotr,

I will let others critique those two images.

To be clear:

If you create a noisy/detailed image with your camera, the Sony EX1/EX3 will mask some of that noise/detail when you record it in camera at 35 Mbps.

If you record it in the nanoFlash, more of that noise/detail, will be preserved when recorded at higher bit-rates.

I currently see no reason to suspect that your nanoFlash is malfunctioning.

Dear Dan,

I'd like to make it clear that it's not necessarily the nanoFlash, but my camera HD-SDI port, or cables, that might be malfunctioning. My problem is I have no means of excluding any from the equations, hence my request to have somebody say "yes, this increase in noise is normal" or "no, there should be no such an increase in noise"...

Thanks, anyway.

Piotr, thank you for posting the images. I'm using an image viewer that allows me to toggle between the two images, so it would be good to see the same frame from the EX1 and Nano. This is sometimes not easy to confirm with little or no motion, so consider swinging a stick in the frame somewhere. This would also provide an example for noise due to motion.

In your examples, I concur there is less EX1 noise on constant color areas such as the painted house, but there is also less detail and more "mosquito noise". In both the shadows (boards above door, wood fence on right) and the midtones (roof in sun), the Nano 100 MBPs shows a real increase in detail and less mosquito noise on the high contrast edges such as the top and bottom of the fence.

I would like to see the scene with your 220+ i-frame-only encoding and 140+ MBs LongGOP. 100 MBps may not be enough to see sufficient increases in detail for you to see value in the Nanoflash.

PP settings
 

Hello Piotr,
I understand the dilemma you are in regarding equipment malfunctioning.
Regarding last posted pictures, they are too milky and I couldn't see it properly. Piotr, is there any way, that you could clear your PP to Sony Camera PMW-EX1 default settings and shot, record and record to NF at 35 MBPS just as the camera records at the same rate. Do that with the NF on all possible settings of BitRate at the same light and object and we could talk about it after-words. The thread is very informative but it seems not clear, at least for me simply because the settings on your camera are not the default settings.
Just an idea...
Luben
Cheers

Only conclusion I can draw from my very limited tests is that the noise *level* as recorded by the EX1 and as fed out the HD-SDI port is roughly (+/-20%) the same. As I said I lack easy access to the expensive test equipment to make a more detailed and accurate test.

Alisters comments are entirely true from my past experience and my understanding of how mpeg-2 works. Encoding very noisy VHS and grainy 8mm film to mpeg-2 really shows this up. Under extreme stress the encoder will simply only encode the noise pattern at the I frames. Most of my clients thought it looked better than the original, years after that experience I still don't know if I should laugh or cry.

As to your more general issue I cannot understand why you are using underexposed images. Of course better recording systems will preserve more detail and more detail is more noise. More detail also means more line twitter, you are quite likely pushing very close to the Nyquist limit as well.This is video science 101 stuff. If you want to test if the NF or something else in your signal chain is *adding* noise then record the in camera test patterns. They should have no noise to start with.

As an interesting aside the same complaint has been levelled at the ProRes codec. We have a KiPro and I'd wager good money I'd get the same outcome as you're getting recording to that.

Regarding ProRes, except for 4:4:4:4, which looks utterly amazing, FWICT, the ten bit ProRes codecs seem to underperform compared to the competition. Don't want to start a war here, LOL.

Piotr,

I think you bring up an interesting point that other people here w/ nano's could easily test. Namely, "Does Long-GOP add more noise than I-Frame only?"

From what you've said, this behavior is w/ the Long-GOP codec only, not the I-Frame (both similar and higher bit rate), correct?

Gints,

Thanks for your comments. I don't think there is need for me to further tests, and post pictures here. I agree with your observations of the phenomena in both scene versions - the problem is that in order to see the nanoFlash format virtues, one needs to zoom into the pictures (or view them on a really large display). The excessive gran, on the other hand, can be seen without magnification, and on even a computer monitor. What's worse, in case of moving video it's shimmering, thus being even more annoying - and that's at just 100 Mbps! At 180 Mbps l-GoP, it's even worse.

Luben,

I have replaced the 2 pictures with the same 2, but taken after converting from sRGB to cRGB in Vegas - something I forgot to do originally, which caused the pics to look "milky".

I don't think there is a need for even more comparisons - if anything, I'd expect other users to confirm or deny my findings, so that I can be sure all my hardware is OK.

Yes Peter - for some reason, the I-Frame only picture looks cleaner.

Well - they cannot be the same, if the theory (which I also believe in) that the 35 Mbps EX1 codes smooths out the noise is true (unless your wide +/-20% margin accounts for that).

All true - but EX series cameras tend to be particularly noisy in underexposed or just shaded, uniform color areas, and in real life you cannot avoid them - especially in run and gun shooting style. This is why I deliberately chose such areas to test for noise...

Well, I have more or less come to terms with the conclusion that all this is normal. However, in order to easy my mind that none of my hardware components is malfunctioning, I'd be grateful if somebody just confirmed or denied my tests with his own ones. The other way would be of course to try my EX1 with another nanoFlash, and my nanoFlash with another EX1 - but I don't have such opportunity at the moment.

Thanks, Bob.

Piotr, while I agree that the noise can be seen, I see very similar noise in the video on a computer monitor for both SxS and Nano 100+ MBps. There is an obvious difference in the (macro) blocking noise in many scenes that I've shot, particularly flowing water, and it's more obvious in the video than in the frame grabs.

While I did present some samples zoomed in by 2x or 4x, it was easy enough to see the difference between the 35 MB SxS and 140 MB Nano in static unzoomed frames when switching between the two. If you compare the exact same frames, you'll see the difference as well.

Gints,

I guess you're right - in the same nanoFlash 100 Mbps L-GoP testing session I posted grabs from yesterday, I can find frames where the virtues (less macroblocking, better color resolution) are equally easy to see in the nano clips played back on my 50" plasma without any magnification, as the increased noise is....

In the examples attached below, one can see more noise on dark, uniform color (as ususally), areas - like the green waste bin, or the brown wood elements on the left. At the same time, however, there are many improvements in the 100 Mbps nanofile that can be see easily as well - like the black writing on the orange label of the basket, and in many more areas...

So I guess 100 Mbps can even be used in run&gun; the higher bitrates of the Long-GoP format, however, should be reserved to controlled lighting / camera setting only (at least, with the EX-series cameras).

Okay, but I was able to easily get the nano picture to look as smooth as the native EX-1 picture by simply adding a very light blur in post. Also, in post you can apply a matte to "de-noise" only those parts of the image you feel need correction. The EX1 smooths everything.

Peter, almost everything is possible in post...However, that's not the point of this thread.

:)

Buried within my clips, I have just come across something I didn't even intentionally record, but this raises my doubts again:(

In another thread on this forum (http://www.dvinfo.net/forum/converge...s-100mbps.html), it has been stressed how great the 100 Mbps Long-GoP is with motion, when compared to native EX1. I don't know how Jim Arthurs' samples were prepared, but I'm posting comparison of a single frame with a lot of motion blur. Please don't pay attention to the content - I know it's rubbish; I didn't shoot it carefully - just randomly moving the camera between two proper shots. Also, these are enlarged crops again. Nevertheless, it's the difference between native EX1 (left) and nanoFlash at 100 Mbps Long-GoP (right) that matters.

The nano clip is so pixelated it almost looks like falling apart!

Please comment.

I'm starting to think there is something wrong with your nano! You should probably send to it CD for them to fix it.

Several posts above, Dan stated:

"I currently see no reason to suspect that your nanoFlash is malfunctioning."

I really don't know what to think now!

Piotr,

If you want to compare pics from the SxS with LGOPs from the Nano, you must compare an "I" frame from both.

If you are comparing an "I" frame from the SxS with a "B" or "P" frame of the Nano, the test is not fair.

This may be the reason why you find no problems with the Intraframe: All are "I" frames.

I really don't see possible that the Nano introduces any noise over the HD/SD-SDI.

Rafael

A very good point, Rafael - thanks!

At last, some technical interpretation... I'll try and analyse my comparisons so far, keeping in mind what you said.

Nevertheless, what matters is the video on the big screen, right? And watching it, you don't analyze whether the current frame is an I, B, or P frame - what counts is the perceived quality. And that is poor - the shimmering noise is simply more visible in the nano clips :(

Thanks anyway.
Piotr

Dear Piotr,

In post 103, on the left side of the image are what appears to be a vertical wall, made out of red, rough wood. At least the texture looks rough.

What does it look like in real life?

Is the wood rough?

Does the 35 Mbps image represent reality (smooth wood with little detail) or does the 100 Mbps image represent reality (rough wood with lots of detail).

How did you create this image?

Was the camera on a stable tripod?

Was someone or the wind shaking the tree?

Or was someone shaking the camera?

What were the camera details? Frame Rate, Shutter, Aperature, gain, picture profile?

Specifically, what is your F Stop? With a 1/2" sensor it should not be greater than F8.

In previous examples, you did not always show the exact same frame. Are these the same frame?

If you want to test if the nanoFlash is working or not, I suggest you set it to 35 Mbps, Long-GOP, 1920 x 1080 mode, and compare the exact same frames. But, as Rafael pointed out, one may be a B or P frame and one may be an I-Frame making it hard.

I know of no way to ensure that the nanoFlash and EX1, on the same frame, will both be on an I-Frame.

If you want to show the effect of "smoothing" due to a low frame rate, record a properly exposed image, with lots of details at 220 Mbps versus 100 Mbps, both using I-Frame Only.

The Codec used in the nanoFlash, as you are probably well aware, is the one used in the Sony PDW-F800 camera. And, all of the compression work is done in this Sony Codec. It is highly unlikely that the Sony codec is failing you.

Just to be clear, you may, of course, send us your nanoFlash and we will be happy to check it out for you. But, we have never had a Sony Codec module fail, or even be suspect, as far as I know.

Dan would know better than me.

By the way, are you shooting straight with the EX-1, or are you using a 35 mm adapter? Because if the latter, the ground glass of the adapter may be causing similar issues.

I agree with you that what it counts is the video in the big scree, but if we compare stills we must to take in to account the issue of the "I" frames.
Piotr, I really appreciate what you are doing to try to put some light on this matter.
Tomorrow I will try to make few test recording in the SxS, Nano and in Prores with the ioHD.
The suggestion of Dan about recording 35Mbps in the Nano is also to consider.
rafael

No Adam - I'm not using the adapter in these tests. But thanks - a good point.

I've been making some tests that may be not the most scientistic, but I thing they point what is going on.
I have used NeatVideo.
The advantage of NeatVideo is that it works independently in the Luma and in the Chroma channels.
This allow to see what kind of noise prevails in the picture.
The pictures from the Nano (I used the Piotrs pics of the house), shows much more Chroma noise than the SxS picture.
That is not to surprise because the Nano pictures have double Chroma info than the SxS pictures.
About the noise in the Luma chanel, I see a bit more of noise in the Nano picture too, but the SxS picture is much more blurry.
For the one that have no NeatVideo, I propose a simple test: Drop any Color Correction filter and bring the Chroma all the way down, then compare the pictures.
As you can see, the SxS recording gets rid of a lot of the noise, but smudge the picture absolutely.
rafael
PS: since I bought my EX-1 I de-noise everything. Before I used VideoPurifier, now NeatVideo.

Dear Dan,

The wood is rather smooth. Of course there is some texture on it, but certainly not containing any other color than various shades of brown - while the grain I'm complaining about consists of dots in all possible colors.

This particular frame comes from a hand-held shoot (I wanted to check camera whip pans and shake), so it's quite different to the previous ones which were shot from the tripod.

The settings were: 1080/25p, 1/50th, aperture never outside the 2.8 - 5.6 range

As you know, it's almost impossible to align an EX1 native clip with its nanoFlash counterpart on two Vegas tracks - even if you switch Quantize to Frames Off.

Yes - with nanoFlash bitrates of 35 or even 50 Mbps, the noise problem is not existing.

Dan, in real life, in most images you will find both bright and dark areas (otherwise those images would represent a very narrow dynamic range). The dark areas tend to be noisy in the EX-series cameras, and this noise is augmented by the nanoFlash at higher bitrates.

Dan, I'm not saying it has failed on me - I'm asking why everybody is raving on how much cleaner the nanoFlash images are when they aren't. That is to say, they tent to amplify the source shortcomings (the noise) in a very nasty fashion. My problem is that while i often need to zoom into a nano frame in order to see its virtues (like less mosquito noise, and better color resolution - I created a separate thread on that, too), the shimmering noise I can see in the normal scale image, on all less-than-bright surfaces of more or less uniform color.

Thanks for this option; let me treat it as the last resort. For now, I'll appreciate technical explanations (with examples) much more. I hope they would prove your previous point that nothing is wrong with my nanoFlash...

Cheers

Piotr

Dear Rafael,

If you could try similar tests with your EX1/nanoFlash to confirm mine are working to specs, I'd be very grateful indeed. Just a single test of the native EX1 vs. nanoFlash in 180 Mbps L-GoP will suffice.

Then, with regained confidence in my equipment, I'll probably start using some de-noiser as well (and since I work with Vegas Pro, it'll most probably be NeatVideo).

Thanks again,

Piotr

Dear Piotr,

Thank you very much for the extra information.

I helps us understand.

If I may add my .02, it seems that the real issue is the difference between how clean Long-GOP vs I-Frame recording is.

If the nano is just recording noise as extra detail, then you would think the I-Frame recording would do the same. But that doesn't seem to be the case, if I've understood this thread correctly (which is not an easy thing, LOL).

IIUC, at the same and higher data rates, the I-Frame recording is significantly cleaner than the Long-GOP. This doesn't sound like a defect in the nano, but rather a subtlety w/ how the Sony chip and MPEG-2 encode Long-GOP vs. I-Frame. And it seems possible that this difference may have gone unnoticed or unmentioned until now.

I would humbly suggest doing a test of Long-GOP vs I-Frame at various data rates of a scene that includes shadows. If there is no "noise" difference between the two recording methods, than it seems that Piotr's nano is defective. But if the test shows extra shadow "noise" in the Long-GOP recording, then Piotr's nano is behaving normally and this behavior is a function of how the Sony chip encodes.

In the best case, Sony can shed some light on this. In the worse case, it's information users can keep in mind when balancing data rate, quality and scene composition.

Hope that makes sense :).

This makes perfect sense, Peter - and it's this type of output result I've been counting on when creating this thread (and NOT ranting, or bashing anybody).

:)

Piotr

Dear Dan,

I'm glad I made myself more clear now. I'd like to stress it again that my intention in this thread is only to better understand things, and get confidence my hardware is not malfunctioning (not necessarily the nanoFlash, but also my EX1 and the HD-SDI ports and cable). Therefore, I'd really appreciate some technical follow-up to my points from the Convergent Design engineers, which would help achieve the above goals. I'm sure other EX/nanoFlash users will benefit, as well.

At the moment, just two additional qusetions, if you will:

1. Here is what Mike wrote in another thread:

Now, is there a possibility that something could have gone wrong in this "cranking up" of the Sony codec chip?

2. On yet another subject:

We have both agreed it's very difficult to synchronize the picture from EX1 and the nano perfectly when on separate tracks, one above the other, in an NLE. I do a lot of multicamera projects, and when only using material from several EX cameras, I'm able to synchronize them with frame accuracy using spikes in the audio waveforms. This method doesn't work with an EX1 and nanoFlash files, recorded simultaneously - could you please explain why, in technical terms?

Thanks and Kind Regards

Piotr

I second that!! I use PluralEyes and FCP does not permit multiclip with different codec (EX1 35Mbps and nanoFlash 50mbps) It is a huge hassle....

Dear Piotr and Luben,

I wonder if all of the souces were the same, or if using the Sony SxS 35 Mbps recordings and the nanoFlashes 35 Mbps recordings, would match up find in PuralEyes.

I can easily speculate that PuralEyes was not designed to work with different codecs or bit-rates, but I really do not have any actual knowledge on this subject.