Peter,

Thanks for encouragement. You are so right in saying:

"The posted images fairly represent what he's seeing, otherwise he wouldn't be wasting his time and everyone else's with this thread"...

As to your suggestions, I don't think comparing Component with HD-SDI /HDMI makes sense; the high frequency detail tend to be filtered out in Component connection (also depending on the cable quality), so in my experience, Component is always less noisy/detailed than SDI/HDMI. Apart from my 24" computer monitor, I have a 50" plasma hooked up to my editing PC, but that doesn't have SDI input. So, what I'm going to do in order to make sure where exactly the noise is increased in the chain, is to connect my nanoFlash HDMI output directly to the 50" plasma (now it drives my Marshall monitor), and - having set up a low-light scene likely to have considerable noise - try to compare:

- the HDMI output from the nanoFlash when not recording (live; the nano will serve as HD-SDI -> HDMI converter)
- the nanoFlash recorded picture (at various bitrates)
- the SxS recorded picture, with the EX1 in playback mode and the nano acting as the HD-SDI -> HDMI converter again).

Will report my findings.

But nobody from CD reacting for 4 days now (nor replaying to my email from 2 days ago) is something I just cannot understand, and am disappointed with...

Piotr,

Thanks for going through all this effort and sharing it with us. In CD's defense, it was a holiday weekend that extended into Monday here in the US, so they very well may be playing catch-up and a lot of times people use that weekend to plan vacations around.

But I hope they respond. I would think they would.

Thanks for the info about the holiday in the US, Peter - I had no idea about the extended weekend you're having.

In that case, I'm still hoping Dan will respond the way he always did, when only he's back...

Dear Piotr,

I am very sorry that I have not been able to respond to your post until now. I saw your post Monday evening, just before I left on a business trip. Due to my travels, I have been unable to respond to emails until now. After reading your original post, I immediately discussed it with one of our engineers and asked them to respond.

My take on this is simple.

Codecs, when compressing images remove some detail. Lower bit-rate codecs remove more detail. Some high bit-rate codecs may remove no detail or almost no detail.

If one creates an image that has noise in it, on purpose or not, then the nanoFlash, when using high-bit rate options, will preseve that noise (or a great deal of that noise).

If one has a high quality HD-SDI monitor, a professional monitor, then one can monitor the live HD-SDI signal out of a camera. Run a test, and carefully examine the image, while recording the image to the nanoFlash.

Then playback the image on the same monitor. Using our high bit-rate options, the image should appear the same or very close.


As a side note, I had the opportunity to test some consumer HDMI televisions and monitors recently. I was surprised to see quite a lot of noise in the images. I was not expecting to see noise. Frankly, I have been spoiled by the images from a very nice Sony LMD-2450wHD 24" monitor.

With the same images, the Sony monitor is noise free to my eye, but two consumer monitors (consumer grade, under $375 for one, and $250 for the other, were very noisy.

I was testing these monitors to see if they were good enough to use at trade shows to demonstrate the nanoFlash. They were not.

Thanks Dan. This more or less confirms one of my own possible explanations (expressed also by a couple of others in this thread), that EX1's own, 35 Mbps 4:2:0 codec removes more noise along with quite some detail, while the nanoFlash'es codec retains the more detail (and noise, if present in the source picture), the higher bit rate it works at.

In other words, the GiCo rule at work :)

Considering the above, I guess that even though CD's latest information confirmed the potential of substantial quality increase with bitrates over 100 Mbps, there is a good reason to stick at the 100 Mbps as the "sweet spot" - and not just for storage consideration alone.

Especially with cameras like the EX1, which - under some circumstances (low light, uniform color areas in the scene, certain gamma curves and detail settings) - tend to produce a rather noisy picture...

But this experiment also made me think about another wish for the future nanoFlash firmware releases, which I'd like to know your opinion about:

- is it viable that CD adds some (user selectable from the menu), mild noise reduction circuitry?

This would make it possible to use all the benefits of higher bitrates (less compression artefacts), while not augmenting the level of noise from the source camera. Of course, at the cost of some detail reduction - but sometimes it could be beneficial (provided the NR can be switched on and off by the conscious user).

What do you think?

My observation was that CD and Dan had been offering outstanding support for many months before I purchased mine. It was a prime reason I felt comfortable making the purchase. Realtime support on DVInfo.net is above and beyond what is required, and clearly they have been supportive of the owners, answering questions, and with ongoing updates and free upgrades. Kudos to Dan and Mike, was never in doubt with me and I'm sure many others including Piotr.

I absolutely agree - have been in contact with Mike long before their products were launched.

As to Dan's responsiveness, I guess I've got spoiled by how great it has been - hence being a little impatient now, when it was lacking for just a couple of days :)

I would rather record it and do it in post using a processor with several times the power of that contained in the NF and a program that I can control the amount and parameters for noise suppression. Ads time to the post production workflow but would ultimately give much more control over the quality of the picture.

Garrett

Doing it in post is always an option - but as you said, can be very tedious and time consuming.

Also, I have yet to find an NR tool for Vegas that I'd really be satisfied with...

Dear Piotr,

One problem of removing noise in the nanoFlash is that it would be permanent. Doing it in post allows one to adjust the noise reduction.

Also, I doubt if we have the extra horsepower to do this inside the Flash XDR or nanoFlash while recording at high bit-rates.

If possible, I would recommend setting up the camera to achieve a good, low-noise image, as verified by a professional HD-SDI monitor. However, I realize that this is not always possible.

Dear Tom,

Thanks for the kind works. We try hard to provide prompt support.

Sometimes when traveling, it is difficult to respond promptly.

Fair enough, Dan.

Now that I can be sure that what I'm seeing is normal (and neither my camera's HD-SDI nor my nano is malfunctioning), the only thing I can to do is find a new, optimum combination of my EX1 settings and nanoFlash bitrate.

Thanks for addressing my doubts, and apologies for getting a little impatient before you did:)

Piotr

Agree with Bob Grant. Would be interesting to see how the noise would fare at comparable bit rates. Of course I'm curious now how the 100mbps rate would look, since that is actually the only bit rate I have used...(cough..ahem)...I've alway thought I saw a noticeable improvement particularly with interlaced footage.

Thant's exactly what I'm going to do, Tom - shoot exactly the same scene again, but with nanoFlash at 35 Mbps. If there is no noise increase from SxS to the nano, my camera's HD-SDI output is fine.

Since I want the results to be easily comparable, I'm waiting now for the same kind of light there was during my 180 Mbps test.

Will keep you posted.

Piotr, it would be good to see 35 and 50 MBps on the Nano.

EX1 vs. nanoFlash at 50 Mbps
 

OK, since the wheather here has changed so that it would take some time to get identical lighting - . I have browsed my older shots, and found a couple of scenes where EX1 can be compared directly to 100 Mbps Long-GoP and 220 Mbps I-frame only from nanoFlash. I will not post all of them, as they've been taken without the test in mind so they represent different scenery, lighting, and camera settings - but I can tell you that the conclusions can be summarized as follows:

1. The higher bit rate of the nanoFlash, the more detail and grain in its picture when compared to the EX1's own encoder.

2. Generally, the above tendency is more pronounced with long-GoP than with I-frame only compression

3. When comparing some perfectly lit and exposed scenes, containing negligible noise - the difference is still visible, but this time the advantages of nano's high data rate prevail: the picture contains more detail with less mosquito noise, and comparable grain level as the EX1.

To conclude: as summarized before, the "Garbage in, Compost out" (or GiCo - the slightly modified version of the GiGo rule, copyrights with Adam Stanislav :)) rule applies. Also, it's difficult to say whether the EX1 does use some NR just before compression, or its encoder has been so designed that it also smooths out the grain a little (along with fine detail, of course).

In order to take full advantage of the nanoFlash (at 100 Mbps and above) - the picture must be perfect from the camera!

I'm posting just one pair of pictures, comparing EX1 compression (left) to that of the nanoFlash at 50 Mbps. As you can see, the difference is much less pronounced than in the 180 Mbps example, which was to be expected - as with more color resolution, 50 Mbps is just about the same compression level as the 35 Mbps of the EX1.

PS. By no means I'd like this discussion to cease - to the contrary, opinions and examples from other EX/NF combo users are welcome - as well as from CD, of course !

More testing, and conclusions...
 

OK, I've done some more tests today; quite different lighting (full sunshine) - so don't worry, I'm not going to scare you with my ugly pictures :)

However, I'd like to share with you some more conclusions on the subject (most of them may be obvious for some people, but I hope they'll be of some use to those considering buying nanoFlash, and got scared by my complaining about the noise). Important clarification: in spite of bright sunshine, I set up the testing so that areas of the same wooden barn fell into deep shadow (see the screen grab), and considerable noise was generated.

So, I tested several NF bitrates again for noise - but this time, I also played with detail settings of my EX1: I tested both 50 and 180 Mbps (NF) against the EX1 with both detail off and on, the latter with crispenig at zero (default) and +35. I was hoping that at the highest bitrate, the nanoFlash would compensate for detail loss due to detail off/positive crispening, while not suffering from the (reduced) source noise...

Unfortunately, this is not happening. Even with detail off, or with detail on and crispening at 35 - at the highest long-GoP bitrate the noise is still augmented and really distracting when watching on my 50" plasma. With 50 Mbps, it's at the same level as on SxS, with color resolution enhanced and mosquito noise reduced.

So, I'll repeat: those using the EX-series cameras in run&gun (or ENG) fashion, should stick to the 50 Mbps data rate of their nanoFlashes. The bitrates of 100 and above are for those shooting in controlled environment (or those lucky ones having cameras with higher S/N).

Well - this was the bad news (sort of). The good news (for Vegas users, like myself) is that I have tried Mike Crash's Dynamic Noise Reduction filter in Vegas, and yes - it does remove most of the 180 Mbps nanofiles' excessive grain, while retaining most of the benefits (better edge definition, color resolution, and less mosquito noise). Let's hope it'll become available for the 64bit version of Vegas soon! Of course, I'm now talking about a free noise filter; those using more advanced NLEs with more robust NR, will be even better off.

To round it up: frankly, I feel quite relaxed now that I'm more aware of the problem's nature. And limiting my nanoFlash to 50 Mbps, 4:2:2 in everyday shooting is not a bad thing, after all - at least, I'll spare some CF and HDD space.

While with the live classical music recordings I do as my "serious" projects, I hope I'll have enough light and time to optimize my EX1 settings so that I can safely use 100 Mbps on the nanoFlash, as well.

It this is the filter you are talking about, it is not likely to come out in 64 bits. I just downloaded its source code and it uses inline assembly language, which is 32-bit specific. So it is not a matter of simply recompiling the code for 64 bits, it would require a rewrite of some of the code.

Not impossible, of course, but we are talking about code not updated since 2004...

Thanks Adam - do you know of a better, and 64bit, NR filer for Vegas?

Thanks,

Piotr

"Neat Video" works in Vegas 32 & 64 bit.

Request to Dan (CD)...
 

Thanks Steve; I read on the SCS forum that Neat Video is even slower than the free DNR from Mike Crash...

Which provokes me to one more comment on the whole matter (sorry guys - I really thought I already could just accept the facts, and settle down with the conclusions I mentioned in my previous posts):

It really bugs me that in order to take advantage of what is supposed to be the most important nanoFlash strength - the high data rates - without substantial side-effects, one ever needs to consider using additional software tool... And with quite a good camera like the EX1/3, too!

Noise removal in post is not only a lengthy process; it also is very susceptible to user errors, leading to not always optimal, and sometimes unpredictable results. Also, using a tool like this is impossible without some side effects, partially negating the very purpose of recording to nanoFlash in the first place!

Therefore, I'd be very grateful indeed to Dan (or anybody else from Convergent Design), if the following was addressed firmly, and after giving it some thought is deserves in my opinion:

1. Can you confirm my conclusions in this thread - i.e., is what I've described confirmed with your own testings? Dan - I'd really like to exclude hardware malfunction from the equation, and I still seem to be the only one bothered with the problem :) Alister Chapman, for instance, said that "he rarely sees a huge difference between noise in his 35Mb footage and NanoFlash footage" in this thread: http://www.dvinfo.net/forum/sony-xdc...ml#post1534295

If however the answer to point 1 is indeed "yes", then let me ask again:

2. Could you re-consider my original idea of adding some NR to the future NanoFlash firmware? I hear what you say, Dan, that once smoothed in hardware it couldn't be undone - but I don't agree it makes the idea invalid, as the NR could be a menu *option* for the conscious user, OFF by default (perhaps even returning to the OFF setting with each power-cycling).

A moderate noise reduction in hardware wouldn't take any more time than the recording itself, and could be much more consistent than anything software. Of course, the algorithm could be more or less comprehensive and intelligent, depending on the horsepower available...

What do you say?

What is the source of the noise that you are talking about when shooting in bright sunlight? The signal should be strong enough and have enough color to make internal camera noise negligible, no?

What makes you think there is noise in bright sunshine?

The problem is increasing the noise generated in low-light situations, particularly on large areas of uniform color. Such areas may also be present in the scene even in bright sunshine - it's enough that a wall, for instance, happen to be in a deep shadow, and the exposure used is such that the other, bright areas are not blow out.

But of course bright areas of the picture, where the signal is strong enough, do NOT pose any problems with the 54 dB of the S/N ratio. If you understood otherwise, my apologies - English is not my native language :)

This may be the strangest thread in the forum; my 2¢.

It seems to me that you're on the wrong path here. Noise reduction does NOT belong in the recorder. The recorder's job is to record and playback your pictures without changing their quality.

If I were pursuing your quest, I would be devoting more of my time learning out how to make my camera look better in less than ideal situations, via the camera settings available to you. You don't like the noise in the pictures in some situations? Adjust the camera settings to reduce the noise. Or get a less noisy camera. Or remove the noise in post. Once you throw away information, you can't get it back again. Don't crush your peds. Don't over expose your whites. Don't suppress your resolution/detail.

Philosophically, wouldn't you want to adjust your camera's picture as best you can when you're shooting, with care taken to throw away as little information as possible, while still maintaing the look you want? Then you want your recorder to record your pictures as identically to what the camera is outputting as possible. If you want to alter the picture your camera is making, you want to do it in post, where you can always go back to the camera master and try different things. If a company came out with a spectacular noise reducer next month, and it didn't reduce the resolution/sharpness of the image at all, but it removed 99% of the noise, would you rather your camera masters were sharp and noisy, or mushy, full of artifacts, but quieter?

Even the very best, most expensive cameras impose tradeoffs depending on the shooting conditions and what you're shooting. I'm not familiar with the EX series; the Sony cameras I use are part of the professional broadcast line. Their detail circuits have dozens of adjustments, as well as the best noise reduction circuits in the business. There are essentially unlimited gamma tables and adjustments. I don't expect an EX to have all these adjustments, but it must have some. Next time you're shooting a house in the woods with most of the picture in the shadows, take a minute or two to play with your crisp control and watch on a really good monitor. There is somewhere between minimum and maximum that will reduce the amount of noise without throwing away all the fine detail. If you can't critically evaluate in the field, then make a handful of recordings with different settings so you can evaluate later. Just make sure you use a nano (in 100Mb or higher). If you use the EX's built in recorder, it will throw away so much information that you won't be able to tell very much.

Billy

What you say directly above makes perfect sense to me. But further up you talked about a new set of tests you did in bright sunlight and said the noise in those shots still presented a problem.

Billy, you're fee to your own opinion of course, but this thread has proved very useful (to me anyway). It brings out some very interesting "issues" that come along with the obvious benefits of the nano. I never thought about how poorer codecs can actually mask noise.

Strange it may be, but this thread is also very interesting. We're all just exploring this very new tool and how it mates with different cameras.

BTW, I DO agree with you that NR should be saved for post. There is no way a device like the nano could do any type of sophisticated NR, so you'd be stuck w/ simple blur... which can be done in post as a realtime effect.

Peter,

As I said - our communication problem arise from my poor English; sorry.

And to make it clear: yes, I did more tests in bright sunshine, but I was judging results on the same portion of the wooden barn, taking advantage of the fact that it was partially in the shadow (see addition to the unclear post you're referring to). So while the picture as a whole were bright and generally noise-free, this particular crop was buzzing with noise...

Thanks for pointing this confusion out, and creating an opportunity for me to explain :)

Piotr

Edit
To clarify even more, I'm posting similar blown-up crops of the shadowy wood areas; as usual the SxS is at the LHS, and the 180 Mbps nano - at the RHS. This time, crispening was set to +35, which softens the picture quite a lot, but still doesn't prevent nanoFlash from adding grain. Looking at the static crops doesn't give full idea of how bad the grain is - even though they've been blown-up again; on my 50" plasma the added (or should I say: not removed) grain is shimmering at full screen playback (lower 2 grabs), thus becoming even more noticeable.

Dear Billy,

If you care to search my posts in the EX1 forum, you will find I know how to use my camera controls, and minimize noise.

If so, you really shouldn't find my quest "strange" at all; I wrote:

"those using the EX-series cameras in run&gun (or ENG) fashion, should stick to the 50 Mbps data rate of their nanoFlashes. The bitrates of 100 and above are for those shooting in controlled environment (or those lucky ones having cameras with higher S/N)."

I guess you fall into the category of those lucky ones - but please let us mortals find the optimum ways of using our EX1/3 with the nanoFlash - this is what I'm trying to achieve here...

Cheers Billy,

Piotr

Piotr, just for kicks, what happens if you go up to 280 but don't blur the image using the Crispening setting? I'm wondering if some of the "noise" starts to resemble detail.

Sorry mate - my CF cards won't do 280 Mbps :)

Cheers

Piotr

I think what Billy points out though is that a well adjusted EX1/3 does not equal the quality and amount of processing and control available in a true broadcast camera (which can cost 10 times as much).

In this forum EX1/3 cameras are considered a great camera, in the broadcast world I would consider them great cameras compared to their extremely cheap cost. They have drawbacks and limitations. And the Nano seems to capture that.

Luckily the Nano also seems to enhance these cameras in many positive ways as well.

Jeff

Fully agree - it's amazing that the nano offers the flexibility (like the wide choice of bit rates available) that allow for optimum use with cameras of any class, and picture quality.

I'm just trying to find and confirm those optimum nanoFlash settings for the EX series cameras, with their limitations in mind.

Indeed I do consider myself one of the lucky ones, though I'm just as mortal as everyone else. I'm glad to hear that you're on top of wringing the most from your camera, no matter where it fits in the range of horrible to spectacular. All I was really trying to say was that degrading your pictures in the recorder strikes me as a bad idea, even if the degradation also has the side effect of masking one of the limitations of the camera. Perhaps "strange" wasn't the right word; I didn't mean to be (or appear) condescending.

With what I've seen from the stills you've uploaded, I do not agree that "those using the EX should stick to the 50Mb codec". I find the mushing of the fine detail and the presence of artifacts ("mosquito noise", etc) far more disturbing than the shadow noise. But maybe that's just me. Starving a codec causes more problems, in most cases, than it solves; and none of your examples take into account motion. Once you add motion to the equation, restricting yourself to 50Mb will add much more degradation. Sure there are times that using a low bit rate codec makes sense, but as a general rule, I think it's misguided.

In any case, I admire your quest to understand and to get the best from your equipment, I just don't think camera noise reduction belongs in the recorder, nor do I believe it's a good general rule to limit yourself to a 50Mb codec when you record to the nano with an EX. The EX camera section is far more capable than the recorder section, and as far as I can tell, one of the main reasons for the nano is to provide a better end result in exactly that situation (better camera section than recorder section).

Billy

Perhaps my choice of the word "strange" was ill conceived. I was trying not to appear condescending, though I seem to have failed. :) What I found "strange" was not the content or intent of the discussion, but the direction it was taking (i.e., "build noise reduction into your recorder and use a low bitrate codec").

So I find the thread interesting as well. (I've not missed reading a single message here for almost a year, though I rarely post, unless I find something particularly interesting). Codecs do play a huge part in how the final picture looks, as well as how easy or difficult it is to edit. I find the Sony MPEG2 hardware codec used in the nano to be particularly spectacular, especially when setting it to 100Mb or higher.

Billy

Dear Billy,

First of all, I didn't find your comments "condescending" - no bad feelings at all, as we're all equally entitled to express our opinions here.

Please consider that my conclusion you quoted - "those using the EX should stick to the 50Mb codec" - has indeed been conditional, i.e. I recommended the above for those shooting with EX-series cameras in uncontrolled environment only (like run&gun of the ENG shooting). However, I also suggested that those who have time and means to provide adequate lighting, scene composition, and camera setting tweaks, will certainly benefit from 100 Mbps (or even higher) bitrate recording on their nanoFlashes.

Also, the mosquito noise etc may look more disturbing than the shimmering grain I'm investigating in the stills I posted, However - and this I also stressed many times - when watching actual video (i.e. moving pictures) on a big enough display device, it's the shimmering noise that is most noticeable, and hence - disturbing.

To confirm my above conclusions, the point 2 in particular:

I have repeated extensive tests today, comparing SxS noise against that of the nanoFlash at 220 Mbps - and can assure you the nanoFlash is much cleaner. So, regarding the camera noise in low-lit or over-shadowed areas, here are my findings:

1. The nanoFlash I-frame only at 220 Mbps (my cards are not fast enough to test 280) is the cleanest by far !!!

2. The SxS (i..e. EX1's own 35 Mbps 4:2:0) is as clean as nanoFlash Long-GoP at up to 50 Mbps

3. With bitrates of 100 Mbps and above, the nanoFlash Long-GoP becomes more and more noisier than SxS.

My previous conclusions/recommendations, based around Long-GoP recordings mainly, should be adjusted accordingly.

Hope it helps,

Piotr

Hi Piotr,

2 questions:

1. Can you get hold of another nanoFlash where you live for a comparison?
2. You recommend 50Mbps for run & gun and 100+ for controlled situations. What about 80Mbps? Does that not offer any advantage over 50, but still is just shy of the 100Mbps?

Cheers

Hi David,

To answer your questions:

1. I know of only one other nanoFlash owner in Poland (Dan, is there more?), but haven't contacted him yet. I probably will, just to make sure my unit is up to the specs (but frankly, I'm 99.9% sure it is OK)

2. Frankly, I didn't test all bitrates as carefully as those extreme or typical ones (hence I did the 50 as a minimum that makes sense and is accepted for broadcast, the long-GoP 180 / I-Frame 220 as the maximums for my CF, needed for heavy grading in post, and the long-GoP 100 as the sweet spot, recommended by CD.

The noise increase (or lack of it) depends on so many other variables, that introducing additional nF speeds (like 80 or 140) was not of particular interest to me at this stage - but perhaps I'll include those speeds in the future tests.

1. The nanoFlash I-frame only at 220 Mbps (my cards are not fast enough to test 280) is the cleanest by far !!!

sorry for the dumb question, but you mean by this: 220 Mbit is cleanest than the native ex-1 codec or the cleanest of the nanoflash flavors that you are able to test by the cards speed.

greetings

Here are two examples from the same scene shot with 720p60 and Gain at + 6dB. The Nanoflash (140 MBps, LongGOP) is showing greater detail/less macro-blocking on the face and less mosquito-noise on the moving blade than the SxS (35 MBps) . Left side is SxS and right side in Nano. Overall, I'd say that the Nano is like adding a better lens.

http://www.gentlemensfightingclub.co...e_RVface4x.tif

http://gentlemensfightingclub.com/Im...re_Blade2x.png

Full frame grabs here :
Index of /Images/Video/NanoflashTest