Alister,

Thanks for chiming in.

Your explanation of what's going on with nanoFlash vs. EX1 encoding, and viewing results, is the most convincing so far. The more so that you also admit the EX1 is noisy in the first place...

The excerpt I quoted has drawn my attention because - even though I have all the user-accessible "enhancements" turned off - there still may be some doing their lousy job in my plasma.

To all that might be concerned:

When I started these tests I initially focused on the augmented noise in low-light areas. I concluded that from this viewpoint, the L-GoP format is only good for up to 100 Mbps; otherwise the I-Frame only format should be used. This was also confirmed by Dan and Mike of Convergent Design.

Later, when I started using I-Frame at 140 Mbps, I noticed the shimmering grain all over the picture - not just in the shadow areas. Its nature has been explained and confirmed by Alister. In order to get it mitigated, very high datarate is required - in fact, only at 220 Mbps does the overall level of shimmering (low-light and otherwise) become comparable with L-GoP at 100 Mbps.

Therefore, I'd like to inform all those interested that when using with a noisy cameras like the EX series, the original recommendations of Convergent Design are correct, after all:

- to keep storage low, use L-GoP at 100 Mbps
- if you need I-Frame only, record at 220 Mbps or higher.

I'd like to apologize to Dan and Mike for introducing some confusion here. That said, I still am a little dismayed with the results one can obtain with the nanoFlash, if either the camera source or the lighting and scene composition are not of the highest quality imaginable.

Thanks all,

Piotr

Piotr, a couple thoughts.

In my suite with JVC DT series pro evaluation monitors the footage from the EXcams looks much better than my Sony 46" Plasma and my Panasonic consumer LCD's. I think Alister's point is very valid, others have noted this in this thread before.

If you view a large plasma at less than 3X the distance to screen size, you will see all kinds of junk.

Grading footage before delivering product is a big part of production. This is where the NanoFlash footage really shines. It will hold up much better than the SxS footage. There are also many ways of removing "grain" in post.

Also bear in mind how the product is delivered. SD DVD, BluRay, Video server, Internet streaming or broadcast. There are probably more options as well. But the finished product is really what counts.

I think the NanoFlash footage has an advantage in all these cases as the basic GIGO effect is always multiplied.

This is not to say that a better camera does not help. Of course it does. But it really is pretty remarkable that we can spend less than $10,000.00 on a camera package that produces the kind of images we now take for granted.

Thanks Olof for your remarks - appreciated. I realize the true advantage of the nanoFlash encoding only comes into play with heavy grading, multiple renders, etc. But I must admit I still haven't established my best workflow using Vegas (where the output possibilities are limited, and even "raw" clips at anything above 100 Mbps can hardly play back at full speed/quality).

Olof and Alister - could I ask you for a favor of proposing some typical workflows for nanoFlash material, with different editing extent and delivery methods in mind? Of course, it could be a separate thread... I'm sure many nanoFlash users could benefit!

Back to this thread: I have done a very extensive and exhaustive series of tests by now, the conclusions being what I have written above.

While I realize that - in the first part of my thread (mainly on the low-light noise augmenting) - I have speculated I-Fo might actually be better than L-GoP for bitrates above 100, it still beats me why CD guys (whose participation and support I value very high indeed) jumped so quickly from their former recommendations of L-GoP over I-Fo, to stating explicitly that I-Fo from 140 Mbps up is superior (?!?).

This is absolutely not the case; in fact, nanoFlash I-Fo at 100 or even 140 Mbps has more artifacts (not just noise) than the native EX, 35 Mbps 4:2:0 codec! I said "not just noise", because the EX codec is known to be very good at suppresing it (at the expense of detail and macro-blocking). But even the mosquito noise and macro-blocking is worse from the nanoFlash at those speeds using I-Fo!

Only at some 180 Mbps, I-Fo is beginning to look as good as the XDCAM EX. Only at 220 Mbps, can it be compared to the L-GoP at 100 Mbps. Anything above these datarate (respectively for both formats), the quality gain becomes irrelevant against the increase in storage space required.

Alistair Chapman wrote "If you consider that the grain in the NanoFlash sky is moving from frame to frame it is easy to see that this could react with the detail enhancement circuits of a consumer TV to create "shimmer" while the blocky artefacts from the 35Mb/s EX would not have the same effect."

My Sony Bravia 52" (2007, 1st gen) 1080p TV excessively amplifies the block edges even after disabling all possible image DSP controls from the user menu. A convincing test is to record LongGOP from 18 MBps, 35 MBps and 100 MBps. 18 MBps is horrendous on that TV, and there is no way that this is due to noise in my EX1.

Piotr is testing on a large plasma while I test with a large LCD. Many LCD reviews include statements about "sparkle" when viewing DVD. LCDs in the past did not have adequate reaction times, and I thought these sparkles mentioned were byproducts of algorithms used to increase the perceived reaction time of the LCDs. Sometimes the discussion about sparkle is really about macroblock edge, deinterlacing and sharpening artifacts. Plasma TVs are known for faster response times, but as Alistair writes, who knows that the image DSP is doing.

These are my conclusions as well, but for the scenes I've recorded, I would still say that I-Frame100 offers more detail, less crunchiness and less mosquito noise in motion areas than 35 Mbps. If you're picking on noise on a solid color, then I would agree with you. But when I-Frame220 is used, most of the noise and MPEG LongGOP-induced shimmering artifacts Piotr has pointed out are reduced.

I think what we really want is constant quality encoding rather than constant bitrate. We are noticing the interframe quality fluctuation starting at 100 Mbps LongGOP and are dismayed that the solution is to run I-Frame MPEG2 at 220+ Mbps, which is double the bitrate. Flash cards have increased in speed immensely in the last year and can handle variations in data rates at the higher Nanoflash encoder bitrates. Is constant quality an option on the Sony XDCAM chipset ?

I Frame 220 + 280 is it !
 

Hi Gints & Piotr:
I have been the lone voice in the wilderness maintaining from earliest times that I frame 220 and in particular 280 is far superior to any Long GOP bitrate you care to use. In fact, if we could get CD to give us a 300 or 320 Mbps I-Frame setting, then you would have a perfect MPEG 2 8 bit encode ! The CF cards are bigger, faster, and now cheap enough to justify this rate. Long GOP 50 is the only setting in Long GOP which matters - PERIOD !

Yes, Mark. I remember your first posts about 220 MBps. Keep in mind that the rest of us were just getting used to 3-4x storage requirements when moving from HDV and Sony SxS at a merely LongGOP 100 Mbps.

Would 300-320 Mbps be enough? You probably want 400-500 MBps (only 50-60 MBytes/second) to see a difference, right? Uncompressed 1920x1080p30 4:4:4 is about 1.5 BGps and 4:2:2 is about 1.0 GBps.

MPEG 2 Encoding for 52 Inch Flat Screens
 

Hi Gints:
I think I-Frame Intra encoding scheme is the only way to reduce the artifacts which translate as "translucent" on some of the larger screen TV's (?) Perhaps VBR encoding method could also be used to reduce overall file size somewhat for the higher data rate files encoded @ 220 & 280 Mbps I-Frame. Then again, I do not know if the Sony XDCAM HD encoding engine contained in the Nano & XDR allow for VBR encoding method ? I haven't gone back to the Convergent Design web site to see if VBR was part of their original spec. Everything I'm getting @ Long GOP 50 and I-Frame 220 and 280 looks pretty spectacular to my eye on my Sony HD SDI broadcast monitor, although after reading Piotr's posts I'm very curious to go out and watch my stuff on some really big TV's.

I already asked for this and was brushed off !
 

...Yeah, I asked for 300 & 320 I-Frame data rates a few months back and Dan was not impressed.

EDIT: Then again, I asked to put a switch on the menu to bypass encoding as an option altogether (At least possible on the old XDR), and that was even a more popular request at CD then my high data rate request ;-)

Mark, given the CF interface, what would you record uncompressed HD stream to? We would have to wait for a CF device capable of sustained 120-150 MBytes/second, right?

Mark, try recording some small water geyers at a nearby park for 10 seconds each at bitrates from 18-280 MBps, plugin them into to a TV via the HDMI port and sit back with one of those potent 9-10% ales, such as Sierra Nevada Bigfoot or Dogfish 90 Minute Pale Ale.

Try Raid 0
 

...Hey Gints:
Not necessary. Try Raid 0. Problem solved.

The Real Meaning of 2001: A Space Odessy
 

....Hey Gints:
I'll better that ! ;-) Try drinking 16 cups of fresh bean Moca Java slow percolated and sit down and watch the same playback. By then end of the playback you will understand the full meaning of 2001:A Space Odessy ;-) !

Mark, the most I've ever done is an Octo Espresso, which is two Quattros. How would you even order sixteen? "I'd like a HexaDeco Espresso, please?"

How would the XDR or Nano record these? My Nano only has CF slots for recording, and I don't see the appropriate connectors for external storage in the XDR pictures.

Dear Mark,

We like to be very open with our communications.

However, our upper recording bit-rate, what is actually possible, is something that we prefer not to divulge for competitive reasons.

No Disrespect Intended
 

Dear Dan:
I was never fishing for private company internal information of this nature about CD's products, I was only requesting a higher data rate option for I-Frame recording for special high end digital cinema or special effects shooting. I don't understand how requesting 300 Mbps or 320 Mbps could trespass on this. Sorry.

External Storage Not Needed
 

Hey Gints: No, no - You don't require any external storage for this feat. I'm not sure the Nano Flash can even do uncompressed, but I think the old XDR is built slightly differently internally (But I really do NOT know ???) You Raid 0 all 4 cards in an XDR or 2 cards in a Nano (If possible- I do NOT know). You use those newer 64 GB CF Cards which can do 60 and 90 MB per second playback from Sandisk (Forget their name - sorry), and you record in a striped configuration of 2 or 4 cards (Should be end user choice as option in menu). You shoot, but do NOT remove the striped CF card media from the Nano or XDR and you either....

1. Playout in realtime via HD-SDI from Nano or XDR.
2. Transfer files from XDR/Nano to computer via built in FireWire (Slow but do-able - Faster CF card media will transfer in Raid 0 config faster anyway)

You bypass the internal encoding engine with this option, thus you are inherently now up to 10 bits from 8.
* You could even stripe for *Compressed MPEG 2 recording ! Why not ? Then you can really boost your I-Frame encoding data rate

Sandisk Extreme does 60 MBytes/second write while Sandisk Extreme Pro does 90 MBytes/second . Now that I'm moving to 200 Mbps I-frame only, my wallet will become friends with the only fast 64 GB card Sandisk makes. I bought two spanking new 32 GByte Sandisk extreme cards eight months ago. How many times have I "re-promised" that I will never buy another $400+ flash card again ? Hey wait, there's no swearing at work.

CD would provide utility to interleave the files after specifying two directories from the CF cards that have been copied to your computer.

Those Pricey But Excellent Sandisk Cards
 

Hey Gints:
I have a hodge - podge of those Sandisk cards myself. I have 2 x 32 GB Sandisk Extreme III's, 1 x 16 GB Sandisk Extreme IV, and one 8 GB Sandisk Extreme III. I shove them all into my XDR and I can even record at 280 Mbps I-Frame across all 4 cards. The only eccentricity is that the fastest card (16 GB Extreme IV @ 45 MB per sec) has to be in the first slot, or else XDR will automatically down set everything to 220 Mbps.

It's good some humor, and relaxed comments, have been introduced to this somewhat delicate (and thus, tense) thread :)

But back to the topic: after Alister's comments, I don't feel isolated any more in my assessment of things, so I will tell you this (I wonder how many of you guys will agree):

- the EX1 is indeed inherently noisy; in fact (contrary to the generally adopted belief) this inherent noise is much more dependent on the scene nature than the level of lighting - in fact, many scenes shoot with gain at -3dB suffer from the shimmering more than some low-light scenes, shoot at +3dB, or even higher gain!

- unfortunately, nanoFlash is not the holy grail for curing this; at low bitrates (and I-Fo especially) it can be a complete mess, while at the highest bitrates (and in L-GoP especially) it augments this noise. I believe it's fair to make all the current and potential users aware of that.

Also, while I am sceptical about the nanoFlash potential to record with bitrates significantly higher than it's capable of currently (it's just not been designed to do so), I'm very much for the idea of Convergent Design considering to use the Sony encoder chip in the "constant quality, automatically variable max bitrate" mode (if at all possible).

What do you say, Dan and Mike?

I'd also suggest removing the options for I-Fo below 180 Mbps from the menu of available recording formats.

Dear Friends,

We love to be responsive to customer suggestions and implement them if possible.

We may be able to implement VBR (Variable Bit-Rate). However, we have to be cautious, in that the files have to work in the various NLE's.

(It would be best if the VBR files work in every NLE. If not, then we would be creating a "trap" in which a user could select an option, but then his or her files would not work in their NLE.)

We are trying to keep our menu options limited, so that the nanoFlash is not complicated to use or intimidating to first time users.

(But, this is not an issue, if all NLE's support VBR files).

We could use some information.

For example, if we just change our files to VBR from CBR (Constant Bit Rate), would these files work in your NLE's?

Please let us know which type of files will work in VBR mode.

Dear Dan,

Some CBR formats must be left for sure (e.g. the L-GoP, XDCAM HD 50/422), to ensure the widest NLE compatibility with the well established formats (there are sure more of them).

As to other formats (especially I-Fo), my knowledge is not deep enough - let's wait for our gurus' comments.

Thanks anyway, for your instant response!

Piotr

VBR Encoding does not affect NLE usage
 

....Hi Dan: I don't understand your premise here. VBR only affects a file in the way it is encoded, but has nothing to do with the finally completed file being manipulated in any NLE.

...How would this complicate the menu system ?

...Dan, I think this needs to be additive and not one OR the other.

Dear Mark,

For Final Cut Pro, our files are identified as Sony XDCam 4:2:2 50 Mbps CBR, for our bit-rates 50 Mbps or higher.

I wonder if we encode the files as VBR and then indicate that they are VBR, whether the NLE's will recognize the files?

Or, whether if we encode in VBR, and specify that they are CBR, if the NLE's willl work with the files?

In response to your post: If we add another menu item, and there is a chance that it could be set wrong, then we are complicating things by adding the new menu item.

We will be discussing this.

Dan, I'm willing to test the VBR files for the Final Cut Pro, Sony Vegas 9 and provide comments with convenience software such as VLC Media Player, Windows Media Player, Western Digital TV and Sony PS3.

After stepping through my own and Piotr's footage, I'm biased against using 140+ Mbps LongGOP due to the quality drop from the I-frame, and the noise from that variation. There is a visible increase in quality almost every 6 frames when viewed on my Apple Cinema 23" display.

CD is creative enough to provide a default for Quality variation parameter. A single parameter could control both B-frame and P-frame "spoilage".

I'm not sure if you're implying that the Sony noise level is less at Gain=-3dB, but there are a number of posts on the Sony XDCAM group that -3dB does not provide a higher SNR - only a signal attenuated by 3 dB.

Personally, I always use 0 dB gain. That is what comes out of the hardware, and I do not feel like messing with it. At least not until post.

Minus 3 DB Gain On Canon XL H1
 

Hi Gints & Adam:
In my visual testing & experimentation, I have noticed a slight decrease in overall noise if I shoot exteriors at -3 db gain.

This should be up to the user. We already have the trap of shimmering MPEG at 140-180 Mbps. I want 140 or 180 Mbps LongGOP VBR exceed the quality of 220-280 Mbps I-Frame only. Besides, after reckoning with the Force of the Flash, NLE's will be scrambling to add new Nano/XDR file compatibility to their editing suites ASAP. The Nano is unleashing a slew of scalable recording options.

It wasn't too long ago (pre-2008) that nearly all major news networks would refuse downloading my DV (.avi) and HDV (.m2t) files and instead insisted on my writing the clips/concatenations to tape to be sent via Overnight Fedex. Some then lost 1/2 to a full day sending my tapes to a conversion house so that they can be written to yet another tape.

And since the user is most likely a pro, he would much rather have the "trappy" option available than be treated like someone who does not needs to be protected from himself.

I mean how is it different from having manual focus available? We know what we are doing, and when there is something we do not know, we are willing to learn it. If we were scared of such traps, we would be using $130 camcorders.

Dear Fiends,

My comment was not meant to be condescending.

My comment was directed to the fact that we want our nanoFlash to be a reliable recording device.

I hope everyone will realize that many crews do not own a nanoFlash, and they receive it the morning of the shoot, or the day before. Quite often they are not in a position to record files and test them in their NLE and workflow. Others, of course, have time to learn the nanoFlash.

Of course we do not want to "dumb down" the nanoFlash.

In an ideal world, we would just test VBR, and it would work, be better, and be accepted in every NLE without problems. We have not run these tests yet.

Another thing that we have considered is adding an "Advanced Settings" menu area, and a way to quickly set all "Advanced Settings" to the defaults.

We are also working on Profiles which will help in this area.

The current standard for Sony XDCam footage is 50 Mbps 4:2:2 CBR. This is what is accepted by the NLE's that support XDCAM footage.

CS5 officially supports other bit rates that we offer.

The Mainconcept Codec for Adobe CS3 and CS4 adds support for all of our modes and bit-rates and we greatly appreciate their efforts to support our products.

Final Cut Pro and others only suppport Sony XDCam footage at 50 Mbps 4:2:2 CBR.
Thus, we label our higher bit-rate footage, such as our 280 Mbps footage, as 50 Mbps so that FCP will recognize it.

Our engineers feel that we we cannot label VBR footage as CBR in the file header. They feel that the footage will not work if we do. We, at this moment, have not run this test.

In our next release, our MPG footage will be VBR. We just made that change.

That's what I mean, Gints - sorry if was unclear.

Sounds like a good idea. Or you could offer an optional VBR firmware update that is not recommended for rental units along with an explanation. That way a pro who wanted VBRs could choose to download the VBR enabled firmware and would be well aware of the consequences.

That would protect the naive and enable the wise.

Dear Mike,

Thank you for your suggestion.

I'll second that.
I suspect many here do not understand what it can be like for a rental company.

Port Over VBR to XDR Too ?
 

...Hi Dan: Could we also have that advanced menu and VBR *Option* ported to the XDR ?

Dear Mark,

Yes.

I need to clarify. We have three file types: MOV, MXF, and MPG.

We have enabled VBR for MPG only.

I doubt that you are using MPG, but maybe you are.