Noise comparison: 35/4:2:0 vs. 180/4:2:2

[Lance Librandi]

Sorry guys I did not intend to send this thread into a different direction.
DAN
For your info the NLE was M100 iFinish which came with the real time Sony MPEG encoder SD the menu option allowed the use of 15 to 50 Mb I-frame 4:2:2 in CBR. It also allowed the selection of MPEG-1, MPEG 2 4.2.0 or MPEG 4.2.2. This system was only used to create archival copies of finished projects and not used as an editing medium. I used the VBR option for storage allowing control of file size vs quality. I now use the NanoFlash to store my finished projects.

[Dan Keaton]

Dear Lance,

Thanks for the extra information.

[Rafael Amador]

Originally Posted by Mark Job

Hi Rafael: Yes I understood this was so, all I was wondering if there is room in the Sony XDCAM HD specification for the utilization of a VBR encoding scheme. Does anyone know or have a link to any possible web page which would display the full Sony XDCAM HD 4:2:2 spec so we could check ?

Hi Mark,

To cheat the NLEs shouldn't be a problem. In fact the NANO files are off-standard (data rate, I-frames,..) but they work in most of the applications. At least FC can manage ot of stuff that in theory is not supported.
rafael

[Piotr Wozniacki]

I agree that "cheating" NLE's should not be a problem. After all, all we need is the highest possible quality - and NOT a strict compliance with any particular "industry standard"...

Why?

Well - simply because it's our delivery formats that ultimately counts; this must be absolutely compliant with some industry standard (be it HDTV broadcast, BluRay disk, ....).

So, does it make sense to bother Convergent Design an push them to upgrade their encoding schemes from the "constant bitrate" to the "constant quality"?

Well, I think no, it does not (strange does it sound from somebody who initiated this thread, doesn't it).

I'll explain: given that all cameras will inevitably feed the nanoFlash with some noise (XDCAM EX in particular, as this is their Ahilles heel when compared to other great specs) , the only solution seems to be getting rid of it in post.

I'm editing in Vegas Pro, and I've just purchased the Neat Video Pro plug-in. I have run it on both the SxS clips, and the simultaneously recorded, 220 Mbps I-Fo nanoFlash clips...

And I can tell you the results from the nanoFlash clips (not just I-Fo 220, but also L-GoP 100) are much, muuuuch better than those from the native XDCAM EX. The obvious reason being that with much less compression artifacts, the ingenious Neat Video filter can really show all its power on what it's supposed to deal with: the noise.

Add to it that Neat Video can also sharpen your video at the same rendering run, and you can shoot with ALL detail enhancement OFF - thus further relieving the nanoFlash codec from dealing with detail enhancement, and allowing it to encode even more efficiently. The motion, as well as static high frequency details, can be better handled by the encoder - resulting with even less compression artifacts.

Do you hate noise, that looks augmented along with detail in the nanoFlash files? I do. Do you want your video sharper? Well, I do too.

The answer for the time being is:

- record in L-GoP, at 100 Mbps max, or
- record in I-Fo, at 220 Mbps min, and

- render your material with the Neat Video (or any other, equally capable, tool)!

Well, of course Convergent Design may come with a better solution any time now, but frankly, I'd rather they spent their man-hours on finally implementing CF card hot-swapping, SDI embedded + analog audio mixing, and alike...

Just my opinion, YMMV.

Piotr

[Dean Harrington]

I've been using this software and it does very good noise reduction. I use it on everything out of the EX3.

[Luben Izov]

Originally Posted by Piotr Wozniacki

Guys,

Thank you very much for your advice. Of course I hope you realize I'm talking subtleties here - just like with the (dark area) noise, the shimmering fine detail ("translucent" grain) can only be seen on a fairly large display, like my 50" plasma.

Kris,

Could you please send me a PM? I tried emailing you, but for some reason I can't. It so happens I do live in the SE of Poland, so I'd be more than happy to meet and share experience.

Hello Piotr, I would love to know what is the outcome of the NF trade/test with Kris? You never come back to clear up the suspicion of faulty NF or camera ensue?
Cheers

[Piotr Wozniacki]

Hi Luben,

My meeting with Kris is planned as soon as he comes to Poland (he's in UK now, and supposed to give me a phone call when he arrives here). Of course I will update this thread accordingly - but frankly, I'm 99.99% positive both my EX1 and my nanoFlash are working up to specs.

It's just that - unlike most "real pros" here who assess their material using super expensive production monitors with HD-SDI input - I always base my opinions on the results I'm getting with my audience in mind, and this means a really big display. Not necessarily with professional inputs (just HDMI), but of 50" and up in size - while most persons trying to convince me there is no noise problem are using something like a 20" incher... What they don't realize is that it's not the type and quality of the input, but the very size of a pixel which decides whether or or not some artifacts will be noticeable or not.

I have almost come to terms with it, and am right now working a new workflow involving obligatory use of the Neat Video filter in Vegas. The additional benefit is that it can de-noise and sharpen at a single go, thus making it the obvious choice to shoot with in-camera sharpening (detail) off. This further enhances results by making the nanoFlash encoder less burden (no sharpening artifacts), while the noise can be removed in post and the picture sharpened to taste.

It's a pity though that all those ideas myself and other people presented in this thread (like fine tuning the Long GoP structure, or even introducing VBR) have not even been commented or considered seriously by CD...

[Mark Job]

Hi Piotr & Luben:
I can now confirm that VBR encoding *is* part of the *Official Sony XDCAM HD 4:2:2 SPEC ! I have a copy of the spec straight from Sony entitled,

"Codec Technology for XDCAM Tapeless Products and Systems
By Hugo Gaggioni, Chief Technology Officer, Sony Broadcast & Production Systems Division"

On Page 5 it States:

"3. Adoption of VBR
For HQ and LP mode, a Variable Bit Rate compression method was adopted in order to take advantage of the random access nature of tapeless media. It is different from the 2 pass* VBR compression technique used in DVD authoring. This codec was specifically developed for MPEG to perform high-speed efficient picture coding.
The XDCAM product line is based on the MXF file format and MPEG-2 Long GOP for exchanging and transmitting Video, Audio and Metadata. In the VBR (Variable Bit Rate) encoding method adopted for XDCAM HD products, the maximum bit rate that can be processed in real-time is fixed. This results in the high level of picture quality exhibited by XDCAM HD products over different picture material while minimizing the file size and maximizing transfer speed."

Apparently, VBR encoding was introduced by Sony as soon as the XDCAM *HD* version of cameras came out.

[Dan Keaton]

Dear Piotr,

We have to maintain compatibilty with a wide range of Non-Linear Editors.

Using a non-standard number of frames in a Group of Pictures would require substantial amount of testing to ensure compatibility.

We use a "Closed GOP". Switching to an "Open GOP" has ramifications when editing. With Closed GOP, the I-Frame that starts each group does not depend on or need information from the preceeding GOP.

Dear Mark,

There are three flavors of XDCam:

XDCam HD
XDCam EX
XDCam

We support both XDCam EX (35 Mbps, 4:2:0 35 Mbps VBR) and XDCam (50 Mbps, 4:2:2, CBR).

XDCam HD is different.

XDCAM - Wikipedia, the free encyclopedia

[Piotr Wozniacki]

Any chances of supporting XDCAM HD format - at least with the higher bit rates, Dan?

[Dan Keaton]

Dear Piotr,

I have not been able to determine if XDCam HD is 1440 x 1080. The PDW-350 is a 1440 x 1080 camera and it is a XDCam HD camera.

We want to continue to record in 1920 x 1080, thus we are using XDCam format and not XDCam HD format.

[Piotr Wozniacki]

Dear Dan,

I'm a bit confused with what you're saying. Even in the Wikipedia article you linked to, it reads:
"In August 2009, Convergent Design began shipping the nanoFlash Portable Recorder, which uses the Sony XDCAM HD422 codec at bit rates ranging from 18 Mbit/ second to 35 Mbit / second (in the 4:2:0 colorspace), and bit rates ranging from 50 Mbit/ second to 280 Mbit/ second (in the 4:2:2 colorspace).

So, why are you saying nanoFlash is only capable of XDCAM EX and XDCAM, but not XDCAM HD?

As to your other point: of course I'd be for the full raster - just VBR option for the high bitrates. From what Mark quoted, the Sony chip you're using is capable of it...

Thanks,

Piotr

[Mark Job]

Hi Dan:

Uhh, I understood the Nano Flash & Flash XDR are using the Sony XDCAM HD 4:2:2 codec, therefore, VBR is also possible. (??) So what codec are you using ? XDCAM or XDCAM HD ? XDCAM is an SD only codec according to the Sony White Paper on the spec.

[Piotr Wozniacki]

Dear Dan,

Could you please explain your confusing post? I'm sure it's not just myself or Mark who are waiting...

Thanks,

Piotr

[Dan Keaton]

Dear Friends,

As I understand it, there are three flavors of Sony XDCam:

XDCam HD, 1440 x 1080 4:2:0
XDCam, 1920 x 1080 4:2:2
XDCam EX, 1920 x 1080 4:2:0

Here is an excerpt from an article on Wikipedia:

XDCAM is a tapeless professional video system introduced by Sony in 2003.

The first two generations, XDCAM and XDCAM HD, use the Professional Disc as recording media. This disc is similar to Blu-ray Disc and holds either 23 GB of data (PFD23, single-sided) or 50 GB (PFD50, double-sided).

The third generation, XDCAM EX, uses solid-state SxS cards instead.

XDCAM - Wikipedia, the free encyclopedia

--------------------------------------------------------------------------------------------------

The nanoFlash uses XDCam, 1920 x 1080, 4:2:2 at all bit-rates 50 Mbps and higher.

And XDCam EX, 1920 x 1080, 4:2:0 at bit-rates of 18 Mbps and 35 Mbps.

And we have an option to record in 1440 x 1080, 4:2:0 at 35 Mbps.

--------------------------------------------------------------------------------------------------

1. The Sony PDW-F330, PDW-F350, PDW-F355 are "XDCam HD".

Here is some text from an offical Sony site discussing the Sony PDW-F330:

HD 1080 Recording at Selectable Frame & Bit Rates
XDCAM HD products can record video signals in 1080/59.94i, 50i, 29.97P, 25P and native 23.98P - using a "MPEG HD" codec with industry standard MPEG-2 MP@HL compression.

The user can also select the following bit rates; 35, 25 or 18Mb/s depending on picture quality and recording length requirements. Choosing the highest bit rate of 35Mb/s results in a recording time of approximately 60 minutes, while choosing 18Mb/s provides a recording time of approximately 120 minutes - the longest recording time offered by any current HD camcorder.

MPEG HD images are recorded at 1440x1080 pixels with 4:2:0 sampling.

Sony : PDW-F330K (PDWF330K) : Features : United Kingdom


--------------------------------------------------------------------------------------------------

2. The Sony PDW-700 and PDW-F800 are "XDCam".

The Sony PDW-700 is titled:

"PDW700 Sony Professional XDCAM®
HD Camcorder"

The Sony PDW-F800 is titled:

"PDWF800 Sony Professional XDCAM®
HD422 Camcorder".

While the title for the Sony PDW-700 camera includes "HD" on the next line, it is in the same class as the PDW-F800 camera which uses "HD422" on the next line. I interpret these to indicate that these two cameras are "XDCam" cameras as opposed to the "XDCam HD" cameras that are 1440 x 1080.

The following is from an offical Sony website for the PDW-700:

MPEG HD422 (MPEG-2 422P@HL) (CBR: 50 Mb/s)
MPEG HD (MPEG-2 MP@HL):
HQ mode (VBR, maximum bit rate: 35 Mb/s)
SP mode (CBR, 25 Mb/s)
LP mode (VBR, maximum bit rate: 18 Mb/s) (Playback only)
MPEG IMX (MPEG-2 422P@ML) (50/40/30 Mb/s)
DVCAM (CBR,25 Mb/s)

Sony Product Detail Page - PDW700


The follow is from an official Sony website for the PDW-F800

MPEG HD422 (MPEG-2 422P@HL) (CBR: 50 Mb/s)
MPEG HD (MPEG-2 MP@HL):
HQ mode (VBR, maximum bit rate: 35 Mb/s)
SP mode (CBR, 25 Mb/s)
LP mode (VBR, maximum bit rate: 18 Mb/s) (Playback only)
MPEG IMX (MPEG-2 422P@ML) (50/40/30 Mb/s)
DVCAM (CBR,25 Mb/s)

Sony Product Detail Page - PDWF800

--------------------------------------

3. The Sony PMW-EX1, PMW-EX1R, PMW-EX3, PMW-320 and PMW-350 use "XDCam EX".

The following is from an offical Sony site, and describes the Sony EX1R:

Video:
MPEG-2 Long GOP HD HQ mode: VBR, maximum bit rate: 35 Mb/s,
MPEG-2 MP@HL HD SP mode: CBR, 25 Mb/s,
MPEG-2 MP@H14 SD mode: DVCAM
Audio: Linear PCM (2ch, 16-bit, 48-kHz)

Sony Product Detail Page - PMWEX1R