Noise comparison: 35/4:2:0 vs. 180/4:2:2

Lance Librandi · August 13th, 2010, 06:33 PM

Hello Dan,
I agree with Mark VBR and CBR selection surely has to be the next feature that is enabled on the NanoFlash/XDR. I have used other Sony MPEG encoder boards in the past which have offered the selection of VBR or CBR and I found that I used the VBR option all time tailoring file size vs quality.

Rafael Amador · August 13th, 2010, 08:12 PM

The only point of VBR is to optimize the files size. If you are not constrained about file size, VBR adds nothing. Aside of this, a proper VBR compression needs a multi-pass process.
Better than VBR would be to implement Open or Short GOPs (I don't know if is possible with this processor and in a RT process).
rafael
PS: If people is having problem choosing a proper data rate in CBR, wait for them to decide the Average and peak data rates for a VBR compression.

Dan Keaton · August 13th, 2010, 10:48 PM

Dear Lance,

When you create VBR files, what NLE's are you using to read the files?

Could you please provide more info?

Bit-Rate, Long-GOP or I-Frame Only, 4:2:0 or 4:2:2, and what problems have you had reading the files?

Piotr Wozniacki · August 14th, 2010, 08:32 AM

Quote:

Originally Posted by Rafael Amador

The only point of VBR is to optimize the files size. If you are not constrained about file size, VBR adds nothing. Aside of this, a proper VBR compression needs a multi-pass process.
Better than VBR would be to implement Open or Short GOPs (I don't know if is possible with this processor and in a RT process).
rafael
PS: If people is having problem choosing a proper data rate in CBR, wait for them to decide the Average and peak data rates for a VBR compression.

AFAIK, single-pass VBR also has its advantages (I'm not sure how much less efficient datarate "distribution" is in this case, but the file size can be decreased for sure). But true - the most efficient is multiple pass, whereby one pass is needed to analyze the video quality needs, and another one to encode those demanding parts of it (motion etc.) at a higher, and the remaining part - at a lower bitrate, just to minimize the output file size.

As to the potential improvement in L-GoP quality arising from using shorter and more sophisticated GoP structure, I was mentioning this many times earlier in this thread.

I wouldn't worry about the added complexity - if some settings do indeed bring much better PQ than others, people would learn very fast how to use them ...

Gints Klimanis · August 14th, 2010, 01:03 PM

My interest in VBR is control of image quality variation from frame to frame. The Sony XDCAM 140 Mbps has a problem with quality variation, and we hope that VBR will address this problem . Of course, this will also reduce file size if I'm able to use 140 Mbps VBR to vary between 120-180 Mbps and yield the quality of 280 Mbps I-Frame only.

Piotr Wozniacki · August 14th, 2010, 01:43 PM

Gints, I hear what you're saying. In fact, we have both noticed and analyzed this quality variation that causes the noise shimmer, making it more distracting. BUT, I am not so sure VBR can take care of it - please try to convince me, and than we could try together and ask CD to try and implement it :).

What I - with my current, limited knowledge - believe to be a better cure for the intra-frame inconsistency, is some fine-tuning of the GoP length and structure. There's a number of parameters controlling this - what I'd expect CD to do is at least let us know which of them are available for their code to be used on the Sony encoding chip.

Who knows - perhaps the two approaches can complement each other?

I'd say the strategy should be:

- leave the I-Fo as is, but limit it to some really high bitrates where it shines (220 and up)
- try to improve the L-GoP by either optimizing the GoP structure/length, or adding VBR encodig, or both.

What do you say, Dan?

Piotr Wozniacki · August 14th, 2010, 04:33 PM

Quote:

Originally Posted by Dan Keaton

In an ideal world, we would just test VBR, and it would work, be better, and be accepted in every NLE without problems. We have not run these tests yet.

Dear Dan,

Of course my knowledge is limited, but frankly I don't see reasons why NLE should not recognized the VBR L-GoP files if nanoFlash had an option to record in this format.

With Vegas, one can encode either a CBR or a VBR MPEG-2 file from any source in the timeline, and such a file is always recognized, read-in, and played back in Vegas. Just tried it with Edius - same story.

Mark Job · August 14th, 2010, 04:55 PM

Quote:

Originally Posted by Piotr Wozniacki

Dear Dan,

Of course my knowledge is limited, but frankly I don't see reasons why NLE should not recognized the VBR L-GoP files if nanoFlash had an option to record in this format.

With Vegas, one can encode either a CBR or a VBR MPEG-2 file from any source in the timeline, and such a file is always recognized, read-in, and played back in Vegas. Just tried it with Edius - same story.

Hi Piotr & Gints: Once a Sony XDCAM HD 4:2:2 MPEG 2 file is created, then the NLE should treat it as any other completed and ready to play XDCAM HD file. How the file is made makes no difference as long as the finished XDCAM HD file meets the editing specification of the NLE - In the case of Avid Media Composer, the known standard is Long GOP 50 Mbps or I-Frame at any data rate you please.

* As long as the file header information remains in a form the NLE was previously able to read, then, in theory, there shouldn't be any issues.

Gints Klimanis · August 14th, 2010, 05:31 PM

It would be good to find a manual on the Sony XDCAM encoder, and admittedly, I don't know what I'm searching for. I see a qpweighting parameter mentioned in the AVC encoder specs. I wonder if this is the parameter for setting the relative quality of the I, B and P frames which are often specified as IQSCALE, PQSCALE and BQSCALE in MPEG encoder parameter files :

http://www.sonic.com/images/products...parameters.jpg

Cees van Kempen · August 15th, 2010, 03:58 AM

Piotr,

Followed your thread only incidently, but was triggered by it in the first place because I also wonderd why at some circunstances noice was moving through my images like it was 'snowing'. Specially at lower light circumstances. I did not fully check the whole thread now, and discussion seems sometimes to move towards other directions. However, I have been playing with my gamma settings and noticed that this terrible noise out of the EX3 into the nano was very pronounced with gamma STD3 (which I used before), and is almost not there with CINE2. Maybe basic knowledge you know since long, but a revelation for me.

Piotr Wozniacki · August 15th, 2010, 05:24 AM

Cees, of course some EX's PP settings are better than others as far as noise is concerned. Usually however, no "best" setting exists - it's always a trade-off.

Let me stress it however that this thread is not about how noisy the EX cameras can be (some even say they are not that noisy at all, considering chip size and price tag). What we're trying to find out here is the best setting for the nanoFlash (or even modifying the way it encodes, if possible and viable to Convergent Design), so that the advantages of this marvellous device always outweigh the main disadvantage: the tendency to augment the noise, by exaggerating its shimmering nature.

Everybody participating in this thread realize perfectly that camera noise cannot be eliminated by the nanoFlash, as the device cannot tell noise from detail, and enhancing detail is one of its very purposes (along with minimizing compression artifacts). However, we've achieved some consensus on that the problem is aggravated by the L-GoP-inherent, frame quality fluctuation. Thus, minimizing this fluctuation has become the goal of our debate here :)

Thanks for your input, anyway. If in response to it, I have taken the liberty to summarize the essence of this (somehat convoluted) thread , isn't because I'm patronizing on you at all (please forgive me if it sounds that way). I simply wanted to help those new ones lurking into it, to understand what is being talked here...

:)

Piotr

Rafael Amador · August 15th, 2010, 09:44 AM

Quote:

Originally Posted by Mark Job

Hi Piotr & Gints: Once a Sony XDCAM HD 4:2:2 MPEG 2 file is created, then the NLE should treat it as any other completed and ready to play XDCAM HD file. How the file is made makes no difference as long as the finished XDCAM HD file meets the editing specification of the NLE

* As long as the file header information remains in a form the NLE was previously able to read, then, in theory, there shouldn't be any issues.

By definition XDCAM HD 422 is CBR.
rafael

Mark Job · August 15th, 2010, 11:55 AM

Quote:

Originally Posted by Rafael Amador

By definition XDCAM HD 422 is CBR.
rafael

Hi Rafael: Yes I understood this was so, all I was wondering if there is room in the Sony XDCAM HD specification for the utilization of a VBR encoding scheme. Does anyone know or have a link to any possible web page which would display the full Sony XDCAM HD 4:2:2 spec so we could check ?

Andrew Stone · August 15th, 2010, 03:14 PM

Sony's pro video british site has a lot of detailed docs relating to their pro gear. I would start there if you are looking for XDCAM specification related material.

Mark Job · August 15th, 2010, 04:36 PM

Hi Andrew:
OK. Will do :-)

August 13th, 2010, 10:48 PM	#288
Dan Keaton Inner Circle Join Date: Dec 2002 Location: Augusta Georgia Posts: 5,421	Dear Lance, When you create VBR files, what NLE's are you using to read the files? Could you please provide more info? Bit-Rate, Long-GOP or I-Frame Only, 4:2:0 or 4:2:2, and what problems have you had reading the files? __________________ Dan Keaton Augusta Georgia

August 14th, 2010, 01:43 PM	#291
Piotr Wozniacki Inner Circle Join Date: Aug 2006 Location: Poland Posts: 4,086	Gints, I hear what you're saying. In fact, we have both noticed and analyzed this quality variation that causes the noise shimmer, making it more distracting. BUT, I am not so sure VBR can take care of it - please try to convince me, and than we could try together and ask CD to try and implement it :). What I - with my current, limited knowledge - believe to be a better cure for the intra-frame inconsistency, is some fine-tuning of the GoP length and structure. There's a number of parameters controlling this - what I'd expect CD to do is at least let us know which of them are available for their code to be used on the Sony encoding chip. Who knows - perhaps the two approaches can complement each other? I'd say the strategy should be: - leave the I-Fo as is, but limit it to some really high bitrates where it shines (220 and up) - try to improve the L-GoP by either optimizing the GoP structure/length, or adding VBR encodig, or both. What do you say, Dan? __________________ Sony PXW-FS7 \| DaVinci Resolve Studio; Magix Vegas Pro; i7-5960X CPU; 64 GB RAM; 2x GTX 1080 8GB GPU; Decklink 4K Extreme 12G; 4x 3TB WD Black in RAID 0; 1TB M.2 NVMe cache drive

August 15th, 2010, 05:24 AM	#296
Piotr Wozniacki Inner Circle Join Date: Aug 2006 Location: Poland Posts: 4,086	Cees, of course some EX's PP settings are better than others as far as noise is concerned. Usually however, no "best" setting exists - it's always a trade-off. Let me stress it however that this thread is not about how noisy the EX cameras can be (some even say they are not that noisy at all, considering chip size and price tag). What we're trying to find out here is the best setting for the nanoFlash (or even modifying the way it encodes, if possible and viable to Convergent Design), so that the advantages of this marvellous device always outweigh the main disadvantage: the tendency to augment the noise, by exaggerating its shimmering nature. Everybody participating in this thread realize perfectly that camera noise cannot be eliminated by the nanoFlash, as the device cannot tell noise from detail, and enhancing detail is one of its very purposes (along with minimizing compression artifacts). However, we've achieved some consensus on that the problem is aggravated by the L-GoP-inherent, frame quality fluctuation. Thus, minimizing this fluctuation has become the goal of our debate here :) Thanks for your input, anyway. If in response to it, I have taken the liberty to summarize the essence of this (somehat convoluted) thread , isn't because I'm patronizing on you at all (please forgive me if it sounds that way). I simply wanted to help those new ones lurking into it, to understand what is being talked here... :) Piotr __________________ Sony PXW-FS7 \| DaVinci Resolve Studio; Magix Vegas Pro; i7-5960X CPU; 64 GB RAM; 2x GTX 1080 8GB GPU; Decklink 4K Extreme 12G; 4x 3TB WD Black in RAID 0; 1TB M.2 NVMe cache drive

August 15th, 2010, 04:36 PM	#300
Mark Job Trustee Join Date: Sep 2006 Location: Montreal, Quebec, Canada Posts: 1,138	Thanks Hi Andrew: OK. Will do :-)

August 13th, 2010, 06:33 PM	#286
Lance Librandi Major Player Join Date: Aug 2009 Location: South Australia Posts: 374	Hello Dan, I agree with Mark VBR and CBR selection surely has to be the next feature that is enabled on the NanoFlash/XDR. I have used other Sony MPEG encoder boards in the past which have offered the selection of VBR or CBR and I found that I used the VBR option all time tailoring file size vs quality.

August 13th, 2010, 08:12 PM	#287
Rafael Amador Major Player Join Date: Jun 2009 Location: Vientiane (Lao PDR) Posts: 349	The only point of VBR is to optimize the files size. If you are not constrained about file size, VBR adds nothing. Aside of this, a proper VBR compression needs a multi-pass process. Better than VBR would be to implement Open or Short GOPs (I don't know if is possible with this processor and in a RT process). rafael PS: If people is having problem choosing a proper data rate in CBR, wait for them to decide the Average and peak data rates for a VBR compression.

August 14th, 2010, 01:03 PM	#290
Gints Klimanis Inner Circle Join Date: Jun 2003 Location: San Jose, CA Posts: 2,222	My interest in VBR is control of image quality variation from frame to frame. The Sony XDCAM 140 Mbps has a problem with quality variation, and we hope that VBR will address this problem . Of course, this will also reduce file size if I'm able to use 140 Mbps VBR to vary between 120-180 Mbps and yield the quality of 280 Mbps I-Frame only.

August 14th, 2010, 05:31 PM	#294
Gints Klimanis Inner Circle Join Date: Jun 2003 Location: San Jose, CA Posts: 2,222	It would be good to find a manual on the Sony XDCAM encoder, and admittedly, I don't know what I'm searching for. I see a qpweighting parameter mentioned in the AVC encoder specs. I wonder if this is the parameter for setting the relative quality of the I, B and P frames which are often specified as IQSCALE, PQSCALE and BQSCALE in MPEG encoder parameter files : http://www.sonic.com/images/products...parameters.jpg

August 15th, 2010, 03:58 AM	#295
Cees van Kempen Major Player Join Date: Jan 2010 Location: Nieuw-Vossemeer, The Netherlands Posts: 455	Piotr, Followed your thread only incidently, but was triggered by it in the first place because I also wonderd why at some circunstances noice was moving through my images like it was 'snowing'. Specially at lower light circumstances. I did not fully check the whole thread now, and discussion seems sometimes to move towards other directions. However, I have been playing with my gamma settings and noticed that this terrible noise out of the EX3 into the nano was very pronounced with gamma STD3 (which I used before), and is almost not there with CINE2. Maybe basic knowledge you know since long, but a revelation for me.

August 15th, 2010, 03:14 PM	#299
Andrew Stone Major Player Join Date: Sep 2008 Location: Vancouver, Canada Posts: 975	Sony's pro video british site has a lot of detailed docs relating to their pro gear. I would start there if you are looking for XDCAM specification related material.