Nano 220Mb/s I-Frame question

[Paul Cronin]

Hi Dan,

Is the 220Mb/s I-Frame recorded on the Nano the same codec as the ProRes HQ?

[Piotr Wozniacki]

Ha!

I've been waiting for somebody asking this question at last. Since the very beginning of the nanoFlash existence, we've been touted about the virtues and advantages of the Sony Long-GOP codec - but I don't remember mentioning any details about the I-Frame "innards"...

[Paul Cronin]

Piotr I am sure Dan will have the answer.

[Dan Keaton]

Dear Paul,

No, Apple ProRes HQ, while it is also 220 Mbps, is different.

First of All, Apple ProRes is 10 bit.

The nanoFlash is 8-Bit as all MPEG-2 is 8-Bit and all XDCam HD422 is also 8-Bit as is EXCam Ex.

While 10-Bit is desirable, there is a price to pay for 10-Bit, it always takes up more space.

Note: in the following discussions, Apple ProRes is 10-Bit and the nanoFlash is 8-bit.
If one is using a good clean 10-bit source, then the 10-Bit recordings are desirable.
If not, then the 10-bit recordings cause the files to be larger with no apparent benefit.



Due to Apple ProRes being 10-bit, it would have to be 275 Mbps to equal the efficiency of 220 Mbps 8-Bit (with everything else being equal).

Or to put it differently, the nanoFlash, being 8-bit, at 176 Mbps would yield approximately the same efficiency as 10-Bit Apple ProRes at 220 Mbps (with everything else being equal).



Our files can be either Quicktime or MXF.

Thus, we write out Sony XDCam HD422 files (if 50 Mbps or higher) in a Quicktime wrapper.
Under 50 Mbps, we use Sony XDCam EX codec.

Our files can be dragged and dropped into a Sony XDCam HD422 timeline without any rendering.

Or, our files can be dragged and dropped into a Apple ProRes timeline.


So Paul, the answer is no, our files are not the same as Apple ProRes HQ.

And of course, we offer 280 Mbps, which would require the 10-Bit Apple ProRes to be 350 Mbps to be equivalent (with everything else being equal).

[Dan Keaton]

Dear Piotr,

Our Long-GOP is Sony's sixth or seventh generation of the MPEG-2 codec.

It is a very sophisticated, professional codec. And it is efficient.

This Long-GOP consists of I-Frames, B-Frames and P-Frames.
It has quite a few advantages.

Our "I-Frame Only" mode, has all I-Frames.

There are multiple advantages to this.

1. Some people prefer to edit in an IntraFrame Codec, and this is an IntraFrame codec.

2. It requires a lot of sophistication, and computing horsepower to implement a Long-GOP codec.

We limited Long-GOP to 180 Mbps, after than we use I-Frame Only, which requires much less computing horsepower.

This allows us to take the I-Frame Only to 280 Mbps.

We use Long-GOP, in HD, from 18 to 180 Mbps and I-Frame Only from 100 to 280 Mbps.

[Piotr Wozniacki]

Dear Dan,

The CD Long-GOP codec is certainly very efficient - no doubts about it. At 100 Mbps, the I-Frame only is unwatchable - while for the Long-GoP, this is the "sweet spot" (with low-noise cameras; if the camera tends to generate noisy picture, I'd say the true sweet spot is at 50 Mbps: less noise, while already of Broadcast Quality).

But, as Rafael points out in another thread:

Originally Posted by Rafael Amador

So lets go for the few little things that still can be improved on our beautiful NANOs, and lets give a try to that GOPs structure :-)

- I'd also like to very politely remind you that the L-GOP structure could be improved in terms of obtaining more consistency in the I,B and P frames quality.

This is why I usually only use L-GOP at 50 Mbps; when I need that extra quality to start with for heavy grading, I'll always choose I-Frame Only. Whether you support it or not, it now works at up to ad including 280 Mbps on my Transcend 400x cards (with the current Public Beta firmware)...

[Paul Cronin]

Thanks Dan I thought that was the case. Appreciate you letting me know your thoughts.

[Dan Keaton]

Dear Piotr,

There is one thing that I have not pointed out before.

With our Long-GOP, when there is an in-balance due to the amount of bandwidth allocated to the B & P frames, the codec module can detect this and make automatic adjustments.

But, if you evaluate the images on a frame by frame basis, you may see some frames before the codec senses the problem and makes these automatic adjustments.

In cases of high noise images the I, B, and P Frames all become about the same size over time.
In normal video, I Frames are about 2X size of P Frames and 3-4X size of B Frames.

These types of problems should only occur on cases of excessive noise in the images or very high motion.

But then, the nanoFlash does a very good job with excessive detail in an image, or excessive motion.

Purposely sending excessive noise to the nanoFlash for testing purposes will only show abnormal conditions.

Purposely sending excessive detail or excessive motion in the images is a very valid test, in my opinion.
But sending excessive noise is not a valid test, in my opinion.

With excessive noise throughout the image, one is just overwhelming the codec.

A full uncompressed recorder would keep all of this noise, and then playback an image with excessive noise.

A low-bandwidth recorder will remove some of this noise, but low-bandwidth recorders also cannot distinguish between noise and detail, so the fine details and the noise are both reduced.



The CODEC inserts numerous I Frame type macroblocks whenever parts of the image are new to the stream (like in high noise environments).

To balance the bitrate, the I Frames must get smaller.

In effect, in very high noise environments, the video is recorded as if it were I frame only, thus negating the advantages of the Long-GOP efficiencies.

Luckily, normal video hardly ever generates this situation.

My comments have been approved by our Chief Engineer.

[Piotr Wozniacki]

Dear Dan,

Thank you very much for this insightful explanation. I agree with most of its points but one:

- nobody (me neither) has been sending "excessive noise" to the nanoFlash in order to prove its "flaws" (or should I say "features"). All my observations, described in the "thread that refused to die", were based on real-life video from the EX1 camera, which - while not the quietest in the world - has a decent S/N ratio of 54dB.

The EX cameras are ENG-type machines, often used in less-than-perfect lighting conditions. Sadly, recording from them to the nanoFlash in the L-GOP format at bitrates higher than 100 Mbps, is denying the very purpose of the wonderful device that the nanoFlash truly is.

But as I said earlier - L-GOP at 50 Mbps for ENG style shooting, and I-Frame only at 220 Mbs+ for controlled environment recording, is the recipe that has worked fine for me for a long time now.

[Dan Keaton]

Dear Piotr,

I will word it differently.

To obtain the best results out of the nanoFlash, optimize your camera to produce the cleanest images before sending them to the nanoFlash.

[Rafael Amador]

Just to add that the only similarity are on the DTC compression side of both codecs.
Prores is a QT codec proprietary of Apple (10/12b/YUB/RGB/422/444-4) with presets data-rates/qualities, and always Intraframes. Is mostly a production codec, now with Alpha available.

MPEG-2 is quite different. Although we work with Intraframe MPEG-2, in fact still being GOPs.The trick is to use GOPs just one frame long.
MPEG-2 can be 420/422, but do not supports RGB color space, or 10b, neither 444 patterns (to fix those shortcomings has been developed the MP4 standard. Well, no RGB on MP4). MPEG-2 can not support Alpha.
rafael

[Piotr Wozniacki]

Originally Posted by Dan Keaton

In effect, in very high noise environments, the video is recorded as if it were I frame only, thus negating the advantages of the Long-GOP efficiencies.

Dear Dan,

Please don't think I'm beginning to split hair again - my background being engineering, I'm just curious. Also, your in-depth explanation has been backed up by CD's Chief Engineer - please let me try and take this opportunity to get some more understanding of the subject :)

When I re-read you explanation, the above statement caught my attention. I must say I either do not understand it properly, or - if my understanding is correct - I do not agree with it.

We all know that - both in theory and in practice - the higher the bitrate, the better the quality of the nanoFlash I-Frame only mode. The same cannot be said about Long-GOP, which tends to exaggerate noise at bitrates higher than 100 Mbps. But that means that if - in presence of noise from the camera, and recording in L-GOP format - the nanoFlash video "was recorded as if it were I frame only", its quality should increase with bitrate. This unfortunately is not the case...

Also the second part of the above statement of yours calls for some follow-up. The "L-GOP efficiency" can be simplistically defined as the PQ to file size ratio. The mechanism of increasing the proportion of I frames content against that of B and P frames that you attribute to excessive noise content does NOT force higher bitrate of the Long-GOP recoding, as all L-GOP modes from 50 Mbps up are CBR. Therefore, one cannot say the efficiency is suffering, as file sizes remain the same. Unless. one admits that the PQ is getting lower, which has always been my point!

All in all, I'm afraid Dan that your own statement is - in fact - making my point that to keep the I-P-B frames proportion constant, the "Constant Quality" mode (called for in the other thread) should be adopted. This would increase the bitrate when necessary. Yes, file sizes would become greater - but the quality could benefit.

So, to summarize: I-Frame Only mode should always be used when PQ has priority over storage needs/costs (the higher PQ, the greater file sizes). On the other hand, CBR L-GOP quality is obviously better at lower bitrates/file sizes....With VBR, we could obtain higher L-GOP quality at the cost of somewhat greater file sizes (of course, with less detail/movement/noise, the file sizes could shrink again).

Please let us know what you and the Chief Engineer think.

Piotr

[Peter Moretti]

Piotr,

While what you are saying may be correct in terms of keeping the proportion of I, P and B frames constant, I think you really have to ask yourself if this a worthy endeavor for CD to undertake.

1) There may be realities of the Sony chip that can't be avoided, making such an undertaking not possible.

2) If possible, it still might wind up creating a Long-GOP structure that NLE's can't work with. I know Avid Media Composer only recently deals with Long-GOP 100, anything over 50 would cause a buffer overrun. Will a larger size Long-GOP work in Vegas, Avid, FCP, etc.? IDK. But this is a very important point. So even if CD gets things to work on their side, the NLE's have to adopt the codec on their side. Will you have to wait for that to happen? How long? (It took Avid years to adopt Long-GOP 100). In the meantime storage will be getting cheaper and faster. And each NLE maker has to decide if it's worth their time supporting yet another codec variation.

3) Let's for argument's sake even say that this new Long-GOP codec will work with NLE's right now. This new choice will be something like Long-GOP 180 and probably be visually equal to I-Frame only 220. This is a somewhat minor savings in storage size. Is this savings worth it as CF cards get cheaper and faster by the month?

So you may be correct about using a constant frame proportion. But even when stipulating that you are, I don't see how this really justifies CD's time. I don't mean to be contrary, and I find your posts very valuable. But I think this suggestion would be a lot of work to create an in-between storage size that NLE's might not even be able to deal with.

JMHO, and again I appreciate all your posts.

[Piotr Wozniacki]

Hi Peter.

All I wrote/speculated about has been out of my technical curiosity, as stated at the beginning of the post.

Ever since Vegas Pro (starting with v.10) has been capable of playing all nanoFlash formats (including 280 Mbps I-Fo) full quality / full speed, I don't have any problem with the fact that with my particular camera, the L-GoP is only usable up to 100 Mbps (frankly, I'm only using 50/422 - just to stay at the Broadcast Quality side). When I need more headroom for pushing and pulling in CC/grading, I record I-Fo at 220 Mbps (and recently, sometimes even 280)...

So, my Friend, all is well - my questions and suggestions are purely academic.

:)

Piotr

PS. One thing in your otherwise perfectly sound reasoning I don't quite agree with, though: yes, with the method I proposed the L-GoP effective bitrate would go up considerably - but only in the case of lots of detail/noise/movement. With clean and static scenes, it could even drop down to 50 from the nominal of say 140 Mbps - thus, the average storage space required could as well remain unchanged :)

[Rafael Amador]

Dear Dan and Piotr,
I don't want to get on another technical discussion because I haven't done yet proper tests to compare quality issues, but I think that, if after proper test, I would reach the conclusion that the "NANO Sweet Spot" is 50Mbps, THIS WOULD BE A HUGE DECEPTION for my self.

I bought the NANO for:
- LGOP-shooting options (for general shooting) better than the SONY HD 422 50 Mbps (the NANO pictures at 100 Mbps MUST look better than at 50 Mbps and at 140 Mbps MUST look better than at 100 Mbps).
- High quality Intraframe options, for some specific situations (Intraframe pictures at high data-rate, should be virtually 8b Uncompressed).


I understand the effects of noise on the GOPs efficiency (In effect, in very high noise environments, the video is recorded as if it were I frame only, thus negating the advantages of the Long-GOP efficiencies).
BUT I understand that this effect MUST be palliated by increasing the data-rate.

In fact this is the only reason of having scalable data-rate: To counteract that "efficiency reduction" on picture with high noise, high detail and fast motion.

So, if there is not an obvious improvement at 100Mbps over 50 Mbps: There is something wrong.
This whatever the signal comes from (noisy EX-1 included).

Addressing high details and fast moving pictures have no much science: Add MORE data-rate.
But If Noise is our real life issue, is the one that needs to be addressed.
I understand that noise is the most stressing issue for a codec, therefore when managing NOISE, is when a good codec should shine.


Dan wrote:
"Our Long-GOP is Sony's sixth or seventh generation of the MPEG-2 code"

Right, but SONY seems to be very conservative and haven't pushed further than the 50Mbps.
The NANO is the first real opportunity to try to get the most of the codec.

I understand that CD may have some legal limits about what he can try to do or implement on the Processor/Codec.
I understand, that in CD you have been really busy fixing other issues.
What I would like to know in short, is if there is any room for improvement.
if the LGOPs structure can be optimized (if there is any thing to be optimized), or if CD considers that is closed and there is nothing else to do.

Piotr,
I propose you a proper test.
First we have to compose a test signal including video with some motion graphics generated by AE or so.
The point would be to feed the NANO from an AJA, BlackMagic or so, instead of from a camera.
Then will be easy compare and see how data rate-affect the NANO files.
Cheers,
rafael