Format Wars - Intraframe vs Long-GOP at DVinfo.net

Dan Keaton · October 31st, 2009, 09:47 AM

Dear Friends,

For years there has been a "Format War" between major camera manufacturers.

Camera manufacturers typically choose one side of the other: Intraframe or Long-GOP.

With all of the best intentions, and with their best engineers and designers, each camera manufacturer usually chose one camp or the other, and then stayed with it for many years. There are exceptions.

We try to remain fairly neutral in this "Format War".

This is very easy to do since we support both.

We do have our opinions, even strong opinions, but we feel that it is best to offer both choices, so that our users can choose whatever is best for their situation.

And, what is best for one shoot may not be best for the next one.

We have found that a lot of misinformation has been spread, and a lot of misconceptions still exist.

1. Some feel that when using Long-GOP, the only real, true frames are the "I-Frames" inside of the group of pictures.

For example, I was told that if a fast moving object did not appear in the initial I-Frame, inside of a group of pictures, that the object would not appear in the video.

The example was given that in a hockey game, a fast moving hockey puck could not be seen in Long-GOP.

I was able to quickly produce a video, demonstrating that Long-GOP handles fast motion, and the person was completely satisfied.

2. Another misconception is that one cannot edit a Long-GOP effectively since one can not cut unless on a Long-GOP boundary.

This is also false. The Long-GOP is fully decoded by the NLE into full frames, and one can cut on any frame desired.

3. Another strongly held belief is that Long-GOP is harder to edit, or editing system will be less responsive, a belief from the HDV days.

We are in a fairly unique situation in that our Long-GOP is always "Full Raster". For 1080, this means 1920 x 1080, even if a camera's sensor is 1440 x 1080.

This means that the NLE does not have to do any "heavy lifting" to work with our footage, in that the footage does not have to be converted to 1920 x 1080 from 1440 x 1080.

Since we support both Long-GOP and Intraframe equally well, I am just posting this to help clear up the misconceptions.

If one prefers to shoot in I-Frame Only (our version of Intraframe), then we fully support their choice.

The same applies to Long-GOP.

The selection of Long-GOP or I-Frame Only is very easy, it is just a menu selection in our Flash XDR or nanoFlash.

It does not mean that one has to switch camera vendors and purchase new cameras.

If one has an Intraframe camera and wants to shoot Long-GOP, our nanoFlash fully supports this choice.

If one wants to shoot Intraframe from a Long-GOP camera, this is also fully supported.

Mark Job · October 31st, 2009, 10:08 AM

Hi Dan:
There is one aspect to this discourse which has thrown me for a loop mentally, and I'm hoping you can clarify this point for me Dan. My question lies in an earlier explanation you gave on this forum, whereby you stated that CD writes its I-Frame MPEG video in a Long-GOP of 15 frames. (??) OK. This blows my mind ! Why do you do it this way and can we really conclude the Flash XDR and Nano Flash are writing true I-Frame video structure, if you are writing the I-Frames as Long GOP structure ? Isn't this just Long GOP by another name ???

Respectfully,

Dan Keaton · October 31st, 2009, 11:19 AM

Dear Mark,

We have always called our modes "Long-GOP" and "I-Frame Only".

Long-GOP starts with an I-Frame, and then has B and P Frames.

Our I-Frame Only is in a Long-GOP structure in which every frame is an I-Frame.
Every frame stands on its on for compression and decompression just as every Intraframe codec does.

This allows the editors that work with XDCam 50 Mbps 4:2:2 footage work with our "I-Frame Only".

Your Avid system should work well with our I-Frame Only footage, as should your Final Cut Pro system.

Mark Job · October 31st, 2009, 12:12 PM

Hi Dan:
So you're stating that it's not CD's version of I-Frame per se, rather, this has always been the way I-Frame MPEG is produced ?

Alister Chapman · October 31st, 2009, 01:12 PM

Mpeg 2 was originally developed as a long GoP codec. Long GoP codecs can have different combinations of I, B and P frames depending on their chosen use. In recent years it has become common practice to make use of I frames only and a simple way to do this is to have a GoP with nothing but I frames. The beauty of this is that you can use a single codec chip and matching codec software to work with both the traditional long GoP IBP type GoP's as well as the I frame GoP's. By doing this CD can make use of the XDCAM codecs installed in FCP, Avid etc to decode the I frame only codec without needing to write any additional software.

Dan, Mike.. correct me if I am wrong.

Daniel Symmes · October 31st, 2009, 01:24 PM

Recommended reading:

MPEG-2 - Wikipedia, the free encyclopedia

Dan Keaton · October 31st, 2009, 02:19 PM

Dear Alister,

I fully agree with your post.

Mark Job · October 31st, 2009, 03:06 PM

Hi Dan & Alister:
I thought there was a form of editable MPEG which did not use the Long GOP structure ?

Gints Klimanis · October 31st, 2009, 04:47 PM

Quote:

Originally Posted by Dan Keaton

2. Another misconception is that one cannot edit a Long-GOP effectively since one can not cut unless on a Long-GOP boundary.

This is also false. The Long-GOP is fully decoded by the NLE into full frames, and one can cut on any frame desired.

This misconception may actually be true regarding cutting and saving MPEG without recompression/rerendering.

Dan Keaton · October 31st, 2009, 04:57 PM

Dear Gints,

It is my opinion, if one is using any of the modern Non-Linear Editors, the ones that are typically discussed on this forum, if a cut is made in the middle of a Group of Pictures, a new Group of Pictures will be created starting with the first frame after the cut.

To be fair, I do not know the details of how this is done. For example, I do not know if the NLE creates a smaller than normal Group of Pictures. and if there is no other reason to recompress and render. Or it could recreate the Group of Pictures until it gets to the end of this segment (source video segment).

All of this depends on how sophisticated the NLE is.

I assume that the most basic of editors, the very low cost editors, may do it differently.

Mark Job · October 31st, 2009, 05:40 PM

Hi Dan:
In the latest iteration of Media Composer, Avid breaks up the entire Long GOP into a series of single frame MPEG's. I don't know how this is done, but I do know Matrox Digi-Suite was doing the same as early as 1997-98 is standard Definition. I know Avid has changed the way it handles MPEG once again, since version 3.5.x, because it is no longer necessary to make a video mixdown or transcode to output Long GOP based HDV projects. Now you can use a Quicktime Reference File to export to DVD or Web encoders, such as Sorensen Squeeze.

Rafael Amador · October 31st, 2009, 08:31 PM

Quote:

Originally Posted by Gints Klimanis

This misconception may actually be true regarding cutting and saving MPEG without recompression/rerendering.

NO.
Only the GOP that is cut is rebuild.
As Dan points, a smaller GOP is created.
GOPs don't necessarily have the same length.

The fully Intraframe structure of the NANO-I files can be tested with CinemaTool.
You can "Conform" NANO-Intra footage, while you can not conform NANO-LGOP footage.
A bit of reading would be very convenient.
Cheers,
Rafael

Mark Job · October 31st, 2009, 11:27 PM

Hi Rafael:
You wrote: "NO.
Only the GOP that is cut is rebuild.
As Dan points, a smaller GOP is created.
GOPs don't necessarily have the same length.

The fully Intraframe structure of the NANO-I files can be tested with CinemaTool.
You can "Conform" NANO-Intra footage, while you can not conform NANO-LGOP footage.
A bit of reading would be very convenient.
Cheers,
Rafael"

.......Huh ?

Alister Chapman · November 1st, 2009, 02:59 AM

Quote:

Originally Posted by Mark Job

Hi Dan:
In the latest iteration of Media Composer, Avid breaks up the entire Long GOP into a series of single frame MPEG's.

That's impossible as in the case of HDV when exporting to HDV tape or streaming over firewire to a deck the deck would not be able to decode the non-standard output. It also implies that everything is forced into a decompression-re-compression step with it's resultant loss of quality and I doubt Avid would force that upon it's users. Avid as far as I know still handles material natively, which is why AMA is able to work with footage straight of SxS cards or XDCAM discs without actually needing to import the footage. Matrox Digi-Suite used a pair of on-board hardware Mpeg codecs to decompress in real time for output over the SDi or Component connections. Any cuts or edits had to be rendered before you could export a Mpeg stream.
As Dan and others have said, only the GoP's immediately either side of any cuts or grades are processed. The way this is done seems to vary from NLE to NLE and may involve truncated GoPs. I have also heard of GoPs with frames of zero duration but never been able to confirm whether this is possible.

Mark Job · November 1st, 2009, 03:34 AM

Hi Alister:
You wrote: "That's impossible as in the case of HDV when exporting to HDV tape or streaming over firewire to a deck the deck would not be able to decode the non-standard output. It also implies that everything is forced into a decompression-re-compression step with it's resultant loss of quality and I doubt Avid would force that upon it's users. Avid as far as I know still handles material natively, which is why AMA is able to work with footage straight of SxS cards or XDCAM discs without actually needing to import the footage. Matrox Digi-Suite used a pair of on-board hardware Mpeg codecs to decompress in real time for output over the SDi or Component connections. Any cuts or edits had to be rendered before you could export a Mpeg stream.
As Dan and others have said, only the GoP's immediately either side of any cuts or grades are processed. The way this is done seems to vary from NLE to NLE and may involve truncated GoPs. I have also heard of GoPs with frames of zero duration but never been able to confirm whether this is possible."

.......No, I'm not 100 % positive, but the new Avid is now totally format and resolution independent and I'm pretty sure it strips *ALL* MPEG of its Long GOP structure *without* transcoding it. I know Avid no longer transcodes HDV or XDCAM HD projects because they say in their literature it is no longer necessary to perform video mix-downs of an HDV or XDCAM sequence or clip in order to export it back out of the NLE. Somehow the new Media Composer (4.0.2) has the ability to see each piece of video (Any video) as individual frames, then re-assenble the Long GOP on the way back out-all in *real time !* Now this kind of data processing is a real accomplishment in programming. Alister, it doesn't matter where the cuts are made, because Avid will re-assemble the Long GOP structure appropriately. In fact, it may not even be as difficult as we think for the NLE to break down the Long GOP structure on the fly and then re-assemble it on the fly. I know it's CPU and GPU intensive, because the PC and MAC hardware requirements are hefty and strict. Avid only guarantees its software to run on approved machines in PC and MAC. The approved list is very short.

October 31st, 2009, 09:47 AM	#1
Dan Keaton Inner Circle Join Date: Dec 2002 Location: Augusta Georgia Posts: 5,421	Format Wars - Intraframe vs Long-GOP Dear Friends, For years there has been a "Format War" between major camera manufacturers. Camera manufacturers typically choose one side of the other: Intraframe or Long-GOP. With all of the best intentions, and with their best engineers and designers, each camera manufacturer usually chose one camp or the other, and then stayed with it for many years. There are exceptions. We try to remain fairly neutral in this "Format War". This is very easy to do since we support both. We do have our opinions, even strong opinions, but we feel that it is best to offer both choices, so that our users can choose whatever is best for their situation. And, what is best for one shoot may not be best for the next one. We have found that a lot of misinformation has been spread, and a lot of misconceptions still exist. 1. Some feel that when using Long-GOP, the only real, true frames are the "I-Frames" inside of the group of pictures. For example, I was told that if a fast moving object did not appear in the initial I-Frame, inside of a group of pictures, that the object would not appear in the video. The example was given that in a hockey game, a fast moving hockey puck could not be seen in Long-GOP. I was able to quickly produce a video, demonstrating that Long-GOP handles fast motion, and the person was completely satisfied. 2. Another misconception is that one cannot edit a Long-GOP effectively since one can not cut unless on a Long-GOP boundary. This is also false. The Long-GOP is fully decoded by the NLE into full frames, and one can cut on any frame desired. 3. Another strongly held belief is that Long-GOP is harder to edit, or editing system will be less responsive, a belief from the HDV days. We are in a fairly unique situation in that our Long-GOP is always "Full Raster". For 1080, this means 1920 x 1080, even if a camera's sensor is 1440 x 1080. This means that the NLE does not have to do any "heavy lifting" to work with our footage, in that the footage does not have to be converted to 1920 x 1080 from 1440 x 1080. Since we support both Long-GOP and Intraframe equally well, I am just posting this to help clear up the misconceptions. If one prefers to shoot in I-Frame Only (our version of Intraframe), then we fully support their choice. The same applies to Long-GOP. The selection of Long-GOP or I-Frame Only is very easy, it is just a menu selection in our Flash XDR or nanoFlash. It does not mean that one has to switch camera vendors and purchase new cameras. If one has an Intraframe camera and wants to shoot Long-GOP, our nanoFlash fully supports this choice. If one wants to shoot Intraframe from a Long-GOP camera, this is also fully supported. __________________ Dan Keaton Augusta Georgia

October 31st, 2009, 10:08 AM	#2
Mark Job Trustee Join Date: Sep 2006 Location: Montreal, Quebec, Canada Posts: 1,138	Convergent Design's Version of I-Frame Hi Dan: There is one aspect to this discourse which has thrown me for a loop mentally, and I'm hoping you can clarify this point for me Dan. My question lies in an earlier explanation you gave on this forum, whereby you stated that CD writes its I-Frame MPEG video in a Long-GOP of 15 frames. (??) OK. This blows my mind ! Why do you do it this way and can we really conclude the Flash XDR and Nano Flash are writing true I-Frame video structure, if you are writing the I-Frames as Long GOP structure ? Isn't this just Long GOP by another name ??? Respectfully,

October 31st, 2009, 11:19 AM	#3
Dan Keaton Inner Circle Join Date: Dec 2002 Location: Augusta Georgia Posts: 5,421	Dear Mark, We have always called our modes "Long-GOP" and "I-Frame Only". Long-GOP starts with an I-Frame, and then has B and P Frames. Our I-Frame Only is in a Long-GOP structure in which every frame is an I-Frame. Every frame stands on its on for compression and decompression just as every Intraframe codec does. This allows the editors that work with XDCam 50 Mbps 4:2:2 footage work with our "I-Frame Only". Your Avid system should work well with our I-Frame Only footage, as should your Final Cut Pro system. __________________ Dan Keaton Augusta Georgia

October 31st, 2009, 01:12 PM	#5
Alister Chapman Inner Circle Join Date: Dec 2005 Location: Bracknell, Berkshire, UK Posts: 4,957	Mpeg 2 was originally developed as a long GoP codec. Long GoP codecs can have different combinations of I, B and P frames depending on their chosen use. In recent years it has become common practice to make use of I frames only and a simple way to do this is to have a GoP with nothing but I frames. The beauty of this is that you can use a single codec chip and matching codec software to work with both the traditional long GoP IBP type GoP's as well as the I frame GoP's. By doing this CD can make use of the XDCAM codecs installed in FCP, Avid etc to decode the I frame only codec without needing to write any additional software. Dan, Mike.. correct me if I am wrong. __________________ Alister Chapman, Film-Maker/Stormchaser http://www.xdcam-user.com/alisters-blog/ My XDCAM site and blog. http://www.hurricane-rig.com

October 31st, 2009, 02:19 PM	#7
Dan Keaton Inner Circle Join Date: Dec 2002 Location: Augusta Georgia Posts: 5,421	Dear Alister, I fully agree with your post. __________________ Dan Keaton Augusta Georgia

October 31st, 2009, 12:12 PM	#4
Mark Job Trustee Join Date: Sep 2006 Location: Montreal, Quebec, Canada Posts: 1,138	Hi Dan: So you're stating that it's not CD's version of I-Frame per se, rather, this has always been the way I-Frame MPEG is produced ?

October 31st, 2009, 01:24 PM	#6
Daniel Symmes Regular Crew Join Date: Jun 2007 Location: Los Angeles, CA U.S.A. Posts: 146	Recommended reading: MPEG-2 - Wikipedia, the free encyclopedia

October 31st, 2009, 03:06 PM	#8
Mark Job Trustee Join Date: Sep 2006 Location: Montreal, Quebec, Canada Posts: 1,138	The Structure of MPEG Hi Dan & Alister: I thought there was a form of editable MPEG which did not use the Long GOP structure ?

October 31st, 2009, 04:57 PM	#10
Dan Keaton Inner Circle Join Date: Dec 2002 Location: Augusta Georgia Posts: 5,421	Dear Gints, It is my opinion, if one is using any of the modern Non-Linear Editors, the ones that are typically discussed on this forum, if a cut is made in the middle of a Group of Pictures, a new Group of Pictures will be created starting with the first frame after the cut. To be fair, I do not know the details of how this is done. For example, I do not know if the NLE creates a smaller than normal Group of Pictures. and if there is no other reason to recompress and render. Or it could recreate the Group of Pictures until it gets to the end of this segment (source video segment). All of this depends on how sophisticated the NLE is. I assume that the most basic of editors, the very low cost editors, may do it differently. __________________ Dan Keaton Augusta Georgia

October 31st, 2009, 05:40 PM	#11
Mark Job Trustee Join Date: Sep 2006 Location: Montreal, Quebec, Canada Posts: 1,138	Avid Handles Long GOP Like This.... Hi Dan: In the latest iteration of Media Composer, Avid breaks up the entire Long GOP into a series of single frame MPEG's. I don't know how this is done, but I do know Matrox Digi-Suite was doing the same as early as 1997-98 is standard Definition. I know Avid has changed the way it handles MPEG once again, since version 3.5.x, because it is no longer necessary to make a video mixdown or transcode to output Long GOP based HDV projects. Now you can use a Quicktime Reference File to export to DVD or Web encoders, such as Sorensen Squeeze.

October 31st, 2009, 11:27 PM	#13
Mark Job Trustee Join Date: Sep 2006 Location: Montreal, Quebec, Canada Posts: 1,138	???????? Hi Rafael: You wrote: "NO. Only the GOP that is cut is rebuild. As Dan points, a smaller GOP is created. GOPs don't necessarily have the same length. The fully Intraframe structure of the NANO-I files can be tested with CinemaTool. You can "Conform" NANO-Intra footage, while you can not conform NANO-LGOP footage. A bit of reading would be very convenient. Cheers, Rafael" .......Huh ?

November 1st, 2009, 03:34 AM	#15
Mark Job Trustee Join Date: Sep 2006 Location: Montreal, Quebec, Canada Posts: 1,138	Avid 4/0/2 Hi Alister: You wrote: "That's impossible as in the case of HDV when exporting to HDV tape or streaming over firewire to a deck the deck would not be able to decode the non-standard output. It also implies that everything is forced into a decompression-re-compression step with it's resultant loss of quality and I doubt Avid would force that upon it's users. Avid as far as I know still handles material natively, which is why AMA is able to work with footage straight of SxS cards or XDCAM discs without actually needing to import the footage. Matrox Digi-Suite used a pair of on-board hardware Mpeg codecs to decompress in real time for output over the SDi or Component connections. Any cuts or edits had to be rendered before you could export a Mpeg stream. As Dan and others have said, only the GoP's immediately either side of any cuts or grades are processed. The way this is done seems to vary from NLE to NLE and may involve truncated GoPs. I have also heard of GoPs with frames of zero duration but never been able to confirm whether this is possible." .......No, I'm not 100 % positive, but the new Avid is now totally format and resolution independent and I'm pretty sure it strips ALL MPEG of its Long GOP structure without transcoding it. I know Avid no longer transcodes HDV or XDCAM HD projects because they say in their literature it is no longer necessary to perform video mix-downs of an HDV or XDCAM sequence or clip in order to export it back out of the NLE. Somehow the new Media Composer (4.0.2) has the ability to see each piece of video (Any video) as individual frames, then re-assenble the Long GOP on the way back out-all in real time ! Now this kind of data processing is a real accomplishment in programming. Alister, it doesn't matter where the cuts are made, because Avid will re-assemble the Long GOP structure appropriately. In fact, it may not even be as difficult as we think for the NLE to break down the Long GOP structure on the fly and then re-assemble it on the fly. I know it's CPU and GPU intensive, because the PC and MAC hardware requirements are hefty and strict. Avid only guarantees its software to run on approved machines in PC and MAC. The approved list is very short.