Smart rendering of m2t-files possible? at DVinfo.net

Michael Mann · December 23rd, 2005, 08:47 AM

I like to render my HDV-files (.m2t) to MPEG2 (.m2t), using the MainConcept templates. But Vegas does not smart-render, instead it re-renders all unaltered m2t-files. Why ist that?

Peter Jefferson · December 25th, 2005, 09:35 PM

y do that wen u can render it cineform intermediate, edit then render to mpg2 for delivery?if ur editing native m2t hdv files, ur jsut chuggin away at the edit, plus its recompressing, so it takes up ALOT of rendering grunt (and time)

John Rofrano · January 1st, 2006, 12:50 PM

Quote:

Originally Posted by Michael Mann

Vegas does not smart-render, instead it re-renders all unaltered m2t-files. Why ist that?

Because M2T files do not contain all full frames so it is impossible to smart render. The MPEG format is a Group Of Pictures (GOP) which contains 15 frames. The first frame (I-Frame) is a full frame. The other 14 frames (B & P-Frames) are delta and predictive frames. They only contain partial information. When you edit the M2T file and add or delete one frame, ALL of the 15 frame sequences MUST be recalculated so that each one contains exactly 15 frames.

That’s why you don’t want to edit M2T files. You want to use CineForm intermediary files. These can be smart rendered because each frame is a full frame of information.

~jr

Tom Roper · January 1st, 2006, 01:46 PM

You can smart render native m2t without re-encoding it, but Vegas can't.

You can with Womble MPEG Video Wizard.

Make your cuts/splits with Womble to break up the file into smaller clips. Use Vegas to re-encode only the few clips that need color correction filters for example. Use Womble to re-assemble them on the timeline. It will quickly output a 100% non-rerendered m2t file with no generational loss, no intermediate codec.

Vegas has wonderful filters. Womble is better at scrubbing, cutting and joining losslessly.

Peter Jefferson · January 1st, 2006, 10:09 PM

Quote:

Originally Posted by John Rofrano

Because M2T files do not contain all full frames so it is impossible to smart render. The MPEG format is a Group Of Pictures (GOP) which contains 15 frames. The first frame (I-Frame) is a full frame. The other 14 frames (B & P-Frames) are delta and predictive frames. They only contain partial information. When you edit the M2T file and add or delete one frame, ALL of the 15 frame sequences MUST be recalculated so that each one contains exactly 15 frames.

That’s why you don’t want to edit M2T files. You want to use CineForm intermediary files. These can be smart rendered because each frame is a full frame of information.

~jr

THis is why DVCProHD is a much better format for HD than HDV.. HDV IS Good, (hell i use Z1s, but hanging for the HVX) anyways, when u consider that the compression of HDV is as described above, you really do lose alot of information on teh acquisition side of things. Even intermediate formats can only work with your source material, they wont improve the source material. And as ur source material is HIGHLY compressed (as described above again) ur still left with fairly substandard image quality (for HD) no matter what format its in.
I do LOE the colour space of the format, i just dont like the Z1...
DVCProHD doesnt have this high compression issue, so what u see is exactly what u get without the IBP issue.

Anywyas gettin back to the topic, Main Concept have released (a while back now) a HDV smart rendering codec. I dont know if Sony are going to license this for vegas 7, but i would say it would be a good idea.

Alot of "realtime" NLEs like Edius SP/NX Liquid and Axio dont require intermediate formats as they edit in the native m2t, and run a realtime process, so in case u cut on an I frame, the 14 other frames arent nuked.. the software writes in the "new" temporary Iframe... (unlike a dropout in HDV where if ur tape dies at an I-frame, you lose half a second of footage...)
Either way, intermediate is the best way to edit M2t, irrespective of teh system u use, reason i say this is that you can render and re-render and not affect the video for up to 15 or so generations.. whereby with native editing ur left with one, and everytime u render u will slowly kill off ur file as u go along..

John Rofrano · January 2nd, 2006, 11:45 AM

I guess these other smart-rendering formats tweak the GOP somehow when you edit it. There is no way you can keep it at 15 frames and not have to re-render it but perhaps shorter GOPs are allowed.

~jr

Laurence Kingston · January 4th, 2006, 04:51 PM

Quote:

Originally Posted by John Rofrano

I guess these other smart-rendering formats tweak the GOP somehow when you edit it. There is no way you can keep it at 15 frames and not have to re-render it but perhaps shorter GOPs are allowed.

~jr

Yeah, that's exactly how Womble works. It smartrenders HDV, but the GOP sequence is altered at the edit points. There's quite a clear explanation of this in the Womble help files.

The other, perhaps even more impressive thing about the Womble editor is how well it previews mpeg encoded video.

Yi Fong Yu · January 5th, 2006, 12:44 PM

how do i backup m2t files that are larger than 8.5 (DVD DL)? i recently rented Canon XL H1 but i don't own it. it's the only device that records/playsback 24F. i'd like to back my files up to DVD DL (8.5 gigabyte) if possible. since i can't just "chop" m2t files off. how do i back it up? cineform .avi's?

Michael Mann · January 5th, 2006, 12:56 PM

In terms of data security and time effort I recommend backups on hard disc.

John Rofrano · January 5th, 2006, 07:53 PM

Quote:

Originally Posted by Yi Fong Yu

how do i backup m2t files that are larger than 8.5 (DVD DL)?

It looks like Womble will help you do this. If you cut your M2T file on 8GB boundaries and do so on an I-Frame, it will simply copy the data at full quality to smaller files. You will have no loss at all.

I agree with Michael. Hard Drives are just as cheap as DVD’s. Dual layer DVD’s are about $0.32 per GB and hard drives cost about $0.42 per GB. Get an external firewire case (I picked one up for $50) and just buy 200GB hard drives to archive. When one gets full, place it on the shelf (label what’s on it) and install a new one. This way you can archive an entire project and call it back quickly to work on it again in the future.

~jr

Yi Fong Yu · January 5th, 2006, 09:06 PM

yeah... that's why we're all anxiously awaiting BluRay/HD-DVD Burners. 1 or 2 tapes per disc =).

Tom Roper · January 5th, 2006, 09:50 PM

Uh John, it sound like you tried MVW...(gasp)!
:-)

Laurence Kingston · January 5th, 2006, 10:48 PM

Mpeg Wizard has some shortcut keys that will take you to the next GOP boundry, but you can cut and join files everywhere. This is not a perfect solution however. You end up with an mpeg file that has unstandard frame orders. Some players won't play these mpegs at all and if you're PC or the source you are streaming the mpeg from is straining at all it will likely glitch at these points even if the media player can handle the odd sequence. Mpegs with an odd frame sequence can't be directly copied to HDV tape as M2T files either.

None the less, even with these limitations the Womble editor is really cool. It works without intermediates or proxies, hardly strains a moderately powered PC at all and renders (thanks to smartrendering) almost as fast as a straight file copy. The files, frame oddities and all, play back just fine from Nero Showtime, and any time this isn't good enough, I can just load the Womble edited file back in a fresh project timeline and render it with some kind of null filter so that the whole thing gets freshly rerended in a single properly frame sequenced mpeg. When you do this, you are still only one generation in aside from the transitions which are second generation at this point. It still looks great.

Another great trick is to throw the Womble edited mpeg on a Vegas timeline and rerender it. That clears out any frame order irregularities and Vegas shines at converting the HDV Mpeg into whatever other format you may need.

I still use Vegas for major stuff, but for things like throwing together event footage with minor edits and simple titles the Womble software is hard to beat.

Tom Roper · January 6th, 2006, 10:16 PM

It's really not necessary Laurence. MVW includes the following tools that you can use in any combination to get, fix or maintain perfect GOP cadence.

*******************************

GOP Trim:

Check this box to trim a video clip on GOP (I-frame) boundaries. With this option checked (ON), the saved file will be on an I-frame boundary and the packet structures of the original file will be preserved except at the beginning and the end of the saved file. Therefore, a "GOP Trim" is almost like a binary cut to an MPEG file.

GOP size compliance for DVD recording:

Check this box to require that all exported MPEG movies will have the GOP size compliance for DVD recording. This will introduce partial video re-encoding to those GOP's that exceed the maximum GOP size set by the DVD standard.

Scene-Change Detection
This will improve compression efficiency by starting a new GOP at a scene change.

GOP FIXER

This tool lets you check and correct GOP time code errors, video and audio PTS (presentation time stamp) errors, and DVD GOP size violations, which may be present in an MPEG data file.

GOP Time Code Error

In MPEG compression, a video sequence is divided into groups of image frames. A typical sequence of picture frames looks like: "I B B P B B P B B P B B P B B". This is called a GOP (group of pictures). A header is usually stored before the I-picture, and is called a GOP header, and it is usually an 8-byte data segment. One of the important items of information stored in this header is a 25-bit integer, called an SMPTE time code (SMPTE: Society of Motion Picture and Television Engineers). This time code refers to the first picture of the GOP in display order.

This GOP header and its time code provide very convenient points for managing a coded MPEG movie. It allows the MPEG editor to randomly access a lengthy MPEG movie very efficiently. However, this also makes the editor highly dependent on the accuracy of time codes in the GOP header, and especially dependent on the continuity of the time codes in a complete MPEG movie. Unfortunately, an MPEG movie may have errors in the time code, or the time code may be disrupted, or the time code may not correctly reflect the coded image sequence. When any of these happens to an MPEG file, it is considered a GOP time code error.

The GOP fixer tool will correct the GOP time code errors based on its calculation from the complete video sequence. Very few bits of original data will be changed.

Video PTS and Audio PTS Error

Video and audio Presentation Time Stamps are 33-bit integers inserted in an MPEG stream to allow an MPEG decoder to synchronize the decoded audio sound with the decoded video image display. Those PTS values should match the playback time length of the decoded audio sound with the video frame rate. When they are in error, the MPEG movie cannot be played correctly, and the audio and the video will be out of sync.

The GOP fixer tool will correct those PTS errors based on its calculation from the complete audio data. It will only modify the erroneous 33-bit integers.

DVD GOP Size Violation

The DVD standard has special restrictions on the size of a GOP for a movie to be compressed into MPEG. Specifically, it requires that no GOP should have more than 18 frames for NTSC TV system and 15 frames for PAL TV system, even though the MPEG standards have no such limitations.

To make an MPEG file compliant with the DVD restriction on GOP size, the GOP fixer tool will find the violator and break it into multiple GOP's of smaller sizes. GOP's of the correct size will just be copied without modification.

December 23rd, 2005, 08:47 AM	#1
Michael Mann Major Player Join Date: Dec 2005 Location: Kelkheim, Germany Posts: 375	Smart rendering of m2t-files possible? I like to render my HDV-files (.m2t) to MPEG2 (.m2t), using the MainConcept templates. But Vegas does not smart-render, instead it re-renders all unaltered m2t-files. Why ist that?

January 2nd, 2006, 11:45 AM	#6
John Rofrano Sponsor: VASST Join Date: May 2005 Location: New York Posts: 516	I guess these other smart-rendering formats tweak the GOP somehow when you edit it. There is no way you can keep it at 15 frames and not have to re-render it but perhaps shorter GOPs are allowed. ~jr __________________ Developer: VASST Ultimate S, Scattershot 3D, Mayhem, FASST Apps, and other VASST Software plug-ins Web Site: www.johnrofrano.com

January 5th, 2006, 12:44 PM	#8
Yi Fong Yu Trustee Join Date: Sep 2002 Location: Vulcan Posts: 1,564	how do i backup m2t files that are larger than 8.5 (DVD DL)? i recently rented Canon XL H1 but i don't own it. it's the only device that records/playsback 24F. i'd like to back my files up to DVD DL (8.5 gigabyte) if possible. since i can't just "chop" m2t files off. how do i back it up? cineform .avi's? __________________ bow wow wow

January 5th, 2006, 09:06 PM	#11
Yi Fong Yu Trustee Join Date: Sep 2002 Location: Vulcan Posts: 1,564	yeah... that's why we're all anxiously awaiting BluRay/HD-DVD Burners. 1 or 2 tapes per disc =). __________________ bow wow wow

December 25th, 2005, 09:35 PM	#2
Peter Jefferson Inner Circle Join Date: Apr 2003 Location: Aus Posts: 3,884	y do that wen u can render it cineform intermediate, edit then render to mpg2 for delivery?if ur editing native m2t hdv files, ur jsut chuggin away at the edit, plus its recompressing, so it takes up ALOT of rendering grunt (and time)

January 1st, 2006, 01:46 PM	#4
Tom Roper Trustee Join Date: Jun 2004 Location: Denver, Colorado Posts: 1,891	You can smart render native m2t without re-encoding it, but Vegas can't. You can with Womble MPEG Video Wizard. Make your cuts/splits with Womble to break up the file into smaller clips. Use Vegas to re-encode only the few clips that need color correction filters for example. Use Womble to re-assemble them on the timeline. It will quickly output a 100% non-rerendered m2t file with no generational loss, no intermediate codec. Vegas has wonderful filters. Womble is better at scrubbing, cutting and joining losslessly.

January 5th, 2006, 12:56 PM	#9
Michael Mann Major Player Join Date: Dec 2005 Location: Kelkheim, Germany Posts: 375	In terms of data security and time effort I recommend backups on hard disc.

January 5th, 2006, 09:50 PM	#12
Tom Roper Trustee Join Date: Jun 2004 Location: Denver, Colorado Posts: 1,891	Uh John, it sound like you tried MVW...(gasp)! :-)

January 5th, 2006, 10:48 PM	#13
Laurence Kingston Major Player Join Date: Aug 2004 Location: Clermont, FL. Posts: 941	Mpeg Wizard has some shortcut keys that will take you to the next GOP boundry, but you can cut and join files everywhere. This is not a perfect solution however. You end up with an mpeg file that has unstandard frame orders. Some players won't play these mpegs at all and if you're PC or the source you are streaming the mpeg from is straining at all it will likely glitch at these points even if the media player can handle the odd sequence. Mpegs with an odd frame sequence can't be directly copied to HDV tape as M2T files either. None the less, even with these limitations the Womble editor is really cool. It works without intermediates or proxies, hardly strains a moderately powered PC at all and renders (thanks to smartrendering) almost as fast as a straight file copy. The files, frame oddities and all, play back just fine from Nero Showtime, and any time this isn't good enough, I can just load the Womble edited file back in a fresh project timeline and render it with some kind of null filter so that the whole thing gets freshly rerended in a single properly frame sequenced mpeg. When you do this, you are still only one generation in aside from the transitions which are second generation at this point. It still looks great. Another great trick is to throw the Womble edited mpeg on a Vegas timeline and rerender it. That clears out any frame order irregularities and Vegas shines at converting the HDV Mpeg into whatever other format you may need. I still use Vegas for major stuff, but for things like throwing together event footage with minor edits and simple titles the Womble software is hard to beat.

January 6th, 2006, 10:16 PM	#14
Tom Roper Trustee Join Date: Jun 2004 Location: Denver, Colorado Posts: 1,891	It's really not necessary Laurence. MVW includes the following tools that you can use in any combination to get, fix or maintain perfect GOP cadence. ******************************* GOP Trim: Check this box to trim a video clip on GOP (I-frame) boundaries. With this option checked (ON), the saved file will be on an I-frame boundary and the packet structures of the original file will be preserved except at the beginning and the end of the saved file. Therefore, a "GOP Trim" is almost like a binary cut to an MPEG file. GOP size compliance for DVD recording: Check this box to require that all exported MPEG movies will have the GOP size compliance for DVD recording. This will introduce partial video re-encoding to those GOP's that exceed the maximum GOP size set by the DVD standard. Scene-Change Detection This will improve compression efficiency by starting a new GOP at a scene change. GOP FIXER This tool lets you check and correct GOP time code errors, video and audio PTS (presentation time stamp) errors, and DVD GOP size violations, which may be present in an MPEG data file. GOP Time Code Error In MPEG compression, a video sequence is divided into groups of image frames. A typical sequence of picture frames looks like: "I B B P B B P B B P B B P B B". This is called a GOP (group of pictures). A header is usually stored before the I-picture, and is called a GOP header, and it is usually an 8-byte data segment. One of the important items of information stored in this header is a 25-bit integer, called an SMPTE time code (SMPTE: Society of Motion Picture and Television Engineers). This time code refers to the first picture of the GOP in display order. This GOP header and its time code provide very convenient points for managing a coded MPEG movie. It allows the MPEG editor to randomly access a lengthy MPEG movie very efficiently. However, this also makes the editor highly dependent on the accuracy of time codes in the GOP header, and especially dependent on the continuity of the time codes in a complete MPEG movie. Unfortunately, an MPEG movie may have errors in the time code, or the time code may be disrupted, or the time code may not correctly reflect the coded image sequence. When any of these happens to an MPEG file, it is considered a GOP time code error. The GOP fixer tool will correct the GOP time code errors based on its calculation from the complete video sequence. Very few bits of original data will be changed. Video PTS and Audio PTS Error Video and audio Presentation Time Stamps are 33-bit integers inserted in an MPEG stream to allow an MPEG decoder to synchronize the decoded audio sound with the decoded video image display. Those PTS values should match the playback time length of the decoded audio sound with the video frame rate. When they are in error, the MPEG movie cannot be played correctly, and the audio and the video will be out of sync. The GOP fixer tool will correct those PTS errors based on its calculation from the complete audio data. It will only modify the erroneous 33-bit integers. DVD GOP Size Violation The DVD standard has special restrictions on the size of a GOP for a movie to be compressed into MPEG. Specifically, it requires that no GOP should have more than 18 frames for NTSC TV system and 15 frames for PAL TV system, even though the MPEG standards have no such limitations. To make an MPEG file compliant with the DVD restriction on GOP size, the GOP fixer tool will find the violator and break it into multiple GOP's of smaller sizes. GOP's of the correct size will just be copied without modification.