Is lossless joining of XA10 clips really possible?

[Nick Mirro]

I only own Videostudio X5 and assume the new X7 recompresses joined clips with similar results. Is there some simple app that can do this losslessly while preserving audio sync?

Was wondering about capturing a 'story' from the camcorder but this seems slow. Anyway I do not own a capture device.

[Bryan Worsley]

I don't know about VideoStudio X5, as I only started using Video Studio with VS Pro x6, and have since upgraded to VS Pro x7 (with Win 64-bit support.....at last !). But there is no problem joining your MTS clips "losslessly". I do exactly the same thing with clips from my HF-G10 and can assure you that (following the steps below) there is absolutely no re-compression of the video (i.e. no GOP re-construction at or around the splice-points) and audio streams. In other words, the full integrity of the original clips is preserved and with no loss in audio sync.

So, here's what you do - really straightforward:

1. In the 'Edit' view - browse for and load/open your clips into the media library.
2. Insert the selected clips into the timeline Video Track. Note - if you selected multiple clips to open in the media library, they will all appear as highlighted in the media library pane - so it's just a question of dragging the first highlighted clip of the series down to the timeline and the rest will follow - or else right click on the first clip thumbnail and select 'Insert to Video track'.
3. Now, don't bother to set up a project or anything. Simply go straight to the Share view. Under 'Create a video that you can play on your computer' there is a tick box, which by default is blank. Tick the box and from the adjacent drop-down menu select "Same as First Video Clip".
4. Make sure that the 'Enable Smart Render' option box (at the bottom) is ticked. Very important.
5. Name your destination file and location, click start, and away you go. Your MTS clips will be joined (smart-rendered) as a single file with an M2T extension.

Simple as that. I think you can probably you can do exactly the same in VS Pro x5. Just make sure you have Smart Rendering enabled and select 'Same as First Video Clip' in the file output options.

Hope that helps.

[Don Palomaki]

Is the issue joining spanned clips of a continuous shoot that were split by the camcorder at the 2 GB file size?

[Bryan Worsley]

I'm pretty sure he's referring, more generally, to joining the individual clips generated for each record episode, but the same principle and method applies (in VS Pro) to joining multiple files split by the camcorder at the 2GB mark during a long uninterrupted recording episode.

Edit: This all assumes of course that his objective is just to join native clips, without any pruning involved. Once you start trimming clips in the timeline there is bound to be some re-encoding. It's difficult to know exactly how VS Pro handles 'smart cutting' of H264 streams as the program provides no visual analysis to reveal those portions that will be re-encoded. But using a stream analysis tool, like Elecard Stream Eye (trial version), it looks like VS Pro only re-constructs (always starting with an I-frame) the particular GOP (15-frame for 30p streams) affected by a cut and doesn't appear to change the adjacent GOP's at all. It was from such stream analyses that I satisfied myself that merely joining native clips does not involve any re-encoding around the splice-points.

That said, I don't do any 'native editing' of my HF-G10 MTS clips and instead use a fast 'lossless' VFW codec intermediate (UTVideo, and now MagicYUV) for all editing in VS Pro x7.So my only reason for joining clips is to create a single file for transcoding to the lossless intermediate, which I do outside of VS Pro x7 using AVISynth.

I did at one point consider using (Pegasys) Smart Render 4 for joining and preliminary (frame-accurate) cut editing, but the program always re-encodes GOP's around the clip splice points. Works wonderfully with HDV MPEG-2 files, but with AVCHD clips (and H264 streams in general) my judgment is that the re-encoded segments tend to be sub-quality (with noticeable compression artifacts) by virtue of being encoded at a lower bitrate than the source. Again, you can see that in the Stream Eye analyses.

So, I'd say, yes, Video Studio Pro X7 is a good tool for joining native clips.

[Bryan Worsley]

Originally Posted by Don Palomaki

Is the issue joining spanned clips of a continuous shoot that were split by the camcorder at the 2 GB file size?

Ah, I see now why you would asking that:

http://www.dvinfo.net/forum/general-...ml#post1864362

Didn't realize that - mind you, I've not, so far, been in a continuous shoot situation with the HF-G10 that incurs 2GB split files, and neither have I used the Canon software for importing files. Probably should look into that for such an eventuality.

[Don Palomaki]

Splits can happen every 2 GB (10 minutes or so depending on the data rate). Driven by limits of the FAT32 format used on SD cards and the need to be compatible with various operating systems. A simple drag and drop using Windows explorer copies the files OK. But if you drop them on a time line you may encounter glitches/gaps/breaks were the files butt together. (Probably a result of the continuation files not having the same header as the first file in the shot. The NLE skips what it thinks is not-needed header information.)

I usually use the good old DOS COPY command with the /B (binary) option to rejoin split files. It avoids ending up with files I do not want/need on the edit machine that can result from methods that copy the whole card content. Some NLE can do this for you as part of their clip load/import/copy utilities.

[Bryan Worsley]

Originally Posted by Bryan Worsley

Ah, I see now why you would asking that:

http://www.dvinfo.net/forum/general-...ml#post1864362

Didn't realize that - mind you, I've not, so far, been in a continuous shoot situation with the HF-G10 that incurs 2GB split files, and neither have I used the Canon software for importing files. Probably should look into that for such an eventuality.

So, I tried, for the first time, the Canon software (that came with the HF-G10) and ran tests with some "spanned/split" clip recordings (i.e. split in two by the camcorder at the 1.9 GB mark). Strange that the Canon Transfer Utility on it's own does not automatically rejoin the split clips and that you need the Video Browser utility for that.

Anyhow, I compared the re-combined clips generated by the Canon Video Browser with those produced by Video Studio Pro x7. Indeed, if one merely copies over the split MTS clips located in the AVCHD/BDMV/Stream folder on the SD card and drops them onto the VS Pro x7 timeline, following the procedure I described above, the rendered stream loses two frames at the splice point, corresponding to the last two frames (B-frames, I think) of the first 'split' clip. Yet, there is no loss in audio sync, so VS Pro x7 does at least make the corresponding adjustments to the audio stream.

However, if the 'Import From Digital Media' function (under Capture section) in VS Pro x 7 is used to import the files from the card AVCHD/BDMV folder, then the split clips are automatically recombined and the resulting stream suffers no drop frames. Clearly then, in the case of spanned/split clips, VS Pro x7 does need the Clip Info metadata located in the BDMV folder to "correctly" rejoin the clips, without dropping frames.

But for joining shorter ("un-split") clips, I could find no evidence of dropped frames and/or audio async using the VS Pro x7 procedure I described above. The rendered streams were no different from those produced using the 'Edit' feature of the Canon Video Browser utility to join clips.

That said, I will change to using the Canon Video Browser utility for transfer and joining of clips from now on. Easier all round. And an option for the OP to maybe consider, if he's looking in.

Couldn't see myself using the 'Edit' function (in VideoBrowser) for trimming clips though. Like VS Pro X7, it re-encodes only the GOP's directly affected by the cut, but the re-encoded frames are lower quality.

[Bryan Worsley]

Ah, now this is puzzling.

Let's say you join some native MTS clips together using the Edit function of Canon VideoBrowser. If you take the resulting 'combined' MTS file, put it in the VS Pro x7 timeline and render it out as per the procedure I described above (Post #2), VS Pro x7 re-encodes all of it. Yet if you take the original source MTS clips (from the VideoBrowser import folder or directly from the AVCHD/BDMV/Stream folder on the SD card) and put them in the VS Pro x7 timeline and do the same, they are joined and smart-rendered with no re-encoding.

Just why there should be this difference, I don't understand. What does the VideoBrowser Editor do to these MTS clips to make them "resistant" to smart-rendering in VS Pro x7 ??

Possibly this is the issue the OP was facing?

Edit: Re-muxing the joined mts clips as m2ts files resolves the issue (simply changing the file extension name from .mts to .m2ts doesn't), but......if you try and join one such re-muxed m2ts clip to a native mts clip (or a m2t clip produced by VS Pro x7) using VS Pro x7 (set to render out in the format of the first clip) it will always re-encode the second clip. Similarly, if you re-mux a m2t clip produced by VS Pro x7 to m2ts, so that it can be opened by the Canon VideoBrowser editor, and attempt to join it to imported (or joined) mts clips, it will be re-encoded. Weird, considering that both mts and m2t are just abbreviated forms of the m2ts file extension and represent the same transport stream container. Evidently different editing programs recognize and treat these file extensions in different ways, at least when it comes to smart-rendering.

So, my conclusion on this is - as long as you work with the original MTS clips imported from the camcorder, joining the clips 'losslessly' is not a problem with VS Pro x7 or the Canon VideoBrowser Editor, but if you start mixing together 'smart-rendered' clips from the two programs, there will likely be issues, with some re-encoding involved. So, probably best to stick with one or the other for joining clips.

[Bryan Worsley]

Oh yes, I forgot, there is also a freeware application that works well for joining MTS clips - TSMuxer.

tsMuxeR 2.6.12

It's fast and gives the option of outputting as .m2ts or .ts files, but you don't have the benefit of seeing the clip thumbnails in a timeline - it just lists the files as you add them.

But you can "losslessly" join together .mts clips (coming from Canon VideoBrowser editor) and .m2t clips (coming from VS Pro x7), with a little workaround. The .m2t file extension is not recognized by TSMuxer as such and needs to be renamed as .m2ts. The mts files however must first be remuxed (using TSMuxer) to m2ts. If they are merely renamed, when it comes to rendering the 'mixed' clips, TSMuxer throws an error "Bad SEI detected. SEI too short" when it reaches the re-named mts clips and aborts the render. From what I can glean, SEI dictates the buffer timing of the video bit-stream and has something to do with the way the stream is recognized as Variable Bitrate (VBR) vs Constant Bitrate (CBR). Remuxing the mts clips to m2ts appears to "correct" this incompatibility, and when the files are joined TSMuxer re-sets the SEI (along with the Pict Timing) for the entire stream. Can't say I understand this fully, but it works perfectly and there is no loss of audio sync (or video jumping/skipping) in the rendered 'joined' m2ts file.

Quite probably this is the reason why VS Pro x7 and the Canon VideoBrowser Editor have issues when it comes to joining mts and m2t source clips and elect instead to re-encode those clips deemed (by the "smart rendering" algorithms employed) to be "incompatible" with the leading clip format. Arguably then TSMuxer is "smarter" in this respect !

Why would I want to join mts and m2t clips anyway, one might ask? Well, if I do change over to using the Canon software for importing and joining clips, I have archived m2t clips that I might want to include in the mix.

Anyhow, the OP seems to have lost interest, but this has been a very useful exercise for me and is maybe of interest to others facing the same issues !