Combining many nano clips

Tim Polster · October 26th, 2009, 08:31 AM

On a related note, a Panasonic tech gave me some DOS commands to stitch together AVCHD clips from the HMC-150.

This did not re-encode anything, but was not instant as it seemingly had to go through the files bit by bit.

This is for the PC btw.

Here is what he sent:

<Procedure to merge files>
1. Copy all divided files in a specific folder on the NTFS drive(ex.
C:\AVCHD Temp)
2. If the divided files are 1000.m2ts, 1001.m2ts and 1002.m2ts, Run
"copy /b 1000.m2ts+1001.m2ts+1002.m2ts AVCHDfile.m2ts" via [Command
Prompt].
3. Then divided files are merged into a single file (AVCHDfile.m2ts in
the example).
4. Import the merged file into EDIUS.

I then showed this and a member wrote a little utility with a windows interface that let one pick all of the files to merge and the choose a destination and filename.

This is really useful for live events where the record is running for a while. One clip in the editor means one filter and adjustment application before cutting.

Hope this helps.

Daniel Symmes · October 26th, 2009, 09:53 AM

Tim -

Can you provide a link to the fellow's program? I'd like to test it.

Ah. Did a little detective work and found it.

Testing time.

Billy Steinberg · October 26th, 2009, 01:51 PM

Thanks for all the info; I'll be testing it out later this week and see how it goes.

I could have sworn that all QT .mov files had header info, and so thought that a simple concatenation from the command line wouldn't work, but I'll be delighted to see that I was wrong. :)

Or could it just be that QT can play the files and deal with a bunch of headers transparently to us... (Though running the files through MPEG Streamclip will take care of that, as will running them through FCP). I'll try combining both ways (command line and streamclip) and see if the output files are the same.

Billy

Aaron Newsome · October 26th, 2009, 02:15 PM

My initial testing shows the timecode is in the new file exactly how it should be. Which makes me question once again, if you would want to do this with files there were not a continuous shot.

Some software can not handle timecode breaks in the mpeg stream properly and if it could, would you want it to?

I'll be combining files from continuous shots only.

Billy Steinberg · October 26th, 2009, 04:46 PM

I have no intention of combining non-sequential files (not sure why you thought that was what I had in mind). What I was talking about was whether QT .mov files had headers, (I thought they did) and if so, how come concatenating smaller (sequential) files worked. If they do have headers, then QT is smart enought to be able to deal with multiple headers in a single file. More likely, QT doesn't have/need headers for this type of file. In any case:

I'll know more (maybe...) later this week.

Billy

Tommy Schell · October 27th, 2009, 09:02 AM

Hi,

perhaps I'm missing something, but I tried concatenating (gluing together) 2 Quicktime files with the terminal command line cat function, the result simply plays the 1st file and that's it, in Qt Player.
Kind of what I expected, as each file has indexing info and a simple concatenation of files wouldn't account for this.

I concatenated 2 mxf files and it wouldn't even open in XDCAM Viewer.

I can believe that m2ts files (as mentioned above by Tim Polster) would concatenate successfully, since they don't have any indexing info in the file.
The same is probably true of nanoFlash .MPG files.

Tommy Schell

Aaron Newsome · October 27th, 2009, 09:15 AM

Quote:

Originally Posted by Tommy Schell

Hi,

perhaps I'm missing something, but I tried concatenating (gluing together) 2 Quicktime files with the terminal command line cat function, the result simply plays the 1st file and that's it, in Qt Player.
Kind of what I expected, as each file has indexing info and a simple concatenation of files wouldn't account for this.

I concatenated 2 mxf files and it wouldn't even open in XDCAM Viewer.

I can believe that m2ts files (as mentioned above by Tim Polster) would concatenate successfully, since they don't have any indexing info in the file.
The same is probably true of nanoFlash .MPG files.

Tommy Schell

stitching together MXF would probably not work. Quicktime files from XDR work fine for me though.

Tim Polster · October 28th, 2009, 08:52 AM

So is there any hope of an MXF clip-join utility?

Bear in mind that this would only be useful if it is quick without needing to render anything.

Billy Steinberg · October 28th, 2009, 09:14 AM

Quote:

Originally Posted by Tommy Schell

Hi,

perhaps I'm missing something, but I tried concatenating (gluing together) 2 Quicktime files with the terminal command line cat function, the result simply plays the 1st file and that's it, in Qt Player.
Kind of what I expected, as each file has indexing info and a simple concatenation of files wouldn't account for this.

I concatenated 2 mxf files and it wouldn't even open in XDCAM Viewer.

I can believe that m2ts files (as mentioned above by Tim Polster) would concatenate successfully, since they don't have any indexing info in the file.
The same is probably true of nanoFlash .MPG files.

Tommy Schell

I'm with Tommy here. There are definitely headers on the QT .mov files, and simply concatenating multiple files together does NOT work. Quicktime simply plays back the first segment and doesn't know about the subsequent ones. I recorded a bunch of clips yesterday and tested them out last night.

I also ran the files through MPEG Streamclip, and it was able to losslessly process the multiple clips into a single clip that Quicktime was happy with. (Use "Save As"). Examining the combined file with a hex editor revealed that MPEG Streamclip had done the right thing, with a single header defining the combined file properly, and no extra headers buried throughout the file.

So I don't know exactly what Aaron is seeing, but all my testing shows that simply concatenating nano QT .mov files fdoes NOT work.

Billy

October 26th, 2009, 09:53 AM	#17
Daniel Symmes Regular Crew Join Date: Jun 2007 Location: Los Angeles, CA U.S.A. Posts: 146	Tim - Can you provide a link to the fellow's program? I'd like to test it. Ah. Did a little detective work and found it. Testing time. Last edited by Daniel Symmes; October 26th, 2009 at 10:52 AM.

October 26th, 2009, 08:31 AM	#16
Tim Polster Inner Circle Join Date: Dec 2004 Location: Arlington, TX Posts: 2,231	On a related note, a Panasonic tech gave me some DOS commands to stitch together AVCHD clips from the HMC-150. This did not re-encode anything, but was not instant as it seemingly had to go through the files bit by bit. This is for the PC btw. Here is what he sent: <Procedure to merge files> 1. Copy all divided files in a specific folder on the NTFS drive(ex. C:\AVCHD Temp) 2. If the divided files are 1000.m2ts, 1001.m2ts and 1002.m2ts, Run "copy /b 1000.m2ts+1001.m2ts+1002.m2ts AVCHDfile.m2ts" via [Command Prompt]. 3. Then divided files are merged into a single file (AVCHDfile.m2ts in the example). 4. Import the merged file into EDIUS. I then showed this and a member wrote a little utility with a windows interface that let one pick all of the files to merge and the choose a destination and filename. This is really useful for live events where the record is running for a while. One clip in the editor means one filter and adjustment application before cutting. Hope this helps.

October 26th, 2009, 01:51 PM	#18
Billy Steinberg Regular Crew Join Date: Nov 2008 Location: NYC Posts: 122	Thanks for all the info; I'll be testing it out later this week and see how it goes. I could have sworn that all QT .mov files had header info, and so thought that a simple concatenation from the command line wouldn't work, but I'll be delighted to see that I was wrong. :) Or could it just be that QT can play the files and deal with a bunch of headers transparently to us... (Though running the files through MPEG Streamclip will take care of that, as will running them through FCP). I'll try combining both ways (command line and streamclip) and see if the output files are the same. Billy

October 26th, 2009, 02:15 PM	#19
Aaron Newsome Major Player Join Date: Jul 2009 Location: Rancho Santa Margarita Posts: 590	My initial testing shows the timecode is in the new file exactly how it should be. Which makes me question once again, if you would want to do this with files there were not a continuous shot. Some software can not handle timecode breaks in the mpeg stream properly and if it could, would you want it to? I'll be combining files from continuous shots only.

October 26th, 2009, 04:46 PM	#20
Billy Steinberg Regular Crew Join Date: Nov 2008 Location: NYC Posts: 122	I have no intention of combining non-sequential files (not sure why you thought that was what I had in mind). What I was talking about was whether QT .mov files had headers, (I thought they did) and if so, how come concatenating smaller (sequential) files worked. If they do have headers, then QT is smart enought to be able to deal with multiple headers in a single file. More likely, QT doesn't have/need headers for this type of file. In any case: I'll know more (maybe...) later this week. Billy

October 27th, 2009, 09:02 AM	#21
Tommy Schell Regular Crew Join Date: Nov 2007 Location: Colorado Springs, CO Posts: 65	Hi, perhaps I'm missing something, but I tried concatenating (gluing together) 2 Quicktime files with the terminal command line cat function, the result simply plays the 1st file and that's it, in Qt Player. Kind of what I expected, as each file has indexing info and a simple concatenation of files wouldn't account for this. I concatenated 2 mxf files and it wouldn't even open in XDCAM Viewer. I can believe that m2ts files (as mentioned above by Tim Polster) would concatenate successfully, since they don't have any indexing info in the file. The same is probably true of nanoFlash .MPG files. Tommy Schell

October 28th, 2009, 08:52 AM	#23
Tim Polster Inner Circle Join Date: Dec 2004 Location: Arlington, TX Posts: 2,231	So is there any hope of an MXF clip-join utility? Bear in mind that this would only be useful if it is quick without needing to render anything.