Rogue QT reference file at DVinfo.net

Scott Brickert · January 2nd, 2009, 01:56 PM

I have a QT reference movie, 2h18m, in HDV, which I'm trying to get converted to MPEG2 for delivery on dual layer DVD.

Whether from the FCP timeline, or direct into Compressor, the render begins, then just disappears after around 2000 frames.

The files are on an external eSATA drive, connected to the MBP via a Sonnet Tempo Sata Express 34, or a Sonnet card in the Mac Pro.

On the MBP, the file plays in QuickTime, although it hung really completely at 2h2m. The mouse and keyboard were non-responsive, while the audio stuttered. I had to use the hard switch. I restarted and tried again at 1:58, and it played through to the end no problem.

I moved the drive to the Mac Pro in order to leverage the processors. It still disappears even before the barber pole starts moving, although the estimation timer is running.

I installed and ran Compressor Repair, just in case. No Help.

i guess there's some issue with the file itself.

While rendering the file out of FCP using Export>QuickTime Movie, there was a General Error. I originally thought there was no result, but then saw the file and played it in QT. When it sorta played, I as hoping to be able to get the conversion done.

Any ideas?

Battle Vaughan · January 2nd, 2009, 02:12 PM

I'm just pulling a guess, here, but I think the reference movie part is the problem --- it's referring to files somewhere else. Why not export it as a self-contained .mov and see if that solves the problem? I'm assuming you are outputting the .mov from your FC timeline and all your clips are present? We have made (ok, single layer, I have no double layer experience) dvd's from hdv quicktime files and just let DVD Studio sort out the conversion with nice results....Battle Vaughan/miamiherald.com video team

David Knaggs · January 2nd, 2009, 04:17 PM

Hi Scott.

It's possible that one of the individual clips (or more than one) within your timeline is corrupted. It can happen occasionally with a clip captured as an .m2t file which is then made into a QuickTime movie with ClipWrap. Or at least this has occasionally happened to me. A solution, in such a case, is to re-capture the problem clip natively using FCP.

Battle's suggestion of exporting as a self-contained movie is a good one, although I would recommend that you export it as a ProRes movie (it's best to avoid another compression with MPEG-2) and then import the ProRes movie into Compressor.

Scott Brickert · January 3rd, 2009, 11:03 AM

Compressor logged a 'QuickTime Error:0' which seems to point toward a corrupt file, based on other forum threads.

I tried rendering to ProRes, but don't have enough disk space. I have two 1TB drives on the way, but want to finish sooner.

So I'm trying a self-contained export, maintaining HDV. If successful, maybe Compressor will convert it to MPEG2.

This is really not fun.

Shaun Roemich · January 3rd, 2009, 11:32 AM

Scott, have you tried doing a Export Using Compressor from the Browser? Right Click (Ctrl-Click) on the Sequence in question in the Browser, select Export Using Compressor and set your export criteria when Compressor comes up.

Not sure if this will help, but it sure beats having to export the entire timeline and then re-encode. Better use of time, better use of disk space IMHO.

Robert Lane · January 3rd, 2009, 04:33 PM

Scott,

Try this:

Go back to the timeline you're trying to output from, do a "select all" on the timeline and use the "reconnect media" function and see if anything comes up as missing in the timeline and if it does, reconnect it.

Try copying the entire finished timeline to a new one, render it for playback "full render" and then output a reference file and see if it still fails. Make sure your timeline settings match exactly what you need first before importing.

If possible, try encoding this in another program such as Episode Pro; if it fails in that app also then there's definitely a clip-corruption problem.

David Knaggs · January 3rd, 2009, 08:26 PM

Quote:

Originally Posted by Scott Brickert

Compressor logged a 'QuickTime Error:0' which seems to point toward a corrupt file, based on other forum threads.

If you have no luck with the HDV export (and I forgot to mention AIC as a good alternative to ProRes - AIC takes up a lot less disk space and is supposed to be visually lossless) or any of the other suggestions, perhaps try the following (based on my occasional experiences with corrupt HDV files):

1/ The detective work. FCP might hang or crash when it gets to the corrupt file (or to within a few clips of the corrupt file). If it's playing okay from 1 hr 58m onwards, play it from 1 hr 50, then 1 hr 40, etc. till you isolate the 10 minute segment and home in from there. If no luck with that,

2/ Export in segments. On another occasion when I had Compressor crash (with a next day deadline to deliver as a 125 minute dual-layer DVD), I started exporting in 30 minute long segments (by placing in and out points in the timeline) as self-contained AIC QuickTime movies and assembled them in a new AIC timeline. when you get to the 30 minute segment that won't export, break it into 2 X 15 minute segments and export, etc., etc. - all the while assembling the new pieces in your AIC timeline - until you isolate the very small segment and then you can recapture and replace that problem clip or clips if necessary. Then export to Compressor from the new AIC timeline. Crude, but that general approach actually worked for me.

3/ Replacing the corrupt file (or files). When you finally isolate the problem area, you'll find that you won't be able to delete the problem clip because FCP will hang up or crash every time you get to it. Simply disconnect your external media hard drive and then locate the problem clip(s) in the browser and note down its reel number and timecode (so you can easily recapture the clip natively). Then delete (Command X) the corrupt file from both the browser and the timeline before reconnecting your media drive.

These tips may or may not prove useful in your circumstances but, as I said, they worked for me.

Bob Hart · January 4th, 2009, 03:37 AM

I have just been successfully through an exercise of "repairing" a quicktime file which was damaged by my stupidity of switching off a SI2K camera before it had finished doing what it had to do.

The software I used was written by Wafian Corporation who make the Wafian recorder. The Wafian evidently creates Cineform .avi files or Quicktime files.

It seems to have a good look at each frame and at the end of its journey through the clip writes a reference file for this damaged clip. The reference file and the original file must travel together and it is the reference file you work with.

If you are lucky, this software may help you out. It is however a script based application. You have to go into Microsoft Command Prompt (MSDOS) to use it. I don't know if there is a Mac version.

I was directed to it on the Silicon Imaging website.

Scott Brickert · January 5th, 2009, 10:42 PM

Well, the hits just keep on rollin'

So I read somewhere maybe it was a bad render. Deleted all renders and immediately exported to a self-contained QT-HDV. Still got a General Error and bad file.

Next, I began the reconstruction. The final piece consists of three sections. The first three minute Intro, was exported to a reference QT-AIC file and imported back. Don't expect any trouble there. Next is a nested filed built of source and multiclips. I collapsed the Multiclips and exported it to self-contained QT-AIC in 5 minute segments--no crashes and no General Errors. Imported each segment and placed on the Master T/L above it's corresponding nested sequence. Finally I moved to the last piece, about 1h40m, where I deleted the Multiclip and replaced with the source clip, then exported in 10 minutes segments, importing those and placing on the Master T/L. This whole process took less than 4 hours, which I found surprising since exporting either nested piece had always taken 4-8 hours.

Once complete, it flowed smoothly, as it always did in FCP (there never were any crashes when playing from any timeline in FCP, although there were hiccups until everything was rendered).

So now I have an AIC 1440x1080i60 sequence, filled with media with the same format. I exported to a self-contained QT with current settings and markers for DVDSP. That took longer than expected, around 3 hours. I played the first 10 minutes in QT, and it looked fine.

I imported that into Compressor for conversion to MPEG2. It was chugging along, foretelling about 3 hours to process. I expanded everything in the Batch Monitor, and saw 12 segments. At first 6 video and 1 audio streams were processing, which I also saw in Leopard's activity monitor. One process hung, so I forced it to Quit. The other continued. Eventually the audio finished successfully. Now we have 6 segments processing the first of the two passes (I'd selected a 2-pass transcode at around 6Mbps--the setting for Best Quality, 90 minute DVD, modified to 16:9), while the other six showed 'Waiting'. The total completion strip showed about 70% complete the last I looked. This seemed a bit much since there were several segments that had not yet successfully completed.

The next thing I know, the entire Batch disappeared from the Batch Monitor. The Activity Monitor still shows qmasterd with 8 threads, one of which is Not Responding, two show zero, five show CPU around 150 (what does that mean, I thought it was percent of CPU usage, not to exceed 100?) and memory usage around 90MB. The CPU graph is almost all green, showing around 80% User and 15% System. There is still Read and Write activity going to the disks.

If anyone can shed light on this, I'd be SO GRATEFUL.

Should I let the processes run or kill them? Why does Batch Monitor show no activity? The destination folder shows segments 6,7,8,9,10, averaging around 500MB apiece. There's no sign of the final complete conversion, nor of segments 1-5,12.

If this gets any more interesting, I'll ......... I"ll,,,,,,,,,,,, well, I'll just...............

Apologies for the long involved post. What did I overlook? I didn't try reconnecting the files since none showed up missing when opening the project...

OK, Activity Monitor still shows qmasterd but all processes show zero under CPU usage. Still can't find the final file, nor the AC3 file. Get Info show 46GB free on the drive.

Any ideas?

David Knaggs · January 6th, 2009, 01:38 AM

Maybe take the self-contained AIC QuickTime movie straight to DVD SP and encode it there (bypassing Compressor). If DVD SP encodes it okay, then it's probably been a problem with Compressor. If DVD SP won't encode it either, then it's likely to be a problem with the movie. That's all I can think of.

Scott Brickert · January 14th, 2009, 10:29 PM

Skipping Compressor and letting DVDSP render to MPEG2 worked!

I delivered the DVD's today. What a relief. Compressor seems pretty finicky, or downright unreliable, but then again I haven't read the book yet.

David Knaggs · January 16th, 2009, 12:45 AM

I'm very glad to find there was a happy ending! Thanks for posting the follow-up.

January 2nd, 2009, 01:56 PM	#1
Scott Brickert Major Player Join Date: Jun 2003 Location: Crested Butte, CO Posts: 576	Rogue QT reference file I have a QT reference movie, 2h18m, in HDV, which I'm trying to get converted to MPEG2 for delivery on dual layer DVD. Whether from the FCP timeline, or direct into Compressor, the render begins, then just disappears after around 2000 frames. The files are on an external eSATA drive, connected to the MBP via a Sonnet Tempo Sata Express 34, or a Sonnet card in the Mac Pro. On the MBP, the file plays in QuickTime, although it hung really completely at 2h2m. The mouse and keyboard were non-responsive, while the audio stuttered. I had to use the hard switch. I restarted and tried again at 1:58, and it played through to the end no problem. I moved the drive to the Mac Pro in order to leverage the processors. It still disappears even before the barber pole starts moving, although the estimation timer is running. I installed and ran Compressor Repair, just in case. No Help. i guess there's some issue with the file itself. While rendering the file out of FCP using Export>QuickTime Movie, there was a General Error. I originally thought there was no result, but then saw the file and played it in QT. When it sorta played, I as hoping to be able to get the conversion done. Any ideas? __________________ Scott Shot-By-Scott

January 3rd, 2009, 11:03 AM	#4
Scott Brickert Major Player Join Date: Jun 2003 Location: Crested Butte, CO Posts: 576	Compressor logged a 'QuickTime Error:0' which seems to point toward a corrupt file, based on other forum threads. I tried rendering to ProRes, but don't have enough disk space. I have two 1TB drives on the way, but want to finish sooner. So I'm trying a self-contained export, maintaining HDV. If successful, maybe Compressor will convert it to MPEG2. This is really not fun. __________________ Scott Shot-By-Scott

January 3rd, 2009, 11:32 AM	#5
Shaun Roemich Inner Circle Join Date: Sep 2002 Location: Vancouver, British Columbia (formerly Winnipeg, Manitoba) Canada Posts: 4,088	Scott, have you tried doing a Export Using Compressor from the Browser? Right Click (Ctrl-Click) on the Sequence in question in the Browser, select Export Using Compressor and set your export criteria when Compressor comes up. Not sure if this will help, but it sure beats having to export the entire timeline and then re-encode. Better use of time, better use of disk space IMHO. __________________ Shaun C. Roemich Road Dog Media - Vancouver, BC - Videographer - Webcaster www.roaddogmedia.ca Blog: http://roaddogmedia.wordpress.com/

January 4th, 2009, 03:37 AM	#8
Bob Hart Inner Circle Join Date: Dec 2003 Location: PERTH. W.A. AUSTRALIA. Posts: 4,477	I have just been successfully through an exercise of "repairing" a quicktime file which was damaged by my stupidity of switching off a SI2K camera before it had finished doing what it had to do. The software I used was written by Wafian Corporation who make the Wafian recorder. The Wafian evidently creates Cineform .avi files or Quicktime files. It seems to have a good look at each frame and at the end of its journey through the clip writes a reference file for this damaged clip. The reference file and the original file must travel together and it is the reference file you work with. If you are lucky, this software may help you out. It is however a script based application. You have to go into Microsoft Command Prompt (MSDOS) to use it. I don't know if there is a Mac version. I was directed to it on the Silicon Imaging website. Last edited by Bob Hart; January 4th, 2009 at 03:41 AM. Reason: error

January 5th, 2009, 10:42 PM	#9
Scott Brickert Major Player Join Date: Jun 2003 Location: Crested Butte, CO Posts: 576	Funner and FUNNER Well, the hits just keep on rollin' So I read somewhere maybe it was a bad render. Deleted all renders and immediately exported to a self-contained QT-HDV. Still got a General Error and bad file. Next, I began the reconstruction. The final piece consists of three sections. The first three minute Intro, was exported to a reference QT-AIC file and imported back. Don't expect any trouble there. Next is a nested filed built of source and multiclips. I collapsed the Multiclips and exported it to self-contained QT-AIC in 5 minute segments--no crashes and no General Errors. Imported each segment and placed on the Master T/L above it's corresponding nested sequence. Finally I moved to the last piece, about 1h40m, where I deleted the Multiclip and replaced with the source clip, then exported in 10 minutes segments, importing those and placing on the Master T/L. This whole process took less than 4 hours, which I found surprising since exporting either nested piece had always taken 4-8 hours. Once complete, it flowed smoothly, as it always did in FCP (there never were any crashes when playing from any timeline in FCP, although there were hiccups until everything was rendered). So now I have an AIC 1440x1080i60 sequence, filled with media with the same format. I exported to a self-contained QT with current settings and markers for DVDSP. That took longer than expected, around 3 hours. I played the first 10 minutes in QT, and it looked fine. I imported that into Compressor for conversion to MPEG2. It was chugging along, foretelling about 3 hours to process. I expanded everything in the Batch Monitor, and saw 12 segments. At first 6 video and 1 audio streams were processing, which I also saw in Leopard's activity monitor. One process hung, so I forced it to Quit. The other continued. Eventually the audio finished successfully. Now we have 6 segments processing the first of the two passes (I'd selected a 2-pass transcode at around 6Mbps--the setting for Best Quality, 90 minute DVD, modified to 16:9), while the other six showed 'Waiting'. The total completion strip showed about 70% complete the last I looked. This seemed a bit much since there were several segments that had not yet successfully completed. The next thing I know, the entire Batch disappeared from the Batch Monitor. The Activity Monitor still shows qmasterd with 8 threads, one of which is Not Responding, two show zero, five show CPU around 150 (what does that mean, I thought it was percent of CPU usage, not to exceed 100?) and memory usage around 90MB. The CPU graph is almost all green, showing around 80% User and 15% System. There is still Read and Write activity going to the disks. If anyone can shed light on this, I'd be SO GRATEFUL. Should I let the processes run or kill them? Why does Batch Monitor show no activity? The destination folder shows segments 6,7,8,9,10, averaging around 500MB apiece. There's no sign of the final complete conversion, nor of segments 1-5,12. If this gets any more interesting, I'll ......... I"ll,,,,,,,,,,,, well, I'll just............... Apologies for the long involved post. What did I overlook? I didn't try reconnecting the files since none showed up missing when opening the project... OK, Activity Monitor still shows qmasterd but all processes show zero under CPU usage. Still can't find the final file, nor the AC3 file. Get Info show 46GB free on the drive. Any ideas? __________________ Scott Shot-By-Scott

January 2nd, 2009, 02:12 PM	#2
Battle Vaughan Trustee Join Date: Nov 2008 Location: Miami, FL USA Posts: 1,505	I'm just pulling a guess, here, but I think the reference movie part is the problem --- it's referring to files somewhere else. Why not export it as a self-contained .mov and see if that solves the problem? I'm assuming you are outputting the .mov from your FC timeline and all your clips are present? We have made (ok, single layer, I have no double layer experience) dvd's from hdv quicktime files and just let DVD Studio sort out the conversion with nice results....Battle Vaughan/miamiherald.com video team

January 2nd, 2009, 04:17 PM	#3
David Knaggs Major Player Join Date: Apr 2006 Location: Melbourne, Australia Posts: 826	Hi Scott. It's possible that one of the individual clips (or more than one) within your timeline is corrupted. It can happen occasionally with a clip captured as an .m2t file which is then made into a QuickTime movie with ClipWrap. Or at least this has occasionally happened to me. A solution, in such a case, is to re-capture the problem clip natively using FCP. Battle's suggestion of exporting as a self-contained movie is a good one, although I would recommend that you export it as a ProRes movie (it's best to avoid another compression with MPEG-2) and then import the ProRes movie into Compressor.

January 3rd, 2009, 04:33 PM	#6
Robert Lane Go Go Godzilla Join Date: Nov 2004 Location: Scottsdale, AZ USA Posts: 2,825 Images: 15	Scott, Try this: Go back to the timeline you're trying to output from, do a "select all" on the timeline and use the "reconnect media" function and see if anything comes up as missing in the timeline and if it does, reconnect it. Try copying the entire finished timeline to a new one, render it for playback "full render" and then output a reference file and see if it still fails. Make sure your timeline settings match exactly what you need first before importing. If possible, try encoding this in another program such as Episode Pro; if it fails in that app also then there's definitely a clip-corruption problem.

January 6th, 2009, 01:38 AM	#10
David Knaggs Major Player Join Date: Apr 2006 Location: Melbourne, Australia Posts: 826	Maybe take the self-contained AIC QuickTime movie straight to DVD SP and encode it there (bypassing Compressor). If DVD SP encodes it okay, then it's probably been a problem with Compressor. If DVD SP won't encode it either, then it's likely to be a problem with the movie. That's all I can think of.

January 14th, 2009, 10:29 PM	#11
Scott Brickert Major Player Join Date: Jun 2003 Location: Crested Butte, CO Posts: 576	Skipping Compressor and letting DVDSP render to MPEG2 worked! I delivered the DVD's today. What a relief. Compressor seems pretty finicky, or downright unreliable, but then again I haven't read the book yet. __________________ Scott Shot-By-Scott

January 16th, 2009, 12:45 AM	#12
David Knaggs Major Player Join Date: Apr 2006 Location: Melbourne, Australia Posts: 826	I'm very glad to find there was a happy ending! Thanks for posting the follow-up.