Repairing Corrupt AVI when Capture is nuked by a crash at DVinfo.net

Roger Moore · October 2nd, 2005, 08:03 PM

File under: Hope this helps someone.

I was using DVRACK to capture when suddenly the laptop ran out of power and died. Upon rebooting I found that the video inside my DVRACK folder was still there but it was corrupt because none of the players (bsplayer, mcplayer, not even vlc) could open it, "Could not render stream". Virtualdub could not open it either: movi list missing. I tried every avi repair tool that I could find but none of them worked.

I was about to delete this file but decided before doing that - it was 2hrs worth of video - I would google around a bit, ie. procrastinate. The best info I found was at AVIEDIT's forum http://www.am-soft.ru and http://www.pk3.org/K3CCDTools/

Neither of them had that magic repair program that would instantly fix my AVI, but at least they showed me what I had to do - namely, because my RIFF headers did not get written when the laptop crashed I would have to go into the file with a HEX editor and replace the missing headers.

The best advice came from the creator of AVI Repair Tool http://www.pk3.org/K3CCDTools :

"The principle of the utility (AVI Repair Tool) is copying of good AVI header from another (good) AVI file. The source file must have the same resolution and must have at least the same number of frames (no problem to use larger AVI files).

If you don't have source AVI file, you can create it by capturing any object now."

But it won't work if you use his program because the software is buggy :(

I could open the file in Virtualdub's hex editor and see that the file was missing the movi list even after the headers had been supposedly "repaired" by AVI Repair Tool. The Tool needs some work, but the logic behind it is excellent - I followed that logic to fix my AVI.

Now here's what you need to do to repair an AVI file that is missing headers because of a crash during capture:

1. Get a Hex Editor that can 1. Open huge files (my AVI was 22 gb), and 2. Have the ability to cut and paste...Virtualdub's built-in Hex editor handles big files but it can't hilight/cut and paste across pages and pages of data. I used HexEdit.

2. In Hex editor, open the corrupt AVI in one window and open the good AVI (mine was 27 gb) in another window.

IMPORTANT STEP:

3. Copy ALL the headers from the good AVI and Paste exactly over the corresponding data (check location on the left column) in the corrupt AVI. Sure it's the wrong headers but don't worry, Virtualdub will take care of that detail later.

Save the AVI.

It's about 40 pages worth of headers, so forget about using VdB's hex editor, or else you'll be typing "01 00 EA 14 DF 69 00-30 etc." for weeks...I did the character by character replacment for about an hour before scrolling down to see the dozens of pages of headers left to go and realized...I'm still on page two.

4. Virtualdub: File/Open/extended options, open the corrupt AVI...

If Virtualdub still says MOVI list missing, go back to step 3 and compare headers from the good and bad AVI's, maybe you overlooked one line.

5. Tick on reflag keyframes, then OK

6. Vdub will start repairing AVI! Then it will lay down the keyframes (this step takes longer).

About 2hrs later it will give you the report of what it did:

AVI: Index not found or damaged -- reconstructing via file scan
AVI: Invalid chunk detected at 511589384. Enabling aggressive recovery mode.

7. Your file is back!

IMPORTANT STEP:

8. Remember to save the new AVI, if you close Virtualdub now you will lose all the changes and have to redo the recovery thing again. Use direct stream copy for both video and audio.

THE END

ps

I did all that and yet at every moment I was ready to throw in the towel and delete the file...especially after performing step 3 several times and still virtualdub kept saying MOVI list was still missing...DELETE IT NOW! Even though at no point was I convinced that I'd get the file back, I persisted for some reason, stupidity probably. Or maybe I'm a sucker for lost causes.

Chris Yole · October 28th, 2005, 02:52 AM

hi
to cut a long story short, i recovered some avi files from my damaged partition and tried playing them.
No luck.
I managed to play the first minute or so after trying to fix them with several avi fixers.
Now after using hex edit and following your instructions, virtual dubmod says there is a missing movi block.
There is a movi block in hex edit.
Any ideas?
Bearing in mind i am a complete novice at this.
Is it likely that the files are too badly damaged for repair?

Anyway if you need more info i will try to pass on if you can help.

Thanks in advance.

Chris

Daniel Kohl · October 28th, 2005, 10:31 AM

You guys are madmen!

Thanks for posting this experience! I can't make any constructive comment or suggestion to this process, I can just relate, and commend you for your persistence. It will pay off some day, in some unexpected way.

This situation reminds of the cartoon of a frog who has already been halfway swallowed by a stork, but the frog is just far enough out of the storks beak to grab the stork by the throat, preventing the stork from completely swallowing.

I would also like to use this occasion to highlight one of the advantages to capture devices that split the capture files into lots of small files. It looks like a mess after you have done a long capture, but it has the great advantage of that if there is an interruption in recording, only the last file is corrupted.

Good luck to all those you who have to resort to these measures to recover their video footage. At least there is hope though (right?).

Peter Ferling · October 28th, 2005, 12:19 PM

So what's the $6 tape for? Hmmmm.

I always run tape, even if I'm capturing to disk. One out of every three shoots I get a faulty capture, be it a hiccup in the labtop, loose cable, whatever. On the other hand, I get a few dropped frames on the tape (including a few eaten and requiring some mending). In any case, I get a complete project between the two methods.

If the shoot it THAT important, I'll have a second camera on hand with a wider shot and pulling the same audio. An extra 50$ in rental fee for a single chip DV has saved my butt before, and makes for a nice edit when switching views to keep the show interesting.

Bottom line. Direct to disk is a timesaver, but it's not a replacement for tape, (those of you naysayers, ready to fire back - don't worry I was on your side once, until Murphy's law struck me sqaure in the middle of a live event...) Fool me once...

Roger Moore · October 29th, 2005, 10:41 PM

Chris: My avi was corrupt because the headers were not written, but everything else was where it was supposed to be, ie. other than missing headers it was a clean avi.

Your avi sounds like it may be corrupt in other areas of the avi, ie the main body of the avi and not just the headers. If that is the case I'm not sure that the above technique would work. You'd be able to play the file up to the corrupt part I guess.

HOWEVER, the symptom you've described, that you can play the first minute before it stops, sounds like what my file did after the first couple of tries I made replacing the headers. Virtualdub said missing MOVI. So you'll have to copy ALL of the MOVI headers from the good avi that you made. Even if the corrupt file seems to have MOVI headers they are probably bad, hence the error. GOOD LUCK!

Peter: seems like I'm following your example for I have now stopped using dv rack and moved over to tape. Not just because of what happened, but because it was getting really tiring carrying a laptop.

My camera is a board camera (ccd, lens, all on a board the size of a stamp) that I connected to a Canopus firewire analog/digital converter and that to the laptop.

For simulatneous tape/hd capture I could've plugged the camera into my camcorder and then the camcorder to the laptop as my camcorder has analog/digital passthru, but did not do this for some reason. Even if I had, I'm not sure if my camcorder is capable of recording auxillary analog signal to tape AND outputing this signal via the firewire *at the same time*. Never tried it yet, and probably not going to as there's no way I'd consider carrying all that with me, the laptop was already too much!

Anyways I've decided to simplify and just use the board camera with a handycam. Or I could get the firestore!

Like this:

Board Camera > Canopus analog/digital converter > Firestore.... Naaaah, forget about it.

Daniel: You are right. Capturing to several smaller avi's would have saved me all that trouble, but for technical reasons this was not *practical*. It was possible, yes, but to do this I would have had to stop what I was doing, take the laptop out of the backpack, and access DVRACK, hilight the clip, press delete, etc..possible, but not practical.

Though, since that time I've discovered X-keys and Genovation programmable keypads...with those wired devices you can keep the laptop in the bag, just hit a key on the device to execute a macro that would perform the commands. But their smallest units (16 keys for the x-keys, 24 keys for the genovation) are slightly too big to fit into a shirt pocket - but maybe not! If I find something similar but smaller, say with 3 keys, I might give this another go!

Daniel Kohl · October 30th, 2005, 03:47 AM

Quote:

Originally Posted by Roger Moore

Daniel: You are right. Capturing to several smaller avi's would have saved me all that trouble, but for technical reasons this was not *practical*. It was possible, yes, but to do this I would have had to stop what I was doing, take the laptop out of the backpack, and access DVRACK, hilight the clip, press delete, etc..possible, but not practical.

Hi Roger,

I was referring to file size restriction because of the FAT format on the FS-4. There has been some criticism of this limitation which causes the FS-4 to break long takes into lots of individual files. Your situation illustrates how this "limitation" can be an asset in a crisis.
I am not familiar with DV Rack because I am mac user, but it would surprise me if there was not the option to set a file size limitation, forcing DV Rack to break the files into smaller packages. FCP offers this option when capturing.

Quote:

Originally Posted by Roger Moore

Though, since that time I've discovered X-keys and Genovation programmable keypads...with those wired devices you can keep the laptop in the bag, just hit a key on the device to execute a macro that would perform the commands. But their smallest units (16 keys for the x-keys, 24 keys for the genovation) are slightly too big to fit into a shirt pocket - but maybe not! If I find something similar but smaller, say with 3 keys, I might give this another go!

Have you considered using an USB mouse with one of the buttons assigned to a key combination which executes a command. I mention doing this in the a previous thread.

http://www.dvinfo.net/conf/showthrea...t=frankenstore

The thread gets into the triggering of the laptop somewhere around #7.

November 12th, 2005, 02:33 PM

About six months ago, I used DV Rack to record a long training course and had an AVI file corrupted and unretrievable. It was a long file (over an hour) but was critical to what I was doing. I could see the file was there, but couldn't get it play.

I contacted Serious Magic who told me to send them my hard drive and they would look it over. Previous to sending them the drive, I'd used various tools in an attempt to retrieve the file.

The SM folks were very interested in looking at what had happened because they thought that they'd fixed the problem of "long catures" (e.g., an hour) in the most recent release, but apparently hadn't.

A couple of points:

The Serious Magic folks were absolutely super folks to work with...fixed my file, returned the drive and even threw in a T-short to boot - even called to make sure the drive arrived back safely. All in all, superb customer service.

The second point - from a production standpoint, head off the problem at the pass...keep your capture files "smaller". Using DV Rack, hitting "F2" (the keyboard shortcut to "record") will create a new file that will edit seamlessly to your last file. I've found that about every 20 minutes or so I reach over and hit F2 and all has been well. You can do it more often (or less) but I've found 20 minutes works for me.

FWIW,
Guy Smith

Wade Spencer · May 15th, 2007, 12:46 PM

Bumping up an old thread here because this just happened to me...ugh.

I've downloaded Hexedit, but I can't seem to open anything up in that software.

Is there anyone who has done this that I could contact by phone to possibly walk me through it? I've called Adobe, but they are no help whatsoever. I'm not even sure the tech support person I talked to even knew what the program did.

Michael Conn · July 12th, 2007, 06:57 AM

I am also looking to repair an avi header. I can't get a hex editor to open the file. My laptop powered off before finishing the avi (windows media). The file is 97 gigabytes. Is there a program available ($50 or less) to repair headers? I've tried some shareware programs but they didn't work.

October 2nd, 2005, 08:03 PM	#1
Roger Moore Regular Crew Join Date: Jun 2004 Location: toronto Posts: 99	Repairing Corrupt AVI when Capture is nuked by a crash File under: Hope this helps someone. I was using DVRACK to capture when suddenly the laptop ran out of power and died. Upon rebooting I found that the video inside my DVRACK folder was still there but it was corrupt because none of the players (bsplayer, mcplayer, not even vlc) could open it, "Could not render stream". Virtualdub could not open it either: movi list missing. I tried every avi repair tool that I could find but none of them worked. I was about to delete this file but decided before doing that - it was 2hrs worth of video - I would google around a bit, ie. procrastinate. The best info I found was at AVIEDIT's forum http://www.am-soft.ru and http://www.pk3.org/K3CCDTools/ Neither of them had that magic repair program that would instantly fix my AVI, but at least they showed me what I had to do - namely, because my RIFF headers did not get written when the laptop crashed I would have to go into the file with a HEX editor and replace the missing headers. The best advice came from the creator of AVI Repair Tool http://www.pk3.org/K3CCDTools : "The principle of the utility (AVI Repair Tool) is copying of good AVI header from another (good) AVI file. The source file must have the same resolution and must have at least the same number of frames (no problem to use larger AVI files). If you don't have source AVI file, you can create it by capturing any object now." But it won't work if you use his program because the software is buggy :( I could open the file in Virtualdub's hex editor and see that the file was missing the movi list even after the headers had been supposedly "repaired" by AVI Repair Tool. The Tool needs some work, but the logic behind it is excellent - I followed that logic to fix my AVI. Now here's what you need to do to repair an AVI file that is missing headers because of a crash during capture: 1. Get a Hex Editor that can 1. Open huge files (my AVI was 22 gb), and 2. Have the ability to cut and paste...Virtualdub's built-in Hex editor handles big files but it can't hilight/cut and paste across pages and pages of data. I used HexEdit. 2. In Hex editor, open the corrupt AVI in one window and open the good AVI (mine was 27 gb) in another window. IMPORTANT STEP: 3. Copy ALL the headers from the good AVI and Paste exactly over the corresponding data (check location on the left column) in the corrupt AVI. Sure it's the wrong headers but don't worry, Virtualdub will take care of that detail later. Save the AVI. It's about 40 pages worth of headers, so forget about using VdB's hex editor, or else you'll be typing "01 00 EA 14 DF 69 00-30 etc." for weeks...I did the character by character replacment for about an hour before scrolling down to see the dozens of pages of headers left to go and realized...I'm still on page two. 4. Virtualdub: File/Open/extended options, open the corrupt AVI... If Virtualdub still says MOVI list missing, go back to step 3 and compare headers from the good and bad AVI's, maybe you overlooked one line. 5. Tick on reflag keyframes, then OK 6. Vdub will start repairing AVI! Then it will lay down the keyframes (this step takes longer). About 2hrs later it will give you the report of what it did: AVI: Index not found or damaged -- reconstructing via file scan AVI: Invalid chunk detected at 511589384. Enabling aggressive recovery mode. 7. Your file is back! IMPORTANT STEP: 8. Remember to save the new AVI, if you close Virtualdub now you will lose all the changes and have to redo the recovery thing again. Use direct stream copy for both video and audio. THE END ps I did all that and yet at every moment I was ready to throw in the towel and delete the file...especially after performing step 3 several times and still virtualdub kept saying MOVI list was still missing...DELETE IT NOW! Even though at no point was I convinced that I'd get the file back, I persisted for some reason, stupidity probably. Or maybe I'm a sucker for lost causes.

October 28th, 2005, 02:52 AM	#2
Chris Yole Tourist Join Date: Oct 2005 Posts: 1	missing movi block hi to cut a long story short, i recovered some avi files from my damaged partition and tried playing them. No luck. I managed to play the first minute or so after trying to fix them with several avi fixers. Now after using hex edit and following your instructions, virtual dubmod says there is a missing movi block. There is a movi block in hex edit. Any ideas? Bearing in mind i am a complete novice at this. Is it likely that the files are too badly damaged for repair? Anyway if you need more info i will try to pass on if you can help. Thanks in advance. Chris

October 28th, 2005, 10:31 AM	#3
Daniel Kohl Contributor Join Date: Dec 2001 Location: Frankfurt, Germany Posts: 571	You guys are madmen! Thanks for posting this experience! I can't make any constructive comment or suggestion to this process, I can just relate, and commend you for your persistence. It will pay off some day, in some unexpected way. This situation reminds of the cartoon of a frog who has already been halfway swallowed by a stork, but the frog is just far enough out of the storks beak to grab the stork by the throat, preventing the stork from completely swallowing. I would also like to use this occasion to highlight one of the advantages to capture devices that split the capture files into lots of small files. It looks like a mess after you have done a long capture, but it has the great advantage of that if there is an interruption in recording, only the last file is corrupted. Good luck to all those you who have to resort to these measures to recover their video footage. At least there is hope though (right?). __________________ Daniel Kohl Frankenstein meets XL1

October 28th, 2005, 12:19 PM	#4
Peter Ferling Trustee Join Date: May 2005 Location: Wyomissing, PA Posts: 1,141 Images: 57	So what's the $6 tape for? Hmmmm. I always run tape, even if I'm capturing to disk. One out of every three shoots I get a faulty capture, be it a hiccup in the labtop, loose cable, whatever. On the other hand, I get a few dropped frames on the tape (including a few eaten and requiring some mending). In any case, I get a complete project between the two methods. If the shoot it THAT important, I'll have a second camera on hand with a wider shot and pulling the same audio. An extra 50$ in rental fee for a single chip DV has saved my butt before, and makes for a nice edit when switching views to keep the show interesting. Bottom line. Direct to disk is a timesaver, but it's not a replacement for tape, (those of you naysayers, ready to fire back - don't worry I was on your side once, until Murphy's law struck me sqaure in the middle of a live event...) Fool me once...

October 29th, 2005, 10:41 PM	#5
Roger Moore Regular Crew Join Date: Jun 2004 Location: toronto Posts: 99	Chris: My avi was corrupt because the headers were not written, but everything else was where it was supposed to be, ie. other than missing headers it was a clean avi. Your avi sounds like it may be corrupt in other areas of the avi, ie the main body of the avi and not just the headers. If that is the case I'm not sure that the above technique would work. You'd be able to play the file up to the corrupt part I guess. HOWEVER, the symptom you've described, that you can play the first minute before it stops, sounds like what my file did after the first couple of tries I made replacing the headers. Virtualdub said missing MOVI. So you'll have to copy ALL of the MOVI headers from the good avi that you made. Even if the corrupt file seems to have MOVI headers they are probably bad, hence the error. GOOD LUCK! Peter: seems like I'm following your example for I have now stopped using dv rack and moved over to tape. Not just because of what happened, but because it was getting really tiring carrying a laptop. My camera is a board camera (ccd, lens, all on a board the size of a stamp) that I connected to a Canopus firewire analog/digital converter and that to the laptop. For simulatneous tape/hd capture I could've plugged the camera into my camcorder and then the camcorder to the laptop as my camcorder has analog/digital passthru, but did not do this for some reason. Even if I had, I'm not sure if my camcorder is capable of recording auxillary analog signal to tape AND outputing this signal via the firewire at the same time. Never tried it yet, and probably not going to as there's no way I'd consider carrying all that with me, the laptop was already too much! Anyways I've decided to simplify and just use the board camera with a handycam. Or I could get the firestore! Like this: Board Camera > Canopus analog/digital converter > Firestore.... Naaaah, forget about it. Daniel: You are right. Capturing to several smaller avi's would have saved me all that trouble, but for technical reasons this was not practical. It was possible, yes, but to do this I would have had to stop what I was doing, take the laptop out of the backpack, and access DVRACK, hilight the clip, press delete, etc..possible, but not practical. Though, since that time I've discovered X-keys and Genovation programmable keypads...with those wired devices you can keep the laptop in the bag, just hit a key on the device to execute a macro that would perform the commands. But their smallest units (16 keys for the x-keys, 24 keys for the genovation) are slightly too big to fit into a shirt pocket - but maybe not! If I find something similar but smaller, say with 3 keys, I might give this another go!

November 12th, 2005, 02:33 PM	#7
Guy Smith Posts: n/a	Serious Magic helped me out About six months ago, I used DV Rack to record a long training course and had an AVI file corrupted and unretrievable. It was a long file (over an hour) but was critical to what I was doing. I could see the file was there, but couldn't get it play. I contacted Serious Magic who told me to send them my hard drive and they would look it over. Previous to sending them the drive, I'd used various tools in an attempt to retrieve the file. The SM folks were very interested in looking at what had happened because they thought that they'd fixed the problem of "long catures" (e.g., an hour) in the most recent release, but apparently hadn't. A couple of points: The Serious Magic folks were absolutely super folks to work with...fixed my file, returned the drive and even threw in a T-short to boot - even called to make sure the drive arrived back safely. All in all, superb customer service. The second point - from a production standpoint, head off the problem at the pass...keep your capture files "smaller". Using DV Rack, hitting "F2" (the keyboard shortcut to "record") will create a new file that will edit seamlessly to your last file. I've found that about every 20 minutes or so I reach over and hit F2 and all has been well. You can do it more often (or less) but I've found 20 minutes works for me. FWIW, Guy Smith

May 15th, 2007, 12:46 PM	#8
Wade Spencer Major Player Join Date: Sep 2005 Location: Wellsboro, Pa Posts: 285	Bumping up an old thread here because this just happened to me...ugh. I've downloaded Hexedit, but I can't seem to open anything up in that software. Is there anyone who has done this that I could contact by phone to possibly walk me through it? I've called Adobe, but they are no help whatsoever. I'm not even sure the tech support person I talked to even knew what the program did.

July 12th, 2007, 06:57 AM	#9
Michael Conn New Boot Join Date: Sep 2006 Location: Cowpens, South Carolina Posts: 8	I am also looking to repair an avi header. I can't get a hex editor to open the file. My laptop powered off before finishing the avi (windows media). The file is 97 gigabytes. Is there a program available ($50 or less) to repair headers? I've tried some shareware programs but they didn't work.