How to check integrity of back-up of video project? at DVinfo.net

Adi Head · September 30th, 2011, 12:09 PM

I'm backing up a large amount of video files (about 240 GB) by just copying the files in Windows XP from one hard drive to another.
Is there a way to verify the integrity of all the files... some way to make sure that no video file somehow became corrupt during the transfer?

I've found a program called ExactFile which compares the contents of two different folders to see that they are exactly the same, but I don't know much about the techy side of how exactly it does this and what exactly it checks. Could there be a case where a program such as this verifies that two files are exactly the same, yet the video of one file will turn out bad when attempting to play/edit it?

Looking for suggestions from experienced members of the forum. Thanks!

Frans Meijer · October 1st, 2011, 05:24 PM

If you use the command-line and copy or xcopy you can add the /V switch to make them verify that the copy is identical to the original. There is also a command to compare files, FC (after File Compare), it compares two files, byte by byte and reports any differences found. I assume your program works in a similar way, if the to copies are not exactly the same, it will find the differences.

Adi Head · October 1st, 2011, 06:22 PM

Thanks Frans. I know what the command line is but I'm not familiar enough with how to use it... so I'm afraid I didn't quite understand your explanation on how to test the integrity of a copied file using xcopy or /v and that stuff (which I know nothing about).
But if you think the software I have is good enough - then that's fine. I just wasn't sure - if two files are incidental byte-byte whether or not it is necessarily an indication that one is not corrupt (specifically a video file).
I gather from your reply that the answer is 'yes'.
:)

Harm Millaard · October 2nd, 2011, 02:47 AM

Even easier: Beyond Compare - The file compare, folder compare and remote directory compare utility from Scooter Software

Ian Firth · October 3rd, 2011, 08:33 PM

Hi,

you can also download freeware MD5Sum.

This will let you calculate the 128-bit MD5 hash (i.e. checksum) of your source and copy and you can compare them. The program takes a while to run (my 5 GB video trial file took around 80% of one core of my i7 and ran for 2 mins 30 secs.

If I remember correctly you can compare at the file level or at the folder level, so one calculation for each would suffice. If there was a difference found, you could then track down to eventually the different file.

Once done, you could go back and check their integrity every week.

This process is obviously not automated and might not suit your needs/level of effort/inclination, but I hope it helps.

Cheers, Ian

Richard Davidson · October 28th, 2011, 06:54 PM

If you use Teracopy it will do a CRC check for you after the transfer is done, just hit the Verify button and it will make sure all match

Wesley Cardone · November 15th, 2011, 12:08 PM

I recommend that you consider backup software. Doing a file copy is hard as you have pointed out with the verification. I have found that routinely the file copy fails for unknown reasons in WinXP with the error message that disk is full. When your backup strategy is complicated the net result is that you neglect backups more. Dependable backups are so incredibly important for us in this business.

Frans Meijer · November 15th, 2011, 01:41 PM

I recommend not using backup software as their backups are typically a lot less convenient to restore files from. Backup software may be appropriate for special needs, but for video, plain explorer-work or command-line copy / xcopy work simpler and best.

But I agree, keep it simple and understandable, whatever backup software you choose, organise your data folders to make working with your clips and the backups as clear and straightforward as possible.

Jon Shohet · November 17th, 2011, 08:22 AM

Quote:

Originally Posted by Frans Meijer

I recommend not using backup software as their backups are typically a lot less convenient to restore files from. Backup software may be appropriate for special needs, but for video, plain explorer-work or command-line copy / xcopy work simpler and best.

That's true if you are referring to a "imaging" or "cloning" software, that backups your files into a single, propriety image file. However, many programs, usually labeled as "Sync" software, will make a plain vanilla backup of your files and folders, and will save you the insanity of manually going over all your files and figuring out what has changed and what to back up. Like the software Harm suggested. There are many more like it, some are even free. (for example : FreeFileSync).
I have been using "SyncBack Pro" Backup software that works - SyncBackPro and SyncBackSE for a good few years, and I couldn't imagine going back to backing up complex project folders with explorer/command line.

At the very least, I second the recommendation for "TeraCopy" as good replacement for the native windows copy.

September 30th, 2011, 12:09 PM	#1
Adi Head Major Player Join Date: Aug 2002 Location: Brooklyn, NY Posts: 642	How to check integrity of back-up of video project? I'm backing up a large amount of video files (about 240 GB) by just copying the files in Windows XP from one hard drive to another. Is there a way to verify the integrity of all the files... some way to make sure that no video file somehow became corrupt during the transfer? I've found a program called ExactFile which compares the contents of two different folders to see that they are exactly the same, but I don't know much about the techy side of how exactly it does this and what exactly it checks. Could there be a case where a program such as this verifies that two files are exactly the same, yet the video of one file will turn out bad when attempting to play/edit it? Looking for suggestions from experienced members of the forum. Thanks!

October 1st, 2011, 05:24 PM	#2
Frans Meijer Regular Crew Join Date: May 2011 Location: Netherlands Posts: 37	Re: How to check integrity of back-up of video project? If you use the command-line and copy or xcopy you can add the /V switch to make them verify that the copy is identical to the original. There is also a command to compare files, FC (after File Compare), it compares two files, byte by byte and reports any differences found. I assume your program works in a similar way, if the to copies are not exactly the same, it will find the differences.

October 1st, 2011, 06:22 PM	#3
Adi Head Major Player Join Date: Aug 2002 Location: Brooklyn, NY Posts: 642	Re: How to check integrity of back-up of video project? Thanks Frans. I know what the command line is but I'm not familiar enough with how to use it... so I'm afraid I didn't quite understand your explanation on how to test the integrity of a copied file using xcopy or /v and that stuff (which I know nothing about). But if you think the software I have is good enough - then that's fine. I just wasn't sure - if two files are incidental byte-byte whether or not it is necessarily an indication that one is not corrupt (specifically a video file). I gather from your reply that the answer is 'yes'. :)

October 2nd, 2011, 02:47 AM	#4
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	Re: How to check integrity of back-up of video project? Even easier: Beyond Compare - The file compare, folder compare and remote directory compare utility from Scooter Software

October 3rd, 2011, 08:33 PM	#5
Ian Firth Regular Crew Join Date: Jan 2007 Location: Ottawa, Ontario Posts: 31	Re: How to check integrity of back-up of video project? Hi, you can also download freeware MD5Sum. This will let you calculate the 128-bit MD5 hash (i.e. checksum) of your source and copy and you can compare them. The program takes a while to run (my 5 GB video trial file took around 80% of one core of my i7 and ran for 2 mins 30 secs. If I remember correctly you can compare at the file level or at the folder level, so one calculation for each would suffice. If there was a difference found, you could then track down to eventually the different file. Once done, you could go back and check their integrity every week. This process is obviously not automated and might not suit your needs/level of effort/inclination, but I hope it helps. Cheers, Ian

October 28th, 2011, 06:54 PM	#6
Richard Davidson Regular Crew Join Date: Oct 2011 Location: McAllen Texas Posts: 85	Re: How to check integrity of back-up of video project? If you use Teracopy it will do a CRC check for you after the transfer is done, just hit the Verify button and it will make sure all match

November 15th, 2011, 12:08 PM	#7
Wesley Cardone Regular Crew Join Date: Jul 2006 Location: Chelsea, Michigan Posts: 104	Re: How to check integrity of back-up of video project? I recommend that you consider backup software. Doing a file copy is hard as you have pointed out with the verification. I have found that routinely the file copy fails for unknown reasons in WinXP with the error message that disk is full. When your backup strategy is complicated the net result is that you neglect backups more. Dependable backups are so incredibly important for us in this business. __________________ Wesley Cardone www.WCardoneProductions.com, Detroit and Lansing, Michigan

November 15th, 2011, 01:41 PM	#8
Frans Meijer Regular Crew Join Date: May 2011 Location: Netherlands Posts: 37	Re: How to check integrity of back-up of video project? I recommend not using backup software as their backups are typically a lot less convenient to restore files from. Backup software may be appropriate for special needs, but for video, plain explorer-work or command-line copy / xcopy work simpler and best. But I agree, keep it simple and understandable, whatever backup software you choose, organise your data folders to make working with your clips and the backups as clear and straightforward as possible.