CRC is for wimps

Ola Christoffersson · January 21st, 2009, 04:15 PM

First of all - I knooow that you are supposed to activate CRC in Clipbrowser for safer copying, but c'mon really...

I have never ever had a problem with a file being copied in Windows without error messages and then later descover it to be corrupt!
Activating CRC just makes copying so much slower and since CB ver.1.0 did not even have this feature I find it really hard to motivate me to turn it on.

I know, I know...better safe than sorry. But still, is it worth it?

What do you guys think? What is really the risk of a copy going through without errors and later proving to have failed. Really?!

Bob Jackson · January 21st, 2009, 10:44 PM

Ask your clients if they will still pay you if something goes wrong?

Ross Herewini · January 22nd, 2009, 01:00 AM

Hi Ola,

In DOS days I was taught to do a copy with /v on the end to validate that the file was written correctly.

I assume the CRC is a similar concept, that the written file is checked for consistency as it is written.

As a minimum I would advise to use it, the consequences of finding out too late that it didn't work are somewhat serious.

Ola Christoffersson · January 22nd, 2009, 02:26 AM

You are of course right, both of you! I intentionally put it a bit harsh to see what reactions I'd get. I am aware of how disasterous a corrupt file could be. The reason I started this thread though was to find out what the actual real life risk is of corrupting a file not using CRC.
Has anyone at all had any corrupt files during the seven month when only ClipBrowser 1.0 was available and did not have this feature?
I'd be interested to now, so I can make a decision on weather to keep on living on the edge or start wearing a helmet and life vest.

Oh - and since you are following this thread Ross. Thanks for your great adapters! I got five of them a couple of weeks ago and I love them.

Vincent Oliver · January 22nd, 2009, 03:05 AM

I just lost the file numbers 0170 - 0184, Ok so I may not have selected all the files to copy over, just the ones that were visible at the time. In my case it was a simple shoot which can be repeated, but it has taught me a good lesson in ensuring everything is copied and verified.

In short, yes, CRC is worth turning on.

Shaun R Walker · January 22nd, 2009, 03:09 AM

I've never used Clipbrowser. I use XDCAM transfer which appears to have no backups in case of damaged data. I do however check each file after transfer and depending on the job copy over the BPAV folder to two separate HDD's.

David Issko · January 22nd, 2009, 05:56 AM

If you are in a position to have a MacBook Pro (I have the latest 15" version), then the SxS transfer to HDD via Clip Browser enabled with both Data Protection and CRC takes maybe about 45 mins for a full 32Gb card. It is much faster than from the camera.

I don't mind being a wimp when it comes to backing up my SxS recordings onto 2 Lacie drives via my MacBook Pro. I let the transfer operate in the background and then take care of another job in the meantime.

I trust everyone has a great 2009.

Craig Seeman · January 22nd, 2009, 07:14 AM

You can drives for YEARS without wearing a seatbelt and have no problem. It only takes one accident to be maimed or killed.

These are your camera masters. It only takes once to lose a client or to have to pay for a reshoot (and crew and rentals, etc).

CRC slows things down but it has almost zero time/dollar workflow issue. Even with SDHC and 2X with CRC on (that's how fast it is for me) you can offload and have it back in camera before your next card is full. CRC with SxS is even faster. So field work isn't impacted. In post work card offload (unlike tape) doesn't prevent you from using the computer for other functions and your still not going to screen "footage" much faster than real time in most cases so you can continue to screen while cards load. Once you hit 2x there's a diminishing return on faster speeds. Sure 7x-10x is nice but look at how we've been willing to lose that for SDHC and cost. Cost rules. Time is money too but how much time are your really saving once you hit 2x and both field and post work are minimally impacted.

If you are in a serious race against the clock and the clock is the boss in a given situation, take off the seatbelt but know that the car accident is often deadliest in this circumstance too. The clock wins if the file is corrupted.

Ola Christoffersson · January 22nd, 2009, 07:55 AM

I am happy I got this discussion going and I agree with all of you on safety being crucial. I have lost two 8 GB cards of footage in the 13 months I have owned my EX1 and both times it was due to human error.
I have two rugged USB-drives that I offload onto when travelling and I ALWAYS have at least two copies of everything. I also try to keep the material on my SxS/SDHC-cards until I have imported it into my Media Composer.

However, my workflow involves copying my media several times. First from the card to one of the drives. I then copy from one drive to the other using Windows (no CRC?!). When I get home I copy the footage from one of the USB-drives onto my footage RAID5-stack. And from there I convert it to MXF onto another drive. Finally I import into Avid.

It is not practiacl to use Clipbrowser when doing all of this copying mainly because it does not allow copying folders within folders so I am left with using Windows to copy.

Now - since I am trusting Windows when doing backups why shouldn't I trust normal file copying when offloading the cards?

Can anybody give me a techincal explanation of what can go wrong when NOT using CRC? Is Windows ever using CRC?

I guess what I am getting at is that I have a feeling CRC is there just because it is possible. Of course there can be a theoretical situation where CRC could detect an error but is there not a risk that these errors occurs anyway when backing up clips or importing MXF:s into your edit system.

Also - in what other situations is CRC being used? When burning a Bluray or DVD? When backing up to an external USB-drive? If not, these weak links should brake the chain.

Sorry if I am rambelling - I'd be happy if anybody with the technical know how could enlighten me!

Craig Seeman · January 22nd, 2009, 08:19 AM

Ola, use ShotPut if ClipBrowser doesn't fit your needs. ShotPut can copy to multiple locations and uses CRC.

Yes CRC check for errors. It's what it does and NO it's not something invented for Sony file transfers. It's an industry standard for safe and secure file copying.
http://en.wikipedia.org/wiki/Cyclic_redundancy_check

http://computer.howstuffworks.com/encryption7.htm

http://errorcrc.windowsreinstall.com/

http://www.softwarepatch.com/tips/cy...edundancy.html

I'm not going to recite ShotPut marketing material. Do read it.
ShotPut EXpress - Video Offloading for Mac and Windows

NEVER USE FINDER/EXPLORER unless you don't mind risking your masters. Double backup means nothing if the corruption is in both copies and depending on the cause, that can happen.

Ola Christoffersson · January 23rd, 2009, 02:14 PM

Ok - I've got CRC turned on again. But how do I backup a whole folder containing other folders from one disk to another safely. There is no way to do this using Clip Browse is there? I only seem to be able to copy one folder at the time. Is there any way to make Windows use CRC or is there a third party application I can use?

Vincent Oliver · January 23rd, 2009, 02:17 PM

Using ClipBrowser Open a folder containing your clips to be copied. Highlight one clip then press Ctrl + A to Select All. Now drag and drop all the selected clips to a destination folder. It's as simple as that.

Joe Lawry · January 23rd, 2009, 02:27 PM

Or just set you copy location in prefs and hit the 'All' Button..

Vincent Oliver · January 23rd, 2009, 03:53 PM

we learn something new each day - I never noticed that button before, maybe I will read the manual.

Thanks

Joe Lawry · January 25th, 2009, 01:34 PM

Heres a question for everyone, if we've got CRC checking on.. do you think the Priority in Move setting NEEDS to be set to Data Protection? or can we leave it set too processing speed?

January 21st, 2009, 04:15 PM	#1
Ola Christoffersson Major Player Join Date: Nov 2007 Location: Stockholm, Sweden Posts: 462	CRC is for wimps - or? First of all - I knooow that you are supposed to activate CRC in Clipbrowser for safer copying, but c'mon really... I have never ever had a problem with a file being copied in Windows without error messages and then later descover it to be corrupt! Activating CRC just makes copying so much slower and since CB ver.1.0 did not even have this feature I find it really hard to motivate me to turn it on. I know, I know...better safe than sorry. But still, is it worth it? What do you guys think? What is really the risk of a copy going through without errors and later proving to have failed. Really?! __________________ Ola Christoffersson, Kamrat produktion, Stockholm - Sweden www.kamrat.tv

January 21st, 2009, 10:44 PM	#2
Bob Jackson Major Player Join Date: Sep 2008 Location: Republic of Vancouver Island Posts: 200	Ask your clients if they will still pay you if something goes wrong? __________________ Any sufficiently advanced technology is indistinguishable from magic. AC Clarke

January 22nd, 2009, 01:00 AM	#3
Ross Herewini Sponsor: eFilms (MxR) Join Date: Jan 2008 Location: Sydney, Australia Posts: 171	Hi Ola, In DOS days I was taught to do a copy with /v on the end to validate that the file was written correctly. I assume the CRC is a similar concept, that the written file is checked for consistency as it is written. As a minimum I would advise to use it, the consequences of finding out too late that it didn't work are somewhat serious. __________________ Ross Herewini - E-films Makers of the new MxR Extreme

January 22nd, 2009, 02:26 AM	#4
Ola Christoffersson Major Player Join Date: Nov 2007 Location: Stockholm, Sweden Posts: 462	You are of course right, both of you! I intentionally put it a bit harsh to see what reactions I'd get. I am aware of how disasterous a corrupt file could be. The reason I started this thread though was to find out what the actual real life risk is of corrupting a file not using CRC. Has anyone at all had any corrupt files during the seven month when only ClipBrowser 1.0 was available and did not have this feature? I'd be interested to now, so I can make a decision on weather to keep on living on the edge or start wearing a helmet and life vest. Oh - and since you are following this thread Ross. Thanks for your great adapters! I got five of them a couple of weeks ago and I love them. __________________ Ola Christoffersson, Kamrat produktion, Stockholm - Sweden www.kamrat.tv

January 22nd, 2009, 03:05 AM	#5
Vincent Oliver Trustee Join Date: Sep 2004 Location: Chislehurst, London Posts: 1,724	I just lost the file numbers 0170 - 0184, Ok so I may not have selected all the files to copy over, just the ones that were visible at the time. In my case it was a simple shoot which can be repeated, but it has taught me a good lesson in ensuring everything is copied and verified. In short, yes, CRC is worth turning on. __________________ Eyes are a deaf man’s ears. Ears are a blind man’s eyes

January 22nd, 2009, 03:09 AM	#6
Shaun R Walker Regular Crew Join Date: Jan 2008 Location: Central Coast, NSW Australia Posts: 119	I've never used Clipbrowser. I use XDCAM transfer which appears to have no backups in case of damaged data. I do however check each file after transfer and depending on the job copy over the BPAV folder to two separate HDD's.

January 22nd, 2009, 05:56 AM	#7
David Issko Major Player Join Date: Jul 2008 Location: Melbourne, Australia Posts: 393	If you are in a position to have a MacBook Pro (I have the latest 15" version), then the SxS transfer to HDD via Clip Browser enabled with both Data Protection and CRC takes maybe about 45 mins for a full 32Gb card. It is much faster than from the camera. I don't mind being a wimp when it comes to backing up my SxS recordings onto 2 Lacie drives via my MacBook Pro. I let the transfer operate in the background and then take care of another job in the meantime. I trust everyone has a great 2009. __________________ David Issko Edit 1 Video Productions

January 22nd, 2009, 07:14 AM	#8
Craig Seeman Inner Circle Join Date: Feb 2004 Location: Brooklyn, NY, USA Posts: 3,841	You can drives for YEARS without wearing a seatbelt and have no problem. It only takes one accident to be maimed or killed. These are your camera masters. It only takes once to lose a client or to have to pay for a reshoot (and crew and rentals, etc). CRC slows things down but it has almost zero time/dollar workflow issue. Even with SDHC and 2X with CRC on (that's how fast it is for me) you can offload and have it back in camera before your next card is full. CRC with SxS is even faster. So field work isn't impacted. In post work card offload (unlike tape) doesn't prevent you from using the computer for other functions and your still not going to screen "footage" much faster than real time in most cases so you can continue to screen while cards load. Once you hit 2x there's a diminishing return on faster speeds. Sure 7x-10x is nice but look at how we've been willing to lose that for SDHC and cost. Cost rules. Time is money too but how much time are your really saving once you hit 2x and both field and post work are minimally impacted. If you are in a serious race against the clock and the clock is the boss in a given situation, take off the seatbelt but know that the car accident is often deadliest in this circumstance too. The clock wins if the file is corrupted.

January 22nd, 2009, 07:55 AM	#9
Ola Christoffersson Major Player Join Date: Nov 2007 Location: Stockholm, Sweden Posts: 462	I am happy I got this discussion going and I agree with all of you on safety being crucial. I have lost two 8 GB cards of footage in the 13 months I have owned my EX1 and both times it was due to human error. I have two rugged USB-drives that I offload onto when travelling and I ALWAYS have at least two copies of everything. I also try to keep the material on my SxS/SDHC-cards until I have imported it into my Media Composer. However, my workflow involves copying my media several times. First from the card to one of the drives. I then copy from one drive to the other using Windows (no CRC?!). When I get home I copy the footage from one of the USB-drives onto my footage RAID5-stack. And from there I convert it to MXF onto another drive. Finally I import into Avid. It is not practiacl to use Clipbrowser when doing all of this copying mainly because it does not allow copying folders within folders so I am left with using Windows to copy. Now - since I am trusting Windows when doing backups why shouldn't I trust normal file copying when offloading the cards? Can anybody give me a techincal explanation of what can go wrong when NOT using CRC? Is Windows ever using CRC? I guess what I am getting at is that I have a feeling CRC is there just because it is possible. Of course there can be a theoretical situation where CRC could detect an error but is there not a risk that these errors occurs anyway when backing up clips or importing MXF:s into your edit system. Also - in what other situations is CRC being used? When burning a Bluray or DVD? When backing up to an external USB-drive? If not, these weak links should brake the chain. Sorry if I am rambelling - I'd be happy if anybody with the technical know how could enlighten me! __________________ Ola Christoffersson, Kamrat produktion, Stockholm - Sweden www.kamrat.tv

January 22nd, 2009, 08:19 AM	#10
Craig Seeman Inner Circle Join Date: Feb 2004 Location: Brooklyn, NY, USA Posts: 3,841	Ola, use ShotPut if ClipBrowser doesn't fit your needs. ShotPut can copy to multiple locations and uses CRC. Yes CRC check for errors. It's what it does and NO it's not something invented for Sony file transfers. It's an industry standard for safe and secure file copying. http://en.wikipedia.org/wiki/Cyclic_redundancy_check http://computer.howstuffworks.com/encryption7.htm http://errorcrc.windowsreinstall.com/ http://www.softwarepatch.com/tips/cy...edundancy.html I'm not going to recite ShotPut marketing material. Do read it. ShotPut EXpress - Video Offloading for Mac and Windows NEVER USE FINDER/EXPLORER unless you don't mind risking your masters. Double backup means nothing if the corruption is in both copies and depending on the cause, that can happen.

January 23rd, 2009, 02:14 PM	#11
Ola Christoffersson Major Player Join Date: Nov 2007 Location: Stockholm, Sweden Posts: 462	Ok - I've got CRC turned on again. But how do I backup a whole folder containing other folders from one disk to another safely. There is no way to do this using Clip Browse is there? I only seem to be able to copy one folder at the time. Is there any way to make Windows use CRC or is there a third party application I can use? __________________ Ola Christoffersson, Kamrat produktion, Stockholm - Sweden www.kamrat.tv

January 23rd, 2009, 02:17 PM	#12
Vincent Oliver Trustee Join Date: Sep 2004 Location: Chislehurst, London Posts: 1,724	Using ClipBrowser Open a folder containing your clips to be copied. Highlight one clip then press Ctrl + A to Select All. Now drag and drop all the selected clips to a destination folder. It's as simple as that. __________________ Eyes are a deaf man’s ears. Ears are a blind man’s eyes

January 23rd, 2009, 02:27 PM	#13
Joe Lawry Major Player Join Date: Jul 2004 Location: Auckland, New Zealand Posts: 418	Or just set you copy location in prefs and hit the 'All' Button..

January 23rd, 2009, 03:53 PM	#14
Vincent Oliver Trustee Join Date: Sep 2004 Location: Chislehurst, London Posts: 1,724	we learn something new each day - I never noticed that button before, maybe I will read the manual. Thanks __________________ Eyes are a deaf man’s ears. Ears are a blind man’s eyes

January 25th, 2009, 01:34 PM	#15
Joe Lawry Major Player Join Date: Jul 2004 Location: Auckland, New Zealand Posts: 418	Heres a question for everyone, if we've got CRC checking on.. do you think the Priority in Move setting NEEDS to be set to Data Protection? or can we leave it set too processing speed?