HDSLR Archiving strategies at DVinfo.net

Jeremy Nicholl · January 24th, 2012, 07:14 AM

What is people’s archiving strategy for video shot on HDSLRs? I’m trying to put a rational system in place & this is what I’ve come up with:

Archive:
• All the original raw h264 clips from the camera & the separate audio recorder. That way if everything else gets lost, or better transcoders come along in the future, I still have all the basic material to go back to.
• The transcoded ProRes clips that are actually used in the final edit. These are larger than the originals & have had the audio track replaced, so although perhaps not essential, it seems good to keep these for convenience sake.
• The final FCP project file.
• The final video saved as QT movie; ie uncompressed, that can be done later as needed.

And three copies of everything in separate locations.

Does that sound about right? Overkill? Or have I missed anything?

Thanks in advance,

Jeremy Nicholl

Bill Pryor · January 24th, 2012, 10:10 AM

Here's what I do.

--I have a 1.5TB USB drive, and after the shoot I copy all my H.264 clips there.

--Then I immediately copy them to a separate firewire drive for editing. When I was using FCP, I transcoded to ProResHQ for that purpose but no need to with Premiere Pro.

So after the files are in two places, I reformat my cards.

As time permits, I take all the files from the USB drive and copy them to Blu-ray and store those safely. Those Blu-rays become like the old original tape.

I think it's best to store the original as what it is, ie., H.264, than the ProRes. What if you move to Avid or some other system where you might not have ProRes. Or in our case, we added another editor and new system with Premiere Pro, since FCP is no longer sold (except for FCPX which doesn't work for us). Without FCP on that system, the ProRes files wouldn't have worked.

For my purposes the Blu-ray storage is good enough. The likelihood of both my drives dying at the same time is not very high, so I feel pretty safe. I think Blu-ray discs, properly stored, will last as long as tape, maybe longer. A burner is under $200 now, and the discs are only a couple of dollars each.

When I travel on a shoot, I take my MacBook Air and a portable USB drive. At night I copy all the footage to the computer and then to the portable drive. Then reformat the cards if necessary. I've never run out of cards on an out of town shoot, but I always copy over and backup the footage anyway.

The weak spot in my system is that sometimes I get behind on making the Blu-rays. As I said, it's unlikely that both drives would fail at the same time...but anything that spins at 7200 rpm day in and day out is subject to failure. In the tape days we didn't have to worry about a tape dying, but with age came dropouts.

Greg Fiske · January 24th, 2012, 10:27 AM

In addition to what was stated, I do images of my dvd's and blu-rays. Couple years down the road you will get a call that the dvd was lost, etc. So I keep a master folder with all those so that I can easily create a new dvd/bluray.

Bill Pryor · January 24th, 2012, 02:31 PM

Good idea.

Taky Cheung · January 25th, 2012, 06:58 PM

I keep footage on two separate harddrives then pull them offline to store in different location. bare HDDs are cheaper (more expensive after the flood in thailand). Using them with HDD dock works well for me.

Trayless Removable Harddrive System | L.A. Color Blog

Roger Shealy · January 25th, 2012, 07:13 PM

I keep source files on one HDD, high quality H264 renders (15Mbps) of finished projects on another HDD and duplicate the most valuable renders onto SD cards. I also upload all renders less than 5GB to Vimeo so I can view or download from anywhere and serve as a cloud based back up.

Taky Cheung · January 25th, 2012, 07:14 PM

Footage uploaded to Vimeo will be transcoded I believe.

John Wiley · January 25th, 2012, 08:39 PM

One thing not mentioned yet, is that it can be very helpful to log/rename your footage before transcoding. That way if you somehow lose or if you choose to delete the ProRes/Cineform/intermediate files then need them later you can simply re-transcode them and point your NLE to the folder, rather than having to rename them and reconnect them one by one.

Roger Shealy · January 25th, 2012, 09:08 PM

Quote:

Originally Posted by Taky Cheung

Footage uploaded to Vimeo will be transcoded I believe.

Taky,

I just checked and downloaded a file from Vimeo and it returned the same file size as the original and looks identical when viewed on my monitor. Give it a try and see what results you get.

Taky Cheung · January 25th, 2012, 10:21 PM

Roger, oh yes you are right. Vimeo transcoded the video file for online streaming display, but it keeps your original uploaded file. The downloaded file would be the same file as the uploaded one.

Bob Krieger · January 26th, 2012, 11:12 AM

On location, I have a laptop, external eSata 1TB HDD and an eSata G-Raid external drive. Once a card is full (I use 16GB cards) I'll copy that card over to the G-Raid and the 1TB drive and then delete/reformat the card. I usually carry enough cards for two days of shooting, and will usually NOT reformat the card unless necessary on the shoot.

Back at the shop, I'll copy the files from either one of the external drives over to the RAID in my editing box. (It's a simple RAID 0). I also utilize a Drobo FS device for archiving, and copy the original camera files there as well. I'll edit using the files on the internal RAID and have a program called "Goodsync" that will automatically copy over any changes to the Drobo device. If something happens to the RAID, everything I'm working with is there and can easily be copied back again. If I lose anything, it'll only be about 5-10 minutes worth of work.

The 1TB size external drive, with the original camera files is then placed in storage offsite... Just in case. $89.00 approximately is a small price to pay to ensure files are safe. So, after a shoot and edit, I have the files and program files on my networked Drobo device and on a separate external drive off site. The final edited video is burned to multiple data DVDs, as well as copied to the Drobo and the offsite drive.

Finished video DVDs are also made and stored in both locations. It works well for me, keeps me relatively safe in case of fire and will make it easy to take with me in case of another evacuation like Katrina and Gustav.

Mike Watson · January 28th, 2012, 09:49 PM

Quote:

Originally Posted by Jeremy Nicholl

Archive:
• All the original raw h264 clips from the camera & the separate audio recorder. That way if everything else gets lost, or better transcoders come along in the future, I still have all the basic material to go back to.
• The transcoded ProRes clips that are actually used in the final edit. These are larger than the originals & have had the audio track replaced, so although perhaps not essential, it seems good to keep these for convenience sake.
• The final FCP project file.
• The final video saved as QT movie; ie uncompressed, that can be done later as needed.

I archive:
- The raw h.264 files
- The "media managed" FCP file with :01 handles on either end of the clip. This gives you just the media that you used (plus :01 on either side), plus a project file
- The final export.

I don't have triple backups, but I do have two (and both in the same location).

January 24th, 2012, 07:14 AM	#1
Jeremy Nicholl Regular Crew Join Date: Mar 2007 Location: Moscow, Russian Federation Posts: 50	HDSLR Archiving strategies What is people’s archiving strategy for video shot on HDSLRs? I’m trying to put a rational system in place & this is what I’ve come up with: Archive: • All the original raw h264 clips from the camera & the separate audio recorder. That way if everything else gets lost, or better transcoders come along in the future, I still have all the basic material to go back to. • The transcoded ProRes clips that are actually used in the final edit. These are larger than the originals & have had the audio track replaced, so although perhaps not essential, it seems good to keep these for convenience sake. • The final FCP project file. • The final video saved as QT movie; ie uncompressed, that can be done later as needed. And three copies of everything in separate locations. Does that sound about right? Overkill? Or have I missed anything? Thanks in advance, Jeremy Nicholl

January 24th, 2012, 10:10 AM	#2
Bill Pryor Contributor Join Date: Jan 2003 Location: Kansas City, MO Posts: 4,449	Re: HDSLR Archiving strategies Here's what I do. --I have a 1.5TB USB drive, and after the shoot I copy all my H.264 clips there. --Then I immediately copy them to a separate firewire drive for editing. When I was using FCP, I transcoded to ProResHQ for that purpose but no need to with Premiere Pro. So after the files are in two places, I reformat my cards. As time permits, I take all the files from the USB drive and copy them to Blu-ray and store those safely. Those Blu-rays become like the old original tape. I think it's best to store the original as what it is, ie., H.264, than the ProRes. What if you move to Avid or some other system where you might not have ProRes. Or in our case, we added another editor and new system with Premiere Pro, since FCP is no longer sold (except for FCPX which doesn't work for us). Without FCP on that system, the ProRes files wouldn't have worked. For my purposes the Blu-ray storage is good enough. The likelihood of both my drives dying at the same time is not very high, so I feel pretty safe. I think Blu-ray discs, properly stored, will last as long as tape, maybe longer. A burner is under $200 now, and the discs are only a couple of dollars each. When I travel on a shoot, I take my MacBook Air and a portable USB drive. At night I copy all the footage to the computer and then to the portable drive. Then reformat the cards if necessary. I've never run out of cards on an out of town shoot, but I always copy over and backup the footage anyway. The weak spot in my system is that sometimes I get behind on making the Blu-rays. As I said, it's unlikely that both drives would fail at the same time...but anything that spins at 7200 rpm day in and day out is subject to failure. In the tape days we didn't have to worry about a tape dying, but with age came dropouts.

January 24th, 2012, 10:27 AM	#3
Greg Fiske Major Player Join Date: Nov 2010 Location: Santa Ana, CA Posts: 499	Re: HDSLR Archiving strategies In addition to what was stated, I do images of my dvd's and blu-rays. Couple years down the road you will get a call that the dvd was lost, etc. So I keep a master folder with all those so that I can easily create a new dvd/bluray.

January 24th, 2012, 02:31 PM	#4
Bill Pryor Contributor Join Date: Jan 2003 Location: Kansas City, MO Posts: 4,449	Re: HDSLR Archiving strategies Good idea.

January 25th, 2012, 06:58 PM	#5
Taky Cheung Inner Circle Join Date: May 2007 Location: Los Angeles, USA Posts: 2,114	Re: HDSLR Archiving strategies I keep footage on two separate harddrives then pull them offline to store in different location. bare HDDs are cheaper (more expensive after the flood in thailand). Using them with HDD dock works well for me. Trayless Removable Harddrive System \| L.A. Color Blog __________________ LA Color Pros Blog RODE Authorized Reseller . Comer LED Camera Lights . TakyBox HTML5 Menu Generator

January 25th, 2012, 07:13 PM	#6
Roger Shealy Major Player Join Date: May 2007 Location: Fayetteville, GA Posts: 772	Re: HDSLR Archiving strategies I keep source files on one HDD, high quality H264 renders (15Mbps) of finished projects on another HDD and duplicate the most valuable renders onto SD cards. I also upload all renders less than 5GB to Vimeo so I can view or download from anywhere and serve as a cloud based back up.

January 25th, 2012, 07:14 PM	#7
Taky Cheung Inner Circle Join Date: May 2007 Location: Los Angeles, USA Posts: 2,114	Re: HDSLR Archiving strategies Footage uploaded to Vimeo will be transcoded I believe. __________________ LA Color Pros Blog RODE Authorized Reseller . Comer LED Camera Lights . TakyBox HTML5 Menu Generator

January 25th, 2012, 08:39 PM	#8
John Wiley Trustee Join Date: Oct 2008 Location: Byron Bay, Australia Posts: 1,155	Re: HDSLR Archiving strategies One thing not mentioned yet, is that it can be very helpful to log/rename your footage before transcoding. That way if you somehow lose or if you choose to delete the ProRes/Cineform/intermediate files then need them later you can simply re-transcode them and point your NLE to the folder, rather than having to rename them and reconnect them one by one.

January 25th, 2012, 10:21 PM	#10
Taky Cheung Inner Circle Join Date: May 2007 Location: Los Angeles, USA Posts: 2,114	Re: HDSLR Archiving strategies Roger, oh yes you are right. Vimeo transcoded the video file for online streaming display, but it keeps your original uploaded file. The downloaded file would be the same file as the uploaded one. __________________ LA Color Pros Blog RODE Authorized Reseller . Comer LED Camera Lights . TakyBox HTML5 Menu Generator

January 26th, 2012, 11:12 AM	#11
Bob Krieger Major Player Join Date: Jan 2010 Location: Abita Springs, LA Posts: 245	Re: HDSLR Archiving strategies On location, I have a laptop, external eSata 1TB HDD and an eSata G-Raid external drive. Once a card is full (I use 16GB cards) I'll copy that card over to the G-Raid and the 1TB drive and then delete/reformat the card. I usually carry enough cards for two days of shooting, and will usually NOT reformat the card unless necessary on the shoot. Back at the shop, I'll copy the files from either one of the external drives over to the RAID in my editing box. (It's a simple RAID 0). I also utilize a Drobo FS device for archiving, and copy the original camera files there as well. I'll edit using the files on the internal RAID and have a program called "Goodsync" that will automatically copy over any changes to the Drobo device. If something happens to the RAID, everything I'm working with is there and can easily be copied back again. If I lose anything, it'll only be about 5-10 minutes worth of work. The 1TB size external drive, with the original camera files is then placed in storage offsite... Just in case. $89.00 approximately is a small price to pay to ensure files are safe. So, after a shoot and edit, I have the files and program files on my networked Drobo device and on a separate external drive off site. The final edited video is burned to multiple data DVDs, as well as copied to the Drobo and the offsite drive. Finished video DVDs are also made and stored in both locations. It works well for me, keeps me relatively safe in case of fire and will make it easy to take with me in case of another evacuation like Katrina and Gustav.