View Full Version : Back up 5D footage
Anton Chernenko October 26th, 2010, 09:54 AM Hi guys
How do you back up your footage? I notice that all my backups are getting super heavy, as I convert files to PRORES for editing and also keep raw footage.
What is the best way? Keep them both or delete native files (since I use PRORES for projects and never use raw)
Perrone Ford October 26th, 2010, 10:53 AM Are you talking about backups (short term) or are you talking about archives(long term)?
Peer Landa October 26th, 2010, 11:19 AM How do you back up your footage? What is the best way? Keep them both or delete native files (since I use PRORES for projects and never use raw)
Since nowadays big external drives are quite affordable -- a drive with 2 terabytes can be had for less than $180 -- I just buy a drive for each project I'm working on and then just store/back it up that way.
-- peer
Anton Chernenko October 26th, 2010, 11:21 AM I guess I am talking about short term. For archiving I burn the raw files to dvds and keep it in several places as well as keep files on a separate hard-drive
Bill Pryor October 26th, 2010, 03:34 PM I load my original onto one drive and when I transcode to Prores, that goes to another drive, or drives. Then I backup the original to Blu-ray.
Harry Simpson October 26th, 2010, 03:50 PM I'm filling up several terabyte usb external drives where I have copies of both the RAW and the Converted files. It real kinda makes sense almost to just save the RAWs - you can always convert but you can't get a RAW from a converted file....
I know I have truckloads of wasted space in RAWs and converted for editing AVIs that will never see the light of day but storage is cheap now.....I shoot so much it's been years since DVD archival was even an option. As the Terabyte drives fill I retire one of the redundant pair and keep in a safe offsite place for safety.
Dave Partington October 27th, 2010, 07:05 AM I gave up buying more and more and more hard disks that I know will eventually fail because they are sat on the shelf not moving. Hard Disks need to be used or their bearings seize up. Sometimes that's quick (a couple of years) other times it takes longer. Either way, eventually they fail.
I made the investment in an LTO drive and now I simply back up to industry standard tapes that have a shelf life exceeding 20 years. Since taped are a third (or less) of the price of one HDD I can make three backups (different locations) for the same price and so have better redundancy that a single HDD can every provide.
Anton Chernenko October 27th, 2010, 09:47 AM I gave up buying more and more and more hard disks that I know will eventually fail because they are sat on the shelf not moving. Hard Disks need to be used or their bearings seize up. Sometimes that's quick (a couple of years) other times it takes longer. Either way, eventually they fail.
I made the investment in an LTO drive and now I simply back up to industry standard tapes that have a shelf life exceeding 20 years. Since taped are a third (or less) of the price of one HDD I can make three backups (different locations) for the same price and so have better redundancy that a single HDD can every provide.
wow, this sounds interesting!
Nigel Barker October 28th, 2010, 06:22 AM I gave up buying more and more and more hard disks that I know will eventually fail because they are sat on the shelf not moving. Hard Disks need to be used or their bearings seize up. Sometimes that's quick (a couple of years) other times it takes longer. Either way, eventually they fail.
I made the investment in an LTO drive and now I simply back up to industry standard tapes that have a shelf life exceeding 20 years. Since taped are a third (or less) of the price of one HDD I can make three backups (different locations) for the same price and so have better redundancy that a single HDD can every provide.
wow, this sounds interesting!Especially as earlier generations of LTO tape drives are available very cheaply on eBay. LTO-1 gives 100GB per tape & can be bought for $100. LTO-2 gives 200GB per tape & can be found for under $200.
The older drives don't have such a good transfer rate as LTO-3/4/5 so it takes longer to write each individual tape but 1-200GB would easily cover most of our projects.
Daniel Browning October 28th, 2010, 12:15 PM How do you back up your footage?
I hand it to my editor and never see it again. :) Actually, I do keep a lot of the footage. I have three 6TB external RAID-5 arrays that connect via E-SATA (much faster than USB or NAS), each array cost about $540 (with shipping). That comes down to about $0.09/GB, or $0.27/GB to backup in triplicate (two on-site, one off-site). So if my 5D2 MOV files average about 18 GB/hour, that's $5 per hour of footage backed up in triplicate, and I can backup up to 341 hours per array. The reason it's so cheap is that I use software raid:
Newegg.com - Mediasonic HF2-SU2S2 3.5" Black USB2.0 & eSATA Pro Box 4 Bay Enclosure (http://www.newegg.com/Product/Product.aspx?Item=N82E16817576001)
What is the best way? Keep them both or delete native files (since I use PRORES for projects and never use raw)
"Delete native files"? I don't know the meaning of them horrible words! :) Personally, I always delete the intermediates, because they are larger than the out-of-camera files and can be regenerated. Furthermore, unless the intermediates are truly lossless (prores is not for sure), then a certain amount of image quality will be lost forever.
Daniel Browning October 28th, 2010, 12:39 PM Since tapes are a third (or less) of the price of one HDD I can make three backups (different locations) for the same price and so have better redundancy that a single HDD can every provide.
Well, I wont comment on the reliability aspect, but I will say something about the cost. Every time I've looked into tape backup, it came out much more expensive.
HP LTO4 800 GB backup drive and SAS card: $2,400
20 tapes (16 TB): $600
=====
$3,000
That's $188/TB, over three times as much as 2TB hard drives:
External e-sata hard drive docking station: $30
Eight 2TB drives ($100/each at the time of this writing): $800
====
$830
The less tapes you use, the more lopsided the comparison:
HP LTO4 800 GB backup drive and SAS card: $2,400
7 tapes (5.6 TB): $175
=====
$2,575
That's $460/TB, while the 6TB of hard drives is only $55/TB, so in that scenario it's 8 times cheaper. The tape drive would have to be $155 in order to match the cost of hard drives for 6 TB of backup (2 TB in triplicate). But even the cheapest ones are ten times that amount.
Anton Chernenko October 29th, 2010, 02:25 AM I hand it to my editor and never see it again. :) Actually, I do keep a lot of the footage. I have three 6TB external RAID-5 arrays that connect via E-SATA (much faster than USB or NAS), each array cost about $540 (with shipping). That comes down to about $0.09/GB, or $0.27/GB to backup in triplicate (two on-site, one off-site). So if my 5D2 MOV files average about 18 GB/hour, that's $5 per hour of footage backed up in triplicate, and I can backup up to 341 hours per array. The reason it's so cheap is that I use software raid:
Newegg.com - Mediasonic HF2-SU2S2 3.5" Black USB2.0 & eSATA Pro Box 4 Bay Enclosure (http://www.newegg.com/Product/Product.aspx?Item=N82E16817576001)
"Delete native files"? I don't know the meaning of them horrible words! :) Personally, I always delete the intermediates, because they are larger than the out-of-camera files and can be regenerated. Furthermore, unless the intermediates are truly lossless (prores is not for sure), then a certain amount of image quality will be lost forever.
Sorry for confusing you, I delete native files from my laptop, but keep them in dvds and extra HDDs.
Marcello Mazzilli November 3rd, 2010, 04:20 AM This is my work chain
As soon as I shoot I copy the card contents to the hard drive
I then convert footage to ProRes but keep also the original
I then archive the original on BluRay
Then i edit in ProRes (sometimes footage is there for months)
Once the job is finished I trim the project keeping only used footage with some short handles
I then archive the job with the used/trimmed footage
I prefer not to use HD for space reasons mainly. I agree that today they are cost effective allthough you need to keep many jobs on the same HD multiplying the risk of damage. Blu Ray is probably not safe either but at least (in theory) if you brake one disk the others will work
Nigel Barker November 3rd, 2010, 06:08 AM Well, I wont comment on the reliability aspect, but I will say something about the cost. Every time I've looked into tape backup, it came out much more expensive.
<snip>
That's $460/TB, while the 6TB of hard drives is only $55/TB, so in that scenario it's 8 times cheaper. The tape drive would have to be $155 in order to match the cost of hard drives for 6 TB of backup (2 TB in triplicate). But even the cheapest ones are ten times that amount.You obviously missed my earlier post where I pointed out that earlier generations of LTO tape drives are available very cheaply on eBay. Even a brand new LTO-2 drive isn't much more expensive than your $155 target like this one for $276 delivered Dell 3-01032-04 LTO2 IBM SCSI LVD/SE DRIVE MODULE LTO2 - eBay (item 290470501480 end time Nov-29-10 20:15:41 PST) (http://cgi.ebay.com/Dell-3-01032-04-LTO2-IBM-SCSI-LVD-SE-DRIVE-MODULE-LTO2-/290470501480?pt=PCC_Drives_Storage_Internal&hash=item43a1641c68#ht_1804wt_1048)
Daniel Browning November 3rd, 2010, 09:11 AM You obviously missed my earlier post where I pointed out that earlier generations of LTO tape drives are available very cheaply on eBay.
Yes, and that would work great if 200 GB/tape is sufficient. For 2TB backed up in triplicate, that takes 30 tapes, compared to just three $99 hard drives. Personally, I have 6 TB backed up in triplicate, and I don't consider that to be a lot.
Nigel Barker November 3rd, 2010, 09:28 AM Yes, and that would work great if 200 GB/tape is sufficient. For 2TB backed up in triplicate, that takes 30 tapes, compared to just three $99 hard drives. Personally, I have 6 TB backed up in triplicate, and I don't consider that to be a lot.Used tapes can be found for less than $10 delivered but I can see that I am not going to convince you of the merits of tape:-) One of the biggest arguments in favour of tapes is their long shelf life & one of the biggest arguments against is speed compared to disks. Tapes are also nicer for backing up on a per project basis as 200GB would amount to about 10 hours of video from a Canon DSLR which is far more than we would have for a typical project but I would be reluctant to use just one hard disk per project as it would be more expensive & bulky.
Dave Partington November 3rd, 2010, 05:47 PM Well, I wont comment on the reliability aspect, but I will say something about the cost. Every time I've looked into tape backup, it came out much more expensive.
HP LTO4 800 GB backup drive and SAS card: $2,400
20 tapes (16 TB): $600
=====
$3,000
That's $188/TB, over three times as much as 2TB hard drives:
External e-sata hard drive docking station: $30
Eight 2TB drives ($100/each at the time of this writing): $800
====
$830
The less tapes you use, the more lopsided the comparison:
HP LTO4 800 GB backup drive and SAS card: $2,400
7 tapes (5.6 TB): $175
=====
$2,575
That's $460/TB, while the 6TB of hard drives is only $55/TB, so in that scenario it's 8 times cheaper. The tape drive would have to be $155 in order to match the cost of hard drives for 6 TB of backup (2 TB in triplicate). But even the cheapest ones are ten times that amount.
If all you ever need is 16TB then tape is not for you. I have much more than 16TB and it's growing rapidly. HDDs are simply not cost effective once you get in to large amounts of footage.
Also, recalculate for more than one backup, maybe three. So 16TBx3 = 48TB. Now imagine 64TBx3 = 192TB. Hard Disks = $10,560
The Tape solution (using your numbers) = $2,400 + $7,200 = $9,600.
Now imagine 150TB or 300TB backup up x3.
If you don't shoot much, or don't need to retain footage for more than the length of the project then tape is probably not for you. If you need to archive footage for future reference then tape is a viable and possible cheaper solution than hard disks.
Imagine broadcasters than need to store petabytes of footage. Do you imagine they have lots of hard disks sat on a shelf? ;)
|
|