Archiving video.

[Eric Peltier]

Originally Posted by Sergio Perez

Really scary, Eric. Bottom line, what we all really needed is a DVCPRO HD deck, but that's just too expensive. How about renting a deck, buying some tapes and copy your cards back to p2, then use the camera's FW port to copy to the deck? I really don't know how much a dvcpro HD tape goes for, but, concerning safety, having the footage on a playable tape were you can visually see the footage on a deck, always seemed the most reliable...

My ideal solution was to have a MXF backup and and a dvcpro HD TAPE backup- but that's insane and overbudget...

Yep, as much as I like the P2 workflow, archival is definitelly the weak link.
I decided to invest in the new Quantum SDLT 600a (http://www.quantum.com/Products/Tape...00A/Index.aspx)
It's not ideal but it is to me the safest solution for long term (30 years) backup.
Make a tape and store on a shelf in a dry place, it will also save space in our server room from all the Raid we would have needed over time.
So, in the end it's ironical to me to still end in a tape in this tape-less workflow.
Again, this is what I think will work best for our post house, other people might prefer backing up to Raid or single hardrive, after all, it's a free country :)
e.

[David Tamés]

A very interesting thread.

And here's a very important point to remember: RAID 1 DOES NOT EQUAL BACKUP. RAID 1 is good to make it less likely that a drive failure will cause you to lose data, however, if you delete a file from a RAID 1 volume by mistake, it's gone.

A truly reliable ARCHIVE requires the use of something that lives offline that is not prone to accidental deletion or electro-mechanical device failure. Like DLT and LTO tape, or DVD-R, or Blu-Ray are all reasonably good archive mediums. It's also a good idea to keep them off site.

For ultimate protection, follow the rule of three: (1) working copy on hard drive, (2) hot backup on second hard drive (ideally set volume to READ-ONLY), (3) archive on tape. Statistically, it would be a very rare event for all three copies to become unreadable at the same time.

And...
One way to make RAID 1 ultra reliable is to use three disks and rotate one in and out of an offsite backup location. This third drive would not be prone to accidental human error deletions.

[Mike Schrengohst]

Originally Posted by Eric Peltier

Yep, as much as I like the P2 workflow, archival is definitelly the weak link.
I decided to invest in the new Quantum SDLT 600a (http://www.quantum.com/Products/Tape...00A/Index.aspx)
It's not ideal but it is to me the safest solution for long term (30 years) backup.
Make a tape and store on a shelf in a dry place, it will also save space in our server room from all the Raid we would have needed over time.
So, in the end it's ironical to me to still end in a tape in this tape-less workflow.
Again, this is what I think will work best for our post house, other people might prefer backing up to Raid or single hardrive, after all, it's a free country :)
e.

Hello Eric,
How much do you shoot? If P2 is a problem to archive how do you think the RED users will fare when they need to back-up 160 gig drives vs. a 4 gig P2 card?

[Eric Peltier]

We shoot quite a lot, long interviews, multicam concerts, and some narrative.
Just to give you an idea, I worked on a project last week where we ended up with 146 8gig content folders between 3 cameras. that's 1TB of masters right there.
Obviously RED users will face the same dilemma X 10.
The one thing I think most indie filmmaker (myself included) don't realize, is that a reliable archival process of a tapeless workflow is quite expensive in the end.
Unless you're David Fincher and have big studio money to back you off.
here's a interesting article on his latest film and tapeless workflow.http://digitalcontentproducer.com/vi...oing_tapeless/

[Eric Peltier]

Originally Posted by David Tames

A very interesting thread.

And here's a very important point to remember: RAID 1 DOES NOT EQUAL BACKUP. RAID 1 is good to make it less likely that a drive failure will cause you to lose data, however, if you delete a file from a RAID 1 volume by mistake, it's gone.

A truly reliable ARCHIVE requires the use of something that lives offline that is not prone to accidental deletion or electro-mechanical device failure. Like DLT and LTO tape, or DVD-R, or Blu-Ray are all reasonably good archive mediums. It's also a good idea to keep them off site.

For ultimate protection, follow the rule of three: (1) working copy on hard drive, (2) hot backup on second hard drive (ideally set volume to READ-ONLY), (3) archive on tape. Statistically, it would be a very rare event for all three copies to become unreadable at the same time.

And...
One way to make RAID 1 ultra reliable is to use three disks and rotate one in and out of an offsite backup location. This third drive would not be prone to accidental human error deletions.

David,
I just checked out Kino-Eye.com for the first time, great blog! full a great articles. thanks for the work.

[Mike Schrengohst]

Originally Posted by Eric Peltier

We shoot quite a lot, long interviews, multicam concerts, and some narrative.
Just to give you an idea, I worked on a project last week where we ended up with 146 8gig content folders between 3 cameras. that's 1TB of masters right there.
Obviously RED users will face the same dilemma X 10.
The one thing I think most indie filmmaker (myself included) don't realize, is that a reliable archival process of a tapeless workflow is quite expensive in the end.
Unless you're David Fincher and have big studio money to back you off.
here's a interesting article on his latest film and tapeless workflow.http://digitalcontentproducer.com/vi...oing_tapeless/

Yes I read that.....your heading into the great unknown....
Hopefully storage will only go down in price as capacity goes up.

[David Tamés]

For the indie producer, tape is still a viable "archive" format, however, with the constant drop in hard drive prices and Blu-Ray on the verge of becoming standard read/write media on new computers, I will say that 2007 will be "the year of tapeless workflow for indies" and I already think 2006 was "the year of tapeless workflow" for all but indie and low-budget producers.

Some hold on to tape, however, I live most comfortable on the leading edge just behind the bleeding edge. You sometimes get spashed with the blood but it's most often not your own. But I've seen old tapes go through video decks leaving much of the oxide inside the deck. No love for videotape here. On the other hand, hard drives die.

Maybe we can get 10 good years from Blu-Ray and then copy to something else, if what we have is worth preserving. Some say properly stored tapes are good for decades, but who's going to have the drives around? It's all a nightmare. Just ask any media archivist.

[David Tamés]

Originally Posted by Eric Peltier

David,
I just checked out Kino-Eye.com for the first time, great blog! full a great articles. thanks for the work.

Thanks for the shout out :-)

[Phillip Palacios]

What about the so-called "100 year" achival gold DVD-R's?

Anyone have info on how "archival" these really are?

[Dean Sensui]

I've had a commercially pressed CD go bad on me, and that CD wasn't handled all that much. I've also had various DVD-R's become unreliable over a few years.

Hard drives have failed. Tapes have gone bad.

The best bet is distributed storage. It would be impractical to make multiple copies of everything and put everything in dispersed locations, but for the really important stuff you might want to make copies onto a couple of different media just in case. And have a copy located elsewhere should a small disaster occur in your neighborhood.

[David Tamés]

Originally Posted by Phillip Palacios

What about the so-called "100 year" achival gold DVD-R's? Anyone have info on how "archival" these really are?

Ask me in a hundred years.

Seriously, in theory the use of gold instead of silver or aluminum as the reflective layer in a DVD provides better resistance to chemical breakdown from oxidation, which is a significant cause of disc failure, however, disc longevity is also threatened by delamination of the layers and fading of the dyes.

By using a combination of gold for the reflective layer, high quality bonding agents, and fade resistant dyes (e.g. Phthalocyanine) for the coloring layer, an "archive grade" DVD-R can be made that will probably last over 100 years. MAM-A (formerly known as Mitsui) claims to to make such a thing in their (relatively) new 4.7GB 8x Gold "Archive Grade" DVD-R. Controlling exposure to humidity, temperature, and light are still important factors in increasing disc life.

Since accelerated testing in the lab is never the same as the ravages of time, we'll only know for sure a hundred years from now, but by then, I'm confident we'll have a reliable alternative medium to which to copy our data to, assuming global warming or thermonuclear war does not make the whole practice of archiving moot.

You might be interested to know that the Library of Congress and National Institute of Standards and Technology (NIST) have been involved in an Optical Media Longevity Study and public test results will be available when the study is completed.

[Phillip Palacios]

Originally Posted by Dean Sensui

I've had a commercially pressed CD go bad on me, and that CD wasn't handled all that much. I've also had various DVD-R's become unreliable over a few years.

Hard drives have failed. Tapes have gone bad.

The best bet is distributed storage. It would be impractical to make multiple copies of everything and put everything in dispersed locations, but for the really important stuff you might want to make copies onto a couple of different media just in case. And have a copy located elsewhere should a small disaster occur in your neighborhood.

I see you are from Honolulu, I lived in Hilo on the Big Island for a while. The salt air and tropic humidity wreck havoc on anything there!

Of course "archival" grade anything can easily be ruined, when I took my still photography courses in college we used archival grade everything, but if you didn't rinse your prints long enough a year later, your prints turning brown. (I ALWAYS rinsed long enough, others...not so much)

I guess my question is more: has anyone used the "archival" gold DVD-r's or had any experiance with them?

sorry for the confusion.
Phil

[David Tamés]

Originally Posted by Phillip Palacios

[...] I guess my question is more: has anyone used the "archival" gold DVD-r's or had any experiance with them? [...]

I've burned some gold discs, however, I will probably not know in my lifetime whether they are truly archival or not. That's for the next generation to decide/discover.

[Jeff Kilgroe]

FWIW, I have some "archival 100+ year rated discs" that I've burned as far back as '93. They all still work... Then again, All my standard dye-based discs from that era still work too... With the exception of one 50-pack of discs from about '95 that didn't last. But those were weird -- half of them didn't work to start with.


For now, nobody knows anything and all these lifespans claimed by manufacturers are just a guess.

[David Tamés]

Originally Posted by Jeff Kilgroe

[...] for now, nobody knows anything and all these lifespans claimed by manufacturers are just a guess.

It's true that lifetime estimates are best-guesses based on accelerated longevity testing, however, I would not go so far as to say that nobody knows anything.

We know that some dyes are more fade resistant than other. We know that some bonding techniques are better than other. We know that gold makes a better reflective surface for archival storage. We know that not all disks are created equal and that we should choose the discs we use based on how they are manufactured and that MAM-A (formerly known as Mitsui) makes some very high-quality archive-grade discs.

And check out the link to the Optical Media Longevity Study in my earlier post. They will be publishing their results in the near future.