DVCPRO HD storage for masters

[Pete Bauer]

One vote for Dan as the Most Organized Pakrat of the Year!

Dude, your diligence in actually organizing, backing up, and safely archiving your footage (not just saying you should get around to it, like me), forces me to hang my head in shame.

My poorly executed plan is to:
- keep all raw footage tapes
- leave anything I might want to use on the hard drive
- back up key stuff to DVD-R, including complex processing intermediates (quicker and easier to save the files than recapture from tape)
- back up all but the really useless raw footage and final products to tape

Sadly, most of that is still on my to-do list...after barely dodging Hurricane Rita, archiving all our photos and videos HAS moved up the list, though!

[Edwin Hernandez]

I know that most people think CD's or DVD's are a better way to archive over any form of tape. And I do think the same except for one thing: I've seen older discs that all of a sudden are not recognized by a newer computers, or whose files are now not supported or corrupted. It happened to me with some photographs that were scanned and everytime I try to open them, they just don't. In the case of tape, maybe not with the same quality, but you can save a portion or some of it.

Also, optical media has gone sooooo cheap, that it is hard for the bad brand to go any cheaper than the good brand and all of a sudden you get confused on which one to buy, you buy the one available because it costs almost the same as the good one and there you go... trusting your work to a $0.60 disc!

So... how reliable are optical media nowadays?

[Steev Dinkins]

I vote for Dan as well. I try for triple backup, but damn man, you are the master at mastering. :P

I know this may sound ridiculous, but the crazy part about all of this is to attempt to archive, to hold onto things "forever". Meanwhile, I'm starting to get the real odd sense that the lifetime is flying by real fast and soon it's really not going to matter anymore. From there, it will be up to our ancestors/archivists to either hang onto our work, figure out how to decipher, use, repurpose, present our work, or just junk it.

So the question remains, what is a viable working solution for the present day?

DVDs, Hard Drives, and Tape.

I haven't had many CDs or DVDs fail me. When I converted my CDs this year, the oldest one was from 1998 I believe. So that's 7 year reliability. If you have redundant copies, I think you're pretty safe there.

Hard Drives, I have less experience leaving them alone for years, then cranking them up and using them. Anyone with more experience with this?

Regarding tape, we all know tape can really get screwed up over time, magnetic field hashing, signal loss, tape surface sticking, and with one bad slip on a deck, the old "this thing ate my f'n tape!!" I think tape is solid if stored correctly (at a former job, they had the tape archive around where the janitor would come by twice a week and neerrooom past with the vacuum!). But I am intrigued by Dan's notion of renting a deck and archiving once or twice a year. Great idea, and free up the hard drives space.

[Jeff Kilgroe]

Originally Posted by Damon Botsford

Jeff,
I have a bunch of 8mm video tapes which are close to 12 years old. They contain priceless family footage that I don't want to risk losing. You think they have already started to degrade considerably?

...Simple laws of physics - yes, they have degraded. How far gone they are will depend on lots of factors like the environment they were stored in. Airtight boxes are one thing, but there's also the consideration of temperature variations, electromagnetic fields and radio waves. Even what the tapes themselves are made of... Some of the real old tapes in my archives even show signs of the plastic case and tape material starting to break down. I've got some old 8mm and beta tapes as well as some audio cassettes that had clear plastic shells on them that have really turned yellow now that they are 15~20+ years old. Even being stored in a cool, dry environment away from light.

If you have taken care of your tapes, they should last a considerable amount of time, but I would be a little nervous about magnetic media of that age. You may want to take a look at what's on those tapes and see for yourself... It may be time to get that video archived digitally before its gone forever.

Originally Posted by Damon Botsford

I wonder how vulnerable HD-DVD's will be for archiving? It appears that one of the formats has a protective case for the disk.

HD-DVD will have the same shelf life of current DVD-R/RW as it will be constructed of the same materials and will just carry a much smoother substrate surface for the laser to write to. Writable Blu-Ray media is still somewhat of a mystery. 25GB Blu-Ray drives are seeing limited availablilty in some Japanese markets, they have been for nearly a year now. Sony has been beta testing them with larger corporations and educational institutions. They claim 100+ years the shelf-life of their media.

Originally Posted by Damon Botsford

I also wonder if the disk coating durability has improved. The 15 year shelf life you mentioned for regular DVD's is scary. I'm curious what it actually is in a protected environment.

Shelf life of current CD and DVD media is estimated to be 100 years or more (both for the writeable -R and +R variants as well as the manufactured disc types. The 15 year lifespan I mentioned is that's all anyone can confirm in reality at this point for CD-R/DVD-R. CD-Rs have only been around for about 15 years and they really didn't become a consumer product until about 10 years ago. We won't know for at least another 80 years or so whether or not our current CD/DVD media really has a 100 year shelf-life, will we? Laboratory simulations and stress testing conducted by the manufacturer can only give us an estimate. These manufacturers are all claiming this 100+ year shelf-life for disc media even though several of them use different types of dyes and films to create their discs... None of them really offer an ironclad guaranty either. In 100 years are people going to file suit against DVD-R manufacturers because great-grandpa's DVD home movies don't play anymore? Doubtful... 100 years from now, DVD will be so obsolete that if any still exist in working order, they will be relics in an antique shop. The reality of digital storage solutions is that an archive of your digital video or other data should continuously evolve with the available standards. I have gone through all the work to convert all my analog video into digital form. Even the old 8mm and other formats of filmstrips shot by my grandparents when I was still a wee lad.

But what we do know for sure right now is that CD-R and apparently DVD-R can be stored at least 15~20 years without worry and the manufacturers claim 100+ years. You won't see any such claims for a magnetic tape media. Tape media consists of a sprayed-on metallic coating (that's way thinner than a human hair) on a plastic film holding a magnetic charge. The odds of this holding that charge stable for extended periods of time (10 to 20 years or more) is highly unlikely. Magnetic charges on microscopic particles will tend to shift, bleed and crawl over time. The magnetic charge or polarization of the particles in these coatings can also dissipate. Tapes, if properly stored, should last at least 10 years... 20 years is possible, but on an analog recording like your old 8mm tapes, I will be very surprised if after 20 years you don't see a noticeable amount of signal decay.

[Jeff Kilgroe]

Originally Posted by Kevin Shaw

It's not that simple because the codecs don't work the same way. Also note that DVCProHD has fewer recorded pixels per frame at 1080i resolution, so that's a trade-off in terms of image quality. Overall DVCProHD should yield better image quality, but nowhere near four times the perceived quality.

Yes, I simlified or generalized my statement too much. The perceived quality level of HDV is going to be a lot closer to DVCProHD than what the bitrates want to indicate. But I was attempting to look at it from an overall information point of view... 100Mbps vs. 25Mbps still allows for 4x the data flow when comparing DVCProHD to HDV. In some ways DVCProHD isn't as efficeint as MPEG2 and that helps bridge the gap between the two formats somewhat. HDV does allow for more pixels, but also sacrifices color information. DVCProHD will provide a lot more information to work with in terms of compositing or other post production tasks. I don't expect the HVX200 to rival the Varicam in total image quality, the lower cost fixed lens and smaller sensor block probably won't allow for that. But I also expect the HVX200 to knock the socks off of any HDV solution out there right now or in the near future. Canon's XLH1 will probably provide a superior solution for certain tasks, but to take full advantage of that camera, you'll need an external deck as opposed to just recording to HDV.

[Graeme Nattress]

I can't render HDV back to HDV even when there's no filters applied without seeing artifacts increase. DVCproHD to HDV would stand up better than this, but once you've gone down that low bitrate MPEG2 path, it's very hard to come back.

Graeme

[Brian Sargent]

Well, I can lend a little perspective to this discussion since I am in the archive business.

Funny that someone brought up painters- there is a major show of Van Gogh's drawings at the Met right now and they are worth quite a bundle. Of course he didn't see any $, but we get to see some amazing examples of a working process. Another interesting thing to me is that B&W pics of my grandfather when he was a infant are better preserved than the Sears Portrait Color pics of my sister and I from the 60's (if Sears had done Dye Transfer we'd be ok). Most of them are almost entirely faded and if the technology hadn't come along when it did they would have been totally gone in a few short years.

I've also worked in the News Biz and we are now making money off our archives. Trouble is you cannot sell footage from a pre-produced package because of duration of the shots and change in editorial POV. The increase in the amount of production companies and outlets for content are driving footage sales right now and clients are looking for high quality images that also don't look overproduced, so your b-roll can potentially be very valuable. I would save it if at all possible.

Dan has it covered about as well it can be right now and from my perspective he has the potential to make as much money with his outtake material if he owns the rights to it as he does from the final product.

Just something to think about...

[Kevin Shaw]

Originally Posted by Jeff Kilgroe

...I also expect the HVX200 to knock the socks off of any HDV solution out there right now or in the near future.

We'll see, we'll see. We'd certainly hope it will be better, but since it's still a 1/3" sensor with (apparently) 720 pixel vertical resolution and probably a comparable lens to current HDV cameras, I'm not sure the difference will be all that significant. Color depth should be better and artifacts should be fewer, with better ability to handle motion, but will that translate to significantly better quality by the time it gets to the viewer's screen? Can't wait to see some video samples...

[Graeme Nattress]

Even 720p without pixel shift vertically is the equal of 1080i in resolution due to the interlace factor to stop 1080i flickering. It is thought that the cam will use both horizontal and vertical pixel shift, although this is not confirmed by any means. Having the largest pixels of the bunch should make for better contrast / noise / low light, and that will be visible, but we've got to wait and see what design decisions were actually made.

Graeme

[David Andrews]

Originally Posted by Damon Botsford

Jeff,
I have a bunch of 8mm video tapes which are close to 12 years old. They contain priceless family footage that I don't want to risk losing. You think they have already started to degrade considerably? I've always kept them safe in a dry, airtight box. I don't have nearly enough drive space to digitize all of them right now. I'll probably have to wait for HD-DVD's. .

In the past year I have had 35 to 40 year old super8 film converted to miniDV for editing and saving to DVD. These old films survived quite well. If there is any doubt about the survival of your 8mm video tapes I would think it worth getting them converted to (at least) miniDV. I used a professional service to do mine.

[Paul Doherty]

One angle not covered in this thread, but which is perhaps worth mentioning is that in my experience there is a huge difference between the archival quality of a cheap DVD burnt on a cheap burner compared to quality media burnt on a good burner.

I have got a Lite-On drive which I use for its bitsetting abilities, but the burns it produces (even with good quality media) are seldom of archival quality.

For archival stuff I stick to my Pioneer drives and I test all the burns with Nero DVD Speed (though DVDInfoPro or KProbe will also do the job). A few get rejected, but I do then have reasonable confidence in the discs that are left.

[Edwin Hernandez]

How about a glass master?
Instead of just "burning" it into a polycarbonate DVD, would it be more reliable to take it to a replication facility and have them make a glass master, as if you were going to replicate discs (but of course you wouldn't have to replicate any)?

-EDWIN

[Kevin Shaw]

Originally Posted by Graeme Nattress

Even 720p without pixel shift vertically is the equal of 1080i in resolution due to the interlace factor to stop 1080i flickering...Having the largest pixels of the bunch should make for better contrast / noise / low light...

But then the JVC HD100 also has a 720p sensor and recording, so any advantages to that should come across there as well. Point being that it's not clear the HVX200 will "knock the socks off" of HDV, but we'll find out soon enough. If I was prepared to spend close to $10K for an HD camera, I might be inclined to wait for next year's models with larger sensors.

[Graeme Nattress]

When shot in a very controlled manner, the JVC can make for a very nice image, but the stock manual lens is rather poor and holds back the quality of the camera, and the HDV compression really doesn't show the pictures it can make in it's best light. I'd expect the HVX to be at least as good in controlled conditions and noticibly less compromised in more demanding conditions - and if not, I won't be buying one. Should be getting a HVX soon for testing....

Graeme

[Steven Thomas]

Graeme, Looking forward to you sharing your thoughts on the HVX200!
I sure hope it lives up to some or most of the hype.

Steve