Tape or Tape-Less This time?

[Erik Norgaard]

Originally Posted by Gene Gajewski

On RAID hard drive systems, well, that's a fault tolerant *online* system - but I think the purpose of tape in terms of video is for permanent *archival* purposes - offline. It's not necessary to back up an entire computer to a tape drive - only the video files need be tranferred to tape, and as Keith pointed out, you can get many, many hours onto a tape.

Nothing keeps you from disconnecting your external RAID hard drive once full. and use it as cheap offline storage for permanent archival.

You don't consider your mini DV tapes an extra cost for backup because you already paid for the tape for recording in the first plase, where as you might think the hard drive is an overhead cost for archival purposes, yes sort of, but if you go tapeless you save the cost of the tape.

As I argued previously the cost of HDD storage is lower per hour recording than tapes, even when recording at a higher bit rate.

Originally Posted by Gene Gajewski

We can point out that hard drive storage is getting cheaper all the time. Of course, in an *offline* sense, who wants to store a bunch of USB drives at an offsite warehouse location? They're certainly more bulky than tape, and there's no economic benefit to the hard drive form as the likelyhood of actual usuage of the hard drive is low - we're merely archiving here and buying insurance. The tape is less bulky, and there's no electronics and little mechanicals to break down. (Of course *tape* drives can fail, but then so do hard drives - you simply replace them).

Read again: Your can store 83hs of video on a 2TB harddrive for less than $100, you can store the same on tape for $300, do you see that economic benefit? Who cares if the harddrive is connected or not. What you want is storage, cheap. Regardless of whether you read data, you use the disk for storage as much as you use the tapes for storage.

As for bulky, the right comparison would take into account the storage density. Compare the external 2TB harddrive that holds 83hs of recording with the 83 miniDV casettes and tell me which is less bulky.

For offsite storage, ok I'll give you: Your external hard drives are more likely to be stolen than obsolete miniDV tapes. Physical security is of course one of the parameters you must check when deciding how to keep your archives safe.

Originally Posted by Gene Gajewski

As for cloud based storage, it presents it's own issue to consider. For one, it's not a one time deal; you've got to keep on paying *rent* every month. Secondly, you dont *own* the data, meaning you don't control it. The cloud service you use could have a failure, the company could go out of business, etc. Third, cloud storage isn't exactly speedy.

Indeed you still own your data even if you store it externally, just as you have your copyrights and other rights protected. What you don't own is the storage media, but who cares, what matters is data availability, integrity and confidentiality, not who owns the hardware.

Yes, it comes at a subscription cost, and yes, for the amount of data it may not be suitable for you given the average connection speed. And evidently, before going with such a service you need to consider the pros and cons study their service level agreement etc...

Just as you have to do with any other storage solution you choose. Do you think the data center with multi-site storage, RAID, dual power, dual network, high availability, continously being upgraded is more prone to faults than the cardbord box you store your miniDV tapes in?

I agree, most won't need or pay for cloud storage, it's just yet another option you have.

Now, another question presses on: Which storage solution is better for long term archival? None actually.

1st: Technology evolves and whatever format you stored your data on will become obsolete. The device required to retrieve data will become obsolete and it will become costly to recover data. So, you need to keep your archives fresh and bring data with you onto new storage equipment if you want to be sure to keep it for the future.

2nd: How do you know your storage is actually good if you don't check it? You need to check your archives regularly to check that your data is readable. What do you do if you can't read data? Ups, we didn't think about that. Basically when you just archive, be it on tape or hard drive, without doing checks regularly or using some sort of redundant storage, then you blindly hope and trust your archive until the day you find you can´t.

BR, Erik

[Glen Vandermolen]

I've shot tapeless for the last 6 years with nary an issue. None, not once.
P2, SxS, CF, SD cards, Firestore, whichever, never an issue.
Tape is a dying medium. I would never invest in a tape based system.

But if you are still undecided, there are systems that allow you to record to tape and solid state at the same time. Consider the Sony Z5 or Z7. They have an MRC1 recording unit that attaches directly to the cameras, so you can roll on both HDV tape and the recorder at the same time. You can import the video from the hard drive to edit, and keep the HDV tapes as archive.
My guess is, once you go tapeless, you'll never go back to tape again.

Storage issues? The cost of 1TB portable hard drives is more than reasonable. Yiou can store a heck of a lot of video on those.

[Gene Gajewski]

Originally Posted by Erik Norgaard

Nothing keeps you from disconnecting your external RAID hard drive once full. and use it as cheap offline storage for permanent archival.

Of course not, but in a RAID configuration, you're storing multiple drives in some configurations. In addition, RAID drives aren't quite nearly cheap as consumer models, with the added expense of a drive configured to fit the RAID rack you're using.

Originally Posted by Erik Norgaard

You don't consider your mini DV tapes an extra cost for backup because you already paid for the tape for recording in the first plase, where as you might think the hard drive is an overhead cost for archival purposes, yes sort of, but if you go tapeless you save the cost of the tape.

Six of one, half dozen of another in this case. Either way, it's an expense. A RAID-able hard drive isn't going to have a cost advantage over tape - and it's more likely to *break down* when stored.

Originally Posted by Erik Norgaard

As I argued previously the cost of HDD storage is lower per hour recording than tapes, even when recording at a higher bit rate.

Barely, and only in a general sense, treating hard drives as a commodity. What's important is an examination of an archival system as a whole. As I pointed out, tape is more secure from a failure perspective vs single drives, and a RAID system completely loses any value if it's sitting in a warehouse - which is where we intend to place our long-term data storage.

We're talking about an archival *system*, not an online or near-line data system.

Originally Posted by Erik Norgaard

Read again: Your can store 83hs of video on a 2TB harddrive for less than $100, you can store the same on tape for $300, do you see that economic benefit?

No, but I can see you're not a very astute shopper. It took me all of two seconds to google up an LTO-5 media tape for less than 80 dollars. LTO-1 through four are even less, some under $25.

Originally Posted by Erik Norgaard

Who cares if the harddrive is connected or not. What you want is storage, cheap. Regardless of whether you read data, you use the disk for storage as much as you use the tapes for storage.

It matters very much if that hard disk sits unused for 10-15 years. Like automobiles, they have a tendency to fail if not used.

We don't intend to pull up that media and read it just for the sake of reading it. It's in long term storage for a reason - insurance. We may never read that tape again - but in the off chance that some opportunity arises that we could not have forseen, we can retrieve it. Instant access is not an issue, any expense to provide that capability are wasted in an archival application.

Originally Posted by Erik Norgaard

As for bulky, the right comparison would take into account the storage density. Compare the external 2TB harddrive that holds 83hs of recording with the 83 miniDV casettes and tell me which is less bulky.


For offsite storage, ok I'll give you: Your external hard drives are more likely to be stolen than obsolete miniDV tapes. Physical security is of course one of the parameters you must check when deciding how to keep your archives safe.

Bait and switch. We we're talking about LTO as an archival system, not DV.

Originally Posted by Erik Norgaard

Indeed you still own your data even if you store it externally, just as you have your copyrights and other rights protected. What you don't own is the storage media, but who cares, what matters is data availability, integrity and confidentiality, not who owns the hardware.

It's a matter of trust - and if you can't trust yourself to maintain an archival system, why would you trust anyone else?

Originally Posted by Erik Norgaard

Just as you have to do with any other storage solution you choose. Do you think the data center with multi-site storage, RAID, dual power, dual network, high availability, continously being upgraded is more prone to faults than the cardbord box you store your miniDV tapes in?

Assuming that I'd store them in a cardboard box, no. But no one with any business sense in their head would do that. They'd be stored at a temp-controlled facility.


Basically, media cost by itself is not the only factor to consider in an *archival* system. You must assign a valuation to the security and reliability of that media. Given that tape beats a hard drive system in long-term reliability, a raid system put in long term is *more* expensive vs tape - it's hard not to make a business case for LTO archival of video.

[Erik Norgaard]

Originally Posted by Gene Gajewski

We're talking about an archival *system*, not an online or near-line data system.
...
Bait and switch. We we're talking about LTO as an archival system, not DV.

Nope, stick to Original Topic:

If you're out to buy a new cam would you go tapeless? Going tape I understand miniDV, going tapeless requires some sort of alternative storage whether for intermediate NLE or for archival. HDD compared to miniDV for intermediate and short and midterm archival wins hands down on all parameters: cost, space, speed, convenience, efficiency etc.

miniDV is a dying format and for archival even if the tapes are stable for storage it still requires you to get hold of a reader, try friending a friends friend at a mayor broadcast station that have the equipment for reading ancient formats.

For long term archival purposes the problem I think is really not the stability of the the media but whether you can get access to it. You actually need to figure a way to keep your archive fresh, verify it's accessibility and integrity and move it on to new media as technology advances, or you might find yourself in a technological dead end with data on a media no one can read as the last vender is long gone out of business. Regardless of media stability I wouldn't expect to rely on it for more than 5-10 years.

And just an off-topic comment on LTO: The tape you found is about the same price/GB as the easily available hard disk based solution with RAID that I found - yes you can get portable external drives with RAID 1 for less than $100, no extra investment needed. But did you check the price of a tape station? $3-4000!

BR, Erik

[Pete Cofrancesco]

Originally Posted by Erik Norgaard

As regards extra encoding time, you should balance that against the extra transfer time for tape. If you shoot 7hs tape you need another 7hs to import it into your project. With CF this takes about an hour, so now you've got a 6hs head start. As long as you don't need to transcode to an intermediate codec I very strongly believe that the CF workflow will bring you results faster, and it just happens that today you can work fine in mpeg2 natively

I didn't fully explain my current workflow, you misunderstood me, and you last statement is wrong mpeg2 can't be imported into fcp it must be transcoded.

I currently shoot to tape while recording to a firestore. The tape is a live backup, the firestore records in dv, which can be edited natively without transcoding or edit it directly off the drive if you don't want take the time to copy the files to the computer. Now thats what I call a time saver. If the job is for an out of state client, we keep the files from the firestore and mail them the tapes. Can't do that with tapeless.

What I'm really mad about is a $4,000 camera that only records SD in mpeg2. That's a consumer format on a pro camera. Plain and simple you should be given the option to record in dv.

What I refer to as not worth paying for was, I charge $20 a dvd for standard def. I doubt I could pass on the extra cost of HD to a dance recital customer and charge them $40 for a blu-ray.

[Gene Gajewski]

Originally Posted by Erik Norgaard

Nope, stick to Original Topic:

Really?

I was responding to you in terms referencing your *first* response to *my* suggestion post - which was about considering going tapeless and using LTO for archival. In your initial response, you simply shot down the idea with some anecdotal observations about cheap hard drives, but nowhere in *that* post did you mention DV.

For long term archival purposes the problem I think is really not the stability of the the media but whether you can get access to it.

Well, without a stable media, you're left with nothing - zip, zero, zilch nada, no go. You can attempt to "access" all you want - but if you can't reproduce the data, you're dead in the water - SOL.

You actually need to figure a way to keep your archive fresh, verify it's accessibility and integrity and move it on to new media as technology advances.

There's no need need to keep an archive 'fresh'. I'm not sure where you're getting that idea from. Verification is done when a tape is created.

Now, this isn't to say that in 15 years or so, you might want to reproduce your archives on newer media, possible using newer technology to keep these archives accessible in the future. But we're talking a *long* time frame here

And just an off-topic comment on LTO: The tape you found is about the same price/GB as the easily available hard disk based solution with RAID that I found - yes you can get portable external drives with RAID 1 for less than $100, no extra investment needed. But did you check the price of a tape station? $3-4000!

Baloney! I'm not off topic, and it was *you* that responded to me on LTO.

As far as RAID one, it's a not a germaine issue. You'd only stash one drive since you're dealing with a mirrored configuration. Whether or not that drive came from a RAID system matters not.


As far as the as the cost of LTO tape drives - yes that is certainly not an insignificant factor. But the total cost of ownership is juxtaposed against the the total benefit derived. The fact that tape has superior permanency weighs against hard drive storage - giving it less of a return on investment.

Oh yeah, shopping again. Google. Search. Find. Enlightenment.

Bet you a wooden nickel if you can't find a drive for under 1800 - less than the cost of a gonzo PC workstation.

[Jim Andrada]

@Erik

Quote "Not to mention the overhead cost of retrieving data from tape backups: When you backup to tape you create a disk image on the tape, you have to read the entire disk image before you can retrieve the file you were looking for."

No longer true - LTO Gen 5 (particularly the IBM drives) supports a file system model called LTFS on the cartridge, ie you can drag and drop to tape and retrieve individual files.(I believe Quantum did something similar a couple of years back on DLT - which as tape formats go really is pretty dead)

To support the LTFS model they actually divide the tape itself into two partitions and implement a directory structure on one partition. There has also been some discussion about MXF awareness that would let you retrieve based on timecode ranges by storing the MXF meta data in the directory partition - don't remember exactly where this stands re being released. The IBM LTO development manager responsible for the LTFS (LTO File System) project will be at NAB so anyone who's there can talk to the guy who really knows.

Re price being prohibitive -see below. I don't think that's what I'd call prohibitive - maybe a little pricey, but no more so that some of the Flash memory add-ons for the camera

DELL LTO-3-060 LTO-3 EXTERNAL SCSI HALF HEIGHT TAPE DRIVE LVD BLACK

Disclosure - I've been working on LTO since well before its initial appearance in the market so I'm biased - very biased.

[Chris Harding]

Hi Guys

Going back to the original topic ..I went to card based cameras about two years back and I have never had a card problem nor a storage problem. Be aware that I shoot around 30 full weddings a season and then around 4 or 5 Realty Condition Reports a week so my cameras work hard!!!

All card contents are copied immediately to a 2TB external drive and when it's nearly full, I put it away and take out a new one...at under $100 it's secure and cheap and I never had a failure on a drive.

With my volume of work I made a rough estimate of tape/card usage and I probably save around $1000 a year by using cards (with tape I always used a new pro tape on every shoot)

Yes, I was very dubious but thanks to Bruce Foreman I took the plunge and never looked back!!!

I used Panasonic's HMC72 cameras for 18 months and now I have HMC82's (PAL models) I just happen to like Panasonic and like shoulder mount cameras!!!

If Darryn wants to go tape-less then go for it...and I'm sure he will have virtually no card or storage issues...I certainly haven't had any!!!

Chris

[Erik Norgaard]

Originally Posted by Pete Cofrancesco

I didn't fully explain my current workflow, you misunderstood me, and you last statement is wrong mpeg2 can't be imported into fcp it must be transcoded.

Sorry, I work with mpeg2 natively in FCP, it works.

HDV is mpeg2 and supported at least since FCP6, I just switched to tapeless HD and while the default transfer setting does transcode into ProRes422 you can change that, I import my mpeg2 and work on it natively without any problems. MPEG2 is not a consumer format, it's a well defined industry standard used by pros as well as consumers and accepted by major broadcast stations.

The shortcoming of mpeg2 is if you need to do colour correction and the like, this is particularly a problem shooting mpeg2 4:2:0. New cameras can shoot in 4:2:2. But as with anything, time is saved if you get it right while recording rather than relying on correcting bad footage in post. There are plenty of preset options to tweak that.

Originally Posted by Pete Cofrancesco

I currently shoot to tape while recording to a firestore. The tape is a live backup, the firestore records in dv, which can be edited natively without transcoding or edit it directly off the drive if you don't want take the time to copy the files to the computer. Now thats what I call a time saver. If the job is for an out of state client, we keep the files from the firestore and mail them the tapes. Can't do that with tapeless.
on a pro camera. Plain and simple you should be given the option to record in dv.

What I refer to as not worth paying for was, I charge $20 a dvd for standard def. I doubt I could pass on the extra cost of HD to a dance recital customer and charge them $40 for a blu-ray.

I'm departing from the situation of the original poster and can't really take into account the individual workflows. As I said before, if you've got a tape based workflow and you're happy with it, no need to change. But, consider this: If you're starting from scratch going to buy or upgrade all equipment without an established workflow and without the backward compatibility concerns, which is what I understand OP is: Would you decide for a tape based solution?

BR, Erik

[Brian Drysdale]

I've just come from a talk on tapeless work flow. Obviously they wanted to underline any problems, but hard drives aren't the safest way to store, unless they're used regularly the bearings can dry out and quite a few people have had one go and lost their data. They are also prone to mechanical damage and can become worn out like any other mechanical device.

The recommended method is to store on 3 hard drives, not one. For rushes, that's to be done during download from the original recording media and ideally they're to be stored in separate locations.

[Arnaldo Paixao]

Hi there.

I'v been using for some years now a tape based cam + a Firestore unit, and that has given me good results and peace of mind. If for some reason any of the Firestore files get corrupted, and it has ocurred more than once, I still have the tape as backup.

The posts by other members stating that they have not experienced problems with tapeless cams are reassuring, but, I like to play it safe. It gives me the shivers to think of this scenario: Shooting an event for hours, looking at your cam viewfinder, REC is on, timecode is runing, lighting is good, relax everything is good... Then, later on, wile trying to access the files in the card you realise in horror that you can't access the files. You try card recovery programs and still nothing because your memory card has died (or you just misplaced the card as they are realy easy to lose beeing small). That is the moment you wish that you had a backup.

So, ence my question, is there a tapeless cam or tapeless cam + accessory that would give me simultaneous recording to two medias (memory card + HDD or any combination of these)?

Would apreciate your insight on this.

Take care,
Arnaldo

[Erik Norgaard]

Originally Posted by Brian Drysdale

I've just come from a talk on tapeless work flow. Obviously they wanted to underline any problems, but hard drives aren't the safest way to store, unless they're used regularly the bearings can dry out and quite a few people have had one go and lost their data. They are also prone to mechanical damage and can become worn out like any other mechanical device.

The recommended method is to store on 3 hard drives, not one. For rushes, that's to be done during download from the original recording media and ideally they're to be stored in separate locations.

Hard drives as with any other type of storage may fail, failure may be local and affect only one or few files, or affect the whole drive, failures may be recoverable or not.

Whether it makes sense to have three master copies depends on your business, how much is that footage worth? How long do you expect to keep your archive? What's the probability of a failure? and what's the cost of mitigating the problem?

This is very subjective depending on what you do. But, those "invaluable" personal family videos are not invaluable after all.

BR, Erik

[Erik Norgaard]

Originally Posted by Arnaldo Paixao

Hi there.
So, ence my question, is there a tapeless cam or tapeless cam + accessory that would give me simultaneous recording to two medias (memory card + HDD or any combination of these)?

Canon XF series cameras have double slot CF and you can either record to one and continue on the other allowing you effectively indefinite uninterrupted recording, or record to both simultaneously for the added security. I don't think this feature is particular to these cameras.

BR, Erik

[Erik Norgaard]

Originally Posted by Jim Andrada

@Erik
No longer true - LTO Gen 5 (particularly the IBM drives) supports a file system model called LTFS on the cartridge, ie you can drag and drop to tape and retrieve individual files.

Re price being prohibitive -see below. I don't think that's what I'd call prohibitive - maybe a little pricey, but no more so that some of the Flash memory add-ons for the camera

DELL LTO-3-060 LTO-3 EXTERNAL SCSI HALF HEIGHT TAPE DRIVE LVD BLACK

Disclosure - I've been working on LTO since well before its initial appearance in the market so I'm biased - very biased.

You're mixing different LTO specs, LTO-3 drive cost and LTO-5 features.

LTO-3 have max 400GB per tape, don't know the tape price but even before you get your first tape you can have quite a beefy disk storage running for that money. LTO-5 and you get storage capacity at the same price as readily available disk drives, but the tape deck runs in thousands of dolars.

[Arnaldo Paixao]

Thanks for your reply Erik.

Looked up the Canon XF series and those cams would indeed give me the redundancy I was looking for.

Thanks again,
Arnaldo