I am so excited about this camera that I could pop! I will be getting mine at the beginning of the year hopefully and am looking forward to all of the footage that will be hitting the net. One question that comes up a lot, but gets lost fairly quickly, is archiving. I've though about buying lots of hard-drives, but those can fail and aren't convenient to hand over to clients. I don't believe that Sony currently has a deck that is affordable(for us small guys) and will back up your 35Mbps footage in 1920x1080. I've thought about LTO drives which could be a great option, but they are very expensive and would work great for personal use, but not for clients. I've been reading about write-once compact flash cards that could be very handy and easy to pass out to clients but I haven't gotten any pricing or capacity information yet. So, all of you early adopters, what are your plans for backing up all of your footage? I would like to hear both about your archiving solutions, and also about deliverable solutions for clients.

I am so excited about this camera that I could pop! I will be getting mine at the beginning of the year hopefully and am looking forward to all of the footage that will be hitting the net. One question that comes up a lot, but gets lost fairly quickly, is archiving. I've though about buying lots of hard-drives, but those can fail and aren't convenient to hand over to clients. I don't believe that Sony currently has a deck that is affordable(for us small guys) and will back up your 35Mbps footage in 1920x1080. I've thought about LTO drives which could be a great option, but they are very expensive and would work great for personal use, but not for clients. I've been reading about write-once compact flash cards that could be very handy and easy to pass out to clients but I haven't gotten any pricing or capacity information yet. So, all of you early adopters, what are your plans for backing up all of your footage? I would like to hear both about your archiving solutions, and also about deliverable solutions for clients.

Burning a 8GB card to DL DVD would work.

60-80 Gb 2.5" usb2 drives are fairly cheap, $50-$70 - it seems like these could be good for handing off a day's footage to the client at the end of the day.

I would like to hear both about your archiving solutions, and also about deliverable solutions for clients.

Even Achilles had a weak spot.

LOL! I hear what you are saying because it feels like a weak spot to me. Although I am thrilled to see tape go, it was a fairly good archive situation at a cheap price. Of course, now I want full-raster 1920x1080, and I want to record in in HQ 35Mbs because that is the "best", and I am not sure how to store it all.

Considering all of my projects would at least take a minimum of 10 DVDs of raw footage, I would need to rent space to archive all of the footage I would shoot in a year(and a filing system equivalent to a working library). USB drives are a good cost/GB solution, but they are still a rotating platter that if damaged or dropped you would loose your work.

I am leaning heavily towards LTO drives if I could find a decent used one. But these cameras are getting bought up by the truckload, so for all of the pros buying 8 of these, what is your archiving and deliverable workflow going to include?

I've just bought a Western Digital "My Book" 2 Terrabyte Ext. hard drive which I have mirrored so I have one Terrabyte available for AU$799.
(If one drive gives up the ghost, you can put in a new one and it can repair).

This seems to be a good solution for me as I have two copies and it's under AU$1.00 a gig.

As I do mainly events, once I have finished the projects I only keep a master disc and don't need the original footage again.

Cheers Vaughan

Thanks for your input Vaughn. If I were just working on one project at a time, I think this would be a great solution, and one that will work for many. But I have been toying with the idea of doing some stock footage work which would mean that I would have to be able to access all of my footage from a library which is why I am looking for a long term archiving solution.

Were looingk at using a server with a mirrored raid setup. SATA disc is cheap so you can get quite a big storage.

For safety were thinking of using Blue Ray Disc or XDCAM HD Disc that we put on a shelf in case the mirrored drive fails or if a file get corrupted.

A mirrored drive doesn't have to crash, files can be corrupted anyways and then the mirrored disc will have the same corrupted file so there is no way to save it.

We would like to save the "masters" in EX1 mp4 files because they are good enough and only 35Mbit so they don't get to big. And on a server/Nas we have easy access to all masters made if we need to do a DVD/Blue Ray or something else. I will also build a ffmpeg solution so when we upload a master it is automagicaly encoded into all formats we need like ipod, 3gp, youtube, DV25, DVD and so on.

Were thinking of getting the new Sony USB drive for our safety copys, but for the moment its read only and not EX1 compatible. This is said to be fixed by a firmware upgrade.

Raid 3, 4TB to work off and Firewire 800 Raid for backup.

Get a dual layer blu ray burner ($600 or so) and then you can put up to 50 gigs on a disc. The discs run around $25 or so I think.

Single layer discs (25 gig) run around $10.

This is a long term solution that will work without the expense of getting an LTO drive.

Hopefully Macs will have blu ray drives in them soon.

Daniel Weber

I would take two 500 GByte quality HDs (seagate, samsung), each approx. 85 EUR - there you can backup approx. 31 x 16 GByte cards (per disc, one disc master backup, one is copy). This is backup costs of aprox. 5,50 EUR per Card with 100% redundancy, exactly the cost of DV tapes.
500 GByte Discs currently give the best bang for the bug.

In some months, when blueRay is as cheap as DVD-R now simply burn the footage to get additional security.

Not that hard drives are a good solution for archiving, but the other day I picked up a 500GB USB drive at Walmart for $85. Unreal!

Not that hard drives are a good solution for archiving, but the other day I picked up a 500GB USB drive at Walmart for $85. Unreal!

why are hard drives not good for archiving?

The obvious ------------------ They Crash!
Like many, I've had it happen and lost it ALL !
They are good if you can mirror drives. In the event one fails you have a backup.
But as someone mentioned before, if the data somehow gets corrupted on one drive, it
will also be corrupted on the mirror drive.

A outboard bluray reader/writter sounds to be a nice ticket....

that way it can be moved from more than one computer if need be...

I own a lot of large hard drives that I use for the actual editing, but from a business standpoint, I don't see hard drives as a good archiving solution. A client calls and says "Hey, we want our show re-rendered out in HD instead of SD", and you tell him "uh, we had a power surge and lost the footage on our drives" or "the janitor knocked it off the shelf while cleaning"... well, you have just lost a client.

Blu-ray burning is not a bad solution, although I am a little hesitant to go that direction because there has been no clear-cut HD format winner yet, and won't be for a few more years which is too long to wait. If you go with blu-ray and HD DVD wins, then you have a nice expensive paper weight. Plus, so far, the cheapest that I have found any 50GB blu-ray media has been in the $30 range. Although, this would make a nice deliverable format depending on if your clients went blu-ray or HD DVD.

I am still leaning toward LTO. You can get a version on ebay for about the price of blu-ray burner, but it will backup 100GB for $30/tape or up to 400GB for $60/tape - 800GB if you go with a compressed format. I am hoping that someone knows of a technology that I am not familiar with that will fit the bill, so keep the ideas coming guys. Aparently, there is a guy who did an EX1 review that was posted on the B&H website, and he is selling a book about it and has an archiving solution inside his book. Don't know what it says, my guess coming from Sony is a blu-ray solution. Any other technologies that I have not thought of yet? What are all the HVX200 guys doing?

How about the firmware update from Sony for the XDCAM disk to do 1920?

that would be nice!

I own a lot of large hard drives that I use for the actual editing, but from a business standpoint, I don't see hard drives as a good archiving solution. ...
Blu-ray burning is not a bad solution, although

of course a single Harddisc is very risky, but If you use two or three external, independent discs with identical data which are locked away e.g. in a firesafe safe and are only temporarily connected to the computer for backup purposes, it is very secure.
Nobody can say how reliable e.g. blue rays are for archiving currently. A small scratch may destroy masses of data.
I know that DVD-R is much less reliable then the good old CD-R. I had some backups just a few years old which had serious errors. The data quality on optical media degreades with the years. Maybe this affects BlueRay, too.
And a single BlueRay costs about 10 EUR for 25 GBytes, thats 200 EUR for 500 GByte and 400 EUR If you do the essential copies. Thats more than double the price for HDs, and also slow. And you need the BlueRay recorder, also not that cheap.

Carlos, two independent hard drives is more to my way of thinking. I do most of my work for myself, I have no concerns of showing anyone else the footage. I just need a way to save it forever :)

I have a question to those archiving their footage on HDDs. Frankly, this post could go to another forum as it is more general than just the XDCAM EX workflow; with this forum being the hottest now I decided to put it here.

From the point of view of the MPEG file structure, is it possible that some minor HDD's' errors (developing with time) degrade the playback quality? I don't mean corrupted drives; in fact Windows tools say all my drives are OK and defragmented, yet I do see some pixelation here and there in my HDV files (BTW copied from my DR60 drive not captured). I'm 100% sure they all played back smoothly only some 2 months ago!

I'd appreciate anyone's experience in this.

Get a dual layer blu ray burner ($600 or so) and then you can put up to 50 gigs on a disc. The discs run around $25 or so I think.

Single layer discs (25 gig) run around $10.

This is a long term solution that will work without the expense of getting an LTO drive.

Hopefully Macs will have blu ray drives in them soon.

Daniel Weber

I would not go with this solution. Number 1 expense and number 2 BD-Rs are not very durable. At least this is from what I've heard...it is worse than DVD-R when it comes to scratches. The data layer is closer to the surface vs. DVD-R (which is closer to the label side of discs) and that really concerns me.

Double hard drive backup I think is the best way to go. Two external hard drives. Use one to edit and when you're finished for the night copy the one to the other. This is like having a manual mirror system and IMO works better tto prevent corruption issues.

For long-term archiving digital linear tape is still the best way to go. Kind of ironic, huh?

If you go back and look at posts when the HVX200 came out, this same debate raged. At point I raised the possibility of harddrive tape backup storage systems..... funny, huh...???

As much as I believe that tape will not be the dominant capture medium for much longer, I wonder how many will be backing up their cards and hard-drives to a tape format for a long time!!!! :)

The more things change, the more they stay the same.

Mike

While I am ediiting, I use a raid system where the data is mirrored across 2 harddrives for redundancy. I also will make a copy of the raw footage onto a separate harddrive. The end product will be stored on a backup harddrive as well as a blu-ray disc. I don't have to give the client footage at the end of the day as I'm shooting weddings and edit everything then provide on DVD.

Matt, if you are shooting weddings, then you are likely giving clients standard definition DVDs. But what if they asked you to shoot all raw footage on a High Def camera which you can charge more money for? They will accept DVDs for now, but when Blu-ray or HD DVD wins the format war and they purchase a player and want their wedding footage rendered out as Hi Def(which you can charge more money for), what will you do then? Because of reasons like this, and for delivery of stock footage clips that I will need to have backed up for long periods of time is why I started this thread. I really need a solid archiving solution so that I don't have to throw any original footage away and that I don't have to fear that anything will be lost if a hard drive fails. I got a PM from Barry Green stating that he currently uses hard drives, but he agrees with me that currently, LTO3 is about the only long term solution currently available. If Sony updates their current xdcam software to burn full raster 1920x1080 35mbps to blu-ray, that is probably the direction I would go. But so far, they haven't.

Will be attending a High Def Summit next week and data archiving will certainly be a topic well covered there.

The concensus last year for cost effectiveness was to archive onto HDD and also to one other medium, maybe a tape based archival medium (DLT/LTO3), but using an intermediate, "mezzanine" intraframe file format that would be more flexible in the longer term. I seem to remember jpeg2000 being mentioned but don't quote me on that. Obviously the choice between predominantly HDD or LTO3 would depend on the cost/terra for storage.

I myself use removable eSATA drives for the moment, but most of my stuff is short form. YMMV. Plus my HD deliveries have to date been on HDD to a dupe or mastering house. I do the SD DVDs myself but will do HDDVD probably when the burners are more reasonable.

Will update after the event.

Bill, very good point. I should have mentioned. I will be recording and editing in full HD (35mbs). I will then export to SD for the client, however will keep all of the footage and project files in HD for future exporting to blu-ray or HD DVD.

I will actually be trying to sell this option to clients most likely middle of next year so that I can get an early hold on the technology so that when it is in more demand I will be totally up to speed with it etc. When I do I will be suggesting to the clients that they purchase both SD and HD discs at the same time (thus reducing my costs to get the project out later and produce a BR or HDDVD.

Are you suggesting a tape backup system similar to what most data centres would use where the tapes store like 100 gb or more and backup each night? Or simply recording back to miniDV tape via an exisiting HD camera?

I have also thought about factoring into the cost of each wedding package a new harddrive which I will use to store all files, be it raw video, edited video, and music, photos etc. So far I haven't had any issues with harddrives over the past 7 years. Touch wood this won't happen in the future.

My other option I thought would be to build some type of server(s) that I can use to store video files. I would probably keep them on there for a few months after I have delivered the final product, then move to backup solution. It really is a hard one and I guess the more conservative we go the the more copies and expenses we have. At the end of the day, once I have delivered the product I will have my own DVD copy as well as archived copy on harddrive. I don't find that many couples come back asking for more copies after a few months. Chances are they make their own copies!!

I'd be interested in some type of tape backup system so I might look into that, it could backup my server once a week or something. And just rotate tapes and store in a fireproof safe or offsite.

Matt, I hear you. People are so excited to get away from tape based solutions, as am I, but tapes were a fairly cheap and reliable archive system that we do not have by going with the EX1 or any of the Panasonic cameras. I have thought about setting up a raid solution myself, but it cost money to keep all of those electronics running and raid solutions are not cheap. LTO3 is a tape-based archive solution for servers that big companies use to back up their systems. They can store 400 GB uncompressed on a $60 tape, or up to 800 GB of compressed data. And these tapes would keep your footage safe for years if stored properly. But the LTO3 drives are not cheap. If you can find one on ebay or used, then that would be a good solution. Or you can go with smaller solutions like LTO or LTO2. I still hope the Sony will offer a blu-ray solution with their xdcam software, but I guess I could also just get a blu-ray burner and back up to blu-ray data discs, but they are like $25/disc for 25GB, which is not a good dollar/GB ratio. But this EX1 camera looks to be a great little camera, so I am determined to come up with a solution for it.

but I guess I could also just get a blu-ray burner and back up to blu-ray data discs, but they are like $25/disc for 25GB, which is not a good dollar/GB ratio.

that maybe the price for the rewritable discs, but the BR-R discs are only around $10 for 25 gigs.

Daniel Weber

Chris ,
I have bought an eSATA swapable drive system, but have no good solution to how to physically store the drives themselves when they are not being used. They look fragile and have circuit board stuff exposed, etc. - how do you store them? Haven't seen anyone selling something for this.

Lenny Levy

They say VHS cases are an almost perfect fit for bare hard drives... I have yet to check on on that, but it sounds like a nice standard form factor for archival, there must be cheap furniture designed for itt.

I'd probably put the hard disk into its original anti static bag, to save it from ESD, and then throw a little piece of foam to cushion the VHS case, so the disk doesn't rattle inside. Maybe also some silica gel.

Does that even make sense? Excuse my English, it's not my first language and right now i'm lacking sleep :)

Daniel, that is a great idea with the VHS tape covers. I have heaps left over from years ago when I stopped offering VHS and moved onto DVD.

Also I like the idea of a BR-R, if your backing it up you will only need to archive it anyway, so that would be perfect for finished products and raw files. Plus a once write disc should burn much faster than a re-write?

Chris ,
I have bought an eSATA swapable drive system, but have no good solution to how to physically store the drives themselves when they are not being used. They look fragile and have circuit board stuff exposed, etc. - how do you store them? Haven't seen anyone selling something for this.

Lenny Levy

I mostly buy segate drives and they are shipped in a plastic protection enclosure. Other drives comes in a ESD protection bag but they are of course less protected to a good beating.

Our plan is to use a server for storage but also have it on XDCAM HD discs when the USB Drive from Sony will write EX Formats (if it ever will).

I've seen nice padded leather cases for bare eSATA drives, so I know they're out there. Now if I could only remember where...

I shot a couple hours of footage on the EX1 this weekend and currently have all of that on both my laptop and an external bus-powered USB drive. For my purposes I'd say redundant hard drive backups are as good a solution as any, and have the advantage of being readily accessible for editing. If hard drives make you nervous then you'll have to think of something else, but I can't think of a more convenient or affordable option.

P.S. I finally get it about the whole solid-state workflow concept, now that I've seen a version of that which works smoothly enough to be useful. Being able to transfer 30 minutes of footage in under 4 minutes is great, and the EX1 footage plays fine in Windows with the camera software installed. Much better than a certain other solid-state video option as far as I'm concerned...

If only Sony would stop screwing-about and make whatever updates are required to the PDW-U1. It would be so great to burn the EX1's full-monty 1920x1080 video to a Sony professional disk and not have to worry about re-archiving again for 50 years. I get warm and tingly when I think about that. Lol.

For the moment I am using 2 USB hard drives. I simply copy the contents of the card to both drives.

When Sony release the F700 50Mb XDCAM HD camera next year (spring??) they are going to have to update the U1 to work with full raster 1920x1080 XDCAM MXF's so at that time I will switch to Professional Discs for my backups.

Alister, I am a little less depressed after reading what you had to say about the U1.

Regarding archival: Do you feel that your data is safe on just two usb drives? Do you back-up to dual-layer dvd or something? I am wary of HDs, although, I have only had one fail on me, and that was most likely due to technical ineptitude more than error on the drives behalf.

On a more random note, I was thinking about what it would be like trying to back-up, say....25 hours of full-resolution video from the ex1 onto single layer dvds. (I know it's illogical, but I'm thinking of a worse-case scenario here). Let's say for the sake of organization you use 4 dvds for every 50 mins (16 gbs) of footage. That wouldn't quite fill up the space of a blank dvd, but it would keep everything consistent and "reasonably" manageable. That would equal 120 dvds and many hours of boredom spent burning them. I personally always back-up to dvd twice...so that means 240 dvds. Haha! Despite such a ridiculous amount of disks, I want this camera so badly I would be willing to do that to archive. The good thing is that because I am shooting features and not doing any weddings etc. I won't have as much footage to save. It is unlikely that I will shoot over 25 hours of footage for a 90 minute feature, especially considering that one can just delete unwanted takes on the fly with this camera.

I agree the the U1 will be the best price/reliability option once it's upgraded to full-raster but for now, HD should be safe. Remember, hard drives *usually* only crash when they're spinning. Fill it up, put it on a shelf and pull it off once the U1 comes to town.

Also... rethink your archives. I currently shoot on miniDV and have over 1000 tapes in my cataloged library. I still need access to them but I need to change my thinking about archiving. Do I need to archive every minute I shoot? I do with tape. However, with tape-less workflow, I can delete the bad takes and only archive the good takes. If I shoot a 10:1 ratio then my storage needs will decrease by 90%.

This is a good way of thinking for commercial and training video producers like myself... not so good for documentary guys, they need to keep every minute.

The bad thing about video tape as archive is the lack of random access and the inability to see "clips."

Imagine have to go through hours of tape to find a shot. Looking at thumbnails on hard drive or optical disc is much faster.

While the tapes are good if you need to load back in an entire project, it's not good for hunting shots.

Also as time marches on and formats change, tape decks for a given format become scarce. Trying to find a D2 machine to play a tape from a project from 1992 is a research project in itself. On the other hand playing a CD from the same year is still easy.

It seems optical disc format playback has much greater longevity than tape decks.

Once your ancient Hi8 camera goes do you have a Hi8 tape deck to play back those tapes. Will you find one in 5 to 10 years?

HDV will be even worse given the number of incompatible variations not playable on other cameras or decks.

Give me an optical disc any day.

Anyone who relies on a hard drive for long term archiving is setting themselves up for DISASTER.

Hard drives are a MID-TERM archiving solution, NOT a long-term solution.

I will NOT trust my important client's footage on a drive whose spindle dries out from non use, thereby making the data unreadable.

SDLT, LTO3 and LTO-4 tapes are good for 30-40 YEARS, and is what is on my archiving shopping list.

What about dl dvds? Are they as reliable as sl dvds? I wouldn't use them for long-term archiving; just until the U1 can handle full 1920x1080.

Anyone who relies on a hard drive for long term archiving is setting themselves up for DISASTER.

Hard drives are a MID-TERM archiving solution, NOT a long-term solution.

I will NOT trust my important client's footage on a drive whose spindle dries out from non use, thereby making the data unreadable.

SDLT, LTO3 and LTO-4 tapes are good for 30-40 YEARS, and is what is on my archiving shopping list.

You might want to consider that its moisture that causes the grease to thicken up and eventually stall the motor. Keeping a drive in a humidity and temperature controlled enclosure will increase its shelf life. I do agree that hard drives are not reliable for long term storage.

When a drive is operating, the heat generated keeps the moisture driven off and the grease in better shape.

Storing a drive unprotected in an area where temperature and humidity swings about is the worst thing you can do and it will shorten its life. Our homes are certainly bad places to store drives without protection. The best thing you can do is put them in a sealed case (like a Pelican case) with a desiccant bag in the case along with it. Be sure to save the antistatic bag the drive came in and put it in there with it open to allow the desiccant to do its job.

Here is what I'm talking about - http://www.sprucemtsurplus.com/051503/P5150013.jpg

They are cheap and reusable. Just bake them in the oven for a bit and they are recharged.

Regardless,

Drives are NOT long term storage solutions. If you rely on them to be so, you will get burned.

Its your money.

Regardless,

Drives are NOT long term storage solutions. If you rely on them to be so, you will get burned.

Its your money.

Matt,

I was agreeing with you. I also wanted to pass along some info to maximize storage better regardless of duration.

When affordable high density optical storage is a reality most of this discussion will disappear and we will all be using our hard drives online again.

Also to point out a different opinion. Here is a snip from Cineforms website:

Quote:

"Some other workflow benefits of are worth noting:

* An entire two hour feature can be mastered to a single 350GB hard drive for long term archive which offers longer shelf life than tape"

Full text here - http://www.cineform.com/technology/12Bit-RGB-QualityAnalysis/12Bit-RGB-QualityAnalysis.htm

I was talking with a friend about this today and we came up with an interesting thought ...

When considering a Solid State future,many are wondering how to archive, because we previously kept camera tapes.

What about the development of WRITE ONLY SxS cards, which could be produced in huge numbers and become very cheap, so that we can do the same thing - shoot, then keep 'em on the shelf.

Or is this not economically feasible?

There are those of us in the video business who are ready to make the leap from SD to HD and our product will look better. I think the EX1 is a good way to go for small video producers like myself to go what way. You can bill more for a better looking product. Plus you can go to your clients and say,

"Back when I was shooing in SD, I didn't charge you to archive your footage because my cost was really only about $7 per hour of footage. The good news is that now that I'm shooting in high-def, your footage is essentially future proof. The bad news is now it costs significantly more money to archive. Would you be willing to pay our fee of $50 per hour of footage for archiving? That way, it's available if we need it again."

Look at that... all of the sudden the 'problem' of archiving becomes an opportunity for another revenue stream.

I'm looking at archiving a bit differently.

Burn to DL-DVD in 8GB increments. First copy is for me. Then 2nd copy if the client wants, the client pays. I can do the same with blu-ray at a higher price. In that case I can change the extension to .ts so they can play on Blu-ray player or PS3. I can charge even more than that (and there's no need to encode).

The thing to be careful about when marketing archival to clients, many might chose to go with HDV if you're going to charge them a fee. Many clients don't know or get the technical difference between 25mbps CBR (HDV) and 35mbps VBR (XDCAM HD). You have to present it as an advantage.

For me the advantage is that you can now hand them an optical disc they can use in their computer vs needing a tape deck (that must be compatible with the specific form of HDV) or camera (to play back the tape). A Blu-ray player can be had for under $400 and a computer drive for under $600. Both way cheaper than an HDV deck. DL-DVD they may have in their computer already. You've now presented the shooting with the EX1 and backing up to optical disc as a great CONVENIENCE to the client over tape.

That's an excellent way of looking at it, Craig.

For me, showing the advantage over HDV is easy. Hold up a miniDV tape and tell them that this is what I've been shooing on for 10 years, it holds an hour of footage. Then put the tape down and pick it up again. Now say, this is an HDV tape, the image it records is twice the size as SD and this tape also holds an hour of footage. How do they get all that extra information on there? I could bore you with the technical mumbo-jumbo but that's why I chose to go with this higher quality solid state recording codec. HDV was (is) a great stepping-stone high-def format but this is the way of the future.