Like many folks who have been shooting AVCHD for a while, I now have many years worth of footage from a variety of cameras stuffed into my backup/archive workflow. For all those years, I've been keeping perfect disk images of every single one of my AVCHD shoots. In the early days of the format, when support was far, far more twitchy and frustrating than it is today, I felt it made sense to have a little "healthy" paranoia.

These days, however, I wonder if this isn't a colossal waste of time and storage space?

Here's the thing about this method...nothing ever gets deleted. You end up sucking every little take, every borked shot, every mishap, into your diskspace, cloud storage, tape library, or whatever you use for archival, forever. When I stop to think objectively about this, it seems nuts. I mean, if I did this with photography, my photo library would be 2x to 3x its current size and full of garbage. It would also be hard to navigate, which come to think of it is a major annoyance with all my AVCHD disk images. I keep them well organized, but still, having to mount them to look through videos (and then having to climb down into the "STREAM" subfolder each and every time) feels like a hassle.

I wonder if I should just go back, bust open those old disk images, use something like ClipWrap or Media Converter to repackage all those .MTS clips in something more portable (like Quicktime or MP4 containers) and just keep around the ones I actually want? There's no quality loss in doing so, and the video/audio content would continue to be usable in pretty much any modern NLE, right?

What, if any, good argument is there in 2012 for keeping complete disk images of AVCHD?

I have all Sony cameras and transfer to the PC with the Sony PMB or the CMU.I thus have all the clips as defined by the start and stop on the camera and all the meta data. These I backup to LTO3 tape. The CMU and PMB have the library and I can reload quickly if I need to from the backup library software. A 400G LTO3 tape will of course store about 30+ hours of AVCHD and costs about $25. The PMB, CMU and backup libray are backup in the normal PC backup. I have been happy with this solution. Not expensive and keeps everything just like I did with video tape except a lot more reliable , less space and costs a lot less. The LOT3 tape is about the size of 3 DV tapes. Backup finished projects in the same way too.

Ron Evans

Backups are the bain of the digitial existence. As for what to backup, it depends if you want to be able to re-edit or just store the finished product. Kepping all of the footage allows you to add something to the edit at a later date which might prove to be useful. If the project was complete and no further editing is needed then I see no need in keeping all of the canon fodder so to speak.

For events I backup source files before the edit and keep those as long as I can. I then make a master BD/DVD of the finished product of which I make my copies from for delivery. Events usually do not need any re-editing, especially as time does forward.

For scripted projects I have the client provide a hard drive and I put everything on the drive. They then store the files and we can access the drive in the future if any changes need to be made.

I use a hot swap tray in my computer to slide drives in and out for the backups. I then store the drives on the shelf.

I have the same problem as the thread starter. What isa n LTO3 tape? What is a Hot Swap Tray?

Thanks.

Mike

LTO3 is a data backup tape as used by IT departments. This is the one I have LTO Tape Drives Data Storage Product - LTO 3HH Overview (http://www.quantum.com/Products/TapeDrives/LTOUltrium/LTO-3HH/Index.aspx). It fits in a 5.25 slot in my PC next to the Bluray burners. I use Retrospect 7.7 as the backup software. Retrospect: for Windows (http://www.retrospect.com/en/products/win). The LTO3 is the lowest cost lowest performance unit and if I were to buy now I would get the LTO4 unit which is a few hundred dollars more. My drive cost about $1100 Can when I bought a few years ago. Considering the cost of all my cameras and PC's it is not much to keep secure backups. Backup and restore is as fast as the hard drives will go, about 65MBps sustained on my system so will backup and verify a SD DVD file in 3 mins. I don't use a Hot Swap tray but these are removeable tray ( 5.25 slot like the tape drive or CD/DVD burners ) that house hard drives and have connectors with leading grounds so that they can be inserted or taken out with power on the PC.

Ron Evans

Thanks for the information, Ron. Appreciate it.

Mike

Drew
I archive just about everything as well & at times have wondered why I'm doing it.
However, I have repeatedly had the experience of needing (or just wanting) to revisit projects that I thought would never see the light of day again.
To be able to do that is very gratifying.
My current solution is to use inexpensive 7200 rpm SATA hard drives via a hot swap tray.
Hard drives are cheap, and, although not totally bulletproof, will typically provide years of reliable, easily accessed storage.
I shoot AVCHD almost exclusively nowdays, and often edit in Cineform format- particularly for long or complicated projects
The AVCHD files are small and I will store all of the original project clips in a seperate folder.
When I complete a project, I will have Premiere Pro project manager save a "trimmed" version of the project. This saves only the media actually used in the project timeline and will trim the clips to their timeline length- this will greatly reduce the amount of storage required.
I will save all of the final master renders of the project, including DVD, Blu Ray, web, etc.
If there is a written script, artwork for DVD, and so forth, that all gets saved.
All this stuff goes on a hard drive in a single folder with the project name.
When the drive eventually gets filled with projects, it goes up on the shelf.
It's pretty fast and cheap to do this- and when the day does come that you need it... priceless :)

I like to save all the original video( I kept all the HDV, DV and Hi8 tapes at much greater expense) as if I want to re edit or look for something I missed it will likely not be in the trimmed stuff as all my projects are multicam with 4 or more cameras so most is not used. Most of the cameras are AVCHD but sometimes with the addition of an EX3. I edit with all native files in Edius. I save all the project stuff in a directory by project too so can just reload the project anytime. I usually keep the finished DVD and Bluray files on a hard drive for quick access but they are also backed up to tape just in case the drive fails. Not sure I would trust my only backup copy to a single hard drive as I have had too many hard drives fail. Putting on the shelf for too long without running is also not good.

A typical 2 hour show with 5 cameras, source AVCHD, project data and finished SD DVD and BLuray files is about 130G. So I could get 3 complete shows on one backup tape for $25. In fact I keep the source AVCHD and project and finished files on separate tapes.

Ron Evans

I think, no question, the LTO3/4 tape back up is the most secure way to go.
Myself- I'm a victim of lazy habit.
I've had good luck with the hard drives so far...
When I eventually get smacked, I'll move on to the new best thing.

I used to store all my info in the original AVCHD format, but lately I convert them to .mov files to play on my ipad. I see no reason to keep the original files. I found a number of converters at AVCHD Player (http://www.avchdplayer.com)

Isn't that a bit like taking a photo with a SLR, getting a 4x6 print from the automatic printer at the local store and throwing the original high quality film away ?

Ron Evans

What, if any, good argument is there in 2012 for keeping complete disk images of AVCHD?

The main argument for keeping the complete disk image is to keep the metadata. Also, if you keep the disk image you can copy it back to the camera for viewing and further in camera editing.

Isn't that a bit like taking a photo with a SLR, getting a 4x6 print from the automatic printer at the local store and throwing the original high quality film away ?

Ron Evans

Not if you use something like Clipwrap. All it does is 'rewrap' the .MTS files into .MOV files. No
recompression or anything, just more easily playable on a Mac. I do this with all my footage before
archiving to blu ray data discs. This way I can preview clips even without opening up a NLE.

Not if you use something like Clipwrap. All it does is 'rewrap' the .MTS files into .MOV files. No
recompression or anything, just more easily playable on a Mac. I do this with all my footage before
archiving to blu ray data discs. This way I can preview clips even without opening up a NLE.

That's it exactly, Gabe. The biggest advantage for me, personally, is not only that they're more easily playable but that I can easily lose all my screwups and otherwise unusable clips! Bye bye garbage data, hello reclaimed usable storage capacity.

Having to keep all of that junk around forever easily doubles my backup/archive needs in terms of disk and cloud space. The whole point of digital to me is the ability to shoot shoot shoot like crazy and then delete delete delete later on, mining for the gold. I guess everyone has their own styles but I am NOT an old school inside-the-camera guy at all. I prefer to capture like crazy and sort it all out after the fact (largely the reason I was never a film buff, I could never have afforded to act that way with developing costs and wait times).

I've been thoroughly testing the "clipped out" version of my archive for the past several weeks and I'm pleased enough with how versatile the Quicktime files are that I'm about ready to lose the old pile of fat bulky disk images for good. I can't wait to send a drive full of this slimmer, more usable version of all my stuff out to Amazon to dump into some Glacier space...then I can start shooting newer stuff with even more freedom (and the peace of mind that comes with knowing I have a smoother and leaner workflow in place).

I probably sound like a paid commercial for ClipWrap, but dude, this really has made me overwhelmingly happy. I just got my first RAW camera for stills and I can tell you, diskspace is starting to look precious. ;)

I understand the advantage of Clipwrap for a MAC user and if it just wraps the AVCHD to make it easier for the MAC to read then you are not likely saving any space just formatting. Not needed for a PC though and I misunderstood that you were reducing the resolution to save space at the same time.

AVCHD doesn't take up a lot of space for the quality so personally I keep everything !!!

Ron Evans

I understand the advantage of Clipwrap for a MAC user and if it just wraps the AVCHD to make it easier for the MAC to read then you are not likely saving any space just formatting. Not needed for a PC though and I misunderstood that you were reducing the resolution to save space at the same time.

MAC = a networking term, referring to hardware ethernet addressing
Mac = most computers made by Apple after 1984

That aside, the purpose isn't "to make it easier for a Mac to read" really, though certain Mac users might use ClipWrap or a similar tool for that reason (both the native video playback tools in Windows and Mac OS X are pretty crappy with AVCHD if you have an older/weaker GPU).

No, rather my primary annoyance with AVCHD is the tendency toward maintaining complete disk images of the AVCHD filesystem, which especially in the early days of the format was necessary out of paranoia because the few tools that did deal well with the content did so in very picky ways (not true today).

AVCHD is a random access affair, and the storage space it takes up is multipurpose. I agree that it's wise to keep whole tapes of old projects around, because with most tape workflows it would be damn near impossible to easily rebuild a project if all the clips weren't exactly in the position on the tape that they started out at. With random access, that's different. There's no sensible reason to keep an entire capture session, SD card, etc. in tact if there's a large portion of it that's pure junk. Think of it this way, if you bought a box of tapes, and you shot one of them, would you want to have to put the entire box on a shelf and never touch the unused portions? That's what a disk image forces you to do, if there are a lot of junk shots inside of it.

AVCHD doesn't take up a lot of space for the quality so personally I keep everything !!!

I definitely don't want to keep everything. I'd want to keep everything that I used in a project, everything that might have even the most remote value someday. But do I want to keep around shots of the floor, shots of my feet, shots in which the camera was set up wrong or some other error occurred? Nope. That's why I'm breaking my disk images up, not just to make the clips more usable...but also to allow me to delete individual clips that have no value to me. I'm not saying everyone should do this, I was just checking to make sure I hadn't overlooked some benefit of keeping the whole disk image.

The one good point that was made was AVCHD metadata...if you maintain it actively and need it for anything then yes, one would want to keep the complete archive in tact. I've not come up with any use for it personally though, so that's not a big deal for me. The only details that are important to me are the timestamp on each clip and the lossless preservation of the audio and video streams inside. This process meets both those requirements (ClipWrap optionally sets the filesystem timestamp on each file of output to exactly match the AVCHD media that was used as input).

All my cameras are Sony and I transfer with the Sony utility so only the file itself and the metadata are transferred to the PC. With the utility keeping track of most of the metadata anyway. All my projects are multicam events so keeping all the camera data is the only sensible way as if I want to re-edit at some time I need all the data since all the camera are in sync and of course only the equivalent of 1 of the 4 cameras is used. The only single camera files are family and yes I throw away stuff as I transfer that !!! I back up everything to LTO3 tape.

Ron Evans