H264 for backup???

[Jose Milan]

Hello,

We have to make a lot of videos (HDV) during the year, we are almost tapeless and we use huge amount of storage. My question is if you people think if it would be a good idea to render the projects when there done in a high H264 profile and big size and discard the .m2t's or avi's (after a prudential time and when everything is check by the client)?

This way we could always render in smaller sizes and profiles (using the high h264) without a great loss of quality and possible make little edits within the .mp4 file, right?

Any insights?

Thanks in advance

[Jose Milan]

Nobody can say nothing about this? Does anybody uses .mp4 for backing up media?

Any thoughts?

[David Chilson]

I use H264 for backup for some projects, it depends on the job and how it was delivered.

I usually decide by the needs of the client. If it's a one time thing, I've been paid and I fullfilled my contract, I save what I delivered in the format delivered. If the off chance they come back I can duplicate it. I tell this to the client.

This accomplishes two goals. It frees up your disk space and lets the customer know that any further work is going to require a new contract and more money. Saving it in H264 if it fits their needs is fine. It's a format that is quite universal, good quality and should be around for a while.

For ongoing accounts I save the files in a format in which they were captured that I can access quickly and output to the needs of that client.

So if saving them in H264 fits your needs and those of your clients, it's a good choice.

The reason no one answered your question was because we were instructed by our government to boycott questions from Spain. Glad that ordeal is over.........:)

[Tripp Woelfel]

When I'm done with a project, I'll render out the final version to Quicktime using the Avid DNxHD codec which is effectively lossless. I haven't done the computation or any tests, but I think using the highest quality 8 bit setting comes in between 20-25GB/hr. Yes, that's GB.

This gives me a gold master that I can save to a USB drive, and I have the option to convert to any delivery format I want in the future with a minimum of quality loss.

[Jose Milan]

Hello guys,

Finally!!! Thanks for the replies...glad to hear USA and Spain are friends again...;-).

Its fundamental to tell clients about this of course, but the good thing is you can edit in h264 (good bit rate) and still maintain quality, right?

Thanks again.

[Tripp Woelfel]

Didn't know that we weren't. (Grin)

Editing a lossy format like h.264 has some issues. If you don't cut on an I frame boundary, you will recompress some of the subsequent frames and quality will be lost. This will happen with any interframe compression. If you apply effects or transitions, quality will be lost too. Dissolves are toughest on interframe codecs and many have complained here about it.

As always, the bit rate will have a big effect on quality so you want to keep the bit rate as high as possible. Also, if you have the option of going I frames only (also known as intraframe only) you should retain more of the source quality through any editing you might do. Of course, your file size will be larger but nothing like DNxHD, Animation or uncompressed.

[Ervin Farkas]

Although you may not immediately see the difference, you loose a little bit of quality every time you transcode to another format. In addition, depending on the type of work you do, you may need to change things on your edit.

I edited a year's worth of TV show (one hour weekly) and archived the final output in the same format as the input. Later I have been asked to remove a logo... and I had to say: sorry, can't do it.

So now I'm archiving the source clips together with all additional audio and video files used, AND the project file - this way I can go back and change absolutely anything I may need to change.

With storage so dirt cheap, it is worth archiving everything - you can put 70 hours on a sub-$100 1TB hard drive...

[Jiri Fiala]

The bad news is that HDDs are very prone to errors and faults. Tape is, like it or not, far more safe for long-term storage. Even if you don`t use tape-based workflow, you could rent or borrow a tape-based camcorder or deck and backup this way.

As for H.264, you will bypass difficulties introduced by editing interframe material by transcoding it to some intraframe codec (DV, QT...) prior to editing. This will indtroduce a small dip in quality, but you will be able to edit without massive losses due to editing interframe.

[Tripp Woelfel]

Originally Posted by Jiri Fiala

The bad news is that HDDs are very prone to errors and faults. Tape is, like it or not, far more safe for long-term storage. Even if you don`t use tape-based workflow, you could rent or borrow a tape-based camcorder or deck and backup this way.

While I'm not a big fan of using rotating magnetic storage (hard drives) for archiving, I don't think you're giving it a fair shake. If disk drives were that prone to faults, I doubt that major corporations would keep their critical data on them.

Since you're living in an HDV world, your choices are a bit limited. You could try transcoding back to HDV and output to tape, but that could involve quality loss. How much, I cannot speculate as I've never done it.

As I said before, I keep QT files with a lossless codec on USB drives. For long form projects, it's the only cost effective option for me at the moment.

[Jiri Fiala]

But major corporations don`t keep their data on single drives. They keep it on massive arrays with multiple levels of redundancy and several backups. That`s the crucial difference. Drives die all the time, and tapes mostly don`t.

But I agree with you on QT lossless (or, for backup purposes, you can use QT PNG, but that is more taxing to encode and decode).

[Tripp Woelfel]

Good point. Redundant disk arrays are used not only for data protection but for continuous operation so servers don't crash when a drive fails.

While drives do fail, I think of you look at the MTBF (mean time between failure) data for Seagate Barracuda, WD Caviar or other high end desktop drives that number will run into many thousands of hours. Now, that doesn't mean that you cannot have a new drive fail within a week. However, it's unfair to expect drive failures with any regularity.

That said, should you slavishly adhere to a comprehensive backup regimen? Darned skippy!

[Stephen Armour]

We have much the same scenerio and need longterm storage for our 1920x1080p material.

For several years, we have used an intermediate codec (Cineform HD) with great results and with very excellent masters by saving everything to duplicate HDD's. One set is saved locally for ease of use/reference, the others are in hardcases, in another easily accessed location for backup security. All HDD's used for backup are new and of good quality.

We use mirroring software to maintain constant exact duplicate copies while editing (we have 3 workstations), then also keep backups on NAS's temporarily just to make sure.

The CFHD codec not only saves us space, it has allowed us to do realtime editing with pretty basic hardware, especially if you use multiple monitors. All our workstations have RAID 5's.

All editing material (except previews) is on the duplicated HDD's, so access/changes are easy and fast. Just plug in both HDD's, re-mirror one to the other for the changes, then unplug and send them back to their storage locations.

Works pretty well, until something better comes along.

BTW, bumping the HDV material to 1920x1080p square-pixel, gives very good scaling results from the full-size CF masters. The best I've seen. Certainly better than any H.264 master results we experimented with! It's not a cheap codec, but more than worth the money in several ways. Guess that's why Hollywood, RED's, Viper's and ARRI's are now paying attention to it....

[Ervin Farkas]

Originally Posted by Stephen Armour

We use mirroring software to maintain constant exact duplicate copies while editing...

Stephen, what is the best mirroring software? I suppose you tried out a few...

Thanks,

[Stephen Armour]

Originally Posted by Ervin Farkas

Stephen, what is the best mirroring software? I suppose you tried out a few...

Thanks,

MirrorFolder 4.1 from Techsoft seems to work quite well for us, and we've got a pretty good mix here.

I especially like the various options for ways you can "mirror" what you need, plus being able to mirror anything on the LAN to anything else, and being able to turn it on or off, or schedule for sync. It can be one way sync, or bidirectional, exactly sync'ed like RAID 1, or just a "new changes" mirror, etc.

Very flexible, very powerful for what it does and easy to use as well. Worth the price. Wish we had it years ago...

So far, no complaints. We've had some expensive solutions that didn't work near as well. They've got a 30 day tryout, so you can make sure it's problem free for your setup. Our mix is 32bit and 2 - x64 machines, with 5 RAIDS, a USB NAS and a GbE NAS, and to confuse things totally...an external hot plug eSATA drive!

Someone commented that it runs more like a system driver than a "backup" software for ease of use, and that's the truth. Very smooth programmers, those guys in Calcutta! No wonder we're losing work to them!

[Ervin Farkas]

My Western Digital MyBook 2TB RAID came with sync software which I did not install at the time, but now I need it for a large project.

Thanks for the info, will try both of them and report back.