Archiving, search and find footage

[Per Johan Naesje]

Hi all!
As subject says, how do you do it?

I´m about to start shooting with 7D after many years in a tape based environment where the original tapes was my archive.
Do you use any software to help you out or are you just organizing disk in folders, what about searching options then?

I heard that the latest adobe lightroom v3 is able to organize dslr footage, but how good is this, has anybody tried it?

[Chuck Spaulding]

I too would be interested in hearing about Lightroom v3's capabilities.

Here are some suggestions and things to think about as you try to figure out how you want to manage your tapeless workflow.

There are a lot of different ways to manage data, in part because people prioritize things differently. Some people believe its important to keep the original h264 clips that come from the CF Cards, others delete them after they're encoded and work with and archive the encoded files.

Either way there are things you need to keep in mind. If your planning on using the Canon plug-in for FCP you need to make sure that when you copy the contents of the CG Card you clone it exactly as it is. It is a file system, if there are any changes to the file system the plug-in will not recognize it. If your not going to archive the H264s then you can use the pug-in to make the ProRes clips directly from the CF card.

The ProRes clips are as much as four times the size of the H264 clips, so I archive the H264s and then if I corrupt a ProRes clip or need to encode for any reason I can easily do that. The catch is you need to be meticulous with your file naming conventions.

So here's one way you might create a workflow that archives, encodes and renames. Start by naming each CF Card, create a folder ARCHIVE, this is the place that you'll store your H264 clips. Inside create another folder AR_RACE_01A.

AR identifies the folder so that you know it contains H264 files so if it accidentally gets moved you'll recognize what it is by this acronym, RACE in this example is the project name you'll be working on, 01 is the camera number it was shot on and the A is the card identifier. If you don't use multiple CF Cards per camera then obviously you can omit the last letter.

Copy the DCIM folder exactly as it is into this folder. You need to pay attention to this because ALL of these folders will be named the same, "DCIM" from ALL cards so you need to make sure you placed it into the correct folder that you previously created.

People manage there capture scratch differently, I'm just going to mention how I do it. Once I have all of the cards transferred into the corresponding AR folders, in this case assuming that all the cards are for the same project, I open a new FCP project name it RACE (project name) and then save. This automatically creates a new corresponding folder in my capture scratch, launch "Log & Transfer" CAUTION if you have a CF Card mounted it will automatically appear in the L&T list, in my workflow I make sure there are no CF cards mounted - because I've already copied the contents into the ARCHIVE folders, so as a rule of thumb if any clips automatically appear in the L&T window I highlight them and click on the EJECT button which clears the list.

Then click on the button with the gear icon to add a custom path then navigate to the AR folder I want to encode, note, do not click on the DCIM folder itself, you must click on the AR folder that contains the DCIM folder for all of the clip in the DCIM folder to appear in the L&T window.

In the logging pain there is a drop down for NAME PRESET. Click on it and drop down to EDIT. There you can edit the parameters that make up your custom name. It is very flexible but you need to make sure that you specify a parameter that is unique to the clip your transferring, original name, time of day etc.. Otherwise the name becomes generic creating the possibility to overwrite already existing files in your capture scratch.

For example my custom naming parameters are:Reel (underscore)Original Name Partial Span. So what I get in this example is AR_RACE_01A_MVI_003. So at a glance I know where the original (RAW) clip is located, it was shot on camera 1, recorded on CF Card A and which clip it is. So if I were doing multiple CF cards they would all have a unique identifier, for example the next CF card would have been AR_RACE_01B_MVI_015 or AR_RACE_02A_MVI_001.

I've probably made this sound much more complicated than it is, after doing it a couple of times it becomes second nature and make it very easy to manage and more importantly know what you've copied and what you haven't. Anyway I hope this helps someone

[Per Johan Naesje]

Chuck, thanks a lot for your great suggestions! I was thinking almost the same as you regarding storing my videoclips to disks.

As a wildlifephotographer as time pass by, I will get a huge library with different kind of clips which I will use in different kind of projects. My thoughts was to find a way to search for these clips in an easy way. Problem is that I might not only shoot with a DSLR but also with other tapeless systems.

143 views for this thread and only one replay tells me that there maybe not so much options out there?

[Declan Smith]

I too like to backup my source clips in a similar way to a tape workflow, so I did a couple of tutorials which you may find useful for fcp. The second one makes use of the eos plugin for fcp to archive the sd card.

YouTube - Canon 7D Final cut pro EOS plugin Ingest Workflow Part 1
YouTube - Canon 7D Final cut pro EOS plugin Ingest Workflow Part 2

[Chuck Spaulding]

Declan, great find about creating a disk image directly using the Canon plug-in. Greta tutorials too.

Its not so much that there aren't good options available I think its more that your original question is about asset management which isn't simply an issue of DSLR's but all file based formats.

It also depends on how you run your business, if your a single artisan working alone there's solutions like Grid Iron Software's Flow - Visual workflow maps & creative file management for designers | Gridiron Flow Product Page. Not only does it track the relationship between links (where the file came from and where its stored) but it also tracks the file at the application level providing you sort of a version control.

If you work as part of a creative team there's also Final Cut Studio. Generally it requires an Xserve which is probably more money than an editor working on their own wants to spend.

The challenge for developers of these types of applications is that everyone works differently, therefore these applications can get quite complicated to set up and manage. It really depends on the volume of stuff your trying to track. I have used both applications and they worked as advertised, but it really starts with you setting up a naming convention that is simple to understand and works for you.

My rule of thumb is that no matter what kind of database or asset management application you use, if it fails can you look at you file system and by reading the file names determine where a file came from and where does it belong. Also depending on volume and capacity you may need different levels of storage, online - which provides high speed access to data, near line - which is immediately accessible but on lower level slower drives and offline- which you have to mount in order to use [external FW drive, DVD, CD for example].

Flow actually can track all of this even the offline data, so I think its worth evaluating.

[Per Johan Naesje]

Declan, many thanks for the links to your tutorials! I will look through them later this week as I am out in the wild collecting more wildlifefootage :-)

Also thanks to Chuck, appreciate your comments very much!