P2 card dump reading problems on Mac and FCP 5.1, Help! at DVinfo.net

Steve Reid · July 26th, 2006, 04:35 AM

Hi all,

We did a shoot other week on our new HXV200 and dumped cards to PC laptop and external HD, but now that has turned bad, corrupted and unreadable.
But we did have insight to make a 2nd back-up of cards to DVD. Now we have put on Mac HD and can access them using Final Cut Pro 5.1, we can see thumbnails in P2 import window, but when select clip thumbnail trying to actually import footage I get an error message

- "one or more P2 clips failed to import, probable incomplete or corrupt clips."

Has anyone else had difficulty in reading transported/archived/spit P2 folders in Final Cut Pro 5.1 or from any OS.

How I archived and reasembled was: Dump P2 card to laptop (7-8GB) Open "contents" folder then "Video" folder and selected lower half of video .mxf clips aprox 4GB and burnt that to DVD, then moved those to another location on PC HD and burnt another DVD with all the P2 contents folders in same file structure, nothing moved.
Reasemble consited of, First copy the DVD with main contents to Mac HD, then copy DVD contents containing just half the video .mxf files and place those back in "Video" folder. Now this should be exactly as how it was when dumped originally, nothing lost added, moved just reasembled as per orig.

Any ideas folks would be great. Even involving PC software, any way to get these files readable really!
It seems like everything is all there just FCP doesn't like the files directory sttructure or how files have been moved and recompiled.

Is there a Mac program - pure .mxf reader and exporter to a .mov etc or PC even but would need to be exported to .movs not avi's or wmvs.

Thanks

Shane Ross · July 26th, 2006, 04:13 PM

You cannot grab just pieces of footage from the CONTENTS folder. In order for FCP to be able to import the footage, it needs the ENTIRE contents folder, nothing moved, nothing changed. And the CONTENTS folder cannot be named anything else. Even adding a letter to the name makes FCP not see the footage. CONTENTS-a or CONTENTS1...won't work.

And look into P2 LOG by www.imagineproducts.com. But I still think you need the CONTENTS folder. Video is tied to the AUDIO, which is in another folder. You simply cannot mess with the file structure at all if you want things to work.

Bill Pryor · July 26th, 2006, 09:13 PM

Interesting. So...if a guy used 8 gig cards and wanted to archive the footage onto DVDs, he would have to open the contents folder and pull out half the files, copy them to a DVD...then take the contents folder and the other document (I forgot its name already) and copy those to a second DVD.

THEN, when using those DVDs...you would pull the contents folder and other document off DVD2 first onto your computer's hard drive. Then pop in the second DVD and move its contents back into that contents folder where they came from before you separated them out for space purposed. And then it should work. Right...?

Once we have Blu-ray and/or HD-DVD burners, the 4.7 gig limit of a DVD will cease to be an issue. But until then, would my solution work ok?

Shane Ross · July 26th, 2006, 10:06 PM

Don't break up the files in the folder. Just plain don't. There is so much room for error there that I find it VERY unwise.

Back up the files to hard drives. You can get an empty firewire case for $30, and 250GB drives for $60-$80 and have PLENTY of room. Use the one case and just swap out the drives.

I'd never archive to DVD...scratches.

Bill Pryor · July 26th, 2006, 10:13 PM

I won't archive to firewire drives--I've seen too many of them die for no apparent reason. DVD is a lot safer. But if you can't break up the files, then that solution won't work, which is too bad. I thought I had something going there for awhile.

Barry Green · July 26th, 2006, 10:43 PM

Dual-layer DVDs might work for the 8gb's.

But seriously -- hard disks are the only way to go right now. They're so much faster, larger, and hassle-free. You can easily copy a card to two hard disks simultaneously, with no (or virtually no) slowdown when doing so, and then you have instant redundancy. And instant editability.

I think most of us are currently keeping the footage on hard disks, and for long-term storage at the end of a project, archiving onto something else (like LTO3 tape, 80 megabytes per second to a 400gb cartridge gives you the capability to back up 16 hours of 720/24pN footage on a single $65 tape, and those 16 hours could be theoretically, potentially backed up in as little as 80 minutes.

Hard drives aren't really a permanent storage medium, but they're the right medium for working with the footage, and they're a reasonable mid-term storage system.

DVD seemed at first glance like the way to go, since they're so cheap and a 4GB card fits on a 4.7GB DVD, and an 8GB card fits on a dual-layer DVD. But as a practical matter DVDs take too long to burn and you start collecting too many of them. Duplicated hard disks are just a much better media system to work with -- make a copy and store it at a second location, and the odds of you losing your footage drop to an incredibly low percentage (and, if the drive fails to "spin up", repair costs shouldn't be bad at all. I mean, they can recover data off of hard disks that have been in fires, off of hard disks that have been formatted and written over... how hard can it be for these data management companies to get a drive to "spin up" again?)

Eventually (as in, hopefully by the end of this year) we'll be archiving onto holographic blue laser drives from InPhase/Maxell/Toshiba. These'll be 300gb to 1.6 terabytes of storage, at 160mbps up to 1gbps, on a thin, affordable sheet of film. Permanent, fast, inexpensive archiving. Until that day arrives I'm plenty happy leaving the footage on hard disks or, were it necessary, getting ahold of an LTO2 or LTO3 drive and spitting out some cheap tape copies.

Shane F. Kelly · July 27th, 2006, 09:48 AM

Download P2 log
http://www.imagineproducts.com/P2log.htm
It should let you read and export the mxf files

Shane.

Bill Pryor · July 27th, 2006, 10:36 AM

It now appears to me that the P2 concept just won't quite work for the way we work. We need to file individual tapes. Those "tapes" could be DVDs, they could be something like XDCAM discs--something that can be used at different times like tapes. I thought I had something going with the DVD idea, because we could pop in a DVD and move the files off at any time. But if it's such a hassle to break up the files and store them that way, then it really isn't feasible. We have several editing computers and different people stick tapes in decks all the time and watch footage, or pull takes for various purposes. Any new system has got to be that simple.

Any transfer of data after a shoot is a hassle, but I've been trying to figure out if it might be worth the hassle for the benefits of the camera. After I saw the HVX, I really liked it, and the ability to do slow motion up to 60fps and better chroma keying of DVCPRO HD are two big deals that made it worth considering over XDCAM HD when we move out of DVCAM. However, it's starting to look as if the combination of low capacity, high price cards combined with the hassle of data managment won't work out in this case. I've really got to be able to go to a shelf, pick up a tape or disc and load it up quickly and easily.

Barry Green · July 27th, 2006, 11:02 AM

That's what the LTO tapes would be for. Archive 'em on data tape, restore at 6x to 24x realtime. You're looking at a cost of less than $4 per hour of storage, and restoring clips will be way faster than from video tape. Plus you don't need a dedicated video deck either.

Or keep footage on a Lacie 2-terabyte ethernet drive. Instead of restoring anything, just play the file. It's already online. No time spent looking for a tape, searching through the tape to find a clip, digitizing, etc. It's all active, all the time. A 2-terabyte Lacie ethernet disk would hold over 80 hours of 720/24pN footage, at a cost of $20/hour (and that's based on today's prices; check back in three months and that cost should have plummeted). And because it's a network drive, the footage is available to everyone in the organization, right on their desktop.

What would tape costs be to archive that on videotape? Consider DVCPRO-HD tape at about $40/hour, and then the cost of needing a $25,000 deck to play it from. And the storage space to fit about 120 tapes somewhere (assuming 46-minute tapes). Versus having 5 LTO3 tapes and a single small ethernet terabyte drive.

So actually you could implement both solutions -- keep the footage live on 2-terabyte ethernet network drives, and archive off LTO3 tapes to store somewhere else (in case of fire/flood/vandalism/theft etc). You'd have your "permanent copy" stored somewhere at a cost of $4/hour, and you'd have your instant-access online vault at a cost of around $20/hour. You could implement both solutions for less than half the cost of going with a traditional tape-based system.

Once you start to factor in the cost savings versus tape, and then the hours saved in not having to find & retrieve footage (because it's already instantly available, it's already currently on-line), and then multiply that by how exponentially larger and dramatically cheaper computer storage becomes over time...

... well, maybe it works for you, maybe it doesn't. It takes a new way of thinking to approach it, that's for sure. Frankly I can't see being saddled with tapes anymore, I think the new paradigm just makes a whole lot more sense, from a time and money and workflow perspective. But that doesn't make it ideal for everyone.

Bill Pryor · July 27th, 2006, 11:26 AM

What would be the difference between LTO and DLT tapes in this type of thing? We already have a DLT deck.

Barry Green · July 27th, 2006, 01:38 PM

Speed speed speed. LTO3 is 80 megabytes per second, DLT is around 3 megabytes per second.

And capacity, capacity, capacity: LTO3 is 400 gigabytes per cartridge, LTO is 40.

So it depends on what DLT drive you're talking about, there are also Super-DLT drives and such. But in a base comparison of LTO3 vs. regular DLT, we're talking Ferrari vs. golf cart.

Bill Pryor · July 27th, 2006, 01:56 PM

We just use an ordinary DLT for backup. Tapes are 40 gigs, 80 uncompressed. Speed wouldn't be an issue because things could run at night. This could be a temporary thing that might work until we decided to move all the way into P2 with a second camera and LTO drives or whatever we might need in a year or so. What I'm trying to figure out now is if there's a way I can make it work on a limited basis in the short term.

July 26th, 2006, 04:35 AM	#1
Steve Reid New Boot Join Date: Oct 2005 Location: Edinburgh, Scotland Posts: 5	P2 card dump reading problems on Mac and FCP 5.1, Help! Hi all, We did a shoot other week on our new HXV200 and dumped cards to PC laptop and external HD, but now that has turned bad, corrupted and unreadable. But we did have insight to make a 2nd back-up of cards to DVD. Now we have put on Mac HD and can access them using Final Cut Pro 5.1, we can see thumbnails in P2 import window, but when select clip thumbnail trying to actually import footage I get an error message - "one or more P2 clips failed to import, probable incomplete or corrupt clips." Has anyone else had difficulty in reading transported/archived/spit P2 folders in Final Cut Pro 5.1 or from any OS. How I archived and reasembled was: Dump P2 card to laptop (7-8GB) Open "contents" folder then "Video" folder and selected lower half of video .mxf clips aprox 4GB and burnt that to DVD, then moved those to another location on PC HD and burnt another DVD with all the P2 contents folders in same file structure, nothing moved. Reasemble consited of, First copy the DVD with main contents to Mac HD, then copy DVD contents containing just half the video .mxf files and place those back in "Video" folder. Now this should be exactly as how it was when dumped originally, nothing lost added, moved just reasembled as per orig. Any ideas folks would be great. Even involving PC software, any way to get these files readable really! It seems like everything is all there just FCP doesn't like the files directory sttructure or how files have been moved and recompiled. Is there a Mac program - pure .mxf reader and exporter to a .mov etc or PC even but would need to be exported to .movs not avi's or wmvs. Thanks

July 27th, 2006, 09:48 AM	#7
Shane F. Kelly Tourist Join Date: Jan 2006 Posts: 3	reading mxfs Download P2 log http://www.imagineproducts.com/P2log.htm It should let you read and export the mxf files Shane.

July 27th, 2006, 11:26 AM	#10
Bill Pryor Contributor Join Date: Jan 2003 Location: Kansas City, MO Posts: 4,449	What would be the difference between LTO and DLT tapes in this type of thing? We already have a DLT deck. Last edited by Bill Pryor; July 27th, 2006 at 12:12 PM.

July 26th, 2006, 04:13 PM	#2
Shane Ross Major Player Join Date: Apr 2004 Location: Los Angeles, USA Posts: 539	You cannot grab just pieces of footage from the CONTENTS folder. In order for FCP to be able to import the footage, it needs the ENTIRE contents folder, nothing moved, nothing changed. And the CONTENTS folder cannot be named anything else. Even adding a letter to the name makes FCP not see the footage. CONTENTS-a or CONTENTS1...won't work. And look into P2 LOG by www.imagineproducts.com. But I still think you need the CONTENTS folder. Video is tied to the AUDIO, which is in another folder. You simply cannot mess with the file structure at all if you want things to work.

July 26th, 2006, 09:13 PM	#3
Bill Pryor Contributor Join Date: Jan 2003 Location: Kansas City, MO Posts: 4,449	Interesting. So...if a guy used 8 gig cards and wanted to archive the footage onto DVDs, he would have to open the contents folder and pull out half the files, copy them to a DVD...then take the contents folder and the other document (I forgot its name already) and copy those to a second DVD. THEN, when using those DVDs...you would pull the contents folder and other document off DVD2 first onto your computer's hard drive. Then pop in the second DVD and move its contents back into that contents folder where they came from before you separated them out for space purposed. And then it should work. Right...? Once we have Blu-ray and/or HD-DVD burners, the 4.7 gig limit of a DVD will cease to be an issue. But until then, would my solution work ok?

July 26th, 2006, 10:06 PM	#4
Shane Ross Major Player Join Date: Apr 2004 Location: Los Angeles, USA Posts: 539	Don't break up the files in the folder. Just plain don't. There is so much room for error there that I find it VERY unwise. Back up the files to hard drives. You can get an empty firewire case for $30, and 250GB drives for $60-$80 and have PLENTY of room. Use the one case and just swap out the drives. I'd never archive to DVD...scratches.

July 26th, 2006, 10:13 PM	#5
Bill Pryor Contributor Join Date: Jan 2003 Location: Kansas City, MO Posts: 4,449	I won't archive to firewire drives--I've seen too many of them die for no apparent reason. DVD is a lot safer. But if you can't break up the files, then that solution won't work, which is too bad. I thought I had something going there for awhile.

July 26th, 2006, 10:43 PM	#6
Barry Green Barry Wan Kenobi Join Date: Jul 2003 Location: North Carolina Posts: 3,863	Dual-layer DVDs might work for the 8gb's. But seriously -- hard disks are the only way to go right now. They're so much faster, larger, and hassle-free. You can easily copy a card to two hard disks simultaneously, with no (or virtually no) slowdown when doing so, and then you have instant redundancy. And instant editability. I think most of us are currently keeping the footage on hard disks, and for long-term storage at the end of a project, archiving onto something else (like LTO3 tape, 80 megabytes per second to a 400gb cartridge gives you the capability to back up 16 hours of 720/24pN footage on a single $65 tape, and those 16 hours could be theoretically, potentially backed up in as little as 80 minutes. Hard drives aren't really a permanent storage medium, but they're the right medium for working with the footage, and they're a reasonable mid-term storage system. DVD seemed at first glance like the way to go, since they're so cheap and a 4GB card fits on a 4.7GB DVD, and an 8GB card fits on a dual-layer DVD. But as a practical matter DVDs take too long to burn and you start collecting too many of them. Duplicated hard disks are just a much better media system to work with -- make a copy and store it at a second location, and the odds of you losing your footage drop to an incredibly low percentage (and, if the drive fails to "spin up", repair costs shouldn't be bad at all. I mean, they can recover data off of hard disks that have been in fires, off of hard disks that have been formatted and written over... how hard can it be for these data management companies to get a drive to "spin up" again?) Eventually (as in, hopefully by the end of this year) we'll be archiving onto holographic blue laser drives from InPhase/Maxell/Toshiba. These'll be 300gb to 1.6 terabytes of storage, at 160mbps up to 1gbps, on a thin, affordable sheet of film. Permanent, fast, inexpensive archiving. Until that day arrives I'm plenty happy leaving the footage on hard disks or, were it necessary, getting ahold of an LTO2 or LTO3 drive and spitting out some cheap tape copies.

July 27th, 2006, 10:36 AM	#8
Bill Pryor Contributor Join Date: Jan 2003 Location: Kansas City, MO Posts: 4,449	It now appears to me that the P2 concept just won't quite work for the way we work. We need to file individual tapes. Those "tapes" could be DVDs, they could be something like XDCAM discs--something that can be used at different times like tapes. I thought I had something going with the DVD idea, because we could pop in a DVD and move the files off at any time. But if it's such a hassle to break up the files and store them that way, then it really isn't feasible. We have several editing computers and different people stick tapes in decks all the time and watch footage, or pull takes for various purposes. Any new system has got to be that simple. Any transfer of data after a shoot is a hassle, but I've been trying to figure out if it might be worth the hassle for the benefits of the camera. After I saw the HVX, I really liked it, and the ability to do slow motion up to 60fps and better chroma keying of DVCPRO HD are two big deals that made it worth considering over XDCAM HD when we move out of DVCAM. However, it's starting to look as if the combination of low capacity, high price cards combined with the hassle of data managment won't work out in this case. I've really got to be able to go to a shelf, pick up a tape or disc and load it up quickly and easily.

July 27th, 2006, 11:02 AM	#9
Barry Green Barry Wan Kenobi Join Date: Jul 2003 Location: North Carolina Posts: 3,863	That's what the LTO tapes would be for. Archive 'em on data tape, restore at 6x to 24x realtime. You're looking at a cost of less than $4 per hour of storage, and restoring clips will be way faster than from video tape. Plus you don't need a dedicated video deck either. Or keep footage on a Lacie 2-terabyte ethernet drive. Instead of restoring anything, just play the file. It's already online. No time spent looking for a tape, searching through the tape to find a clip, digitizing, etc. It's all active, all the time. A 2-terabyte Lacie ethernet disk would hold over 80 hours of 720/24pN footage, at a cost of $20/hour (and that's based on today's prices; check back in three months and that cost should have plummeted). And because it's a network drive, the footage is available to everyone in the organization, right on their desktop. What would tape costs be to archive that on videotape? Consider DVCPRO-HD tape at about $40/hour, and then the cost of needing a $25,000 deck to play it from. And the storage space to fit about 120 tapes somewhere (assuming 46-minute tapes). Versus having 5 LTO3 tapes and a single small ethernet terabyte drive. So actually you could implement both solutions -- keep the footage live on 2-terabyte ethernet network drives, and archive off LTO3 tapes to store somewhere else (in case of fire/flood/vandalism/theft etc). You'd have your "permanent copy" stored somewhere at a cost of $4/hour, and you'd have your instant-access online vault at a cost of around $20/hour. You could implement both solutions for less than half the cost of going with a traditional tape-based system. Once you start to factor in the cost savings versus tape, and then the hours saved in not having to find & retrieve footage (because it's already instantly available, it's already currently on-line), and then multiply that by how exponentially larger and dramatically cheaper computer storage becomes over time... ... well, maybe it works for you, maybe it doesn't. It takes a new way of thinking to approach it, that's for sure. Frankly I can't see being saddled with tapes anymore, I think the new paradigm just makes a whole lot more sense, from a time and money and workflow perspective. But that doesn't make it ideal for everyone.

July 27th, 2006, 01:38 PM	#11
Barry Green Barry Wan Kenobi Join Date: Jul 2003 Location: North Carolina Posts: 3,863	Speed speed speed. LTO3 is 80 megabytes per second, DLT is around 3 megabytes per second. And capacity, capacity, capacity: LTO3 is 400 gigabytes per cartridge, LTO is 40. So it depends on what DLT drive you're talking about, there are also Super-DLT drives and such. But in a base comparison of LTO3 vs. regular DLT, we're talking Ferrari vs. golf cart.

July 27th, 2006, 01:56 PM	#12
Bill Pryor Contributor Join Date: Jan 2003 Location: Kansas City, MO Posts: 4,449	We just use an ordinary DLT for backup. Tapes are 40 gigs, 80 uncompressed. Speed wouldn't be an issue because things could run at night. This could be a temporary thing that might work until we decided to move all the way into P2 with a second camera and LTO drives or whatever we might need in a year or so. What I'm trying to figure out now is if there's a way I can make it work on a limited basis in the short term.