Corrupted hd100 footage - What is up?

Bob Diesso · July 25th, 2006, 05:28 AM

Hi Keith - My personal experience with the HD100 has been virtually perfect and I'm hoping your experiences with the replacement camera are more successful.

Please consider one point: "Dropouts" in MPEG-encoded recordings have a very different character than dropouts in analog or DV recording systems. MPEG uses long GOP recording, meaning that compression is spread over many frames (15 in Sony's case, 6 in JVC's case). Loss of an "I" frame can corrupt the subsequent 14 (or 5) frames. Loss of a "P" (predictive) frame can cause loss of motion vectors, resulting in "fractured" frames (examples posted recently elsewhere on dvinfo.com).

HDV cameras and decks have powerful error correction and error concealment technology that, within limits, compensate for flaws. But once HDV data is spooled out to an NLE, the NLE's error correction / concealment takes over. Some NLE's are better than others in this area, which may be why some people see errors displayed on their NLE that they do not see during tape playback via a camera or deck.

Visible "glitches" during tape playback may mean the camera or deck is struggling to conceal gross errors. In this case, I have found the tape stock is frequently the source of the problem. I personally have had good experience with Panasonic and JVC tape stock, but even the best brands can and DO have occasional bad "batches". During manufacture, individual tapes are slit from a huge sheet (called a "web"). From experience, if one tape from a box of a dozen has problems, there is a high probability ALL the tapes from that box have the same problems. I've found it's best to return (or abandon) the remaining tapes, buying a NEW box with a DIFFERENT run number (stamped on the master carton and each tape).

You've posted more than 30 messages detailing your unfortunate experiences and I've read them all, privately wondering if the underlying cause of your problem is the tape stock being used. As a suggestion, you may wish to consider buying completely FRESH stock to use with the new camera.

Hopefully your HD100 experiences will soon become as successful as mine.

No, I'm not an engineer. I just play one on the Internet.

Bob Diesso · July 25th, 2006, 05:54 AM

Reading the Canon, Sony, JVC and Panasonic boards, it seems no HD system - including optical media and flash memory - is completely immune from data corruption. As I read this and similar HDV strings, it brings to mind a learning curve that many of us may be going through. Dropouts in MPEG recording have very different character than dropouts in DV or analog recordings. It would seem that some people may simply be experiencing what data loss can look like in MPEG recording.

For those who might claim "tapeless is the way to go", reading other forums it appears folks working with optical and flash media have their own issues. Seems the world's rarely perfect for any of us.

Though I lack universal experience, some HDV cameras check for data errors and will alert the operator of problems (pressing "return" on some cameras quickly plays a few seconds of a previously-recorded segment, including checking for data errors).

One thought possibly worth consideration, especially for "mission critical" shoots.

K. Forman · July 25th, 2006, 06:38 AM

You know what? I really don't care what the problem is. Fact of the matter is very simple. I spent a buttload of money on a camera, with the understanding that it will work properly. I either want that camera to work, my money back, or somebody's actual head on a pike, to warn other foolish people away. Not too much to ask, is it? An "occasional' glitch is one thing, a wasted tape is unacceptable.

I just recieved the replacement yesterday, and because of bad weather, didn't get a chance to do much more than adjust menu settings. As soon as the coffee kicks in, I'll go chase the birds or something.

Dennis Stevens · July 25th, 2006, 07:48 AM

Quote:

Originally Posted by Bob Diesso

Please consider one point: "Dropouts" in MPEG-encoded recordings have a very different character than dropouts in analog or DV recording systems.

I'm the first to admit I haven't worked with MPEG very much, so I'm totally open to the idea that some of this stuff is my unfamiliarity with a new camera, new format, etc.

However, I did use a totally different tape stock, and got the same results.

I'm also open to the idea 'every camera has it's quirks you need to learn'. OK, what are they? If there are work arounds to the camera, things I can be aware of and account for, fine.

But it's sort of hard to use the camera when you can't count on 10 minutes of video without some major drop outs.

It's hard to interrupt the ceremony and yell 'Cut! Had a frame drop out. Can you guys go through the vow again? And prop up Uncle Fred, he hadn't passed out in the earlier footage, and I need it to match.'

K. Forman · July 25th, 2006, 07:52 AM

Quote:

Originally Posted by Dennis Stevens

It's hard to interrupt the ceremony and yell 'Cut! Had a frame drop out. Can you guys go through the vow again? And prop up Uncle Fred, he hadn't passed out in the earlier footage, and I need it to match.'

Problem is, half the time you don't even notice any issues until you go to capture the footage. It's even harder to get everybody back the next day, because last night you noticed huge gaps and need to reshoot the whole thing again.

Jerry Porter · July 25th, 2006, 08:34 AM

Quote:

Originally Posted by Keith Forman

Problem is, half the time you don't even notice any issues until you go to capture the footage. It's even harder to get everybody back the next day, because last night you noticed huge gaps and need to reshoot the whole thing again.

Hey shouldn't you be out finding a bird to chase. I WANNA KNKOW IF THE NEW ONE WORKS FOR YOU!!! I haven't been able to view any of my footage on the computer, because it's being assembled and you guys are killing me!! GO PLAY WITH THE THING!! Oh and thanks for this thread because as soon as my new editing system is up there will be TONS of test footage reviewed.

K. Forman · July 25th, 2006, 08:41 AM

Geez... you're a bigger nag than my wife! Cut me some slack, dude, I'll get to it ;)

Bill Edmunds · July 25th, 2006, 08:44 AM

Quote:

Originally Posted by Jerry Porter

Oh and thanks for this thread because as soon as my new editing system is up there will be TONS of test footage reviewed.

Sweet. You had better keep that promise!

K. Forman · July 25th, 2006, 10:23 AM

Quote:

Originally Posted by Jerry Porter

Hey shouldn't you be out finding a bird to chase. I WANNA KNKOW IF THE NEW ONE WORKS FOR YOU!!! I haven't been able to view any of my footage on the computer, because it's being assembled and you guys are killing me!! GO PLAY WITH THE THING!! Oh and thanks for this thread because as soon as my new editing system is up there will be TONS of test footage reviewed.

Ok, the first results are in- Aside from the footage looking like it had been shot by Katherine Hepburn, it looks beautiful. No corruption, no red, just very shaky extreme zooming from the shoulder. I even have some focused shots ;)

And I just recieved a callback from Jason at T.T., also eager to find out the results. We all be pleased at this point, and Jason is even tossing in a few HD tapes for the trouble! Again, you like, totally ROCK Dudes!

Don Donatello · July 25th, 2006, 10:43 AM

RE: RED FRAMES

are you using Vegas ?
Vegas uses the color red on clips/frames that have errors/corrupt (which is usually head/tails of m2t clips that have been split/cut depending on if they are 15 GOP or 6 GOP= not a big deal .. it's when the whole clip is RED you should be concerned ) ..another NLE may or may not play the same clip/frames ..

K. Forman · July 25th, 2006, 10:58 AM

Even if I had been using Vegas, as opposed to actually using PP2, the red would not have been on the tape, unless I exported them there. But, the problem was elsewhere.

And I just found out via Jason, our cameras will be disected, making us test subjects! Yaaaaaayyy GUINEA PIGS! ;)

Jonathan Nelson · July 25th, 2006, 01:56 PM

Quote:

Originally Posted by Keith Forman

Ok, the first results are in- Aside from the footage looking like it had been shot by Katherine Hepburn, it looks beautiful. No corruption, no red, just very shaky extreme zooming from the shoulder. I even have some focused shots ;)

And I just recieved a callback from Jason at T.T., also eager to find out the results. We all be pleased at this point, and Jason is even tossing in a few HD tapes for the trouble! Again, you like, totally ROCK Dudes!

Yay, now we can all be happy again!

Too bad you guys live so far away, it would be cool to have some kind of party with lots of beer.

I did some freelance stuff yesterday and all the footage turned out beautiful as well.

Tim Dashwood · July 25th, 2006, 03:38 PM

Quote:

Originally Posted by Keith Forman

Even if I had been using Vegas, as opposed to actually using PP2, the red would not have been on the tape, unless I exported them there. But, the problem was elsewhere.

EXACTLY!
Keith and Dennis, I have been following your "red frame" problems closely and attempting to get to the bottom of it through the information provided by you, talking with some video engineers I know (JVC and Sony specialists,) and by independent testing.

By the way, here are the four other threads that seem to discuss the same topic. These issues are much easier to solve if we maintain single threads on each topic (I should probably merge these double-posts all into one thread to avoid future cross-postings. Man, that's going to take a long time! :)
http://www.dvinfo.net/conf/showthread.php?t=71554
http://www.dvinfo.net/conf/showthread.php?t=72137
http://www.dvinfo.net/conf/showthread.php?t=72084
http://www.dvinfo.net/conf/showthread.php?t=71195

Keith & Dennis, I have come to the conclusion that your problem could not have possibly been the CCDs. Even though that is what you were told by JVC tech support via TT, it just doesn't make any sense to me or the engineers I've had contact with.

There are two possible logical causes for the origination of your dropouts:

1) You have a head-clog which could probably be solved by running a head cleaning cassette.

2) You have a bad batch of tape.

Now on to the question of the "red frames." Since these red frames do not appear on the camera original tapes, and they do not appear on a m2t captured with "CapDVHS," and since both of you only see the red frames when capturing footage with dropouts within Premiere Pro 2, it seems logical to me that Premiere Pro 2 is creating the red frames everytime it encounters a dropout. Out of curiousity have you both updated the application to the latest version?

Dennis was kind enough to capture an m2t for me to test and yes indeed there are a couple of dropouts on it causing minimal data loss on a few frames, but this doesn't cause any similar problems in other NLEs (other than the data-loss itself.)
My logical conclusion is that the culprit is partial data loss due to a dropout (clogged heads or bad batch of tape) and the problem was exacerbated by the way Premiere Pro 2 handles dropouts. Can anyone else with PP2 please test this theory if you happen to have a tape with dropouts?
Maybe it would help to post this problem on the Official Adobe PP2 forum.

I'm still waiting until I actually find a dropout on a tape of my own to fully test this theory. Fortunately I've used over 90 of the JVC ProHD tapes this summer without a dropout problem. "Knock on wood."
I followed procedure by running a head cleaning tape as soon as the brand-new HD100A body was taken out of the box, and I have since only used JVC tape. I also run the head cleaner once for every 10 tapes (approximately.)
As Bob Diesso explained earlier, all tape stock can be victim to data loss, and unfortunately the characteristics of HDV dropouts can potentially be more disasterous for our footage than we were used to with the interframe DV codec.

I'm glad your new unit is working well for you Keith, and I suggest sticking with JVC ProHD stock, and get yourself a head-cleaning cassette.
Dennis, I hope you get your new camera soon as well and also follow the same advise. As we have discussed many times before, "cross-pollination" of tapes stocks on any camera can lead to head clogs and therefore dropouts.

Good luck.

Jonathan Nelson · July 25th, 2006, 04:16 PM

I don't know much about how ppro2.0 handles data but I made a mistake yesterday and captured hdv-SD60p and hdv-30p within the same capture.

I captured the SD60p in a timeline designed for hdv-30p. In ppro2.0, the SD footage looked great but the hdv-30p was all totally corrupted looking with horizontal lines everywhere. This is not a problem becuase I knew that I should not have captured both SD60p and hdv-30p in the same capture session. There was nothing wrong with the original hdv-30p footage, I started a new session and it came out great as expected.

The odd thing is that when I go back to the original m2t files, the hdv-30p is downrezed to SD and it looks GREAT! Its so funny because it downrezed it perfectly but it looks completely glitched out on the ppro2.0 timeline.

So odd, maybe you guys know something about this?

Ram Ganesh · July 25th, 2006, 06:35 PM

When I capture using PPro2 - I get the red frames - but in Liquid Capture (I think) I didnt see any (same footage)

I assumed it was a drop frame placeholder...

July 25th, 2006, 05:28 AM	#61
Bob Diesso New Boot Join Date: Oct 2005 Posts: 16	You may wish to consider fresh tape Hi Keith - My personal experience with the HD100 has been virtually perfect and I'm hoping your experiences with the replacement camera are more successful. Please consider one point: "Dropouts" in MPEG-encoded recordings have a very different character than dropouts in analog or DV recording systems. MPEG uses long GOP recording, meaning that compression is spread over many frames (15 in Sony's case, 6 in JVC's case). Loss of an "I" frame can corrupt the subsequent 14 (or 5) frames. Loss of a "P" (predictive) frame can cause loss of motion vectors, resulting in "fractured" frames (examples posted recently elsewhere on dvinfo.com). HDV cameras and decks have powerful error correction and error concealment technology that, within limits, compensate for flaws. But once HDV data is spooled out to an NLE, the NLE's error correction / concealment takes over. Some NLE's are better than others in this area, which may be why some people see errors displayed on their NLE that they do not see during tape playback via a camera or deck. Visible "glitches" during tape playback may mean the camera or deck is struggling to conceal gross errors. In this case, I have found the tape stock is frequently the source of the problem. I personally have had good experience with Panasonic and JVC tape stock, but even the best brands can and DO have occasional bad "batches". During manufacture, individual tapes are slit from a huge sheet (called a "web"). From experience, if one tape from a box of a dozen has problems, there is a high probability ALL the tapes from that box have the same problems. I've found it's best to return (or abandon) the remaining tapes, buying a NEW box with a DIFFERENT run number (stamped on the master carton and each tape). You've posted more than 30 messages detailing your unfortunate experiences and I've read them all, privately wondering if the underlying cause of your problem is the tape stock being used. As a suggestion, you may wish to consider buying completely FRESH stock to use with the new camera. Hopefully your HD100 experiences will soon become as successful as mine. No, I'm not an engineer. I just play one on the Internet.

July 25th, 2006, 06:38 AM	#63
K. Forman Capt. Quirk Join Date: Apr 2002 Location: Middle of the woods in Georgia Posts: 3,596	You know what? I really don't care what the problem is. Fact of the matter is very simple. I spent a buttload of money on a camera, with the understanding that it will work properly. I either want that camera to work, my money back, or somebody's actual head on a pike, to warn other foolish people away. Not too much to ask, is it? An "occasional' glitch is one thing, a wasted tape is unacceptable. I just recieved the replacement yesterday, and because of bad weather, didn't get a chance to do much more than adjust menu settings. As soon as the coffee kicks in, I'll go chase the birds or something. __________________ www.SmokeWagonLeather.us

July 25th, 2006, 08:41 AM	#67
K. Forman Capt. Quirk Join Date: Apr 2002 Location: Middle of the woods in Georgia Posts: 3,596	Geez... you're a bigger nag than my wife! Cut me some slack, dude, I'll get to it ;) __________________ www.SmokeWagonLeather.us

July 25th, 2006, 10:58 AM	#71
K. Forman Capt. Quirk Join Date: Apr 2002 Location: Middle of the woods in Georgia Posts: 3,596	Even if I had been using Vegas, as opposed to actually using PP2, the red would not have been on the tape, unless I exported them there. But, the problem was elsewhere. And I just found out via Jason, our cameras will be disected, making us test subjects! Yaaaaaayyy GUINEA PIGS! ;) __________________ www.SmokeWagonLeather.us

July 25th, 2006, 06:35 PM	#75
Ram Ganesh Regular Crew Join Date: Mar 2005 Location: Seattle, WA Posts: 175	When I capture using PPro2 - I get the red frames - but in Liquid Capture (I think) I didnt see any (same footage) I assumed it was a drop frame placeholder... __________________ http://www.wizedot.com

July 25th, 2006, 05:54 AM	#62
Bob Diesso New Boot Join Date: Oct 2005 Posts: 16	Reading the Canon, Sony, JVC and Panasonic boards, it seems no HD system - including optical media and flash memory - is completely immune from data corruption. As I read this and similar HDV strings, it brings to mind a learning curve that many of us may be going through. Dropouts in MPEG recording have very different character than dropouts in DV or analog recordings. It would seem that some people may simply be experiencing what data loss can look like in MPEG recording. For those who might claim "tapeless is the way to go", reading other forums it appears folks working with optical and flash media have their own issues. Seems the world's rarely perfect for any of us. Though I lack universal experience, some HDV cameras check for data errors and will alert the operator of problems (pressing "return" on some cameras quickly plays a few seconds of a previously-recorded segment, including checking for data errors). One thought possibly worth consideration, especially for "mission critical" shoots.

July 25th, 2006, 10:43 AM	#70
Don Donatello Hawaiian Shirt Mogul Join Date: Nov 2001 Location: northern cailfornia Posts: 1,261	RE: RED FRAMES are you using Vegas ? Vegas uses the color red on clips/frames that have errors/corrupt (which is usually head/tails of m2t clips that have been split/cut depending on if they are 15 GOP or 6 GOP= not a big deal .. it's when the whole clip is RED you should be concerned ) ..another NLE may or may not play the same clip/frames ..

July 25th, 2006, 04:16 PM	#74
Jonathan Nelson Major Player Join Date: Apr 2006 Location: Kingman Arizona Posts: 298	I don't know much about how ppro2.0 handles data but I made a mistake yesterday and captured hdv-SD60p and hdv-30p within the same capture. I captured the SD60p in a timeline designed for hdv-30p. In ppro2.0, the SD footage looked great but the hdv-30p was all totally corrupted looking with horizontal lines everywhere. This is not a problem becuase I knew that I should not have captured both SD60p and hdv-30p in the same capture session. There was nothing wrong with the original hdv-30p footage, I started a new session and it came out great as expected. The odd thing is that when I go back to the original m2t files, the hdv-30p is downrezed to SD and it looks GREAT! Its so funny because it downrezed it perfectly but it looks completely glitched out on the ppro2.0 timeline. So odd, maybe you guys know something about this?