24F vs 30F in A1 vs HV20

[Mikko Lopponen]

Originally Posted by Geoff Dills

that is interesting that it shows up 23.976 interlaced as that's not a supported HDV format from what I can tell.

That's the reason it only works with canons equipment. I believe that the stream is 23.976. If someone would upload a raw clip, I can check it out.

[Brian Brown]

The "F" footage is clearly not interlaced, captured as discrete frames and not fields. I can play back m2t's or mpegs, created by my A1 or others', and in any viewer and see nary a field or interlacing artifact.

I have downloaded a few raw HV20 clips and I can see the interlacing, however.

Since 1080p/24f footage isn't HDV spec, maybe this is why we're seeing various applications report various things. The big question is whether pulldown is being applied to that stream via Firewire on only the HV20 or also on the A1.

I vote for "only on the HV20", BTW. And now I'm concerned on cutting my A1 footage with any HV20 footage.

And this is made all the more maddening because the HV20 is a truly a progressive camera!

Brian

[Tom Roper]

Originally Posted by Mikko Lopponen

That's the reason it only works with canons equipment. I believe that the stream is 23.976. If someone would upload a raw clip, I can check it out.

It only works with Canon HDV *cameras*. The 24F m2t cuts perfectly for me on a 60i timeline with ANY mpeg video.

And this is why I think Barlow has it right, flags. Yes it's 24 frames per second progressive encoded, but with repeat flags. Pulldown! Pulldown should not ever be actually added as encoded frames because it wastes compression efficiency. The repeat flags are there, and the decoder does with them what it needs to do. If it needs to display at 60 hz refresh, then the decoder reads the flags and adds pulldown. If the refresh is a multiple of 24hz then it doesn't.

One mpeg editor I use has an input window and an output window. On the input window, you can step through 24 discrete frames, but the frame counter skips frame numbers but indexes to 30, like 1,2,4,5,6,7,9...Meanwhile, the frame counter for the output window also counts to 30, but instead of skipping counts, the 4th frame in every sequence is repeated, so 30 whole frames are displayed. This is without re-encoding anything, just looking at the editor windows.

It cuts perfectly with 60i and is not re-encoded. How would it do that without pulldown flags?

[Tom Roper]

As for the footage clearly not interlaced? If both fields originate from the same progressively captured image, then segmenting them into two separate fields that are combined by the decoder should reveal no deinterlace artifacts. Thus it's reported by various software apps to be top field first, or even bottom field first, because field order is irrelevant for 24 discrete progressive frames.

So that's my vote, 24 PsF with pulldown flags. It would seem to explain everything.

[Brian Brown]

Tom, that's a great explanation and makes sense now. Maybe my Premiere Pro removes the flags and pulldown of the original stream from the Firewire. If I capture the stream with something else, it'd be interesting to see if the pulldown (and 29.97fps) remains.

I'm still wondering if the HV20's 24p footage is streamed differently than the A1's 24f.

Thanks,
Brian

[Pete Bauer]

Tom, what you're postulating is possible but leaves these nagging thoughts:

- From a bit budget standpoint, it is inefficient to take a progressive frame and pull it down to 2:3 or 2:3:3:2 interlaced; Canon has done really well with minimizing artifacts at HDV bitrates. When they were designing F-Mode why would they choose to strain the bit budget and make their job that much harder?

- Canon explicitly says in their manual that F-Mode, except for SD 24F, is sent out through 1394 as progessive.

- How do you know it isn't the other way around: that the camera sends a data stream that is progressive, but some NLEs pull it down to 60i and report it as such (whereas others don't)?

I guess only Canon and software engineers for the NLE's and third party vendors like Cineform really know, since they have the tech data.

[Brian Brown]

Perhaps the pulldown is added post-tape. So the tape is encoded at 24p or 30p, but pulldown is added before it hits the Firewire port. I know that pulldown must be added to the analog component out, since F modes play fine on any HDTV. Maybe Canon does the same to the streams from the Firewire port for maximum compatibility. And some NLEs are able to discard/ignore the added fields and pulldown.

Is this possible????

Brian Brown

[Tom Roper]

Originally Posted by Pete Bauer

Tom, what you're postulating is possible but leaves these nagging thoughts:

- From a bit budget standpoint, it is inefficient to take a progressive frame and pull it down to 2:3 or 2:3:3:2 interlaced; Canon has done really well with minimizing artifacts at HDV bitrates. When they were designing F-Mode why would they choose to strain the bit budget and make their job that much harder?

If the repeat flag is just a particular hex bit address within a word in side the frame header, then setting or clearing that bit would have no consequence on the overhead since the entire word would define the instruction.

- Canon explicitly says in their manual that F-Mode, except for SD 24F, is sent out through 1394 as progessive.

Agree. Just not sure if PsF'd.

- How do you know it isn't the other way around: that the camera sends a data stream that is progressive, but some NLEs pull it down to 60i and report it as such (whereas others don't)?

It's reported by a standalone capture utility, CapDVHS to be 29.97 fps. I posted a screen shot of this earlier. I'm not looking at it now, (writing this from my cell phone) but I think it gives a frame count as well.

[Tom Roper]

I believe I can close the loop by answering the one final remaining open question. Thank you all for making this thread lively and civil.

The question was, why on page 40 of the Canon XH-A1/G1 English language manual is there wording that states for HDV, 24 progressive frames are encoded but for SD pulldown is added?

The answer is that HDV is mpeg and SD is DV. HDV uses one I-Frame (information frame) for each 15 GOP structure. With DV, there are no GOPs, all frames are I-frames.

DV has no 24 fps spec. All DV is 29.97 frames per second, 60i. In order to wrap 24F inside of DV, you have to add pulldown frames so that all 30 frames in the DV stream are filled. With DV 60i, there are always 30 frames anyway, so there is no compression penalty, no difference whatsoever in file size whichever way you go. You’ve always got 30 frames to fill whether its 60i or 24F.

But with HDV and its mpeg 15 GOP compression scheme, you can have 24 or 30 frames per second. But rather than forcing the codec to write redundant frames for the pulldown, it sets a flag to tell the decoder that the next frame in sequence is to be repeated for the pulldown.

When you place the m2t file onto the 24fps timeline of your NLE, it clears the flag that was set inside each header. If you place the 24F m2t file onto a 30fps timeline of your NLE, it sets the repeat flag, (which was already set anyway by the Canon mpeg encoder). No re-encoding of the frame is required either way.

This is a very efficient scheme, because it allows the decoder of the playback device to decide whether it needs to read pulldown flags to match the refresh rate, or does not need to.

There is no disputing that 24F is progressive encoding. There never has been any question. The question is, is it segmented frames or whole frames. They have to be segmented for DV, it’s completely logical to me that it is segmented for HDV also, particularly in view of the interlaced CCD image sensors used. But either way, there is NO NEGATIVE consequence to splitting a frame into segmented fields that have no temporal shift. Every last single bit is accounted for, and reconstructed into the whole from which it was first sired.

Why then the 10-12% loss in vertical resolution in 24F mode compared to 60i? Most likely it is from the 24F green chroma channel pixel shift.

One particular mpeg editor I use does not require you to choose a 24 or 30fps timeline. You just drop whatever mpeg you have onto the generic timeline. You can even mix them on the same timeline. It does not set or clear the pulldown flags. It leaves each flag in the state it found it. It’s up to the playback device to decide whether to read or ignore the repeat flag, depending on whether it needs to repeat the pulldown frames to match the refresh rate of the display, or not.

[Brian Brown]

Great explanation, Tom. Thanks for all of your time spent posting on this pulldown issue.

Brian Brown

[Salah Baker]

Originally Posted by Pete Bauer

I guess only Canon and software engineers for the NLE's and third party vendors like Cineform really know, since they have the tech data.

http://www.dvinfo.net/conf/showthread.php?t=90148
cool

love CineForm Btw

[Pete Bauer]

Tom, I guess I still don't get where this leads us? Even if it turns out that F-Mode uses PsF, that would mean that HDV 24F ought to appear as 47.952 fields/sec or 23.976 frames/sec in the NLE -- unless 2:3 or 2:3:3:2 pull-down was applied, which of course would be very inefficient and I personally think highly unlikely.

Even if 24F is sent by 1394 as PsF at 23.976, it started progressive and is read by F-Mode-aware NLEs as 23.976 so it really doesn't matter and we're all just speculating anyway. If it helps us end-users to know the data protocol beyond what's described in the manual and by the documentation of the NLEs we use, let me know and I'll see if I can get some additional factual details (no promises, as a lot of the detail may still be propriety secrets).

EDIT: Well, Salah posted while I was typing. There it is. Kudos for using the SEARCH feature...shoulda done that myself, since I'm usually first in line to remind people to do so!

[Tom Roper]

Thanks Brian. I credit Barlow Elton for getting us on track though. Your observations are excellent as well.

[Dino Leone]

Originally Posted by Tom Roper

I believe I can close the loop by answering the one final remaining open question. Thank you all for making this thread lively and civil.

Me too, I'd like to thank everybody for the valuable and most interesting and educating contributions! When I posted the original question I never thought I'd raise so much dust / questions and disagreements. It's incredible how much I learned from this. Thanks again!

Dino

[Mikko Lopponen]

Originally Posted by Tom Roper

It's reported by a standalone capture utility, CapDVHS to be 29.97 fps. I posted a screen shot of this earlier. I'm not looking at it now, (writing this from my cell phone) but I think it gives a frame count as well.

There are lots of standalone capture utilities that report fps numbers that are wrong. I've watched a couple of clips that were 23.97 reported as 29.97 in different software.