After reading several threads in the HV20 subforum about 2-3 pulldown removal being necessary in order to get good-looking 24F, I'm confused as to what exactly the 24F mode is in both HV20 and A1.

Am I correct in the assumption that the HV20 will always output 29.97fps (60i), even when 24F is selected (thus the pulldown necessary - in order to be compatible with consumer NTSC format)? I assume the 24fps get 3-2 "pulldowned" internally, resulting in 60i?

On the A1, on the other hand, having 24F selected will indeed output 24F (23.976 in fact) over firewire, correct? (That being the reason why on the Mac, FCP is necessary since FCE is not supporting 24F).

Not sure if I should have posted this in the HV20 forum as it seems more relevant to that cam?
Thanks in advance,
Dino

Yup, you've got it. 24F will go to tape as 23.976 progressive. If you send an F Mode signal out by SDI or analog component, though, it'll be pulled down in-camera to 60i to maintain NTSC compatibility.

Maybe 24F goes to tape as 23.976 but it goes from the tape out the firewire as 60i with pulldown.

now im confused! what if you want to deliver something as 24P?
will the encoding back to 23.976 look the same?

Hmmm... now I'm confused,. too!

24F footage shot on my A1 shows up as "non-interlaced" in my ProCoder Express encoder.

Is the HV20 different... or is ProCoder wrong (or both)??

I was hoping to get an HV20 to use as a deck and for occasional b-roll. But if it does capturing differently on 24F... I might have to pass.

Brian Brown

Non-interlaced yes, but progressive frames carried inside a 60i stream with pulldown added.

Now you've got me confused. I shoot HDV at 24P (F), capture it in FCP's 1080P24 mode and edit in a 24P timeline. It all stays 24 fps progressive all the way. No pulldown.

Now you've got me confused. I shoot HDV at 24P (F), capture it in FCP's 1080P24 mode and edit in a 24P timeline. It all stays 24 fps progressive all the way. No pulldown.

Cool. Now I get everybody confused. :-)
I did some more reading in the meantime - with the following results:
- the A1 is true 24fps (records to tape and outputs 24f)
- HV20 is always(*) 60i. If one captures as 24p, it gets stored to tape (and outputs through firewire and HDMI) as 60i using pulldown (for compatibility with consumer equipment).

(*) one exception: if a 24f-recorded tape from the A1 is played back on the HV20, it actually outputs true 24f on its firewire port! This is according to page 101 in the HV20 user's manual (specifications at the very end - well hidden).

Sorry for the confusion.
Dino

HV20 Records 24p not PsF
Plays Back 24PsF,30PsF ..etc from other Canon cameras
A-1 does 24PsF

HV20 Records 24p not PsF
Plays Back 24PsF,30PsF ..etc from other Canon cameras
A-1 does 24PsF

Wrong and misleading.

24PsF...what's that?

Use something else besides your NLE to capture with and you'll see. The XH-A1 firewire is exporting 24F as 29.97fps, pulldown added. CapDVHS works well for this with PCs.

Working with a native m2t stream, not with an intermediate codec, and because pulldown is added, I put 24F on the same timeline with 60i, not *re-encoding it* . It mixes because pulldown is already added to the 24F. If it wasn't, it wouldn't playback on the Tohsiba HD-DVD player, which doesn't support 24 fps mpeg, just VC1.

I think what's happening is the NLEs some of you are using is making the pulldown transparent when you import into a 24p timeline, so you just edit frame by frame. And I think you print to tape so you don't see just what the output format really is. I could be wrong.

Wrong and misleading.

Which part is wrong Piotr? Can you set this thread straight?

Wrong and misleading.

OK Im wrong fix it in post(IE tell me why?)

Dino's got it right. Unfortunately, the guessing has led to some misinformation in this thread.

F-mode is recorded to tape as progressive, not interlaced. The XL H1 and the XH cameras record and play F-mode.

The HV20 records 24p to tape with 3:2 pulldown as 60i...sometimes called "Progessive Segemented Frame." The HV20 can PLAYBACK F-Mode tapes as F-Mode, just can't record F-Mode. That makes it a good VTR for someone who doesn't want to use an XL or XH for capture and other playback purposes.

Firewire (1394) is simply a protocol for transferring digital data. The bits come out one end just the same as they went in the other end; it doesn't do any kind of conversions. What a NLE does once it receives a video clip is up to the coding of the NLE and the choices the editor makes with the NLE's features. For instance, when the XL H1 came out, presets needed to be released for PPro so it could work with F-Mode. Various NLEs can mix n match different kinds of clips and export in a wide variety of formats, frame rates and resolutions. So with most current NLEs, you can import a progressive clip and export 60i to DVD if you wish, or even vice versa.

Dino's got it right.

On the A1, on the other hand, having 24F selected will indeed output 24F (23.976 in fact) over firewire, correct?

Doesn't have it right. It's 29.97 over firewire. Pulldown is added.

Sorry Tom, that is incorrect. When recording HDV, the XH A1 records and outputs true 24fps progressive. When recording in SD at 24F you have the option to add pulldown, just as the DVX100 does. HDV, however, at 24F, is always true 24p. No pulldown. If you take that into a 60i timeline in FCP (and presumably in other NLEs as well, the pulldown will be added by the NLE (and it will have to, therefore, render). You keep it in a 24p timeline, it stays 24P, no rendering.

...If you take that into a 60i timeline in FCP (and presumably in other NLEs as well, the pulldown will be added by the NLE (and it will have to, therefore, render). You keep it in a 24p timeline, it stays 24P, no rendering.

http://vsdrives.com/graphics/XH-A1/24F.png

Bill, respectfully but I think you have it backwards. My earlier post was directed at you when I said you need to take the NLE out of the equation and just consider the capture stream. Please click on the link above. The capture is a 24F stream from an XH-A1, but the frame rate is 29.97. The capture already has the pulldown in it even before it goes to the NLE.

I just checked the XH User Manual (see page 40 of the English language manual). The way I read it, SD 24F gets sent out to 1394 as either 2:3 or 2:3:3:2 (shooter's choice), otherwise F-Mode over 1394 is progressive. Oddly enough, it looks like SD 30F get recorded to tape as 60i PsF, but then sent out as progressive frames. Both 24F and 30F (and I assume 25F in PAL-land) are progressive in HDV.

Tom, I use PPro so am not familiar with the dialog box you linked. Not sure why the software reports HDV F-Mode as 29.97 unless it was 30F? Canon 24F should be 23.976. I do know that PPro reports HDV F-Mode footage correctly based on progressive frame rates.

Pete, I used Vegas to capture from tape the same 24F clip that I captured earlier with the CapDVHS utility. Vegas reports the clip as HDV 23.976 fps progressive. But the same clip captured by Vegas is reported with clip properties 29.97 fps mpeg2 bottom field first in several other apps.

I'm certain it's 29.97 fps mpeg Psf. I'm trying to understand the semantics of Vegas calling it 23.976 fps progressive. And I think the answer is just semantics. HDV versus Mpeg. In other words, HDV 23.976 fps progressive is another name for mpeg2 29.97 fps Psf with pulldown added.

Below, click on the link for a screen capture from Ulead MF clip properties. This is the clip captured from tape by Vegas and reported by Vegas as "HDV 23.976 fps progressive."

http://vsdrives.com/graphics/XH-A1/24F_3.png

It's important to note that Ulead MF authored that clip to HD-DVD format without re-encoding it, not possible if it were true 24p. Mpeg 24p is not a supported format by HD-DVD, just VC1 24p. But it plays.

Salah Baker noted in this thread that HV20 records 24p (not PsF), and sent a clip (which I am downloading now) to David Newman of Cineform. David said that clip had pulldown added inside of 60i, i.e. it *is* PsF.

Hmm... well I also have screen grabs from 24F footage from the XH-A1. However, on mine both ProCoder Express and MainConcept's encoder show it as progressive, 23.976fps footage.

http://www.brownland.org/video/24F.jpg

I did capture this clip in Premiere Pro... so does this NLE ignore/ remove pulldown when putting streams on my hard drive?

I'll have to try to capture with HDVslpit or something and see what happens.

This is strange indeed!
Brian Brown

A1 24f clip captured using HDVSplit, opened in MPEG Streamclip. Attached is the readout showing indeed it is 23.976 fps. Everything I've read on the forum indicates the A1 records 24 frames to tape with NO pulldown.

24F is a 24 fps stream with no pulldown HOWEVER...I believe there are repeat flags in the stream intended to make it easier for the decoder to add pulldown if necessary. Before I had native HDV 24F support in FCP, I captured raw m2t's with MPEG Streamclip and it indicated that the files were 29.97 and VLC player would play 24F clips with repeat frames that had a noticeably odd cadence if you knew what to look for.

24F is 23.976 fps.

24F is a 24 fps stream with no pulldown HOWEVER...I believe there are repeat flags in the stream intended to make it easier for the decoder to add pulldown if necessary. Before I had native HDV 24F support in FCP, I captured raw m2t's with MPEG Streamclip and it indicated that the files were 29.97 and VLC player would play 24F clips with repeat frames that had a noticeably odd cadence if you knew what to look for.

24F is 23.976 fps.


Some apps are reporting 23.976 and some are reporting 29.97. I think you are right about the repeat flags inside the stream, but a repeated frame is never actually encoded because it wastes compression efficiency. The flags themselves are the repeat mechanism for pulldown, so I think you make a case for it being there.

A1 24f clip captured using HDVSplit, opened in MPEG Streamclip. Attached is the readout showing indeed it is 23.976 fps. Everything I've read on the forum indicates the A1 records 24 frames to tape with NO pulldown.

Your readout also says "upper field first." <---

Your readout also says "upper field first." <---

Why don't you guys just take a player and go through a file frame by frame? It should be blatantly obvious if its 29.97 or 23.97.

Which part is wrong Piotr? Can you set this thread straight?

Sorry for having been so sparse in words, I didn't want to be rude - just was going to bed:)

What was wrong was using the term PsF in the Canon A1 context; it's a term reserved for true progressive, written within interlaced stream in the form of two half-frames (fields or segments) of half-resolution, but no temporal difference, per each frame (Progressive Segmented Frames).

Canon A1 uses "F" which also is progressive, but named differently because it comes from interlaced chips, and is achieved with some very clever signal processing.

Also, I had the impression "PsF' was confused with "FpS", which would of course stand for frame rate.

Your readout also says "upper field first." <---

and is that wrong?

and is that wrong?

Geoff, "upper field first" would indicate interlaced footage. "True" progressive would presumably make no mention of fields.

Boy, this is an interesting thread. It seems like we're all getting different indications from various programs about 24f streams.

Brian Brown

(voice of Homer Simpson) DOH!!!


that is interesting that it shows up 23.976 interlaced as that's not a supported HDV format from what I can tell. Maybe this is a reason Canon can't call it "P" .

that is interesting that it shows up 23.976 interlaced as that's not a supported HDV format from what I can tell.

That's the reason it only works with canons equipment. I believe that the stream is 23.976. If someone would upload a raw clip, I can check it out.

The "F" footage is clearly not interlaced, captured as discrete frames and not fields. I can play back m2t's or mpegs, created by my A1 or others', and in any viewer and see nary a field or interlacing artifact.

I have downloaded a few raw HV20 clips and I can see the interlacing, however.

Since 1080p/24f footage isn't HDV spec, maybe this is why we're seeing various applications report various things. The big question is whether pulldown is being applied to that stream via Firewire on only the HV20 or also on the A1.

I vote for "only on the HV20", BTW. And now I'm concerned on cutting my A1 footage with any HV20 footage.

And this is made all the more maddening because the HV20 is a truly a progressive camera!

Brian

That's the reason it only works with canons equipment. I believe that the stream is 23.976. If someone would upload a raw clip, I can check it out.

It only works with Canon HDV *cameras*. The 24F m2t cuts perfectly for me on a 60i timeline with ANY mpeg video.

And this is why I think Barlow has it right, flags. Yes it's 24 frames per second progressive encoded, but with repeat flags. Pulldown! Pulldown should not ever be actually added as encoded frames because it wastes compression efficiency. The repeat flags are there, and the decoder does with them what it needs to do. If it needs to display at 60 hz refresh, then the decoder reads the flags and adds pulldown. If the refresh is a multiple of 24hz then it doesn't.

One mpeg editor I use has an input window and an output window. On the input window, you can step through 24 discrete frames, but the frame counter skips frame numbers but indexes to 30, like 1,2,4,5,6,7,9...Meanwhile, the frame counter for the output window also counts to 30, but instead of skipping counts, the 4th frame in every sequence is repeated, so 30 whole frames are displayed. This is without re-encoding anything, just looking at the editor windows.

It cuts perfectly with 60i and is not re-encoded. How would it do that without pulldown flags?

As for the footage clearly not interlaced? If both fields originate from the same progressively captured image, then segmenting them into two separate fields that are combined by the decoder should reveal no deinterlace artifacts. Thus it's reported by various software apps to be top field first, or even bottom field first, because field order is irrelevant for 24 discrete progressive frames.

So that's my vote, 24 PsF with pulldown flags. It would seem to explain everything.

Tom, that's a great explanation and makes sense now. Maybe my Premiere Pro removes the flags and pulldown of the original stream from the Firewire. If I capture the stream with something else, it'd be interesting to see if the pulldown (and 29.97fps) remains.

I'm still wondering if the HV20's 24p footage is streamed differently than the A1's 24f.

Thanks,
Brian

Tom, what you're postulating is possible but leaves these nagging thoughts:

- From a bit budget standpoint, it is inefficient to take a progressive frame and pull it down to 2:3 or 2:3:3:2 interlaced; Canon has done really well with minimizing artifacts at HDV bitrates. When they were designing F-Mode why would they choose to strain the bit budget and make their job that much harder?

- Canon explicitly says in their manual that F-Mode, except for SD 24F, is sent out through 1394 as progessive.

- How do you know it isn't the other way around: that the camera sends a data stream that is progressive, but some NLEs pull it down to 60i and report it as such (whereas others don't)?

I guess only Canon and software engineers for the NLE's and third party vendors like Cineform really know, since they have the tech data.

Perhaps the pulldown is added post-tape. So the tape is encoded at 24p or 30p, but pulldown is added before it hits the Firewire port. I know that pulldown must be added to the analog component out, since F modes play fine on any HDTV. Maybe Canon does the same to the streams from the Firewire port for maximum compatibility. And some NLEs are able to discard/ignore the added fields and pulldown.

Is this possible????

Brian Brown

Tom, what you're postulating is possible but leaves these nagging thoughts:

- From a bit budget standpoint, it is inefficient to take a progressive frame and pull it down to 2:3 or 2:3:3:2 interlaced; Canon has done really well with minimizing artifacts at HDV bitrates. When they were designing F-Mode why would they choose to strain the bit budget and make their job that much harder?

If the repeat flag is just a particular hex bit address within a word in side the frame header, then setting or clearing that bit would have no consequence on the overhead since the entire word would define the instruction.

- Canon explicitly says in their manual that F-Mode, except for SD 24F, is sent out through 1394 as progessive.
Agree. Just not sure if PsF'd.

- How do you know it isn't the other way around: that the camera sends a data stream that is progressive, but some NLEs pull it down to 60i and report it as such (whereas others don't)?

It's reported by a standalone capture utility, CapDVHS to be 29.97 fps. I posted a screen shot of this earlier. I'm not looking at it now, (writing this from my cell phone) but I think it gives a frame count as well.

I believe I can close the loop by answering the one final remaining open question. Thank you all for making this thread lively and civil.

The question was, why on page 40 of the Canon XH-A1/G1 English language manual is there wording that states for HDV, 24 progressive frames are encoded but for SD pulldown is added?

The answer is that HDV is mpeg and SD is DV. HDV uses one I-Frame (information frame) for each 15 GOP structure. With DV, there are no GOPs, all frames are I-frames.

DV has no 24 fps spec. All DV is 29.97 frames per second, 60i. In order to wrap 24F inside of DV, you have to add pulldown frames so that all 30 frames in the DV stream are filled. With DV 60i, there are always 30 frames anyway, so there is no compression penalty, no difference whatsoever in file size whichever way you go. You’ve always got 30 frames to fill whether its 60i or 24F.

But with HDV and its mpeg 15 GOP compression scheme, you can have 24 or 30 frames per second. But rather than forcing the codec to write redundant frames for the pulldown, it sets a flag to tell the decoder that the next frame in sequence is to be repeated for the pulldown.

When you place the m2t file onto the 24fps timeline of your NLE, it clears the flag that was set inside each header. If you place the 24F m2t file onto a 30fps timeline of your NLE, it sets the repeat flag, (which was already set anyway by the Canon mpeg encoder). No re-encoding of the frame is required either way.

This is a very efficient scheme, because it allows the decoder of the playback device to decide whether it needs to read pulldown flags to match the refresh rate, or does not need to.

There is no disputing that 24F is progressive encoding. There never has been any question. The question is, is it segmented frames or whole frames. They have to be segmented for DV, it’s completely logical to me that it is segmented for HDV also, particularly in view of the interlaced CCD image sensors used. But either way, there is NO NEGATIVE consequence to splitting a frame into segmented fields that have no temporal shift. Every last single bit is accounted for, and reconstructed into the whole from which it was first sired.

Why then the 10-12% loss in vertical resolution in 24F mode compared to 60i? Most likely it is from the 24F green chroma channel pixel shift.

One particular mpeg editor I use does not require you to choose a 24 or 30fps timeline. You just drop whatever mpeg you have onto the generic timeline. You can even mix them on the same timeline. It does not set or clear the pulldown flags. It leaves each flag in the state it found it. It’s up to the playback device to decide whether to read or ignore the repeat flag, depending on whether it needs to repeat the pulldown frames to match the refresh rate of the display, or not.

Great explanation, Tom. Thanks for all of your time spent posting on this pulldown issue.

Brian Brown

I guess only Canon and software engineers for the NLE's and third party vendors like Cineform really know, since they have the tech data.
http://www.dvinfo.net/conf/showthread.php?t=90148
cool

love CineForm Btw

Tom, I guess I still don't get where this leads us? Even if it turns out that F-Mode uses PsF, that would mean that HDV 24F ought to appear as 47.952 fields/sec or 23.976 frames/sec in the NLE -- unless 2:3 or 2:3:3:2 pull-down was applied, which of course would be very inefficient and I personally think highly unlikely.

Even if 24F is sent by 1394 as PsF at 23.976, it started progressive and is read by F-Mode-aware NLEs as 23.976 so it really doesn't matter and we're all just speculating anyway. If it helps us end-users to know the data protocol beyond what's described in the manual and by the documentation of the NLEs we use, let me know and I'll see if I can get some additional factual details (no promises, as a lot of the detail may still be propriety secrets).

EDIT: Well, Salah posted while I was typing. There it is. Kudos for using the SEARCH feature...shoulda done that myself, since I'm usually first in line to remind people to do so!

Thanks Brian. I credit Barlow Elton for getting us on track though. Your observations are excellent as well.

I believe I can close the loop by answering the one final remaining open question. Thank you all for making this thread lively and civil.


Me too, I'd like to thank everybody for the valuable and most interesting and educating contributions! When I posted the original question I never thought I'd raise so much dust / questions and disagreements. It's incredible how much I learned from this. Thanks again!

Dino

It's reported by a standalone capture utility, CapDVHS to be 29.97 fps. I posted a screen shot of this earlier. I'm not looking at it now, (writing this from my cell phone) but I think it gives a frame count as well.

There are lots of standalone capture utilities that report fps numbers that are wrong. I've watched a couple of clips that were 23.97 reported as 29.97 in different software.

Good explanation, Tom. I just know what it looks like, never knew how it got there.

The last few posts seem confusing to me, so probably will be to those reading this thread in the future looking for facts. To clarify, Canon and Cineform (one of the first third party companies to support Canon F-Mode) are both on record as stating that HDV F-Mode via 1394 is a purely progressive data stream.

Some freeware applets or other software report might possibly report otherwise.

24f footage (xh-a1) will not cut together on the same timeline as 24p footage (HV20).
The 24f is captured (in FCP by the 1080/24p preset) and works great on a 24p timeline without render.
The 24p footage cuts on a 1080/60i timeline fine. But if you want to take it to a 24p time line you must capture it using the Apple Int. Codec and then reverse telecine with Cinema Tools to remove the pulldown. Then it will work on a 24p timeline. BUT not with 24f footage without rendering the 24p footage b/c the 24f is captured with an HDv codec, not AIC. If you try and capture the 24f stuff with the AIC, it captures at 29.97 not 23.98.
Crazy.

Ryan,
There are differences among editors. 24F cuts right with 60i for me on the same timeline, no rendering.

You only have to look at the chroma samples to figure out what the video is.

mpeg2 uses two different methods of 4:2:0 color space depending on if the video is encoded as interlaced or progressive. With progressive the chroma pixels are 2x2 pixels in size for a clean result only not as detailed. Interlaced on the other hand has to alternate the chroma samples every other line because every other line is a different moment in time. If the samples didn't alternate every other line then field A would always look good while field B would always look very bad. By alternating the chroma samples the encoder and decoder treat the video as 1440x540 and then sample at 4:2:0. The video is then interweaved back into 1440x1080. It is a huge pain in the rump but it works great for interlaced video.

The reason why it works well for interlaced video is because only one field is shown at a time anyway. Progressive inside of a interlaced encoded file however can have chroma problems. This is because the chroma is treated as interlaced so it alternates but when viewing as true progressive the lines form the same frame so you do not want the chroma samples to alternate.

All of the F mode footage I have seen has had 2x2 pixel chroma samples without any sign of the chroma ever being in the interlaced form.

Even 24p DVD's can show this. The video is 24p but there are repeat field flags that tell the DVD player to output 60i instead of 24p. The disk itself still has only the 24p video. A normal DVD player will honor those flags and play 60i. To that player it only sees 60i video and deals with it as 60i. A progressive scan DVD player hooked up to a progressive scan display will output the 24p frames.

I'm sure the same thing is happening with the F footage. When played from the camera it works like a DVD where the video is 24p with flags set that output the video as 60i through analog connections. For firewire transfer some decoders only know how to read the stream as 60i (like nornal DVD players) while the special decoders can remove the flags (like a progressive scan DVD player.)