Contacting Canon re: adding 24p pulldown flags

Chris Barcellos · May 3rd, 2007, 10:33 AM

As indicated above, I did the email thing, too. But to those of you who are up on 24p, I have a few Q's:

1. The HV20 capture 24p, laying it into a 60i stream. When I play it back on my HD TV through component (haven't done HDMI yet.) I seem to be getting the effect of 24p. I assume that is because the camera is doing the pulldown (if that is even the proper term.) Can someone explain it better ?

2. IIf I capture to my system, and play the resulting capture on Window Media player, I also get a clean play back that appears to have that same 24 frame look

So what benefit will I get from having 24p pulldown removal. Is it smaller size, playability on other players, easier playing....

Pete Bauer · May 3rd, 2007, 10:55 AM

Yep, that's the pull-down at work. What you're seeing on the TV is a 24p frame split into 3 fields (odd-even-odd for 3/60th of a second), then the next 24p frame split into 2 fields (even-odd for 2/60ths), then the subsequent frame split into 3 fields (even-odd-even), then a frame of 2 fields, etc. So there is a little stutter to the cadence, similar to that you might notice on a Hollywood film played on a TV, especially during rolling credits.

Current versions of most computer players such as WMP and QT will play clips at their native frame rate, so a 24p program ought not to have that slight stutter. And the file size will be smaller since there isn't repeat information -- the two identical odd fields in the "3" part of the 3:2 pulldown -- plus progressive frames are more efficient to compress.

Ian G. Thompson · May 3rd, 2007, 10:57 AM

Chris, I'm no expert on the matter myself but try this. When viewing your material try to pause randomly and examine what the picture looks like. If you see, what a lot of people are calling "ghosting" (where you get double images on top of each other etc) then that is probably an indication that the pulldown was not removed. You are still getting the 24p cadence when watching but those extra frames need to be removed for your NLE (or any other video software) to recognize it as 24p. I have used Cineform HDLink with my footage and compared the before and after and those are the differences that jump out at me. With Cineform (or anything else that can remove the pulldown) whenever I pause the video it is smooth all the way through.

Gavin Ouckama · May 3rd, 2007, 11:31 AM

This explains it really well.

If I understand things correctly ...

As you can see in the example, with the pulldown, one of the frames (in the example it is frame C) gets completely "lost". By adding the pull down flags, an application would then be able to know what frame it is on (A, B, C or D) and put all full progressive frames back together.

As it is now, the HV20 does not add the flags, therefore people have to guess where in the pull down process the selected video is if they want the pure progressive frames. To boot, every time a new video clip is started, the HV20 may start on a different frame, so for each and every clip of video, the guess work has to happen again and again ... what a pain!

Am I making sense? If people want to add corrections, or feel they can do a better job, feel free to jump in ...

Gavin Ouckama · May 3rd, 2007, 11:34 AM

It was also asked what some of the benefits were for removing the pulldown. As per the link above it states:

Benefits of reverse telecine include high-quality non-interlaced display on compatible display devices and the elimination of redundant data for compression purposes.

Ian G. Thompson · May 3rd, 2007, 12:39 PM

I think what Chris is saying is that his video already has that 24P look...what else is he supposed visually see in his video after removing pulldown?

I had the same question at first. When I first used the HV20 I edited in Vegas just like normal and rendered the final product. It had a 24P cadence and looked good. But what I saw in the picture was "ghosting" every now and then whenever there was movement. I thought it had something to do with the issues of HDV. After a while it just irritated me and I eventually learned about "pulldown". I then tried Cineform (which I heard could remove the pulldown...and at that time I didn't know what pulldown was) and what I noticed was the video looked the same but without that horrible "ghosting" effect. I did a side by side comparison and realized that the software actually pulled out those extra frames and got rid of the problem.

I know we are all saying the same thing but I also at first did not know what to visually look for when removing pulldown (besides deinterlacing the video). I can now watch anyones video and determine whether or not their 24P footage was correctly reverse telecined.

Ian G. Thompson · May 3rd, 2007, 12:46 PM

Hopefully this petition to Canon pans out so that they will add the flags to the 24p footage in the cam. That way we could edit within Vegas like normal and properly remove the pulldown during rendering.

I guess this is how the DVX works...similar to the HV20 but it incorporates flags in the footage....which is no problem for any NLE (that supports pulldown removal) to work with.

Jonathan Gentry · May 3rd, 2007, 04:04 PM

I started a new thread on this but I am wondering if the new 7.0e update to Vegas which was updated on May 1 will properly handle the pulldown issue?

-Jonathan

Quote:

Originally Posted by Ian G. Thompson

Hopefully this petition to Canon pans out so that they will add the flags to the 24p footage in the cam. That way we could edit within Vegas like normal and properly remove the pulldown during rendering.

I guess this is how the DVX works...similar to the HV20 but it incorporates flags in the footage....which is no problem for any NLE (that supports pulldown removal) to work with.

Ian G. Thompson · May 3rd, 2007, 05:11 PM

That's a good question. I know it does not handle Panasonic's AVCHD format (only Sony cams). But in regards to the HV20 I think it's more of a Canon problem. If they (Canon) allowed the use of flags in the video the this would be a non-issue within Vegas.

Jonathan Gentry · May 3rd, 2007, 08:10 PM

I thought their new update was to support AVCHD. I just hoped it would also solve the HV20 thing as well.

Quote:

Originally Posted by Ian G. Thompson

That's a good question. I know it does not handle Panasonic's AVCHD format (only Sony cams). But in regards to the HV20 I think it's more of a Canon problem. If they (Canon) allowed the use of flags in the video the this would be a non-issue within Vegas.

Jonathan Gentry · May 3rd, 2007, 08:22 PM

Here are the release notes:

2.0 What’s New
Notable Fixes/Changes in Version 7.0e

Added support for editing files recorded with Sony AVCHD camcorders.
Added snapping to speaker locations for Film style panning.
Added support for multichannel audio in WMA/WMV files. When you add a 5.1 Windows Media file to a stereo project, the stereo downmix stream is loaded by default; when adding to a 5.1 surround project, audio will be added to separate tracks for the center, front, rear, and LFE channels.
Export to PSP® (PlayStation® Portable) now supports writing to the \VIDEO folder and includes a PSP full-screen template (supported in PSP® firmware 3.30 or higher).
Improved encoding quality in the MainConcept AVC/AAC renderer when using a constant bit rate.
Fixed a visual issue with rotated 2D images being scaled down very small.
Fixed a crash that could occur when using Ctrl+NumPad keys with the Surround Panner if no track had focus.
Fixed an issue with normalizing events that had a very small, silent looped portion.
The My Documents folder is now the default capture location for SDI and HDV captures.
Fixed an issue with rendered XDCAM IMX files to ensure they play back from an XDCAM deck.
Fixed an issue that could cause the wrong frame to be displayed on an external monitor when playing back outside of the active loop region.
Fixed a potential buffer overrun with some ASIO drivers.
Fixed an issue opening some Ogg Vorbis audio files.
Fixed a crash that could occur while trying to recapture offline XDCAM media files during project load.
Fixed a crash that could occur while replacing media with media with fewer audio channels.
Fixed a DV video capture issue that could cause captured NTSC clips to be misidentified as PAL.
Fixed a DV video capture issue that could prevent starting timecode from being detected.
Fixed an issue identifying the Plextor-755 drive for CD audio extraction.

Peter J Alessandria · May 4th, 2007, 10:27 AM

Already tried Vegas 7e - doesn't work on 24p issue.

As to the rest of the discussion, I'm no expert but here's my take:

1. The advantage of having flags in the video stream is that software knows exactly which frames are redundant and can remove them. Besides affecting appearance of the rendered video which I'll mention below, this provides a couple of benefits - first, you have the ability to edit based on non-redundant frames meaning "true-er" edit points. Second, it means 20% less data (24 frames v. 30 frames) to render and fit on a DVD or other storage/playback media. So your projects render 20% faster and you can fit 20% more video on a DVD for instance. Personally those are the biggest advantages to me.

2. Again I'm no expert, but I think Cineform is removing pulldown based on something other flags. Maybe it's counting frames/comparing data between frames to decide which are redundant. While I assume the major NLE's can incorporate this method (I'm not sure how time/ processor intensive it is), having flags seems to make the job much easier. I shot exclusively 24p on my DVX100 for 4 years and will shoot exclusively 24p on my HV20 and it is nice to not have to think about the whole pulldown removal issue.

3. All that being said, when the DVX100 came out there was no support for 2:3 pulldown removal as the camera did it. So we edited in 60i and no one was really the wiser. Likewise, I've done a few tests so far with my HV20 shooting 24p, editing as 60i in Vegas 7 and outputting to DVD or WM9 files and it's hard to see the difference between the "true" 24p look of my DVX100 DVD's and the 60i edit/output to DVD from the 24p HV20 files. I did notice some slight ghosting on quick movement on the WM9 files but like I said, the SD DVD's I did look fine and no one other than you or I would notice it.

So... until there are flags or the NLE's find someway to do pulldown without them, I'll edit 24p as 60i files and not lose sleep over it. But I do see a big advantage to having 20% faster renders and 20% more room on my DVD's for longer projects. And for pro-applications, the ability to edit true 24p is probably required.

Chris Barcellos · May 4th, 2007, 01:10 PM

Peter:

Thanks, not having been involved withe 24p before, your post is welcome.

One thing I have in some 24p footage is not so much ghosting, as a pulsing. For instance, I was doing a very slow left to right pan in from a near full frame close up of a grave stone in the forground, to medim ground of several stones. As the at a point during the pan, the grave stone appears to "pulse" like a beating heart.... I'm wondering if that is what caused that. Is that a 24p result. Other thing it could be is OIS, because I don't remember whether I had turned it of, or the famoust rolling shutter issue. I am wondering if anyone else has seen this. I have seen it in other footage with this camera too, only in pans....

Peter J Alessandria · May 4th, 2007, 04:40 PM

Quote:

Originally Posted by Chris Barcellos

One thing I have in some 24p footage is not so much ghosting, as a pulsing. For instance, I was doing a very slow left to right pan in from a near full frame close up of a grave stone in the forground, to medim ground of several stones. As the at a point during the pan, the grave stone appears to "pulse" like a beating heart.... I'm wondering if that is what caused that. Is that a 24p result. Other thing it could be is OIS, because I don't remember whether I had turned it of, or the famoust rolling shutter issue. I am wondering if anyone else has seen this. I have seen it in other footage with this camera too, only in pans....

Chris - I'm sure you'll get several different opinions on this one: here's mine - that is most likely the "dreaded autofocus pulse". (Of course if you're not using autofocus it's the "dreaded-something-else-pulse".) Having not used autofocus in years (my DVX did not have autofocus in 24p mode) I cringed the minute I did a slow zoom on my HV20 and watched to my horror as the image pulsed as the AF hunted to keep up with an otherwise slow zoom ('course it didn't really have to keep up since the zoom was so slow but the AF mech in the camera didn't know this so it constantly checked by microscopically zooming and out - hence the pulsing image.)

Now I know you said "slow pan" not zoom but think it could be equally applicable. Of course some will cry that's "24p! Can't pan faster than [forget what the figure is they always cite]... or else you get strobing". But what they don't tell you is if you go faster or slower than that magic strobing inducing speed you're fine. So I have seen it very few times in shooting 24p for the last four years, though admittedly I don't do a lot of slow pans - I'm a fast pan kind of guy in which case the blur takes care of any strobing issues.

Paul Kepen · May 5th, 2007, 02:29 PM

So DOES Cineform correctly do the 3-2pulldown/ reverse telecine to give you correct 24p from the HV20, even though the HV20 does NOT provide the pulldown flags? If so, then using cineform solves the problem? I sent a response to canon to have the flags added in a firmware, as I think they should provide whatever is normal for 24p in a camera that has 24p. Personally I have cineform, and in my limited time with the HV20, I have been relatively pleased with the 24p results using cineform. Its not perfect, but the resultant DVD's are very clean:)

May 3rd, 2007, 10:33 AM	#31
Chris Barcellos Inner Circle Join Date: Nov 2005 Location: Elk Grove CA Posts: 6,838	As indicated above, I did the email thing, too. But to those of you who are up on 24p, I have a few Q's: 1. The HV20 capture 24p, laying it into a 60i stream. When I play it back on my HD TV through component (haven't done HDMI yet.) I seem to be getting the effect of 24p. I assume that is because the camera is doing the pulldown (if that is even the proper term.) Can someone explain it better ? 2. IIf I capture to my system, and play the resulting capture on Window Media player, I also get a clean play back that appears to have that same 24 frame look So what benefit will I get from having 24p pulldown removal. Is it smaller size, playability on other players, easier playing.... __________________ Chris J. Barcellos

May 3rd, 2007, 10:55 AM	#32
Pete Bauer Wrangler Join Date: May 2003 Location: Eagle River, AK Posts: 4,100	Yep, that's the pull-down at work. What you're seeing on the TV is a 24p frame split into 3 fields (odd-even-odd for 3/60th of a second), then the next 24p frame split into 2 fields (even-odd for 2/60ths), then the subsequent frame split into 3 fields (even-odd-even), then a frame of 2 fields, etc. So there is a little stutter to the cadence, similar to that you might notice on a Hollywood film played on a TV, especially during rolling credits. Current versions of most computer players such as WMP and QT will play clips at their native frame rate, so a 24p program ought not to have that slight stutter. And the file size will be smaller since there isn't repeat information -- the two identical odd fields in the "3" part of the 3:2 pulldown -- plus progressive frames are more efficient to compress. __________________ Pete Bauer The most beautiful thing we can experience is the mysterious. It is the source of all true art and science. Albert Einstein Trying to solve a DV mystery? You may find the answer behind the SEARCH function ... or be able to join a discussion already in progress!

May 4th, 2007, 01:10 PM	#43
Chris Barcellos Inner Circle Join Date: Nov 2005 Location: Elk Grove CA Posts: 6,838	Peter: Thanks, not having been involved withe 24p before, your post is welcome. One thing I have in some 24p footage is not so much ghosting, as a pulsing. For instance, I was doing a very slow left to right pan in from a near full frame close up of a grave stone in the forground, to medim ground of several stones. As the at a point during the pan, the grave stone appears to "pulse" like a beating heart.... I'm wondering if that is what caused that. Is that a 24p result. Other thing it could be is OIS, because I don't remember whether I had turned it of, or the famoust rolling shutter issue. I am wondering if anyone else has seen this. I have seen it in other footage with this camera too, only in pans.... __________________ Chris J. Barcellos

May 5th, 2007, 02:29 PM	#45
Paul Kepen Major Player Join Date: Dec 2004 Location: Minneapolis Minnesota Posts: 347	Cineform So DOES Cineform correctly do the 3-2pulldown/ reverse telecine to give you correct 24p from the HV20, even though the HV20 does NOT provide the pulldown flags? If so, then using cineform solves the problem? I sent a response to canon to have the flags added in a firmware, as I think they should provide whatever is normal for 24p in a camera that has 24p. Personally I have cineform, and in my limited time with the HV20, I have been relatively pleased with the 24p results using cineform. Its not perfect, but the resultant DVD's are very clean:)

May 3rd, 2007, 10:57 AM	#33
Ian G. Thompson Major Player Join Date: Jan 2007 Location: Rockledge, Florida Posts: 351	Chris, I'm no expert on the matter myself but try this. When viewing your material try to pause randomly and examine what the picture looks like. If you see, what a lot of people are calling "ghosting" (where you get double images on top of each other etc) then that is probably an indication that the pulldown was not removed. You are still getting the 24p cadence when watching but those extra frames need to be removed for your NLE (or any other video software) to recognize it as 24p. I have used Cineform HDLink with my footage and compared the before and after and those are the differences that jump out at me. With Cineform (or anything else that can remove the pulldown) whenever I pause the video it is smooth all the way through.

May 3rd, 2007, 11:31 AM	#34
Gavin Ouckama New Boot Join Date: Mar 2007 Location: Nampa, Idaho Posts: 19	This explains it really well. If I understand things correctly ... As you can see in the example, with the pulldown, one of the frames (in the example it is frame C) gets completely "lost". By adding the pull down flags, an application would then be able to know what frame it is on (A, B, C or D) and put all full progressive frames back together. As it is now, the HV20 does not add the flags, therefore people have to guess where in the pull down process the selected video is if they want the pure progressive frames. To boot, every time a new video clip is started, the HV20 may start on a different frame, so for each and every clip of video, the guess work has to happen again and again ... what a pain! Am I making sense? If people want to add corrections, or feel they can do a better job, feel free to jump in ...

May 3rd, 2007, 11:34 AM	#35
Gavin Ouckama New Boot Join Date: Mar 2007 Location: Nampa, Idaho Posts: 19	It was also asked what some of the benefits were for removing the pulldown. As per the link above it states: Benefits of reverse telecine include high-quality non-interlaced display on compatible display devices and the elimination of redundant data for compression purposes.

May 3rd, 2007, 12:39 PM	#36
Ian G. Thompson Major Player Join Date: Jan 2007 Location: Rockledge, Florida Posts: 351	I think what Chris is saying is that his video already has that 24P look...what else is he supposed visually see in his video after removing pulldown? I had the same question at first. When I first used the HV20 I edited in Vegas just like normal and rendered the final product. It had a 24P cadence and looked good. But what I saw in the picture was "ghosting" every now and then whenever there was movement. I thought it had something to do with the issues of HDV. After a while it just irritated me and I eventually learned about "pulldown". I then tried Cineform (which I heard could remove the pulldown...and at that time I didn't know what pulldown was) and what I noticed was the video looked the same but without that horrible "ghosting" effect. I did a side by side comparison and realized that the software actually pulled out those extra frames and got rid of the problem. I know we are all saying the same thing but I also at first did not know what to visually look for when removing pulldown (besides deinterlacing the video). I can now watch anyones video and determine whether or not their 24P footage was correctly reverse telecined.

May 3rd, 2007, 12:46 PM	#37
Ian G. Thompson Major Player Join Date: Jan 2007 Location: Rockledge, Florida Posts: 351	Hopefully this petition to Canon pans out so that they will add the flags to the 24p footage in the cam. That way we could edit within Vegas like normal and properly remove the pulldown during rendering. I guess this is how the DVX works...similar to the HV20 but it incorporates flags in the footage....which is no problem for any NLE (that supports pulldown removal) to work with.

May 3rd, 2007, 05:11 PM	#39
Ian G. Thompson Major Player Join Date: Jan 2007 Location: Rockledge, Florida Posts: 351	That's a good question. I know it does not handle Panasonic's AVCHD format (only Sony cams). But in regards to the HV20 I think it's more of a Canon problem. If they (Canon) allowed the use of flags in the video the this would be a non-issue within Vegas.

May 3rd, 2007, 08:22 PM	#41
Jonathan Gentry Regular Crew Join Date: Jun 2004 Posts: 183	Here are the release notes: 2.0 What’s New Notable Fixes/Changes in Version 7.0e Added support for editing files recorded with Sony AVCHD camcorders. Added snapping to speaker locations for Film style panning. Added support for multichannel audio in WMA/WMV files. When you add a 5.1 Windows Media file to a stereo project, the stereo downmix stream is loaded by default; when adding to a 5.1 surround project, audio will be added to separate tracks for the center, front, rear, and LFE channels. Export to PSP® (PlayStation® Portable) now supports writing to the \VIDEO folder and includes a PSP full-screen template (supported in PSP® firmware 3.30 or higher). Improved encoding quality in the MainConcept AVC/AAC renderer when using a constant bit rate. Fixed a visual issue with rotated 2D images being scaled down very small. Fixed a crash that could occur when using Ctrl+NumPad keys with the Surround Panner if no track had focus. Fixed an issue with normalizing events that had a very small, silent looped portion. The My Documents folder is now the default capture location for SDI and HDV captures. Fixed an issue with rendered XDCAM IMX files to ensure they play back from an XDCAM deck. Fixed an issue that could cause the wrong frame to be displayed on an external monitor when playing back outside of the active loop region. Fixed a potential buffer overrun with some ASIO drivers. Fixed an issue opening some Ogg Vorbis audio files. Fixed a crash that could occur while trying to recapture offline XDCAM media files during project load. Fixed a crash that could occur while replacing media with media with fewer audio channels. Fixed a DV video capture issue that could cause captured NTSC clips to be misidentified as PAL. Fixed a DV video capture issue that could prevent starting timecode from being detected. Fixed an issue identifying the Plextor-755 drive for CD audio extraction.

May 4th, 2007, 10:27 AM	#42
Peter J Alessandria Major Player Join Date: Oct 2003 Location: Los Angeles, CA Posts: 293	Already tried Vegas 7e - doesn't work on 24p issue. As to the rest of the discussion, I'm no expert but here's my take: 1. The advantage of having flags in the video stream is that software knows exactly which frames are redundant and can remove them. Besides affecting appearance of the rendered video which I'll mention below, this provides a couple of benefits - first, you have the ability to edit based on non-redundant frames meaning "true-er" edit points. Second, it means 20% less data (24 frames v. 30 frames) to render and fit on a DVD or other storage/playback media. So your projects render 20% faster and you can fit 20% more video on a DVD for instance. Personally those are the biggest advantages to me. 2. Again I'm no expert, but I think Cineform is removing pulldown based on something other flags. Maybe it's counting frames/comparing data between frames to decide which are redundant. While I assume the major NLE's can incorporate this method (I'm not sure how time/ processor intensive it is), having flags seems to make the job much easier. I shot exclusively 24p on my DVX100 for 4 years and will shoot exclusively 24p on my HV20 and it is nice to not have to think about the whole pulldown removal issue. 3. All that being said, when the DVX100 came out there was no support for 2:3 pulldown removal as the camera did it. So we edited in 60i and no one was really the wiser. Likewise, I've done a few tests so far with my HV20 shooting 24p, editing as 60i in Vegas 7 and outputting to DVD or WM9 files and it's hard to see the difference between the "true" 24p look of my DVX100 DVD's and the 60i edit/output to DVD from the 24p HV20 files. I did notice some slight ghosting on quick movement on the WM9 files but like I said, the SD DVD's I did look fine and no one other than you or I would notice it. So... until there are flags or the NLE's find someway to do pulldown without them, I'll edit 24p as 60i files and not lose sleep over it. But I do see a big advantage to having 20% faster renders and 20% more room on my DVD's for longer projects. And for pro-applications, the ability to edit true 24p is probably required.