View Full Version : jagged edges within certain frames question?


William Graydon
January 24th, 2010, 06:55 PM
I just finished filming and editing a commercial for a local radio station competition and while reviewing the final product I noticed that in some of the frames it looks as if the zebra button is on the actual image (that is the only way I can explain it)

It seems as if certain objects have a fuzz to them and other sharp objects (a printer) has squiggly lines or something of the sorts.

I feel these are due to the lighting scenarios I was shooting in but am not sure so I was hoping I could get some feedback.

FYI I am writing on the XDCAM EX forum because I am using the EX1R so maybe some users have been in similar situations and could provide me a solution or alternative.

Thanks in advance.

Perrone Ford
January 24th, 2010, 07:18 PM
Can you post a sample or screenshot?

William Graydon
January 24th, 2010, 07:29 PM
That would prob. be the best thing to do let me see if I can do that.

William Graydon
January 24th, 2010, 10:44 PM
Perrone Ford, So I went back and looked at all the clips through the camera and an HDMI cable plugged into my TV and saw none of the things I spoke of in the final video. I am using FCP 6 and am wondering now what the problem may be. I use log and transfer to import.

Any suggestions?

Perrone Ford
January 24th, 2010, 10:55 PM
Well, without knowing at what steps the problem was introduced, it's hard to say. Maybe you log and transfer settings were bad. Maybe your final export settings were wrong. Maybe it was a one-time glitch. Who can say...

William Graydon
January 24th, 2010, 11:05 PM
Well from the footage played back by the camera onto the TV I can tell it wasnt that. What settings should I have in FCP? for log and transfer I have it configured to XDCAM EX 1080p 24 frames per second.

In order to have the 1080p would I need an external blue-ray burner?

Im confused so I apologize for the all the questions.

Perrone Ford
January 24th, 2010, 11:09 PM
Well from the footage played back by the camera onto the TV I can tell it wasnt that. What settings should I have in FCP? for log and transfer I have it configured to XDCAM EX 1080p 24 frames per second.

In order to have the 1080p would I need an external blue-ray burner?

Im confused so I apologize for the all the questions.

Well, I am not a Mac user so it's hard for me to give specifics. But if doing log and transfer I'd certainly be using ProRes and not XDCam anything.

Leonard Levy
January 25th, 2010, 12:06 AM
XDCAM EX is fine for your codec in FCP but you need to set it to render in Pro Res.

You have (like many people on the forum) provided far too little information for anyone to tell you what happened. If you are serious about getting help here you need to list every step of your process in great detail.

Alister Chapman
January 25th, 2010, 01:43 AM
How are you viewing the finished video?

William Graydon
January 25th, 2010, 09:39 AM
I shot everything in 1080p24 then went on to log and transfer it into fcp6 straight from the express card slot on the mac after editing it was saved to .mov and because it was quicker and we just wanted to see the finished product on tv we used Idvd to burn it onto a dvd.

In all do honesty Im not quite sure what other information to provide and yes I am serious about getting help.

Alister Chapman
January 25th, 2010, 09:42 AM
It is probably the conversion from HD to SD that's causing the problem. If you start off with too much resolution and downconvert to SD you will get aliasing. You need to soften the HD before downconverting to SD. Take a look at this article: XDCAM-USER.com Getting SD from Hd and the problems of oversampling (http://www.xdcam-user.com/?p=443)

William Graydon
January 25th, 2010, 09:47 AM
Ok great. But if I wanted to keep it 1080p24 how would I export it or what should I do in FCP in order to keep its original resolution?

Keep in mind I have only recently started working with FCP and XDCAM EX so its a grand learning process for me. I do appreciate all the input and help I have received from DVINFO users thus far.

Perrone Ford
January 25th, 2010, 09:49 AM
It is probably the conversion from HD to SD that's causing the problem. If you start off with too much resolution and downconvert to SD you will get aliasing. You need to soften the HD before downconverting to SD. Take a look at this article: XDCAM-USER.com Getting SD from Hd and the problems of oversampling (http://www.xdcam-user.com/?p=443)

Alister, I respect you and your work more than you know. But I simply can't buy this. I've seen films scanned at 8k end up on DVD and look just fine. I've put RED 4K footage onto DVD and it's been just fine. I don't think it's just down to resolution.

Alister Chapman
January 25th, 2010, 10:56 AM
Sure it isn't all about resolution, but it's the bulk of the problem. The frame grabs in my article illustrate it very well. It's all down to Nyquist Theory. Take a transition from light to dark that in HD uses four pixels and drop that down to just 2 pixels in SD and instead of a soft edge you have an instant hard light to dark switch, which doesn't look good or natural. Simply softening the HD image with a 2 two to 3 pixel blur can really improve the quality of the SD downconvert.

Your end result when down converting from HD to SD will depend on whether the down conversion algorithm applies any interpolation or softening. A high quality converter should do a good job, but a basic frame size conversion may just dump pixels resulting in a nasty image. It's one of the reasons DSLR's suffer from so much aliasing when used for video: Too much resolution. It's why quality video cameras have optical low pass filters that reduce the image resolution to around half that of the sensor resolution. It's counter intuitive I know but you really can have too much resolution.

Perrone Ford
January 25th, 2010, 11:12 AM
Ok, I see what you're getting at. It's not that you necessarily have too much resolution, it's that poor handling of it as you move to lower resolutiuon leads to issue. Yes, we see this all the time. And it's why I do my downconversions with Lanczos or bicubic spline algorithms. They seem to produce excellent quality.

It's unfortunate these aren't available in modern NLEs, as it would solve a lot of issues.

-P

Alister Chapman
January 25th, 2010, 11:33 AM
William: To keep it at 1080P24 from FCP you can export the clip back to the camera using the "Export XDCAM" function. There is a limit of 4Gb on the SxS cards so you have to break any edits down into segments less than 12mins long (I think it's 12 mins).

Another option is to convert it to a quicktime H264 at 1920x1080 using a bitrate of 10Mb/s or higher. This will look pretty good played back on a PC or Laptop. There are many, many export options. You can also burn basic Blu-Ray discs by exporting via compressor.

Leonard Levy
January 25th, 2010, 12:56 PM
William,
Lots and lots of information is missing in what you've provided:

What codec and sequence settings did you use?
What were the render settings in that sequence?

Did you need to render the problem scenes (due to color correction or anything else.)

Mostly what are you looking at these images on when you saw the problem? Is it on your computer monitor while editing ( then what kind of monitor is it), or are you looking at an SD DVD made from iDVD?

If its an SD DVD made from IDVD then that's likely the culprit. In that case you should tell us what settings you used for making the iDVD and what settings were used to export the FCP file to iDVD.

I don't know anything about iDVD but this forum is filled with threads from people who had trouble making SD DVD's from EX footage on DVD Studio Pro. It can be very tricky to get it right and can look awful if its done wrong.

( Please don't take it personally but I don't understand how someone doing this kind of work wouldn't realize how important the details of these settings are to your final product. It often happens that someone posts a terrible problem like yours then never bothers to add the relevant info.)

Gints Klimanis
January 25th, 2010, 01:05 PM
t's one of the reasons DSLR's suffer from so much aliasing when used for video: Too much resolution.

Agreed, and their downsampling filtering is inadequate for the job. Some of these manufacturers could solve this by offering a video crop mode in which only the actual 1280x720 or 1920x1080 pixels are used, after demosaicing of course. That "digital zoom" through a crop would be a gift to those using video in longer lens applications. In other situations, shorter focal lengths would be required, and in some cases, that would help the situations with inadequate DOF at shorter focusing distances.

Alister Chapman
January 25th, 2010, 02:07 PM
But it would make the lenses so telephoto that for most people they would be useless, your looking at a 4 to 5x magnification factor.

Leonard Levy
January 26th, 2010, 03:55 PM
Am I overreacting?

I am tired of people posting some terrible problem they have with footage from their camera and desperately seeking advice. Lots of us take the time to chime in with suggestions but the original poster never bothers to list the details of what he did, so all it does is waste people's time. I had a hunch we would never hear more from Mr. Graydon. I hope that is proved wrong, but it makes me just not want to bother at all in the future.

Perrone Ford
January 26th, 2010, 04:12 PM
Leonard.. it's par for the course.

Don't invest too heavily in the answer, until you you have an active conversation going. Otherwise it just leads to frustration. On a more positive note, your excellent responses may really help someone down the road who runs into the same problems.

-P

William Graydon
January 27th, 2010, 07:30 AM
Assuming is never a good thing but Instead of posting aimlessly I tried to go back and investigate some of the things you mentioned to me and also attempted to do some of the research myself here is what I found so far.

IDVD may have caused the problem so I need to export it again through dvd studio pro to see if this solves the problem.

From what Alister Chapman said it looks like it could also be the down conversion of the video which caused it too look so crappy.

My thing is I have only begun to really use FCP maybe for 2 months now so experience and or knowledge is subpar but I can say that the problem resides within FCP because the footage directly from the camera on to my 42" looks perfect.

some of the settings I used were:
- shutter off
- iris between 1.9 and 8 (due to the brightness of day)
- nd2 filter outside and clear inside
- gain 0db
- wb was around 2800k indoors and somewhere around 5000k outdoors
- 1080p24 35mbs
- used some of alister chapman's pp setting in order to get a richer color.
- the sequence was xdcam ex 1080p24 and so was the render setting. (I read a few times that I should switch the render setting to apple prores 422 is that so?)
- used log and transfer in fcp
- exported to quicktime

Perrone Ford
January 27th, 2010, 07:34 AM
- used log and transfer in fcp
- exported to quicktime

What settings did you use for Log and Transfer, and what settings exactly did you use in "export to quicktime". Believe it or not, there's several dozen things at both of those steps that could go wrong.

William Graydon
January 27th, 2010, 07:42 AM
wow I feel like such an amateur here but I honestly don't know. Most likely which ever one was standard is the one I used. Let me see if I can find out and Ill get back to you.

William Graydon
January 27th, 2010, 07:45 AM
I know you don't use fcp but maybe you know what the settings should be? I literally from the beg. just clicked on log and transfer and when my files appeared selected them all and put import to the new project.

Perrone Ford
January 27th, 2010, 07:52 AM
I know you don't use fcp but maybe you know what the settings should be? I literally from the beg. just clicked on log and transfer and when my files appeared selected them all and put import to the new project.

Taking the defaults on an Apple is nearly a surefire way to get substandard results. And without a Mac in front of me, I wouldn't have a clue to tell you how to proceed. Maybe if you post a screenshot...

Hopefully, an FCP guru stops through shortly...

Leonard Levy
January 27th, 2010, 12:55 PM
William,
Sorry I was negative. You have proved me wrong. There are many possible sources for your problem. The camera is extremely unlikely and your original footage is probably fine.

For log and transfer your settings look right to me but not for render.

XDCAM is a compressed format ( like many others) so if you let renders in the timeline also render to the same codec then you are re-compressing every time you render - BAD - big NoNo. render should be set for ProRes and ordinary PreRes422 should be fine.

Eventually you should click always render in high precision YUV though you can do your general work without that checked then clean out all renders and re-render if you like because it is slower. But that's the highest quality.

Exporting from FCP has lots of choices and is a bit of a mine field as are the settings for good DVD's in DVD Studio Pro. When you open up export it wioll offer "current settings" but that is generally wrong. You need to choose how you want it to output from a long list of codecs. Also there is Quicktime conversion, Compressor and Quicktime - all different choices with radically different results. Its a learning process for exporting.

I don't know anything about iDVD but for studio Pro if you Search through these pages will you will find lots of discussion and good advice. I recently (September I think) went through a lot of tests and finally got very good DVD's and it was ultimately simple. If I have time to find it I will post here. It should be a sticky.

Leonard Levy
January 27th, 2010, 01:56 PM
William, I went looking for my final workflow and realized I had never written it down carefully so thanks to you I have reconstructed I think what I did . I'm glad you pushed my button to look because at least I had the info around to reconstruct.
Here goes

EX-1 HD> SD DVD’s final results ( I think these were my good settings)

1. Edit in EX1 Native Sequence, PAL or NTSC
2. Set Renders > Go to "Sequence Settings >Render Control>Codec>Apple ProRes 422 (HDV XDCAM HD/EX only)."
Also click render in high precision YUV
( Some people may add a very slightly blur their EX-1 file or add a flicker filter which does something similar if they feel the DVD comes out too sharp and has aliasing as a result. I didn't find the need to do that)
----------------------------------------

3. Export to “Quicktime for self-contained movie” - Pro Res 422 > This makes a final Pro Res version > Then bring this file to Compressor
OR Export directly from FCP timeline to Compressor ( I do this)

4. In Compressor Select “ DVD Best Quality (90 unless it is longer)
- Copy the "MPEG-2" 6.2 Mbps 2 pass file and change these settings by double clicking and opening the Inspector (When finished name and save this compressor setting)

6. click "Encoder" icon and go through the icons
Under "Quality" tab
-choose bit rates, ( the defaults are prob OK)
-"Two pass VBR Best"
-*IMPORTANT select "Motion Estimation >Better."
(There used to be a bug if you picked Best that ruined the encoding I don’t know if its still there but it works fine at Better.)

Under "Frame Controls"
-Frame Controls "On."
- "Resize Filter "Better (Linear filter).
- CHANGE "Output Fields to "Progressive" (if its a progressive timeline)
-Leave deinterlace to Fast (Line averaging),
- "Adaptive Details" box checked. *Leave other settings *as is.

Be prepared for long renders.

7. Proceed to make DVD in DVDSP (you will also need to make the audio file in compressor) for DVDSP

William Graydon
January 27th, 2010, 03:38 PM
Wow... I cannot thank you enough and I appreciate all the info you and perrone have provided me.

When I get home I will be follow those steps and see if I can achieve the same results.

Steve Shovlar
January 27th, 2010, 04:13 PM
William, I went looking for my final workflow and realized I had never written it down carefully so thanks to you I have reconstructed I think what I did . I'm glad you pushed my button to look because at least I had the info around to reconstruct.
Here goes

EX-1 HD> SD DVD’s final results ( I think these were my good settings)

1. Edit in EX1 Native Sequence, PAL or NTSC
2. Set Renders > Go to "Sequence Settings >Render Control>Codec>Apple ProRes 422 (HDV XDCAM HD/EX only)."
Also click render in high precision YUV
( Some people may add a very slightly blur their EX-1 file or add a flicker filter which does something similar if they feel the DVD comes out too sharp and has aliasing as a result. I didn't find the need to do that)
----------------------------------------

3. Export to “Quicktime for self-contained movie” - Pro Res 422 > This makes a final Pro Res version > Then bring this file to Compressor
OR Export directly from FCP timeline to Compressor ( I do this)

4. In Compressor Select “ DVD Best Quality (90 unless it is longer)
- Copy the "MPEG-2" 6.2 Mbps 2 pass file and change these settings by double clicking and opening the Inspector (When finished name and save this compressor setting)

6. click "Encoder" icon and go through the icons
Under "Quality" tab
-choose bit rates, ( the defaults are prob OK)
-"Two pass VBR Best"
-*IMPORTANT select "Motion Estimation >Better."
(There used to be a bug if you picked Best that ruined the encoding I don’t know if its still there but it works fine at Better.)

Under "Frame Controls"
-Frame Controls "On."
- "Resize Filter "Better (Linear filter).
- CHANGE "Output Fields to "Progressive" (if its a progressive timeline)
-Leave deinterlace to Fast (Line averaging),
- "Adaptive Details" box checked. *Leave other settings *as is.

Be prepared for long renders.

7. Proceed to make DVD in DVDSP (you will also need to make the audio file in compressor) for DVDSP

Interesting post leonard. I do this slightly different. perhaps yours is the better method and I will certainly try it out on my next project.

I edit in my EX3 footage in FCP. I then make a new sequence. Right click on it and change it to a SD sequence, DV Pal ( being in the UK)

I then drop the sequence 1 onto the sequence 2 timeline. A box pops up asking me if I want to change the settings. I click no. The timeline goes red and I render it out. (takes some time)

I then send this timeline to Compressor and do the same as you do, except I chose "best" rather than "better".

If I am making a blu-ray version, I render the HD timeline out as a stand alone QT movie and bring that into Toast, though you could do virtually the same in FCP7.

Perhaps your way is better?

Leonard Levy
January 27th, 2010, 06:16 PM
Steve,

I'm not sure what the difference would be. When I figured this out I ended up writing down every method I found on the web, organized all the differences and then tried out a few.

A friend also was having a awful problems making DVD's at the same time (from his EX) so we compared notes. I'm sure I got this method from this forum and it worked perfectly. Very good DVD.

I was never sure about the Motion Estimation "Better " or "Best" as I was changing alot of things at once. I read a number of posts in various places that " Best" had a bug that produced terrible DVD's. I was never sure if my success came from going to "Better" or some other change.

The fact that you use "Best" may not be relevant though because it may have been a problem with HD original files and you have already downconverted. On the other hand the Best/Better thing could be long fixed by Apple.

At any rate try it out - this method worked very well for me and my friend who was tearing his hair out.