Nasty Jagged Edges at DVinfo.net

Daniel Baker · June 21st, 2006, 02:54 AM

Not sure why I'm getting jagged edges on curved objects...did a search on this form and didn't seem to find anything exactly like this:

http://myspace-377.vo.llnwd.net/0085...51998377_l.jpg

Which setting have I set wrong this time?

Wes Coughlin · June 21st, 2006, 07:39 AM

If you want to get ride of the "jagged" edges or interlace look, i would shoot a progressive mode. 30p or 24p. Just make sure you editor projects are set for progressive and not interlace, as it requires the computer to interpet the footage properly.

Jarrod Whaley · June 21st, 2006, 09:17 AM

This kind of aliasing doesn't look like an interlacing kind of issue to me... It seems to be more of a resolution/compression/color space kind of thing.

Daniel--try dialing down the vertical detail a few notches.

Daniel Baker · June 21st, 2006, 10:06 AM

I'm already in 24p mode, sorry if I didn't mention that.

I'll knock down my verticle detail and see what happens.

Thanks!

Eric Brown · June 21st, 2006, 11:53 AM

enter "xl2 aliasing" in the search box and have at it. Plenty of useful info to be read.

Daniel Baker · June 21st, 2006, 08:35 PM

Thanks,

...but I had already done a search and didn't get anywhere...I tried the suggestions and still have jagged edges. I just went through the search results a second time and still nothing.

Jarrod Whaley · June 21st, 2006, 08:49 PM

Are you only seeing this on your computer monitor? What about on a TV?

Ash Greyson · June 21st, 2006, 09:28 PM

Settings? That does not look normal...

ash =o)

Daniel Baker · June 21st, 2006, 09:37 PM

Well...just did some testing on the TV, and you really can't see it as much...but then again you can't freeze frame...it's not a perfectly clean edge but at least it doesn't look like frogger.

I tried with and without verticle detail, and with softening the image...i'm also using a .5 black diffusion. I'm thinking this may just be the look on a comp. monitor...which is a bummer cause I'd like to display the video on the internet.

Gah...

I like my guitar...it doesn't have jagged edges...just pointy edges ;)

Jarrod Whaley · June 21st, 2006, 09:46 PM

Is your monitor an LCD? I hear CRT's have less problems with aliasing, as a general rule.

Keep playing around. There are ways to minimize stuff like this, but sometimes it requires a lot of trial and error.

For what it's worth, your jaggies might go away when you re-size for the web. Or they may get worse... Depends on the final output resolution. Rescaling can either cause or alleviate aliasing, depending on the amount of scaling. In fact, if you were looking at your footage on your monitor at a scale beyond the NTSC scale, that might have been a big part of the problem.

Anyway, tell Ash what your settings were. He really knows this camera extremely well.

Daniel Baker · June 21st, 2006, 10:21 PM

Ah, sorry, I missed the settings question...:

I'm on a CRT monitor.

I call this preset The Vice! ;P

24p @ both 3:4 and 16:9 (jaggies on both settings)
Gain -3

Gamme: Cine
Knee: Low
Black: Press

cMatrix: Cine
cGain: 0
cPhase: 0

rGain: 0
gGain: 0
bGain: 0

vDetail (more problems on Normal, but still a jagged on Low)
Sharp: -4 (on the recent shots for sure...old, not so sure, maybe higher)
Coring: 6

SetupL: -6
MasterPed: -4
NR: Low

I was intentionally crunching the blacks really hard for some low light shots where I wanted a little more contrast. I will probably bring this back up before the shoot so I dont kill my shadow detail...

Ash Greyson · June 22nd, 2006, 01:00 AM

Is that +6 or -6 on the coring? With the gain at 0 or -3db you do not need to turn up the coring and should probably leave it at zero. The more you raise it the more it removes noise from micro-details, the more you lower it, the more the microdetails get noisy. This is almost a sharpness setting for fine detail only, think something like mesh, lace, etc.

Also, your noise reduction should be OFF, as there should be no noise at all with a lowered sharpness and default coring setting at 0 gain.

ash =o)

Daniel Baker · June 22nd, 2006, 02:10 AM

Sorry, coring was at +6

Yiesh...it's like my guitar amp...I had to go buy one of those fancy modeling amps that simulates everything under the sun...and now I have too many buttons to worry about...what I wouldn't give for a solid 1x12 tube amp with three knobs that just sounded good by itself!!!

J/K

I'm actually enjoying this little discovery process.

Let me ask you this then now:

1. If I have a .5 black diff. filter, what does that mean for the sharpness and coring? Would I just start loosing definition if I turn down my sharpness? I'd like to get a really gritty detailed image for this video, which I guess is why I had the coring cranked up. Think good shots of old worn and warped lumber.

2. If I wanted to get the light to fall off rather quickly into darkness, yet retain some information in the shadows, do i keep my black on press and raise the MasterPed or vice verse or change the setupLevel...so on so forth...?

I actually have some look/feel/vibe shots from other videos and such that I'm using as a reference if you reeeeeeeeeally want to see them.

As alwayse, thanks so much!

Ash Greyson · June 22nd, 2006, 12:58 PM

Looks like your settings are really for high gain, low light situations... the coring at +6 is normally something you do to take the edge off of grain caused by the gain being up. The black diffuser will have a similar effect, making textured surfaces look smooth. So I think it is not proper for your application since you want detail and grit. Most of what you are adjusting is going against the "gritty detailed" image you are seeking.

Sure post a grab and I will help you out. There are SOOO many ways to handle blacks... You can stretch them, then turn the setup level all the way down. You can leave them in the middle, etc. etc. I only recommend bumping up the MP when you are in a high contrast situation and have to lighten up part of the image that becomes too dark when you get good exposure on the overall frame...

ash =o)

Daniel Baker · June 22nd, 2006, 03:13 PM

Well...maybe detailed and gritty aren't the best words to use...take a look:

Here are some screen grabs from other videos (lighting_ideas.jpg). I'm going more for the look of b1-3/c3...with smatterings of the others.

SSPX0083.JPG is a shot of the foyer I'll be using.

Here are some test shots I've done...They have been edited heavily and I'd like to get as close to this with the camera and lights as possible:

http://www.greenbeetlestudio.com/tes...st_guitar1.jpg
http://www.greenbeetlestudio.com/tes...st_guitar2.jpg
http://www.greenbeetlestudio.com/tes...st_guitar3.jpg
http://www.greenbeetlestudio.com/tes...st_guitar5.jpg

So far I really like the .5 Black Diff...it gives this very subtle organic glow to things...I'm just worried that if I turn down my sharpness on the camera I'd loose clarity.

June 21st, 2006, 02:54 AM	#1
Daniel Baker Regular Crew Join Date: Apr 2006 Location: Memphis, TN Posts: 28	Nasty Jagged Edges Not sure why I'm getting jagged edges on curved objects...did a search on this form and didn't seem to find anything exactly like this: http://myspace-377.vo.llnwd.net/0085...51998377_l.jpg Which setting have I set wrong this time?

June 21st, 2006, 07:39 AM	#2
Wes Coughlin Major Player Join Date: Jun 2005 Location: Bend, Oregon Posts: 206	If you want to get ride of the "jagged" edges or interlace look, i would shoot a progressive mode. 30p or 24p. Just make sure you editor projects are set for progressive and not interlace, as it requires the computer to interpet the footage properly. __________________ http://wildlookout.com

June 21st, 2006, 09:17 AM	#3
Jarrod Whaley Major Player Join Date: Jun 2006 Location: chattanooga, tn Posts: 721	This kind of aliasing doesn't look like an interlacing kind of issue to me... It seems to be more of a resolution/compression/color space kind of thing. Daniel--try dialing down the vertical detail a few notches. __________________ -->jarrod whaley. www.oakstreetfilms.com

June 21st, 2006, 08:49 PM	#7
Jarrod Whaley Major Player Join Date: Jun 2006 Location: chattanooga, tn Posts: 721	Are you only seeing this on your computer monitor? What about on a TV? __________________ -->jarrod whaley. www.oakstreetfilms.com

June 21st, 2006, 09:46 PM	#10
Jarrod Whaley Major Player Join Date: Jun 2006 Location: chattanooga, tn Posts: 721	Is your monitor an LCD? I hear CRT's have less problems with aliasing, as a general rule. Keep playing around. There are ways to minimize stuff like this, but sometimes it requires a lot of trial and error. For what it's worth, your jaggies might go away when you re-size for the web. Or they may get worse... Depends on the final output resolution. Rescaling can either cause or alleviate aliasing, depending on the amount of scaling. In fact, if you were looking at your footage on your monitor at a scale beyond the NTSC scale, that might have been a big part of the problem. Anyway, tell Ash what your settings were. He really knows this camera extremely well. __________________ -->jarrod whaley. www.oakstreetfilms.com

June 21st, 2006, 10:06 AM	#4
Daniel Baker Regular Crew Join Date: Apr 2006 Location: Memphis, TN Posts: 28	I'm already in 24p mode, sorry if I didn't mention that. I'll knock down my verticle detail and see what happens. Thanks!

June 21st, 2006, 11:53 AM	#5
Eric Brown Major Player Join Date: Mar 2005 Location: Seattle Posts: 427	enter "xl2 aliasing" in the search box and have at it. Plenty of useful info to be read.

June 21st, 2006, 08:35 PM	#6
Daniel Baker Regular Crew Join Date: Apr 2006 Location: Memphis, TN Posts: 28	Thanks, ...but I had already done a search and didn't get anywhere...I tried the suggestions and still have jagged edges. I just went through the search results a second time and still nothing.

June 21st, 2006, 09:28 PM	#8
Ash Greyson Trustee Join Date: May 2003 Location: Tulsa, OK Posts: 1,689	Settings? That does not look normal... ash =o)

June 21st, 2006, 09:37 PM	#9
Daniel Baker Regular Crew Join Date: Apr 2006 Location: Memphis, TN Posts: 28	Well...just did some testing on the TV, and you really can't see it as much...but then again you can't freeze frame...it's not a perfectly clean edge but at least it doesn't look like frogger. I tried with and without verticle detail, and with softening the image...i'm also using a .5 black diffusion. I'm thinking this may just be the look on a comp. monitor...which is a bummer cause I'd like to display the video on the internet. Gah... I like my guitar...it doesn't have jagged edges...just pointy edges ;)

June 21st, 2006, 10:21 PM	#11
Daniel Baker Regular Crew Join Date: Apr 2006 Location: Memphis, TN Posts: 28	Ah, sorry, I missed the settings question...: I'm on a CRT monitor. I call this preset The Vice! ;P 24p @ both 3:4 and 16:9 (jaggies on both settings) Gain -3 Gamme: Cine Knee: Low Black: Press cMatrix: Cine cGain: 0 cPhase: 0 rGain: 0 gGain: 0 bGain: 0 vDetail (more problems on Normal, but still a jagged on Low) Sharp: -4 (on the recent shots for sure...old, not so sure, maybe higher) Coring: 6 SetupL: -6 MasterPed: -4 NR: Low I was intentionally crunching the blacks really hard for some low light shots where I wanted a little more contrast. I will probably bring this back up before the shoot so I dont kill my shadow detail...

June 22nd, 2006, 01:00 AM	#12
Ash Greyson Trustee Join Date: May 2003 Location: Tulsa, OK Posts: 1,689	Is that +6 or -6 on the coring? With the gain at 0 or -3db you do not need to turn up the coring and should probably leave it at zero. The more you raise it the more it removes noise from micro-details, the more you lower it, the more the microdetails get noisy. This is almost a sharpness setting for fine detail only, think something like mesh, lace, etc. Also, your noise reduction should be OFF, as there should be no noise at all with a lowered sharpness and default coring setting at 0 gain. ash =o)

June 22nd, 2006, 02:10 AM	#13
Daniel Baker Regular Crew Join Date: Apr 2006 Location: Memphis, TN Posts: 28	Sorry, coring was at +6 Yiesh...it's like my guitar amp...I had to go buy one of those fancy modeling amps that simulates everything under the sun...and now I have too many buttons to worry about...what I wouldn't give for a solid 1x12 tube amp with three knobs that just sounded good by itself!!! J/K I'm actually enjoying this little discovery process. Let me ask you this then now: 1. If I have a .5 black diff. filter, what does that mean for the sharpness and coring? Would I just start loosing definition if I turn down my sharpness? I'd like to get a really gritty detailed image for this video, which I guess is why I had the coring cranked up. Think good shots of old worn and warped lumber. 2. If I wanted to get the light to fall off rather quickly into darkness, yet retain some information in the shadows, do i keep my black on press and raise the MasterPed or vice verse or change the setupLevel...so on so forth...? I actually have some look/feel/vibe shots from other videos and such that I'm using as a reference if you reeeeeeeeeally want to see them. As alwayse, thanks so much!

June 22nd, 2006, 12:58 PM	#14
Ash Greyson Trustee Join Date: May 2003 Location: Tulsa, OK Posts: 1,689	Looks like your settings are really for high gain, low light situations... the coring at +6 is normally something you do to take the edge off of grain caused by the gain being up. The black diffuser will have a similar effect, making textured surfaces look smooth. So I think it is not proper for your application since you want detail and grit. Most of what you are adjusting is going against the "gritty detailed" image you are seeking. Sure post a grab and I will help you out. There are SOOO many ways to handle blacks... You can stretch them, then turn the setup level all the way down. You can leave them in the middle, etc. etc. I only recommend bumping up the MP when you are in a high contrast situation and have to lighten up part of the image that becomes too dark when you get good exposure on the overall frame... ash =o)

June 22nd, 2006, 03:13 PM	#15
Daniel Baker Regular Crew Join Date: Apr 2006 Location: Memphis, TN Posts: 28	Well...maybe detailed and gritty aren't the best words to use...take a look: Here are some screen grabs from other videos (lighting_ideas.jpg). I'm going more for the look of b1-3/c3...with smatterings of the others. SSPX0083.JPG is a shot of the foyer I'll be using. Here are some test shots I've done...They have been edited heavily and I'd like to get as close to this with the camera and lights as possible: http://www.greenbeetlestudio.com/tes...st_guitar1.jpg http://www.greenbeetlestudio.com/tes...st_guitar2.jpg http://www.greenbeetlestudio.com/tes...st_guitar3.jpg http://www.greenbeetlestudio.com/tes...st_guitar5.jpg So far I really like the .5 Black Diff...it gives this very subtle organic glow to things...I'm just worried that if I turn down my sharpness on the camera I'd loose clarity. Attached Thumbnails