Mixing HD (709) and SD (601) clips at DVinfo.net

Jon Fairhurst · February 19th, 2008, 04:50 PM

What plug in should I use to convert a Rec. 709 clip to Rec. 601 and vice versa?

In my current project there was a cuts-only DVD edit that was done with content from various sources. I have no idea of the processing used. I am now doing an (mostly) HD version, and the levels need to match the DVD version. This is a technical and not an artistic project.

With a mix of clips/tracks on the timeline - some are 16-235, others are 0-255, how do I apply filters that will accurately conform the content to the same colorspace?

Note that a search of the Vegas help file on "601" or "709" yields no results.

Thanks!

Emre Safak · February 19th, 2008, 09:58 PM

Good question. I would recommend converting the offending 601 clips to 709 using avisynth: http://forum.doom9.org/showthread.php?s=&threadid=82217

Tom Barry has a 709 to 601 filter: http://www6.impacthosting.com/trbarry/

Someone else has the reverse: http://rennfast.dns2go.com/Public/HT...601tobt709.zip

It is too bad that the video world does not yet have proper color management!

Douglas Spotted Eagle · February 20th, 2008, 12:15 AM

Downconverting HD to SD automatically sets up the correct color range, but even so, there are presets used in GearShift for this conversion, or you can use the Cineform recommended presets from long ago.
You can use the broadcast filter as well, to clamp everything down, if that's you primary concern.

Jon Fairhurst · February 20th, 2008, 12:39 AM

Thanks everybody.

I've been working on a spreadsheet with all of the coefficients. Once I confirm the values, I'll plug them into the Channel Blend plug-in. I'll create presets for 0-255 to 16-235 and back, as well as various flavors of 709 and 601 conversions. I'll post my results to this thread.

One example would be if somebody took a 0-255 709 stream and squished it to 16-235 without converting to 601 color space. I'll make a preset to fix that sort of stuff.

Since I don't know what conversions and tools my source video has been through, I'll need to do some trial and error checks in order to get things to match. There are only so many combinations. Hopefully my presets will do the trick.

One thing for sure, just messing with offsets and gains doesn't work. And I don't believe that anybody applied gamma changes or other artistic tweaks.

I'm booked with other work all day tomorrow. I should get to try it out on Thursday...

Jon Fairhurst · February 22nd, 2008, 01:32 PM

Well, I calculated and confirmed my values using Excel. I plugged the values into the converter, and that wasn't the problem. I'll have to match things by hand as best I can.

However, here are the values I promised. Simply plug these into the Channel Blend plugin to convert between 601 and 709 color spaces. Both assume 0-255 ranges. You can use the Levels plugin before/after as needed to convert between 0-255 and 16-235. There is also a very slight saturation difference between 0-255 and 16-236 to consider.

709 to 601
R = 1.086*R - 0.072*G - 0.014*B
G = 0.097*R + 0.845*G + 0.058*B
B = -0.014*R - 0.028*G +1.041*B

601 to 709
R = 0.914*R + 0.079*G + 0.008*B
G = -0.105*R + 1.172*G - 0.067*B
B = 0.010*R + 0.032*G + 0.958*B

You can confirm these by using full field color bars and running these conversions back to back. The two conversions cancel one another, and the bars look normal.

With color bars, the green bars show the most difference. On natural video, I see the main differences as the red tones shift to/from magenta and orange.

This is all based on these equations:

Y' = 0.587*G' + 0.114*B' + 0.299*R' (601), and
Y' = 0.7152*G' + 0.0722*B' + 0.2126*R' (709).

(The ' indicates the gamma corrected domain.)

Glenn Chan · February 22nd, 2008, 02:03 PM

Jon, Vegas should be handling the luma co-efficients for you.

Bring color bars from your camera and check them on Vegas' scopes. If they line up, you don't have a problem.

*Vegas 7 has a bug with some JVC cameras. I calculated the numbers some while back, but it's easier to use Vegas 8.

2- In regards to 16-235 range versus 0-255 (or studio versus computer RGB), see my article about that:
http://glennchan.info/articles/vegas...or/v8color.htm

3- Modern HD (i.e. 709) and SD material also differ in their transfer functions and primaries. I would pretend that these differences don't exist and you'll get reasonable results that way. This is the way it's done in many high-end facilities. (To do things right can create more problems, whereas it doesn't really solve any problems because no one will tell the difference.)

Bill Ravens · February 22nd, 2008, 02:31 PM

whoa!
I bring the color bars in from my camera, either an HD110 or an EX1...they line up fine in the vegas wfm; and I got BIG problems after rendering to an SD format....well, big is a relative term. lets just say the colors are off by ~10-15 R'G'B' value

Jon Fairhurst · February 22nd, 2008, 03:03 PM

Quote:

Originally Posted by Glenn Chan

Jon, Vegas should be handling the luma co-efficients for you.

Glenn, as I wrote above, I'm working on a unique project. The content isn't from my camera. I have some SD source material that went through unknown processes. I also have the same material in HD, and again the process is unknown. And for technical reasons, I have to get the levels to match.

In my case, I need manual control. This has become more of a forensic task than anything.

To give you an idea of how divergent the SD and HD materials are, I have some stuff from BBC Research that seems to have been converted from 50 Hz to 60 Hz. The SD and HD cuts have the frames a different phases! I have one scene with a slow moving train. The spokes of the wheels don't even line up on individual frames between the cuts!

And that's before I even get to cropping, levels and gamma curves.

The equations above aren't needed in normal use, but can be helpful in case the content went through an NLE that applied (or failed to apply) the conversion appropriately.

Bill Ravens · February 22nd, 2008, 07:09 PM

Jon...

I tried your coefficients. Beleive me, they make matters worse.
I may not have much credibility with you, but, Vegas is really hosed. In the end, it's made me crazy trying to figure it all out. Glenn has tried to help, but, he doesn't want to admit to the problems with Vegas.

Emre Safak · February 22nd, 2008, 09:25 PM

Bill, let's be fair: how many video editing applications support ICC profiles--and not just for previewing, but assigning to clips?

Jon Fairhurst · February 22nd, 2008, 11:25 PM

Quote:

Originally Posted by Bill Ravens

I tried your coefficients. Beleive me, they make matters worse.

If the problem isn't a 601 <-> 709 color shift, these coefficients definitely won't help.

Still, they're nice to have in the toolbox in the case that you get stuff from another NLE that processed things wrong.

I used to think that in a pure digital workflow it wasn't important to lay down bars. Now that we have 0-255 and 16-235 as well as 601 and 709 color space, bars are as important as ever.

Glenn Chan · February 23rd, 2008, 12:31 AM

Quote:

Originally Posted by Jon Fairhurst

Glenn, as I wrote above, I'm working on a unique project. The content isn't from my camera. I have some SD source material that went through unknown processes. I also have the same material in HD, and again the process is unknown. And for technical reasons, I have to get the levels to match.

In that case there is a small possibility that you are working with HD material that's not 709... e.g. the footage might be using SMPTE 240M luma co-efficients. (I've encountered this myself because it's possible to set some cameras up wrong.)

Your numbers seem to match mine (actually I just realized that I copied my original numbers into a forum post wrong). I would round differently (e.g. round and not truncate), but that may or may not be important.

2- If there aren't bars on the source tapes, then... it might be difficult to know what the original colors are supposed to be. On top of that, there are color inaccuracies inherent to many video systems (e.g. metamerism, video knee, gamma 'correction', cheating in the color matrix, etc.).

Quote:

With a mix of clips/tracks on the timeline - some are 16-235, others are 0-255, how do I apply filters that will accurately conform the content to the same colorspace?

You might (or might not) need to apply the studio<-->computer RGB presets in the Color Corrector filter (or Levels; levels may be less accurate in 8-bit).

The article I mentioned earlier explains it:
http://glennchan.info/articles/vegas...or/v8color.htm

The whole 16-235 studio RGB versus 0-255 computer RGB issue is a Vegas thing / unique to Vegas.

Quote:

Glenn has tried to help, but, he doesn't want to admit to the problems with Vegas.

The problem with Vegas is that:
A- You have to do the color space conversions manually.
B- You have to know what color space conversion (if any) is appropriate.

I don't think it's fair to characterize me as not "admitting" to the problems with Vegas... I think I've always been clear that you have to do color space conversions manually sometimes.

Bill Ravens · February 23rd, 2008, 08:24 AM

Emre...

I surely can't argue with you about Vegas' seeming good points. It has many. However, it has one fatal flaw. There's NO WAY to do CCing in Vegas and have the ability to see what your final output will be. It's confirmed that the scopes are wrong and the preview window is wrong. So, how is one to do any kind of CC with finesse? Sure vegas can take the sledge hammer approach. But, it's certainly not the precision that a professional app needs to have.

Glenn Chan · February 23rd, 2008, 02:16 PM

Quote:

There's NO WAY to do CCing in Vegas and have the ability to see what your final output will be.

That's simply not true.

Anyways, I'm getting sick and tired of discussing this.

I've laid out how to get correct results in Vegas in my article(s). And I've checked my results against hardware scopes. I haven't seen any compelling evidence that what I've written is wrong.

Bill Ravens · February 23rd, 2008, 03:26 PM

Well, tell ya what. Nothing you've told me has helped.
Now what?The monitor is wrong and the scope is wrong. I'm supposed to CC on pure faith? I don't think so.
Maybe I'm just too dumb to be using Vegas. In that case there should be a disclaimer on the box, "dumb people need not waste their money".
Or, maybe, Sony should issue hardware scopes to all users so the users can figure out how to get decent output from their software.

February 19th, 2008, 04:50 PM	#1
Jon Fairhurst Inner Circle Join Date: May 2006 Location: Camas, WA, USA Posts: 5,513	Mixing HD (709) and SD (601) clips What plug in should I use to convert a Rec. 709 clip to Rec. 601 and vice versa? In my current project there was a cuts-only DVD edit that was done with content from various sources. I have no idea of the processing used. I am now doing an (mostly) HD version, and the levels need to match the DVD version. This is a technical and not an artistic project. With a mix of clips/tracks on the timeline - some are 16-235, others are 0-255, how do I apply filters that will accurately conform the content to the same colorspace? Note that a search of the Vegas help file on "601" or "709" yields no results. Thanks! __________________ Jon Fairhurst

February 20th, 2008, 12:15 AM	#3
Douglas Spotted Eagle VASST Join Date: Jan 2004 Location: Stockton, UT Posts: 5,648	Downconverting HD to SD automatically sets up the correct color range, but even so, there are presets used in GearShift for this conversion, or you can use the Cineform recommended presets from long ago. You can use the broadcast filter as well, to clamp everything down, if that's you primary concern. __________________ Douglas Spotted Eagle/Spot Author, producer, composer Certified Sony Vegas Trainer http://www.vasst.com

February 20th, 2008, 12:39 AM	#4
Jon Fairhurst Inner Circle Join Date: May 2006 Location: Camas, WA, USA Posts: 5,513	Thanks everybody. I've been working on a spreadsheet with all of the coefficients. Once I confirm the values, I'll plug them into the Channel Blend plug-in. I'll create presets for 0-255 to 16-235 and back, as well as various flavors of 709 and 601 conversions. I'll post my results to this thread. One example would be if somebody took a 0-255 709 stream and squished it to 16-235 without converting to 601 color space. I'll make a preset to fix that sort of stuff. Since I don't know what conversions and tools my source video has been through, I'll need to do some trial and error checks in order to get things to match. There are only so many combinations. Hopefully my presets will do the trick. One thing for sure, just messing with offsets and gains doesn't work. And I don't believe that anybody applied gamma changes or other artistic tweaks. I'm booked with other work all day tomorrow. I should get to try it out on Thursday... __________________ Jon Fairhurst

February 22nd, 2008, 01:32 PM	#5
Jon Fairhurst Inner Circle Join Date: May 2006 Location: Camas, WA, USA Posts: 5,513	Well, I calculated and confirmed my values using Excel. I plugged the values into the converter, and that wasn't the problem. I'll have to match things by hand as best I can. However, here are the values I promised. Simply plug these into the Channel Blend plugin to convert between 601 and 709 color spaces. Both assume 0-255 ranges. You can use the Levels plugin before/after as needed to convert between 0-255 and 16-235. There is also a very slight saturation difference between 0-255 and 16-236 to consider. 709 to 601 R = 1.086R - 0.072G - 0.014B G = 0.097R + 0.845G + 0.058B B = -0.014R - 0.028G +1.041B 601 to 709 R = 0.914R + 0.079G + 0.008B G = -0.105R + 1.172G - 0.067B B = 0.010R + 0.032G + 0.958B You can confirm these by using full field color bars and running these conversions back to back. The two conversions cancel one another, and the bars look normal. With color bars, the green bars show the most difference. On natural video, I see the main differences as the red tones shift to/from magenta and orange. This is all based on these equations: Y' = 0.587G' + 0.114B' + 0.299R' (601), and Y' = 0.7152G' + 0.0722B' + 0.2126R' (709). (The ' indicates the gamma corrected domain.) __________________ Jon Fairhurst

February 19th, 2008, 09:58 PM	#2
Emre Safak Trustee Join Date: Feb 2004 Location: Brookline, MA Posts: 1,447	Good question. I would recommend converting the offending 601 clips to 709 using avisynth: http://forum.doom9.org/showthread.php?s=&threadid=82217 Tom Barry has a 709 to 601 filter: http://www6.impacthosting.com/trbarry/ Someone else has the reverse: http://rennfast.dns2go.com/Public/HT...601tobt709.zip It is too bad that the video world does not yet have proper color management!

February 22nd, 2008, 02:03 PM	#6
Glenn Chan Inner Circle Join Date: Jun 2003 Location: Toronto, Canada Posts: 4,750	Jon, Vegas should be handling the luma co-efficients for you. Bring color bars from your camera and check them on Vegas' scopes. If they line up, you don't have a problem. *Vegas 7 has a bug with some JVC cameras. I calculated the numbers some while back, but it's easier to use Vegas 8. 2- In regards to 16-235 range versus 0-255 (or studio versus computer RGB), see my article about that: http://glennchan.info/articles/vegas...or/v8color.htm 3- Modern HD (i.e. 709) and SD material also differ in their transfer functions and primaries. I would pretend that these differences don't exist and you'll get reasonable results that way. This is the way it's done in many high-end facilities. (To do things right can create more problems, whereas it doesn't really solve any problems because no one will tell the difference.)

February 22nd, 2008, 02:31 PM	#7
Bill Ravens Join Date: Nov 2001 Location: santa fe, nm Posts: 3,264 Images: 10	whoa! I bring the color bars in from my camera, either an HD110 or an EX1...they line up fine in the vegas wfm; and I got BIG problems after rendering to an SD format....well, big is a relative term. lets just say the colors are off by ~10-15 R'G'B' value

February 22nd, 2008, 07:09 PM	#9
Bill Ravens Join Date: Nov 2001 Location: santa fe, nm Posts: 3,264 Images: 10	Jon... I tried your coefficients. Beleive me, they make matters worse. I may not have much credibility with you, but, Vegas is really hosed. In the end, it's made me crazy trying to figure it all out. Glenn has tried to help, but, he doesn't want to admit to the problems with Vegas.

February 22nd, 2008, 09:25 PM	#10
Emre Safak Trustee Join Date: Feb 2004 Location: Brookline, MA Posts: 1,447	Bill, let's be fair: how many video editing applications support ICC profiles--and not just for previewing, but assigning to clips?

February 23rd, 2008, 08:24 AM	#13
Bill Ravens Join Date: Nov 2001 Location: santa fe, nm Posts: 3,264 Images: 10	Emre... I surely can't argue with you about Vegas' seeming good points. It has many. However, it has one fatal flaw. There's NO WAY to do CCing in Vegas and have the ability to see what your final output will be. It's confirmed that the scopes are wrong and the preview window is wrong. So, how is one to do any kind of CC with finesse? Sure vegas can take the sledge hammer approach. But, it's certainly not the precision that a professional app needs to have.

February 23rd, 2008, 03:26 PM	#15
Bill Ravens Join Date: Nov 2001 Location: santa fe, nm Posts: 3,264 Images: 10	Well, tell ya what. Nothing you've told me has helped. Now what?The monitor is wrong and the scope is wrong. I'm supposed to CC on pure faith? I don't think so. Maybe I'm just too dumb to be using Vegas. In that case there should be a disclaimer on the box, "dumb people need not waste their money". Or, maybe, Sony should issue hardware scopes to all users so the users can figure out how to get decent output from their software.