OK, So when are we hooking up to compare these cameras?

Mike Teutsch · January 4th, 2006, 02:13 PM

Quote:

Originally Posted by Shannon Rawls

Ok. I'm learning. I'm new at this, so bear with me.....
Do I have this right?

"RESOLUTION CHARTS AREN'T GOOD ENOUGH FOR TESTING CAMERAS ANYMORE."
The results are a lie.

correct?

- ShannonRawls.com

I don't believe that is what they are saying or meaning. It is just that there is more to testing a camera than checking what the lenses and CCD's can do. There are motion artifacts, light level conciderations, and on and on.

They just want the tests to be more than a Resolution Chart tests, and more like real world tests of all or more aspects.

After all, my Nikon D70 would out rez all of these cameras if I figure this right, but I sure can't make a movie with it.

I look forward to see the results of these tests, and would love to be there.

Mike

Kevin Shaw · January 4th, 2006, 02:17 PM

Quote:

Originally Posted by Chris Barcellos

If you need any muscle at this "test", I would be willing to help just to get a look at all of the rigs..

Chris: this will be taking place on Feb. 15th in Sacramento, California, as part of an event being sponsored by WEVA. If you think you can make it send me an email and I'll give you the details.

Shannon Rawls · January 4th, 2006, 02:24 PM

Quote:

Originally Posted by Mike Teutsch

I don't believe that is what they are saying or meaning. It is just that there is more to testing a camera than checking what the lenses and CCD's can do. There are motion artifacts, light level conciderations, and on and on.

HENCE......

Quote:

Originally Posted by Shannon Rawls

Let's shoot some flowers & birds for color testing and film some weirdo strangers in Hollywood for motion comparisons! We can even do some run gun type film making. I'll hire an actress and actor for the day to do some public dramatic stuff in the streets of Hollywood. Maybe we'll have them pull up in a car and get out slamming the doors and begin arguing as they walk up the street!

*scratching my head* OK, so what's the problem? Did you guys miss that part?

- ShannonRawls.com

Les Dit · January 4th, 2006, 02:25 PM

Res charts are just one useful data point. Nobody is saying that it's all that matters. Motion compression artifacts can be an issue, the flames and birds that someone shot over on the JVC thread really tell me that the mpeg2 is showing no problems on the JVC 100 .
Can't wait to see the charts!
-Les

Kevin Shaw · January 4th, 2006, 02:28 PM

Quote:

Originally Posted by Charles Papert

My suggestion to you guys is that if you are going to the bother of doing the tests, put as much energy into leveling the playing field as possible.

Any suggestions anyone can provide along these lines would be appreciated. Some things I'm particularly wondering about are how to establish a fair exposure comparison, how to ensure an equivalent white balance, and what modes to run the cameras in that will be most useful to people. I probably won't have time to run all the cameras through all of their paces, so my plan is to pick a single "standard" recording mode for each camera for the main event and then run whatever experiments I can beyond that. I also hope to do a comparison of encoding the resulting footage to widescreen SD and compressed HD formats, so any recommendations of encoders to test are welcome.

Mike Marriage · January 4th, 2006, 02:40 PM

Quote:

Originally Posted by Shannon Rawls

"RESOLUTION CHARTS AREN'T GOOD ENOUGH FOR TESTING CAMERAS ANYMORE."
The results are a lie.

correct?

I could be wrong BTW, I am just saying it is how I would do it if I was designing the camera. :)

I wasn't trying to knock your test plans or anything, just suggesting additional considerations so the test is DEFINITIVE :)

I am still trying to think of a way to test resolution objectively if intelligent deinterlacing is being used... any ideas guys?

Charles Papert · January 4th, 2006, 02:56 PM

It's a tricky one, Kevin...take white balance for instance. Nominally you can just point all of the cameras at a white or grey object and do a white balance. However, each manufacturer has a different bias to their cameras DSP, so one might appear cooler than another, one more green than another etc. Ideally one would "paint" the cameras to match each other as much as possible, using a vectorscope and a broadcast monitor. Then there's the exposure issue--as has been discussed in another thread, simply setting all the cameras at the same gain setting (say, 0 db) does not mean they have the same gain processing internally.

The very least that one can do to achieve parity between the cameras is to A/B them through a trusty monitor and adjust the second camera to match the first by eye, then put the third camera in place of the second and adjust it, and so on. Of course, this requires a lot of fiddling with menus and each camera has different nomenclature and features, so a lot of trial and error will result.

If the cameras are not matched, inevitably a certain amount of false result may apply--anyone viewing the footage may decide a preference for one camera's image over another when under more controlled circumstances, they may choose differently.

To me, the only way to qualitatively judge two cameras is to match their images as closely as possible AND have the framing as similar as possible. It makes for a tedious shoot day; it's not as spontaneous as shooting a wedding or as sexy as shooting two actors walking down the street having a fight, but there it is.

Quick example--say you are shopping for a 42" plasma; it's pretty obvious that all 42" plasmas next to each other in the store should be exhibiting the same program in the same mode...think how much harder it is would be if one was zoomed in, another was stretched, etc. Plus you hope that they are adjusted as well as possible; you'd hate to plunk down your money for the Sony only to find out the next time you went into the store that the Panasonic next to it had subsequently been dialed in and you actually preferred it.

Kevin Shaw · January 4th, 2006, 03:54 PM

Thanks Charles -- sounds like it would be useful to try to line up a decent broadcast monitor and maybe a vectorscope. We will have some time before the shoot to try to match everything up, and framing will probably be locked in place, so we could at least attempt to make this an equal test.

Brian Wengrofsky · January 4th, 2006, 07:12 PM

Wish I was on the left coast to help out! If any of you could include the following in your test, it would be extremely useful:

In at least one of your shots, include some well-lit primary colors - ESPECIALLY RED. I've seen HDV and even possibly some HVX200 material appear blocky, as if it were losing resolution in those areas, such as auto taillights (and parts of Kaku's footage in the bike store). This is an issue I haven't seen addressed anywhere as of yet - perhaps I missed it?

I did tests with the Z1 shooting some custom charts I made in Photoshop, and it revealed these artifacts - mostly in primary colors. I would be interested in seeing how these different cameras perform in this regard. (Also - be happy to send these charts as image files to anyone if they would want to replicate my test).

John McGinley · January 5th, 2006, 09:45 PM

Quote:

Originally Posted by Shannon Rawls

So you mean to tell me if the Canon yields 100,000 lines of resolution and we are seeing this with our own optical eyeballs, you're saying...."don't beleive it, it was done electronically in the camera so it don't count??" LOL

Hey if it yields 100,000 lines of resolution, forget the Origin, I'm buying a Canon. :-)

January 4th, 2006, 02:56 PM	#22
Charles Papert Wrangler Join Date: Jan 2002 Location: Los Angeles, CA Posts: 6,810	It's a tricky one, Kevin...take white balance for instance. Nominally you can just point all of the cameras at a white or grey object and do a white balance. However, each manufacturer has a different bias to their cameras DSP, so one might appear cooler than another, one more green than another etc. Ideally one would "paint" the cameras to match each other as much as possible, using a vectorscope and a broadcast monitor. Then there's the exposure issue--as has been discussed in another thread, simply setting all the cameras at the same gain setting (say, 0 db) does not mean they have the same gain processing internally. The very least that one can do to achieve parity between the cameras is to A/B them through a trusty monitor and adjust the second camera to match the first by eye, then put the third camera in place of the second and adjust it, and so on. Of course, this requires a lot of fiddling with menus and each camera has different nomenclature and features, so a lot of trial and error will result. If the cameras are not matched, inevitably a certain amount of false result may apply--anyone viewing the footage may decide a preference for one camera's image over another when under more controlled circumstances, they may choose differently. To me, the only way to qualitatively judge two cameras is to match their images as closely as possible AND have the framing as similar as possible. It makes for a tedious shoot day; it's not as spontaneous as shooting a wedding or as sexy as shooting two actors walking down the street having a fight, but there it is. Quick example--say you are shopping for a 42" plasma; it's pretty obvious that all 42" plasmas next to each other in the store should be exhibiting the same program in the same mode...think how much harder it is would be if one was zoomed in, another was stretched, etc. Plus you hope that they are adjusted as well as possible; you'd hate to plunk down your money for the Sony only to find out the next time you went into the store that the Panasonic next to it had subsequently been dialed in and you actually preferred it. __________________ Charles Papert www.charlespapert.com

January 4th, 2006, 07:12 PM	#24
Brian Wengrofsky New Boot Join Date: Jan 2006 Location: New York City Posts: 8	Test Request... Wish I was on the left coast to help out! If any of you could include the following in your test, it would be extremely useful: In at least one of your shots, include some well-lit primary colors - ESPECIALLY RED. I've seen HDV and even possibly some HVX200 material appear blocky, as if it were losing resolution in those areas, such as auto taillights (and parts of Kaku's footage in the bike store). This is an issue I haven't seen addressed anywhere as of yet - perhaps I missed it? I did tests with the Z1 shooting some custom charts I made in Photoshop, and it revealed these artifacts - mostly in primary colors. I would be interested in seeing how these different cameras perform in this regard. (Also - be happy to send these charts as image files to anyone if they would want to replicate my test).

January 4th, 2006, 02:25 PM	#19
Les Dit Major Player Join Date: Oct 2003 Location: Southern Cal-ee-for-Ni-ya Posts: 608	Res charts are just one useful data point. Nobody is saying that it's all that matters. Motion compression artifacts can be an issue, the flames and birds that someone shot over on the JVC thread really tell me that the mpeg2 is showing no problems on the JVC 100 . Can't wait to see the charts! -Les

January 4th, 2006, 03:54 PM	#23
Kevin Shaw Inner Circle Join Date: Sep 2004 Location: Sacramento, CA Posts: 2,488	Thanks Charles -- sounds like it would be useful to try to line up a decent broadcast monitor and maybe a vectorscope. We will have some time before the shoot to try to match everything up, and framing will probably be locked in place, so we could at least attempt to make this an equal test.