Camcorder comparisons?

Ben Wiens · January 11th, 2004, 08:36 PM

I think it is great what you are proposing Michael. Comments are generally too subjective. We need standardized tests. Robin Liss at CamcorderInfo said she was going to do such tests, but they ended up being too simplistic. And your suggestion to make a standardized test so individuals can even do the test is the best suggestion I have heard so far. Here are some things I would like included in the tests and evaluation. But these should be video and not still tests. I'm still adding to these:

PICTURE AND SOUND
(1) Picture brightness and clarity as % of normal at 1, 10, 100, 1000 LUX, using readily available standardized objects or scenes.
(2) Grain at 1, 10, 100, 1000 LUX
(3) Dynamic range of colors at 1, 10, 100, 1000 LUX
(4) Sharpness
(5) JPG compression, inter-line twitter and other artifacts
(6) Color correctness compared to test sheet at 1, 10, 100, 1000 LUX
(7) Lense flare and aberation
(8) Resolution
(9) Sound quality, stereo available, motor noise, wind cut, jacks, hot shoe microphone

CAMERA (major features)
(1) 1CCD, 1CCD RGB, or 3CCD
(2) Manual controls available, focus, shutter, aperature, sharpening etc.
(3) Stabilization, OIS or DIS, and effectiveness
(4) Bottom or top loading
(5) Optical zoom ratio
(6) Mass and size
(7) Photosite area
(8) Ratio 4:3 and/or 16:9 with photosite to recorded pixel ratio
(9) Viewfinder and LCD viewability and details
(10) Interlaced and/or progressive mode available
(11) Environmental evaluation, shock, water, cold, heat, handling. I wouldn't want to damage my camera or a borrowed one, but in case people do drop their camera by mistake they could report this.

Michael Wisniewski · January 11th, 2004, 10:12 PM

Cool! Thanks, that's a great list

Bryan Beasleigh · January 11th, 2004, 11:42 PM

Ben you are kidding, aren't you. I rather doubt most profesional reviews go to that much trouble.

How come you didn't do all that with your 852?

Frank Granovski · January 12th, 2004, 12:02 AM

Trouble is, few, if any present day cams don't even come close to our expectations, unless one replaces the PV-DV953's CCDs with the DVX100's CCDs, drill a hole to add an XLR socket and spray paint the shell black. :-))

Ben Wiens · January 12th, 2004, 12:53 AM

Bryan, professional reviews don't include tests of all the above. That is why we are left guessing. The above list is only my short list. And no I didn't check all the things in the list before buying my 852 but I had most of them in mind. I can't do all these tests myself and no one had produced a report like that for my camcorder.

Bryan Beasleigh · January 12th, 2004, 01:53 AM

"Bryan, professional reviews don't include tests of all the above. That is why we are left guessing."

I'm not left guessing. I know my equipments capabilitie's, it's limitations and it's shortfalls and I make do. Put a fully charged battery in the camera and a tape and let's see what it will do. Same with the audio.

Now if we were talking a several hundred thousand dollar camera i could see some of the information would be valid, but a consumer or prosumer camera. Who would want to waste the time even trying.

There are those that would waste a half a lifetime selecting the perfect widget only to find that it didn't really exist.

To some that is a hobby in itself. I suspect that you enjoy testing a product more than actually using it for the purpose it was designed for and i'll wager that you thoroughly enjoy yourself doing so. There is no negative connetation attached to my statment, other than to say that we don't all have the same need for perfection.

This i

Jeff Donald · January 12th, 2004, 02:41 AM

Who is supplying, or purchasing, the tens of thousands of dollars of test gear required for some of the tests you purpose? Even the $200,000 cameras Bryan mentions, aren't put through reviews like you purpose? Why? Because the people who would buy and use those type of cameras will want to shoot with the camera and subjectively evaluate the image quality.

When I used to purchase cameras for my production company, I rarely looked at the technical specs. I reviewed the list of features to see if the camera had my required feature set. But the technical specs like signal to noise ratio of 60db vs. 59.9db made little difference to me. The picture quality was much more important than a .1db difference in a spec. The subjectiveness of image quality can't be condensed to a single number or even a series of numbers or test.

Ben Wiens · January 12th, 2004, 11:05 AM

Phil Askey at dpreview.com has created an extreemly popular website for digital still cameras. He tests about as many equivalent things as I was proposing. People flock to this site to get reviews. Thousands of people all around the world choose camera based on these extensive tests. Even $600 cameras. There are thousands of digital still cameras. Who would get to test most of these in a camera store or show? There are also too many camcorders out there to be able to always find one to test in person, in my humble opinion.

Bryan Beasleigh · January 12th, 2004, 11:19 AM

I can see an argument for a standardized review. Many of the reviews are simply somebody regurgatating the manufacturers promotional information. The review criteria you suggest is extreme overkill though especially for consumer , prosumer cameras.

It would be an excellent make work project if someone were to obtain funding. Sounds like something the Canadian Government would go for.

If such a review was mandated then think of the consequences. the price of the camera would be increased and their would be a need for books and even seminars to decipher the findings. Chris would have to add several new forums.

Nah, I'll just flip a coin, thank you very much.

Michael Wisniewski · January 12th, 2004, 01:34 PM

Yeah dpreview.com definitely sets a good precedent for this and Ben's list is a good place to start.

Bryan's got a good point, those tests might be a costly and time consuming process but I think if I start with the highest possible standards and then work down from there to the most practical and easiest tests to conduct we can get some good information for comparison - and we'll end up with a lot of good eyeball material for people to judge themselves.

Manufacturer technical spec comparisons are easy enough, in fact I'm almost done with the spreadsheet, for what it's worth, and I'll post it in about a week. So far I've got all the current camcorders for Sony, Panansonic, Canon, and JVC specs matched up, but I was having problems with the higher end cams (above US$5,000) because I didn't understand some of the specs.

Ben Wiens · January 12th, 2004, 05:13 PM

Actually I'm open to many things. Bryan has a suggestion that I've not thought of before. Tomorrow, I'll consider checking it out. I'll flip a coin to see whether I should even get out of bed, then whether I should eat breakfast and so on. It might lead to some interesting things actually.

Bryan Beasleigh · January 12th, 2004, 08:17 PM

Ok
I give up, but then i have a camera and equipment that i'm happy with. I responded with the coin toss because I thought it better than incurring a masssive debt for a questionable review protocol.

It would be better to ask a happy customer how they made their decision rather than one who is unhappy. Jeff has had years of expereince, operating as well as overseeing the repair of video cameras, yet his methods for selection are simple.

It happens all too often that we get mired in an unrealistic world of specifications that have absolutely no baering on our actual usage of the product.

How does it look, how does it sound, is it dependable and well built? Will it do the job that I want it to? Is it reasonably priced?

It works for me and I am a tech, everything I do and enjoy has a technical base. Why make things more complicated than they really have to be.

I'm flown all over North America to find the answers to problems that others can't solve. Most of the problems I find are very basic. People have overlooked the simple reason and made the whole process more difficult than it should be.

January 11th, 2004, 08:36 PM	#16
Ben Wiens Regular Crew Join Date: Dec 2002 Location: Vancouver BC Canada Posts: 56	Standardized tests are needed I think it is great what you are proposing Michael. Comments are generally too subjective. We need standardized tests. Robin Liss at CamcorderInfo said she was going to do such tests, but they ended up being too simplistic. And your suggestion to make a standardized test so individuals can even do the test is the best suggestion I have heard so far. Here are some things I would like included in the tests and evaluation. But these should be video and not still tests. I'm still adding to these: PICTURE AND SOUND (1) Picture brightness and clarity as % of normal at 1, 10, 100, 1000 LUX, using readily available standardized objects or scenes. (2) Grain at 1, 10, 100, 1000 LUX (3) Dynamic range of colors at 1, 10, 100, 1000 LUX (4) Sharpness (5) JPG compression, inter-line twitter and other artifacts (6) Color correctness compared to test sheet at 1, 10, 100, 1000 LUX (7) Lense flare and aberation (8) Resolution (9) Sound quality, stereo available, motor noise, wind cut, jacks, hot shoe microphone CAMERA (major features) (1) 1CCD, 1CCD RGB, or 3CCD (2) Manual controls available, focus, shutter, aperature, sharpening etc. (3) Stabilization, OIS or DIS, and effectiveness (4) Bottom or top loading (5) Optical zoom ratio (6) Mass and size (7) Photosite area (8) Ratio 4:3 and/or 16:9 with photosite to recorded pixel ratio (9) Viewfinder and LCD viewability and details (10) Interlaced and/or progressive mode available (11) Environmental evaluation, shock, water, cold, heat, handling. I wouldn't want to damage my camera or a borrowed one, but in case people do drop their camera by mistake they could report this.

January 12th, 2004, 12:53 AM	#20
Ben Wiens Regular Crew Join Date: Dec 2002 Location: Vancouver BC Canada Posts: 56	My short list Bryan, professional reviews don't include tests of all the above. That is why we are left guessing. The above list is only my short list. And no I didn't check all the things in the list before buying my 852 but I had most of them in mind. I can't do all these tests myself and no one had produced a report like that for my camcorder.

January 12th, 2004, 02:41 AM	#22
Jeff Donald Warden Join Date: Mar 2002 Location: Clearwater, FL Posts: 8,287	Who is supplying, or purchasing, the tens of thousands of dollars of test gear required for some of the tests you purpose? Even the $200,000 cameras Bryan mentions, aren't put through reviews like you purpose? Why? Because the people who would buy and use those type of cameras will want to shoot with the camera and subjectively evaluate the image quality. When I used to purchase cameras for my production company, I rarely looked at the technical specs. I reviewed the list of features to see if the camera had my required feature set. But the technical specs like signal to noise ratio of 60db vs. 59.9db made little difference to me. The picture quality was much more important than a .1db difference in a spec. The subjectiveness of image quality can't be condensed to a single number or even a series of numbers or test. __________________ Jeff Donald Carpe Diem Search DVinfo.net for quick answers \| Where to Buy? From the best in the business: DVinfo.net sponsors

January 12th, 2004, 11:05 AM	#23
Ben Wiens Regular Crew Join Date: Dec 2002 Location: Vancouver BC Canada Posts: 56	dpreview.com Phil Askey at dpreview.com has created an extreemly popular website for digital still cameras. He tests about as many equivalent things as I was proposing. People flock to this site to get reviews. Thousands of people all around the world choose camera based on these extensive tests. Even $600 cameras. There are thousands of digital still cameras. Who would get to test most of these in a camera store or show? There are also too many camcorders out there to be able to always find one to test in person, in my humble opinion.

January 12th, 2004, 05:13 PM	#26
Ben Wiens Regular Crew Join Date: Dec 2002 Location: Vancouver BC Canada Posts: 56	Fipping A Coin Actually I'm open to many things. Bryan has a suggestion that I've not thought of before. Tomorrow, I'll consider checking it out. I'll flip a coin to see whether I should even get out of bed, then whether I should eat breakfast and so on. It might lead to some interesting things actually.

January 11th, 2004, 10:12 PM	#17
Michael Wisniewski Wrangler Join Date: Jul 2002 Location: Makati, Metro Manila Posts: 2,706 Images: 32	Cool! Thanks, that's a great list

January 11th, 2004, 11:42 PM	#18
Bryan Beasleigh Trustee Join Date: May 2002 Location: Barrie, Ontario, Canada Posts: 1,922	Ben you are kidding, aren't you. I rather doubt most profesional reviews go to that much trouble. How come you didn't do all that with your 852?

January 12th, 2004, 12:02 AM	#19
Frank Granovski Outer Circle Join Date: May 2002 Location: Hope, BC Posts: 7,524	Trouble is, few, if any present day cams don't even come close to our expectations, unless one replaces the PV-DV953's CCDs with the DVX100's CCDs, drill a hole to add an XLR socket and spray paint the shell black. :-))

January 12th, 2004, 01:53 AM	#21
Bryan Beasleigh Trustee Join Date: May 2002 Location: Barrie, Ontario, Canada Posts: 1,922	"Bryan, professional reviews don't include tests of all the above. That is why we are left guessing." I'm not left guessing. I know my equipments capabilitie's, it's limitations and it's shortfalls and I make do. Put a fully charged battery in the camera and a tape and let's see what it will do. Same with the audio. Now if we were talking a several hundred thousand dollar camera i could see some of the information would be valid, but a consumer or prosumer camera. Who would want to waste the time even trying. There are those that would waste a half a lifetime selecting the perfect widget only to find that it didn't really exist. To some that is a hobby in itself. I suspect that you enjoy testing a product more than actually using it for the purpose it was designed for and i'll wager that you thoroughly enjoy yourself doing so. There is no negative connetation attached to my statment, other than to say that we don't all have the same need for perfection. This i

January 12th, 2004, 11:19 AM	#24
Bryan Beasleigh Trustee Join Date: May 2002 Location: Barrie, Ontario, Canada Posts: 1,922	I can see an argument for a standardized review. Many of the reviews are simply somebody regurgatating the manufacturers promotional information. The review criteria you suggest is extreme overkill though especially for consumer , prosumer cameras. It would be an excellent make work project if someone were to obtain funding. Sounds like something the Canadian Government would go for. If such a review was mandated then think of the consequences. the price of the camera would be increased and their would be a need for books and even seminars to decipher the findings. Chris would have to add several new forums. Nah, I'll just flip a coin, thank you very much.

January 12th, 2004, 01:34 PM	#25
Michael Wisniewski Wrangler Join Date: Jul 2002 Location: Makati, Metro Manila Posts: 2,706 Images: 32	Yeah dpreview.com definitely sets a good precedent for this and Ben's list is a good place to start. Bryan's got a good point, those tests might be a costly and time consuming process but I think if I start with the highest possible standards and then work down from there to the most practical and easiest tests to conduct we can get some good information for comparison - and we'll end up with a lot of good eyeball material for people to judge themselves. Manufacturer technical spec comparisons are easy enough, in fact I'm almost done with the spreadsheet, for what it's worth, and I'll post it in about a week. So far I've got all the current camcorders for Sony, Panansonic, Canon, and JVC specs matched up, but I was having problems with the higher end cams (above US$5,000) because I didn't understand some of the specs.

January 12th, 2004, 08:17 PM	#27
Bryan Beasleigh Trustee Join Date: May 2002 Location: Barrie, Ontario, Canada Posts: 1,922	Ok I give up, but then i have a camera and equipment that i'm happy with. I responded with the coin toss because I thought it better than incurring a masssive debt for a questionable review protocol. It would be better to ask a happy customer how they made their decision rather than one who is unhappy. Jeff has had years of expereince, operating as well as overseeing the repair of video cameras, yet his methods for selection are simple. It happens all too often that we get mired in an unrealistic world of specifications that have absolutely no baering on our actual usage of the product. How does it look, how does it sound, is it dependable and well built? Will it do the job that I want it to? Is it reasonably priced? It works for me and I am a tech, everything I do and enjoy has a technical base. Why make things more complicated than they really have to be. I'm flown all over North America to find the answers to problems that others can't solve. Most of the problems I find are very basic. People have overlooked the simple reason and made the whole process more difficult than it should be.