Why no serious professional comparisons of prosumer equipment?

This site is an incredible resource, so this is not meant to be a slam, but why is no one doing serious (dare I say scientific) reviews and head-to-head comparisons of these cameras the way that sites like dpreview.com do for still cameras?

Is the low-end pro video/indie film market that much smaller than the prosumer photography market?

Surely I'm not the only one dying for identical res charts and studio shots to accurately, and objectively, evaluate these cameras?

Mainly because the cameras either just have, or are just about to, reach the hands of shooters. No doubt a shoot out will happen pretty soon.

They are preparing a big test with all HD and HDV cams next to each other. But they want to take it very seriously, so they are taking the time they need.
(Barry Green even proposed to test the Varicam and Cinealta next to them, that would be great!)

That will be great, I surely look forward to it!

But the still camera review sites are loaned cameras even before official release dates and usually have detailed reviews day and date. In fact, this is the case with almost all gear. Why is "pro" video equip treated so differently?

Again, I'm not disparaging the incredible work that those on this board do for us all ... I'm just curious as to why the situation is so different with video cams versus still cameras? Are the manufacturers trying to hide? Is there not a big enough audience to support that kind of review site?

Are the manufacturers trying to hide? Is there not a big enough audience to support that kind of review site?

No conspiracy to be found here. First of all, no manufacturer is happy to have a pre-production camera examined as a "test" and it's not fair to do so when being tested against other cams.
Second, they're not all that keen to get cameras out there for testing by independent parties prior to shipping. It doesn't necessarily help sales, and can certainly hurt sales. Look at what happened to JVC in the first month of sales alone.
Finally, none of the camera manufacturers have really allotted time for cameras going out early. The moment a cam comes off the line, it's been already long-ago paid for by an excited user.
Professional products are rarely pre-reviewed in most any industry, so the clear answer to your question is that yes, professional products are marketed and managed differently than consumer products.

So many of the tests that I have seen on other sites tend to concentrate on feature lists and test charts. What we really need is unbiased tests by experienced camera operators who have no ties to any brand or product and no bias towards any manufacturer.

This is hard to find, as an example I have been using Sony kit almost exclusivly for 20 years, and being honest it has served me well. It's not always been the best or the greatest, but I have made a living using it, as a result if someone gave me 2 products to review and one was a Sony, my opinion would be biased before I had even opened the box.

This site does however give us the opportunity to read others real world experiences and that is worth more than any scientific test. You just have to figure out who likes which brands!

I'd have to dissagree with the statement that professional equipment is rarely available for early review.

Pro level DSLR's routinely are. (Production models, not pre.) The same is true of a lot of computer equipment. For that matter, most cars are a lot more expensive than these rigs and they're made available for serious testing before their street dates.

At the very least review equipment is made available day and date. I rarely see pro video cameras being gifted to anybody but fanboys.

So I'm left to ponder why the same companies (Sony, Canon) treat their DSLR customers so differently?

I think this situation is peculiar to this industry because we have a history of puting up with it. It's "pro-level" and we shouldn't question them beyond that. ;-)

At the very least review equipment is made available day and date. I rarely see pro video cameras being gifted to anybody but fanboys.


I truly wish this were either correct, or that folks that are getting free cameras or long-term "loaners" would say exactly what they're getting if they're doing reviews.

As far as pro gear, I've been reviewing professional audio, video, and musical instruments for nearly 25 years now, published in print and webzines, and have yet to see an early release of pro audio gear, video gear, or musical instruments. They're usually not made available until the day the product starts shipping to resellers/retailers. There are a couple rare exceptions, but they're rare.

The video industry is microcosmically small to what the photo industry is, so it's not at all an accurate comparison. Video is growing, but so is high end photo markets.

Further, it's a much bigger deal to be sending around a few 5,000.00 cameras, or higher cost cameras, than it is to send around a few 500.00 cameras. They only allot one or two units for review even after they're shipping in most cases. As a reviewer, on many occasions lately, I've had to use a credit card as a means of assuring that I'll return the product once it's been reviewed. Many big companies like Grass Valley, Ikegami, Sony, and others are turning to this policy because they can't afford to have a 7500.00 switcher or similar not coming back within an allotted time.

They will show whatever the tester wants... I can PROMISE you that there will be tests that will show JVC, Sony, Panny and Canon are EACH the best. The people who gravitate toward one or the other will focus on the data that supports their natural loyalty. Human nature, I mean, if I go drop $10K on an HVX and some P2 cards and someone evaluates that it has the lowest rez of all the aforementioned... I naturally wont be too happy. I may decide to focus on color space, dynamic range or someone elses test that shows otherwise.

I saw a test done by someone last night between a Z1 and an XLH1 and the EXACT SAME RESULTS were used to show both were the better camera! XLH1 advocate says, "look how much more resolution and dynamic range the Canon has." Z1 advocate says, "The Z1 has better resoultion to the corners, not just in the middle and has dramatically less noise."

Bottom line, these cameras are different. Choose them based on a workflow you like, what the project is most suited for, or what is within your budget. There is no best camera...period endstop...



ash =o)

Hehe, and I wish a Canon EOS 5D or a BOXX workstation were $500!!! ;-)

I understand what you're driving at, though.

Still, lets limit ourselves to shipping equipment (all of the major contenders are in the marketplace now) there still is not a site (that I've found) that offers objective, unbiased, critical (repeatable) testing of this gear.

And that's a shame. We're left relying on the kindess of a few early adopters to try to divine the truth. I don't do that with $10k workstations, and I shouldn't have to for $10k cameras.
<End of rant.>

I think the Internet, with places like this, will solve the problem. Pre-Internet, we had magazine reviews. But they never had 'scientific' , to use your term, tests. Most of the time, they said the good things about how great the camera worked. Sell Sell Sell ! Guess who pays the bills? The same companies placing ads in the mag. The Icing on the cake was the review itself being confirmation of a fantastic product.
Perhaps I'm being overly critical of the bias.

Another reason they are lacking in quantitative tests is the public they are selling to are not educated to understand and have trust in the numbers.

Maybe I'm wrong.
-Les


Why no serious professional comparisons of prosumer equipment?

This site is an incredible resource, so this is not meant to be a slam, but why is no one doing serious (dare I say scientific) reviews and head-to-head comparisons of these cameras the way that sites like dpreview.com do for still cameras?

Is the low-end pro video/indie film market that much smaller than the prosumer photography market?

Surely I'm not the only one dying for identical res charts and studio shots to accurately, and objectively, evaluate these cameras?

Tests are usefull but you have to learn how to read between lines. The same goes for any type of products, cars, light bulbs, or whatever. Often it is the company that pays for the test, directly or indirectly, is the one, whose products come on top.

In the UK, at the moment, a PAL DVX100BE is on offer at <£2K and according to tests can record a 540x540 resolution signal on a 25 Mbps tape

How can justification be made for 100 Mbps recording on special media for a 600x600 resolution signal from a HVX200E?

I just dont get it, it seems like its more to do with selling P2 than selling a decent camera head.

Surely I'm not the only one dying for identical res charts and studio shots to accurately, and objectively, evaluate these cameras?

Woohoo! Looks like we found a volunteer to take care of all the planning, coordination, travel, leg-work, clean-up, and most importantly, financial backing for an HD shoot-out! ;-)

But silliness aside, Joseph, a number of us have expressed interest in putting together something (this has been all off-line discussion), but it really does take a lot of time, planning, work, and money to make something like that happen and be meaningful to devotees of digital video...not to mention the HVX200 is barely starting to hit the streets. Even when we know we can get all four major contenders together in one place at one time, from a business standpoint, what financial incentive is there for anyone interested in video to put forth that kind of effort? I think that's why you don't see real head-to-head in-depth comparisons out there...it isn't in anyone's business interest to do so.

So it is left to us DVinfo'ers to put something good together, for no reason more than the love of digital moving art. No doubt some sort of shoot-out will happen in due course...

That's why I asked about audience size. i'm sure dpreview and similar sites don't do their reviews out of the goodness of their hearts! ;-)

If the audience is sufficient, then the cost can be dealt with. (Can you tell I'm an advertising CD by day?)

In the UK, at the moment, a PAL DVX100BE is on offer at <£2K and according to tests can record a 540x540 resolution signal on a 25 Mbps tape

How can justification be made for 100 Mbps recording on special media for a 600x600 resolution signal from a HVX200E?

I don't think they are quite that similar. More like 540x500 vs 600x650 (maybe a little higher). But still, is that enough?

DVX has 4x3 CCD's doesn't it? So if you shoot 16x9, you letterbox and lose 25% vertical resolution; not so with Canon XL2. HVX shoots up to 60p for slow motion. Does it not have 4-channel audio? P2 cards have advantage for fast news downloads to satellite link, etc. The lens is longer and is HD and it may have better zoom and focus rings. It shoots 4:2:2 DVCPRO50 for better color. There are many advantages in the new camera.

Ash, I understand what you are saying but Barry Green and Shannon Rawls (and a couple of others) said they were going to do a big test.
Shannon has the Canon, Barry will have the HVX (and even said would try to get a Varicam and Cinealta to test along the others!), Others will have their camera...

That could conclude in a big big article here on DVinfo, with an OBJECTIVE rapport on everything: resolution, color, motion, ...
So, I for sure, am really looking forward to it.

I've always had a lot of respect for Adam Wilt, when he takes a look at a piece of gear. It would be good to get his take on it.

I've always had a lot of respect for Adam Wilt, when he takes a look at a piece of gear. It would be good to get his take on it.

I think he is in on Barry's test last I heard.

This thread and the one that Ash Greyson started which was titled "I can make a prediction right now about what real tests will show!" have been merged together into one, at the suggestion of our forum moderators.

Makes sense Chris...


ash =o)

Apart from testing resolution, color rendition, lens accuracy/warping, extreme latitude and low light, we should test compression artifacts (using accurate uncompressed/lossless blowups to illustrate problems.

One thing I have noticed in motion artifact tests, is that people squint at it and say it looks fine, but if you blow up the frame you discover it is full of fine to medium blocking. So your 1440*1080 frame has 720*540 and less resolution in parts. People probably don't notice it until it gets to 360*270, but when you blow up to a really big screen everything is noticeable, even just discontenting so. So can I suggest two sorts of tests, one movement based, and the other changed based (movement retains a lot of moving previous information, compared to an change test that has lots of changing areas ion screen). One thing that has been noticed on consumer/pro-sumer camera, is changing the camera mode or shutter, has a profound effect on motion artifacts to.

To recreate them for each camera, they could be 60fps+ visually lossless footage at 10+bits put on a big screen running at high enough refresh. Maybe one of the higher end cameras could provide it in that format. Failing that, getting a loaner of the new Sony SHD projector might be able to do it by showing 4 HD feeds on one screen.

Two SHD screens:

http://www.dvinfo.net/conf/showthread.php?t=57770