Pdw-700, pdw-800, pmw-350

[Mike Marriage]

Hey, where's all the hostility come from :( We're all just trying to help each other, let's keep it friendly.

Just been toying with various settings and shots on my 350 including a few torture tests. I don't own an EX1/3 but I use them a lot and think the 350 has it beat in every way. The EX1/3 is noisier and just seems to make a more "mucky" picture for want of a better word. Having said that, I have often intercut EX3 and PDW700 material and later been unable to spot which is which. The stock EX lens is normally the giveaway.

I've found the 350 to produce a very clean image by comparison. Maybe it lacks a certain organic tone, it's all very subjective.

What I'd like to do is shoot a few test scenes simultaneously with a few cameras at once. One exterior, a controlled set, mixed WB etc. Anyone fancy it? Not sure when I'll have the time mind!

[Alister Chapman]

If there was one thing a learnt when I did the 6 camera shoot-out, it is that anything like that takes a lot more time than you would expect. Even using stock camera settings you have to be sure that you are using a lighting range that is fair to all the cameras, for example you can easily come up with a scene that favors one camera or another. Then you need to ensure the light level is one that is neutral to all the cameras involved. Too bright and one camera may need ND, which might skew your results as the aperture on the lens will be different, too dark and one camera may need gain, which is clearly unfair. In my tests I had to come up with a light range that was bright enough for the HVX201 yet not so bright that the HPX301's iris was stopped down into diffraction limiting. Arguably you should test with charts as these will reveal the true colorimetry, dynamic range and resolution of a camera, but these don't really reveal the real world picture "look" which is far more subjective, but as I said earlier different scenes may favor different cameras.

Following a recent experience when testing a Canon DSLR against an EX I would still at least like to see bitmap frame grabs from each camera to compliment any highly compressed video clips. With the Canon/EX test the difference in resolution and aliasing between the cameras stood out like a sore thumb in the original footage and frame grabs, but when compressed for Vimeo the Canon barely looked any different to the EX and anyone judging the two by the video clip alone would have gained the impression that they are similar in performance when they are not. This would I guess be fine if your end product was clips for Vimeo, but I suspect most of us are aiming a little higher than that!

You see these same discussions every time someone or some organisation does a side by side camera shootout as everyone has their own view as to how it should be done, the people that take the time to do the tests are criticised for doing them, because in some way the test is viewed as flawed. We have to keep reminding ourselves that as well as a technical exercise, this is also an art form and what works for one person may not work for the next. As has been said the best way to see if a camera is right for you is to try it for yourself. All that those of us that do compare cameras can do is to try and provide an un biased opinion based on our own experiences.

[Alister Chapman]

Here are a couple of frame grabs from my PDW-700 and PMW-350. These were done a little while ago, but shows how close these two cameras are (or can be).

[Doug Jensen]

Alister,
Out of those two screen grabs, I prefer the 350. If that's the way your two cameras look, then I can understand the quandry you face in deciding which one to keep. Our experience has been very different.

Tom,
I can assure you that there was nothing "insincere" about my offer to test your scene files for you side-by-side with an F800. You challenged my opinion about what I have observed with my own eyes, so I offered to re-do the tests with your settings so there would be no questions. Sorry I wasted my time even making the offer. If the situation was reversed, I'd jump at the chance to have you run some tests with my settings. Do happen to have an F800 available?

[Mike Marriage]

Originally Posted by Alister Chapman

Here are a couple of frame grabs from my PDW-700 and PMW-350. These were done a little while ago, but shows how close these two cameras are (or can be).

What lenses? It appears to be the lens making the biggest difference, in favour of the 350.

[Alister Chapman]

The 700 had a Canon KJ20 and the 350 had it's kit lens. The KJ20 is not the best of HD lenses.

There are more reasons why the PDW-700/F800 and PMW-350 should look the same than there are reasons for them to be different. After all they both have 2/3" sensors of the same resolution with similar pixel size, very similar noise, sensitivity and dynamic range. That is then processed by 14 bit DSP's (so dynamic range should be the same) in both cameras, using the same gamma curves and color matrix and what should be the same detail levels (as we have discovered there are definitely differences here). CMOS and CCD sensors both use very similar MOS capacitor devices to capture the light, it is the read out methods that are different. With CMOS this used to mean more noise due to heat and the high frequency clock signals needed for readout and on chip analog to digital conversion. Sony have however now managed to tame this noise with on chip analog and digital noise sampling/reduction circuits so other than the read out method the difference in the signals produced by CCD and CMOS sensors should be minimal.

So other than motion, these cameras should look similar. Oh one more thing, the grabs were taken from the camera recordings, so the 350 is 4:2:0 35Mb/s and the 700 4:2:2, 50Mb/s. Both set to progressive, both cameras using the same gamma and matrix settings but different detail settings.

[Tom Roper]

Originally Posted by Doug Jensen

I can assure you that there was nothing "insincere" about my offer to test your scene files for you side-by-side with an F800. You challenged my opinion about what I have observed with my own eyes, so I offered to re-do the tests with your settings so there would be no questions. Sorry I wasted my time even making the offer. If the situation was reversed, I'd jump at the chance to have you run some tests with my settings. Do happen to have an F800 available?

I did not challenge your opinion. I asked to see the results of the tests, whereupon I saw they were not really tests but ad hoc observations, but you were using them to counter what really were formal tests with rules (rightly or wrongly), put forth by Alister in his 6 cam test.

That said, I apologize for characterizing the offer to review my scene files as insincere, I meant that in the context that it was offered with a hurried deadline and in lieu of just laying out the information I was really trying to get at, that you used a particular gamma on camera A, a particular scene setting on camera B, and in a particular situation you had a particular observation about it. Instead, you went straight to the conclusions that I never would have said were wrong anyway because I don't know, with the minor exception of concluding even the EX1 was better, which since I do own that cam as well, made it hard to take much of anything seriously. That and I don't have any complaints or problems with my scene files to lose sleep over.


I also was not, am not holding out a particular scene file of my own as nirvana. To me, they are starting points, I freely adjust them as I feel the need, and I don't subscribe that by dialing in just one size fits all situations for everybody. There are other variables to take control of as well, particularly exposure.

My reasons for purchasing the EX350 were affordable 2/3 inch performance in a light weight package, long battery life, a kit lens with autofocus, color viewfinder and good sensitivity. I can understand how one takes umbrage at claims that such is equal or better than a 700/800 if you can understand I have not been making such claims. Accordingly, although I am financially equipped to rent or buy a 700/800, doing so while the result would be interesting does not change fundamentally why the 350 was the right choice for me. I have not had problems achieving the desired look (in my hands), and I'm sure I would be equally pleased or more so with the image of the 700/800/F23/F35/Red et al.

[Paul Cronin]

No Hostility here Mike just trying to make the camera scene setting better with great help from Doug now on three test.

Tom I still stand by our testing method. I have been involved in testing multi million dollar boats against each other for 25 years, and all the lab or standardize testing in the world does not compare to real world field testing. That is where they spend the real money. We are doing real world easy on the camera field-testing. No fast motion (YET although I have shot the 350 for 42 hrs in the helicopter), and with the most natural color we can find that is easy to reach for both of us with our very busy businesses/lives. We use STD as a start on each test then multi scene files we think are the best for the cameras as we learn. Yes they change daily, and that is why we have not posted results because we are not happy with the results yet. If you do not like how we are testing fine no problem but don’t tell us it is wrong. This is how we shoot our jobs and that is what we deliver to our clients. So PLEASE lets just move on and talk about settings between the cameras. That is why I started this thread!

“You see these same discussions every time someone or some organization does a side by side camera shootout as everyone has their own view as to how it should be done, the people that take the time to do the tests are criticized for doing them, because in some way the test is viewed as flawed. We have to keep reminding ourselves that as well as a technical exercise, this is also an art form and what works for one person may not work for the next. As has been said the best way to see if a camera is right for you is to try it for yourself. All that those of us that do compare cameras can do is to try and provide an un biased opinion based on our own experiences” Very well said Alister.

Alister I do like your 350.bmp over the 700.bmp for the buildings but not the sky. That is similar to what I have been seeing the latitude in the 350 seems limited compared to the 800 but better then the EX1/3.

Tom agree the 350 is a great deal on a 2/3” HD camera. It is a nice step up from the other EX series and right in line. I guess I have just been spoiled by looking at Doug’s 800 which is why I started this thread.

I need a great outdoor setting for bright light and one for low light then another for 3.2K lighting range. And as we have discussed it needs to please me and I am not there yet. Tweaking Alister’s Natural is getting close to my liking but a little more color punch is what I am working on. All while keeping nice detail while holding that detail at 100kts in the helicopter with normal motion blur. Now that is a challenge for any camera. My scene on the ground on the tripod is a whole lot easier for the camera and I tend to make that sharper. So moving forward learning and more soon.

Thank you to Doug for his hard work in moving my 350 scene files forward.

[Doug Jensen]

Okay, for anyone who is interested, here's the results of the testing Paul and I did this afternoon. I will not make any comments or analysis. You can come to your own conclusions and take into account that our testing methods may not be the same as what you would do in the same situation.

We tried to adjust the exposre for each scene file to make it look it's best. If we had left the iris on auto, or locked it at the same f-stop for all of them, the results would be meaningless. Once again, we are trying to make each camera looks as good as we can.
All I can say is that it was damn cold, windy, and not everything is as perfectly exposed as I'd lilke it. But we did the best we could. It is what it is.
It's be nice to have some sun, but we can't control the weather and we were lucky it wasn't raining.

The scene files called "standard" and "no PP" are the camera's default settings out of the box. The two paint settings for the EX350 are listed below, but I do not care to post my F800 settings at this time. This is a 720P version of a 1080P video. I may decide to post a full-resolution version later.

PMW-350 vs. PDW-F800 vs. PMW-EX1 testing on Vimeo

-----------------------------
Alister's Natural (new version)
Black Gamma On, Level -30, Range High)
Master Black -3
Gamma On
Gamma select 4 4609
Gamma Category HG
Detail On
Level -12
H/V Ratio +15
Crispening 0
Frequency +30
White Limit +30
Black Limit +40
Aperture On
Level -20
Matrix On
Preset Matrix On
Preset select 2
User Matrix On, R-G 0, R-B +5, G-R -6, G-B +8, B-R -15, B-G -9

-----------------------------

Tom's Best:
Gamma ON
Gamma Step .45
Gamma Select 5 R709 Std
Black Gamma Off
Knee On
Knee point 84%
Knee slope +15
Knee sat On
Knee sat level 0
White Clip ON
Level 109%
Detail +1
H/V Ratio +35
Crisping +3
Level Depend ON
LD Level 0
Frequency +65
Knee Aperture Off
Limit 0
White Limit +75
Black Limit +75
V-BLK limit 0
V Detail Creation R+G
Aperture ON
Level -20
Multi Matrix ON
Saturation +5

-----------------------------

[Paul Cronin]

A couple of points I will make to help answer a few questions.

For the 350 all shots were with my ZA 17 x 7.6 BERM M58H which is sharper then the kit lens. As we said were are trying to make the cameras look their best with the gear we own.

Tom's best looks nice at times but I have never had such a hard time exposing a shot. This setting has positives but I would never use it due to how touchy it is to get proper exposure.

Have at it.

[Mike Marriage]

Thanks Paul and Doug, very generous of you guys!

Of course you can't critically evaluate footage off Vimeo but seeing as what most end viewers see is highly compressed I think it has value.

To me, all three cameras match pretty closely. Doug's Lizard settings look nicest to my eye on most shots but Lizard 2 doesn't really work with the skin tone at the end. I think you could set any of the 3 cameras up so that the end audience would not be aware of any difference at all when shooting in this environment. Obviously there are further variables such as DoF, lenses and codec to consider.

The 350 and 700/800 would come into their own in more challenging situations and the 700/800 would have further advantages in terms of its global shutter. I may be able to run an interior side-by-side tomorrow under very different conditions, depends on timing as it is a paid shoot. I probably won't have much time to tweak camera settings either.

[Steve Phillipps]

I'd love to see the Varicams (2700 and 3700) in the same test too, and maybe also an HDW790 as an industry workhorse and universally well-known comparison.
Steve

[Alister Chapman]

Thanks for doing the test guys and thanks for using the variety of settings, it was interesting to see the different looks.

There are some varying exposures, but I know how long these tests take, so that is not a criticism of you guys, but it makes making a judgement harder. All in all the camera performance appears remarkably similar. The EX1's noise appears to be causing a little more macro blocking in the vimeo clip and some of the reds look a little "EX".

I don't like Doug's Lizard settings (sorry Doug), too much sharpening with crushed blacks for me. Looks like you are making use of high contrast to boost apparent image sharpness. Lizard 2 seems to be causing moire and aliasing, especially on the truck sides, on some of the ropes and cables (Shot D, wire rope). Doug is this in the original material or is it a Vimeo thing? As I keep saying different looks will appeal to different people, this one's not for me.

In shot B the F800 standard appears to be crushing the blacks compared to the 350 standard, I see a lot more detail in the truck wheel arches with the 350.

In shot E I'm seeing aliasing from the standard F800 on the grill of the trailer chiller units (vimeo??) and I see no detail in the shadows under the trailers, while the 350 looks clean and you can clearly see the legs of the trailer. Are you sure this is F800 default? it looks to be crushing the blacks to me, adding extra contrast which may be triggering the aliasing.

My Natural setting certainly looks flat, but I think it could have been exposed a little lower, but I do like the way it retains detail well into the shadows of the truck wheel arches making it a good candidate for grading and post work, which is what I want. Adding in some more negative black Gamma would certainly make the blacks deeper, bringing the image closer to Toms best, but at the expense of grade ability. Doug - are you sure you had Black Gamma ON, it doesn't look like it. This looks more like my "to-be-graded" setting without Black Gamma.

If anything this test makes my dilemma worse! If anything I still prefer the 350 shots. The 350 seems to be retaining better shadow detail than the F800. I saw this behaviour when I used the pre-production 350 to shoot the airshow at Duxford. The aircraft showed much more detail against the bright sky and were less silhouetted compared to what we were used to seeing. Does the 350 have greater dynamic range? Whites and highlights appear similar but there looks to be more shadow detail, this suggest greater dynamic range.
What the test illustrates more than anything is how big a difference, different scene files make and given that the 350 and F800 have pretty much the same range of settings, if you choose either you should be able to roll your own look that you will be happy with.

Thanks for taking the time to do this Doug - Paul.

[Simon Wyndham]

When the ASC does their HD camera shootouts they assign an operator to each camera who's job it is to get the best possible look the camera is capable of

Doing that would be subjective.

A better way would be to set up each camera to match with scopes using a calibrated chart such as the DSC ChromaDuMonde. That way each camera would be replicating precise colours as accurately as possible and the real differences in performance can be seen.

I would also turn the detail circuits off completely in order to show the real detail and resolution being captured by the camera.

What struck me was how amazing the Panasonic 3700 looked out of the box. The default Sony setup really was very poor in comparison.

[Alister Chapman]

I agree Simon, setting up each camera using a ChromaDuMonde chart should level the playing field, but that takes a lot of time to do correctly.

Once again the camera that comes up smelling of roses is the EX1, amazing camera for the money.