View Full Version : Thoughts on Judging, Some Extra Commentary, and...A Surprise!


Meryem Ersoz
November 27th, 2007, 11:41 PM
First of all, let's take a little tour through the judging process. This was a great exercise, in many ways. Of 18 judge-able entries, (mine and DJ's were non-competes), 11 received at least one vote. That's a broad range of opinion and right in line with my experiences as the first screener when i have acted as a judge--in almost every single contest, i have passed half of the entries on to the primary judge, for final judging. So how you all chose was quite consistent with how I choose, which I was happy to see.

Also noteoworthy, as the votes rolled in, there was no consistency on the selection of the first place winner until the seventh set of votes rolled in, and then, from that point forward, some overlap kicked in. What this tells me is that we have very diverse opinions, much like our outside judges have displayed, and consensus is not easy to come by. Realizing the diversity of opinion gave me even more respect for the difficulty faced by an outside judge. One of the persistent critiques of outside judges has been, "There's no consistency!" Riiiight....we didn't have a lot either. There are so many factors that go into making a final call on these entries...it is actually a fairly complex process, as many of you discovered and clearly difficult to predict.

I gained a great deal of understanding about how you, the players think and feel about the contest and the type of films we are making here. I believe you, the players, gained more insight and respect for how difficult these choices are for a judge to make.

The primary downside that I saw to this form of judging: in my opinion, it really works when everyone votes, and not everyone did. I chased people around a bit and extended the deadline for a day, because I wanted 100% participation--it seemed the only way to make this a useful experiment, one from which we could all learn. One thing I learned is that by withholding votes, a player can actually positively influence their own positioning in the ranks....think about it, if they don't vote, but the other leaders do vote, then they have the advantage of receiving votes from their competitor while simultaneously withholding votes from that same competitor...that can create up to a six-point difference.

That seems most unfair to me of all, worse than any outside judging snafu, and is the one possible outcome that really makes me think twice about making this our regular form of judging. It's a very uncomfortable situation to mediate on my end, but, on the other hand, I don't want to be the vote police, that's no fun either. Blech.

I will be very interested to hear back from you players about how this process went for you and if you feel that this should be our new judging format. I reserve the right to make the final decision, based on your feedback, because the workload for it, whether it is finding outside judges and instructing them, or collating votes and chasing around reluctant player-judges, will ultimately be my responsibility. I must say, that, in this round, I was thrilled not to carry this responsiblity for the judging, to dump it on you guys and sit back and just count up the votes. But I do have mixed feelings about this process, which I have just shared. It's not perfect either. (although I kind of was hoping that it would be....)

I'm tentatively thinking about keeping the outside judging, but doing this once a year, at an unspecified time, when I feel lazy, as a kind of check and balance, for helping players and judges (or at least me, ahem) maintain empathy with each other. But I am very open to hearing your ideas on it, including fresh ideas for judging protocols that I may not have considered.

It's your turn to tell me how it worked for you.

And one last surprise: one of the coolest things to come out of this whole process was opening up the commentary. We received some very extensive and thoughtful comments, shared below. I wish we could do this with all the entries.

The most pleasant surprise of them all was our free t-shirt winner for best commentary: Marj Atkins!

I always knew there were many people out there silently watching our contest--the hits on our dedicated website alone, support the fact that's it's not only the players downloading the films-- and guess what, one of these lurkers sprung to life and wrote up 3 pages of comments (2 of which are included below) on the players' entries. Her comments were wonderfully worded and she gave a great deal of careful thought to the films she reviewed. Check it out, players, you have a fan base! How cool is that??

In Marj's own words: "I have been following your UWOL challenge since its inception with a great deal of interest and enjoyment and have learned so much from it."

So congratulations to Marj for de-lurking and for her comprehensive and thoughtful comments., Welcome her aboard.

T-shirt people, including Marj: send me your shipping info to: uwol@comcast.net--the shirt design is near completion. Once we fill those, we will then be taking orders for anyone else who would like official UWOL swag, once we have figured out the cost.

Meryem Ersoz
November 27th, 2007, 11:43 PM
whoops, too blathery, i have to break it into 2 separate posts:::

-------------------------------------------------------------------------------------

And now, finally, a cross-section of comments on each film:

"Isopod" by Markus Nord

Markus did a great job of editing Isopod, using nice tricks and technically
savvy transitions. But most of all, he went after a collection of images,
with an informative purpose. The lengths some people go to to get a shot
always amazes me. Almost freezing water, at night. Astounding.

He took us to a place that it’s
highly unlikely many of us will ever go.

The camera work was excellent with really good close ups of all kinds of marine creatures, & he was very informative about all those creatures in his voice over. The sound track he used complemented his work perfectly, never distracting, only adding to the pictures.

The introduction as a host on the beach was charming, and additional supporting
footage pre and post dive gave a real you are there effect. I really felt as
if I was diving along with Markus, and I think that is the sign of a great
film - a suspension of disbelief- so for at least a few moments you can be
taken somewhere else.

The shot of the shrimp showing the delicate looking transparent nature of that organism showed it in a way I had never seen.

Technicaly this was a masterpiece both under and over land. Some of the
underwater shoots was in the class of Discovery and National Geographic.

The soundtrack is a great mix between background sounds, music and narrating, in my opinion essential to wildlife films. you have a great way of handling the camera as well over land as under water, steady (as can be under water) and it all give a relaxing feeling of, “I’m right there with you in the depth of The Baltic Sea”. By participating/narrating in your own film it gives a more personal touch, which I like very much.

The winning place on my list is the night-dive movie “Isopod” by Markus Nord who has done a superb job of bringing to us a glimpse of an amazingly beautiful world that most of us will never get to see in our life-times – under water night life in the cold Baltic Sea and in particular a view of the large sea louse that is endemic only to that area and a few icy lakes of Scandinavia. Even more special is the fact that he managed to capture this nocturnal creature whose internal clock only brings it to that area at this time of the year – when the sea temperature drops below 10 deg. This movie is technically outstanding, particularly as specialized skills are required to do this type of video. The clear, colourful night shots done underwater must surely make any aspiring underwater videographer envious. Markus obviously has a thorough understanding of his camera, lighting and underwater gear, never mind an ability to go diving in freezing conditions! The video of the fish and plants and the transparent, fragile shrimp, looking to all intents and purposes like a delicate glass ornament, simply blew me away.
The movie has a clear structure with beginning middle and end .
I like the sound effects used to open his ‘UWOL TV show’ – a novel, catchy way to introduce an interesting and informative movie into which the night theme is appropriately woven. His use of a strong, well-positioned line as a transition to split then wipe in the next scene worked well here. The appropriately selected music and sound effects complement the excellent video work. The sound is clear. I especially liked the pinging of the sub. The fast-moving, stunning sunset was given added energy by the music and one got the feeling of time moving into place for the scene to unfold.
Markus has a good feel for composition. The movie “Isopod” is a well-structured, to-the-point short video in which Markus has managed to capture the depth, colour and textures of a normally black underwater world. Without his knowledge and ability to use light this movie would not have been possible – a wonderful achievement and an excellent contribution to the UWOL Challenge ‘Night” theme.

"Art of the Night" by Per Johan
He had every subject perfectly in focus. His sense of
composing the images is top notch. His choice of music
helps set the mood. Technically this is the best video in this round, and a true
artwork.

For sheer cinematography, this was amazing, and I was really swept up in the
drama of his filming choices. I especially loved the slow fade from the
reflection to the building, an ethereal effect where the film became
surreal, and for the few seconds of transition I shared in his exploration
of the ghostly mirror- he really evoked the spooky atmosphere of an urban
night. This same feeling was echoed later in the dogs running out of
darkness in the park, where for an instant there was a sense of
anticipation, to discover what was approaching. It was full of amazing
moments, my favorite being the meeting of the real and statue dogs in the
beautifully lit fog. The film captured so many amazing scenes, and embodied
the beauty and mystery of the night. The music helped enhance the feeling of
a slightly off kilter but beautiful night atmosphere, so in that way for me
all the pieces tied together well into supporting the theme.

“The Art of Night” by Per Johan Naesje is my first runner-up - unquestionably the most technically and artistically superior video entered this round. Per masterfully demonstrates how to find and video different subjects at night with different lighting conditions, incorporating scenes involving different types of lighting. Each scene presented has been carefully considered and constructed to give a flawless rendition of the subject and lighting. The result is a series of truly stunning and powerful images that are nothing less than works of art flowing together to form a breathtaking whole.
Per visits various sites in the city of Oslo and turns on the magic. He explores his subject from different angles and different viewpoints. Lights of the city, moving car lights, reflections in water, buildings and statues receive the same masterful treatment and after he has done, nothing but a highly polished result is achieved. Per has an incredible ability to capture his subjects skillfully, from razor sharp, images to hazy images softened by the fog.
The scenes flow easily, with a variation in tempo and mood. The music is slow, heavy and almost depressing though, and I can’t help thinking that Per did himself a disservice in the choice of music, even if it does suit the heavy, solid architecture and grotesque statues. The mood is emphasized by the colours in this video that are predominantly browns, blacks, greys and red.

"Get to Bed" by Cat Russell

My favorite film is Get to Bed, by Catherine Russell. Right
away, Geir's judging form went out the window. Technically, there was a lot
of grain and not great lighting on this film. To be honest, this was a bit
of a disappointment, since shooting in a barn, good lighting could have been
available. But, and a large but for me- I thought it really held together
and told a complete story. I felt the text kept the flow of the film moving,
and the story moved seamlessly from one shot to the next, showing a real
excellence in editing choices. The music also enhanced the mood of the
piece, so the audio was great in that sense. I liked the mixture of humor
and story, and was totally entertained by this piece.

This film did this for me - through the eyes of a farm girl talking to her farm yard friends in such a comical and innocent way. I would imagine Catherine would talk to her animals in this way in real life and she would probably treat these animals with the same love and respect she would give to a close friend. These values and this type of feeling came through in her movie - which was just beautiful for me.

The score
substantiated the film in a way that almost got my tears to appear in the corner
of my eye. The poem who took us through the film was nicely written and
performed.

This is a great story from a great storyteller and very entertaining. Minor issue is the grain, but as this is a night video, I will not let it down for that matter. A fiction from a human, drawn into the animal world, what a great story this would have been on the children’s TV. The grain gives it a touch of mysterious adventure and the text on screen is neat and easy to read. Editing skills are way over average and every clip is put together to a whole and complete story. Not a dull moment. Soundtrack is great too, and suits the story well. If you had let some animal sounds in it, I would have easily picked this as the winner. You even managed to let a wild animal into your little adventure – the skunk, just awesome.

Per Johan Naesje
November 28th, 2007, 02:53 AM
I will be very interested to hear back from you players about how this process went for you and if you feel that this should be our new judging format.
Meryem, my biggest concern about this method are that well known players (like myself) is going to get more votes than new players. I'm saying this because it can be real hard to be objective in the judging procedure! What do you others players think?
In the other hand, if we stay objective and ALL players joining the judging, I think this is the easiest way of running this Challenge. I don't know how large your network of contacts are Meryem, but finding a new judge for every second month for our challenge, will (at least for me) be a real challenge!

Your suggestion of finding "an outside judge" from time to time is great too. That way we don't know who's the judge before we almost finished our entries!

Mat Thompson
November 28th, 2007, 04:55 AM
I have to agree with Per on this point. If there was some way of the films being judges anonymously then I think community voting would be the way to go for sure. Unfortunately this is just not possible with our format, self narration, presentation and just obvious visual styles.

I do think throwing this in the pot once a year is an interesting process and yields lots and lots of meaty feedback which as you've proven really helps you 'as the filmaker' both appreciate the judging procedure and become a better balanced film maker by showing you what’s strong and what needs building on in your personally skill set.

Just to throw another possibility and I don't know how I feel about this completely but I thought I'd throw up the idea. Could we do both! The community decide on the top5 for instance and a judge finishes the process........just a thought, as I say I'm not quite sure how I feel about that and it could be even more work I guess.

Bryce Comer
November 28th, 2007, 06:01 AM
I found the judging process very thought provoking. It made me think of things that i probably otherwise would not have thought about, & i think i learnt a lot about my own filmaking because of it. I am not apposed at all to the idea of we as a community having to judge eachothers work, as long as i am providing my humble opinion, & am not totally off the mark with my feedback. I was very happy to see that for this round i was not far off the mark, since i had picked 2 of the top 3 that all the others had picked, including the 1st place getter.
Matt, i like your last idea that the community picks the top 5. This may be an idea that could work.
Per, although i think you may well be right, i'm not sure we got it wrong with this round of judging, like others have commented, 3 deserved winners indeed.

Bryce.

Meryem Ersoz
November 28th, 2007, 08:08 AM
Meryem, my biggest concern about this method are that well known players (like myself) is going to get more votes than new players. I'm saying this because it can be real hard to be objective in the judging procedure!

That's why I instituted outside judging in the first place, because, from my experience with DVC, players develop reputations very quickly. DVC has never had the same winner twice, I don't think, but I was trying to introduce total objectivity to the process.

Player picks combined with outside judges essentially doubles my workload around the judging, first finding judges and then chasing around the votes, so I'm a little reluctant to put that in play.

I definitely think you all did a very thoughtful job of picking winners. I have no complaints whatsoever about the outcomes. That part was great. My 2 main reservations are the first one, voiced by Per, the issue of reputation, and the 2nd one of how players can, quite inadvertantly I think, manipulate the outcome just by abstaining from the voting (this only comes into play if a top contender does not vote, but if you are all called upon to do this every contest, voting complacency could set in quickly...).

I'm getting some good ideas from your feedback, though, and formulating some new thoughts, so keep it coming....

Benjamin Durin
November 28th, 2007, 09:05 AM
T-shirt people, including Marj: send me your shipping info

But the t-shirt people who are they? Except for Marj, did you name them? Who can win a t-shirt exactly? The best comment, ok you said it. I guess the winner. Are there others? Why do I ask so many questions? Someone will stop me?

Geir Inge
November 28th, 2007, 09:10 AM
There may be different opinions about the judging system, or how to judge a film. Just take a look in the news papers :)
I think, the critical reviews from so called pro's, are mostly influenced by their personal feeling and taste, and not least what's "the trend right now".
But most of them follow a set of rules, similar to the ones we follow when we're making the films, and it goes in the terms of the same guidlines Meryem sent us by mail for this competition. Some critical reviews are good, some are not so good. In this community we have to make sure everyone participate in the judging prosess too, so maybe we should look at our rules and see if we can do some changes? If "we" are to be the judge, maybe we should make it a compulsory part of the game?

I've been going through the films, started from uwol#1, and there are many good films but also many different approaches to the theme so it's not easy to be a judge. I think Mat's suggestion about trowing in a top 5 may do the trick, but some how everyone do have to participate in the judging prosess and then we have to look at the rules.
It's not everyone who takes part in the feedback tread either, or maybe just trow in a "nice film comment". This is a learning prosess too and I think most of us want to be judged by the last film we made and not our previous work.

Last; For me, taking part in the judging prosess has been the most challenging part of uwol, so far. I did a research on the internet, on different movie sites, and read a lot about how to do a film analysis. Just to know what it's all about and I must say I feel much more comfortable behind the camera :)

Geir Inge

Brian McKay
November 28th, 2007, 09:27 AM
" No Judge No entry" if you enter a film but don't take the time to judge the others and pick your top three your entry is disqualified.

However if there is someone out there that would still use the "vote for the worst three" to enhance there chance of winning.....I don't what to play with him....I think it would become obvious in one hell of a hurry. Damn I hope were above that crap.

Brian

Meryem Ersoz
November 28th, 2007, 09:36 AM
" No Judge No entry" if you enter a film but don't take the time to judge the others and pick your top three your entry is disqualified.

However if there is someone out there that would still use the "vote for the worst three" to enhance there chance of winning.....I don't what to play with him....I think it would become obvious in one hell of a hurry. Damn I hope were above that crap.

Brian

No, no, no, no! I don't want to mislead anybody...nobody did anything that Machiavellian or deliberate, and I don't think anyone would, not in our little UWOL love-fest...in fact, quite the opposite, and the issue was pretty readily resolved. It was quite by accident...but it did point to a big hole in what I was hoping would be our perfect solution to judging. But maybe "no judge, no entry" is a good addition to the rules of self-judging. That's the kind of stuff I'm trying to worm out of you...to help me plug the holes in my blind spots.

And, oh, Benjamin, to answer your question, "the t-shirt people" are the people who finished all six of this year's Challenges--at the beginning of the year, I promised a freebie to everyone who managed to finish the full year cycle. That was a pretty good sub-Challenge. I'm thinking about doing it again next year....

Brian McKay
November 28th, 2007, 09:46 AM
I completely understood...that's why I broke it into seperate statements. The first being a possible solution to someone not taking the time to judge....no hidden agenda.

The second was to simply point out that if someone wants to be an ass there is no way to stop it.

Like you I don't believe it could ever happen with this group.

Brian

Trond Saetre
November 28th, 2007, 10:30 AM
Meryem, my biggest concern about this method are that well known players (like myself) is going to get more votes than new players. I'm saying this because it can be real hard to be objective in the judging procedure!
This can be a concern. But I believe that from the good friendship we have developed here, I don't think it will be a problem.

What I hope will "change", is that all, instead of most of, the players (at least those who submit an entry) will give feedback to eachother. Most do that already, but it would be even better if everyone did it.
So I agree with Geir Inge, maybe we should look into the rules and make this part mandatory?

Personally I think it could be a good idea combine us, the players, judging once in a while, with an external judge who doesn't know us or our reputation.

To me, the judging was a much more difficult than actually coming up with a video. Watching the videos over and over again, trying to find the 3 entries that really stood out and gave me something extra. But I liked doing it.

Mat's suggestion about having us to pick the top5 is an idea I think we could try sometime, and see how it goes. But I understand if you Meryem don't want this because of the increased workload on your end.


Trond

Chris Barcellos
November 28th, 2007, 10:53 AM
In this vote Meryem gave arbitrary point values to each judges evaluations. I don't have a concern about any of this...., but since we are in a discussion, I think we ought to consider this.

As it stood, in the voting, if I voted for one film as number one, the point values assigned to number two said that I thought it was only 2/3 of the number one film. And my #3 pick was only 1/3 of the number 1 film. This is not what I may have thought, but it is what was reflected in the voting.

So my thought is we should be doing just what is done in the DV Challenge, ie., everyone pick three top films, each getting one point for a vote, and the one with the most votes wins. This seems to be a fairer reflection of the consensus of the voting participants, because in my mind, any of six films was good enough to warrant consideration.

Markus Nord
November 28th, 2007, 11:04 AM
One idea could be that we had a silent 3-4 days with no comment on any video. During that time we have to send in a top three. After that the forum is open for comment and after 3-4 days the winner is announced. The role of “No Judge - No Entry” would work in this way too.
In this way no one would me “misled” upon what other are saying about a specific films.

One way of taking the work of Myrem could be that we hade a formulary on the UWOL website, connected to a program that would recount our judging and fill out the total top three. To keep this safe we would need a password (that we would be given after uploading) to fill in the formulary with our top three and comment.

Markus

Jim Montgomery
November 28th, 2007, 11:23 AM
Just my two cents.

It is my opinion that the original intent of this contest was to further my skills as a movie maker. By having a theme we would all be focused in one direction and the end result would be the our interpertation of that direction and theme.

By selecting a "winner" we have defeated the purpose of the challenge. We all seemed to be focused on winning, judging, etc. and not the original intention, to improve our craft.

Lets do without winners and return to focus on how to become a better film maker. This can be accomplished just by using the feed back threads that are generated.

For what it is worth

Jim

Kevin Railsback
November 28th, 2007, 11:26 AM
Judging is a bear because it's so subjective.

One person may look for the story and if it's not there pick a film that may not be as visually exciting because it has a story.

Someone else may pick a film because it's visually exciting but has no story.

Even with a spreadsheet that says give X number of points for this, this and this, it's still going to come down to what film moves you the most. At least for me it would.

In all the entries I've watched,since UWOL1, my top pics range between visual and story. But all were films that made me go Whoa!

I think there is no perfect system.

What if we just picked the top film and that's it? Everyone that enters and votes gets one point for their own film. You can't vote for your own film.
The top three films are the ones with the most points. No Vote, no entry.

Trond Saetre
November 28th, 2007, 11:27 AM
Jim, I understand very well your point here. I agree that the purpose of the challenge is to have fun, to get out in the nature, and of course as you say to improve our skills as video makers.

Maybe the judging has become a too important part of this challenge? But the judging/having a winner, can (and probably will) also give many of us a little extra encourage to do our very best.
Still, the feedback is most important in my opinion too.

Kevin Railsback
November 28th, 2007, 11:38 AM
Like I always have said, judging is subjective. That is why I never get upset if my film doesn't make it in the UWOl Challenge, or doesn't make it in a film festival or whatever.

But, it's always nice to recognized. I like to win, who doesn't? But I don't cry in my cereal if I think my film was better and everyone likes something different.

I can tell you when I'm sitting here editing and I'm not getting what I want or think I can, it's nice to look at my awards and know that someone else appreciated my work.

For me, the feedback on what works and what doesn't is more important to me. Also, seeing through someone else's eyes opens my vision to see things in a new way as well.

That's why I like the common theme so much. I saw so many different ways to tackle a single subject.

And above all that, I made some great friends that I never would have met otherwise.

So, I think the judging is important cause it's always good to get an atta boy ( or girl ) but I think we're all here to become better filmmakers.

Trond Saetre
November 28th, 2007, 11:42 AM
And above all that, I made some great friends that I never would have met otherwise.
VERY true!
I have never met such awesome friendly people on any forums as I have here on dvinfo and the uwol family! It's amazing and remarkably.
I have learned a lot from all of you!

Per Johan Naesje
November 28th, 2007, 12:17 PM
Lets do without winners and return to focus on how to become a better film maker. This can be accomplished just by using the feed back threads that are generated.
We already have that possibility in the UWOL forum, where everybody can put up a film for others to view.
I think it's important, that others also have stated, that this is a challenge with a winner at the end. This force us (at least for me) to take out our best for making a real winner entry.
The question will be to make this as fair as possible to all participants if we gonna do the judging ourself.

Ruth Happel
November 28th, 2007, 12:23 PM
Well, it looks like our opinions on the judging are as diverse as the judging itself. If we want to continue judging, I think it would probably be best as a couple have said here to have a rule that requires judging, to keep things fair. That never occurred to me how not judging would influence the outcome. I don't think our group would do this, given the warm and supportive community that has evolved. But planning for the future, if it really takes off into something large, there should be some rules to avoid that.

I would also like if there was some way to have a mandatory feedback for all films. As part of this challenge, I provided feedback on every film, every challenge. It did take some time, but it seems an important part of the process to look at and comment on each film, since the main purpose for me in participating is getting suggestions, good and bad. I learn not only from feedback on my films, but from suggestions about other films- even after I comment, I go back to all the feedback threads to see what advice others have for each film. I learn from not only my own mistakes, but the mistakes and successes of others.

One thing that occurs to me, and I am not really sure if it would be popular, but just a thought since we are throwing out ideas, is to have a slightly greater focus on how to improve. I am as guilty of this as anyone- when I give feedback, I tend to focus only on the positive. But if we had to say, for example, the three things that are best- and worst- about each film, even though some of that might sting, I think that is where we could really improve. That might be too much effort for the individual feedback threads, but at least for the top films we choose to judge, it would be valuable suggestions for the winners.

As far as the debate about judging or not, it doesn't matter much to me. I am here to learn from feedback about the films I make, and the films the other participants produce. I am happy either way, the main thing I appreciate is having a positive and supportive community of like minded wildlife filmmakers.

Meryem, since you already work so hard at this, I would say go with whatever lightens your workload! This has evolved into a great community in such a short time, it really has helped me focus on new techniques and ideas for making videos. I have learned way more than I expected when I first signed up. And kept to the last rule- have fun!


Ruth

Chris Barcellos
November 28th, 2007, 12:33 PM
As a six time contributor, the fact that I have not won creates no issues with me. It actually is an incentive for me to improve my skills. Each of the competitions has taught me a lesson. The competition aspect spurs me to make a film, and the possibility that I might be recognized in a particular competition spurs me on to make the best film I can given the circumstances, the moment and the theme. I also enjoy being able to "commune" with those more accomplished than me, with the hope of learning from them, and getting to their level some day. Please don't go away UWOL and Meryem !!

I am curious, though, if there would be any difference in the voting if each top three voted for was assigned one point, but I would not want create a controversy by learning that. From that standpoint, I just wonder whether in the future if it would be proper to actually publish the ultimate points given. With everyone having knowledge of the actual scoring, it might help us give better input regarding the way scoring should be accomplished.

Catherine Russell
November 28th, 2007, 12:33 PM
Hi everyone:

First, I would like to say that after reading the "cross-cuts" of judging commentaries, I am astounded at the professionalism and integrity of the UWOL community. The critical thinking, the thoughtfulness, the careful explanation of why the choice was made ... it made me realize that this is an exceptional group of people with a shared passion for creative film and all aspects of it: the production, the sharing and the discovering and lifting up of each other's talent and inspiration. The comments justifiably covered in detail all the merits of Markus' and Per's films but also justifiably covered the weaknesses of my film and defended why other strengths could still lift it up and salvage it. I had to laugh in spite of myself at the ... "this film is actually a disappointment, BUT..." This is a testimony to sound critical judging that goes far beyond the sentiment that the person with the best camera, highest technical ability and the greatest reputation will always inevitably win.

That being said, my input on the judging would be this:

Like Olympic ice skating, it is impossible to get around the fact that personal tastes and preferences will always play a major role in the voting process. How could it not? So I think the fairest outcome in judging should be a culmination of both Mat and Brian's suggestions:

1) The UWOL community takes on the responsibility of judging the top 5 (or whatever) films. Since we are participating with the intention of improving our filming skills, should we not also pursue our critical judging skills as well? I think we have shown that we are up to that task. I don't think there is a person in the group that would vote with a strategy that would better their chances at winning.

2) The people that skip out of this responsibility of judging forfeit their films of being judged. Hopefully this will help Meryem's work load a bit and keep the mathematical probability statistics fair. Perhaps special consideration should be given those that end up with an unexpected emergency, such as what happened to Bruce and his family this round.

3) An outside judge picks the top 3 out of the 5 (or whatever) films. This way, this helps even the score where Per was concerned that reputation gets unintentional biases over newcomers.

I think this way, we cover most of the fairness issues in a very subjective process and allows the UWOL community to also have a say in what they feel are winning films.

As a final insight as judges, deep critical analysis of the films as well as first impressions may have equal merit so that if you are pressed for time a certain round, long lengthy analysis may not get you any farther than a simple selection of the top films. A case in point is my experience this round. When it was time to judge the films, I fell ill and was unable to go through in depth each film several times with critical analysis. I emailed Meryem and told her I was taking myself out of the judging because there was no way I could be fair in my choices, having gone through everyone's film only once or so before getting sick. She responded that she was looking for 100% participation and asked me to vote without commentary. I did so off of overall impression and I too picked the ultimate top two finishers chosen by the group. I think when all is said and done, outstanding films tend to rise to the top that most feel good about them claiming that status. That being said, the fact that there is quite a bit of variability in winning film choices shows how close we all are to winning in a particular round. All of our films bring to the table a wealth of talent and inspiration unique to our own filming styles and we are all winners. It's like the slalom ski races... winners are decided by 100ths of a second after a long downhill run! Picking winners is important, but the best part of the forum to me is the comment threads on our films. It would not be worth making these films vying only for a potential a win with no comment forum. It's what makes it so worthwhile... and of course the friends we have all become through it.

Cat

Meryem Ersoz
November 28th, 2007, 12:39 PM
As it stood, in the voting, if I voted for one film as number one, the point values assigned to number two said that I thought it was only 2/3 of the number one film. And my #3 pick was only 1/3 of the number 1 film. This is not what I may have thought, but it is what was reflected in the voting.

So my thought is we should be doing just what is done in the DV Challenge, ie., everyone pick three top films, each getting one point for a vote, and the one with the most votes wins. This seems to be a fairer reflection of the consensus of the voting participants, because in my mind, any of six films was good enough to warrant consideration.

we can that in future self-judging rounds...i'm not that picky about how points are assigned--actually, having voted in several of these DVC rounds, i didn't even realize that's how it's done...i submitted my picks in a 1,2,3 hierarchy and assumed that mattered. i don't see why it would have to, though.

like i said, i'm trying to plug holes in my blind spots.

this is and always will be a contest, though, that won't change. to me, the most important part of it is the feedback threads--to tell you the truth, i had to learn how important judging is to *you*, the players, because i never signed up, personally, for a DV Challenge, with an eye to win it...i always signed up with the idea that i was going to incorporate one new thing in every video, try something i had never tried before. in my dog video, it was making a ghost image, in my zombie film, it was working with an actual cast instead of being a lone wolf shooter, in my burning memories video, i experimented with a cineframe feature that i had never tried before...that sort of thing. winning has never been a motive, camera and editing experiments are, so i had to actually wrap my brain around this concept. it changed my perspective a lot.

that's probably more about my process than anyone needs to know, really, but it's why i initiate these discussions, to get in your heads better....

Chris Barcellos
November 28th, 2007, 12:57 PM
Hi everyone:

A case in point is my experience this round. When it was time to judge the films, I fell ill and was unable to go through in depth each film several times with critical analysis.

Cat

I was in a similar situation, going through surgery on Wednesday, but by Saturday or Sunday, though in a Vicodyn induced haze, I did manage to send my vote. I was the one who included Meryem's film in my voting, and I was contacted by Meryem, and properly chastized, and I added another film to my vote....

So this raises an issue-- why can't Meryem's films be included in the judging ? Just because she has the administrative responsibilities in this challenges, why can't she also participate in the competition, if she desires to do so ?

As far as my illness, and dealing with judging, in my case, I reviewed all films, pulled the eight or so that made the biggest impression to a separate folder, and rewatched each, more than once, finally arriving at an order, using both productions values and story as my primary guide.

In the end, I tend to look at the films and ask myself which production is closer to what would be acceptable as airable for TV or film production, while staying on theme.

Kevin Railsback
November 28th, 2007, 01:02 PM
The reason why Meryem can't enter and Mat and I can is that Meryem is the one that picks the themes.
Mat and I find out the same time everyone else does. Believe me, she takes a huge amount of enjoyment in torturing Mat and I behind the scenes.

Catherine Russell
November 28th, 2007, 01:05 PM
Hi all:

Chris, at first I agreed with your sentiment that a third place finish with 1 point implies that it is 1/3 as good as the first place finish, which may not be your intention. But I think you have to have different points representing each place or you could come out with bad results. For instance, what if someone got 10 votes for 3rd place and another 8 votes for first place. If each place had equal points associated with it, the 3rd place votes of 10 would win over the 8 votes for 1st place... instead of the correct outcome of a really strong 1st place winner and a really strong 3rd place winner.

Also, I don't think it adds to anyone's knowledge or benefit to know how many points people earned in a particular round. I think a first, second or third finish is information enough.

And by the way Chris... I'm so thankful that you are not bothered by not winning yet, because I love some of your techniques (I find them very powerful) and I have started to use them! I am learning from you!

Ruth: I too look for comment from everyone in the thread and am a little stung when someone's input doesn't show up. I can't help but think... was it that bad to not even get any input? However, I think it's up to the inspiration of the person to choose to write or not write in a thread. I too make certain I comment on everyone's films. But I think it would be sort of legalistic to require people to comment if they don't feel like it, do you think?

Cat

Chris Barcellos
November 28th, 2007, 01:19 PM
Cat: Thank you for the nice compliment about learning from something I did..

As far as commenting on each entry, I try to make comment on each film too, but I failed this time, due to my health situation..... In other Challenges, I am sure I have dropped the ball, too. And when I see a lot of comments similar to what I might say, I tend not to want to pile on, so may elect to say nothing.

I have thought about just starting a thread like "Chris Barcellos Comments on UWOL6 films", to make it easier on me, and maybe make it easier for those not interested in my comments to ignore them...

Meryem Ersoz
November 28th, 2007, 01:21 PM
why can't Meryem's films be included in the judging...

because she knows the themes before anyone else, that's why, and doesn't want to be accused of any more improprities than she already has been!....also, you weren't the only one i had to properly chastise, chris, you were only the one i derived the most pleasure from properly chastising...heh.

i don't think making a requirement to comment is necessarily a good thing...it's something we should do from good will and the desire to learn. i mean, that's the spirit we most want to keep alive...

Trond Saetre
November 28th, 2007, 01:27 PM
good will and the desire to learn. i mean, that's the spirit we most want to keep alive...
The most important thing about uwol!!!

Per Johan Naesje
November 28th, 2007, 01:30 PM
Ruth: I too look for comment from everyone in the thread and am a little stung when someone's input doesn't show up. I can't help but think... was it that bad to not even get any input? However, I think it's up to the inspiration of the person to choose to write or not write in a thread. I too make certain I comment on everyone's films. But I think it would be sort of legalistic to require people to comment if they don't feel like it, do you think?

Cat
I have been commenting everyone else's entry so far. But if this community and number of entries grows, I don't think any of us are able to comment all entries!
I think it should be up the every single participants to make comments or not.

Meryem Ersoz
November 28th, 2007, 01:32 PM
these entries are coming so fast and furiously, i can't keep up....

but yes, we do hope to keep growing the contest...so i think it will be inevitable that keeping up with every single film might get harder....per makes a good point.

something else to think about!

everyone online together, all around the world!

Carl Middleton
November 28th, 2007, 01:54 PM
I'm planning on getting involved next year - I wanted to watch one first before jumping in the middle, see how things worked!

So that's one more entry to deal with. I'm sure there will be plenty of others to get involved after, as there have been plenty before me. :)

Carl

Trond Saetre
November 28th, 2007, 01:58 PM
Carl, welcome to the uwol family!
Always nice when new people join us.

Ruth Happel
November 28th, 2007, 03:08 PM
Yes, Cat and Per, I agree that feedback shouldn't be required. But I think maybe there isn't enough emphasis on it- the contest has evolved to a greater focus on judging than feedback. To me, the feedback is way more valuable than winning or losing. So given that I always comment on all the films, but generally get feedback from less than half the participants, I feel maybe we need to do something to emphasize feedback. Meryem, your comments on your participation the dvinfo contest mirror my own feelings about UWOL- I use these contests to challenge myself and learn new techniques. And the winning or losing just isn't a factor for me- but as you expressed, it seems to be a more important aspect of UWOL. I appreciate you giving a t-shirt for feedback, that was a great way to emphasize it! There aren't any easy answers, but I would at least like to explore ways to formalize and encourage feedback somehow.

Ruth

Meryem Ersoz
November 28th, 2007, 03:52 PM
the contest has evolved to a greater focus on judging than feedback.

i think we do a pretty good job of the feedback, considering how many entries we put up. it is a lot of work. i appreciate all that the players do, in this regard. we all have jobs and families.

but feedback and judging, both, are only parts of a holistic process that also includes watching each others' work and making our own internal notes about how we can improve our own. there's really no where on on the web, possible in the world, which serves its target the way this contest does.

i don't want to get too distracted in this discussion--the reason i raised it is that this part of the process--the judging--is the single most criticized part of the contest, and my intent is to improve upon it. i think the overall experience is a solid one...actually, it is so solid that the judging, by comparison, seemed like it had a few holes to plug.

Benjamin Durin
November 28th, 2007, 08:21 PM
I too look for comment from everyone in the thread and am a little stung when someone's input doesn't show up. I can't help but think... was it that bad to not even get any input? However, I think it's up to the inspiration of the person to choose to write or not write in a thread. I too make certain I comment on everyone's films. But I think it would be sort of legalistic to require people to comment if they don't feel like it, do you think?

Cat, I don't write feedback for each entry. There are several reasons for that:
1) If one or more persons already wrote what I wanted to input then I don't want to repeat what they said.
2) I can't help but emphasize what I didn't like in a movie. Even for one I loved, I may write 20% positive feedback and 80% negative feedback. It often sounds harsher than I would like and I don't want to let other people think I don't like any movies, because it is not true.
3) Time. It takes time writing feedback. I already had to squeeze time to make this entry, it's difficult taking more time for feedback and judging. But I do my best I swear.

I also don't expect everyone to write feedback for my entry for the same reasons I cited above. Don't get me wrong, I love to have feedback and I am grateful to those who wrote one for my entry.


Meryem, for the judging, keep it simple (for you). I only had one out of the three in my list but I don't think it's a big deal. There are different tastes in there and it is fine.
Oh, I see here the point to have a different judge each time. If we vote ourselves each time, it will be more or less the same taste dominating. But with an external judge, it is a different taste each time so more variety in the judging. Does it make sense? Maybe alternating with an external judge every other challenge would be a good idea?

Per Johan Naesje
November 29th, 2007, 12:56 AM
Oh, I see here the point to have a different judge each time. If we vote ourselves each time, it will be more or less the same taste dominating. But with an external judge, it is a different taste each time so more variety in the judging. Does it make sense? Maybe alternating with an external judge every other challenge would be a good idea?
That's what I think too, altenate every second time will be nice. Meryem, what do you think about this for next year judging?

Trond Saetre
November 29th, 2007, 01:55 AM
This sounds like a good idea to me. Or maybe using an external judge 4 of 6 times a year if that reduce the workload for Meryem?

Meryem Ersoz
November 29th, 2007, 07:16 AM
it's kind of along the lines of what i was thinking, after listening to everybody's ideas...a mix of outside judging with self-judging. after trying both ways, i think both methods of judging have advantages and disadvantages--neither is perfect. so mixing it up might be a good way to keep things fresh.

Meryem Ersoz
November 29th, 2007, 07:41 AM
and one last word regarding feedback--i didn't mean to put you off, ruth, regarding your feedback about feedback--it's just that it's another issue which has a broad diversity of opinion. some of our players feel we should be more more hard-nosed in our criticism, that our challenge has too much "happy talk" and not enough hard-line critique, and other people see feedback as a community-building experience, as a social experience, as a place for finding support for what they do.

some people have thicker skins than other people, and some folks have thinner skins about receiving criticism. sometimes a friendly wave of the hand is all that is left to say, as benjamin has pointed out....sometimes people post entries just to play along, to be part of it, not because they even want feedback--i certainly do that. sometimes, if your name is kevin, you make something really pretty, post it, and then try to hide it under a rock. et cetera.

we need to respect the diversity of players' motives for playing. for some people, it's feedback, for some people it's community, for some people, it's as simple as HAVE FUN. right, trond? i think feedback is multi-purposed. "feedback" may even be a misnomer, actually....

everyone has a little bit of a different emphasis on why they sign up, so it is something that just has to play out in the relational field, rather than in the field of rules, in my opinion.

now that i've re-opened that can o' worms myself!....

Ruth Happel
November 29th, 2007, 09:33 AM
Meryem,

Okay, it seems there is such a diverse community, any form of feedback will be equally varied. And though comments are important to me, not so much to everyone else, necessarily. I like the concensus view so far of switching between self judging and outside judges. In the rounds where we do the judging, I'll note my most and least favorite aspects of the winning films, since that should be helpful feedback to the winners, anyway. Thanks for opening up this dialogue, it was helpful to discuss judging.

Ruth

Geir Inge
November 29th, 2007, 10:26 AM
I have trown in my point of view in this judging matter, and it's great to see we all are dedicated to the question. I just want to quote one of my "old" friends:
"You can't stay in your corner of the forest, waiting for others to come to you; you have to go to them sometimes".
Winnie the Pooh



Geir Inge

Catherine Russell
November 29th, 2007, 07:29 PM
Hey Geir:

It puts a smile on my face imagining Winnie the Pooh speaking in Norwegian ;-).

Cato

Catherine Russell
November 29th, 2007, 07:41 PM
And Benjamin.... We should be joined at the hip on these feedback threads... I'm just about the opposite than you are in that I think I'm more 80% positive and maybe 20% negative in my comments. So between the two of us, some constructive truths might be gleaned ;-). You and Chris have pointed out lots of good reasons why you may not comment on every thread... I won't worry about it next time!

Dale Guthormsen
November 30th, 2007, 08:14 PM
Well,

I finally have a moment to get on the computer again and find this amazing thread.

So diverse meryem's head must be spinning. After reading them all I almost do not want to add to the fodder.

I have participated as a means to force myself to do more than just shoot wildlife footage.

I found I have learned more from all than the participants and the threads than all of the videos and shooting I have done.

In this regard I feel that I am a huge winner.

I think a few of you i should send coaching fees to!!

I do think picking a winner is of value. I picked my personal top three in all the challenges and had picked the winner among them 5 of 6 times.

I will bet others did the same.

In this regard the dv challenge method would seem to have merit .

I do like the idea of an outsider every other challenge.


In the end, I think whatever meryem decides to go with will be just fine with me.

And i will even title them correctly next year!!!!

I bet your Pesos that meryem never knew just how much she took on when she bit off this job!!!!!

Catherine Russell
November 30th, 2007, 09:57 PM
Hi Dale:

Always glad to hear what you have to say. Thanks for adding to the fodder. I'm sure Meryem likes the input from everyone.

Here's to film-making in 2008.

Your friend,

Cat

Trond Saetre
December 1st, 2007, 12:02 AM
I found I have learned more from all than the participants and the threads than all of the videos and shooting I have done.

In this regard I feel that I am a huge winner.

Same here, I have learned so much from everyone.
Dale, you, me and every other member of the uwol family, we are all winners!
Every single one of us have improved our video skills during this year.

A huge THANK YOU to Meryem, Mat and Kevin for all the work you do to run the coolest challenge on the planet!


your friend
Trond

Meryem Ersoz
December 1st, 2007, 10:38 AM
I bet your Pesos that meryem never knew just how much she took on when she bit off this job!!!!!

you got that right, dale. of course, it is largely my own fault. i keep tinkering with the darn thing and coming up with new moving parts. but hey, it keeps sucking in new players from all corners of the world, and that could not be more exciting to me.

from a personal standpoint, I am especially happy to see Turkey represented. my Turkish dad, if he were still alive, would be pretty psyched to see Turkey in the game....I was raised, partly, by my Turkish ane, so it is awesome to me that the energy of the game sucked in a turk. baklava all around!

Marj Atkins
December 1st, 2007, 12:17 PM
Well you certainly got me in Meryem - and as I'm in, I might as well add my bit to the pot.
If we accept the fact that this is a challenge and therefore it needs to be judged, then it would probably be a good idea to make sure that everyone is playing by the same rules - players and judge/s included to make it as objective as possible.
A good starting point would be some to lay out the judging parameters (Meryem’s for example) as well as the theme and mission - then everyone would know from the start what is expected of them.
I, personally, would also appreciate knowing up front when the theme is announced, where the judge’s emphasis is going to be when judging (eg on sound, light, story, technique) rather than try to second-guess this, because it would force me into thinking about and experimenting with another angle or skill that I would normally not consider.
I have really valued the comments made by the judges this year and appreciate the time that it took them to do that.