Nielsen Ratings: Fact, Fiction or Wishful Thinking?

[James Emory]

The question I have is how could supposedly educated, successful corporate leaders continue to buy into this method of viewership measurement? The link below is detailed about how this system works, but the fact that the words estimated and theoretically are used so often when the decision to spend hundreds or millions on an airtime slot is at stake is troubling. Almost only counts in horseshoes and hand grenades. According to the info, around 5000 peoples' viewing habits are monitored to determine what the rest of us like. It is just aggravating and insulting to hear reports that 20 million people watched the Super Bowl when they truly don't know just how many did. They should say ALOT of people watched. But these marketing people just keep buying into it so I guess they deserve the 2.5 million dollar bill for their :30. How could you take someone seriously that even tried to give you a viewership number like that without literally visiting millions of homes for verification? Radio is the same situation. At least the print industry can say, we have this many subscribers, but that doesn't mean that all of them see your ad. Why don't they try to put a black box in as many homes as possible to get the most accurate data? Are they afraid of the truth? Is this not a racket? Give me a break!!!

www.nielsenmedia.com/whatratingsmean

www.nielsenmedia.com

[Keith Loh]

James, 5000 is a statistically large amount. Federal election polls are frequently around 1000 people.

//don't they try to put a black box in as many homes as possible to get the most accurate data? //

Cost and the probability that the accuracy won't go up. It's been awhile since I've taken a statistics course but I was convinced then that pools really don't have to be all that large.

That said, pollsters were totally taken aback when the results of last night's Canadian Federal election came in and took a swing at their credibility. But in that case, the pollsters are recording people's opinions which isn't the same as actually recording activity in the case of the Nielsen families.

Back to the Nielsen's. Broadcasters have long recognized that the Nielsen Family Ratings aren't the be all and end all. That is why they do a hell of a lot of testing including audience testing, focus groups and gathering other data like letters from fans, analyzing DVD and other merchandising sales, monitoring boards and taking polls of their own.

You should know that the Nielsen's and other similar viewer habit services are influential in more than just recording the mass numbers, they also record when audiences drop off. I attended a seminar where a TV show runner told us she knew exactly when her show was in trouble because the numbers began dropping off at a certain time during the show. So then she knew she had to rejig the format of it. So even if you don't trust the audience totals, the reporting at least can tell you patterns during the show.

[Rob Lohman]

Keith: 5000 may be a statistically large amount but that ofcourse
totally depends on which 5000 people they are using. In other
words, is the spread of types of people in real life the same in
the 5000 people group?

Anyways I'm outside the states so what do I know on this <g>

[Rob Belics]

Keith is correct. Interviews are done, I believe, with those selected for the ratings so they fit a profile but it is known that the accuracy of the system does not improve much with the number of people used for the survey.

[Barry Gribble]

James,

The math involved is rather complex, but the fact of the matter is that if you are trying to establish the average opions of one million people and you select 10 at random, you are going to get inaccurate results. However, if you select 1000 truly at random and get answers from them then you get pretty darned close to the average opinions of the million. And if you go to 10,000 instead, it doesn't actually get much closer.

You get problems, of course, because the subset of people willing to allow Nielson in to their house is not a perfectly representative set of the general viewing audience, but there are ways to normalize that out to reduce the error.

Fundamentally there is soooo much money at stake, that I have to believe that the networks trust Nielson's methodology and are not just giving them control of billions of dollars out of tradition. And the networks certainly know more than you or I about the accuracy.

I admit that it does seem strange tho....

I am guessing that the Nielson numbers are more useful in determining relative numbers than absolute ones.


There is a great entry on Gallup's website about sample size and polling methods that has some applicability here... you may want to check it out - it is a great overview of general polling:

http://www.gallup.com/help/FAQs/poll1.asp

also, a Wired article on Nielson:
http://www.wired.com/news/print/0,1294,63080,00.html

and an article about the recent uproar over a Nielson methodology change that some say is going to undercount minorities:
http://www.plastic.com/article.html;...06/05/14031275

[Rob Belics]

And what is the alternative? You can't poll everyone so that leaves guessing. Neither of which would be acceptable.

[James Emory]

Guessing is just what this ratings thing is. These folks are trusting that the statistical info is correct. They deserve to get financially screwed if they are that naive. I think it's safe to say that there are experts in every industry. They are often used in court to decide someone's guilt or innocence. The average person doesn't understand how DNA is processed for evidence but we trust these experts which they MAY very well be. This is just for argument and for no particular procedure, but what if they were wrong about a procedure and the guilty were released and innocent ones were held. Experts have been found to be wrong you know. I guess the question is this. If you were a potential advertiser, and a sales person said that there was a space available for $100,000 for :30 would you write them a check on, "well 10 million people saw last week's episode." I would send them right out the front door. Why don't they just go ahead and say 20 million people saw it and get $500,000 out of these suckers because the truth is NOONE knows the true number of viewers! It should be done accurately or not at all.

[Keith Loh]

James you wrote:
//It should be done accurately or not at all.//

And it has been already said in this thread that it would be a question of resources and statistically it doesn't improve the accuracy much. The alternative, guessing, as has already been said, is guessing.

You're babbling about 'experts' and 'DNA'. Stay on the topic.

In any case, we may be getting to toward the 'box in every home' idea when more and more people install set-top boxes, satellites, (pay-per-view obviously gives the exact data) which *can* track viewing patterns. And then, as now, you will have people wanting to opt out because they are not crazy about having some corporation know what they are viewing.

[James Emory]

I am not babbling, I am rambling! I only used DNA as an example of a complex test that not everyone understands but we trust as we're told by the experts, just like this viewer statistics racket.

As far as a black box recording viewing habits, didn't Tivo get in trouble a couple of years ago for recording its subscribers' viewing habits without telling them. They said something like they never assigned the stats to individual accounts or identities so noone's privacy was violated.

[Keith Loh]

DNA test methodology are open to scientific review. In the past year there have been more than a handful of articles publiched in New Scientist about DNA testing so you don't have to be an expert to be able to look back and see what the issues are.

We are both recalling the same TiVo story. That certainly is an issue, especially as not everyone views the same 'safe' programmes that others do. I certainly wouldn't want John Ashcroft knowing that my viewing patterns and it has been shown that the DoJ and other agencies has been able to get that information from corporations without much of a fight (I'm speaking of the airlines specifically).

Anyway, I don't see how your criticisms can be satisfied. If you don't like the methodology and there are privacy concerns with tracking all viewership that leaves you with notions and guesses.

Where exactly were you going with all of this? Were you pitching a show and some executive threw back negative data?

[James Emory]

Keith said "Where exactly were you going with all of this? Were you pitching a show and some executive threw back negative data?"

No, I just think it's a joke. Especially how millions of dollars are spent on what ifs. If I had proposed a plan that said I am going to tell you how many people watched your show by only monitoring 5000 out of a potential 15 million + viewers and then dare to put a million dollar price tag on it, I would have been laughed out of the room. I think it's probably the best of the worst estimated, maybe, guess method available.

Like I said earlier, if you were a potential advertiser, and a sales person said that there was a space available for $100,000 for :30 would you write them a check on, "well 10 million people saw last week's episode" and hope that they were right. I'd have to say, go visit those 10 million homes and then show me the reports with the signatures of each viewer next week and then I'll pay you. Ha, if they actually had to do that, then it just wouldn't get done! Why should they make any extra effort when it's too easy to estimate and/or guess and take these people's money.

They sure didn't ask me what I was watching!

[Rob Belics]

James,

These audience numbers are based on statistics, one of the mathematical sciences. This is nothing new and a proven method.

There are actual formulas created to determine these results and prove their accuracy. You can go to the library and find books on the topic.

Search the web and you'll find sites that explain how it all works. It's not a hidden topic and the method is easy to learn, at least if you're into that sort of thing.

[Michael Wisniewski]

Also, I'm sure the Nielsen ratings are just a part of the data that the advertisers use to figure out how much different time slots are worth.

A sample of 5000 homes could get you a pretty good guess as to what's going on if you cross reference it against all kinds of other viewer statistics.

For example, start with U.S. Census data, cross reference against historical sales and inventory data from companies, and then throw in the Nielsen data. That would give you a pretty good estimate.

I used to work with a statistical data company that would buy all kinds of databases from hundreds of public, private, and governmental institutions. They could probably figure out any trend within an acceptable range of error.

[Jeff Donald]

the last time I checked the Law of Large Numbers was still in effect.

[James Emory]

I don't think there's much more I can say without repeating myself again. One thing is for certain though. I'm not putting any of my millions on that ratings method anytime soon!!