Reading Polls

Reading Polls

A Guide

There have been some recent murmurs in the press about how Senator Coleman is in a weak position right now because he’s polling “within the margin of error” in the latest polls against his likely Democrat opponent. Here’s my source material for this post but the actual numbers are not important. This is a lesson in how to read polls, not a commentary on recent ones.

According to Survey USA Norm Coleman is leading in the polls over Al Franken 48% to 44%. The “margin of error” for this particular poll is 4%. Does this mean the race is statistically tied? We hear that a lot whenever a race in within the margin of error. The problem I would pose to the normal person reporting on these polls is simply “what do you mean by “margin of error”?

They probably won’t know. The “margin of error” in these polls is actually an expression of the 2SD Rule (two standard deviations rule). A “margin of error” of 4% really means “We are 95% confident that there is a 95% probability that the ‘real’ level of support is within 4 percentage points of our answer.” If you remember your normal distributions, 95% of any population (if the population is normally distributed of course) is within two standard deviations up and down from the mean.

So really, a poll simply reports two overlapping probability distributions.

What’s this mean? This means a poll provides us all the information we need to find the probability one candidates’ real support is greater than that of his opponent. In our example, Coleman has a mean support of 48% which means there’s a 95% chance his actual support is somewhere between 44% and 52%. Franken sits at 44% which means his support is most likely somewhere between 40% and 48%. In this case the standard deviation is 2% (half of 4%)

To find out what the odds are Coleman will win the race (or, more accurately, that Coleman has a level of support higher than Al Franken’s) we simply need to find out what percentage of the two bell curves overlap.

Here’s what we need to do using a simple equation and the graph below:

First, we must find the SD (or standard deviation) of our polls. Since we understand that the margin of error is just a re-statement of the 2SD Rule, we know half the margin of error equals the SD. In this case it is 2.

To utilize the graph we need to first get our Y-axis. This is SD2 (Standard deviation of the second distribution) divided by SD1. Since both have the same SD the ratio is 1.

To find the X-axis takes a bit more work. We must subtract the mean of our first distribution from the mean of our second distribution (M1-M2) and divide that by SD1 which is 2 in our case.

M1=48 (Coleman’s level of support) and M2=44 (Franken’s).

48-44=4

4/2=2

Our axis points for this graph are 1,2 and finding that point on the graph we see that approximately 35% of the two distributions overlap. We can then see 65% of Norm Coleman’s distribution is beyond Al Frankens and the bulk of Franken’s support is below Coleman’s.

We can use these numbers to give us a rough estimate of the chances Coleman would win the election were it held today. One might think “65%” is the right answer but that would be ignoring the overlap. Since the overlap is a toss up, we divide this number by two to get 17.5. The reason we do this is because in the area of overlap it is just as likely Coleman could win as Franken.

We add 17.5 to 65 (the amount of the distribution beyond Franken’s) and come to 82.5%, which is the approximate likelihood Coleman’s “real support” is higher than his opponent’s. Since we’re using approximations I round up to 83%.

So, despite the fact Franken’s numbers and Coleman’s numbers are “within the margin of error” there is an 83% chance Coleman beats Franken. This race isn’t a toss up at all, Coleman is firmly in the lead.

(You can even use this method to compare polls from competing groups, one just has to keep track of the differences in SDs)

Mike Ciresi was also included in the Survey USA poll. In a head to head against Norm Coleman he received 43% support to Coleman’s 49% support. There’s the same 4% margin of error. Going through the same process we find there’s a 95% chance Coleman has greater support than Ciresi despite the fact there’s only a 1% difference between Ciresi and Franken.

If you’re a Democrat, based on the polls seen here, who should you support? Since there’s an 85% overlap between Ciresi and Franken you could call this a tie. Really, there’s a 58% chance Franken is “really” better than Ciresi. If you add the lead Franken has in the polls to his fundraising ability a “rational” Democrat (A Democrat who cares only about winning and not about the policy differences between the two candidates) is going to support Franken for the nomination.

(This is a bit crude as Franken has greater name ID, a smaller amount of “undecideds” and higher negatives than Ciresi. Franken has gotten most of the press so far and he has a lot of money which skews his score. On top of that, another poll puts Ciresi ahead of Franken by the same 1% difference but they didn’t include a margin of error so I couldn’t use it for this post. Ciresi may be the stronger candidate once he gets more coverage which will allow him to raise enough money which will cause the “rational” Democrat to change their view.)

There you go, polls are just that easy.

Just remember, polls are important but they are done by humans and are fallible. The numbers will change as news and coverage changes in the election cycle. Always note where a poll comes from, the number of “undecided” voters and where you are in the election cycle. Polls are a valuable tool and one must be able to look at them critically to get the best information out of them.