Items of Statistical Significance In Maine's Fall Election

[1]

Many political observers view the margin of error in opinion polls as the most important metric of a poll’s potential accuracy. This isn’t always the case. Often the question wording, voter screen, and weighting of the poll may be more important in understanding the survey than the MOE, which is almost always simply a measure of how many people were interviewed.

The margin of error is, however, an important and easily verifiable first check on accuracy and noting and explaining the margin of error is the very least that any reputable news organization should do when reporting on a poll.

That’s why I was so disappointed to see this article [2] in the Portland Press Herald earlier this week on the 2nd District congressional race, which stated:

The Maine Poll, with a margin of error of 4.9 percentage points at a 95 percent confidence level, was conducted Monday and surveyed 218 likely voters in the 2nd Congressional District.

Without getting into the formulas that are used to determine a margin of error and concepts like standard error and confidence intervals, suffice to say that one can determine the maximum margin of error for a poll (assuming the poll sampled fewer than 5 percent of the target population) simply by knowing the number of people interviewed.

(A handy formula for determining the approximate margin of error of a poll at the 95 percent confidence level is .98 / √n where n is the sample size.)

A survey of 218 voters would have a margin of error of around 6.64 percent, 95 times out of 100. This describes the bell curve in which the actual voting preferences of the population of the second district (assuming the pollster chose a representative, random sample of this population) might fall if the results of the poll were split 50-50 (giving the maximum error).

The difference between 4.9 percent and 6.64 percent may not be as important in a poll with one candidate leading the other by 12 points, but reporting the error for the entire poll rather than this question is a sign that someone at the paper either doesn't understand the research that they're using or doesn't care.

I mentioned previously that the MaineToday papers have reported the results of their polling on the governor's race without mentioning that their pollster, Critical Insights, has done some work for the Cutler campaign. This time when they reported their polling results [3] on the Casino referendum, they did note that Critical Insights had worked for the pro-casino side. CasinosNo frontman Dennis Bailey, however, sent a release urging "full disclosure" of the extent of their work, mainly that CI has been paid more than $80,000 this cycle by Black Bear Entertainment, the backers of the referendum.