Polling Methodology And Politics - What's Up With Ron Paul's Polling Numbers?

If you're a Ron Paul supporter, you probably have read a few comments, blog posts and articles attempting to justify Ron's poor performance in polls. Some make sense and others are plainly excuses. I'm not certain theories about cell-phone-only households are good enough to explain Dr. Paul's poor showing in these polls though it may be valid for a certain demographic (18-24 year olds).

One of the better explanations I have read is that pollsters are selecting responses which only include loyal republicans and democrats who are likely to vote. Because they are ruling out other possibilities, they are not able to measure what is really happening amongst the general population.

For the most part, poll respondents are discovered by dialing numbers in a phone directory (listed numbers). The pollsters continue calling until they get enough respondents who match the criteria they've selected (x republicans who answer they are likely to vote - x democrats who answer they are likely to vote).

Out of curiosity, I decided to do some googling in order to get a better idea of the mechanics of polling. I'm certainly not going to become an expert on polling using google but there are things that I'd like to know.

Besides using standard directory listings, pollsters also use a method called Random Digit Dialing. This attempts to reach people who have unlisted numbers and won't be found in the directory listings. Some people have theorized that people with unlisted phone numbers might offer significantly different results but according to a Zogby study on RDD, advances in technology (CallerID) and changing demographics (unlisted numbers aren't just for the affluent anymore), people with unlisted phone numbers are less likely to participate then they were 20 years ago.

“When I started in 1984, response rates averaged 65%,” Zogby says. “You were twice as likely to get someone to answer the telephone. Today they don’t answer the phone and they are much more likely to refuse to participate in a survey.”

Zogby has concluded that RDD doesn't offer a significantly better sample and so doesn't use RDD. However, the Zogby study conclusions talk more about costs and "interviewer fatigue" as a rationale then it does the actual polling results. Not being an expert, I merely note this because it is interesting. Zogby admits that academic studies haven't found RDD any more or less reliable then the old phone book method.

In the academic world, no consensus has been reached among public opinion researchers on the superiority of RDD over listed samples. However, the results from a number of studies are consistent with Zogby’s assessment that listed samples are more efficient, less costly to administer, and produce results similar to those of RDD.

Zogby's study does say that in measuring differences between the two polling techniques, there are moderate differences in responses 35% of the time and significant differences 2-3% of the time. Given the margin of error, one *might* conclude that RDD may produce more accurate results. But at what cost? Zogby thinks it's too expensive for the results given.

A poll cannot predict anything by definition. It only attempts to get the current opinions of the people contacted. It can't read the minds of those contacted and it cannot contact the same people intentionally ( to track trends ) lest it invalidate the randomness of the samples used.

Rather, the alleged randomness. For some reason, the media and politicians seem quite obsessed with the opinions of very small numbers of Americans extrapolated to conclude that therefore the general population would react similarly by percentage. I'm not convinced. As much as the media is liable to promote a point of view using polls as "evidence", I am suspicious that such reliance is because polls are so easily manipulated.

Most everyone who follows the gun debate remembers Arthur Kellermann and Donald Reay's infamous epidemiological farce which concluded (on the first pass) that people who keep guns in the home are 43 times more likely to have a family member killed by a gun than those who do not keep a gun in the home. This number kept getting revised downward as time went on and as more and more reputable reviewers poked holes in the data until the study was altogether discredited. The basis of the study, for which they claimed properly ruled out bias, was homes of people where a gun homicide had occurred. Apparently it didn't occur to either Kellerman or Reay that study "participants" were more likely to be endangered due to their criminal behavior then they were by idle, inanimate objects. Because the sample used was weighted so heavily with people involved in crime (prior felonies) the sample was biased in the extreme.

So why isn't it considered bias when respondents are limited to those who are likely democrat or republican registered voters? There are certainly states which allow crossover primary voting though not all states do. So, in those states where it is allowed, why not include responses from members of "third" parties?

Again, I'm not an expert in polling, but I am familiar with testing methodologies. I've been testing software for over 20 years. I have seen some very interesting and brilliant uses for random sampling that proved extremely useful. However, people are not like software or hardware. When applying random sampling techniques in simulations, the data being sampled is finite and well-defined in its characteristics. Humans are far more complex and infinitely less predictable than the opinion makers would have you believe.

The polling results we are seeing also doesn't account for claims made by RP supporters and noted by observers. These fall into several distinct categories:

1. Young voters who were not old enough to have voted in the prior election cycle.
2. Young voters who could have registered in the last cycle but did not find a candidate attractive enough to bother.
3. Older voters who had given up on the electoral process.
4. Independent voters who aren't always loyal to a specific "third party"
5. Independent voters who have shown loyalty to a particular third party but will not be loyal to that party this election cycle.
6. Democrats who plan on switching parties or will not be voting for a Democratic candidate for President.

Zogby, Rasmussen, Newsweek, Bloomberg et al, do not include these categories. If they show up, they eliminate these from their reported results in order to keep their samples "pure". For "second tier" candidates including Ron Paul, Tom Tancredo, Mike Gravel and Denis Kucinich - who offer at least some significant variations on the issues from their "first tier" counterparts - the omission of this data could be significant. It may even be significant enough to skew the numbers unrealistically toward the "top tier" candidates since substantial percentages of respondents may not be considered.

Another factor that may be significant is the formulation of the questions. The questions do not attempt to discover any rationale for respondents choices but only the "seriousness" of the respondent (are you not sure, somewhat sure, pretty sure, extremely sure, super extremely sure....)

Are those sampled choosing an answer based on name recognition alone or based upon a specific issue which they agree with the candidate's view? Can the respondent even list the positions of the candidate they have chosen? Since polling questions as to motive are saved for another random sample, we don't have the answers to such questions by the original respondents. It is pure conjecture to attempt any linking of motives to the original responses. (ie; People are angry with Bush because the war isn't going well therefore) I am reminded of daily news reports which attempt to tell us why the market went up or down in spite of the fact that there are often news reports which contradict those reasons. Yesterday, Yahoo claimed that the market went up on retail sales reports. On the same page, there were two other news headlines, one which said that Walmart reported a 9% increase in sales last quarter, the other claimed that there was a general retail sales slump for the past quarter. While not equivalent, the claim by Yahoo that markets went up on reports of stronger retails sales is as reliable as claiming Ron Paul is getting 0-3% "real world" support.

It is also extremely early in the cycle to put any stock in polls that have been conducted thus far. As mentioned before, polls cannot be used to offer predictions. The only reliable political polls are those which include openly counted paper ballots.

One other important factor is the intentional omission of certain candidates. In spite of the fact that Ron Paul has surpassed the one-time front-runner McCain in cash on hand, neither Newsweek nor La Times/Bloomberg include Paul in its poll.

Anyone using poll results as a means to prove or disprove a candidate's chances next fall is revealing an agenda. The polling data that you aren't seeing would prove interesting I think. Candidates often hire polling firms to help them measure the results of certain policy statements. Clinton was notorious for commissioning polls to see which way the wind is blowing.

If polls were so good at predicting voter opinions, what happened with the recent fiasco over immigration? Both parties had to run for cover and the amnesty bill was abandoned. Wouldn't it be interesting to see privately commissioned polling data immediately after "The Decider" got decisive and demanded an amnesty bill?

I don't intend to completely dismiss polling data. However, the growth of third party membership and the decline of those participating in both telephone surveys and voting in elections suggests that the pickings are getting slimmer and slimmer for pollsters. To put any faith in them 16 months prior to the general election is unrealistic. To use them as a means of predicting the outcome of the general election? Completely absurd.

Tags:ronpaul polls politics demidog