Understanding Margin of Error in Polls

This plot tries to explain on feature of reading polling numbers that many people over look, that statisticians call Poisson Noise.

There are many reasons why an opinion poll is not an accurate predictor of an election result. For example

1. The election is at some point in the future, and people may change their minds
2. The poll may undercount a segment of the population that tend to vote for one candidate over the other. For example, a poll that doesn't include Black people will likely over state support for a Republican candidate
3. The pollsters may just get unlucky, and happen to talk to more people on one side than the other.

This last point is a well studied issue and well understood mathematically. If point number 3 was the only source of error in a poll (and it never is), we can easily calculate the probability that a second poll would show the same person leading.

The figure above shows the probability that two polls, with the same margin of error, will show the same candidate leading.

If the margin of error in the poll is 3% (the blue line), and candidate A leads candidate B 52-48%, the probability that candidate A will lead candiddate B in a second poll with the same margin or error is about 75%.

If you assume that the pollsters properly accounted for point 2 above (did they ask the right mix of people), and you assume that no one changes their mind, then candidate A has a 75% chance of winning the election.

In reality, it is impossible for pollsters to be 100% sure they have correctly accounted for undercounting a demographic. Typically, if a poll interviews 1000 people it will claim a margin of error of 3% (the blue line) above, but the actual margin of error will be closer to 5% (purple line).

Smaller polls (400-600 interviews) will claim an accuracy of 5%, but are probably no better than 7%.