Every election season, the same objection appears: “They only asked a thousand people. Nobody asked me. How can that possibly represent a country of sixty million?”
It is a reasonable question, and the answer is one of the more counter-intuitive results in statistics. A well-drawn sample of a thousand people really can describe a population of millions with useful precision — and, crucially, the size of the population barely matters. What matters is how the sample was chosen and how large it is in absolute terms.
The soup analogy
The classic illustration is a pot of soup. To know whether it needs salt, you do not eat the whole pot. You stir thoroughly and taste one spoonful. If the pot is properly stirred, one spoonful tells you about the whole thing — and it tells you just as reliably whether the pot holds two litres or two hundred.
The stirring is the important part. If the soup is not mixed and all the salt has settled at the bottom, your spoonful is worthless no matter how big it is. In survey terms, stirring is random sampling: every member of the population must have a known, non-zero chance of being selected.
Why population size hardly matters
This is the part that surprises people most. The precision of an estimate depends almost entirely on the number of people you actually asked, not on the proportion of the population they represent.
A sample of 1,000 gives you roughly the same precision whether you are studying a city of 500,000 or a country of 50 million. The proportion sampled is wildly different — 0.2% versus 0.002% — but the resulting margin of error is nearly identical. Population size only starts to matter when your sample is a large fraction of the whole group, which is rare in practice.
What margin of error means
When a poll reports “52% support, margin of error ±3 percentage points”, it is saying that the plausible range for the true value runs from about 49% to about 55%. The reported figure is the centre of a range, not a precise measurement.
That range comes with a stated confidence level, usually 95%. The standard interpretation is that if the same survey were repeated many times under identical conditions, about 95 out of 100 of the resulting intervals would contain the true population value.
Two practical consequences follow. First, a single poll showing 52% to 48% in a two-way race with a ±3 point margin is not evidence of a lead — the intervals overlap. Journalists call this a statistical tie, and it is a genuinely important distinction. Second, differences within the margin of error should not be reported as change. A shift from 47% to 49% between two polls is usually noise.
Diminishing returns of a bigger sample
Precision improves with sample size, but slowly. Because the margin of error shrinks in proportion to the square root of the sample size, cutting the error in half requires roughly four times as many respondents.
| Sample size | Approximate margin of error (95% confidence) |
|---|---|
| 100 | about ±10 points |
| 400 | about ±5 points |
| 1,000 | about ±3 points |
| 2,500 | about ±2 points |
| 10,000 | about ±1 point |
These figures assume a simple random sample and the least favourable case, where opinion is split near 50-50. They explain why 1,000 is such a common sample size: it lands at the point where precision is respectable and cost is still reasonable. Going to 10,000 multiplies the budget by ten to gain two percentage points of precision.
Bias: the error that sample size cannot fix
Margin of error only describes random sampling variation. It says nothing about systematic error, and systematic error is what actually ruins surveys. A biased sample of a million people is worse than a random sample of a thousand, because the large sample gives you false confidence in a wrong answer.
- Coverage bias. Your sampling method cannot reach part of the population — for example, a landline-only survey missing younger households.
- Non-response bias. The people who decline to answer differ systematically from those who agree.
- Self-selection bias. Online polls where anyone can click sample the motivated, not the representative.
- Question wording. Leading or emotionally loaded phrasing changes answers before any statistics are involved.
- Question order. Earlier questions prime how later ones are answered.
Serious survey organisations correct for some of this by weighting — adjusting responses so the sample’s demographic profile matches the known population. Weighting helps, but it relies on assumptions and cannot rescue a fundamentally unrepresentative sample.
How to read a survey critically
Before you accept a headline number, look for five things:
- Who was sampled, and how were they contacted? Random selection from a known list is very different from volunteers on a website.
- What is the sample size and margin of error? If neither is reported, treat the result as an anecdote.
- Are subgroup results being quoted? A breakdown of a 1,000-person poll into age brackets may leave 150 people per group, with a much wider margin of error than the headline figure.
- What exactly was asked? The full question wording often explains a surprising result.
- Who paid for it? Not disqualifying on its own, but worth knowing.
Conclusion
A thousand people can speak for millions, but only if they were chosen properly. Sample size controls precision; sampling method controls whether the answer is aimed at the right target at all. When you read the next poll, check the method first and the headline number second — and remember that any single figure is really a range in disguise.
If you would like to understand the mathematics behind these ideas properly, Cursa offers free statistics courses covering sampling, confidence intervals and hypothesis testing from the ground up.
