Omid57.info

How to spot misleading statistics and charts in the news

By · Published · 5 min read

Cut-off axes, relative instead of absolute numbers, averages that hide the majority and cherry-picked time frames – the most common tricks with numbers, and how to see through them.

Numbers and charts make news look objective. Yet the same figures can tell very different stories depending on how they are presented – sometimes on purpose, often by accident. You do not need to be a statistician to notice the most common problems. This guide shows the typical tricks and the questions that expose them.

The cut-off axis

A bar chart shows two values: 50 and 52. If the vertical axis starts at zero, the bars look almost equal – which they are. If the axis starts at 49, the second bar suddenly looks three times as tall as the first.

So when a chart looks dramatic, look at the numbers on the axis before you look at the shapes.

Relative versus absolute numbers

"Risk doubles" sounds alarming. But if the risk rises from 1 in 10,000 to 2 in 10,000, the absolute change is one additional case per 10,000 people. Both statements are correct; only the second tells you how much it matters to you.

The same applies to percentage increases from small numbers: "crime up 50%" can mean 2 cases last year and 3 this year. Whenever you read a percentage change, ask: percent of what?

A related trap is the difference between percent and percentage points, explained in our guide on economic news.

Totals versus rates

A large city has more crimes, more accidents and more hospital patients than a small town – simply because more people live there. To compare places of different size, you need rates: cases per 100,000 inhabitants, per car, per household. A headline that ranks countries or cities by total numbers often just ranks them by population.

Averages that hide the majority

The average (mean) is sensitive to extremes. If nine people earn 3,000 a month and one earns 100,000, the average income is 12,700 – a figure that describes nobody in the group. The median – the value in the middle when everyone is lined up – is 3,000 and much closer to reality. Good reporting on incomes, rents or house prices therefore often uses the median. If only an average is given, ask whether a few very large values could distort it.

The chosen time frame

Every trend depends on where it starts. A share price that "rose 40%" may simply have recovered half of an earlier crash. Economic growth after a crisis year looks strong because the starting point was low – a base effect. And a period can be chosen precisely because it shows what the author wants to show.

Useful checks: What happens if you look at a longer period? Is the starting year unusual? Is the comparison with the same month a year earlier, or with an unusual month?

Correlation is not causation

Two things that rise at the same time are not necessarily connected. Ice cream sales and drowning accidents both increase in summer – not because ice cream causes drowning, but because both depend on warm weather. When a report says that A "leads to" B, ask whether the study really shows cause and effect, or only that both happen together. Often a third factor explains both.

Small samples and selected cases

A survey of 50 people, a study with 20 patients or "three experts agree" can be interesting, but they rarely support general conclusions. Equally, a single striking example – one family, one company, one incident – shows that something can happen, not how often it happens. Our guide on opinion polls explains how sample size affects precision.

Maps that mislead by area

On a map coloured by election results or statistics, large but thinly populated regions dominate the picture. A country can look almost entirely one colour although most people live in a few small, densely populated areas with the other colour. Maps that size areas by population, or simple tables, often give a fairer picture.

Money without inflation

Comparisons of wages, prices or budgets over many years are misleading if they ignore inflation. A salary that doubled in twenty years may buy less than before if prices tripled. Look for "real" or "inflation-adjusted" figures; our guide on inflation explains why.

Seven questions for any number in the news

  1. Who produced the figure, and does that person or organisation have an interest in a particular result?
  2. Percent of what? What are the absolute numbers behind a percentage?
  3. Total or rate? Are places or groups of different size compared fairly?
  4. Average or median? Could a few extreme values distort the picture?
  5. Which time frame, and what happens with a longer one?
  6. Cause or coincidence? Does the evidence show that one thing causes the other?
  7. Where does the chart's axis start, and is the scale labelled?

Numbers on Omid57.info

Omid57.info shows headlines from many outlets, and many of them contain figures: growth rates, poll results, casualty numbers, prices. Seeing several outlets side by side often reveals that the same number is presented quite differently – as a rise or a fall, in percent or in absolute terms. Our guide on how to read the news critically has more general checks for any story.

Conclusion

Most misleading statistics are not lies but choices: which axis, which comparison, which time frame, which average. Once you know these choices, you can ask the right questions – and the numbers in the news become what they should be: useful information rather than a tool of persuasion.