杏吧原创

In his superlative The Improbability Principle, David J. Hand makes sense of bizarre patterns in Bible codes, lightning strikes and even drug trials

PEOPLE thought it was a fix when the Bulgarian lottery announced the same six winning numbers on two consecutive draws in 2009. The national sports minister launched an investigation, but found no tampering; the repetition was mere chance.

How are such coincidences possible? In The Improbability Principle, statistician David J. Hand delves into extreme unlikelihoods, from duplicate lottery results to global financial meltdowns. 鈥淲e should expect the unexpected,鈥� he explains, in this lucid overview of the mathematics of chance and the psychological phenomena that can make probability seem counter-intuitive to so many.

Understanding the Bulgarian lottery coincidence, it turns out, is relatively straightforward. Although the odds of two specific draws matching is 1 in 13,983,816, the chance of any two draws matching increases with the number of draws, and reaches a probability of greater than 50 per cent by the 4404th round.

Hand concedes that the odds of consecutive draws being identical are significantly less, but points out that there are many lotteries globally, so we shouldn鈥檛 be surprised when winning numbers occasionally repeat consecutively somewhere.

He dubs this 鈥渢he law of truly large numbers鈥�, and applies it to lightning strikes, plane crashes and even Bible 鈥渃odes鈥�, where the sheer number of letters means anyone searching for a 鈥渉idden鈥� message will inevitably find one. 鈥淲ith a large enough number of opportunities, any outrageous thing is likely to happen,鈥� writes Hand. Because we tend to focus on specific instances instead of the broader context, we fail to recognise the true probability of an event.

鈥淭he law of large numbers applies to lightning strikes, plane crashes and even the Bible鈥�

Not all outrageous events depend on truly large numbers. Even small numbers will suffice, if probabilities are ascribed after the fact. Consider how dreams can seem to miraculously foretell future events. US president Abraham Lincoln and the Roman emperor Caligula both dreamed of dying shortly before they were assassinated. That seems pretty impressive until you consider the likelihood they dreamed of death on other occasions. Who would bother to record them? Unfulfilled prophesies are not newsworthy.

Referring to this as the 鈥渓aw of selection鈥�, Hand notes it doesn鈥檛 only afflict doomed heads of state. In fact, selection bias is one of the most pernicious problems in research. Take drug trials, where people whose symptoms don鈥檛 improve have a tendency to drop out, potentially making the drug look more effective than it is.

Even if the dropout bias is accounted for, publication bias may skew the data. Researchers rarely write up experiments that fail to show an effect, and few journals will publish such papers. We only care about dramatic results. But knowing about only recorded dreams or published experiments, you have no basis for measuring probability.

Other improbable phenomena are explained by different facets of Hand鈥檚 improbability principle. His 鈥渓aw of near enough鈥�, for example, explains what happens when 鈥渆vents that are sufficiently similar are regarded as identical鈥� 鈥� a slip exploited by parapsychologists who support their vague hypotheses with pseudoscientific experiments.

In a classic case from the 1930s, a researcher at University College London asked 160 people to guess which card he had chosen from a deck. When they failed to guess, he included the cards before and after his selection. He continued to loosen his standards until the experiment appeared to demonstrate psychic ability.

The concepts underlying Hand鈥檚 laws will be familiar to all who have studied probability. His improbability principle is simply a conceit for assembling them in a book. But as an organisational scheme, it serves the purpose exceptionally well.

Hand has written a superlative introduction to critical thinking, accessible to everybody, regardless of mathematical ability 鈥� and essential if your probability of taking a university-level maths class is less than zero.

This article appeared in print under the headline 鈥淓xpect the unexpected鈥�