Two of the world鈥檚 top scientific journals, Nature and the British Medical Journal, have been found guilty of routinely publishing figures that do not add up.
The evidence comes from Spanish researchers who found a surprisingly large number of statistical errors in the two journals. They warn that the same may well be true of other mainstream journals.
鈥淚t鈥檚 a warning to be more careful,鈥� says Emili Garc铆a-Berthou of the University of Girona, who discovered the discrepancies. 鈥淥ur concern is that these kinds of errors are probably present in all numerical results and all steps of scientific research 鈥� with potentially important consequences.鈥�
Advertisement
The analysis revealed that at least one error appeared in 38 per cent of the Nature papers and 25 per cent of the BMJ papers looked at. Furthermore, the study estimates that four per cent of results reported to be statistically 鈥渟ignificant鈥� may not be significant after all.
Rounding up
Garc铆a-Berthou and his colleague, Carles Alcaraz, analysed 32 papers published in 2001 in Nature and 12 in the BMJ. When they calculated numerical 鈥減 value鈥� results from supporting numerical data, they found inconsistencies in 11 per cent of cases in both journals.
They noticed that the frequency of the numbers four and nine in data was much lower than expected, backing up the idea that many of the errors are caused by incorrect rounding-up of numbers. Other errors probably arose from typographical errors.
鈥淲e think there鈥檚 a lot of publication pressure these days, and sometimes people write too quickly,鈥� he says.
As well as warning researchers and editors to be more careful with data, they also urge the publication of raw data online. 鈥淚f we had that, we could check the results,鈥� Garc铆a-Berthou says. 鈥淪ome journals already publish supplements online, but it鈥檚 rare, and I think it should become commonplace.鈥�
鈥淭丑别 Nature papers鈥� conclusions are not affected by the errors suggested to be in them,鈥� notes Maxine Clarke, Publishing Executive Editor of Nature. But she says that 鈥渟ome awareness-raising about statistical accuracy鈥� might be necessary, and says the issue will be discussed at the journal.
She adds that raw data for Nature papers can already be published: 鈥淲e already offer supplemental information online, to accompany any paper. So there is nothing stopping any author making any supporting raw data available.鈥�
Journal reference: BMC Medical Research Methodology (vol 4, no 13)
