It was one of last year鈥檚 most astonishing scientific stories: a leading psychology journal accepted a paper presenting evidence for precognition 鈥� an ability to perceive future events. What鈥檚 more, mainstream psychologists had pored over of the paper and found no fatal flaw.
Bold scientific claims need to be replicated before gaining widespread acceptance, however, and now the Journal of Personality and Social Psychology, which in its March 2011 issue, has touched off controversy by rejecting the first attempts to repeat the work without sending them out for peer review.
The incident exposes a problem that may be biasing the entire body of psychological literature, argues Richard Wiseman of the University of Hertfordshire, UK, one of the authors of a manuscript that describes three failed attempts to repeat an experiment in the original paper. If failed replications languish unpublished, he says, 鈥測ou don鈥檛 know whether the effects that are published are genuine. It鈥檚 a problem in psychology, and it鈥檚 a particular problem in parapsychology.鈥�
Advertisement
The original paper described nine experiments conducted over eight years by , a social psychologist at Cornell University in Ithaca, New York, with a long-standing interest in parapsychology. Bem鈥檚 strategy was to take well-established psychological phenomena and reverse the sequence of events, so that the 鈥渃ause鈥� happened after the 鈥渆ffect鈥�, rather than before.
Delighted and perplexed
When news emerged that the Journal of Personality and Social Psychology had vetted the work and decided to publish it, believers in the paranormal were delighted, sceptics perplexed, and bystanders fascinated.
New 杏吧原创鈥荣 initial story about the paper was among the most widely read articles we published online last year. Bem ended up a minor celebrity, being interviewed on Comedy Central TV鈥檚 鈥� where host Stephen Colbert homed in on experiments into 鈥渢ime-travelling porn鈥�, in which volunteers seemed to anticipate the position where erotic images would appear on a computer screen.
Frivolity aside, confirmation of Bem鈥檚 findings would turn established ideas about time, cause and effect on their head. 鈥淲e openly admit that the reported findings conflict with our own beliefs about causality and that we find them extremely puzzling,鈥� the Journal of Personality and Social Psychology鈥荣 editors said in an .
Other researchers began to try to repeat the results as soon as news of Bem鈥檚 findings began to spread. Wiseman set up of such attempts, which has so far documented five.
Primed for random choices
Three of these were described in a paper from Wiseman, of Goldsmiths, University of London, and of the University of Edinburgh, UK, which was sent to the Journal of Personality and Social Psychology.
Each tried to repeat an experiment of Bem鈥檚 which was based on a well-tested experiment for memory priming. In Bem鈥檚 back-to-front version, participants were shown a list of words and then asked to recall words from it. Later Bem showed them words randomly selected from the same list, and it turned out that they had been better at recalling these words in the prior test. The subsequent display seemed to have influenced their earlier memory.
In the conventional psychological experiment on which Bem鈥檚 experiment was based, people are shown particular words, and then are given a list of words that include the ones they have previously experienced. The participants are next asked to recall as many words as possible from the list, allowing the experimenters to quantify the effect of the prior priming on the recall of those words.
In contrast to Bem鈥檚 results, Wiseman, French and Ritchie failed to find that the subsequent typing facilitated the volunteers鈥� earlier recall. But of Indiana University in Bloomington, the Journal of Personality and Social Psychology editor who handled the submitted paper, declined to send it out to review. 鈥淭his journal does not publish replication studies, whether successful or unsuccessful,鈥� he wrote.
Journal of Bem Replication
Smith defends the decision, noting that he made the same ruling on another paper that, by contrast, supported Bem鈥檚 findings. 鈥淲e don鈥檛 want to be the Journal of Bem Replication,鈥� he says, pointing out that other high-profile journals have similar policies of publishing only the best original research.
鈥淚 certainly agree that it鈥檚 desirable that replications are published,鈥� Smith told New 杏吧原创. 鈥淭he question is where. There are hundreds of journals in psychology.鈥�
Bem stressed the importance of replication in his original paper. However, he argues that firm answers will come only when it is possible to conduct a meta-analysis of multiple attempts at replication. 鈥淚 understand the journal鈥檚 position,鈥� he says. 鈥淚t almost never publishes a single study.鈥�
Wiseman is unconvinced, however, arguing that a meta-analysis may miss the whole picture if journals are reluctant to publish replication studies: 鈥淢y feeling is that the whole system is out of date and comes from a time when journal space was limited.鈥� He argues that journals could publish only abstracts of replication studies in print, and provide the full manuscript online.
