鈥淭here is always a well-known solution to every human problem 鈥� neat, plausible, and wrong.鈥�
Yesterday the Australian government rediscovered H. L. Mencken鈥檚 century-old wisdom, when Sally Brinkman and her colleagues at the University of Western Australia . These mechanised dolls are meant to reduce teenage pregnancy by providing a cautionary experience: the dolls cry, wee and need to be 鈥渇ed鈥�, interrupting sleep schedules and annoying the whole family when teens take them home for a school assignment.
That they must work was so self-evident that no one checked, and over the past decades they have been rolled out in school districts in 89 countries.
Advertisement
This was set to happen in Western Australia too when someone asked the question that tends to lead to all major scientific shake-ups: 鈥淗ang on 鈥� is there any evidence that this actually works?鈥�
The answer, incredibly, was no. About but they were all non-randomised and didn鈥檛 follow through to see whether the dolls actually reduced pregnancy rates, says Julie Quinlivan of the University of Notre Dame Australia, who reviewed Brinkman鈥檚 work.
So Brinkman and her colleagues set out to find the answer. Starting in 2003, they tested the intervention on more than 2800 girls at 57 schools and found that the intervention group actually had higher rates of pregnancy and abortion than the control group.
Behind the alluring narrative of the off-putting doll, Brinkman and her colleagues found a web of more complex realities. It may have played into an idealised view of pregnancy that girls in disadvantaged communities are already vulnerable to 鈥� giving these girls a cute, fun doll to take home for a weekend is hardly an accurate reflection of the torments of parenthood. Then there was the positive attention that the dolls create. 鈥淕irlfriends would gather around and say 鈥榦h! you鈥檝e got the doll鈥�,鈥� says Quinlivan.
Uncommon sense
This cautionary tale is unfortunately not an unusual one. There are many seemingly obvious 鈥渃ommon sense鈥� interventions that crumble under the slightest probing. New 杏吧原创 recently documented many examples of U-turns over medical interventions. And Brinkman鈥檚 work is just the most recent example of how things can go unexpectedly wrong in social policy as well.
For example, in the UK, several areas have piloted the use of incentives to promote attendance at adult literacy classes 鈥� pay people a little money for going to their classes, the thinking went, and attendance and literacy would go up. Several such programmes went into effect without rigorous evaluations. In 2008, the first randomised controlled trial to properly test the idea gave adult learners the typical 拢5 per class for attendance 鈥� and found that attendance was worse than in control groups. Further investigation revealed this to be down to the fact that paying very small amounts for something can actually decrease its perceived value.
Sometimes, the 鈥渃ommon sense鈥� is so alluring that even when contradicted by multiple studies, it resists correction. For example, for a long time the official policy in many hospitals after an emergency worker had suffered a potentially traumatising event was a to head off any chance of post-traumatic stress. It turns out that going over what happened in detail can actually bring on post-traumatic stress disorder. Nonetheless, the idea that it is common sense to do these debriefs is so 鈥渢ruthy鈥� that despite 20 randomised controlled trials, compulsory debriefing has not been entirely eradicated, says Quinlivan.
This has also been the case for the 鈥淪cared Straight鈥� programme, which took US children to local prisons to deter them from crime and delinquency. The reasoning was that showing them the realities of prisoners doing hard time would make them toe the line. The opposite was true; their delinquency increased. 鈥淒oing nothing would have been better than exposing juveniles to the program鈥� . Nonetheless, these .
Perhaps the worst magnet for dud silver bullets is philanthropy, particularly in developing countries. This is mainly because cultural differences can make it hard for remote do-gooders to understand on-the-ground realities. For example, malaria nets are often dismantled for purposes deemed more immediate. This is fairly benign 鈥� worse is the effort to donate used clothes. Common sense would tell you that this could have no downside. The reality is more complicated undermines local clothes makers, fabric makers, dye manufacturers, cotton growers, and so on until whole economic sectors have been demolished.
Is playtime over?
Brinkman鈥檚 study isn鈥檛 without its own methodological flaws but it highlights the need to look more closely. Even if its findings are corroborated, however, there鈥檚 no guarantee that baby simulators will fall out of fashion. In the US, the simulators are said to be in use at 67 per cent of US school districts, according to manufacturer Realityworks. A spokesman at the National Institutes of Health says that even if a policymaker decides that a rethink is in order, it will be a long road before a proposal is written, accepted, funded, carried out and published. Even then, US school systems are under the control of local jurisdictions, and the decision to use the dolls is ultimately theirs to make individually.
In Australia, says Quinlivan, the dolls and their attendant curriculum can cost schools about A$2500 (拢1500), and it is unlikely that the providers will give up the niche without a fight. Neither are politicians going to be happy to give up the prospect of a voter-friendly techno-fix. 鈥淲hether it works is less important than saying 鈥業鈥檓 doing something鈥�,鈥� Quinlivan says.
So just how sure do we need to be of that something? Should interventions ever be rolled out before sufficient evidence has accumulated for their efficacy? Sometimes. But only when it鈥檚 clear no harm will be done. In the US, Congress mandated for chain restaurants; these were proven in , but even if they hadn鈥檛 been, the only cost would have been financial.
They should also go ahead when the harm being done is so severe that not intervening would be unethical. Take those malaria nets. Even if only a few people use them for their intended purposes, they save enough lives to make the entire endeavour worthwhile.
Expecting the unexpected
But when 鈥� as in the literacy and doll examples 鈥� there are possibilities for hidden variables, it鈥檚 crucial to do randomised controlled trials. This is not only to prove that the intervention works, but also to keep verifying that it works in the way you think it does, says David Torgerson of the University of York, who collaborates with the UK government鈥檚 Behavioural Insights Team.
And in some cases, doing trials can be crucial to rescuing good ideas that have failed the common-sense test. For example, when the UK looked at whether text messaging might encourage people to pay their court fines, the initial reaction was scepticism. But a trial showed that not only did it work, , increasing not just repayment rates but also the amount recovered.
What may be most important is to make sure you鈥檙e not trying to solve the wrong problem. The baby simulator is an excellent example of 鈥渇orest for the trees policy鈥�, says Torgerson: it鈥檚 easy to look like you鈥檙e doing something about a social ill. But on closer examination, teenage pregnancy is a symptom, not a cause. 鈥淎 teenage girl becomes pregnant because there are no other options in her life,鈥� says Quinlivan. To really solve it in any meaningful way, you have to give young girls an identity they can realistically pursue beyond 鈥渕other鈥�.
But that requires abandoning the search for a silver bullet. Perhaps it鈥檚 worth rethinking a policy culture that often seems to be predicated on the maxim 鈥淒on鈥檛 just stand there 鈥� do something!鈥� A better idiom for public policy might be 鈥淒on鈥檛 just do something.鈥�
Journal reference: The Lancet, DOI:
