(Image: Renaud Vigourt)
When robots fight in place of soldiers, will wars have fewer casualties? Or will the world slide into non-stop conflict?
QANDI AGHA used to be a cashier in Afghanistan鈥檚 Ministry of Culture. But he claims that in 2012 he was arrested by an elite US Special Forces unit and tortured for six and a half weeks. He was held under water until he felt like he was dying, says Joanne Mariner, an expert in humanitarian law for Amnesty International. 鈥淭hey also tied a cord around his penis so he couldn鈥檛 urinate,鈥 she says. 鈥淭hey left it on for four days.鈥
Advertisement
Agha鈥檚 torture is one of many such cases recorded by Mariner for a recent . Demonstrations of inhumanity by trained soldiers are not uncommon, from the infamous abuse of prisoners at Baghdad鈥檚 Abu Ghraib to the . 鈥淥bviously this doesn鈥檛 go anywhere, fellas鈥 I鈥檝e just broken the Geneva Convention,鈥 the marine said after pulling the trigger. 鈥淲hen you start getting into the business of killing people it takes a lot of care and control to make sure that you鈥檙e killing only the people that you鈥檙e legally allowed to kill,鈥 says Mariner.
Could machines do better? For many, it鈥檚 an outrageous idea. But others not only think machines could, but that they must. 鈥淗umans are currently slaughtering other humans unjustly on the battlefield,鈥 says Ronald Arkin, a robotics expert at the Georgia Institute of Technology in Atlanta. 鈥淚 can鈥檛 sit idly by and do nothing. I believe technology can help.鈥
Unlike humans, robots won鈥檛 break the rules. For this reason, the argument goes, wars fought by machines would be more humane. And sending robots in place of human soldiers would also save lives, particularly for the nation possessing such advanced tech. So the development of so-called lethal autonomous weapons systems 鈥 or 鈥渒iller robots鈥 鈥 is accelerating, with many of the world鈥檚 armies looking for ways to keep their soldiers out of the line of fire.
But there is strident opposition. This week, representatives of dozens of nations will gather at the UN headquarters in Geneva, Switzerland, to discuss whether the UN should consider a ban. The aim is to get killer robots classed alongside chemical weapons, and blinding lasers. With fully autonomous weapons a technical possibility, campaigners are keen to halt their development 鈥 and soon. But what鈥檚 at stake?
Robots currently play several roles on the battlefield. Some carry equipment, others dismantle bombs and provide surveillance. And remote-controlled drones let their operators control attacks on targets from thousands of kilometres away. The latest machines, though, take drones to the next level. Capable of selecting and engaging targets with little or no human intervention, the authorisation to open fire is sometimes all that remains under human control.
The US navy鈥檚 Phalanx anti-missile system on board its Aegis ships can perform its own 鈥渒ill assessment鈥 鈥 weighing up the likelihood that a target can be successfully attacked. UK firm BAE is developing an uncrewed jet called Taranis. It can take off, fly to a given destination, and identify objects of interest with little intervention from ground-based human operators unless required. The jet is a prototype and carries no weapons, but it demonstrates the technical feasibility of such aircraft.
Meanwhile, Russia鈥檚 鈥渕obile robotic complex鈥 鈥 an uncrewed tank-like vehicle that guards ballistic missile installations 鈥 and South Korea鈥檚 Super Aegis II gun turret are reportedly able to detect and fire on moving targets without any human supervision at all. The Super Aegis II can pinpoint an individual from 2.2 kilometres away.
Arms manufacturers don鈥檛 like to talk about the details. The specifics, in general, are classified. What鈥檚 clear, though, is that technology is no longer the limiting factor. 鈥淭echnology is not the likely restriction as to what is feasible in the future,鈥 says a spokesman for UK missile manufacturers . Instead, he says, autonomous weapons will be constrained by policy, not capability.
鈥淭he specifics are classified but it鈥檚 clear that technology is no longer the limiting factor鈥
What, then, are the relevant rules of war? , but all weapons must comply with existing conventions. One key principle is that civilians and civilian property must not be intentionally targeted. Weapons must also be capable of discriminating between civilians and soldiers. And the use of force must be proportional 鈥 the expected military advantage of an attack cannot be outweighed by collateral damage. 鈥淚 would argue there is a robust body of law that applies to autonomous weapons systems,鈥 says at Columbia Law School in New York. 鈥淎ny weapons system that is developed has to be reviewed for its basic compliance.鈥
Within such a framework, Arkin believes that if a lethal autonomous system can be shown to be better than a human soldier at limiting civilian casualties, then a ban on such technology would be a mistake. 鈥淲e have to remember the fallibility and frailty of humans in modern warfare,鈥 he says. 鈥淚f we can do better than them, we are saving lives.鈥
Others make the case just as strongly. Erik Schechter, a mathematician at Vanderbilt University in Nashville, Tennessee, , for example: 鈥淚f the goal of international humanitarian law is to reduce non-combatant suffering in wartime, then using sharpshooting robots would be more than appropriate, it would be a moral imperative.鈥
Robots could spare soldiers鈥 lives, too. Instead of calling in an airstrike to bomb a suspected enemy base, which may be situated in a densely populated urban area, for example, robots could enter the building ahead of human soldiers, allowing them to take the initial risks. Where aspects of a mission are especially risky, machines could take the lead.
The right to dignity
The idea is deeply divisive, however. For many, the prospect of a computer chip having the power of life or death over someone is discomforting. According to , the UN鈥檚 Special Rapporteur on extrajudicial, summary or arbitrary executions, it could contravene humanitarian law and the human right to dignity.
鈥淗umans need to be quite closely involved in the decision for it not to violate your human dignity,鈥 says Heyns. He notes that a robot鈥檚 targets don鈥檛 have the option of an appeal to humanity the way they might if a person was behind the weapon. It would be like being exterminated, he says. Remote-controlled drones already give little opportunity to make such appeals. But they at least have a human operator, however distant, who can make ethical judgements. 鈥淭he hope that this is possible is at least not completely absent,鈥 says Heyns. 鈥淎nd hope is part of a dignified life.鈥
Heyns isn鈥檛 totally against the rise of autonomous systems, however. Imagine a team of snipers with their sights trained on members of a militant group such as Boko Haram, he says. Hostages are nearby, so the snipers all need a clean shot at the same time for the hostages to escape unharmed. As long as the decision to kill is taken by a human commander, the actual shots could be triggered by a computer controlling the rifles. That may be the only way to get each gun to fire at the exact moment all the snipers have a clean shot.
Ultimately, however, Heyns is wary of what he calls the 鈥渄epersonalisation of force鈥. In a 2013 , he warned that 鈥渢ireless war machines, ready for deployment at the push of a button鈥 could lead to a future of permanent conflict. If governments don鈥檛 have to put boots on the ground, going to war could become too easy. Even in scenarios where machines fight machines, significant collateral damage could destroy a nation鈥檚 infrastructure. And as casualty numbers will be lower, wars could go on for longer, preventing post-war reconstruction.
鈥淚f governments don鈥檛 have to put boots on the ground, going to war could become too easy鈥
Fears of depersonalising war have convinced some that weaponised robots should be vetoed. 鈥淧eople die in wars,鈥 says US Congressman . 鈥淎nd we ought to never get to the point where we forget how horrible that is.鈥 Concerned that technologically advanced nations are distancing themselves from the reality of war, McGovern wants the US to take the lead in prohibiting autonomous weapons.
鈥淚 think people ought to be concerned about the impact of bombing, of drone attacks,鈥 he says. 鈥淵ou take the human out of that, you desensitise people. I want people to know that war is awful and ugly and terrible.鈥
There鈥檚 substance to McGovern鈥檚 fear. Research suggests drone strikes against human targets may radicalise members of a population previously unconnected to the conflict, for . 鈥淚t鈥檚 possible that the sense of injustice will be enhanced even more because the enemy you鈥檙e fighting is so disconnected from the conflict that it鈥檚 not even bothering to send soldiers,鈥 says , an expert in terrorism at London鈥檚 Royal United Services Institute.
This summer, McGovern organised a briefing for US politicians by members of the 鈥 a boon for , one of the campaign鈥檚 leaders, who has been trying to bring the issue to international attention for nearly a decade.
Sharkey is an expert in artificial intelligence and robotics at the University of Sheffield, UK. A key driver of his dogged campaigning is his awareness of the shortcomings of current technology. While Arkin has an eye on next-generation tech, Sharkey is concerned with the present. So is Mariner. 鈥淎 calculation is only as good as the data that goes into it,鈥 she says. Can a drone tell the difference between a bearded taxi driver and a bearded Taliban member? 鈥淭here鈥檚 no perfect computer that can do this assessment,鈥 she says. Sharkey agrees. 鈥淚 could make you a killer robot within weeks that detects human body signatures and fires at them,鈥 he says. 鈥淭he problem is being able to discriminate between a civilian and a combatant.鈥
Bottling that ability is hard. 鈥淚t鈥檚 a very formidable problem,鈥 says Glynn Wright, founder of , a UK-based firm that provides image-analysing software for security applications, 鈥渁nd it鈥檚 not solved by any means.鈥 Aralia鈥檚 software can highlight suspicious activity in CCTV footage. A couple of years ago the system flagged up the activity of a group of people who were later found to have been scoping out a public area for a suitable place to plant a bomb. The individuals were apprehended and successfully prosecuted, says Wright. But he readily admits that making snap decisions in busy urban environments is a long way off.
Even if a machine was capable of such discrimination, what of its ability to make moral decisions based on such data? Arkin has argued that we could develop software that acts as an 鈥渆thical governor鈥 to guide a robot鈥檚 response to various circumstances. The required complexity of such software, though, means that these proposals remain on the drawing board. For now, a human must remain in control.
Calling the shots
But what does that mean? The debate on killer robots turns on this question and the call for a ban will succeed or fail depending on what counts as having a human calling the shots (see 鈥In the loop鈥). There is no accepted definition. And there is another issue: focusing only on the question of killer robots conceals more fundamental problems with the way humans and machines interact. Just because a human is in the loop doesn鈥檛 mean the problems with high-tech killing go away.
In 2003, for example, an operator at a US Patriot missile battery received an automated alert that an incoming Iraqi missile had been detected. She had a split second to make a decision and chose to take defensive action, authorising the battery to fire. But the target turned out to be an RAF Tornado jet whose two pilots were killed when the Patriot missile hit. What went wrong? The system certainly misidentified the jet. But an inquest found that this was because . As a result, the system hadn鈥檛 been connected to a wider network that would have told it that the jet was in contact with air traffic controllers and not a threat. In this case, having humans in the loop was arguably the root of the problem.
Whatever the outcome in the UN, the issues raised in the debate over autonomous weapons systems aren鈥檛 as simple as they first seem. Whether fully autonomous or not, machines are already part of war and the promise to keep people in the loop is certainly no guarantee that mistakes won鈥檛 happen. Machines don鈥檛 streamline war, they complicate it. It鈥檚 something they have in common with us.
Leader: 鈥Automated killer robots come with enormous risks鈥
In the loop
The surge in drone warfare has raised the profile of remotely piloted military vehicles. Humans remain ostensibly 鈥渋n the loop鈥, but that doesn鈥檛 mean there aren鈥檛 problems. And 鈥渋n the loop鈥 can mean anything from controlling almost every facet of a remotely piloted vehicle, to simply saying 鈥測es鈥 or 鈥渘o鈥 when asked if an automatically selected target should be destroyed.
Nancy Cooke is an expert in the psychological impact of drone piloting at Arizona State University in Tempe. Her contract with the US military is to consult on 鈥渉uman systems integration鈥 鈥 how well people are in control of the machines at their fingertips. For pilots of Predator and Reaper drones, that means the 鈥渃ockpit鈥 deep inside a military base that directs aircraft flying thousands of kilometres away.
Maintaining control
Cooke鈥檚 research helps identify aspects of the drone-piloting interface that could be improved, aspects of training that can be changed to benefit pilots, and situations in which social and psychological support is most important.
The whole aim is to put the human in fullest control, says Cooke 鈥 but that isn鈥檛 easy. Drone operators themselves don鈥檛 make kill decisions, though they carry out all the functions necessary to do so. With endless video footage of their activities rolling in front of their eyes, and orders constantly issued from a nearby commander, the effects of this aren鈥檛 trivial.
Cooke says operators describe themselves as 鈥渂us drivers鈥 or feel roboticised, while their commanders load up on 鈥淧redator crack鈥 鈥 the video feed beamed back from the drones. The removal of people from the front line has created a strange situation, in which pilots are more acutely exposed to the effects of what they do, while feeling less in control overall, says Cooke.
鈥淭he video imagery is of body parts and injuries, fatalities and lots of destruction,鈥 she explains. 鈥淲hat鈥檚 relayed to me, at least from operators, is that that鈥檚 really a horrible thing because it鈥檚 only too real, what you鈥檝e just done. Unlike other cases where you鈥檙e just in and out and don鈥檛 really see what you did.鈥
Arms control from crossbows to cluster bombs
1139: Crossbow banned by Pope Innocent II for use against Christians
The pope said: 鈥淲e prohibit under anathema that murderous art of crossbowmen and archers, which is hateful to God, to be employed against Christians and Catholics from now on.鈥 The use of crossbows against non-Christians was of course acceptable.
1675: Strasbourg agreement
This outlawed the use of poison bullets.
1899: Hague convention
This included:
- Declaration concerning the Prohibition of the Discharge of Projectiles and Explosives from Balloons or by Other New Analogous Methods
- Declaration concerning the Prohibition of the Use of Projectiles with the Sole Object to Spread Asphyxiating Poisonous Gases
- Declaration concerning the Prohibition of the Use of Bullets which can Easily Expand or Change their Form inside the Human Body such as Bullets with a Hard Covering which does not Completely Cover the Core, or containing Indentations
All major powers at the time ratified the prohibition of aerial bombing, except the UK and the US. This ban expired in 1905.
1925: Geneva protocol
This is the 鈥淧rotocol for the Prohibition of the Use in War of Asphyxiating, Poisonous or other Gases, and of Bacteriological Methods of Warfare鈥.
1928: Kellogg-Briand pact
This was the 鈥淕eneral Treaty for Renunciation of War as an Instrument of National Policy鈥 and was signed by the leading powers of the time. Although it evidently failed in its stated purpose, it formed the basis of some important international laws that relate to the use of force.
1949: Geneva conventions
These updated previous treaties of 1864, 1906 and 1929. They established wartime rights for prisoners of war, wounded combatants and civilians.
1967: Outer space treaty
This banned the installation of weapons of mass destruction in orbit around the Earth or on the moon.
1970: Treaty on the Non-Proliferation of Nuclear Weapons
This was intended to prevent the spread of nuclear weapons technology beyond those countries that already had it and to promote the use of nuclear technology for non-violent purposes.
1975: Biological Weapons Convention
This bans the production and stockpiling of biological weapons (the use of which was already illegal under the 1925 Geneva protocol). One hundred and seventy states have ratified it 鈥 Egypt and Syria are among those that have signed but not ratified, and Israel is perhaps the most significant state that has not signed.
1978: Environmental Modification Treaty
This bans 鈥渨eather warfare鈥, the use of weather modification techniques for destructive purposes.
1995: Protocol on Blinding Laser Weapons
This bans weapons that aim to cause permanent blindness
1997: Chemical Weapons Convention
This bans the production and stockpiling of chemical weapons (as with biological weapons, their use was already illegal under the 1925 Geneva protocol); 190 states have ratified it 鈥 Israel and Burma have signed but not ratified, while Egypt and North Korea have not signed.
1997: Ottawa treaty
This bans the use, production and stockpiling of anti-personnel landmines. One hundred and sixty-two states are party to the treaty; 34 UN states, including China, Egypt, Israel, North and South Korea, Russia and the US, are not.
2010: Convention on Cluster Munitions
This bans the use and stockpiling of cluster bombs 鈥 explosives that scatter 鈥渂omblets鈥 over an area. Eight-eight states have ratified it so far 鈥 those who have not include China, Egypt, Israel, North and South Korea, Russia and the US.
2014: Lethal Autonomous Weapons Systems
This year, the question of whether to ban robot warriors was discussed at the UN as part of the ongoing Convention on Conventional Weapons, updates to which included the ban on landmines and blinding lasers.
This article appeared in print under the headline 鈥淲orld War R鈥
