Military drones may have autonomously attacked humans for the first time ever last year, according to a United Nations report. While the full details of the incident, which took place in Libya, haven鈥檛 been released and it is unclear if there were any casualties, the event suggests that international efforts to ban lethal autonomous weapons before they are used may already be too late.
The robot in question is a Kargu-2 quadcopter produced by STM, a Turkish firm. It is equipped with an explosive charge and can be flown manually, but in autonomous mode the drone uses on-board cameras with artificial intelligence to find and identify targets. It can then attack by flying into the target and detonating. STM claims the robot has sophisticated object and facial recognition capability. The firm didn鈥檛 respond to a request for comment.
Details of the apparent attack have emerged in a report by the UN Security Council鈥檚 Panel of Experts on Libya, . It details a civil war conflict in March 2020 between forces allied to Libya鈥檚 Government of National Accord (GNA), which has since been dissolved, and those affiliated to Khalifa Haftar, commander of the Libyan National Army.
Advertisement
The report says that retreating Haftar forces were 鈥渉unted down鈥� by Kargu-2 drones operating autonomously, which were 鈥渉ighly effective鈥�. 鈥淭he lethal autonomous weapons systems were programmed to attack targets without requiring data connectivity between the operator and the munition: in effect, a true 鈥榝ire, forget and find鈥� capability,鈥� notes the report. In other words, the drones were able to seek and attack targets without a human in the loop.
The panel says in its report that this information comes from a confidential source and declined to disclose further details to New 杏吧原创.
at the National Consortium for the Study of Terrorism and Responses to Terrorism in Maryland, says this appears to be the first time that autonomous drones have found and attacked humans. UN talks on restricting such lethal autonomous weapons and these weapons aren鈥檛 currently restricted, with many nations including the UK arguing against an outright ban.
Kallenborn says he is concerned about the Kargu-2鈥檚 reliance on machine vision. 鈥淗ow brittle is the object recognition system?鈥� he says. 鈥淎nd how often does it misidentify targets?鈥� While there is no public information on the drone鈥檚 AI, researchers have previously found that general machine vision algorithms can be fooled into misidentifying a turtle as a rifle.
at the United Nations Institute for Disarmament Research in Geneva, Switzerland, also voiced concerns. 鈥淲hether any such system could execute such functions reliably and to a high degree of precision under complex and dynamic conditions is a totally open question,鈥� he says.
at Royal United Services Institute, a UK defence think tank, wonders whether the Kargu-2 drones were really in autonomous mode, because only their operators would know for sure. This highlights a challenge of legal restrictions: unlike chemical weapons or landmines, autonomous weapons cannot easily be distinguished from operator-controlled ones.
鈥淭his does not show that autonomous weapons would be impossible to regulate,鈥� says Holland. 鈥淏ut it does show that the discussion continues to be urgent and important. The technology isn鈥檛 going to wait for us.鈥�
