The US Air Force can now equip autonomous drones with face recognition technology, raising fears that they could be used to find and kill specified people.
The drones will be employed by special operations forces for intelligence gathering and for missions in foreign countries, according to 聽and Seattle-based firm The company鈥檚 software, based on machine learning, is designed to work on a drone that is piloting itself, with limited or no communication with a human.
The contract has been fulfilled, but it isn鈥檛 known whether the software has yet been rolled out or what drones might be running it. RealNetworks said it was unable to comment.
Advertisement
A big area of concern, says at the University of California, Berkeley, lies in the contract document saying the software will 鈥渙pen the opportunity for real-time autonomous response by the robot鈥�.
鈥淗ard to see what else it refers to, other than lethal action,鈥� he says.
In 2017, Russell released a viral video presenting a fictional future in which small quadcopter drones called that were armed with explosive charges hunted down specific individuals using face recognition. The new project appears to make this a practical possibility, he says.
鈥淭he difference between a lethal autonomous weapon, and a weapon capable of assassinating individuals by facial recognition, is just one of software,鈥� says Russell. 鈥淚t鈥檚 probably not possible to detect or prevent software updates that would convert one to the other.鈥�
The US policy on lethal autonomous weapons, which the country in a conference in the Netherlands earlier this month, doesn鈥檛 require direct human supervision, but calls for 鈥渁ppropriate levels of human judgment鈥�. The Pentagon hasn鈥檛 explained what exactly this means.
Russell says a drone finding, identifying and attacking someone from a list of targets might be allowed under these guidelines. 鈥淪ome might argue it鈥檚 not the weapon that鈥檚 selecting the target, but the human operator 鈥榩re-approving鈥�.鈥�
, director of the artificial intelligence governance project at the Center for Strategic and International Studies think tank in Washington DC, doesn鈥檛 believe the technology is intended for robotic assassination. 鈥淚 sincerely doubt this has anything to do with lethal autonomous capabilities,鈥� he says. 鈥淚 have never met anyone in the DoD who is even slightly interested in using facial recognition for autonomous anti-personnel targeting capabilities.鈥�
A drone鈥檚 recognition of a face might not be the sole factor used in identifying whether someone is a person of interest. The military and police services also use other methods, such as the聽 known to be associated with someone. But regardless of whether the drones act against a target or just raise an alert, there is a risk they could identify the wrong person.
In theory, automated face recognition should be capable of identifying subjects with a high degree of confidence, says at the University of Lincoln, UK. 鈥淭he world standard test of face recognition algorithms now puts the top algorithms as having a false positive rate 鈥� that is, incorrectly identifying the wrong person 鈥� of less than 0.5 per cent.鈥�
However, these results rely on high-quality images. Achieving good results with a small drone camera at long range and in variable lighting would be more challenging, she says. 鈥淭he algorithm would need to be specifically trained to function well under these conditions.鈥�
There may be other issues, too, making misidentification more likely. Some algorithms are less effective at distinguishing between people of certain ethnicities, says Ritchie.
鈥淭he main point here is that it is so important for users of face recognition technology to understand the capabilities of the algorithm they are using,鈥� she says.
A spokesperson for the says: 鈥淒etails such as those involving safety or other operational measures聽involving this capability are not releasable with respect to operational聽security considerations. However, DoD personnel are directed to exercise appropriate levels of judgement and care, while remaining responsible for the development, deployment, and use of AI capabilities.鈥�
