HAVE you been turned down by a computer? Perhaps your online credit application was refused. Maybe you were denied a job, or even parole.
the right to ask the inscrutable algorithms involved to explain themselves.
In April this year, the European parliament approved the General Data Protection Regulation (GDPR), a new set of rules governing personal data. Due to go into , it introduces a 鈥渞ight to explanation鈥�: the opportunity for European Union citizens to question the logic of an algorithmic decision 鈥� and contest the results.
Advertisement
We should cheer this development. The world is increasingly run by algorithms that calculate credit scores, read medical scans, drive our cars and tell our police forces where to patrol. But algorithms can behave in mysterious ways, sometimes even surprising the programmers who created them. It鈥檚 crucial that ordinary people whose lives they affect have the ability to examine and challenge decisions.
The GDPR is a significant step forward compared with existing laws, says Bryce Goodman at the University of Oxford鈥檚 Internet Institute. It creates new rules about how data is used and explicitly states how they affect any company working with data belonging to European citizens, whether or not that company is based in Europe. (We鈥檙e looking at you, Google.)
鈥淎n arbitrary decision made by a designer in Silicon Valley dictates the policies we live our lives by鈥�
It also has teeth. Organisations in breach of the GDPR can expect fines of up to 4 per cent of their yearly turnover or 鈧�20 million 鈥� whichever is greater.
The GDPR also specifically calls for companies to prevent discrimination based on personal characteristics such as race, religious beliefs or health data. This matters because experiments already show that online ad services preferentially show details of higher-paying jobs to male users; criminal justice algorithms suggest harsher sentences for African Americans.
Technology didn鈥檛 create institutionalised discrimination. But, filtered through algorithms, familiar biases can be hidden behind the guise of mathematical impartiality.
Concerns about algorithmic prejudice resonated at a White House symposium on artificial intelligence held in New York City last week. 鈥淲e live in a technocracy,鈥� of Harvard University told the meeting. 鈥淎n arbitrary decision made by a designer in Silicon Valley dictates the policies we live our lives by.鈥� Protection requires independent oversight.
Welcome as it is, the GDPR won鈥檛 be easy to enforce. In a draft paper published last month, Goodman and colleague Seth Flaxman identified some of the obstacles to making a .
For example, machine learning algorithms spit out results by giving more weight to certain factors and making calculations that even programmers struggle to articulate. Who will explain their opaque workings to those who aren鈥檛 technically literate?
But it鈥檚 important to try. 鈥淗istory teaches us that human decisions can all too easily be biased, whether consciously or unconsciously,鈥� said Ed Felten, deputy chief technology officer at the White House Office of Science and Technology Policy. 鈥淎s we build automated systems, we have a responsibility to do better.鈥�
This article appeared in print under the headline 鈥淚nterrogating the algorithms鈥�
