Worries about artificial intelligence seem to be everywhere. Will our jobs survive? Can we trust thinking machines to do the right thing? Common concerns such as these put AI high up the agenda at last week鈥檚 meeting of the in Switzerland, a gathering of the world鈥檚 business and political elites.
To coincide with this, IBM, one of the leading developers of these kind of systems, used the spotlight to try to reassure us that the new 鈥渃ognitive era鈥� of computing will be smooth and safe.
In a , the technology giant proposed three guiding principles: the purpose of these systems must be to support and extend human capabilities, not supplant them; trust in machine decisions requires transparency on how they are trained and how they work; and human skills must be expanded to use and shape such systems.
Advertisement
I鈥檝e been writing and speaking about the ethical development and deployment of AI for years, and I was pleased to see IBM鈥檚 statement. These principles are thoughtful and very much on-target. However, there are reasons why they may not be enough to allay worries until this technology is more mature and better understood.
Weak AI
Underpinning IBM鈥檚 statement is its declaration that 鈥渃ognitive systems will not realistically attain consciousness or independent agency鈥�. At a time when luminaries such as Stephen Hawking, Elon Musk and Bill Gates worry out loud about the risks of too-powerful artificial general intelligence, reassurances like this are welcome. But this implicitly downplays the : machine analysis and recommendations within a narrow field but with deep knowledge may be counter-productive, even dangerous, in a wider context.
As a relatively benign example, a system designed to diagnose disease might not concern itself with costs of care, differential treatment of minority groups, or availability of skilled specialists.
More troublingly, philosophers concerned with artificial intelligence often bring up unintentionally malevolent outcomes of poorly designed narrow AI, such as the 鈥溾�� problem; this imagines an AI that causes global harm in its unrelenting pursuit of stationery. An intelligence without consciousness is not automatically devoid of dangers.
Secondly, as we鈥檝e seen time and again, people often trust a superficially intelligent computer over their own instincts and knowledge, and can ascribe an unwarranted objectiveness to the system 鈥� the machine must be right. But even if a computer reaches its conclusions by 鈥渓earning鈥� on its own, humans and institutions select the sources and methods upon which the system relies. This can introduce unintended cultural biases. Cognitive computing may be brilliant, but it won鈥檛 be objective.
Public trust
As these systems mature 鈥� along with our understanding of how to use them appropriately 鈥� these issues of narrow intelligence without context and embedded subjectivity will be more easily recognised and handled. But while we are still figuring out how best to develop, deploy and make use of cognitive systems, such problems could have a disproportionate impact, not least on public trust.
The era of cognitive computing will, in the long run, be a good thing. The ability of machines to learn and reason, coupled with real-time access to petabytes of data, will allow our digital systems to provide unprecedentedly deep and useful analysis. We鈥檙e already seeing this emerge in the realm of diagnostic medicine, where cognitive systems increasingly do better than human doctors in identifying a patient鈥檚 disease, not least Watson, IBI鈥檚 AI.
But as with most fundamental shifts 鈥� in this case, from human-driven to machine-driven analysis 鈥� the transition is likely to be surprisingly awkward and occasionally painful, even with the best principles.
Jamais Cascio is a distinguished fellow at the and writes about the impact of innovation
听
