IT decision makers and CIOs must start considering how to develop ethical programming for smart machines, according to Gartner.
Realising the potential of smart machines — and ensuring successful outcomes for the businesses that rely on them — will hinge on how trusted smart machines are and how well they maintain that trust.
Central to establishing this trust will be ethical values that people recognize and are comfortable with.
"Clearly, people must trust smart machines if they are to accept and use them," says Frank Buytendijk, research vice president and distinguished analyst, Gartner.
"The ability to earn trust must be part of any plan to implement artificial intelligence (AI) or smart machines, and will be an important selling point when marketing this technology.
"CIOs must be able to monitor smart machine technology for unintended consequences of public use and respond immediately, embracing unforeseen positive outcomes and countering undesirable ones."
To ensure that smart machines behave ethically and deserve our trust in the future, CIOs must prepare for ethical discussion and programming.
As a result, Gartner has identified five levels of programming and system development, based on their ethical impact:
Level 0: Non-Ethical Programming
On this level, there are no explicit ethical considerations for the behaviour of technology.
The technology manufacturer assumes very limited ethical responsibility, other than that the technology must provide the promised functions safely and reliably.
Examples include "vaporware" (a technology that is announced to the public but never delivered) that can reduce customer trust in a manufacturer.
The first release of any software is seldom complete, which means customers may have limited expectations of "version 1.0" software.
Gartner recommends that technology manufacturers communicate openly on what they will deliver and any changing circumstances, altering what can be delivered and what cannot.
This should include service-level agreements (SLAs) that specify what is delivered and how.
Level 1: Ethical Oversight
The next degree of sophistication has no ethical programming, but the deployment and use of technology may have ethical consequences.
Smart machines may be used, but it's essentially up to users what they do with the results. The main ethical responsibility is in the hands of those who use the smart machines.
Some companies have established an ethics board and some end-user organisations — particularly in financial services — have also established such boards but they are a small minority.
Gartner recommends that organisations establish governance practices that ensure no laws are broken, as a bare minimum.
They should also seek to make ethics a part of governance by identifying and discussing dilemmas posed by using new technologies.