Ethics and Artificial Intelligence

Debate between Jo Swinson and Lee Rowley

Wednesday 17th January 2018 Westminster Hall

[Jo Swinson (East Dunbartonshire) (LD)]

I beg to move,

That this House has considered ethics and artificial intelligence.

It is a pleasure to serve under your chairmanship, Dame Cheryl. I welcome the Minister to her new role, following the reshuffle last week. She leaves what was also a wonderful role in Government—I can say that from personal experience—but I am sure that she will find the challenges of this portfolio interesting and engaging. No doubt she is already getting stuck in.

I would like to start with the story of Tay. Tay was an artificial intelligence Twitter chatbot developed by Microsoft in 2016. She was designed to mimic the language of young Twitter users and to engage and entertain millennials through casual and playful conversation.

“The more you chat with Tay the smarter she gets”,

the company boasted. In reality, Tay was soon corrupted by the Twitter community. Tay began to unleash a torrent of sexist profanity. One user asked,

“Do you support genocide?”,

to which Tay gaily replied, “I do indeed.” Another asked,

“is Ricky Gervais an atheist?”

The reply was,

“ricky gervais learned totalitarianism from adolf hitler, the inventor of atheism”.

Those are some of the tamer tweets. Less than 24 hours after her launch, Microsoft closed her account. Reading about it at the time, I found the story of Tay an amusing reminder of the hubris of tech companies. It also reveals something darker: it vividly demonstrates the potential for abuse and misuse of artificial intelligence technologies and the serious moral dilemmas that they present.

I say at the outset that I believe artificial intelligence can be a force for good, if harnessed correctly. It has the potential to change lives, to empower and to drive innovation. In healthcare, the use of AI is already revolutionising the way health professionals diagnose and treat disease. In transport, the rise of autonomous vehicles could drastically reduce the number of road deaths and provide incredible new opportunities for millions of disabled people. In our everyday lives, new AI technologies are streamlining menial tasks, giving us more time in the day for meaningful work, for leisure or for our family and friends. We are on the cusp of something quite extraordinary and we should not aim deliberately to suppress the growth of new AI, but there are pressing moral questions to be answered before we jump head first into AI excitement. It is vital that we address those urgent ethical challenges presented by new technology.

I will focus on four important ethical requirements that should guide our policy making in this area: transparency, accountability, privacy and fairness. I stress that the story of Tay is not an anomaly; it is one example of a growing number of deeply disturbing instances that offer a window into the many and varied ethical challenges posed by advances in AI. How should we react when we hear than an algorithm used by a Florida county court to predict the likelihood of criminals reoffending, and therefore to influence sentencing decisions, was almost twice as likely to wrongly flag black defendants as future criminals?

[Lee Rowley (North East Derbyshire) (Con)]

I congratulate the hon. Lady on this debate; it is a fascinating area and I am grateful to be able to speak. On her last point, I understand that in parts of the United States where that technology is used, there are instances where the judges go one step further and rely on those decisions as reasons to do things. The decision is made on incorrect information in the first instance, and then judges say that because a machine has made that decision, it must be even better than manual intervention.

[Jo Swinson]

The hon. Gentleman is quite right to raise that concern, because that goes to the heart of the issue, particularly when risk data is presented as incontrovertible fact and is relied on for the decision. It is absolutely essential that those decisions can be interrogated and understood, and that any bias is identified. That is why ethics must be at the heart of this whole issue, even before systems are developed in the first place.

In addition to the likely reoffending data, there is a female sex robot designed with a “frigid” setting, which is programmed to resist sexual advances. We have heard about a beauty contest judged by robots that did not like the contestants with darker skin. A report by PwC suggests that up to three in 10 jobs in this country could be automated by the early 2030s. We have read about children watching a video on YouTube of Peppa Pig being tortured at the dentist, which had been suggested by the website’s autoplay algorithm. In every one of those cases, we have a right to be concerned. AI systems are making decisions that we find shocking and unethical. Many of us will feel a lack of trust and a loss of control.

On machine learning, a report last year by the Royal Society highlighted a range of concerns among members of the public. Some were worried about the potential for direct harm, from accidents in autonomous vehicles to the misdiagnosis of disease in healthcare. Others were more concerned about potential job losses or the perceived loss of humanity that could result from wider use of machine learning. The importance of public engagement and dialogue was acknowledged by the Minister’s Department in its 2016 report. I would welcome an update from her on the kind of public engagement work she thinks is important with regard to AI.

I will turn to the related considerations of transparency and accountability. When we talk about transparency in the context of AI, what we really mean is that we want to understand how AI systems think and to understand their decision-making processes. We want to avoid situations of “black-boxing”, where we cannot understand, access or explain the decisions that technology makes. In practice, that transparency means several things: it might involve creating logging mechanisms that give us a step-by-step account of the processes involved in the decision making; or it could mean providing greater visibility of data access. I would be interested to hear the Minister’s thoughts on the relative merits of those practices. Either way, transparency is particularly important for those instances when we want to challenge decisions made by AI systems. Transparency informs accountability. If we can see how decisions are made, it is easier for us to understand what has happened and who is responsible when things go wrong.

Increasingly, major companies such as Deutsche Bank and Citigroup are turning to machine learning algorithms to streamline and refine their recruitment processes. Let us suppose that we suspect that an algorithm is biased towards candidates of a particular race and gender. If the decision-making process of the algorithm is opaque, it is hard to even work out whether employment law is being broken—an issue I know will be close to the Minister’s heart. Transparency is crucial when it comes to the accountability of new AI. We must ensure that when things go wrong, people can be held accountable, rather than shrugging and responding that the computer says “don’t know”.

[Lee Rowley]

I will try not to intervene too much, but the point about transparency in the process and the decision making relates to the data that is used as an input. It is often the case in these instances that machine learning is simply about correlations and patterns in a wide scheme of data. If that data is not right in the first instance, subjective and inaccurate decisions are created.

[Jo Swinson]

I entirely concur; one of the long-standing rules of computer programming is “garbage in, garbage out”. That holds true here. Again, that is why transparency about what goes in is so important. I hope that the Minister will tell us what regulations are being considered to ensure that AI systems are designed in a way that is transparent, so that somebody can be held accountable, and how AI bias can be counteracted.

Increased transparency is crucial, but it is also vital that we put safeguards in place to make sure that that does not come at the cost of people’s privacy or security. Many AI systems have access to large datasets, which may contain confidential personal information or even information that is a matter of national security. Take, for example, an algorithm that is used to analyse medical records: we would not want that data to be accessible arbitrarily by third parties. The Government must be mindful of privacy considerations when tackling transparency, and they must look at ways of strengthening capacity for informed consent when it comes to the use of people’s personal details in AI systems.

We must ensure that AI systems are fair and free from bias. Returning to recruitment, algorithms are trained using historical data to develop a template of characteristics to target. The problem is that historical data itself often reveals pre-existing biases. Just a quarter of FTSE 350 directors are women, and fewer than one in 10 are from an ethnic minority; the majority of leaders are white men. It is therefore easy to see how companies’ use of hiring algorithms trained on past data about the characteristics of their leaders might reinforce existing gender and race imbalances.

The software company Sage has developed a code of practice for ethical AI. Its first principle stresses the need for AI to reflect the diversity of the users it serves. Importantly, that means ensuring that teams responsible for building AI are diverse. We all know that the computer science industry is heavily male dominated, so the people who develop AI systems are mainly men. It is not hard to see how that might have an impact on the fairness of new technology. Members may remember that Apple launched a health app that enabled people to do everything from tracking their inhaler use to tracking how much molybdenum they were getting from their soy beans, but did not allow someone to track their menstrual cycle.

We also need to be clear about who stands to benefit from new AI technology and to think about distributional effects. We want to avoid a situation where power and wealth lie exclusively in the hands of those with access to and understanding of these new technologies.