Automatic Computer-based Decisions: Legal Status

Debate between Matt Warman and Darren Jones

Wednesday 10th March 2021 Westminster Hall

[The Parliamentary Under-Secretary of State for Digital, Culture, Media and Sport (Matt Warman)]

I congratulate the Chair of the Select Committee on securing this important debate. He is absolutely right to say that the potential of technology to enhance the decision-making process, in the public sector just as much as the private sector, is something that this Government are absolutely committed to not only getting the maximum out of, but getting right as well. He is also absolutely right to highlight that legislation from decades ago is perhaps not 100% where we would wish it to be.

First, let me say that I share the concerns raised by him and other Members about the specific example he has raised and the treatment of postmasters, who are vital members of the community, in this whole affair. I also acknowledge that it highlights essential legal issues. I will address those shortly, although I should perhaps start by saying that he has been comprehensive in his own circumnavigation of the issues at hand.

On Horizon, the Government recognise that the dispute has had a hugely damaging effect on the lives of the affected postmasters and their families. Its repercussions are still being felt today. Over the years, the Horizon accounting system recorded shortfalls in cash in branches. At the time, the Post Office believed that those shortfalls were caused by postmasters, leading to dismissals, recovery of losses and, in some cases, criminal prosecution. Many hon. Members, me included, have listened to the stories of the postmasters affected and have been deeply moved by the impact on their livelihoods, their finances and often their health.

A group of 555 of those postmasters, led by former postmaster Alan Bates, brought a group litigation claim against the Post Office in 2017. In the findings of Mr Justice Fraser, it is clear just how wrong the Post Office was in its relationship with postmasters and that there were clear failings in the Horizon system. As I will explain, the Government are taking steps through an independent inquiry to ensure that lessons are learned and that a full analysis takes place.

The Post Office reached a full and final settlement with the group litigation claimants in December 2019 and apologised for its failings. That settlement was an important step towards addressing the wrongs of the past, but it was only the start of a long journey for the Post Office to repair and strengthen its relationship with postmasters.

As part of the settlement, the Post Office agreed to set up the historical shortfall scheme, open to current and former postmasters who may have experienced and repaid Horizon shortfalls but did not participate in the group litigation. That is an important step in ensuring that all those who were affected have the opportunity to seek resolution.

A number of postmasters with criminal convictions have applied to the Criminal Cases Review Commission to have their cases referred for appeal. To date, the commission has referred 51 cases either to the Court of Appeal or to the Crown court. The Government welcome the decision made by the Crown court in December 2020 to overturn six of those convictions.

However, a number of cases—42 in total—are still to be heard in the relevant Appeal Court at the end of March. It would not be appropriate for the Government to comment on those cases while the courts are still considering them, but I assure hon. Members that the Post Office is co-operating with the commission to the fullest extent.

More broadly, we must ensure that such a situation can never be allowed to occur again. In September 2020, therefore, the Government launched the Post Office Horizon IT inquiry, an independent inquiry led by Sir Wyn Williams. Sir Wyn’s inquiry will work to understand fully what happened, gather available evidence and ensure that lessons have been learned so that this cannot occur again. The inquiry will look specifically at whether the historical shortfall scheme is being delivered properly. The Government look forward to receiving that report in the summer.

In recent years, however, a lot has changed on standards and ethics relating to the management of algorithms and data in general. The hon. Member for Bristol North West (Darren Jones) rightly pointed out the work of the Centre for Data Ethics and Innovation. Crucially, that centre has not only “data ethics” but “innovation” in its title—those two things go hand in hand.

The centre was established by my Department in 2017, but that is not the only area in which we have implemented change. Substantial steps have been taken to consider and address deficiencies in the application of algorithms where that lies within the remit of the DCMS and, crucially, beyond. I am confident that we are in a much stronger position than when the worst excesses of the Horizon affair took place, but there is more work to do.

If an automated decision is based on personal data, the UK general data protection regulation already applies. It provides regulatory tools to safeguard data subjects and identified or identifiable persons in automated decision making. Organisations processing personal data must also adhere to strong transparency requirements. Organisations, including public authorities, should ensure that the algorithms they deploy and procure, where based on personal data, generate sound and impartial decisions, and that that should be considered before such algorithms are used.

The UK GDPR contains provisions for protecting the interests of data subjects and their data. In particular, data protection impact assessments are mandatory for data processing that is high risk and require organisations to weigh up the impacts on privacy of data processing activities, including automated decision making.

In addition, the Government have introduced non-legislative tools that will be important as we move towards a world where not just algorithms but the ability for computers to amend algorithms—artificial intelligence—become more commonplace. Let me run through some of them. We were the first Government to publish a data ethics framework, which is a set of principles to guide the design of appropriate data use in the public sector, aimed at anyone working with data in the public sector. We published an ethics, transparency and accountability framework for automated decision making, and we have commissioned the Government Digital Service to deliver the review of artificial intelligence adoption in the public sector. We have also published an AI guide for Government.

There are also published guidelines on AI procurement in collaboration with the World Economic Forum’s Centre for the Fourth Industrial Revolution. It will inform and empower buyers in the public sector, helping them to evaluate suppliers and then confidently and responsibly procure the right AI technologies for the benefit of citizens. We have also published, along with the Information Commissioner’s Office and the Alan Turing Institute, “Explaining decisions made with AI”. This guidance gives organisations practical advice to help them explain the processes, services and decisions delivered or assisted by AI to the individuals affected by them. That is a crucial action that the hon. Member for Bristol North West mentioned.

Those various documents are updated with new thinking and insight from our public sector, civil society, industry and academic partners. We have also launched the new AI dynamic purchasing system, which is a framework that offers public sector customers a direct route to AI services in an emerging market, addressing ethical considerations when organisations buy AI services for use in the public sector.

The new and independent Regulatory Horizons Council has been appointed to scan the horizons for new technological innovations and provide the Government with impartial, expert advice on the regulatory reform required to support their rapid and safe introduction. More broadly, the Government are always monitoring how algorithms and data affect people’s lives. As they grow in importance in all our lives, we will consider what more we can do. That is why we are active in the international debates on algorithm and artificial intelligence regulations at the Council of Europe and, beyond that, at the OECD and in the Global Partnership on Artificial Intelligence.

The hon. Gentleman specifically asked whether the status of algorithms in the courts might be referred to the Law Commission, especially given the role played by the commission in first adjusting the Police and Criminal Evidence Act 1984 on this topic. It is a suggestion worth very serious consideration, and my colleagues in the Ministry of Justice and I are grateful for it. He will know that it is not in the Law Commission’s current three-year plan of work, and it will take considerable time to establish the necessary work in order to address the underlying legal issue.

While we consider that route, the Government are also investigating whether there may be faster methods that we can use to address the legal status of algorithms in a court of law—the hon. Gentleman mentioned that himself. For example, once the Court of Appeal has made a determination in respect to the Criminal Cases Review Commission, the judiciary Criminal Procedure Rule Committee could consider making changes in this area. The courts are expected to make their determination shortly, after which I look forward to taking up the matter with the Ministry of Justice and the Lord Chief Justice, the chair of that committee.

To close, I thank you, Mr Hollobone, and the hon. Gentleman. This is the beginning of the next phase in an ongoing debate. It is a hugely important issue, and seizing these opportunities for the benefit of citizens and everyone around the world is in all our interests. It will be a complex and involving conversation, and I look forward to having more conversations with the hon. Gentleman.

[Darren Jones]

rose—

5G Network and Huawei

Debate between Matt Warman and Darren Jones

Monday 27th January 2020 Commons Chamber

[Matt Warman]

My right hon. Friend is very experienced in these matters and gets right to the heart of the issue. The issue of this country’s relationship with other countries of varying friendliness around the world will only become more pressing. We have to make the right decision now.

[Darren Jones (Bristol North West) (Lab)]

I declare my interest as set out in the Register of Members’ Financial Interests.

The issue of internet-connected devices in our critical national infrastructure is related not just to 5G and Huawei, but to water, electricity and supermarket food distribution systems—every part of our way of life. Yet we are caught in the middle of a China, European Union and United States policy approach to developing these technologies. The Minister has been asked a few times today—he has not quite answered the question—what representations he has made to the Department for Business, Energy and Industrial Strategy to include in the Government’s industrial strategy sovereign capability in the manufacturing of technologies. We want absolute reassurance that technologies are safe in our critical infrastructure.

[Matt Warman]

I hoped that I had hinted at an answer earlier. The hon. Gentleman is absolutely right that Britain has to have an eye on the importance to our strategic interests of certain areas of our economy and of certain small companies growing in this country. We will continue to do that. It is a statement of the obvious that the areas where we will have to take an interest will grow over time.

Cashless Transactions

Debate between Matt Warman and Darren Jones

Tuesday 22nd January 2019 Westminster Hall

[Matt Warman]

The hon. Gentleman is currently right, but we will all be better off if we hasten the transition, so that people do not have to rely on cash and all of our constituents can use more secure and efficient digital means, whether they live in rural areas such as my constituency and his, or in big cities.

[Darren Jones (Bristol North West) (Lab)]

I declare my interest in this issue. Does the hon. Gentleman agree that the increasing amount of online fraudware—for example, one of my constituents was defrauded of £40,000 by telephone fraud using her online banking—and the use of crypto-assets for online cashless transactions means that we need more regulation from the Government, so that consumers can trust the system to enforce the rules? Do we not need to give police more powers to tackle those who use cashless transactions for criminal purposes?

[Matt Warman]

I completely agree. Much of this is about trust. It is true, as I said, that 46% of people do not trust a cashless society, whether that involves crypto-currencies—although I suspect most of those 46% are not wholly familiar with every detail of that—or simply contactless cards. Part of this is about regulation; much of it is about trust.

It is worth examining in a small amount of detail why people want to continue to use cash. About 20% say it makes them feel more in control, 16% say it helps them with budgeting, and 3% want to hide their guilty pleasures—perhaps we had best not dwell on those. Some 5% do not trust online transactions at all, and 3% just do not trust the banks. That is a real issue if this trend is to be mitigated for vulnerable groups where needed and if the broader society is to take advantage of cashless transactions.

Countries such as Sweden already have twice the level of cashless transactions as we have in the UK, and their authorities have taken conscious action to slow down the pace of cashless-ising, to ensure that vulnerable groups are not left behind. It is also reported that 4,000 people in Sweden have had chips inserted under their skin, so that they do not even have to use cards—I am not sure that I would go quite that far, although I know the Minister might want to consider it as a personal experiment.

We are at risk of ending up with two cultures: those who embrace a wholly digital way of living, and those who do not. There has been an 8.5% decline in the use of ATMs in London, but just a 2% decline in Northern Ireland and a 2.9% decline in my area of the east midlands. There are very different views on what is important for people and on the pace of change.

It is instructive to look at what people use cash for. Figures from the LINK report show that just 13% of people pay their rent in cash—disproportionately those on lower incomes. Some 85% said that they use cash to pay for taxis, which is a particularly instructive example. That is obviously a nationwide figure rather than a conversation about London taxis, on which we could perhaps spend many hours. Taxis are a particularly interesting example because the giving of cash to a driver makes them more vulnerable to theft and to being a target for crime. It also means that they are responsible for ensuring that they have change, so they have to carry a float even before they have taken any cash. It is of course true to say that it would be naive in today’s society to get into a cab outside London and expect the driver to take a card transaction. This is a complex landscape.

Some four out of five people say that they pay their gardener—if they have one—in cash. I am sure that neither the Minister nor I wish to cast aspersions on gardeners, but there is a suspicion that there are parts of the economy where cash is used to avoid the taxation that I know he is very eager to collect at every possible opportunity. There are a whole host of reasons to promote cashless transactions, whether it is ensuring that people are at less risk of the crime that goes with cash or that businesses are at less risk of the increasingly expensive costs of handling cash.

[Matt Warman]

I agree. In a sense, I make the same point that I made before, which is that currently there are a large number of exceptions and it would actually be in the interests of the many seasonal, low-paid and often zero-hours contract workers in my constituency to be paid digitally, because they would be less at risk of crime and the businesses that they work for would have less of the handling costs associated with cash. We are already at the point where the declining ATM network that those people rely on is struggling to make a viable business case to those who use it with such diminishing enthusiasm.

[Darren Jones]

The case study of First Bus in Bristol—I would not speak accurately on behalf of my constituents if I said it was a cheap network—demonstrates that the use of digital payments has allowed a reduction in the cost of tickets because cash is no longer handled on the bus network. Does the hon. Gentleman recognise that that must be a positive for consumers, whether or not they use cash?

[Matt Warman]

I agree. That is an example of where digital payments are in the interests of both the business and the consumer—not simply in terms of convenience, but, in the case the hon. Gentleman mentioned, in terms of the viability of the bus route. Companies such as Square have tried to encourage marketplaces to embrace digital technology in such a way that businesses that often operate at the very limit of viability are able to find new and innovative ways of changing their business models slightly. Of course, it is no good for a business to accept cards only when people wish to show up and pay in cash. The London bus network experienced some controversy when it moved to using a card-only payment mechanism, and I suspect that the hon. Gentleman may have experienced the same in his constituency.

As the hon. Member for Strangford said, it comes back to ensuring that a one-size-fits-all force is not exerted either by Government or by business, but that a number of things are done to encourage businesses to understand that there are huge advantages, and to encourage consumers to understand that there are very limited opportunities for fraud when it comes to contactless cards or, indeed, chip and pin. We should be careful not to stoke those sorts of fears unnecessarily.

There are costs associated with handling cash and there are associated costs for consumers—socioeconomic risks for those who are often most reliant on cash but would benefit most from digital payments, for example. We therefore need to ask the Minister for a number of things from the perspective of Government. The first would be to consider whether action should be taken to encourage shops and businesses to go cashless and whether there should be a safety net to ban them from becoming entirely cashless, as has happened in some states in America. I would argue that in most circumstances, a retailer of any kind is perfectly entitled to make that decision for itself. We should not sleepwalk into a cashless society, however; we should understand the risks and the benefits.

With that in mind, my single largest request to the Minister is not that we try to invent some system in which, as someone proposed, a business rates scheme encourages cashless-ising in town centres, to encourage businesses to become more viable and eliminate costs to which they might be sentimentally attached, but simply to build on the success that Britain has already shown both in FinTech and in technology more generally. That means putting the governmental shoulder to the wheel and recognising that ultimately, the cost of taking cash is already very close—if not over the line—to outweighing the benefits. We are certainly already at the point where many businesses looking at that in detail would think very hard about whether taking cash was in their interests at all. Clearly, the moment when far more businesses go over that line is fast approaching.

We need to do two things. The first is to consider much more carefully the impact that eliminating cash can have on vulnerable groups in certain circumstances. Then, we must say what we can do to help those groups to embrace a cashless economy with much greater enthusiasm. Some of that means reassuring people about their concerns on the risks of fraud, while another part means defining—as the hon. Member for Strangford mentioned—what can always be paid in cash, at least for a long time into the future.

Would the Minister consider setting out a roadmap for the future proportion of cash in the economy, to reassure us that it has been entirely considered and that its impact has been thought through? If I had my way, I would set a target for the elimination of cash from the economy, in much the same way that—in an ideal world—I would think carefully about when cheques might be eliminated entirely from the economy. I appreciate that setting any type of target in that regard is probably down to the market as much as it is to Government, but whether or not there is a target, the trajectory is ultimately very clear: there will be less and less cash in the economy and that will mean certain things for vulnerable groups. Our job as politicians is to provide some leadership and tell people and businesses that it is economically advantageous for there to be less cash in the economy, however sentimental some people might feel about where we are in today’s society.

I will leave that gentle request there, rather than demand anything more of the Minister, but I would say that the direction of travel is clear: cash was king, but it is now coming towards the end of its reign.