Automatic Computer-based Decisions: Legal Status

Matt Warman Excerpts

Wednesday 10th March 2021 Westminster Hall

[The Parliamentary Under-Secretary of State for Digital, Culture, Media and Sport (Matt Warman)]

I congratulate the Chair of the Select Committee on securing this important debate. He is absolutely right to say that the potential of technology to enhance the decision-making process, in the public sector just as much as the private sector, is something that this Government are absolutely committed to not only getting the maximum out of, but getting right as well. He is also absolutely right to highlight that legislation from decades ago is perhaps not 100% where we would wish it to be.

First, let me say that I share the concerns raised by him and other Members about the specific example he has raised and the treatment of postmasters, who are vital members of the community, in this whole affair. I also acknowledge that it highlights essential legal issues. I will address those shortly, although I should perhaps start by saying that he has been comprehensive in his own circumnavigation of the issues at hand.

On Horizon, the Government recognise that the dispute has had a hugely damaging effect on the lives of the affected postmasters and their families. Its repercussions are still being felt today. Over the years, the Horizon accounting system recorded shortfalls in cash in branches. At the time, the Post Office believed that those shortfalls were caused by postmasters, leading to dismissals, recovery of losses and, in some cases, criminal prosecution. Many hon. Members, me included, have listened to the stories of the postmasters affected and have been deeply moved by the impact on their livelihoods, their finances and often their health.

A group of 555 of those postmasters, led by former postmaster Alan Bates, brought a group litigation claim against the Post Office in 2017. In the findings of Mr Justice Fraser, it is clear just how wrong the Post Office was in its relationship with postmasters and that there were clear failings in the Horizon system. As I will explain, the Government are taking steps through an independent inquiry to ensure that lessons are learned and that a full analysis takes place.

The Post Office reached a full and final settlement with the group litigation claimants in December 2019 and apologised for its failings. That settlement was an important step towards addressing the wrongs of the past, but it was only the start of a long journey for the Post Office to repair and strengthen its relationship with postmasters.

As part of the settlement, the Post Office agreed to set up the historical shortfall scheme, open to current and former postmasters who may have experienced and repaid Horizon shortfalls but did not participate in the group litigation. That is an important step in ensuring that all those who were affected have the opportunity to seek resolution.

A number of postmasters with criminal convictions have applied to the Criminal Cases Review Commission to have their cases referred for appeal. To date, the commission has referred 51 cases either to the Court of Appeal or to the Crown court. The Government welcome the decision made by the Crown court in December 2020 to overturn six of those convictions.

However, a number of cases—42 in total—are still to be heard in the relevant Appeal Court at the end of March. It would not be appropriate for the Government to comment on those cases while the courts are still considering them, but I assure hon. Members that the Post Office is co-operating with the commission to the fullest extent.

More broadly, we must ensure that such a situation can never be allowed to occur again. In September 2020, therefore, the Government launched the Post Office Horizon IT inquiry, an independent inquiry led by Sir Wyn Williams. Sir Wyn’s inquiry will work to understand fully what happened, gather available evidence and ensure that lessons have been learned so that this cannot occur again. The inquiry will look specifically at whether the historical shortfall scheme is being delivered properly. The Government look forward to receiving that report in the summer.

In recent years, however, a lot has changed on standards and ethics relating to the management of algorithms and data in general. The hon. Member for Bristol North West (Darren Jones) rightly pointed out the work of the Centre for Data Ethics and Innovation. Crucially, that centre has not only “data ethics” but “innovation” in its title—those two things go hand in hand.

The centre was established by my Department in 2017, but that is not the only area in which we have implemented change. Substantial steps have been taken to consider and address deficiencies in the application of algorithms where that lies within the remit of the DCMS and, crucially, beyond. I am confident that we are in a much stronger position than when the worst excesses of the Horizon affair took place, but there is more work to do.

If an automated decision is based on personal data, the UK general data protection regulation already applies. It provides regulatory tools to safeguard data subjects and identified or identifiable persons in automated decision making. Organisations processing personal data must also adhere to strong transparency requirements. Organisations, including public authorities, should ensure that the algorithms they deploy and procure, where based on personal data, generate sound and impartial decisions, and that that should be considered before such algorithms are used.

The UK GDPR contains provisions for protecting the interests of data subjects and their data. In particular, data protection impact assessments are mandatory for data processing that is high risk and require organisations to weigh up the impacts on privacy of data processing activities, including automated decision making.

In addition, the Government have introduced non-legislative tools that will be important as we move towards a world where not just algorithms but the ability for computers to amend algorithms—artificial intelligence—become more commonplace. Let me run through some of them. We were the first Government to publish a data ethics framework, which is a set of principles to guide the design of appropriate data use in the public sector, aimed at anyone working with data in the public sector. We published an ethics, transparency and accountability framework for automated decision making, and we have commissioned the Government Digital Service to deliver the review of artificial intelligence adoption in the public sector. We have also published an AI guide for Government.

There are also published guidelines on AI procurement in collaboration with the World Economic Forum’s Centre for the Fourth Industrial Revolution. It will inform and empower buyers in the public sector, helping them to evaluate suppliers and then confidently and responsibly procure the right AI technologies for the benefit of citizens. We have also published, along with the Information Commissioner’s Office and the Alan Turing Institute, “Explaining decisions made with AI”. This guidance gives organisations practical advice to help them explain the processes, services and decisions delivered or assisted by AI to the individuals affected by them. That is a crucial action that the hon. Member for Bristol North West mentioned.

Those various documents are updated with new thinking and insight from our public sector, civil society, industry and academic partners. We have also launched the new AI dynamic purchasing system, which is a framework that offers public sector customers a direct route to AI services in an emerging market, addressing ethical considerations when organisations buy AI services for use in the public sector.

The new and independent Regulatory Horizons Council has been appointed to scan the horizons for new technological innovations and provide the Government with impartial, expert advice on the regulatory reform required to support their rapid and safe introduction. More broadly, the Government are always monitoring how algorithms and data affect people’s lives. As they grow in importance in all our lives, we will consider what more we can do. That is why we are active in the international debates on algorithm and artificial intelligence regulations at the Council of Europe and, beyond that, at the OECD and in the Global Partnership on Artificial Intelligence.

The hon. Gentleman specifically asked whether the status of algorithms in the courts might be referred to the Law Commission, especially given the role played by the commission in first adjusting the Police and Criminal Evidence Act 1984 on this topic. It is a suggestion worth very serious consideration, and my colleagues in the Ministry of Justice and I are grateful for it. He will know that it is not in the Law Commission’s current three-year plan of work, and it will take considerable time to establish the necessary work in order to address the underlying legal issue.

While we consider that route, the Government are also investigating whether there may be faster methods that we can use to address the legal status of algorithms in a court of law—the hon. Gentleman mentioned that himself. For example, once the Court of Appeal has made a determination in respect to the Criminal Cases Review Commission, the judiciary Criminal Procedure Rule Committee could consider making changes in this area. The courts are expected to make their determination shortly, after which I look forward to taking up the matter with the Ministry of Justice and the Lord Chief Justice, the chair of that committee.

To close, I thank you, Mr Hollobone, and the hon. Gentleman. This is the beginning of the next phase in an ongoing debate. It is a hugely important issue, and seizing these opportunities for the benefit of citizens and everyone around the world is in all our interests. It will be a complex and involving conversation, and I look forward to having more conversations with the hon. Gentleman.

[Darren Jones]

rose—