(1 month, 1 week ago)
Lords ChamberEssentially, I agree with the right reverend Prelate the Bishop of Hereford. I could almost leave it there, but I will briefly say, in the spirit of the amendments, that the tablers are right to raise general concerns about the possibility of abuse through bias—as we heard from the noble Baroness, Lady O’Loan—and hallucination. After all, we have had the first high-profile resignation of a public sector leader in the form of the chief constable of the West Midlands praying in aid the fabrication of a non-existent football match as the reason why Parliament was misled.
In addition to bias and hallucination, there is the risk of what is called scheming. The results from some of the LLMs—published, for example, in the journal Nature in October—show some pretty disturbing examples. In the article, headed “AI Models that Lie, Cheat, and Plot Murder”, there are examples where models have attempted to write self-propagating worms, fabricate legal documentation and leave hidden notes to future instances of themselves. The punchline, essentially, is that, in regard to some of these technologies,
“the world is in a lucky period in which models are smart enough to scheme but not smart enough to escape monitoring”.
That is scary because, in five years’ time, that may no longer be true. So there are good reasons for generalised concerns about AI and wanting to circumscribe the role it might play in this legislation.
However, for the reasons that others have mentioned— I suspect the noble Baroness, Lady Coffey, herself would accept this—this probing amendment is written too broadly. It says:
“Artificial intelligence must not be used to carry out any functions in any section or schedule of this Act”.
Given that, for example, under Clauses 5(5) and 12(2), a doctor has to discuss with a person their diagnosis, their prognosis, any treatments available, the likely effects of them, and palliative, hospice and other care, it is highly likely that those will be informed by machine learning. It will interpret, for example, CT scans or MRIs, and AI tools will personalise and optimise therapies, potentially with predictive AI for better prognosis. So, were this to come back on Report, there would be a good case for ensuring greater precision in the firepower that is aimed at this particular concern.
However, all that should not in any way excuse or divert us from an equivalent worry: we must not kid ourselves that the gold standard is human expert judgment on many of the questions posed by the Bill. As we discussed, Clause 2(1)(b) requires an assessment of whether somebody with a terminal illness will live longer than six months. Unfortunately, as we have heard, that turns out to be a clinically irrelevant threshold that is very hard for expert judgment to get right.
I have just pulled the data from a large study looking at 98,000 people across London over the last decade and at prognostic accuracy, and the answer was that clinicians were able to be accurate about whether somebody was going to live for two weeks with about 74% accuracy, and they were able to be accurate about whether somebody was going to live more than a year with 83% accuracy, but, in terms of being able to predict whether somebody is going to survive for weeks or months, accuracy was only 32%. So, whatever our concerns about AI, human expert judgment, which underpins the Bill, is itself highly fallible.
To follow on from that, as my noble friend said right at the beginning, the amendment was put down in such a blunt fashion absolutely to stimulate this sort of debate. What has been really useful in this debate is finding that there is a broad degree of consensus that AI can be valuable as an input to decision-making, but it should not be used as the output: as the final decision-maker. As mentioned, AI can detect the progression of cancers and can probably do better prognosis or improve, especially over the time that we are looking at here, so that you can get better assessments of how long someone is likely to live.
On the AI in the chat box, there are very many instances where it could be very useful in terms of detecting coercion if it is talking to someone over quite a long period of time. Therefore, in all of this we see that, with inputs to the decision-making process, AI has a valuable part to play, but I think we would also absolutely agree that the final decision-maker in terms of an output clearly has to be a human; obviously they will be armed with the inputs from AI, but the human will make the final decision. I think that is what the Bill does, if I am correct, in that it is very clear that the decision-makers, the panels, the doctors and everything are those people, but at the same time—although I guess the Bill is silent on this—obviously it enables AI as an input.
I hope this debate is useful in that it shows a degree of consensus and that in this instance we probably have the right balance, but, again, I would be interested to hear from the Bill sponsor in his response whether that is the case.
(2 years, 3 months ago)
Lords ChamberMy Lords, the Government are committed to giving patients better, more joined-up healthcare services. To do so, we need to ensure that we have the right procurement regime so that the NHS can best allocate resources which meet the needs of patients. These regulations do that. They would establish the provider selection regime on 1 January 2024.
This House knows that the challenges we face as a country are changing, and the NHS is changing to address them—an ageing population, an increase in people with multiple health conditions, and persistent inequalities in health outcomes. We must respond to these challenges. To meet them, we need to provide an enabling and empowering framework that allows the NHS to combine the value of competition with the benefits of collaboration in the interests of patients.
In March last year, the Health and Care Act 2022 was passed. It sought to bring together NHS organisations and partners to tackle issues in our health and care system. This instrument builds on that progress. In 2019, engagement across the NHS identified that the use of the current rules on procurement presented a bureaucratic barrier to bringing NHS organisations and partners together. NHS colleagues wanted a framework that allowed them to use the right approach for different scenarios; a framework that included competition without defaulting to it and which supported the increased need for the alignment of services, including those provided by non-statutory organisations in the voluntary sector, to join up care for patients. The Government developed the legislative framework in the light of these requests. Furthermore, in June 2019, the Health and Social Care Committee also agreed that this was the right approach to
“ease the burden procurement rules have placed on the NHS, ensuring commissioners have discretion over when to conduct a procurement process”.
As our colleagues in the NHS and across the health system have emphasised, we must seek to balance a system-driven approach to planning services while recognising the importance of provider diversity for service innovation and value. That is also why my officials have worked closely with a broad range of colleagues and organisations across the system, including both commissioners and providers of healthcare services, to prepare the instrument before you today. This work has included extensive consultation. In 2021, NHS England published a consultation on the detail of the policy behind this instrument. Of 420 responses received from NHS representative bodies and individuals, 70% of respondents agreed or strongly agreed with the detailed proposals set out in that consultation. In 2022, the department published a further consultation to help inform the detail of our regulations.
Finally, we have not neglected to do the analysis of impacts associated with this regime change. Our voluntary impact assessment shows that, in the most likely scenarios, introducing this instrument will deliver savings to the NHS by reducing bureaucracy. Although it is difficult to provide a precise figure ahead of monitoring this regime, those noble Lords who have read the assessment will be aware that our central estimate suggests that savings of up to £230 million are possible. While I am on this subject, I was very glad to see that the Secondary Legislation Scrutiny Committee welcomed our consultation and voluntary impact assessment in its report on this instrument.
To summarise, the instrument reflects engagement and careful balancing to present commissioners with the right options for procurement so that they can find the most collaborative, value-add solutions that will work for patients. Engagement with providers has told us that both more collaborative approaches to healthcare—where those with services to offer can get around the table, help break down barriers and promote provider diversity—and putting a contract out to tender are valuable and need to be in the commissioner’s toolkit. That is why this instrument reaffirms the role of competition in arranging services by providing explicitly for those processes, while also providing some flexibility to commissioners to adopt a more direct approach.
As many noble Lords will know, getting the balance of a framework right to promote the best culture and behaviour on the ground is tricky. I am glad, therefore, that we have worked so closely with providers and commissioners to find and test that balance. One result of that engagement was to agree to establish an independently chaired panel which will act as a non-statutory advisory body for contested decisions made under this regime. We intend that this will help commissioners think carefully about the approach that they take to procurement, and its justifications.
Furthermore, we must ensure that the system understands these rules so that it can have the best chance of promoting the right behaviour on the ground. That is why NHS England is leading an extensive programme of familiarisation with those draft regulations and the draft statutory guidance, which is available online. Of course, legislation and guidance are only part of the story of how the new legislation will influence outcomes. That is why the department is committed to monitoring and evaluating this new regime from its implementation.
For these reasons, I am content to move these draft regulations, which, subject to the approval of the House, would bring the provider selection regime into force. I beg to move.
My Lords, I welcome these regulations. They get the NHS off the hook from inappropriate compulsory competitive tendering of clinical services but also avoid throwing the baby out with the bathwater. Open procurement will remain an option where it is in patients’ and taxpayers’ interests.
In my previous experience, there have been several problems with the way in which the accretion of UK procurement rules and the EU procurement regime have tied the hands of the NHS. We have often had to go through the motions of competitive clinical procurements for services that would quite obviously be provided only in one place and by one part of the NHS—for example, billions of pounds-worth of specialised cardiac and cancer services for which it was blindingly obvious that the Germans and Italians would not turn up and try to replace Leeds General Infirmary or St Thomas’ Hospital. These regulations make these processes honest, in that when we embark on procurements, it will be for a good reason.
A related problem is that the legacy procurement rules have tended to lead to too much service fragmentation. We have seen examples where community nursing services have had to be tendered out but core general practice services have not, so getting the community nurses and GP practices working together has been much harder. One of the fragmenting consequences of the 2012 Act was that a lot of what had previously been NHS services became local authority-procured, and so sexual health services and health visitors were operating on a different procurement process through local authorities rather than through the local NHS. The Health and Care Act 2022 and these regulations overcome that problem. The NHS will still be subject to transparent and fair procurement, but it will now be much more flexible and proportionate.
The regulations are quite complex. Those noble Lords who have read through the materials may agree that it is fair to say that they will not command the attention of the pubs and clubs of Barnsley or Barnstaple, but they will make a huge difference to the way in which care is delivered right across the country.