International Relations and Defence Committee
Wednesday 11th November 2026
Skip to latest contribution
This debate has concluded
(began 3 hours, 8 minutes ago)
Watch LiveSkip to latest contribution
This debate has concluded
11:05
Lord Robertson of Port Ellen
-
-
Lord Robertson of Port Ellen
The technical stuff done first. Before you go live on Parliament television. Okay. Thank you, and welcome to this meeting of the International Relations and Defense Committee of the of the House of Lords. And this further session on our discussion and our potential report about AI governance at multilateral organizations outside the UN system and international fora. And you are very welcome today to this meeting. It's live on Parliament TV. A transcript will be taken of the proceedings, which will be sent to you. And maybe any small correction might be made to that. Members of the committee will ask questions, and if they have any interest to declare, they should do so before.
And I therefore declare that I am a senior counselor with the Cohen Group in Washington, D.C. , but we will proceed with you, and we're very grateful to you for giving up your time in order to inform us. So my question at the beginning is that a variety of organizations are already meddling or discussing the whole question about artificial intelligence, the G7, the OECD, the Global partnership on AI are all involved in , in, in the idea of governance, which is what we are particularly looking at. So I'd like to know from you how significant these organizations are within that global governance governance debate and how are they cooperating together?
Many people say they're just a talking shop, that they're soaking up a lot of time, but not actually producing very much. So I wonder if you could give us your views on how these organizations are dealing with the whole question of governance. They. Mr..
And I therefore declare that I am a senior counselor with the Cohen Group in Washington, D.C. , but we will proceed with you, and we're very grateful to you for giving up your time in order to inform us. So my question at the beginning is that a variety of organizations are already meddling or discussing the whole question about artificial intelligence, the G7, the OECD, the Global partnership on AI are all involved in , in, in the idea of governance, which is what we are particularly looking at. So I'd like to know from you how significant these organizations are within that global governance governance debate and how are they cooperating together?
Many people say they're just a talking shop, that they're soaking up a lot of time, but not actually producing very much. So I wonder if you could give us your views on how these organizations are dealing with the whole question of governance. They. Mr..
11:08
11:08
11:08
Speaker 1
-
Speaker 1
Here. By way of introduction, my name is Florian Ostman. I'm a distinguished policy fellow at the London School of Economics in the Data Science Institute. I've worked in the AI governance space for about ten years previously at the Alan Turing Institute, leading the institute's work on AI governance and regulation. And I should also mention that I'm a member and have been a member for a number of years of the OECD and Global Partnership on AI, a community of experts on AI. Now, with regards to the question, I think my sort of my short answer would be, I think across the organizations you mentioned and the four you mentioned, G7, G, OECD, all of those fora have played a really important role and continue to play an important role in AI governance.
One point I would maybe start with is, is the fact that there is a lot of coordination actually between these three fora. And so, you know, rather than thinking of them as a, you know, independent actors in a busy landscape, I think it's worth recognizing that there is, you know, a way of thinking of them as concentric circles as it were. So the G7 obviously bringing together seven countries, the OECD, including all G7 members, and an additional range of countries. And then, as you may be aware, the global partnership on AI and the OECD a couple of years ago entered an integrated partnership, which means that the global partnership on AI, in a sense, is now a bigger circle, you know, comprising OCD members and an additional number of countries.
And so in terms of membership, there are, you know, subsets of of each other. Also, in terms of the work that's happening, we often see, you know, work being initiated in one forum and carried over in a, in a, in a larger forum. So, for example, the G7, one of the most significant outputs there has been the Hiroshima process and the reporting process around advanced AI systems that was initiated in the G7 and is now taking forward by the OECD, for example. I would also emphasize that while the AI governance space internationally has gotten a lot more crowded, all three of these fora were among the first movers, as it were.
So the work in the OECD on AI started in 2018. Similarly, the global partnership on AI discussions around that started in 2018, by comparison, for example, you know, the AI act negotiations on that started three years later. And in the G7, we've seen, you know, attention to AI matters for for a long time in communiques. Now, one important feature, I think, of the work in those fora in all three of them, is that generally outputs take the form of non-binding instruments or non-binding outputs. That does imply limitations. And I, I think, want to be clear and expressing the view that there is a need for binding, you know, requirements in certain areas of AI governance.
But these non-binding instruments play a really, really important role, first of all, because in some areas, non-binding instruments are sufficient and are the right tool. And the fact that both the OECD and are, you know, pursue a mode of working that includes a lot of stakeholder input. They're both sort of characterized by a strong commitment to multi-stakeholder assessments and how work is being developed. And then, you know, that underpins to some extent the uptake of the outputs, even though they're they're not binding. So there is an important role for binding instruments in their own right.
But then, you know, insofar as binding rules are needed, this non-binding work can play a really important role as, as providing a foundation, be it, you know, in the case of the OCD, for example, informing policy making and regulatory rule making at the national level and working towards alignment in that. And we've seen, for example, the OECD AI definition being reflected in national policies in a number of jurisdictions, including the AI act and the UK's approach to AI regulation, but also at international level. So thinking about, you know, emerging discussions around transparency requirements, incident reporting requirements, in a sense, the OECD has done important work in developing the frameworks that are that are now broadly agreed by, by important ranges of stakeholders that can then be adopted and or sort of turned into binding requirements.
One point I would maybe start with is, is the fact that there is a lot of coordination actually between these three fora. And so, you know, rather than thinking of them as a, you know, independent actors in a busy landscape, I think it's worth recognizing that there is, you know, a way of thinking of them as concentric circles as it were. So the G7 obviously bringing together seven countries, the OECD, including all G7 members, and an additional range of countries. And then, as you may be aware, the global partnership on AI and the OECD a couple of years ago entered an integrated partnership, which means that the global partnership on AI, in a sense, is now a bigger circle, you know, comprising OCD members and an additional number of countries.
And so in terms of membership, there are, you know, subsets of of each other. Also, in terms of the work that's happening, we often see, you know, work being initiated in one forum and carried over in a, in a, in a larger forum. So, for example, the G7, one of the most significant outputs there has been the Hiroshima process and the reporting process around advanced AI systems that was initiated in the G7 and is now taking forward by the OECD, for example. I would also emphasize that while the AI governance space internationally has gotten a lot more crowded, all three of these fora were among the first movers, as it were.
So the work in the OECD on AI started in 2018. Similarly, the global partnership on AI discussions around that started in 2018, by comparison, for example, you know, the AI act negotiations on that started three years later. And in the G7, we've seen, you know, attention to AI matters for for a long time in communiques. Now, one important feature, I think, of the work in those fora in all three of them, is that generally outputs take the form of non-binding instruments or non-binding outputs. That does imply limitations. And I, I think, want to be clear and expressing the view that there is a need for binding, you know, requirements in certain areas of AI governance.
But these non-binding instruments play a really, really important role, first of all, because in some areas, non-binding instruments are sufficient and are the right tool. And the fact that both the OECD and are, you know, pursue a mode of working that includes a lot of stakeholder input. They're both sort of characterized by a strong commitment to multi-stakeholder assessments and how work is being developed. And then, you know, that underpins to some extent the uptake of the outputs, even though they're they're not binding. So there is an important role for binding instruments in their own right.
But then, you know, insofar as binding rules are needed, this non-binding work can play a really important role as, as providing a foundation, be it, you know, in the case of the OCD, for example, informing policy making and regulatory rule making at the national level and working towards alignment in that. And we've seen, for example, the OECD AI definition being reflected in national policies in a number of jurisdictions, including the AI act and the UK's approach to AI regulation, but also at international level. So thinking about, you know, emerging discussions around transparency requirements, incident reporting requirements, in a sense, the OECD has done important work in developing the frameworks that are that are now broadly agreed by, by important ranges of stakeholders that can then be adopted and or sort of turned into binding requirements.
11:12
11:12
Speaker 2
-
Speaker 2
Thank you. And thank you to the committee for having me here today. My name is Michael Birtwistle. I am the associate director for law and policy at the Ada Lovelace Institute, which is an independent research institute with a mission to make AI and data work for people and society. And we have been a fairly active participant in, I suppose, global fora for a number of years. We've been participants in some sense at all of the sort of AI summit series. And recently in New York for the UN General Assembly and Global Global Dialogue. I would agree with a lot of Florens remarks.
I think our starting point on this question is that it's probably helpful to distinguish what different kinds of contributions you might make to the idea of global governance. Normally, at the national level, sort of, we often mean by governance sort of binding rules, processes, the incentives that follow. So this isn't a particularly rigorous way of breaking it down, but might be useful today. You have that sort of normative role that global institutions can play in terms of making recommendations, setting binding rules, sometimes setting terms and standards, potentially providing venues for legal redress. Most of those around AI do exist in the softer form, not the harder form.
You have sort of epistemic and knowledge based roles where you are doing scientific assessment, monitoring and measurement, technical evaluation of models, and then you have the sort of political and strategic roles where you're convening agenda setting some of the things Florian referred to. I think one of the spaces and roles that that is has been more challenging for some of these institutions has been around the operationalizing of, of principles. So building implementation guidance and tools, disclosure reporting, regulatory cooperation, capacity building, which some of which is in train. And then I suppose just, just comparing to some of the other comparators that you might have at the international level, we don't have a sort of verification or inspection role really around AI compared to, say, the IEAIAEA, you don't have like an incident response process as you do with cert for cyber, and you don't really have cross-border liability redress enforcement regimes.
So I guess the first point, global governance doesn't always mean enforcement. So, you know, having, for example, distributed networks of collaborative AI expert bodies can, can be very helpful and make a contribution. I'd agree with Florian that the sort of G7 into OECD process through things like the Hiroshima principles has been a very useful contribution. It has some limitations in the sense that, I mean, if you're thinking about, for example, its role in transparency, the the sort of the commitments to publish data are useful in theory, but the information is often very heterogeneous, very difficult to compare over time.
And more work is needed to make sure that that that can actually be usefully consumed, for example, by regulators so that they can improve their understanding of national regulators, so they can improve their understanding of of their object of regulation in AI. And then I suppose a final observation would be that that I guess the role of those bodies has changed over time. So G7, for example, was much more in a normative space 2023. And has sort of passed, I think, the baton on to, to OCD for for operationalization. And it now plays much more of a kind of agenda setting role.
And that conversation has moved more towards things like economic security and resilience, for example. So. The roles have changed, I suppose, in response to the geopolitical picture.
I think our starting point on this question is that it's probably helpful to distinguish what different kinds of contributions you might make to the idea of global governance. Normally, at the national level, sort of, we often mean by governance sort of binding rules, processes, the incentives that follow. So this isn't a particularly rigorous way of breaking it down, but might be useful today. You have that sort of normative role that global institutions can play in terms of making recommendations, setting binding rules, sometimes setting terms and standards, potentially providing venues for legal redress. Most of those around AI do exist in the softer form, not the harder form.
You have sort of epistemic and knowledge based roles where you are doing scientific assessment, monitoring and measurement, technical evaluation of models, and then you have the sort of political and strategic roles where you're convening agenda setting some of the things Florian referred to. I think one of the spaces and roles that that is has been more challenging for some of these institutions has been around the operationalizing of, of principles. So building implementation guidance and tools, disclosure reporting, regulatory cooperation, capacity building, which some of which is in train. And then I suppose just, just comparing to some of the other comparators that you might have at the international level, we don't have a sort of verification or inspection role really around AI compared to, say, the IEAIAEA, you don't have like an incident response process as you do with cert for cyber, and you don't really have cross-border liability redress enforcement regimes.
So I guess the first point, global governance doesn't always mean enforcement. So, you know, having, for example, distributed networks of collaborative AI expert bodies can, can be very helpful and make a contribution. I'd agree with Florian that the sort of G7 into OECD process through things like the Hiroshima principles has been a very useful contribution. It has some limitations in the sense that, I mean, if you're thinking about, for example, its role in transparency, the the sort of the commitments to publish data are useful in theory, but the information is often very heterogeneous, very difficult to compare over time.
And more work is needed to make sure that that that can actually be usefully consumed, for example, by regulators so that they can improve their understanding of national regulators, so they can improve their understanding of of their object of regulation in AI. And then I suppose a final observation would be that that I guess the role of those bodies has changed over time. So G7, for example, was much more in a normative space 2023. And has sort of passed, I think, the baton on to, to OCD for for operationalization. And it now plays much more of a kind of agenda setting role.
And that conversation has moved more towards things like economic security and resilience, for example. So. The roles have changed, I suppose, in response to the geopolitical picture.
11:16
11:16
Speaker 1
-
Speaker 1
So, so I think I think they can in important ways. So I think there are certain examples of work that can be thought of as the building blocks for binding requirements in the sense that they don't necessarily, you know, they're not a blueprint for what how to formulate the regulation, right. But they provide the element of how things should be done. So for example, you know, we've got the transparency framework, the Hiroshima reporting framework. I agree with the limitations, right. But it's an important first step that commands broad agreement. So if regulators or countries at international level thinking about, you know, introducing mandatory transparency requirements, it'd be a natural starting point to think about what those should look like or similarly, an important area.
You know, currently the question of incident reporting, you know, where the OCD has done important work on a conceptual framework for what incident reporting should look like. So if mandatory incident reporting requirements are introduced, they can draw on on that work.
You know, currently the question of incident reporting, you know, where the OCD has done important work on a conceptual framework for what incident reporting should look like. So if mandatory incident reporting requirements are introduced, they can draw on on that work.
11:17
Speaker 2
-
Speaker 2
Yes. I'd say that that there's sort of processes like the Hiroshima reporting framework are useful for establishing a floor of, of sort of agreement around AI transparency and accountability. It doesn't obviously provide the the hard incentive mechanisms that are needed to make its addressees comply with, with the principles. And, you know, those will probably require. Coordinated, coordinated action internationally. But, but but regulation at the national level in order to to be achieved. But you do see, I suppose, the DNA flowing down from some of those agreements into things like the EU's AI act, Code of Practice and SB 53 in California.
So the sort of the some of the wording and consensus over terms and what you're measuring, for example, that is that is set by those non-binding processes can eventually flow down, but it does require, I suppose, the the political will at the at the national level to implement.
So the sort of the some of the wording and consensus over terms and what you're measuring, for example, that is that is set by those non-binding processes can eventually flow down, but it does require, I suppose, the the political will at the at the national level to implement.
11:19
11:19
Baroness Crawley
-
-
Baroness Crawley
Just a very quick. one to either of our speakers. Whoever feels they want to answer it. You talked about the G7 Hiroshima process and you referred to work that the the EU is doing. They've recently, I think last month had the Dublin Declaration on AI. Where do these recommendations come together? How do you get the G7 and the EU , both very recent recommendations coming together to to form.
You talked about it trickling down, but there must be more than just hope that there's a trickle down.
You talked about it trickling down, but there must be more than just hope that there's a trickle down.
11:20
11:20
Speaker 1
-
Speaker 1
I think I think one way to think about it is to think in terms of the timeline for these different things happening. So if you think about the Hiroshima process that started when the OCD had already done its work on, you know, establishing principles and policy recommendations, those were developed before generative AI entered the scene. Right. So then there was a sense in the policy community that there was a need, you know, to for a framework that sort of address the specific risks around language models. Right. And so it was natural for that to be sensed that something additional is needed.
The G7 provided a forum for doing that as my, my, my view on the history. But then there was, you know, as we touched on, you know, follow up work and sort of a path to integration, right? So there's no disagreement between the OECD and the OECD's community and the G7 that these parts need to be thought of as as, as being elements of one whole. And, and then, in fact, the OECD later on also updated its principles and its policy recommendations. And I think we see, you know, similarly, when we look at current calls, we see the the call from a number of countries around, you know, control for frontier AI that was initiated by Finland and Norway.
The declaration you just mentioned, I think they're all reactions to the, you know, needs that are identified in response to further advancements in AI development. So there's a logic for them being, you know, introduced. But I think there's also reason to assume that over time there will be integrated. You know, they won't need to there's no interest, I think, in any of the stakeholders introducing those among like minded partners for that to lead to this alignment.
But it's a contribution that, you know, in building the same, you know, edifice, as it were. Yeah.
The G7 provided a forum for doing that as my, my, my view on the history. But then there was, you know, as we touched on, you know, follow up work and sort of a path to integration, right? So there's no disagreement between the OECD and the OECD's community and the G7 that these parts need to be thought of as as, as being elements of one whole. And, and then, in fact, the OECD later on also updated its principles and its policy recommendations. And I think we see, you know, similarly, when we look at current calls, we see the the call from a number of countries around, you know, control for frontier AI that was initiated by Finland and Norway.
The declaration you just mentioned, I think they're all reactions to the, you know, needs that are identified in response to further advancements in AI development. So there's a logic for them being, you know, introduced. But I think there's also reason to assume that over time there will be integrated. You know, they won't need to there's no interest, I think, in any of the stakeholders introducing those among like minded partners for that to lead to this alignment.
But it's a contribution that, you know, in building the same, you know, edifice, as it were. Yeah.
11:22
Speaker 1
-
Speaker 1
I think, I mean, I'm sure that's a question we might come back to, you know, and of course, in the G7 and G20, a lot depends on the presidency. I think the UK has an interesting, you know, two years coming up with, first of all, leading the G20 presidency next year and the G7 presidency the year after. So maybe there is an important role for the UK there.
11:22
11:22
11:22
Baroness Blackstone
-
-
Baroness Blackstone
I wonder if you could give the committee an overview of what the part played by the UK is in discussions on AI in organisations like OECD or bodies like the G7, what particular capacities does it have? Are technical or other. Does it help shape the agenda? If you could tell us a bit about its role and how effective it is, maybe starting with Michael Birtwhistle?
11:22
Speaker 2
-
Speaker 2
Sure. Thank you. Thank you for the question. So I think it has changed over time. The UK did quite a lot in the run up to 2023. It was a founding member of G P. It was obviously the host of Bletchley and it coordinated with Japan on the Hiroshima principles, I believe. And it, I believe, is quite an active participant in sort of OECD subcommittees and the sort of technical work that goes on there. I think what we have seen sort of since about 2025 is, is a bit of a realignment in its posture to align with the with with the US.
It declined to sign the Paris Summit Declaration in February 2025. That was the same time that it renamed the the AI Safety Institute to the AI Security Institute and sort of narrowed its scope. It sort of framed its future contributions around the sort of bilateral relations with the US in terms of the US, UK prosperity deal. And very recently at the UN General Assembly, it didn't sort of sign this sort of middle powers statement that was put out by by a number of state nations. It has made other contributions. So it's sort of driven the international AI safety report up until sort of the UN's processes were sort of set up to to carry on that baton, which was helpful contribution on many levels.
It wasn't comprehensive. It didn't cover some notably contentious AI issues, such as climate data protection, copyright issues, but it was nevertheless an important sort of evidential contribution. It did endorse the Delhi Declaration, and sort of since I think in April of this year, it sort of Liz Kendall, who was then secretary of state, gave a gave a speech on how the UK wanted to work with middle powers, France, Germany, Canada, which in practice meant sort of closer work on AI standards, on joint investment. But I think we would say and sorry, the thing I should also mention is the enormous technical contribution that that the AI Security Institute does make on a narrow set of sort of extreme risk and understanding the propensity for extreme risk of, of AI models.
But there is sort of a disconnect between what the government state is. Ambitions are sort of in public in terms of the way that it says it wants to work with middle powers and what it's actually signaling during its participation in these in these global fora to date.
It declined to sign the Paris Summit Declaration in February 2025. That was the same time that it renamed the the AI Safety Institute to the AI Security Institute and sort of narrowed its scope. It sort of framed its future contributions around the sort of bilateral relations with the US in terms of the US, UK prosperity deal. And very recently at the UN General Assembly, it didn't sort of sign this sort of middle powers statement that was put out by by a number of state nations. It has made other contributions. So it's sort of driven the international AI safety report up until sort of the UN's processes were sort of set up to to carry on that baton, which was helpful contribution on many levels.
It wasn't comprehensive. It didn't cover some notably contentious AI issues, such as climate data protection, copyright issues, but it was nevertheless an important sort of evidential contribution. It did endorse the Delhi Declaration, and sort of since I think in April of this year, it sort of Liz Kendall, who was then secretary of state, gave a gave a speech on how the UK wanted to work with middle powers, France, Germany, Canada, which in practice meant sort of closer work on AI standards, on joint investment. But I think we would say and sorry, the thing I should also mention is the enormous technical contribution that that the AI Security Institute does make on a narrow set of sort of extreme risk and understanding the propensity for extreme risk of, of AI models.
But there is sort of a disconnect between what the government state is. Ambitions are sort of in public in terms of the way that it says it wants to work with middle powers and what it's actually signaling during its participation in these in these global fora to date.
11:25
Speaker 1
-
Speaker 1
Yeah, sorry. I think I agree with the, the overview and the assessment. The middle power statement is the statement that I mentioned previously as well, you know, that was initiated by, by Norway and, and, and Finland. And so it's noteworthy that the UK decided not, not, not to sign that, but there is a history, you know, of engagement. I think, you know, from all I know and what I've witnessed, the UK government is very active in at the table, you know, in the OECD and the G7. The question is, you know, what are to what extent does that translate into, you know, decisions and, you know, sort of , yeah, outcomes that sort of, you know, push the boundary and advance, advance progress.
Right. And I think there are certain geopolitical. Factors that have become more prominent, as Michael said, in the last in the last two years, I would maybe just slightly expand on the role of the AI Security Institute in the sense that that really, you know, was the UK was sort of provided a proof of concept there, you know, and has the strongest institute of that kind by far. So it plays an international leadership role. And it's also the custodian of the network of similar institutes around the world, which includes, you know, ten countries, I believe.
So that's an interesting, you know, point to add in the sense that it's, you know, that coalition of countries is, is not identical to any of the groups that we discussed. Right? It's smaller than the OECD, it's wider than the G7, but it's I think it's an important coalition. It's focused on sort of the technical understanding and advancing safety research rather than than rule making. But of course, that has a really important role to play in, in, in informing decisions about, about rule making and governance. And then the last thing I would maybe mention is sort of the fact that setting aside the role of the UK government in international fora, I think there is a.
It's worth recognizing that there's a history and a strong record for UK experts and non-government stakeholders to be very actively involved in making important contributions , both in the OECD and GPA. For example, you know, UK experts have been prominent members, including as chairs of of working groups, for example, but also in other international contexts such as the standard setting organizations, the European standards bodies, for example, that develop the standards to support implementation, implementation of the EU AI act. UK experts have been involved in that. They've. Some UK experts have played a leading role in developing the Code of Conduct or Code of Practice for general purpose models in the EU.
So in terms of, you know, broader UK expertise, I think, you know, also thinking beyond government, there's important it's an important history and and line of continuity.
Right. And I think there are certain geopolitical. Factors that have become more prominent, as Michael said, in the last in the last two years, I would maybe just slightly expand on the role of the AI Security Institute in the sense that that really, you know, was the UK was sort of provided a proof of concept there, you know, and has the strongest institute of that kind by far. So it plays an international leadership role. And it's also the custodian of the network of similar institutes around the world, which includes, you know, ten countries, I believe.
So that's an interesting, you know, point to add in the sense that it's, you know, that coalition of countries is, is not identical to any of the groups that we discussed. Right? It's smaller than the OECD, it's wider than the G7, but it's I think it's an important coalition. It's focused on sort of the technical understanding and advancing safety research rather than than rule making. But of course, that has a really important role to play in, in, in informing decisions about, about rule making and governance. And then the last thing I would maybe mention is sort of the fact that setting aside the role of the UK government in international fora, I think there is a.
It's worth recognizing that there's a history and a strong record for UK experts and non-government stakeholders to be very actively involved in making important contributions , both in the OECD and GPA. For example, you know, UK experts have been prominent members, including as chairs of of working groups, for example, but also in other international contexts such as the standard setting organizations, the European standards bodies, for example, that develop the standards to support implementation, implementation of the EU AI act. UK experts have been involved in that. They've. Some UK experts have played a leading role in developing the Code of Conduct or Code of Practice for general purpose models in the EU.
So in terms of, you know, broader UK expertise, I think, you know, also thinking beyond government, there's important it's an important history and and line of continuity.
11:28
11:28
Lord Grocott
-
-
Lord Grocott
Yes. I mean, you've covered a good deal of it already, for which. Thank you for that. Just say that the picture that emerges from the evidence we've had so far is very much of a a myriad of different international and national organizations, all looking at matters connected with the governance of of AI, with little coordination, as far as anyone can see. I don't know whether again, one of our early witnesses says it's pretty tricky to expect agreement on the answers when there isn't even agreement on the questions I. If I can distill this, I'd just ask you this where's.
Would you be prepared to say where you think the most productive venue might be for discussions of this sort and maybe decisions to be made? I mean, is it at the level of the UN, which does seem to me have a pretty fundamental problem in that the two superpowers are at different US and China at different ends of the spectrum. As far as governments have of AI is concerned, or is it best trying to achieve some kind of coordination and agreement? Is that best done at again, through. You've mentioned this a bit through like minded organizations or to an extent like minded organizations like the the G7.
Where could we as a country most productively place our priority?
Would you be prepared to say where you think the most productive venue might be for discussions of this sort and maybe decisions to be made? I mean, is it at the level of the UN, which does seem to me have a pretty fundamental problem in that the two superpowers are at different US and China at different ends of the spectrum. As far as governments have of AI is concerned, or is it best trying to achieve some kind of coordination and agreement? Is that best done at again, through. You've mentioned this a bit through like minded organizations or to an extent like minded organizations like the the G7.
Where could we as a country most productively place our priority?
11:30
Speaker 1
-
Speaker 1
I think my, my general sort of answer would be not to think about these different fora as a as a matter of choosing between them, but recognizing that they all have their role to play, I think, I don't think we should think of AI governance, you know, guided by the the idea of ideally having a single forum where all matters of AI governance are settled or agreed, let alone a single, you know, international forum that includes all countries. I think, you know, depending on the AI governance problem that we're looking at , different stakeholders, different countries need to be at the table the range of countries that are needed.
You know, currently at the sharp end of frontier AI risks, it's clear we need an agreement that includes China and the US. And potentially there's a lot to be gotten out of an agreement that focuses just on China and the US. And there are other matters where, you know, broader agreement is needed. And there's also, I think, the point that, you know, agreement that is reached in a smaller fora over time can find its way and, you know, into work that's happening in broader fora like the United Nations. Right? So, you know, OCD work, for example, informing debates in the United Nations.
Now, having said all, all of that, I think fora like the OECD have have the advantage of a number of advantages. The first one is, is sort of the, the pace in which dialogue can advance and, and associated with that, the ease of finding agreement, you know, associated as a result of a smaller number of countries being there in the countries having sort of a similar policy outlook. Also, the fact that, as I mentioned earlier, the OECD has a strong mechanism for engaging non-government stakeholders, which I think strengthens the work and the fact that the OSCE and that's maybe a contrast to the G7, you know, has a has an institutional structure that allows work to continue over time that isn't dependent on, for example, presidency changing.
And so I do think that, you know, that means the OECD isn't necessarily the only body of that kind, but organizations that have those features lend themselves particularly well to, you know, going back to what Michael mentioned earlier, work that is thinking about sort of operational operationalizing or, you know, implementation questions that move from principles to sort of, you know, practical, practical questions. Having said that, you know, going back to what I started with, a clear limitation of the OECD is the fact that it doesn't include China and it doesn't include global South perspectives. And so clearly, there are AI governance areas where the inclusion of both of those or one of those are critical.
As I mentioned, you know, US-China agreement is critical. That doesn't mean that, you know, those discussions need to happen happen at the UN level because the UN comes with, you know, the the fact that discussions move more slowly, simply as a matter of, you know, the number of countries involved. So I do think there's a need for a dialogue that involves the US and and China and the urgency of some of the questions are such that that shouldn't happen or probably won't happen in the UN, but needs to happen in a in a different forum.
And again, perhaps the G20, you know, is a forum to look at for that purpose, among others, and the UK's role as a G20 chair next year might be an opportunity to do that.
You know, currently at the sharp end of frontier AI risks, it's clear we need an agreement that includes China and the US. And potentially there's a lot to be gotten out of an agreement that focuses just on China and the US. And there are other matters where, you know, broader agreement is needed. And there's also, I think, the point that, you know, agreement that is reached in a smaller fora over time can find its way and, you know, into work that's happening in broader fora like the United Nations. Right? So, you know, OCD work, for example, informing debates in the United Nations.
Now, having said all, all of that, I think fora like the OECD have have the advantage of a number of advantages. The first one is, is sort of the, the pace in which dialogue can advance and, and associated with that, the ease of finding agreement, you know, associated as a result of a smaller number of countries being there in the countries having sort of a similar policy outlook. Also, the fact that, as I mentioned earlier, the OECD has a strong mechanism for engaging non-government stakeholders, which I think strengthens the work and the fact that the OSCE and that's maybe a contrast to the G7, you know, has a has an institutional structure that allows work to continue over time that isn't dependent on, for example, presidency changing.
And so I do think that, you know, that means the OECD isn't necessarily the only body of that kind, but organizations that have those features lend themselves particularly well to, you know, going back to what Michael mentioned earlier, work that is thinking about sort of operational operationalizing or, you know, implementation questions that move from principles to sort of, you know, practical, practical questions. Having said that, you know, going back to what I started with, a clear limitation of the OECD is the fact that it doesn't include China and it doesn't include global South perspectives. And so clearly, there are AI governance areas where the inclusion of both of those or one of those are critical.
As I mentioned, you know, US-China agreement is critical. That doesn't mean that, you know, those discussions need to happen happen at the UN level because the UN comes with, you know, the the fact that discussions move more slowly, simply as a matter of, you know, the number of countries involved. So I do think there's a need for a dialogue that involves the US and and China and the urgency of some of the questions are such that that shouldn't happen or probably won't happen in the UN, but needs to happen in a in a different forum.
And again, perhaps the G20, you know, is a forum to look at for that purpose, among others, and the UK's role as a G20 chair next year might be an opportunity to do that.
11:33
Speaker 2
-
Speaker 2
Just thinking about what I might add, because I agree with a lot of that. Usefully, I think I think it very much is the case of like the most productive venue depends on on which problem you are trying to solve. And you have a sort of fairly tight coupling at the moment with, you know, questions of risk mitigation, of frontier AI risks, which is often where the sort of conversation about global governance goes. But a lot of these fora are also talking about dependency and about sovereignty and about, I suppose, the, the risks, the strategic risks that fall out for countries of being dependent on technology, largely controlled by by another country.
And that is going to inform which of in part, which of these venues is is most useful. The other thing that that might also help is sort of thinking about what what problems actually in the AI governance space actually require an international solution. We ran a , a workshop at the UN General Assembly, a fringe event, and to, to sort of highlight some of the sort of the things that fell out of basically sort of things that don't work if you don't do them at an international level, you've got questions like cross border externality. So there's harm in one jurisdiction, fall on people in another who had no say.
Does the harm causing actor sit outside the jurisdictional reach of where the harm lands? The sort of regulatory arbitrage questions about does uneven protection in one place mean that you sort of push everyone towards the weakest regime? Right. To to the committee with the the sort of the full framework, but it sort of gives you a sense of, of a discriminator, I suppose, to sort of decide which of these forums, given the properties that Florian was talking about, are actually well set up to, to solve those because yes, some of them are, are more tightly focused on, I guess, solution spaces, be they technical standards, be they developing evaluation frameworks and so on that are actually able to understand and address those risks more directly.
And that is going to inform which of in part, which of these venues is is most useful. The other thing that that might also help is sort of thinking about what what problems actually in the AI governance space actually require an international solution. We ran a , a workshop at the UN General Assembly, a fringe event, and to, to sort of highlight some of the sort of the things that fell out of basically sort of things that don't work if you don't do them at an international level, you've got questions like cross border externality. So there's harm in one jurisdiction, fall on people in another who had no say.
Does the harm causing actor sit outside the jurisdictional reach of where the harm lands? The sort of regulatory arbitrage questions about does uneven protection in one place mean that you sort of push everyone towards the weakest regime? Right. To to the committee with the the sort of the full framework, but it sort of gives you a sense of, of a discriminator, I suppose, to sort of decide which of these forums, given the properties that Florian was talking about, are actually well set up to, to solve those because yes, some of them are, are more tightly focused on, I guess, solution spaces, be they technical standards, be they developing evaluation frameworks and so on that are actually able to understand and address those risks more directly.
11:36
Lord Bruce of Bennachie
-
-
Lord Bruce of Bennachie
Thank you very much. It really follows on from that because in terms of what governance is trying to achieve, I suppose we're trying to minimize risks while maximizing opportunities and protecting sovereignty. And obviously different players are going to have different emphasis on that. So my question is, to what extent should countries broaden the participation to try and have it as wide as possible to take account of the wider range of interest, or alternatively, concentrate on those countries which have the most expertise and are the best able to come up with the best solutions? In theory, maybe not in practice.
I noticed that the global partnership , which has been expanding, but at least I've got at the moment, has only one African member, which is Senegal, and that seems. And the EU has its own initiative on AI, and there's a South African based organization. So Africa has an interest, but they're not involved. So to what extent is it, if I can repeat that appropriate for the expert countries to lead the way and cooperate, or to broaden it out to ensure that all the wider range of interests are covered? And indeed, are those mutually exclusive given? You know, there's a limitation of resources.
You said you've participated in all these various organizations, but there's only so many people can attend so many conferences, so many times. So it it is a constraint.
I noticed that the global partnership , which has been expanding, but at least I've got at the moment, has only one African member, which is Senegal, and that seems. And the EU has its own initiative on AI, and there's a South African based organization. So Africa has an interest, but they're not involved. So to what extent is it, if I can repeat that appropriate for the expert countries to lead the way and cooperate, or to broaden it out to ensure that all the wider range of interests are covered? And indeed, are those mutually exclusive given? You know, there's a limitation of resources.
You said you've participated in all these various organizations, but there's only so many people can attend so many conferences, so many times. So it it is a constraint.
11:37
11:37
Speaker 2
-
Speaker 2
Maybe make a start? So I think there are very good arguments for broadening participation. I'm conscious we're not focusing on the UN today, but there are very good arguments for for participating in the UN processes. It's you know, it's the sort of the fora that represents sort of most of the fora we've been talking about today represent only a fraction of the world. And the UN has the sort of international human rights law mandate. It also carries the mandate for international peace and security. And it obviously has 193 member states versus, I think, 46 for GPA.
I think we're at the stage where the sort of question of how we ensure AI produces the outcomes that we want is getting very, very highly technical. And so understanding sort of how they work and why they produce particular outcomes and having visibility of that is increasingly going to sit in sort of technical processes and evaluations of AI systems, the forums in which those happen, those sort of things get decided have historically been relatively narrow and also seen very high, if you like, representation rates from the companies that those processes are seeking to sort of set the evaluation terms for.
So you sort of are potentially moving from a model where the companies, in safety terms , are marking their own homework to one where they are writing their own exams. But there are sort of other other reasons. So so this idea of like, you know, cross border impact, the statement yesterday that France, Canada and a bunch of the companies signed up to explicit knowledge, the idea that sort of AI impacts are cross border. One way, for example, in which that can manifest is you could have a South African, for example, foundation model that is fine tuned to the local language, and it will then probably demonstrably be the safeguards put in place, for example, around refusals to build weapons will be measurably worse.
So if you don't include all of the countries that have an interest in building sovereign models, for example, then there is sort of increased risk across the system in that sense. And then lastly, I'd say, you know, if the goal is some form, if one of the goals that countries like the UK might be pursuing is some form of, of sovereignty, you do need viable alternatives to sort of a lot of the current AI stack. And that means building with, with trusted partners that sort of want to avoid the risks of, of rent extraction and strategic dependency.
Open sources. I think about 9 to 12 months behind the frontier. I think that might be an AC stat even. And so for some things like cyber, you are may always be the case that you need the frontier. But for many other purposes, open source fast flow is going to be a viable strategy for countries to pursue. And that sort of broadens the the pool of people that it makes sense to collaborate with.
I think we're at the stage where the sort of question of how we ensure AI produces the outcomes that we want is getting very, very highly technical. And so understanding sort of how they work and why they produce particular outcomes and having visibility of that is increasingly going to sit in sort of technical processes and evaluations of AI systems, the forums in which those happen, those sort of things get decided have historically been relatively narrow and also seen very high, if you like, representation rates from the companies that those processes are seeking to sort of set the evaluation terms for.
So you sort of are potentially moving from a model where the companies, in safety terms , are marking their own homework to one where they are writing their own exams. But there are sort of other other reasons. So so this idea of like, you know, cross border impact, the statement yesterday that France, Canada and a bunch of the companies signed up to explicit knowledge, the idea that sort of AI impacts are cross border. One way, for example, in which that can manifest is you could have a South African, for example, foundation model that is fine tuned to the local language, and it will then probably demonstrably be the safeguards put in place, for example, around refusals to build weapons will be measurably worse.
So if you don't include all of the countries that have an interest in building sovereign models, for example, then there is sort of increased risk across the system in that sense. And then lastly, I'd say, you know, if the goal is some form, if one of the goals that countries like the UK might be pursuing is some form of, of sovereignty, you do need viable alternatives to sort of a lot of the current AI stack. And that means building with, with trusted partners that sort of want to avoid the risks of, of rent extraction and strategic dependency.
Open sources. I think about 9 to 12 months behind the frontier. I think that might be an AC stat even. And so for some things like cyber, you are may always be the case that you need the frontier. But for many other purposes, open source fast flow is going to be a viable strategy for countries to pursue. And that sort of broadens the the pool of people that it makes sense to collaborate with.
11:40
Speaker 1
-
Speaker 1
I agree, I agree with with everything Michael shared there, I think. You know, going back also to some of my earlier comments, I would I would say, you know, the answer depends on, you know, the issue that we're trying to solve in terms of, you know, should we try to broaden the circle or should we focus on a narrower circle? If we think about the narrower circle, I think it's worth thinking about. We started the session with with talking about like minded fora that bring together like minded countries in the traditional sense, the, you know, the G7 or the ECD as a larger community.
If we think about reasons why we might or might governments look at working with smaller circles of countries, I think it's worth reflecting on the fact that those aren't necessarily circles of like minded countries in that traditional sense. So, for example, as we touched on, you know, there is a need in certain areas for dialogue and agreement that involves China and the US, right? There's two parties that are that are critical. You won't get that through, you know, the OECD or the G7. But it may well mean, you know, that there is a need for a smaller rather than, you know, looking for that agreement at the UN level for for smaller forum or venue for those discussions to happen.
I think increasingly there's also the question around the role of middle powers as a group, right, which is again, separate, you know, to the four that we touched on. And we've we've touched on, you know, the call for control for frontier AI as a middle power initiative. I think there's an important question of the role that middle powers as a group coming together can play in acting as a catalyst and maybe also to some extent, you know, pushing the US and China, you know, to come together for the discussions that need to happen in the in the frontier AI space.
So of course, that's, you know, diplomatically, there's a, you know, there's a delicate balance there. But again, if you think about the G20 as a forum that includes both countries and it includes a range of middle powers, you know, to what extent is in terms of agenda setting, there are an opportunity to to highlight those those questions and, and try to. Yeah, try to try to make progress. I think there's an interesting dynamic at the moment on the US side, in the sense that the US government is more hesitant to engage in those discussions than US companies themselves.
Right. And so I think that that also provides an opportunity. And and also, I think, again, going back to an earlier point, highlights the importance of a multi-stakeholder approach and those discussions, right, rather than relying on fora that purely bring together government partners, which then may mean that that discussions don't progress or are shut down at an early point, having fora that include industry voices in this case, and of course, civil society voices, I think can make a difference in terms of which what the agenda looks like and, and how far discussions may may advance.
If we think about reasons why we might or might governments look at working with smaller circles of countries, I think it's worth reflecting on the fact that those aren't necessarily circles of like minded countries in that traditional sense. So, for example, as we touched on, you know, there is a need in certain areas for dialogue and agreement that involves China and the US, right? There's two parties that are that are critical. You won't get that through, you know, the OECD or the G7. But it may well mean, you know, that there is a need for a smaller rather than, you know, looking for that agreement at the UN level for for smaller forum or venue for those discussions to happen.
I think increasingly there's also the question around the role of middle powers as a group, right, which is again, separate, you know, to the four that we touched on. And we've we've touched on, you know, the call for control for frontier AI as a middle power initiative. I think there's an important question of the role that middle powers as a group coming together can play in acting as a catalyst and maybe also to some extent, you know, pushing the US and China, you know, to come together for the discussions that need to happen in the in the frontier AI space.
So of course, that's, you know, diplomatically, there's a, you know, there's a delicate balance there. But again, if you think about the G20 as a forum that includes both countries and it includes a range of middle powers, you know, to what extent is in terms of agenda setting, there are an opportunity to to highlight those those questions and, and try to. Yeah, try to try to make progress. I think there's an interesting dynamic at the moment on the US side, in the sense that the US government is more hesitant to engage in those discussions than US companies themselves.
Right. And so I think that that also provides an opportunity. And and also, I think, again, going back to an earlier point, highlights the importance of a multi-stakeholder approach and those discussions, right, rather than relying on fora that purely bring together government partners, which then may mean that that discussions don't progress or are shut down at an early point, having fora that include industry voices in this case, and of course, civil society voices, I think can make a difference in terms of which what the agenda looks like and, and how far discussions may may advance.
11:44
Baroness Crawley
-
-
Baroness Crawley
Thank you very much, chair. Could I ask our speakers about the. We talked. You've just been talking about the US and my question is about the influence of the US on the future agenda for AI. Given the US's leading role in AI. And that's not to be disputed. How dependent are these very important international and mini lateral initiatives on US participation? And perhaps you can, if you've got a judgment, talk about whether the midterm results in Congress and the Senate would make a difference in. A plausible difference in the US's influence on the future of AI.
11:45
Speaker 1
-
Speaker 1
So I think maybe starting with the last point, which I think is an interesting one, I think it's too early to say to what extent, you know, the midterm results will have an impact on the current administration's position. But I think what is clear is that there's a strong contrast between the current administration's position, you know, up to this point and popular opinion in the US, right? If you look at popular opinion, there's strong support for AI regulation across a number of areas of regulation. And then coupled with that, there is also the fact that, as I mentioned, you know, US industry and in particular the leading AI labs, you know, are calling for stronger forms of governance and regulation.
So they're also, in a sense, in contrast to to the federal government's view. And so, you know, it wouldn't be surprising given that constellation. It wouldn't be surprising if there's a change in in the administration's position or at least a change in position, you know, with the next administration in terms of the impact that sort of or the dependence in in the fora that we've been talking about on US involvement. I think it's without doubt, you know, there is a dependence and, you know, the the extent of US engagement makes a difference, I think in terms of the exact impact, it's hard to answer that at a general level.
You know, the question of there's a distinction between existing work continuing and new work starting, right? New work starting often is more dependent on parties agreeing to it. But even that depends on detail. You know, not all new work being started is dependent on consensus. It may be that, you know, the US government is less reluctant to support initiatives that are non-binding, as they have been in the past, for example, in fora fora like the OECD. I don't think, you know, there is US support at the moment for, you know, for binding measures at international level.
There might be a sense, you know, there's a hope at least that that will change on the frontier end of risks potentially. Also, you know, the cybersecurity related risks that are relevant to national security. But sort of, you know, if you think about AI governance more broadly, you know, I think there's it's clear that, you know, the current US administration, there won't be an appetite for binding rules. So I think the fora that we've been talking about, you know, insofar as work continues and new work begins, it will continue to be in sort of a non-binding domain.
But as I mentioned in the beginning, that doesn't mean that the work isn't meaningful or doesn't have impact. And in fact, the fact that, you know, the leading AI labs in the US are sort of, you know, taking a stance that is supportive and actually calling for greater governance measures, the fact that that measures are voluntary may actually not make that much of a of a difference for the time being. So we see, for example, with the Hiroshima AI reporting framework that we've talked about, you know, all major AI labs in the US are participating in that, even though it's a voluntary initiative.
So they're also, in a sense, in contrast to to the federal government's view. And so, you know, it wouldn't be surprising given that constellation. It wouldn't be surprising if there's a change in in the administration's position or at least a change in position, you know, with the next administration in terms of the impact that sort of or the dependence in in the fora that we've been talking about on US involvement. I think it's without doubt, you know, there is a dependence and, you know, the the extent of US engagement makes a difference, I think in terms of the exact impact, it's hard to answer that at a general level.
You know, the question of there's a distinction between existing work continuing and new work starting, right? New work starting often is more dependent on parties agreeing to it. But even that depends on detail. You know, not all new work being started is dependent on consensus. It may be that, you know, the US government is less reluctant to support initiatives that are non-binding, as they have been in the past, for example, in fora fora like the OECD. I don't think, you know, there is US support at the moment for, you know, for binding measures at international level.
There might be a sense, you know, there's a hope at least that that will change on the frontier end of risks potentially. Also, you know, the cybersecurity related risks that are relevant to national security. But sort of, you know, if you think about AI governance more broadly, you know, I think there's it's clear that, you know, the current US administration, there won't be an appetite for binding rules. So I think the fora that we've been talking about, you know, insofar as work continues and new work begins, it will continue to be in sort of a non-binding domain.
But as I mentioned in the beginning, that doesn't mean that the work isn't meaningful or doesn't have impact. And in fact, the fact that, you know, the leading AI labs in the US are sort of, you know, taking a stance that is supportive and actually calling for greater governance measures, the fact that that measures are voluntary may actually not make that much of a of a difference for the time being. So we see, for example, with the Hiroshima AI reporting framework that we've talked about, you know, all major AI labs in the US are participating in that, even though it's a voluntary initiative.
11:48
11:48
11:48
Speaker 2
-
Speaker 2
And just I suppose to add to that, I think on the on the question of the the midterms. The only thing I'd note is that there's been some interesting analysis on how AI generally, and data centers in particular were appeared to be a factor in voting patterns. So that might well be something that sort of shapes political discourse domestically in the US. And therefore, whether where the administration decides to go, I'd echo Florian's point that the company appetite now does seem to be greater than the administration's, but in very specific ways. So we, again, for example, have seen reporting on draft bills that some AI companies have been sharing and lobbying around with various national administrations because their views on what kind of regulation they would deem appropriate or are advantageous to them is quite specific.
Agree that it's honestly a very difficult problem to root around. If the US continues to be reluctant to engage. I think it does explain some of the focus on standards because, you know, sort of the development of standards is, is, is historically been a forum in which opposed countries can sort of cooperate more. There's less sort of normative or political charge to the debate on that. And maybe, maybe the last remark I'd make is that there is some leverage in the sort of the market power procurement investment and industrial strategy behaviors of large economies in this.
And in those interested in developing alternatives. And if that sort of economic demand signal starts being driven towards safe alternatives, perhaps sovereign alternatives, then that may also sort of change the behavior of of the companies or the US administration because. It will be clear that that's where the market is moving.
Agree that it's honestly a very difficult problem to root around. If the US continues to be reluctant to engage. I think it does explain some of the focus on standards because, you know, sort of the development of standards is, is, is historically been a forum in which opposed countries can sort of cooperate more. There's less sort of normative or political charge to the debate on that. And maybe, maybe the last remark I'd make is that there is some leverage in the sort of the market power procurement investment and industrial strategy behaviors of large economies in this.
And in those interested in developing alternatives. And if that sort of economic demand signal starts being driven towards safe alternatives, perhaps sovereign alternatives, then that may also sort of change the behavior of of the companies or the US administration because. It will be clear that that's where the market is moving.
11:50
11:50
11:50
Speaker 3
-
Speaker 3
Thank you. Chairman. You've just said that there are forces. I'm sure you're right about this. Forces in the US, for example, in the commercial sector, that would favor a more positive American approach to to AI regulation. Reality is, though, that that the Americans sometimes do just stay out of things. If you want two examples, they've never joined the International Criminal Court. I think they're still outside the UN law of the Sea Convention, which has been around like forever. So I just wonder if we were to speculate, just supposing, no matter if the government changes to Democrats next time round or whatever, the Americans choose to stay out long term, say there are some examples where they do that.
I mean, is there any value at all? Do you think in, in in AI binding AI regulation without the Americans participating? I'm surely there. Participation, given their dominant role in AI development, is absolutely critical to anything that happens in that without it that all of these initiatives, worthy though they are, and helpful and important though they are actually just run into the sand.
I mean, is there any value at all? Do you think in, in in AI binding AI regulation without the Americans participating? I'm surely there. Participation, given their dominant role in AI development, is absolutely critical to anything that happens in that without it that all of these initiatives, worthy though they are, and helpful and important though they are actually just run into the sand.
11:51
Speaker 1
-
Speaker 1
So I would say if you think about the range of, you know, issues that that exist in, in the domain of AI governance and therefore, and potentially in the domain of regulation, introducing binding requirements, you know, we can draw a broad distinction between frontier AI related matters and what you might think of as frontier AI, AI regulation, right? Sort of thinking about systems that are currently on the market and the way they're adopted and so on and so forth. And I would certainly say, you know, there there are a wide range in the frontier space, you know, a wide range of issues that can effectively be regulated in through through regimes that don't involve us, you know, sign up.
And in fact, I think to some extent and quite, quite a few of them can be regulated at a national level and don't require any, you know, international coordination with other countries. And, you know, Michael alluded to some of the criteria of factors earlier, right? Sort of, you know, questions. To what extent do the risks emerge from, you know, producers of of AI systems that are situated outside jurisdictions currently, what we're seeing, you know, at the frontier end is risks that are associated with the R&D space, right? So if you think about the hugging face incident, right, those were models that aren't on the market that are tested in the lab.
What that means is to, to tackle those risks, any regulatory regime that relies on regulating market access, which is what traditional regulatory regimes do, won't be effective at addressing those risks. Right? So their US participation is absolutely critical. But there are a wide range of issues that can be managed via market access.
And in fact, I think to some extent and quite, quite a few of them can be regulated at a national level and don't require any, you know, international coordination with other countries. And, you know, Michael alluded to some of the criteria of factors earlier, right? Sort of, you know, questions. To what extent do the risks emerge from, you know, producers of of AI systems that are situated outside jurisdictions currently, what we're seeing, you know, at the frontier end is risks that are associated with the R&D space, right? So if you think about the hugging face incident, right, those were models that aren't on the market that are tested in the lab.
What that means is to, to tackle those risks, any regulatory regime that relies on regulating market access, which is what traditional regulatory regimes do, won't be effective at addressing those risks. Right? So their US participation is absolutely critical. But there are a wide range of issues that can be managed via market access.
11:53
Speaker 2
-
Speaker 2
I was going to say something very similar. If the goal is frontier risk mitigation, then there is a certain amount you can do with with market power as the EU is doing. I think the the open question is whether or not that market power is leverageable for the pre-deployment phase. So can can the EU credibly say to US companies will actually. We are not going to allow you to participate in EU markets at all unless you meet these. These requirements that apply even before something is a product or service that you release out into the market, the challenge still becomes enforcement.
Even if you're if you're setting those requirements because it is very difficult to enforce cross Jurisdictionally. Yeah.
Even if you're if you're setting those requirements because it is very difficult to enforce cross Jurisdictionally. Yeah.
11:54
11:54
Lord Houghton of Richmond
-
-
Lord Houghton of Richmond
Yes. And I need to declare some commercial defense AI related interests with Blackstone. Tal is Defense Holdings and Secure Cloud Plus in many respects, the aggregate of your answers so far have largely been the sandwich of of my formal question, which is are there specific areas of AI governance that, in your view, offer greater scope for agreement? But can I therefore, if you've got nothing more to add on that, but please do if you have, could I just also ask something more specific on AI in autonomous weapon systems and particularly sort of. Thus far, although this is not there's no sort of international agreement on all this.
There appears to be two different tracks, if you like, of regulation. One is down the track of technical regulation, then the other is the, as it were, the retention of human agency in the decision loops of weapon systems. And the idea that human agency will protect the worst extremes of technical AI wreaking havoc on a battlefield. But when we look at the world as it is, is the Achilles heel of regulating autonomous weapon systems. The failure of effective human agency that we see before us in the world at the moment. I've come to Florian first.
There appears to be two different tracks, if you like, of regulation. One is down the track of technical regulation, then the other is the, as it were, the retention of human agency in the decision loops of weapon systems. And the idea that human agency will protect the worst extremes of technical AI wreaking havoc on a battlefield. But when we look at the world as it is, is the Achilles heel of regulating autonomous weapon systems. The failure of effective human agency that we see before us in the world at the moment. I've come to Florian first.
11:56
Speaker 1
-
Speaker 1
I think starting with the the last point. So and I'm aware that, you know, the question of lethal autonomous weapons is a topic that's come up in previous sessions of the committee. And I think for completeness, you know, in this discussion, I think it's important to recognize that that's, you know, it's a topic that hasn't been addressed effectively, and that in order to be effectively addressed, you know, requires broad international agreement, right? So it's not an issue that won't be will be able to be resolved unilaterally by a smaller group like the G7 or the OECD.
So, and you're probably aware there are, you know, processes in the UN that have been going on for for several years that haven't led to progress so far. There's an important meeting coming up in November. We'll see if there's a change there. But but I'm personally not expecting a change in position to your question. You know, there is when it comes to sort of the risks, you know, around and around the use of AI in the military, there are certainly the two levels that you that you mentioned. There's a question of risks that are associated with technical design of systems and correspondingly, you know, questions about technical solutions to those.
And then there's the question of or the view that, you know, fully autonomous weapons should just be prohibited on the basis of, you know, the risks, you know, that that may may be associated or as a matter of principle, because we think, you know, in military decisions, there should always be a human that's accountable. And that's been driving, I think, a lot of the UN debate, right? Sort of the autonomy of weapons question. I think the if the view is that, you know, we need to ensure that there's human involvement in decision making, we need to think about both the human in the loop question and the technical layer, right?
So at the technical layer, as I'm sure you're aware, you know, there are concerns around, you know, the loss of control over systems as systems become more advanced. And so simply having a human in the loop, you know, won't necessarily ensure that we keep control. If there's a risk of losing control from a technical design perspective. So we need to think about both, both of those, those levels.
So, and you're probably aware there are, you know, processes in the UN that have been going on for for several years that haven't led to progress so far. There's an important meeting coming up in November. We'll see if there's a change there. But but I'm personally not expecting a change in position to your question. You know, there is when it comes to sort of the risks, you know, around and around the use of AI in the military, there are certainly the two levels that you that you mentioned. There's a question of risks that are associated with technical design of systems and correspondingly, you know, questions about technical solutions to those.
And then there's the question of or the view that, you know, fully autonomous weapons should just be prohibited on the basis of, you know, the risks, you know, that that may may be associated or as a matter of principle, because we think, you know, in military decisions, there should always be a human that's accountable. And that's been driving, I think, a lot of the UN debate, right? Sort of the autonomy of weapons question. I think the if the view is that, you know, we need to ensure that there's human involvement in decision making, we need to think about both the human in the loop question and the technical layer, right?
So at the technical layer, as I'm sure you're aware, you know, there are concerns around, you know, the loss of control over systems as systems become more advanced. And so simply having a human in the loop, you know, won't necessarily ensure that we keep control. If there's a risk of losing control from a technical design perspective. So we need to think about both, both of those, those levels.
11:58
Speaker 2
-
Speaker 2
I'm trying to think what I can add because we don't study laws at at Ada. What I can perhaps comment on is the pressure that systems of human decision making that rely on human decision making to provide accountability, to provide safety and review, are going to be sort of challenged by a future of multi-agent systems. So the next generation of, I suppose, way that these systems are, are being harnessed. For example, we ran a workshop with with the NHS recently exploring the potential use of multi-agent systems in the NHS, the productivity benefits of using AI systems in kind of swarms of agents that sort of all talk to each other are very much based on not having a human in the loop.
So our. You are going to have a tension between, in this case, the economic or public service benefits of potentially deploying those technologies in that way. And all of the reasons that we do have humans in the loop , and whether or not you care about sort of data protection laws and the rules around automated decision making there, or you're simply thinking about, how can I be sure as a one actor in a system of a large bureaucratic system like the NHS, that the set of communications and decisions that have come to me are, are sort of accurate or safe, that's constantly going to run into tension with the sort of the the.
Proposed benefits of adoption. And I imagine that the same is going to be true in in laws.
So our. You are going to have a tension between, in this case, the economic or public service benefits of potentially deploying those technologies in that way. And all of the reasons that we do have humans in the loop , and whether or not you care about sort of data protection laws and the rules around automated decision making there, or you're simply thinking about, how can I be sure as a one actor in a system of a large bureaucratic system like the NHS, that the set of communications and decisions that have come to me are, are sort of accurate or safe, that's constantly going to run into tension with the sort of the the.
Proposed benefits of adoption. And I imagine that the same is going to be true in in laws.
12:00
12:00
12:00
Lord Houghton of Richmond
-
-
Lord Houghton of Richmond
Well, it's I mean, the reality is in the the world in which we live, we deploy autonomous weapon systems almost at the very extreme, well beyond the realms of legality. That's the, you know, the facts of the realities of warfare. Take what? Well, take a non-contentious one. What Russia has done in Ukraine in terms of what is admissible in international humanitarian law and the laws of armed conflict, and therefore, what we are some people are proposing as, as it were, the fail safe, which is human agency, is actually one of the most fundamental flaws now in the system, because people will naturally, in high intensity conflict, go to the, you know, to the very extremes, but beyond the extremes legally.
And therefore, unless you have a regulatory body that harnesses both the legal dimension, the moral dimension and the technical dimension, you're never going to get effective regulation at all. But I perceive of no initiative to attempt to combine these.
And therefore, unless you have a regulatory body that harnesses both the legal dimension, the moral dimension and the technical dimension, you're never going to get effective regulation at all. But I perceive of no initiative to attempt to combine these.
12:01
Speaker 2
-
Speaker 2
So I think we would say that we think the regulation and evaluation of AI systems needs to be socio technical, because you do not understand the impact of a technology like AI until it hits the real world. And so that means that you're always going to need a lot of post-market, not just pre-market. Monitoring and, and research into that impact. And I think there is certainly when it comes to generative AI systems, to LLMs and interactions with them, there is a growing body of evidence around how automation biases are
12:02
Speaker 1
-
Speaker 1
You generative AI systems to LLMs and interactions with them. There is a growing body of evidence around how automation bias is working well in in the way that they operate. And I think some recent research I saw that was looking at how often, I suppose operators of AI systems simply take an LMS. recommendations because of the the sort of plausibility and persuasiveness and the ease with which it is, you know, a human tends to sort of flow and accept.
recommendations and measuring that. So.
recommendations and measuring that. So.
12:02
12:03
12:03
12:03
12:03
Speaker 2
-
Speaker 2
Right? There are crucial questions around sort of the competence of the human in the loop, right. And in the absence of the competence or the informed ability to make informed decisions, sufficient understanding of the system, you know, a human either. interfering or deciding one way or the other, you know, may lead to, even though it's well intentioned, might lead to worse outcomes than human not being involved. So and, and, and related aspects. So I think that's the fact that, you know, we need to think about competence in humans being involved. That's sort of well established in the field of, you know, responsible AI and applies to the use of AI in any domain.
12:03
Speaker 1
-
Speaker 1
And of course, the stakes are particularly high in the military domain. Yeah. There are very interesting parallels from the the Data Use and Access Act that passed through this House last year. And some of the. amendments laid to that and the sort of the section on automated decision making and which some of which have sort of been reflected in the ICO guidance since, speak about a person with sufficient competence and authority, I think, is the is the rough wording being in that position to to make that decision and that that gets you part of the way.
But .
But .
12:04
Lord Robertson of Port Ellen
-
-
Lord Robertson of Port Ellen
It doesn't solve the automation bias problem. Okay. Apart from autonomous weapons, which you've you've dealt with, can you be more. specific about the. areas that you think need to have a. multilateral approach, an international approach to them in terms of regulation? You you've talked about the. principles generally, but, you know, I would like you to be a bit more. specific about the areas that you think really should be the subject of, of, of international regulation.
12:05
12:05
12:05
12:05
12:05
12:05
12:05
Speaker 1
-
Speaker 1
Routinely collected evidence base on, on AI opportunities, risks and harms as a sort of enabling condition. And then lastly, kind of process commitments with regular review, state reporting, civil society participation. One of the conversations that was quite loud at the India Impact AI Summit in February this year was, I suppose, the, the absence of, of, of clear. continuity between particularly that summit series and the, you know, if you think about something like Cop, there are. established sort of.
procedures for. review and.
procedures for. review and.
12:06
12:06
12:06
Speaker 2
-
Speaker 2
For some of the other forums we've been talking about today. Maybe I'll just add, you know, a couple of more specific aspects on the question of frontier AI risks. I think, you know, think about the current landscape of, of policy. proposals that fall into the category of things that. require broad international agreement and certainly agreement between the leading AI nations. I think one is the need for red lines, you know, and the I think an agreed. prohibition on the release of models that have certain. capabilities that are. considered unacceptable or, you know, post pose unacceptable risks and associated with that, you know, an evaluation ecosystem.
I mean, there's some progress in anthropic and OpenAI, for example, now having committed to having external.
I mean, there's some progress in anthropic and OpenAI, for example, now having committed to having external.
12:07
12:07
Speaker 2
-
Speaker 2
But we need to get that right. Right. There's a question of what kinds of access will external evaluators have , and will the results of the findings be meaningful and reliable enough. And so I think that in order to ensure that and in order to ensure that evaluation results in different countries are comparable, they need to be put on a, you know, on a regulatory footing. So agreed red lines, you know, certain things that are considered. unacceptable up to the point where, you know, those capabilities can be managed and, and, and the risks. Sort of mitigated.
I think another, you know, more ambitious but equally important agenda in the policy space, and I think Michael alluded to this earlier, is sort of the, the, the need to shift the burden of proof for systems to be safe on developers. Right. So at the moment, I think, you know, we're dealing with the narrative. You know, we look at the again, the OpenAI incident, right, of, you know, and the point is often made if you compare what happened there with what might happen in, in other industries or in non-ai space, you know, clearly that, you know, that would have legal ramifications.
So evaluation needs to be the first step. Red lines need to be first step. But in order to actually have the guarantees that, that we need for, you know, severe frontier risks not to occur or not to.
I think another, you know, more ambitious but equally important agenda in the policy space, and I think Michael alluded to this earlier, is sort of the, the, the need to shift the burden of proof for systems to be safe on developers. Right. So at the moment, I think, you know, we're dealing with the narrative. You know, we look at the again, the OpenAI incident, right, of, you know, and the point is often made if you compare what happened there with what might happen in, in other industries or in non-ai space, you know, clearly that, you know, that would have legal ramifications.
So evaluation needs to be the first step. Red lines need to be first step. But in order to actually have the guarantees that, that we need for, you know, severe frontier risks not to occur or not to.
12:08
12:08
12:09
12:09
Speaker 1
-
Speaker 1
Dealing with systems that we ultimately don't understand. Yeah. And just very quickly build on that. I think there's sort of two, two. categories of, I mean, we talk about frontier model risk. Part of that is risk that exists because that system running anywhere could potentially affect somewhere else, as we saw with, with the sort of. The attacks on, on. Australian government servers. Part of that is also because they are general purpose systems. So even when you're talking about last year's model, technically not the frontier anymore, the safety of those things is important because they are built upon within, for example, the UK.
So heavily further systems are built on them or with open source models where they're sort of fine tuned. So those yeah,
So heavily further systems are built on them or with open source models where they're sort of fine tuned. So those yeah,
12:10
12:10
12:10
Baroness Fraser of Craigmaddie
-
-
Baroness Fraser of Craigmaddie
Area, it's difficult to to pin it down. Baroness Fraser thank you, chair. I'm glad you just mentioned OpenAI and anthropic, because I want to ask about the private sector. And my question is, how. engaged are private sector actors in the. governance initiatives that we've been. discussing here in the UK? We've got the AI Security Institute, and they highlight their. collaboration with AI companies. But testing systems. prior to public launch. But, Mr. Birtwhistle, I love the fact that you raised the specter of companies not only marking their own homework, but setting their own exams. So how should the.
international fora engage with the private.
international fora engage with the private.
12:11
12:11
12:11
12:11
Speaker 2
-
Speaker 2
So, Doctor Osman, do you want to kick off? Yeah, thank you for the question. So I think in terms of, you know. engagement in simple terms, we see strong. engagement from the majority of AI labs. There are differences. So for example, meta is less engaged in many discussions compared to compared to some of the other leading AI labs. But if you look at, you know, for example, the work that AC is doing, there are strong .
12:12
12:12
Speaker 2
-
Speaker 2
As I mentioned earlier, you know, the OECD, for example, and the global partnership on AI relies heavily on. multi-stakeholder expert groups. And, you know, the large AI labs are typically reliably at the table for those discussions. Now, I would add, and I agree with the point that Michael made earlier, that, you know, that's a really important precondition. But of course, it's also important to be mindful of the fact that AI labs have their own interests, potentially interests in shaping, you know, the.
12:12
12:12
Speaker 2
-
Speaker 2
That take shape. I think a concrete. example for that. And, you know, an area where we really need attention at the moment is this. proposal or the move in US labs to open up, open the door, as it were, to external evaluators, right. And to suggest that having external evaluators evaluate models, including.
12:12
12:12
Speaker 2
-
Speaker 2
Is part of the. solution to the frontier AI governance challenge. The point there is that, you know, I think everyone can agree that that's an important step. But, you know, the devil is in the details, as it were, right, as it so often is. So, you know, the question is, what access exactly will evaluators get? Who are the evaluators? What is the pool of evaluators? You know, it's sufficiently broad.
Does it represent all the ranges of.
Does it represent all the ranges of.
12:13
12:13
Speaker 2
-
Speaker 2
You know, including from civil society, including, you know, from the Global South and so on, that we think need to be represented to have a comprehensive evaluation and so on. And so there is now an emerging, lively debate on, you know, all sorts of, you know, what does a profession of external evaluators look like? And it's clearly important, you know, to, to shape that conversation in a way that takes account of, of the broad range of interests rather than, you know, taking, you know, simply adopting a proposal put forward by AI labs, you know, without, without scrutiny or indeed, you know, by simply assuming that access is, is, is, is, is, is sufficient, right?
Sort of access is, is the first step. But the. effectiveness of, of this proposal depends on what happens after the access is granted and what, what you know, what data and what evaluations are .
Sort of access is, is the first step. But the. effectiveness of, of this proposal depends on what happens after the access is granted and what, what you know, what data and what evaluations are .
12:14
12:14
12:14
12:14
Speaker 1
-
Speaker 1
We've still got an issue there. So yes. So I suppose I think your question was, how responsive are they being? I think they're, they're, they're very much trying to define the terms on which they are governed. We, I mentioned the sort of reported draft bills shared by one company, I think, with a couple of national governments. We've seen very heavy lobbying at the state level in the US on the EU, AI act itself, on the codes of practice, and some delays that were in part due to their sort of influencing efforts. When we're thinking about.
collaboration with with bodies like AC. A couple of reflections.
collaboration with with bodies like AC. A couple of reflections.
12:15
12:15
Speaker 1
-
Speaker 1
It seems to be less certain than it was. I think it felt like there was some kind of. understanding between. The labs, the US. government and and UK AC. We have seen the. withdrawal of. pre-deployment sort of. access for a couple of models earlier this year. That seems to have been restored. It is unclear whether that is subject to some new arrangement. It's also worth understanding that when we're thinking about what. collaboration with AC means, AC has a very, very narrow set of. catastrophic and. security risks that it looks at, so it doesn't think about psychological and wellbeing harms.
It doesn't think about child safety, synthetic media abuse, bias.
It doesn't think about child safety, synthetic media abuse, bias.
12:16
12:16
12:16
12:16
Lord De Mauley
-
-
Lord De Mauley
Question from Laura Demoly. Thank you, chairman. You've been extremely helpful so far. Thank you very much. And we're approaching the end of the marathon. Let's focus on specifically a UK perspective. If you were a UK policymaker responsible for managing engagement with international initiatives on AI, what would be your three policy priorities?
12:17
12:17
12:17
Speaker 2
-
Speaker 2
Across different. initiatives and, you know, playing a proactive role in that. I think, broadly speaking, the, you know, again, this idea of concentric circles that work that's happening in concentric circles, acknowledging the important role of the G20, I think that's, you know, in the past been under under. utilized going back to the fact that the UK will have the G20 .
12:17
12:17
Speaker 2
-
Speaker 2
Next year, the G7 afterwards, I think there is a really important potential role for the UK to make sure there are lots of competing agenda items, of course, for for these fora right, and to make sure that AI is sufficiently high on the agenda and that there is concrete progress, right, that we move the needle forward rather than, you know, it's easy to have some language in a communique that that touches and acknowledges AI. The need for AI governance, but to really make sure and work towards, you know, advancing. You know, the, the addressing, addressing the needs in a way that that goes beyond current governance regimes.
And that leads me maybe to the two other priorities, sort of in substantive terms. So, you know, what are the, I think, areas where there's currently the most urgent need? And also, given the UK's background, the greatest. opportunity to be seen as a national leader in advancing the debate, I think the first one is.
And that leads me maybe to the two other priorities, sort of in substantive terms. So, you know, what are the, I think, areas where there's currently the most urgent need? And also, given the UK's background, the greatest. opportunity to be seen as a national leader in advancing the debate, I think the first one is.
12:18
12:18
12:19
12:19
Speaker 2
-
Speaker 2
That coalition currently doesn't include China. So there's an opportunity to bring in China and there's a need for, you know, for commonly agreed standards. There's a lot of, I mean, important work in AC that that contributes to that. But we currently don't have that international agreement on, you know, standards for evaluation. And I think that's a really important opportunity to make progress on that by building on the frameworks that AC has developed and then linking to that. You know, the point that we touched on when it comes to external evaluation, building, you know, touching and addressing this point of, you know, what kind of access to external evaluators need working towards a future where, you know, AC isn't the only external evaluator.
And we, and we need to end up with a, in a world where there's a wide range of external evaluation parties and, and how, how can we formalize the kinds of the ways of working and the modes of access that they need. And so. evaluation as one area and, you know, again, using different international fora to, to. Share and make progress on, on the questions that the UK has been working on. And then the second one is the, the other point that I mentioned earlier is, is this need to, I think, change the. paradigm of how we think about safety in the frontier AI space and to think about, you know, the burden of proof for for safety and where that should be placed and shift the policy debate to one where we think of safety as a.
requirement rather than, you know, a good faith effort that is being.
And we, and we need to end up with a, in a world where there's a wide range of external evaluation parties and, and how, how can we formalize the kinds of the ways of working and the modes of access that they need. And so. evaluation as one area and, you know, again, using different international fora to, to. Share and make progress on, on the questions that the UK has been working on. And then the second one is the, the other point that I mentioned earlier is, is this need to, I think, change the. paradigm of how we think about safety in the frontier AI space and to think about, you know, the burden of proof for for safety and where that should be placed and shift the policy debate to one where we think of safety as a.
requirement rather than, you know, a good faith effort that is being.
12:20
12:20
Speaker 2
-
Speaker 2
That that requires sort of confronting the the racing dynamic, right? I mean, at the moment, we're sort of stuck. geopolitically in a situation where it seems political decision makers are prepared to gamble, you know, with the severe, significant risks on the one hand and the potential, you know, significant gains of of very advanced AI systems. But I think we need to move away from that gamble. And we need to in order to do that, we need to , yeah, shift the burden on safety and, and.
introduce a paradigm for thinking .
introduce a paradigm for thinking .
12:21
12:21
Speaker 1
-
Speaker 1
Guarantees rather than good faith efforts. It's very much agree with, with, with, with that last point in particular. I think so agree that the UK has an opportunity to lead at the G20. And then with the G7, it said that it wants to focus on principles and standards. I think we think that's not. necessarily quite where like developing those in the first place is not necessarily where the the gap is. And the gap is more about sort of because those have been in development for a number of years. The gap is really about. operationalizing them.
12:22
12:22
12:22
12:22
Speaker 1
-
Speaker 1
As we talked about before, what can only be solved at the international and global level and picking forums on that basis. I think the second is to. understand the. limitations of operationalized principles and and technical. standards without harder incentives to comply. I think the sort of the, the, the debate around technical standards is going to absorb a lot of energy and political will over the next year. And there is a question about how far can advancing technical standards and. evaluations get you to. incentivizing the risk management to actually happen? So I think this is sort of one of the ways of managing that is, is sort of flipping the burden, as Florian was saying, and also who has had a say in writing them, as we as we said earlier.
And then the other the other kind of on the standards question is sort of shifting the. mindset and the. debate also. towards the idea of socio technical evaluation. And that can feel like quite an academic term. But what I mean is, you know, a lot of these. evaluations are very much about how systems. behave in lab settings before they interact with human systems, before they have humans operating them in the real world. And a lot more could be gained by thinking about them in that sort of more post-deployment context. And the last thing I think I'd probably be thinking about is, is the point I made earlier about sort of continuity.
So how can the UK build something concrete across those two. presidencies it has, including sort of processes, institutions to make sure that the torch gets passed on. And there is continuity in those international .
And then the other the other kind of on the standards question is sort of shifting the. mindset and the. debate also. towards the idea of socio technical evaluation. And that can feel like quite an academic term. But what I mean is, you know, a lot of these. evaluations are very much about how systems. behave in lab settings before they interact with human systems, before they have humans operating them in the real world. And a lot more could be gained by thinking about them in that sort of more post-deployment context. And the last thing I think I'd probably be thinking about is, is the point I made earlier about sort of continuity.
So how can the UK build something concrete across those two. presidencies it has, including sort of processes, institutions to make sure that the torch gets passed on. And there is continuity in those international .
12:24
12:24
12:24
Lord Robertson of Port Ellen
-
-
Lord Robertson of Port Ellen
Important social debates. Thank you. Thank you very much. And we're very grateful to you for the information that you've given us today, which will help us with the. consideration that we have to to give and the advice that we might give to the to the British government. So , Doctor Osman and Mr. Birtwhistle, we thank you very much for your contribution.
12:24