(12 years, 11 months ago)
Lords ChamberRead Hansard Text Read Debate Ministerial Extracts
Lord Hennessy of Nympsfield
To ask Her Majesty’s Government how extensions in the scope of the Freedom of Information Act 2000 and their transparency agenda will affect the availability of government records to the public.
Lord Hennessy of Nympsfield
My Lords, I very much welcome this evening’s debate on access to official information and I am grateful to so many noble Lords for wishing to speak. It is, I think, the first time that your Lordships’ House has had a chance to discuss the plans for additional openness announced by the noble Lord, Lord McNally, on 7 January 2011. He outlined four changes that the coalition proposed to make: increasing the number of organisations to which freedom of information requests can be made by bringing in such bodies as the Association of Chief Police Officers and the Financial Services Ombudsman; consulting on drawing in a range of further bodies, such as examination boards; undertaking post-legislative scrutiny to see how the Freedom of Information Act 2000 has worked in practice—a task which the Justice Committee in the other place will take up next month; and—especially dear to my historian’s heart—making most public records available at the National Archives after 20 years instead of the current 30 years.
I know that the noble Lord, Lord McNally, is keen on this. He has been a good friend to historians. He has always been willing to talk to my students about the 1976 IMF crisis when he became a prime historical exhibit while working in No. 10 for the greatly missed Lord Callaghan of Cardiff.
Perhaps I may this evening mirror the Government’s approach by taking a wider-lens view of freedom of information and look at it as a question of access to official information as a whole. To do so, our field of vision needs to encompass not just the workings of the Freedom of Information Act, for access is a matter of linkages embracing with FOI the output of the public records system and the often neglected, although related, matter of government-commissioned official histories—a subject on which the noble Lord, Lord Rodgers of Quarry Bank, led a debate in your Lordships’ House four years ago.
Before turning to public records policy, I must first declare an interest as president of the Friends of the National Archives and as Attlee Professor of Contemporary British History at Queen Mary, University of London. Given my trade, I especially welcome the coalition’s pledge to implement the provisions of Part 6 of the Constitutional Reform and Governance Act 2010, which was passed in the last days of the Labour Government and created a new 20-year norm to replace the 30-year rules that have been operational since 1972. The plan is that from January 2013 an extra year’s worth of declassifications will be added to the 30-year process until the 10-year gap has been closed.
I understand that the statutory instrument for triggering the new archival flow has yet to be signed. I am neither a pessimist nor a sniffer of conspiracies but I would welcome reassurance from the Minister on this point. In the same spirit, I would welcome publication of the 20-year rule guidelines for record reviewers in government departments—who are fondly known as “the weeders” —so that Parliament can check that they will be no more restrictive than the current ones for 30-year releases.
I am confident that the new 20-year rule will stimulate a cataract of fine theses and excellent books, for a fresh run of documents is like giving the historical profession a new currency with which to trade. This is precisely what happened in the 1990s after the noble Lord, Lord Waldegrave of North Hill, who I am delighted to see in his place, as Sir John Major’s Minister for Open Government, put in place what we historians called, and still do, the “Waldegrave initiative” whereby departments were encouraged to re-examine particularly sensitive files that had been retained for longer than 30 years.
By 1998, when Whitehall stopped measuring its yield, 96,000 files had been declassified as a result, which filled gaps in the defence, intelligence and nuclear elements of the post-1945 secret state. I am full of admiration for the departmental records teams in Whitehall and the staff of the National Archives. I am equally aware of the pressure on budgets and manpower, but I urge the Government to consider commissioning a “Waldegrave 2” to run alongside preparations for the 20-year rule to ensure that as little possible remains in departmental strong rooms, including files that were still too sensitive to release in the 1990s but that might safely be declassified now. If the Minister agrees, we might even call the initiative “McNally 1”.
Those 30-year releases are a form of delayed freedom of information. FOI disclosures are welcome and often highly revealing, but they are fragmentary and it is runs of documents that historians need. FOI, to be candid, is not an unmixed blessing for scholars because it has led to greater caution in what is written down.
I turn now to official histories. I have not written one myself, but I am very grateful to those who have, not least for providing a window into Whitehall short of 30 years, and, in Christopher Andrew’s authorised history of MI5 and Keith Jeffery’s history of MI6 up to 1949, opening up windows into the necessarily most opaque parts of the secret state. The Cabinet Office, under the energetic guidance of Mrs Tessa Stirling, is the engine room of official histories. Thirteen have been commissioned since 2000, eight of which have already been published. The Pilling report of 2009 urged still more and suggested enhancements in the commissioning process, while the Hamilton report of the same year laid out improvements in the marketing of the books produced. The Government have not yet pronounced on Pilling-Hamilton. I hope the noble Lord, Lord McNally, will this evening be able to accept the recommendations and undertake to implement them when funds allow.
For the general public the most visible manifestation of the public records system is the annual festival of 30-year revelations in the media between Christmas and the new year. This time, understandably, it was the riots of 1981 that attracted most attention. Few noticed a security file in the No. 10 papers of the noble Baroness, Lady Thatcher, dealing with a leak inquiry instigated by the noble Lord, Lord Armstrong of Ilminster, then Cabinet Secretary, into a story on civil contingencies planning that I had written in the Times as that paper’s Whitehall correspondent. I am glad to say that the investigation got absolutely nowhere, but its declassification, as I think my noble friend Lord Armstrong would agree, has brought a frisson of amusement to both of us—a kind of bond between us after all these years.
There is, however, a truly hidden treasure in the latest releases for those with a taste for personal and political drama, after, perhaps, seeing that remarkable film “The Iron Lady”. It is a Cabinet Office file containing the records of the third world war that never was of March 1981: a transition-to-war drill exercised in great secrecy in Whitehall every two years in which officials role-played Ministers. It ends with the United Kingdom under conventional and chemical weapons attacks from the Soviet Union and its allies, and the British War Cabinet reaching and crossing the nuclear threshold, with the role-played—I emphasise that—Mrs Thatcher declaring that never before had a Cabinet been faced with such a grim choice between capitulating to a powerful and malevolent aggressor and embarking on a course of action that could end with the destruction of civilisation. It is gripping, desperate and—mercifully—fictional stuff, which cries out, I think, to be converted into a film script.
To finish by returning to reality, I stress that well organised and sustained access to official information, current and past, is crucial to the accountability of our system of government and the richness of the historical residue that clings to the Velcro of our collective memory. Such practices enhance the depth and quality of the rolling national conversation about government policy and politics without which no open society can flourish.
Baroness Shephard of Northwold
My Lords, I have two rather narrow points to make in this valuable debate so brilliantly introduced by the noble Lord, Lord Hennessy.
First, it is obviously in the interests not only of historical accuracy but, I would add, of transparency in the democratic process that there should be access to Cabinet and other government papers.
Secondly, if such access is not well structured and organised, and accompanied by well observed conventions, the public will inevitably be more interested in the political disclosures involved than in greater historical understanding of the events concerned. I believe the increasing use of FOI requests illustrates this point.
The conventions are all-important. Without them, the noble Lord’s objectives of a clearer and more accurate understanding of the past could be at risk. The convention that a Government cannot examine the files of their predecessor was breached by Ruth Kelly as Education Secretary in 2006, when she sought to make public her predecessors’ decisions on teachers debarred from working with children—the famous List 99—with no prior consultation. This was regrettable. It was an enormously sensitive issue, which was, inevitably, hyped up by the media. This resulted in inaccurate press coverage, problems for children, for schools and for individuals and, in the end, a negligible increase in public understanding of the issues concerned.
There may not have been time to develop conventions with the Scottish Parliament. Something tells me that the Scottish Government might not be too interested in discussing these things, but their decision unilaterally to reduce the no-disclosure period to 15 years—again with no consultation with their Westminster counterparts, who, by definition, are the ones who will be affected—raises a number of questions, not least about political motivation.
It is obvious that the public interest is served by the orderly and properly structured publication of government decisions, but without observed conventions the more accurate and informed public understanding of past events that we all want could be threatened by short-term sensationalism and even political manipulation. The conventions and their observance are key in achieving the noble aims so eloquently described by the noble Lord, Lord Hennessy.
Lord Thomas of Gresford
My Lords, “Transparency”, said Sir Humphrey Appleby cynically,
“afflicts all incoming administrations. It used to be called ‘open government’, and reflects the frustrations they felt when they were in opposition and could not find out what was going on, combined with an eagerness to discover and publicise the deception, distortions and disasters of their predecessors … But it does not last beyond the first few months. As time passes they realise they have more to lose than to gain from public knowledge of what they are up to. Each month increases their tally of catastrophic misjudgments, pathetic deceptions, humiliating retreats and squalid compromises. They very soon come to understand that sound and effective government is only possible if people do not know what you are doing”.
I am grateful to the noble Lord, Lord Hennessy, for introducing this debate. It is to the credit of this Government that they have maintained the commitment in the coalition agreement to build on the Freedom of Information Act and to,
“extend transparency to every area of public life”.
The Act has been in full operation for upwards of seven years. When it was introduced, local authorities recorded information in a more traditional way. The right to data now being introduced will ensure that public authorities make electronically stored data readily available in a reusable form. I spent 10 minutes this afternoon looking at complaints about the potholes in the road outside my home in Gresford, which are published on the local open website and was happy to realise that, in the past six months, my very good local Lib Dem councillor had managed to resurface the whole of that area.
The Act has been extended, as the noble Lord, Lord Hennessy, said, and proposals in the Protection of Freedoms Bill will extend it to companies owned by a number of authorities. There are real social and economic benefits, apart from for the writing of history, to be gained by enabling businesses, non-profit organisations, volunteers and others to use freely the datasets held by public authorities for social and commercial purposes.
For my own part, I would extend the provisions of the Act to private companies carrying out public work paid for by the taxpayer. I agree with the Public Accounts Committee which said recently in its 44th report published last July:
“Transparency on the full costs and benefits of PFI projects to both the public and private sectors has been obscured … commercial confidentiality should not restrict the ability of the public, Parliament and decision makers to access information”.
I hope that the Minister will be able to comment upon that view; I am sure that Sir Humphrey would not agree.
Viscount Colville of Culross
I thank my noble friend Lord Hennessy for having secured this debate on such an important issue. I declare an interest, having been a journalist, both on newspapers and in television, for most of my career. Since the introduction of the Freedom of Information Act, I have seen at first hand the beneficial effect of its principal aim, that of improving the openness of public bodies to wider scrutiny.
The Ministry of Justice will consult 200 bodies carrying out public functions and receiving public funds which are not covered by the Act. I should like to focus your Lordships’ attention on several of these bodies.
There will be consultation with two examination boards, AQA and Edexcel. Edexcel is a private company which is the UK’s largest award-giving body; AQA is an education charity providing GCSE and A-levels to English and Welsh schools. I remind the Minister of the concern felt across the educational establishment about the aggressive manner in which these examination bodies compete for GSCE and A-level business. An investigation last year by the Daily Telegraph exposed how privileged information was being given by these bodies to teachers at feedback seminars. Ofqual is now looking into the matter. There is concern also about the uneven quality of marking by these boards. A survey last year by the National Association for the Teaching of English expressed great anxiety about the qualityof the marking of English at A-level. These processes and these boards need to be made open and public.
The Local Government Association and the NHS Confederation are also being consulted on the extension of the Act. The LGA has campaigned against the vexatious and expensive effect of FOI requests on its local authority members, citing a fourfold increase during the past six years. I would argue that this figure is evidence of the success of the Act. Certainly, it has revealed stories such as the 6,000 council houses lying empty in London last year and the £31 million in fines paid by utility companies for overrunning roadworks. In fact, the Constitution Unit found that 95 per cent of local authority FOI officers felt that the Act had brought increased openness to local government. If the LGA's members have so benefited, I suggest that their representative body would garner similar advantages.
The NHS Confederation is also up for consideration. In autumn last year, its trustees agreed a new transparency and accountability policy which would ensure that it was open and transparent to the public. However, the voluntary nature of this agreement means that there is no right of appeal to the Information Commissioner. An extension of the Act would rectify this omission.
I urge the Minister to bring the Justice Ministry’s consultation on these bodies to a speedy conclusion. The last exercise to extend the FOI Act was launched on 25 October 2007, but was not implemented until four years later. I hope that this new consultation will be carried out in a fraction of the time.
Lord Armstrong of Ilminster
My Lords, I am glad to be contributing to this short debate initiated by my noble friend and former adversary Lord Hennessy of Nympsfield. I say “former adversary”, because when he was the Whitehall correspondent of the Times and I was the Principal Private Secretary at 10 Downing Street, I was required by my political masters to see that Whitehall did all it could to frustrate his knavish tricks, designed to extract information about the working of government which government would have preferred not to disclose. He collected nuggets of information with indefatigable diligence, like Squirrel Nutkin collected nuts, but, unlike Squirrel Nutkin, he always knew where he had stored his nuggets and where to find them when he needed them.
Now that the noble Lord is no longer a mischievous journalist but a learned professor, and I am a mandarin long since put out to grass, we are firm friends. I can acknowledge that, though he did not win them all, he did win more than we could have wished, and that much of what he succeeded in extracting was relatively harmless if occasionally a little embarrassing.
I am much in favour, and always have been, of the greatest degree of transparency in government as is reasonably possible. That is owed by government, central and local, to Parliament and to the councils to which they are accountable, and to the people they represent. But freedom of information is not, at least in its current legislative form, an unmitigated boon and blessing to men. Tony Blair expressed the point in his memoirs in his own characteristically vivid style:
“You idiot”—
he says, addressing himself—
“You naive, foolish, irresponsible nincompoop”—
his words, not mine—
“I quake at the imbecility of it”.
“It” is the Freedom of Information Act. He goes on:
“Where was Sir Humphrey when I needed him? We had legislated in the first throes of power. How could you—
“you” is now Sir Humphrey, I think—
“knowing what you know, have allowed us to do such a thing so utterly undermining of sensible government?”
It is a well attested fact that, if I was the model for anyone, it was for Sir Arnold Robinson, not for Sir Humphrey Appleby. None the less, I am sorry that I was not around to be consulted in Mr Blair’s hour of need. I was already, unfortunately, well into my retirement.
I am afraid that there is no doubt that the risk of unwarrantable disclosure created by the Freedom of Information Act is liable to be damaging to the quality of governance. My successor, the noble Lord, Lord O'Donnell, has gone on record with his anxieties about the threat that the Freedom of Information Act presents to the usefulness of the minutes of the meetings of the Cabinet. Those minutes are not a verbatim record; they are none the less a comprehensive and accurate account of what the Cabinet decides and why. They are a valuable tool of administration. Their value depends upon their comprehensiveness and their accuracy. Their value would be diminished—they could even be misleading—if they had to be edited or bowdlerised to minimise risks of unacceptable disclosure under the Freedom of Information Act.
More generally, Ministers and officials, and indeed other people, now hesitate to put in writing things which are important, and which ought to be conveyed to their readers, but which they would not want to see having to be disclosed in response to Freedom of Information Act requests. The result is not only that discussion among colleagues is less candid than it should be but also that policy decisions may be taken on inadequate information.
The perverse effect of freedom of information legislation is thus to make important information less freely available where it is most needed and to impair the quality of governance. This is a problem that may well need to be addressed by amending legislation if its effects are not to become cumulatively more damaging.
I just add my support for the programme of official histories and my hope that the Minister will be able to assure the House that, despite the need for austerity in public spending, the programme will be maintained.
Lord Black of Brentwood
My Lords, I declare an interest in this subject as the executive director of the Telegraph Media Group. It is worth remembering the opening words of the 1997 White Paper on freedom of information, which began thus:
“Unnecessary secrecy in government leads to arrogance in governance and defective decision-making. The perception of excessive secrecy has become a corrosive influence in the decline of public confidence in government”.
Fifteen years on, such sentiments remain as forceful as ever. That FOI has perhaps not lived up to all the expectation is perhaps because, in my view, the 2000 Act did not go far enough. The Government’s transparency agenda, including the increase in scope of that Act, is a welcome step to remedying some of its initial deficiencies. I wish it went further now, to include all contractors performing public functions on behalf of public authorities—an extension all the more vital because of the contracting-out provisions of the Localism Act and the health service reforms currently before this House. Will the Minister look at that?
There are concerns—eloquently expressed by the noble Lord, Lord Armstrong—that FOI has had a negative impact on government decision-making by increasing the practice of what is termed sofa government, with damaging consequences for government record-keeping. As a historian, I am only too well aware that a comprehensive and accessible archive of government decision-making is a precious legacy to future generations. However, I believe that those concerns are misplaced. The last inquiry that looked into them—the review of the 30-year rule led by Mr Paul Dacre, working with the distinguished historian Sir David Cannadine—concluded that:
“We accept that ‘sofa government’ may have occurred at certain times in twentieth-century British history– indeed, long before FOI and the introduction of the 30 year rule; but we believe that it is more likely to be a reflection of leadership style and political circumstances than to be motivated by any concern regarding the timing of the future disclosure of official documents”.
For those interested in maintaining a comprehensive record of the deeds and doings of government, which is available to the public under FOI or the 30-year rule, there are far greater concerns than FOI, of which the most vital is the vulnerability of digital records. Most business is now recorded digitally. These records are subject to rapid obsolescence, with often a natural life of only five years. Already the oldest digital government records from the early 1990s are lost for ever, because appropriate software is no longer available or storage media is corrupted.
There is also the problem of digital information overload, with, in the words of the 30-year review,
“vast amounts of ephemeral information paralysing the system”.
Although digital technology may assist in making government more open, it has the paradoxical effect of making a permanent archival record far more difficult to establish, because documents disappear into digital landfill. Those two problems present a huge issue for the long-term maintenance of government records. That is a deeply worrying problem to tackle. It would help to hear from the Minister about what the Government are doing to ensure that electronic record capture is an integral part of government IT infrastructure—the key point for all of us who are concerned about the maintenance of government records.
Lord Wilson of Tillyorn
My Lords, in this very short debate, for which we are all grateful to my noble friend Lord Hennessy, I want to touch briefly on two issues. One is the effect on the Foreign and Commonwealth Office in particular of the Freedom of Information Act and the other is the consequences, again essentially for the Foreign and Commonwealth Office, of reducing the period of restriction on government records down to 20 years. I fear that on both I have a more suspicious and restrictive view than my noble friend Lord Hennessy.
Maximum openness of government is something that everybody wants, historians particularly, although my noble friend Lord Hennessy has managed to produce fantastic accounts of British policy and the way the government machinery operates without any changes in the way in which the records are dealt with. However, there are dangers, particularly for departments of state such as the Foreign Office. We always say we want officials to speak truth to power. However, will those officials be speaking as much truth if they think they are not talking just to power but to the whole population as well? Will they not, as my noble friend Lord Armstrong suggested, avoid written communications and get into huddles in corridors? Is that really for the benefit of the nation as a whole? Perhaps the Minister could comment on how this sort of danger might be avoided.
Then there is the problem of reducing the restriction of access to public records from 30 years to 20 years. Legislation is already in place to do that. Some years ago, I did a PhD on British policy in the 1920s towards the rise of nationalism in China. Most records were open, but there was one fascinating file that was closed for 70 years. It was the response by a senior Foreign Office official to a paper from the number two in the British mission in Peking, Owen O’Malley, around 30 pages long, about what British policy to China should be. I puzzled about what this reply from the Foreign Office said—clearly it revealed key things about British policy. I went to see Sir Owen O’Malley later. He said, “You must see the reply I got to my paper. It was so abusive that the Foreign Office has closed it for 70 years”.
That did not need closing, but I suggest that other things do. Imagine a conversation a young person in a British embassy has with a friend in a country. The friend is frank about the failings of his country and the people who are responsible. Later, he becomes a senior official. Does he want those sorts of things to be in the hands of his enemies, only 20 years later, when he has risen to a high position? I think there is a serious issue there. Perhaps again the Minister could suggest how “McNally 1”—which I think it is going to be—could somehow deal with this issue; lest, as I fear we might, with the best possible intentions, we finish by shooting ourselves in the foot.
Lord Bew
My Lords, I thank my noble friend Lord Hennessy for initiating this debate. My remarks will follow broadly in the spirit of his remarks and, like him, I have to declare an interest as a practising university historian, at Queen’s University Belfast, and as secretary of the All-Party Group on Archives and History.
There is an important Irish dimension to this question, relating to the decades of commemorations that are about to come upon us in Ireland: of the signing of the Ulster covenant, the Easter Rising and the war of independence. There are still important documents in Kew that have not been released in this context. I am interested because historians need to be armed. Quite rightly, there is a certain nervousness in both Governments about the emotional consequences of some of these commemorations. For some, they are a rather bad model—the argument is that the commemoration in 1966 of the Easter Rising played a role in triggering the subsequent Troubles. Historians need help, and we have formed an ad-hoc group of historians on a north-south basis, under the chairmanship of Professor Eunan O’Halpin of Trinity College, Dublin. We all feel that the more help that we get from Governments to release documents, the more valuable we can actually be.
I am not talking about spasmodic release. For example, when Tony Blair was Prime Minister, Bertie Ahern, the Irish Taoiseach, wrote to him and said, “Could we please have Roger Casement’s SIS files released?” and they were released. I am talking about something somewhat more systematic. There is one thing in particular that Her Majesty’s Government could do: they could get in touch with Commonwealth states—Canada, Australia, New Zealand—and say to them that they could safely declassify Irish material from the period 1913-23. We strongly suspect that it remains locked away because it contains security-related exchanges with London that no longer have any particularly poisonous dimension to them. I very much hope that the Minister, who has been a friend to historians, will help us in this matter.
However, I would like to add something slightly more cautious in the spirit of my noble friend Lord Wilson’s remarks about the move to a 20-year rule. I am certain that before long our political class will agree to this, and there is a reason for that. Even the great figures of our current political class, a Thatcher or a Blair, are at the top for only 10 or 11 years, so a 20-year rule is something that they can feel quite comfortable with. For good or for ill, there are no longer any Mr Gladstones, who sat in Peel’s Cabinet in the 1840s and was putting through the third Home Rule Bill in 1893. If we had politicians who spent 50 years at the top, I am certain that we would not be talking about a 20-year rule.
It is not politicians or their reputations that worry me; it is the young officials. I have talked to members of the committee that made this recommendation, I believe that it is the spirit of the times and I support it broadly. However, even under the 30-year rule young civil servants have been embarrassed by material that has been released. We need to be a little cautious about this matter.
Lord Turnbull
My Lords, the FOI Act has, as intended, brought important benefits to citizens by giving them information on decisions affecting their lives. However, one area of the Act is not working as intended—the so-called safe space. In all the discussions leading to the Act, the code of practice, the White Paper, the Bill itself and reports by committees, the need for a safe space was repeatedly acknowledged. The noble and learned Lord, Lord Falconer, speaking on the Bill in 2000, said:
“I should … make clear that many people on both sides of the debate consider that it is appropriate that policy making should not take place in a goldfish bowl: that there should be a process which allows Ministers, public authorities and civil servants to exchange views in a way that they feel will be private to give them that space to think and make decisions”.—[Official Report, 24/10/00; col. 282.]
When taken with the principle of Pepper v Hart that the courts should be able to look at the parliamentary record as well as at the specific wording of the Act, it is crystal clear what Parliament intended. Why, then, has the safe space been under repeated attack? It is because Section 35 does not confer an absolute exemption but requires a balancing public interest test. In applying that test, however, the commissioners and the tribunal have tended to focus narrowly on the information sought in the request, not the wider signal that the disclosure produces. Any release contains two forms of information: that inherent in the document, and that which provides signals how about the commissioner/tribunal are expected to respond in future cases. There will be cases where the information itself may cause little harm but where releases of similar documents could have a big effect on the behaviour of Ministers and officials. The commissioners and the tribunal appear to place little weight on this wider impact despite the advice of many distinguished people.
There is a further dimension. The Civil Service is required to give its best advice to Governments of different complexions. This will inevitably become more difficult if the advice given by named officials is revealed to successor Governments, a danger referred to in the papers by none other than the noble Lords, Lord Heseltine, Lord Mandelson and Lord Butler. There are two ways out of this: either the commissioner and the tribunal pay more heed to the original intention of Parliament or the use of the ministerial veto, always intended as a backstop, will, sadly, become more frequent.
Lord Cormack
My Lords, I can think of nowhere else where such a brief debate could contain so much. We are all enormously grateful to the noble Lord, Lord Hennessy, for the manner in which he introduced the debate, just as many of us are grateful to him for all that he has done for contemporary history.
I speak as one who for some 24 years was on the Royal Commission on Historical Manuscripts and sat on the council of archives, and who occasionally had to chair a panel to adjudicate on whether a certain document should indeed be released. I therefore have a great deal of sympathy with the general case that the noble Lord so brilliantly and wittily made when he opened the debate.
There are other things that we have to consider, though, and some of them have been touched upon in this debate. The importance of archives is such that we must not endanger them or their preservation. My noble friend Lord Black talked about the digital and electronic archive, which is something to which we have not yet devoted sufficient attention.
My noble friend Lady Shephard of Northwold talked very eloquently about the need for conventions. It is truly important that we have those. What we do not want is an incentive to destroy or an incentive for people to go into the back garden of an embassy, which I was once told about when I visited an iron curtain country. That was the only place where people dared talk. We need to have the conversations between Ministers and civil servants recorded and released at the appropriate time.
However, the appropriate time is not always necessarily after 15 or 20 years. Sometimes it has to be longer. Although the noble Lord, Lord Bew, said that we no longer live in the age of Gladstone, in which a statesman could be active for 50 years, maybe he is being a bit premature. We live in an age of longevity. At the moment we have three party leaders who could well still be active in politics in 20 years’ time. I should not like to think that they were being driven to make difficult decisions on the sofa or somewhere that are not adequately recorded so that the Hennessys of the future and those who look back, not as contemporary historians but as historians viewing a whole sweep of history, are deprived of essential evidence.
In this, as in all things, we have to get the balance right. If we take the judicious approach of the noble Lord, Lord Hennessy, tempered by the cautious words of the noble Lords, Lord Wilson and Lord Armstrong, we will have the ideal solution. I know that my noble friend Lord McNally has a reputation for being something of a Solomon. We will have to hear what he says to us tonight. However, I hope he will be able to assure us that he has a passionate care for archives, but complete archives, and that he will do nothing to damage in any way the material that future historians will need.
Lord Wright of Richmond
My Lords, I, too, congratulate my noble friend on initiating this debate. However, I should like to draw attention, as others have, to rather a different problem—namely, the extent to which the Freedom of Information Act in its present form and application, and the Government’s existing transparency agenda, may affect the readiness of public servants at home and diplomats abroad to record their frank advice to Ministers, even though there are exemptions in the Freedom of Information Act designed to cover confidential advice to Ministers and damage to international relations.
Does the Minister agree that there is a real potential constraint on officials if they believe that their confidential advice is liable to be open to early public access in spite of these exemptions, as illustrated by the recent decision to stop valedictory dispatches by departing heads of mission? I personally deplore this, having benefited three times from the experience and assessments of my ambassadorial predecessors. I regard the problem as being not so much the availability of government records, but whether important advice to Ministers is failing to be recorded for eventual inclusion in the National Archives. There is also a potential problem for future historians, unrelated to the Freedom of Information Act, in the failure to register many of the constant official exchanges by e-mail.
It is, I hope, an exaggeration—although I have heard it said—that the application of the Freedom of Information Act has led to a climate of fear in Whitehall, and to a marked reluctance by officials to put their frank and confidential advice in writing, whether by minute, letter, dispatch or valedictories. The Minister knows well, from our service together in 10 Downing Street many years ago, the importance that we both attach to a relationship of mutual trust between Ministers and their advisers. Does he accept that there is a real danger of that mutual trust being eroded by the excessive application of the Freedom of Information Act?
Lord Thomas of Swynnerton
My Lords, like other noble Lords I am grateful to the noble Lord, Lord Hennessy, for introducing this debate. I am also grateful to him for his work over many years in elucidating problems of great interest in English politics, particularly his discussion of how the nuclear weapons of this country were developed. He also developed well the idea of how we would manage to survive a nuclear attack in his book The Secret State, to which he alluded.
I have on one or two occasions come up against restrictions on the freedom of information. I remember writing a book and needing the text of the Non-Intervention Committee in Spain in the 1930s. I was told it was an official secret. I argued but I did not get very far until I found out that the Dutch version of these minutes and documents could be available to me if I went to Amsterdam. I discussed this with the Public Record Office, which eventually relaxed its control.
I had another experience once when, already a Member of this House, I tried to get the late Lord Dainton, then chairman of the British Library, to tell me how many people had visited the British Library— the old, noble British Library, which I still regret—the previous year. He told me—this is difficult to believe—that it was an official secret and I could not be informed. Recently, I have come across a different problem—if it is a problem. Letters that I wrote to the noble Baroness, Lady Thatcher, in the 1980s are now available for scrutiny in Churchill College, Cambridge. I do not mind that: I know that on one occasion I wrote a very important paper about the Ming dynasty in China because she was just about to go to China.
In this debate, we are trying to discuss where the line lies between the need for confidentiality: the need for Ministers, for civil servants and for private persons to have private conversations which are not leaked immediately; and the need which public persons, private persons, historians, journalists and others have for information. The difficulty of deciding this line has been touched upon very well by many noble Lords. I was particularly interested by the speeches of the noble Lords, Lord Wilson and Lord Armstrong, who pointed out the difficulty of immediate indiscretion, so to speak.
I feel have to sit down in a minute—I can feel a glare upon me. In conclusion, it is essential for public servants and politicians to be able to write down statements of policy and not just commit them to the telephone or to conversation. Dr Kissinger makes that point very strongly in an introduction to one of his volumes of memoirs and I very much agree with him. It is the written document which we need.
Baroness Royall of Blaisdon
My Lords, I, too, thank the noble Lord, Lord Hennessy of Nympsfield, both for securing this debate and for his own role, not just as an historian, but as a constitutional activist who has done a great deal more than most in pushing at the boundaries of official information.
First, I would like to endorse the proposal from the noble Lord, Lord Hennessy, for a “Waldegrave 2”. Just as the initiative of the noble Lord, Lord Waldegrave, helped historians understand our recent past, so, too, would a reconsideration of the issues now. Secondly, I would also endorse his call for the Pilling and Hamilton reviews on the official history programme to be implemented.
However, my main concern is the Government’s proposal to review the operation of the FOI Act 2000. Although the FOI Act was passed by a Labour Government, this does not of course mean that everything about it is perfect. The wriggling of this Government in relation to the Department of Health’s risk register is clear evidence of that. So we on these Benches—well, my colleague and I—welcome the establishment of a review of the 2000 Act, in the form of a post-legislative scrutiny of the legislation.
There is a continuing need for consideration about whether the Act has got right the balances it seeks to strike: for instance, between disclosure and operation in government; between transparency and the need to reduce regulatory burdens; between, in effect, good government and open government. In that respect, I welcome the intervention by the noble Lord, Lord O’Donnell of Clapham—whom I am delighted to see in his place—in his final remarks as Cabinet Secretary. My own experience in government suggests that there is indeed a need for proper policy-making space in government and also suggests that FOI, as a piece of legislation, has had some negative as well as positive effects. Discussion in government can be less open as a result of FOI. Fewer things are now written down in government as a result of FOI. These are not good outcomes either for good governance or for future historians.
Set against that are the clear and real successes of FOI, as detailed in the Government’s helpful memorandum on the Act, published last month, which will form an important part of the review of that Act. At present, the review is to be carried out by the House of Commons Justice Select Committee, chaired by Sir Alan Beith, the Member for Berwick-upon-Tweed. The Select Committee is a very fine body and its chairman a very fine chairman, but there is a case—a strong case—for the form of that inquiry to be expanded.
Today’s debate gives me the opportunity to propose that even at this late stage the vehicle for the review of the operation of the FOl Act 2000, first suggested by the Government in January last year, should be extended to a review carried out by a committee of both Houses. That would also be in line with the spirit of the Goodlad report, and the Leader of this House has always said that he is in favour of this House undertaking post-legislative scrutiny. The expertise of your Lordships’ House across a whole range of activities is clear.
A Joint Committee of both Houses might well be the best means of carrying out a review. Or perhaps, given that the work of the Common’s Justice Committee on the matter is already under way, there might be scope for that Committee to co-opt or include in some way as part of its process Members of your Lordships’ House, and this House should explore and pursue this matter further with the House of Commons.
The Minister of State, Ministry of Justice (Lord McNally)
My Lords, in the spirit in which the Leader of the Opposition intervened, perhaps I may say that it is an interesting idea to broaden the post-legislative scrutiny. The rules are that it is within the scope of the relevant committee to do that, but I will draw her remarks to the attention of Sir Alan. It would be useful if that committee made use of the experience and expertise available. One of my memories of the original Act was the ludicrous situation at that time whereby there was pre-legislative scrutiny by two committees—one from this House and one from the Commons—sitting at either end of the Corridor, with witnesses moving from one to the other.
First, I apologise to all noble Lords who have taken part. This should have been a three-hour debate that gave the eminent people who have contributed a proper opportunity. I will not do my usual practice of trying to refer to the individual points raised. Instead, I should like to provide a considered omnibus reply that I will send to each Member who has participated and put it in the Library of the House, because the issues raised are too important. If I tried to reply to each in turn, I would look up and all my time would have gone. With the permission of the House, I will do it that way.
I must say in passing that I have never seen so many mandarins in one place since my school production of “Chu Chin Chow”, and it was great to see them all out. I am of course immensely grateful to my old friend and colleague, the noble Lord, Lord Hennessy. He is quite right—it is about time that I got a PhD from Queen Mary college because of the number of times I see a letter beginning with, “Professor Hennessy has suggested you might be able to help with my research”. I am grateful in one way because I am in several footnotes to history around the various work that is done. My time goes back even further than his clash with the noble Lord, Lord Armstrong, because I was working for Harold Wilson when the Times, with great fanfare, announced this new creature that was going to pace the corridors of power: a Whitehall correspondent, one P Hennessy—to which Harold announced that any civil servant found talking to this new creature would be fired on the spot. Such was the spirit of the age.
The Leader of the Opposition pointed out that the Freedom of Information Act was the work of the previous Government. Let me put on record my admiration for the noble Lord, Lord Clark of Windermere, who produced the first White Paper on freedom of information and on which the Act was built.
I am grateful to the noble Lord, Lord Thomas of Gresford, for the quotation from “Yes, Prime Minister”. I do not think that any of us who have watched “Yes, Prime Minister” and “Yes Minister” and have actually worked in Whitehall and Westminster have ever considered it as comedy; we watched it more as documentary. In passing, I should say that I still have not worked out how in those days Sir John Hunt could manage to get from his office way down in 70 Whitehall to the foyer of No. 10 to greet the Prime Minister without any of the electronic devices that we have today to know that the Prime Minister was about to return, but now as I watch it on the cable channels I find the show still very pertinent.
As many Members are aware, I am a long-standing enthusiast for transparency and freedom of information. I therefore continue to be proud to have an opportunity to help shape the next phase of freedom of information in this country. I note what the noble Lord, Lord Armstrong, said with his vivid quotation from Tony Blair’s memoirs and indeed the valedictory from the noble Lord, Lord Macdonald, who I am very pleased to see in his place. They are matters and opinions certainly to be taken in evidence, although it leaves me, as a proponent of the Act, also to argue that when Prime Ministers and mandarins object, this Act might actually be doing something that it was intended to do.
The noble Lord, Lord Hennessy, has placed considerable emphasis on the Government’s pledge to replace the current 30-year rule with a 20-year rule. I hope the noble Lord will appreciate the enormity of the task. Central government departments alone hold more than 3 million files that will need to be reviewed during this process.
However, just before I depart from the points that were made by a number of noble Lords about the battle—and this has been the battle throughout—between the safe space and the culture of secrecy, all I would say is: let the Parliament system work. The Act has the process of post-legislative scrutiny built into it. Let this process tease out some of the weaknesses that have been named today, and let the Government look at them, and let those who believe that those weaknesses exist give evidence of them. That can be nothing but healthy.
The detailed work has now been done to develop an implementation plan that will balance our intention to reduce the 30-year rule with the burdens that this will impose. I reassure noble Lords, and in particular the noble Lord, Lord Hennessy—who asked for my guarantees on this—that the Government remain committed to this course of action and that further details will be announced in due course. Let me also give him a guarantee that that is not Whitehallese for some time, never. Not on my watch it won’t; we will press ahead with this.
I know that the noble Lord, Lord Hennessy, is also keen for there to be a new Waldegrave initiative and an expansion of the current programme of official histories. I fully recognise the benefits to historians that such initiatives bring and I am sympathetic to his view. However, the subsequent introduction of FOI since that time means that the public—including historians—are free to request any records. Moreover, the reduction in the 30-year rule will result in more and more government records being made available earlier year by year.
I hope noble Lords will recognise that such a significant undertaking as the reduction in the 30-year rule would make additional simultaneous initiatives very difficult at this time of financial restraint. I also hope that noble Lords will recognise the unprecedented level of transparency that was not available at the time of the Waldegrave initiative. However, I will continue to keep the suggestion under review and I pledge here and now that it will be called Waldegrave 2. I have no ambitions for it to be called anything else.
As for the official history programme, a good deal of work is already in progress, and I hope that we can review future work in happier economic circumstances. I emphasise again my enthusiasm for the programme of official histories. It would be a tragedy if we were to allow them to wither on the vine after 2013.
Hand in hand with FOI is the transparency agenda being pushed through government by my right honourable friend Francis Maude. The transparency agenda is about much more than historical information; it is about much more than government records in the traditional meaning of the phrase. It is about the information and data that we deal with day to day to inform our decisions and provide our public services. Making that information available is what makes the transparency agenda truly revolutionary. More information than ever is being published proactively by this Government. In excess of 7,500 data sets have been made available to increase accountability, empower the public and foster innovation and economic growth.
As noble Lords will be aware, the Protection of Freedoms Bill includes provisions that introduce a new right to data to ensure that public authorities, including government departments, make data sets available in a reusable format where they can and make them available for reuse when releasing them in response to requests or through publication schemes.
The Government will also be developing a transparency and open data strategy and plan to publish their response to consultation on the form that it might take early this year. There is a strong public interest in increased transparency by all bodies in receipt of public funds, including those in the private sector. The Cabinet Office is considering the type of bodies to which an open data policy will apply.
Many noble Lords will also be aware of the significant steps that we are taking to extend the Freedom of Information Act to more bodies through the Protection of Freedoms Bill and secondary legislation under Section 5 of the Act.
As I said in my opening remarks, I am a long-standing supporter of freedom of information and transparency. I am proud that, on my watch, the Act has been extended and the independence of the Information Commissioner strengthened. I am well aware that the FOI still has its critics among both Ministers and officials. It was never meant to make those groups comfortable, but as the Minister responsible I have played this matter by the book. My department has consulted widely on how the act works in practice. We have produced a report that, I believe—I am grateful for the noble Baroness’s comment—is a model of objectivity in the post-legislative scrutiny process.
That is how matters should work. It is now in the hands of the Justice Select Committee, which has in turn asked for written comment by 3 February and urged interested parties to give the committee their views. As I said, I will pass on the noble Baroness’s comments about widening the basis of the group considering that.
I am confident that FOI is robust enough to survive rigorous post-legislative scrutiny. It is that process that will expose any flaws, if there are any. In the mean time, I thank the noble Lord, Lord Hennessy, for initiating the debate and all noble Lords who have spoken. As I said at the beginning, the tragedy is that a speakers list of this quality and a topic such as this should be crammed in to the dinner hour, but I will write to noble Lords on the points raised. In the mean time, I am most grateful for all contributions.