Higher Education and Research Bill

Lord Lipsey Excerpts

Committee: 4th sitting (Hansard): House of Lords
Wednesday 18th January 2017 Lords Chamber

[Viscount Hanworth (Lab)]

My Lords, I should like to testify that there is something utterly perverse in the current system of rating the quality of the provisions of individual departments within universities and of universities as a whole. The system depends on the National Student Survey, which aims to determine the degree of customer satisfaction. Because the ratings of the NSS are determined within these organisations, and because they can make no reference to what is happening elsewhere, they cannot possibly serve as a valid standard for comparison across the sector.

The NSS is subject to the social dynamics of small groups of students, and it can produce highly variable results from year to year. It is well known that it can be strongly influenced by the interaction of staff with students. There is a strong temptation for academics to appeal to their students, in ways that may be more or less subtle, to give ratings that will be beneficial both to themselves and to their students. This has often swayed the outcomes. Quite apart from these difficulties in assessing the true degree of customer satisfaction, it is questionable whether customer satisfaction should be the principle to guide the provision of teaching. It is now a principle that also guides many other aspects of the provision to students. The quality of sports facilities, catering, entertainment and much else besides has been influenced by the need to increase student satisfaction.

However, the effects on teaching of an adherence to this principle can be dire. It has been a common experience that, the more difficult a course and the more vigorously it is taught, the lower is its NSS rating. University administrators, who nowadays control the activities of academic staff, have requested the removal of courses that have scored badly. Among such courses have been some of the essential STEM courses, which often form the backbones of academic disciplines. I propose that we cease to use the NSS as a basis for assessing the qualities of universities. We should cease to make such assessments, or to use them, until we can be sure of their validity.

[Lord Lipsey (Lab)]

My Lords, I chair the Trinity Laban Conservatoire of Music and Dance, which I think is a very effective conservatoire. On Monday night I was closeted with my board, making one of the most difficult decisions that as chairman I have faced: should we go in to the TEF, which I think is supposed to close in about a week’s time, or not? The situation was simple. None of us thinks anything of it, particularly because of the presence within it of the metric of the National Student Survey, on which I will say a bit more in a minute and a lot more in our next debate.

But if we did not go in for it, we would have £250 less per student to spend on teaching, on instruments and on bringing them up to our very high standard. The board decided to go ahead. I very much hope that, before we finish with the Bill, they will be shown to have been right for a different reason—because the Government have backed off from these really very ill-considered decisions.

Incidentally, I endorse what the noble Lord, Lord Kerslake, said about Chris Husbands: if there is a man who can sort out TEF, it is Chris, and we should wish him every power and a fair wind from Ministers at his back.

I am a bit of a statistician; I chair the All-Party Group on Statistics. I will go into this in more detail on a subsequent occasion, as I said, but the NSS seems to be a statistic that makes the statement on the side of the Leave buses an exemplar of statistical validity. It is just frightful. In particular, for a small institution such as mine, the sample sizes are tiny. It has had the most coruscating reviews from the Royal Statistical Society. The Office for National Statistics put it more cautiously but nevertheless said the same thing: you cannot use it to compare institutions—which is exactly what the gold, silver and bronze ratings do.

This is the first time that a piece of legislation for the post-fact era, where facts no longer matter, has made it to the statute book. It must be changed. Fortunately, it can relatively easily be changed, because I think we are all after the same thing: we are after a true measure of teaching effectiveness. I do not mean just whether students like it. At one stage, I joked to my board that I was thinking of withdrawing all music teaching at Trinity Laban and instead providing free beer in the bar every night. They would be jolly satisfied with the quality of their courses if they had free beer every night, but they would not be learning to play their instruments—which is bloody hard work, I can tell noble Lords who have not tried it. For that reason, this metric is dotty.

I have one or two other points to make. Information is very important in the new era. It is difficult enough to choose an institution now and, if the Government get their way and there is a proliferation of institutions, it will be more difficult in future for students to choose institutions. One thing that does not help is misinformation. We did not do terribly well in the National Student Survey this year. It was fine for me because I was able to say, as I had pointed out every year to the board, that the previous year had been completely different, because this number fluctuates almost completely randomly. But I had members of staff who were reduced to tears and considering resignation because we had a bad NSS score. Think how much more that will be so if it is incorporated into the midst of the TEF. Managers would then say, “You have a very bad NSS score, so we will do badly in the TEF, so we will have less grant”. The pressure will be enormous, crushing and based on wholly false information. We need proper information and a proper TEF based on the kind of assessment that Chris, with his team, is well capable of undertaking. New metrics are being developed that would help with this, although whether they will be available under the Government’s timetable is not yet clear.

We can get a TEF that works, which I would welcome. There are institutions that have not been as successful in their teaching as they have in other aspects of their work. If it fulfilled the Conservative election manifesto in the process, that is the sort of thing that we have to put up with in life. But please do not let us take this false step of a phony TEF that will reward only those who are good at gaming these things, not those who are doing what we really want: teaching well.

[Lord Desai (Lab)]

My Lords, I was present at Second Reading, when I did not speak, and I was not going to speak on the amendment, but I would like to make some contrary observations to what has been said so far. The first time I saw students rating teachers was in 1961, when I was at the University of Pennsylvania. The anger of teachers then was more or less the same as the anger being expressed now: “How dare anybody judge us, especially our students? They are so stupid that they will not like difficult courses. They are so stupid that they will always go for the soft option”. I do not want to comment on the quality of the National Student Survey, but we ought to reflect on whether we are not respecting our students enough if we think that they are stupid and likely to hurt themselves by grading soft courses higher than hard ones.

Several problems are getting mixed up here. First, can teaching be evaluated at all? Some people think it cannot. I was involved in the first round of the research assessment exercise, and virtually the same arguments were made by academics: “You cannot grade research or compare it; it is very difficult”, and so on. This was being evaluated by their peer group but, by and large, we academics are rather conservative people when it comes to being judged by others. Ultimately, I think that the research assessment exercise performed a very good function. It mattered that some universities were five star and others were three or two: if they were three star or two star, they had to get their act together and improve. There is no reason to believe that something as important as teaching cannot be judged and therefore that there can be no competition because it is such a pure product that it is impossible to find a methodology to judge it.

--- Later in debate ---

[Lord Willetts (Con)]

My Lords, I shall comment briefly on some of the remarks about the NSS and perhaps try to address some of the concerns and offer noble Lords on both sides of the Committee a bit of assurance about what is happening here.

I should begin by drawing attention to the fact that I am a visiting professor at King’s College London, which sadly scores rather low on the NSS. I will not detain the Committee with a special pleading of why I think that is a completely misleading picture of the excellent work done at King’s. I also chair the advisory board of the Times Higher, which itself produces university rankings.

Surely what we are trying to do is embark on a journey towards what should be reliable metrics of teaching quality and learning gain. Of course, we do not have those yet. The question is whether we do anything now or wait until we have these superior and trusted metrics. The dilemma that one faces is that, back in 2010, there really was only the NSS, and it has been caricatured as simply a question of a student’s kind of, “What’s it like for you?”. We have already seen changes in the NSS and, if I may get into the technical language, it is becoming much more like the National Survey of Student Engagement which does try to get closer to the academic experience of the student.

The measure that will be used in formulating the TEF is not the generic question, “How was it for you?”. My understanding is that that is not what will appear in the TEF. There will be the earlier questions in the NSS. The NSS has more than 20 questions, and incidentally is completed by hundreds of thousands of students. It is the earlier questions that are closest to engagement that will be the ones used in the TEF. They are particularly questions about teaching on their course and on assessment and feedback.

The noble Lord who spoke for the Opposition when he opened said there had been no evidence that anything had been getting better. I can tell him that the fact that many universities have done disappointingly badly on assessment and feedback has led universities to change their practice and give students much more prompt reactions on their essays or other forms of work than they used to receive. I would argue that assessment and feedback are regarded as having genuine value and significance in the world of universities. Those measures are the measures extracted from the NSS which will be part of the overall metric for the TEF. The others which I think will have higher weight are the learning environment and student outcomes.

These are not perfect measures. We are on a journey, and I look forward to these metrics being revised and replaced by superior metrics in the future. They are not as bad as we have heard in some of the caricatures of them, and in my experience, if we wait until we have a perfect indicator and then start using it, we will have a very long wait. If we use the indicators that we have, however imperfect, people then work hard to improve them. That is the spirit with which we should approach the TEF today.

[Lord Lipsey]

Before the noble Lord sits down, will he explain what consolation he will offer to those institutions which are put out of business, at worst, while we perfect the metric that is being used in this case?

[Lord Willetts]

There are genuine questions, including the impact on overseas students, and I understand that issue. But I think that it would not be possible to envisage fees increasing without some kind of measures of the teaching performance in universities.

Given the difficulty of getting any measures, my view is that the measures we have are the best ones currently available. I think that the message that should go out from your Lordships’ House is surely that we would see them improved, changed and reviewed—and improved rapidly. It would be particularly regrettable—I know that I am turning to a later stage in our debate—if we bring in measures, if we amend the legislation, to make future changes in the metrics harder rather than easier by requiring a more elaborate process for them to be changed in the future. I am absolutely not saying that we now have a reliable and authoritative measure of teaching quality.