In 2017 I applied to be an assessor for the new Teaching Excellence Framework, and was appointed. I wanted to be part of it because I felt that the process would not represent a distance education establishment like the OU very well, and as someone who likes to promote widening participation I feared it would favour the usual Russell Group suspects.
In the end, my role as assessor had little impact on those areas, the OU did a lot of consultation with the TEF team, although the metrics still proved problematic for us. Before I list some of the criticisms, I will also highlight just how well the process was conducted. The TEF team, led by Chris Husbands genuinely cared about the process being fair, and the OfS team really, really understood data. I’m pretty confident that had they been consulted the problems with the A levels algorithm in the summer of this year would have been avoided.
So the good parts of the process were that we all got together for a few days in Manchester and worked in teams to assess the submissions. These had been evaluated by two independent assessors prior to this, and over the course of the days we would come to a decision that the entire team were happy with. Experts in employment, widening participation and statistics were on hand for any queries we might have.
It was an intense couple of days, and you could find yourself suddenly passionate about whether a small FE college deserved a Silver or a Bronze. There is one college (which shall remain anonymous) that was the hill I would die on, as I felt the PVC of a posh university wanted to mark them down. Similarly, there was at least one prestigious uni/college that deservedly ended up with a Bronze because their submission was basically “we’re X, fuck you”. But when you looked at the data, they really didn’t do a very good job of supporting students. So there was a sense of democracy about the outcomes. The process was very thorough and I’m confident that within the parameters of the exercise, all institutions ended up with a fair assessment.
But the process was inevitably problematic. At its very core it is the epitome of a neo-liberal approach to something as complex as education. That it could be reduced to a bunch of scores and then a rating. But even accepting this fundamental issue, the problem I have with it, is that it operates in distinct opposition to other desirable directions for higher ed. For instance, there is often talk about the need for a more flexible education system, where people can study what they need, in different sized chunks, or meeting the needs of different audiences, or experimenting with new approaches. All of these will have an adverse effect on your TEF rating however. Politicians, commentators, employers etc can’t complain about a system being slow to change while a system is foisted upon it that directly rewards a lack of innovation in many respects.
The OU for instance scores poorly on some metrics, such as student continuation because our students often take a break between study or may only study one or two modules. This is because that was all they ever intended to study, and so our offering has entirely met their needs. But in TEF metrics that counts as a failure.
On the plus side though, I was doing the round of university open days with my daughter last year. One of them had a Gold TEF which they proudly declared on banners around campus and the VC boasted about in their address to the prospective students. I nudged my daughter and whispered “I gave them that Gold”.
And also it allowed me to dig out this classic 1980s quiz show theme: