Progressive Geographies

Lord Nicholas Stern’s review of the Research Excellence Framework is now available here. For those outside UK higher education, the REF is the means by which academic research is evaluated for league tables and funding.

This independent review makes recommendations on the future operation of the Research Excellence Framework (REF). The review examines how university research funding can be allocated more efficiently so that universities can focus on carrying out world-leading research.

The key recommendations seem to be that –

• all research active staff should be returned
• instead of four pieces for all people, an average number (which may be different), with some above and below that number
• outputs are not portable – in other words, while you can hire someone, publications accepted prior to that date stay with the previous institution.
• metrics should be used to supplement peer review, not supplant it
• ‘impact’ should be understood more broadly

There are more recommendations, of course, but those seem to me to be the most important. It will take some time to digest. The idea that all research-active staff are returned is good, though there are still ways around this. But it might end some of the game-playing and difficulties of comparison – 66% return at 3.2 vs. 84% at 2.9, etc. The others all make sense (though I benefitted from the portability of outputs in the last submission), and let’s hope the government takes these very seriously. It doesn’t address some much more serious and deeper issues, and there is plenty making that case already. There is also the claim here that there should be no increased administration as a result of the REF and the new Teaching Excellence Framework. Let’s see that happen!

I’ve not seen many assessments of this report yet – please feel free to share links in comments.

Update: the ‘not portable’ idea will be a problem for people wanting to move, since their selling point of pieces accepted will no longer work in their favour. For high-profile hiring this may be the entire point, but it would count against anyone wanting to move, including it seems early career people coming to the end of a fixed-term position.

On looking at paragraphs 69 and 70, it seems that there is a recommendation to reduce the number of outputs assessed – if all research-active staff are assessed, it doesn’t want to increase the number of outputs assessed. ‘This may require the average number of outputs submitted per faculty member to be below 2, depending on the number of research active staff to be submitted.’ A drop to an average of 2, while likely to be applauded by some, could create some other difficulties. If two members of staff submit one and three pieces, respectively, how does workload take account of this?

There is a long standing madness, which I’m sure I’ve mentioned on the blog before, that each output is counted as one piece. So a journal article is one, as is a book. Obviously this bears no relation to the work involved in, or contribution to scholarship of these different types of outputs. In exceptional cases, a single piece of work could be submitted as double-weighted. I tried to make the case that, if anything I ever wrote was to be double-weighted, The Birth of Territory should be. But no, advice was taken and it was submitted as a single item. The argument was that I had more than enough other pieces, so it wasn’t necessary to double-weight it.

The Stern review does make this claim in paragraph 38:

Finding ways to ensure that the REF can encourage researchers to explore big or fundamental problems, in ways that may not deliver a steady stream of papers or a quick monograph; to deliver academically excellent synthesis of evidence and meta-analysis to support policy making; and to develop game changing ideas that, for example, can lead to the development of new disciplines, or that have significant impact outside their discipline, is a priority.

What a ‘quick monograph’ is remains unclear, though I wonder if they mean these increasingly popular short books (Palgrave Pivots, Sage Swifts, Chicago Shorts, Stanford Briefs, Minnesota Forerunners…) I’m not sure those are exactly quick, but they are a different type of output to a more standard length book of c.80-100,000 words.

Edited and translated books still don’t seem to figure as valuable outputs, despite the importance of such work, and the way such work is enabling of other work.

Some further thoughts are at Academic Irregularities and Wonkhe (neither can resist puns on Stern’s name, unfortunately).

Update 2: There is a very good detailed list of responses and other resources here.

Phil Steinberg has some thoughts on impact here; Campaign for the Public University here. More reflections – Martin Eve, Adam Golberg, Paul Kirby, Andrew McRae.