At New APPS Catarina Dutilh Novaes reflects on a problem – ‘Reliable referees: a rare commodity‘.
With this post, I want to bring up for discussion what I think is one of the main issues with the peer-reviewing system… namely the extreme difficulties journal editors encounter at finding competent referees willing to take up new assignments. Until two years ago, my experience with the peer-review system was restricted to the role of author (and I, as everybody else, got very frustrated with the months and months it often took journals to handle my submissions) and the role of referee (and I, as so many others, got very frustrated with the constant outpour of referee requests reaching my inbox). Two years ago I became one of the editors of the Review of Symbolic Logic, and thus acquired a third perspective, that of the journal editor. I can confirm that it is one of the most thankless jobs I’ve ever had.
I can certainly relate to this – ten or more requests to get two useable reports is not at all uncommon. I wrote an editorial for Society and Space back in 2008, after I’d been editor for a couple of years. It was entitled ‘The Exchange Economy of Peer Review‘ (open access) – basically making the point that it is not editors, or publishers, who create the need for reviews, it is authors. And authors are also reviewers, which means that to make the system work we should do three times as many reviews as we make submissions – submissions, not published papers.
There are distortions in the system of course, because more senior people are probably asked more, and more junior people less. But I know I am not alone in reviewing far far more than that. In an average year, I accept and deliver on 30-40 review requests (articles, books or proposals, grants). This doesn’t count the papers I process for Society and Space, the ones I review for Theory, Culture and Society as a board member, or the grant applications I review as International Officer for the Geography section in the British Academy. I regularly think I need to say ‘no’ to review requests much more often, but I still feel guilty, because I know editors are struggling to get reports. I’m unconvinced by suggestions for paying referees or other incentives, but it is a problem. It’s undoubtedly part of the wider issue – recently raised in The Times Higher – about how ‘academic citizenship’ is under increased pressure.