A post on the difficulty of getting reports, here; and a post on how it’s not really a huge amount of work, with interesting comments, here. Another post raises the issue of who is asked to referee. And finally, a proposal to abolish it altogether.
I’m unconvinced by the last. Yes, peer review has its problems, not least the generalised unwillingness of people to do it, often by the same people who are creating the need for it, i.e. authors. I’ve written about this in an editorial before, and recently wrote a short piece for the Society and Space open site about referees, following a piece by Felix Driver in Journal of Historical Geography. I sense that this will run and run, but here’s a paragraph from the Society and Space open site piece:
It is easy to be critical of reviewers, and there are, of course, bad reports and bad referees. Getting reports on a paper, getting those reports on time, and good reports, is one of the biggest challenges for journal editors and managers. But the idea is one that is difficult to envision being abandoned. It is increasingly fashionable to complain about the review, production and editing process of journals, and to praise open access publishing and blogs as means of disseminating information. While there can be very good editorial standards in open access journals – in related fields, see, for example Surveillance and Society and Acme: An International E-Journal for Critical Geographies – these are never ‘free’. They exist on the good will of editors, voluntary labour from others, and of course, on referees. If you think that the refereeing, editorial and publisher production process adds little, consider a journal that includes the next ten papers submitted to it, inappropriate or not, rough and ready, unedited and simply in the form they arrive, and compare that to the finished next issue of that journal. I can guarantee you are more likely to read the latter. Refereeing is an essential part of that process.
I wonder if there is something else the status getting into a peer-reviewed journal means – exposure for earlier career researchers. If everything was up on personal sites, or general, unreviewed depositories, then perhaps established people would have readers. But how would you tell whose personal site was worth reading; and in the general site among the huge amounts, why would you read X when you can read Y, perhaps on the same subject? You may have read a less good piece, but how would you know unless you read everything? I think that anything in Society and Space, regardless of author, is worth reading. And in terms of having different modes of review concerning material uploaded, it looks like it would be re-inventing the wheel: “other scholars (perhaps a pre-selected board?)”… “journal editors pick papers”… “what would a referee’s motivation be?”… “what if sites allowed qualified professionals to submit brief comments”…
The one thing that is compelling is the question of why review things twice. I’ve thought the same thing as I review tenure files for the US, write evaluation letters on published scholarship, not to mention the British research assessment exercise…
Volume 1 of Georges Canguilhem’s 
