Progressive Geographies

A interesting proposal here, trading on here.

Accepted, refereed papers and books should be accompanied with the name of the referees and, ideally, their reports, if only, in the online edition. (Note rejections can still be done anonymously.)

I hope this will have four positive consequences:

1. People can get credit (including for tenure and grants) for work done and this should increase number of responsive referees. Given the increased reliance on objective performance metrics (and the ongoing, general expansion of higher education in the world with Brazilian and Chinese universities soon sporting there own research orientation), there is an avelange of papers joining the refereeing pipeline.  This alone is putting stress on the system. The most likely referees are being overwhelmed with requests. (Short of charging submission fees and paying referees, this is probably best way to improve the system.)

2. It should increase transparency about intellectual networks. Small groups of incrowds have a tendency (perhaps unconsciously) to favor work done by friends or that cites them.

3. With publicity becoming the norm referees have an incentive to do diligent work, that is, try to catch plagiarism, make sure that relevant scholarship is cited, objections are raised, etc.

4. It makes the role and choices of of the editor subject of more careful scrutinity. Now they often appear as a neutral clearinghouse–but in their choice of referees and their interpretations of them they have considerable leeway.

Right now refereeing is organized like a secret priestcraft. The underlying message is, ‘trust us.’ But this will not do anymore. We should not give up on expert review, and we should promote the gift-giving culture that is a quickly vanishing within academia. Some mild transparency may actually go a long way here.

I’m open to the idea, because as I’ve said before (here and here), the current system is not working well. But, as an editor, my biggest concern is the difficulty in getting reports, and getting them in a timely manner, so we can process papers efficiently.

But I’m sceptical of the benefits here. I am not sure that 1 will work in the way intended; 2 and 3 sound good with cautions; 4 is the most interesting though has several problems.

1. Many journals already post lists of referees used in a year, so there is already one record of thanks. I keep a log of journals I’ve reviewed for, which is on my cv. Do we need to make the direct link between a single paper and a named referee to give appropriate credit? And, put crudely, are the benefits from being a referee under this model really likely to increase willingness to referee? I think that some pretty serious benefits would be needed to tilt things away from the place they are now.

2. This is a potential problem, but is dependent, in part, on editors to stop this. Transparency might make this better. But if you can reject anonymously, does the argument not fall down from the other side if this author isn’t part of the club, or is challenging a dominant position that referees wish to protect?

3. Perhaps, but is knowing that your report may become public going to improve things? Yes, in one respect, because if you agree to do it, you may do it more thoroughly, carefully and responsibly. At the very least, some referees may learn to spellcheck and proof their reports. It might also stop referees insisting that authors reference the referee’s own work. It might stop the needlessly negative (though not if rejections are still anonymous). But equally, more conscientious reports will take more time. And so, even good referees may end up saying no to more requests in order to spend more time on those they do, which would be counterproductive.

4. I have no problem in principle with my, and my co-editors, suggestions for referees on a paper being made public. But this cannot simply be the ones who actually did the review. It would need to be those who were asked, because the ones who actually do the reports might, to an outside observer, look a slightly odd mix. Why them, and why did you not ask Professor X who is the world expert on the case study, or Dr Y whose theoretical work would fit the aims of the paper? Well, maybe we did, maybe they said no, for whatever reasons, or maybe they didn’t reply. Maybe we asked five or six excellent referees before we found ones who were willing to do it. Unless the whole process is made public – which has serious downsides – then the view from the outside on the editors’ choices risks being seriously skewed.

Most generally, do editors have to secure agreement from all referees that their reports will become public? And what happens if an otherwise willing referee refuses that permission? Do we disqualify them from being a referee? Would we need to abolish the practice of reports for an editor and separate comments for an author? (I’m actually in favour of this generally, but there are exceptions.) Would junior scholars worry that their comments might count against them if they are critical of more senior scholars?

With books I see fewer problems, but again, there might need to be some careful thinking here. I’ve suggested books be rejected by presses that have then been published. I’ve been asked to endorse books, agreed, and then when I’ve read the ms. said that I’m not able to give a truthful, positive comment, and indeed recommended the book is substantially revised before publication. But by that point it’s usually too late and the book appears anyway. How much of that is made public?

I’m sure there are more issues. I’m really glad this is being discussed, and don’t want to dismiss suggestions like this out of hand. But there are some serious problems with the proposals. Most significantly it gets rid of the idea of anonymous peer-review. Anonymity is only for a limited period – until acceptance. The benefits that come from anonymity are going to be lost. That needs careful thought.