Responding to a proposal to move the journal Political Analysis from double-blind to single-blind reviewing (that is, authors would not know who is reviewing their papers but reviewers would know the authors’ names), Tom Palfrey writes:
I agree with the editors’ recommendation. I have served on quite a few editorial boards of journals with different blinding policies, and have seen no evidence that double blind procedures are a useful way to improve the quality of articles published in a journal. Aside from the obvious administrative nuisance and the fact that authorship anonymity is a thing of the past in our discipline, the theoretical and empirical arguments in both directions lead to an ambiguous conclusion. Also keep in mind that the editors know the identity of the authors (they need to know for practical reasons), their identity is not hidden from authors, and ultimately it is they who make the accept/reject decision, and also lobby their friends and colleagues to submit “their best work” to the journal. Bias at the editorial level is far more likely to affect publication decisions than bias at the referee level, and double blind procedures don’t affect this. One could argue then that perhaps the main thing double blinding does is shift the power over journal content even further from referees and associate editors to editors. It certainly increases the informational asymmetry.
Another point of fact is that the use of double blind procedures in economics and political science shares essentially none of the justifications for it with the other science disciplines from which the idea was borrowed. In these other disciplines, like biology, such procedures exist for different (and good) reasons. Rather than a concern about biasing in favor of well-known versus lesser-known authors, in these other fields it is driven by a concern of bias because of the rat-race competition over a rapidly moving frontier of discovery. Because of the speed at which the frontier is moving, authors of new papers are intensely secretive (almost paranoid) about their work. Results are kept under wrap until the result has been accepted for publication – or in some cases until it is actually published. [Extra, Extra, Read All About It: PNAS article reports that Caltech astronomer Joe Shmoe discovered a new planet three months ago…] Double blind is indeed not a fiction in these disciplines. It is real, and it serves a real purpose. Consider the contrast with our discipline, in which many researchers drool over invitations from top places to present their newest results, even if the paper does not yet exist or is in very rough draft form. Furthermore, financial incentives for bias in these other disciplines are very strong, given the enormous stakes of funding. [Think how much a new telescope costs.] Basically none of the rationales for double blinding in those disciplines applies to political science. One final note. In those disciplines, editors are often “professional” editors. That is, they do not have independent research careers. This may have to do with the potential bias that results from intense competition in disciplines where financial stakes are enormous and the frontier of discovery moves at ‘blinding’ speed.
Tom’s comparison of the different fields was a new point to me and it seems sensible.
I’d also add that I’m baffled by many people’s attitudes toward reviewing articles for journals. As I’ve noted before, I don’t think people make enough of the fact that editing and reviewing journal articles is volunteer work. Everyone’s always getting angry at referees and saying what they should or should not do, but, hey–we’re doing it for free. In this situation, I think it’s important to get the most you can out of all participants.
I don't get it. What would be the advantage of single blind review? All reasons that I can think of why someone would want to know the author of a paper are exactly the types of things we should want to prevent. I agree that it's usually not a problem for the reviewer to know the authors- in fact, I just had a very positive experience with a reviewer who knew who wrote the paper – but what possible reason could there be to make that a general rule?
Why _would_ we want to introduce biases – bias in favor of senior scholars, against people Rs don't like personally, potentially even (unconscious) racial and/or gender biases?
If an R really wants to know the author of a paper – especially in polMeth – she can just google.
I'm not sure there's any strong evidence in favour of double-blind reviewing, even in biology. For me, a big argument against it is that it doesn't work – if a third of the time the author(s) can be identified correctly, is it really working? Especially as it's likely to be the Big Names that are identified.
As discussed in one of my own posts, I would rather favour the no-blind reviewing… Double-blind reviewing does not work in a World with on-line forums, on-line technical reports, and a growing tendency to deposit everything on arXiv.
As someone who reviews for economics & political science journals (amongst others) I think double-blind, while not perfect is preferable. Sure the editor knows the authors but the reviewer has an important input. So the best should not be the enemy of the good. What possible advantage can there be in the reviewer knowing who the author is? None. What business is it of the reviewer who the author is? None.
The idea that double-blind somehow shifts the power further from the reviewer to the editor is doubtful: the editor has all the power anyway.
Personally I prefer not to know the identity of the authors since I know then that I can't be biased, even subconsciously.
In economics we make work available as working papers a lot largely because of the long and uncertain lags to publication. So often, but not always, one could work out who wrote the paper if you felt so inclined. But this is much less common in other disciplines.
The additional hassle of making the process double-blind seems small for the possibility of making things fairer. Moreover it should be possible to make the process entirely blind by devolving communication with the authors to an administrator.
Maybe I misunderstand the point, not having read the original post, but a detailed problem with this analysis is that the proferred example of astronomy is NOT a discipline which currently uses double-blind reviewing. We always know the authors' identity, for all of the journals of which I am aware (e.g., The Astrophysical Journal, Monthly Notices of the Royal Astronomical Society, a nd of course Nature and Science). Not to mention the fact that most work in the field builds on the authors' previous work, so authorship is usually blindingly obvious anyway.
Sebastian, Kevin:
The point is that double-blind reviewing is costly. The key reason we want to get rid of double-blind reviewing for Political Analysis is that it takes a lot of effort that we'd rather spend in other forms of community service (for example, in writing this blog comment).
Even better than out-in-the-open reviews would be to get rid of journals altogether. Everyone could publish on something open like Arxiv. Everyone would be free to comment on anything. The bean counters could still calculate citation counts.
Like Andrew Jaffe, I find in my subfield of natural language processing that it's almost always obvious who the author of a halfway decent paper is.