enter search term and/or author name
(written by Richard T. Snodgrass, March 2007)
As announced in an editorial, ACM Transactions on Database Systems has adopted double-blind reviewing, in which the identities of the author and the reviewer are not known to each other. That editorial provided a comprehensive analysis of the costs of double-blind reviewing (DBR) and a detailed examination of the decision.
The present document is an easy-to-read summary. For those readers who prefer TLAs (three-letter acronyms), the title may be restated as a "DBR FAQ."
The following is now TODS policy.
This policy is not dependent on absolute or even relative blinding efficacy. The central and unambiguous message is that every submission should be judged solely on its own merits. This message applies even when reviewers know exactly who the authors are. The other important message is that TODS so values fairness that it is willing to undertake additional effort by AEs to make the process more fair.
During the long gestation of this policy, several questions and concerns were raised. The guidelines provided on the TODS web page were designed to address these concerns.
Here, we provide responses to these questions to explain the policy, starting with questions from authors and then turning to questions from reviewers.
Details may be found in the author guidelines, included in the editorial and on the TODS web site.
Double-blind reviewing has been shown through numerous scientific studies to be more fair. TODS goes to great lengths to ensure fairness in the review process. DBR is just one component of this.
Authors need only apply six simple steps to blind their submission.
Yes, certainly. TODS continues to encourage such submissions, subject to the existing novelty and disclosure requirements listed in the author instructions on the TODS web site. In summary, the paper must have 30% new material, must explicitly describe the differences between the submission and the paper(s) it was extended from, and reference those papers, as anonymous citations.
Yes, you can. There are no restrictions on the dissemination of drafts or technical reports prior to or after submission.
Such papers continue to be encouraged. The anonymizing steps listed above have been designed to allow such papers to be submitted.
The paper should be first written with no concern for DBR. Then, right before the paper is submitted, it can be blinded by following those six steps, which will generally affect only a few sentences of your manuscript.
Blinding efficacy (the percentage of papers for which blinding was successful, that is, that a reviewer could not determine the identity of an author) varies greatly, depending on a mix of factors. Certainly if the submission is an extension of a paper published in a prevalent conference, blinding efficacy may be considerably reduced. Note however that in the worst case, the process merely devolves back to single-blind reviewing, which has been the practice for many years. And even in that case, the blinded cover page still emphasizes to the reviewer that the identity of the author should not be taken into account.
Hence, when the identity is not known or is uncertain, DBR affords the benefit of increased fairness. When the identity is known, we're back to the previous process, with the added reminder to the reviewer.
The reviewing process in place at TODS explicitly favors comprehensiveness of the review over blinding efficacy, via a set of stated principles.
This in my view is a source of much of the concern about DBR.
There is a prevalent feeling among prolific authors that their name on a submitted manuscript increases the chance of that paper being accepted. The scientific evidence does not support that view. A review of the scientific studies concludes that "These contradictory results render it impossible to say anything definitive about the impact of blinding on prolific authors."
So where did this perception come from? My guess is that prolific authors are more likely to write high quality papers (there is scientific evidence to support this); it is then the higher quality of their average submission rather than the identity of the author that increases the acceptance rate.
If you are a prolific author (or even if you are not), you should have nothing to fear from DBR, because such a process focuses the reviewer on the quality of your submission.
Yes, you should retain these citations. Steps 3 through 4 require such citations to be anonymized, again, following the guidelines on the TODS web site.
No. If you follow the six stated steps your manuscript will meet the anonymity requirements for submission.
The entire process of blinding your manuscript and preparing the cover letter should take only an hour or two.
If you as a reviewer accidentally discover the identity of an author, just continue the review. (Of course, you should not strive to unmask the paper.)
It is TODS policy that every submission should be judged on its own merits. Your task is to review the submission, not the author.
In that case, you should check the conflict of interest rules in the editorial or the TODS web site; these rules are identical to those for the US National Science Foundation. If you have any concerns, please contact the Associate Editor handling this submission.
If this work is included as an anonymous citation in the submitted manuscript and you need to check it, just contact the Associate Editor and request the full citation. Additionally, you are permitted to search the web and other sources if you feel that there is related work that has not been cited.
Over the past few years, TODS has adopted many innovative policies: a call for short papers and directed surveys, a limit of one review per year per reviewer, a guaranteed turn-around time of five months, full implementation of the ACM Rights and Responsibilities policy, and reviewing statistics published on its web site. Most of these innovations were at the time unique to TODS; some other journals are now following up.
These policies were somewhat controversial when first considered. What if the reviewer pool dried up given this promised limit? What if reviewers weren't responsive, causing the Editorial Board to violate its five-month guarantee? Over time, the community responded and everything worked out.
These past policies were enacted to increase fairness and quality. The double-blind policy furthers both of these objectives. The expectation is that over time the database community will become accustomed to the process and benefits of double-blind reviewing, as has occurred in other scientific communities.