Peer Review

Background
PSU Policies
PowerPoint Presentations
Case Studies
Online Learning Tools
Articles

Journal editors, especially in the sciences, rely on the expert opinions of knowledgeable researchers to ensure the quality of papers they publish. Granting agencies utilize a similar system in determining grant awards. The process of having submitted manuscripts or grant applications reviewed by objective experts in the field is known as peer review, and this process has become a cornerstone of scientific and scholarly publication. A recent international survey of over 3000 academics reported that 93% felt that peer review was necessary, and 90% said that the process improved the quality of published papers (1). On the other hand, a report published in the New England Journal of Medicine stated that ‘Editorial peer review, although widely used, is largely untested and its effects are uncertain' (2). And an experimental study in the journal Brain found that two reviewers agreed about a paper only slightly more often than would be expected by chance alone (3).

Peer review and confidentiality
Most peer reviews are conducted with some degree of anonymity. Typically, papers in the sciences are reviewed in a single-blind manner: that is, the author(s) of the paper is known to the reviewer, who remains anonymous. The humanities and social science fields more often use a double-blind system, where neither the authors' nor the reviewers' identities are revealed (1). There are arguments both in favor of and opposed to each system. Keeping the reviewer's identity secret allows the reviewer to be honest in their comments without fear of repercussions; it also may permit bias on the part of the reviewer. Double-blinded review may avoid the problem of bias, but it may be impossible to keep the identity of an author secret—a knowledgeable colleague, working in the same field of study, is likely to be able to deduce the author's identity from their work.

The fact that manuscripts are reviewed prior to publication, by researchers in the same field who may be competitors (or whose colleagues may be competitors of the author) can lead to ethical concerns about confidentiality or the data or research results. Reviewers working in a closely related field may obtain insights or ideas from the manuscript that could unfairly benefit their own research. In an extreme situation, a reviewer could delay publication in order to allow his or her own work to be published first. In a more typical situation, however, it can simply be difficult for a busy researcher to keep track of the source of ideas—did that great insight come during a conversation with my colleague, or from that grant proposal I read?

Conflicts of interest and peer review
Experts in the same field may be actively working in the same area, with overlapping research goals. Researchers in the same field may know each other, be friends, collaborators, or enemies. Financial conflicts of interest arise when reviewers may benefit from publication of papers concerning a product or process produced by a company in which they have a financial stake (e.g. own shares or stock, or are a paid consultant). Family members of the reviewer, or the institution with which the reviewer is affiliated, may also be a source of a financial (or other) conflict or interest for the reviewer. Reviewers are often busy academics, and reviewing (which is usually unpaid) may present a conflict of commitment, leading reviewers to spend too little time on the review, or pass the job on to a graduate student.

References:

1. Ware, Mark (2008) "Peer review: benefits, perceptions and alternatives" (PDF)
2. Jefferson et al. (2002) Effects of Editorial Peer Review: A Systematic Review. Journal of the American Medical Association 287: 2784-2786.
3. Rothwell and Martyn (2000) Reproducibility of peer review in clinical neuroscience: is agreement between reviewers any greater than would be expected by chance alone? Brain 123: 1964-1969

Resources