Publication & Peer Review

Publication

In academic life, it is said, one must publish or perish. Indeed, publication is central to many disputes about responsible conduct of research. Publication facilitates the open exchange of information among researchers and exposes findings and methods to the the scrutiny of the community. It also documents who is first with new ideas or discoveries, shows productive use of research funds, and provides a record by which a research career can be judged. For these latter reasons, publication has a prominent role in advancement, promotion, and continued research funding. Scientists are under considerable pressure to publish.

In pursuit of publication and, more specifically, of credit, prospective authors can have serious differences of opinion about when to publish, what to publish, and how credit should be apportioned. Some of these issues are discussed below. Authorship and peer review demand special attention and are included as separate topics. Ultimately, the centrality of publication in academic life means that it is implicated to some degree in nearly all aspects of the responsible conduct of research.
 

Peer Review

For much of the last century, peer review has been the principal mechanism by which the quality of research is judged. In general, the most respected research findings are those that are known to have faced peer review. Most funding decisions in science are based on peer review. Academic advancement is generally based on success in achieving peer-reviewed publications and funding; further it involves direct peer review of the candidate's academic career. In short, research and researchers are judged primarily by peers.

The process is based on the notion that, because much of academic inquiry is relatively specialized, peers with similar expertise are in the best position to judge one another's work. This mechanism was largely designed to evaluate the relative quality of research. However, with appropriate feedback, it can also be a valuable tool to improve a manuscript, a grant application, or the focus of an academic career. Despite these advantages, the process of peer review is hampered by both perceived and real limitations.

Critics of peer review worry that reviewers may be biased in favor of well-known researchers or researchers at prestigious institutions, that reviewers may review the work of competitors unfairly, that reviewers may not be qualified to provide an authoritative review, and even that reviewers will take advantage of ideas in unpublished manuscripts and grant proposals that they review. Many attempts have been made to examine these assumptions about the peer review process. Most have found such problems to be, at worst, infrequent [e.g., Abby et al., 1994; Garfunkel et al., 1994; Godlee et al., 1998; Justice et al., 1998; van Rooyen et al., 1998; Ward and Donnelly, 1998]. Nonetheless, problems do occur.

Because the process of peer review is highly subjective, it is possible that some people will abuse their privileged position and act based on unconscious bias. Reviewers may be less likely to criticize work that is consistent with their own perceptions [Ernst and Resch, 1994] or to award a fellowship to a woman rather than a man [Wennerds and Wold, 1997]. Further, peer review does not do well at detecting innovative research or filtering out fraudulent, plagiarized, or redundant publications [reviewed by Godlee, 2000].

Despite its flaws, peer review does work to improve the quality of research. Considering the possible failings of peer review, the potential for bias and abuse, how can the process be managed so as to minimize problems while maintaining the advantages?
 

Rules and Regulations

Publication

Other than copyright law and federal definitions of research misconduct, nearly all aspects of authorship and publication are covered only by guidelines and unwritten standards. Many professional societies, scientific journals, and institutions have guidelines, the depth and scope of which are quite variable.

One of the most widely cited guidelines for publication is a document from the International Committee of Medical Journal Editors (ICMJE). In 1978, a group of medical journal editors met in Vancouver to establish guidelines for the format of manuscripts submitted to their journals. A product of that meeting was the ICMJE guidelines, Uniform Requirements for Manuscripts Submitted to Biomedical Journals, which have been expanded and revised to address ethical questions (ICMJE, 1997). These guidelines have been adopted by more than 500 biomedical journals.

Publication:  Copyright law

Extensive international and U.S. copyright laws regulate protection for writings such as research publications. For most published articles and book chapters, authors are required to transfer the copyright to the publisher. In practice, this means that authors of a published paper are in violation of federal law, not just ethical standards, if they attempt to re-publish without first getting permission from the copyright holder, the publisher. Therefore, unless one is legally advised otherwise, it is best to assume that it is never acceptable to reproduce previously published work without permission from both the author(s) and the publisher.

Peer Review

Peer review is governed by federal regulations in three respects. First, federal misconduct regulations can be invoked if a reviewer seriously abuses the review process. Second, peer review for the grant review process prohibit review by individuals with conflicts of interest. Third, a proposed requirement would make discussion of peer review part of instruction in the responsible conduct of research [Office of Research Integrity, 2000].

Despite these regulations, much of peer review is not directly regulated. It is governed instead by guidelines and custom.
 

Principles

Research is not complete until it has been reported.   Research is no contribution to the scientific community until it is reported, whether through publication or some other means. Reports are a necessary first step in the dialog with other scientists about the approach and significance of the work. Unless research is reported, it is impossible for others to build on what has been learned.

Most organizations reviewing research have specific guidelines regarding confidentiality and conflicts of interest. In addition, many organizations and institutions have guidelines dealing explicitly with the responsibilities of peer reviewers, such as those of the American Chemical Society (1996), the Society for Neuroscience (1999), and the Council of Biology Editors (CBE Peer Review Retreat Consensus Group, 1995).

Publication is not merely a matter of credit. One may wish to publish so as not to perish, but this alone is not enough to justify a report's being published. Publications should present some substantive and new result or analysis, and should not serve merely to increase the author's number of publications.

Credit for important new contributions to the scientific literature is assigned to the authors. Along with the privilege of credit, however, authors assume a responsibility for the integrity of what is being published. By placing their names on a publication, authors are attesting that the work was done as described. If this is found not to be the case, for instance if some of the data are found to be falsified, then the names of all the authors will be associated with a fraudulent paper.

Authors have a responsibly for assuring not only that the paper is free of outright fraud, but for assuring generally that it is free of misrepresentation. Although errors can occur, authors should endeavor to publish an accurate, complete, clear, and unbiased representation of their work, including background for their work, the methods used, the findings, the significance and contributions of the work, and fair assignment of authorship and credit.

Academic integrity depends on peer review. A published paper reflects not only on the authors of that paper, but also on the scientific community as a whole. High standards for academic work can only be maintained if scientists critically assess one another's work.

Effective peer review depends on academic integrity. Peer review must be conducted so that better scientific work is the result. Candidates for publication, grant funding, and career advancement have a right to a timely response from competent, unbiased reviewers.

Responsible peer review is a researcher's responsibility. By definition, peer review depends on the willingness of peers to participate as reviewers, usually without financial compensation. Participation in the research community thus involves a responsibility to share in the task of reviewing the work of peers. In addition to being an ethical responsibility, it should be noted that experience as a reviewer also has practical advantages. These include the opportunity to better understand the peer review system, to become more aware of the work of peers, and to develop lines of communication with other peer reviewers.

Publication Guidelines

Contributions to the literature

Dividing research findings into the smallest publishable units might increase an investigator's total number of publications, but works against the interests of science. Minimally, this is an inefficient use of scarce resources, including space in journals and the time of authors, editors, and reviewers. Furthermore, fragmentation of one study into many small publications can give the false impression that a line of research has been extensively pursued. Even so, a scientist's reputation is likely to be more respected if based on a few widely respected studies rather than on many small, fragmentary reports. No simple formula can determine the point at which a body of data warrants publication, but scientific literature benefits from the publication of manuscripts that represent new and substantial findings.

Redundant publication

Publication of data in more than one location gives the findings more visibility, but it may also mislead readers into believing that the publications represent distinct data sets. In the case of clinical findings, this could contribute to a false impression that some greater number of patients have been studied. In the case of basic research, readers could be misdirected into thinking that the study had been successfully replicated. Any data set, either in whole or in part, should not be published twice without making explicitly clear which of the data have been published previously and where and when the work was published.

Some exceptions to the injunction against redundant publication may be justified. It's acceptable to publish a paper that was published previously only as an abstract, although authors should disclose the prior publication and avoid the suggestion that the two represent distinct results. In some circumstances, the case can be made that two completely different audiences can be reached only by separate publications: for example, when a work warrants publication in two languages. Even in such cases, however, the editor and the publisher for both papers must approve the arrangement. In all of these cases, it is essential that the later publication make clear reference to the earlier work. Without such reference, the second publication constitutes a falsification of the research record.

For the submission of papers, most journals require that the work not be submitted simultaneously elsewhere for consideration. Submission of a paper is tantamount to provisionally giving the selected journal copyright to the work, and it initiates considerable expense of time and effort in reviewing the manuscript. Only when an article has been rejected by or withdrawn from consideration in one journal may it be submitted elsewhere.

Plagiarism

Authors take responsibility for both the ideas and words in a publication. For this reason, co-opting an entire manuscript is a clear example of research misconduct. While taking credit for someone else's research findings is clearly a greater wrong than copying their methods section, both are examples of plagiarism-- taking personal credit for someone else's words or ideas. To use the words of another author, either state where the original words can be found or reproduce the original text with clear and well-cited attribution to the original author. Even with proper citation, repeating the words of other authors is constrained by the fair use provisions of copyright law.

A difficult case of plagiarism is when it occurs between colleagues. When text has been co-authored, the question of ownership may be a matter of dispute. In some research groups, jointly written text is assumed to be shared property available for use by any of the original authors. Opinions about the extent of collective ownership of jointly written materials varies and it may be impractical to determine when plagiarism has occurred (see the Office of Research Integrity, 1994). It should never be assumed that it is acceptable to take credit for words written by someone else, and this issue should be openly addressed among collaborators.

Citation

The references cited in a research publication form the path that connects new work with the work on which it is built. Just as a thorough reference search is the foundation for responsible research, appropriate reference citation is the foundation for responsible reporting. Because future readers will rely on the references listed, an author has responsibilities to assure the accuracy of each citation so that readers can locate the referenced work, to include adequate references that document the origins of ideas, to verify that referenced works are consistent with the ideas and information credited to them, and to cite original sources whenever possible.

Statistical methods

Statistical methods are often used to describe data samples, to summarize results and relationships, and to test hypotheses. Usually, readers will not have access to the raw data and, therefore, will base any conclusions on the outcome of the statistical analyses. Because the assumptions and meaning of statistical tests vary widely, simply reporting a final P value (the probability of a false positive) tells readers very little unless they also know the methods of data collection and analysis. For these reasons, it is essential that authors not only design and analyze experiments appropriately, but also that they clearly and openly describe their methods (American Statistical Association, 1999).

Errata, corrections, and retractions

The decision to submit a correction or retraction is not an easy choice to make, and all authors of the paper should be informed. Despite any perceived risks, there are also advantages to an appropriate correction or retraction. Admitting error is typically perceived as a sign of integrity and concern for the highest standards in work entered into the published record. Conversely, failure to admit to an error can be devastating if the problem with the manuscript is first discovered by others. The integrity of the scientific literature is best served by rapid correction of misleading or mistaken information.

If errors are discovered after a manuscript has been published, then authors have several options, depending on the significance of the errors: (1) If minor errors are found to have been included in a manuscript, then a letter (singular erratum or plural errata) describing the errors should be submitted to the journal that published the article. (2) If unintentional errors are great enough to undermine part of a report, then the authors should submit a letter to the journal explaining the errors as a correction to the publication. (3) If unintentional errors are of such a magnitude as to invalidate or seriously undermine the entire report or if misconduct affecting the work on the part of one or more authors is found to have occurred, then the authors should retract the paper by writing to the editor of the publication.

Peer Review Guidelines

Timely response

Reviewers should make every effort to complete a review in the time requested. If it is not possible to meet the conditions for the review, then the reviewer should promptly decline or should see if some accommodation can be made. Research reports, grant applications, and academic files submitted for review all represent a significant investment of time and effort. Frequently, documents under review will contain timely results that suffer for a delay in the review process.

Competence

Reviewers who realize that their expertise is limited have a responsibility to make their degree of competence clear to the editor, funding agency, or academic institution asking for their opinion. A reviewer who does not have the requisite expertise is at risk of accepting a submission that has substantial deficiencies or rejecting one that is meritorious. Such errors are a waste of resources and hamper the scientific enterprise.

Bias

Reviewers' comments and conclusions should be based on a consideration of the facts, exclusive of personal or professional bias. To the extent possible, the system of review should be designed to minimize actual or perceived bias on the reviewers' part. If reviewers have any interest that might interfere with an objective review, then they should either decline a role as reviewer or declare the conflict of interest to the editor, funding agency, or academic institution and ask how best to manage the conflict of interest.

Confidentiality

Material under review should not be shared or discussed with anyone outside the designated review process unless approved by the editor, funding agency, or academic institution. Material submitted for peer review is a privileged communication that should be treated in confidence. While it is expected that the administrators and reviewers will have access to the material submitted, authors, grant applicants, and candidates for academic review have a right to expect that the review process will remain confidential. Reviewers unsure about policies for enlisting the help of others should ask.

Security

A reviewer should not take advantage of material available through the privileged communication of peer review. One exception is that if reviewers become aware on the basis of work under review that a line of their own research is likely to be unprofitable or a waste of resources, then they may ethically discontinue that work [American Chemical Society, 1996; Society for Neuroscience, 1999]. In such cases, this decision should be communicated to the parties requesting the review.

Beyond this exception, every effort should be made to avoid even the appearance of taking advantage of information obtained through the review process. Potential reviewers concerned that their participation would represent a substantial conflict of interest should decline the request to review.

Constructive criticism

Reviewers' comments should acknowledge positive aspects of the material under review, assess negative aspects constructively, and indicate clearly the improvements needed. The purpose of peer review is not merely to judge the submitted work, but also to promote better work within the scientific community. A review does not exist to demonstrate the reviewer's proficiency in identifying flaws, but to help the authors or candidates identify and resolve weaknesses in their work.
 

UAF Specific Information

No applicable information available at this time.
 

Works Cited

  • Abby M, Massey MD, Galandiuk S, Polk HC (1994): Peer review is an effective screening process to evaluate medical manuscripts. JAMA 272: 105-107.
  • American Chemical Society (1996): C. Ethical obligations of reviewers of manuscripts. In: ACS Ethical Guidelines
    http://pubs.acs.org/instruct/ethic.html
  • American Statistical Association (1999): Ethical Guidelines for Statistical Practice. Prepared by the Committee on Professional Ethics.
    http://www.amstat.org/profession/ethicalstatistics.html
  • CBE Peer Review Retreat Consensus Group (1995): Peer review guidelines - A working draft. CBE Views 18(5): 79-81.
  • Eastwood S, Fike JR, Cogen PH, Rosegay H, Berens M (2001): BTRC Guidelines on Research Data and Manuscripts (Brain Tumor Research Center, University of California San Francisco, 1989). Revised and updated in 2000 and reprinted in Bulger RE, Heitman E, Reiser SJ, eds.: The Ethical Dimensions of the Biological Sciences, 2nd ed. Cambridge University Press, New York.
  • Ernst E, Resch KL (1994): Reviewer bias: a blinded experimental study. Journal of Laboratory and Clinical Medicine 124(2): 178-82.
  • Garfunkel JM, Ulshen MH, Hamrick HJ, Lawson EE (1994): Effect of institutional prestige on reviewers' recommendations and editorial decisions. JAMA 272: 137-138.
  • Godlee F, Gale CR, Martyn CN (1998): Effect on the quality of peer review of blinding reviewers and asking them to sign their reports. JAMA 280: 237-240.
  • Godlee F (2000): The ethics of peer review. In (Jones AH, McLellan F, eds.): Ethical Issues in Biomedical Publication. Johns Hopkins University Press, Baltimore, MD, pp. 59-84.
  • International Committee of Medical Journal Editors (1997): Uniform Requirements for Manuscripts Submitted to Biomedical Journals. JAMA 277:927-34
    http://www.icmje.org
  • Justice AC, Cho MK, Winker MA, Berlin JA, Rennie D, PEER Investigators (1998): Does masking author identity improve peer review quality? JAMA 280: 240-242.
  • NIH (1997): Guidelines for the Conduct of Research in the Intramural Research Programs at NIH.
    http://www.nih.gov/campus/irnews/guidelines.htm
  • Office of Research Integrity (1994): Working definition of plagiarism. Office of Research Integrity Newsletter 3(1).
    http://ori.dhhs.gov/html/misconduct/plagiarism.asp
  • Office of Research Integrity (2000): PHS Policy on Instruction in RCR.
    http://ori.hhs.gov/html/programs/finalpolicy.asp
  • Society for Neuroscience (1999): Guidelines: Responsible Conduct Regarding Scientific Communication
    http://www.sfn.org/guidelines
  • Society for Neuroscience (1999): 2. Reviewers of manuscripts. In: Responsible Conduct in Scientific Communication.
    http://www.sfn.org/guidelines
  • van Rooyen S, Godlee F, Evans S, Smith R, Black N (1998): Effect of blinding and unmasking on the quality of peer review. JAMA 280: 234-237.
  • Ward JE, Donnelly N (1998): Is there gender bias in research fellowships awarded by the NHMRC? Medical Journal of Australia 169: 623-624.
  • Wennerds C, Wold A (1997): Nepotism and sexism in peer review. Nature 307: 341.