While still young, I was editor-in-chief of a new journal. Two papers were submitted by a young author who was in a running conflict with the established view in his field. I chose two members of the editorial board, leaders in the field, to review the papers. Since I was familiar with the topic, I also reviewed the papers.

Both editorial board members recommended rejection because the conclusions did not fit the established view. Though one reviewer was adamant, neither objected to the methodology or the data. Their quarrel was with the author's interpretation. Meanwhile, I had concluded from my own reading that the second paper was primarily a follow-up, adding little to the first. Now, with the reviewers' recommendations before me, I decided to reject the second paper and submit the first to another reviewer, an experienced member of the editorial board not expert in the field. I gave him the comments of the two reviewers without telling him what I thought. He recommended acceptance.

One of the other reviewers, the adamant one, must have heard what I was doing. He called about this time and threatened to resign from the editorial board if I accepted the papers. Did I feel tremendous pressure to give in to his threat? Certainly. For a young scientist attempting to build a journal and a career, what could be more dangerous than rejecting the advice of those who will soon sit in judgment of one's work? Nevertheless, 1 went ahead and published the first paper. The caller did not resign. And I heard nothing more of the decision.

I felt then, as I still do, that the purpose of peer review is not to censor the interpretation of data but to evaluate the methodology and the data, to determine whether the author's subject comes within the scope of the journal, and to suggest ways to improve the paper. The peer reviewer should judge the originality and thoroughness of the paper.

Still, the question remains, why didn't I give in to the caller's threats? Perhaps I did not because I myself had been the victim of peer review a few years earlier. I was then teaching at the Medical School of the University of Rochester. I had submitted a grant proposal to a federal agency. The official in charge of selecting reviewers called to say that it appeared that I already had a grant from another federal agency and that this grant was sufficient for a newly appointed assistant professor: would I withdraw my proposal?
Naive and ambitious, I said: No. The two grants were for different purposes. He and I knew many scientists who had more than one grant from more than one federal agency.

Polite, but blunt, he replied that he could select reviewers who would kill any grant proposal.

The discussion went like this a bit longer. Then I demanded that he put his request in writing and send it to me. No, he concluded, that was not necessary. The grant proposal would go through the normal "fair" process of peer review.

The proposal was denied funding. 1 submitted three more proposals on the same topic during the next two years. All went to the same official; all were rejected. Finally, I caught on. With my fifth proposal. I sent a covering letter asking that the proposal be assigned to someone else. The proposal was funded.

Now, a seasoned journal editor, I cannot reflect on my first hard decision as an editor without being aware of how much was at stake.

The future of my journal was at stake, of course. The journal was new, fragile. Rejecting the advice of established scientists could have meant their departure from the editorial board, loss of manuscripts submitted by them and their friends, declining subscriptions, and eventually the journal's death. Scientists in my field are a small community. They know who counts. They try to read what the leaders read and publish where the leaders publish. A journal can lose its reputation quickly.

The advancement of my science might also have been at stake. Science seems to advance by the constant assault of new ideas upon the old. Science is forever eating its children. The established leaders of a science, that is, the parents of those children, may be forgiven for finding such an assault objectionable, especially since most such assaults fail. All new ideas must begin unpopular, but most remain so. Still, the paper I accepted might have been important. An editor's duty is not to suffocate a good idea in its cradle.

That duty, though in part owed to science, is also, in part, owed to society at large. New science can have an impact on society, though the impact may be small and slow in being felt. An editor of a science journal serves the public by adopting a reliable procedure for determining what is worth publishing. Peer review, when it works, is certainly such a procedure. But, like any other procedure, it can fail.

The editor's role in peer review is to provide a check on reviewers, to keep an eye out for signs of bias, sloppiness, incompetence, and so on. An editor must be willing to step in-for example, by using a third reviewer-when the ordinary procedures seem to have gone awry. An editor must, of course, be careful not to go too far. Editors are as fallible as reviewers. The editor's role is to judge impartially between reviewers and authors, not himself to replace the reviewers.

How can an editor know that he has done this right? I don't think there is any general answer to that question. I myself take a "pragmatic approach". I ask: do I, taking into account everything I know about this decision, feel comfortable with it? It is this "comfort level" that decides what I will do. After all, whatever else is at stake in an editorial decision, my oven peace of mind, my own conscience, is.

Can we improve peer review in journals? Yes, of course. I believe we can make improvements on three different levels.

First, we can change public policy. Public: policy can be quite influential in setting standards across fields. A single regulation of the National Science Foundation or the National Institutes of Health can have a much greater impact on society than can several hundred journal editors acting individually. Think, for example, of recent regulations concerning plagiarism. Of course, the public policy enacted must be good. Bad public policy is as widely corrosive as good public policy is healthful.

Second, editors themselves can cooperate in setting standards. An individual editor may have trouble resisting the pressure of an important member of the scientific community. But he will have less trouble if he can cite a rule that all editors in his field have agreed to follow. We have a few such rules now (for example, on who should be listed as author). We should develop others.

Third, an individual editor can develop general practices that tend to head off trouble. While I like the current structure of having two reviewers, I think it important for the editor actually to read both reviewers thoroughly even if they agree. An editor should never be a rubber stamp.

I also think every journal should use an evaluation form that virtually forces reviewers to state specific reasons for acceptance or rejection. No one liners. If reviewers state their reasons, an editor can make a judgment on the reviewer's reasons rather than on whether the reviewer checked the "accept box" or the "reject box".

Editors should also routinely use some reviewers not in the paper's field but adjacent to it. Given how small many fields of science are, such a practice helps protect against the groupthink that small fields seem to nourish. Such a practice may also encourage the editor to think about the paper's importance to science generally rather than getting overly concerned about the minutia of interpretation within a field. Editors should have an explicit policy of not rejecting a paper based solely on its controversial interpretation of data so long as the "standard" interpretation is included as an alternative and the reasons for rejecting that alternative are given.

These practices could eliminate censorship and drastically reduce bias.

We also need to teach science graduate students, and even undergraduates, about the ethics of editing science journals. I might have found my first hard decision easier if, as a student, I had had a chance to think calmly about decisions like mine.