"Getting in" revisited: an analysis of manuscript characteristics, reviewers' ratings, and acceptance of manuscripts in Psychological Bulletin.

Abstract

Arizona State UniversityGetting published in the premier journals in psychology can bea daunting task, especially for the novice. Young scholars or thosewith limited experience publishing often ask editors what factorsaffect the probability of a manuscript being accepted and what theyshould do to enhance their chances of getting published. Of course,many factors affect whether a manuscript is accepted, includingwho the acting editor and reviewers happen to be and the qualityand orientation of the journal to which a paper is submitted.Nonetheless, it is highly likely that certain characteristics of manu-scripts contribute in a systematic manner to reviewers’ evaluationsof them and, ultimately, to a final editorial decision.At the close of Robert J. Sternberg’s editorship of PsychologicalBulletin, Sternberg, Hojjat, Brigockas, and Grigorenko (1997)published an analysis of reviewers’ ratings and their relation withthe final editorial decisions for manuscripts handled by the editorand one of the associate editors between January 1, 1993, andFebruary 29, 1996. In their analyses, they considered each item onthe reviewer rating form as addressing an analytical, creative, orpractical aspect of the manuscript. The analytically oriented itemsconcerned the extent to which “the authors completely and cor-rectly analyzed, evaluated, and reflected on the extant literaturereviewed” (p. 321). These items included completeness of cover-age, balance and fairness in the coverage of alternative views, andaccuracy of information. Thus, to score high on this dimension,authors needed to accurately determine what articles should bereviewed, analyze their views and findings in a balanced manner,and provide an accurate summary of the issue in question.Sternberg et al. (1997) included among the creativity items thosethat assessed whether the article made a novel contribution to thefield. The items that tapped this dimension included scientificimportance of the topic, theoretical orientation of manuscript, andcontribution to the field reviewed. A creative article, then, wasviewed as one that involved “taste and judgment in topic selection;. . . the formation of a new theory or application of an existingtheory to diverse scientific findings;...and asubstantial incre-mental contribution to a field at the time of publication” (p. 321).The items Sternberg et al. (1997) labeled as practically orientedconcerned the extent to which manuscripts reflected authors’ tacitknowledge of the features of a manuscript that make it publishablein a particular journal. For Psychological Bulletin, these included“generalized writing skills needed to communicate effectively andpersuasively to a diverse audience” (p. 321). The items consideredas tapping this dimension were scope of review; appropriateness ofmanuscript for journal; interest for a broad audience of psycholo-gists; existence and clarity of a take-home message; existence ofand persuasiveness in arguing for a well-articulated point of view;organization of manuscript; and clarity, coherence, and concise-ness of prose.Although Sternberg et al. (1997) initially classified their ratingitems into the aforementioned three classes, an exploratory factoranalysis of reviewers’ item ratings for initial manuscript reviews(not revisions) suggested two factors, which they described as theoverall appropriateness of manuscripts for any scientific journaland the overall appropriateness of manuscripts specifically forPsychological Bulletin. In addition, they found that all items hadstatistically significant polyserial correlations with the final edito-rial decision regarding acceptance versus rejection of a manuscript.The items most strongly correlated with the final editorial decision(in order of strength) were the reviewer’s recommendation regard-ing acceptance/rejection and contribution to the field reviewed,followed by appropriateness of the manuscript for the journal,existence of and persuasiveness in arguing for a well-articulatedpoint of view, completeness of coverage, and theoretical orienta-tion of the manuscript.Our goal in this article is to reexamine manuscript characteris-tics that affect reviewers’ recommendations regarding acceptanceversus rejection of manuscripts and the acting editor’s final deci-sion. Early in Nancy Eisenberg’s editorship, she devised a ratingsheet for reviewers that was used subsequently by all associateeditors. Many items on the full 13-item scale used under hereditorship paralleled those used on Robert J. Sternberg’s ratingform with minor wording changes, although multiple items werereplaced or rewritten to capture more specific technical aspects ofthe manuscript (e.g., quality of the analyses [if relevant]). Buildingon Sternberg et al.’s (1997) analysis and considering the changesin the rating form, we expected to find three dimensions ofmanuscript characteristics to be useful for predicting publication in