Anonymous Peer Research Articles

From the Editor

Dear Readers, It is a great honor for me to be chosen as editor of the Financial Analysts Journal. From its genesis in 1945, the FAJ has established a proud and strong tradition as one of the leading investment practitioner journals in the world. Not many journals can boast that they have published 14 articles by Benjamin Graham, 19 articles by Fischer Black, 13 articles by Bill Sharpe, and 11 articles by Peter Bernstein. If one considers the many other authors and articles represented in the journal's 57-year history, the accumulated wisdom in the FAJ is staggering. The FAJ is, at its core, a journal for you, the community of investment practitioners. AIMR and I are unswerving in our commitment that the FAJ will continue to play a key role in our industry as a link between innovative researchers and thinkers and the broad practitioner community. The FAJ's purpose is spelled out in its mission and standards: FAJ Mission and Editorial Standards The goal of the Financial Analysts Journal is to advance the knowledge and understanding of the practice of investment management through the publication of high-quality, practitioner-relevant research. The Financial Analysts Journal should serve asa bridge between academic research and practice by publishing academically rigorous articles that have direct relevance to practitioners, a forum for presenting practitioner research, and a forum for the perspectives of leading practitioners, academics, and regulators about our industry. Striving to meet these high expectations, the FAJ operates according to written quality, relevance, and readability standards. As an example, articles submitted to the FAJ are subject to an anonymous peer review process that provides an independent assessment of quality and relevance. Additionally, we work with authors to produce readable articles written in a style that is appropriate for a practitioner (rather than an academic) audience. We will, of course, continue to seek papers of the highest academic standards; we will be particularly alert to such opportunities when they are targeted to the practitioner community. I urge you to visit the FAJ Website at aimrpubs.org for details about our editorial standards. Without departing from the double-blind process, in which authors of articles do not know who their referees are and referees are not told the identities of the authors, we intend to increase the transparency of the review process. We will inform authors how their papers were judged by the referees and how the final decisions on their papers were reached. Naturally, it takes several issues for a journal to fully reflect the goals and priorities of any new editor. In this case, the situation is complicated by the fact that I and various co-authors had several articles that were submitted before the start of my tenure as editor. I will play no role whatsoever in the final editorial process for these papers; I will leave those decisions to the capable people at AIMR. I will not be submitting any new articles to the FAJ, but I may contribute some short perspectives pieces, which will be subject to independent review without my involvement in any way. In addition, I will not vote on the Graham and Dodd awards for 2002 or 2003. All of us working on the FAJ recognize that the reputation of the journal depends, in part, on its commitment to high standards of ethics and professional conduct. To help us maintain these standards in the selection and publication of articles, all authors, advisors, and reviewers, as well as the editor and editorial staff, will continue to abide by the FAJ's conflict-of-interest policies and ethical standards. These policies and standards are available on the FAJ Website. I am delighted to introduce to you a new group of people who will help the FAJ maintain its standards—the FAJ Advisory Council. They are listed on this issue's masthead and on FAJ's Website. The Advisory Council consists of luminaries who have made a substantial contribution to our industry; their key role is to advise the editor and AIMR on the proper direction of the FAJ. The masthead also lists our hard-working Editorial Board, which serves as our main source of reviewers. Some Advisory Council and Editorial Board members have previously served on the board; some names are new. All are leaders in their fields, and I thank both groups for their willingness to serve. In addition to their other duties, we are encouraging Advisory Council and Editorial Board members to submit perspectives pieces based on their decades of experience in the field, to submit cutting-edge, relevant research articles, and to help us identify outstanding practitioner-oriented papers and innovative ideas for future publication. I am looking forward to serving you.

Problem-based writing with peer review improves academic performance in physiology.

The aim of this study was to determine whether problem-based writing with peer review (PW-PR) improves undergraduate student performance on physiology exams. Didactic lectures were replaced with assignments to give students practice explaining their reasoning while solving qualitative problems, thus transferring the responsibility for abstraction and generalization to the students. Performance on exam items about concepts taught using PW-PR was compared with performance on concepts taught using didactic lectures followed by group work. Calibrated Peer Review, a Web-delivered program, was used to collect student essays and to manage anonymous peer review after students "passed" three calibration peer reviews. Results show that the students had difficulty relating concepts. Relationship errors were categorized as (1) problems recognizing levels of organization, (2) problems with cause/effect, and (3) overgeneralizations. For example, some described cells as molecules; others thought that vesicles transport materials through the extracellular fluid. With PW-PR, class discussion was used to confront and resolve such difficulties. Both multiple-choice and essay exam results were better with PW-PR instead of lecture.

The publishing cycle

As I prepare this editorial during the last week of January, I am aware that it will not be published and therefore read until June 2002. It is remarkable to think that when this issue is distributed some of you will be fortunate enough to be on your way to Stockholm for the 13th World Congress of Occupational Therapists. I plan to take the opportunity to attend a meeting of international occupational therapy journal editors while there. This meeting will provide an opportunity for the editors to discuss the common issues relevant to maintaining a high quality professional journal. One of the major problems we are encountering here in Australia is the time taken for manuscripts to be published. This results in frustration for authors and also the Editorial Board, and means that, at times, published research has become less current. Part of the problem is due to the fact that we have such enthusiastic and prolific authors and, because of high costs of publishing, a limited page allocation per issue. This year, due to increased membership, we have gained an extra eight pages per issue, equivalent to one feature article. Because of the demand for space, we are currently investigating the feasibility of publishing more issues per year. It is important for readers to know why there is such a long lead-time when publishing a quarterly journal. As the journal is a peer-reviewed publication, this contributes to the length of time from submission to acceptance for publication. The process is as follows. Following submission, as Editor I select two referees for anonymous peer review of the article, these referees then have six weeks allocated to review the manuscript and return their comments to the editorial office. We have found over the last 12 months that the referees have been very efficient, with few reminders being required. If minor or major changes are recommended, the manuscript is then returned to the author with the referees’ reports. Manuscripts which are not of the required standard or relevance are also returned to the author with a covering letter. It is at the revision stage that delays appear to be occurring, with authors taking a long time to return their amended manuscripts. If a major rewrite is required, then the manuscript is reviewed again by the referees, and the process is repeated until the paper is accepted for publication. There are many stages involved in the editorial process once a manuscript has been accepted for publication, this is the Production Process. For example, for this current June issue, we began planning in December 2001. Articles were selected based on a variety of criteria, and sent to the Production Editor by mid January, any queries were then emailed to authors by early February with replies required within 10 days. The authors then received proofs by email in late March, with corrections being required by early April. The Synergy online publication date for the June issue was 23 May, and the June issue was dispatched to subscribers at the end of May. As can be seen, the production process takes time and a great deal of hard work by the editorial staff and the authors. There are many reasons for the length of time taken for manuscripts to reach the publication stage. However, with all the due processes in place and carefully adhered to, it ensures that we have a high quality and well-respected journal. As occupational therapists have now become more involved in research, the dissemination of their findings becomes essential: to provide evidence, for the sharing of information and to generate further areas of research. Through presentation of research at professional meetings and conferences, and publication in professional journals we will increase the occupational therapy research profile and add to the evidence base of our profession. In addition to research findings, theoretical and feature articles, we have other sections in our Journal, such as book and media reviews, ‘In Practice’ and ‘Viewpoint’, which, we hope, provide a wide range of topics to stimulate the interest of occupational therapists in their many areas of practice.

Teaching peer review and the process of scientific writing.

Many undergraduate and graduate students understand neither the process of scientific writing nor the significance of peer review. In response, some instructors have created writing assignments that teach or mimic parts of the scientific publishing process. However, none fully reproduced peer review and revision of papers together with the writing and publishing process from research to final, accepted draft. In addition, most have been instituted at the graduate rather than undergraduate level. We present a detailed method for teaching undergraduate students the full scientific publishing process, including anonymous peer review, during the process of writing a "term paper." The result is a review article in the format for submission to a major scientific journal. This method has been implemented in the course Cell and Molecular Biology for Engineers at the University of Virginia. Use of this method resulted in improved grades, much higher quality in the final manuscript, greater objectivity in grading, and improved understanding of the importance of peer review.

Web based peer assessment: attitude and achievement

The specific features of the Web-based peer assessment are to utilize Internet resources to facilitate contacts between individuals and information, to assist in brainstorming among individuals, and to generate more meaningful learning at the higher education level. In this research, authors focus predominantly on attitudes of computer science students toward Web-based peer assessment using NetPeas as the interactive channel and management center. NetPeas is a Web-based peer assessment system implemented for two-way anonymous peer assessment. In an evaluation held in spring 1999, this study recruited a sample of fifty-eight computer science undergraduate students enrolled in an operating systems class in a research university of Taiwan. Attitudes toward Web-based peer assessment were measured by a post-test questionnaire, including several affective components, for example, "satisfied" or "unsatisfied" about the Web-based peer assessment. The result demonstrated that; (1) significantly more students favored this new learning strategy; and (2) students with positive attitude outperformed those with negative attitude. No matter positive attitude toward Web-based peer assessment brings about higher achievement or higher achievement promotes positive attitude, teachers must take care of students' subjective feelings in enhancing effective Web-based peer assessment.

Online Peer Evaluation in Basic Pharmacology

Objective: An online, Web-based peer evaluation was designed to enable students enrolled in a basic pharmacology course to complete an anonymous peer evaluation reflecting students' contributions to a variety of small-group class activities. Description: As part of the ongoing curriculum renewal at the University of Missouri—Kansas City (UMKC) School of Medicine, small-group activities were incorporated into a historically successful lecture-based course in basic pharmacology in 2000. The class was divided into six small groups (six or seven students per group) and over a two-month period each group completed three activities: They solved six problem-based learning cases, they wrote a group paper, and they delivered a formal group presentation on an assigned topic using an evidence-based medicine approach. Small-group activities were introduced into the class to address three objectives in UMKC's competency-based curriculum: problem solving, effective communication, and lifelong learning. A peer evaluation was introduced into the course to allow each student to complete an anonymous peer evaluation for every student participating in the group. The evaluation allows students to rate the contribution of each group member to the overall success or failure of the group effort, and it addresses a concern expressed by students that all group members receive equal credit regardless of individual effort. The results of the peer evaluation were used in assigning student grades. The online evaluation was prepared using FrontPage 2000, and students were able to complete it on any computer with Internet access. The evaluation consists of 15 items, which students rate using scores of 1 to 5. To complete an evaluation, each student enters his or her social security number and selects a group member by selecting the group member's name from a drop-down box. These two pieces of information are collected to ensure that each student submits only one evaluation for each student in the group and to facilitate data analysis. Many of the items address issues of professionalism. Each student receives a summary of his or her peer evaluations within 72 hours via e-mail, and in addition to the personal scores the students receives an aggregate score for their group and for the class for each evaluation item. The online peer evaluation allows students to add anonymous written comments, which are also returned to each student within 72 hours via e-mail. Discussion: The online peer evaluation was well accepted by the students, based upon written comments on the evaluation. There was a strong correlation between the students' written comments and numerical ratings of their fellow group members. The evaluations enabled faculty to identify students who had made outstanding or marginal contributions to small-group activities; they enabled students to share their assessments with their peers anonymously. Because of this success, the process will be continued in future classes and will probably be adopted by other courses. A significant modification planned for upcoming classes is that aggregated comments and rankings for each student will be shared with the student's faculty mentor.

Guidelines for Authors

Guidelines for Authors

A letter from the executive editor

AbstractColleagues,One of the attractions of the position of Executive Editor is the opportunity to consider creative and experimental approaches to publishing findings from research on the frontiers of insect science. In future issues of Archives, we will be more active in publishing special issues. On one hand, we will continue working to publish special issues emerging from symposia presentations at scientific meetings. With cooperation from reviewers and from the publisher, we can arrange expedited reviews and rapid publication of issues that will appear in a timely manner relative to symposia presentations. On the other hand, we are also experimenting with Special Topics Issues, to appear no more than once each year. For these issues, papers from previous volumes of Archives will be published as a set of articles devoted to a single topic. The topics will emphasize research opportunities that are emerging or perhaps re‐emerging on the horizons of our field.For this first Special Topics Issue, we have collected several papers devoted to insect cell culture. Studies of insect cell culture enjoy a lengthy history, dating back to the 1930’s when the practical goal of culturing insect cells related to research on insect‐borne viruses. In the years since this early work, we have seen marvelous progress in cell culture techniques and in the use of cell cultures to forge important advances in basic and applied research. The papers in this Special Topics Issue highlight some of the contemporary research with insect cell cultures, including analysis of an ion transport peptide, studies of glandular secretion, interactions of hormones and cells, the influence of an insect growth regulator on insect cells and activation of cellular immunity. We see, also, that basic work on cellular nutrition continues to yield important insights. It is hoped that Special Topics Issues will help facilitate further research. We welcome your suggestions for future Special Topics Issues.In another innovative experiment in publishing, we have an opportunity to help bolster the careers of our colleagues in postdoctoral positions. As we all know, the postdoctoral phase of careers in insect science is growing in length and in the numbers of postdoctoral scientists. We invite nominations from mentors of active postdoctoral scientists (who have not yet accepted a permanent professional position), who may benefit from the experience of developing a sole‐authored mini‐review centered around their current research investigations. We are now developing guidelines for mini‐reviews, which will undergo the standard scrutiny of anonymous peer review. As always, your comments, and certainly your manuscripts are welcome. We look forward to working with you on these new initiatives.

Comment When did peer review become anonymous?

Anonymous peer review is an important part of the process by which academic information is evaluated. Some of the problems associated with the process are discussed below. Although the history of peer review is well documented, there appears to be no record of when it became anonymous. Since most criticisms associated with the practice result from the anonymity of referees, this is significant. A number of well‐known and long‐established journals were approached in an attempt to determine when they adopted the process but none of the correspondents at the journals knew.

Henry Knowles Beecher and the development of informed consent in anesthesia research.

Henry Knowles Beecher and the development of informed consent in anesthesia research.

Lifting the veil on the editorial process

Recently a large group of medical editors from around the world met in Prague to discuss the current state of peer review and the future of the editorial process in biomedical journals. A common theme running through the International Congress on Biomedical Peer Review and Global Communications was that the traditional secrecy and lack of transparency of the editorial process had to cease. Powerful arguments were made to open up the process of peer review to easier scrutiny, to make editors more accountable for their decisions and to ensure the system of reviewing and accepting papers for publication is fair, equitable and reproducible. The body of scientific knowledge being built on research into peer review is in its infancy. Few of the processes involved in peer review have been studied and fewer still validated. The development of instruments to measure the quality of papers, the quality of peer review, and the reliability of reviewer assessments is as yet poor. The evidence on whether reviewers give better reports when blinded to the identity of authors is now conflicting, despite initial work from Journal of General Internal Medicine1 suggesting this was so. Indeed a variety of journals showed that attempted blinding of reviewers to the identity of authors was successful as little as 50% of the time2,3,4, especially in smaller specialties. The question whether reviewers' reports would be better if authors knew their identity has not been investigated. Despite the lack of evidence that better quality papers result, there is a strong feeling amongst medical editors that biomedical journals must move towards opening up their processes and lifting the veil on the peer review process. Anonymous peer review is an example of privilege and power disconnected from reviewers' responsibilities to fellow scientists. Justice is poorly served by secrecy. I am convinced that revealing the identity of reviewers to authors is essential. Providing a report in the knowledge that authors will not know the reviewers' identity can result in negative rather than positive criticism. While most reviewers refrain from making disparaging remarks under the current system, knowing that authors will be aware of the reviewers' identity can only encourage accountability for comments. Some editors went further and suggested that perhaps the reviewers' identities should be published with the article. Indeed the Medical Journal of Australia has pioneered an even more innovative idea, opening up the peer review process to comment on the Internet, effectively increasing the pool of reviewers for a given paper. The MJA has posted some of its papers accepted for publication on the Internet, inviting comment from interested readers. A selection of these (edited) comments has been fed back to authors where the editors felt the comments were pertinent. Craig Bingham of the MJA showed that around 30% of papers were modified following additional comments received on the Net. Richard Horton, editor of the Lancet, reported that in July 1996 his journal established an ombudsman to deal with complaints related to delays in handling manuscripts, editorial discourtesy, and failure to take reasonable account of authors' concerns5. While Emergency Medicine does not yet have the resources for similar initiatives, there is little doubt that we should be moving to more transparency and accountability in our peer review processes. To this end, the Editorial Board recently approved a proposal to reveal the identities of reviewers of a paper to the authors. I believe we should not do this immediately though; rather we should now begin to survey authors on their satisfaction with the reviewing process, and then in 12 months time begin advising authors of the identity of the reviewers. We will then be able to compare authors' satisfaction with the peer review process before and after the change. We will also be measuring the quality of the reviews produced to assess whether this intervention improves quality, but given the lack of reliable measures of quality this is likely to be unproductive. This study should at least show us whether contributing authors feel they are getting a better deal from a more open review system than the present one. Although readers buy journals and hence determine its viability, without authors there would be no journals. We must encourage researchers in our field to submit their work to our journal, in the knowledge that they will be dealt with fairly and openly.

Soapbox Discourse: Tribal Historiography, Indian-White Relations, and Southeastern New England Powwows

ANN MCMULLEN is curator of North American ethnology at the Milwaukee Public Museum and holds a Ph.D. in anthropology from Brown University. She has written widely on Northeastern woodsplint basketry and began dissertation research on Native American conceptions of history in southeastern New England as preparation for the exhibition Entering the Circle: Native Traditions in Southeastern New England at the Haffenreffer Museum of Anthropology, Bristol, Rhode Island (1992-93). The data and conclusions presented here are dealt with in much greater depth in Ann McMullen, Culture by Design: Native Identity, Historiography, and the Reclamation of Tradition in Twentieth-Century Southeastern New England (Ph.D. diss., Brown University, 1996), based on ethnographic and ethnohistorical fieldwork done between June 1990 and May 1994. Research was funded by the Haffenreffer Museum of Anthropology, Brown University; the Wenner-Gren Foundation for Anthropological Research; and the Center for the Study of Race and Ethnicity in America, Brown University. I gratefully acknowledge this assistance here. I am indebted to my dissertation committee-Shepard Krech III, George L. Hicks, William O. Beeman, Patricia E. Rubertone, and William S. Simmons-for discussion on this paper, and to Dan Odess for his critical reading of earlier versions, subsequent discussions, and support. In addition, I thank Barry O'Connell and two anonymous peer reviewers for their critical comments on an earlier draft, and Ann Marie Plane and Clara Sue Kidwell for their encouragement. Within the text and notes, I have used italicized pseudonyms to disguise informants' identity, but individuals deceased during the period of my research retain their true names.

How to compare and report department specific mortality rates for peer review using the Perinatal Database of The Netherlands

Since characteristics of patient populations of obstetric departments vary substantially with respect to the pathology presented, (perinatal) mortality rates as such can not be used for a fair peer review without adjustment for those differences. Using the Perinatal Database of The Netherlands (LVR), data on approximately 80 000 newborns annually from 1985 to 1991 inclusive were used in statistical models to predict the perinatal mortality risks of four subpopulations of different gestational age in about 125 obstetric departments. As predictors for perinatal mortality we used only those risk factors which were judged to reflect the ‘pathology’ of the patient; risk factors associated with or resulting from hospital care and/or policy are (by definition) excluded. The statistical technique of logistic regression was used. The expected and observed mortality within each of these four subpopulations (for 1985–1991) was sent anonymously to each department as a specially designed graphical overview. This method of (anonymous) peer review was met favourably. In peer review the difference between observed and expected mortality should be used, rather than mortality per se, to adjust for differences in basic risk in the intake populations on which the departmental policy has no influence.

"This Is Hard, Isn't It?": Children Evaluating Writing

This paper describes the evaluation criteria that nonnative-English-speaking children (Grades 2–6) employed when evaluating writing. Specifically, the paper discusses: (a) the range of evaluation criteria that children used, (b) whether authorship influenced evaluation criteria (not all the stories were written by the children), and (c) whether the evaluation criteria used by the children varied according to age. The study is grounded in 14 in-depth interviews of 9 students, in which they rated pieces of writing and explained why they had given each story its particular rating. An analysis of the data reveals that the students (a) were critical evaluators, (b) tended to focus on meaning regardless of their age and whether the piece of writing had been written by themselves or an anonymous peer, (c) were highly idiosyncratic in the range of evaluation criteria that they employed, and (d) were influenced by the pedagogical focus in their ESOL classes.

Anonymous peer refereeing

Anonymous peer refereeing

Anonymous peer refereeing

Anonymous peer refereeing

TM or Not TM?

For a (now formerly) anonymous reviewer of the previous article, International Peace Project in the Middle East: The Effects of the Maharishi Technology of the Unified Field, the manuscript presents somewhat of a dilemma. On the one hand, the claimed theoretical basis for the activity described, and the results purported in the analysis, are anathema to one who considers himself a strong advocate of the application of scientific method in the social sciences. On the other hand, both the level of exposition and the application of statistical methods for hypothesis testing are commensurate with this reviewer's standards for scientific research. The disturbing aspect of this piece is not that it is in print, but that it got there via the very mechanism that, one would suppose, should have screened it out-an anonymous peer review and editorial process that evaluates manuscripts based upon unarticulated but widely accepted professional standards for scientific research. There are, I feel, a number of lessons in this. Some of these lessons are but reminders of the basic elements of philosophy of science one encounters in graduate school but pays little heed to in the everyday practice of research. Others are more practical matters suggesting caution concerning our willingness to accept results reported in more conventional literature. A brief look at the implications of this article for research will, in the eyes of this reviewer, be instructive. What I find of value in this piece has little, if anything, to do with the substance of the article. For this reviewer, the primary lesson learned is that if a piece this unconventional can be supported with measurement and method, then there is perhaps a notable volume of research of a

Peering into the bowels of the MRC. II: Review systems.

Many of the medical researchers whom I met think of the Medical Research Council as a vast sausage machine that slowly and laboriously but usually fairly processes grant applications. Here I take a critical look at the council's mechanisms for deciding which research will be supported. They hinge around peer review, a time honoured system that is increasingly being questioned.14 Most flamboyant of the critics is the American materials scientist, Rustum Roy, who argues that peer review is extremely wasteful of a resource in limited supply, the time of talented scientists and engineers... is inherently anti-innovatory... [and] presupposes an unrealistic view of human nature. 4 Although many other bodies in Britain fund medical research, the MRC's mechanisms are important as they are a model for other organisations, and many of the points that can be made about the MRC's mechanisms apply elsewhere. Key features of the MRC's mechanisms are that they use anonymous peer review and do not (formally) give reasons for turning down grant applications. Nor does the council have an appeal mechanism. This was roughly the state of affairs at the National Institutes of Health in the United States until about 10 years ago, when after a heated debate the institutes decided to give reasons for turning down an application and to develop appeal mechanisms.

An anonymous peer counseling system for use with pregnant students.

An anonymous peer counseling system for use with pregnant students.