Experience with "Select Crowd Review" in Peer Review for The Thoracic and Cardiovascular Surgeon: 1-Year Experience.

Abstract

To evaluate the experience with a new peer review method, "Select Crowd Review" (SCR): anonymized PDFs of manuscripts are accessible to a reviewer crowd via an online platform. It has access for 10 days to enter anonymized comments directly into the manuscript. A SCR-Editor summarizes the annotations, giving a recommendation. Both reviewed PDF and summary are sent back to authors. Upon submission, authors are given a choice to accept or decline SCR. All manuscript submissions since introduction in July 2021 until July 2022 were analyzed regarding acceptance and quality. Manuscripts were sent to a crowd of 45 reviewers and regular double-blinded peer review at the same time. Efficiency and performance of the crowd's reviews were compared with those of regular review. For thoracic manuscripts, a crowd was not yet available. SCR was accepted by the authors for 73/179 manuscripts (40.8%). After desk rejections, 51 cardiac manuscripts entered SCR. For five manuscripts, the crowd did not respond. In all remaining papers, the crowd's recommendation concurred with that of the normal reviewers. Regular peer review took up to 6 weeks. Twelve manuscripts underwent repeated SCR after revision. A median of 2 (0-9) crowd members sent in reviews. In revisions, average response was one reviewer responding. SCR encountered good acceptance by authors. As the first experience showed concordant recommendations compared with traditional review, we have extended SCR to thoracic manuscripts for more experience. SCR may become the sole review method for eligible manuscripts. Efficiency should be increased, especially for re-review of revisions.