How to detect indications of potential sources of bias in peer review: A generalized latent variable modeling approach exemplified by a gender study

Abstract

The universalism norm of the ethos of science requires that contributions to science are not excluded because of the contributors’ gender, nationality, social status, or other irrelevant criteria. Here, a generalized latent variable modeling approach is presented that grant program managers at a funding organization can use in order to obtain indications of potential sources of bias in their peer review process (such as the applicants’ gender). To implement the method, the data required are the number of approved and number of rejected applicants for grants among different groups (for example, women and men or natural and social scientists). Using the generalized latent variable modeling approach indications of potential sources of bias can be examined not only for grant peer review but also for journal peer review.