Latent Markov modeling applied to grant peer review

Abstract

In the grant peer review process we can distinguish various evaluation stages in which assessors judge applications on a rating scale. Research on the grant peer review process that considers its multi-stage character scarcely exists. In this study we analyze 1954 applications for doctoral and post-doctoral fellowships from the Boehringer Ingelheim Fonds (B.I.F.), which are evaluated in three stages (first: evaluation by an external reviewer; second: internal evaluation by a staff member; third: final decision by the B.I.F. Board of Trustees). The results of a latent Markov model (in combination with latent class analysis) show that a fellowship application has a chance of approval only if it is recommended for support already in the first evaluation stage, that is, if the external reviewer's evaluation is positive. Based on these results, a form of triage or pre-screening of applications seems desirable.