Six-fold over-representation of graduates from prestigious universities does not necessitate unmeritocratic selection in the faculty hiring process.

Michael Miuccio,Ka-Yuet Liu,Hakwan Lau,Megan A K Peters

doi:10.1371/journal.pone.0185900

Abstract

To achieve faculty status, graduating doctoral students have to substantially outperform their peers, given the competitive nature of the academic job market. In an ideal, meritocratic world, factors such as prestige of degree-granting university ought not to overly influence hiring decisions. However, it has recently been reported that top-ranked universities produced about 2–6 times more faculty than did universities that were ranked lower [1], which the authors claim suggests the use of un-meritocratic factors in the hiring process: how could students from top-ranked universities be six times more productive than their peers from lower-ranked universities? Here we present a signal detection model, supported by computer simulation and simple proof-of-concept example data from psychology departments in the U.S., to demonstrate that substantially higher rates of faculty production need not require substantially (and unrealistically) higher levels of student productivity. Instead, a high hiring threshold due to keen competition is sufficient to cause small differences in average student productivity between universities to result in manifold differences in placement rates. Under this framework, the previously reported results are compatible with a purely meritocratic system. Whereas these results do not necessarily mean that the actual faculty hiring market is purely meritocratic, they highlight the difficulty in empirically demonstrating that it is not so.

Highlights

We evaluated the similarity between the remaining Higher tier and Lower tier groups using Receiver Operating Characteristic (ROC) analysis, which plots the hit rate versus false alarm rate at varying criterion values [9,10]: for all possible criterion values, Impact Factor Sum (IFS) values are defined as hits if they are correctly classified as belonging to the Higher tier group, and false alarms if they are classified as belonging to the Higher tier group but came from the Lower tier group
The theoretical model does not depend on particularities of distribution shape, distribution parameters, or sample size. (The exception to this is when the probability distributions in question are a priori highly unlikely in the real world, such as uniform distributions which would lead the difference in p(hire|tier) to be linearly related to the difference in means between the two distributions, or some cases where two distributions have qualitatively different shapes.) put, as long as the faculty job market is competitive—i.e., a high meritocratic criterion exists because the probability of being hired is low—small differences in productivity as a function of university rank can be magnified into manifold differences in faculty production rates
The signal detection theoretic model—demonstrating that a small difference in meritocratic measures across university rank can lead to a manifold difference in hiring rates under an extreme criterion—holds in almost any distribution, and so the sample we gathered serves primarily to provide a concrete example of our theoretical argument

Summary

Introduction

Is academia a pure meritocracy? If it is not, what makes it deviate from the ideal? Doctoral students seem to have to substantially outperform their peers in the competitive academic job market to get a faculty position, and the prestige of the degree-granting institution appears. Just as in the main analysis, despite a higher IFS Cutoff (criterion) at 57.05, resulting from the shift of probability density towards higher IFS values, the observed similarity between Higher and Lower tier distributions is maintained (albeit a little lower, AUC = 0.633), and resultant ratios of Higher to Lower tier faculty hiring rates are starkly asymmetric: mean p(hire| Higher tier) = .108, mean p(hire|Lower tier) = .025 This result demonstrates that while our sample may be small and IFS Score may not capture all possible meritocratic elements, the proof of concept of our signal detection theoretic argument does not depend on any specific sample. These results demonstrate that, except in scenarios that are a priori highly unlikely in the real world (e.g., when the probability distributions in question are uniform, or when distributions significantly deviate from each other in shape), the signal detection theoretic argument here will hold regardless of the exact shape or family of probability distributions used

Findings

Discussion

Conclusions