Closing the GAP: Group-Aware Parallelization for Online Selection of Candidates with Biased Evaluations

Abstract

The $k$-secretary problem deals with online selection of at most $k$ numerically scored candidates, where selection decisions are immediate upon their arrival and irrevocable, with the goal of maximizing total score. There is, however, wide prevalence of bias in evaluations of candidates from different demographic groups (e.g., gender, age, race), and the assumption of an algorithm observing their true score is unreasonable in practice. In this work, we propose a biased variant of the secretary problem where selection decisions must be made by observing biased scores only, and the goal is to maximize true score. We consider two models of bias: one where the amount of bias experienced is demographic group-dependent, and propose one which takes into account implicit bias and limitations of testing at the individual level. We posit that a desirable property for selection of candidates is ranked demographic parity (RDP), that is, parity of selection rates across within-group ranks and monotonicity in selection rates with rank. We show that bias-agnostic methods can be suboptimal in terms of performance and RDP, and propose group-aware parallelization (GAP), as an effective counter measure. We tweak GAP for both models of bias and for stochastic and adversarial group membership and score assignments, and develop algorithms that are order-optimal in terms of their competitive ratios. We give the first order-optimal algorithm for the $k$-secretary problem under partial ordinal information, which may be of independent interest. Finally, we perform a case study on real-world data to demonstrate the significant impact of our methods on selection rates across groups.