Too Many Fairness Metrics: Is There a Solution?

Abstract

With the growing literature on new definitions and metrics to measure fairness, the central question decision-makers face in many applications is which metric to select. When there is no consensus on a universal definition of fairness, we propose finding solutions that are approximately-optimal using tools from multi-objective optimization. To demonstrate our framework, we consider the placement of emergency rooms in two neighboring California counties. We consider with respect to different groups of people (e.g. grouped by race, income etc.). The distance traveled an emergency room for groups of people is the basis of the fairness metrics, and there are at least 24 group metrics of fairness in the literature. In this, one must ensure equity across multiple groupings of the population. For example, it is important to simultaneously consider equity across both racial groups as well as across economic groups. In this paper we discuss: (i) a problem formulation with composite objectives that capture interplay of equity and efficiency, (ii) problem instances when two or more equity metrics can be simultaneously minimized, (iii) solutions that are approximately-optimal for multiple definitions of fairness simultaneously. We find that in practice, rather than deciding on a single equity metric, one can compute approximately-optimal solutions that may not be perfect with respect to any single metric, yet can be near-perfect with respect to multiple metrics. We hope that these tools can be useful for policymakers to find operational solutions amidst the myriad of fairness definitions.