When the discussion about algorithmic fairness first started to receive academic attention, much of the debate was about criteria that use aggregate statistics of observed data to determine whether a predictive model is fair. At the center of the debate were the two criteria equalized odds and predictive parity. Equalized odds requires that false positive and false negative error rates be equal for different protected groups (e.g., men and women) in order for a predictive model to be considered fair. Predictive parity, on the other hand, stipulates that a model must have equal predictive values for different protected groups. At first glance, both criteria seem like reasonable and intuitive conditions for algorithmic fairness. As it turned out, however, the two criteria are (under realistic circumstances) mutually incompatible.1 This means, it is in most cases impossible to satisfy both—when one is satisfied, the other must be violated. This was a frustrating result since both conditions have some intuitive appeal. At the same time, these impossibility results inadvertently provided a justification for companies, governments, and other organizations to use predictive models which violate one of the fairness criteria: they could simply argue that the model cannot but violate the criterion since it satisfies the other. To illustrate the impossibility of satisfying both equalized odds and predictive parity in a practical setting, consider a predictive model that estimates a person's risk of defaulting on a loan. Let us say this model is used by a bank to renegotiate the loan terms. To retrospectively compare the model's performance for two different demographic groups, we could examine how many people in each group who defaulted on their loan were predicted to do so. Equalized odds requires that this proportion be the same for both groups. Alternatively, we could check how many people in each group who were predicted to default actually did default. Predictive parity requires that this proportion be the same for both groups. However, if one group has a lower default rate than the other, it is impossible to satisfy both equalized odds and predictive parity simultaneously. If equalized odds is satisfied, the proportion of those who do in fact default among those predicted to default will differ between the two groups, violating predictive parity. Conversely, if predictive parity is satisfied, equalized odds will be violated. The bank could exploit this fact to justify the use of a discriminatory predictive model. Various approaches have been proposed to address the issue of the impossibility theorem. Some authors have suggested abandoning one of the fairness criteria,2 while others have proposed that the choice of criterion should depend on the context.3 Still, others have argued for the abandonment of both equalized odds and predictive parity in favor of an alternative criterion.4 However, given the intuitive appeal of these two criteria, it is hard to accept any of these options. As a result, there remains a lack of consensus on how best to deal with the impossibility theorem. In this article, I will argue that both criteria can be modified in a way that retains their intuitive appeal and renders them universally compatible. Instead of requiring that error rates or predictive values be equal across protected groups, the modified equalized odds and predictive parity criteria require that the protected characteristic does not cause discrepancies in these metrics across groups. To formalize these modified versions of equalized odds and predictive parity, I will use a method called matching, which is typically used for causal inference in observational studies. The remainder of this article is organized as follows. In Section II, I introduce the two statistical fairness criteria, equalized odds and predictive parity, and present the Kleinberg-Chouldechova impossibility theorem that involves them. I then present criticisms of the two criteria. In Section III, I turn away from fairness for a moment to introduce the method of matching. In Section IV, I utilize this method to define versions of equalized odds and predictive parity that better capture their intended interpretation. As will be shown, the two modified criteria are universally compatible. Section V concludes the article. Let us begin by providing more precise definitions of the two statistical fairness criteria equalized odds and predictive parity.5 We will here focus on binary predictions (or classifications) and take the predictive model to be a function from a set of input variables to the prediction. We will denote the variable representing the protected characteristic with A (which will also be assumed to be binary), the prediction with Y ̂ $$ \hat{Y} $$ , and the target variable that is to be predicted with Y. Let P be a joint probability distribution function of A, Y ̂ $$ \hat{Y} $$ , and Y, such that (i) P(E) ≥ 0 for all events E in the event space defined on the sample space; (ii) P(Ω) = 1, where Ω is the sample space, which is the set of all possible outcomes; and (ii) P(E1∪E2) = P(E1) + P(E2) for all events E1 and E2 that are mutually exclusive in the event space, meaning they cannot both occur simultaneously. The event space is a collection of subsets of the sample space that represent different possible events. We will also denote conditional probability (i.e., the probability of an event occurring given another event) by P E 1 E 2 $$ P\left({E}_1|{E}_2\right) $$ , which is defined as P E 1 | E 2 = P E 1 ∩ E 2 P E 2 $$ P\left({E}_1|{E}_2\right)=\frac{P\left({E}_1\cap {E}_2\right)}{P\left({E}_2\right)} $$ . The probability of a variable V taking on the value v is denoted as P(V = v), abbreviated as P(v) when there is no ambiguity. The probability of two events occurring simultaneously, V = v and U = u, will be abbreviated as P(v, u). We can now formally define the two criteria as follows: Definition 1.(Equalized odds). A predictive model satisfies equalized odds (relative to protected characteristics a1, a2∈DA) if and only if for all y ̂ ∈ D Y ̂ $$ \hat{y}\in {D}_{\hat{Y}} $$ and y∈DY, P y ̂ a 1 y = P y ̂ a 2 y $$ P\left(\hat{y}|{a}_1,y\right)=P\left(\hat{y}|{a}_2,y\right) $$ . Definition 2.(Predictive parity). A predictive model satisfies predictive parity (relative to protected characteristics a1, a2∈DA) if and only if for all y ̂ ∈ D Y ̂ $$ \hat{y}\in {D}_{\hat{Y}} $$ and y∈DY, P y a 1 y ̂ = P y a 2 y ̂ $$ P\left(y|{a}_1,\hat{y}\right)=P\left(y|{a}_2,\hat{y}\right) $$ . In most contexts, equalized odds and predictive parity cannot be satisfied simultaneously. This follows from a theorem of which two versions were simultaneously and independently proved by Chouldechova6 and Kleinberg.7 More precisely, the theorem states that whenever the prevalence, that is, the relative frequency of an occurrence of the event represented by the target variable, is different for different protected groups, a predictive model which satisfies equalized odds must violate predictive parity, and vice versa.8 For example, a predictive model intended to predict whether a defendant will reoffend (i.e., commit a future crime) cannot at the same time produce equal error rates and have equal predictive values for different ethnic groups, if the prevalence of reoffence (i.e., the relative frequency of defendants committing another crime in the future) differs across these groups. To state this more concisely, the theorem can be formulated as follows: Theorem 1.(The Kleinberg-Chouldechova impossibility). If the prevalence differs across protected groups, no (imperfect) predictive model can satisfy both equalized odds and predictive parity. If both equalized odds and predictive parity were deemed universally necessary conditions for predictive fairness, this impossibility theorem would suggest that truly fair predictive models are an unattainable goal. There is, however, another possible interpretation of this impossibility: namely that it highlights a flaw in our formalization of what constitutes fair predictive models. The impossibility result can then be seen as an indicator that we have to rethink the definitions of the two fairness criteria and reevaluate whether they actually formalize the more intuitive ideas they are intended to formalize. The argument pursued here is along those lines. I will argue that while the intuitive appeal of both fairness criteria is undeniable, they impose stronger requirements than what is necessary in order to avoid certain types of unfairness in predictive models. Before we examine both criteria, let us first consider how we can cash out the broader idea of fairness for predictive models. Algorithmic fairness is, in the context of such models, generally taken to mean the absence of discrimination. Discrimination, in turn, can be understood as the wrongful disadvantageous treatment of an individual on the basis of a sensitive characteristic like ethnicity, gender, or religion.9 There is disagreement on what constitutes wrongful disadvantageous treatment in this context. However, the most widely accepted definitions are based on the ideas that the sensitive characteristic is irrelevant in most situations10 and that a person should be treated as an individual.11 Wrongfulness then arises from basing disadvantageous treatment on an irrelevant sensitive characteristic, or from treating someone disadvantageously based on presumed statistical patterns that associate their sensitive characteristic with some other trait in a way that disregards their individuality. Predictive models, by definition, are mathematical models used to predict the value of a specific variable. The output of such a model can be seen as representing a belief-like propositional attitude. If the possession of a certain sensitive characteristic leads to disadvantageous predictions, such as in terms of accuracy or their impact on the decision for which the model is used, this process can be considered discriminatory. We can consequently understand equalized odds and predictive parity as criteria that prevent discriminatory outcomes in predictive models. Equalized odds can be interpreted as a criterion that prevents discriminatory outcomes by preventing systematic cognitive bias with regard to a sensitive characteristic. By systematic cognitive bias, we here mean misjudging how informative a certain trait is in predicting another trait.12 Assume, for example, that in making predictions about whether someone will get lung cancer, we overestimate how informative it is that the person smokes. More precisely, if someone is a smoker, we predict that they will get lung cancer, and if not, we predict that they will not get lung cancer. We are clearly biased with regard to smoking in predicting lung cancer: not everyone who smokes gets lung cancer, and some people get it without ever having touched a cigarette. It is easy to see that this will result in different error rates for the group of smokers and the group of non-smokers. The smokers will have a false negative rate of 0 (simply for the fact that no smoker was predicted to not get lung cancer) but a false positive error rate above 0 (some smokers do not get lung cancer). Vice versa, the non-smokers will have a false negative rate above 0 (there are some who get lung cancer, but we never predict a non-smoker to get lung cancer) but a false positive rate of 0 (because no non-smoker is predicted to get lung cancer). One could say that this model is systematically biased with regard to smoking in predicting lung cancer. If, however, instead of using the predictive model just described we used a predictive model which guarantees that the error rates across smokers and non-smokers are equal, then we could be sure that the predictor contains no such bias. While being a smoker is typically not considered a sensitive characteristic, overestimating the informativeness of a sensitive characteristic like gender could lead to disadvantageous predictions on the basis of an irrelevant (or less relevant than warranted) sensitive characteristic.13 Yet, it is important to note that a violation of equalized odds across protected groups is an indicator and not a definition of systematic cognitive bias. To see this, note that the statement relating error rates to bias is a conditional: if there is bias with regard to trait A, there will be disparities in error rates between those with trait A and those without. By simple logic, this implies that whenever there are no disparities in error rates, there is no bias. Yet, it does not imply that whenever we observe disparities in error rates between those with trait A and those without, we can conclude that the predictor is biased with regard to A. In other words, equalized odds relative to A is a sufficient condition for the absence of bias with regard to A, but not a necessary one. Hence, trying to deduce that a predictor is biased from the observation that error rates among groups differ amounts to committing the well-known fallacy of affirming the consequent. At best, observing disparities in error rates allows one to make an inference to the best explanation: when disparities in error rates between two groups are observed, and there is no other plausible explanation, then one is justified in suspecting that this is due to bias with regard to the trait that distinguishes the groups. While this inference may seem plausible in many cases, it is important to note that it is a fallible inference, and that is what matters for our purposes (Figure 1). To illustrate this with an example, imagine a health insurance company that tries to predict the healthcare costs an individual incurs in a given year in order to decide how to set their customers' premiums. To simplify things, imagine the company is trying to predict only whether an individual's annual costs are above a certain threshold. This allows us to represent the target variable and the prediction as binary variables. Now imagine that in country C, citizens of religion R1 are, on average, younger than citizens of religion R2 (we can imagine that this is due to the fact that many people of religion R1 in C have recently immigrated, and that people generally tend to immigrate when they are somewhat younger). Suppose that, upon examination, the predictions turn out to have a higher false positive rate for people of religion R1 than for people of religion R2. Can we conclude that the predictive model the insurance company used has a discriminatory bias against people of religion R1? The observation of different error rates does not conclusively establish this. Different explanations for this discrepancy are conceivable. Imagine first a scenario in which the insurance company uses a predictive model which solely takes the individual's age into account. Now imagine further that the predictor is biased with regard to age, in that it overestimates how informative young age is of risky behavior, and hence of increased health costs. This, as we have shown above, will obviously lead to higher false positive rates for predictions of high health costs among young people. Because, on average, people of religion R1 are younger, and the predictive model is biased with regard to age, it will produce predictions with a higher false positive rate for people of religion R1. However, it can be argued that the outcomes of this predictive model do not discriminate against people of religion R1. To see this, consider the following. Imagine that instead of C, the health insurance company operated in a different country D, where citizens of religion R2 are, on average, younger than citizens of religion R1. Again, we can assume that this is because in this context people of religion R2 are mostly recent immigrants to D. It is easy to see that here, the predictive model would produce predictions with a higher false positive rate for people of religion R2. Remember that this model is exactly the same as the one above and that the higher false positive rate for R2 would not occur in C, where the reverse was the case. If we define bias as disparities in observed error rates, we would come to the somewhat contradictory conclusion that the predictive model is biased against people of religion R1, but that, had the insurance company applied the exact same predictive model in a different country, the model would be biased against people of religion R2. We can see that which religion a person has does not, in any sense, influence the predictions (or, for that matter, the error rates). It only happens to be the case that, in the given context, the predictive model works on average less well for one religious group than for another. While this might be worrisome in its own right, it can hardly be considered discrimination on the basis of religion, as there is no explanatory relation between a customer's religion and the predicted health costs. Compare this with a second scenario, depicted in 1(b), in which the insurance's predictor takes a person's religion into account in order to make a healthcare cost prediction. From an observational point of view, the two predictors' performances might be indistinguishable, as they could both produce the same discrepancies in error rates between different religious groups. Yet, on a narrow understanding of systematic cognitive bias, only the latter can be said to be biased against people of religion R1. This is of course not to say that disparities in error rates among different protected groups are of no moral concern by themselves. Disparities in error rates can, if tied to decision-making, lead to unjust distributions of resources and burdens. Yet, we are here following Eidelson14 in claiming that (direct and structural) discrimination is conceptually distinct from matters of distributive justice, and trying to subsume one under the other will get in the way of a clear analysis of each. Especially in the context of algorithmic decision-making, it seems that distinguishing between discriminatory predictions and unjust decisions is necessary in order to determine an appropriate method of mitigating the unfairness without introducing other unanticipated biases into the decision-making process.15 We are here solely concerned with fairness criteria for predictions. A discussion of distributive justice arising from algorithmic decision-making, although important, is beyond the scope of this article. Let us now turn to predictive parity. Predictive parity is often interpreted as a criterion that prohibits the meaning of a prediction from depending on a person's sensitive characteristic, as doing so could incentivize discriminatory behavior.16 Here, a similar observation can be made. Imagine a medical device that tests for a specific disease. Given a person has the disease, there is a 95 percent probability that the test turns out positive. When applied to a person who is healthy, there is a 5 percent probability that the test nonetheless turns out positive. This, we can imagine, can be shown to robustly hold across genders. There is no difference whatsoever in the likelihood of receiving an erroneous result, no matter whether a patient is male or female. Intuitively, it seems, there is no difference in meaning of the prediction for men and for women. But now imagine that the disease happens to occur more frequently in men. More specifically, we can imagine that one in every 10 men has the disease, but only one in every 100 women does. Then the positive predictive value, that is, the probability of actually having the disease given that one receives a positive test result, is different for men and women. For men it is roughly 68 percent, whereas for women it is only about 16 percent.17 This means, in this intuitively fair case, predictive parity is not satisfied. But it seems that this is not due to bias in the testing device, but just to the prevalence of the disease, which differs across genders. In other words, it is not gender which causes the difference in predictive value (since the testing device works, by assumption, equally well for a randomly chosen man as for a randomly chosen woman). So it seems that here, too, we want to distinguish between discrepancies in predictive value which are (causally) explained by gender, and discrepancies in predictive value which are due to external factors, such as differences in the prevalence of a disease. In light of these criticisms, it seems that the definitions of both equalized odds and predictive parity do not adequately explicate the underlying moral intuitions they were designed to capture.18 This, in turn, could mean that the Kleinberg-Chouldechova impossibility result is not so disastrous after all. If neither equalized odds nor predictive parity are, as they are currently defined, necessary conditions for fairness, the impossibility loses its bite. There is a chance that the impossibility theorem is just an artifact of the way the criteria are defined. The remainder of this article will examine this possibility by trying to provide modified definitions of equalized odds and predictive parity that retain all the intuitively plausible aspects of the current definitions but avoid the impossibility. We can use matching—a method for causal inference on the basis of observational data19—to define modified versions of equalized odds and predictive parity. In this section, I will explain the method. The motivation behind matching stems from the following problem. In many scenarios, it would be useful to be able to infer whether and to which degree a given variable has a causal effect on some other variable. The “gold standard” for estimating causal effects is the so-called randomized controlled trial—a specific type of experimental study. Often, however, it is practically impossible or unethical to run experiments, or the only available data is observational data. Think, for instance, about studying the health effects of passive smoking on children. It would be unethical to actively expose a group of children to secondhand smoke. Yet, there might be observational data on the health of children that live in a home where at least one of the parents smokes. Matching aims to replicate the properties of a randomized controlled trial for observational data as best as possible, to allow for the estimation of causal effects in cases like the above, where experimental data is unavailable. Randomized controlled trials estimate causal effects by comparing the results of an intervention on one group of randomly selected participants (the treatment group) to a control group who do not receive the intervention. This ensures that any observed differences between the groups are due to the intervention and not other factors. For example, in a drug trial to test the effectiveness of an anti-rheumatic drug, participants could be randomly assigned to receive either the drug or a placebo, and the effectiveness of the drug could be determined by comparing the frequency of rheumatic symptoms in the two groups. A key advantage of randomized controlled trials over observational studies is that they can control for confounding factors that may affect both who receives the treatment and the outcome being measured. Contrast this with a trial without randomization. If the anti-rheumatic drug were administered to anyone willing to take it, and the results were compared to those who did not receive the drug, the results could be confounded by various factors. One such factor is the age of the participants. Older people are more likely to suffer from rheumatic arthritis and might hence be more willing to take anti-rheumatic drugs. At the same time, age obviously also affects whether and with which frequency someone experiences rheumatic symptoms. This means that the treatment group may have a higher average baseline frequency of rheumatic symptoms than the control group, which could lead to the incorrect conclusion that the drug is not effective. In a randomized controlled trial, however, the age distributions of the treatment and control groups would be roughly equal, allowing for a more accurate assessment of the drug's effectiveness. When we only have observational data, we can try to mimic randomization via matching. Instead of randomly assigning individuals to the treatment or control group, observational data can be used to create a synthetic control group that does not systematically differ from the treatment group on any observed or unobserved variables other than the treatment (i.e., causal) variable, effectively achieving randomization. This is done as follows. Assume that the data consists of information on the causal variable, the effect variable, and a number of other variables, which, in this context, will be called the covariates. For each individual in the treatment group,20 we pick the individual from the initial control group whose covariate values are as similar as possible to the covariate values of the individual from the treatment group. These individuals constitute the synthetic control group. We end up with a treatment and a synthetic control group that have identical or very similar distributions over the covariates, and which—given certain assumptions that we will get to in a moment—allow for the conclusion that any significant difference between the groups with regard to the effect variable is caused by the difference in the causal variable. To illustrate this, we can apply matching to the example of evaluating the effectiveness of the supposedly anti-rheumatic drugs on the presence of rheumatic symptoms. To create a synthetic control group, we would try to find individuals in the initial control group with ages that are as close as possible to the ages of the individuals in the treatment group, as well as similar levels of pretreatment rheumatic symptoms. By creating a synthetic control group with these individuals, we would ensure that the treatment and control groups have similar distributions of age and symptomaticity, allowing us to conclude that any significant differences in the presence of rheumatic symptoms between the groups is likely due to the treatment rather than age or pretreatment health. Assuming that the drug is effective, we would expect the frequency of rheumatic symptoms to be lower in the matched control group than in the treatment group. Of course, there may still be other unobserved confounding factors that could affect the results, but matching allows us to control for observed confounding factors and provide a more accurate assessment of the treatment's effectiveness. Let us now address a central methodological question, namely how to choose covariates. In order for matching to be as good a method for estimating causal effects as randomization, it is crucial that the set of covariates contains all the variables that influence both the causal and the effect variable (i.e., all confounding variables). This is important because it entails that there are no unobserved differences between the control and treatment group conditional on the observed covariates.21 The assumption that this is the case is typically called ignorability. In our example above, ignorability is most likely not satisfied: there could be other variables that influence both treatment and effect variables, like for instance weight, previous injuries, and so on. Even though we have removed some confounding by matching on age and pretreatment rheumatic symptoms, we could still get a better estimate of the drug's effect if we had moreover matched the data on these other confounding factors. Another important norm for choosing covariates is not to include any variables that are causally influenced by the causal variable. This might lead to an underestimation of the causal effect and thereby distort the analysis.22 Lastly, let us mention that there is an entire family of matching methods. The make-up of the obtained synthetic control group depends on the distance metric used, whether one uses 1:1 or 1:n matching (where the matched data point is the average of the n most similar data points), and whether matching is done with replacement or without (i.e., whether a data point can be matched more than once). The specific choice of matching method, however, is of no importance to the overall argument presented here. Let us now return to the attempt to modify equalized odds and predictive parity to avoid the Kleinberg-Chouldechova impossibility. We will consider both fairness criteria in turn. Let us begin by laying out the intended interpretation of the fairness criterion which is to replace equalized odds. We will call this criterion matched equalized odds. Matched equalized odds requires that the protected characteristic has no direct effect on the predictor in a way that affects its error rates. For instance, if a recidivism predictor satisfies matched equalized odds, it means that the fact that a defendant is African- American does not increase the probability of receiving a false positive recidivism prediction.23 How can we determine if a predictor satisfies matched equalized odds? We can approach this as a causal inference problem by identifying the causal effect of the protected characteristic on a predictor's error rates. This problem, as explained in the previous section, can be addressed using matching. We treat the protected characteristic as the treatment variable and select an appropriate set of covariates. We then create a matched control group, such that the treatment and control group (i.e., the two protected groups) exhibit no systematic differences other than in their protected characteristics and the predictions they receive. Finally, we compare the error rates of the two groups. If (and