'Un'Fair Machine Learning Algorithms

Abstract

Machine Learning algorithms are becoming widely deployed in real world decision-making. Ensuring fairness in algorithmic decision-making is a crucial policy issue. Current legislation ensures fairness by barring algorithm designers from using demographic information in their decision-making. As a result, the algorithms need to ensure equal treatment to be legally compliant. However, in many cases, ensuring equal treatment leads to disparate impact particularly when there are differences among groups based on demographic classes. In response, several ``fair” machine learning algorithms that require impact parity (e.g., equal opportunity) have recently been proposed to adjust for the societal inequalities; advocates propose changing the law to allow the use of protected class-specific decision rules. We show that these ``fair'' algorithms that require impact parity, while conceptually appealing, can make everyone worse off, including the very class they aim to protect. Compared to the current law, which requires treatment parity, these ``fair'' algorithms, which require impact parity, limit the benefits of a more accurate algorithm for a firm. As a result, profit maximizing firms could under-invest in learning, i.e., improving the accuracy of their machine learning algorithms. We show that the investment in learning decreases when misclassification is costly, which is exactly the case when greater accuracy is otherwise desired. Our paper highlights the importance of considering strategic behavior of stake holders when developing and evaluating ``fair'' machine learning algorithms. Overall, our results indicate that ``fair'' algorithms that require impact parity, if turned into law, may not be able to deliver some of the anticipated benefits.