Source Of Unfairness Research Articles

Outlier Detection Bias Busted: Understanding Sources of Algorithmic Bias through Data-centric Factors

The astonishing successes of ML have raised growing concern for the fairness of modern methods when deployed in real world settings. However, studies on fairness have mostly focused on supervised ML, while unsupervised outlier detection (OD), with numerous applications in finance, security, etc., have attracted little attention. While a few studies proposed fairness-enhanced OD algorithms, they remain agnostic to the underlying driving mechanisms or sources of unfairness. Even within the supervised ML literature, there exists debate on whether unfairness stems solely from algorithmic biases (i.e. design choices) or from the biases encoded in the data on which they are trained. To close this gap, this work aims to shed light on the possible sources of unfairness in OD by auditing detection models under different data-centric factors.By injecting various known biases into the input data---as pertain to sample size disparity, under-representation, feature measurement noise, and group membership obfuscation---we find that the OD algorithms under the study all exhibit fairness pitfalls, although differing in which types of data bias they are more susceptible to. Most notable of our study is to demonstrate that OD algorithm bias is not merely a data bias problem. A key realization is that the data properties that emerge from bias injection could as well be organic---as pertain to natural group differences w.r.t. sparsity, base rate, variance, and multi-modality. Either natural or biased, such data properties can give rise to unfairness as they interact with certain algorithmic design choices. Our work provides a deeper understanding of the possible sources of OD unfairness, and serves as a framework for assessing the unfairness of future OD algorithms under specific data-centric factors. It also paves the way for future work on mitigation strategies by underscoring the susceptibility of various design choices.

Fair space for life: A dynamic care monitor working up to growth and flourishing for all.

Mental health disparities persistently cause inequity and social exclusion. Extensive research underpins the need to embrace the social determinants of health and facilitate network learning at various ecosystem levels. Despite valuable quality frameworks and ratified conventions, local practices which counter health inequity are scarce. The Dutch HOP-TR study collected health and needs of Homeless Service Users (HSU) in a rights-based, transdiagnostic, recovery framework. We assessed the survival modes and conducted a socio-ecological analysis, exploring what happened in care pathways at three ecosystem levels: individual HSU, caregiver networks, society. While documenting vital conditions for growth and citizenship, we explore major opportunities to develop 'fair space for life'. Under low distress levels, prosocial behavior is prominent (32.9%). High distress levels are found with an avoidant (42.0%) or aggressive mode (24.9%). Rising distress levels give more frictions in relations, psychiatric admissions, and police-justice contacts. The distress-induced descent in the social hierarchy causes social withdrawal, alienation, and marginalization. At society level, fair conditions for growth and citizenship are challenged by the cumulative impact of distress over the HSU' lives. This care monitor uncovers the impact of distress on caregiver interactions. The care pathways reveal that the survival strategies reflect a systematic, pervasive neglect. Unfair representations hold HSU personally responsible of their situation, disregarding the cumulative impact of environmental conditions over their lives. The diverse sources of unfairness are intrinsic to the health care system and culture. Therefore, the survival modes ask for profound culture transformations in a whole-system-whole-society approach. Given the need for action on health equity and the social determinants of health, this paper provides an example of a dynamic care monitor. The actionable data elicit dialogs and stimulate to enrich opportunities for inclusion and growth in communities and societies.

Perceived Unfairness in University Settings: Findings from a Qualitative Study among Students in Germany

In universities, perceived unfairness is associated with negative outcomes, such as dissatisfaction, lower engagement, higher drop-out and stress, and poorer mental health. It is, however, still unclear which aspects constitute perceived unfairness in students. Therefore, this qualitative study explored students’ perceptions of unfairness with twenty semi-structured interviews. The interviews were audiotaped, transcribed verbatim, and analyzed following Mayring’s qualitative content analysis. A set of main codes and subcodes was developed and refined. The identified sources of unfairness can be summarized as organizational aspects, such as the registration and administration processes, the treatment of students, including the behavioural patterns of lecturers and university staff, and the individual situation of students. This study identified different sources of perceived unfairness in the university setting, potentially related to less engagement, higher drop-out rates, and worse mental health of the students. To keep students engaged and healthy, it seems beneficial to address the sources of unfairness.

Unveiling Data Fairness Functional Requirements in Big Data Analytics Through Data Mapping and Classification Analysis

Aims: In the realm of Big Data Analytics, ensuring the fairness of data-driven decisionmaking processes is imperative. This abstract introduces the Learning Embedded Fairness Interpretation (LEFI) Model, a novel approach designed to uncover and address data fairness functional requirements with an exceptional accuracy rate of 97%. The model harnesses advanced data mapping and classification analysis techniques, employing Explainable-AI (xAI) for transparent insights into fairness within large datasets Methods: The LEFI Model excels in navigating diverse datasets by mapping data elements to discern patterns contributing to biases. Through systematic classification analysis, LEFI identifies potential sources of unfairness, achieving an accuracy rate of 97% in discerning and addressing these issues. This high accuracy empowers data analysts and stakeholders with confidence in the model's assessments, facilitating informed and reliable decision-making. Crucially, the LEFI Model's implementation in Python leverages the power of this versatile programming language. The Python implementation seamlessly integrates advanced mapping, classification analysis, and xAI to provide a robust and efficient solution for achieving data fairness in Big Data Analytics. Results: This implementation ensures accessibility and ease of adoption for organizations aiming to embed fairness into their data-driven processes. The LEFI Model, with its 97% accuracy, exemplifies a comprehensive solution for data fairness in Big Data Analytics. Moreover, by combining advanced technologies and implementing them in Python, LEFI stands as a reliable framework for organizations committed to ethical data usage. Conclusion: The model not only contributes to the ongoing dialogue on fairness but also sets a new standard for accuracy and transparency in the analytics pipeline, advocating for a more equitable future in the realm of Big Data Analytics.

Unfairness in AI Anti-Corruption Tools: Main Drivers and Consequences

This article discusses the potential sources and consequences of unfairness in artificial intelligence (AI) predictive tools used for anti-corruption efforts. Using the examples of three AI-based anti-corruption tools from Brazil—risk estimation of corrupt behaviour in public procurement, among public officials, and of female straw candidates in electoral contests—it illustrates how unfairness can emerge at the infrastructural, individual, and institutional levels. The article draws on interviews with law enforcement officials directly involved in the development of anti-corruption tools, as well as academic and grey literature, including official reports and dissertations on the tools used as examples. Potential sources of unfairness include problematic data, statistical learning issues, the personal values and beliefs of developers and users, and the governance and practices within the organisations in which these tools are created and deployed. The findings suggest that the tools analysed were trained using inputs from past anti-corruption procedures and practices and based on common sense assumptions about corruption, which are not necessarily free from unfair disproportionality and discrimination. In designing the ACTs, the developers did not reflect on the risks of unfairness, nor did they prioritise the use of specific technological solutions to identify and mitigate this type of problem. Although the tools analysed do not make automated decisions and only support human action, their algorithms are not open to external scrutiny.

Tiny, Always-on, and Fragile: Bias Propagation through Design Choices in On-device Machine Learning Workflows

Billions of distributed, heterogeneous, and resource constrained IoT devices deploy on-device machine learning (ML) for private, fast, and offline inference on personal data. On-device ML is highly context dependent and sensitive to user, usage, hardware, and environment attributes. This sensitivity and the propensity toward bias in ML makes it important to study bias in on-device settings. Our study is one of the first investigations of bias in this emerging domain and lays important foundations for building fairer on-device ML. We apply a software engineering lens, investigating the propagation of bias through design choices in on-device ML workflows. We first identify reliability bias as a source of unfairness and propose a measure to quantify it. We then conduct empirical experiments for a keyword spotting task to show how complex and interacting technical design choices amplify and propagate reliability bias . Our results validate that design choices made during model training, like the sample rate and input feature type, and choices made to optimize models, like light-weight architectures, the pruning learning rate, and pruning sparsity, can result in disparate predictive performance across male and female groups. Based on our findings, we suggest low effort strategies for engineers to mitigate bias in on-device ML.

A LEGAL FRAMEWORK FOR ARTIFICIAL INTELLIGENCE FAIRNESS REPORTING

AbstractClear understanding of artificial intelligence (AI) usage risks and how they are being addressed is needed, which requires proper and adequate corporate disclosure. We advance a legal framework for AI Fairness Reporting to which companies can and should adhere on a comply-or-explain basis. We analyse the sources of unfairness arising from different aspects of AI models and the disparities in the performance of machine learning systems. We evaluate how the machine learning literature has sought to address the problem of unfairness through the use of different fairness metrics. We then put forward a nuanced and viable framework for AI Fairness Reporting comprising: (1) disclosure of all machine learning models usage; (2) disclosure of fairness metrics used and the ensuing trade-offs; (3) disclosure of de-biasing methods used; and (d) release of datasets for public inspection or for third-party audit. We then apply this reporting framework to two case studies.

RAWLSNET: Altering Bayesian Networks to Encode Rawlsian Fair Equality of Opportunity

We present RAWLSNET, a system for altering Bayesian Network (BN) models to satisfy the Rawlsian principle of fair equality of opportunity (FEO). RAWLSNET's BN models generate aspirational data distributions: data generated to reflect an ideally fair, FEO-satisfying society. FEO states that everyone with the same talent and willingness to use it should have the same chance of achieving advantageous social positions (e.g., employment), regardless of their background circumstances (e.g., socioeconomic status). Satisfying FEO requires alterations to social structures such as school assignments. Our paper describes RAWLSNET, a method which takes as input a BN representation of an FEO application and alters the BN's parameters so as to satisfy FEO when possible, and minimize deviation from FEO otherwise. We also offer guidance for applying RAWLSNET, including on recognizing proper applications of FEO. We demonstrate the use of our system with publicly available data sets. RAWLSNET's altered BNs offer the novel capability of generating aspirational data for FEO-relevant tasks. Aspirational data are free from the biases of real-world data, and thus are useful for recognizing and detecting sources of unfairness in machine learning algorithms besides biased data.

Substantive Unfairness as Unconscionable

Contrary to the conventional view that the doctrine of unconscionability requires procedural defects and is based on bargaining inequality, this paper argues that, in principle, substantive unconscionability alone shall be sufficient to constitute unconscionability and so vitiate a contract. It is also my contention that Aristotelian idea of contract of exchange as an act of voluntary commutative justice explains the two sources of unfairness. They are not understood by contemporary legal scholarship, namely, economic unfairness and involuntariness. Economic unfairness prevents gross disparity, which alone triggers unconscionability; it has been observed that when a transaction is truly stupid or one-sided, courts will set aside the contract as unconscionable. When a contract violates the autonomy of a party by hiding offensive or oppressive terms in a boilerplate, relief is given by both the Second Restatement of Contracts and European Directive. Yet, there is no doctrinal basis in our theoretical framework that justifies either one of the two reliefs. We stand a better chance of having a coherent doctrine of unconscionability by understanding what unfairness means and seeing contract of exchange as an act of voluntary commutative justice as in Aristotelian tradition.

Evaluation of Performance Unfairness in NUMA System Architecture

NUMA (Non-uniform memory access) system architectures are commonly used in high-performance computing and datacenters. Within each architecture, a processor-interconnect is used for communication between the different sockets and examples of such interconnect include Intel QPI and AMD HyperTransport. In this work, we explore the impact of the processor-interconnect on overall performance-in particular, we explore the impact on performance fairness from the processor-interconnect arbitration. It is well known that locally-fair arbitration does not guarantee globally-fair bandwidth sharing as closer nodes receive more bandwidth in a multi-hop network. However, this paper is the first to demonstrate the opposite can occur in a commodity NUMA servers where remote nodes receive higher bandwidth (and perform better). This problem occurs because router micro-architectures for processor-interconnects commonly employ external concentration. While accessing remote memory can occur in any NUMA system, performance unfairness (or performance variation) is more critical in cloud computing and virtual machines with shared resources. We demonstrate how this unfairness creates significant performance variation when executing workload on the Xen virtualization platform. We then provide analysis using synthetic workloads to better understand the source of unfairness.

Fairness-Based Retributivism Reconsidered

In this paper, I defend fairness-based retributivism against two important objections, the no-benefit objection and the social injustice objection. I argue that the theory can defeat the no-benefit objection by developing an account of how crimes can be sources of unfairness by inflicting losses on people, and that it can blunt the social injustice objection by toning down the theory’s distributive aspirations. I conclude that fairness-based retributivism, contrary to received wisdom, merits further attention from legal and political philosophers.

Why Do We Care What Others Pay? The Effect of Other Consumers’ Prices on Inferences of Seller (Dis)Respect and Perceptions of Deservingness Violation

Comparisons to other consumers have been identified as particularly meaningful in shaping price fairness judgments, but why they should be so is not clear. The current work argues that fairness judgments reflect both the extent to which consumers receive what they believe they deserve and inferences about seller respect. We suggest that comparisons to consumers paying lower prices are a particularly potent source of unfairness because they highlight the violation of deservingness and readily communicate information about sellers’ opinions of consumers. Three experiments investigate these ideas. Implications for fairness theory and marketing are discussed.

Fair and Open Evaluation May Call for Temporarily Hidden Authorship, Caution When Counting the Votes, and Transparency of the Full Pre-publication Procedure

The problem of imbalance The traditional system of manuscript evaluation has created a certain imbalance bordering with unfairness: while the authors of submitted papers typically have had their identity disclosed already at the outset, the reviewers have remained mostly anonymous. With a new open evaluation system being currently envisaged, the main difference would be that evaluators become disclosed as well, which is a significant step toward balance and fairness. More openness and constructive interactivity in the reviewing process have become to be practiced increasingly more, including some noteworthy success (e.g., the Shepherding system at the European Conference on Pattern Languages of Programming and Computing and the Frontiers initiative). However, while both the authors and the reviewers have become disclosed, the past collective “instincts” and traditions of reviewers as evaluators could often remain unaffected by this change. Thus, if the new envisaged evaluation system fully discloses both the authors and evaluators, it nevertheless cannot tackle all potential sources of bias and unfairness. Although for the majority of researchers this system seems to be suitable, there are also researchers who feel that not all possible sources of unfairness would be eliminated. Therefore there should be also an additional optional format of review and publishing that goes even further in pursuit for minimizing the impact of subjectivity. Why so?

An Examination of the Effects of Situational Constraints on Appraised Performance

Most workers are subject to factors in the work place that are beyond their power to control and that act to restrict their performances. Despite the uncontrollable nature of these ‘“situational constraints,” there is evidence that appraisals of work performance tend lo unfairly blame the worker for the shortfalls in performance that these constraints cause. The magnitude of this tendency may be unclear due to the ways in which constraints have been assessed in previous research. A lab study was conducted to obtain a better estimate of the magnitude of situational constraint “bias” using a relativistic rating method that factors out the influence of constraints from performance appraisals. The results indicated that this bias is considerably stronger than previous reports had indicated and may be the dominant source of unfairness in performance appraisals.

Customer (In)Justice and Emotional Labor: The Role of Perspective Taking, Anger, and Emotional Regulation†

This study investigates the impact of customer interpersonal and informational injustice on service workers' emotional labor (surface acting). Results from a study conducted on 152 bank tellers in Germany showed that customers are evaluated by employees as a potential source of unfairness. Perceptions of customer justice were found to interact with individual differences in perspective taking in the prediction of surface acting such that the negative effect of customer injustice on surface acting was stronger for those low in perspective taking (compared to those high in perspective taking). Although anger was expected to mediate this moderated effect, this hypothesis was not confirmed. Considering the results post hoc, a revised theoretical model is proposed based on Cropanzano, Weiss, Suckow, and Grandey's model of justice and emotional regulation. Future research is needed to test this model and determine what leads employees to deploy emotional regulation strategies when faced with unfair customers.

The origins of network server latency & the myth of connection scheduling

We investigate the origins of server-induced latency to understand how to improve latency optimization techniques. Using the Flash Web server [4], we analyze latency behavior under various loads. Despite latency profiles that suggest standard queuing delays, we find that most latency actually originates from negative interactions between the application and the locking and blocking mechanisms in the kernel. Modifying the server and kernel to avoid these problems yields both qualitative and quantitative changes in the latency profiles -- latency drops by more than an order of magnitude, and the effective service discipline also improves.We find our modifications also mitigate service burstiness in the application, reducing the event queue lengths dramatically and eliminating any benefit from application-level connection scheduling. We identify one remaining source of unfairness, related to competition in the networking stack. We show that adjusting the TCP congestion window size addresses this problem, reducing latency by an additional factor of three.

Testing methods, testing consequences: are they ethical? are they fair?

Language tests employing methods which are not fair to all test takers are unethi cal. Equally, uses of language tests which aim to exercise control and manipulate stakeholders rather than provide information regarding proficiency levels are also unethical. Ways of reducing these sources of unfairness are discussed and language testers are urged to exercise vigilance at all times to ensure that the tests they develop are democratic and fair.

Remnants of Parens Patriae in the Adjudicatory Hearing: Is a Fair Trial Possible in Juvenile Court?

Traditionally, adjudicatory hearings in juvenile court operated under the direction of the parens patriae doctrine, the state's obligation and license to care for children. Adjudications were achieved in informal, clinic-like sessions. The Supreme Court purportedly transformed juvenile court hearings into criminal-like trials via the Gault and Winship decisions. This research examines the extent to which juvenile court personnel currently perceive remnants of parens patriae in the adjudicatory hearing. One hundred workers (judges, prosecutors, defense attorneys, and probation officers) from three different juvenile courts (urban, suburban, and rural) were interviewed concerning sources of unfairness in the contemporary adjudicatory hearing. The results indicate that court workers see numerous obstacles to fairness in the juvenile court adjudicatory hearing.