Demographic Bias Research Articles

Responsible AI in Healthcare: Best Practices to Improve Healthcare Delivery

A candid discussion that dives into the recesses of AI and ethics including topics such as what lessons can be used in healthcare from tech players implementing responsible AI and unbiased systems, how can responsible AI help build a clinical support system, examples from providers and payers and learnings from responsible AI which can be used in Obamacare and clinical trial implementation.    Discussion topics will include: What is responsible AI? Is it the same as Ethical AI?  What lessons can be used in healthcare from tech players implementing responsible AI and unbiased systems? Why is responsible AI important for healthcare? Is there demographic bias in the treatment part of responsible AI? How can responsible AI help build a clinical support system?  What are some examples of responsible AI in the healthcare industry with respect to providers and payers?  What are learnings from responsible AI which can be used in Obamacare implementation? HHS introduced an interoperability rule between EMR/EHR systems in 2019. How does it help in the implementation of responsible AI?   How responsible AI is currently covered in new US government initiative such as AI.gov Responsible AI & data bias in clinical trials 

A104 PATIENT AND PUBLIC INVOLVEMENT (PPIN) IN IBD RESEARCH - A SCOPING REVIEW

Abstract Background Over the past 10 years, interest in patient and public involvement (PPIn) in research has grown. Several arguments support the engagement of patients as partners in the research process. Patients with lived experience of a condition can offer their knowledge to study design as experience-based experts, helping researchers incorporate patient-pertinent outcomes. PPIn has also been shown to boost patient enrolment and retention in clinical trials. Benefits, challenges, and best practices of PPIn have been examined in other fields. However, to date, no study has examined PPIn in inflammatory bowel disease (IBD) research. Many factors amenable to research involvement may impact IBD patients’ quality of life, including disease morbidity, complications, and efficacy/side effects of therapy. Aims This review aims to characterize methods of PPIn in IBD research and highlight themes relating to best practices, benefits, and challenges. Methods We ran a systematic search on MEDLINE, EMBASE, and Cochrane for all IBD research studies in which IBD patients were involved in the research process. PPIn included but was not limited to patient input in one of the following 3 stages: Study Design (prioritization of research topics, outcome selection, study tool development), Study Execution (recruitment, data collection & analysis), and Dissemination of Research. After abstract and full-text screening, 14 studies were selected. Results Patients were recruited for PPIn through IBD and patient organizations (7/14), outpatient clinics (4/14), tertiary care sites (2/14), and pre-existing patient advisory groups (1/14). The majority of studies (11/14) engaged patients in the development of study materials, which included a physical activity intervention for stoma patients, an IBD pregnancy decision aid, and a quality of life questionnaire. Two studies interviewed patients to determine comprehensibility of survey items and guide revisions. One study involved patients in data analysis and manuscript development. Most consultations were open-ended, including focus groups (8/14) and semi-structured interviews (3/14). According to study authors, PPIn helps guide IBD research priorities by focusing on patient-relevant issues. Authors also cited the role of PPIn in designing patient-friendly study tools. One challenge reported by 2 studies was that PPIn requires patients to have access to high-quality information and requires a significant time commitment, which may contribute to demographic biases. Conclusions The majority of IBD studies engaged patients in an open-ended format and were engaged in study design, particularly in developing study materials. Authors recommend continuous involvement of patients throughout the research process to address their research priorities. Funding Agencies None

Cohort profile: biological pathways of risk and resilience in Syrian refugee children (BIOPATH)

The BIOPATH cohort was established to explore the interplay of psychosocial and biological factors in the development of resilience and mental health problems in Syrian refugee children. Based in Lebanon, a middle-income country significantly impacted by the refugee crisis, it is the first such cohort of refugees in the Middle East. Families were recruited from informal tented settlements in the Beqaa region using purposive cluster sampling. At baseline (October 2017–January 2018), N = 3188 individuals participated [n = 1594 child–caregiver dyads; child gender, 52.6% female; mean (SD) age = 11.44 (2.44) years, range = 6–19]. Re-participation rate at 1-year follow-up was 62.8%. Individual interviews were conducted with children and primary caregivers and biological samples collected from children. Measures include: (1) children’s well-being and mental health problems (using tools validated against clinical interviews in a subsample of the cohort); (2) psychosocial risk and protective factors at the level of the individual (e.g. coping strategies), family (e.g. parent–child relationship), community (e.g. collective efficacy), and wider context (e.g. services); (3) saliva samples for genetic and epigenetic (methylation) analyses; (4) hair samples to measure cortisol, dehydroepiandrosterone (DHEA) and testosterone. This cohort profile provides details about sampling and recruitment, data collection and measures, demographic data, attrition and potential bias, key findings on resilience and mental health problems in children and strengths and limitations of the cohort. Researchers interested in accessing data should contact Professor Michael Pluess at Queen Mary University of London, UK (e-mail: m.pluess@qmul.ac.uk).

Enhancing Participation in Probability-Based Online Panels: Two Incentive Experiments and Their Effects on Response and Panel Recruitment

This article investigates how mail-based online panel recruitment can be facilitated through incentives. The analysis relies on two incentive experiments and their effects on panel recruitment, and the intermediate participation in the recruitment survey. The experiments were implemented in the context of the German Emigration and Remigration Panel Study and encompass two samples of randomly sampled persons. Tested incentives include a conditional lottery, conditional monetary incentives, and the combination of unconditional money-in-hand with conditional monetary incentives. For an encompassing evaluation of the link between incentives and panel recruitment, the article further assesses the incentives’ implications for demographic composition and panel recruitment unit costs. Multivariate analysis indicates that low combined incentives (€5/€5) or, where unconditional disbursement is unfeasible, high conditional incentives (€20) are most effective in enhancing panel participation. In terms of demographic bias, low combined incentives (€5/€5) and €10 conditional incentives are the favored options. The budget options from the perspective of panel recruitment include the lottery and the €10 conditional incentive which break-even at net sample sizes of 1000.

A Cnn-Based Multi-Level Face Alignment Approach for Mitigating Demographic Bias in Clinical Populations

A Cnn-Based Multi-Level Face Alignment Approach for Mitigating Demographic Bias in Clinical Populations

A CNN-Based Multi-Level Face Alignment Approach for Mitigating Demographic Bias in Clinical Populations

A CNN-Based Multi-Level Face Alignment Approach for Mitigating Demographic Bias in Clinical Populations

Unifying telescope and microscope: A multi-lens framework with open data for modeling emerging events

Open data is becoming ubiquitous as governments, companies, and even individuals have the option to offer more or less unrestricted access to their non-sensitive data. The benefits of open data, such as accessibility and transparency, have motivated and enabled a large number of research studies and applications in both academia and industry. However, each open data only offers a single perspective, and its potential inherent limitations (e.g., demographic biases) may lead to poor decisions and misjudgments. This paper discusses how to create and use multiple digital lenses empowered by open data, including census data (macro lens), search logs (meso lens), and social data (micro lens), to investigate general real-world events. To reveal the unique angles and perspectives brought by each open lens, we summarize and compare the underpinning open data from eleven dimensions, such as utility, data volume, dynamic variability, and demographic fairness. Then, we propose an easy-to-use and generalized open data driven framework, which automatically retrieves multi-source data, extracts features, and trains machine learning models for the event specified by answering what, when, and where questions. With low labor efforts, the framework’s generalization and automation capabilities guarantee an instant investigation of general events and phenomena, such as disasters, sports events, and political activities. We also conduct two case studies, i.e., the COVID-19 pandemic and Great American Eclipse (see Appendix), to demonstrate its feasibility and effectiveness at different time granularities.

Measuring Changes in Perceptions of Access to Pet Support Care in Underserved Communities.

Understanding social, economic, and structural barriers to accessing pet care services is important for improving the health and welfare of companion animals in underserved communities in the U.S. From May 2018-December 2019, six questions from the validated One Health Community Assessment were used to measure perceptions of access to pet care in two urban and two rural zip codes. One urban and one rural community received services from a pet support outreach program (Pets for Life), while the other served as a comparison community. After propensity score matching was performed to eliminate demographic bias in the sample (Urban = 512 participants, Rural = 234 participants), Generalized Estimating Equations were employed to compare the six measures of access to pet care between the intervention and comparison communities. The urban community with the Pets for Life intervention was associated with a higher overall measure of access to pet care compared to the urban site that did not have the Pets for Life intervention. When assessing each of the six measures of access to care, the urban community with the Pets for Life intervention was associated with higher access to affordable pet care options and higher access to pet care service providers who offer payment options than the community without the Pets for Life intervention. Further analyses with a subset of Pets for Life clients comparing pre-intervention and post-intervention survey responses revealed statistically significant positive trends in perceptions of two of the six measures of access to pet care. This study provides evidence that community-based animal welfare programming has the potential to increase perceptions of access to pet support services.

Increasing planning targets is not enough: health workforce management in rheumatology in Germany

Abstract Background Health workforce shortage in German rheumatology and its negative impact in healthcare are increasingly recognised. Health policy has increased staffing targets, but effective intervention strategies are lacking. This paper aims to systematically map the rheumatology workforce and to explore opportunities for intervention, thus contributing to better health workforce management. Methods The WHO National Health Workforce Accounts (NHWA) provided a conceptual framework for the mapping exercise. Four major sets of indicators were selected, comprising staffing levels, health labour market flows, skill-mix and education/training. Comparison of age-groups and time series was applied to explore trends. Public statistics and other secondary sources served the analysis, using descriptive methodology. Results Rheumatologists nearly doubled in absolute numbers since 2000, but this trend shows a problematic demographic bias. Between 2000 and 2019 numbers increased markedly in the groups aged +50 years, but only about 9% in younger groups under 50 years. Most alarming, since 2010 the group 40 to 50 years showed a reverse trend and strong decreases. In 2019, absolute numbers of rheumatologists working in healthcare after retirement-age exceeded those aged 40 and under. Since 2015 an expanding workforce trend overall flattened, but strongest in the hospital sector; annual inflows do not show relevant growths. Conclusions Health workforce assessment reveals negative demographic trends and decreasing appeal of the specialty especially to men, thus highlighting a need for more effective health workforce management. Intervention strategies should target innovation in resident training and task shifting and improve gender equality. Key messages Health workforce trends indicate that new health policy planning targets cannot be met with an available stock of rheumatologist. The WHO NHWA provide a useful toolbox to map health workforce trends and identify weaknesses and gaps.

Reduced false positives in autism screening via digital biomarkers inferred from deep comorbidity patterns.

Here, we develop digital biomarkers for autism spectrum disorder (ASD), computed from patterns of past medical encounters, identifying children at high risk with an area under the receiver operating characteristic exceeding 80% from shortly after 2 years of age for either sex, and across two independent patient databases. We leverage uncharted ASD comorbidities, with no requirement of additional blood work, or procedures, to estimate the autism comorbid risk score (ACoR), during the earliest years when interventions are the most effective. ACoR has superior predictive performance to common questionnaire-based screenings and can reduce their current socioeconomic, ethnic, and demographic biases. In addition, we can condition on current screening scores to either halve the state-of-the-art false-positive rate or boost sensitivity to over 60%, while maintaining specificity above 95%. Thus, ACoR can significantly reduce the median diagnostic age, reducing diagnostic delays and accelerating access to evidence-based interventions.

A Study of Gender Bias in Face Presentation Attack and Its Mitigation

In biometric systems, the process of identifying or verifying people using facial data must be highly accurate to ensure a high level of security and credibility. Many researchers investigated the fairness of face recognition systems and reported demographic bias. However, there was not much study on face presentation attack detection technology (PAD) in terms of bias. This research sheds light on bias in face spoofing detection by implementing two phases. First, two CNN (convolutional neural network)-based presentation attack detection models, ResNet50 and VGG16 were used to evaluate the fairness of detecting imposer attacks on the basis of gender. In addition, different sizes of Spoof in the Wild (SiW) testing and training data were used in the first phase to study the effect of gender distribution on the models’ performance. Second, the debiasing variational autoencoder (DB-VAE) (Amini, A., et al., Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure) was applied in combination with VGG16 to assess its ability to mitigate bias in presentation attack detection. Our experiments exposed minor gender bias in CNN-based presentation attack detection methods. In addition, it was proven that imbalance in training and testing data does not necessarily lead to gender bias in the model’s performance. Results proved that the DB-VAE approach (Amini, A., et al., Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure) succeeded in mitigating bias in detecting spoof faces.

The Role of Demographic Data Bias in the Under-Provision of Retail: A Quasi-Experimental Case Study of Low Food Access Neighborhoods in Twenty-Five Southern Metros

Municipal policy attempting to remediate low food access neighborhoods tends to focus on improving demand conditions in these neighborhoods. We investigate the role of two fundamental measures of food demand (population and income) and the biases inherent in these data in creating low food access neighborhoods. Population and income data were collected for a 1-mile radius surrounding seventy-one grocery desert sites in Southern U.S. metro areas, those results were compared to thirty-eight low-income, non-desert sites in the same metros. No significant difference between the demand characteristics of desert and non-desert sites was found in our sample—suggesting that policy may need to be refocused on issues other than demand metrics. In addition, we detected significant demand underestimation bias from one source commonly used by grocery stores. Given these findings we believe that parcel level characteristics such as visibility, accessibility, and buildability may play a larger role in remediating low food access than addressing demand conditions.

O32 Leveraging Every Door Direct Mail for Remote Recruitment of a Rural Study Sample: Response Rate and Representativeness

Background Travel restrictions imposed by the COVID-19 pandemic forced researchers to recruit remotely, further challenging the recruitment of “hard-to-reach” populations, such as rural communities. Every Door Direct Mail (EDDM) is a service of the USPS in which mail is sent to all ‘postal customers’ in a designated region. While relatively underexplored as a research tool, 1 study reported similar response rates to addressed survey mailings with lower cost and potentially lower demographic bias. Objective To evaluate the utility of EDDM as a tool for recruiting a representative sample of rural households in Appalachia Ohio. Study Design, Setting, Participants As part of a larger effort to evaluate community resource utilization in relation to diet, health and food security, a study recruitment postcard was sent via EDDM to all residential addresses (n = 31,201) within an 18 zip code region in Southeastern OH in June 2020. Adult residents were invited to complete a survey online via QR code or short link or to call for a mailed survey. Measurable Outcome/Analysis Respondent demographic characteristics were generated using SPSS and compared with the region's 2019 U.S. Census Bureau statistics. Results A total of 724 households completed the survey, reflecting a response rate lower than estimates for addressed mailings (2.3% vs 6%). Compared to census data, a greater proportion of respondents were female (68% vs 51%), and highly educated (60% vs 36% college graduates); a smaller proportion were non-Hispanic (90% vs 98%), White (88% vs 92%), had ≥1 child in the household (23% vs 28%), and a household income Conclusion EDDM was a viable method for remote recruitment of a rural sample. Further work is needed to explore its efficacy in recruiting representative samples, and studies to inform best practices for its use are warranted. Funding NIH, The Ohio State University's Initiative for Food and AgriCultural Transformation (InFACT), a Discovery Themes program (learn more at discovery.osu.edu/infact); Award Number UL1TR002733 National Center for Advancing Translational Sciences (#5591); and, the Clinical Research Center/Center for Clinical Research Management of The Ohio State University Wexner Medical Center and The Ohio State University College of Medicine in Columbus, Ohio. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Advancing Translational Sciences, the National Institutes of Health, or The Ohio State University.

Face presentation attack detection. A comprehensive evaluation of the generalisation problem

Face recognition technology is now mature enough to reach commercial products, such as smart phones or tablets. However, it still needs to increase robustness against imposter attacks. In this regard, face Presentation Attack Detection (face-PAD) is a key component in providing trustable facial access to digital devices. Despite the success of several face-PAD works in publicly available datasets, most of them fail to reach the market, revealing the lack of evaluation frameworks that represent realistic settings. Here, an extensive analysis of the generalisation problem in face-PAD is provided, jointly with an evaluation strategy based on the aggregation of most publicly available datasets and a set of novel protocols to cover the most realistic settings, including a novel demographic bias analysis. Besides, a new fine-grained categorisation of presentation attacks and instruments is provided, enabling higher flexibility in assessing the generalisation of different algorithms under a common framework. As a result, GRAD-GPAD v2, a comprehensive and modular framework is presented to evaluate the performance of face-PAD approaches in realistic settings, enabling accountability and fair comparison of most face-PAD approaches in the literature.

Health workforce development in rheumatology : Amapping exercise and wake-up call for health policy

Hintergrund und FragestellungFachkräftemangel in der Rheumatologie in Deutschland ist als Versorgungsproblem erkannt. Die Gesundheitspolitik hat mit neuen Planungszielen reagiert, aber es fehlen effektive Interventionsstrategien. Ziel dieser Studie ist ein systematischer berufsstruktureller Überblick, um die Grundlage für Interventionen zu verbessern und Möglichkeiten für ein effektives Fachkräftemanagement aufzuzeigen.MethodeDie WHO National Health Workforce Accounts (NHWA) dienen als konzeptioneller Rahmen. Ausgewählt werden 4 Indikatoren: Personalbestand, Arbeitsmarktbewegungen, Komposition und Weiterbildung. Die Exploration von Entwicklungstrends stützt sich auf vergleichende Analysen von Altersgruppen und Zeitreihen. Die Erhebung nutzt öffentliche Statistiken und andere Sekundärliteratur; die Auswertung erfolgt deskriptiv.ErgebnisseIn Deutschland sind 1076 Ärzt*innen mit einer Facharztqualifikation oder Schwerpunktbezeichnung in der internistischen Rheumatologie ärztlich tätig. Die absolute Zahl verdoppelte sich seit 2000 deutlich (91 %), aber mit einem demografischen Bias. Im Zeitraum 2000 bis 2019 stieg die Zahl der über 50-Jährigen deutlich, aber die der unter 50-Jährigen nur um 9 %; seit 2010 sind die Zahlen in der Gruppe 40 bis 50 Jahre rückläufig. Im Jahr 2019 waren mehr Rheumatolog*innen im Rentenalter als unter 40-Jährige ärztlich tätig. Seit 2015 schwächt sich der steigende Trend insgesamt ab, aber am stärksten im Krankenhaussektor; die Weiterbildungen lassen keine konstante Steigerung erkennen.SchlussfolgerungenBerufsstrukturelle Trends zeigen, dass die gesundheitspolitischen Planziele mit den verfügbaren Humanressourcen nicht zu erreichen sind. Gefordert ist ein besseres Fachkräftemanagement, insbesondere durch Innovation der Weiterbildung, Aufgabenverschiebung und verbesserte Geschlechtergerechtigkeit.

Mitigating Demographic Bias in Facial Datasets with Style-Based Multi-attribute Transfer

Deep learning has catalysed progress in tasks such as face recognition and analysis, leading to a quick integration of technological solutions in multiple layers of our society. While such systems have proven to be accurate by standard evaluation metrics and benchmarks, a surge of work has recently exposed the demographic bias that such algorithms exhibit–highlighting that accuracy does not entail fairness. Clearly, deploying biased systems under real-world settings can have grave consequences for affected populations. Indeed, learning methods are prone to inheriting, or even amplifying the bias present in a training set, manifested by uneven representation across demographic groups. In facial datasets, this particularly relates to attributes such as skin tone, gender, and age. In this work, we address the problem of mitigating bias in facial datasets by data augmentation. We propose a multi-attribute framework that can successfully transfer complex, multi-scale facial patterns even if these belong to underrepresented groups in the training set. This is achieved by relaxing the rigid dependence on a single attribute label, and further introducing a tensor-based mixing structure that captures multiplicative interactions between attributes in a multilinear fashion. We evaluate our method with an extensive set of qualitative and quantitative experiments on several datasets, with rigorous comparisons to state-of-the-art methods. We find that the proposed framework can successfully mitigate dataset bias, as evinced by extensive evaluations on established diversity metrics, while significantly improving fairness metrics such as equality of opportunity.

Efficacy of Topical High Volume Budesonide Nasal Irrigation in Post FESS Patients of Chronic Rhinosinusitis With or Without Nasal Polyposis.

Chronic rhinosinusitis (CRS) is a chronic inflammatory process of nasal mucosa and paranasal sinuses, lasting more than 12weeks, without complete resolution of symptoms. CRS is treated medically, followed by Endoscopic sinus surgery (ESS) if necessary, and supplemented by post-operative topical treatment with highly variable clinical outcomes. However, till date there is no consensus on the composition and duration of maximal medical treatment. Despite proven role of topical steroids, the mode of delivery, dose and duration of topical intranasal corticosteroids still remains debatable. Studies found that high volume sinonasal irrigation (> 50ml) using budesonide is most efficient method but still there is not sufficient data to prove this and results are variable with multiple modifiable factors therefore, this study has been conducted. (1) To determine the difference in mean decrease in Lund-Kennedy endoscopic scores and SNOT-22 scores among post ESS patients with high volume budesonide nasal irrigation nasal cavity and control nasal cavity of chronic rhinosinusitis patients. (2) To determine safety by measuring serum cortisol levels and intra ocular pressure. This is hospital based interventional, randomised, double blind, control trial study. A total of 66 patients of CRS with previous failed medical therapy were included. Same patients nasal cavities were divided into control and case nasal cavities, to avoid demographic bias. All subjects had a baseline SNOT-22 scores (Sino Nasal Outcome Test scores), Lund Kennedy endoscopy score, NCCT PNS score. All patients were undergone ESS procedure. After nasal pack removal, nasal cavities were randomly assigned 1:1 to receive normal saline irrigation (control group) or 1mg of budesonide irrigation (case group) for transnasal irrigation twice daily for 12weeks. A total of 66 patients with 132 nasal cavities were included in the study. Out of which 16 were female and 50 were male with mean age 33year and mean duration of symptoms was 38.19months. Mean duration of follow up was for 3months. Mean decrement in SNOT-22 score in control nasal cavity from 52.54(16.309) to 30.06 (18.16) and in endoscopic score from 6.53 (1.33) to 3.93 (1.6) which is statically significant (p value < 0.05) in both scores. Mean decrement in SNOT-22 score in case nasal cavity from 53.73 (15.75) to 21.15 (13.52) and in endoscopic score from 6.74 (1.8) to 2.77 (1.4) which is statically significant (p value < 0.05) in both scores. Decrement in SNOT-22 scores and endoscopy scores in case nasal cavity in comparision to control nasal cavity were compared by student 't' test and found to be statically significant (p value equals to 0.0001). In subjective outcomes 57.57% shows total improvement in control nasal cavities while 72.73% case nasal cavities shows total improvement. Our study shows high volume budesonide irrigation is safe and superior over normal saline irrigation and results were statistically comparable. Still further studies with larger sample size and longer duration of irrigation needed.Based on available evidence, high volume budesonide irrigation is statically safe and superior over normal saline irrigation.

Demographic Bias in Referral and Evaluation for Advanced Cardiac Therapies

<h3>Purpose</h3> Cardiovascular disease is the leading cause of death for women in the US. However, 80% of ventricular assist devices (VADs) are implanted in men suggesting gender bias in access or delivery to care for those with advanced heart failure (HF). Racial disparities in disease management also exist. <h3>Methods</h3> In a retrospective review of patients referred for advanced HF therapy at our center, evaluation status was classified as approved for evaluation or as terminated due to access to insurance, other or unspecified reasons. Race and gender distributions were compared to statewide proportions adjusted for incidence of HF and by evaluation status. A multivariable model of the proportion approved, accounting for gender, age (<60, 60-69, 70+), race, program (destination VAD, transplant/bridge to transplant VAD), and interactions, was constructed. <h3>Results</h3> Of 2893 referred patients, 2085 (72%) were approved for evaluation, 138 (5%) terminated due to insurance access, 108 (4%) other, and 560 (19%) unspecified. Three in 4 referrals were men, indicating a bias relative to statewide data (74% vs. 48%; p<0.001). Moreover, the proportion of men approved was 7% greater than women (95% CI: 3% - 11%; Fig. 1A). The racial composition was 25% African American and 73% Caucasian. Relative to statewide data, representation of African Americans was 5% greater (25% vs. 20%; <0.001) and comparable for Caucasians (73% vs. 74%; p = 0.55) with no differences between Caucasians vs. non-Caucasians approved (Fig. 1B). Accounting for variables in Fig. 1C, significant differences in the odds of acceptance were found by gender, race for 60-69 year-olds, and program for 70+ year-olds (Fig. 1D). <h3>Conclusion</h3> In this cohort, incoming referral bias toward males was further skewed at entry into the evaluation process for advanced HF therapy. Incoming referral bias toward African Americans is suggested. In 60-69 year-olds, approval for evaluation is skewed toward Caucasians. Understanding this data may help ensure equitable access to advanced HF therapies.

Network-Biased Technical Change: How Modern Digital Collaboration Tools Overcome Some Biases but Exacerbate Others

Using three years’ data from more than 1,000 employees at a large professional services firm, we find that adopting an expertise search tool improves employee work performance in billable revenue, which results from improvements in network connections and information diversity. More importantly, we also find that adoption does not benefit all employees equally. Two types of employees benefit more from adoption of digital collaboration tools than others. First, junior employees and women benefit more from the adoption of digital collaboration tools than do senior employees and men, respectively. These tools help employees overcome the institutional barriers to resource access faced by these employees in their searches for expertise. Second, employees with greater social capital at the time of adoption also benefit more than others. The tools eliminate natural barriers associated with traditional offline interpersonal networks, enabling employees to network even more strategically than before. We explore the mechanisms for these differential benefits. Digital collaboration tools increase the volume of communication more for junior employees and women, indicating greater access to knowledge and expertise than they had before adoption. The tools also decrease the volume of communication for people with greater social capital, indicating more efficient access to knowledge and expertise. An important implication of our findings is that digital collaboration tools have the potential to overcome some of the demographic institutional biases that organizations have long sought to change. It does so, however, at the expense of potentially creating new biases toward network-based features—a characteristic we call “network-biased technical change.”

BiasFinder: Metamorphic Test Generation to Uncover Bias for Sentiment Analysis Systems

Artificial Intelligence (AI) software systems, such as Sentiment Analysis (SA) systems, typically learn from large amounts of data that may reflect human biases. Consequently, the machine learning model in such software systems may exhibit unintended demographic bias based on specific characteristics (e.g., gender, occupation, country-of-origin, etc.). Such biases manifest in an SA system when it predicts a different sentiment for similar texts that differ only in the characteristic of individuals described. Existing studies on revealing bias in SA systems rely on the production of sentences from a small set of short, predefined templates. To address this limitation, we present BisaFinder, an approach to discover biased predictions in SA systems via metamorphic testing. A key feature of BisaFinder is the automatic curation of suitable templates based on the pieces of text from a large corpus, using various Natural Language Processing (NLP) techniques to identify words that describe demographic characteristics. Next, BisaFinder instantiates new text from these templates by filling in placeholders with words associated with a class of a characteristic (e.g., gender-specific words such as female names, she, her). These texts are used to tease out bias in an SA system. BisaFinder identifies a bias-uncovering test case when it detects that the SA system exhibits demographic bias for a pair of texts, i.e., it predicts a different sentiment for texts that differ only in words associated with a different class (e.g., male vs. female) of a target characteristic (e.g., gender). Our empirical evaluation showed that BisaFinder can effectively create a large number of realistic and diverse test cases that uncover various biases in an SA system with a high true positive rate of up to 95.8\%.