Contextual Bias Research Articles

Towards equitable AI in oncology.

Artificial intelligence (AI) stands at the threshold of revolutionizing clinical oncology, with considerable potential to improve early cancer detection and risk assessment, and to enable more accurate personalized treatment recommendations. However, a notable imbalance exists in the distribution of the benefits of AI, which disproportionately favour those living in specific geographical locations and in specific populations. In this Perspective, we discuss the need to foster the development of equitable AI tools that are both accurate in and accessible to a diverse range of patient populations, including those in low-income to middle-income countries. We also discuss some of the challenges and potential solutions in attaining equitable AI, including addressing the historically limited representation of diverse populations in existing clinical datasets and the use of inadequate clinical validation methods. Additionally, we focus on extant sources of inequity including the type of model approach (such as deep learning, and feature engineering-based methods), the implications of dataset curation strategies, the need for rigorous validation across a variety of populations and settings, and the risk of introducing contextual bias that comes with developing tools predominantly in high-income countries.

Cognitive bias and contextual information management: Considerations for forensic handwriting examinations

AbstractAs for other largely subjective perceptual and cognitive processes, the forensic analysis and interpretation of handwriting is potentially susceptible to bias. Cognitive bias may subconsciously influence the outcome of specific tasks at any point in an examination, during pre‐analysis, analysis, comparison, interpretation and/or reporting of results. One class of cognitive bias, contextual bias, may arise from information provided by the client, results reported from other forensic disciplines within a laboratory or the content of the questioned and/or known material itself. The base rate information (i.e., examiner knowledge, experience, and innate beliefs) associated with the type and range of examinations an examiner has previously encountered may also lead to bias. Bias introduced at any stage of the examination process, and even prior to the examination (e.g., when deciding what is collected for examination) may affect the examination outcome.This Overview focuses on contextual information management (CIM) protocols that are used to remove potentially biasing task‐irrelevant case information from the view of the reporting forensic handwriting examiner (FHE). Where laboratory resources and logistics allow for these procedures to be in place, they should be applied as part of the pre‐analysis stage of the examination process that is, at a stage in the examination prior to the reporting FHE accessing the examination material or associated documentation. Also highlighted are some additional procedures that have been proposed by others to minimize the opportunity for cognitive bias to arise in handwriting examinations.This article is categorized under: Jurisprudence and Regulatory Oversight > Expert Evidence and Narrative

Analysis of Oncology and Radiation Therapy Representation on the National Board of Medical Examiners Official Practice Material for the United States National Standardized Medical Board Examinations.

Radiation therapy (RT) is a critical component of multidisciplinary cancer care, but has inconsistent curricular exposure. We characterize the radiation oncology (RO) content on the standardized undergraduate medical examinations by comparing its context and prevalence with other domains in oncology. National Board of Medical Examiners (NBME) self-assessments and sample questions for the United States Medical Licensing Exam (USMLE) Steps 1-3 and NBME clinical science shelf examinations were accessed (n = 3878). Questions were inductively analyzed for content pertaining to oncology and treatment modalities of RT, systemic therapy (ST), and surgical intervention (SI). Questions were coded using USMLE Physician Tasks/Competencies and thematic analysis. Descriptive statistics and analyses using the Kruskal-Wallis test are reported. A total of 337 questions (8.6%) within the USMLE and shelf exams included oncology content, with 101 questions (2.6%) referencing at least one cancer treatment modality (n = 35 RT, 45 ST, 57 SI). Treatment questions were more common on USMLE Step 2 CK (n = 35/101, 32%) compared to Step 1 (n = 23/101, 23%) and Step 3 (n = 8/101, 8%) (p < 0.001). RT was significantly less likely to be the correct answer (2/35, 6%) compared to ST (4/45, 9%) and SI (18/57, 32%) (p = 0.003). Therapeutic oncology questions are uncommon on the examination material, with an under-representation of radiation-related content, and contextual bias favoring surgical approaches. We advocate for greater RO involvement in the content creation of such examinations to help trainees better understand multidisciplinary cancer care.

The effects of cognitive bias, examiner expertise, and stimulus material on forensic evidence analysis.

Forensic examiners have come under scrutiny due to high-profile exonerations, highlighting the consequences that contextual bias can have on investigations. Researchers have proposed solutions to reduce the effects of bias including blind testing and redacting task-irrelevant information. Practitioners have concerns over the limitations of some of this research that uses untrained students to examine complex pieces of forensic evidence (e.g., fingerprints) (1; but see 2 for studies including trained experts and/or actual casework). This study sought to (a) examine the effect of contextual bias on examiners' evaluation of forensic evidence by varying the amount of pre-comparison information available to participants, (b) compare student and expert examiners' performance and their vulnerability to contextual bias, and (c) examine the effects of contextual bias on examiners' evaluation of different types of forensic evidence. Expert fingerprint examiners and student participants were presented with varying amounts of pre-comparison case information and compared matching and non-matching fingerprint and footwear impression evidence. Results suggest no effects of blinding examiners from case information or redacting task-irrelevant information. As expected, expert fingerprint examiners were more likely to correctly identify matching fingerprints and correctly exclude non-matching fingerprints than students. However, expert fingerprint examiners were no better than student participants at comparing footwear impression evidence. These findings suggest that sample, stimulus selection, and discipline-specific training matter when investigating bias in forensic decision making. These findings suggest caution when using forensic stimuli with student samples to investigate forensic decision-making and highlight the need for more research on redaction procedures.

Contextual bias by Forensic Document Examination trainees: An empirical study from China

The impact of contextual bias has been repeatedly demonstrated across forensic domains; however, research on this topic in China is scarce. To examine the prevalence of contextual bias in pattern feature-comparison disciplines, we conducted an experiment involving 24 forensic document examination students. The aim was to determine whether knowledge of different contextual information influenced their forensic decision-making. Participants were divided into different context groups and tasked with examining whether questioned signatures with ambiguous features matched reference signatures. The results of independent-samples t-tests for their decision score data in the two context groups exhibited a statistically significant difference (p < 0.05, Cohen’s d > 0.8). Moreover, the submitted forensic reports by participants disclosed a biased evaluation of handwriting features. These findings show how contextual information can bias forensic decision-making in handwriting examination. Context management with complementary strategies such as case triage, cognitive training and decision-making transparency must be implemented to minimize bias in handwriting examination.

PARAMETRIC DIFFERENCES BETWEEN ENGLISH AND KOREAN RESPONSE PARTICLES: A DISCOURSE‐BASED HPSG APPROACH1

AbstractAcross languages, polarity particles like yes and no can be used as a response to the polar (yes/no) question. This paper offers a discourse‐based account for the distribution and interpretation of response particles in languages like English and Korean. The paper supports the view that response particles are anaphoric in nature and their interpretation is determined by the salient antecedent evoked by the context in question. The paper also suggests that the parametric differences between English‐style polarity‐based and Korean‐style truth‐based answering systems have to do with tight interactions between the anaphoric nature of response particles and discourse structure, rather than with the position of negation in syntactic structures. This direction, formalized in the framework of HPSG, offers a simpler account for the parametric differences between the two different answering systems as well as other related puzzles concerning variations in contextual bias.

Contextual bias in forensic toxicology decisions: A follow-up empirical study from China.

The impact of contextual bias has been demonstrated repeatedly across forensic domains; however, research on this topic in forensic toxicology is very limited. In our previous study, experimental data from only one context version were compared with the actual forensic biasing casework. As a follow-up, this controlled experiment with 159 forensic toxicology practitioners was conducted, to test whether knowledge of different contextual information influenced their forensic decision-making. Participants in different context groups were tasked to identify testing strategies for carbon monoxide and opiate drugs. The results of chi-squared tests for their selections and two context groups exhibited statistically significant differences (p < 0.05 or p < 0.01). These findings show contextual information can bias forensic toxicology decisions about testing strategies, despite it is a relatively objective domain in forensic science.

The Most Common Types of Bias in a Human Bitemark Analysis

Given that some suspected perpetrators were wrongly convicted, a defective bitemark analysis is comparable to dentists’ most crucial clinical decisions regarding assessment. Bias affects human bitemark analysis beyond the limitation of the evidence itself. The aim of this study was to explore the potential for different types of bias in bitemark analysis and the methods involved in that analysis by conducting a scoping review. Results showed that the 14 articles that explore the topic of bias in bitemark analysis were published from 2006-2022. Publications were from the USA mainly (n = 7), followed by the UK (n = 3), Australia (n = 2), New Zealand, (n = 1) and the Netherlands (n = 1). Of these publications, 36% addressed contextual bias, while 57% acknowledged cognitive bias. According to the findings, preventive measures consist of limiting the availability of unrelated data during research, employing several comparison samples for a more impartial assessment, and repeating the analysis while being blind to past findings. Nevertheless, the physical limitations of the evidence such as distortions are still strongly present.

Can professionals "keep the tiller straight" in organizations? Resistance to reframing and decoy alternatives in workplace decision-making.

So far, little is known about the ability to contrast contextual bias as a protective factor in an ever-changing organizational environment. This study assessed whether professionals with different seniority can resist the reframing and the decoy effect under decision-making conditions and whether decision-making styles can predict the resistance to such covert influence tactics. To reach this aim, two groups of professionals divided into senior and junior professionals performed two novel tasks, a Resistance to Reframe Task (RRT) and a Resistance to Alternatives Task (RAT), which, by including ecological scenarios that represent typical decision situations that could arise in the company, can measure the resistance to such covert influence tactics. Decision-making styles were measured through the General Decision-Making Style (GDMS) and the Maximization Scale (MS). Results showed that all professionals were able to resist more to the reframing (at the RRT) than the decoy alternatives (RAT), without any difference between groups. In addition, higher GDMS-dependent subscale scores predict lower RRT scores, especially in the group of senior professionals. However, in the group of junior professionals, the GDMS-dependent subscale and MS high standards subscale predicted lower RAT scores. To conclude, this study showed that professionals know how to "keep the tiller straight" in organizations, especially when facing reframing conditions, rather than decoy alternatives; however, the predominance of dependent decision-making styles (for both senior and junior professionals) and the tendency to hold high standards in decisions (mainly for juniors) could undermine their resistance capacity and make them vulnerable to these covert influence tactics.

Simple Primitives With Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-Shot Learning.

The task of Open-World Compositional Zero-Shot Learning (OW-CZSL) is to recognize novel state-object compositions in images from all possible compositions, where the novel compositions are absent during the training stage. The performance of conventional methods degrades significantly due to the large cardinality of possible compositions. Some recent works consider simple primitives (i.e., states and objects) independent and separately predict them to reduce cardinality. However, it ignores the heavy dependence between states, objects, and compositions. In this paper, we model the dependence via feasibility and contextuality. Feasibility-dependence refers to the unequal feasibility of compositions, e.g., hairy is more feasible with cat than with building in the real world. Contextuality-dependence represents the contextual variance in images, e.g., cat shows diverse appearances when it is dry or wet. We design Semantic Attention (SA) to capture the feasibility semantics to alleviate impossible predictions, driven by the visual similarity between simple primitives. We also propose a generative Knowledge Disentanglement (KD) to disentangle images into unbiased representations, easing the contextual bias. Moreover, we complement the independent compositional probability model with the learned feasibility and contextuality compatibly. In the experiments, we demonstrate our superior or competitive performance, SA-and-kD-guided Simple Primitives (SAD-SP), on three benchmark datasets.

Shifting decision thresholds can undermine the probative value and legal utility of forensic pattern-matching evidence

Forensic pattern analysis requires examiners to compare the patterns of items such as fingerprints or tool marks to assess whether they have a common source. This article uses signal detection theory to model examiners' reported conclusions (e.g., identification, inconclusive, or exclusion), focusing on the connection between the examiner's decision threshold and the probative value of the forensic evidence. It uses a Bayesian network model to explore how shifts in decision thresholds may affect rates and ratios of true and false convictions in a hypothetical legal system. It demonstrates that small shifts in decision thresholds, which may arise from contextual bias, can dramatically affect the value of forensic pattern-matching evidence and its utility in the legal system.

Reassessing referral of touch following peripheral deafferentation: The role of contextual bias

Some amputees have been famously reported to perceive facial touch as arising from their phantom hand. These referred sensations have since been replicated across multiple neurological disorders and were classically interpreted as a perceptual correlate of cortical plasticity. Common to all these and related studies is that participants might have been influenced in their self-reports by the experimental design or related contextual biases. Here, we investigated whether referred sensations reports might be confounded by demand characteristics (e.g., compliance, expectation, suggestion). Unilateral upper-limb amputees (N = 18), congenital one-handers (N = 19), and two-handers (N = 22) were repeatedly stimulated with computer-controlled vibrations on 10 body-parts and asked to report the occurrence of any concurrent sensations on their hand(s). To further manipulate expectations, we gave participants the suggestion that some of these vibrations had a higher probability to evoke referred sensations. We also assessed similarity between (phantom) hand and face representation in primary somatosensory cortex (S1), using functional Magnetic Resonance Imaging (fMRI) multivariate representational similarity analysis. We replicated robust reports of referred sensations in amputees towards their phantom hand; however, the frequency and distribution of reported referred sensations were similar across groups. Moreover, referred sensations were evoked by stimulation of multiple body-parts and similarly reported on both the intact and phantom hand in amputees. Face-to-phantom-hand representational similarity was not different in amputees' missing hand region, compared with controls. These findings weaken the interpretation of referred sensations as a perceptual correlate of S1 plasticity and reveal the need to account for contextual biases when evaluating anomalous perceptual phenomena.

A Probabilistic Formalisation of Contextual Bias: from Forensic Analysis to Systemic Bias in the Criminal Justice System

Abstract Researchers have found evidence of contextual bias in forensic science, but the discussion of contextual bias is currently qualitative. We formalise existing empirical research and show quantitatively how biases can be propagated throughout the legal system, all the way up to the final determination of guilt in a criminal trial. We provide a probabilistic framework for describing how information is updated in a forensic analysis setting by using the ratio form of Bayes’ rule. We analyse results from empirical studies using this framework and employ simulations to demonstrate how bias can be compounded where experiments do not exist. We find that even minor biases in the earlier stages of forensic analysis can lead to large, compounded biases in the final determination of guilt in a criminal trial.

The evidential strength of a combination of corresponding class features in tire examination

Tire marks are an important type of forensic evidence as they are frequently encountered at crime scenes. When the tires of a suspect’s car are compared, the evidence can be very strong if so-called ‘acquired features’ are observed to correspond. When only ‘class characteristics’ such as parts of the tire pattern are observed to correspond, it is obvious that many other tires will exist that also correspond, and so this evidence is usually considered very weak or is simply ignored. Like Benedict et al. (2014) we argue that such evidence can still be strong and should be taken into account. We describe a method for assessing the evidential strength of a set of corresponding class characteristics by presenting a case example from the Netherlands in which tire marks were obtained. Only part of two different tire patterns were visible, in combination with measurements on the axes width. Suitable databases were found already existing and accessible to forensic experts. We show how such data can be used to quantify the strength of evidence and how it can be reported. We also show how the risk of bias due to information surrounding the case may be minimized in cases like this. Our ‘blind’ procedure enables the expert to report a correspondence between class features in a more convincing way than standard procedures allow. In the particular exemplar case quite strong evidence was obtained, which was accepted and used by the Dutch court. We generalize this procedure for quantifying the evidential value of an expert’s opinion of a correspondence. This examination procedure can be applied directly to other types of pattern evidence such as shoeprints, fingerprints, or images. Furthermore, it is ‘blind’ in the sense that the risk of contextual bias is minimized.

Context Aware Recommender Systems: A Novel Approach Based on Matrix Factorization and Contextual Bias

In the world of Big Data, a tool capable of filtering data and providing choice support is crucial. Recommender Systems have this aim. These have evolved further through the use of information that would improve the ability to suggest. Among the possible exploited information, the context is widely used in literature and leads to the definition of the Context-Aware Recommender System. This paper proposes a Context-Aware Recommender System based on the concept of embedded context. This technique has been tested on different datasets to evaluate its accuracy. In particular, the use of multiple datasets allows a deep analysis of the advantages and disadvantages of the proposed approach. The numerical results obtained are promising.

Contextual bias on decision-making in forensic toxicology: First survey from China

In forensic science, contextual bias has been used to describe the tendency for a forensic analysis to be influenced by task-irrelevant background information. Contextual bias is not limited to subjective pattern matching disciplines based on visual recognition (e.g., fingerprints, handwritings, and tool marks), and has been demonstrated in objective analytical disciplines based on quantitative instruments (e.g., DNA typing). Recently, several publications suggested that contextual bias can even affect decision-makings in forensic toxicology. To obtain more extensive evidence of contextual bias, we surveyed 200 forensic toxicology practitioners from China for 1) unconscious bias in two hypothetical forensic toxicology cases with contextual information, 2) understanding of the concept and nature of contextual bias, 3) communication between investigators and forensic examiners, 4) and perception of the task-relevance of contextual information. The results revealed that 1) most of the participants made a decision deviating from standard processes under the potentially biasing context; 2) they also showed a low level of familiarity with the concept and nature of contextual bias; 3) they had a close contact with police investigators and even a dual role as a crime scene investigator and laboratory examiner, especially those affiliated with police departments; 4) and there was a general opinion that all available case information should be considered in the forensic toxicological analysis, even if the information should be specifically identified as task-irrelevant. As the first empirical work in China, this study attempted to draw more attention to the potential impact of Chinese forensic scientists’ lack of appreciation of cognitive factors in forensic practice, and promote more extensive empirical research in various forensic domains.

Ethical Considerations Related to Using Machine Learning-Based Prediction of Mortality in the Pediatric Intensive Care Unit

Machine learning shows promise for developing prediction models that could improve care in the pediatric intensive care unit (PICU). Advocates claim these systems enhance prognostic accuracy and can adapt to changing clinical practices by adding more and new large-scale child health data. Accurate predictive models using machine learning could benefit decision-making and care delivery and, in turn, outcomes for patients and families. Despite their potential, some of these models may replicate the biases of their training datasets or may be biased in other ways (eg, label bias or contextual bias), and are built without the capacity to explain how they reach decisions (so-called black boxes).

World Heart Day 2021: COVID-19, digital health, and tackling cardiovascular disease

World Heart Day 2021: COVID-19, digital health, and tackling cardiovascular disease

The biasing impact of irrelevant contextual information on forensic odontology radiograph matching decisions

The potential biasing effect of irrelevant context information on the forensic odontology method of radiograph-based identification has never been empirically investigated despite being a recognized problem in other forensic science disciplines. This study examines the effect of irrelevant context information on the probability judgment of match (JOM) of practicing forensic odontologist and dentist participants who were asked to match pairs of dental radiographs supplemented with irrelevant case information.The irrelevant case information contained domain task-irrelevant context information which varied in strength (strong or weak). It suggested either supportive or contradictory bias relative to the actual match status of the radiograph pairs. The dental radiographs consisted of verified match and non-match radiographs pairs sampled and de-identified from actual forensic cases. Changes in accuracy and JOM between supportive and contradictory contexts conditions revealed a contextual bias. Mixed model analysis showed that strong supportive context increased the odds ratio of correct decisions by a factor of 2.4 [1.23, 4.46]; p = 0.0097. Consistent with the biasing effect, the JOM score differences between strong supportive and contradictory irrelevant context information were 1.03 and 0.43 respectively for the non-match and match decisions. The direction of context suggestion (p = 0.0067), the radiograph match status (p = 0.014), and their interactions (p = 0.0061), were all found to impact the participants’ decision. The weak context information was not strong enough to have a significant effect on accuracy or JOM scores.This study demonstrates that radiograph match judgment is affected and can be biased by strong irrelevant contextual information.

Contextual bias, the democratization of healthcare, and medical artificial intelligence in low- and middle-income countries.

Medical artificial intelligence (MAI) creates an opportunity to radically expand access to healthcare across the globe by allowing us to overcome the persistent labor shortages that limit healthcare access. This democratization of healthcare is the greatest moral promise of MAI. Whatever comes of the enthusiastic discourse about the ability of MAI to improve the state-of-the-art in high-income countries (HICs), it will be far less impactful than improving the desperate state-of-the-actual in low- and middle-income countries (LMICs). However, the almost exclusive development of MAI in HICs risks this promise being thwarted by contextual bias, an algorithmic bias that arises when the context of the training data is significantly dissimilar from potential contexts of application, which makes the unreflective application of HIC-based MAI in LMIC contexts dangerous. The use of MAI in LMICs demands careful attention to context. In this paper, I aim to provide that attention. First, I illustrate the dire state of healthcare in LMICs and the hope that MAI may help us to improve this state. Next, I show that the radical differences between the health contexts of HICs and those of LMICs create an extraordinary risk of contextual bias. Then, I explore ethical challenges raised by this risk, and propose policies that will help to overcome those challenges. Finally, I sketch a wide range of related issues that need to be addressed to ensure that MAI has a positive impact on LMICs-and is able to improve, rather than worsen, global health equity.