Dataset Bias Research Articles

Revisiting the Importance of Amplifying Bias for Debiasing

In image classification, debiasing aims to train a classifier to be less susceptible to dataset bias, the strong correlation between peripheral attributes of data samples and a target class. For example, even if the frog class in the dataset mainly consists of frog images with a swamp background (i.e., bias aligned samples), a debiased classifier should be able to correctly classify a frog at a beach (i.e., bias conflicting samples). Recent debiasing approaches commonly use two components for debiasing, a biased model fB and a debiased model fD. fB is trained to focus on bias aligned samples (i.e., overfitted to the bias) while fD is mainly trained with bias conflicting samples by concentrating on samples which fB fails to learn, leading fD to be less susceptible to the dataset bias. While the state of the art debiasing techniques have aimed to better train fD, we focus on training fB, an overlooked component until now. Our empirical analysis reveals that removing the bias conflicting samples from the training set for fB is important for improving the debiasing performance of fD. This is due to the fact that the bias conflicting samples work as noisy samples for amplifying the bias for fB since those samples do not include the bias attribute. To this end, we propose a simple yet effective data sample selection method which removes the bias conflicting samples to construct a bias amplified dataset for training fB. Our data sample selection method can be directly applied to existing reweighting based debiasing approaches, obtaining consistent performance boost and achieving the state of the art performance on both synthetic and real-world datasets.

Social Bias Meets Data Bias: The Impacts of Labeling and Measurement Errors on Fairness Criteria

Although many fairness criteria have been proposed to ensure that machine learning algorithms do not exhibit or amplify our existing social biases, these algorithms are trained on datasets that can themselves be statistically biased. In this paper, we investigate the robustness of existing (demographic) fairness criteria when the algorithm is trained on biased data. We consider two forms of dataset bias: errors by prior decision makers in the labeling process, and errors in the measurement of the features of disadvantaged individuals. We analytically show that some constraints (such as Demographic Parity) can remain robust when facing certain statistical biases, while others (such as Equalized Odds) are significantly violated if trained on biased data. We provide numerical experiments based on three real-world datasets (the FICO, Adult, and German credit score datasets) supporting our analytical findings. While fairness criteria are primarily chosen under normative considerations in practice, our results show that naively applying a fairness constraint can lead to not only a loss in utility for the decision maker, but more severe unfairness when data bias exists. Thus, understanding how fairness criteria react to different forms of data bias presents a critical guideline for choosing among existing fairness criteria, or for proposing new criteria, when available datasets may be biased.

Sourcing and Bias in the Study of Coups: Lessons from the Middle East

Abstract The last two decades have seen an increased focus on reporting bias in large-N datasets. Research on coups d’etat has similarly increased given the availability of coup datasets. This essay argues that while the availability of such data has pushed scholarship forward, the data collection process behind these efforts remains plagued with limitations common to event datasets. Rather than building on what previous projects have accomplished, researchers have invariably developed “new” datasets that suffer from the same problems as earlier efforts. Specifically, we point to reliance on international news sources such as The New York Time and Keesing’s Record of World Events - without the adequate consultation of regional sources and expertise - as a source of concern. We explore this issue by assessing the coverage of coup events from three country cases from the post-colonial Middle East: Syria, Jordan, and Saudi Arabia. Our findings show that while existing data on successful and failed coups are largely adequate, scholars interested in coup plots and rumors will require a wider breadth of source material to identify such cases.

#5490 GENERATIVE ARTIFICIAL INTELLIGENCE FOR CREATION OF SYNTHETIC HYPERTENSION TRIAL DATA

Abstract Background and Aims Synthetic data can be an effective supplement or alternative to real data for the training of machine learning models. Synthetic data may also be used to evaluate new tools, develop educational curricula, or remove undesirable biases in datasets. We aim to evaluate four synthetic data generation methods applied to hypertension randomized clinical trial data. Method The Systolic Blood Pressure Intervention Trial (SPRINT) trial showed that intensive BP control to SBP <120 mm Hg results in significant cardiovascular benefits in high-risk patients with hypertension compared with routine BP control to <140 mm Hg. The Synthetic Data Vault (SDV) is a Synthetic Data Generation ecosystem of libraries that allows users to easily generate new Synthetic Data that has the same format and statistical properties as the original dataset. SDV supports multiple types of data, including date-times, discrete-ordinal, categorical, and numerical. SPRINT data was pre-processed to create a single table of 140,000 patient visits with baseline variables (age, sex, race, aspirin use, estimated Glomerular Filtration Rate (eGFR)) and visit level variables (systolic and diastolic blood pressure, heart rate and total number of antihypertensive medications at end of visit). Using the SDV library for python, we used four generative models to create synthetic SPRINT data, 1. Gaussian copula model, 2. Conditional Tabular Generative adversarial network (CTGAN), 3. CopulaGan model, and 4. Tabular Variational Auto-encode (TVAE). We evaluated the results using the SDMetrics library which includes the shapes of the columns (marginal distributions), the pairwise trends between the columns (correlations), reproduce mathematical properties from your original data and new row synthesis. Finally, an overall quality score which represents an amalgamation of the marginal distribution and correlations was computed, where 0 indicates the lowest quality and 1 indicates the highest. Results Two hundred thousand synthetic patient visits were created for each method. The overall quality scores in order were 90.67% for Gaussian copula, 86.77% for TVAE, 81.03% for CTGAN’, and 79.7% for CopulaGAN. The column shape score which represents the marginal distribution was highest for Gaussian Copula (94.54%), followed by TVAE (88.44%), CTGAN (82.35%), and Copula GAN (80.27%). The column pair trend which corresponds to correlations was highest for Gaussian Copula (86.8%), followed by TAVE (85.1%), CTGAN (79.72%), and Copula GAN (79.12%). Conclusion Gaussian copula created the highest scoring synthetic SPRINT data based on the marginal distribution, correlations, and overall score. The Synthetic Data Vault is a feasible collection of methods for generation of synthetic clinical trial data for training future machine learning and AI models.

A Deep‐Learning Ensemble Method to Detect Atmospheric Rivers and Its Application to Projected Changes in Precipitation Regime

AbstractThis study aims to detect atmospheric rivers (ARs) around the world by developing a deep‐learning ensemble method using AR catalogs of the ClimateNet data set. The ensemble method, based on 20 semantic segmentation algorithms, notably reduces the bias of the testing data set, with its intersection over union score being 1.7%–10.1% higher than that of individual algorithms. This method is then applied to the Coupled Model Intercomparison Project Phase 6 (CMIP6) datasets to quantify AR frequency and its related precipitation in the historical period (1985–2014) and future period (2070–2099) under the Shared Socioeconomic Pathways 5–8.5 warming scenario. The six key regions, which are distributed in different continents of the globe and greatly influenced by ARs, are particularly highlighted. The results show that CMIP6 multi‐model mean with the deep‐learning ensemble method reasonably reproduces the observed AR frequency. In most key regions, both heavy precipitation (90–99 percentile) and extremely heavy precipitation (>99 percentile) are projected to increase in a warming climate mainly due to the increased AR‐related precipitation. The AR contributions to future heavy and extremely heavy precipitation increase range from 145.1% to 280.5% and from 36.2% to 213.5%, respectively, indicating that ARs should be taken into account to better understand the future extreme precipitation changes.

Movedesign: Shiny R app to evaluate sampling design for animal movement studies

Abstract Projects focused on movement behaviour and home range are commonplace, but beyond a focus on choosing appropriate research questions, there are no clear guidelines for such studies. Without these guidelines, designing an animal tracking study to produce reliable estimates of space‐use and movement properties (necessary to answer basic movement ecology questions), is often done in an ad hoc manner. We developed ‘movedesign’, a user‐friendly Shiny application, which can be utilized to investigate the precision of three estimates regularly reported in movement and spatial ecology studies: home range area, speed and distance travelled. Conceptually similar to statistical power analysis, this application enables users to assess the degree of estimate precision that may be achieved with a given sampling design; that is, the choices regarding data resolution (sampling interval) and battery life (sampling duration). Leveraging the ‘ctmm’ R package, we utilize two methods proven to handle many common biases in animal movement datasets: autocorrelated kernel density estimators (AKDEs) and continuous‐time speed and distance (CTSD) estimators. Longer sampling durations are required to reliably estimate home range areas via the detection of a sufficient number of home range crossings. In contrast, speed and distance estimation requires a sampling interval short enough to ensure that a statistically significant signature of the animal's velocity remains in the data. This application addresses key challenges faced by researchers when designing tracking studies, including the trade‐off between long battery life and high resolution of GPS locations collected by the devices, which may result in a compromise between reliably estimating home range or speed and distance. ‘movedesign’ has broad applications for researchers and decision‐makers, supporting them to focus efforts and resources in achieving the optimal sampling design strategy for their research questions, prioritizing the correct deployment decisions for insightful and reliable outputs, while understanding the trade‐off associated with these choices.

DiPPS: Differentially Private Propensity Scores for Bias Correction

In surveys, it is typically up to the individuals to decide if they want to participate or not, which leads to participation bias: the individuals willing to share their data might not be representative of the entire population. Similarly, there are cases where one does not have direct access to any data of the target population and has to resort to publicly available proxy data sampled from a different distribution. In this paper, we present Differentially Private Propensity Scores for Bias Correction (DiPPS), a method for approximating the true data distribution of interest in both of the above settings. We assume that the data analyst has access to a dataset D' that was sampled from the distribution of interest in a biased way. As individuals may be more willing to share their data when given a privacy guarantee, we further assume that the analyst is allowed locally differentially private access to a set of samples D from the true, unbiased distribution. Each data point from the private, unbiased dataset D is mapped to a probability distribution over clusters (learned from the biased dataset D'), from which a single cluster is sampled via the exponential mechanism and shared with the data analyst. This way, the analyst gathers a distribution over clusters, which they use to compute propensity scores for the points in the biased D', which are in turn used to reweight the points in D' to approximate the true data distribution. It is now possible to compute any function on the resulting reweighted dataset without further access to the private D. In experiments on datasets from various domains, we show that DiPPS successfully brings the distribution of the available dataset closer to the distribution of interest in terms of Wasserstein distance. We further show that this results in improved estimates for different statistics, in many cases even outperforming differential privacy mechanisms that are specifically designed for these statistics.

Bias Analysis in Healthcare Time Series (BAHT) Decision Support Systems from Meta Data.

One of the hindrances in the widespread acceptance of deep learning-based decision support systems in healthcare is bias. Bias in its many forms occurs in the datasets used to train and test deep learning models and is amplified when deployed in the real world, leading to challenges such as model drift. Recent advancements in the field of deep learning have led to the deployment of deployable automated healthcare diagnosis decision support systems at hospitals as well as tele-medicine through IoT devices. Research has been focused primarily on the development and improvement of these systems leaving a gap in the analysis of the fairness. The domain of FAccT ML (fairness, accountability, and transparency) accounts for the analysis of these deployable machine learning systems. In this work, we present a framework for bias analysis in healthcare time series (BAHT) signals such as electrocardiogram (ECG) and electroencephalogram (EEG). BAHT provides a graphical interpretive analysis of bias in the training, testing datasets in terms of protected variables, and analysis of bias amplification by the trained supervised learning model for time series healthcare decision support systems. We thoroughly investigate three prominent time series ECG and EEG healthcare datasets used for model training and research. We show the extensive presence of bias in the datasets leads to potentially biased or unfair machine-learning models. Our experiments also demonstrate the amplification of identified bias with an observed maximum of 66.66%. We investigate the effect of model drift due to unanalyzed bias in datasets and algorithms. Bias mitigation though prudent is a nascent area of research. We present experiments and analyze the most prevalently accepted bias mitigation strategies of under-sampling, oversampling, and the use of synthetic data for balancing the dataset through augmentation. It is important that healthcare models, datasets, and bias mitigation strategies should be properly analyzed for a fair unbiased delivery of service.

Summaries of Articles

Summaries of Articles

Turning observations into biodiversity data: Broadscale spatial biases in community science

AbstractBiodiversity community science projects are growing rapidly in popularity. The enormous amounts of data generated by these programs are transforming how we conduct ecological research and conservation management. However, as with other biodiversity surveys, community science datasets suffer from biases in time and locations of observations. To better use these data, we modeled the spatial biases present in the popular community science platform, iNaturalist. iNaturalist uses crowdsourcing to collect georeferenced and time‐stamped observations of all taxa worldwide. With its wealth of biodiversity data, iNaturalist is now being used to answer a broad range of questions in ecology and conservation, but little is known about the platform's spatial biases. We focus on the more than 1.75 million iNaturalist observations available (as of December 2021) from British Columbia, Canada, a region with a strong community science presence and diversity of ecosystems. Using machine learning and species distribution modeling, we examined which landscape factors (e.g., protected areas, roads, human population density, habitat zones, elevation) were most important in determining where observations are taken, and we created a predicted probability map revealing how likely different regions are to be sampled by community scientists. We found strong road biases for observations in iNaturalist, with over 94% of observations within 1 km of roads. In addition, human population density and broad habitat ecosystem zones played a large role in predicting where iNaturalist observations occur across the landscape. These methods demonstrate tools for modeling the effects of spatial biases in large opportunistic datasets that can then be used to produce more accurate species distribution and biodiversity models from community science data.

Dataset Bias Prediction for Few-Shot Image Classification

Dataset bias is a significant obstacle that negatively affects image classification performance, especially in few-shot learning, where datasets have limited samples per class. However, few studies have focused on this issue. To address this, we propose a bias prediction network that recovers biases such as color from the extracted features of image data, resulting in performance improvement in few-shot image classification. If the network can easily recover the bias, the extracted features may contain the bias. Therefore, the whole framework is trained to extract features that are difficult for the bias prediction network to recover. We evaluate our method by integrating it with several existing few-shot learning models across multiple benchmark datasets. The results show that the proposed network can improve the performance in different scenarios. The proposed approach effectively reduces the negative effect of the dataset bias, resulting in the performance improvements in few-shot image classification. The proposed bias prediction model is easily compatible with other few-shot learning models, and applicable to various real-world applications where biased samples are prevalent, such as VR/AR systems and computer vision applications.

Availability of information needed to evaluate algorithmic fairness — A systematic review of publicly accessible critical care databases

BackgroundMachine learning (ML) may improve clinical decision-making in critical care settings, but intrinsic biases in datasets can introduce bias into predictive models. This study aims to determine if publicly available critical care datasets provide relevant information to identify historically marginalized populations. MethodWe conducted a review to identify the manuscripts that report the training/validation of ML algorithms using publicly accessible critical care electronic medical record (EMR) datasets. The datasets were reviewed to determine if the following 12 variables were available: age, sex, gender identity, race and/or ethnicity, self-identification as an indigenous person, payor, primary language, religion, place of residence, education, occupation, and income. Results7 publicly available databases were identified. Medical Information Mart for Intensive Care (MIMIC) reports information on 7 of the 12 variables of interest, Sistema de Informação de Vigilância Epidemiológica da Gripe (SIVEP-Gripe) on 7, COVID-19 Mexican Open Repository on 4, and eICU on 4. Other datasets report information on 2 or fewer variables. All 7 databases included information about sex and age. Four databases (57%) included information about whether a patient identified as native or indigenous. Only 3 (43%) included data about race and/or ethnicity. Two databases (29%) included information about residence, and one (14%) included information about payor, language, and religion. One database (14%) included information about education and patient occupation. No databases included information on gender identity and income. ConclusionThis review demonstrates that critical care publicly available data used to train AI algorithms do not include enough information to properly look for intrinsic bias and fairness issues towards historically marginalized populations.

Adversarial domain adaptation using contrastive learning

In recent research, problems with biased datasets or domain shift have presented challenges to the practical applications of deep learning methods. In this paper, we propose a simple method using adversarial learning combined with contrastive learning and domain adaptation to solve the domain-shift problem. Because domain shift is caused by differences in the distribution of data across domains, different approaches have been proposed to resolve this by rendering the data distributions closer through adversarial training. Contrastive learning is a method of acquiring valid feature representations for downstream tasks. By combining these approaches, we aim to achieve better domain adaptation. The proposed method is simple and intuitive. By introducing a domain discriminator into SimCLR, which is a typical contrastive learning model, and training it in an adversarial manner, the feature vectors of the source and target domains are rendered closer to acquire domain-invariant features. The proposed approach facilitates high-performance pretraining without labels and demonstrates a significant improvement in accuracy in comparison to standard benchmark methods, including conventional supervised models and SimCLR.

Malaria Cell Classification Using Transfer Learning

Abstract: In recent years, machine learning techniques, particularly transfer learning, have shown promise in automating malaria cell classification using digital images of blood smears. Transfer learning involves leveraging pre-trained neural networks to extract relevant features from large datasets and applying them to smaller, specialized datasets for improved performance. This abstract provides an overview of the concept of malaria cell classification using transfer learning. It highlights the advantages of using transfer learning, including reduced training time and improved classification accuracy, while addressing challenges such as dataset bias and model interpretability. Further research and development in this area could potentially contribute to the automation and scalability of malaria diagnosis, particularly in resource-limited settings. This study describes a method for classifying malaria cells using convolutional neural networks. It has been evaluated using a standard dataset obtained from the National Library of Medicine, which consists of 27,000microscopic pictures. The accuracy of the neural network has been improved while the total loss has been reduced using the Adam optimizer. Overfitting has been avoided by using dropout regression

Can Machine Learning Be Better than Biased Readers?

Background: Training machine learning (ML) models in medical imaging requires large amounts of labeled data. To minimize labeling workload, it is common to divide training data among multiple readers for separate annotation without consensus and then combine the labeled data for training a ML model. This can lead to a biased training dataset and poor ML algorithm prediction performance. The purpose of this study is to determine if ML algorithms can overcome biases caused by multiple readers' labeling without consensus. Methods: This study used a publicly available chest X-ray dataset of pediatric pneumonia. As an analogy to a practical dataset without labeling consensus among multiple readers, random and systematic errors were artificially added to the dataset to generate biased data for a binary-class classification task. The Resnet18-based convolutional neural network (CNN) was used as a baseline model. A Resnet18 model with a regularization term added as a loss function was utilized to examine for improvement in the baseline model. Results: The effects of false positive labels, false negative labels, and random errors (5-25%) resulted in a loss of AUC (0-14%) when training a binary CNN classifier. The model with a regularized loss function improved the AUC (75-84%) over that of the baseline model (65-79%). Conclusion: This study indicated that it is possible for ML algorithms to overcome individual readers' biases when consensus is not available. It is recommended to use regularized loss functions when allocating annotation tasks to multiple readers as they are easy to implement and effective in mitigating biased labels.

A case study of improving a non-technical losses detection system through explainability

Detecting and reacting to non-technical losses (NTL) is a fundamental activity that energy providers need to face in their daily routines. This is known to be challenging since the phenomenon of NTL is multi-factored, dynamic and extremely contextual, which makes artificial intelligence (AI) and, in particular, machine learning, natural areas to bring effective and tailored solutions. If the human factor is disregarded in the process of detecting NTL, there is a high risk of performance degradation since typical problems like dataset shift and biases cannot be easily identified by an algorithm. This paper presents a case study on incorporating explainable AI (XAI) in a mature NTL detection system that has been in production in the last years both in electricity and gas. The experience shows that incorporating this capability brings interesting improvements to the initial system and especially serves as a common ground where domain experts, data scientists, and business analysts can meet.

Automatic Essay Scoring Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses

Deep-learning based Automatic Essay Scoring (AES) systems are being actively used in various high-stake applications in education and testing. However, little research has been put to understand and interpret the black-box nature of deep-learning-based scoring algorithms. While previous studies indicate that scoring models can be easily fooled, in this paper, we explore the reason behind their surprising adversarial brittleness. We utilize recent advances in interpretability to find the extent to which features such as coherence, content, vocabulary, and relevance are important for automated scoring mechanisms. We use this to investigate the oversensitivity (i.e., large change in output score with a little change in input essay content) and overstability (i.e., little change in output scores with large changes in input essay content) of AES. Our results indicate that autoscoring models, despite getting trained as “end-to-end” models with rich contextual embeddings such as BERT, behave like bag-of-words models. A few words determine the essay score without the requirement of any context making the model largely overstable. This is in stark contrast to recent probing studies on pre-trained representation learning models, which show that rich linguistic features such as parts-of-speech and morphology are encoded by them. Further, we also find that the models have learnt dataset biases, making them oversensitive. The presence of a few words with high co-occurrence with a certain score class makes the model associate the essay sample with that score. This causes score changes in ∼95% of samples with an addition of only a few words. To deal with these issues, we propose detection-based protection models that can detect oversensitivity and samples causing overstability with high accuracies. We find that our proposed models are able to detect unusual attribution patterns and flag adversarial samples successfully.

Abstract 1922: PhyloFrame: A machine learning framework for ancestry agnostic disease signatures

Abstract Rapid advancements in genomic sequencing technologies have catalyzed the molecular identification and treatment of previously elusive diseases. Despite these genomic advancements, disparities in precision medicine access and data have resulted in disparate impacts on different socioeconomic groups. Many of the gold standard datasets largely exclude minority populations, compounding their exclusion from medical research. This has resulted in a restricted understanding of cancer and other complex diseases. We demonstrate the effects of ancestral bias in gold standard genomics datasets through ancestral analysis of cancer-related genes in the Cancer Gene Census, spanning 17 ancestral populations. Additionally, we present a machine learning framework, PhyloFrame, that incorporates population genomics data to correct for ancestral bias by creating disease signatures representative of all ancestries. Our ancestral analysis results show that while a majority of the current cancer-related genes in the Cancer Gene Census have below average mutation frequency in non-European populations, there are peaks of ancestrally enriched mutations in ancestry-specific genes related to cancer, which can be targeted using PhyloFrame. PhyloFrame prioritizes gene expression from the cancer genes with high frequency mutations in a given human population in order to capture genes driving disease in each ancestry. It builds on existing disease gene signatures and big-data functional interaction networks to identify ancestry-relevant genes related to a disease, outputing an ancestry-agnostic disease signature. We test PhyloFrame on TCGA cancers with diverse patient populations, such as breast cancer, and compare PhyloFrame's disease signature output to the disease signature output of elastic net runs on cancer samples from a single ancestry. Our results demonstrate that the incorporation of ancestral information allows PhyloFrame to recapitulate disease signatures trained on only one ancestry in a dataset with individuals from many unquantified ancestries. With the incorporation of ancestral information, PhyloFrame is able to create disease signatures with genes pertinent to each ancestral population, even when individuals from those populations are not included in the training data. This work offers a quick and cheap alternative to the mass sequencing that would be required to capture disease-driving genes in minority populations in hopes to contribute to equitable representation in medical research. Citation Format: Leslie A. Smith, James A. Cahill, Kiley Graim. PhyloFrame: A machine learning framework for ancestry agnostic disease signatures [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 1922.

Medical image data augmentation: techniques, comparisons and interpretations.

Designing deep learning based methods with medical images has always been an attractive area of research to assist clinicians in rapid examination and accurate diagnosis. Those methods need a large number of datasets including all variations in their training stages. On the other hand, medical images are always scarce due to several reasons, such as not enough patients for some diseases, patients do not want to allow their images to be used, lack of medical equipment or equipment, inability to obtain images that meet the desired criteria. This issue leads to bias in datasets, overfitting, and inaccurate results. Data augmentation is a common solution to overcome this issue and various augmentation techniques have been applied to different types of images in the literature. However, it is not clear which data augmentation technique provides more efficient results for which image type since different diseases are handled, different network architectures are used, and these architectures are trained and tested with different numbers of data sets in the literature. Therefore, in this work, the augmentation techniques used to improve performances of deep learning based diagnosis of the diseases in different organs (brain, lung, breast, and eye) from different imaging modalities (MR, CT, mammography, and fundoscopy) have been examined. Also, the most commonly used augmentation methods have been implemented, and their effectiveness in classifications with a deep network has been discussed based on quantitative performance evaluations. Experiments indicated that augmentation techniques should be chosen carefully according to image types.

Measurements, Algorithms, and Presentations of Reality: Framing Interactions with AI-Enabled Decision Support

Bringing AI technology into clinical practice has proved challenging for system designers and medical professionals alike. The academic literature has, for example, highlighted the dangers of black-box decision-making and biased datasets. Furthermore, end-users’ ability to validate a system’s performance often disappears following the introduction of AI decision-making. We present the MAP model to understand and describe the three stages through which medical observations are interpreted and handled by AI systems. These stages are Measurement, in which information is gathered and converted into data points that can be stored and processed; Algorithm, in which computational processes transform the collected data; and Presentation, where information is returned to the user for interpretation. For each stage, we highlight possible challenges that need to be overcome to develop Human-Centred AI systems. We illuminate our MAP model through complementary case studies on colonoscopy practice and dementia diagnosis, providing examples of the challenges encountered in real-world settings. By defining Human-AI interaction across these three stages, we untangle some of the inherent complexities in designing AI technology for clinical decision-making, and aim to overcome misalignment between medical end-users and AI researchers and developers.