Dataset Bias Research Articles

BASE: a web service for providing compound-protein binding affinity prediction datasets with reduced similarity bias.

Deep learning-based drug-target affinity (DTA) prediction methods have shown impressive performance, despite a high number of training parameters relative to the available data. Previous studies have highlighted the presence of dataset bias by suggesting that models trained solely on protein or ligand structures may perform similarly to those trained on complex structures. However, these studies did not propose solutions and focused solely on analyzing complex structure-based models. Even when ligands are excluded, protein-only models trained on complex structures still incorporate some ligand information at the binding sites. Therefore, it is unclear whether binding affinity can be accurately predicted using only compound or protein features due to potential dataset bias. In this study, we expanded our analysis to comprehensive databases and investigated dataset bias through compound and protein feature-based methods using multilayer perceptron models. We assessed the impact of this bias on current prediction models and proposed the binding affinity similarity explorer (BASE) web service, which provides bias-reduced datasets. By analyzing eight binding affinity databases using multilayer perceptron models, we confirmed a bias where the compound-protein binding affinity can be accurately predicted using compound features alone. This bias arises because most compounds show consistent binding affinities due to high sequence or functional similarity among their target proteins. Our Uniform Manifold Approximation and Projection analysis based on compound fingerprints further revealed that low and high variation compounds do not exhibit significant structural differences. This suggests that the primary factor driving the consistent binding affinities is protein similarity rather than compound structure. We addressed this bias by creating datasets with progressively reduced protein similarity between the training and test sets, observing significant changes in model performance. We developed the BASE web service to allow researchers to download and utilize these datasets. Feature importance analysis revealed that previous models heavily relied on protein features. However, using bias-reduced datasets increased the importance of compound and interaction features, enabling a more balanced extraction of key features. We propose the BASE web service, providing both the affinity prediction results of existing models and bias-reduced datasets. These resources contribute to the development of generalized and robust predictive models, enhancing the accuracy and reliability of DTA predictions in the drug discovery process. BASE is freely available online at https://synbi2024.kaist.ac.kr/base .

PhyKIT: A Multitool for Phylogenomics.

Multiple sequence alignments and phylogenetic trees are rich in biological information and are fundamental to research in biology. PhyKIT is a tool for processing and analyzing the information content of multiple sequence alignments and phylogenetic trees. Here, we describe how to use PhyKIT for diverse analyses, including (i) constructing a phylogenomic supermatrix, (ii) detecting errors in orthology inference, (iii) quantifying biases in phylogenomic data sets, (iv) identifying radiation events or lack of resolution using gene support frequencies, and (v) conducting evolution-based screens to facilitate gene function prediction. Several PhyKIT functions that streamline multiple sequence alignment and phylogenetic processing-such as renaming FASTA entries or tree tips-are also discussed. These protocols demonstrate how simple command-line operations in the unified framework of PhyKIT facilitate diverse phylogenomic data analysis and processing, from supermatrix construction and diagnosis to gaining clues about gene function. © 2024 The Author(s). Current Protocols published by Wiley Periodicals LLC. Basic Protocol 1: Installing PhyKIT and syntax for usage Basic Protocol 2: Constructing a phylogenomic supermatrix Basic Protocol 3: Detecting anomalies in orthology relationships Basic Protocol 4: Quantifying biases in phylogenomic data matrices and related measures Basic Protocol 5: Identifying polytomies Basic Protocol 6: Assessing gene-gene coevolution as a genetic screen.

G4 & the balanced metric family – a novel approach to solving binary classification problems in medical device validation & verification studies

BackgroundIn medical device validation and verification studies, the area under the receiver operating characteristic curve (AUROC) is often used as a primary endpoint despite multiple reports showing its limitations. Hence, researchers are encouraged to consider alternative metrics as primary endpoints. A new metric called G4 is presented, which is the geometric mean of sensitivity, specificity, the positive predictive value, and the negative predictive value. G4 is part of a balanced metric family which includes the Unified Performance Measure (also known as P4) and the Matthews’ Correlation Coefficient (MCC). The purpose of this manuscript is to unveil the benefits of using G4 together with the balanced metric family when analyzing the overall performance of binary classifiers.ResultsSimulated datasets encompassing different prevalence rates of the minority class were analyzed under a multi-reader-multi-case study design. In addition, data from an independently published study that tested the performance of a unique ultrasound artificial intelligence algorithm in the context of breast cancer detection was also considered. Within each dataset, AUROC was reported alongside the balanced metric family for comparison. When the dataset prevalence and bias of the minority class approached 50%, all three balanced metrics provided equivalent interpretations of an AI’s performance. As the prevalence rate increased / decreased and the data became more imbalanced, AUROC tended to overvalue / undervalue the true classifier performance, while the balanced metric family was resistant to such imbalance. Under certain circumstances where data imbalance was strong (minority-class prevalence < 10%), MCC was preferred for standalone assessments while P4 provided a stronger effect size when evaluating between-groups analyses. G4 acted as a middle ground for maximizing both standalone assessments and between-groups analyses.ConclusionsUse of AUROC as the primary endpoint in binary classification problems provides misleading results as the dataset becomes more imbalanced. This is explicitly noticed when incorporating AUROC in medical device validation and verification studies. G4, P4, and MCC do not share this limitation and paint a more complete picture of a medical device’s performance in a clinical setting. Therefore, researchers are encouraged to explore the balanced metric family when evaluating binary classification problems.

The role of artificial intelligence in higher medical education and the ethical challenges of its implementation

Artificial intelligence (AI) is penetrating higher medical education; however, its adoption remains low. A PRISMA-S search of the Web of Science database from 2020 to 2024, utilizing the search terms &amp;ldquo;artificial intelligence,&amp;rdquo; &amp;ldquo;medicine,&amp;rdquo; &amp;ldquo;education,&amp;rdquo; and &amp;ldquo;ethics,&amp;rdquo; reveals this trend. Four key areas of AI application in medical education are examined for their potential benefits: Educational support (such as personalized distance education), radiology (diagnostics), virtual reality (VR) (visualization and simulations), and generative text engines (GenText), such as ChatGPT (from the production of notes to syllabus design). However, significant ethical risks accompany AI adoption, and specific concerns are linked to each of these four areas. While AI is recognized as an important support tool in medical education, its slow integration hampers learning and diminishes student motivation, as evidenced by the challenges in implementing VR. In radiology, data-intensive training is hindered by poor connectivity, particularly affecting learners in developing countries. Ethical risks, such as bias in datasets (whether intentional or unintentional), need to be highlighted within educational programs. Students must be informed of the possible motivation behind the introduction of social and political bias in datasets, as well as the profit motive. Finally, the ethical risks accompanying the use of GenText are discussed, ranging from student reliance on instant text generation for assignments, which can hinder the development of critical thinking skills, to the potential danger of relying on AI-generated learning and treatment plans without sufficient human moderation.

GPT-Driven Radiology Report Generation with Fine-Tuned Llama 3.

The integration of deep learning into radiology has the potential to enhance diagnostic processes, yet its acceptance in clinical practice remains limited due to various challenges. This study aimed to develop and evaluate a fine-tuned large language model (LLM), based on Llama 3-8B, to automate the generation of accurate and concise conclusions in magnetic resonance imaging (MRI) and computed tomography (CT) radiology reports, thereby assisting radiologists and improving reporting efficiency. A dataset comprising 15,000 radiology reports was collected from the University of Medicine and Pharmacy of Craiova's Imaging Center, covering a diverse range of MRI and CT examinations made by four experienced radiologists. The Llama 3-8B model was fine-tuned using transfer-learning techniques, incorporating parameter quantization to 4-bit precision and low-rank adaptation (LoRA) with a rank of 16 to optimize computational efficiency on consumer-grade GPUs. The model was trained over five epochs using an NVIDIA RTX 3090 GPU, with intermediary checkpoints saved for monitoring. Performance was evaluated quantitatively using Bidirectional Encoder Representations from Transformers Score (BERTScore), Recall-Oriented Understudy for Gisting Evaluation (ROUGE), Bilingual Evaluation Understudy (BLEU), and Metric for Evaluation of Translation with Explicit Ordering (METEOR) metrics on a held-out test set. Additionally, a qualitative assessment was conducted, involving 13 independent radiologists who participated in a Turing-like test and provided ratings for the AI-generated conclusions. The fine-tuned model demonstrated strong quantitative performance, achieving a BERTScore F1 of 0.8054, a ROUGE-1 F1 of 0.4998, a ROUGE-L F1 of 0.4628, and a METEOR score of 0.4282. In the human evaluation, the artificial intelligence (AI)-generated conclusions were preferred over human-written ones in approximately 21.8% of cases, indicating that the model's outputs were competitive with those of experienced radiologists. The average rating of the AI-generated conclusions was 3.65 out of 5, reflecting a generally favorable assessment. Notably, the model maintained its consistency across various types of reports and demonstrated the ability to generalize to unseen data. The fine-tuned Llama 3-8B model effectively generates accurate and coherent conclusions for MRI and CT radiology reports. By automating the conclusion-writing process, this approach can assist radiologists in reducing their workload and enhancing report consistency, potentially addressing some barriers to the adoption of deep learning in clinical practice. The positive evaluations from independent radiologists underscore the model's potential utility. While the model demonstrated strong performance, limitations such as dataset bias, limited sample diversity, a lack of clinical judgment, and the need for large computational resources require further refinement and real-world validation. Future work should explore the integration of such models into clinical workflows, address ethical and legal considerations, and extend this approach to generate complete radiology reports.

Less can be more: representational vs. stereotypical gender bias in facial expression recognition

AbstractMachine learning models can inherit biases from their training data, leading to discriminatory or inaccurate predictions. This is particularly concerning with the increasing use of large, unsupervised datasets for training foundational models. Traditionally, demographic biases within these datasets have not been well-understood, limiting our ability to understand how they propagate to the models themselves. To address this issue, this paper investigates the propagation of demographic biases from datasets into machine learning models. We focus on the gender demographic component, analyzing two types of bias: representational and stereotypical. For our analysis, we consider the domain of facial expression recognition (FER), a field known to exhibit biases in most popular datasets. We use Affectnet, one of the largest FER datasets, as our baseline for carefully designing and generating subsets that incorporate varying strengths of both representational and stereotypical bias. Subsequently, we train several models on these biased subsets, evaluating their performance on a common test set to assess the propagation of bias into the models’ predictions. Our results show that representational bias has a weaker impact than expected. Models exhibit a good generalization ability even in the absence of one gender in the training dataset. Conversely, stereotypical bias has a significantly stronger impact, primarily concentrated on the biased class, although it can also influence predictions for unbiased classes. These results highlight the need for a bias analysis that differentiates between types of bias, which is crucial for the development of effective bias mitigation strategies.

News Ninja: Gamified Annotation of Linguistic Bias in Online News

Recent research shows that visualizing linguistic bias mitigates its negative effects. However, reliable automatic detection methods to generate such visualizations require costly, knowledge-intensive training data. To facilitate data collection for media bias datasets, we present News Ninja, a game employing data-collecting game mechanics to generate a crowdsourced dataset. Before annotating sentences, players are educated on media bias via a tutorial. Our findings show that datasets gathered with crowdsourced workers trained on News Ninja can reach significantly higher inter-annotator agreements than expert and crowdsourced datasets with similar data quality. As News Ninja encourages continuous play, it allows datasets to adapt to the reception and contextualization of news over time, presenting a promising strategy to reduce data collection expenses, educate players, and promote long-term bias mitigation.

Artificial Intelligence &amp; the Capacity for Discrimination: The Imperative Need for Frameworks, Diverse Teams &amp; Human Accountability

The increasing integration of Artificial Intelligence (AI) in various industries has led to concerns about how these systems can perpetuate discrimination, particularly in fields like employment, healthcare, and public policy. Multiple academic and business perspectives on AI discrimination, focusing on the need for global policy coordination and ethical oversight to mitigate biased outcomes, ask for our technical innovators to create contingencies that will better protect humanity’s experience with AI’s ever-expanding reach. Central to the key constructs such as biased datasets, algorithmic transparency, and the global governance of AI systems can function as a harmful drawback to these systems. Without adequate data governance and transparency, AI systems can perpetuate discrimination. AI's ability to discriminate stems primarily from biased data and the opacity of machine learning models, necessitating proactive research and policy implementation on a global scale. These frameworks must transcend the limitations of the experiences or perspectives of their programmers to ensure that AI innovations are ethically sound and that their use in global organizations adheres to principles of fairness and accountability. This synthesis will explore how these articles advocate for comprehensive, continuous monitoring of AI systems and policies that address both local and international concerns, offering a roadmap for organizations to innovate responsibly while mitigating the risks of AI-driven discrimination.

Correcting for Observation Bias in Cancer Progression Modeling.

Tumor progression is driven by the accumulation of genetic alterations, including both point mutations and copy number changes. Understanding the temporal sequence of these events is crucial for comprehending the disease but is not directly discernible from cross-sectional genomic data. Cancer progression models, including Mutual Hazard Networks (MHNs), aim to reconstruct the dynamics of tumor progression by learning the causal interactions between genetic events based on their co-occurrence patterns in cross-sectional data. Here, we highlight a commonly overlooked bias in cross-sectional datasets that can distort progression modeling. Tumors become clinically detectable when they cause symptoms or are identified through imaging or tests. Detection factors, such as size, inflammation (fever, fatigue), and elevated biochemical markers, are influenced by genomic alterations. Ignoring these effects leads to "conditioning on a collider" bias, where events making the tumor more observable appear anticorrelated, creating false suppressive effects or masking promoting effects among genetic events. We enhance MHNs by incorporating the effects of genetic progression events on the inclusion of a tumor in a dataset, thus correcting for collider bias. We derive an efficient tensor formula for the likelihood function and apply it to two datasets from the MSK-IMPACT study. In colon adenocarcinoma, we observe a significantly higher rate of clinical detection for TP53-positive tumors, while in lung adenocarcinoma, the same is true for EGFR-positive tumors. Compared to classical MHNs, this approach eliminates several spurious suppressive interactions and uncovers multiple promoting effects.

Massive experimental quantification of amyloid nucleation allows interpretable deep learning of protein aggregation.

Protein aggregation is a pathological hallmark of more than fifty human diseases and a major problem for biotechnology. Methods have been proposed to predict aggregation from sequence, but these have been trained and evaluated on small and biased experimental datasets. Here we directly address this data shortage by experimentally quantifying the amyloid nucleation of >100,000 protein sequences. This unprecedented dataset reveals the limited performance of existing computational methods and allows us to train CANYA, a convolution-attention hybrid neural network that accurately predicts amyloid nucleation from sequence. We adapt genomic neural network interpretability analyses to reveal CANYA's decision-making process and learned grammar. Our results illustrate the power of massive experimental analysis of random sequence-spaces and provide an interpretable and robust neural network model to predict amyloid nucleation.

Multimodal biometric integration: Trends and insights from the past quinquennial

Recent years have seen a notable breakthrough in multimodal biometric systems, which combine numerous biometric modalities for increased security and accuracy. This paper offers a thorough analysis of recent advances in the field, addressing the many multimodal biometric strategies that have been put out by different researchers. Numerous modalities are covered by the evaluation, such as palm print, speech, iris, facial features, palm print, fingerprint, body form, gait, and more. To obtain amazing results in terms of accuracy and security, researchers have used cutting-edge methods like convolutional neural networks (CNNs), support vector machines (SVM), recurrent neural networks (RNNs), deep learning, and optimization algorithms. Many fusion strategies have been investigated to efficiently merge data from many modalities, such as feature-level fusion, decision-level fusion, and score-level fusion. Developments in template protection systems have also addressed security issues related to transmission and storage of biometric data. Even though a lot of the suggested systems have shown great accuracy rates, there are still issues with hardware restrictions, dataset biases, privacy problems, and computational costs. Prospective study avenues encompass investigating more extensive and heterogeneous datasets, crafting resilient fusion algorithms, and amalgamating cutting-edge technologies like deep learning-based biometric cryptosystems and federated learning. All things considered, the literature study demonstrates the enormous potential of multimodal biometric systems in offering safe and dependable authentication solutions for a wide range of applications, from on-line student authentication and proctoring systems to customized healthcare networks.

Overcoming Dataset Bias: A Deep Learning Approach to Improve Cross-Cultural Image Classification

The difficulties and strategies concerning cross-ethnic image categorization are discussed throughout this report based on deep learning techniques for preventing model bias towards specific ethnic groups. In this part of the project, we continued our discussion of the theory and practice of bias in training datasets and how it impacts the model in the case of emotion recognition and facial analysis using simulations. Three real-time scenarios were explored: public security applications, health status evaluations, education performance, and awareness, all of which must include culturally sensitive data to maximize performance. Some of the issues we defined include the data bias problem, the problem that different cultures exhibit different levels of emotions, and technical concerns such as the limitations of machine learning algorithms, among others that we have described in the paper. Some solutions we provided include data augmentation, real-time learning, and ethical concerns such as using machine learning in making decisions. In doing so, we propose solutions that include building models that are not only precise but also culturally sensitive to avoid prejudice in applications, ranging from credit scoring to complaints handling.

OligoFormer: an accurate and robust prediction method for siRNA design.

RNA interference (RNAi) has become a widely used experimental approach for post-transcriptional regulation and is increasingly showing its potential as future targeted drugs. However, the prediction of highly efficient siRNAs (small interfering RNAs) is still hindered by dataset biases, the inadequacy of prediction methods, and the presence of off-target effects. To overcome these limitations, we propose an accurate and robust prediction method, OligoFormer, for siRNA design. OligoFormer comprises three different modules including thermodynamic calculation, RNA-FM module, and Oligo encoder. Oligo encoder is the core module based on the transformer encoder. Taking siRNA and mRNA sequences as input, OligoFormer can obtain thermodynamic parameters, RNA-FM embedding, and Oligo embedding through these three modules, respectively. We carefully benchmarked OligoFormer against six comparable methods on siRNA efficacy datasets. OligoFormer outperforms all the other methods, with an average improvement of 9% in AUC, 6.6% in PRC, 9.8% in F1 score, and 5.1% in PCC compared to the best method among them in our inter-dataset validation. We also provide a comprehensive pipeline with prediction of siRNA efficacy and off-target effects using PITA score and TargetScan score. The ablation study shows RNA-FM module and thermodynamic parameters improved the performance and accelerated convergence of OligoFormer. The saliency maps by gradient backpropagation and base preference maps show certain base preferences in initial and terminal region of siRNAs. The source code of OligoFormer is freely available on GitHub at: https://github.com/lulab/OligoFormer. Docker image of OligoFormer is freely available on the docker hub at https://hub.docker.com/r/yilanbai/oligoformer.

Linking transcriptome and morphology in bone cells at cellular resolution with generative AI

Abstract Recent advancements in deep learning (DL) have revolutionized the capability of artificial intelligence (AI) by enabling the analysis of large-scale, complex datasets that are difficult for humans to interpret. However, large amounts of high-quality data are required to train such generative AI models successfully. With the rapid commercialization of single-cell sequencing and spatial transcriptomics platforms, the field is increasingly producing large-scale datasets such as histological images, single-cell molecular data, and spatial transcriptomic data. These molecular and morphological datasets parallel the multimodal text and image data used to train highly successful generative AI models for natural language processing and computer vision. Thus, these emerging data types offer great potential to train generative AI models that uncover intricate biological processes of bone cells at a cellular level. In this Perspective, we summarize the progress and prospects of generative AI applied to these datasets and their potential applications to bone research. In particular, we highlight three AI applications: predicting cell differentiation dynamics, linking molecular and morphological features, and predicting cellular responses to perturbations. To make generative AI models beneficial for bone research, important issues, such as technical biases in bone single-cell datasets, lack of profiling of important bone cell types, and lack of spatial information, need to be addressed. Realizing the potential of generative AI for bone biology will also likely require generating large-scale, high-quality cellular-resolution spatial transcriptomics datasets, improving the sensitivity of current spatial transcriptomics datasets, and thorough experimental validation of model predictions.

Ethical Considerations and Challenges in the Integration of Artificial Intelligence in Education: A Systematic Review

This systematic review examines those challenges in light of data privacy, algorithmic bias, ethical implications, technological hurdles, and acceptance of AI by educators and students. First, data privacy should be a primary concern, as AI systems require extensive data, bringing up the potential for breach and misuse. Secondly, there must be a robust mechanism concerning data protection and against the application of GDPR. Another critical point is algorithm bias: biased training data sets may lead to discriminative decisions that will increase inequalities in education. It talks about AI's impact on teachers and classroom dynamics because the takeover of responsibilities may lower the intensity of necessary human contact. From a technical perspective, there is so much infrastructure and expertise required that too many educational institutions lack, especially in developing countries. In addition, educators themselves may feel that the change resists and fears job loss and therefore acts as a deterrent to AI integration. The review underscores the imperative for extensive training of teachers to support enabling the integration of AI. It now demands a collaborative effort on the part of all stakeholders to maximize the gains and reduce the drawbacks of AI in educational aspects. Continuous research in, policy-making for, and ethical guidelines on AI are required to benefit all aspects of education equitably and effectively.

Tracking aquatic non-native macroinvertebrate species in Germany using long-term data

Biological invasions pose a global challenge, threatening both biodiversity and human well-being. Projections suggest that as invasions increase, the financial costs associated with management and the ecological harm they cause will also escalate. Here, we examined whether long-term biomonitoring strategies were adequate to identify and track benthic aquatic non-native macroinvertebrate species by using the German subset (151 time series; 129 of which reported non-native species) of the currently most comprehensive European long-term dataset of 1816 macroinvertebrate community time series from 22 European countries. The detection of aquatic non-native species was directly linked to the availability of long-term sites and thus, monitoring effort, having identified the spatio-temporal occurrence of 32 non-native species. The available long-term monitoring site data were mostly concentrated in the western part of Germany, predominantly covering the Rhine River and its tributaries. The spatially biased network of long-term monitoring sites, therefore, naturally skews the detection and reporting of aquatic non-native species toward this area and underestimates Eastern and Southern regions, impeding the comprehension of invasion dynamics. However, based on the available data, we found that the absolute number of non-native species increased and the proportion of non-native species relative to native species decreased over time. This indicates complex ecological interactions between native and non-native species and underlines the value of long-term data for investigating invasion dynamics. Considering the value of comprehensive monitoring networks, a spatially biased network delays the application of management and mitigation plans, possibly worsening the ecological and economic effects of biological invasions in Germany. The results provided here indicate the disadvantages of biased datasets, but simultaneously underline the enormous potential of a dense network of long-term monitoring. Our results also highlight the urgent need to increase and diversify long-term biomonitoring efforts throughout Germany to cover the main freshwater resources and their connections where the introduction risk of non-native species is the highest. Centrally collating such data would provide a profound basis for the monitoring of spreading aquatic non-native species and could serve the implementation of national biosecurity efforts.

P068 IDENTIFYING THRESHOLD BIAS IN RECORDED BLOOD PRESSURE

Background and Objective: Even when blood pressure (BP) is measured accurately, BP may be misrecorded. “Threshold bias” occurs when BP measurements are intentionally or unintentionally recorded at values just below BP control goals and may occur due to incentives to improve BP control or to avoid medication titration. Our objectives were to 1) examine the potential for threshold bias in datasets with and without a BP control goal from two countries and 2) propose a method for detecting threshold bias. Methods: We examined the distribution of recorded BP measurements among individuals taking antihypertensive medication in clinical practice or trials with a BP goal &lt;140/90 mmHg and population-based surveillance studies without a BP goal from Bangladesh and the US. Then, we explored whether surveillance data could be used as a reference to evaluate the likelihood of threshold bias in clinical data. We calculated the ratio of BP measurements recorded 10 mmHg below versus above thresholds of interest (i.e., the “threshold ratio,” or number of systolic BP [SBP] measurements 130-139 mmHg over 140-149 mmHg) and used bootstrapping to determine the probability of observing ratios that may indicate threshold bias. We compared the “threshold ratio” from clinical data to the probabilities determined from surveillance data. Results: Visually, we observed a cluster of measurements below SBP 140 mmHg in the clinical but not the surveillance datasets from Bangladesh and the US (Figure). Based on 10,000 random draws of 100 observations in the Bangladesh surveillance dataset, the probability of observing a “threshold ratio” of 3.56 in the clinical dataset was &lt;2%, indicating a high probability of threshold bias. Using the same procedure in the US surveillance dataset, the probability of observing a “threshold ratio” of 2.26 in the clinical dataset was &lt;10%, indicating potential concern. Conclusions: Threshold bias may be an underrecognized but common problem in clinical settings with a BP goal. Our proposed approach to detecting threshold bias should be tested in other settings.

Democratising artificial intelligence in healthcare: community-driven approaches for ethical solutions

The rapid advancement and widespread adoption of artificial intelligence (AI) has ushered in a new era of possibilities in healthcare, ranging from clinical task automation to disease detection. AI algorithms have the potential to analyse medical data, enhance diagnostic accuracy, personalise treatment plans and predict patient outcomes among other possibilities. With a surge in AI's popularity, its developments are outpacing policy and regulatory frameworks, leading to concerns about ethical considerations and collaborative development. Healthcare faces its own ethical challenges, including biased datasets, under-representation and inequitable access to resources, all contributing to mistrust in medical systems. To address these issues in the context of AI healthcare solutions and prevent perpetuating existing inequities, it is crucial to involve communities and stakeholders in the AI lifecycle.This article discusses four community-driven approaches for co-developing ethical AI healthcare solutions, including understanding and prioritising needs, defining a shared language, promoting mutual learning and co-creation, and democratising AI. These approaches emphasise bottom-up decision-making to reflect and centre impacted communities' needs and values. These collaborative approaches provide actionable considerations for creating equitable AI solutions in healthcare, fostering a more just and effective healthcare system that serves patient and community needs.

Acquisition parameters influence AI recognition of race in chest x-rays and mitigating these factors reduces underdiagnosis bias

A core motivation for the use of artificial intelligence (AI) in medicine is to reduce existing healthcare disparities. Yet, recent studies have demonstrated two distinct findings: (1) AI models can show performance biases in underserved populations, and (2) these same models can be directly trained to recognize patient demographics, such as predicting self-reported race from medical images alone. Here, we investigate how these findings may be related, with an end goal of reducing a previously identified underdiagnosis bias. Using two popular chest x-ray datasets, we first demonstrate that technical parameters related to image acquisition and processing influence AI models trained to predict patient race, where these results partly reflect underlying biases in the original clinical datasets. We then find that mitigating the observed differences through a demographics-independent calibration strategy reduces the previously identified bias. While many factors likely contribute to AI bias and demographics prediction, these results highlight the importance of carefully considering data acquisition and processing parameters in AI development and healthcare equity more broadly.

M[formula omitted]ixup: A multi-modal data augmentation approach for image captioning

Despite the great success, most models in image captioning (IC) are still stuck in the dilemma of generating simple and non-discriminative captions. In this paper, we study this problem from the perspective of data augmentation and propose a novel method called Multi-modal Mixup (M3ixup). Compared with the original Mixup strategy designed for image classification, the proposed M3ixup has three novel designs to mix IC samples from the aspects of visual features, sentence embeddings and loss values, respectively. In practice, M3ixup can not only enrich the diversity of IC training data, but also enforce the model to focus more on visual information for captioning, thereby alleviating the negative effect of dataset bias and addressing the issue of simple captioning. To validate M3ixup, we apply it to three baseline models and conduct extensive experiments on MS COCO. The experimental results demonstrate that our proposed M3ixup can not only improve the discriminability and quality of generated captions, but also help the baseline models obtain obvious performance gains, i.e., improving the CIDEr scores of the state-of-the-art model from 133.8 to 135.3 on off-line testing and 135.4 to 137.1 on online testing.