Medical Question Answering Research Articles

PMC-LLaMA: toward building open-source language models for medicine.

Recently, large language models (LLMs) have showcased remarkable capabilities in natural language understanding. While demonstrating proficiency in everyday conversations and question-answering (QA) situations, these models frequently struggle in domains that require precision, such as medical applications, due to their lack of domain-specific knowledge. In this article, we describe the procedure for building a powerful, open-source language model specifically designed for medicine applications, termed as PMC-LLaMA. We adapt a general-purpose LLM toward the medical domain, involving data-centric knowledge injection through the integration of 4.8M biomedical academic papers and 30K medical textbooks, as well as comprehensive domain-specific instruction fine-tuning, encompassing medical QA, rationale for reasoning, and conversational dialogues with 202M tokens. While evaluating various public medical QA benchmarks and manual rating, our lightweight PMC-LLaMA, which consists of only 13B parameters, exhibits superior performance, even surpassing ChatGPT. All models, codes, and datasets for instruction tuning will be released to the research community. Our contributions are 3-fold: (1) we build up an open-source LLM toward the medical domain. We believe the proposed PMC-LLaMA model can promote further development of foundation models in medicine, serving as a medical trainable basic generative language backbone; (2) we conduct thorough ablation studies to demonstrate the effectiveness of each proposed component, demonstrating how different training data and model scales affect medical LLMs; (3) we contribute a large-scale, comprehensive dataset for instruction tuning. In this article, we systematically investigate the process of building up an open-source medical-specific LLM, PMC-LLaMA.

Medical education and artificial intelligence: Question answering for medical questions based on intelligent interaction

SummaryComputer assisted medical diagnosis technology is widely used in the field of medical assistance to assist doctors in making diagnostic decisions. But as the number of patients increases, the diagnostic pressure on doctors gradually increases, and more efficient computer‐aided medical diagnosis technology is needed to improve the accuracy of doctors' diagnosis. Today's computer‐based medically assisted diagnostic technologies suffer from the inability to fully simulate physical attributes and environmental factors, the large computational resources required for high‐precision models, the need for professional training for user operation, and the limited intuition for innovative design. For improving diagnostic efficiency, this study designs a medical Question answering intelligent interaction system in view of artificial intelligence algorithms. The system is constructed with an active interactive intelligent Q&A model consisting of a medical reasoning module and a medical examination recommendation module. Then, it uses local Bayesian network algorithm as the foundation to establish an intelligent strategy optimization network. And it puts forward the answer selection model of medical Question answering in view of hierarchical interaction for natural language processing tasks in the medical context. The performance test results show that when the diagnostic end threshold of the medical reasoning module is 0.8, the shortest diagnostic path is 3.33. When the diagnostic threshold is 0.85, the maximum length of the diagnostic path is 4.66, and the maximum difference between the diagnostic paths is 1.33, which is basically not affected by the diagnostic end threshold. The local Bayesian network algorithm can reduce the impact of noise features and extract more valuable information. The accuracy of the multilevel interactive answer selection model on the Stanford Natural Language Inference dataset without using external resources reached 89.2%. The ablation test results show that the overall accuracy of the model is 89.64%. The visualization results of the attention weight distribution test between interaction layers show that under different levels of interaction, the attention distribution will undergo significant changes.

Development and analysis of medical instruction-tuning for Japanese large language models

In the ongoing wave of impact driven by large language models (LLMs) like ChatGPT, the adaptation of LLMs to the medical domain has emerged as a crucial research frontier. Since mainstream LLMs tend to be designed for general-purpose applications, constructing a medical LLM through domain adaptation is a huge challenge. While instruction-tuning, particularly based on low-rank adaptation (LoRA), has become a frequently employed strategy to fine-tune LLMs recently, its precise roles in domain adaptation remain unknown. Here, we investigated how LoRA-based instruction-tuning improves the performance of Japanese medical question-answering tasks by employing a multifaceted evaluation of multiple-choice questions, including scoring based on “Exact match” and “Gestalt distance” in addition to the conventional accuracy. Our findings suggest that LoRA-based instruction-tuning can partially incorporate domain-specific knowledge into LLMs, with larger models demonstrating more pronounced effects. Furthermore, our results underscore the potential of adapting English-centric models for Japanese applications in domain adaptation, while also highlighting the persisting limitations of Japanese-centric models. This initiative represents a pioneering effort in enabling medical institutions to fine-tune and operate models without relying on external services.

Medical knowledge graph question answering for drug‐drug interaction prediction based on multi‐hop machine reading comprehension

AbstractDrug‐drug interaction (DDI) prediction is a crucial issue in molecular biology. Traditional methods of observing drug‐drug interactions through medical experiments require significant resources and labour. The authors present a Medical Knowledge Graph Question Answering (MedKGQA) model, dubbed MedKGQA, that predicts DDI by employing machine reading comprehension (MRC) from closed‐domain literature and constructing a knowledge graph of “drug‐protein” triplets from open‐domain documents. The model vectorises the drug‐protein target attributes in the graph using entity embeddings and establishes directed connections between drug and protein entities based on the metabolic interaction pathways of protein targets in the human body. This aligns multiple external knowledge and applies it to learn the graph neural network. Without bells and whistles, the proposed model achieved a 4.5% improvement in terms of DDI prediction accuracy compared to previous state‐of‐the‐art models on the QAngaroo MedHop dataset. Experimental results demonstrate the efficiency and effectiveness of the model and verify the feasibility of integrating external knowledge in MRC tasks.

MedChatZH: A tuning LLM for traditional Chinese medicine consultations

Generative Large Language Models (LLMs) have achieved significant success in various natural language processing tasks, including Question-Answering (QA) and dialogue systems. However, most models are trained on English data and lack strong generalization in providing answers in Chinese. This limitation is especially evident in specialized domains like traditional Chinese medical QA, where performance suffers due to the absence of fine-tuning and high-quality datasets. To address this, we introduce MedChatZH, a dialogue model optimized for Chinese medical QA based on transformer decoder with LLaMA architecture. Continued pre-training on a curated corpus of Chinese medical books is followed by fine-tuning with a carefully selected medical instruction dataset, resulting in MedChatZH outperforming several Chinese dialogue baselines on a real-world medical dialogue dataset. Our model, code, and dataset are publicly available on GitHub (https://github.com/tyang816/MedChatZH) to encourage further research in traditional Chinese medicine and LLMs.

Systematic analysis of ChatGPT, Google search and Llama 2 for clinical decision support tasks

It is likely that individuals are turning to Large Language Models (LLMs) to seek health advice, much like searching for diagnoses on Google. We evaluate clinical accuracy of GPT-3·5 and GPT-4 for suggesting initial diagnosis, examination steps and treatment of 110 medical cases across diverse clinical disciplines. Moreover, two model configurations of the Llama 2 open source LLMs are assessed in a sub-study. For benchmarking the diagnostic task, we conduct a naïve Google search for comparison. Overall, GPT-4 performed best with superior performances over GPT-3·5 considering diagnosis and examination and superior performance over Google for diagnosis. Except for treatment, better performance on frequent vs rare diseases is evident for all three approaches. The sub-study indicates slightly lower performances for Llama models. In conclusion, the commercial LLMs show growing potential for medical question answering in two successive major releases. However, some weaknesses underscore the need for robust and regulated AI models in health care. Open source LLMs can be a viable option to address specific needs regarding data privacy and transparency of training.

A Liver Cancer Question-Answering System Based on Next-Generation Intelligence and the Large Model Med-PaLM 2

Chronic diseases are the "number one killer" threatening human life and health. In recent years, the incidence of chronic diseases has been rising, and the trend is younger, and the prevention and control situation is very serious. This study introduces a Liver Cancer Question-Answering System (LCQAS) that leverages next-generation artificial intelligence and the large model Med-PaLM 2 The Med-PaLM 2 medical question answering system, powered by next-generation intelligence and large language models, represents a cutting-edge tool in the healthcare domain. Leveraging advanced artificial intelligence techniques and the capabilities of large language models like Med-PaLM 2, this system aims to provide accurate and comprehensive responses to medical inquiries and queries. With its intuitive interface and contextually relevant answers, LCQAS represents a significant advancement in medical information retrieval for liver cancer management.

The accuracy of AI-assisted chatbots on the annual assessment test for emergency medicine residents

BackgroundThe incorporation of natural language processing (NLP) models into medical education has accelerated with the introduction of ChatGPT. This study aimed to evaluate and compare the performance of ChatGPT-3.5, Bard and the residents in the annual assessment test for emergency medicine (EM) residents. MethodA total of 90 questions covering 18 distinct topics within the field of EM were asked to residents. The same questions were directed to ChatGPT-3.5 and Bard without excluding those with images. The percentage of correct answers were calculated and represented through a histogram showcasing the distribution of test scores across various bins. Questions were categorized as medical knowledge and clinical reasoning to further assess chatbots' performance based on question types. ResultsChatGPT-3.5 demonstrated a 60% accuracy, securing the 10th position, while Bard achieved an accuracy of 55.5%, placing 21st in the rankings among 46 residents. ChatGPT-3.5 performed better in the 16 subtopics. Bard outperformed ChatGPT-3.5 only on cardiovascular and pulmonary emergencies, accounting 24.4% of questions. Analysis of question types revealed ChatGPT-3.5's higher accuracy in medical knowledge (66%) compared to Bard (56%), while Bard performed better in clinical reasoning (55%) than ChatGPT-3.5 (52.5%). Despite lacking access to ECG images, both models answered several questions by providing text-based ECG interpretations. ConclusionChatGPT-3.5 and Bard demonstrated impressive performance in the task of medical question answering. On the other hand, issues related to ECG interpretations raises questions about the reliability of these models. Our findings highlight the importance of verifying the outputs generated by these models.

A Dual-Attention Learning Network With Word and Sentence Embedding for Medical Visual Question Answering.

Research in medical visual question answering (MVQA) can contribute to the development of computer-aided diagnosis. MVQA is a task that aims to predict accurate and convincing answers based on given medical images and associated natural language questions. This task requires extracting medical knowledge-rich feature content and making fine-grained understandings of them. Therefore, constructing an effective feature extraction and understanding scheme are keys to modeling. Existing MVQA question extraction schemes mainly focus on word information, ignoring medical information in the text, such as medical concepts and domain-specific terms. Meanwhile, some visual and textual feature understanding schemes cannot effectively capture the correlation between regions and keywords for reasonable visual reasoning. In this study, a dual-attention learning network with word and sentence embedding (DALNet-WSE) is proposed. We design a module, transformer with sentence embedding (TSE), to extract a double embedding representation of questions containing keywords and medical information. A dual-attention learning (DAL) module consisting of self-attention and guided attention is proposed to model intensive intramodal and intermodal interactions. With multiple DAL modules (DALs), learning visual and textual co-attention can increase the granularity of understanding and improve visual reasoning. Experimental results on the ImageCLEF 2019 VQA-MED (VQA-MED 2019) and VQA-RAD datasets demonstrate that our proposed method outperforms previous state-of-the-art methods. According to the ablation studies and Grad-CAM maps, DALNet-WSE can extract rich textual information and has strong visual reasoning ability.

Development and application of Chinese medical ontology for diabetes mellitus

ObjectiveTo develop a Chinese Diabetes Mellitus Ontology (CDMO) and explore methods for constructing high-quality Chinese biomedical ontologies.Materials and methodsWe used various data sources, including Chinese clinical practice guidelines, expert consensus, literature, and hospital information system database schema, to build the CDMO. We combined top-down and bottom-up strategies and integrated text mining and cross-lingual ontology mapping. The ontology was validated by clinical experts and ontology development tools, and its application was validated through clinical decision support and Chinese natural language medical question answering.ResultsThe current CDMO consists of 3,752 classes, 182 fine-grained object properties with hierarchical relationships, 108 annotation properties, and over 12,000 mappings to other well-known medical ontologies in English. Based on the CDMO and clinical practice guidelines, we developed 200 rules for diabetes diagnosis, treatment, diet, and medication recommendations using the Semantic Web Rule Language. By injecting ontology knowledge, CDMO enhances the performance of the T5 model on a real-world Chinese medical question answering dataset related to diabetes.ConclusionCDMO has fine-grained semantic relationships and extensive annotation information, providing a foundation for medical artificial intelligence applications in Chinese contexts, including the construction of medical knowledge graphs, clinical decision support systems, and automated medical question answering. Furthermore, the development process incorporated natural language processing and cross-lingual ontology mapping to improve the quality of the ontology and improved development efficiency. This workflow offers a methodological reference for the efficient development of other high-quality Chinese as well as non-English medical ontologies.

Developing ChatGPT for biology and medicine: a complete review of biomedical question answering.

ChatGPT explores a strategic blueprint of question answering (QA) to deliver medical diagnoses, treatment recommendations, and other healthcare support. This is achieved through the increasing incorporation of medical domain data via natural language processing (NLP) and multimodal paradigms. By transitioning the distribution of text, images, videos, and other modalities from the general domain to the medical domain, these techniques have accelerated the progress of medical domain question answering (MDQA). They bridge the gap between human natural language and sophisticated medical domain knowledge or expert-provided manual annotations, handling large-scale, diverse, unbalanced, or even unlabeled data analysis scenarios in medical contexts. Central to our focus is the utilization of language models and multimodal paradigms for medical question answering, aiming to guide the research community in selecting appropriate mechanisms for their specific medical research requirements. Specialized tasks such as unimodal-related question answering, reading comprehension, reasoning, diagnosis, relation extraction, probability modeling, and others, as well as multimodal-related tasks like vision question answering, image captioning, cross-modal retrieval, report summarization, and generation, are discussed in detail. Each section delves into the intricate specifics of the respective method under consideration. This paper highlights the structures and advancements of medical domain explorations against general domain methods, emphasizing their applications across different tasks and datasets. It also outlines current challenges and opportunities for future medical domain research, paving the way for continued innovation and application in this rapidly evolving field. This comprehensive review serves not only as an academic resource but also delineates the course for future probes and utilization in the field of medical question answering.

Counterfactual Causal-Effect Intervention for Interpretable Medical Visual Question Answering.

Medical Visual Question Answering (VQA-Med) is a challenging task that involves answering clinical questions related to medical images. However, most current VQA-Med methods ignore the causal correlation between specific lesion or abnormality features and answers, while also failing to provide accurate explanations for their decisions. To explore the interpretability of VQA-Med, this paper proposes a novel CCIS-MVQA model for VQA-Med based on a counterfactual causal-effect intervention strategy. This model consists of the modified ResNet for image feature extraction, a GloVe decoder for question feature extraction, a bilinear attention network for vision and language feature fusion, and an interpretability generator for producing the interpretability and prediction results. The proposed CCIS-MVQA introduces a layer-wise relevance propagation method to automatically generate counterfactual samples. Additionally, CCIS-MVQA applies counterfactual causal reasoning throughout the training phase to enhance interpretability and generalization. Extensive experiments on three benchmark datasets show that the proposed CCIS-MVQA model outperforms the state-of-the-art methods. Enough visualization results are produced to analyze the interpretability and performance of CCIS-MVQA.

Integration of Multi-Source Medical Data for Medical Diagnosis Question Answering

Integration of Multi-Source Medical Data for Medical Diagnosis Question Answering

Consistency Conditioned Memory Augmented Dynamic Diagnosis Model for Medical Visual Question Answering

Consistency Conditioned Memory Augmented Dynamic Diagnosis Model for Medical Visual Question Answering

Are my answers medically accurate? Exploiting medical knowledge graphs for medical question answering

Are my answers medically accurate? Exploiting medical knowledge graphs for medical question answering

BIR: Biomedical Information Retrieval System for Cancer Treatment in Electronic Health Record Using Transformers

The rapid growth of electronic health records (EHRs) has led to unprecedented biomedical data. Clinician access to the latest patient information can improve the quality of healthcare. However, clinicians have difficulty finding information quickly and easily due to the sheer data mining volume. Biomedical information retrieval (BIR) systems can help clinicians find the information required by automatically searching EHRs and returning relevant results. However, traditional BIR systems cannot understand the complex relationships between EHR entities. Transformers are a new type of neural network that is very effective for natural language processing (NLP) tasks. As a result, transformers are well suited for tasks such as machine translation and text summarization. In this paper, we propose a new BIR system for EHRs that uses transformers for predicting cancer treatment from EHR. Our system can understand the complex relationships between the different entities in an EHR, which allows it to return more relevant results to clinicians. We evaluated our system on a dataset of EHRs and found that it outperformed state-of-the-art BIR systems on various tasks, including medical question answering and information extraction. Our results show that Transformers are a promising approach for BIR in EHRs, reaching an accuracy and an F1-score of 86.46%, and 0.8157, respectively. We believe that our system can help clinicians find the information they need more quickly and easily, leading to improved patient care.

Asymmetric cross-modal attention network with multimodal augmented mixup for medical visual question answering

Insufficient training data is a common barrier to effectively learn multimodal information interactions and question semantics in existing medical Visual Question Answering (VQA) models. This paper proposes a new Asymmetric Cross Modal Attention network called ACMA, which constructs an image-guided attention and a question-guided attention to improve multimodal interactions from insufficient data. In addition, a Semantic Understanding Auxiliary (SUA) in the question-guided attention is newly designed to learn rich semantic embeddings for improving model performance on question understanding by integrating word-level and sentence-level information. Moreover, we propose a new data augmentation method called Multimodal Augmented Mixup (MAM) to train the ACMA, denoted as ACMA-MAM. The MAM incorporates various data augmentations and a vanilla mixup strategy to generate more non-repetitive data, which avoids time-consuming artificial data annotations and improves model generalization capability. Our ACMA-MAM outperforms state-of-the-art models on three publicly accessible medical VQA datasets (VQA-Rad, VQA-Slake, and PathVQA) with accuracies of 76.14 %, 83.13 %, and 53.83 % respectively, achieving improvements of 2.00 %, 1.32 %, and 1.59 % accordingly. Moreover, our model achieves F1 scores of 78.33 %, 82.83 %, and 51.86 %, surpassing the state-of-the-art models by 2.80 %, 1.15 %, and 1.37 % respectively.

Medical visual question answering with symmetric interaction attention and cross-modal gating

The purpose of medical visual question answering (Med-VQA) is to provide accurate answers to clinical questions related to visual content of medical images. However, previous attempts neglect to take full advantage of the information interaction between medical images and clinical questions, which hinders the further progress of Med-VQA. The above issue requires the efforts to focus on critical information interaction within each modality and relevant information interaction between modalities. In this paper, we utilize the multiple meta-model quantifying model as visual encoder and the GloVe word embedding followed by the LSTM as textual encoder to form our feature extraction module. Then, we design a symmetric interaction attention module to construct dense and deep intra- and inter-modal information interaction on medical images and clinical questions for the Med-VQA task. Specifically, the symmetric interaction attention module consists of multiple symmetric interaction attention blocks that contain two basic units, i.e., self-attention and interaction attention. Technically, self-attention is introduced for intra-modal information interaction, while interaction attention is constructed for inter-modal information interaction. In addition, we develop a multi-modal fusion scheme that leverages the cross-modal gating to effectively fuse multi-modal information and avoid redundant information after sufficient intra- and inter-modal information interaction. Experimental results on the VQA-RAD dataset and PathVQA dataset show that our method outperforms other state-of-the-art Med-VQA models, achieving 74.7% and 48.7% on accuracy, 73.5% and 46.0% on F1-score, respectively.

CHQ- SocioEmo: Identifying Social and Emotional Support Needs in Consumer-Health Questions

General public, often called consumers, are increasingly seeking health information online. To be satisfactory, answers to health-related questions often have to go beyond informational needs. Automated approaches to consumer health question answering should be able to recognize the need for social and emotional support. Recently, large scale datasets have addressed the issue of medical question answering and highlighted the challenges associated with question classification from the standpoint of informational needs. However, there is a lack of annotated datasets for the non-informational needs. We introduce a new dataset for non-informational support needs, called CHQ-SocioEmo. The Dataset of Consumer Health Questions was collected from a community question answering forum and annotated with basic emotions and social support needs. This is the first publicly available resource for understanding non-informational support needs in consumer health-related questions online. We benchmark the corpus against multiple state-of-the-art classification models to demonstrate the dataset’s effectiveness.

Medical Visual Question Answering via Conditional Reasoning and Contrastive Learning.

Medical visual question answering (Med-VQA) aims to accurately answer a clinical question presented with a medical image. Despite its enormous potential in healthcare services, the development of this technology is still in the initial stage. On the one hand, Med-VQA tasks are highly challenging due to the massive diversity of clinical questions that require different visual reasoning skills for different types of questions. On the other hand, medical images are complex in nature and very different from natural images, while current Med-VQA datasets are small-scale with a few hundred radiology images, making it difficult to train a well-performing visual feature extractor. This paper addresses above two critical issues. We propose a novel conditional reasoning mechanism with a question-conditioned reasoning component and a type-conditioned reasoning strategy to learn effective reasoning skills for different Med-VQA tasks adaptively. Further, we propose to pre-train a visual feature extractor for Med-VQA via contrastive learning on large amounts of unlabeled radiology images. The effectiveness of our proposals is validated by extensive experiments on existing Med-VQA benchmarks, which show significant improvement of our model in prediction accuracy over state-of-the-art methods. Our source code and pre-training dataset are released at https://github.com/Awenbocc/CPCR.