Medical Chatbot Research Articles

Systematic Analysis of Retrieval-Augmented Generation-Based LLMs for Medical Chatbot Applications

Artificial Intelligence (AI) has the potential to revolutionise the medical and healthcare sectors. AI and related technologies could significantly address some supply-and-demand challenges in the healthcare system, such as medical AI assistants, chatbots and robots. This paper focuses on tailoring LLMs to medical data utilising a Retrieval-Augmented Generation (RAG) database to evaluate their performance in a computationally resource-constrained environment. Existing studies primarily focus on fine-tuning LLMs on medical data, but this paper combines RAG and fine-tuned models and compares them against base models using RAG or only fine-tuning. Open-source LLMs (Flan-T5-Large, LLaMA-2-7B, and Mistral-7B) are fine-tuned using the medical datasets Meadow-MedQA and MedMCQA. Experiments are reported for response generation and multiple-choice question answering. The latter uses two distinct methodologies: Type A, as standard question answering via direct choice selection; and Type B, as language generation and probability confidence score generation of choices available. Results in the medical domain revealed that Fine-tuning and RAG are crucial for improved performance, and that methodology Type A outperforms Type B.

Enhancing Clinical Accuracy of Medical Chatbots with Large Language Models.

The rapid advancement of large language models (LLMs) has opened up new possibilities for transforming healthcare practices, patient interactions, and medical report generation. This paper explores the application of LLMs in developing medical chatbots and virtual assistants that prioritize clinical accuracy. We propose a novel multi-turn dialogue model, including adjusting the position of layer normalization to improve training stability and convergence, employing a contextual sliding window reply prediction task to capture fine-grained local context, and developing a local critical information distillation mechanism to extract and emphasize the most relevant information. These components are integrated into a multi-turn dialogue model that generates coherent and clinically accurate responses. Experiments on the MIMIC-III and n2c2 datasets demonstrate the superiority of the proposed model over state-of-the-art baselines, achieving significant improvements in perplexity, BLEU-2, recall at K scores, medical entity recognition, and response coherence. The proposed model represents a significant step in developing reliable and contextually relevant multi-turn medical dialogue systems that can assist patients and healthcare professionals.

FP5.8 - Bridging the Trust Divide: How Medical AI Outperforms yet Battles to Win Public Confidence - A Study Conducted Among 74 UK Residents

Abstract Background AI is being incorporated into healthcare by major tech companies, but public acceptance remains challenging. The study aims to understand resistance to AI despite its increasing accuracy and potential to improve patient waiting times. Method In partnership with the University of Hull and the Academic Health Science Network (AHSN) in the UK, a study was conducted that involved 111 adult patients or carers. 9 did not respond while 28 demonstrated poor digital literacy and therefore excluded. An interactive user page was designed for the remaining users (n=74) to engage with medical chatbot. The participants were asked whether participants would use an AI-powered medical chatbot or nurse for triage at their nearest A&E. Unstructured open-ended interviews were conducted to understand the participants' reasoning behind their answers. Finding Participants ranged in age from 21 to 74, with a slight female majority (40 female vs 34 male). The majority, 79.8% (n=59) respondents, expressed their comfort with a nurse-led care while a mere 20.2% (n=15) showed readiness to interact with a medical chatbot. Notably, the age group of 20-40 was most open to the idea of consulting a chatbot. The four primary objections were: 90.5% (n = 67) believed that the chatbot fails to justify its decisions, 59.5% (n = 44) doubt its accuracy, 78.4% (n = 58) felt that the chatbot was inflexible, and 79.7% (n = 59) found it unemotional and detached. Conclusion Effective treatment relies on trust, respect and understanding, crucial elements needed when incorporating AI in medicine. It should be introduced progressively, considering patients' emotions and societal circumstances.

Smart Voice Based Medical Chatbot for Disease Prediction

The global shortage of medical resources has intensified in recent years, exacerbated by a growing and aging population, particularly following the pandemic. However, advancements in robotics and artificial intelligence have empowered healthcare professionals to address these challenges. Intelligent Speech Technology (IST) is one such innovation that has helped doctors and patients enhance medical efficiency and reduce cumbersome tasks. Despite this, complex hospital environments with background noise and variations in patient pronunciation have limited IST's broader application. Yet, as machine learning technologies rapidly advance intelligent speech recognition capabilities, these obstacles are expected to be overcome in the near future. This project focuses on the concept of an AI-based voice- assisted chatbot for medical disease prediction. Deployed on a Raspberry Pi as a standalone hardware device, it utilizes natural language processing (NLP) and machine learning (ML) techniques to detect diseases by interacting with users through a series of questions. A groundbreaking innovation, the Speech-Assisted Chatbot for Medical Infectious Disease Prediction, promises to transform healthcare delivery. By leveraging AI-driven algorithms and natural language processing, it facilitates quick diagnosis and prescription generation through spoken interactions with patients. This simplifies and accelerates procedures, reduces error risks, improves patient care standards, saves valuable time for healthcare professionals, and enhances service availability, all contributing to better outcomes and more efficient healthcare processes.. Key Words: chatbot, NLP, Machine learning, Medical diagnosis, Speech, Artificial Intelligence etc.

Network-integrated medical chatbot for enhanced healthcare services

Technology has a significant role in improving medical care, such as enhancing communication between patients and healthcare virtual assistants. By including artificial intelligence and natural language processing (NLP), the main aim of this research paper is to design a medical chatbot framework, and application design prototype, and use the Dialog Flow tool to build an auto-response system known as a chatbot. The goal is to provide medical services and meet patients' needs. It will communicate with the user in the language they follow. With the help of keywords such as symptoms entered by the patients, it will help to identify the disease. Moreover, the application provides the patients with the ability to record their medicine to remind them to take it. Additionally, it will make it easier for patients to learn more about their disease and motivate them to take the necessary precautions to stay healthy.

Development of a ChatBot model for health telecare: Integration of LangChain, embeddings with OpenAI, and Pinecone using the question answering technique

This study was dedicated to the development of an advanced model for a medical ChatBot, aimed at optimizing Telecare in healthcare. Cutting-edge technologies such as LangChain, Embeddings with OpenAI, and Pinecone were incorporated, applying Question Answering techniques to enhance user-Bot interaction. Google Colab was essential, serving as the main environment for both running the code and the ChatBot, in addition to facilitating intensive data processing. The methodology involved a manual and rigorous data collection from official medical sources to ensure the accuracy and reliability of the information integrated into the ChatBot. The results showed a notable variability in response times between GPT-3.5-turbo and GPT-4, with a comprehensive cost evaluation to analyze the profitability of both models in specific medical contexts. In conclusion, the study provided significant insights into the implementation and efficacy of advanced medical ChatBot models, revealing crucial considerations about optimizing response times and cost evaluations in the development of innovative Telecare solutions in healthcare.

AI-Based Medical Chatbot for Disease Prediction

AI-Based Medical Chatbot for Disease Prediction

Systematic review: The use of large language models as medical chatbots in digestive diseases.

Interest in large language models (LLMs), such as OpenAI's ChatGPT, across multiple specialties has grown as a source of patient-facing medical advice and provider-facing clinical decision support. The accuracy of LLM responses for gastroenterology and hepatology-related questions is unknown. To evaluate the accuracy and potential safety implications for LLMs for the diagnosis, management and treatment of questions related to gastroenterology and hepatology. We conducted a systematic literature search including Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus and the Web of Science Core Collection to identify relevant articles published from inception until January 28, 2024, using a combination of keywords and controlled vocabulary for LLMs and gastroenterology or hepatology. Accuracy was defined as the percentage of entirely correct answers. Among the 1671 reports screened, we identified 33 full-text articles on using LLMs in gastroenterology and hepatology and included 18 in the final analysis. The accuracy of question-responding varied across different model versions. For example, accuracy ranged from 6.4% to 45.5% with ChatGPT-3.5 and was between 40% and 91.4% with ChatGPT-4. In addition, the absence of standardised methodology and reporting metrics for studies involving LLMs places all the studies at a high risk of bias and does not allow for the generalisation of single-study results. Current general-purpose LLMs have unacceptably low accuracy on clinical gastroenterology and hepatology tasks, which may lead to adverse patient safety events through incorrect information or triage recommendations, which might overburden healthcare systems or delay necessary care.

Advancing Medical Assistance: Developing an Effective Hungarian-Language Medical Chatbot with Artificial Intelligence

In recent times, the prevalence of chatbot technology has notably increased, particularly in the realm of medical assistants. However, there is a noticeable absence of medical chatbots that cater to the Hungarian language. Consequently, Hungarian-speaking people currently lack access to an automated system capable of providing assistance with their health-related inquiries or issues. Our research aims to establish a competent medical chatbot assistant that is accessible through both a website and a mobile app. It is crucial to highlight that the project’s objective extends beyond mere linguistic localization; our goal is to develop an official and effectively functioning Hungarian chatbot. The assistant’s task is to answer medical questions, provide health advice, and inform users about health problems and treatments. The chatbot should be able to recognize and interpret user-provided text input and offer accurate and relevant responses using specific algorithms. In our work, we put a lot of emphasis on having steady input so that it can detect all the diseases that the patient is dealing with. Our database consisted of sentences and phrases that a user would type into a chatbot. We assigned health problems to these and then assigned the categories to the corresponding cure. Within the research, we developed a website and mobile app, so that users can easily use the assistant. The app plays a particularly important role for users because it allows them to use the assistant anytime and anywhere, taking advantage of the portability of mobile devices. At the current stage of our research, the precision and validation accuracy of the system is greater than 90%, according to the selected test methods.

Accuracy, readability, and understandability of large language models for prostate cancer information to the public.

Generative Pretrained Model (GPT) chatbots have gained popularity since the public release of ChatGPT. Studies have evaluated the ability of different GPT models to provide information about medical conditions. To date, no study has assessed the quality of ChatGPT outputs to prostate cancer related questions from both the physician and public perspective while optimizing outputs for patient consumption. Nine prostate cancer-related questions, identified through Google Trends (Global), were categorized into diagnosis, treatment, and postoperative follow-up. These questions were processed using ChatGPT 3.5, and the responses were recorded. Subsequently, these responses were re-inputted into ChatGPT to create simplified summaries understandable at a sixth-grade level. Readability of both the original ChatGPT responses and the layperson summaries was evaluated using validated readability tools. A survey was conducted among urology providers (urologists and urologists in training) to rate the original ChatGPT responses for accuracy, completeness, and clarity using a 5-point Likert scale. Furthermore, two independent reviewers evaluated the layperson summaries on correctness trifecta: accuracy, completeness, and decision-making sufficiency. Public assessment of the simplified summaries' clarity and understandability was carried out through Amazon Mechanical Turk (MTurk). Participants rated the clarity and demonstrated their understanding through a multiple-choice question. GPT-generated output was deemed correct by 71.7% to 94.3% of raters (36 urologists, 17 urology residents) across 9 scenarios. GPT-generated simplified layperson summaries of this output was rated as accurate in 8 of 9 (88.9%) scenarios and sufficient for a patient to make a decision in 8 of 9 (88.9%) scenarios. Mean readability of layperson summaries was higher than original GPT outputs ([original ChatGPT v. simplified ChatGPT, mean (SD), p-value] Flesch Reading Ease: 36.5(9.1) v. 70.2(11.2), <0.0001; Gunning Fog: 15.8(1.7) v. 9.5(2.0), p < 0.0001; Flesch Grade Level: 12.8(1.2) v. 7.4(1.7), p < 0.0001; Coleman Liau: 13.7(2.1) v. 8.6(2.4), 0.0002; Smog index: 11.8(1.2) v. 6.7(1.8), <0.0001; Automated Readability Index: 13.1(1.4) v. 7.5(2.1), p < 0.0001). MTurk workers (n = 514) rated the layperson summaries as correct (89.5-95.7%) and correctly understood the content (63.0-87.4%). GPT shows promise for correct patient education for prostate cancer-related contents, but the technology is not designed for delivering patients information. Prompting the model to respond with accuracy, completeness, clarity and readability may enhance its utility when used for GPT-powered medical chatbots.

Healthcare Chatbot Using AI

Abstract: Medical chatbots mark a major advancement in patient care and dissemination of medical information. These AIpowered virtual assistants provide a variety of services, from providing medical information, facilitating appointments, and monitoring symptoms to providing psychological support and making it easier to find the nearest medical centre. It provides users with 24/7 access by easily advertising and pre-registering, improving user experience and making the healthcare process more efficient. Although medical chatbots have many advantages, it is important to know that their role is complementary rather than replacing, and therefore they should continue to be in the care of doctors. The report outlines the future of healthcare chatbots and highlights their potential to increase access and efficiency for patients and doctors

A Research Paper of a Medical Chatbot using Llama 2

Abstract: Embracing the knowledge era in healthcare, this research explores a novel medical chatbot empowered by the formidable Llama 2 language model. Transcending the limitations of conventional chatbots, it leverages a trifecta of technological prowess: Llama 2's vast knowledge base and nuanced understanding of medical language enable pinpointaccurate responses, even to intricate inquiries. Chainlit's web framework sculpts an intuitive interface, empowering users to effortlessly navigate and visualize pertinent medical information. Faiss, the master of information retrieval, swiftly delivers curated medical resources tailored to the user's specific needs This research paves the way for a future where medical chatbots, wielding the power of knowledge, empower individuals to navigate the complexities of healthcare with confidence.

Artificial Intelligence Healthcare Chatbot System (Medimate-AI)

With the increasing population of India, increasing birth rate and decreasing death rate due to advancement in the medical field, it's found that the number of doctors is less to serve the needs of the increasing population. This scenario can be better understood while walking through the city's government hospitals where the less availability of the doctors is the major cause behind the improper treatment of the patients and in certain scenarios the resultant death. To encounter such cases there is a need for the smart and Intelligent chatbot who can provide advice to the doctors and sometimes even patients about what to do in such cases which ultimately results in saving the life of hundreds of people. The AI based medical chatbot (Medimate-AI) on which this research topic is based deals with providing medical. The Medimate-AI interact with potential patients visiting the site, assisting them with finding specialists and getting them admittance to the right treatment

HEALTH CARE MEDIBOT IN MEDICAL FIELD USING MACHINE LEARNING

Health is considered an important part of human health, including physical, mental and social health, and plays an important role in our lives. Hospitals are the most common way for patients to receive physical examinations ,diagnosis , treatment recommendations. However, it Is very difficult to take every minor health problem to a doctor and get approval from the doctor. The idea is to use artificial intelligence to create a “medical chatbot” that can diagnose disease and provide useful information about the disease before going to the doctor. Chatbots are computers that use natural language to interact with users. The main goal of the project is to create a Medibot that can assist doctors and patients with tasks such as symptom assessment, diagnostic support, treatment recommendations and publication of medical information. Medibot will use machine learning algorithms that learn from big data of medical conversations to understand users' questions and ideas and respond accordingly. medical software or existing platforms and perform rigorous testing to ensure accuracy and reliability. A continuous evaluation and improvement process will be used to improve Medibot's performance over time. Through this project, the team is focusing on the development of AI driven solutions in healthcare while solving real-world problems faced by doctors and patients. Deploying Medibot has the potential to improve clinical processes, improve access to medical information, and ultimately improve patient care outcomes Key Words:: Medibot, Rasa framework, Health, Artificial intelligence, Natural language processing, Queries, Deployment of ML algorithms, intent, entity

AI-Based Medical Chatbot for Disease Prediction By IJISRT

This research paper presents the development and implementation of an AI-based medical chatbot for disease prediction. Leveraging machine learning and artificial intelligence technologies, the chatbot utilizes natural language processing (NLP) to understand user queries and provide accurate information, guidance, and assistance for various infectious diseases. Motivated by the global spread of infectious diseases and the need for accessible healthcare support, the paper outlines the objectives, algorithm design, dataset description, and application of the chatbot. The algorithm involves receiving user input, extracting symptoms, classifying diseases, and suggesting prevention measures. The dataset, structured in JSON format, facilitates training and pattern recognition. The chatbot interface, accessible across platforms, offers information on symptoms, prevention measures, hospital bed availability, and medication options. In conclusion, the research highlights the potential of AI-based chatbots in revolutionizing healthcare accessibility and personalized diagnosis, thereby bridging the gap between users and healthcare systems.

Generative Pre-Trained Transformer-Empowered Healthcare Conversations: Current Trends, Challenges, and Future Directions in Large Language Model-Enabled Medical Chatbots

This review explores the transformative integration of artificial intelligence (AI) and healthcare through conversational AI leveraging Natural Language Processing (NLP). Focusing on Large Language Models (LLMs), this paper navigates through various sections, commencing with an overview of AI’s significance in healthcare and the role of conversational AI. It delves into fundamental NLP techniques, emphasizing their facilitation of seamless healthcare conversations. Examining the evolution of LLMs within NLP frameworks, the paper discusses key models used in healthcare, exploring their advantages and implementation challenges. Practical applications in healthcare conversations, from patient-centric utilities like diagnosis and treatment suggestions to healthcare provider support systems, are detailed. Ethical and legal considerations, including patient privacy, ethical implications, and regulatory compliance, are addressed. The review concludes by spotlighting current challenges, envisaging future trends, and highlighting the transformative potential of LLMs and NLP in reshaping healthcare interactions.

MedLingua: A conceptual framework for a multilingual medical conversational agent

This study introduces a hybrid model for an advanced medical chatbot addressing crucial healthcare communication challenges. Leveraging a hybrid ML model, the chatbot aims to provide accurate and prompt responses to users' health-related queries. The proposed model will overcome limitations observed in previous medical chatbots by integrating a dual-stemming approach, P-Stemmer and NLTK-Stemmer, accommodating both semitic and non-semitic languages. The system prioritizes the analysis of cognates, identification of symptoms, doctor recommendations, and prescription generation. It integrates an automatic translation module to facilitate a smooth multilingual diagnostic experience. Following the Scrum methodology for agile development, the framework ensures adaptability to evolving research needs and stays current with recent medical discoveries. This groundbreaking idea aims to improve the effectiveness and availability of healthcare services by introducing an intelligent, multilingual chatbot. This technology enables patients to communicate with doctors from diverse linguistic backgrounds through an automated language translation model, eliminating language barriers and extending healthcare access to rural regions worldwide.•A simple but efficient hybrid conceptual model for advancement in smart medical assistance.•This conceptual model can be applied to implement a medical chatbot that can understand multiple languages.•This method can be utilized to address medical chatbot limitations and enhance accuracy in response generation.

Healthcare : A Transformer Network Based Chatbot

Medical chatbots are becoming more and more common as the healthcare industry changes and places more and more focus on automated and remote services. These digital assistants provide cost savings, prompt answers to healthcare inquiries, and access to medical guidance around-the-clock. The effectiveness of these chatbots depends on how well-informed they are about healthcare. Our project's main goal is to employ Transformer network architecture to create a sophisticated chatbot for healthcare. It includes gathering data, choosing a model, recognizing intent and entities, managing conversations, producing responses, integrating knowledge databases, creating user-friendly designs, doing thorough testing, adhering to regulations, and continuously improving. Our objective is to develop a smart conversational agent that can respond to a broad range of healthcare inquiries, provide advice and recommendations tailored to individual diseases, and eventually improve patient satisfaction and healthcare experiences

BERT-Based Medical Chatbot: Enhancing Healthcare Communication through Natural Language Understanding

The advent of modern technologies like Artificial Intelligence(AI), Internet of Things(IoT) and Deep Learning(DL) has ushered in a transformative era in healthcare, offering innovative solutions towards personalized healthcare by enhancing the quality of various medical services. Our proposed methodology involves the development of a BERT-based medical chatbot, leveraging cutting-edge deep learning technology to significantly enhance healthcare communication and accessibility. The traditional challenges faced by medical chatbots, such as imprecise understanding of medical conversations, inaccurate responses to jargon, and the inability to offer personalized feedback, are addressed through the utilization of Bidirectional Encoder Representations from Transformers (BERT). The performance metrics of our chatbot underscore its effectiveness. With an accuracy of 98%, the chatbot ensures a high level of precision in handling medical queries. The precision score of 97% attests to the accuracy and reliability of its responses. The AUC-ROC score of 97% indicates the chatbot's exceptional ability to predict specific diseases based on user queries and symptoms, showcasing its robust predictive power. Furthermore, a recall of 96% demonstrates the chatbot's capability to avoid missing cases in medical diagnoses, ensuring comprehensive coverage of potential conditions. The F1 score of 98% showcases the chatbot's proficiency in delivering accurate and personalized healthcare information, striking a harmonious balance between precision and recall. Our BERT-based medical chatbot not only addresses the limitations of traditional approaches but also achieves a remarkable performance with high accuracy, precision, predictive power, and comprehensive coverage, making it a valuable tool for advancing the quality of healthcare services.

Performance of AI chatbots on controversial topics in oral medicine, pathology, and radiology

ObjectivesIn this study, we assessed six different artificial intelligence (AI) chatbots (Bing, GPT-3.5, GPT-4, Google Bard, Claude, Sage) responses to controversial and difficult questions in oral pathology, oral medicine, and oral radiology. Study DesignThe chatbots’ answers were evaluated by board-certified specialists using a modified version of the global quality score on a 5-point Likert scale. The quality and validity of chatbot citations were evaluated. ResultsClaude had the highest mean score of 4.341 ± 0.582 for oral pathology and medicine. Bing had the lowest scores of 3.447 ± 0.566. In oral radiology, GPT-4 had the highest mean score of 3.621 ± 1.009 and Bing the lowest score of 2.379 ± 0.978. GPT-4 achieved the highest mean score of 4.066 ± 0.825 for performance across all disciplines. 82 out of 349 (23.50%) of generated citations from chatbots were fake. ConclusionsThe most superior chatbot in providing high-quality information for controversial topics in various dental disciplines was GPT-4. Although the majority of chatbots performed well, it is suggested that developers of AI medical chatbots incorporate scientific citation authenticators to validate the outputted citations given the relatively high number of fabricated citations.