Evaluating the efficacy of major language models in providing guidance for hand trauma nerve laceration patients: a case study on Google's AI BARD, Bing AI, and ChatGPT

Abstract

This study evaluated three prominent Large Language Models (LLMs)-Google’s AI BARD, Bing’s AI, and ChatGPT-4 in providing patient advice for hand laceration. Five simulated patient inquiries on hand trauma were prompted to them. A panel of Board-certified plastic surgical residents evaluated the responses for accuracy, comprehensiveness, and appropriate sources. Responses were also compared against existing literature and guidelines. This study suggests that ChatGPT outperforms BARD and Bing AI in providing reliable, evidence-based clinical advice, but they still face limitations in depth and specificity. Healthcare professionals are essential in interpreting LLM recommendations, and future research should improve LLM performance by integrating specialized databases and human expertise to advance nerve injury management and optimize patient-centred care.