Abstract

This study proposes the utilization of a Natural Language Processing tool to create a semantic search engine for dental education while addressing the increasing concerns of accuracy, bias, and hallucination in outputs generated by AI tools. The paper focuses on developing and evaluating DentQA, a specialized question-answering tool that makes it easy for students to seek information to access information located in handouts or study material distributed by an institution. DentQA is structured upon the GPT3.5 language model, utilizing prompt engineering to extract information from external dental documents that experts have verified. Evaluation involves non-human metrics (BLEU scores) and human metrics for the tool's performance, relevance, accuracy, and functionality. Non-human metrics confirm DentQA's linguistic proficiency, achieving a Unigram BLEU score of 0.85. Human metrics reveal DentQA's superiority over GPT3.5 in terms of accuracy (p=0.00004) and absence of hallucination (p=0.026). Additional metrics confirmed consistent performance across different question types (X2(4,N=200)=13.0378,p=0.012). User satisfaction and performance metrics support DentQA's usability and effectiveness, with a response time of 3.5 s and over 70% satisfaction across all evaluated parameters. The study advocates using a semantic search engine in dental education, mitigating concerns of misinformation and hallucination. By outlining the workflow and the utilization of open-source tools and methods, the study encourages the utilization of similar tools for dental education while underscoring the importance of customizing AI models for dentistry. Further optimizations, testing, and utilization of recent advances can contribute to dental education significantly.

Full Text
Paper version not known

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.