Research on the model of automatic recognition and natural language question-answer system for traditional Chinese medicine tongue images based on LLMs

Abstract

Large Language Models (LLMs) have recently demonstrated their potential in clinical applications by providing valuable medical knowledge and recommendations. Traditional Chinese tongue diagnosis, one of the "Four Diagnoses," is an essential method for traditional Chinese medicine diagnosis. This paper builds upon tongue image classification technology and utilizes natural language processing and image recognition techniques to enhance the discrimination and analysis of traditional Chinese tongue images through learning and inference. We propose a method to integrate LLMs into the tongue image automatic recognition model and use them in interactive question-answering by summarizing and reorganizing information in natural language text format.