Development of a mobile phone based tongue image acquisition system

Abstract

Background. Tongue diagnosis is one of the main diagnostic methods in Chinese Medicine (CM). In addition to being used with other techniques to generate a CM diagnosis, methods derived from CM tongue diagnosis may have new applications, especially in early detection of certain diseases. Therefore, we believe that automating the collection and analysis of tongue images is of great benefit. While several methods for capturing images, extracting features and analyzing the data have been proposed over the years, most have limitations in reliability, accuracy, cost and usability. Nonetheless, recent advances in the field of image processing and machine learning (ML), together with improved hardware capabilities are rapidly enabling novel solutions to the problem. This paper describes a new mobile phone-based image acquisition method and reports our assessment of image quality from a clinical perspective. Methodology. A mobile phone-based system that uses a phone's flash and back camera to collect images with automatic white balance was developed. The usability and image quality were tested on forty (40) volunteers. Information pertaining to CM diagnosis of the tongue was collected through direct visual inspection, and by inspecting images produced by a commercially available tongue image diagnostic instrument, and by our newly developed system. The data collected amongst the three methods were compared using Cohen's kappa coefficient to evaluate the practical value of the images collected by the new system. Result. Our system is able to collect good quality tongue images and in general provides information comparable to those obtained through direct visual inspection or by the commercial instrument. Compared with the commercial instrument, our system has better inter-rater agreement with direct visual inspection in detecting the presence of tooth marks (kappa value: 0.633 vs 0.259), but inferior in detecting tongue coating greasiness (kappa value: 0.217 vs 0.739). Main contributions of this paper. We have presented a new mobile phone-based tongue image acquisition system that is low cost and convenient to use. Because of these desirable features, the system has the potential for mass public use to collect big data, paving the way for development of automatic tongue analysis using deep learning techniques.