Toward Recognition of Easily Confused TCM Herbs on the Smartphone Using Hierarchical Clustering Convolutional Neural Network

Abstract

Background and Objective. The use of Chinese herbal medicines (CHMs) for treatment plays an important role in traditional Chinese medicine (TCM). However, some herbs are easily confused with the others because their shapes/textures look similar and they could have totally different utilities. Recently, deep learning has attracted great attention for the application of image recognition and could be useful for TCM herb identification. Methods. For recognizing easily‐confused TCM herbs on a smartphone, we propose a CHM recognition system using hierarchical clustering convolutional neural networks (HCNNs) based on the affinity propagation clustering method. Results. We implement our system on the smartphone and show recognition accuracy close to 98%, based on a dataset of 65 kinds of herbs (including 12 easy‐confused herbs pairs). We also investigate the effect of different parameters (e.g., selection of clustering algorithms for HCNNs, types of smartphone, and number of layers in the neural network) on the system performance. Conclusions. In this work, we proposed a hierarchical clustering convolutional neural network (HCNN) method to distinguish similar TCM herbs with a high accuracy. We also showed the usefulness of applying the data augmentation techniques when implementing the proposed system for a variety of smartphones.