Tomato leaf segmentation algorithms for mobile phone applications using deep learning

Abstract

Leaf disease recognition using image processing techniques is currently a hot research topic. High disease recognition accuracies have been reported in literature particularly in studies that relied on lab images captured under controlled lighting and uniform background conditions. When these systems were tested under real field conditions, however, their performance dropped sharply. Studies have demonstrated that complex backgrounds significantly contribute to this drop in performance and that background removal enhances disease recognition accuracy. Unfortunately, a fast and accurate means to perform automatic background subtraction for leaf images has so far not yet been developed. In this paper, we propose fully convolutional neural networks to perform automatic background subtraction for leaf images captured in mobile applications. In a mobile application use case, the target leaf would typically dominate an image captured by the farmer. The leaf would also be surrounded by various background features including other leaves, stems, fruits, soil and mulch. The goal of the segmentation network is to remove these background features so that only the target leaf remains. A dataset that is representative of this scenario was prepared in order to train and test the proposed networks. It consists of 1,408 tomato leaf images captured under challenging field conditions and their respective ground truth masks. We report state-of-the-art leaf image segmentation performance of over 0.96 mean weighted intersection over union and over 0.91 boundary F1 score. In particular, our proposed segmentation network KijaniNet outperforms all competitors scoring 0.9766 mean weighted intersection over union and 0.9439 boundary F1 score. The proposed technique supersedes competing background subtraction algorithms yet does not require user intervention nor does it place constraints on the orientation, shape or illumination of the target leaf. Furthermore, all CNN models are able to perform segmentation of a 256x256 pixel RGB image in under 0.12 s when running on a GPU and in less than 2.1 s when running on a CPU; which is much faster than any of the competing techniques.