Deep leaf: Mask R-CNN based leaf detection and segmentation from digitized herbarium specimen images

Abstract

• A modified mask RCNN named Deep Leaf is developed to identify the leaves from the digitized herbarium specimens. • Deep Leaf measures automatically the morphological traits of the extracted leaves. • Deep features are extracted through an improved ResNet50/101, which is chosen as the backbone network of the feature extraction. • We achieved better performance compared with the original mask RCNN algorithm for leaves detection. The generation of morphological traits of plants such as the leaf length, width, perimeter, area, and petiole length are fundamental features of herbarium specimens, thus providing high-quality data to investigate plant responses to ongoing climatic change and plant history evolution. However, the existing measurement methods are primarily associated with manual analysis, which is labor-intensive and inefficient. This paper proposes a deep learning-based approach, called Deep Leaf, for detecting and pixel-wise segmentation of leaves based on the improved state-of-the-art instance segmentation approach, Mask Region Convolutional Neural Network (Mask R-CNN). Deep Leaf can accurately detect each leaf in the herbarium specimen and measure the associated morphological traits. The experimental results indicate that our automated approach can segment the leaves of different families. Compared to manual measurement done by ecologist and botanist experts, the average relative error of leaf length is 4.6%, while the average relative error of leaf width is 5.7%.