Detection and segmentation of lettuce seedlings from seedling-growing tray imagery using an improved mask R-CNN method

Abstract

Real-time monitoring and management of lettuce plants, especially during the seedling growth stage, is important in order to protect them from various environmental stresses and to prevent yield loss. Various and similar symptoms may occur as a result of different unfavorable environmental stresses. Visual inspection alone may result in an incorrect and late diagnosis, obstructing subsequent corrective action for the damaged seedlings in an early stage. The purposes of this study were to detect and segment lettuce seedlings from the background of seedling-growing trays using an improved mask R-CNN and to estimate seedling size based on the output of the proposed technique. Lettuce seedling images were taken from a plant factory using an automatic image acquisition tool. The seedlings in the images were 1 to 3 weeks old. An annotation of 1,000 lettuce seedling image datasets was prepared using an online annotation tool. An improved mask R-CNN was implemented using ResNet-101 with CB-Net, which provided enhanced image feature extraction. Transfer learning was used to train the model network with a smaller dataset and reduce the processing time. Bounding boxes and annotations were set as inputs for the training model. Training and test datasets were prepared for the model evaluation. The sizes of the seedlings were determined by applying a binary mask to the output-masked seedling images. The total number of pixels was used to determine the seedling size. The training loss of the selected method was less than 0.20 %. The identification of lettuce seedlings from 150 randomly chosen test photos revealed that the best-fit image had a performance F1 score of 93 %, corresponding to 92 % precision and 95 % recall. A strong linear relationship (R2>0.99) between manual and model-estimated leaf area confirmed the proposed model accuracy throughout the lettuce growth stages. The improved mask R-CNN could detect lettuce seedlings in the tray image background and also extract the leaf area of identified lettuce seedlings.