Real-time statistical algorithm for cherry tomatoes with different ripeness based on depth information mapping

Abstract

Rapid and non-destructive automatic statistics of cherry tomatoes at different ripeness stages help better manage resources during harvesting, storage, and transportation processes. Currently, the inspection of cherry tomatoes (ripeness assessment and counting) still faces challenges, such as excluding background cherry tomatoes, detecting heavily obscured ones, and tracking similar feature extraction across frames. This study presented a statistical algorithm for cherry tomatoes with different ripeness. Firstly, a complete field of view was achieved by stitching images from dual cameras. Then, during the detection phase, the proposed depth information mapping and morphological operations were employed to filter out background cherry tomatoes effectively. Secondly, the SimAM attention module was introduced to enhance the focus of the YOLO v7-tiny model on small and occluded targets. The ReID feature extraction model was replaced with the lighter and more powerful MobileNeXt model, with the input resolution adapted to 64 × 64 based on the morphological characteristics of cherry tomatoes. Finally, the statistics of cherry tomatoes at different ripeness levels were conducted using the improved DeepSORT algorithm. The ablation experimental results prove the effectiveness of the proposed algorithm. The improved YOLO v7-tiny has a mAP of 87.3 % on the dataset, combining depth information mapping with morphological operations. Compared with the original DeepSORT algorithm, the improved DeepSORT algorithm has RMSE decreased by 15.46 to 3.11, and R2 increased by 0.071 to 0.998. The statistical algorithm enables real-time statistical of the number of cherry tomatoes at different ripeness levels during inspection.