High-throughput instance segmentation and shape restoration of overlapping vegetable seeds based on sim2real method

Abstract

The high-throughput measurement of morphological parameters for seeds is an essential procedure in vegetable phenotyping. This study drew on sim2real thought to design a novel image simulation method based on multi-feature randomization to synthesize simulated images and generate labels automatically. The Mask R-CNN model was optimized based on seed characteristics to improve the segmentation performance of small vegetable seeds. The optimized Mask R-CNN models trained on simulated datasets achieved superior segmentation performance on real-world datasets, and AP@[0.5:0.95] reached more than 0.85. Furthermore, the Mask R-CNN models had a satisfactory performance in the shape restoration of occluded seeds, and AP@[0.5:0.95] were 0.81, 0.79, and 0.80 for eggplant, tomato, and pepper seeds, respectively. The combination of sim2real and Mask R-CNN realized high-throughput instance segmentation and shape restoration of overlapping vegetable seeds, and it could be used as an alternative for vegetable seed phenotyping.