Cucumber Fruits Detection in Greenhouses Based on Instance Segmentation

Xiaoyang Liu,Chengzhi Ruan,Yueping Sun,Wei Ji,Weikuan Jia,Dean Zhao

doi:10.1109/access.2019.2942144

Xiaoyang Liu, Chengzhi Ruan + Show 4 more

Open Access

https://doi.org/10.1109/access.2019.2942144

Copy DOI

Abstract

The cucumber fruits have the same color with leaves and their shapes are all long and narrow, which is different from other common fruits, such as apples, tomatoes, and strawberries, etc. Therefore, cucumber fruits are more difficult to be detected by machine vision in greenhouses for special color and shape. A pixel-wise instance segmentation method, mask region-based convolutional neural network (Mask RCNN) of an improved version, is proposed to detect cucumber fruits. Resnet-101 is selected as the backbone of Mask RCNN with feature pyramid network (FPN). To improve the detection precision, region proposal network (RPN) in original Mask RCNN is improved. Logical green ( LG ) operator is designed to filter non-green background and limit the range of anchor boxes. Besides, the scales and aspect ratios of anchor boxes are also adjusted to fit the size and shape of fruits. Improved Mask RCNN has a better performance on test images. The test results are compared with that of original Mask RCNN, Faster RCNN, you only look once (YOLO) V2 and YOLO V3. The $F_{1}$ score of improved Mask RCNN in test results reaches 89.47%, which is higher than the other methods. The average elapsed time of improved Mask RCNN is 0.3461 s, which is only lower than the original Mask RCNN. Meanwhile, the mean value and standard deviation of location deviation in improved Mask RCNN are 2.10 pixels and 1.73 pixels respectively, which are lower than the other methods.

Highlights

Fruit detection is an important research filed in precision agriculture, which is widely applied to yield estimation, and fruit picking robot [1]–[3]
TRAINING AND TESTING MASK RCNN In addition to improved region proposal network (RPN), the selection of backbone is an important factor to affect the precision of cucumber detection
Resnet-101 is selected as the backbone of improved Mask RCNN

Summary

INTRODUCTION

Fruit detection is an important research filed in precision agriculture, which is widely applied to yield estimation, and fruit picking robot [1]–[3]. Cucumber fruits have the same color with leaves and their shapes are long and narrow, which is different from other common fruits. Zhang et al [13] adopted a three-layer back propagation (BP) neural network to segment cucumber fruits from the background. The researches listed above adopted different methods to detect cucumbers, but the general strategy is similar. Two kinds of CNNs, ZFnet and VGG16, were used as the backbone network of Faster RCNN to detect all kinds of fruits mentioned above respectively and the detection precisions were all more than 90%. Mask RCNN of an improved version, is proposed to detect cucumber fruits in pixel level in our study. Compared with our previous works [26]–[28], this work focuses on the detection of cucumber fruits and is more challenging

MATERIALS AND METHODS

RESULTS AND DISCUSSIONS

COMPARISONS OF FRUIT DETECTION

CONCLUSION