Measuring pecan nut growth utilizing machine vision and deep learning for the better understanding of the fruit growth curve

Abstract

Knowing pecan nut growth dynamic and its thermal requirements from bloom to ripening are essential to predict nut size and harvest date. However, monitoring of pecan growth (shuck, shell, and embryo) is a time consuming manual task, requiring multiple measurements for each nut. This process also ignores any deformation of the nut. A computer vision system can facilitate such a task by performing pixel-by-pixel semantic segmentation for all parts of the nut. This study uses the Mask-RCNN algorithm for object detection and semantic segmentation for area estimation of shuck, shell, and embryo on pecans in multiple growth stages. The dataset was divided in two stages: small (young) and big (older) pecans. The network chosen achieved an F1 score of 95.3% to 100% on the models for the object detection task. The area estimation achieved a mean absolute percentage error of 10.14% to 28.06% on these tasks. The resulting growth plot presents a sigmoid curve for all parts of the nut, which demonstrates the precision of the measurement. This automated measuring system makes it possible to have a multidimensional understanding of each aspect of nut development, and gives us the ability of modeling nut growth phenology and predicting nut productions.