Maturity estimation of mangoes using hyperspectral imaging from a ground based mobile platform

Abstract

Monitoring the maturity of fruit in commercial orchards can help growers optimise the time of harvest. Dry matter content (DM) of fruit is used as an indicator of mango maturity, measured in-field with a hand-held spectrometer. This approach is labour intensive, limiting the extent to which DM variability can be measured across an orchard block, which would enable selective harvesting. This paper proposes an alternative approach that utilises a hyperspectral camera, LIDAR sensor and navigation system mounted to a ground vehicle to predict fruit DM individually for hundreds of trees in a mango orchard block. First, the challenges faced due to tree geometry and shadows in mango orchards are addressed. Then the ability to predict DM at a distance using hyperspectral imaging (411.3–867.0 nm) was demonstrated. Two regression methods, partial least squares (PLS) and a convolutional neural network (CNN) were compared and tested against DM results from a hand-held NIR spectrometer using harvested (n=468,σ=2.32%w/w) and on-tree fruit (n=662,σ=1.79%w/w). The CNN achieved a cross validation RCV2=0.64 and RMSECV=1.08%w/w in fruit on tree, while PLS achieved RCV2=0.58 and RMSECV=1.17%w/w. In order to discriminate mango and non-mango pixels, PLS discriminant analysis (PLS-DA) and a CNN were also compared, where both methods achieved good classification performance with a mean F1>0.97. Having established mango classification and DM prediction performance, hyperspectral data were processed for a full orchard block and projected to world coordinates using AGV position and orientation as provided by the navigation system. Trees were segmented using corresponding LIDAR data, which allowed association of projected DM predictions to individual trees. Repeated scans of the orchard block over two days allowed a measure of repeatability, which was achieved with an RMSE<0.29%w/w. The results provide strong evidence that predicting maturity at a distance for all trees in an orchard is feasible using a hyperspectral camera, which will be an important management tool for growers to optimise harvest timing and yield.