Crop Residue Cover Percentage Estimation from RGB Images Using Transfer Learning and Ensemble Ordinal Regression

Abstract

Highlights A transfer learning strategy improved residue estimates from high-resolution RGB imagery. The best method used probabilistic estimates of expert classifiers to estimate residue cover. This research confirms the utility of RGB imagery to quantify residue cover in agricultural fields. Abstract. Plant residue on the soil surface increases the sustainability of food and fiber production in agricultural systems. Automated assessments of residue cover based on imagery have the potential to reduce labor and human bias associated with in-field measurements. We evaluate the capacity of a transfer learning strategy to improve the determination of residue level from high-resolution RGB images. The imagery for the study was collected from 88 field locations in 40 row crop fields in five Missouri counties between mid-April and early July in 2018 and 2019. At each field location, 50 contiguous 0.3 m × 0.2 m region of interest (ROI) images (ground sampling distance of 0.014 cm pixel--1) were extracted from imagery, resulting in a dataset of 4,400 ROI images; 3,000 were used for cross-validation and training (data collected in 2018) and 1,400 were used for testing (data collected in 2019). The percentage residue for each ROI image (ground truth) was determined by a bullseye grid method (n = 100). Features were extracted from ROI images using the VGGNet-16 CNN model, a pre-trained convolutional neural network model. We extracted 1,472 features per ROI using a global averaging and pooling strategy. The optimum feature set was identified using recursive feature elimination using a support vector machine (RFE-SVM). To estimate crop residue percentage using selected features, expert two-class SVMs were trained to separate adjacent levels of residue cover, where the rationale of the ensemble was to allow each of the two-class SVMs to find the hyperplanes that maximize the margin between the corresponding two consecutive classes. Based on the distance of the samples to these hyperplanes, probabilistic estimates of the data-point belonging to the class were computed. With the combined knowledge of probabilistic estimates from each expert classifier, the percentage crop residue cover of each ROI image was calculated. We tested our approach with 3-, 4-, 5-, and 8-class problems, achieving the best results with the 8-class problem with r2 = 0.93 at the ROI level, r2 = 0.97 at the field-location level, and minimal bias in residue estimates in low residue conditions. These results are superior to other reported estimates of percent residue derived from imagery. This research confirms the utility of high-resolution RGB imagery to quantify residue cover in agricultural systems. Keywords: Convolutional neural network, Soil erosion, Support vector machine, Transfer learning.