Estimation of soil organic matter by in situ Vis-NIR spectroscopy using an automatically optimized hybrid model of convolutional neural network and long short-term memory network

Abstract

It is attractive nowadays to estimate soil information based on in situ Visible and Near-infrared (Vis-NIR) spectroscopy. However, there exist a lot of errors, mainly due to modeling between soil properties and spectra. A hybrid of deep learning (DL) models seems an ideal approach for the modeling, e.g., convolutional neural network (CNN) and long short-term memory (LSTM). Nevertheless, it is not easy for a user, or not efficient for an automatic technique, such as Bayesian optimization (BO), to select hundreds of parameters of a hybrid of DL models. Recently, tree Parzen Estimator (TPE) based HyperBand within BO (BOHB) has been widely used in other fields for establishing a hybrid of DL models efficiently and automatically. Further, it is not quite clear if a DL model or a hybrid DL model performs well for different sample sizes and different numbers of spectral data. This study aimed to investigate these issues by evaluating performances of two hybrids of DL models (CNN-LSTM and LSTM-CNN) established using BOHB, based on soil spectra and organic matter contents of 670 soil samples collected from a forest in Nanning, southwest China. The performances were also compared with those of separate DL models and two commonly used methods, i.e., partial least square regression (PLSR) and random forest (RF), while the effect of sample sizes on the performances was also investigated. Besides, the effect of different numbers of spectra was also investigated, and the spectra were augmented using two methods, one by stacking scans of all measurement points of a sample and one by stacking preprocessed spectra using several methods. Results showed that, given less than 600 samples, accuracies of CNN-LSTM and LSTM-CNN were not higher than those of separate DL models, PLSR and RF. However, BOHB optimization largely improved the accuracies of the two hybrid models which were close to or higher than those of separate DL models, PLSR and RF. With the increase of sample sizes, accuracies of all models used in this study gradually increased and the DL models seemed to be more promising for large sample sizes. Furthermore, augmenting spectra with preprocessed data provided much more benefits than sample sizes, modeling methods, and optimization methods. Thus, it is promising to estimate soil using in situ Vis-NIR spectra based on a hybrid of DL models optimized using BOHB, particularly for large datasets or augmented spectra.