Automated spectroscopic modelling with optimised convolutional neural networks

Zefang Shen,R A Viscarra Rossel

doi:10.1038/s41598-020-80486-9

Abstract

Convolutional neural networks (CNN) for spectroscopic modelling are currently tuned manually, and the effects of their hyperparameters are not analysed. These can result in sub-optimal models. Here, we propose an approach to tune one-dimensional CNN (1D-CNNs) automatically. It consists of a parametric representation of 1D-CNNs and an optimisation of hyperparameters to maximise a model’s performance. We used a large European soil spectroscopic database to demonstrate our approach for estimating soil organic carbon (SOC) contents. To assess the optimisation, we compared it to random search, and to understand the effects of the hyperparameters, we calculated their importance using functional Analysis of Variance. Compared to random search, the optimisation produced better final results and showed faster convergence. The optimal model produced the most accurate estimates of SOC with hbox {RMSE} = 9.67 pm 0.51 (s.d.) and {R}^2 = 0.89 pm 0.013 (s.d.). The hyperparameters associated with model training and architecture critically affected the model’s performance, while those related to the spectral preprocessing had little effect. The optimisation searched through a complex hyperparameter space and returned an optimal 1D-CNN. Our approach simplified the development of 1D-CNNs for spectroscopic modelling by automatically selecting hyperparameters and preprocessing methods. Hyperparameter importance analysis shed light on the tuning process and increased the model’s reliability.

Highlights

We propose an approach to tune one-dimensional convolutional neural networks (CNNs) (1D-CNNs) automatically
In the Methods section below, we describe the dataset used, the parametric representation of the 1D-CNN, the hyperparameters, their values, and the optimisation
The Bayesian optimisation plateaued at 80 trials (9 h and 51 min) whereas random search needed 250 trials to level-off (24 h and 12 min)

Summary

Introduction

We propose an approach to tune one-dimensional CNN (1D-CNNs) automatically. It consists of a parametric representation of 1D-CNNs and an optimisation of hyperparameters to maximise a model’s performance. Since a spectrum contains information on the fundamental compositing of soil, spectra can be used to estimate functional soil properties such as soil organic carbon (SOC). To estimate soil properties with spectra, one needs to develop spectroscopic models using a spectral library. Various spectral preprocessing methods are commonly used to remove noise, extract information from the spectra and improve the performance of the spectroscopic models, which describe the establishment of the relationship between the spectra and soil p roperties[2]. Statistical methods, such as partial least squares r egression[12], have been used to model soil properties with spectra. They include methods such as random forest[14], support vector machine[15], artificial neural networks[16]

Objectives

Methods

Results

Conclusion