Derivation of biophysical variables from Earth observation data: validation and statistical measures

Abstract

Evaluation is an essential step of model development. However, there is a missing definition of appropriate validation strategies, needed to guarantee reproducibility and generalizability of modeling results. Also, there is a lack of a generally agreed set of 'optimal' statistical measure(s) to assess model accuracy. The objective of the present study is to provide for remote sensing practitioners (i.e., non-statisticians) guidance for model validation strategies and to propose an optimal set of statistical measures for the quantitative assessment of model performance in the context of vegetation biophysical variable retrieval from Earth observation (EO) data. For these purposes, main terms and concepts were reviewed. Then, validation strategies were tested on a polynomial regression model and discussed. Moreover, a literature review was carried out, summarizing the statistical measures used to evaluate model performances. Supported by some exemplary datasets, these measures were calculated and their meanings discussed in view of several model validation criteria. From the results, we recommend to further exploit cross-validation and bootstrapping strategies to guarantee the development/validation of reliable models. An 'optimal' statistic set is suggested, including root mean square error (RMSE), coefficient of determination (R 2 ), slope and intercept of Theil-Sen regression, relative RMSE, and Nash-Sutcliffe efficiency index. A wide acceptance and use of these statistics should enable a better intercomparison of scientific results, urgently needed in times of increasing model development activities that are carried out with respect to upcoming EO missions.