Comment on "How Good is Your Model Fit? Weighted Goodness-of-Fit Metrics for Irregular Time Series".

Abstract

The technical commentary of Collenteur (2021) touches an important aspect in the use of groundwater head data in the conversion from manual measurements (and sampling) to sensors with automatic dataloggers (e.g., Post and von Asmuth 2013; Retike et al. 2022). Collenteur offers a practical solution that improves evaluation for time series with a transition from regular manual measurements to high(er) frequency automatic logged groundwater heads. The weighting he proposes may also be useful for calibration of time series models. However, scientific underpinning is needed for true advancement in the analysis of such data, and data with other frequency variations. This comment considers the problem from two perspectives: the model Collenteur presented and the head measurements used for the model. Collenteur suggests using the timestep of the lowest frequency for Δ t max . However, a more rigorous approach is needed for usage in model calibration. The response time of the groundwater system or the autocorrelation of the groundwater heads could provide a more physical basis for Δ t max . This will also make the weighting applicable for series with other frequency variations. Collenteur does not mention correlation—which is obviously present in the high frequency part and to a lesser extent in the low frequency part. Weights are needed when there is (variable) correlation between measurements (Hill and Tiedeman 2005) to ensure that equal amounts of information have equal weight in a calibration. The formal solution is a full weight matrix (Hill and Tiedeman 2005). However, this requires information that usually is unknown and thus would require a model. This would lead to an iterative calibration procedure. Also, the matrix can become very large, which makes this approach further impractical. The effect of correlation is that an individual measurement contains less additional information if the correlation with other measurements is higher. If all measurements are weighed equally, information in the measurements with higher correlation is given more importance than the information from measurements with less correlation. This definitely plays a role in the examples of Collenteur. As an illustration, I analyzed the time series of the same piezometer (from the Dutch national subsurface information database at https://www.DINOloket.nl/en/) as Collenteur (2021) with precipitation and Makkink evaporation series from the same meteorological stations of the Royal Dutch Meteorological Institute (KNMI) using the Metran software (Berendrecht and van Geer 2016; Zaadnoordijk et al. 2019). The initial model based on all measurements (orange line in Figure 1) matches the yearly fluctuation reasonably well, and the average level reflects more the average of the high frequency part than of the entire series. Next, the frequency of the part with daily measurements has been reduced by selecting only the measurements on the 14th and the 28th day of each month, resulting in 24 measurements per year. The model for this series gives the same fluctuation, but a better average level (green line in Figure 1). Recognizing that the residuals of the first two models have a multiyear fluctuation that could be due to a much slower response to precipitation and evaporation, a new model has been created in which the responses of the second model have been included with fixed parameters and a second Gamma function has been added for the response of precipitation and evaporation with initial parameters such that the response is slower. This leads to a model that fits the data much better (red line in Figure 1). As a last step, the parameters of the third model have been specified as initial values without fixing any of them and they have been optimized using all measurements (purple line in Figure 1). This example goes beyond the Commentary of Collenteur on the use of weights in calculating statistics for model evaluation. It shows steps that can be taken to arrive at an acceptable model. Working with a reduced set of measurements, which has a similar effect to the weighting scheme of Collenteur, may help during this model development. Weighting proposed by Collenteur is useful in the exploratory phase, but lacks theoretical underpinning and should therefore be avoided for prediction or decision support. Alternative options include the development of structures for the noise model that do a better job of removing autocorrelation in the residuals of a time series model. The example is based on time series modeling carried out in the RESOURCE project within the GeoERA programme, which has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 731166.