Root Mean Square Error Statistics Research Articles

Development and testing of updated curve number models for efficient runoff estimation in steep-slope watersheds

The globally adopted and computationally efficient curve number (CN) model fills an active hydrological professional niche and has a well-documented history. However, it is structurally inconsistent and fails to reliably estimate runoff from rainfall. This is mainly due to the much-debated fixed initial abstraction (λ) and associated sudden jumps in runoff based on CN obtained from the documented tables. In this study, three new variants (M4, M5, M6) of the CN model are proposed that consider the hydrological imbalance between pre-storm soil moisture and initial abstraction after a rainfall event. A total of 1837 rainfall-runoff events were analyzed from 41 steep-slope watersheds in South Korea to test the robustness of the proposed CN models. The results were compared to the recently updated original CN model (M1) and two other recent variants (M2, M3) of this model. These models were evaluated using root mean squared error (RMSE), Nash-Sutcliffe efficiency (NSE), percent bias (PB), Kling-Gupta efficiency (KGE), a proposed overall weighted score, 1:1 line graph, and the FITEVAL tool. Using data from 41 watersheds, the lowest mean (median) RMSE of M5, M6, M4 [15.54(16.56), 15.84(11.45), 18.00(16.56)], PB for M4, M6, M5 [−1.28(−1.03), 1.29(0.03), −7.06(−7.27)]; the highest mean (median) NSE for M5, M6, M4 [0.86(0.87), 0.85(0.87), 0.81(0.83)], KGE for M6, M5, M4 [0.81(0.84), 0.79(0.81), 0.79(0.81)] and other graphical assessments show a better agreement between the observed and the runoff estimated by the proposed models. The corresponding mean (median) of RMSE, PB, NSE, and KGE statistics for M2 [19.26(17.69), 9.50(10.34), 0.78(0.80), 0.77(0.79)], and M3 [19.99(18.47), 05.56(7.17), 0.76(0.79), 0.77(0.79)] models show comparatively inferior results. Based on the same statistics, the M1 [24.60(22.48), 33.62(32.77), 0.63(0.68), 0.60(0.64)] model yields unrealistic results. It is inferred that both λ and CN should be kept flexible for a systematic and region-specific revision of the CN model to improve runoff estimation. In addition, a structurally consistent model with a stable soil moisture accounting (SMA) procedure is vital to get more reliable runoff estimates without compromising the simplicity and applicability to ungauged watersheds.

Chasing Volatility of USD/TRY Foreign Exchange Rate: The Comparison of CARR, EWMA, and GARCH Models

This paper aims to make a comparison between range-based and return-based volatility models. For this purpose, we compare the Conditional Autoregressive Range (CARR) type and Generalized Autoregressive Conditional Heteroskedastic (GARCH) type models with different innovation distributions and the Exponential Weighted Moving Average (EWMA) model with fixed and estimated lambda parameters. The out-of-sample forecasts obtained from the volatility processes are compared according to the Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Heteroskedastic Root Mean Square Error (HRMSE), and Heteroskedastic Mean Absolute Error (HMAE) statistics. We use the USD-TRY exchange rate data for real-life applications since estimating the volatility of forex helps to determine prices for goods and services to avoid the uncertainty created by exchange rate shocks in developing countries such as Turkiye. Although MAE and RMSE show Gumbel CARR and Weibull CARR have the minimum error statistics, respectively, the HMAE and HRMSE statistics indicate that among the range-based models, the EWMA model, in which the lambda parameter is estimated, performs better. Furthermore, we find that Exponential CARR according to RMSE and MAE statistics, and Weibull CARR according to HMAE and HRMSE statistics appear as the return-based volatility models with minimum error.

Do you know your r2?

The prediction of solubility of drugs usually calls on the use of several open-source/commercially-available computer programs in the various calculation steps. Popular statistics to indicate the strength of the prediction model include the coefficient of determination (r2), Pearson’s linear correlation coefficient (rPearson), and the root-mean-square error (RMSE), among many others. When a program calculates these statistics, slightly different definitions may be used. This commentary briefly reviews the definitions of three types of r2 and RMSE statistics (model validation, bias compensation, and Pearson) and how systematic errors due to shortcomings in solubility prediction models can be differently indicated by the choice of statistical indices. The indices we have employed in recently published papers on the prediction of solubility of druglike molecules were unclear, especially in cases of drugs from ‘beyond the Rule of 5’ chemical space, as simple prediction models showed distinctive ‘bias-tilt’ systematic type scatter.

Assessment of Ordinary Kriging and Inverse Distance Weighting Methods for Modeling Chromium and Cadmium Soil Pollution in E-Waste Sites in Douala, Cameroon.

Background.Several studies have demonstrated that chromium (Cr) and cadmium (Cd) have adverse impacts on the environment and human health. These elements are present in electronic waste (e-waste) recycling sites. Several interpolation methods have been used to evaluate geographical impacts on humans and the environment.Objectives.The aim of the present paper is to compare the accuracy of inverse distance weighting (IDW) and ordinary kriging (OK) in topsoil analysis of e-waste recycling sites in Douala, Cameroon.Methods.Selecting the proper spatial interpolation method is crucial for carrying out surface analysis. Ordinary kriging and IDW are interpolation methods used for spatial analysis and surface mapping. Two sets of samples were used and compared. The performances of interpolation methods were evaluated and compared using cross-validation.Results.The results showed that the OK method performed better than IDW prediction for the spatial distribution of Cr, but the two interpolation methods had the same result for Cd (in the first set of samples). Results from Kolmogorov-Smirnov and Shapiro-Wilk tests showed that the data were normally distributed in the study area. The p value (0.302 and 0.773) was greater than 0.05 for Cr and for Cd (0.267 and 0.712). In the second set of samples, the OK method results (for Cd and Cr) were greatly diminished and the concentrations dropped, looking more like an average on the maps. However, the IDW interpolation gave a better representation of the concentration of Cd and Cr on the maps of the study area. For the second set of samples, OK and IDW for Cd and Cr had more similar results, especially in terms of root mean square error (RMSE).Conclusions.Many parameters were better identified from the RMSE statistic obtained from cross-validation after exhaustive testing. Inverse distance weighting appeared more adequate in limited urban areas.Competing Interests. The authors declare no competing financial interests

Comparison of regression-based and combined versions of Inverse Distance Weighted methods for spatial interpolation of daily mean temperature data

This paper focuses on the performance of two regression-based and one Inverse Distance Weighted (IDW) and two combined versions of IDW methods for interpolation of daily mean temperature at the Black Sea Region of Turkey. Simple linear regression (SLR) and multiple linear regression (MLR) are used as regression-based methods. Combinations of IDW with TLR (temperature lapse rate) and gradient plus inverse distance squared (GIDS) are used as combined versions of IDW. This study targets to compare five spatial interpolation methods based on RMSE (root-mean-square error) statistics of interpolation errors for daily mean temperatures from 1981 to 2012. In order to compare the interpolation errors of the five methods, the leave-one-out cross-validation method was applied over long periods of 32 years on 52 different sites. The algorithms of the five interpolation methods’ codes were written in MATLAB by the authors of the paper.

CEREF: A hybrid data-driven model for forecasting annual streamflow from a socio-hydrological system

Hydrological forecasting is complicated by flow regime alterations in a coupled socio-hydrologic system, encountering increasingly non-stationary, nonlinear and irregular changes, which make decision support difficult for future water resources management. Currently, many hybrid data-driven models, based on the decomposition–prediction-reconstruction principle, have been developed to improve the ability to make predictions of annual streamflow. However, there exist many problems that require further investigation, the chief among which is the direction of trend components decomposed from annual streamflow series and is always difficult to ascertain. In this paper, a hybrid data-driven model was proposed to capture this issue, which combined empirical mode decomposition (EMD), radial basis function neural networks (RBFNN), and external forces (EF) variable, also called the CEREF model. The hybrid model employed EMD for decomposition and RBFNN for intrinsic mode function (IMF) forecasting, and determined future trend component directions by regression with EF as basin water demand representing the social component in the socio-hydrologic system. The Wuding River basin was considered for the case study, and two standard statistical measures, root mean squared error (RMSE) and mean absolute error (MAE), were used to evaluate the performance of CEREF model and compare with other models: the autoregressive (AR), RBFNN and EMD–RBFNN. Results indicated that the CEREF model had lower RMSE and MAE statistics, 42.8% and 7.6%, respectively, than did other models, and provided a superior alternative for forecasting annual runoff in the Wuding River basin. Moreover, the CEREF model can enlarge the effective intervals of streamflow forecasting compared to the EMD–RBFNN model by introducing the water demand planned by the government department to improve long-term prediction accuracy. In addition, we considered the high-frequency component, a frequent subject of concern in EMD-based forecasting, and results showed that removing high-frequency component is an effective measure to improve forecasting precision and is suggested for use with the CEREF model for better performance. Finally, the study concluded that the CEREF model can be used to forecast non-stationary annual streamflow change as a co-evolution of hydrologic and social systems with better accuracy. Also, the modification about removing high-frequency can further improve the performance of the CEREF model. It should be noted that the CEREF model is beneficial for data-driven hydrologic forecasting in complex socio-hydrologic systems, and as a simple data-driven socio-hydrologic forecasting model, deserves more attention.

Calibration and Validation of the Crop Growth Model DAISY for Spring Barley in the Czech Republic

In this paper, the crop growth model DAISY for spring barley (cultivar “Tolar“) was calibrated and subsequently validated in three different soil-climate locations in the Czech Republic – Lednice (48°48'51'' N, 16°48'46'' E, altitude 180 m), Věrovany (49°27'39'' N, 17°17'42'' E, altitude 210 m) and Domanínek (49°31'42'' N, 16°14'13'' E, altitude 560 m). The calibration and validation were based on data from a multi-year field experiment from the Central Institute for Supervising and Testing in Agriculture and from a two-year field experiment in Domanínek (2011 and 2012) that was conducted by the Institute of Agrosystems and Bioclimatology in cooperation with the Global Change Research Centre AS CR. The calibration for Lednice, Věrovany and Domanínek was performed using 4 growth seasons from each station, the subsequent validation for Lednice and Věrovany was performed based on 3 growth seasons from each station, and that for Domanínek was based on 6 growth seasons. The value of the RMSE (root mean square error) statistic for flowering was 2 days for calibration and 4 days for validation on average; for maturity, the RMSE was 11 days for both calibration and validation. The average RMSE for the yields was 0.9 t·ha−1 for calibration and 1.6 t·ha−1 for validation. According to the statistical index MBE (mean bias error) for the flowering phenological phase, the crop growth model DAISY showed a delay of 2 days in both calibration and validation. There was also delay of 6 days in calibration and of 8 days in validation for maturity. According to the MBE, the crop growth model DAISY underestimates the yield by 0.2 t·ha−1 for calibration and underestimates the yield by 0.4 t·ha−1 for validation.

Evaluation of infiltration models with different numbers of fitting parameters in different soil texture classes

In this study, the ability of eight different infiltration models (i.e. Green and Ampt, Philip, SCS (US-Soil Conservation Service), Kostiakov, Horton, Swartzendruber, Modified Kostiakov (MK) and Revised Modified Kostiakov (RMK) models) were evaluated by least-squares fitting to measured infiltration data. Six comparison criteria including coefficient of determination (R2), mean root mean square error (MRMSE), root mean square error (RMSE), the F-statistic (F), Cp statistic of Mallows (Cp) and Akaike information criterion (AIC) were used to determine the best performing model with the least number of fitting parameters. Results indicated that R2 and MRMSE were not suitable for model selection. A more valid comparison was achieved by F, Cp, AIC and RMSE statistics. The RMK model including four parameters had the best performance with the majority of soils studied. RMK was better than the MK model in approximately 51.6, 57, 68.5 and 70.6% of soils, when using F, Cp, AIC and RMSE statistics, respectively, and for the other models, a higher per cent of soils was obtained. The RMK model was the best for loam, clay loam and silty clay loam soils, but the MK model was the best for silty loam soils.

Testing Alternative Models for Forecasting Volatility in Stock Futures Market: A Linear Approach

Thanks to numerous empirical research studies, a general consensus has been reached on examining the various alternative models for forecasting and modeling volatility in stock futures contracts by using out-of-sample forecast according to statistic and risk management evaluation criteria. The dataset were retrieved from National Stock Exchange (NSE) website terminal for the period from April 1, 2003 and ending on December 31, 2008. The forecasting models that are considered here ranges from Random Walk, Linear Regression, Moving Average, Autoregressive Models. In order to evaluate the forecasting performance of different models we use two forecasting error statistics by considering the root mean square error (RMSE) and the mean absolute percentage error (MAPE) for testing the return characteristics. Our findings suggest that, according to RMSE statistics the autoregressive model and linear regression models rationally shared and ranked first for out-of-sample forecasts in the linear models. In addition, one cannot conclude that the success or failure of a particular type of forecasting model applied to one market carries over to a different market, because the size and liquidity of a market can affect the quality of volatility forecasts. Finally, one can learn more about the market movement and return volatility through studying the non linear approach.

The Role of Financial Variables in Predicting Economic Activity in the Euro Area

The U.S. business cycle typically leads the European cycle by a few quarters and this can be used to forecast euro area GDP. We investigate whether financial variables carry additional information. We use vector autoregressions (VARs) which include the U.S. and the euro area GDPs as a minimal set of variables as well as growth in the Rest of the World (an aggregation of seven small countries) and selected combinations of financial variables. Impulse responses (in-sample) show that shocks to financial variables influence real activity. However, according to out-of-sample forecast exercises using the Root Mean Square Error (RMSE) metric, this macro-financial linkage would be weak: financial indicators do not improve short and medium term forecasts of real activity in the euro area, even when their timely availability, relative to GDP, is exploited. This result is partly due to the 'average' nature of the RMSE metric: when forecasting ability is assessed as if in real time (conditionally on the information available at the time of the forecast), we find that models using financial variables would have been preferred, ex ante, in several episodes, in particular between 1999 and 2002. This result suggests that one should not discard, on the basis of RMSE statistics, the use of predictive models that include financial variables if there is a theoretical prior that a financial shock is affecting growth.

The Role of Financial Variables in Predicting Economic Activity in the Euro Area

The U.S. business cycle typically leads the European cycle by a few quarters and this can be used to forecast euro area GDP. We investigate whether financial variables carry additional information. We use vector autoregressions (VARs) which include the U.S. and the euro area GDPs as a minimal set of variables as well as growth in the Rest of the World (an aggregation of seven small countries) and selected combinations of financial variables. Impulse responses (in-sample) show that shocks to financial variables influence real activity. However, according to out-of-sample forecast exercises using the Root Mean Square Error (RMSE) metric, this macro-financial linkage would be weak: financial indicators do not improve short and medium term forecasts of real activity in the euro area, even when their timely availability, relative to GDP, is exploited. This result is partly due to the 'average' nature of the RMSE metric: when forecasting ability is assessed as if in real time (conditionally on the information available at the time of the forecast), we find that models using financial variables would have been preferred, ex ante, in several episodes, in particular between 1999 and 2002. This result suggests that one should not discard, on the basis of RMSE statistics, the use of predictive models that include financial variables if there is a theoretical prior that a financial shock is affecting growth.

Positional Accuracy of Spatial Data: Non‐Normal Distributions and a Critique of the National Standard for Spatial Data Accuracy

AbstractSpatial data quality is a paramount concern in all GIS applications. Existing spatial data accuracy standards, including the National Standard for Spatial Data Accuracy (NSSDA) used in the United States, commonly assume the positional error of spatial data is normally distributed. This research has characterized the distribution of the positional error in four types of spatial data: GPS locations, street geocoding, TIGER roads, and LIDAR elevation data. The positional error in GPS locations can be approximated with a Rayleigh distribution, the positional error in street geocoding and TIGER roads can be approximated with a log‐normal distribution, and the positional error in LIDAR elevation data can be approximated with a normal distribution of the original vertical error values after removal of a small number of outliers. For all four data types considered, however, these solutions are only approximations, and some evidence of non‐stationary behavior resulting in lack of normality was observed in all four datasets. Monte‐Carlo simulation of the robustness of accuracy statistics revealed that the conventional 100% Root Mean Square Error (RMSE) statistic is not reliable for non‐normal distributions. Some degree of data trimming is recommended through the use of 90% and 95% RMSE statistics. Percentiles, however, are not very robust as single positional accuracy statistics. The non‐normal distribution of positional errors in spatial data has implications for spatial data accuracy standards and error propagation modeling. Specific recommendations are formulated for revisions of the NSSDA.