An Improved Version of the Prewhitening Method for Trend Analysis in the Autocorrelated Time Series

Abstract

Nonparametric trend detection tests like the Mann–Kendall (MK) test require independent observations, but serial autocorrelation in the datasets inflates/deflates the variance and alters the Type-I and Type-II errors. Prewhitening (PW) techniques help address this issue by removing autocorrelation prior to applying MK. We evaluate several PW schemes—von Storch (PW-S), Slope-corrected PW (PW-Cor), trend-free prewhitening (TFPW) proposed by Yue (TFPW-Y), iterative TFPW (TFPW-WS), variance-corrected TFPW (VCTFPW), and newly proposed detrended prewhitened with modified trend added (DPWMT). Through Monte Carlo simulations, we constructed a lag-1 autoregressive (AR(1)) time series and systematically assessed the performance of different PW methods relative to sample size, autocorrelation, and trend slope. Results indicate that all methods tend to overestimate weak trends in small samples (n < 40). For moderate/high trends, the slopes estimated from the VCTFPW and DPWMT series close (within a ± 20% range) to the actual trend. VCTFPW shows slightly lower RMSE than DPWMT at mid-range lag-1 autocorrelation (ρ1 = 0.3 to 0.6) but fluctuates for ρ1 ≥ 0.7. Original series and TFPW-Y fail to control Type-I error with increasing ρ1, while VCTFPW and DPWMT maintained Type-I errors below the significance level (α = 0.05) for large samples. Apart from TFPW-Y, all PW methods resulted in weak power of the test for weak trends and small samples. TFPW-WS showed high power of the test but only for strong autocorrelated data combined with strong trends. In contrast, VCTFPW failed to preserve the power of the test at high autocorrelation (≥0.7) due to slope underestimation. DPWMT restores the power of the test for 0.1 ≤ ρ1 ≤ 0.9 for moderate/strong trends. Overall, the proposed DPWMT approach demonstrates clear advantages, providing unbiased slope estimates, reasonable Type-I error control, and sufficient power in detecting linear trends in the AR(1) series.