Analysis and application of the ensemble Kalman filter for the estimation of bounded quantities

Abstract

The Kalman filter and its Monte Carlo approximation, the ensemble Kalman filter (EnKF), are best suited to problems involving unbiased, Gaussian errors. Non-Gaussian error distributions induced by bounded quantities make the EnKF sub-optimal and cause biased estimates. Further, EnKF estimates of bounded quantities may violate physical bounds and lead to a failure of the involved model. Extending the EnKF with a nonlinear variable transformation technique can mitigate the first and solve the second problem. Motivated by a parameter estimation problem from land surface modelling, we analyse the effects of non-Gaussian distributions and non-zero mean errors on EnKF estimates theoretically and experimentally. For the first time, we use a linear regression framework to qualitatively examine and explain errors in the EnKF estimates and we analyse their behaviour with and without variable transformations. From theoretical considerations, we derive a covariance scaling approach for the estimation of the transformed observation error covariance that ensures a constant transformed observation error covariance, independent of the observed value. Comparing estimates derived with the new covariance scaling approach, with two other transformation-based approaches, and with the EnKF without variable transformation, we find that covariance scaling is superior to the other methods with respect to the quality of the estimates (for all other methods) and with respect to its computational cost (for all methods except the EnKF without anamorphosis). We verify these findings in a series of data assimilation experiments using synthetic land surface albedo observations and a newly implemented data assimilation framework based on the dynamic global vegetation model JSBACH and the Data Assimilation Research Testbed.