Data Assimilation Window Research Articles

An ensemble-based data assimilation system for forecasting variability of the Northwestern Pacific ocean

An adjoint-free four-dimensional variational (a4dVar) data assimilation (DA) is implemented in an operational ocean forecast system based on an eddy-resolving ocean general circulation model for the Northwestern Pacific. Validation of the system against independent observations demonstrates that fitting the model to time-dependent satellite altimetry during a 10-day DA window leads to substantial skill improvements in the succeeding 60-day hindcast. The a4dVar corrects representation of the Kuroshio path variation south of Japan by adjusting the dynamical balance between amplitude/wavelength of the meander and flow advection. A larger ensemble spread tends to reduce the skill in representing the observed sea surface height anomaly, suggesting that it is possible to use the ensemble information for quantifying the forecast error. The ensemble information is also utilized for modification of the background error covariance (BEC), which improves the accuracy of temperature and salinity distributions. The modified BEC yields the skill decline of the Kuroshio path variation during the 60-day hindcast period, and the ensemble sensitivity analysis shows that changes in the dynamical balance caused by the ensemble BEC result in such skill deterioration.

A Comparison of 3DEnVar and 4DEnVar for Convective-Scale Direct Radar Reflectivity Data Assimilation in the Context of a Filter and a Smoother

Abstract The Gridpoint Statistical Interpolation (GSI)-based four- and three-dimensional ensemble–variational (4DEnVar and 3DEnVar) methods are compared as a smoother and a filter, respectively, for rapidly changing storms using the convective-scale direct radar reflectivity data assimilation (DA) framework. Two sets of experiments with varying DA window lengths (WLs; 20, 40, 100, and 160 min) and radar observation intervals (RIs; 20 and 5 min) are conducted for the 5–6 May 2019 case. The RI determines the temporal resolution of ensemble perturbations for the smoother and the DA interval for the filter spanning the WL. For experiments with a 20-min RI, evaluations suggest that the filter and the smoother have comparable performance with a 20-min WL; however, extending the WL results in the outperformance of the filter over the smoother. Diagnostics reveal that the degradation of the smoother is attributed to the increased degree of nonlinearity and the issue of time-independent localization as the WL extends. Evaluations for experiments with different RIs under the same WL indicate that the outperformance of the filter over the smoother diminishes for most forecast hours at thresholds of 30 dBZ and above when shortening the RI. Diagnostics show that more frequent interruptions of the model introduce model imbalance for the filter, and the increased temporal resolution of ensemble perturbations enhances the degree of nonlinearity for the smoother. The impact of model imbalance on the filter overwhelms the enhanced nonlinearity on the smoother as the RI reduces. Significance Statement The background uncertainties of rapidly changing storms suffer from fast error growth and high degrees of nonlinearities during the data assimilation (DA) period. Two variants of the ensemble-based DA method can account for such temporal evolution. The smoother uses background ensemble from multiple observation times over an assimilation period to estimate the propagation of statistics. The filter frequently calculates the statistics at multiple observation times over the same period. Current comparisons of the smoother and the filter were mostly performed using simple models; however, unknowns remain for convection-allowing forecasts with additional complexities. This study compares the filter and the smoother for the convective-scale analysis and prediction using a real-data study and finds that the comparison varies with the assimilation period and the observation interval.

An improved estimate of daily precipitation from the ERA5 reanalysis

AbstractPrecipitation is an essential climate variable and a fundamental part of the global water cycle. Given its importance to society, precipitation is often assessed in climate monitoring activities, such as in those led by the Copernicus Climate Change Service (C3S). To undertake these activities, C3S predominantly uses ERA5 reanalysis precipitation. Research has shown that short‐range forecasts for precipitation made from this reanalysis can provide valuable estimates of the actual (observed) precipitation in extratropical regions but can be less useful in the tropics. While some of these limitations will be reduced with future reanalyses because of the latest advancements, there is potentially a more immediate way to improve the precipitation estimate. This is to use the precipitation modelled in the Four‐Dimensional Variational (4D‐Var) data assimilation window of the reanalysis, and it is the aim of this study to evaluate this approach. Using observed 24‐h precipitation accumulations at 5637 stations from 2001 to 2020, results show that smaller root‐mean‐square errors (RMSEs) and mean absolute errors are generally found by using the ERA5 4D‐Var precipitation. For example, for all available days from 2001 to 2020, 87.5% of stations have smaller RMSEs. These improvements are driven by reduced random errors in the 4D‐Var precipitation because it is better constrained by observations, which are themselves sensitive to or influence precipitation. However, there are regions (e.g., Europe) where larger biases occur, and via the decomposition of the Stable Equitable Error in Probability Space score, this is shown to be because the 4D‐Var precipitation has a wetter bias on ‘dry’ days than the standard ERA5 short‐range forecasts. The findings also highlight that the 4D‐Var precipitation does improve the discrimination of ‘heavy’ observed events. In conclusion, an improved ERA5 precipitation estimate is largely obtainable, and these results could prove useful for C3S activities and for future reanalyses, including ERA6.

Formulation and use of 3D‐hybrid and 4D‐hybrid ensemble covariances in the Météo‐France global data assimilation system

AbstractThe global data assimilation (DA) system at Météo‐France is currently based on a 4D‐Var formulation relying on wavelet‐based 3D background‐error covariances. These covariances are specified at the beginning of the DA window and are evolved implicitly in the DA window through tangent linear and adjoint model integrations. Further research and development steps on data assimilation at Météo‐France are conducted in the framework of the Object‐Oriented Prediction System (OOPS), which is developed in collaboration with the European Centre for Medium‐Range Weather Forecasts (ECMWF). For instance, 3D background‐error covariances can be made hybrid through a linear combination between wavelet‐based covariances and ensemble‐based covariances that are filtered through spatial localisation. This allows covariances to be made more anisotropic in a flow‐dependent way, and implementation of this hybridation in the OOPS framework is shown to have general positive impacts on the forecast quality. This 3D‐hybrid approach can also be extended to a 4D‐hybrid approach in the OOPS framework: this relies on a linear combination between 4D ensemble covariances on the one hand and 4D linearly propagated covariances on the other hand, corresponding to initial covariances that are evolved more explicitly by tangent linear and adjoint versions of the model. This provides a unifying framework for implementations of DA schemes that correspond to 4DEnVar, 4D‐Var, and new 4D‐hybrid formulations. This is thus considered as a novel way to combine the respective attractive features of 4D‐Var and 4DEnVar approaches, leading in particular to a new 4D‐hybrid formulation of 4DEnVar. Its properties and implementation in the OOPS framework are presented, and first experimental results show that this new formulation of 4DEnVar is competitive with 4D‐Var, in relation with the improved hybridisation.

A Regional multi-Air Pollutant Assimilation System (RAPAS v1.0) for emission estimates: system development and application

Abstract. Top-down atmospheric inversion infers surface–atmosphere fluxes from spatially distributed observations of atmospheric composition in order to quantify anthropogenic and natural emissions. In this study, we developed a Regional multi-Air Pollutant Assimilation System (RAPAS v1.0) based on the Weather Research and Forecasting–Community Multiscale Air Quality (WRF–CMAQ) modeling system model, the three-dimensional variational (3D-Var) algorithm, and the ensemble square root filter (EnSRF) algorithm. This system can simultaneously assimilate hourly in situ CO, SO2, NO2, PM2.5, and PM10 observations to infer gridded emissions of CO, SO2, NOx, primary PM2.5 (PPM2.5), and coarse PM10 (PMC) on a regional scale. In each data assimilation window, we use a “two-step” scheme, in which the emissions are inferred first and then input into the CMAQ model to simulate initial conditions (ICs) of the next window. The posterior emissions are then transferred to the next window as prior emissions, and the original emission inventory is only used in the first window. Additionally, a “super-observation” approach is implemented to decrease the computational costs, observation error correlations, and influence of representative errors. Using this system, we estimated the emissions of CO, SO2, NOx, PPM2.5, and PMC in December and July 2016 over China using nationwide surface observations. The results show that compared to the prior emissions (2016 Multi-resolution Emission Inventory for China – MEIC 2016)), the posterior emissions of CO, SO2, NOx, PPM2.5, and PMC in December 2016 increased by 129 %, 20 %, 5 %, 95 %, and 1045 %, respectively, and the emission uncertainties decreased by 44 %, 45 %, 34 %, 52 %, and 56 %, respectively. With the inverted emissions, the RMSE of simulated concentrations decreased by 40 %–56 %. Sensitivity tests were conducted with different prior emissions, prior uncertainties, and observation errors. The results showed that the two-step scheme employed in RAPAS is robust in estimating emissions using nationwide surface observations over China. This study offers a useful tool for accurately quantifying multi-species anthropogenic emissions at large scales and in near-real time.

Impacts of Northeastern Pacific Buoy Surface Pressure Observations

Abstract Under the Atmospheric River Reconnaissance (AR Recon) Program, ocean drifting buoys (drifters) that provide surface pressure observations were deployed in the northeastern Pacific Ocean to improve forecasts of U.S. West Coast high-impact weather. We examine the impacts of both AR Recon and non-AR Recon drifter observations in the U.S. Navy’s global atmospheric data assimilation (DA) and forecast system using data-denial experiments and forecast sensitivity observation impact (FSOI) analysis, which estimates the impact of each observation on the 24-h global forecast error total energy. Considering all drifters in the eastern North Pacific for the 2020 AR Recon season, FSOI indicates that most of the beneficial impacts come from observations in the lowest quartile of observed surface pressure values, particularly those taken late in the DA window. Observations in the upper quartile have near-neutral impacts on average and are slightly nonbeneficial when taken late in the DA window. This may occur because the DA configuration used here does not account for model biases, and innovation statistics show that the forecast model has a low pressure bias at high pressures. Case studies and other analyses indicate large beneficial impacts coming from observations in regions with large surface pressure gradients and integrated vapor transport, such as fronts and ARs. Data-denial experiments indicate that the assimilation of AR Recon drifter observations results in a better-constrained analysis at nearby non-AR Recon drifter locations and counteracts the NAVGEM pressure bias. Assimilating the AR Recon drifter observations improves 72- and 96-h Northern Hemisphere forecasts of winds in the lower and middle troposphere, and geopotential height in the lower, middle, and upper troposphere. Significance Statement The purpose of this study is to understand how observations of atmospheric pressure at the ocean surface provided by drifting buoys impact weather forecasts. Some of these drifting buoys were deployed under a program to study atmospheric rivers (ARs) to improve forecasts of high-impact weather on the West Coast. We find that these observations are most effective at reducing forecast errors when taken in regions near fronts and cyclones. The additional drifting buoys deployed under the AR Reconnaissance project reduce forecast errors at 72 and 96 h over North America and the Northern Hemisphere. These results are important because they illustrate the potential for improving forecasts by increasing the number of drifting buoy surface pressure observations over the world oceans.

Ensemble-based 4DVarNet uncertacinty quantification for the reconstruction of sea surface height dynamics

Abstract Uncertainty quantification (UQ) plays a crucial role in data assimilation (DA) since it impacts both the quality of the reconstruction and near-future forecast. However, traditional UQ approaches are often limited in their ability to handle complex datasets and may have a large computational cost. In this paper, we present a new ensemble-based approach to extend the 4DVarNet framework, an end-to-end deep learning scheme backboned on variational DA used to estimate the mean of the state along a given DA window. We use conditional 4DVarNet simulations compliant with the available observations to estimate the 4DVarNet probability density function. Our approach enables to combine both the efficiency of 4DVarNet in terms of computational cost and validation performance with a fast and memory-saving Monte-Carlo based post-processing of the reconstruction, leading to the so-called En4DVarNet estimation of the state pdf. We demonstrate our approach in a case study involving the sea surface height: 4DVarNet is pretrained on an idealized Observation System Simulation Experiment (OSSE), then used on real-world dataset (OSE). The sampling of independent realizations of the state is made among the catalogue of model-based data used during training. To illustrate our approach, we use a nadir altimeter constellation in January 2017 and show how the uncertainties retrieved by combining 4DVarNet with the statistical properties of the training dataset lead to a relevant information providing in most cases a confidence interval compliant with the Cryosat-2 nadir alongtrack dataset kept for validation.

Reconstruction of the Rheological Parameters in a Sea Ice Model with Viscoplastic Rheology

Abstract We analyzed the feasibility of the reconstruction of the spatially varying rheological parameters through the four-dimensional variational data assimilation of the sea ice velocity, thickness, and concentration into the viscoplastic (VP) sea ice model. The feasibility is assessed via idealized variational data assimilation experiments with synthetic observations configured for a 1-day data assimilation window in a 50 × 40 rectangular basin forced by the open boundaries, winds, and ocean currents and should be viewed as a first step in the developing the similar algorithms which can be applied for the more advanced sea ice models. It is found that “true” spatial variability (∼5.8 kN m−2) of the internal maximum ice strength parameter can be retrieved from observations with reasonable accuracy of 2.3–5.3 kN m−2, when an observation of the sea ice state is available daily in each grid point. Similar relative accuracy was achieved for the reconstruction of the compactness strength parameter α. The yield curve eccentricity e is found to be controlled by the data with less efficiency, but the spatial mean value of e could be still reconstructed with a similar degree of confidence. The accuracy of , α, and e retrievals is found to increase in regions of stronger ice velocity convergence, providing prospects for better processing of the observations collected during the recent MOSAiC experiment. The accuracy of the retrievals strongly depends on the number of the control variables characterizing the rheological parameter fields.

A Comparison of HWRF Six-Hourly 4DEnVar and Hourly 3DEnVar Assimilation of Inner Core Tail Dopper Radar Observations for the Prediction of Hurricane Edouard (2014)

Six-hourly three-dimensional ensemble variational (3DEnVar) (6H-3DEnVar) data assimilation (DA) assumes constant background error covariance (BEC) during a six-hour DA window and is, therefore, unable to account for temporal evolution of the BEC. This study evaluates the one-hourly 3DEnVar (1H-3DEnVar) and six-hourly 4DEnVar (6H-4DEnVar) DA methods for the analyses and forecasts of hurricanes with rapidly evolving BEC. Both methods account for evolving BEC in a hybrid EnVar DA system. In order to compare these methods, experiments are conducted by assimilating inner core Tail Doppler Radar (TDR) wind for Hurricane Edouard (2014) and by running the Hurricane Weather Research and Forecasting (HWRF) model. In most metrics, 1H-3DEnVar and 6H-4DEnVar analyses and forecasts verify better than 6H-3DEnVar. 6H-4DEnVar produces better thermodynamic analyses than 1H-3DEnVar. Radar reflectivity shows that 1H-3DEnVar produces better structure forecasts. For the first 24–48 h of the intensity forecast, 6H-4DEnVar forecast performs better than 1H-3DEnVar verified against the best track. Degraded 1H-3DEnVar forecasts are found to be associated with background storm center location error as a result of underdispersive ensemble storm center spread. Removing location error in the background improves intensity forecasts of 1H-3DEnVar.

Parameter optimization in sea ice models with elastic–viscoplastic rheology

Abstract. The modern sea ice models include multiple parameters which strongly affect model solution. As an example, in the CICE6 community model, rheology and landfast grounding/arching effects are simulated by functions of the sea ice thickness and concentration with a set of fixed parameters empirically adjusted to optimize the model performance. In this study, we consider the extension of a two-dimensional elastic–viscoplastic (EVP) sea ice model using a spatially variable representation of these parameters. The feasibility of optimization of the landfast sea ice parameters and rheological parameters is assessed via idealized variational data assimilation experiments with synthetic observations of ice concentration, thickness and velocity. The experiments are configured for a 3 d data assimilation window in a rectangular basin with variable wind forcing. The tangent linear and adjoint models featuring EVP rheology are found to be unstable but can be stabilized by adding a Newtonian damping term into the adjoint equations. A set of observation system simulation experiments shows that landfast parameter distributions can be reconstructed after 5–10 iterations of the minimization procedure. Optimization of sea ice initial conditions and spatially varying parameters in the stress tensor equation requires more computation but provides a better hindcast of the sea ice state and the internal stress tensor. Analysis of inaccuracy in the wind forcing and errors in sea ice thickness observations show reasonable robustness of the variational DA approach and the feasibility of its application to available and incoming observations.

An Iterative Ensemble Kalman Smoother in Presence of Additive Model Error

Ensemble variational methods are being increasingly used in the field of geophysical data assimilation. Their efficiency comes from the combined use of ensembles, which provide statistics estimates, and a variational analysis, which handles nonlinear operators through iterative optimization techniques. Taking model error into account in four-dimensional ensemble variational algorithms is challenging because the state trajectory over the data assimilation window (DAW) is no longer determined by its sole initial condition. In particular, the control variable dimension scales with the DAW length, which yields a high numerical complexity. This is unfortunate since accuracy improvement is expected with longer DAWs. Building upon the work of [P. Sakov and M. Bocquet, Tellus A, 70 (2018), 1414545], this paper discusses how to algorithmically construct and numerically test an iterative ensemble Kalman smoother with additive model error (IEnKS-Q) which is thought to be the natural weak constraint generalization of the IEnKS [M. Bocquet and P. Sakov, Quart. J. Roy. Meteorol. Soc., 140 (2014), pp. 1521--1535], as well as the generalization of IEnKF-Q [P. Sakov, J. Haussaire, and M. Bocquet, Quart. J. Roy. Meteorol. Soc., 144 (2018), pp. 1297--1309] to general DAWs. The number of model evaluations per cycle of the IEnKS-Q is also examined. Solutions based on perturbation decomposition are proposed to dissociate those numerically costly evaluations from the control variable dimension.

Ensemble of 4DVARs (En4DVar) data assimilation in a coastal ocean circulation model, Part I: Methodology and ensemble statistics

The ocean state off Oregon-Washington, U.S. West coast, is highly variable in time. Under these conditions the assumption made in traditional 4-dimensional variational data assimilation (4DVAR) that the prior model (background) error covariance is the same in every data assimilation (DA) window can be limiting. A DA system based on an ensemble of 4DVARs (En4DVar) has been developed in which the background error covariance is estimated from an ensemble of model runs and is thus time-varying. This part describes details of the En4DVar method and ensemble statistics verification tests. The control run and 39 ensemble members are forced by perturbed wind fields and corrected by DA in a series of 3-day windows. Wind perturbations are represented as a linear combination of empirical orthogonal functions (EOFs) for the larger scales and Daubechies wavelets for the smaller scales. The variance of the EOF expansion coefficients is based on estimates of the wind field error statistics derived using scatterometer observations and a Bayesian Hierarchical Model. It is found that the variance of the wind errors relative to the natural wind variability increases as the horizontal spatial scales decrease. DA corrections to the control run and ensemble members are calculated in parallel by the newly developed, cost-effective cluster search minimization method. For a realistic coastal ocean application, this method can generate a 30% wall time reduction compared to the restricted B-conjugate gradient (RBCG) method. Ensemble statistics are generally found to be consistent with background error statistics. In particular, ensemble spread is maintained without inflating. However, sea-surface height background errors cannot be fully reproduced by the ensemble perturbations.

Evaluation of iterative Kalman smoother schemes for multi-decadal past climate analysis with comprehensive Earth system models

Abstract. Paleoclimate reconstruction based on assimilation of proxy observations requires specification of the control variables and their background statistics. As opposed to numerical weather prediction (NWP), which is mostly an initial condition problem, the main source of error growth in deterministic Earth system models (ESMs) regarding the model low-frequency response comes from errors in other inputs: parameters for the small-scale physics, as well as forcing and boundary conditions. Also, comprehensive ESMs are non-linear and only a few ensemble members can be run in current high-performance computers. Under these conditions we evaluate two assimilation schemes, which (a) count on iterations to deal with non-linearity and (b) are based on low-dimensional control vectors to reduce the computational need. The practical implementation would assume that the ESM has been previously globally tuned with current observations and that for a given situation there is previous knowledge of the most sensitive inputs (given corresponding uncertainties), which should be selected as control variables. The low dimension of the control vector allows for using full-rank covariances and resorting to finite-difference sensitivities (FDSs). The schemes are then an FDS implementation of the iterative Kalman smoother (FDS-IKS, a Gauss–Newton scheme) and a so-called FDS-multistep Kalman smoother (FDS-MKS, based on repeated assimilation of the observations). We describe the schemes and evaluate the analysis step for a data assimilation window in two numerical experiments: (a) a simple 1-D energy balance model (Ebm1D; which has an adjoint code) with present-day surface air temperature from the NCEP/NCAR reanalysis data as a target and (b) a multi-decadal synthetic case with the Community Earth System Model (CESM v1.2, with no adjoint). In the Ebm1D experiment, the FDS-IKS converges to the same parameters and cost function values as a 4D-Var scheme. For similar iterations to the FDS-IKS, the FDS-MKS results in slightly higher cost function values, which are still substantially lower than those of an ensemble transform Kalman filter (ETKF). In the CESM experiment, we include an ETKF with Gaussian anamorphosis (ETKF-GA) implementation as a potential non-linear assimilation alternative. For three iterations, both FDS schemes obtain cost functions values that are close between them and (with about half the computational cost) lower than those of the ETKF and ETKF-GA (with similar cost function values). Overall, the FDS-IKS seems more adequate for the problem, with the FDS-MKS potentially more useful to damp increments in early iterations of the FDS-IKS.

Adjoint-Based Observation Impact Estimation with Direct Verification Using Forward Calculation

Abstract The adjoint-based observation impact estimation method has been providing essential information to improve data assimilation systems (DASs) in numerical weather prediction (NWP). This paper has two purposes. The first is to verify the four approximations used in the method: iterative construction of the Kalman gain adjoint operator, the tangent linear (TL) approximation of a forecast model, ignoring cross terms of observation impacts, and approximations for incremental DASs. The second is to add new information to our knowledge of observation impacts. For the verification of the adjoint-based method, we use the TL-based observation impact estimation method that can calculate the same quantity as the adjoint-based method without adjoint calculations. Results of these verifications and observation impact estimations performed on the global NWP system of the Japan Meteorological Agency are as follows. First, observation impacts calculated using the adjoint-based method agree well with those from the TL-based method, with the correlation coefficient between them exceeding 0.97. Second, estimated observation impacts are consistent with previous studies in many aspects. There are also system-dependent properties, such as relatively small impacts from GPS radio occultation data above 13 km. Furthermore, new aspects of observation impacts are found: 1) the probability density function of the observation impact agrees well with the scalar theory when giving the experimental value of the observation and the forecast error standard deviations; 2) later observations in the data assimilation window have larger positive impacts; and 3) impacts of AMSU-A in the Southern Hemisphere are more than double those in the Northern Hemisphere.

Quasi-static ensemble variational data assimilation: a theoretical and numerical study with the iterative ensemble Kalman smoother

Abstract. The analysis in nonlinear variational data assimilation is the solution of a non-quadratic minimization. Thus, the analysis efficiency relies on its ability to locate a global minimum of the cost function. If this minimization uses a Gauss–Newton (GN) method, it is critical for the starting point to be in the attraction basin of a global minimum. Otherwise the method may converge to a local extremum, which degrades the analysis. With chaotic models, the number of local extrema often increases with the temporal extent of the data assimilation window, making the former condition harder to satisfy. This is unfortunate because the assimilation performance also increases with this temporal extent. However, a quasi-static (QS) minimization may overcome these local extrema. It accomplishes this by gradually injecting the observations in the cost function. This method was introduced by Pires et al. (1996) in a 4D-Var context. We generalize this approach to four-dimensional strong-constraint nonlinear ensemble variational (EnVar) methods, which are based on both a nonlinear variational analysis and the propagation of dynamical error statistics via an ensemble. This forces one to consider the cost function minimizations in the broader context of cycled data assimilation algorithms. We adapt this QS approach to the iterative ensemble Kalman smoother (IEnKS), an exemplar of nonlinear deterministic four-dimensional EnVar methods. Using low-order models, we quantify the positive impact of the QS approach on the IEnKS, especially for long data assimilation windows. We also examine the computational cost of QS implementations and suggest cheaper algorithms.

Assimilating Doppler radar observations with an ensemble Kalman filter for convection-permitting prediction of convective development in a heavy rainfall event during the pre-summer rainy season of south China

This study examines the effectiveness of an ensemble Kalman filter based on the weather research and forecasting model to assimilate Doppler-radar radial-velocity observations for convection-permitting prediction of convection evolution in a high-impact heavy-rainfall event over coastal areas of South China during the pre-summer rainy season. An ensemble of 40 deterministic forecast experiments (40 DADF) with data assimilation (DA) is conducted, in which the DA starts at the same time but lasts for different time spans (up to 2 h) and with different time intervals of 6, 12, 24, and 30 min. The reference experiment is conducted without DA (NODA).To show more clearly the impact of radar DA on mesoscale convective system (MCS) forecasts, two sets of 60-member ensemble experiments (NODA EF and exp37 EF) are performed using the same 60-member perturbed-ensemble initial fields but with the radar DA being conducted every 6 min in the exp37 EF experiments from 0200 to 0400 BST. It is found that the DA experiments generally improve the convection prediction. The 40 DADF experiments can forecast a heavy-rain-producing MCS over land and an MCS over the ocean with high probability, despite slight displacement errors. The exp37 EF improves the probability forecast of inland and offshore MCSs more than does NODA EF. Compared with the experiments using the longer DA time intervals, assimilating the radial-velocity observations at 6-min intervals tends to produce better forecasts. The experiment with the longest DA time span and shortest time interval shows the best performance. However, a shorter DA time interval (e.g., 12 min) or a longer DA time span does not always help. The experiment with the shortest DA time interval and maximum DA window shows the best performance, as it corrects errors in the simulated convection evolution over both the inland and offshore areas. An improved representation of the initial state leads to dynamic and thermodynamic conditions that are more conducive to earlier initiation of the inland MCS and longer maintenance of the offshore MCS.

The Local Ensemble Tangent Linear Model: an enabler for coupled model 4D‐Var

A leading Data Assimilation (DA) technique in meteorology is 4D‐Var which relies on the Tangent Linear Model (TLM) of the nonlinear model and its adjoint. The difficulty of building and maintaining traditional TLMs and adjoints of coupled ocean–wave–atmosphere–etc. models is daunting. On the other hand, coupled model ensemble forecasts are readily available. Here, we show how an ensemble forecast can be used to construct an accurate Local Ensemble TLM (LETLM) and adjoint of the entire coupled system. The method features a local influence region containing all the variables that could possibly influence the time evolution of some target variable(s) near the centre of the region. We prove that high accuracy is guaranteed provided that (i) the ensemble perturbations are governed by linear dynamics, and (ii) the number of ensemble members exceeds the number of variables in the influence region. The approach is illustrated in a simple coupled model. This idealized coupled model has some realistic features including reasonable predictability limits in the upper atmosphere, lower atmosphere, upper ocean and lower ocean of 10, 96, 160 and 335 days, respectively. In addition, the length‐scale of eddies in the ocean is about one fifth of those in the atmosphere. The easy manner in which the adjoint is obtained from the LETLM is also described and illustrated by demonstrating how the LETLM adjoint predicts the high sensitivity of oceanic boundary‐layer evolution to changes in the atmosphere. Finally, the feasibility of LETLMs for 4D‐Var is demonstrated. Specifically, a case is considered with a 5‐day data assimilation window in which nonlinear terms play a significant role in the evolution of forecast error; it is shown that the posterior mode delivered by 4D‐Var with an LETLM, its adjoint and ten outer loops approximately recovers the true state in spite of a spatially sparse observational network.

Four-dimensional ensemble variational data assimilation and the unstable subspace

The performance of (ensemble) Kalman filters used for data assimilation in the geosciences critically depends on the dynamical properties of the evolution model. A key aspect is that the error covariance matrix is asymptotically supported by the unstable–neutral subspace only, i.e. it is spanned by the backward Lyapunov vectors with non-negative exponents. The analytic proof of such a property for the Kalman filter error covariance has been recently given, and in particular that of its confinement to the unstable–neutral subspace. In this paper, we first generalize those results to the case of the Kalman smoother in a linear, Gaussian and perfect model scenario. We also provide square-root formulae for the filter and smoother that make the connection with ensemble formulations of the Kalman filter and smoother, where the span of the error covariance is described in terms of the ensemble deviations from the mean. We then discuss how this neat picture is modified when the dynamics are nonlinear and chaotic, and for which analytic results are precluded or difficult to obtain. A numerical investigation is carried out to study the approximate confinement of the anomalies for both a deterministic ensemble Kalman filter (EnKF) and a four-dimensional ensemble variational method, the iterative ensemble Kalman smoother (IEnKS), in a perfect model scenario. The confinement is characterized using geometrical angles that determine the relative position of the anomalies with respect to the unstable–neutral subspace. The alignment of the anomalies and of the unstable–neutral subspace is more pronounced when observation precision or frequency, as well as the data assimilation window length for the IEnKS, are increased. These results also suggest that the IEnKS and the deterministic EnKF realize in practice (albeit implicitly) the paradigm behind the approach of Anna Trevisan and co-authors known as the assimilation in the unstable subspace.

To what extent is your data assimilation scheme designed to find the posterior mean, the posterior mode or something else?

Recently there has been a surge in interest in coupling ensemble-based data assimilation methods with variational methods (commonly referred to as 4DVar). Here we discuss a number of important differences between ensemble-based and variational methods that ought to be considered when attempting to fuse these methods. We note that the Best Linear Unbiased Estimate (BLUE) of the posterior mean over a data assimilation window can only be delivered by data assimilation schemes that utilise the 4-dimensional (4D) forecast covariance of a prior distribution of non-linear forecasts across the data assimilation window. An ensemble Kalman smoother (EnKS) may be viewed as a BLUE approximating data assimilation scheme. In contrast, we use the dual form of 4DVar to show that the most likely non-linear trajectory corresponding to the posterior mode across a data assimilation window can only be delivered by data assimilation schemes that create counterparts of the 4D prior forecast covariance using a tangent linear model. Since 4DVar schemes have the required structural framework to identify posterior modes, in contrast to the EnKS, they may be viewed as mode approximating data assimilation schemes. Hence, when aspects of the EnKS and 4DVar data assimilation schemes are blended together in a hybrid, one would like to be able to understand how such changes would affect the mode- or mean-finding abilities of the data assimilation schemes. This article helps build such understanding using a series of simple examples. We argue that this understanding has important implications to both the interpretation of the hybrid state estimates and to their design.

Localization and the iterative ensemble Kalman smoother

The iterative ensemble Kalman smoother (IEnKS) is a data assimilation method meant for tracking the state of nonlinear geophysical models efficiently. It combines an ensemble of model states to estimate the errors similarly to the ensemble square‐root Kalman filter, with a four‐dimensional variational analysis performed within the ensemble space. As such, it belongs to the class of four‐dimensional ensemble variational methods. It could require the use of localization of the analysis when the state‐space dimension is high. However, its localization needs to be defined across time and to be as consistent as possible with the dynamical flow within the data assimilation window where the four‐dimensional variational analysis is performed. We show that a Liouville equation governs the time evolution of the localization operator, which is linked to the evolution of the error correlations. It is argued that the time evolution of the localization operator depends strongly on the forecast dynamics. Using either covariance localization or domain localization, we propose and test several localization strategies meant to address the issue: (i) a static and uniform localization, (ii) propagation through the window of a restricted set of dominant modes of the error covariance matrix and (iii) the approximate propagation of the localization operator using covariant local domains that are moved in accordance with the dynamical flow. These schemes are illustrated with the one‐dimensional Lorenz 40 variable model and with a two‐dimensional barotropic vorticity model. In both cases, local analysis based on the covariant local domains leads to a systematic improvement of the data assimilation performance.