Observation Error Covariance Matrices Research Articles

A comparison of channel selection for the assimilation of CrIS radiances for NWP

Abstract With hyperspectral instruments measuring radiation emitted by the earth and its atmosphere in the thermal infrared range in multiple channels, several studies were made to select a subset of channels in order to reduce the number of channels to be used in a data assimilation system. An optimal selection of channels based on information content depends on several factors related to observation and background error statistics and the assimilation system itself. An optimal channel selection for CrIS was obtained and then compared to selections made for different NWP systems. For instance, the channel selection of Carminati (2022) has 224 channels also present in to our optimal selection which includes 455 channels. However, in terms of analysis error variance, the difference between the two selections is small. Integrated over the whole profile, the relative difference is equal to 15.3 % and 4.5 % for temperature and humidity respectively. Also, different observation error covariance matrices were considered to evaluate the impact of this matrix on channel selection. Even though the channels selected optimally were different in terms of which channels were selected for the various R matrices.

Comparative Study of the Atmospheric Gas Composition Detection Capabilities of FY-3D/HIRAS-I and FY-3E/HIRAS-II Based on Information Capacity

Fengyun-3E (FY-3E)/Hyperspectral Infrared Atmospheric Sounder-II (HIRAS-II) is an extension Fengyun-3D (FY-3D)/HIRAS-I. It is crucial to fully explore and analyze the detection capabilities of these two instruments for atmospheric gas composition. Based on the observed spectral data from the infrared hyperspectral detection instruments FY-3D/HIRAS-I and FY-3E/HIRAS-II, simulated radiance data and Jacobian matrices are obtained using the Rapid Radiative Transfer Model RTTOV (Radiative Transfer for TOVS (TIROS Operational Vertical Sounder)). By perturbing temperature (T), surface temperature (Tsurf), water vapor (H2O), ozone (O3), carbon dioxide (CO2), methane (CH4), carbon monoxide (CO), and nitrous oxide (N2O), the brightness temperature differences before and after the perturbations are calculated to analyze the sensitivity of temperature and various atmospheric gas components. The Improved Optimal Sensitivity Profile (OSP) algorithm is used to select the channels for atmospheric gas retrieval. The observation error covariance and background error covariance matrices are calculated, and then the information capacity is calculated, specifically the degrees of freedom for signal(DFS) and the entropy reduction (ER). Based on this, a comparative analysis is conducted on the information capacity of atmospheric water vapor and ozone components contained in the hyperspectral detection data from HIRAS-I and HIRAS-II instruments, respectively, to explore the retrieval capabilities of the two instruments for atmospheric gas components. We selected clear-sky data from the African oceanic region and the Chinese Yangtze River Delta terrestrial region for quantitative analysis of the information capacity of HIRAS-I and HIRAS-II. The results show that FY-3D/HIRAS-I and FY-3E/HIRAS-II exhibit different sensitivities to atmospheric gas components. In different experimental regions, temperature and water vapor show the most dramatic sensitivity changes, followed by ozone, methane, and nitrous oxide, while carbon monoxide and carbon dioxide exhibit the lowest variability. Regarding channel selection, HIRAS-II identifies more gas channels compared to HIRAS-I. The experiments concluded that HIRAS-II has a significantly higher information capacity than HIRAS-I, and the information capacity of atmospheric gas components varies across different experimental regions. Water vapor and ozone exhibit the highest information capacity, followed by nitrous oxide and methane, while carbon monoxide and carbon dioxide demonstrate the lowest capacity. The H2O ER (DFS) contained in FY-3E/HIRAS-II is 1.51 (0.35) higher than that in FY-3D/HIRAS-I, the O3 ER (DFS) in FY-3E/HIRAS-II is 1.51 (0.36) higher than that in FY-3D/HIRAS-I, while the N2O ER (DFS) in FY-3E/HIRAS-II is 0.17 (0.19) higher and the CH4 ER (DFS) is 0.07 (0.04) higher than that in FY-3D/HIRAS-I.

Continuous Hyper-parameter OPtimization (CHOP) in an ensemble Kalman filter

Practical data assimilation algorithms often contain hyper-parameters, which may arise due to, for instance, the use of certain auxiliary techniques like covariance inflation and localization in an ensemble Kalman filter, the re-parameterization of certain quantities such as model and/or observation error covariance matrices, and so on. Given the richness of the established assimilation algorithms, and the abundance of the approaches through which hyper-parameters are introduced to the assimilation algorithms, one may ask whether it is possible to develop a sound and generic method to efficiently choose various types of (sometimes high-dimensional) hyper-parameters. This work aims to explore a feasible, although likely partial, answer to this question. Our main idea is built upon the notion that a data assimilation algorithm with hyper-parameters can be considered as a parametric mapping that links a set of quantities of interest (e.g., model state variables and/or parameters) to a corresponding set of predicted observations in the observation space. As such, the choice of hyper-parameters can be recast as a parameter estimation problem, in which our objective is to tune the hyper-parameters in such a way that the resulted predicted observations can match the real observations to a good extent. From this perspective, we propose a hyper-parameter estimation workflow and investigate the performance of this workflow in an ensemble Kalman filter. In a series of experiments, we observe that the proposed workflow works efficiently even in the presence of a relatively large amount (up to 103) of hyper-parameters, and exhibits reasonably good and consistent performance under various conditions.

Data Assimilation of Nowcasted Observations

Abstract We investigate the assimilation of nowcasted information into a classical data assimilation cycle. As a reference setup, we employ the assimilation of standard observations such as direct observations of particular variables into a forecasting system. The pure advective movement extrapolation of observations as a simple nowcasting (NWC) is usually much better for the first minutes to hours, until outperformed by numerical weather prediction (NWP) based on data assimilation. Can nowcasted information be used in the data assimilation cycle? We study both an oscillator model and the Lorenz 63 model with assimilation based on the localized ensemble transform Kalman filter (LETKF). We investigate and provide a mathematical framework for the assimilation of nowcasted information, approximated as a local tendency, into the LETKF in each assimilation step. In particular, we derive and discuss adequate observation error and background uncertainty covariance matrices and interpret the assimilation of nowcasted information as assimilation with an H1-type metric in observation space. Further, we show numerical results that prove that nowcasted information in data assimilation has the potential to significantly improve model based forecasting.

New bounds on the condition number of the Hessian of the preconditioned variational data assimilation problem

AbstractData assimilation algorithms combine prior and observational information, weighted by their respective uncertainties, to obtain the most likely posterior of a dynamical system. In variational data assimilation the posterior is computed by solving a nonlinear least squares problem. Many numerical weather prediction (NWP) centers use full observation error covariance (OEC) weighting matrices, which can slow convergence of the data assimilation procedure. Previous work revealed the importance of the minimum eigenvalue of the OEC matrix for conditioning and convergence of the unpreconditioned data assimilation problem. In this article we examine the use of correlated OEC matrices in the preconditioned data assimilation problem for the first time. We consider the case where there are more state variables than observations, which is typical for applications with sparse measurements, for example, NWP and remote sensing. We find that similarly to the unpreconditioned problem, the minimum eigenvalue of the OEC matrix appears in new bounds on the condition number of the Hessian of the preconditioned objective function. Numerical experiments reveal that the condition number of the Hessian is minimized when the background and observation lengthscales are equal. This contrasts with the unpreconditioned case, where decreasing the observation error lengthscale always improves conditioning. Conjugate gradient experiments show that in this framework the condition number of the Hessian is a good proxy for convergence. Eigenvalue clustering explains cases where convergence is faster than expected.

A Review of Innovation-Based Methods to Jointly Estimate Model and Observation Error Covariance Matrices in Ensemble Data Assimilation

AbstractData assimilation combines forecasts from a numerical model with observations. Most of the current data assimilation algorithms consider the model and observation error terms as additive Gaussian noise, specified by their covariance matrices and , respectively. These error covariances, and specifically their respective amplitudes, determine the weights given to the background (i.e., the model forecasts) and to the observations in the solution of data assimilation algorithms (i.e., the analysis). Consequently, and matrices significantly impact the accuracy of the analysis. This review aims to present and to discuss, with a unified framework, different methods to jointly estimate the and matrices using ensemble-based data assimilation techniques. Most of the methods developed to date use the innovations, defined as differences between the observations and the projection of the forecasts onto the observation space. These methods are based on two main statistical criteria: 1) the method of moments, in which the theoretical and empirical moments of the innovations are assumed to be equal, and 2) methods that use the likelihood of the observations, themselves contained in the innovations. The reviewed methods assume that innovations are Gaussian random variables, although extension to other distributions is possible for likelihood-based methods. The methods also show some differences in terms of levels of complexity and applicability to high-dimensional systems. The conclusion of the review discusses the key challenges to further develop estimation methods for and . These challenges include taking into account time-varying error covariances, using limited observational coverage, estimating additional deterministic error terms, or accounting for correlated noise.

Harmonization of Space-Borne Infra-Red Sensors Measuring Sea Surface Temperature

Sea surface temperature (SST) is observed by a constellation of sensors, and SST retrievals are commonly combined into gridded SST analyses and climate data records (CDRs). Differential biases between SSTs from different sensors cause errors in such products, including feature artefacts. We introduce a new method for reducing differential biases across the SST constellation, by reconciling the brightness temperature (BT) calibration and SST retrieval parameters between sensors. We use the Advanced Along-Track Scanning Radiometer (AATSR) and the Sea and Land Surface Temperature Radiometer (SLSTR) as reference sensors, and the Advanced Very High Resolution Radiometer (AVHRR) of the MetOp-A mission to bridge the gap between these references. Observations across a range of AVHRR zenith angles are matched with dual-view three-channel skin SST retrievals from the AATSR and SLSTR. These skin SSTs act as the harmonization reference for AVHRR retrievals by optimal estimation (OE). Parameters for the harmonized AVHRR OE are iteratively determined, including BT bias corrections and observation error covariance matrices as functions of water-vapor path. The OE SSTs obtained from AVHRR are shown to be closely consistent with the reference sensor SSTs. Independent validation against drifting buoy SSTs shows that the AVHRR OE retrieval is stable across the reference-sensor gap. We discuss that this method is suitable to improve consistency across the whole constellation of SST sensors. The approach will help stabilize and reduce errors in future SST CDRs, as well as having application to other domains of remote sensing.

Multiscale Representation of Observation Error Statistics in Data Assimilation.

Accounting for realistic observation errors is a known bottleneck in data assimilation, because dealing with error correlations is complex. Following a previous study on this subject, we propose to use multiscale modelling, more precisely wavelet transform, to address this question. This study aims to investigate the problem further by addressing two issues arising in real-life data assimilation: how to deal with partially missing data (e.g., concealed by an obstacle between the sensor and the observed system), and how to solve convergence issues associated with complex observation error covariance matrices? Two adjustments relying on wavelets modelling are proposed to deal with those, and offer significant improvements. The first one consists of adjusting the variance coefficients in the frequency domain to account for masked information. The second one consists of a gradual assimilation of frequencies. Both of these fully rely on the multiscale properties associated with wavelet covariance modelling. Numerical results on twin experiments show that multiscale modelling is a promising tool to account for correlations in observation errors in realistic applications.

The impact of using reconditioned correlated observation‐error covariance matrices in the Met Office 1D‐Var system

AbstractRecent developments in numerical weather prediction have led to the use of correlated observation‐error covariance (OEC) information in data assimilation and forecasting systems. However, diagnosed OEC matrices are often ill‐conditioned and may cause convergence problems for variational data assimilation procedures. Reconditioning methods are used to improve the conditioning of covariance matrices while retaining correlation information. In this article, we study the impact of using the “ridge regression” method of reconditioning to assimilate Infrared Atmospheric Sounding Interferometer (IASI) observations in the Met Office 1D‐Var system. This is the first systematic investigation of how changing target condition numbers affects convergence of a 1D‐Var routine. This procedure is used for quality control, and to estimate key variables (skin temperature, cloud‐top pressure, cloud fraction) that are not analysed by the main 4D‐Var data assimilation system. Our new results show that the current (uncorrelated) OEC matrix requires more iterations to reach convergence than any choice of correlated OEC matrix studied. This suggests that using a correlated OEC matrix in the 1D‐Var routine would have computational benefits for IASI observations. Using reconditioned correlated OEC matrices also increases the number of observations that pass quality control. However, the impact on skin temperature, cloud fraction, and cloud‐top pressure is less clear. As the reconditioning parameter is increased, differences between retrieved variables for correlated OEC matrices and the operational diagonal OEC matrix reduce. As correlated choices of OEC matrix yield faster convergence, using stricter convergence criteria along with these matrices may increase efficiency and improve quality control.

Improving the condition number of estimated covariance matrices

High dimensional error covariance matrices and their inverses are used to weight the contribution of observation and background information in data assimilation procedures. As observation error covariance matrices are often obtained by sampling methods, estimates are often degenerate or ill-conditioned, making it impossible to invert an observation error covariance matrix without the use of techniques to reduce its condition number. In this paper, we present new theory for two existing methods that can be used to ‘recondition’ any covariance matrix: ridge regression and the minimum eigenvalue method. We compare these methods with multiplicative variance inflation, which cannot alter the condition number of a matrix, but is often used to account for neglected correlation information. We investigate the impact of reconditioning on variances and correlations of a general covariance matrix in both a theoretical and practical setting. Improved theoretical understanding provides guidance to users regarding method selection, and choice of target condition number. The new theory shows that, for the same target condition number, both methods increase variances compared to the original matrix, with larger increases for ridge regression than the minimum eigenvalue method. We prove that the ridge regression method strictly decreases the absolute value of off-diagonal correlations. Theoretical comparison of the impact of reconditioning and multiplicative variance inflation on the data assimilation objective function shows that variance inflation alters information across all scales uniformly, whereas reconditioning has a larger effect on scales corresponding to smaller eigenvalues. We then consider two examples: a general correlation function, and an observation error covariance matrix arising from interchannel correlations. The minimum eigenvalue method results in smaller overall changes to the correlation matrix than ridge regression but can increase off-diagonal correlations. Data assimilation experiments reveal that reconditioning corrects spurious noise in the analysis but underestimates the true signal compared to multiplicative variance inflation.

Estimating the Correlated Observation-Error Characteristics of the Chinese FengYun Microwave Temperature Sounder and Microwave Humidity Sounder

In operational data assimilation systems, observation-error covariance matrices are commonly assumed to be diagonal. However, inter-channel and spatial observation-error correlations are inevitable for satellite radiances. The observation errors of the Microwave Temperature Sounder (MWTS) and Microwave Humidity Sounder (MWHS) onboard the FengYun-3A (FY-3A) and FY-3B satellites are empirically assigned and considered to be uncorrelated when they are assimilated into the WRF model’s Community Variational Data Assimilation System (WRFDA). To assimilate MWTS and MWHS measurements optimally, a good characterization of their observation errors is necessary. In this study, background and analysis residuals were used to diagnose the correlated observation-error characteristics of the MWTS and MWHS. It was found that the error standard deviations of the MWTS and MWHS were less than the values used in the WRFDA. MWTS had small inter-channel errors, while MWHS had significant inter-channel errors. The horizontal correlation length scales of MWTS and MWHS were about 120 and 60 km, respectively. A comparison between the diagnosis for instruments onboard the two satellites showed that the observation-error characteristics of the MWTS or MWHS were different when they were onboard different satellites. In addition, it was found that the error statistics were dependent on latitude and scan positions. The forecast experiments showed that using a modified thinning scheme based on diagnosed statistics can improve forecast accuracy.

Optimal Interpolation of Precipitable Water Using Low Earth Orbit and Numerical Weather Prediction Data

The National Meteorological Satellite Center/Korean Meteorological Administration (NMSC/KMA) receives data directly from low Earth orbit (LEO) satellites (including NOAA-18,19; MetOp-A,B; and Suomi-NPP), and generates Level 2 products (e.g., temperature and humidity profile) in near real time. Total precipitable water (TPW) and layer precipitable water (LPW) are also generated using the retrieved humidity profiles. Today, forecasters need meteorologically-significant data fields composited from all available data sources, not multiple maps of observations from individual sources. Hence, TPW and LPW are reproduced using the optimal interpolation (OI) method with numerical weather prediction (NWP) data, in order to generate composite precipitable water (PW) products. In the OI procedure, PW data retrieved from the LEO satellites serve as observation data, while PW data from NWP serve as background data. Error covariances are estimated using a new approach, which considers correlations between observation errors to describe the characteristics of the errors better. Both background and observation error covariance matrices may have non-zero off-diagonal components. The composite PW products are validated using radiosonde data. The validation results for optimally-interpolated LPW (OI LPW) are much better than those for optimally-interpolated TPW (OI TPW). Generally, the OI LPW validation results are better than those for observation and background data; OI LPW data are ~5–10% more accurate than background data. Optimally-interpolated PW (OI PW) fields are applied to the correction of NWP forecast fields and the prediction of severe weather.

The conditioning of least‐squares problems in variational data assimilation

SummaryIn variational data assimilation a least‐squares objective function is minimised to obtain the most likely state of a dynamical system. This objective function combines observation and prior (or background) data weighted by their respective error statistics. In numerical weather prediction, data assimilation is used to estimate the current atmospheric state, which then serves as an initial condition for a forecast. New developments in the treatment of observation uncertainties have recently been shown to cause convergence problems for this least‐squares minimisation. This is important for operational numerical weather prediction centres due to the time constraints of producing regular forecasts. The condition number of the Hessian of the objective function can be used as a proxy to investigate the speed of convergence of the least‐squares minimisation. In this paper we develop novel theoretical bounds on the condition number of the Hessian. These new bounds depend on the minimum eigenvalue of the observation error covariance matrix and the ratio of background error variance to observation error variance. Numerical tests in a linear setting show that the location of observation measurements has an important effect on the condition number of the Hessian. We identify that the conditioning of the problem is related to the complex interactions between observation error covariance and background error covariance matrices. Increased understanding of the role of each constituent matrix in the conditioning of the Hessian will prove useful for informing the choice of correlated observation error covariance matrix and observation location, particularly for practical applications.

Impacts on sea ice analyses from the assumption of uncorrelated ice thickness observation errors: Experiments using a 1D toy model

Sea ice prediction centres are moving toward the assimilation of ice thickness observations under the simplifying assumption that the observation errors are uncorrelated. The assumption of uncorrelated observation errors is attractive because the errors can be represented by a diagonal observation error covariance matrix, which is inexpensive to invert. In this paper a set of idealized experiments are carried out to investigate the impact of this assumption on sea ice analyses. A background error covariance matrix is generated using a 1D toy model for sea, i.e. forced with idealized models of the ocean and atmosphere. Analysis error covariance matrices are then calculated using this matrix for both correlated and uncorrelated observation error covariance matrices, . The results indicate when the true is correlated, using a diagonal approximation results in an analysis that is overconfident at the large scales, in that the analysis error standard deviation at the large scales is underestimated. It is also shown that for the largest observation error correlation length scale tested, 150 km, the analysis error standard deviation for ice thickness is reduced by 10.8% relative to the background error standard deviation when has the correct correlation length scale of 150 km, whereas when a diagonal approximation to is used in combination with an inflation factor, the reduction is to 6.3%.

Accounting for Correlated Observation Error in a Dual-Formulation 4D Variational Data Assimilation System

Appropriate specification of the error statistics for both observational data and short-term forecasts is necessary to produce an optimal analysis. Observation error stems from instrument error, forward model error, and error of representation. All sources of observation error, particularly error of representation, can lead to nonzero correlations. While correlated forecast error has been accounted for since the early days of atmospheric data assimilation, observation error has typically been treated as uncorrelated until relatively recently. Thinning, averaging, and/or inflation of the assigned observation error variance have been employed to compensate for unaccounted error correlations, especially for high-resolution satellite data. In this study, the benefits of accounting for nonzero vertical (interchannel) correlation for both the Advanced Technology Microwave Satellite (ATMS) and Infrared Atmospheric Sounding Interferometer (IASI) in the NRL Atmospheric Variational Data Assimilation System-Accelerated Representer (NAVDAS-AR) are assessed. The vertical observation error covariance matrix for the ATMS and IASI instruments was estimated using the Desroziers method. The results suggest lowering the assigned error variance and introducing strong correlations, especially in the moisture-sensitive channels. Strong positive impact on forecast skill (verified against both the ECMWF analyses and high-quality radiosonde data) is shown in both the ATMS and IASI instruments. Additionally, the convergence of the iterative solver in NAVDAS-AR can be improved by small modifications to the observation error covariance matrices, resulting in further reduction in RMS error.

An Efficient Way to Account for Observation Error Correlations in the Assimilation of Data from the Future SWOT High-Resolution Altimeter Mission

AbstractMost data assimilation algorithms require the inverse of the covariance matrix of the observation errors. In practical applications, the cost of computing this inverse matrix with spatially correlated observation errors is prohibitive. Common practices are therefore to subsample or combine the observations so that the errors of the assimilated observations can be considered uncorrelated. As a consequence, a large fraction of the available observational information is not used in practical applications. In this study, a method is developed to account for the correlations of the errors that will be present in the wide-swath sea surface height measurements, for example, the Surface Water and Ocean Topography (SWOT) mission. It basically consists of the transformation of the observation vector so that the inverse of the corresponding covariance matrix can be replaced by a diagonal matrix, thus allowing to genuinely take into account errors that are spatially correlated in physical space. Numerical experiments of ensemble Kalman filter analysis of SWOT-like observations are conducted with three different observation error covariance matrices. Results suggest that the proposed method provides an effective way to account for error correlations in the assimilation of the future SWOT data. The transformation of the observation vector proposed herein yields both a significant reduction of the root-mean-square errors and a good consistency between the filter analysis error statistics and the true error statistics.

EnKF and 4D-Var data assimilation with chemical transport model BASCOE (version 05.06)

Abstract. We compare two optimized chemical data assimilation systems, one based on the ensemble Kalman filter (EnKF) and the other based on four-dimensional variational (4D-Var) data assimilation, using a comprehensive stratospheric chemistry transport model (CTM). This work is an extension of the Belgian Assimilation System for Chemical ObsErvations (BASCOE), initially designed to work with a 4D-Var data assimilation. A strict comparison of both methods in the case of chemical tracer transport was done in a previous study and indicated that both methods provide essentially similar results. In the present work, we assimilate observations of ozone, HCl, HNO3, H2O and N2O from EOS Aura-MLS data into the BASCOE CTM with a full description of stratospheric chemistry. Two new issues related to the use of the full chemistry model with EnKF are taken into account. One issue is a large number of error variance parameters that need to be optimized. We estimate an observation error variance parameter as a function of pressure level for each observed species using the Desroziers method. For comparison purposes, we apply the same estimate procedure in the 4D-Var data assimilation, where both scale factors of the background and observation error covariance matrices are estimated using the Desroziers method. However, in EnKF the background error covariance is modelled using the full chemistry model and a model error term which is tuned using an adjustable parameter. We found that it is adequate to have the same value of this parameter based on the chemical tracer formulation that is applied for all observed species. This is an indication that the main source of model error in chemical transport model is due to the transport. The second issue in EnKF with comprehensive atmospheric chemistry models is the noise in the cross-covariance between species that occurs when species are weakly chemically related at the same location. These errors need to be filtered out in addition to a localization based on distance. The performance of two data assimilation methods was assessed through an 8-month long assimilation of limb sounding observations from EOS Aura MLS. This paper discusses the differences in results and their relation to stratospheric chemical processes. Generally speaking, EnKF and 4D-Var provide results of comparable quality but differ substantially in the presence of model error or observation biases. If the erroneous chemical modelling is associated with moderately fast chemical processes, but whose lifetimes are longer than the model time step, then EnKF performs better, while 4D-Var develops spurious increments in the chemically related species. If, however, the observation biases are significant, then 4D-Var is more robust and is able to reject erroneous observations while EnKF does not.

Investigating the role of prior and observation error correlations in improving a model forecast of forest carbon balance using Four-dimensional Variational data assimilation

Efforts to implement variational data assimilation routines with functional ecology models and land surface models have been limited, with sequential and Markov chain Monte Carlo data assimilation methods being prevalent. When data assimilation has been used with models of carbon balance, prior or “background” errors (in the initial state and parameter values) and observation errors have largely been treated as independent and uncorrelated. Correlations between background errors have long been known to be a key aspect of data assimilation in numerical weather prediction. More recently, it has been shown that accounting for correlated observation errors in the assimilation algorithm can considerably improve data assimilation results and forecasts. In this paper we implement a Four-dimensional Variational data assimilation (4D-Var) scheme with a simple model of forest carbon balance, for joint parameter and state estimation and assimilate daily observations of Net Ecosystem CO2 Exchange (NEE) taken at the Alice Holt forest CO2 flux site in Hampshire, UK. We then investigate the effect of specifying correlations between parameter and state variables in background error statistics and the effect of specifying correlations in time between observation errors. The idea of including these correlations in time is new and has not been previously explored in carbon balance model data assimilation. In data assimilation, background and observation error statistics are often described by the background error covariance matrix and the observation error covariance matrix. We outline novel methods for creating correlated versions of these matrices, using a set of previously postulated dynamical constraints to include correlations in the background error statistics and a Gaussian correlation function to include time correlations in the observation error statistics. The methods used in this paper will allow the inclusion of time correlations between many different observation types in the assimilation algorithm, meaning that previously neglected information can be accounted for. In our experiments we assimilate a single year of NEE observations and then run a forecast for the next 14 years. We compare the results using our new correlated background and observation error covariance matrices and those using diagonal covariance matrices. We find that using the new correlated matrices reduces the root mean square error in the 14 year forecast of daily NEE by 44% decreasing from 4.22gCm−2day−1 to 2.38gCm−2day−1.

Accounting for observation errors in image data assimilation

This paper deals with the assimilation of image-type data. Such kinds of data, such as satellite images, have good properties (dense coverage in space and time), but also one crucial problem for data assimilation: they are affected by spatially correlated errors. Classical approaches in data assimilation assume uncorrelated noise, because the proper description and numerical manipulation of non-diagonal error covariance matrices is complex. This paper proposes a simple way to provide observation error covariance matrices adapted to spatially correlated errors. This is done using various image transformations: multiscale (Wavelets, Fourier, Curvelets), gradients, and gradient orientations. These transformations are described and compared to classical approaches, such as pixel-to-pixel comparison and observation thinning. We provide simple yet effective covariance matrices for each of these transformations, which take into account the observation error correlations and improve the results. The effectiveness of the proposed approach is demonstrated on twin experiments performed on a 2-D shallow-water model.

Dynamic statistical optimization of GNSS radio occultation bending angles: advanced algorithm and performance analysis

Abstract. We introduce a new dynamic statistical optimization algorithm to initialize ionosphere-corrected bending angles of Global Navigation Satellite System (GNSS)-based radio occultation (RO) measurements. The new algorithm estimates background and observation error covariance matrices with geographically varying uncertainty profiles and realistic global-mean correlation matrices. The error covariance matrices estimated by the new approach are more accurate and realistic than in simplified existing approaches and can therefore be used in statistical optimization to provide optimal bending angle profiles for high-altitude initialization of the subsequent Abel transform retrieval of refractivity. The new algorithm is evaluated against the existing Wegener Center Occultation Processing System version 5.6 (OPSv5.6) algorithm, using simulated data on two test days from January and July 2008 and real observed CHAllenging Minisatellite Payload (CHAMP) and Constellation Observing System for Meteorology, Ionosphere, and Climate (COSMIC) measurements from the complete months of January and July 2008. The following is achieved for the new method's performance compared to OPSv5.6: (1) significant reduction of random errors (standard deviations) of optimized bending angles down to about half of their size or more; (2) reduction of the systematic differences in optimized bending angles for simulated MetOp data; (3) improved retrieval of refractivity and temperature profiles; and (4) realistically estimated global-mean correlation matrices and realistic uncertainty fields for the background and observations. Overall the results indicate high suitability for employing the new dynamic approach in the processing of long-term RO data into a reference climate record, leading to well-characterized and high-quality atmospheric profiles over the entire stratosphere.