Correlated Observation Errors Research Articles

Assimilating Observations with Spatially Correlated Errors Using a Serial Ensemble Filter with a Multiscale Approach

AbstractThe serial ensemble square root filter (EnSRF) typically assumes observation errors to be uncorrelated when assimilating the observations one at a time. This assumption causes the filter solution to be suboptimal when the observation errors are spatially correlated. Using the Lorenz-96 model, this study evaluates the suboptimality due to mischaracterization of observation error spatial correlations. Neglecting spatial correlations in observation errors results in mismatches between the specified and true observation error variances in spectral space, which cannot be resolved by inflating the overall observation error variance. As a remedy, a multiscale observation (MSO) method is proposed to decompose the observations into multiple scale components and assimilate each component with separately adjusted spectral error variance. Experimental results using the Lorenz-96 model show that the serial EnSRF, with the help from the MSO method, can produce solutions that approach the solution from the EnSRF with correctly specified observation error correlations as the number of scale components increases. The MSO method is further tested in a two-layer quasigeostrophic (QG) model framework. In this case, the MSO method is combined with the multiscale localization (MSL) method to allow the use of different localization radii when updating the model state at different scales. The combined method (MSOL) improves the serial EnSRF performance when assimilating observations with spatially correlated errors. With adjusted observation error spectral variances and localization radii, the combined MSOL method provides the best solution in terms of analysis accuracy and filter consistency. Prospects and challenges are also discussed for the implementation of the MSO method for more complex models and observing networks.

Update of Infrared Atmospheric Sounding Interferometer (IASI) channel selection with correlated observation errors for numerical weather prediction (NWP)

Abstract. The Infrared Atmospheric Sounding Interferometer (IASI) is an essential instrument for numerical weather prediction (NWP). It measures radiances at the top of the atmosphere using 8461 channels. The huge amount of observations provided by IASI has led the community to develop techniques to reduce observations while conserving as much information as possible. Thus, a selection of the 300 most informative channels was made for NWP based on the concept of information theory. One of the main limitations of this method was to neglect the covariances between the observation errors of the different channels. However, many centres have shown a significant benefit for weather forecasting to use them. Currently, the observation-error covariances are only estimated on the current IASI channel selection, but no studies to make a new selection of IASI channels taking into account the observation-error covariances have yet been carried out. The objective of this paper was therefore to perform a new selection of IASI channels by taking into account the observation-error covariances. The results show that with an equivalent number of channels, accounting for the observation-error covariances, a new selection of IASI channels can reduce the analysis error on average in temperature by 3 %, humidity by 1.8 % and ozone by 0.9 % compared to the current selection. Finally, we go one step further by proposing a robust new selection of 400 IASI channels to further reduce the analysis error for NWP.

Use of Uncertainty Inflation in OSTIA to Account for Correlated Errors in Satellite-Retrieved Sea Surface Temperature Data

Sea surface temperature (SST) analysis systems such as the Operational Sea Surface Temperature and Ice Analysis (OSTIA) use statistical methods to combine observations together with a first guess field to create spatially complete maps of SST. These commonly assume that observation errors are uncorrelated, yet some errors (such as due to retrieval issues) can be correlated. Information about errors is used by the analysis system to determine the weighting to apply to the observations, hence this incorrect assumption could degrade the analysis. A common technique to mitigate for this is to inflate the observation uncertainties. Using information on observation error correlations provided with data produced by the European Space Agency (ESA) SST Climate Change Initiative (CCI) project, idealised tests were carried out to determine how this inflation technique can best be applied. These showed that applying inflation in situations where the observation errors are correlated over similar or larger distances to the errors in the background can cause unpredictable and sometimes negative results. However, in situations where the observation error correlation length scale is relatively small, inflation should improve the analysis. These findings were adapted to the OSTIA system and various configurations were tested. It was found that the inflation methods did not affect statistics of differences between the analyses and independent Argo reference data. However, the SST gradients were affected, particularly if some observation uncertainties were inflated but others were not. The results from both the idealised tests and the application to the real system therefore highlight that it is challenging to implement the inflation method in the case of an SST analysis system and show the need for assimilation schemes that can make full use of observation error correlation information.

A practical assimilation approach to extract smaller‐scale information from observations with spatially correlated errors: An idealized study

AbstractIt is still common to neglect the spatial error correlations of assimilated observations in numerical weather prediction systems because no practical approach is available to account for them when the number of observations with correlated error is large or when these observations are non‐uniformly distributed. Instead, it is common practice to inflate observation error variances to avoid overfitting large scales and spatially thin observations to reduce error correlations between remaining observations, although both methods generally sacrifice small‐scale information. Inspired by previous work on assimilating the difference between adjacent observations (so‐called spatial difference observations), this study aims at combining direct observations with spatial difference observations in the assimilation to extract both large‐ and smaller‐scale information from observations with spatially correlated errors, while still neglecting these error correlations. Experiments performed in a simplified 1D context over a periodic domain show that the combined approach is numerically equivalent to directly assimilating observations available at every grid point using non‐diagonal observation error covariances based on a first‐order autoregressive correlation function. In a case where observation error correlations have a different structure (e.g. Gaussian), the true observation error correlations are not perfectly taken into account by the combined approach, but it still efficiently extracts information to correct the scales that have the most errors. Combining direct observations with spatial difference observations proves complementary and experimental results show lower analysis errors for both large and intermediate scales. More specifically, while neglecting spatial observation error correlations, the combined approach provides results with lower analysis error than the direct approach, especially when spatial error correlations are large and when direct observations are spatially thinned.

Evaluating prior predictions of production and seismic data

It is common in ensemble-based methods of history matching to evaluate the adequacy of the initial ensemble of models through visual comparison between actual observations and data predictions prior to data assimilation. If the model is appropriate, then the observed data should look plausible when compared to the distribution of realizations of simulated data. The principle of data coverage alone is, however, not an effective method for model criticism, as coverage can often be obtained by increasing the variability in a single model parameter. In this paper, we propose a methodology for determining the suitability of a model before data assimilation, particularly aimed for real cases with large numbers of model parameters, large amounts of data, and correlated observation errors. This model diagnostic is based on an approximation of the Mahalanobis distance between the observations and the ensemble of predictions in high-dimensional spaces. We applied our methodology to two different examples: a Gaussian example which shows that our shrinkage estimate of the covariance matrix is a better discriminator of outliers than the pseudo-inverse and a diagonal approximation of this matrix; and an example using data from the Norne field. In this second test, we used actual production, repeat formation tester, and inverted seismic data to evaluate the suitability of the initial reservoir simulation model and seismic model. Despite the good data coverage, our model diagnostic suggested that model improvement was necessary. After modifying the model, it was validated against the observations and is now ready for history matching to production and seismic data. This shows that the proposed methodology for the evaluation of the adequacy of the model is suitable for large realistic problems.

Non-local Observations and Information Transfer in Data Assimilation

Nonlocal observations are observations that cannot be allocated one specific spatial location. Examples are observations that are spatial averages of linear or nonlinear functions of system variables. In conventional data assimilation such as (ensemble) Kalman Filters and variational methods information transfer between observations and model variables is governed by covariance matrices that are either preset or determined from the dynamical evolution of the system. In many science fields the covariance structures have limited spatial extent, and this paper discusses what happens when this spatial extent is smaller then the support of the observation operator that maps state space to observations space. It is shown that information is carried beyond the physical information in the prior covariance structures by the nonlocal observational constraints, building an information bridge (or information channel) that has not been studied before: the posterior covariance can have nonzero covariance structures where the prior has a covariance of zero. It is shown that in standard data-assimilation techniques that enforce a covariance structure and limit information transfer to that structure the order in which local and nonlocal observations are assimilated can have a large influence on the analysis. Local observations should be assimilated first. This relates directly to localisation used in Ensemble Kalman Filters and Smoothers, but also to variational methods with a prescribed covariance structure where observations are assimilated in batches. This suggests that the emphasis on covariance modelling should shift away from the prior covariance and towards the modelling of the covariances between model and observation space. Furthermore, it is shown that local observations with non-locally correlated observation errors behave in the same way as uncorrelated observations that are nonlocal. Several theoretical results are illustrated with simple numerical examples. The significance of the information bridge provided by nonlocal observations is highlighted further through discussions of temporally nonlocal observations, and new ideas on targeted observations.

Seismic data assimilation with an imperfect model

Data assimilation methods often assume perfect models and uncorrelated observation error. The assumption of a perfect model is probably always wrong for applications to real systems, and since model error is known to generally induce correlated effective observation errors, then the common assumption of uncorrelated observation errors is probably almost always wrong, too. The standard approach to dealing with correlated observation errors, which simply ignores the correlation, leads to suboptimal assimilation of observations. In this paper, we examine the consequences of model errors on assimilation of seismic data. We show how to recognize the existence of correlated error through model diagnostics modified for large numbers of data, how to estimate the correlation in the error, and how to use a model with correlated errors in a perturbed observation form of an iterative ensemble smoother to improve the quantification of a posteriori uncertainty. The methodologies for each of these aspects have been developed to allow application to problems with very large number of model parameters and large amounts of data with correlated observation error. We applied the methodologies to a small toy problem with linear relationship between data and model parameters, and to synthetic seismic data from the Norne Field model. To provide a controlled investigation in the seismic example, we investigate an application of data assimilation with two sources of model error—errors in seismic resolution and errors in the petro-elastic model. Both types of model errors result in effectively correlated observation errors, which must be accounted for in the data assimilation scheme. Although the data are synthetic, parameters of the seismic resolution and the observation noise are estimated from the actual inverted acoustic impedance data. Using a structured approach, we are able to assimilate approximately 115,000 observations with correlated total observation error efficiently without neglecting correlations. We show that the application of this methodology leads to less overfitting to the observations, and results in an ensemble estimate with smaller spread than the initial ensemble of predictions, but that the final estimate of uncertainty is consistent with the truth.

Correlated observation error models for assimilating all-sky infrared radiances

Abstract. The benefit of hyperspectral infrared sounders to weather forecasting has been improved with the representation of inter-channel correlations in the observation error model. A further step would be to assimilate these observations in all-sky conditions. However, in cloudy skies, observation errors exhibit much stronger inter-channel correlations, as well as much larger variances, compared to clear-sky conditions. An observation error model is developed to represent these effects, building from the symmetric error models developed for all-sky microwave assimilation. The combination of variational quality control with correlated errors is also introduced. The new error model is tested in all-sky assimilation of seven water vapour sounding channels from the Infrared Atmospheric Sounding Interferometer (IASI). However, its initial formulation degrades both tropospheric and stratospheric analyses. To explain this, the eigendeparture and eigenjacobian are introduced as a way of understanding the effect of correlated observation errors in data assimilation. The trailing eigenvalues can be problematic because they strongly amplify high-order harmonic combinations of the water vapour channels, which could have at least three consequences. First, if there are small inter-channel biases, these can be greatly amplified. Second, the trailing eigenjacobians map onto features resembling gravity waves that the data assimilation may not be able to handle. Finally, these harmonic combinations can amplify trace sensitivities, for example, revealing a strong upper stratospheric sensitivity over high cloud in what are usually mid- to upper-tropospheric water vapour channels. A likely explanation is the sensitivity to gravity wave features that are present in the observations but hard for the data assimilation to handle. After reducing the sensitivity to the trailing eigenjacobians, the new error covariance matrix gives good results in all-sky infrared assimilation.

A pragmatic strategy for implementing spatially correlated observation errors in an operational system: An application to Doppler radial winds

Recent research has shown that high‐resolution observations, such as Doppler radar radial winds, exhibit spatial correlations. High‐resolution observations are routinely assimilated into convection‐permitting numerical weather prediction models assuming their errors are uncorrelated. To avoid violating this assumption, observation density is severely reduced. To improve the quantity of observations used and the impact that they have on the forecast requires the introduction of full, correlated, error statistics. Some operational centres have introduced satellite inter‐channel observation‐error correlations and obtained improved analysis accuracy and forecast skill scores. Here we present a strategy for implementing spatially correlated observation errors in an operational system. We then provide the first demonstration of the practical feasibility of incorporating spatially correlated Doppler radial wind error statistics in the Met Office numerical weather prediction system.Inclusion of correlated Doppler radial winds error statistics has little impact on the computation cost of the data assimilation system, even with a fourfold increase in the number of Doppler radial winds observations assimilated. Using the correlated observation‐error statistics with denser observations produces increments with shorter length‐scales than the control. Initial forecast trials show a neutral to positive impact on forecast skill overall, notably for quantitative precipitation forecasts. There is potential to improve forecast skill by optimizing the use of Doppler radial winds and applying the technique to other observation types.

Modelling spatially correlated observation errors in variational data assimilation using a diffusion operator on an unstructured mesh

We propose a method for representing spatially correlated observation errors in variational data assimilation. The method is based on the numerical solution of a diffusion equation, a technique commonly used for representing spatially correlated background errors. The discretization of the pseudo‐time derivative of the diffusion equation is done implicitly using a backward Euler scheme. The solution of the resulting elliptic equation can be interpreted as a correlation operator whose kernel is a correlation function from the Matérn family.In order to account for the possibly heterogeneous distribution of observations, a spatial discretization technique based on the finite element method (FEM) is chosen where the observation locations are used to define the nodes of an unstructured mesh on which the diffusion equation is solved. By construction, the method leads to a convenient operator for the inverse of the observation‐error correlation matrix, which is an important requirement when applying it with standard minimization algorithms in variational data assimilation. Previous studies have shown that spatially correlated observation errors can also be accounted for by assimilating the observations together with their directional derivatives up to arbitrary order. In the continuous framework, we show that the two approaches are formally equivalent for certain parameter specifications. The FEM provides an appropriate framework for evaluating the derivatives numerically, especially when the observations are heterogeneously distributed.Numerical experiments are performed using a realistic data distribution from the Spinning Enhanced Visible and InfraRed Imager (SEVIRI). Correlations obtained with the FEM‐discretized diffusion operator are compared with those obtained using the analytical Matérn correlation model. The method is shown to produce an accurate representation of the target Matérn function in regions where the data are densely distributed. The presence of large gaps in the data distribution degrades the quality of the mesh and leads to numerical errors in the representation of the Matérn function. Strategies to improve the accuracy of the method in the presence of such gaps are discussed.

Data compression in the presence of observational error correlations

Numerical weather prediction (NWP) models are moving towards km-scale (and smaller) resolutions in order to forecast high-impact weather. As the resolution of NWP models increase the need for high-resolution observations to constrain these models also increases. A major hurdle to the assimilation of dense observations in NWP is the presence of non-negligible observation error correlations (OECs). Despite the difficulty in estimating these error correlations, progress is being made, with centres around the world now explicitly accounting for OECs in a variety of observation types. This paper explores how to make efficient use of this potentially dramatic increase in the amount of data available for assimilation. In an idealised framework it is illustrated that as the length-scales of the OECs increase the scales that the analysis is most sensitive to the observations become smaller. This implies that a denser network of observations is more beneficial with increasing OEC length-scales. However, the computational and storage burden associated with such a dense network may not be feasible. To reduce the amount of data, a compression technique based on retaining the maximum information content of the observations can be used. When the OEC length-scales are large (in comparison to the prior error correlations), the data compression will select observations of the smaller scales for assimilation whilst throwing out the larger scale information. In this case it is shown that there is a discrepancy between the observations with the maximum information and those that minimise the analysis error variances. Experiments are performed using the Ensemble Kalman Filter and the Lorenz-1996 model, comparing different forms of data reduction. It is found that as the OEC length-scales increase the assimilation becomes more sensitive to the choice of data reduction technique.

Estimation of ground‐based GNSS Zenith Total Delay temporal observation error correlations using data from the NOAA and E‐GVAP networks

In preparation for possible higher frequency assimilation of ground‐based Global Navigation Satellite System (GB‐GNSS) Zenith Total Delay (ZTD) observations at Environment and Climate Change Canada (ECCC), two popular diagnostic methods are applied to observation‐minus‐background and observation‐minus‐analysis departures from the ECCC Global Deterministic Prediction System (GDPS) to estimate temporal ZTD observation error correlations within the 6 h assimilation window. The GDPS uses a four‐dimensional ensemble‐variational (4D‐EnVar) data assimilation system with a background error covariance matrix (B‐matrix) obtained from an equal blend of 3D static and 4D flow‐dependent (ensemble‐based) background error covariances. The two diagnostic methods, the Desroziers method and the Hollingsworth–Lönnberg method, are applied using ZTD observations from the North American NOAA network and the European EIG EUMETNET GNSS Water Vapour Programme (E‐GVAP) network with similar results obtained for both networks. Correlations estimated with the Desroziers method drop off rapidly with time difference becoming significantly negative for observations separated by more than 2–3 h. The results are unexpected as there is no obvious physical basis for negative correlations. It is shown how suboptimal specification of the background error covariances in the 4D‐EnVar assimilation system B‐matrix could explain the unexpected aspects of the results obtained with this method. In contrast, diagnosed observation error covariances without negative temporal correlations are obtained with the Hollingsworth–Lönnberg method, where a decorrelation time‐scale on the order of 4 h is found. Both methods capture some expected features of the error correlations, such as highly correlated errors between observations within the same 1 h data processing blocks. While considerable uncertainty is associated with the results due to the inherent assumptions and limitations of both methods, the results do suggest that ZTD observation error decorrelation time‐scales are on the order of a few hours rather than days as suggested by early pioneering work with different error correlation estimation methods.

Considering the impact of observation error correlation in ensemble square-root Kalman filter

Data assimilation has been developed into an effective technology that can utilize a large number of multisource unconventional data. It cannot only provide the initial field for the ocean numerical prediction model, but also construct the ocean reanalysis datasets and provide the design basis for the ocean observation plan. In data assimilation, the estimation of the observation error is of paramount importance, because the quality of the analysis depends on it. In general, the observation error covariance matrix is diagonal or assumed to be diagonal, which means that the observation errors are independent from one another. However, there are indeed correlations in the observation errors. A diagnostic method has been developed, which can estimate a correlated and more accurate observation error covariance matrix. The proposed method combines an ensemble squareroot Kalman filter with the diagnostic method, providing an estimation of the observation error covariance matrix. In order to test the performance of the method, the numerical experiments are performed with the Lorenz 96 model and a Shallow water model. The more accurate observation error covariance matrix can be obtained to use in ensemble square-root Kalman filter by using the new method. We could find using the estimated correlated observation error in the data assimilation improves the analysis.

Data fusion and data assimilation of ice thickness observations using a regularisation framework

Accurate estimates of sharp features in the sea ice cover, such as leads and ridges, are critical for shipping activities, ice operations and weather forecasting. These sharp features can be difficult to preserve in data fusion and data assimilation due to the spatial correlations in the background error covariance matrices. In this article, a set of data fusion and data assimilation experiments are carried out comparing two objective functions, one with a conventional l2-norm and one that imposes an additional l1-norm on the derivative of the ice thickness state estimate. The latter is motivated by analysis of high resolution ice thickness observations derived from an airborne electromagnetic sensor demonstrating the sparsity of the ice thickness in the derivative domain. Data fusion and data assimilation experiments (using a 1 D toy sea-ice model) are carried out over a wide range of background and observation error correlation length scales. Results show the superiority of using an l1–l2 regularisation framework. For the data fusion experiments it was found when both background and observation error correlation length scales are zero, the ice thickness root mean squared error for the l1–l2 method was 0.16 m as compared to 0.20 m for the l2 method. The differences between the methods were greater when the background error correlation length scale was relatively short (approximately five times the analysis grid spacing), and were not significant for larger background error correlation length scales (e.g. 10 times the analysis grid spacing). For data assimilation experiments it was found that openings in the ice cover were captured better with the l1–l2 regularisation, with reduced errors in ice thickness, concentration and velocity. In addition, the ice thickness derivatives in the analyses were found to be more sparse when the l1–l2 method was used and are closer to the those from the true model run.

Convergence Issues in the Estimation of Interchannel Correlated Observation Errors in Infrared Radiance Data

Abstract A posteriori consistency diagnostics have been used in recent years to estimate correlated observation error. These diagnostics provide an estimate of what the observation error covariances should be and could, in turn, be introduced in the assimilation to improve the statistical consistency between the error statistics used in the assimilation and those obtained from observation departures with respect to the background and the analysis. To estimate the observation error covariances, it is often assumed that the background error statistics are optimal, an assumption that is open to criticism. The consequence is that if the background error covariances are in error, then the estimated observation error statistics will adjust accordingly to fit the innovation error covariances. In this paper, the RTTOV radiative transfer model is used as the observation operator. Using controlled experiments, the background error is considered fixed, and it is shown that the iterative procedure to estimate the observation error may require more than one iteration. It is also shown that the underlying matrix equation being solved can be factorized, and the exact solution can be obtained. If the true background error covariances are used in the assimilation, the estimated observation error covariances are then obtained by subtracting the background error covariances from those of the innovations. This can be applied to the full set of assimilated observations. Using the Environment Canada assimilation system, the results for several types of observations indicate that the background error estimation would deserve additional attention.

Estimating the Correlated Observation-Error Characteristics of the Chinese FengYun Microwave Temperature Sounder and Microwave Humidity Sounder

In operational data assimilation systems, observation-error covariance matrices are commonly assumed to be diagonal. However, inter-channel and spatial observation-error correlations are inevitable for satellite radiances. The observation errors of the Microwave Temperature Sounder (MWTS) and Microwave Humidity Sounder (MWHS) onboard the FengYun-3A (FY-3A) and FY-3B satellites are empirically assigned and considered to be uncorrelated when they are assimilated into the WRF model’s Community Variational Data Assimilation System (WRFDA). To assimilate MWTS and MWHS measurements optimally, a good characterization of their observation errors is necessary. In this study, background and analysis residuals were used to diagnose the correlated observation-error characteristics of the MWTS and MWHS. It was found that the error standard deviations of the MWTS and MWHS were less than the values used in the WRFDA. MWTS had small inter-channel errors, while MWHS had significant inter-channel errors. The horizontal correlation length scales of MWTS and MWHS were about 120 and 60 km, respectively. A comparison between the diagnosis for instruments onboard the two satellites showed that the observation-error characteristics of the MWTS or MWHS were different when they were onboard different satellites. In addition, it was found that the error statistics were dependent on latitude and scan positions. The forecast experiments showed that using a modified thinning scheme based on diagnosed statistics can improve forecast accuracy.

Justification for estimating observation‐error covariances with the Desroziers diagnostic

The convergence properties of the iterative Desroziers method, which is commonly used to estimate observation error statistics and correlations based on analysis and background departures, is examined. In the derivation of the Desroziers diagnostic, it is assumed that the error statistics are correctly specified in the analysis. While this is not true in practice, it is possible to apply the diagnostic iteratively to obtain increasingly better estimates of the error statistics and correlations. It is proved here that these iterations converge to the truth only when the background error statistics are correct in the analysis. Bounds on the eigenvalues of the assumed background error covariance matrix determine when this matrix is over‐estimated. If the background covariance is not exact, but is not over‐estimated, it is shown that the iterations converge to a symmetric positive definite matrix that depends on the true and assumed error covariances, and diverge to a singular matrix otherwise. The use of the Desroziers method when background and observation errors are correlated with each other is also addressed, as well as the application of this method to unassimilated observations, with examples.

Triple collocation-based estimation of spatially correlated observation error covariance in remote sensing soil moisture data assimilation

Spatially correlated errors are typically ignored in data assimilation, thus degenerating the observation error covariance R to a diagonal matrix. We argue that a nondiagonal R carries more observation information making assimilation results more accurate. A method, denoted TC_Cov, was proposed for soil moisture data assimilation to estimate spatially correlated observation error covariance based on triple collocation (TC). Assimilation experiments were carried out to test the performance of TC_Cov. AMSR-E soil moisture was assimilated with a diagonal R matrix computed using the TC and assimilated using a nondiagonal R matrix, as estimated by proposed TC_Cov. The ensemble Kalman filter was considered as the assimilation method. Our assimilation results were validated against climate change initiative data and ground-based soil moisture measurements using the Pearson correlation coefficient and unbiased root mean square difference metrics. These experiments confirmed that deterioration of diagonal R assimilation results occurred when model simulation is more accurate than observation data. Furthermore, nondiagonal R achieved higher correlation coefficient and lower ubRMSD values over diagonal R in experiments and demonstrated the effectiveness of TC_Cov to estimate richly structuralized R in data assimilation. In sum, compared with diagonal R, nondiagonal R may relieve the detrimental effects of assimilation when simulated model results outperform observation data.

The conditioning of least‐squares problems in variational data assimilation

SummaryIn variational data assimilation a least‐squares objective function is minimised to obtain the most likely state of a dynamical system. This objective function combines observation and prior (or background) data weighted by their respective error statistics. In numerical weather prediction, data assimilation is used to estimate the current atmospheric state, which then serves as an initial condition for a forecast. New developments in the treatment of observation uncertainties have recently been shown to cause convergence problems for this least‐squares minimisation. This is important for operational numerical weather prediction centres due to the time constraints of producing regular forecasts. The condition number of the Hessian of the objective function can be used as a proxy to investigate the speed of convergence of the least‐squares minimisation. In this paper we develop novel theoretical bounds on the condition number of the Hessian. These new bounds depend on the minimum eigenvalue of the observation error covariance matrix and the ratio of background error variance to observation error variance. Numerical tests in a linear setting show that the location of observation measurements has an important effect on the condition number of the Hessian. We identify that the conditioning of the problem is related to the complex interactions between observation error covariance and background error covariance matrices. Increased understanding of the role of each constituent matrix in the conditioning of the Hessian will prove useful for informing the choice of correlated observation error covariance matrix and observation location, particularly for practical applications.

The Effect of Thinning and Superobservations in a Simple One-Dimensional Data Analysis with Mischaracterized Error

A one-dimensional (1D) analysis problem is defined and analyzed to explore the interaction of observation thinning or superobservation with observation errors that are correlated or systematic. The general formulation might be applied to a 1D analysis of radiance or radio occultation observations in order to develop a strategy for the use of such data in a full data assimilation system, but is applied here to a simple analysis problem with parameterized error covariances. Findings for the simple problem include the following. For a variational analysis method that includes an estimate of the full observation error covariances, the analysis is more sensitive to variations in the estimated background and observation error standard deviations than to variations in the corresponding correlation length scales. Furthermore, if everything else is fixed, the analysis error increases with decreasing true background error correlation length scale and with increasing true observation error correlation length scale. For a weighted least squares analysis method that assumes the observation errors are uncorrelated, best results are obtained for some degree of thinning and/or tuning of the weights. Without tuning, the best strategy is superobservation with a spacing approximately equal to the observation error correlation length scale.