Spatial Error Correlation Research Articles

The Spatial Structure of Observation Errors in Atmospheric Motion Vectors from Geostationary Satellite Data

This study investigates and quantifies in detail the spatial correlations of random errors in atmospheric motion vectors (AMVs) derived by tracking structures in imagery from geostationary satellites. A good specification of the observation error is essential to assimilate any kind of observation for numerical weather prediction in a near-optimal way. For AMVs, height assignment, tracking of similar cloud structures, or quality control procedures may introduce spatially correlated errors. The spatial structure of the error correlations is investigated based on a 1-yr dataset of pairs of collocations between AMVs and radiosonde observations. Assuming spatially uncorrelated sonde errors, the spatial AMV error correlations are obtained over dense sonde networks. Results for operational infrared and water vapor wind datasets from Meteosat-5 and -7, Geostationary Operational Environmental Satellite-8 and -10 (GOES-8 and -10), and Geostationary Meteorological Satellite-5 (GMS-5) are presented. Winds from all five datasets show statistically significant spatial error correlations for distances up to about 800 km, with little difference between satellites, channels, or vertical levels. Even broader correlations are found for tropical regions. The correlations exhibit considerable anisotropic structures with, for instance, longer correlation scales in the south–north direction for the υ-wind component, and are comparable to error correlations for short-term forecasts. The study estimates the spatially correlated part of the annual mean AMV wind component error for high-level Northern Hemisphere winds to be about 2.7–3.5 m s−1. Some seasonal variation is found for these errors with larger values in winter. The findings have a number of important implications for the use of AMVs in data assimilation.

Error analysis of TMI rainfall estimates over ocean for variational data assimilation

AbstractAn intercomparison of retrieval errors from different Tropical Rainfall Measuring Mission (TRMM) passive microwave rainfall products was carried out to assess the definition of observation error for experiments of rainfall assimilation in a variational framework. Depending on algorithms and their spatial resolution and sampling, a large variety of error estimates occurred. The error propagation to the European Centre for Medium‐Range Weather Forecasts (ECMWF) model grid (here 45 and 60 km) was investigated from error simulations and observed data with and without accounting for spatial error correlation.All algorithms used in this study (TRMM standard product 2A12 V.5 and two alternative algorithms, namely PATER and BAMPR) employ a Bayesian retrieval framework. The Bayesian errors obtained from each algorithm from different case‐studies showed values well above 100% at low rain rates (0.1 mm h−1) and around 50% at high rain rates (20–50 mm h−1) at the original product resolution and sampling. These Bayesian errors corresponded very well with those from an independent evaluation which was carried out by comparing TRMM microwave radiometer (TMI) estimates to precipitation radar retrievals at the same (here ≈27×40 km2) resolution.The impact of spatial averaging on retrieval errors was simulated using fits to the Bayesian errors and realistic log‐normal rainfall probability distributions. By neglecting spatial correlation, the range of errors is reduced from 70–200% to 20–50% at low rain rates and from 25–70% to 5–20% at high rain rates. To account for spatial data correlation, TMI observations were first averaged to the ECMWF model grid. Then the decorrelation of rain rates as a function of separation distance from all products was calculated. The introduction of spatial error correlation affected both error reduction and dispersion of errors per rain‐rate interval. The final error estimates ranged from 50–150% at low rain rates to 20–50% at high rain rates. The analysis suggests that once the spatial correlation pattern of a product is known, the probability density distribution of real observations inside the model grid does not produce larger scatter and therefore a simple scaling may suffice to calculate rainfall retrieval errors at the model resolution. Copyright © 2002 Royal Meteorological Society

An assessment of estimates of integrated water vapor from ground-based GPS data

A network of ground-based Global Positioning System (GPS) receivers is used to continuously obtain information on variations in the integrated water vapor (IWV) in the atmosphere. We use GPS data from 1996–1998 to compare the IWV estimates with radiosonde data. The spatial correlation of estimation errors is important when the GPS results are to be used in assimilation models. We study and model this effect by comparing the horizontal correlations of the IWV time series derived from the radiosonde network and the GPS data analysis. Under ideal conditions this method separates the IWV features from the underlying correlation function for the estimation errors in the GPS data. A sparse radiosonde network limits the quality of the obtained results. We foresee continued studies including more data and simulations of the errors associated with the GPS technique.

LM Tests for Functional Form and Spatial Error Correlation

This paper derives Lagrangian multiplier (LM) tests to jointly test for functional form and spatial error correlation. In particular, this paper tests for linear and log-linear models with no spatial error dependence against a more general Box-Cox model with spatial error correlation. Conditional LM tests and modified Rao score tests that guard against local misspecification are also derived. These tests are easy to implement and are illustrated using Anselin’s crime data. The performance of these tests is also compared using Monte Carlo experiments.

Quantitative estimation of surface ozone observation and forecast errors

Five-month time series of hourly ground level ozone data of more than 360 German monitoring sites are investigated to estimate errors in observations and forecasts by a comprehensive three-dimensional regional chemistry transport model. On the assumption of uniform measurement techniques, error variances and spatial error correlations in the gridded background field are derived from the analysis of observation increments which are the differences between observations and modeled data. A thorough estimation of those error characteristics is the basic requirement for all data assimilation techniques. The results indicate how the representativeness of the observation sites is limited by differences in local emission characteristics. Additionally, there is a strong dependency on the time of day. The conclusion is that the description of observation and background error covariances for the analysis of ground level ozone has to be time-dependent and spatially inhomogeneous.

Spatial error correlation of GPS atmospheres as determined from simulations

We have made Monte Carlo simulations in order to quantify the influence of several error sources on the atmospheric delay derived in our regular “Precise Point Positioning” analysis of GPS data. Satellite position and clock errors were found to be the greatest sources of error. The RMS size of the atmospheric delay error due to these sources is suggested to be 4–8 mm, and the correlation of the delay error over 1000 km is 0.5-0.7. In our real-time analysis of the atmospheric delay satellite orbits and clock offsets are modelled in the Kalman filter. The modelled errors suggest an uncertainty in the atmospheric delay of 10–13 mm, and a decrease in the correlation coefficients to about 0.65 over a distance of 1000 km. For both the Precise Point Positioning and the real-time analysis the influence from antenna signal scatter will give additional error contributions.

Contracting in space: An application of spatial statistics to discrete-choice models

We develop tests for spatial-error correlation and methods of estimation in the presence of such correlation for discrete-choice models. The tests, which are based on the notion of a generalized residual, are a set of orthogonality conditions that should be satisfied under the null. When the restrictions are rejected, the full model can be estimated by generalized method of moments. These techniques are used to evaluate spatial patterns in retail-gasoline contracts. We examine whether the spatial configuration is random or whether there is a tendency towards clustering or dispersion of contract types. The data consist of all contracts between integrated oil companies and their branded service stations in the city of Vancouver. Six metrics or measures of closeness are examined: Euclidean distance, competition along streets, a combination of the first two, nearest neighbors along streets, nearest neighbors in Euclidean distance, and neighbors that share a common marked boundary.

Scatterometer data interpretation: Estimation and validation of the transfer function CMOD4

In this paper we estimate the 18 coefficients of the CMOD4 σ0‐to‐wind transfer function using a maximum likelihood estimation (MLE) method in order to improve the prelaunch function. We show that a MLE method has to be used with caution when dealing with a nonlinear relationship or with measurement errors that depend on the measured values. In the transfer function estimation it is crucial to use the components of the wind, rather than wind speed and direction, to use σ0 in logarithmic units rather than physical ones, and to use well‐sampled input data. In Stoffelen and Anderson [1997a] we showed that the triplets of measured backscatter are very coherent and, when plotted in a three‐dimensional measurement space, they lie on a well‐defined conical surface. Here we propose a strategy for validation of a transfer function, the first step of which is to test the ability of a transfer function to fit this conical surface. We derive an objective measure to compute the average fit of the transfer function surface to the distribution of measured σ0 triplets. The transfer function CMOD4, derived in the first part of this paper, is shown to fit the cone surface to within the observed scatter normal to the cone, i.e., within roughly 0.2 dB, equivalent to a root‐mean‐square wind vector error of ∼0.5 m s−1 The second step in the validation strategy is the verification of retrieved scatterometer winds at each position on the cone surface. Scatterometer winds computed from CMOD4 compare better to the European Centre for Medium‐Range Weather Forecasts model winds than real‐time conventional surface wind data (ship, buoy, or island reports) with the root‐mean‐square wind vector difference typically 3.0 m s−1. This surprising result can be explained by the so‐called representativeness error. We further show that no significant spatial wind error correlation is present in scatterometer data and therefore conclude that the ERS 1 scatterometer provides winds useful for weather forecasting and climate studies.

Spatial Correlation of Beam-Filling Error in Microwave Rain-Rate Retrievals

In this paper the authors consider the possibility of correlations between the random part of the so-called beam-filling error between neighboring fields of view in the microwave retrieval of rain rate over oceans. The study is based upon the GARP (Global Atmospheric Research Program) Atlantic Tropical Experiment (GATE) rain-rate dataset, and it is found that there is a correlation of between 0.35 and 0.50 between the errors in adjacent rainy fields of view. The net effect of this correlation is reducing the number of statistically independent terms accumulated in forming area and time averages of rain-rate estimates. In GATE-like rain areas, this reduction can be of the order of a factor of 3, making accumulated standard error percentages increase by a factor of the order of √3. For the Tropical Rainfall Measuring Mission using the microwave radiometer alone. this could increase the accumulated random part of the beam-filling error for month-long 5°×5° boxes from about 1.2% to 2%. The effect is larger for less rainy areas away from the equatorial zone.

A review of mask errors on a variety of pattern generators

The question of how mask dimensional errors impact yield is becoming increasingly critical to VLSI manufacturing. Both the magnitude and the spatial correlation of CD and registration errors are believed important to this issue. As part of a program to characterize these errors, we have had one identical test mask made on 6 different state-of-the-art mask pattern generators with widely different architectures. The test mask provides information on both registration and feature-size errors in both x- and y-directions, and does so over distances of several centimeters with spacing between measurements as small as 1 /spl mu/m. More than 100000 data points have been collected from these test plates using a LMS2000 optical metrology system, and are analyzed in the spatial frequency domain where error contributors as small as 1 nm can be identified. All systems were found to have similar characteristics in that most error contributors occur at a number of machine-specific spatial frequencies correlated to the particular architecture and printing strategy of the machine. Comparison of raster to vector machines show that vector machines tend to have more periodic error contributors, especially in the high spatial frequency range, which is consistent with the more complex writing fields used to achieve higher throughput.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Partially coherent migration

Partially coherent migration reduces the spurious details introduced by velocity macro‐model imperfections. In a partially coherent migration, instead of summing coherently over the full aperture to achieve maximum lateral resolution, (1) a coherent stack is performed over a limited window width, and then (2) the collection of coherent stacks for different windows along the full aperture are summed incoherently to produce an amplitude value for each output point. Amplitude accuracy of a migration is improved with some sacrifice in spatial (lateral) resolution. A parameter in partially coherent migration is the running coherent window width, which accounts for the spatial correlation of errors in the velocity model. The coherent window width controls the trade‐off between signal‐to‐noise ratio and lateral resolution. Assuming that timing errors introduced by imperfections of a velocity macro‐model are from a zero‐mean stationary Gaussian process, partially coherent migration is shown to raise the signal‐to‐noise ratio of the migrated image as compared to a conventional migration. The two competing aspects of signal‐to‐noise ratio and lateral resolution of the partially coherent migration in the presence of timing errors are analyzed in a stochastic framework. The intuitively attractive idea of limiting the coherent window width to the correlation length of the timing errors is confirmed numerically.

Load Forecasting for Transmission Planning

This paper investigates the load forecasting needs of transmission planning, using the planning of two utility systems as an example. Two aspects are investigated. One, the effect of forecasting error on the planning of a transmission system, and two, procedures required to produce forecasts. Several conclusions are developed. Future substation loads are a function of both decisions in future distribution planning and also of the changing, nonuniform geographic distribution of load. Second, spatial correlation of error is as important in determining impact as is the average magnitude of the errors. Third, a transmission system planned from an incorrect forecast may function but fail to meet contingency criteria. A procedure for estimating the benefit due to improved load forecasting is given.

Load Forecasting for Trnasmission Planning

This paper investigates the load forecasting needs of transmission planning, using the planning of two utility systems as an example. Two aspects are investigated. One, the effect of forecasting error on the planning of a transmission system, and two, procedures required to produce forecasts. Several conclusions are developed. Future substation loads are a function of both decisions in future distribution planning and also of the changing, nonuniform geographic distribution of load. Second, spatial correlation of error is as important in deternmining impact as is the average magnitude of the errors. Third, a transmiission system planned from an incorrect forecast may function but fail to meet contingency criteria. A procedure for estimating the benefit due to improved load forecasting is given.

Absolute and Differential Accuracy of Analyses Achievable with Specified Observational Network Characteristics

Abstract The dependence of the root-mean-square analysis error of an observed element, its gradient and its Laplacian upon 1) observational density, 2) observational error and 3) spatial correlation of observational error have been assessed using optimum interpolation theory. The results have been generalized by scaling the observational separation s according to a length scale parameter L, which corresponds to the Gaussian population spatial autocorrelation function μ(s)=exp(−s2L−2) of the forecast error of the observed element. The responses of different synoptic regimes (subpopulations) were discriminated according to the sub-population length-scale parameter. The number of observations considered at a grid point was limited to either 12 or 4. It has been shown that 1) an increasing spatial correlation of observational error has a different effect on the analysis errors of absolute and differential quantities, and 2) that the point of diminishing returns, below which an increasing observational density...