Linear Statistical Estimation Research Articles

A Statistical Interpolation Code for Ocean Analysis and Forecasting

Abstract We present a data assimilation package for use with ocean circulation models in analysis, forecasting, and system evaluation applications. The basic functionality of the package is centered on a multivariate linear statistical estimation for a given predicted/background ocean state, observations, and error statistics. Novel features of the package include support for multiple covariance models, and the solution of the least squares normal equations either using the covariance matrix or its inverse—the information matrix. The main focus of this paper, however, is on the solution of the analysis equations using the information matrix, which offers several advantages for solving large problems efficiently. Details of the parameterization of the inverse covariance using Markov random fields are provided and its relationship to finite-difference discretizations of diffusion equations are pointed out. The package can assimilate a variety of observation types from both remote sensing and in situ platforms. The performance of the data assimilation methodology implemented in the package is demonstrated with a yearlong global ocean hindcast with a 1/4° ocean model. The code is implemented in modern Fortran, supports distributed memory, shared memory, multicore architectures, and uses climate and forecasts compliant Network Common Data Form for input/output. The package is freely available with an open source license from www.tendral.com/tsis/.

Insights on the OAFlux ocean surface vector wind analysis merged from scatterometers and passive microwave radiometers (1987 onward)

AbstractA high‐resolution global daily analysis of ocean surface vector winds (1987 onward) was developed by the Objectively Analyzed air‐sea Fluxes (OAFlux) project. This study addressed the issues related to the development of the time series through objective synthesis of 12 satellite sensors (two scatterometers and 10 passive microwave radiometers) using a least‐variance linear statistical estimation. The issues include the rationale that supports the multisensor synthesis, the methodology and strategy that were developed, the challenges that were encountered, and the comparison of the synthesized daily mean fields with reference to scatterometers and atmospheric reanalyses. The synthesis was established on the bases that the low and moderate winds (<15 m s−1) constitute 98% of global daily wind fields, and they are the range of winds that are retrieved with best quality and consistency by both scatterometers and radiometers. Yet, challenges are presented in situations of synoptic weather systems due mainly to three factors: (i) the lack of radiometer retrievals in rain conditions, (ii) the inability to fill in the data voids caused by eliminating rain‐flagged QuikSCAT wind vector cells, and (iii) the persistent differences between QuikSCAT and ASCAT high winds. The study showed that the daily mean surface winds can be confidently constructed from merging scatterometers with radiometers over the global oceans, except for the regions influenced by synoptic weather storms. The uncertainties in present scatterometer and radiometer observations under high winds and rain conditions lead to uncertainties in the synthesized synoptic structures.

On the Least Median Square Problem

We consider the exact and approximate computational complexity of the multivariate least median-of-squares (LMS) linear regression estimator. The LMS estimator is among the most widely used robust linear statistical estimators. Given a set of n points in \({\Bbb R}^d\) and a parameter k, the problem is equivalent to computing the narrowest slab bounded by two parallel hyperplanes that contains k of the points. We present algorithms for the exact and approximate versions of the multivariate LMS problem. We also provide nearly matching lower bounds for these problems. These lower bounds hold under the assumptions that k is Ω(n) and that deciding whether n given points in \({\Bbb R}^d\) are affinely non-degenerate requires Ω(nd) time.

Monthly Mean Sea-Level Variability Along the West Coast of North America

Abstract Linear statistical estimators are used to examine 29 years of nonseasonal, monthly-mean, tide-gauge sea-level data along the west coast of North America. The objective is exploration of the structure, and causes of nearshore ocean variability over time scales of months to years at 20 stations from Alaska to Mexico. North of San Francisco, 50–60% of the sea-level variability reflects a simple inverse barometric response to local atmospheric pressure. These inverted barometer effects account for only 10–15% of the variance at stations to the south. The dominant signal of inverse-barometer-corrected sea level represents a nearly uniform rise or fall of sea level everywhere along the eastern rim of the North Pacific. The interannual aspects of this large-scale sea-level variability are closely related to El Nino occurrences in the eastern tropical Pacific which appear to propagate poloward with phase speeds of ∼40 cm s−1. Higher frequency aspects of this large-scale sea-level variability appear to re...

The statistical prediction of beach changes in southern California

Changes in natural sand beaches induced by variations in incident waves were predicted by techniques of linear statistical estimation and empirical eigenfunction analysis. A 5‐year set of measured beach profiles and wave statistics from southern California constituted the data base for this two‐faceted statistical study. First, daily beach profile changes were predicted using four different spectral representations of the wave field. These profile changes were predictable using spectral representations of wave energy, radiation stress, energy flux, and wave steepness. Because of constraints on statistical reliability, a longer data set is required to select one of these as an optimal wave parameterization. Second, weekly beach profile changes were predicted using weekly averaged wave characteristics. Weekly beach changes were predictable using weekly mean and maximum values of wave energy and wave height. The best predictor of those tested was the weekly mean wave energy. When combined with a longshore transport model, this onshore/offshore transport estimator should be applicable to other coastal regions with different beach and wave characteristics.

Predictability of Sea Surface Temperature and Sea Level Pressure Anomalies over the North Pacific Ocean

Nonseasonal variability of sea level pressure (SLP) and sea surface temperature (SST) in the mid-latitude North Pacific Ocean is examined. The objective is examination of the basic scales of the variability and determination of possible causal connections which might allow prediction of short-term climatic (time scales between a month and a year) variability. Using empirical orthogonal function descriptions of the spatial structure, it is found that SLP variability is concentrated in a few large-scale modes but has a nearly white frequency spectrum. SST variability is spatially complex (being spread over many spatial modes, some of which have small-scale changes) but is dominated by low-frequency changes. The use of linear statistical estimators to examine predictability is discussed and the importance of limiting the number of candidate data used in a correlation starch is underscored. Using linear statistical predictors, it is found that (A) SST anomalies can be predicted from SST observations several months in advance with measurable skill, (B) the anomalous SLP variability can be specified from simultaneous SST data with significant skill, thus showing the fields are related, and (C) future SLP anomalous variability cannot be predicted from SST data although previous SLP can be specified. The fact that previous SLP variability is better specified by SST data than is simultaneous SLP variability, coupled with a complete inability to predict future SLP anomalies, suggests that, in the region studied and on the time scales of a month to a year, the observed connection between SST and SLP variabilities is the result of the atmosphere driving the ocean.

Stochastic difference equation predictors of population fluctuations

This paper studies the effectiveness of a class of linear statistical estimators called autoregressive and autoregressive-moving averages equations for mimicking and predicting the abundance fluctuations of three species of Drosophila censused at a pine plantation near Bogota, Colombia. A short introductory justification for the use of linear estimators is given followed by a brief discussion of the theoretical basis of statistical prediction. The assumptions of the method, fitting techniques, and use of the equations in forecasting are discussed. The mimicking ability of the equations is tested by comparing monte carlo simulations employing the fitted models to the observed fluctuations of the three species. Of the autoregressive equations fitted to each species two are judged successful and one less than successful. Autoregressive-moving averages models were found to be significantly worse predictors than the simpler autoregressive equations for these three species. The parameter estimates given by the preliminary estimation techniques are compared with the statistically efficient least squares estimates. The estimates compare well for most of the autoregressive models, but the parameter estimates for the autoregressive-moving averages models were misleading.

The correctness of a linear estimation problem

THE CONCEPT of the correctness of the problem of linear optimal statistical estimation is considered for a certain class of random processes with compact correlation operators. The concept of the correctness of the problem of optimal linear statistical estimation, regarded as an extremal problem for a quality criterion, is formulated. It is shown that in the case where the quality criterion is the mean-square error, the problem of optimal estimation is correct. The estimation problem is also considered for inexact initial data.

On a generalization of theFarkas theorem

In this paper we give a generalization of theFarkas theorem [Farkas; Dantzig, p. 159;Dorfman-Samuelson-Solow, pp. 502ff. orGale, pp. 42–46], well known from application in linear and nonlinear programming [Dorfman-Samuelson-Solow, pp. 189ff.;Danskin, pp. 123ff.]. The method of investigation is geometric, the essential tool is the bipolar-theorem. It is shown, that for every closed, convex set of a finite-dimensionalEuclidian vectorspaceH, which contains zero and for each linear mappingX from theEuclidian vector-spaceΘ intoH the relation $$(X^{ - 1} (M))^0 = \overline {X'M^0 } $$ holds. Here we denote beiN0 the polar of a given setN. Applications of this more generalFarkas theorem in linear statistical estimation theory are given. The paper is self-contained in so far, as it contains all the proofs of the important theorems on projections and polars.