Asymptotic Mean Square Error Research Articles

Nonlinear log-periodogram regression for perturbed fractional processes

This paper studies fractional processes that may be perturbed by weakly dependent time series. The model for a perturbed fractional process has a components framework in which there may be components of both long and short memory. All commonly used estimates of the long memory parameter (such as log periodogram (LP) regression) may be used in a components model where the data are affected by weakly dependent perturbations, but these estimates can suffer from serious downward bias. To circumvent this problem, the present paper proposes a new procedure that allows for the possible presence of additive perturbations in the data. The new estimator resembles the LP regression estimator but involves an additional (nonlinear) term in the regression that takes account of possible perturbation effects in the data. Under some smoothness assumptions at the origin, the bias of the new estimator is shown to disappear at a faster rate than that of the LP estimator, while its asymptotic variance is inflated only by a multiplicative constant. In consequence, the optimal rate of convergence to zero of the asymptotic MSE of the new estimator is faster than that of the LP estimator. Some simulation results demonstrate the viability and the bias-reducing feature of the new estimator relative to the LP estimator in finite samples. A test for the presence of perturbations in the data is given.

A Bias-Reduced Log-Periodogram Regression Estimator for the Long-Memory Parameter

In this paper, we propose a simple bias-reduced log-periodogram regression estimator, dr of the long-memory parameter, d, that eliminates the first- and higher-order biases of the Geweke and Porter-Hudak (1983) (GPH) estimator. The bias-reduced estimator is the same as the GPH estimator except that one includes frequencies to the power 2k for k = 1, . . ., r, for some positive integer r, as additional regressors in the pseudo-regression model that yields the GPH estimator. The reduction in bias is obtained using assumptions on the spectrum only in a neighborhood of the zero frequency. Following the work of Robinson (1995b) and Hurvich, Deo, and Brodsky (1998), we establish the asymptotic bias, variance, and mean-squared error (MSE) of dr, determine the asymptotic MSE optimal choice of the number of frequencies, m, to include in the regression, and establish the asymptotic normality of dr. These results show that the bias of dr goes to zero at a faster rate than that of the GPH estimator when the normalized spectrum at zero is sufficiently smooth, but that its variance only is increased by a multiplicative constant. We show that the bias-reduced estimator dr attains the optimal rate of convergence for a class of spectral densities that includes those that are smooth of order s > 1 at zero when r > (s - 2)/2 and m is chosen appropriately. For s > 2, the GPH estimator does not attain this rate. The proof uses results of Giraitis, Robinson, and Samarov (1997). We specify a data-dependent plug-in method for selecting the number of frequencies m to minimize asymptotic MSE for a given value of r. Some Monte Carlo simulation results for stationary Gaussian ARFIMA(1, d, 1) and (2, d, 0) models show that the bias-reduced estimators perform well relative to the standard log-periodogram regression estimator.

Efficient Bandwidth Selection in Non‐parametric Regression

In this paper we use non‐parametric local polynomial methods to estimate the regression function, m(x). Y may be a binary or continuous response variable, and X is continuous with non‐uniform density. The main contributions of this paper are the weak convergence of a bandwidth process for kernels of order (0,k), k=2j, j≥1 and the proposal of a local data‐driven bandwidth selection method which is particularly beneficial for the case when X is not distributed uniformly. This selection method minimizes estimates of the asymptotic MSE and estimates the bias portion in an innovative way which relies on the order of the kernel and not estimation of m2(x) directly. We show that utilization of this method results in the achievement of the optimal asymptotic MSE by the estimator, i.e. the method is efficient. Simulation studies are provided which illustrate the method for both binary and continuous response cases.

Estimation of cumulative incidence functions in competing risks studies under an order restriction

In the competing risks problem an important role is played by the cumulative incidence function (CIF), whose value at time t is the probability of failure by time t for a particular type of failure in the presence of other risks. Its estimation and asymptotic distribution theory have been studied by many. In some cases there are reasons to believe that the CIFs due to two types of failure are order restricted. Several procedures have appeared in the literature for testing for such orders. In this paper we initiate the study of estimation of two CIFs subject to a type of stochastic ordering, both when there are just two causes of failure and when there are more than two causes of failure, treating those other than the two of interest as a censoring mechanism. We do not assume independence of the two types of failure of interest; however, these are assumed to be independent of the other causes in the censored case. Weak convergence results for the estimators have been derived. It is shown that when the order restriction is strict, the asymptotic distributions are the same as those for the empirical estimators without the order restriction. Thus we get the restricted estimators “free of charge”, at least in the asymptotic sense. When the two CIFs are equal, the asymptotic MSE is reduced by using the order restriction. For finite sample sizes simulations seem to indicate that the restricted estimators have uniformly smaller MSEs than the unrestricted ones in all cases.

Bias and Mean Square Error of the Sample Roots Under the Contaminated Gaussian Model

In this article, the bias and mean square error (MSE) of the characteristic roots of the sample covariance matrix are studied when the parent population is nonnormal but is distributed following an ε-contaminated model based on a mixture of two multivariate normal distributions. The expressions for the asymptotic bias and MSE are obtained. An interesting fact is revealed: the bias and MSE don't change in a particular direction of the contamination.

Higher-order kernel semiparametric M-estimation of long memory

Abstract Econometric interest in the possibility of long memory has developed, as a flexible alternative to or compromise between the usual short memory or unit root prescriptions, for example in the context of modelling cointegrating or other relationships and in describing the dependence structure of nonlinear functions of financial returns. Semiparametric methods of estimating the memory parameter can avoid bias incurred by misspecification of the short memory component. We introduce a broad class of such semiparametric estimates that also covers pooling across frequencies. A leading “Box-Cox” sub-class, indexed by a single tuning parameter, interpolates between the popular local log-periodogram and local Whittle estimates, leading to a smooth interpolation of asymptotic variances. The bias of these two estimates also differs to higher order, and we also show how bias, and asymptotic mean-squared error, can be reduced, across the class of estimates studied, by means of a suitable version of higher-order kernels. We thence calculate an optimal bandwidth (the number of low-frequency periodogram ordinates employed) which minimizes this mean-squared error. Finite sample performance is studied in a small Monte Carlo experiment, and an empirical application to intra-day foreign exchange returns is included.

Exploiting sparsity in adaptive filters

This paper studies a class of algorithms called natural gradient (NG) algorithms. The least mean square (LMS) algorithm is derived within the NG framework, and a family of LMS variants that exploit sparsity is derived. This procedure is repeated for other algorithm families, such as the constant modulus algorithm (CMA) and decision-directed (DD) LMS. Mean squared error analysis, stability analysis, and convergence analysis of the family of sparse LMS algorithms are provided, and it is shown that if the system is sparse, then the new algorithms will converge faster for a given total asymptotic MSE. Simulations are provided to confirm the analysis. In addition, Bayesian priors matching the statistics of a database of real channels are given, and algorithms are derived that exploit these priors. Simulations using measured channels are used to show a realistic application of these algorithms.

Tracking properties of a gradient-based second-order adaptive IIR notch filter with constrained poles and zeros

Gradient-type adaptive IIR notch filters have many attractive merits for various real-life applications since they require a small number of computations and yet demonstrate practical performance. However, it is generally quite difficult to assess their performance analytically. Their tracking properties, in particular, have not yet been investigated. In this paper, the tracking performance of a plain gradient (PG) algorithm is analyzed in detail for a second-order adaptive IIR notch filter with constrained poles and zeros, which takes a linear chirp signal as its input. First, two sets of difference equations for the frequency tracking error and mean square error (MSE) are established in the sense of convergence in the mean and convergence in the mean square, respectively. Closed-form expressions for the asymptotic tracking error and MSE are then derived from these difference equations. An optimum step-size parameter for the algorithm is also evaluated based on the minimization of the asymptotic tracking error or the tracking MSE. It is discovered that the asymptotic tracking error may be driven to zero for a positive chirp rate by selecting a proper step size, which is an interesting property for a real-valued adaptive filtering algorithm. Extensive simulations are performed to support the analytical findings.

A dynamic convergence analysis of blind equalization algorithms

Blind equalizers do not require a training sequence to start up or to restart after a communications breakdown, making them particularly useful in applications such as broadcast and point-to-multipoint networks. We study in parallel the dynamic convergence behavior of three blind equalization algorithms: the multimodulus algorithm (MMA), the constant modulus algorithm (CMA), and the reduced constellation algorithm (RCA). Using a conditional Gaussian approximation, we first derive the theoretical mean-squared-error (MSE) trajectory for MMA. This analysis leads to accurate but somewhat cumbersome expressions. Alternatively, we apply a Taylor series approximation to derive MSE trajectories for MMA, CMA, and RCA. This approach yields simpler but somewhat less accurate expressions. For the steady-state operation, however we derive even simpler formulas that accurately predict the asymptotic MSE values. We finally study the convergence rates of the three blind algorithms using their theoretical MSE trajectories, computer simulations, and a laboratory experiment.

A study of local linear ridge regression estimators

Abstract In the case of the random design nonparametric regression, to correct for the unbounded finite-sample variance of the local linear estimator (LLE), Seifert and Gasser (J. Amer. Statist. Assoc. 91 (1996) 267–275) apply the idea of ridge regression to the LLE, and propose the local linear ridge regression estimator (LLRRE). However, the finite sample and the asymptotic properties of the LLRRE are not discussed there. In this paper, upper bounds of the finite-sample variance and bias of the LLRRE are obtained. It is shown that if the ridge regression parameters are not properly selected, then the resulting LLRRE has some drawbacks. For example, it may have a nonzero constant asymptotic bias, may suffer from boundary effects, or may be unable to share the nice asymptotic bias quality of the LLE. On the other hand, if the ridge regression parameters are properly selected, then the resulting LLRRE does not suffer from the above problems, and has the same asymptotic mean-square error as the LLE. For this purpose, the ridge regression parameters are allowed to depend on the sample size, and converge to 0 as the sample size increases. In practice, to select both the bandwidth and the ridge regression parameters, the idea of cross-validation is applied. Simulation studies demonstrate that the LLRRE using the cross-validated bandwidth and ridge regression parameters could have smaller sample mean integrated square error than the LLE using the cross-validated bandwidth, in reasonable sample sizes.

Using a Bootstrap Method to Choose the Sample Fraction in Tail Index Estimation

Tail index estimation depends for its accuracy on a precise choice of the sample fraction, i.e., the number of extreme order statistics on which the estimation is based. A complete solution to the sample fraction selection is given by means of a two-step subsample bootstrap method. This method adaptively determines the sample fraction that minimizes the asymptotic mean-squared error. Unlike previous methods, prior knowledge of the second-order parameter is not required. In addition, we are able to dispense with the need for a prior estimate of the tail index which already converges roughly at the optimal rate. The only arbitrary choice of parameters is the number of Monte Carlo replications.

Wavelet Estimator of Long-Range Dependent Processes

In this contribution, the statistical properties of the wavelet estimator of the long-range dependence parameter introduced in Abry et al. (1995) are discussed for a stationary Gaussian process. This contribution complements the heuristical discussion presented in Abry et al. (1999), by taking into account the correlation between the wavelet coefficients (which is discarded in the mentioned reference) and the bias due to the short-memory component. We derive expressions for the estimator's asymptotic bias, variance and mean-squared error as functions of the scale used in the regression and some user-defined parameters. Consistency of the estimator is obtained as long as the scale index j T goes to infinity and 2 JT /T → 0, where T denotes the sample size. Under these and some additional conditions assumed in the paper, we also establish the asymptotic normality of this estimator.

Statistical estimation in varying coefficient models

Varying coefficient models are a useful extension of classical linear models. They arise naturally when one wishes to examine how regression coefficients change over different groups characterized by certain covariates such as age. The appeal of these models is that the coef .cient functions can easily be estimated via a simple local regression.This yields a simple one-step estimation procedure. We show that such a one-step method cannot be optimal when different coefficient functions admit different degrees of smoothness. This drawback can be repaired by using our proposed two-step estimation procedure.The asymptotic mean-squared error for the two-step procedure is obtained and is shown to achieve the optimal rate of convergence. A few simulation studies show that the gain by the two-step procedure can be quite substantial.The methodology is illustrated by an application to an environmental data set.

Broadband log-periodogram regression of time series with long-range dependence

This paper discusses the properties of an estimator of the memory parameter of a stationary long-memory time-series originally proposed by Robinson. As opposed to ‘‘narrow-band’’ estimators of the memory parameter (such as the Geweke and Porter-Hudak or the Gaussian semiparametric estimators) which use only the periodogram ordinates belonging to an interval which degenerates to zero as the sample size $n$ increases, this estimator builds a model of the spectral density of the process over all the frequency range, hence the name, “broadband.” This is achieved by estimating the ‘‘short-memory’’ component of the spectral density, $f*(x) = |1 - e^{ix}|^{2d}f(x)$, where $d \epsilon (-1/2, 1/2)$ is the memory parameter and $f(x)$ is the spectral density, by means of a truncated Fourier series estimator of log $f*$. Assuming Gaussianity and additional conditions on the regularity of $f*$ which seem mild, we obtain expressions for the asymptotic bias and variance of the long-memory parameter estimator as a function of the truncation order. Under additional assumptions, we show that this estimator is consistent and asymptotically normal. If the true spectral density is sufficiently smooth outside the origin, this broadband estimator outperforms existing semiparametric estimators, attaining an asymptotic mean-square error $O(\log(n)/n)$ .

Sequentially Deciding between Two Experiments for Estimating a Common Success Probability

To estimate a success probability p, two experiments are available: individual Bernoulli (p) trials or the product of τ individual Bernoulli (p) trials. This problem has its roots in reliability where either single components can be tested or a system of τ identical components can be tested. A total of N experiments can be performed, and the problem is to sequentially select some combination (allocation) of these two experiments, along with an estimator of p, to achieve low mean squared error (MSE) of the final estimate. This scenario is similar to that of the better-known group testing problem, but here the goal is to estimate failure rates rather than to identify defective units. The problem also arises in epidemiological applications such as estimating disease prevalence. Information maximization considerations, and analysis of the asymptotic MSE of several estimators, lead to the following adaptive procedure: use the maximum likelihood estimator (MLE) to estimate p, and if this estimator is below (above) the cutpoint aτ , then observe an individual (product) trial at the next stage. In a Bayesian setting with squared error estimation loss and suitable regularity conditions on the prior distribution, this adaptive procedure—replacing the MLE with the Bayes estimator—will be asymptotically Bayes. Exact computational evaluations of the adaptive procedure for fixed sample sizes show that it behaves roughly as the asymptotics predict. The exact analyses also show parameter regions for which the adaptive procedure achieves negative regret, as well as regions for which it achieves normalized MSE superior to that asymptotically possible. An example and a discussion of extensions conclude the work.

On identity reproducing nonparametric regression estimators

Some theoretical results on the performance of the identity reproducing nonparametric regression estimator of Müller and Song (1993) are presented. Compared with Müller and Song's analysis, we develop (a) better asymptotic mean squared error (MSE) results in the ‘interior’ of design space, and (b) ‘partial’ asymptotic MSE results at the boundary, although we do both only in the univariate case. A brief comparison of theoretical boundary results with those of local linear regression estimation is included.

Asymptotic bias and variance of a kernel-based estimator for the location of a discontinuity

A family of kernel-based estimators parametrised by a bandwidth is used to estimate the location of the discontinuity in discretely observed data consisting of a discontinuous regression function with added noise. Asymptotic bias and variance calculations for the estimated location are given as a function of the bandwidth and the sample size. For functions Which are Smooth apart from a single discontinuity, bias is of order h 2, and is not in general negligible compared with variance, which is of order h/n. The asymptotic mean square error rate MSE = O(n −4/3), obtained by balancing squared bias and variance, can usually be improved to O(n −2+δ ) with δ > 0 arbitrarily small by choosing a smaller bandwidth h = O(n −1+δ ). Special cases are treated and bias and variance are calculated explicitly. Only bias depends on the type of discontinuous function considered: For a step function with added noise and for any function which is antisymmetric about the discontinuity point, squared bias becomes negligible compared to variance, and MSE can therefore converge at the faster rate O(n −2+δ )1 For a smooth function with one added step and added white noise, bias is of order h 3 showing that bias decreases more rapidly as the function becomes simpler.

Design of two MUSIC-like estimators based on bias minimization

Two classes of MUSIC-like estimators are considered. One class, called weighted norm MUSIC, possesses an optimizing functional, or null spectrum, which is the product of the MUSIC null spectrum and an angle-dependent weight. The second class, which is denoted the D/sub r/ estimator, has an optimizing functional that is dependent on a parameter r and is a generalized distance between two particular vectors in the signal subspace. It is shown that the asymptotic mean-square errors of these estimators are the same as MUSIC. By determining an appropriate weight, based on a derived large-sample expression for the estimator bias, a weighted norm MUSIC estimator is found that gives zero bias of order N/sup -1/, where N is the sample size. Using an approximate relation between the two types of estimators under consideration, a data-dependent parameter r(/spl theta/) is derived for the D/sub r/ estimator, which results in small bias over a wide range of signal-to-noise ratios (SNRs) for two closely spaced sources.

Order-Determination of Transfer Function Model From Noisy Input-Output Signals

A regularization scheme is investigated to determine the order of a transfer function model in the presence of input and output noises. By introducing multiple regularization parameters to the corrected least squares (CLS) estimate, it is shown that the system order can be determined by comparing each eigenvalue and a corresponding optimal regularization parameter, which minimizes the mean-square error (MSE) of the regularized estimate. An asymptotic MSE is given, and an analytical expression of the optimal regularization parameters is also clarified. Furthermore, a data adaptive iterative algorithm is presented to estimate the system model order and noise variances simultaneously.

Adaptive Choice of Tuning Constant for Robust Regression Estimators

A problem with robust regression estimators is that the tuning constants need to be chosen for the estimators to be completely specified. Here an adaptive method is used to choose the tuning constant and to investigate its variability. The effects that this has on the behaviour of the parameter estimates is also investigated. Using the jackknife the asymptotic mean-squared error of the estimators is estimated and the tuning constant is chosen to minimize this. The Huber and Tukey robust regression estimators are explored by using different error distributions.