Fixed Design Regression Research Articles

Nonlinear wavelet estimation of regression function with random desigm

The nonlinear wavelet estimator of regression function with random design is constructed. The optimal uniform convergence rate of the estimator in a ball of Besov spaceB 3 p,q is proved under quite general assumpations. The adaptive nonlinear wavelet estimator with near-optimal convergence rate in a wide range of smoothness function classes is also constructed. The properties of the nonlinear wavelet estimator given for random design regression and only with bounded third order moment of the error can be compared with those of nonlinear wavelet estimator given in literature for equal-spaced fixed design regression with i.i.d. Gauss error.

On residual empirical processes of stochastic regression models with applications to time series

Motivated by Gaussian tests for a time series, we are led to investigate the asymptotic behavior of the residual empirical processes of stochastic regression models. These models cover the fixed design regression models as well as general AR$(q)$ models. Since the number of the regression coeffi-cients is allowed to grow as the sample size increases, the obtained results are also applicable to nonlinear regression and stationary AR$(\infty)$ models. In this paper, we first derive an oscillation-like result for the residual em-pirical process. Then, we apply this result to autoregressive time series. In particular, for a stationary AR$(\infty)$ process, we are able to determine the order of the number of coefficients of a fitted AR$(q_n)$ model and obtain the limiting Gaussian processes. For an unstable AR$(q)$ process, we show that if the characteristic polynomial has a unit root 1, then the limiting process is no longer Gaussian. For the explosive case, one of our side results also provides a short proof for the Brownian bridge results given by Koul and Levental.

Bootstrapping quantiles in a fixed design regression model with censored data

We consider the problem of estimating the quantiles of a distribution function in a fixed design regression model in which the observations are subject to random right censoring. The quantile estimator is defined via a conditional Kaplan-Meier type estimator for the distribution at a given design point. We establish an a.s. asymptotic representation for this quantile estimator, from which we obtain its asymptotic normality. Because a complicated estimation procedure is necessary for estimating the asymptotic bias and variance, we use a resampling procedure, which provides us, via an asymptotic representation for the bootstrapped estimator, with an alternative for the normal approximation.

Large-sample inference for nonparametric regression with dependent errors

A central limit theorem is given for certain weighted partial sums of a covariance stationary process, assuming it is linear in martingale differences, but without any restriction on its spectrum. We apply the result to kernel nonparametric fixed-design regression, giving a single central limit theorem which indicates how error spectral behavior at only zero frequency influences the asymptotic distribution and covers long-range, short-range and negative dependence. We show how the regression estimates can be Studentized in the absence of previous knowledge of which form of dependence pertains, and show also that a simpler Studentization is possible when long-range dependence can be taken for granted.

Two-stage change-point estimators in smooth regression models

We consider a fixed design regression model where the regression function is assumed to be smooth, i.e., Lipschitz continuous, except for a point where it has only one-sided limits and a local discontinuity occurs. We propose a two-step estimator for the location of this change point and study its asymptotic convergence properties. In a first step, initial pilot estimates of the change point and associated asymptotically shrinking intervals which contain the true change point with probability converging to 1 are obtained. In the second step, a weighted mean difference depending on the assumed location of the change point is maximized within these intervals and the maximizing argument is then the final change point estimator. It is shown that this estimator attains the rate O p ( n −1) in the fixed jump case. In the contiguous case, the estimator attains the rate O p ( n −1 Δ n −2), where Δ n is the sequence of jump sizes which in this case is assumed to converge to 0. For the contiguous case an invariance principle is established. A sequence of appropriately scaled deviation processes is shown to converge to a two-sided Brownian motion with triangular drift.

Nonparametric regression with long-memory errors

The fixed design regression model with long-memory errors is considered. The finite-dimensional asymptotic distributions of the properly normalised kernel estimators of the regression function are shown to be normal when the errors are a linear process.

Weak convergence of the bootstrapped conditional kaplan-meier process and its quantile process

We consider a fixed design regression model in which the observations are subject to random right censoring. We establish weak convergence results for Beran's conditional Kaplan-Meier estimator and the corresponding quantile estimator, as well as for their bootstrapped versions. This enables us to construct bootstrap confidence bands for both the distribution and the quantile function.

Weighted nonparametric regression

In the fixed design regression model, additional weights are considered for the Nad a ray a-Watson and Gasser-Miiller kernel estimators. We study their asymptotic behavior and the relationships between new and classical estimators. For a simple family of weights, and considering the AIMSEAS global loss criterion, we show some possible theoretical advantages. An empirical study illustrates the performance of the weighted kernel estimators in theoretical ideal situations and in simulated data sets. Also some results concerning the use of weights for local polynomial estimators are given.

Fixed-design regression for linear time series

This investigation is concerned with recovering a regression function $g(x_i)$ on the basis of noisy observations taken at uniformly spaced design points $x_i$. It is presumed that the corresponding observations are corrupted by additive dependent noise, and that the noise is, in fact, induced by a general linear process in which the summand law can be discrete, as well as continuously distributed. Discreteness induces a complication because such noise is not known to be strong mixing, the postulate by which regression estimates are often shown to be asymptotically normal. In fact, as cited, there are processes of this character which have been proven not to be strong mixing. The main analytic result of this study is that, in general circumstances which include the non-strong mixing example, the smoothers we propose are asymptotically normal. Some motivation is offered, and a simple illustrative example calculation concludes this investigation. The innovative elements of this work, mainly, consist of compassing models with discrete noise, important in practical applications, and in dispensing with mixing assumptions. The ensuing mathematical difficulties are overcome by sharpening standard arguments.

Comparison of bandwidth selectors in nonparametric regression under dependence

Six selectors for the smoothing parameter of the Gasser-Müller nonparametric kernel estimator are compared for the case of a fixed design regression model with correlated errors. Four cross-validation methods and two plug-in methods are studied. All of them are briefly introduced and then compared by means of a thorough simulation study.

A set-indexed process in a two-region image

We investigate the problem of edge estimation in a two-region image in the setting of a fixed design regression model. The edge estimation problem is equivalent to estimating one of the plateau sets where the regression function is constant, and we define a global set-valued estimator by finding the partition which maximizes a weighted distance measure. An investigation of the weak convergence of the random sets generated by this estimator shows that a properly scaled stochastic process in symmetric differences between estimated and true partitions converges in the limit to a set-indexed Brownian motion with drift in R d .

Distant long-range dependent sums and regression estimation

Consider a stationary sequence G( Z 0), G( Z 1), …, where G(·) is a Borel function and Z 0, Z 1, … is a sequence of standard normal variables with covariance function E( Z 0 Z j ) = j − α L( j), j = 1, 2, …, where E( G( Z 0)) = 0, E( G 2( Z 0)) < ∞, 0 < α < 1 and L(·) varies slowly at infinity. Let S n(t) = ∑ ⌞nt⌋−1 j=0 G(Z j) , t ⩾ 0, be the associated partial-sum process. The main result is that for any fixed k ϵ N and 0 < b < ∞, a suitable norming sequence a n > 0 and sequences of gap-lengths l 1, n , …, l k, n such that l 1, n → ∞ and l j, n − l j−1, n → ∞, j = 2, …, k, arbitrary slowly, the vector process (S n(t 0), S n(l 1,n + t 1) − S n(l 1,n), …, S n(l k,n + t k) − S n(l k,n)) a n , 0 〈 t 0, t 1, …, t k 〈 b, converges in distribution in D[0, b] k+1 to the vector of k + 1 independent Hermite processes with a rank given by G(·). As an application, the asymptotic behavior of the finite-dimensional distributions of kernel estimators is determined in the fixed-design regression model with errors of the form G( Z j ), j = 0, 1, … .

On a Semiparametric Variance Function Model and a Test for Heteroscedasticity

We propose a general semiparametric variance function model in a fixed design regression setting. In this model, the regression function is assumed to be smooth and is modelled nonparametrically, whereas the relation between the variance and the mean regression function is assumed to follow a generalized linear model. Almost all variance function models that were considered in the literature emerge as special cases. Least-squares-types estimates for the parameters of this model and the simultaneous estimation of the unknown regression and variance functions by means of nonparametric kernel estimates are combined to infer the parametric and nonparametric components of the proposed model. The asymptotic distribution of the parameter estimates is derived and is shown to follow usual parametric rates in spite of the presence of the nonparametric component in the model. This result is applied to obtain a data-based test for heteroscedasticity under minimal assumptions on the shape of the regression function.

Nonparametric Regression Under Long-Range Dependent Normal Errors

We consider the fixed-design regression model with long-range dependent normal errors and show that the finite-dimensional distributions of the properly normalized Gasser-Miiller and Priestley-Chao estimators of the regression function converge to those of a white noise process. Furthermore, the distributions of the suitably renormalized maximal deviations over an increasingly finer grid converge to the Gumbel distribution. These results contrast with our previous findings for the corresponding problem of estimating the marginal density of long-range dependent stationary sequences.

A plug-in technique in nonparametric regression with dependence

The problem addressed is that of smoothing parameter selection in kernel nonparametric regression in the fixed design regression model with dependent noise. An asymptotic expression of the optimum bandwidth parameter has been obtained in recent studies, where this takes the form h = C 0 n −1/5. This paper proposes to use a plug-in methodology, in order to obtain an optimum estimation of the bandwidth parameter, through preliminary estimation of the unknown value of C 0.

On variance function estimation with quadratic forms

Abstract We propose a class of general quadratic forms in the dependent variable for estimating the variance function in a nonparametric heteroscedastic fixed design regression model. It is shown that in some instances these estimators achieve improved rates of convergence for the mean squared error as compared to estimators that were considered before. Besides results on consistency and rates of convergence, also the leading terms of the asymptotic mean squared error are obtained for some important cases. Several interesting special estimators are discussed in more detail.

Preaveraged Localized Orthogonal Polynomial Estimators for Surface Smoothing and Partial Differentiation

Abstract We propose a multivariate smoothing method based on products of localized orthogonal polynomial series estimators for a smooth regression surface in the fixed-design regression model. The estimation of partial derivatives is included. The proposed method provides for automatic and efficient boundary modifications near the edges of the surface, assuming that the boundary of the support of the regression function satisfies some regularity conditions. By allowing for a preaveraging step, the corresponding algorithms are speeded up considerably and are easy to implement. Computation of special boundary kernels, as required by the kernel method to avoid edge effects, is not necessary. It is shown that under sufficient smoothness assumptions, the global average mean squared error has the same optimal rate of convergence as the mean squared error at an interior point; that is, the boundary correction is asymptotically effective. The method depends on two smoothing parameters, one determining the amount ...

A law of the iterated logarithm for kernel estimators of regression functions

For fixed design regression data kernel estimation is widely used to estimate μ(v)(t), thev-th derivative of μ(t). Denoting such an estimator by $$\hat \mu _{nv} (t),$$ this paper is concerned with the almost sure convergence of $$\hat \mu _{nv} (t) - E\hat \mu _{nv} (t)$$ . It is shown that under several inserting procedures a law of iterated logarithm holds.

Residuals density estimation in nonparametric regression

For the fixed design regression model, a nonparametric estimate of the probability density function of the residuals is proposed and its basic properties are studied. This estimate should have direct impact on diagnostics of regression models of this type. The estimate proposed here is based on estimating the regression function at the regressor points thus giving estimates of the residuals. Then these residual estimators are used to construct a kernel estimate of residuals density. The estimate is shown, among other things, to be consistent and asymptotically normal.

Fixed design regression for time series: Asymptotic normality

Consider the fixed regression model with general weights, and suppose that the error random variables are coming from a strictly stationary stochastic process, satisfying the strong mixing condition. The asymptotic normality of the proposed estimate is established under weak conditions. The applicability of the results obtained is demonstrated by way of two existing estimates, the Gasser-Müller estimate and that of Priestley and Chao. The asymptotic normality of these estimates is further illustrated by means of a concrete example from the class of autoregressive processes.