Fixed Design Regression Research Articles

Two-stage variational mode decomposition approach to enhance the estimates of variance function

A two-stage variational mode decomposition approach is explored in this article to estimate the variance function in a nonparametric heteroscedastic fixed design regression model. This estimate improves the results obtained by using variational mode decomposition in the case of smooth regression function. We noticed that the amount of improvement is depending on the smoothness of the variance function. Simulations and comparisons with the earlier methods are carried out to evaluate the performance of the proposed method.

Gaussian linear model selection in a dependent context

In this paper, we study the nonparametric linear model, when the error process is a dependent Gaussian process. We focus on the estimation of the mean vector via a model selection approach. We first give the general theoretical form of the penalty function, ensuring that the penalized estimator among a collection of models satisfies an oracle inequality. Then we derive a penalty shape involving the spectral radius of the covariance matrix of the errors, which can be chosen proportional to the dimension when the error process is stationary and short range dependent. However, this penalty can be too rough in some cases, in particular when the error process is long range dependent. In a second part, we focus on the fixed-design regression model assuming that the error process is a stationary Gaussian process. We propose a model selection procedure in order to estimate the mean function via piecewise polynomials on a regular partition, when the error process is either short range dependent, long range dependent or anti-persistent. We present different kinds of penalties, depending on the memory of the process. For each case, an adaptive estimator is built, and the rates of convergence are computed. Thanks to several sets of simulations, we study the performance of these different penalties for all types of errors (short memory, long memory and anti-persistent errors). Finally, we give an application of our method to the well-known Nile data, which clearly shows that the type of dependence of the error process must be taken into account.

Localized Gaussian width of $M$-convex hulls with applications to Lasso and convex aggregation

Upper and lower bounds are derived for the Gaussian mean width of a convex hull of $M$ points intersected with a Euclidean ball of a given radius. The upper bound holds for any collection of extreme points bounded in Euclidean norm. The upper bound and the lower bound match up to a multiplicative constant whenever the extreme points satisfy a one sided Restricted Isometry Property. An appealing aspect of the upper bound is that no assumption on the covariance structure of the extreme points is needed. This aspect is especially useful to study regression problems with anisotropic design distributions. We provide applications of this bound to the Lasso estimator in fixed-design regression, the Empirical Risk Minimizer in the anisotropic persistence problem, and the convex aggregation problem in density estimation.

Rates of contraction with respect to $L_{2}$-distance for Bayesian nonparametric regression

We systematically study the rates of contraction with respect to the integrated $L_{2}$-distance for Bayesian nonparametric regression in a generic framework, and, notably, without assuming the regression function space to be uniformly bounded. The generic framework is very flexible and can be applied to a wide class of nonparametric prior models. Three non-trivial applications of the framework are provided: The finite random series regression of an $\alpha$-Holder function, with adaptive rates of contraction up to a logarithmic factor; The un-modified block prior regression of an $\alpha$-Sobolev function, with adaptive-and-exact rates of contraction; The Gaussian spline regression of an $\alpha$-Holder function, with near optimal rates of contraction. These applications serve as generalization or complement of their respective results in the literature. Extensions to the fixed-design regression problem and sparse additive models in high dimensions are discussed as well.

The mean consistency of the weighted estimator in the fixed design regression models based on m-END errors

The mean consistency of the weighted estimator in the fixed design regression models based on m-END errors

Smoothing the difference-based estimates of variance using variational mode decomposition

ABSTRACTWe propose a variational mode decomposition approach to estimate the variance function in a nonparametric heteroscedastic fixed design regression model. A data-driven estimator is constructed by applying variational mode decomposition technique to the difference-based initial estimates. The numerical results show that the proposed estimator performs better than the existing variance estimation procedures in the mean square sense.

Gasser-Müllerův odhad

Kernel smoothing provides a simple way for finding structure in data. The idea of the kernel smoothing can be applied to a simple fixed design regression model and a random design regression model. This article is focused on kernel smoothing for fixed design regression model with using special type of estimator, the Gasser-Müller estimator, and on choice of the bandwidth. At the end of this article figures for ilustration described methods on two data sets are shown. The first data set contains simulated values of function sin(2πx), the second contains January average temperatures measured in Basel 1755–1855.

Smoothing spline regression estimation based on real and artificial data

In this article we introduce a smoothing spline estimate for fixed design regression estimation based on real and artificial data, where the artificial data comes from previously undertaken similar experiments. The smoothing spline estimate gives different weights to the real and the artificial data. It is investigated under which conditions the rate of convergence of this estimate is better than the rate of convergence of the ordinary smoothing spline estimate applied to the real data only. The finite sample size performance of the estimate is analyzed using simulated data. The usefulness of the estimate is illustrated by applying it in the context of experimental fatigue tests.

Semiparametric Estimator of Mean Conditional Residual Life Function under Informative Random Censoring from Both Sides

In this paper we study estimator of mean residual life function in fixed design regression model when life times are subjected to informative random censoring from both sides. We prove an asymptotic normality of estimators.

Variance Estimation in Heteroscedastic Models by Undecimated Haar Transform

We propose a method in order to maximize the accuracy in the estimation of piecewise constant and piecewise smooth variance functions in a nonparametric heteroscedastic fixed design regression model. The difference-based initial estimates are obtained from the given observations. Then an estimator is constructed by using iterative regularization method with the analysis-prior undecimated three-level Haar transform as regularizer term. We notice that this method shows better results in the mean square sense over an existing adaptive estimation procedure considering all the standard test functions used in addition to the functions that we target. Some simulations and comparisons with other methods are conducted to assess the performance of the proposed method.

Poznámky k neparametrickým odhadům

Kernel smoothers belong to the most popular nonparametric functional estimates. They provide a simple way of finding structure in data. The idea of the kernel smoothing can be applied to a simple fixed design regression model. This article is focused on kernel smoothing for fixed design regresion model with three types of estimators, the Gasser-Müller estimator, the Nadaraya-Watson estimator and the local linear estimator. At the end of this article figures for ilustration of desribed estimators on simulated and real data sets are shown.

A wavelet-based hybrid approach to estimate variance function in heteroscedastic regression models

We propose a wavelet-based hybrid approach to estimate the variance function in a nonparametric heteroscedastic fixed design regression model. A data-driven estimator is constructed by applying wavelet thresholding along with the technique of sparse representation to the difference-based initial estimates. We prove the convergence of the proposed estimator. The numerical results show that the proposed estimator performs better than the existing variance estimation procedures in the mean square sense over a range of smoothness classes.

Adaptive confidence bands in the nonparametric fixed design regression model

In this note, we consider the problem of the existence of adaptive confidence bands in the fixed design regression model, adapting ideas in Hoffmann and Nickl [(2011), ‘On Adaptive Inference and Confidence Bands’, Annals of Statistics, 39, 2383–2409] to the present case. In the course of the proof, we show that sup-norm adaptive estimators exist as well in the regression setting.

Confidence regions for images observed under the Radon transform

Abstract Recovering a function f from its integrals over hyperplanes (or line integrals in the two-dimensional case), that is, recovering f from the Radon transform R f of f , is a basic problem with important applications in medical imaging such as computerized tomography (CT). In the presence of stochastic noise in the observed function R f , we shall construct asymptotic uniform confidence regions for the function f of interest, which allows to draw conclusions regarding global features of f . Specifically, in a white noise model as well as a fixed-design regression model, we prove a Bickel–Rosenblatt-type theorem for the maximal deviation of a kernel-type estimator from its mean, and give uniform estimates for the bias for f in a Sobolev smoothness class. The finite sample properties of the proposed methods are investigated in a simulation study.

The consistency of estimator under fixed design regression model with NQD errors

In this article, based on NQD samples, we investigate the fixed design nonparametric regression model, i.e. Ynk = g(xnk )+ enk for 1 ≤ k ≤ n ,w hereenk are pairwise NQD random errors, xnk are fixed design points, and g(·) is an unknown function. The nonparametric weighted estimator gn(· )o fg(·) will be introduced and its consistency is studied. As a special case, the consistency result for weighted kernel estimators of the model is established. This extends the earlier work on independent random and dependent random errors to the NQD case.

Smoothing dichotomy in randomized fixed-design regression with strongly dependent errors based on a moving average

We consider a fixed-design regression model with errors which form a Borel measurable function of a long-range dependent moving average process. We introduce an artificial randomization of grid points at which observations are taken in order to diminish t

Aggregation of affine estimators

We consider the problem of aggregating a general collection of affine estimators for fixed design regression. Relevant examples include some commonly used statistical estimators such as least squares, ridge and robust least squares estimators. Dalalyan and Salmon [DS12] have established that, for this problem, exponentially weighted (EW) model selection aggregation leads to sharp oracle inequalities in expectation, but similar bounds in deviation were not previously known. While results [DRZ12] indicate that the same aggregation scheme may not satisfy sharp oracle inequalities with high probability, we prove that a weaker notion of oracle inequality for EW that holds with high probability. Moreover, using a generalization of the newly introduced $Q$-aggregation scheme we also prove sharp oracle inequalities that hold with high probability. Finally, we apply our results to universal aggregation and show that our proposed estimator leads simultaneously to all the best known bounds for aggregation, including $\ell_{q}$-aggregation, $q\in(0,1)$, with high probability.

Fixed-design regression estimation based on real and artificial data

In this article, we study the fixed-design regression estimation based on real and artificial data, where the artificial data comes from previously undertaken similar experiments. A least-squares estimate that gives different weights to the real and artificial data is introduced. It is investigated under which condition the rate of convergence of this estimate is better than the rate of convergence of an ordinary least-squares estimate applied to the real data only. The results are illustrated using simulated and real data.

Bernstein-Type Inequality for Widely Dependent Sequence and Its Application to Nonparametric Regression Models

We present the Bernstein-type inequality for widely dependent random variables. By using the Bernstein-type inequality and the truncated method, we further study the strong consistency of estimator of fixed design regression model under widely dependent random variables, which generalizes the corresponding one of independent random variables. As an application, the strong consistency for the nearest neighbor estimator is obtained.

The Strong Consistency of the Estimator of Fixed-Design Regression Model under Negatively Dependent Sequences

We study the strong consistency of estimator of fixed design regression model under negatively dependent sequences by using the classical Rosenthal-type inequality and the truncated method. As an application, the strong consistency for the nearest neighbor estimator is obtained.