Nonparametric Least-squares Research Articles

A simple nonparametric least-squares-based causal inference for heterogeneous treatment effects

Estimating treatment effects is a common practice in making causal inferences. However, it is a challenging task for observational studies because the underlying models for outcome and treatment assignment are unknown. The concept of potential outcomes has been widely adopted in the literature on causal inferences. Building on potential outcomes, we propose a simple nonparametric least-squares spline-based causal inference method to estimate heterogeneous treatment effects in this manuscript. We use empirical process theory to study its asymptotic properties and conduct simulation studies to evaluate its operational characteristics. Based on the estimated heterogeneous treatment effects, we further estimate the average treatment effect and show the asymptotic normality of the estimator. Finally, we apply the proposed method to assess the biological anti-rheumatic treatment effect on children with newly onset juvenile idiopathic arthritis disease using electronic health records from a longitudinal study at Cincinnati Children's Hospital Medical Center.

On a projection estimator of the regression function derivative

ABSTRACT In this paper, we study the estimation of the derivative of a regression function in a standard univariate regression model. The estimators are defined either by derivating nonparametric least-squares estimators of the regression function or by estimating the projection of the derivative. We prove two simple risk bounds allowing to compare our estimators. More elaborate bounds under a stability assumption are then provided. Bases and spaces on which we can illustrate our assumptions and first results are both of compact or noncompact type, and we discuss the rates reached by our estimators. They turn out to be optimal in the compact case. Lastly, we propose a model selection procedure and prove the associated risk bound. To consider bases with a noncompact support makes the problem difficult.

Nonparametric least squares methods for stochastic frontier models

When analyzing productivity and efficiency of firms, stochastic frontier models are very attractive because they allow, as in typical regression models, to introduce some noise in the Data Generating Process. Most of the approaches so far have been using very restrictive fully parametric specified models, both for the frontier function and for the components of the stochastic terms. Recently, local MLE approaches were introduced to relax these parametric hypotheses. However, the high computational complexity of the latter makes them difficult to use, in particular if bootstrap-based inference is needed. In this work we show that most of the benefits of the local MLE approach can be obtained with less assumptions and involving much easier, faster and numerically more robust computations, by using nonparametric least-squares methods. Our approach can also be viewed as a semi-parametric generalization of the so-called “modified OLS” that was introduced in the parametric setup. If the final evaluation of individual efficiencies requires, as in the local MLE approach, the local specification of the distributions of noise and inefficiencies, it is shown that a lot can be learned on the production process without such specifications. Even elasticities of the mean inefficiency can be analyzed with unspecified noise distribution and a general class of local one-parameter scale family for inefficiencies. This allows to discuss the variation in inefficiency levels with respect to explanatory variables with minimal assumptions on the Data Generating Process. Our method is illustrated and compared with other methods with a real data set.

A least-squares approach to anomaly detection in static and sequential data

We describe a probabilistic, nonparametric method for anomaly detection, based on a squared-loss objective function which has a simple analytical solution. The method emerges from extending recent work in nonparametric least-squares classification to include a “none-of-the-above” class which models anomalies in terms of non-anamalous training data. The method shares the flexibility of other kernel-based anomaly detection methods, yet is typically much faster to train and test. It can also be used to distinguish between multiple inlier classes and anomalies. The probabilistic nature of the output makes it straightforward to apply even when test data has structural dependencies; we show how a hidden Markov model framework can be incorporated in order to identify anomalous subsequences in a test sequence. Empirical results on datasets from several domains show the method to have comparable discriminative performance to popular alternatives, but with a clear speed advantage.

Nonparametric Identification of LPV Models Under General Noise Conditions: An LS-SVM Based Approach

Parametric identification approaches in the Linear Parameter-Varying (LPV) setting require optimal prior selection of a set of functional dependencies, used in the parametrization of the model coefficients, to provide accurate model estimates of the underlying system. Consequently, data-driven estimation of these functional dependencies has a paramount importance, especially when very limited a priori knowledge is available. Existing overparametrization and nonparametric methods dedicated to nonlinear estimation offer interesting starting points for this problem, but need reformulation to be applied in the LPV setting. Moreover, most of these approaches are developed under quite restrictive auto-regressive noise assumptions. In this paper, a nonparametric Least-Squares Support Vector Machine (LS-SVM) approach is extended for the identification of LPV polynomial models. The efficiency of the approach in the considered noise setting is shown, but the drawback of the auto-regressive noise assumption is also demonstrated by a challenging LPV identification example. To preserve the attractive properties of the approach, but to overcome the drawbacks in the estimation of polynomial LPV models in a general noise setting, a recently developed Instrumental Variable (IV)-based extension of the LS-SVM method is applied. The performance of the introduced IV and the original LS-SVM approaches are compared in an identification study of an LPV system with unknown noise dynamics.

Nonparametric least squares estimation in derivative families

Cost function estimation often involves data on a function and a family of its derivatives. Such data can substantially improve convergence rates of nonparametric estimators. We propose series-type estimators which incorporate the various derivative data into a single nonparametric least-squares procedure. Convergence rates are obtained and it is shown that for low-dimensional cases, much of the beneficial impact is realized even if only data on ordinary first-order partials are available. In instances where root- n consistency is attained, smoothing parameters can often be chosen very easily, without resort to cross-validation. Simulations and an illustration of cost function estimation are included.

Data Envelopment Analysis as Nonparametric Least-Squares Regression

Data envelopment analysis (DEA) is known as a nonparametric mathematical programming approach to productive efficiency analysis. In this paper, we show that DEA can be alternatively interpreted as nonparametric least-squares regression subject to shape constraints on the frontier and sign constraints on residuals. This reinterpretation reveals the classic parametric programming model by Aigner and Chu [Aigner, D., S. Chu. 1968. On estimating the industry production function. Amer. Econom. Rev. 58 826–839] as a constrained special case of DEA. Applying these insights, we develop a nonparametric variant of the corrected ordinary least-squares (COLS) method. We show that this new method, referred to as corrected concave nonparametric least squares (C2NLS), is consistent and asymptotically unbiased. The linkages established in this paper contribute to further integration of the econometric and axiomatic approaches to efficiency analysis.

Representation theorem for convex nonparametric least squares

Summary We examine a nonparametric least-squares regression model that endogenously selects the functional form of the regression function from the family of continuous, monotonic increasing and globally concave functions that can be nondifferentiable. We show that this family of functions can be characterized without a loss of generality by a subset of continuous, piece-wise linear functions whose intercept and slope coefficients are constrained to satisfy the required monotonicity and concavity conditions. This representation theorem is useful at least in three respects. First, it enables us to derive an explicit representation for the regression function, which can be used for assessing marginal properties and for the purposes of forecasting and ex post economic modelling. Second, it enables us to transform the infinite dimensional regression problem into a tractable quadratic programming (QP) form, which can be solved by standard QP algorithms and solver software. Importantly, the QP formulation applies to the general multiple regression setting. Third, an operational computational procedure enables us to apply bootstrap techniques to draw statistical inference.

A Note on the Nonparametric Least-squares Test for Checking a Polynomial Relationship

Recently, Gijbels and Rousson[6] suggested a new approach, called nonparametric least-squares test, to check polynomial regression relationships. Although this test procedure is not only simple but also powerful in most cases, there are several other parameters to be chosen in addition to the kernel and bandwidth. As shown in their paper, choice of these parameters is crucial but sometimes intractable. We propose in this paper a new statistic which is based on sample variance of the locally estimated pth derivative of the regression function at each design point. The resulting test is still simple but includes no extra parameters to be determined besides the kernel and bandwidth that are necessary for nonparametric smoothing techniques. Comparison by simulations demonstrates that our test performs as well as or even better than Gijbels and Rousson’s approach. Furthermore, a real-life data set is analyzed by our method and the results obtained are satisfactory.

A nonparametric least-squares test for checking a polynomial relationship

In this paper the interest is in testing whether a regression function is a polynomial of a certain degree. One possible approach to this testing problem is to do a parametric polynomial fit and a nonparametric fit and to reject the null hypothesis of a polynomial function if the distance between the two fits is too large. Another approach consists of looking at the residuals from the parametric fit. In this paper we propose an entirely new approach to deal with the testing problem. When testing whether a regression function is a polynomial of degree smaller than or equal to p, the key idea is to first obtain a nonparametric local polynomial estimate of the pth derivative of the unknown regression function, and then to proceed with a classical least-squares test for a general linear model for testing whether this derivative is constant. This is a quite appealing approach since it just relies on ordinary least-squares tests, and hence is simple to use. The performance of the method is illustrated via a simulation study.

Reduced-rank least squares channel estimation

A reduced-rank least squares (RRLS) algorithm based on oversampling the channel output by an integer factor and singular value decomposition (SVD) of a data matrix is shown to have certain advantages over the nonparametric least-squares (LS) and the parametric Evans and Fischl (EF) alternative algorithm for estimating the pulse response of a truncated equivalent baseband communication channel. By sampling the channel output faster than the training symbol rate and applying SVD to the data matrix formed from the observed data, the method is shown to exhibit improved-error performance over existing nonparametric LS methods and the parametric EF iterative algorithm. The RRLS algorithm's performance has been shown to be somewhat sensitive to model order selection and observation noise statistics. The normalized mean squared error (MSE) performance of the RRLS algorithm is shown to be essentially independent of oversampling factors that are not much greater than the span of the truncated channel. It performs well even in severe noise environments.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

4530148 Tool changing mechanism : Walter Flamme, Oskar Rahn, Wegberg Beeckerheide, Federal Republic Of Germany assigned to L Schuler GmbH

Cost function estimation often involves data on a function and a family of its derivatives. Such data can substantially improve convergence rates of nonparametric estimators. We propose series-type estimators which incorporate the various derivative data into a single nonparametric least-squares procedure. Convergence rates are obtained and it is shown that for low-dimensional cases, much of the beneficial impact is realized even if only data on ordinary first-order partials are available. In instances where root-n consistency is attained, smoothing parameters can often be chosen very easily, without resort to cross-validation. Simulations and an illustration of cost function estimation are included.