Vector Of Regressors Research Articles

A robust permutation test for subvector inference in linear regressions

We develop a new permutation test for inference on a subvector of coefficients in linear models. The test is exact when the regressors and the error terms are independent. Then we show that the test is asymptotically of correct level, consistent, and has power against local alternatives when the independence condition is relaxed, under two main conditions. The first is a slight reinforcement of the usual absence of correlation between the regressors and the error term. The second is that the number of strata, defined by values of the regressors not involved in the subvector test, is small compared to the sample size. The latter implies that the vector of nuisance regressors is discrete. Simulations and empirical illustrations suggest that the test has good power in practice if, indeed, the number of strata is small compared to the sample size.

Seemingly unrelated clusterwise linear regression

Linear regression models based on finite Gaussian mixtures represent a flexible tool for the analysis of linear dependencies in multivariate data. They are suitable for dealing with correlated response variables when data come from a heterogeneous population composed of two or more sub-populations, each of which is characterised by a different linear regression model. Several types of finite mixtures of linear regression models have been specified by changing the assumptions on the parameters that differentiate the sub-populations and/or the vectors of regressors that affect the response variables. They are made more flexible in the class of models defined by mixtures of seemingly unrelated Gaussian linear regressions illustrated in this paper. With these models, the researcher is enabled to use a different vector of regressors for each dependent variable. The proposed class includes parsimonious models obtained by imposing suitable constraints on the variances and covariances of the response variables in the sub-populations. Details about the model identification and maximum likelihood estimation are given. The usefulness of these models is shown through the analysis of a real dataset. Regularity conditions for the model class are illustrated and a proof is provided that, when these conditions are met, the consistency of the maximum likelihood estimator under the examined models is ensured. In addition, the behaviour of this estimator in the presence of finite samples is numerically evaluated through the analysis of simulated datasets.

Prediction when fitting simple models to high-dimensional data

We study linear subset regression in the context of a high-dimensional linear model. Consider $y=\vartheta +\theta 'z+\epsilon $ with univariate response $y$ and a $d$-vector of random regressors $z$, and a submodel where $y$ is regressed on a set of $p$ explanatory variables that are given by $x=M'z$, for some $d\times p$ matrix $M$. Here, “high-dimensional” means that the number $d$ of available explanatory variables in the overall model is much larger than the number $p$ of variables in the submodel. In this paper, we present Pinsker-type results for prediction of $y$ given $x$. In particular, we show that the mean squared prediction error of the best linear predictor of $y$ given $x$ is close to the mean squared prediction error of the corresponding Bayes predictor $\mathbb{E}[y\|x]$, provided only that $p/\log d$ is small. We also show that the mean squared prediction error of the (feasible) least-squares predictor computed from $n$ independent observations of $(y,x)$ is close to that of the Bayes predictor, provided only that both $p/\log d$ and $p/n$ are small. Our results hold uniformly in the regression parameters and over large collections of distributions for the design variables $z$.

Impact of feature selection on system identification by means of NARX-SVM

Support Vector Machines (SVM) are widely used in many fields of science, including system identification. The selection of feature vector plays a crucial role in SVM-based model building process. In this paper, we investigate the influence of the selection of feature vector on model’s quality. We have built an SVM model with a non-linear ARX (NARX) structure. The modelled system had a SISO structure, i.e. one input signal and one output signal. The output signal was temperature, which was controlled by a Peltier module. The supply voltage of the Peltier module was the input signal. The system had a non-linear characteristic. We have evaluated the model’s quality by the fit index. The classical feature selection of SVM with NARX structure comes down to a choice of the length of the regressor vector. For SISO models, this vector is determined by two parameters: nu and ny. These parameters determine the number of past samples of input and output signals of the system used to form the vector of regressors. In the present research we have tested two methods of building the vector of regressors, one classic and one using custom regressors. The results show that the vector of regressors obtained by the classical method can be shortened while maintaining the acceptable quality of the model. By using custom regressors, the feature vector of SVM can be reduced, which means also the reduction in calculation time.

Some tests for the allometric extension model in regression

ABSTRACTThe allometric extension model is a multivariate regression model recently proposed by Tarpey and Ivey (2006). This model holds when the matrix of covariances between the variables in the response vector y and the variables in the vector of regressors x has a particular structure. In this paper, we consider tests of hypotheses for this structure when (y′, x′)′ has a multivariate normal distribution. In particular, we investigate the likelihood ratio test and a Wald test.

Goodness-of-fit tests based on series estimators in nonparametric instrumental regression

This paper proposes several tests of restricted specification in nonparametric instrumental regression. Based on series estimators, test statistics are established that allow for tests of the general model against a parametric or nonparametric specification as well as a test of exogeneity of the vector of regressors. The tests’ asymptotic distributions under correct specification are derived and their consistency against any alternative model is shown. Under a sequence of local alternative hypotheses, the asymptotic distributions of the tests are derived. Moreover, uniform consistency is established over a class of alternatives whose distance to the null hypothesis shrinks appropriately as the sample size increases. A Monte Carlo study examines finite sample performance of the test statistics.

Two-component mixtures with independent coordinates as conditional mixtures: Nonparametric identification and estimation

We show how the multivariate two-component mixtures with independent coordinates in each component by Hall and Zhou (2003) can be studied within the framework of conditional mixtures as recently introduced by Henry, Kitamura and Salanié (2010). Here, the conditional distribution of the random variable $Y$ given the vector of regressors $Z$ can be expressed as a two-component mixture, where only the mixture weights depend on the covariates. Under appropriate tail conditions on the characteristic functions and the distribution functions of the mixture components, which allow for flexible location-scale type mixtures, we show identification and provide asymptotically normal estimators. The main application for our results are bivariate two-component mixtures with independent coordinates, the case not previously covered by Hall and Zhou (2003). In a simulation study we investigate the finite-sample performance of the proposed methods. The main new technical ingredient is the estimation of limits of quotients of two characteristic functions in the tails from independent samples, which might be of some independent interest.

Model Specification between Parametric and Nonparametric Cointegration

This paper considers a general model specification between a parametric co-integrating model and a nonparametric co-integrating model in a multivariate regression model, which involves a univariate integrated time series regressor and a vector of stationary time series regressors. A new and simple test is proposed and the resulting asymptotic theory is established. The test statistic is constructed based on a natural distance function between a nonparametric estimate and a smoothed parametric counterpart. The asymptotic distribution of the test statistic under the parametric specification is proportional to that of a local-time random variable with a known distribution. In addition, the finite sample performance of the proposed test is evaluated through using both simulated and real data examples.

Identification and Estimation of Average Partial Effects in "Irregular" Correlated Random Coefficient Panel Data Models

In this paper we study identification and estimation of a correlated random coefficients (CRC) panel data model. The outcome of interest varies linearly with a vector of endogenous regressors. The coefficients on these regressors are heterogenous across units and may covary with them. We consider the average partial effect (APE) of a small change in the regressor vector on the outcome (cf. Chamberlain (1984), Wooldridge (2005a)). Chamberlain (1992) calculated the semiparametric efficiency bound for the APE in our model and proposed a √N-consistent estimator. Nonsingularity of the APE's information bound, and hence the appropriateness of Chamberlain's (1992) estimator, requires (i) the time dimension of the panel (T) to strictly exceed the number of random coefficients (p) and (ii) strong conditions on the time series properties of the regressor vector. We demonstrate irregular identification of the APE when T = p and for more persistent regressor processes. Our approach exploits the different identifying content of the subpopulations of stayers—or units whose regressor values change little across periods—and movers—or units whose regressor values change substantially across periods. We propose a feasible estimator based on our identification result and characterize its large sample properties. While irregularity precludes our estimator from attaining parametric rates of convergence, its limiting distribution is normal and inference is straightforward to conduct. Standard software may be used to compute point estimates and standard errors. We use our methods to estimate the average elasticity of calorie consumption with respect to total outlay for a sample of poor Nicaraguan households.

Orthogonal least squares regression with tunable kernels

A novel technique is proposed to construct sparse regression models based on the orthogonal least squares method with tunable kernels. The proposed technique tunes the centre vector and diagonal covariance matrix of individual regressors by incrementally minimising the training mean square error using a guided random search algorithm, and it offers a state-of-the-art method for constructing very sparse models that generalise well.

Analysis of semi-parametric regression models with non-ignorable non-response.

In this article we consider the problem of making inferences about the parameter beta zero indexing the conditional mean of an outcome given a vector of regressors when a subset of the variables (outcome or covariates) are missing for some study subjects and the probability of non-response depends upon both observed and unobserved data values, that is, non-response is non-ignorable. We propose a new class of inverse probability of censoring weighted estimators that are consistent and asymptotically normal (CAN) for estimating beta zero when the non-response probabilities can be parametrically modelled and a CAN estimator exists. The proposed estimators do not require full specification of the likelihood and their computation does not require numerical integration. We show that the asymptotic variance of the optimal estimator in our class attains the semi-parametric variance bound for the model. In some models, no CAN estimator of beta zero exists. We provide a general algorithm for determining when CAN estimators of beta zero exist. Our results follow after specializing a general representation described in the article for the efficient score and the influence function of regular, asymptotically linear estimators in an arbitrary semi-parametric model with non-ignorable non-response in which the probability of observing complete data is bounded away from zero and the non-response probabilities can be parametrically modelled.

Sensitivity analysis for Box-Cox power transformation model: Contrast parameters

SUMMARY Sensitivity analysis is an important tool for the evaluation of scientific models. We consider the Box-Cox power transformation model and study the sensitivity for the estimated slope vector when the transformation parameter is perturbed. This is a key issue in the debate among Bickel & Doksum (1981), Box & Cox (1982) and Hinkley & Runger (1984) on the application of the power transformation model. Hinkley & Runger (1984) conjectured that Box & Cox's (1964) z transformation would eliminate asymptotically the sensitivity for the estimated slope vector. We establish this conjecture under appropriate symmetry conditions on the joint distribution for the regressors, x. When the true transformation is logarithmic, the conjecture holds if the distribution for x is axially symmetric. For other power transformations, the conjecture holds if x is multivariate normal. We also consider a weaker version of Hinkley & Runger's conjecture, namely, the z transformation would achieve the best possible reduction in the sensitivity. We establish this weaker conjecture under less restrictive symmetry conditions which only involve the marginal distribution for xfl. Both conjectures might fail when those symmetry conditions are not satisfied. Two examples are given to illustrate the limitations of the conjectures. Sensitivity analysis is an important tool for the evaluation of scientific models. Many scientific models include some nuisance parameters which are not known precisely. It is usually desirable for the model output to be stable with respect to those parameters. The performance of the model can be questionable if the model output is highly sensitive to those parameters. It is therefore useful to carry out a sensitivity analysis, in which those parameters are perturbed, the model output is derived under the perturbed parameters and compared with the model output prior to the perturbation. We will study the sensitivity analysis for the Box-Cox power transformation model with respect to the specific transformation taken. Let {(yi, xi), i = 1, . . . , n} be the observed data, where yi denotes a positive scalar outcome, and xi denotes a p-dimensional row vector of regressors. Let the transformed outcome be

A Consistent Conditional Moment Test of Functional Form

In this paper, it will be shown that any conditional moment test of functional form of nonlinear regression models can be converted into a chi-square test that is consistent against all deviations from the null hypothesis that the model represents the conditional expectation of the dependent variable relative to the vector of regressors. Copyright 1990 by The Econometric Society.