Linear Combination Of Explanatory Variables Research Articles

Using Canonical Quantile Regression to predict company performance: better prediction than using CEO compensation

In using multiple regression methods for prediction, one often considers the linear combination of explanatory variables as an index. Seeking a single such index when there are multiple responses is rather more complicated. One classical approach is to use the coefficients from the leading Canonical Correlation. However, methods based on variances are unable to disaggregate responses by quantile effects, lack robustness, and rely on normal assumptions for inference. To address these problems, a novel regression quantile approach will be applied to an empirical study of the performance of large publicly held companies and CEO compensation.

An investigation of penalization and data augmentation to improve convergence of generalized estimating equations for clustered binary outcomes

BackgroundIn binary logistic regression data are ‘separable’ if there exists a linear combination of explanatory variables which perfectly predicts the observed outcome, leading to non-existence of some of the maximum likelihood coefficient estimates. A popular solution to obtain finite estimates even with separable data is Firth’s logistic regression (FL), which was originally proposed to reduce the bias in coefficient estimates. The question of convergence becomes more involved when analyzing clustered data as frequently encountered in clinical research, e.g. data collected in several study centers or when individuals contribute multiple observations, using marginal logistic regression models fitted by generalized estimating equations (GEE). From our experience we suspect that separable data are a sufficient, but not a necessary condition for non-convergence of GEE. Thus, we expect that generalizations of approaches that can handle separable uncorrelated data may reduce but not fully remove the non-convergence issues of GEE.MethodsWe investigate one recently proposed and two new extensions of FL to GEE. With ‘penalized GEE’ the GEE are treated as score equations, i.e. as derivatives of a log-likelihood set to zero, which are then modified as in FL. We introduce two approaches motivated by the equivalence of FL and maximum likelihood estimation with iteratively augmented data. Specifically, we consider fully iterated and single-step versions of this ‘augmented GEE’ approach. We compare the three approaches with respect to convergence behavior, practical applicability and performance using simulated data and a real data example.ResultsOur simulations indicate that all three extensions of FL to GEE substantially improve convergence compared to ordinary GEE, while showing a similar or even better performance in terms of accuracy of coefficient estimates and predictions. Penalized GEE often slightly outperforms the augmented GEE approaches, but this comes at the cost of a higher burden of implementation.ConclusionsWhen fitting marginal logistic regression models using GEE on sparse data we recommend to apply penalized GEE if one has access to a suitable software implementation and single-step augmented GEE otherwise.

Performances of some high dimensional regression methods: sparse principal component regression

Principal component analysis (PCA) is widely used technique in data processing and dimensionality reduction, but it has some drawbacks since each principal component is a linear combination of explanatory variables. As an alternative, the sparse PCA (SPCA) is a very appealing method which produces principal components with sparse loadings. On the other hand, combining PCA on explanatory variables with least squares regression yields to principal component regression (PCR). In PCR, the components are obtained using only explanatory variables, not considered the effect of the dependent variable. Considering the dependent variable, the sparse PCR (SPCR) enables to obtain sparse principal component loadings. But the main drawback of it is the computational cost. Taking into consideration the general structure of PCR, we combine (S)PCA with some sparse regression methods and compared with the classical PCR and last introduced method, SPCR. Extensive simulation studies and real data examples are implemented to show their performances. The results are supported by a reasonable computation time study.

Simultaneous variable selection for the classification of near infrared spectra

This paper addresses the problem of wood wastes recycling automation. We propose variable selection methods based on near infrared spectroscopic data to select a set of wavebands that captures the main spectral peaks of wood materials to improve the sorting performances. The spectra are first jointly modeled as linear combinations of explanatory variables drawn from a collection of Gaussian-shaped functions. The aim is to select a common subset of wavebands shared by several spectra. The variable selection is then formulated as an unconstrained simultaneous sparse approximation problem in which the coefficients related to different spectra are encouraged to be piecewise constant, i.e. the coefficients associated to successive spectra should have comparable magnitudes. We also investigate the case where the coefficients are constrained to be nonnegative. These problems are solved using the fast iterative shrinkage-thresholding algorithm. The proposed approaches are illustrated on a dataset of 290 spectra of wood wastes; each spectrum is composed of 1647 wavelengths. We show that the selected variables lead to better classification performances as compared to standard approaches.

Semiparametric estimation of a class of generalized linear models without smoothing

In a generalized linear model, the mean of the response variable is a possibly non-linear function of a linear combination of explanatory variables. When the nonlinear function is unknown and is estimated nonparametrically from the data, these models are known as single index models. Using the relation of generalized linear models with the exponential family model, this paper shows how to use a modified version of the empirical cumulant generating function to estimate the linear function of the explanatory variables with no need of smoothing techniques. The resulting estimator is consistent and normally distributed. Extensive simulations, partially reported here, show that the method works in practice. The method can also be seen as complementary to existing fully nonparametric methods. In fact, it can provide an initial value that can be used to fine tune a nonparametric estimator of the link function in the first step of the estimation.

엑셀 VBA를 이용한 이분형 로지스틱 회귀모형 교육도구 개발

이분형 로지스틱 회귀분석은 양적 혹은 질적 설명변수를 이용해서 이분형 반응변수를 설명하는 하나의 통계적인 기법이다. 이 모형에서는 반응변수가 1이 될 확률을 설명변수들의 선형결합의 변환(혹은 함수)으로 설명하고자 한다. 이 개념에 대한 이해가 비통계학자들이 이분형 로지스틱 회귀모형을 이해하는데 있어서 넘어야 할 커다란 장벽 중의 하나이다. 이 연구에서는 이분형 로지스틱 회귀모형의 필요성을 엑셀 VBA를 이용하여 설명하는 교육도구를 개발하고자 한다. 반응변수가 1이 될 확률을 설명변수의 선형함수로 모형화 할 때의 문제점과 선형결합에 대한 변환을 통해 이 문제점이 어떻게 해소되는지 보여준다. Binary logistic regression analysis is a statistical technique that explains binary response variable by quantitative or qualitative explanatory variables. In the binary logistic regression model, the probability that the response variable equals, say 1, one of the binary values is to be explained as a transformation of linear combination of explanatory variables. This is one of big barriers that non-statisticians have to overcome in order to understand the model. In this study, an educational tool is developed that explains the need of the binary logistic regression analysis using Excel VBA. More precisely, this tool explains the problems related to modeling the probability of the response variable equal to 1 as a linear combination of explanatory variables and then shows how these problems can be solved through some transformations of the linear combination.

Evaluating explanatory models of the spatial pattern of surface climate trends using model selection and bayesian averaging methods

We evaluate three categories of variables for explaining the spatial pattern of warming and cooling trends over land: predictions of general circulation models (GCMs) in response to observed forcings; geographical factors like latitude and pressure; and socioeconomic influences on the land surface and data quality. Spatial autocorrelation (SAC) in the observed trend pattern is removed from the residuals by a well-specified explanatory model. Encompassing tests show that none of the three classes of variables account for the contributions of the other two, though 20 of 22 GCMs individually contribute either no significant explanatory power or yield a trend pattern negatively correlated with observations. Non-nested testing rejects the null hypothesis that socioeconomic variables have no explanatory power. We apply a Bayesian Model Averaging (BMA) method to search over all possible linear combinations of explanatory variables and generate posterior coefficient distributions robust to model selection. These results, confirmed by classical encompassing tests, indicate that the geographical variables plus three of the 22 GCMs and three socioeconomic variables provide all the explanatory power in the data set. We conclude that the most valid model of the spatial pattern of trends in land surface temperature records over 1979–2002 requires a combination of the processes represented in some GCMs and certain socioeconomic measures that capture data quality variations and changes to the land surface.

A Covariance Regression Model

Classical regression analysis relates the expectation of a response variable to a linear combination of explanatory variables. In this article, we propose a covariance regression model that parameterizes the covariance matrix of a multivariate response vector as a parsimonious quadratic function of explanatory variables. The approach is analogous to the mean regression model, and is similar to a factor analysis model in which the factor loadings depend on the explanatory variables. Using a random-effects representation, parameter estimation for the model is straightforward using either an EM-algorithm or an MCMC approximation via Gibbs sampling. The proposed methodology provides a simple but flexible representation of heteroscedasticity across the levels of an explanatory variable, improves estimation of the mean function and gives better calibrated prediction regions when compared to a homoscedastic model.

Locally Adaptive Semiparametric Estimation of the Mean and Variance Functions in Regression Models

This article proposes a Bayesian method for estimating a heteroscedastic regression model with Gaussian errors, where the mean and the log variance are modeled as linear combinations of explanatory variables. We use Bayesian variable selection priors and model averaging to make the estimation more efficient. The model is made semiparametric by allowing explanatory variables to enter the mean and log variance flexibly by representing a covariate effect as a linear combination of basis functions. Our methodology for estimating flexible effects is locally adaptive in the sense that it works well when the flexible effects vary rapidly in some parts of the predictor space but only slowly in other parts. Our article develops an efficient Markov chain Monte Carlo simulation method to sample from the posterior distribution and applies the methodology to a number of simulated and real examples.

Locally Adaptive Semiparametric Estimation of the Mean and Variance Functions in Regression Models

This article considers the estimation of a regression model with Gaussian errors, where the mean and the log variance are modeled as a linear combination of explanatory variables. We consider Bayesian variable selection priors and model averaging to obtain efficient estimators when the number of explanatory variables is large. To make the model semiparametric using this framework we allow explanatory variables to enter the mean and log variance models flexibly by representing a covariate effect as a linear combination of basis functions. Our methodology for estimating flexible effects is locally adaptive in the sense that it works well when the flexible effects vary rapidly in some parts of the predictor space but only slowly in other parts. The whole model is estimated using a Markov chain simulation method that samples the posterior distribution with coefficients in the mean model integrated out analytically and highly dependent parameters generated in blocks. The methodology in the paper is applied to a number of simulated and real examples and is shown to work well.

Empirical Bayes calculations of concordance between endpoints in environmental toxicity experiments

Hierarchical models are considered for estimating the probability of agreement between two outcomes or endpoints from an environmental toxicity experiment. Emphasis is placed on generalized regression models, under which the prior mean is related to a linear combination of explanatory variables via a monotone function. This function defines the scale over which the systematic effects are modelled as additive. Specific illustration is provided for the logistic link function. The hierarchical model employs a conjugate beta prior that leads to parametric empirical Bayes estimators of the individual agreement parameters. An example from environmental carcinogenesis illustrates the methods, with motivation derived from estimation of the concordance between two species carcinogenicity outcomes. Based on a large database of carcinogenicity studies, the inter-species concordance is seen to be reasonably informative, i.e. in the range 67–84%. Stratification into pertinent potency-related sub-groups via the logistic model is seen to improve concordance estimation: for environmental stimuli at the extremes of the potency spectrum, concordance can reach well above 90%.