Normal Linear Regression Model Research Articles

Bayesian variable selection using the hyper‐g prior in WinBUGS

The hyper‐g prior is a default choice for Bayesian variable selection in normal linear regression models. In this article we provide an overview of the Bayesian variable selection framework and explain in detail the specification for the hyper‐g prior setup. The practical implementation of the methods under consideration is demonstrated through the use of WinBUGS software explaining the correspondence between code and theoretical setup. An illustration of results is considered through a simulated data example.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Bayesian Methods and Theory Statistical Models > Model Selection Statistical Models > Linear Models

On Bayes factors for the linear model

SummaryWe show that the Bayes factor for testing whether a subset of coefficients are zero in the normal linear regression model gives the uniformly most powerful test amongst the class of invariant tests discussed in Lehmann & Romano (2005) if the prior distributions for the regression coefficients are in a specific class of distributions. The priors in this class can have any elliptical distribution, with a specific scale matrix, for the subset of coefficients that are being tested. We also show under mild conditions that the Bayes factor gives the uniformly most powerful invariant test only if the prior for the coefficients being tested is an elliptical distribution with this scale matrix. The implications are discussed.

Minimum-volume confidence sets for normal linear regression models

ABSTRACTIn this article, we establish the minimum-volume confidence sets for normal linear regression models, extending the results in Zhang [Minimum volume confidence sets for parameters of normal distributions. Adv Stat Anal. 2017;101:309–320] on building the minimum-volume confidence sets for parameters of normal distributions. Compared with classical confidence sets, the proposed optimal confidence set is proved to have the smallest volume, for whatever confidence level, sample size and sample data.

Estimation of Okun Coefficient for Algeria

The objective of this paper is to investigate the presence of Okun’s (1962) relationship for Algeria for the 1970- 2015 period. Two methodologies are employed to estimate the Okun coefficient: An Autoregressive Distributed Lag (ARDL) linear model and a Bayesian Normal Linear Regression model. The results indicate an Okun coefficient of about -0.2 which suggests some rigidity of the labour market in Algeria.

Association between blood cholesterol and sodium intake in hypertensive women with excess weight.

Restricted sodium intake has been recommended for more than 1 century for the treatment of hypertension. However, restriction seems to increase blood cholesterol. In women with excess weight, blood cholesterol may increase even more because of insulin resistance and the high lipolytic activity of adipose tissue.The aim of this study was to assess the association between blood cholesterol and sodium intake in hypertensive women with and without excess weight.This was a cross-sectional study with hypertensive and nondiabetic women aged 20 to 59 years, recruited at the primary healthcare units of Maceio, Alagoas, Brazilian Northeast. Excess weight was defined as body mass index (BMI) ≥25.0 kg/m2. Sodium intake was estimated by the 24-hour urinary excretion of sodium. Blood cholesterol was the primary outcome investigated by this study, and its relationship with sodium intake and other variables was assessed by Pearson correlation and multivariate linear regression using a significance level of 5%.This study included 165 hypertensive women. Of these, 135 (81.8%) were with excess weight. The mean sodium intake was 3.7 g (±1.9) and 3.4 g (±2.4) in hypertensive women with and without excess weight, respectively. The multiple normal linear regression models fitted to the “blood cholesterol” in the 2 groups reveal that for the group of hypertensive women without excess weight only 1 independent variable “age” is statistically significant to explain the variability of the blood cholesterol levels. However, for the group of hypertensive women with excess weight, 2 independent variables, age and sodium intake, can statistically explain variations of the blood cholesterol levels.Blood cholesterol is statistically inversely related to sodium intake for hypertensive women with excess weight, but it is not statistically related to sodium intake for hypertensive women without excess weight.

A Bayes Factor for Bounding the Treatment Effect to Address Hidden Bias in Linear Regression

A Bayes factor is introduced for the normal linear regression model, which can be used to estimate bounds of the treatment effect on the dependent variable, from the data. This is done while accounting for hidden omitted-variable bias, due to an unobserved covariate, and adjusting for any other observed covariates. The Bayes factor measures how much the data have changed the odds for some specified hidden bias versus no hidden bias, and is defined by a ratio of residual sums-of-squares raised to a power proportional to half the sample size. Therefore, the estimated bounds for the treatment effect can be determined by values of the hidden bias parameter that attain non-small Bayes factors, while the Bayes factor can be quickly computed in closed-form. The Bayes factor is illustrated through the analysis of real data and simulated data sets. Software code for the Bayes factor method is provided as Supplemental Material (available upon request of the author).

Top-down transformation choice

Simple models are preferred over complex models, but over-simplistic models could lead to erroneous interpretations. The classical approach is to start with a simple model, whose shortcomings are assessed in residual-based model diagnostics. Eventually, one increases the complexity of this initial overly simple model and obtains a better-fitting model. I illustrate how transformation analysis can be used as an alternative approach to model choice. Instead of adding complexity to simple models, step-wise complexity reduction is used to help identify simpler and better interpretable models. As an example, body mass index (BMI) distributions in Switzerland are modelled by means of transformation models to understand the impact of sex, age, smoking and other lifestyle factors on a person's BMI. In this process, I searched for a compromise between model fit and model interpretability. Special emphasis is given to the understanding of the connections between transformation models of increasing complexity. The models used in this analysis ranged from evergreens, such as the normal linear regression model with constant variance, to novel models with extremely flexible conditional distribution functions, such as transformation trees and transformation forests.

Objective Bayesian hypothesis testing and estimation for the intraclass model

ABSTRACTThe intraclass correlation coefficient (ICC) plays an important role in various fields of study as a coefficient of reliability. In this paper, we consider objective Bayesian analysis for the ICC in the context of normal linear regression model. We first derive two objective priors for the unknown parameters and show that both result in proper posterior distributions. Within a Bayesian decision-theoretic framework, we then propose an objective Bayesian solution to the problems of hypothesis testing and point estimation of the ICC based on a combined use of the intrinsic discrepancy loss function and objective priors. The proposed solution has an appealing invariance property under one-to-one reparametrisation of the quantity of interest. Simulation studies are conducted to investigate the performance the proposed solution. Finally, a real data application is provided for illustrative purposes.

Variable Selection via Partial Correlation.

Partial correlation based variable selection method was proposed for normal linear regression models by Bühlmann, Kalisch and Maathuis (2010) as a comparable alternative method to regularization methods for variable selection. This paper addresses two important issues related to partial correlation based variable selection method: (a) whether this method is sensitive to normality assumption, and (b) whether this method is valid when the dimension of predictor increases in an exponential rate of the sample size. To address issue (a), we systematically study this method for elliptical linear regression models. Our finding indicates that the original proposal may lead to inferior performance when the marginal kurtosis of predictor is not close to that of normal distribution. Our simulation results further confirm this finding. To ensure the superior performance of partial correlation based variable selection procedure, we propose a thresholded partial correlation (TPC) approach to select significant variables in linear regression models. We establish the selection consistency of the TPC in the presence of ultrahigh dimensional predictors. Since the TPC procedure includes the original proposal as a special case, our theoretical results address the issue (b) directly. As a by-product, the sure screening property of the first step of TPC was obtained. The numerical examples also illustrate that the TPC is competitively comparable to the commonly-used regularization methods for variable selection.

A Chi-Square Approximation for the <i>F</i> Distribution

F distribution is one of the most frequently used distributions in statistics. For example, it is used for testing: equality of variances of two independent normal distributions, equality of means in the one-way ANOVA setting, overall significance of a normal linear regression model, and so on. In this paper, a simple chi-square approximation for the cumulative distribution of the F-distribution is obtained via an adjusted log-likelihood ratio statistic. This new approximation exhibits remarkable accuracy even when the degrees of freedom of the F distribution are small.

The Log-gamma-logistic Regression Model: Estimation, Sensibility and Residual Analysis

In this paper, we formulate and develop a log-linear model using a new distribution called the log-gamma-logistic. We show that the new regression model can be applied to censored data since it represents a parametric family of models that includes as sub-models several widely-known regression models and therefore can be used more effectively in the analysis of survival data. We obtain maximum likelihood estimates of the model parameters by considering censored data and evaluate local influence on the estimates of the parameters by taking different perturbation schemes. Some global-influence measurements are also investigated. Further, for different parameter settings, sample sizes and censoring percentages, various simulations are performed. In addition, the empirical distributions of some modified residuals are displayed and compared with the standard normal distribution. These studies suggest that the residual analysis usually performed in normal linear regression models can be extended to modified deviance residuals in the proposed regression model applied to censored data. We demonstrate that our extended regression model is very useful to the analysis of real data and may give more realistic fits than other special regression models.

The Kumaraswamy normal linear regression model with applications

ABSTRACTFor any continuous baseline G distribution, Cordeiro and Castro pioneered the Kumaraswamy-G family of distributions with two extra positive parameters, which generalizes both Lehmann types I and II classes. We study some mathematical properties of the Kumaraswamy-normal (KwN) distribution including ordinary and incomplete moments, mean deviations, quantile and generating functions, probability weighted moments, and two entropy measures. We propose a new linear regression model based on the KwN distribution, which extends the normal linear regression model. We obtain the maximum likelihood estimates of the model parameters and provide some diagnostic measures such as global influence, local influence, and residuals. We illustrate the potentiality of the introduced models by means of two applications to real datasets.

Statistical disclosure control via sufficiency under the multiple linear regression model

In this article we show, under the normal multiple linear regression model, how synthetic data can be generated using the principle of sufficiency. An advantage of this approach is that if the regression model assumed by the synthetic data producer is correctly specified, then the synthetic data have the same joint distribution as the original data, and therefore one can use standard regression methodology and software to analyze the synthetic data. If the same regression model used to generate the synthetic data is also used for data analysis, and the data are analyzed using standard regression methodology, then the synthetic data yield identical inference to that of the original data. We also study the effects of overfitting or under-fitting the linear regression model. We show that even if the data producer overspecifies the regression model when creating the synthetic data, the synthetic data will still have the same distribution as the original data, and hence valid inference can be obtained. However, if the data producer underspecifies the linear regression model, then one cannot expect to obtain valid inference from the synthetic data. The disclosure risk of the proposed method relative to a standard synthetic data method is also examined.

The performance of model averaged tail area confidence intervals

ABSTRACTWe investigate the exact coverage and expected length properties of the model averaged tail area (MATA) confidence interval proposed by Turek and Fletcher, CSDA, 2012, in the context of two nested, normal linear regression models. The simpler model is obtained by applying a single linear constraint on the regression parameter vector of the full model. For given length of response vector and nominal coverage of the MATA confidence interval, we consider all possible models of this type and all possible true parameter values, together with a wide class of design matrices and parameters of interest. Our results show that, while not ideal, MATA confidence intervals perform surprisingly well in our regression scenario, provided that we use the minimum weight within the class of weights that we consider on the simpler model.

Improved heteroskedasticity likelihood ratio tests in symmetric nonlinear regression models

In this paper we address the issue of testing inference of the dispersion parameter in heteroscedastic symmetric nonlinear regression models considering small samples. We derive Bartlett corrections to improve the likelihood ratio as well modified profile likelihood ratio tests. Our results extend some of those obtained in Cordeiro (J Stat Comput Simul 74:609–620, 2004) and Ferrari et al. (J Stat Plan Inference 124:423–437, 2004), who consider a symmetric nonlinear regression model and normal linear regression model, respectively. We also present the bootstrap and bootstrap Bartlett corrected likelihood ratio tests. Monte Carlo simulations are carried out to compare the finite sample performances of the three corrected tests and their uncorrected versions. The numerical evidence shows that the corrected modified profile likelihood ratio test, the bootstrap and bootstrap Bartlett corrected likelihood ratio test perform better than the other ones. We also present an empirical application.

A non-iterative Bayesian sampling algorithm for censored Student-t linear regression models

ABSTRACTIn this paper, we consider an effective Bayesian inference for censored Student-t linear regression model, which is a robust alternative to the usual censored Normal linear regression model. Based on the mixture representation of the Student-t distribution, we propose a non-iterative Bayesian sampling procedure to obtain independently and identically distributed samples approximately from the observed posterior distributions, which is different from the iterative Markov Chain Monte Carlo algorithm. We conduct model selection and influential analysis using the posterior samples to choose the best fitted model and to detect latent outliers. We illustrate the performance of the procedure through simulation studies, and finally, we apply the procedure to two real data sets, one is the insulation life data with right censoring and the other is the wage rates data with left censoring, and we get some interesting results.

New Flexible Regression Models Generated by Gamma Random Variables with Censored Data

We propose and study a new log-gamma Weibull regression model. We obtain explicit expressions for the raw and incomplete moments, quantile and generating functions and mean deviations of the log-gamma Weibull distribution. We demonstrate that the new regression model can be applied to censored data since it represents a parametric family of models which includes as sub-models several widely-known regression models and therefore can be used more effectively in the analysis of survival data. We obtain the maximum likelihood estimates of the model parameters by considering censored data and evaluate local influence on the estimates of the parameters by taking different perturbation schemes. Some global-influence measurements are also investigated. Further, for different parameter settings, sample sizes and censoring percentages, various simulations are performed. In addition, the empirical distribution of some modified residuals are displayed and compared with the standard normal distribution. These studies suggest that the residual analysis usually performed in normal linear regression models can be extended to a modified deviance residual in the proposed regression model applied to censored data. We demonstrate that our extended regression model is very useful to the analysis of real data and may give more realistic fits than other special regression models.

Consistency of Bayes factors under hyper [formula omitted]-priors with growing model size

In this paper, we examine Bayes factor consistency in the context of Bayesian variable selection for normal linear regression models. We take a hierarchical Bayesian approach using a hyper-g prior (Liang et al. (2008) JASA). There are two regimes for computing Bayes factors, which differ in the choice of the base model. For both these regimes, we study conditions under which Bayes factors are consistent when the number of all the potential regressors grows with sample size n. This situation is not fully understood in the current literature, but has gained increasing importance recently. In the present case, Bayes factors are not analytically tractable and are calculated via Laplace approximation. A rigorous justification for these high-dimensional Laplace approximations is also provided.

Likelihood Based Finite Sample Inference for Singly Imputed Synthetic Data Under the Multivariate Normal and Multiple Linear Regression Models

In this paper we develop likelihood-based finite sample inference based on singly imputed partially synthetic data, when the original data follow either a multivariate normal or a multiple linear regression model. We assume that the synthetic data are generated by using the plug-in sampling method, where unknown parameters in the data model are set equal to observed values of their point estimators based on the original data, and synthetic data are drawn from this estimated version of the model. Empirical studies are presented to show that the proposed methods do indeed perform as the theory predicts, and to compare the proposed methods for singly imputed synthetic data with the combining rules that are used to analyze multiply imputed partially synthetic data. Some theoretical comparisons between singly and multiply imputed partially synthetic data inference are also provided. A data analysis example and disclosure risk evaluation of singly and multiply imputed partially synthetic data is presented based on public use data from the Current Population Survey. We discuss the specific conditions under which the proposed methodology will yield valid inference, and evaluate the performance of the methodology when certain conditions do not hold. We outline some ways to extend the proposed methodology for certain scenarios where the required set of conditions do not hold.

Inference for Singly Imputed Synthetic Data Based on Posterior Predictive Sampling under Multivariate Normal and Multiple Linear Regression Models

Likelihood-based finite sample inference for singly imputed synthetic data generated via posterior predictive sampling is developed in this paper for multivariate normal and multiple linear regression models. Currently available methodology for drawing valid inference on population parameters using synthetic data is based on concepts of multiple imputation for missing data, and therefore requires the release of multiple synthetic datasets. The methodology developed in this paper demonstrates that, contrary to the usual belief, valid inference about meaningful model parameters can indeed be drawn based on a singly imputed synthetic dataset under the multivariate normal and multiple linear regression models, by fully utilizing the model structure.