Biased Regression Coefficients Research Articles

Reducing bias and mitigating the influence of excess of zeros in regression covariates with multi-outcome adaptive LAD-lasso

Zero-inflated explanatory variables, as opposed to outcome variables, are common, for example, in environmental sciences. In this article, we address the problem of having excess of zero values in some continuous explanatory variables, which are subject to multi-outcome lasso-regularized variable selection. In short, the problem results from the failure of the lasso-type of shrinkage methods to recognize any difference between zero value occurring either in the regression coefficient or in the corresponding value of the explanatory variable. This kind of confounding will obviously increase the number of false positives – all non-zero regression coefficients do not necessarily represent true outcome effects. We present here the adaptive LAD-lasso for multiple outcomes, which extends the earlier work of multi-outcome LAD-lasso with adaptive penalization. In addition to well-known property of having less biased regression coefficients, we show that the adaptivity also improves method’s ability to recover from influences of excess of zero values measured in continuous covariates.

Bias in regression coefficient estimates when assumptions for handling missing data are violated: a simulation study


 Background
 The purpose of this simulation study is to assess the performance of multiple imputation compared to complete case analysis when assumptions of missing data mechanisms are violated.
 Methods
 The authors performed a stochastic simulation study to assess the performance of Complete Case (CC) analysis and Multiple Imputation (MI) with different missing data mechanisms (missing completely at random (MCAR), at random (MAR), and not at random (MNAR)). The study focused on the point estimation of regression coefficients and standard errors.
 Results
 When data were MAR conditional on Y, CC analysis resulted in biased regression coefficients; they were all underestimated in our scenarios. In these scenarios, analysis after MI gave correct estimates. Yet, in case of MNAR MI yielded biased regression coefficients, while CC analysis performed well.Conclusion
 The authors demonstrated that MI was only superior to CC analysis in case of MCAR or MAR. In some scenarios CC may be superior over MI. Often it is not feasible to identify the reason why data in a given dataset are missing. Therefore, emphasis should be put on reporting the extent of missing values, the method used to address them, and the assumptions that were made about the mechanism that caused missing data.

Modelling old‐age retirement: An adaptive multi‐outcome LAD‐lasso regression approach

AbstractUsing unique administrative register data, we investigate old‐age retirement under the statutory pension scheme in Finland. The analysis is based on multi‐outcome modelling of pensions and working lives together with a range of explanatory variables. An adaptive multi‐outcome LAD‐lasso regression method is applied to obtain estimates of earnings and socioeconomic factors affecting old‐age retirement and to decide which of these variables should be included in our model. The proposed statistical technique produces robust and less biased regression coefficient estimates in the context of skewed outcome distributions and an excess number of zeros in some of the explanatory variables. The results underline the importance of late life course earnings and employment to the final amount of pension and reveal differences in pension outcomes across socioeconomic groups. We conclude that adaptive LAD‐lasso regression is a promising statistical technique that could be usefully employed in studying various topics in the pension industry.

Accuracy of random-forest-based imputation of missing data in the presence of non-normality, non-linearity, and interaction

BackgroundMissing data are common in statistical analyses, and imputation methods based on random forests (RF) are becoming popular for handling missing data especially in biomedical research. Unlike standard imputation approaches, RF-based imputation methods do not assume normality or require specification of parametric models. However, it is still inconclusive how they perform for non-normally distributed data or when there are non-linear relationships or interactions.MethodsTo examine the effects of these three factors, a variety of datasets were simulated with outcome-dependent missing at random (MAR) covariates, and the performances of the RF-based imputation methods missForest and CALIBERrfimpute were evaluated in comparison with predictive mean matching (PMM).ResultsBoth missForest and CALIBERrfimpute have high predictive accuracy but missForest can produce severely biased regression coefficient estimates and downward biased confidence interval coverages, especially for highly skewed variables in nonlinear models. CALIBERrfimpute typically outperforms missForest when estimating regression coefficients, although its biases are still substantial and can be worse than PMM for logistic regression relationships with interaction.ConclusionsRF-based imputation, in particular missForest, should not be indiscriminately recommended as a panacea for imputing missing data, especially when data are highly skewed and/or outcome-dependent MAR. A correct analysis requires a careful critique of the missing data mechanism and the inter-relationships between the variables in the data.

Comparison of Ridge Regression and Least Squares Methods in the Presence of Multicollinearity for Body Measurements in Saanen Kids

Least square (LS) method is a common method used to estimate the coefficients in multiple regression models. The least square multiple regression models produce biased regression coefficients when the multicollinearity is encountered in the studied data sets. Multicollinearity problem can be solved by using some methods. As one of the methods, Ridge Regression (RR) is a biased estimation method that enables to obtain models having more reliable coefficient of determination (R2). This study was conducted on 40 Saanen kids in order to determine some morphological measurements (withers height, rump height, body length, chest width, chest girth and chest depth) affecting body weight. In this study, usability of ridge regression method in the presence of multicollinearity was evaluated. Variance Inflation Factor (VIF) values higher than 10 were detected for withers height and rump height. Coefficient of determination (R2) was obtained as 0.88 from LS method and R2 was obtained 0.875 with k=0.0136 from RR method. As a result, the model obtained from RR is more reliable than that obtained from LS.

On the Use of Empirical Bayes Estimates as Measures of Individual Traits.

Empirical Bayes (EB) estimates of the random effects in multilevel models represent how individuals deviate from the population averages and are often extracted to detect outliers or used as predictors in follow-up analysis. However, little research has examined whether EB estimates are indeed reliable and valid measures of individual traits. In this article, we use statistical theory and simulated data to show that EB estimates are biased toward zero, a phenomenon known as "shrinkage." The degree of shrinkage and reliability of EB estimates depend on a number of factors, including Level-1 residual variance, Level-1 predictor variance, Level-2 random effects variance, and number of within-person observations. As a result, EB estimates may not be ideal for detecting outliers, and they produce biased regression coefficients when used as predictors. We illustrate these issues using an empirical data set on emotion regulation and neuroticism.

Applications of Measurement Error Correction Approaches in Statistical Road Safety Modeling

Road safety modelers frequently use average annual daily traffic (AADT) as a measure of exposure in regression models of expected crash frequency for road segments and intersections. Recorded AADT values at most locations are estimated by state and local transportation agencies with significant uncertainty, often by extrapolating short-term traffic counts over time and space. This uncertainty in the traffic volume estimates, often termed in a modeling context as measurement error in right-hand-side variables, can have serious effects on model estimation, including: 1) biased regression coefficient estimates; and 2) increases in dispersion. The structure and magnitude of measurement error in AADT estimates are not clearly understood by researchers or practitioners, leading to difficulties in explicitly accounting for this error in statistical road safety models, and ultimately in finding solutions for its correction. This study explores the impacts of measurement error in traffic volume estimates on statistical road safety models by employing measurement error correction approaches, including regression calibration and simulation extrapolation. The concept is demonstrated using crash, traffic, and roadway data from rural, two-lane horizontal curves in the State of Washington. The overall results show that the regression coefficient estimates with a positive coefficient were larger and those with a negative coefficient were smaller (i.e., more negative) when the measurement error correction methods were applied to the regression models of expected crash frequency. Future directions in applications of measurement error correction approaches to road safety research are provided.

A method to account for and estimate underreporting in crash frequency research

Underreporting is a well-known issue in crash frequency research. However, statistical methods that can account for underreporting have received little attention in the published literature. This paper compares results from underreporting models to models that account for unobserved heterogeneity. The difference in the elasticities between the negative binomial underreporting model and random parameters negative binomial models, which accounts for unobserved heterogeneity in crash frequency models, are used as the basis for comparison. The paper also includes a comparison of the predicted number of unreported PDO crashes based on the negative binomial underreporting model with crashes that were reported to police but were not considered reportable to PennDOT to assess the ability of the underreporting models to predict non-reportable crashes.The data used in this study included 21,340 segments of two-lane rural highways that are owned and maintained by PennDOT. Reported accident frequencies over an eight year period (2005⿿2012) were included in the sample, producing a total of 170,468 segment-years of data. The results indicate that if a variable impacts both the true accident frequency and the probability of accidents being reported, statistical modeling methods that ignore underreporting produce biased regression coefficients. The magnitude of the bias in the present study (based on elasticities) ranged from 0.00⿿16.79%. If the variable affects the true accident frequency, but not the probability of accidents being reported, the results from the negative binomial underreporting models are consistent with analysis methods that do not account for underreporting.

Spatial autocorrelation and the scaling of species–environment relationships

Issues of residual spatial autocorrelation (RSA) and spatial scale are critical to the study of species-environment relationships, because RSA invalidates many statistical procedures, while the scale of analysis affects the quantification of these relationships. Although these issues independently are widely covered in the literature, only sparse attention is given to their integration. This paper focuses on the interplay between RSA and the spatial scaling of species-environment relationships. Using a hypothetical species in an artificial landscape, we show that a mismatch between the scale of analysis and the scale of a species' response to its environment leads to a decrease in the portion of variation explained by environmental predictors. Moreover, it results in RSA and biased regression coefficients. This bias stems from error-predictor dependencies due to the scale mismatch, the magnitude of which depends on the interaction between the scale of landscape heterogeneity and the scale of a species' response to this heterogeneity. We show that explicitly considering scale effects on RSA can reveal the characteristic scale of a species' response to its environment. This is important, because the estimation of species-environment relationships using spatial regression methods proves to be erroneous in case of a scale mismatch, leading to spurious conclusions when scaling issues are not explicitly considered. The findings presented here highlight the importance of examining the appropriateness of the spatial scales used in analyses, since scale mismatches affect the rigor of statistical analyses and thereby the ability to understand the processes underlying spatial patterning in ecological phenomena.

Comparison of techniques for handling missing covariate data within prognostic modelling studies: a simulation study

BackgroundThere is no consensus on the most appropriate approach to handle missing covariate data within prognostic modelling studies. Therefore a simulation study was performed to assess the effects of different missing data techniques on the performance of a prognostic model.MethodsDatasets were generated to resemble the skewed distributions seen in a motivating breast cancer example. Multivariate missing data were imposed on four covariates using four different mechanisms; missing completely at random (MCAR), missing at random (MAR), missing not at random (MNAR) and a combination of all three mechanisms. Five amounts of incomplete cases from 5% to 75% were considered. Complete case analysis (CC), single imputation (SI) and five multiple imputation (MI) techniques available within the R statistical software were investigated: a) data augmentation (DA) approach assuming a multivariate normal distribution, b) DA assuming a general location model, c) regression switching imputation, d) regression switching with predictive mean matching (MICE-PMM) and e) flexible additive imputation models. A Cox proportional hazards model was fitted and appropriate estimates for the regression coefficients and model performance measures were obtained.ResultsPerforming a CC analysis produced unbiased regression estimates, but inflated standard errors, which affected the significance of the covariates in the model with 25% or more missingness. Using SI, underestimated the variability; resulting in poor coverage even with 10% missingness. Of the MI approaches, applying MICE-PMM produced, in general, the least biased estimates and better coverage for the incomplete covariates and better model performance for all mechanisms. However, this MI approach still produced biased regression coefficient estimates for the incomplete skewed continuous covariates when 50% or more cases had missing data imposed with a MCAR, MAR or combined mechanism. When the missingness depended on the incomplete covariates, i.e. MNAR, estimates were biased with more than 10% incomplete cases for all MI approaches.ConclusionThe results from this simulation study suggest that performing MICE-PMM may be the preferred MI approach provided that less than 50% of the cases have missing data and the missing data are not MNAR.

Regression Calibration for Dichotomized Mismeasured Predictors

Epidemiologic research focuses on estimating exposure-disease associations. In some applications the exposure may be dichotomized, for instance when threshold levels of the exposure are of primary public health interest (e.g., consuming 5 or more fruits and vegetables per day may reduce cancer risk). Errors in exposure variables are known to yield biased regression coefficients in exposure-disease models. Methods for bias-correction with continuous mismeasured exposures have been extensively discussed, and are often based on validation substudies, where the "true" and imprecise exposures are observed on a small subsample. In this paper, we focus on biases associated with dichotomization of a mismeasured continuous exposure. The amount of bias, in relation to measurement error in the imprecise continuous predictor, and choice of dichotomization cut point are discussed. Measurement error correction via regression calibration is developed for this scenario, and compared to naïvely using the dichotomized mismeasured predictor in linear exposure-disease models. Properties of the measurement error correction method (i.e., bias, mean-squared error) are assessed via simulations.

Proportional hazards models based on biased samples and estimated selection probabilities

AbstractIn non‐randomized biomedical studies using the proportional hazards model, the data often constitute an unrepresentative sample of the underlying target population, which results in biased regression coefficients. The bias can be avoided by weighting included subjects by the inverse of their respective selection probabilities, as proposed by Horvitz & Thompson (1952) and extended to the proportional hazards setting for use in surveys by Binder (1992) and Lin (2000). In practice, the weights are often estimated and must be treated as such in order for the resulting inference to be accurate. The authors propose a two‐stage weighted proportional hazards model in which, at the first stage, weights are estimated through a logistic regression model fitted to a representative sample from the target population. At the second stage, a weighted Cox model is fitted to the biased sample. The authors propose estimators for the regression parameter and cumulative baseline hazard. They derive the asymptotic properties of the parameter estimators, accounting for the difference in the variance introduced by the randomness of the weights. They evaluate the accuracy of the asymptotic approximations in finite samples through simulation. They illustrate their approach in an analysis of renal transplant patients using data obtained from the Scientific Registry of Transplant Recipients

A Reexamination of Congressional Primary Turnout

Using original data and ordinary least squares regression analysis, this article examines primary turnout for elections to the House of Representatives in 1992 and 1994. These data are compared to earlier estimates of primary turnout that used a substitute measure. The author finds that the previous measure biased regression coefficients, inaccurately estimating the impact of political party and overestimating the influence of some variables while failing to capture the effects of others. Future research ought to rely on the new measure, used here, because substantive conclusions drawn from analyses that use the old measure are in jeopardy.

Software Reviews : Lisrel/Prelis

LISREL/PRELIS has become the statistical package of choice for many social science graduate students and researchers who wish to develop multivariate linear models that handle measurement error and latent variables in a more sophisticated way than do traditional regression models. LISREL models are also partic-ularly appropriate when there is reciprocal causation or interdependence among causal variables. LISREL is a multivariate linear modeling method developed by Karl Joreskog (Uppsala University, Sweden). It addresses the problem of biased regression coefficients caused by use of variables that are measured with error, as is the rule in social science.