Number Of Regression Coefficients Research Articles

Relationship of EEG and tremorogram indicators

The existing facts and hypotheses regarding the localization of the "pacemaker" of tremor are rather controversial. Fundamentally, the development of tremor (TrG) is explained by the formation of an oscillator (a source of rhythmic activity) at any level of the central nervous system. The objective is to study the mutually oriented influences between the frequency and amplitude of the TRH rhythms and the electroencephalogram (EEG), reflecting the work of the generators (oscillators) of both oscillatory processes. It was revealed that more regression coefficients from the EEG indicators of both the right and left hemispheres were determined for the left hand TRG indicators. This may indicate that the generators that form the physiological tremor of the left and right hand are not identical, as well as that the physiological tremor of the left hand is to a greater extent controlled by the generators that form the EEG than the generators that form the tremor of the right hand. The largest number of regression coefficients of multiple regression analysis was determined by the rate of beta-1 rhythm of the left hand TrH from EEG indicators. It was shown that the generators forming the physiological tremor of the left and right hands are not identical. The physiological tremor of the left hand is to a greater extent under the control of the generators that form the EEG than the generators that form the tremor of the right hand. The generator of the alpha rhythm of the TrG of the left hand is an independent formation and is not similar to the generator of the alpha rhythm of the TrG of the right hand.

Clustering Individuals on Limited Features of a Vector Autoregressive Model

Dynamical interplays in emotions have been investigated using vector autoregressive (VAR) models, whose estimates can be used to cluster participants into unknown groups. The present study evaluated a clustering algorithm, the alternating least square (ALS) algorithm, for accuracy in predicting individual group membership. We systematically manipulated (a) the number of variables in a model, (b) the size of group differences in regression coefficients, and (c) the number of regression coefficients that vary across the groups (i.e., effective features). The ALS algorithm works reliably when there are at least 5 effective features with very large group differences in a 5-variable model; and 9 effective features with very large group differences in a 10-variable model. These findings suggest that the ALS algorithm is sensitive to group differences that are present only in several coefficients of a VAR model, but that the group differences have to be large. We also found that the ALS algorithm outperforms another clustering method, Gaussian mixture modeling. The ALS algorithm was further evaluated with unbalanced sample sizes between groups and with a greater number of groups in data (Study 2). A real data application was provided to illustrate how to interpret the detected group differences (Study 3).

Optimal regression parameter-specific shrinkage by plug-in estimation

One benefit of the bias-variance tradeoff is that regression estimators do not have to be strictly unbiased. However, to take full advantage of allowing bias, shrinkage regression estimators require that the appropriate level of bias is chosen carefully. Because the conventional grid search for the shrinkage parameters requires heavy computation, it is practically difficult to incorporate more than two shrinkage parameters. In this paper, we propose a class of shrinkage regression estimators which differently shrink each regression parameter. For this purpose, we set the number of shrinkage parameters to be the same as the number of regression coefficients. The ideal shrinkage for each parameter is suggested, meaning that a burdensome tuning process is not required for each parameter. The -consistency and oracle property of the suggested estimators are established. The application of the proposed methods to simulated and real data sets produces the favorable performance of the suggested regression shrinkage methods without the need for a grid search of the entire parameter space.

ThrEEBoost: Thresholded Boosting for Variable Selection and Prediction via Estimating Equations

ABSTRACTMost variable selection techniques for high-dimensional models are designed to be used in settings, where observations are independent and completely observed. At the same time, there is a rich literature on approaches to estimation of low-dimensional parameters in the presence of correlation, missingness, measurement error, selection bias, and other characteristics of real data. In this article, we present ThrEEBoost (Thresholded EEBoost), a general-purpose variable selection technique which can accommodate such problem characteristics by replacing the gradient of the loss by an estimating function. ThrEEBoost generalizes the previously proposed EEBoost algorithm (Wolfson 2011) by allowing the number of regression coefficients updated at each step to be controlled by a thresholding parameter. Different thresholding parameter values yield different variable selection paths, greatly diversifying the set of models that can be explored; the optimal degree of thresholding can be chosen by cross-validation. ThrEEBoost was evaluated using simulation studies to assess the effects of different threshold values on prediction error, sensitivity, specificity, and the number of iterations to identify minimum prediction error under both sparse and nonsparse true models with correlated continuous outcomes. We show that when the true model is sparse, ThrEEBoost achieves similar prediction error to EEBoost while requiring fewer iterations to locate the set of coefficients yielding the minimum error. When the true model is less sparse, ThrEEBoost has lower prediction error than EEBoost and also finds the point yielding the minimum error more quickly. The technique is illustrated by applying it to the problem of identifying predictors of weight change in a longitudinal nutrition study. Supplementary materials are available online.

Review and evaluation of penalised regression methods for risk prediction in low-dimensional data with few events.

Risk prediction models are used to predict a clinical outcome for patients using a set of predictors. We focus on predicting low‐dimensional binary outcomes typically arising in epidemiology, health services and public health research where logistic regression is commonly used. When the number of events is small compared with the number of regression coefficients, model overfitting can be a serious problem. An overfitted model tends to demonstrate poor predictive accuracy when applied to new data. We review frequentist and Bayesian shrinkage methods that may alleviate overfitting by shrinking the regression coefficients towards zero (some methods can also provide more parsimonious models by omitting some predictors). We evaluated their predictive performance in comparison with maximum likelihood estimation using real and simulated data. The simulation study showed that maximum likelihood estimation tends to produce overfitted models with poor predictive performance in scenarios with few events, and penalised methods can offer improvement. Ridge regression performed well, except in scenarios with many noise predictors. Lasso performed better than ridge in scenarios with many noise predictors and worse in the presence of correlated predictors. Elastic net, a hybrid of the two, performed well in all scenarios. Adaptive lasso and smoothly clipped absolute deviation performed best in scenarios with many noise predictors; in other scenarios, their performance was inferior to that of ridge and lasso. Bayesian approaches performed well when the hyperparameters for the priors were chosen carefully. Their use may aid variable selection, and they can be easily extended to clustered‐data settings and to incorporate external information. © 2015 The Authors. Statistics in Medicine Published by JohnWiley & Sons Ltd.

A Multiobjective Exploratory Procedure for Regression Model Selection

Variable selection is recognized as one of the most critical steps in statistical modeling. The problems encountered in engineering and social sciences are commonly characterized by over-abundance of explanatory variables, nonlinearities, and unknown interdependencies between the regressors. An added difficulty is that the analysts may have little or no prior knowledge on the relative importance of the variables. To provide a robust method for model selection, this article introduces the multiobjective genetic algorithm for variable selection (MOGA-VS) that provides the user with an optimal set of regression models for a given dataset. The algorithm considers the regression problem as a two objective task, and explores the Pareto-optimal (best subset) models by preferring those models over the other which have less number of regression coefficients and better goodness of fit. The model exploration can be performed based on in-sample or generalization error minimization. The model selection is proposed to be performed in two steps. First, we generate the frontier of Pareto-optimal regression models by eliminating the dominated models without any user intervention. Second, a decision-making process is executed which allows the user to choose the most preferred model using visualizations and simple metrics. The method has been evaluated on a recently published real dataset on Communities and Crime Within the United States.

Exact small sample properties of an operational variant of the minimum mean squared error estimator

In this paper, we show a sufficient condition for an operational variant of the minimum mean squared error estimator (simply, the minimum MSE estimator) to dominate the ordinary least squares (OLS) estimator. It is also shown numerically that the minimum MSE estimator dominates the OLS estimator if the number of regression coefficients is larger than or equal to three, even if the sufficient condition is not satisfied. When the number of regression coefficients is smaller than three, our numerical results show that the gain in MSE of using the minimum MSE estimator is larger than the loss.

Limiting Behavior of M-Estimators of Regression-Coefficients in High Dimensional Linear Models II. Scale-Invariant Case

Asymptotics of M-estimators of the regression coefficients in linear models (both scale-variant and scale-invariant) when the number of regression coefficients tends to infinity as the sample size increases are investigated. The main purpose of this study is to establish the asymptotic properties under weaker conditions than those usually assumed, especially to relax the restrictions on the order of the dimension. Also, the conditions assumed and the results obtained seem easy to be extended to the multivariate linear models. In the second part of the paper, the asymptotic behavior of the scale-invariant M-estimates is considered.

Limiting Behavior of M-Estimators of Regression Coefficients in High Dimensional Linear Models I. Scale Dependent Case

Asymptotics of M-estimators of the regression coefficients in linear models (both scale-variant and scale-invariant) when the number of regression coefficients tends to infinity as the sample size increases are investigated The main purpose of this study is to establish the asymptotic properties under weaker conditions than usually assumed, especially to relax the restrictions on the order of the dimension. Also, the conditions assumed and the results obtained seem easy to be extended to the multivariate linear models. In the first part of the paper, the asymptotic behavior of the ordinary (i.e., not scale-invariant) M-estimates is considered.

Indeterminacy of the Chow Test when the Number of Observations Is Insufficient

THE CHOW TEST is a widely used procedure for testing for the equality of sets of coefficients in two linear regression models. However, when the number of observations in one of the models is less than the number of regression coefficients, the Chow test is incapable of testing the hypothesis of equality against that of inequality. It can never be concluded from the Chow test itself that the two sets are equal, although at times it may be possible to conclude that they are unequal. This point is implicit in [1], but has not been specifically discussed heretofore. The indeterminacy of the Chow test results from the insufficient number of observations. The two linear regression models, each of which is assumed to satisfy the conditions of the standard normal linear regression model, can be written as