Stochastic Linear Restrictions Research Articles

Restricted Liu estimator under stochastic linear restrictions in generalized linear models: theory and applications

It is a fact that the problem of multicollinearity adversely effects the estimation of parameters not only in normal linear models but also in generalized linear models (GLMs). In this study, we propose a new estimator by imposing the stochastic restrictions on the parameters and combining the Liu estimator in GLMs to address the problem of multicollinearity. The new estimator is referred to as the stochastic restricted Liu estimator. The novel estimator’s first order approximated form (FOA) is presented to compare it to its competitors using the asymptotic mean square error criterion. Furthermore, two empirical applications using a response variable from the Poisson and Gamma distributions are studied to evaluate the performance of estimators. In addition, a simulation study is conducted by considering a response variable from Binomial distribution for comparisons. Based on simulation study and empirical applications, it is found that the performance of the proposed estimator is better than ML, Liu, and mixed estimators for the appropriate selected biasing parameter.

Lasso regression under stochastic restrictions in linear regression: An application to genomic data

Variable selection approaches are often employed in high-dimensionality and multicollinearity problems. Since lasso selects variables by shrinking the coefficients, it has extensive use in many fields. On the other, we may sometime have extra information on the model. In this case, the extra information should be considered in the estimation procedure. In this paper, we propose a stochastic restricted lasso estimator in linear regression model which uses the extra information as stochastic linear restrictions. The estimator is a generalization of mixed estimator with L 1 type penalization. We give the coordinate descent algorithm to estimate the coefficient vector of the proposed method and strong rules for the coordinate descent algorithm to discard variables from the model. Also, we propose a method to estimate the tuning parameter. We conduct two real data analyses and simulation studies to compare the new estimator with several estimators including the ridge, lasso and stochastic restricted ridge. The real data analyses and simulation studies show that the new estimator enjoys the automatic variable selection property of the lasso while outperforms standard methods, achieving lower test mean squared error.

A generalized difference-based mixed two-parameter estimator in partially linear model

In this paper a generalized difference-based mixed two-parameter estimator in partially linear model is presented, when stochastic linear restrictions are assumed to hold. We also discussed the properties of the new estimator and a method to select the biasing parameters is discussed. Finally a simulation study is given to show the performances of the estimators.

Efficiency of a generalized difference-based weighted mixed ridge estimator in partially linear model

Tabakan and Akdeniz introduced a difference-based ridge estimator in partially linear model. In this article a generalized difference-based weighted mixed ridge estimator in partially linear model is presented by combining the ideas underlying the generalized difference-based weighted mixed estimator and the generalized difference-based ridge estimator, when stochastic linear restrictions are assumed to hold. We also discussed the properties of the new estimator and compared the new estimator to the difference-based ridge estimator and difference-based weighted mixed estimator in the mean squared error matrix. A method to select the biasing parameters is also discussed. Finally, a numerical example and a simulation study are given to show the performances of the estimators.

Mixed Lasso estimator for stochastic restricted regression models

Parameters of a linear regression model can be estimated with the help of traditional methods like generalized least squares and mixed estimator. However, recent developments increased the importance of big data sets, which have much more predictors than observations where some predictors have no impact on the dependent variable. The estimation and model selection problem of big datasets can be solved using the least absolute shrinkage and selection operator (Lasso). However, to the authors’ knowledge, there is no study that incorporates stochastic restrictions, within a Lasso framework. In this paper, we propose a Mixed Lasso (M-Lasso) estimator that incorporates stochastic linear restrictions to big data sets for selecting the true model and estimating parameters simultaneously. We conduct a simulation study to compare the performance of M-Lasso with existing estimators based on mean squared error and model selection performance. Results show that M-Lasso is superior in terms of and it generally dominates compared estimators according to the model selection criteria. We employ M-Lasso to estimate parameters of a widely analysed production function under stochastic restrictions raised from economic theory. Our results show that M-Lasso can provide reasonable and more precise estimates of model parameters that are in line with the economic theory.

The Mixed Liu Estimator in Stochastic Restricted Linear Measurement Error Model

Ghapani and Babdi [1] proposed a mixed Liu estimator in linear measurement error model with stochastic linear restrictions. In this article, we propose an alternative mixed Liu estimator in the linear measurement error model with stochastic linear restrictions. The performance of the new mixed Liu estimator over the mixed estimator, Liu estimator, and mixed Liu estimator proposed by Ghapani and Babdi [1] are discussed in the sense of mean squared error matrix. Finally, a simulation study is given to show the performance of these estimators.

On the stochastic restricted Liu estimator in logistic regression model

In this paper, we study the effects of near-singularity which is known as multicollinearity in the binary logistic regression. Furthermore, we also assume the presence of stochastic non-sample linear restrictions. The well-known logistic Liu estimator is combined with the stochastic linear restrictions in order to propose a new method, namely, the stochastic restricted Liu estimation. Theoretical comparisons between the usual maximum likelihood estimator, Liu estimator, stochastic restricted maximum-likelihood estimator and the new stochastic restricted Liu estimator are derived using matrix mean-squared errors of the estimators. A Monte Carlo simulation experiment is designed to evaluate the performances of the listed estimators in terms of mean-squared error and mean absolute error criteria. Artificial data are used to show how to interpret the theorems. According to the results of the simulation, the new method beats the other estimators when the data matrix has the problem of collinearity along with the stochastic restrictions.

Stochastic Restricted LASSO-Type Estimator in the Linear Regression Model

Among several variable selection methods, LASSO is the most desirable estimation procedure for handling regularization and variable selection simultaneously in the high-dimensional linear regression models when multicollinearity exists among the predictor variables. Since LASSO is unstable under high multicollinearity, the elastic-net (Enet) estimator has been used to overcome this issue. According to the literature, the estimation of regression parameters can be improved by adding prior information about regression coefficients to the model, which is available in the form of exact or stochastic linear restrictions. In this article, we proposed a stochastic restricted LASSO-type estimator (SRLASSO) by incorporating stochastic linear restrictions. Furthermore, we compared the performance of SRLASSO with LASSO and Enet in root mean square error (RMSE) criterion and mean absolute prediction error (MAPE) criterion based on a Monte Carlo simulation study. Finally, a real-world example was used to demonstrate the performance of SRLASSO.

Ridge estimation in linear mixed measurement error models with stochastic linear mixed restrictions

This article is concentrated on the problem of multicollinearity in linear mixed models (LMMs) with measurement error in the fixed effects variables. After introducing a ridge estimator (RE) in these models, we propose a new estimator called the stochastic restricted ridge estimator (SRRE) by combining the ridge estimator (RE) and the stochastic restricted estimator (SRE). Moreover, asymptotic properties of these estimators will be derived and the necessary and sufficient conditions for the superiority of the SRRE over the RE and SRE are obtained using the mean squared error matrix (MSEM). Finally, the theoretical findings of the proposed estimators are also evaluated with a Monte Carlo simulation study and a numerical example.

Outlier Detection in Ridge Regression Model Under Stochastic Linear Restrictions

Outlier Detection in Ridge Regression Model Under Stochastic Linear Restrictions

On OLS Estimation of Stochastic Linear Regression Model

This paper mainly discusses the formulation of stochastic linear statistical model and its assumptions and finally explores an important aspect namely the Ordinary Least Squares (OLS) estimation of stochastic linear regression model. In addition to these inference in stochastic linear regression model is also presented here. Nimitozbay et.al [1], in their paper proposed the weighted mixed regression estimation of the coefficient vector in a linear regression model with stochastic linear restrictions binding the regression coefficients. In 1980, P.A.V.B. Swamy et.al proposed a linear regression model where the coefficient vector is a weekly stationary multivariate stochastic process and that model provides a convenient representation of a general class of non-stationary processes. They proposed prediction and estimation methods which are linear and easy to compute. Daojiang et.al [2] in 2014, in their paper depicted an innovative estimation technique to the multicollinearity in statistical model which is linear in the case of existence of stochastic linear constraints on the parameters and a very different estimation technique was presented by mixing the OME and PCR estimator also known as SRPC regression estimator. In 2014, Shuling Wang et.al [3] in their paper proposed some diagnostic methods in restricted stochastic statistical models which are linear. Gil Gonjalez et.al [4], in 2007, in their paper, derived the LSEs for the simple linear statistical model and examined them from a theoretical perspective.

Optimal stochastic restricted logistic estimator

It is well known that the use of prior information in the logistic regression improves the estimates of regression coefficients when multicollinearity presents. This prior information may be in the form of exact or stochastic linear restrictions. In this article, in the presence of stochastic linear restrictions, we propose a new efficient estimator, named Stochastic restricted optimal logistic estimator for the parameters in the logistic regression models when the multicollinearity presents. Further, conditions for the superiority of the new optimal estimator over some existing estimators are derived with respect to the mean square error matrix sense. Moreover, a Monte Carlo simulation study and a real data example are provided to illustrate the performance of the proposed optimal estimator in the scalar mean square error sense.

Liu Estimates and Influence Analysis in Regression Models with Stochastic Linear Restrictions and AR (1) Errors

In the linear regression models with AR (1) error structure when collinearity exists, stochastic linear restrictions or modifications of biased estimators (including Liu estimators) can be used to reduce the estimated variance of the regression coefficients estimates. In this paper, the combination of the biased Liu estimator and stochastic linear restrictions estimator is considered to overcome the effect of collinearity on the estimated coefficients. In addition, the deletion formulas for the detection of influential observations are presented for the proposed estimator. Finally, a simulation study and numerical example have been conducted to show the superiority of the proposed procedures.

Estimation of parameters in linear mixed measurement error models with stochastic linear restrictions

In this paper, we concentrate on the estimation of parameters in the linear mixed models (LMMs) with stochastic linear restrictions on the fixed and random effects, when the fixed effects are measured with error. In addition, the asymptotic properties of the estimators are derived and their performances are compared with the estimators for no restrictions cases, in the sense of mean square error matrix (MSEM). Finally, a simulation study and numerical example have been conducted to show the superiority of the restricted estimator over the unrestricted estimator.

Amendments of a Stochastic Restricted Principal Components Regression Estimator in the Linear Model

Principal component Analysis (PCA) is one of the popular methods used to solve the multicollinearity problem. Researchers in 2014 proposed an estimator to solve this problem in the linear model when there were stochastic linear restrictions on the regression coefficients. This estimator was called the stochastic restricted principal components (SRPC) regression estimator. The estimator was constructed by combining the ordinary mixed estimator (OME) and the principal components regression (PCR) estimator. It ignores the number of components (orthogonal matrix Tr) that the researchers choose to solve the multicollinearity problem in the data matrix (X). This paper proposed four different methods (Lagrange function, the same technique, the constrained principal component model, and substitute in model) to modify the (SRPC) estimator to be used in case of multicollinearity. Finally, a numerical example, an application, and simulation study have been introduced to illustrate the performance of the proposed estimator.

Stochastic Restricted Estimation in Partially Linear Measurement Error Models

As a generalization of nonparametric regression model, partially linear model has been studied extensively in the last decades. This paper considers estimation of the semiparametric model under the situation that the covariates are measured with additive error in the linear part and some additional stochastic linear restrictions exist on the parametric component. Based on the corrected profile least-squares approach and mixed regression method, we propose a stochastic restricted estimator named the corrected profile mixed estimator for the parametric component, and discuss its statistical properties. We also construct a weighted stochastic restricted estimation for the parametric component. Finally, the proposed procedure is illustrated by simulation studies.

Liu-Type logistic estimator under Stochastic Linear Restrictions

To conquer the multicollinearity problem in logistic regression, many alternative estimators have been proposed in the literature when some linear restrictions on the parameter space are available in addition to the sample model. In this paper, we propose a new two parameter Liu-type estimator called Stochastic Restricted Liu-Type Logistic Estimator (SRLTLE) by combining Liu-type estimator with the logistic model in the presence of stochastic linear restrictions. Further, a Monte Carlo simulation study is done to compare the performance of the proposed estimator with some existing estimators in the scalar mean squared error (SMSE) sense, and a numerical example is given to illustrate the theoretical results.

Estimation in a linear regression model with stochastic linear restrictions: a new two-parameter-weighted mixed estimator

ABSTRACTThe present paper considers the weighted mixed regression estimation of the coefficient vector in a linear regression model with stochastic linear restrictions binding the regression coefficients. We introduce a new two-parameter-weighted mixed estimator (TPWME) by unifying the weighted mixed estimator of Schaffrin and Toutenburg [1] and the two-parameter estimator (TPE) of Özkale and Kaçıranlar [2]. This new estimator is a general estimator which includes the weighted mixed estimator, the TPE and the restricted two-parameter estimator (RTPE) proposed by Özkale and Kaçıranlar [2] as special cases. Furthermore, we compare the TPWME with the weighted mixed estimator and the TPE with respect to the matrix mean square error criterion. A numerical example and a Monte Carlo simulation experiment are presented by using different estimators of the biasing parameters to illustrate some of the theoretical results.

Stochastic restricted estimation in partially linear additive errors-in-variables models

As a generalization of additive model and partially linear model, partially linear additive model has been paid considerably attention in recent years. This paper considers estimation of the parametric component of the semiparametric model when the covariates in the linear part are measured with additive error and some additional stochastic linear restrictions on the parametric component are available. Based on the corrected profile least-squares approach and mixed regression estimation method, we propose a corrected profile mixed estimator for the parametric component, and derive its asymptotic distribution. Finally, some simulation studies are conducted to illustrate the proposed procedure and the results are satisfactory.

Diagnostic Measures in Ridge Regression Model with AR(1) Errors under the Stochastic Linear Restrictions

Outliers and influential observations have important effects on the regression analysis. The goal of this paper is to extend the mean-shift model for detecting outliers in case of ridge regression model in the presence of stochastic linear restrictions when the error terms follow by an autoregressive AR(1) process. Furthermore, extensions of measures for diagnosing influential observations are derived. A numerical example of a real data set is used to illustrate the findings. Finally, a simulation study is conducted to evaluate the performance of the proposed procedure and measures. Results of this study show the efficiency of the proposed mean-shift outlier model for the proposed model. Also, the study resulted in some findings about the behavior of suggested measures for the specified model. In fact, these measures are affected by the degree of collinearity and the size of autocorrelation.