Parameter estimation of multinomial logistic regression model using least absolute shrinkage and selection operator (LASSO)

Abstract

Regression modeling was used to describe the relationship between the response variable and one or more predictor variables. For a categorical response variable, the logistic regression would be more appropriate for this. It is not uncommon that we encounter a response variable with more than two categories. Hence, we end up modeling a multinomial logistic regression. The estimation of the parameters of the model was done using Maximum Likelihood Estimation (MLE). Furthermore, we used Least Absolute Shrinkage and Selection Operator (LASSO) to further facilitate variable selection in the model. Case studies and simulations were studied using the LASSO model, and are implemented in R. We then implement the LASSO model to analyze the data of senior high school preferences by Public Junior High School and Islamic Junior High School students in Trenggalek Regency, East Java, Indonesia. The influencing factors from the model with LASSO were average score during the 8th semester, school type, father’s occupation, mother’s occupation, mother’s last education, father’s income, mother’s income, and long term plan. The response variable was assumed to follow a multinomial distribution, with three levels. A random error was assumed to follow the normal distribution. There were two predictor variables, and various sample sizes were considered. The results showed that the LASSO estimates are similar to those from parametric estimation.