The Cost of Generalizing Logistic Regression

Abstract

Abstract Binary-response regression models in which the link function is a family defined by one or more unknown shape parameters are considered. Detailed attention is given to the two single-parameter families proposed by Aranda-Ordaz (1981) that incorporate the logistic, linear, and complementary log-log link functions as special cases. One model is a symmetric family of alternatives to the logistic shape and the second model is an asymmetric family. The increase in variance of quantities of interest due to the addition of an extra parameter is calculated using asymptotic methods. This inflation in variance is interpreted as the cost of adding an additional parameter to the model. In considering the seriousness of this cost the reduction in bias due to the estimation of the additional parameter should also be taken into account. Three quantities of interest—predicted probabilities for a fixed value of the explanatory variable, predicted value of the explanatory variable to obtain a specific probability, and the ratio of regression parameters—are considered. It is found that when there is a single explanatory variable the cost of adding the extra parameter in terms of inflation in variance (of either predicted probabilities or predicted explanatory-variable values) is 50% on the average across the design points, although it can be appreciably larger or smaller. The inflation in variance for predicted probabilities or covariate values tends to be very large outside the range of the data and closer to 1 within the range of the observations. Nevertheless, the inflation in variance is not typically at a minimum in the center of the data. When there are q – 1 explanatory variables the average inflation in variance is 1 + 1/q, for both models. The small-sample situation is illustrated with an example and some Monte Carlo simulations; the conclusions based on the asymptotic calculations carry over to small samples.