Abstract
AbstractIn small samples, maximum likelihood (ML) estimates of logit model coefficients have substantial bias away from zero. As a solution, we remind political scientists of Firth's (1993,Biometrika,80, 27–38) penalized maximum likelihood (PML) estimator. Prior research has described and used PML, especially in the context of separation, but its small sample properties remain under-appreciated. The PML estimator eliminates most of the bias and, perhaps more importantly, greatly reduces the variance of the usual ML estimator. Thus, researchers do not face a bias-variance tradeoff when choosing between the ML and PML estimators—the PML estimator has a smaller biasanda smaller variance. We use Monte Carlo simulations and a re-analysis of George and Epstein (1992,American Political Science Review,86, 323–337) to show that the PML estimator offers a substantial improvement in small samples (e.g., 50 observations) and noticeable improvement even in larger samples (e.g., 1000 observations).
Highlights
In small samples, maximum likelihood (ML) estimates of logit model coefficients have substantial bias away from zero
We show that the penalized maximum likelihood (PML) estimator nearly eliminates the bias, which can be substantial
We offer Monte Carlo evidence that the PML estimator offers a substantial improvement in small samples (e.g., 100 observations) and noticeable improvement even in large samples (e.g., 1000 observations)
Summary
The statistics literature offers a simple solution to the problem of bias. Firth (1993) suggests penalizing the usual likelihood function L(b|y) by a factor equal to the square root of the determinant of the information matrix |I(b)|1/2, which produces a “penalized” likelihood function L∗(b|y) = L(b|y)|I(b)|1/2 (see Kosmidis and Firth 2009; Kosmidis 2014). It turns out that this penalty is equivalent to Jeffreys’ (1946) prior for the logit model (Firth 1993; Poirier 1994). PML reduces both the bias and the variance of the ML estimates of logit model coefficients. A researcher can implement PML as as ML, but PML estimates of logit model coefficients have a smaller bias (Firth, 1993) and a smaller variance (Copas, 1988; Kosmidis 2007: 49).. A researcher can implement PML as as ML, but PML estimates of logit model coefficients have a smaller bias (Firth, 1993) and a smaller variance (Copas, 1988; Kosmidis 2007: 49).7 The curved (convex) score function pulls low misses closer the true value and pushes high misses even further from the true value This dynamic, which is highlighted by (ii) in Figure 1, implies that low misses and high misses do not cancel and that the ML estimate curvature in the is too score large on average. Though, Monte Carlo simulations show substantial improvements that should appeal to substantive researchers
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
