Abstract
Motivated by recent works on the high-dimensional logistic regression, we establish that the existence of the maximum likelihood estimate exhibits a phase transition for a wide range of generalized linear models with binary outcome and elliptical covariates. This extends a previous result of Candès and Sur who proved the phase transition for the logistic regression with Gaussian covariates. Our result reveals a rich structure in the phase transition phenomenon, which is simply overlooked by Gaussianity. The main tools for deriving the result are data separation, convex geometry and stochastic approximation. We also conduct simulation studies to corroborate our theoretical findings, and explore other features of the problem.
Highlights
We are concerned with the maximum likelihood estimate of generalized linear models [32, 34] with binary outcome
Our work aims to explore to which extent the phase transition occurs in terms of link functions and covariates, including the logit link and Gaussian covariates as a special case
We provide the background on the existence of the maximum likelihood estimate in binary response generalized linear models, and the properties of elliptical distributions
Summary
We are concerned with the maximum likelihood estimate of generalized linear models [32, 34] with binary outcome. Candes and Sur [12] proved a phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression with Gaussian covariates. This extends an earlier result of Cover [14] in the context of information theory. In a recent paper of Montanari, Ruan, Sohn and Yan [33], they considered the max-margin classifier of the random feature problem in the high-dimensional regime They provided a phase transition threshold for the existence of the max-margin classifier, and further studied the limiting max-margin classifier.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
