Abstract
The Alternating Direction Method of Multipliers(ADMM) is an important method for machine learning. A large number of stochastic versions of ADMM continue to emerge. But almost all the algorithm focused on the Steepest Descent, which cause slow convergence rate. In our work, we propose a new stochastic alternative direction method of multipliers with conjugate gradient(SCG-ADMM), and prove its convergence rate in ergodic sense. Conjugate gradient descent has a better performance in traditional cases. Experiments on LASSO and Logistics Regression show that our proposed algorithm can achieve state-of-the-art performance in real applications. And we use the linear search method to replace the manual setting of the step length.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.