Multi-Class Logistic Regression Classifier with BFGS Method

Abstract

The classical multi-class logistic regression classifier uses Newton method to optimize its loss function and suffers the expensive computations and the un-stable iteration process. In our work, we apply the state-of-art optimization techniques BFGS to train multi-class logistic regression and compare them with Newton method on the classification accuracy of 25 datasets experimentally. The results show that BFGS achieves better classification accuracy than the Newton method. Moreover, BFGS have the lower time complexity, in contrast with Newton method. Finally, we also observe that logistic classifier with BFGS method demonstrate comparable performance with the SVM classifier.