Non-Approximation Addictive Update and Multiplicative Update Online Classifier Algorithm

Abstract

Since the online learning framework that make a compromise of the correctness and conservativeness is proposed by Kivinen and Warmuth,the framework have been referenced widely,but in gradient descent and exponentiated gradient algorithms proposed by Kivinen and Warmuth,the approximation step in the derivation of loss function of objection function lead to bad results.In this work,by means of duality theory of optimization,the novel non-approximation classifier algorithms based on square distance and relative entropy loss,relative entropy distance and relative entropy loss are proposed.Experimental results show that the proposed classifiers are always more accurate than the gradient descent and exponentiated gradient algorithms proposed by Kivinen and Warmuth in the real datasets.