A Sparse Stochastic Method With Optimal Convergence Rate for Optimizing Pseudo-Linear Performance Measures

Abstract

Optimizing pseudo-linear performance measures has gained increasing interest in recent years. Compared with the existing works focusing on L2-norm, it is more challenging to design the model with L1-norm, especially, when data are of large scale. To address the issue, in this paper, we propose a sparse stochastic method to optimize the pseudo-linear metrics. The algorithm begins by formulating the problem as a cost-sensitive classification with L1 regularization, then composite objective mirror descent (COMID) is adopted for the inner optimization, which can obtain a sparse solution with promising performance. However, the original COMID method only has $\mathcal {O}(log(T)/T)$ convergence rate with the strong convex objective. To this end, a simple yet efficient polynomial-decay average strategy is suggested. We prove that with this strategy, the proposed algorithm can not only achieve the optimal convergence rate of $\mathcal {O}(1/T)$ but also obtain the classifier with higher sparsity. The empirical studies on several public benchmark data sets demonstrate the superior performance of the proposed algorithm in terms of both efficiency and sparsity.