A two-stage gap safe screening rule for multi-label optimal margin distribution machine

Abstract

Multi-label optimal margin distribution machine (mlODM) is an efficient algorithm for multi-label classification. Although it can achieve great generalization performance, it is inefficient for large-scale datasets due to the huge number of label pairs. Motivated by its sparse solution, in this paper, we propose a two-stage gap safe screening rule for accelerating mlODM, termed as TSSR. First, a sequential safe screening rule (SSSR) based on gap is designed to screen out part of redundant label pairs prior to training, which reduces the scale of mlODM. Compared with the previous DVI rule, our method ensures absolute safety without destroying efficiency. In the second stage, to further speed up the solving process, a dynamic safe screening rule (DSSR) is embedded into the solving algorithm DCDM when training the simplified mlODM. More importantly, the feasible solution generated in the first stage can promote the efficiency of DSSR. To the best of our knowledge, this is the first attempt to create a hybrid screening rule for multi-label model. Our TSSR can greatly reduce the cost and achieve exactly the same accuracy. Experimental results on seven multi-label benchmark datasets and two real-world learning problems including movie genres classification and hypoglycemic drugs prediction verify the superiority of the proposed methods.