McMatMHKS: A direct multi-class matrixized learning machine

Abstract

Multi-class classification learning can be implemented by the decomposition to binary classification or the direct techniques. The decomposition technique simplifies s the original learning problem into a set of binary subproblems, separately learns each one, and then combines their results to make a final decision. While the direct technique learns a set of multi-class classifiers by directly optimizing one single objective function. Plenty of empirical results have shown that the two techniques achieve comparable performance. However, both the techniques are mainly designed for vector-pattern samples at present. These traditional vector-pattern-oriented decomposition technique has been extended to a new type of matrix-pattern-oriented classifiers which obtain better learning performance and reduce the learning time–cost by utilizing the original structural information of the input matrix. To our best knowledge, no direct multi-class learning method for matrix pattern has been proposed so far. Therefore, this paper aims to propose a direct multi-class classification technique to compensate such a missing, which is a natural extension of the vector-based direct multi-class classification technique. Simultaneously, the left or right vector acting on matrix pattern in the multi-class matrixized objective function plays a role of a tradeoff parameter to balance the capacity of learning and generalization. Finally, based on the original binary-classifier Matrix-pattern-oriented Modified Ho-Kashyap classifier named MatMHKS, we design a corresponding Direct Multi-class Matrixized Learning Machine named McMatMHKS. It is the first direct multi-class classification technique for matrix patterns. To validate both feasibility and effectiveness of McMatMHKS, we conduct the comparative experiments on some benchmark datasets with two multi-class support vector machines and MatMHKS with the decomposition technique including both one-vs-one and one-vs-all. The results show that like its vector-oriented counterpart, McMatMHKS not only has comparable classification accuracy and AUC value, but also owns lower time complexity when compared with its corresponding decomposition machines.