Combining One-vs-One Decomposition and Instance-Based Learning for Multi-Class Classification

Jun-Ying Liu,Bin-Bin Jia

doi:10.1109/access.2020.3034448

Jun-Ying Liu, Bin-Bin Jia

Open Access

https://doi.org/10.1109/access.2020.3034448

Copy DOI

Abstract

Multi-class classification is one of the most important supervised learning problems and can be solved by either designing direct multi-class classifiers (direct strategy) or decomposing it into a set of binary classification problems (indirect strategy). Direct strategy only needs training one unified classifier while indirect strategy, especially the one-vs-one decomposition method, has shown its superiority and been utilized by some popular software packages. In this article, a first attempt towards bridging the gap between direct strategy and one-vs-one decomposition for multi-class classification is conducted, and accordingly a novel approach named CODIL is proposed. Specifically, CODIL firstly transforms the class vector into a ternary label matrix (only with {-1, 0, +1}) via one-vs-one rule, where each column of the label matrix corresponds to a pair of classes. Then, CODIL determines the binary label vector (only with {-1, +1}) for unseen instance by exploiting the manifold structure information residing in its k nearest neighbors, where each element of the label vector denotes the unseen instance's prediction for its corresponding class pair. Finally, the class for unseen instance is returned via majority voting based on the binary label vector. Extensive comparative studies are conducted between CODIL and six well-established multi-class approaches over seventeen benchmark multi-class data sets. The experimental results show the superiority of the proposed CODIL approach against the compared approaches.

Highlights

The traditional multi-class classification (MCC) problem is one of the most popular and important learning frameworks in supervised learning, where each object is always classified into one of N classes (N > 2)
In this article, a novel multi-class classification approach named CODIL is proposed which aims at combining the strengths of direct solving strategy and OvO decomposition strategy
Experimental results clearly validate the superiority of the proposed CODIL approach against other well-established multi-class classifiers

Summary

Introduction

The traditional multi-class classification (MCC) problem is one of the most popular and important learning frameworks in supervised learning, where each object is always classified into one of N classes (N > 2). Some other learning frameworks are usually solved by transforming themselves into MCC problems, such as multi-label learning [5], multi-dimensional classification [6], etc. Compared with binary classification where each object is classified into one of two classes, solving MCC problem is more complicated due to the possible multiple classes. Given an MCC training set D = {(xi, yi) | 1 ≤ i ≤ m} with m examples, where xi ∈ X is d-dimensional feature vector and yi ∈ Y is the associated class, the task of multi-class classification is to learn a mapping function f : X → Y from D which can predict a proper class f (x∗) ∈ Y for unseen instance x∗

Objectives

Methods

Results

Conclusion