SVM-Based Multiple Instance Classification via DC Optimization

Annabella Astorino,Antonio Fuduli,Giovanna Miglionico,Giovanni Giallombardo

doi:10.3390/a12120249

Annabella Astorino, Antonio Fuduli + Show 2 more

Open Access

https://doi.org/10.3390/a12120249

Copy DOI

Abstract

A multiple instance learning problem consists of categorizing objects, each represented as a set (bag) of points. Unlike the supervised classification paradigm, where each point of the training set is labeled, the labels are only associated with bags, while the labels of the points inside the bags are unknown. We focus on the binary classification case, where the objective is to discriminate between positive and negative bags using a separating surface. Adopting a support vector machine setting at the training level, the problem of minimizing the classification-error function can be formulated as a nonconvex nonsmooth unconstrained program. We propose a difference-of-convex (DC) decomposition of the nonconvex function, which we face using an appropriate nonsmooth DC algorithm. Some of the numerical results on benchmark data sets are reported.

Highlights

Multiple instance learning (MIL) is a recent machine learning paradigm [1,2,3], which consists of classifying sets of points
We propose a DC optimization model providing a linear classifier for binary MIL
The interesting property of such a model-function is that whenever the sufficient descent is not achieved at points that are close to the stability center, say z + d, ̄ an improved cutting-plane model can be obtained by only updating the bundle of f 1 with the appropriate information related to the point z + d. ̄ On the other hand, it looks obviously difficult to adopt the minimization of the model-function Γk as a building block of any algorithm, given its nonconvexity

Summary

Introduction

Multiple instance learning (MIL) is a recent machine learning paradigm [1,2,3], which consists of classifying sets of points. Some MIL applications are image classification [5,6,7,8], drug discovery [9,10], classification of text documents [11], bankruptcy prediction [12], and speaker identification [13] For this kind of problems, there are various solutions in the literature that fall into three different classes: instance-space approaches, bag-space approaches, and embedding-space approaches. Nonlinear instance-space MIL classifiers have been proposed in the literature, such as in [27] and in [28], where a spherical separation approach is adopted: in particular, in the former a variable neighborhood search method [29] is used, while in the latter a DC (difference of convex) model is solved using an appropriate DC algorithm [30].

A DC Decomposition of the SVM-Based MIL

Solving DC-MIL using a Nonsmooth DC Algorithm

Results

Conclusions