A convex formulation for multiple ordinal output classification

Abstract

Multiple ordinal output classification (MOOC) which specifically refers to learning an association between individual inputs (e.g. face images) and a set of discrete ordinal response/output variables (e.g. facial action units), is a special case of multi-output classification and also a relatively-understudied topic in machine learning. It is very challenging in how to jointly model the relationship among multiple output variables and their discrete ordinal values. In this paper, we propose an effective formulation to address the above challenging problems. Under this formulation, the objective function is convex and thus leads to a convex multiple ordinal output classification (ConMOOC). Specifically, we use a regularization formulation to model the relationships among multiple output variables and an effective threshold-based loss function to fit their ordinal values. To enhance ability of the model, we also apply the kernel trick to provide a nonlinear extension. For efficiency, we use an alternating iteration method to learn the optimal model parameters for each variable as well as the relationships between different variables. Experiments conducted on synthetic and real datasets demonstrate that ConMOOC not only achieves effective classification performance but also reveals the structures among output variables. To the best of our knowledge, MOOC as a general machine learning task is the first time to be studied.