An Iterative Constrained Optimization Approach to Classifier Design

Abstract

In this paper, we propose an iterative constrained optimization (ICO) approach to classifier design. When a set of conflicting objectives needs to be simultaneously satisfied, it is often not easy to combine all the utilities in a single overall objective function for optimization. We instead formulate the problem with conflicting objectives as a single-objective optimization scenario while embedding other competing objectives in constraints so that the original problem can be solved by adopting conventional constrained nonlinear optimization techniques. The bounds needed to constrain each objective are determined based on the objective function values obtained in the previous iterate. The so-formed individual constrained optimization problems are solved until a stable solution is obtained. We illustrate the utility of our framework in the context of designing classifiers for text categorization and automatic language identification. The results of our experiments demonstrate that our approach achieves a significant improvement in one objective with only slight degradation of the other conflicting objective.