An online isotonic separation with cascade architecture for binary classification

Abstract

Isotonic separation (IS) is a non-parametric classification technique which constructs an isotonic function from ordered data. The rationale is to convert partially isotonic data into isotonic using a linear programming problem (LPP) and partition the input space into isotonic and non-isotonic regions to make predictions easier. Despite the widespread applications of IS in diverse domains where monotonicity exists, it has certain limitations: Firstly, computing time and the constraints of the LPP in isotonic separation increase polynomially as size of the data increases and it is highly complex to solve the LPP and obtain the model on large data sets. In order to support dynamic stream data and address the computational overhead and size of the LPP issues, this paper proposes an online isotonic separation algorithm called Cascade-IS (CIS) for binary classification. The rationale behind CIS is that it splits the data set into a sequence of partitions and models are obtained and combined in cascade. Statistical and experimental analysis are done on datasets with isotonic properties and the results prove that CIS is superior to its variants in terms of training time, performance measures and number of constraints in the LPP.