Discrimination Neural Network Model for Binary Classification Tasks on Tabular Data

Abstract

For the classification task, neural network-based approaches attempt to distinguish between two distributions by determining the joint distribution of input variables for each class. However, the most challenging task is still to classify the observations in the overlapping region of two classes. In this work, we propose a new discrimination neural network (DiscNN) architecture to address this issue. Our DiscNN learns to embed the initial input into more informative representations with better discriminability between the two distributions based on the cosine embedding loss. We also train our proposed model using the few-shot learning method to extract better-generalized representations from the initial input. We applied the DiscNN model to 35 tabular datasets from the OpenML-CC18 benchmark for a binary classification task. Our model showed superior performances on 28 datasets of them. In addition, we also performed experiments on 95 imbalanced datasets from the KEEL repository. The experiment results showed that the DiscNN outperformed the state-of-the-art models, including CatBoost, LightGBM, TabNet, VIME and Scarf, by around 0.23% AUC, 0.20% G-mean, and 1.06% F1 score.