Decision function with probability feature weighting based on Bayesian network for multi-label classification

Abstract

The multi-label classification problem involves finding a multi-valued decision function that predicts an instance to a vector of binary classes. Two methods are widely used to build multi-label classifiers: the binary relevance method and the chain classifier. Both can induce a polynomial multi-valued decision function by using Bayesian network-augmented naive Bayes classifiers as base models. In this paper, we propose a feature weighting approach to improve the classification accuracy of the decision function. This method, called probability feature weighting, estimates the conditional probability of the positive class through deep computation of the frequency ratio of features from the training data. Moreover, we identify irrelevant variables in terms of probability to simplify the decision function. Experiments showed that the decision function with a probability feature weighting rarely degrades the quality of the model and drastically improves it in many cases.