Class-dependent Cost Research Articles

Adaptive learning cost‐sensitive convolutional neural network

Real-world classification often encounters a problem called class imbalance. When the data of some classes are redundant than that of other classes, traditional classifiers usually bias their decision boundaries to the redundant majority classes. Most proposed cost-sensitive strategies often ignore the hard-to-learn examples or have a large amount of hyper-parameters. This article proposes an adaptive learning cost-sensitive convolutional neural network to solve this problem. During the training process, the proposed method embeds a class-dependent cost to each class in the global error, making the decision boundary bias to the minority classes. Meanwhile, a distribution weight is assigned to each example to enhance the learning of the hard-to-learn examples. Both the class-dependent costs and distribution weights are learnt automatically in the net. This cost-sensitive approach makes the algorithm focus on the examples in the minority classes as well as the hard-to-learn examples in each class. Besides, this approach can be applied to both binary and multi-class image classification problems without any modification. Experiments are conducted on four image classification datasets to evaluate this algorithm. The experimental results show that the proposed method achieves better performance than the baseline algorithms and some other algorithms.

Cost-Sensitive Learning of Deep Feature Representations From Imbalanced Data.

Class imbalance is a common problem in the case of real-world object detection and classification tasks. Data of some classes are abundant, making them an overrepresented majority, and data of other classes are scarce, making them an underrepresented minority. This imbalance makes it challenging for a classifier to appropriately learn the discriminating boundaries of the majority and minority classes. In this paper, we propose a cost-sensitive (CoSen) deep neural network, which can automatically learn robust feature representations for both the majority and minority classes. During training, our learning procedure jointly optimizes the class-dependent costs and the neural network parameters. The proposed approach is applicable to both binary and multiclass problems without any modification. Moreover, as opposed to data-level approaches, we do not alter the original data distribution, which results in a lower computational cost during the training process. We report the results of our experiments on six major image classification data sets and show that the proposed approach significantly outperforms the baseline algorithms. Comparisons with popular data sampling techniques and CoSen classifiers demonstrate the superior performance of our proposed method.

COST-SENSITIVE MULTI-VIEW LEARNING MACHINE

Multi-view learning aims to effectively learn from data represented by multiple independent sets of attributes, where each set is taken as one view of the original data. In real-world application, each view should be acquired in unequal cost. Taking web-page classification for example, it is cheaper to get the words on itself (view one) than to get the words contained in anchor texts of inbound hyper-links (view two). However, almost all the existing multi-view learning does not consider the cost of acquiring the views or the cost of evaluating them. In this paper, we support that different views should adopt different representations and lead to different acquisition cost. Thus we develop a new view-dependent cost different from the existing both class-dependent cost and example-dependent cost. To this end, we generalize the framework of multi-view learning with the cost-sensitive technique and further propose a Cost-sensitive Multi-View Learning Machine named CMVLM for short. In implementation, we take into account and measure both the acquisition cost and the discriminant scatter of each view. Then through eliminating the useless views with a predefined threshold, we use the reserved views to train the final classifier. The experimental results on a broad range of data sets including the benchmark UCI, image, and bioinformatics data sets validate that the proposed algorithm can effectively reduce the total cost and have a competitive even better classification performance. The contributions of this paper are that: (1) first proposing a view-dependent cost; (2) establishing a cost-sensitive multi-view learning framework; (3) developing a wrapper technique that is universal to most multiple kernel based classifier.

A New Information Measure Based on Example-Dependent Misclassification Costs and Its Application in Decision Tree Learning

This article describes how the costs of misclassification given with the individual training objects for classification learning can be used in the construction of decision trees for minimal cost instead of minimal error class decisions. This is demonstrated by defining modified, cost-dependent probabilities, a new, cost-dependent information measure, and using a cost-sensitive extension of the CAL5 algorithm for learning decision trees. The cost-dependent information measure ensures the selection of the (local) next best, that is, cost-minimizing, discriminating attribute in the sequential construction of the classification trees. This is shown to be a cost-dependent generalization of the classical information measure introduced by Shannon, which only depends on classical probabilities. It is therefore of general importance and extends classic information theory, knowledge processing, and cognitive science, since subjective evaluations of decision alternatives can be included in entropy and the transferred information. Decision trees can then be viewed as cost-minimizing decoders for class symbols emitted by a source and coded by feature vectors. Experiments with two artificial datasets and one application example show that this approach is more accurate than a method which uses class dependent costs given by experts a priori.

Costs sensitive to examples learning in reduced support vector machine

In many machine learning domains,misclassification costs are sensitive to examples.As an extension of class dependent costs,a cost-sensitive reduced Support Vector Machine(SVM) named sd2sSVM that aimed at minimizing all costs was introduced.Firstly,through the use of Generalized SVM(GSVM) framework,the optimization object was converted into unconstrained mathematical programming problems. Secondly,based on smooth piecewise polynomial function that was used to approach the plus function,the unique optimization solution can thus be gained by Newton-YUAN method.Finally,reduced kernel was employed to improve the solution of nonlinear problem.The experimental results show that sd2sSVM is comparable or choicer than traditional example dependent cost-sensitive SVM.It was also discussed that how parameter C influenced the performance of sd2sSVM.

Cost-Effective Hidden Markov Model-Based Image Segmentation

Image segmentation is an important preprocessing step in a sophisticated and complex image processing algorithm. In segmenting real-world images, the cost of misclassification could depend on the true class. For example, in a two-class (negative or positive class) problem, the cost of misclassifying positive to negative class could not be equal to that of misclassifying negative to positive class. However, existing algorithms do not take into account the unequal misclassification cost. In this letter, motivated by recent advances in machine learning theory, we introduce a procedure to minimize the misclassification cost with class-dependent cost. The procedure assumes the hidden Markov model (HMM) which has been popularly used for image segmentation in recent years. We represent all feasible HMM-based segmenters (or classifiers) as a set of points in the receiver operating characteristic (ROC) space. Then, the optimal segmenter (or classifier) is found by computing the tangential point between the iso-cost line with given slope and the convex hull of the feasible set in the ROC space. We illustrate the procedure by segmenting aerial images with different selection of misclassification costs.

Perceptron and SVM learning with generalized cost models

Learning algorithms from the fields of artificial neural networks and machine learning, typically, do not take any costs into account or allow only costs depending on the classes of the examples that are used for learning. As an extension of class dependent costs, we consider costs that are example, i.e. feature and class dependent. We derive a cost-sensitive perceptron learning rule for non-separable classes, that can be extended to multi-modal classes (DIPOL) and present a natural cost-sensitive extension of the support vector machine (SVM). We also derive an approach for including example dependent costs into an arbitrary cost-insensitive learning algorithm by sampling according to modified probability distributions.

Learning Perceptrons and Piecewise Linear Classifiers Sensitive to Example Dependent Costs

This article considers the cost dependent construction of linear and piecewise linear classifiers. Classical learning algorithms from the fields of artificial neural networks and machine learning consider either no costs at all or allow only costs that depend on the classes of the examples that are used for learning. In contrast to class dependent costs, we consider costs that are example, i.e. feature and class dependent. We present a cost sensitive extension of a modified version of the well-known perceptron algorithm that can also be applied in cases, where the classes are linearly non-separable. We also present an extended version of the hybrid learning algorithm DIPOL, that can be applied in the case of linear non-separability, multi-modal class distributions, and multi-class learning problems. We show that the consideration of example dependent costs is a true extension of class dependent costs. The approach is general and can be extended to other neural network architectures like multi-layer perceptrons and radial basis function networks.