Multi-label Classification Algorithms Research Articles

Dual aggregated feature pyramid network for multi label classification

• Feature level aggregation for multi-label classification. • Classifier level aggregation with the one-vs-all strategy. • End-to-end learning for combined feature and classifier aggregation network. • Evaluation on various experiments in comparison with state-of-the-art methods. • Ablation study to validate both aggregation methods. While many deep convolutional neural networks show promising performance in various classification tasks, multiple objects appearing in very different sizes, shapes, and appearances cause difficulty in multi-label classification using conventional neural networks. In this paper, we introduce a dual aggregated network on pyramidal convolutional features for multi-label classification. The proposed method includes both feature- and classifier-level aggregation to learn discriminant multi-scale information of various target objects in the image. First, the feature-level aggregation collects the convolutional activation maps from the multi-scale pyramid network, and then it densely pools them to take localized features of each object. We elaborately design the feature aggregation method so that the responses from the objects with different sizes, aspect ratios, and shapes are properly reflected the aggregated activation map. Unlike conventional methods, this process does not require the region proposal step, which reduces the computational burden significantly. Second, we introduce the classifier level aggregation algorithm for integrating the multi-object classifier modules. To maximize the discrimination power of each class, we train one-vs-all classifiers for individual classes using the class-wise loss function. For each test image, the scores from the class-wise classifiers are aggregated to get the final multi-label classification result. By combining the above feature- and classifier-level aggregation methods, our network can be trained in an end-to-end fashion, which is not possible for the conventional multi-label classification algorithms using region proposals. Extensive evaluations on PASCAL VOC 2007 and PASCAL VOC 2012 demonstrate that the proposed algorithm outperforms the state-of-the-art methods.

Robust Cross-View Embedding With Discriminant Structure for Multi-Label Classification

Label embedding is an important family of multi-label classification algorithms which can jointly extract the information of all labels for better performance. However, few works have been done to develop the multi-label embedding methods that can effectively deal with the interference of noisy data during training process. The noise often makes the labels of a few samples incorrect ( i.e. , missing or mislabeled), which could lead to a poor learning performance. To address this issue, we propose a novel cross-view based model. It performs a robust and discriminant embedding, namely Robust Cross-view Embedding with Discriminant Structure for Multi-label Classification (RCEDS). In RCEDS, a novel hypergraph fusion technique is designed to explore and utilize the complementary between the feature space and the label space to make the proposed RCEDS robust. Meanwhile, we use double-side metric learning to mine the consistency between the feature space and the label space to effectively improve the discriminative ability of our proposed RCEDS. Furthermore, we conduct a deep extension of RCEDS and effectively apply it to image annotation. Extensive experimental results on data sets with many labels demonstrate that our proposed approach can attain better classification performance than the existing label embedding algorithms.

Multi-Label Classification of Arrhythmia for Long-Term Electrocardiogram Signals With Feature Learning

Arrhythmia is a kind of cardiovascular disease that seriously threats human health. Intelligent analysis of electrocardiogram (ECG) is an effective method for the early prevention and precise treatment to arrhythmia. In clinical ECG waveforms, it is common to see the multi-label phenomenon that one patient would be labeled with multiple types of arrhythmia. However, the current research is mainly to use the multiclass methods to solve the multi-label problem, ignoring the correlations between diseases and causing information loss. Therefore, this article aims: 1) to propose a multi-label feature selection method based on ECG (MS-ECG) and design an evaluation criterion of ECG features based on kernelized fuzzy rough sets so as to choose the optimal feature subset and optimize ECG feature space and 2) to propose the multi-label classification algorithm of arrhythmia based on ECG (MC-ECG) by establishing a multiobjective optimization model. This algorithm based on sparsity constraint explores the correlations between arrhythmia diseases and analyzes the mapping relationship between ECG features and arrhythmia diseases, so that one ECG signal would be automatically and accurately given multiple labels. Through sufficient experiments to prove the feasibility of our methods, we obtain the selected feature subset composed of 23 ECG features by MS-ECG. For the six evaluation criterions of MC-ECG, average precision is 0.8462, hamming loss is 0.1041, ranking loss is 0.1313, one-error is 0.2023, coverage is 0.4015, and micro-F1 is 0.6088. The outcome presents optimal to the current algorithms.

An Approach Based on Multilevel Convolution for Sentence‐Level Element Extraction of Legal Text

In the judicial field, with the increase of legal text data, the extraction of legal text elements plays a more and more important role. In this paper, we propose a sentence‐level model of legal text element extraction based on the structure of multilabel text classification. Our proposed model contains an encoder and an improved decoder. The encoder applies multilevel convolutional neural networks (CNN) and Long Short‐Term Memory (LSTM) as feature extraction networks to extract local neighborhood and context information from legal text, and a decoder applies LSTM with multiattention and full connection layer with an improved initialization method to decode and generate label sequences. To our best knowledge, it is one of the first attempts to apply a multilabel classification algorithm for element extraction of legal text. In order to verify the effectiveness of our model, we conduct experiments not only on three real legal text datasets but also on a general multilabel text classification dataset.The experimental results demonstrate that our proposed model outperforms baseline models on legal text datasets, and our model is competitive to baseline models on the general text multilabel classification dataset, which indicates that our proposed model is useful for multilabel classification tasks of ordinary texts and legal texts with an uncertain number of characters in words and short lengths.

A ranking-based feature selection for multi-label classification with fuzzy relative discernibility

Feature selection is a crucial pre-processing step for learning tasks to mitigate the “curse of dimensionality”, which is caused by irrelevant and redundant features in high-dimensional feature space. The fuzzy rough set is an effective tool in feature selection, and the fuzzy discernibility matrix, one of the generalized model of the fuzzy rough set, has attracted significant attention recently. However, with the complicated discrimination relation of multiple labels, the traditional fuzzy discernibility matrix does not fit well for multi-label data. To address this problem, in this paper, the fuzzy label discernibility relation and the fuzzy relative discernibility relation are defined firstly, which are derived from fuzzy discernibility relation. Subsequently, we present the feature discernibility significance which measures the discernibility ability of the conditional feature and selects the most relevant features using the value of discernibility significance. On this basis, we propose a ranking-based feature selection algorithm for multi-label classification with fuzzy relative discernibility. Finally, in terms of six widely-accepted multi-label evaluation metrics, a series of experiments is conducted based on the multi-label classifier with ten different multi-label datasets from Mulan library and MLL resource, to compare the performance of the proposed algorithm with four state-of-the-art multi-label feature selection algorithms. The experimental results demonstrate the superiority and effectiveness of the proposed algorithm in multi-label classification.

A novel emergency situation awareness machine learning approach to assess flood disaster risk based on Chinese Weibo

Social media emerged as an important resource of information to improve the emergency situation awareness of flooding disasters. However, the online microblog text stream is unstructured and unbalanced obviously. Given the big, real-time, and noisy flood disaster microblog text flow, a new regional emergency situation awareness model to automatic assess flood disaster risk is proposed. Firstly, according to the established online disaster event-meta frame, a multi-label classification algorithm for the flood microbloggings is constructed based on the historical dataset. This algorithm helps to assign the relevant event-meta tags to each situation microbloggings. Second, a new machine learning method for dynamic assessment of flood risk for online microbloggings is developed. The flood event-metas are considered to be feature vectors, and the four different levels of flood risk are considered to be four classes. Then, the flood risk assessment task is innovatively transformed into a multi-classification task. By the logistic regression ordered multi-classification algorithm, the dynamic quantitative evaluation of event-meta, users and regional risks is realized. Finally, the proposed model is applied in the case of the Yuyao Flood. The results of the case study show that the Yuyao Flood’s online quantitative risk assessment results are consistent with real accumulated precipitation data, which illustrate that the proposed machine learning model could realize the bottom-up automatic disaster information collecting by processing victim user-generated content effectively. Social media is proven to supplement the deficiencies of traditional disaster statistics and provide real-time, scientific information support for the implementation of flood emergency processes.

Approaches for Multi-View Redescription Mining

The task of redescription mining explores ways to re-describe different subsets of entities contained in a dataset and to reveal non-trivial associations between different subsets of attributes, called views. This interesting and challenging task is encountered in different scientific fields, and is addressed by a number of approaches that obtain redescriptions and allow for the exploration and analyses of attribute associations. The main limitation of existing approaches to this task is their inability to use more than two views. Our work alleviates this drawback. We present a memory efficient, extensible multi-view redescription mining framework that can be used to relate multiple, i.e. more than two views, disjoint sets of attributes describing one set of entities. The framework can use any multi-target regression or multi-label classification algorithm, with models that can be represented as sets of rules, to generate redescriptions. Multi-view redescriptions are built using incremental view-extending heuristic from initially created two-view redescriptions. In this work, we use different types of Predictive Clustering trees algorithms (regular, extra, with random output selection) and the Random Forest thereof in order to improve the quality of final redescription sets and/or execution time needed to generate them. We provide multiple performance analyses of the proposed framework and compare it against the naive approach to multi-view redescription mining. We demonstrate the usefulness of the proposed multi-view extension on several datasets, including a use-case on understanding of machine learning models - a topic of growing importance in machine learning and artificial intelligence in general.

Label distribution feature selection for multi-label classification with rough set

Multi-label learning deals with cases where every instance corresponds to multiple labels. The objective is to learn mapping from an instance to a relevant label set. Existing multi-label learning approaches assume that the significance for all related labels is same for every instance. Several problems of label ambiguity can be dealt with using multi-label learning, but some practical applications with significance among related labels for every instance cannot be effectively processed. To achieve superior results by conducting different significance of labels, label distribution learning is used for such applications. First, the probability model and rough set are embedded in the labeling significance, thus more supervised information can be obtained from original multi-label data. Subsequently, to resolve the feature selection problem of label distribution data, according to the feature dependency and the rough set, a novel feature selection algorithm for multi-label classification is designed. Finally, to verify the effectiveness of the proposed algorithms, an extensive experiment is conducted on 15 real-world multiple label data sets. The performance of the proposed algorithm through the multi-label classifier is compared with seven state-of-the-art approaches, thereby indicating the applicability and effectiveness of label distribution feature selection.

Partial Classifier Chains with Feature Selection by Exploiting Label Correlation in Multi-Label Classification.

Multi-label classification (MLC) is a supervised learning problem where an object is naturally associated with multiple concepts because it can be described from various dimensions. How to exploit the resulting label correlations is the key issue in MLC problems. The classifier chain (CC) is a well-known MLC approach that can learn complex coupling relationships between labels. CC suffers from two obvious drawbacks: (1) label ordering is decided at random although it usually has a strong effect on predictive performance; (2) all the labels are inserted into the chain, although some of them may carry irrelevant information that discriminates against the others. In this work, we propose a partial classifier chain method with feature selection (PCC-FS) that exploits the label correlation between label and feature spaces and thus solves the two previously mentioned problems simultaneously. In the PCC-FS algorithm, feature selection is performed by learning the covariance between feature set and label set, thus eliminating the irrelevant features that can diminish classification performance. Couplings in the label set are extracted, and the coupled labels of each label are inserted simultaneously into the chain structure to execute the training and prediction activities. The experimental results from five metrics demonstrate that, in comparison to eight state-of-the-art MLC algorithms, the proposed method is a significant improvement on existing multi-label classification.

Non-intrusive load monitoring using multi-label classification methods

Non-intrusive load monitoring is a technique to help power companies monitor and analyze residential energy usage. Aggregated power load measurements for a household (i.e., the signal on the main powerline) are disaggregated into individual appliance loads by examining the appliance-specific power consumption characteristics. These data can then be used to modify consumer behaviors via detailed billing and/or demand-pricing tariffs. A number of advances in the field have been reported in the past two decades, many of which apply machine learning algorithms. However, these algorithms usually only assign one label to an example, which is a poor match to the monitoring problem, meaning elaborate encodings or classifier ensembles are needed. A more elegant solution would be to use algorithms that assign multiple labels to a single example. These multi-label classification algorithms have received very little attention in this field to date. We conduct an experimental investigation of four multi-label classification algorithms for non-intrusive monitoring and find that the best one is superior to the existing reported results on multiple real-world household datasets.

Multi-label text classification algorithm based on semi-supervised learning

The standard Co-Training algorithm can effectively improve the performance of the classifier by using unlabeled data. In fact, however, it is difficult to meet the requirement of sufficient and redundant views At the same time, the constraints of Co-Training is further relaxed to solve the problem of inaccurate confidence estimation of unlabeled samples. In this paper, imml-knn and mlplsa-knn are used to train the basic classifiers with great difference. By using unlabeled data in the way of providing Pseudotag sample to each other, a Multi-Label Text Classification Algorithm CT-MLTC based on Co-Training is proposed. Experimental results show that the algorithm has good performance in multi label text classification tasks with more unlabeled data.

Multi-label learning for improving discretely-modulated continuous-variable quantum key distribution

We propose a novel scheme for discretely-modulated continuous-variable quantum key distribution (CVQKD) using machine learning technologies, which called multi-label learning-based CVQKD (ML-CVQKD). In particular, the proposed scheme divides the whole quantum system into state learning process and state prediction process. The former is used for training and estimating classifier, and the latter is used for generating final secret key. Meanwhile, a multi-label classification algorithm (MLCA) is also designed as an embedded classifier for distinguishing coherent state. Feature extraction for coherent state and related machine learning-based metrics for the quantum classifier are successively suggested. Security analysis based on the linear bosonic channel assumption shows that MLCA-embedded ML-CVQKD outperforms other existing discretely-modulated CVQKD protocols, such as four-state protocol and eight-state protocol, as well as the original Gaussian-modulated CVQKD protocol, and it will be further enhanced with the increase of modulation variance.

An improved Chinese text multi-label classification method based on CNN

Text multi-label classification technology can accurately and quickly classify text information into related categories or topics, and help people quickly locate the required content in massive information resources, which is of great significance in application. As the traditional classification algorithm is faced with the problems of low classification accuracy due to the low correlation of data labels, unbalanced label data and few short text feature words, this paper firstly performs hierarchical pre-processing on label data to transform multi-label classification into hierarchical text multi-classification. At the same time, an improved multi-label classification algorithm Multi-label Convolutional Neural Networks (ML-CNN) is proposed. Based on the TensorFlow framework, a CNN model is designed and different training models are constructed for each level of label classification. According to the number of classification levels, the output of the upper level label is stitched to the original input tail as the next level of input. Experiments on the description information of 500,000 Chinese products with labels, show that the improved algorithm will significantly improve the classification accuracy and the accuracy of each level can reach more than 88%, which proves the feasibility and effectiveness of the algorithm.

Joint label-specific features and label correlation for multi-label learning with missing label

Existing multi-label learning classification algorithms ignore that class labels may be determined by some features in the original feature space. And only a partial label of each instance can be obtained for some real applications. Therefore, we propose a novel algorithm named joint Label-Specific features and Label Correlation for multi-label learning with Missing Label (LSLC-ML) and its optimized version to solve the above-mentioned problems. First, a missing label can be recovered by the learned positive and negative label correlations from the incomplete training data sets, then the label-specific features can be selected, finally the multi-label classification task can be modeled by combining the labelspecific feature selections, missing labels and positive and negative label correlations. The experimental results show that our algorithm LSLC-ML has strong competitiveness compared with some state-of-the-art algorithms in evaluation matrices when tested on benchmark multi-label data sets.

Metric learning-guided k nearest neighbor multilabel classifier

Multilabel classification deals with the problem where each instance belongs to multiple labels simultaneously. The algorithm based on large margin loss with k nearest neighbor constraints (LM-kNN) is one of the most prominent multilabel classification algorithms. However, due to the use of square hinge loss, LM-kNN needs to iteratively solve a constrained quadratic programming at a high computational cost. To address this issue, we propose a novel metric learning-guided k nearest neighbor approach (MLG-kNN) for multilabel classification. Specifically, we first transform the original instance into the label space by least square regression. Then, we learn a metric matrix in the label space, which makes the predictions of an instance in the learned metric space close to its true class values while far away from others. Since our MLG-kNN can be formulated as an unconstrained strictly (geodesically) convex optimization problem and yield a closed-form solution, the computational complexity is reduced. An analysis of generalization error bound indicates that our MLG-kNN converges to the optimal solutions. Experimental results verify that the proposed approach is more effective than the existing ones for multilabel classification across many benchmark datasets.

A new framework of multi-objective evolutionary algorithms for feature selection and multi-label classification of video data

There are few studies in the literature to address the multi-objective multi-label feature selection for the classification of video data using evolutionary algorithms. Selecting the most appropriate subset of features is a significant problem while maintaining/improving the accuracy of the prediction results. This study proposes a framework of parallel multi-objective Non-dominated Sorting Genetic Algorithms (NSGA-II) for exploring a Pareto set of non-dominated solutions. The subsets of non-dominated features are extracted and validated by multi-label classification techniques, Binary Relevance (BR), Classifier Chains (CC), Pruned Sets (PS), and Random k-Labelset (RAkEL). Base classifiers such as Support Vector Machines (SVM), J48-Decision Tree (J48), and Logistic Regression (LR) are performed in the classification phase of the algorithms. Comprehensive experiments are carried out with local feature descriptors extracted from two multi-label data sets, the well-known MIR-Flickr dataset and a Wireless Multimedia Sensor (WMS) dataset that we have generated from our video recordings. The prediction accuracy levels are improved by 6.36% and 25.7% for the MIR-Flickr and WMS datasets respectively while the number of features is significantly reduced. The results verify that the algorithms presented in this new framework outperform the state-of-the-art algorithms.

Multi-label Classification Algorithm Based on Label-Specific Features and Instance Correlations

The method for learning label-specific features reduces dimensions by selecting specific features for each label with the consideration of pairwise label correlations and it solves the problem of dimensions of multi-label classification effectively.However,instance correlations are not taken into account in the method.To solve this problem,a multi-label classification algorithm based on label-specific features and instance correlations is proposed.Both label correlations and the correlation of instance features are considered.The similarity map is constructed to learn the similarity of instance feature space.Experimental results on 8 datasets show that the proposed algorithm effectively extracts label-specific features with better classification performance.

Distributed Information-Theoretic Semisupervised Learning for Multilabel Classification.

Multilabel classification (MLC) has received much attention recently. The existing MLC algorithms usually learn multiple classifiers simultaneously by exploiting the correlations among different labels. However, it is difficult and/or expensive to collect a large amount of multilabeled data in practice. The lack of labeled data significantly deteriorates the performance of classification. Moreover, the existing algorithms belong to centralized learning, that is, all the data with their labels must be transmitted to a fusion center for processing. But in many real applications, data may be dispersedly collected/stored in distributed nodes of networks. Due to the concerns of communication cost, processing ability, and data privacy, it is impossible to transmit and/or process the data centrally at one node. Considering this, the problem of distributed MLC over networks is studied, and two distributed information-theoretic semisupervised multilabel learning (dITS2ML2) algorithms are proposed, which are, respectively, used for solving linear and nonlinear MLC problems. In the proposed algorithms, a cost-sensitive objective function is designed, in which a new label correlation term defined on some anchor data is suggested. Besides, to decentralize the global objective function, a distributed matrix completion algorithm is developed to distributively complete the label matrix of the anchor data. Then, by exchanging and combining a few intermediate quantities instead of the original data for both linear and nonlinear cases, the model parameters can be adaptively estimated. The convergence of the proposed dITS2ML2 algorithms is analyzed, and their effectiveness in MLC is verified by simulations on various real datasets.

User Association for Millimeter-Wave Networks: A Machine Learning Approach

Millimeter-wave (mmWave) communication has been regarded as one of the most promising means to improve the cellular system capacity in the fifth-generation (5G) era. Compared with the conventional microwave communication networks, mmWave terminals should connect with multiple base stations (BSs) simultaneously to prevent the signal blockage. Meanwhile, the accurate instantaneous channel state information (CSI) is difficult to estimate and collect due to the densification of mmWave BSs. These unique characteristics pose stiff challenges to user association in mmWave networks. To deal with these issues, we develop a novel machine learning based user association approach to support multi-connectivity in mmWave networks. Specifically, we first formulate the mmWave user association problem as a multi-label classification problem, which is then transformed into a series of single-label classification problems through efficient multi-label classification algorithms. To further reduce the requirement on the amount of training samples, we utilize graphical model to represent the user association scenario and adopt novel feature extraction methods to obtain appropriate features from both geographical location information and topological information. With appropriate features, each single-label classification problem can be trained in a supervised manner. Test results show that the proposed approach can achieve a good performance with only a few training samples and without the need of CSI.

A novel multi-label classification algorithm based on K-nearest neighbor and random walk

The multi-label classification problem occurs in many real-world tasks where an object is naturally associated with multiple labels, that is, concepts. The integration of the random walk approach in the multi-label classification methods attracts many researchers’ sight. One challenge of using the random walk-based multi-label classification algorithms is to construct a random walk graph for the multi-label classification algorithms, which may lead to poor classification quality and high algorithm complexity. In this article, we propose a novel multi-label classification algorithm based on the random walk graph and the K-nearest neighbor algorithm (named MLRWKNN). This method constructs the vertices set of a random walk graph for the K-nearest neighbor training samples of certain test data and the edge set of correlations among labels of the training samples, thus considerably reducing the overhead of time and space. The proposed method improves the similarity measurement by differentiating and integrating the discrete and continuous features, which reflect the relationships between instances more accurately. A label predicted method is devised to reduce the subjectivity of the traditional threshold method. The experimental results with four metrics demonstrate that the proposed method outperforms the seven state-of-the-art multi-label classification algorithms in contrast and makes a significant improvement for multi-label classification.