Multi-label Data Classification Research Articles

Daugiaklasių duomenų klasifikavimo metodų tyrimas

The article analyzes the impact of the chosen method of model application on the classification of multi-label texts written in the Lithuanian language. The article presents a study of mult-label data classification methods in Lithuanian, which includes an analysis of the accuracy of the application of data classification methods for the automatic classification of multiclass texts written in Lithuanian. The classification methods, evaluation criteria, their applicability and the principles of data preparation for classification are reviewed. After preparing the text data for classification tasks, 44 combinations of classifiers were formed for the study and classification was performed using 3 different methods of multi-label data classification: category detection, category membership and category combination detection. The results obtained are compared in terms of time and classification accuracy, identifying the best performing classifiers and identifying the differences and advantages of the classification methods used.

Label correlation guided borderline oversampling for imbalanced multi-label data learning

Multi-label data classification has received much attention due to its wide range of application domains. Unfortunately, a class imbalance problem often occurs in multi-label datasets, causing challenges for classification algorithms. Oversampling is one of the most important approaches, as it generates minority label instances to balance the class distribution. However, existing oversampling methods ignore existing label correlations, resulting in the generation of inappropriate synthetic minority samples and making multi-label data classification tasks harder. In this work, we propose an oversampling method that considers label correlations and identifies two critical boundary regions for generating synthetic minority samples. Moreover, we propose a weighting strategy to assign weights to these instances based on their distance information. To evaluate the performance of our proposed method, we conducted experiments on sixteen public datasets. The results show that our approach outperforms the state-of-the-art approaches in terms of various assessment metrics, such as Macro F1 and Macro AUC. The code is available at https://github.com/IntelliDAL/Multi-label/tree/main/LCOS.

Towards an Optimal KELM Using the PSO-BOA Optimization Strategy with Applications in Data Classification.

The features of the kernel extreme learning machine-efficient processing, improved performance, and less human parameter setting-have allowed it to be effectively used to batch multi-label classification tasks. These classic classification algorithms must at present contend with accuracy and space-time issues as a result of the vast and quick, multi-label, and concept drift features of the developing data streams in the practical application sector. The KELM training procedure still has a difficulty in that it has to be repeated numerous times independently in order to maximize the model's generalization performance or the number of nodes in the hidden layer. In this paper, a kernel extreme learning machine multi-label data classification method based on the butterfly algorithm optimized by particle swarm optimization is proposed. The proposed algorithm, which fully accounts for the optimization of the model generalization ability and the number of hidden layer nodes, can train multiple KELM hidden layer networks at once while maintaining the algorithm's current time complexity and avoiding a significant number of repeated calculations. The simulation results demonstrate that, in comparison to the PSO-KELM, BBA-KELM, and BOA-KELM algorithms, the PSOBOA-KELM algorithm proposed in this paper can more effectively search the kernel extreme learning machine parameters and more effectively balance the global and local performance, resulting in a KELM prediction model with a higher prediction accuracy.

An effective single-model learning for multi-label data

Multi-label data classification (MLC) has become an increasingly active research area over the past decade. MLC refers to a classification problem where each instance can be associated with more than one class label. Capturing the correlation among labels and tackling the label imbalance are the main challenges in MLC. Problem transformation is one of the well-known approaches in this area that became the de-facto approach for MLC. Existing methods in this approach consider MLC as a collection of single-label tasks and solve each of them separately. To consider correlation among labels, some of them consider the combination of labels that appear in the training data as a separate label. The main drawback of these kinds of methods is the complexity of the model, which makes them not applicable in real-world applications. In this paper, we show how MLC can be efficiently and effectively tackled with a single classifier. Our proposed method maps the training data into a new sub-space for each label. Then, it pools all the mapped data together and efficiently trains a single classifier for all the labels together. Experimental results show that our method successfully tackles MLC tasks and outperforms the state-of-the-art methods.

A Bayesian Convolutional Neural Network Model with Uncertainty for Multi-label Text Classification on Mechanisms of Action (MoA) Prediction

With the development of scientific research techniques, drug discovery has shifted from the serendipitous approach of the past to more targeted models based on an understanding of the underlying biological mechanisms of disease. However, there are hundreds or more of mechanism of action (MoA) data in the known drugs, which makes this process faced with complicated multi-label classification of text data. Traditional multi-label text classification algorithms will increase the complexity of the model and reduce the accuracy as the number of labels increases. Although deep learning algorithms can solve the problem of model complexity, they are currently only suitable for processing image format data. To overcome these problems, this study proposes a multi-label classification method based on Bayesian deep learning, which can convert non-image data format into image data, making it suitable for Convolutional neural network algorithm requirements. Then in the PyTorch environment, the Bayesian deep learning algorithm and the EfficientNet convolutional neural network are perfectly combined using the BLiTZ library to construct the Bayesian convolutional neural network model which named BCNNM. Not only improves the classification efficiency, this method also solves the problem of imbalanced classification of multi-label data, and fully considers the uncertainty in the neural network. In the process of drug development, this method has important practical significance for processing the multi-label classification of MoA data.

Imbalanced multi-label data classification as a bi-level optimization problem: application to miRNA-related diseases diagnosis

Imbalanced multi-label data classification as a bi-level optimization problem: application to miRNA-related diseases diagnosis

DEEP LEARNING TRAINING WITH UNBALANCE SAMPLE DISTRIBUTION FOR REMOTE SENSING IMAGE SEGMENTATION

Abstract. The intelligent interpretation of remote sensing images based on deep learning has become a hot spot with the increasing satellite images acquired due to the rapid development of aerospace technology. Sufficient and reasonable distributed samples are essential for the accuracy of deep learning. The spatial distribution of natural features is inhomogeneous in the real world. When people create sample dataset, they often collect within a certain local range, which may bring problems of unbalanced distribution of samples, including the unbalance between training dataset and validation dataset, and the unbalance among different sample categories. This long-tail distribution of samples (i.e., a few classes account for most of the data, while most classes are under-represented) can lead to bias in the training model and make it difficult to ensure accuracy.In this paper we tried to solved the above-mentioned problem in landcover classification with high spatial and spectral resolution (HSSR) remote sensing images. We first adopted an iterative stratification method for multi-label data classification to ensure that both training dataset and validation dataset contain reasonable proportion of landcover classes. Then we proposed a weighted loss algorithm to further strengthen the learning ability of the model for rare categories. Experiments on a large volume HSSR dataset shows that with our methods the accuracy of landcover classification increased by 2%.

Multi-label learning via minimax probability machine

In this paper, we propose Minimax Probability Machine for Multi-label data classification and is termed as Multi-Label Minimax Probability Machine (MLMPM). Based on data mean and covariance information, MLMPM builds a classifier that minimizes an upper bound on the mis-classification probability of unseen future data. For capturing label correlation we have considered asymmetric co-occurrency matrix into the model. The proposed model has also been extended to non-linear settings using the Mercer Kernel trick. To accelerate the training procedure, iterative weighted least squares is used to train the underlying optimization model efficiently. Extensive experimental comparisons of our proposed method with related multi-label algorithms on synthetic as well as real world multi-label datasets, along with Amazon rainforest satellite images dataset, prove its efficacy.

Multi layered Stacked Ensemble Method with Feature Reduction Technique for Multi-Label Classification

Nowadays, multi-label classification can be considered as one of the important challenges for classification problem. In this case instances are assigned more than one class label. Ensemble learning is a process of supervised learning where several classifiers are trained to get a better solution for a given problem. Feature reduction can be used to improve the classification accuracy by considering the class label information with principal Component Analysis (PCA). In this paper, stacked ensemble learning method with augmented class information PCA (CA PCA) is proposed for classification of multi-label data (SEMML). In the initial step, the dimensionality reduction step is applied, then the number of classifiers have to be chosen to apply on the original training dataset, then the stacking method is applied to it. By observing the results of experiments conducted are showing our proposed method is working better as compared to the existing methods.

Kernelized Supervised Laplacian Eigenmap for Visualization and Classification of Multi-Label Data

We had previously proposed a supervised Laplacian eigenmap for visualization (SLE-ML) that can handle multi-label data. In addition, SLE-ML can control the trade-off between the class separability and local structure by a single trade-off parameter. However, SLE-ML cannot transform new data, that is, it has the “out-of-sample” problem. In this paper, we show that this problem is solvable, that is, it is possible to simulate the same transformation perfectly using a set of linear sums of reproducing kernels (KSLE-ML) with a nonsingular Gram matrix. We experimentally showed that the difference between training and testing is not large; thus, a high separability of classes in a low-dimensional space is realizable with KSLE-ML by assigning an appropriate value to the trade-off parameter. This offers the possibility of separability-guided feature extraction for classification. In addition, to optimize the performance of KSLE-ML, we conducted both kernel selection and parameter selection. As a result, it is shown that parameter selection is more important than kernel selection.We experimentally demonstrated the advantage of using KSLE-ML for visualization and for feature extraction compared with a few typical algorithms.

A novel binary many-objective feature selection algorithm for multi-label data classification

For Multi-label classification, redundant and irrelevant features degrade the performance of classification. To select the best features based on several conflicting objectives, feature selection can be modeled as a large-scale optimization problem. However, most existing multi-objective feature selection methods select the features based on minimizing two well-known objectives, the number of features and classification error, additional objectives can be considered to improve the classification performance. In this study, for the first time, a many-objective optimization method is proposed to select the efficient features for multi-label classification based on not only two mentioned objectives, but also maximizing the correlation between features and labels and minimizing the computational complexity of features. Maximizing the correlation could lead to increasing the accuracy of classification. On the other hand, selecting less complex features decreases the computational complexity of feature extraction phase. The most important aim of this paper is to tackle the multi-label feature selection based on the number of features, classification error, correlation between features and labels, and computational complexity of features, simultaneously. The conducted many-objective feature selection problem is solved using a proposed binary version of NSGA-III algorithm. The binary operator improves the exploration power of optimizer to search the large-scale space. In order to evaluate the proposed algorithm (called binary NSGA-III), a benchmarking experiments is conducted on eight multi-label datasets in terms of several multi-objective assessment metrics, including Hypervolume indicator, Pure Diversity, and Set-coverage. Experimental results show significant improvements for proposed method in comparison with other algorithms.

Improved Ensemble Classification for Evolving Data Streams

A major challenge for evolving data stream classification is feature evolution where features of stream instances are dynamically changing as they progress. Existing classification methods considered feature evolution either for fixed-size data or of limited degree with presumed dependence to history, making them unable to work effectively on evolving data streams of unbounded size and arbitrary feature evolution. Particularly, for evolving data streams containing instances of multiple labels, classification coping with feature evolution faces significant challenges. In this article, we present efficient ensemble methods for classifying evolving data streams of both single label and multiple labels through effective model coupling. For single-label classification, we present an improved unsupervised classification algorithm that applies multi-cluster feature selection (MCFS), which was originally proposed for static data classification, in the DXMiner framework to handle each window of instances in a dynamic stream. Our method generates an optimal feature subset and achieves a high classification accuracy. We further improve the time complexity of the feature selection process in MCFS by applying the Ball-tree searching technique. For multi-label classification, we propose an effective fixed-size ensemble classifier based on multi-label KNN, which works only for static multi-label data classification, by incorporating a weight adaptation strategy among the classifiers in the ensemble to dynamically update the model and cope with arbitrary feature evolution of stream instances as the stream progresses. Extensive experiment results on real-life data streams show that our algorithms outperform the existing results for single-label and multi-label classification in classification accuracy and efficiency.

Adaptive Decision Threshold-Based Extreme Learning Machine for Classifying Imbalanced Multi-label Data

Multi-label learning is a popular area of machine learning research as it is widely applicable to many real-world scenarios. In comparison with traditional binary and multi-classification tasks, the multi-label data are more easily impacted or destroyed by an imbalanced data distribution. This paper describes an adaptive decision threshold-based extreme learning machine algorithm (ADT-ELM) that addresses the imbalanced multi-label data classification problem. Specifically, the macro and micro F-measure metrics are adopted as the optimization functions for ADT-ELM, and the particle swarm optimization algorithm is employed to determine the optimal decision threshold combination. We use the optimized thresholds to make decision for future multi-label instances. Twelve baseline multi-label data sets are used in a series of experiments o verify the effectiveness and superiority of the proposed algorithm. The experimental results indicate that the proposed ADT-ELM algorithm is significantly superior to many state-of-the-art multi-label imbalance learning algorithms, and it generally requires less training time than more sophisticated algorithms.

Multi-label classification using a cascade of stacked autoencoder and extreme learning machines

This article introduces a cascade of neural networks for classification of multi-label data. Two types of networks, namely, stacked autoencoder (SAE) and extreme learning machine (ELM) have been incorporated in the proposed system. ELM is a compact and efficient single-label classifier which seems to lose its efficiency while dealing with multi-label data. This happens due to the complex nature of the multi-label data, which makes it difficult for the smaller networks to interpret it accurately. In our proposed work, we attempt to deal with few of the bottlenecks faced while handling multi-label data. Thus, we aim to enhance the performance of a stand-alone multi-label extreme learning machine (MLELM) by collaborating it with other networks. There are three basic phases in the proposed method: feature encoding, soft classification and class score approximation. In the first step, an SAE network is employed to generate a discriminating and reduced input representation of the multi-label data. This makes the data compact and more manageable for the successive stages. This data in turn is used by an MLELM in the next phase for the prediction of soft labels. In the final step, to improve the prediction capability of the previous network, a novel approach of approximating the class score is proposed using an additional MLELM. Comprehensive experimental evaluation of the proposed approach has been performed on seven datasets against eleven relevant algorithms, and overall it displays a promising performance.

Secure multi-label data classification in cloud by additionally homomorphic encryption

In many applications, an instance is associated with a set of labels. Nevertheless, traditional classification algorithm generally assumes an instance has a label, and cannot be suitable for multi-label instance classification. Multi-label learning is thus put forward to predict the associated labels for instance with multiple unclassified labels. Nowadays, many multi-label learning approaches have been proposed, unfortunately, all of the existing multi-label learning approaches did not consider the issue of protecting the sensitive information of private instances. In this paper, we propose a scheme for secure multi-label classification over encrypted data in cloud. Our scheme can outsource the multi-label classification task to the cloud servers which dramatically reduce the storage and computation burden of data owner and data users. Based on the theoretical proof, our scheme can protect the privacy information of data owner and data users, the cloud servers cannot learn anything useful about the input data and output multi-label classification results. Additionally, we analyze our computation complexity and communication overheads in detail, and leverage simulation experiments to evaluate the computation time of our proposed scheme.

Multi-label methods for prediction with sequential data

The number of methods available for classification of multi-label data has increased rapidly over recent years, yet relatively few links have been made with the related task of classification of sequential data. If labels indices are considered as time indices, the problems can often be seen as equivalent. In this paper we detect and elaborate on connections between multi-label methods and Markovian models, and study the suitability of multi-label methods for prediction in sequential data. From this study we draw upon the most suitable techniques from the area and develop two novel competitive approaches which can be applied to either kind of data. We carry out an empirical evaluation investigating performance on real-world sequential-prediction tasks: electricity demand, and route prediction. As well as showing that several popular multi-label algorithms are in fact easily applicable to sequencing tasks, our novel approaches, which benefit from a unified view of these areas, prove very competitive against established methods.