Multi-label Data Research Articles

Performance-based fault detection approach for the dew point process through a fuzzy multi-label support vector machine

Nowadays, there are many fault detection approaches, which are taken into consideration in two categories including the data-driven and the model-based fault detection approaches. One of the well-known data-based fault detection techniques is the support vector machine that is proved to be a powerful approach in the classification. This one has weakness in dealing with a variety of complicated data. To address this concern, based on the investigation presented, an integration of the two approaches including the fuzzy and the multi-label SVM is proposed. In a word, the performance-based approach can classify the noisy and the multi-label data. This one is carried out for the dew point process with the cooling cycle in the real-world plant data in correspondence with the simulated plant via the HYSYS software environment. The proposed performance-based approach is about 10% percent more accurate than the conventional multi-label SVM in identifying the faults of the process.

Multi-label learning with kernel extreme learning machine autoencoder

In multi-label learning, in order to improve the accuracy of classification, many scholars have considered the relationship between features and features, features and labels or labels and labels, but how to combine the correlation among them is rarely studied. Based on this, this paper proposes a multi-label learning algorithm with kernel extreme learning machine autoencoder. Firstly, the label space is reconstructed by using the non-equilibrium labels completion method in the label space. Then, the non-equilibrium labels space information is added to the input node of the kernel extreme learning machine autoencoder network, and the input features are output as the target. Finally, the kernel extreme learning machine is used for classification. Our method implements the information fusion between features and features, between labels and features, and between labels and labels. Compared with the traditional autoencoder network, the extreme learning machine autoencoder has no iterative process, which reduces the network training time and improves the classification accuracy. The experimental results of the proposed algorithm in the opening benchmark multi-label data sets show that the KELM-AE algorithm has some advantages over other comparative multi-label learning algorithms and the statistical hypothesis testing and stability analysis further illustrate the effectiveness of the proposed algorithm.

Efficient Ensemble Classification for Multi-Label Data Streams with Concept Drift

Most existing multi-label data streams classification methods focus on extending single-label streams classification approaches to multi-label cases, without considering the special characteristics of multi-label stream data, such as label dependency, concept drift, and recurrent concepts. Motivated by these challenges, we devise an efficient ensemble paradigm for multi-label data streams classification. The algorithm deploys a novel change detection based on Jensen–Shannon divergence to identify different kinds of concept drift in data streams. Moreover, our method tries to consider label dependency by pruning away infrequent label combinations to enhance classification performance. Empirical results on both synthetic and real-world datasets have demonstrated its effectiveness.

Pemodelan Data Multi-Label dengan Pendekatan Multivariate Generalized Linear Mixed Model (MGLMM)

Multi-label data refers to a type of categorical data where an object may has more than one corresponding label or possible values. Multi-label data are commonly found in many fields, one of them is market research of the sweetened condensed milk (SCM) and sweetened condensed creamer (SCC) products. According to product characteristic, market research for the aforementioned product is appropriately conducted on the outlet level. An outlet may use more than one product’s brand in the same time frame. That condition inflict brand choice information to be represented under multi-label data. This research used problem transformation method by tranforming a multi-label variable into several single-label variables. Multivariate Generalized Linear Mixed Modeling or MGLMM was selected under consideration of binary multiple responses and correlated responses presumption. Five responses of SCM and SCC brand choice modeling resulted correct model without overdispersion and the scaled pearson chi square statistic is 0.99. Tests of fixed effects indicate three factor significantly affect SCM and SCC brand choice at the 5% level. They are purchase total, province, and type of business. The variance of the random effect intercept is 1.53×10−18 or insignificant, hence MGLMM based model was similar compare to separated GLM based model.

Alignment Based Feature Selection for Multi-label Learning

Multi-label learning deals with data sets in which each example is assigned with a set of labels, and the goal is to construct a learning model to predict the label set for unseen examples. Multi-label data sets share the same problems with single-label data sets that usually possess high-dimensional features and may exist redundancy features, which will influence the performance of the algorithm. Thus it is obviously necessary to address feature selection in multi-label learning. Meanwhile, information among labels play an important role in multi-label learning, thereby it is significance to measure information among labels in order to improve the performance of learning algorithms. In this paper, we introduce kernel alignment into multi-label learning to measure the consistency between feature space and label space by which features are ranked and selected. Firstly we define an ideal kernel in label space as a convex combination of ideal kernels defined by each label, and a linear combination of kernels where each kernel corresponds to a feature. Secondly, through maximizing the kernel alignment value between linear combination kernel and ideal kernel, both weights in the two defined kernels are learned in this process simultaneously, and the learned weights of labels can be employed as the degree of labeling importance regarded as a kind of information among labels. Finally, features are ranked according to their weights in linear combined kernel, and a proper feature subset consisting of top ranking features is selected. Thus a novel method of feature selection for multi-label learning is developed which can learn and address importance degree of labels automatically, and effectiveness of this method is demonstrated by experimental comparisons.

A multilabel classification on topics of qur’anic verses in English translation using K-Nearest Neighbor method with Weighted TF-IDF

There are so many information contained in the Qur’an, it will be difficult to bring up the information manually, moreover if someone wants to know more about the Qur’an. Therefore, there is a need to find information with a certain topic that already classified in the Qur’an, especially in one verse of the Qur’an may have more than one topic (multilabel). This research examined how to build classifier to classify multilabel data which is topics of Qur’anic verses with k-Nearest Neighbor method. In this research, there is a comparison between feature extraction, Weighted TF-IDF and TF-IDF. The result of that comparison is that Weigthed TF-IDF has better performance compared to normal TF-IDF. The highest result by finding the most optimal k score is k=25 with the average score of hamming loss = 0.134875. There will be a test to measure the effect of stopword removal and lemmatization with optimal k value, for a case without stopword removal, the result is 0.136375, whereas without the lemmatization the result is 0.13537. For not using stopword removal and lemmatization the average hamming loss is 0.1373125.

Extended adaptive Lasso for multi-class and multi-label feature selection

Feature selection is a basic step and important task in applications of pattern recognition and machine learning. In this paper, we propose a new Extended Adaptive Least absolute shrinkage and selection operator (EALasso) feature selection method for multi-class and multi-label learning problems. It preserves the oracle properties of identifying the correct subset model and having the optimal estimation accuracy. An iterative optimization algorithm, with theoretical proof of convergence, is proposed to solve the optimization problem. Experimental results on several real multi-class and multi-label data sets show that the proposed EALasso method outperforms many state-of-the-art feature selection methods for multi-class single-label classification as well as multi-class multi-label classification.

Supervised representation learning for multi-label classification

Representation learning is one of the most important aspects of multi-label learning because of the intricate nature of multi-label data. Current research on representation learning either fails to consider label knowledge or is affected by the lack of labeled data. Moreover, most of them learn the representations and incorporate the label information in a two-step manner. In this paper, due to the success of representation learning by deep learning we propose a novel framework based on neural networks named SERL to learn global feature representation by jointly considering all labels in an effective supervised manner. At its core, a two-encoding-layer autoencoder, which can utilize labeled and unlabeled data, is adopted to learn feature representation in the supervision of softmax regression. Specifically, the softmax regression incorporates label knowledge to improve the performance of both representation learning and multi-label learning by being jointly optimized with the autoencoder. Moreover, the autoencoder is expanded into two encoding layers to share knowledge with the softmax regression by sharing the second encoding weight matrix. We conduct extensive experiments on five real-world datasets to demonstrate the superiority of SERL over other state-of-the-art multi-label learning approaches.

Zero‐shot multi‐label learning via label factorisation

This study considers the zero-shot learning problem under the multi-label setting where each test sample is associated with multiple labels that are unseen in training data. The authors propose a novel learning framework based on label factorisation for this problem. Specifically, the authors' framework takes three key issues into consideration and addresses them in a unified way. The first is knowledge transfer that utilises information from seen classes to build recognition models for unseen classes. The second is label correlation which means that labels which have different semantics may co-occur frequently. This is an important issue in multi-label learning. The authors propose to learn a shared latent space by label factorisation and use the label semantics as the decoding function, which can address both issues. The third is the predictability which requires the learned latent space to be strongly related to the visual features. It is guaranteed by incorporating a regression model into the learning framework. The authors derive two specific formulations from the general framework and propose the corresponding learning algorithms. The authors conducted extensive experiments on three multi-label data sets. The results demonstrated the effectiveness.

Multi-label classification via label correlation and first order feature dependance in a data stream

Many batch learning algorithms have been introduced for offline multi-label classification (MLC) over the years. However, the increasing data volume in many applications such as social networks, sensor networks, and traffic monitoring has posed many challenges to batch MLC learning. For example, it is often expensive to re-train the model with the newly arrived samples, or it is impractical to learn on the large volume of data at once. The research on incremental learning is therefore applicable to a large volume of data and especially for data stream. In this study, we develop a Bayesian-based method for learning from multi-label data streams by taking into consideration the correlation between pairs of labels and the relationship between label and feature. In our model, not only the label correlation is learned with each arrived sample with ground truth labels but also the number of predicted labels are adjusted based on Hoeffding inequality and the label cardinality. We also extend the model to handle missing values, a problem common in many real-world data. To handle concept drift, we propose a decay mechanism focusing on the age of the arrived samples to incrementally adapt to the change of data. The experimental results show that our method is highly competitive compared to several well-known benchmark algorithms under both the stationary and concept drift settings.

Hybrid Multilabel Feature Selection Using BPSO and Neighborhood Rough Sets for Multilabel Neighborhood Decision Systems

Recently, feature selection for multilabel classification has attracted substantial attention in many fields; however, some of the available methods ignore the correlations among labels and yield low classification performance. In addition, most feature selection algorithms that are based on multilabel neighborhood rough sets (MNRS) can only deal with finite sets for multilabel data. To address the issues, this paper presents a novel hybrid filter-wrapper multilabel feature selection method that is based on binary particle swarm optimization (BPSO) and MNRS with the Lebesgue measure for multilabel neighborhood decision systems. First, to overcome the problem that the traditional correlation-based feature selection (CFS) algorithm ignores the dependencies among labels, two types of average correlation between single labels and label sets and among labels are presented. Via combination with information-entropy-based uncertainty measures, a new average correlation among labels is studied. A novel comprehensive evaluation function of CFS (NCFS) is constructed. Then, NCFS is introduced as a fitness function into the original BPSO and improved BPSO algorithms to optimize multilabel classification in the early and later stages, respectively, and the optimization process is terminated when the maximum number of iterations is reached. Next, the Lebesgue measure of the neighborhood class is developed for investigating the neighborhood approximation accuracy and the dependency degree based on MNRS. Various properties are deduced, and the relationships among these measures are used to evaluate the uncertainty and correlations among labels of multilabel data. Finally, a hybrid filter-wrapper feature selection algorithm using NCFS-BPSO is designed for preliminarily eliminating redundant features to decrease the complexity, and a heuristic forward multilabel feature selection algorithm is proposed for improving the performance of multilabel classification. Experimental results on fifteen multilabel datasets demonstrate that our proposed algorithms are effective in selecting significant features and realizing great classification performance in multilabel neighborhood decision systems.

Deep Multi-Level Semantic Hashing for Cross-Modal Retrieval

With the rapid growth of multimodal data, the cross-modal search has widely attracted research interests. Due to its efficiency on storage and computing, hashing-based methods are broadly used for large scale cross-modal retrieval. Most existing hashing methods are designed based on binary supervision, which transforms complex relationships of multi-label data into simple similar or dissimilar. However, few methods have explored the rich semantic information implicit in multi-label data to improve the accuracy of searching results. In this paper, the multi-level semantic supervision generating approach is proposed by exploring the label relevance. And a deep hashing framework is designed for multi-label image-text cross retrieval tasks. It can simultaneously capture the binary similarity and the complex multi-level semantic structure of data in different forms. Moreover, the effects of three different convolutional neural networks, CNN-F, VGG-16, and ResNet-50, on the retrieval results are compared. The experimental results on an open source cross-modal dataset show that our approach outperforms several state-of-the-art hashing methods, and the retrieval result on the CNN-F network is better than VGG-16 and ResNet-50.

An Efficient Stacking Model of Multi-Label Classification Based on Pareto Optimum

Nowadays, multi-label data are ubiquitous in real-world applications, in which each instance is associated with a set of labels. Multi-label learning has attracted significant attentions from researchers and plenty of algorithms have been proposed. Among those algorithms binary relevance (BR) is a widely used framework for multi-label classification. It constructs binary classifiers for each label by means of one-vs-rest style. BR approach is a simple and straight forward way of problem transformation for multi-label learning, but it ignores label correlations totally. Stacking based BR is a feasible way to tackle this problem. The key issue of stacking based BR is how to select label subset to extend the original features for each label. Existing methods of stacking based BR usually select identical label subset for all labels. It may be suboptimal as each label has its own most related label subset. In this paper, a novel stacking based method is introduced to utilize label correlations based on Pareto Optimum for improving the performance of BR. Our method builds a stack of two layers of BR classifiers. At the first layer, a group of binary classifiers are constructed, one for a label. At the second layer, for each label we employ Pareto Optimum to select most related label subset, then augment the original features by the selected label subset. The final binary classifiers for each label are constructed based on their corresponding reconstructed feature space. Comparing to other well-established stacking multi-label learning algorithms in terms of different multi-label classification criteria, experimental results on several multi-label benchmark datasets testify the superiority of the proposed methods.

A Novel IGBT Health Evaluation Method Based on Multi-Label Classification

The IGBT health evaluation of power semiconductor devices is usually based on the threshold evaluation method, which is usually a single characteristic parameter evaluation system. This kind of evaluation method cannot reflect the internal correlation of the change of multiple characteristic parameters in the deep level. Multi-label classification plays an important role in machine learning and can truly reflect the internal correlation principle of multi-feature parameters. Many studies have proved that multi-label classification (mlc) can effectively increase the actual classification effect of the clustering algorithm. In this paper, a clustering algorithm based on multi-label learning is applied to the health evaluation of IGBT. There are many characteristic parameters that affect each other in the actual work of IGBT, so it is difficult for a single label to reflect its actual health status. At the same time, multi-label data often belong to multiple classifications. Multi-label learning can improve the feature dependence ability of clustering method and improve the accuracy of classification. In this paper, we propose a multi-label classification learning model based on ISODATA for the multi-feature parameters of power semiconductor device IGBT, which can comprehensively consider the multi-level correlation effect of internal parameters in the multi-feature parameter extraction. The experiment results show that the algorithm model can better adapt to the IGBT health classification evaluation compared with the general clustering algorithm.

Joint Label-Density-Margin Space and Extreme Elastic Net for Label-Specific Features

The label-specific features learning is a kind of framework for extracting the specific features of each label for classification. At present, the label-specific features algorithm is generally based on the original label space to find a particular feature set. This kind of extraction method for label-specific features has a general adaptation when the label density is balanced. However, in most multi-label data sets, the number of positive and negative labels varies greatly, and the label density is unbalanced. And there is an inherent correlation between each label. When extracting label-specific features, the method needs to consider issues such as imbalanced label density and label correlations. Based on this, a label-specific features learning method based on Joint Label-density-margin Space and Extreme Elastic Net (JLSE2N) is proposed in this paper. Firstly, to enlarge the margin between the positive and negative labels in the original label space, the positive and negative label density of the total example are calculated. And the information of unbalanced label density is integrated into the label space, thus forming a priori knowledge of the label-density-margin space. Subsequently, to quickly extract the label-specific features, the elastic net regularization extreme learning machine is used for the first time to obtain the label-specific features. The transformed label density space is used to calculate the cosine similarity and is added to the L2 regularization term to consider the correlation between pairs of labels. The L1 regularization term in the elastic net can generate the sparse weight matrix to extract the required label-specific features. We compare the proposed method with five well-known algorithms on 11 benchmark data sets. Experimental analysis shows that our proposed method is superior to the state-of-the-art algorithms for multi-label learning.

Distributed Selection of Continuous Features in Multilabel Classification Using Mutual Information.

Multilabel learning is a challenging task demanding scalable methods for large-scale data. Feature selection has shown to improve multilabel accuracy while defying the curse of dimensionality of high-dimensional scattered data. However, the increasing complexity of multilabel feature selection, especially on continuous features, requires new approaches to manage data effectively and efficiently in distributed computing environments. This article proposes a distributed model for mutual information (MI) adaptation on continuous features and multiple labels on Apache Spark. Two approaches are presented based on MI maximization, and minimum redundancy and maximum relevance. The former selects the subset of features that maximize the MI between the features and the labels, whereas the latter additionally minimizes the redundancy between the features. Experiments compare the distributed multilabel feature selection methods on 10 data sets and 12 metrics. Results validated through statistical analysis indicate that our methods outperform reference methods for distributed feature selection for multilabel data, while MIM also reduces the runtime in orders of magnitude.

Addressing Imbalance in Weakly Supervised Multi-Label Learning

Multi-label learning has been widely used in many fields to solve the problem of assigning multiple related categories to an instance. Nevertheless, the label for each training example is assumed complete in most of the current multi-label learning methods. As a matter of fact, it is often hard to obtain training samples with complete labels, thus weakly supervised multi-label learning is demanded, which has become a hot topic in recent years. Moreover, the missing labels would further aggravate the class imbalance in multi-label learning. In this paper, the asymmetric stage-wise loss function is introduced to make positive class samples farther away from the classification boundary than negative class samples by adjusting ramp as well as the margin parameters. In addition, the general aggregate loss function is replaced with the average top-k aggregate loss so as to protect the non-typical distributed samples from being sacrificed in the aggregation process of the loss function, and therefore improve the identification accuracy of minority labels. The experiments on both standard and large-scale multi-label data sets demonstrate that the proposed algorithm can solve the problem of class imbalance by changing the sample distribution of training set, thus obtaining competitive performance.

Multi-Label Learning With Label Specific Features Using Correlation Information

To deal with the problem where each instance is associated with multiple labels, a lot of multi-label learning algorithms have been developed in recent years. Some approaches have been proposed to select label-specific features to utilize discriminate features for multi-label classification. Although label correlation has been considered in learning label-specific features, the critical correlation among instances was less taken into account. In this paper, we proposed a new approach called multi-label learning with label-specific features using correlation information (LSF-CI) to learn label-specific features for each label with the consideration of both correlation information in label space and correlation information in feature space. In the LSF-CI, the instance correlation in feature space is computed by a probabilistic neighborhood graph model, and label correlation in label space is computed by cosine similarity. For multi-label data, the LSF-CI has the capability to select Label-specific features for each label as well as classify an unseen instance into a set of relevant labels. To validate the effectiveness of LSF-CI, we conducted comprehensive experiments on eight multi-label datasets. The experimental results demonstrate that the LSF-CI is capable of selecting compact label-specific features, and achieving a competitive performance in comparison with the performances of the existing multi-label learning approaches.

Using Data Mining Techniques to Guide Academic Programs Design and Assessment

This paper presents a novel application of data mining techniques to guide academic programs design and assessment. More specifically, it propose using association rule mining techniques to discover a set of rules that govern the relationship between two core components of an academic program, program educational objectives (PEOs) and students outcomes(SOs). As a case study, this paper demonstrates how association rule mining techniques are applied to mine mapping rules between the PEOs and a predefined set of SOs adopted by the American Board for Engineering and Technology-Engineering Accreditation Commission (ABET-EAC) for engineering programs. To this end, a set of 152 ABET-EAC accredited engineering programs’ self-study reports have been collected and the mapping data between their PEOs and ABET-EAC SOs have been extracted. This dataset has been pre-processed and transformed into a representation suitable for applying association rule mining techniques. This involves identifying a set of PEOs labels, annotating data instances with PEOs labels, and projecting each multi-label data instance into a set of single-label instances. Apriopi algorithm is then applied to discover the rules that govern the mapping between PEOs and ABET-EAC SOs. The discovered rules are of particular importance for guiding the design and assessment of engineering academic programs. Besides that, the discovered rules unveil of a number of interesting correlations between PEOs and ABET-EAC SOs that need further investigation by pedagogists.

A classification algorithm based on weighted ML-kNN for multi-label data

The ML-kNN algorithm uses naive Bayesian classification to modify the traditional kNN algorithm to solve multi-label classification problems. However, the ML-kNN algorithm is prone to misjudgement or incomplete judgment of the unseen instance's label set in two special cases: when the number of labels in the training set is not balanced and when the training instances are unevenly distributed in space. Therefore, a weighted ML-kNN algorithm (i.e., wML-kNN) is proposed in this paper. The main idea is to assign different weights to each label according to the proportion of labels and mutual information of the spatial distribution of unseen instances to training instances. This method can reduce the probability of misjudgement of the unseen instance's label set. A comparative study was conducted on four multi-label datasets that included review classification and three other published benchmark multi-label datasets: yeast gene function analysis, natural scene classification, and musical sentiment classification. The results show that the performance of the wML-kNN algorithm is better than the other four multi-label learning algorithms, including ML-kNN.