Multi-label Decision Research Articles

CC<i>k</i>EL: Compensation-based correlated <i>k</i>-labelsets for classifying imbalanced multi-label data

<abstract> <p>Imbalanced data distribution and label correlation are two intrinsic characteristics of multi-label data. This occurs because in this type of data, instances associated with certain labels may be sparse, and some labels may be associated with others, posing a challenge for traditional machine learning techniques. To simultaneously adapt imbalanced data distribution and label correlation, this study proposed a novel algorithm called compensation-based correlated <italic>k</italic>-labelsets (CC<italic>k</italic>EL). First, for each label, the CC<italic>k</italic>EL selects the <italic>k</italic>-1 strongest correlated labels in the label space to constitute multiple correlated <italic>k</italic>-labelsets; this improves its efficiency in comparison with the random <italic>k</italic>-labelsets (RA<italic>k</italic>EL) algorithm. Then, the CC<italic>k</italic>EL transforms each <italic>k</italic>-labelset into a multiclass issue. Finally, it uses a fast decision output compensation strategy to address class imbalance in the decoded multi-label decision space. We compared the performance of the proposed CC<italic>k</italic>EL algorithm with that of multiple popular multi-label imbalance learning algorithms on 10 benchmark multi-label datasets, and the results show its effectiveness and superiority.</p> </abstract>

Predicting and characterising persuasion strategies in misinformation content over social media based on the multi-label classification approach

Persuasion aims at affecting the audience’s attitude and behaviour through a series of messages containing persuasion strategies. In the context of misinformation spread, identifying the persuasion strategies is important in order to warn people to be aware of the analogous persuasion attempts in the future. In this work, we address the prediction of persuasion strategies in micro-blogging posts through a multi-label classification approach based on a variety of lexical and semantic features. We conduct our experiments using a set of well-known multi-label classification algorithms, including multi-label decision tree, multi-label k-nearest neighbours, multi-label random forest, binary relevance and classifier chains. The results show that the model incorporating classifier chains and XGBoost algorithm achieves the best subset accuracy of 0.779 and the highest macro F1-score of 0.847. In addition, we evaluated and compared the features’ importance for different persuasion strategies and analysed the major errors of miss-out prediction. The findings of this article provide a benchmark for the multi-label classification of persuasion strategies in micro-blogging posts and lead to a better understanding of different persuasion attempts contained in social media misinformation.

Intelligent Perception of Multiaroma Types Based on Machine Olfaction

Machine olfactory perception (MOP) is a new type of intelligent perception technology that senses and describes odors. The difference between MOP and machine olfaction is that MOP describes odors by using some semantic labels. One of the challenges in realizing MOP is an appropriate multilabel classifier from the high-dimensional sensor responses. Here, we propose a novel aroma-type (AT) oriented MOP method, which is an MOP method of multiple AT. Wherein, the method designed an intelligent perception of multiple ATs based on a convolutional neural network (IPMAT-CNN), in which an olfactory information-shared mechanism was introduced to enhance the feature extraction of the network. In addition, by analyzing the molecular correlation between multiple ATs, a matrix addition method was designed in IPMAT-CNN to strengthen the information on adjacent ATs. These designs effectively improved the accuracy of model detection. At last, IPMAT-CNN successfully sensed and recognized seven ATs of 38 monomer flavors. The effectiveness of IPMAT-CNN was evaluated by four baseline models [classifier chain based on random forest classifier (CC-RF), label powerset based on random forest classifier (LP-RF), multilabel decision tree (ML-DT), and multilabel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${k}$ </tex-math></inline-formula> -nearest neighbor (ML-kNN)]. The results showed that the IPMAT-CNN had the best performance with a 0.0636 hamming loss, a 93.65% accuracy, and a 0.8149 mean average precision (mAP) value for classifying the seven ATs. It reveals that the IPMAT-CNN had a decent prediction performance and can be used as an effective reference method for the ATs classification.

DAC Stacking: A Deep Learning Ensemble to Classify Anxiety, Depression, and Their Comorbidity From Reddit Texts.

Depression is the most incapacitating disease worldwide, and it has an alarming comorbidity rate with anxiety. The use of social networks to expose personal difficulties has enabled works on the automatic identification of specific mental conditions, particularly depression. In spite of many solutions proposed for the automatic recognition of depression, fewer exist for anxiety and its comorbidity with depression. In this paper, we propose DAC Stacking, a solution that leverages stacking ensembles and Deep Learning (DL) to automatically identify depression, anxiety, and their comorbidity, using data extracted from Reddit. The stacking is composed of single-label binary classifiers, that either distinguish between specific disorders and control users (experts), or between pairs of target conditions (differentiating). A meta-learner explores these base classifiers as a context for reaching a multi-label decision. We assessed alternative ensemble topologies, exploring roles for base models, DL architectures, and word embeddings. All base classifiers and ensembles outperformed the baselines for depression and anxiety (f-measures near 0.79). The ensemble topology with the best performance (Hamming Loss of 0.29 and Exact Match Ratio of 0.46) combines base classifiers of three DL architectures, and includes expert and differentiating base models. The analysis of the influential classification features according to SHAP revealed the strengths of our solution and provided insights on the challenges for the automatic classification of the addressed mental conditions.

HML-RF: Hybrid Multi-Label Random Forest

Multi-label classification is the supervised learning problem in which an instance is associated with a set of labels. In this, labels are correlated, and hence label dependency information plays a vital role. Its always been a question of research to decide the order of labels to exploit their inter-dependency. Hence, to this end, many research works are done that, in general, can be categorized as problem transformation and algorithm adaptation techniques. The problem transformation reconstructs the multi-label problem as a multiple single class problem. The algorithm transformation modifies the existing well-known machine learning approaches to solve the multi-label classification problem. However, these two techniques have their pros and cons. In this paper, we propose a novel approach to consider the merits of both techniques, hence named Hybrid Multi-Label Random Forest (HML-RF). The multi-label decision trees are used as base classifiers in the proposed approach to construct the HML-RF model. Each base classifier is constructed over a randomly selected subset of labels to exploit the label dependency. We also formulate a way to compute the tree strength of a multi-label decision tree, which is used to construct the HML-RF with strength (HML-RFws). The efficacy of the proposed approach is tested over the ten well-known and publicly available datasets. Experimental results show the HML-RF is performing better for at-least six datasets, and the HML-RFws is performing better for at-least nine datasets in comparison to state-of-the-art approaches in terms of accuracy, hamming loss, and zero-one loss. Finally, the statistical test is also validating all the experimental results.

MEGA: Predicting the best classifier combination using meta-learning and a genetic algorithm

Classifier combination through ensemble systems is one of the most effective approaches to improve the accuracy of classification systems. Ensemble systems are generally used to combine classifiers; However, selecting the best combination of individual classifiers is a challenging task. In this paper, we propose an efficient assembling method that employs both meta-learning and a genetic algorithm for the selection of the best classifiers. Our method is called MEGA, standing for using MEta-learning and a Genetic Algorithm for algorithm recommendation. MEGA has three main components: Training, Model Interpretation and Testing. The Training component extracts meta-features of each training dataset and uses a genetic algorithm to discover the best classifier combination. The Model Interpretation component interprets the relationships between meta-features and classifiers using a priori and multi-label decision tree algorithms. Finally, the Testing component uses a weighted k-nearest-neighbors algorithm to predict the best combination of classifiers for unseen datasets. We present extensive experimental results that demonstrate the performance of MEGA. MEGA achieves superior results in a comparison of three other methods and, most importantly, is able to find novel interpretable rules that can be used to select the best combination of classifiers for an unseen dataset.

Optimal Granularity Selection in Multi-label Decision Table

To solve the problem of granularity selection in multi-label decision classification, a global and object-based optimal granularity selection method is proposed. Firstly, the granularity partitioning of multi-label decision tables is analyzed based on multiple granularity levels and the granulation roughness measurement method of multi-label decision tables is introduced. Then the general optimal granularity selection method is discussed for coordinated decision tables and uncoordinated decision tables. Finally, in view of the limitation that the global optimal granularity selection cannot make every object reach the optimal granularity, and the uncertainty of some objects in the uncoordinated decision table regarding the classification of decision markers, the local optimal granularity selection method of objects is discussed, and the effectiveness of the method is verified by an example.

Multi-label feature selection based on information entropy fusion in multi-source decision system

Feature selection plays an important role in high-dimensional multi-source data, which can improve classification performance of learning algorithm. Most of existing multi-source information fusion focus on the single decision system without considering multi-source and multi-label problems together. Nevertheless, data from different sources along with multiple labels simultaneously are absolutely frequent in many real-world applications. For this issue, in this paper, a multi-source multi-label decision system is proposed, which has more than one decision label. To remove some redundant or irrelevant features in multi-source multi-label decision system, a feature selection algorithm based on positive region for multi-source multi-label data is explored, which uses the feature dependency carried on the fusion decision table. Finally, examples are introduced to elaborate the detail process of the proposed algorithm, and experimental results show the effective performance of the proposed algorithm on multi-source and multi-label data.

Feature Selection for Multi-Label Learning Based on F-Neighborhood Rough Sets

Multi-label learning is often applied to handle complex decision tasks, and feature selection is its essential part. The relation of labels is always ignored or not enough to consider for both multi-label learning and its feature selection. To deal with the problem, F-neighborhood rough sets are employed. Different from other methods, the original approximate space is not changed, but the relation of labels is sufficient to consider. To be specific, a multi-label decision system is discomposed into a family of single-label decision tables with the label set(first-order strategy) at first. Secondly, calculate attribute significance in the family of single-label decision tables. Third, construct an attribute significance matrix and improved attribute significance matrices to evaluate the quality of the features, then a parallel reduct is obtained with information fusion. These processes construct F-neighborhood parallel reduction algorithm for a multi-label decision system(FNPRMS). Compared with the state-of-the-arts, experimental results show that FNPRMS is effective and efficient on 9 publicly available data sets.

Non-parametric predictive inference for solving multi-label classification

Decision Trees (DTs) have been adapted to Multi-Label Classification (MLC). These adaptations are known as Multi-Label Decision Trees (ML-DT). In this research, a new ML-DT based on the Nonparametric Predictive Inference Model on Multinomial data (NPI-M) is proposed. The NPI-M is an imprecise probabilities model that provides good results when it is applied to DTs in standard classification. Unlike other models based on imprecise probabilities, the NPI-M is a nonparametric approach and it does not make unjustified assumptions before observing data. It is shown that the new ML-DT based on the NPI-M is more robust to noise than the ML-DT based on precise probabilities. As the intrinsic noise in MLC might be higher than in traditional classification, it is expected that the new ML-DT based on the NPI-M outperforms the already existing ML-DT. This fact is validated with an exhaustive experimentation carried out in this work on different MLC datasets with several levels of added noise. In it, many MLC evaluation metrics are employed in order to measure the performance of the algorithms. The experimental analysis shows that the proposed ML-DT based on NPI-M obtains better results than the ML-DT that uses precise probabilities, especially when we work on data with noise.

Optimal granulation selection for multi-label data based on multi-granulation rough sets

Multi-label decision data serve as an important extension of decision data, where several labels associate with each object. In classical multi-granulation rough set models, granulation selection is always related to positive region. However, some positive region reducts are not appropriate for multi-label data due to the fact that the uncertainties implied by labels are not totally considered. To overcome this deficiency in this paper, we propose the optimal granulation selection in multi-granulation and multi-label decision table. First, two kinds of fine and coarse decision functions are introduced, which represent all certainly and all possibly associated labels for the objects in the same class from optimistic and pessimistic viewpoints, respectively. Some relevant propositions and theorems are also carefully examined. Based on granular significance, a novel approach of optimal granulation selection is discussed. Finally, a heuristic algorithm is designed and an illustrative example is given to show the effectiveness of this algorithm. The main contribution of this paper is to extend the attribute reduction of multi-label data based on rough set theory, and overcome the limitation of the positive region reducts, in which the uncertainties implied by labels are not totally considered.

A wrapped multi-label classifier for the automatic diagnosis and prognosis of Alzheimer’s disease

BackgroundAD is the most frequent neurodegenerative disease, severely impacting our society. Early diagnosis and prognosis are challenging tasks in the management of AD patients. New methodWe implemented a machine-learning classifier for the automatic early diagnosis and prognosis of AD by means of features extracted, selected and optimized from structural MRI brain images. The classifier was designed to perform multi-label automatic classification into the following four classes: HC, ncMCI, cMCI, and AD. ResultsFrom our analyses, it emerged that MMSE and hippocampus-related measures must be included as primary measures in automatic-classification systems for both the early diagnosis and the prognosis of AD. The voting scheme mainly based on the binary-classification performances on the different four groups is the best choice to model the multi-label decision function for AD, when compared with a simple majority-vote scheme or with a scheme aimed at discriminating patients with high vs low risk of conversion to AD and therapy addressing. Comparison with existing method(s)The accuracies of our binary classifications were higher than or comparable to previously published methods. An improvement is needed on the approach we used to combine binary-classification outputs to obtain the final multi-label classification. ConclusionsThe performance of multi-label automatic-classification systems strongly depends on the choice of the voting scheme used for combining binary-classification labels.

Multimodal Biosignal Acquisition System for Remote Health Monitoring

The aim was to summarize the research on remote health monitoring system that comprises of the multimodal biosignal acquisition system, thereby delivering multi-label classification and clinical decision support system (CDSS). Numerous studies were involved effectively which offers the rapid diagnosis for the patients in critical situations. The learning model was established using the multi-label classification, extending to decision support system which can be enriched using the context-awareness mechanisms to foresee the future vital signs, and thus delivering an alert to the patients or doctors to take up the required actions.

Multi-stage optimization of decision and inhibitory trees for decision tables with many-valued decisions

We study problems of optimization of decision and inhibitory trees for decision tables with many-valued decisions. As cost functions, we consider depth, average depth, number of nodes, and number of terminal/nonterminal nodes in trees. Decision tables with many-valued decisions (multi-label decision tables) are often more accurate models for real-life data sets than usual decision tables with single-valued decisions. Inhibitory trees can sometimes capture more information from decision tables than decision trees. In this paper, we create dynamic programming algorithms for multi-stage optimization of trees relative to a sequence of cost functions. We apply these algorithms to prove the existence of totally optimal (simultaneously optimal relative to a number of cost functions) decision and inhibitory trees for some modified decision tables from the UCI Machine Learning Repository.

多值决策表的最小决策树生成

决策树技术在数据挖掘的分类领域应用极其广泛，可以从普通决策表(每行记录包含一个决策值)中挖掘有价值的信息，但是要从多值决策表(每行记录包含多个决策值)中挖掘潜在的信息则比较困难。多值决策表中每行记录包含多个决策值，多个决策属性用一个集合表示。针对已有的启发式算法，如贪心算法，由于性能不稳定的特点，该算法获得的决策树规模变化较大，本文基于动态规划的思想，提出了使决策树规模最小化的算法。该算法将多值决策表分解为多个子表，通过多值决策表的子表进行构造最小决策树，进而对多值决策表进行数据挖掘。 Decision tree is a widely used classification in data mining. It can discover the essential knowledge from the common decision tables (each row has a decision). However, it is difficult to do data mining from the multi-label decision tables (each row has a set of decisions). In a multi-label decision tables, each row contains several decisions, and several decision attributes are represented using a set. By testing the existing heuristic algorithms, such as greedy algorithms, their performance is not stable, i.e., the size of the decision tree might become very large. In this paper, we propose a dynamic programming algorithm to minimize the size of the decision trees for a multi- label decision table. In our algorithm, the multi-label decision table is divided into several subtables, and the decision tree is constructed by using all subtables of the multi-label decision table, then useful information can be discovered from the multi-label decision tables.

A Variable Precision Attribute Reduction Approach in Multilabel Decision Tables

Owing to the high dimensionality of multilabel data, feature selection in multilabel learning will be necessary in order to reduce the redundant features and improve the performance of multilabel classification. Rough set theory, as a valid mathematical tool for data analysis, has been widely applied to feature selection (also called attribute reduction). In this study, we propose a variable precision attribute reduct for multilabel data based on rough set theory, called δ-confidence reduct, which can correctly capture the uncertainty implied among labels. Furthermore, judgement theory and discernibility matrix associated with δ-confidence reduct are also introduced, from which we can obtain the approach to knowledge reduction in multilabel decision tables.

Multi-Label Classification Method for Multimedia Tagging

Community tagging offers valuable information for media search and retrieval, but new media items are at a disadvantage. Automated tagging may populate media items with few tags, thus enabling their inclusion into search results. In this paper, a multi-label decision tree is proposed and applied to the problem of automated tagging of media data. In addition to binary labels, the proposed Iterative Split Multi-label Decision Tree (IS-MLT) is easily extended to the problem of weighted labels (such as those depicted by tag clouds). Several datasets of differing media types show the effectiveness of the proposed method relative to other multi-label and single label classifier methods and demonstrate its scalability relative to single label approaches.Keywords: Automated Multimedia Tagging; Community Tagging; Multi-label Classification; Multi-label Decision Tree; Pattern Classification

Predicting gene function using hierarchical multi-label decision tree ensembles

BackgroundS. cerevisiae, A. thaliana and M. musculus are well-studied organisms in biology and the sequencing of their genomes was completed many years ago. It is still a challenge, however, to develop methods that assign biological functions to the ORFs in these genomes automatically. Different machine learning methods have been proposed to this end, but it remains unclear which method is to be preferred in terms of predictive performance, efficiency and usability.ResultsWe study the use of decision tree based models for predicting the multiple functions of ORFs. First, we describe an algorithm for learning hierarchical multi-label decision trees. These can simultaneously predict all the functions of an ORF, while respecting a given hierarchy of gene functions (such as FunCat or GO). We present new results obtained with this algorithm, showing that the trees found by it exhibit clearly better predictive performance than the trees found by previously described methods. Nevertheless, the predictive performance of individual trees is lower than that of some recently proposed statistical learning methods. We show that ensembles of such trees are more accurate than single trees and are competitive with state-of-the-art statistical learning and functional linkage methods. Moreover, the ensemble method is computationally efficient and easy to use.ConclusionsOur results suggest that decision tree based methods are a state-of-the-art, efficient and easy-to-use approach to ORF function prediction.