Hierarchical Multi-label Classification Research Articles

A top-down supervised learning approach to hierarchical multi-label classification in networks

Node classification is the task of inferring or predicting missing node attributes from information available for other nodes in a network. This paper presents a general prediction model to hierarchical multi-label classification, where the attributes to be inferred can be specified as a strict poset. It is based on a top-down classification approach that addresses hierarchical multi-label classification with supervised learning by building a local classifier per class. The proposed model is showcased with a case study on the prediction of gene functions for Oryza sativa Japonica, a variety of rice. It is compared to the Hierarchical Binomial-Neighborhood, a probabilistic model, by evaluating both approaches in terms of prediction performance and computational cost. The results in this work support the working hypothesis that the proposed model can achieve good levels of prediction efficiency, while scaling up in relation to the state of the art.

Hierarchical Multilabel Ship Classification in Remote Sensing Images Using Label Relation Graphs

Hierarchical multilabel classification (HMC) assigns multiple labels to each instance with the labels organized under hierarchical relations. In ship classification in remote sensing images, depending on the expert knowledge and image quality, the same type of ships in different remote sensing images may be annotated with different class labels from coarse to fine levels such as merchant ship (MS) or container ship (CTS). In this article, we propose a novel deep network with two output channels and their associated loss functions to learn an HMC classifier using samples labeled at different levels in the hierarchy. In the proposed network, a hierarchy and exclusion (HEX) graph is introduced to model the label hierarchy, which satisfies hierarchical constraints by encoding semantic relations between any two labels. The output nodes of the first channel are organized according to the HEX graph, and its corresponding probabilistic classification loss is built to reflect the hierarchical structure of the HEX graph. On the other hand, the output nodes of the second channel only represent the finest grained (last level in the hierarchy) classes, and its multiclass cross-entropy loss is designed to enhance the discriminative power of the HMC classifier on the last level labels, which is also compatible with constraints in the HEX graph. The combination of these two losses from two output channels can effectively transfer the hierarchical information of ship taxonomy during network training. Experimental results on two commonly used ship datasets demonstrate that the proposed method outperforms the state-of-the-art HMC approaches, and is especially advantageous when trained with fewer fine-grained samples.

Weakly Supervised Hierarchical Multi-Label Classification of Lung Cancer Types Based on Rapid On-Site Evaluation in Transbronchial Biopsy

Weakly Supervised Hierarchical Multi-Label Classification of Lung Cancer Types Based on Rapid On-Site Evaluation in Transbronchial Biopsy

Application of Uncertainty Thought Environment in Judicial Adjudication Based on Cognitive Psychology.

The uncertainty of judicial decision-making has a deep and extensive theoretical foundation. Theoretical analysis starts with a reflection on legal rationalism that challenges the legal certainty before delving deeply into the case's facts and the entire legal system. In light of this, this paper explores a novel approach to enhance the reasoning mechanism of trial documents from the viewpoint of modern cognitive psychology, concentrating on the parties' and the public's cognitive processes to justice. It is suggested to use an inert hierarchical multilabel classification algorithm. In order to predict the category of invisible examples, the extended multilabel training set is first searched for adjacent samples of invisible examples, and the classification weight and confidence of each category are then determined in accordance with these adjacent samples. The group of invisible examples is then anticipated. Experimental comparison demonstrates that this algorithm outperforms other prediction techniques; the macro accuracy, macro recall, and macro F1 of this method are, respectively, 0.896, 0.871, and 0.814. It has some advantages in many multilabel evaluation indexes when compared to other multilabel algorithms.

Smart Fashion: A Review of AI Applications in Virtual Try-On & Fashion Synthesis

The rapid progress of computer vision, machine learning, and artificial intelligence combined with the current growing urge for online shopping systems opened an excellent opportunity for the fashion industry. As a result, many studies worldwide are dedicated to modern fashion-related applications such as virtual try-on and fashion synthesis. However, the accelerated evolution speed of the field makes it hard to track these many research branches in a structured framework. This paper presents an overview of the matter, categorizing 110 relevant articles into multiple sub-categories and varieties of these tasks. An easy-to-use yet informative tabular format is used for this purpose. Such hierarchical application-based multi-label classification of studies increases the visibility of current research, promotes the field, provides research directions, and facilitates access to related studies.

Multi-Label Classification Neural Networks with Hard Logical Constraints

Multi-label classification (MC) is a standard machine learning problem in which a data point can be associated with a set of classes. A more challenging scenario is given by hierarchical multi-label classification (HMC) problems, in which every prediction must satisfy a given set of hard constraints expressing subclass relationships between classes. In this article, we propose C-HMCNN(h), a novel approach for solving HMC problems, which, given a network h for the underlying MC problem, exploits the hierarchy information in order to produce predictions coherent with the constraints and to improve performance. Furthermore, we extend the logic used to express HMC constraints in order to be able to specify more complex relations among the classes and propose a new model CCN(h), which extends C-HMCNN(h) and is again able to satisfy and exploit the constraints to improve performance. We conduct an extensive experimental analysis showing the superior performance of both C-HMCNN(h) and CCN(h) when compared to state-of-the-art models in both the HMC and the general MC setting with hard logical constraints.

LA-HCN: Label-based Attention for Hierarchical Multi-label Text Classification Neural Network

Hierarchical multi-label text classification (HMTC) has been gaining popularity in recent years thanks to its applicability to a plethora of real-world applications. The existing HMTC algorithms largely focus on the design of classifiers, such as the local, global, or a combination of them. However, very few studies have focused on hierarchical feature extraction and explore the association between the hierarchical labels and the text. In this paper, we propose a Label-based Attention for Hierarchical Multi-label Text Classification Neural Network (LA-HCN), where the novel label-based attention module is designed to hierarchically extract important information from the text based on the labels from different hierarchy levels. Besides, hierarchical information is shared across levels while preserving the hierarchical label-based information. Separate local and global document embeddings are obtained and used to facilitate the respective local and global classifications. In our experiments, LA-HCN outperforms other state-of-the-art neural network-based HMTC algorithms on four public HMTC datasets. The ablation study also demonstrates the effectiveness of the proposed label-based attention module as well as the novel local and global embeddings and classifications. By visualizing the learned attention (words), we find that LA-HCN is able to extract meaningful information corresponding to the different labels which provides explainability that may be helpful for the human analyst.

Hybrid embedding-based text representation for hierarchical multi-label text classification

Many real-world text classification tasks often deal with a large number of closely related categories organized in a hierarchical structure or taxonomy. Hierarchical multi-label text classification (HMTC) has become rather challenging when it requires handling large sets of closely related categories. The structural features of all categories in the entire hierarchy and the word semantics of their category labels are very helpful in improving text classification accuracy over large sets of closely related categories, which has been neglected in most of existing HMTC approaches. In this paper, we present a hybrid embedding-based text representation for HMTC with high accuracy. First, the hybrid embedding consists of both graph embedding of categories in the hierarchy and their word embedding of category labels. The Structural Deep Network Embedding-based graph embedding model is used to simultaneously encode the global and local structural features of a given category in the whole hierarchy for making the category structurally discriminable. We further use the word embedding technique to encode the word semantics of each category label in the hierarchy for making different categories semantically discriminable. Second, we presented a level-by-level HMTC approach based on the bidirectional Gated Recurrent Unit network model together with the hybrid embedding that is used to learn the representation of the text level-by-level. Last but not least, extensive experiments were made over five large-scale real-world datasets in comparison with the state-of-the-art hierarchical and flat multi-label text classification approaches, and the experimental results show that our approach is very competitive to the state-of-the-art approaches in classification accuracy, in particular maintaining computational costs while achieving superior performance.

Hierarchical multilabel classification by exploiting label correlations

Hierarchical multilabel classification by exploiting label correlations

Oblique predictive clustering trees

Predictive clustering trees (PCTs) are a well-established generalization of standard decision trees, which can be used to solve a variety of predictive modeling tasks, including structured output prediction. Combining them into ensembles of PCTs yields state-of-the-art performance. However, they scale poorly to problems with high-dimensional output spaces and cannot exploit sparsity in data. Both of these issues are typically highlighted in (hierarchical) multi-label classification tasks, where the output can consist of hundreds of labels (high dimensionality), among which only a few are relevant for each example (sparsity). Sparsity is also often encountered in the input space (molecular fingerprints, bag-of-words representations, etc.). In this paper, we propose oblique predictive clustering trees capable of addressing these limitations. We design and implement two methods for learning oblique splits that contain linear combinations of features in the tests, hence a split corresponds to an arbitrary hyperplane in the input space. The resulting oblique trees are efficient for high-dimensional data and are capable of exploiting sparse data. We experimentally evaluate the proposed methods on 60 benchmark datasets for 6 predictive modeling tasks. The results of the experiments show that oblique predictive clustering trees achieve performance on par with state-of-the-art methods and are orders of magnitude faster than standard predictive clustering trees. We also show that meaningful feature importance scores can be extracted from the models learned with the proposed methods.

Condition-CNN: A hierarchical multi-label fashion image classification model

Current state of the art image classifiers predict a single class label of an image. However, in many industry settings such as online shopping, images belong to a class hierarchy where the first level represents the coarse grained or the most abstract class with subsequent levels representing the more specific classes. We propose a novel hierarchical image classification model, Condition-CNN, which addresses some of the shortcomings of the branching convolutional neural network in terms of training time and fine-grained accuracy. It applies the Teacher Forcing training algorithm, where the actual class labels of the higher level classes rather than the predicted labels are used to train the lower level branches. The technique also prevents error propagation, and thereby, reduces the training time. Besides learning the image features for each level of classes, Condition-CNN also learns the relationship between different levels of classes as conditional probabilities, which is used to estimate class predictions during scoring. By feeding the estimated higher-level class predictions as priors to the lower-level class prediction, Condition-CNN achieves a superior prediction accuracy while requiring fewer trainable parameters compared to the baseline CNN models. The validation results of Condition-CNN using the Kaggle Fashion Product Images data set demonstrate a prediction accuracy of 99.8%, 98.1%, and 91.0% for Level 1, 2 and 3 classes respectively, which are greater than that of B-CNN and other baseline CNN models. Moreover, Condition-CNN used only 77.58% of the total number of trainable parameters as that of B-CNN.

Cognitive structure learning model for hierarchical multi-label text classification

The human mind grows in learning new knowledge, which finally organizes and develops a basic mental pattern called cognitive structure. Hierarchical multi-label text classification (HMLTC), a fundamental but challenging task in many real-world applications, aims to classify the documents with hierarchical labels to form a resembling cognitive structure learning process. Existing approaches for HMLTC mainly focus on partial new knowledge learning or the global cognitive-structure-like label structure utilization in a cognitive view. However, the complete cognitive structure learning model is a unity that is indispensably constructed by the global label structure utilization and partial knowledge learning, which is ignored among those HMLTC approaches. To address this problem, we will imitate the cognitive structure learning process into the HMLTC learning and propose a unified framework called Hierarchical Cognitive Structure Learning Model (HCSM) in this paper. HCSM is composed of the Attentional Ordered Recurrent Neural Network (AORNN) submodule and Hierarchical Bi-Directional Capsule (HBiCaps) submodule. Both submodules utilize the partial new knowledge and global hierarchical label structure comprehensively for the HMLTC task. On the one hand, AORNN extracts the semantic vector as partial new knowledge from the original text by the word-level and hierarchy-level embedding granularities. On the other hand, AORNN builds the hierarchical text representation learning corresponding to the global label structure by the document-level neurons ordering. HBiCaps employs an iteration to form a unified label categorization process similar to cognitive-structure learning: firstly, using the probability computation of local hierarchical relationships to maintain partial knowledge learning; secondly, modifying the global hierarchical label structure based on the dynamic routing mechanism between capsules. Moreover, the experimental results on four benchmark datasets demonstrate that HCSM outperforms or matches state-of-the-art text classification methods.

DeepciRGO: functional prediction of circular RNAs through hierarchical deep neural networks using heterogeneous network features

BackgroundCircular RNAs (circRNAs) are special noncoding RNA molecules with closed loop structures. Compared with the traditional linear RNA, circRNA is more stable and not easily degraded. Many studies have shown that circRNAs are involved in the regulation of various diseases and cancers. Determining the functions of circRNAs in mammalian cells is of great significance for revealing their mechanism of action in physiological and pathological processes, diagnosis and treatment of diseases. However, determining the functions of circRNAs on a large scale is a challenging task because of the high experimental costs.ResultsIn this paper, we present a hierarchical deep learning model, DeepciRGO, which can effectively predict gene ontology functions of circRNAs. We build a heterogeneous network containing circRNA co-expressions, protein–protein interactions and protein–circRNA interactions. The topology features of proteins and circRNAs are calculated using a novel representation learning approach HIN2Vec across the heterogeneous network. Then, a deep multi-label hierarchical classification model is trained with the topology features to predict the biological process function in the gene ontology for each circRNA. In particular, we manually curated a benchmark dataset containing 185 GO annotations for 62 circRNAs, namely, circRNA2GO-62. The DeepciRGO achieves promising performance on the circRNA2GO-62 dataset with a maximum F-measure of 0.412, a recall score of 0.400, and an accuracy of 0.425, which are significantly better than other state-of-the-art RNA function prediction methods. In addition, we demonstrate the considerable potential of integrating multiple interactions and association networks.ConclusionsDeepciRGO will be a useful tool for accurately annotating circRNAs. The experimental results show that integrating multi-source data can help to improve the predictive performance of DeepciRGO. Moreover, The model also can combine RNA structure and sequence information to further optimize predictive performance.

Effects of the hierarchy in hierarchical, multi-label classification

The consequences of the choice of a hierarchy in hierarchical multi-label classification (HMLC) have previously not been considered in any detail. Three hierarchy-related factors in HMLC are examined here: hierarchy structure, class location in the hierarchy, and class distribution in feature space. Four general model groups are found to exist in HMLC modeling: "non-informative”, "semi-informative”, "comparable”, and "hierarchical”. Studies of synthetic and real data show that the choice of hierarchy used in the modeling is important in setting the relative error rates of false positives and false negatives. The choice of hierarchy depends upon the relative consequences of false positive and false negative errors produced by the resulting model. A low false negative error rate results from use of a "comparable” HMLC model with a hierarchy designed to maximize intergroup separation. A low false positive error rate results from use of a "hierarchical” HMLC model using any hierarchy. Modest differences in accuracy and F1 measure occur between the best-performing HMLC models built on several external and internal hierarchies for a complex, multiclass dataset. HMLC methods using "comparable” and "hierarchical” HMLC models and phylogenetic hierarchies examined slightly outperform a conventional classification using the same classifier on the Dalbergia data. BriefStudies of synthetic and real data show that the structure of a multi-label hierarchy in hierarchical, multi-label classification (HMLC) is important in setting the relative error rate of false positives and false negatives. A low false negative error rate results from use of a "comparable” HMLC model with a hierarchy designed to maximize inter-class separation. A low false positive error rate results from use of a "hierarchical” HMLC model with any hierarchy.

HMATC: Hierarchical multi-label Arabic text classification model using machine learning

Multi-label classification assigns multiple labels to each document concurrently. Many real-world classification problems tend to employ high-dimensional label spaces, which can be naturally structured in a hierarchy. In this type of problem, each instance may belong to multiple labels and labels are organized in a hierarchical structure. It presents a more complex problem than flat classification, given that the classification algorithm has to take into account hierarchical relationships between labels and be able to predict multiple labels for the same instance. Few studies have investigated multi-label text classification for the Arabic language. Most of these studies have focused mainly on flat classification and have neglected the hierarchical structure. Therefore, this paper explores the hierarchical multi-label classification in the context of the Arabic language. It proposes a hierarchical multi-label Arabic text classification (HMATC) model with a machine learning approach. The impact of feature selection methods and feature set dimensions on classification performance are also investigated. In addition, the Hierarchy Of Multilabel ClassifiER (HOMER) algorithm is optimized via examination of different sets of multi-label classifiers, clustering algorithms and different numbers of clusters to improve the hierarchical classification. Moreover, this study contributes to existing research by introducing a hierarchical multi-label Arabic dataset in an appropriate format for hierarchical classification and making it publicly available. The results reveal that the proposed model outperforms all models considered in the experiments in terms of the computational cost, which consumed less cost (2 h) compared with other evaluated models. In addition, it shows a significant improvement compared with the state-of-the-art model (Fatwa model) in terms of Hamming loss (0.004), hierarchical loss (1.723), multi-label accuracy (0.758), subset accuracy (0.292), micro-averaged precision (0.879), micro-averaged recall (0.828), and micro-averaged F-measure (0.853).

Deep hiearchical multi-label classification applied to chest X-ray abnormality taxonomies.

Chest X-rays (CXRs) are a crucial and extraordinarily common diagnostic tool, leading to heavy research for computer-aided diagnosis (CAD) solutions. However, both high classification accuracy and meaningful model predictions that respect and incorporate clinical taxonomies are crucial for CAD usability. To this end, we present a deep hierarchical multi-label classification (HMLC) approach for CXR CAD. Different than other hierarchical systems, we show that first training the network to model conditional probability directly and then refining it with unconditional probabilities is key in boosting performance. In addition, we also formulate a numerically stable cross-entropy loss function for unconditional probabilities that provides concrete performance improvements. Finally, we demonstrate that HMLC can be an effective means to manage missing or incomplete labels. To the best of our knowledge, we are the first to apply HMLC to medical imaging CAD. We extensively evaluate our approach on detecting abnormality labels from the CXR arm of the Prostate, Lung, Colorectal and Ovarian (PLCO) dataset, which comprises over 198,000 manually annotated CXRs. When using complete labels, we report a mean area under the curve (AUC) of 0.887, the highest yet reported for this dataset. These results are supported by ancillary experiments on the PadChest dataset, where we also report significant improvements, 1.2% and 4.1% in AUC and average precision, respectively over strong "flat" classifiers. Finally, we demonstrate that our HMLC approach can much better handle incompletely labelled data. These performance improvements, combined with the inherent usefulness of taxonomic predictions, indicate that our approach represents a useful step forward for CXR CAD.

Hierarchical Multi-Label Object Detection Framework for Remote Sensing Images

Detecting objects such as aircraft and ships is a fundamental research area in remote sensing analytics. Owing to the prosperity and development of CNNs, many previous methodologies have been proposed for object detection within remote sensing images. Despite the advance, using the object detection datasets with a more complex structure, i.e., datasets with hierarchically multi-labeled objects, is limited to the existing detection models. Especially in remote sensing images, since objects are obtained from bird’s-eye view, the objects are captured with restricted visual features and not always guaranteed to be labeled up to fine categories. We propose a hierarchical multi-label object detection framework applicable to hierarchically partial-annotated datasets. In the framework, an object detection pipeline called Decoupled Hierarchical Classification Refinement (DHCR) fuses the results of two networks: (1) an object detection network with multiple classifiers, and (2) a hierarchical sibling classification network for supporting hierarchical multi-label classification. Our framework additionally introduces a region proposal method for efficient detection on vain areas of the remote sensing images, called clustering-guided cropping strategy. Thorough experiments validate the effectiveness of our framework on our own object detection datasets constructed with remote sensing images from WorldView-3 and SkySat satellites. Under our proposed framework, DHCR-based detections significantly improve the performance of respective baseline models and we achieve state-of-the-art results on the datasets.

Ensembles of extremely randomized predictive clustering trees for predicting structured outputs

We address the task of learning ensembles of predictive models for structured output prediction (SOP). We focus on three SOP tasks: multi-target regression (MTR), multi-label classification (MLC) and hierarchical multi-label classification (HMC). In contrast to standard classification and regression, where the output is a single (discrete or continuous) variable, in SOP the output is a data structure—a tuple of continuous variables MTR, a tuple of binary variables MLC or a tuple of binary variables with hierarchical dependencies (HMC). SOP is gaining increasing interest in the research community due to its applicability in a variety of practically relevant domains. In this context, we consider the Extra-Tree ensemble learning method—the overall top performer in the DREAM4 and DREAM5 challenges for gene network reconstruction. We extend this method for SOP tasks and call the extension Extra-PCTs ensembles. As base predictive models we propose using predictive clustering trees (PCTs)–a generalization of decision trees for predicting structured outputs. We conduct a comprehensive experimental evaluation of the proposed method on a collection of 41 benchmark datasets: 21 for MTR, 10 for MLC and 10 for HMC. We first investigate the influence of the size of the ensemble and the size of the feature subset considered at each node. We then compare the performance of Extra-PCTs to other ensemble methods (random forests and bagging), as well as to single PCTs. The experimental evaluation reveals that the Extra-PCTs achieve optimal performance in terms of predictive power and computational cost, with 50 base predictive models across the three tasks. The recommended values for feature subset sizes vary across the tasks, and also depend on whether the dataset contains only binary and/or sparse attributes. The Extra-PCTs give better predictive performance than a single tree (the differences are typically statistically significant). Moreover, the Extra-PCTs are the best performing ensemble method (except for the MLC task, where performances are similar to those of random forests), and Extra-PCTs can be used to learn good feature rankings for all of the tasks considered here.

Active learning for hierarchical multi-label classification

Due to technological advances, a massive amount of data is produced daily, presenting challenges for application areas where data needs to be labelled by a domain specialist or by expensive procedures, in order to be useful for supervised machine learning purposes. In order to select which data points will provide more information when labelled, one can make use of active learning methods. Active learning (AL) is a subfield of machine learning which addresses methods to build models with fewer, but more representative instances. Even though AL has been vastly studied, it has not been thoroughly investigated in hierarchical multi-label classification, a learning task where multiple class labels can be assigned to an instance and these labels are hierarchically structured. In this work, we provide a public framework containing baseline and state-of-the-art algorithms suitable for this task. Additionally, we also propose a new algorithm, namely Hierarchical Query-By-Committee (H-QBC), which is validated on datasets from different domains. Our results show that H-QBC is capable of providing superior predictive performance results compared to its competitors, while being computationally efficient and parameter free.

Hierarchical multilabel classifier for gene ontology annotation using multihead and multiend deep CNN model

AbstractGene ontology annotation is known to be a very complicated multilabel classification task, and the hierarchical multilabel classification (HMC) approaches with local classifiers have been shown to be effective for the task. In a traditional HMC method, a set of hierarchically organized simple local classifiers are usually used, each of which for one hierarchical level separately. In this paper, we propose a novel hierarchical multilabel classifier implementing the whole set of hierarchically organized local classifiers in one deep convolution neural network (CNN) model with multiple heads and multiple ends (MHME). The proposed MHME CNN model consists of three parts: the body part of a deep CNN model shared by different local classifiers for feature extraction and feature mapping; the multiend part of a set of autoencoders performing feature fusion transforming the input vectors of different local classifiers to feature vectors with the same length so as to share the feature mapping part; and the multihead part of a set of linear multilabel classifiers. Furthermore, a sophisticated recursive algorithm is designed to train the MHME CNN model to realize the functions of a set of hierarchically organized local classifiers. In this way, by sharing a deep CNN with multiple local classifiers, we are able to construct more powerful local classifiers for each level with limited training samples, and to achieve better classification performance. Experiment results on various benchmark datasets show that the proposed deep CNN‐based model has better performance than the state‐of‐the‐art traditional models. Moreover, it gives rather good performance even under a transfer learning. © 2020 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.