Hierarchical Multi-label Classification Research Articles

A consistency-aware deep capsule network for hierarchical multi-label image classification

Hierarchical classification is a significant challenge in computer vision due to the logical order and interconnectedness of multiple labels. This paper presents HD-CapsNet, a novel neural network architecture based on deep capsule networks, specifically designed for hierarchical multi-label classification(HMC). By incorporating a tree-like hierarchical structure, HD-CapsNet is designed to leverage the inherent ontological order within the hierarchical label tree, thereby ensuring classification consistency across different levels. Additionally, we introduce a specialized loss function that promotes accurate hierarchical relationships while penalizing inconsistencies. This not only enhances classification performance but also strengthens the network’s robustness. We rigorously evaluate HD-CapsNet’s efficacy by benchmarking it against existing HMC methods across six diverse datasets: Fashion-MNIST, Marine-Tree, CIFAR-10, CIFAR-100, Caltech-UCSD Birds-200-2011, and Stanford Cars. Our results conclusively demonstrate that HD-CapsNet excels in learning hierarchical relationships and significantly outperforms the competition in various image classification tasks. Our implementation is available at https://github.com/tasrif-khondaker/HD-CapsNet.

Towards ultrasonic guided wave fine-grained damage detection on hierarchical multi-label classification network

This study advocates the hierarchical multi-label classification (HMC) network for fine-grained damage detection using ultrasonic guided wave. Existing deep learning methods only focus on one aspect of the damage type or the degree of damage progression, and both are important for the long-term safe operation of the structure. To address this limitation, an ultrasonic guided wave HMC network (GHmcNet) and its lightweight version (L-GHmcNet) are developed. The essential designs of the proposed methods are derived from two motivations: (1) achieving robust fine-grained damage classification; (2) designing a lightweight network. The hierarchical labels (defect type-depth-size) are constructed on guided wave data. Inspired by the work of image classification, residual connections are introduced to transfer and fuse features between parent and children categories. The GHmcNet has three output channels and a weighted cross-entropy loss function is imposed on each channel. The weights of the corresponding loss are automatically adjusted during training. To reduce the parameters, we improve the structure of GHmcNet by applying different numbers of convolutional layers and channels at different levels based on the number of classes in each level, proposing the lightweight model L-GHmcNet. Both numerical and experimental studies are carried out, in which the dataset contains signals of different damage types, depths and sizes. The results show the superiority of the proposed methods in terms of accuracy at three levels compared to state-of-the-art methods. Moreover, the high accuracy classification results of L-GHmcNet demonstrate that the proposed lightweight network is a successful attempt.

Semi-Supervised Predictive Clustering Trees for (Hierarchical) Multi-Label Classification

Semi-supervised learning (SSL) is a common approach to learning predictive models using not only labeled, but also unlabeled examples. While SSL for the simple tasks of classification and regression has received much attention from the research community, this is not the case for complex prediction tasks with structurally dependent variables, such as multi-label classification and hierarchical multi-label classification. These tasks may require additional information, possibly coming from the underlying distribution in the descriptive space provided by unlabeled examples, to better face the challenging task of simultaneously predicting multiple class labels. In this paper, we investigate this aspect and propose a (hierarchical) multi-label classification method based on semi-supervised learning of predictive clustering trees, which we also extend towards ensemble learning. Extensive experimental evaluation conducted on 24 datasets shows significant advantages of the proposed method and its extension with respect to their supervised counterparts. Moreover, the method preserves interpretability of classical tree-based models.

Self-Paced Unified Representation Learning for Hierarchical Multi-Label Classification

Hierarchical Multi-Label Classification (HMLC) is a well-established problem that aims at assigning data instances to multiple classes stored in a hierarchical structure. Despite its importance, existing approaches often face two key limitations: (i) They employ dense networks to solely explore the class hierarchy as hard criterion for maintaining taxonomic consistency among predicted classes, yet without leveraging rich semantic relationships between instances and classes; (ii) They struggle to generalize in settings with deep class levels, since the mini-batches uniformly sampled from different levels ignore the varying complexities of data and result in a non-smooth model adaptation to sparse data. To mitigate these issues, we present a Self-Paced Unified Representation (SPUR) learning framework, which focuses on the interplay between instance and classes to flexibly organize the training process of HMLC algorithms. Our framework consists of two lightweight encoders designed to capture the semantics of input features and the topological information of the class hierarchy. These encoders generate unified embeddings of instances and class hierarchy, which enable SPUR to exploit semantic dependencies between them and produce predictions in line with taxonomic constraints. Furthermore, we introduce a dynamic hardness measurement strategy that considers both class hierarchy and instance features to estimate the learning difficulty of each instance. This strategy is achieved by incorporating the propagation loss obtained at each hierarchical level, allowing for a more comprehensive assessment of learning complexity. Extensive experiments on several empirical benchmarks demonstrate the effectiveness and efficiency of SPUR compared to state-of-the-art methods, especially in scenarios with missing features.

Dimensionality Reduction for Hierarchical Multi-Label Classification: A Systematic Mapping Study

Hierarchical multi-label classification problems typically deal with datasets with many attributes and labels, which can negatively impact the classifier performance. The application of dimensionality reduction methods can significantly improve the performance of classifiers. Dimensionality reduction can be performed by feature extraction or feature selection, according to the problem domain and datasets characteristics. This work carried out a systematic literature mapping to identify the approaches and techniques of dimensionality reduction that have been used in hierarchical multi-label classification tasks. Searches were performed on 7 important databases for the Computer Science field. From a list of 184 retrieved papers, 12 were selected for analysis, from which it was possible to determine a general overview of studies conducted from 2010 to 2022. It was identified that feature selection was the most frequent reduction method, with filter approach standing out. In addition, it was detected that most of the works used tree hierarchical structure. As its main outcome, this paper presents the state of the art of dimensionality reduction problem for hierarchical multi-label classification, indicating trends and research issues in the field.

Editorial

Editorial

Graph Neural Network Based Multi-Label Hierarchical Classification for Disease Predictions in General Practice.

General practitioners are supposed to be better diagnostics to detect patients with serious diseases earlier, and conduct early interventions and appropriate referrals of patients. However, in the current general practice, primary general practitioners lack sufficient clinical experiences, and the correct rate of general disease diagnosis is low. To assist general practitioners in diagnosis, this paper proposes a multi-label hierarchical classification method based on graph neural network, which integrates medical knowledge and electronic health record (EHR) data to build a disease prediction model. The experimental results based on data consist of 231,783 visits from EHR show that the proposed model outperforms all baseline models in the general disease prediction task with a top-3 recall of 0.865. The interpretable results of the model can effectively help clinicians understand the basis of the model's decision-making.

A new sentence embedding framework for the education and professional training domain with application to hierarchical multi-label text classification

In recent years, Natural Language Processing (NLP) has made significant advances through advanced general language embeddings, allowing breakthroughs in NLP tasks such as semantic similarity and text classification. However, complexity increases with hierarchical multi-label classification (HMC), where a single entity can belong to several hierarchically organized classes. In such complex situations, applied on specific-domain texts, such as the Education and professional training domain, general language embedding models often inadequately represent the unique terminologies and contextual nuances of a specialized domain. To tackle this problem, we present HMCCCProbT, a novel hierarchical multi-label text classification approach. This innovative framework chains multiple classifiers, where each individual classifier is built using a novel sentence-embedding method BERTEPro based on existing Transformer models, whose pre-training has been extended on education and professional training texts, before being fine-tuned on several NLP tasks. Each individual classifier is responsible for the predictions of a given hierarchical level and propagates local probability predictions augmented with the input feature vectors to the classifier in charge of the subsequent level. HMCCCProbT tackles issues of model scalability and semantic interpretation, offering a powerful solution to the challenges of domain-specific hierarchical multi-label classification. Experiments over three domain-specific textual HMC datasets indicate the effectiveness of HMCCCProbT to compare favorably to state-of-the-art HMC algorithms in terms of classification accuracy and also the ability of BERTEPro to obtain better probability predictions, well suited to HMCCCProbT, than three other vector representation techniques.

Interpretable Detection of Diabetic Retinopathy, Retinal Vein Occlusion, Age-Related Macular Degeneration, and Other Fundus Conditions.

Diabetic retinopathy (DR), retinal vein occlusion (RVO), and age-related macular degeneration (AMD) pose significant global health challenges, often resulting in vision impairment and blindness. Automatic detection of these conditions is crucial, particularly in underserved rural areas with limited access to ophthalmic services. Despite remarkable advancements in artificial intelligence, especially convolutional neural networks (CNNs), their complexity can make interpretation difficult. In this study, we curated a dataset consisting of 15,089 color fundus photographs (CFPs) obtained from 8110 patients who underwent fundus fluorescein angiography (FFA) examination. The primary objective was to construct integrated models that merge CNNs with an attention mechanism. These models were designed for a hierarchical multilabel classification task, focusing on the detection of DR, RVO, AMD, and other fundus conditions. Furthermore, our approach extended to the detailed classification of DR, RVO, and AMD according to their respective subclasses. We employed a methodology that entails the translation of diagnostic information obtained from FFA results into CFPs. Our investigation focused on evaluating the models' ability to achieve precise diagnoses solely based on CFPs. Remarkably, our models showcased improvements across diverse fundus conditions, with the ConvNeXt-base + attention model standing out for its exceptional performance. The ConvNeXt-base + attention model achieved remarkable metrics, including an area under the receiver operating characteristic curve (AUC) of 0.943, a referable F1 score of 0.870, and a Cohen's kappa of 0.778 for DR detection. For RVO, it attained an AUC of 0.960, a referable F1 score of 0.854, and a Cohen's kappa of 0.819. Furthermore, in AMD detection, the model achieved an AUC of 0.959, an F1 score of 0.727, and a Cohen's kappa of 0.686. Impressively, the model demonstrated proficiency in subclassifying RVO and AMD, showcasing commendable sensitivity and specificity. Moreover, our models enhanced interpretability by visualizing attention weights on fundus images, aiding in the identification of disease findings. These outcomes underscore the substantial impact of our models in advancing the detection of DR, RVO, and AMD, offering the potential for improved patient outcomes and positively influencing the healthcare landscape.

ProFun-SOM: Protein Function Prediction for Specific Ontology Based on Multiple Sequence Alignment Reconstruction.

Protein function prediction is crucial for understanding species evolution, including viral mutations. Gene ontology (GO) is a standardized representation framework for describing protein functions with annotated terms. Each ontology is a specific functional category containing multiple child ontologies, and the relationships of parent and child ontologies create a directed acyclic graph. Protein functions are categorized using GO, which divides them into three main groups: cellular component ontology, molecular function ontology, and biological process ontology. Therefore, the GO annotation of protein is a hierarchical multilabel classification problem. This hierarchical relationship introduces complexities such as mixed ontology problem, leading to performance bottlenecks in existing computational methods due to label dependency and data sparsity. To overcome bottleneck issues brought by mixed ontology problem, we propose ProFun-SOM, an innovative multilabel classifier that utilizes multiple sequence alignments (MSAs) to accurately annotate gene ontologies. ProFun-SOM enhances the initial MSAs through a reconstruction process and integrates them into a deep learning architecture. It then predicts annotations within the cellular component, molecular function, biological process, and mixed ontologies. Our evaluation results on three datasets (CAFA3, SwissProt, and NetGO2) demonstrate that ProFun-SOM surpasses state-of-the-art methods. This study confirmed that utilizing MSAs of proteins can effectively overcome the two main bottlenecks issues, label dependency and data sparsity, thereby alleviating the root problem, mixed ontology. A freely accessible web server is available at http://bliulab.net/ ProFun-SOM/.

HmcNet: A General Approach for Hierarchical Multi-Label Classification

Hierarchical multi-label classification (HMC) deals with the problem of assigning each entity to multiple classes with a taxonomic structure (e.g., tree). Within this structure, classes at different levels tend to have dependencies under the hierarchy constraints. However, most prior studies for HMC tasks tend to ignore the class dependencies within the hierarchy. Moreover, most existing methods generate incoherent predictions and do not satisfy the hierarchy constraint. To this end, based on previously developed HARNN, we propose a general framework, HmcNet, for introducing explicit and implicit class hierarchy constraints to generate coherent predictions. We develop an efficient Prune-based Coherent Prediction (PCP) strategy for the optimal paths selection, which produces coherent predictions in a principled way. HmcNet can be well explained from two perspectives. First, it develops the Hierarchical Attention-based Memory (HAM) unit with implicit class hierarchy constraints to capture class dependencies more intuitively; Second, it subsumes explicit class hierarchy constraints during training and inference phases and generates coherent predictions in a consistent manner. Finally, extensive experimental results on six real-world datasets demonstrate the effectiveness and interpretability of the HmcNet frameworks. To facilitate future research, our code has been made publicly available.

Multi-Label Classification of Chinese Rural Poverty Governance Texts Based on XLNet and Bi-LSTM Fused Hierarchical Attention Mechanism

Hierarchical multi-label text classification (HMTC) is a highly relevant and widely discussed topic in the era of big data, particularly for efficiently classifying extensive amounts of text data. This study proposes the HTMC-PGT framework for poverty governance’s single-path hierarchical multi-label classification problem. The framework simplifies the HMTC problem into training and combination problems of multi-class classifiers in the classifier tree. Each independent classifier in this framework uses an XLNet pretrained model to extract char-level semantic embeddings of text and employs a hierarchical attention mechanism integrated with Bi-LSTM (BiLSTM + HA) to extract semantic embeddings at the document level for classification purposes. Simultaneously, this study proposes that the structure uses transfer learning (TL) between classifiers in the classifier tree. The experimental results show that the proposed XLNet + BiLSTM + HA + FC + TL model achieves micro-P, micro-R, and micro-F1 values of 96.1%, which is 7.5~38.1% higher than those of other baseline models. The HTMC-PGT framework based on XLNet, BiLSTM + HA, and transfer learning (TL) between classifier tree nodes proposed in this study solves the hierarchical multi-label classification problem of poverty governance text (PGT). It provides a new idea for solving the traditional HMTC problem.

CEHMR: Curriculum learning enhanced hierarchical multi-label classification for medication recommendation

The medication recommendation (MR) or medication combination prediction task aims to predict effective prescriptions given accurate patient representations derived from electronic health records (EHRs), which contributes to improving the quality of clinical decision-making, especially for patients with multi-morbidity. Although in recent years deep learning technology has achieved great success in MR, the performance of current multi-label based MR solutions is unsatisfactory. They mainly focus on improving the patient representation module and modeling the medication label dependencies such as drug–drug interaction (DDI) correlation and co-occurrence relationship. However, the hierarchical dependency among medication labels and diversity of difficulty among MR training examples lack sufficient consideration. In this paper, we propose a framework of Curriculum learning Enhanced Hierarchical multi-label classification for MR (CEHMR). Motivated by the category hierarchy of medications which organizes standard medication codes in a hierarchical structure, we utilize it to provide more trustworthy prior knowledge for modeling label dependency. Specifically, we design a hierarchical multi-label classifier with a learnable gate fusion layer, to simultaneously capture the level-independent (local) and level-dependent (global) hierarchical information in the medication hierarchy. In addition, to overcome the diversity of training example difficulties, and progressively achieve a smoother training process, we introduce a bootstrap-based curriculum learning strategy. Hence, the example difficulty can be measured based on the predictive performance of the MR model, and then all training examples would be retrained from easy to hard under the guidance of a predefined training scheduler. Experiments on the real-world medical MIMIC-III database demonstrate that the proposed framework can achieve state-of-the-art performance compared with seven representative baselines, and extensive ablation studies validate the effectiveness of each component of CEHMR.

Simultaneous Fault Diagnosis Based on Hierarchical Multi-Label Classification and Sparse Bayesian Extreme Learning Machine

This paper proposes an intelligent simultaneous fault diagnosis model based on a hierarchical multi-label classification strategy and sparse Bayesian extreme learning machine. The intelligent diagnosis model compares the similarity between an unknown sample to be diagnosed and each single fault mode, then outputs the probability of each fault mode occurring. First, multiple two-class sub-classifiers based on SBELM are trained by using single-fault samples to extract the correlation between various pairs of single-fault, and the sub-classifiers are integrated with the proposed hierarchical multi-label classification (HMLC) strategy to form the diagnostic model based on HMLC-SBELM. Then, samples of single faults and simultaneous faults are used to generate the optimal discriminative thresholds by using optimization algorithms. Finally, the probabilistic output generated by the HMLC-SBELM-based model is transformed into the final fault modes by using the optimal discriminative threshold. The model performance is evaluated by using actual vibration signals of the main reducer and is compared with several classical models. The contrastive results indicate that the proposed model is more accurate, efficient, and stable.

Leveraging class hierarchy for detecting missing annotations on hierarchical multi-label classification

With the development of new sequencing technologies, availability of genomic data has grown exponentially. Over the past decade, numerous studies have used genomic data to identify associations between genes and biological functions. While these studies have shown success in annotating genes with functions, they often assume that genes are completely annotated and fail to take into account that datasets are sparse and noisy. This work proposes a method to detect missing annotations in the context of hierarchical multi-label classification. More precisely, our method exploits the relations of functions, represented as a hierarchy, by computing probabilities based on the paths of functions in the hierarchy. By performing several experiments on a variety of rice (Oriza sativa Japonica), we showcase that the proposed method accurately detects missing annotations and yields superior results when compared to state-of-art methods from the literature.

Hierarchical multi-label taxonomic classification of carbonate skeletal grains with deep learning

Recent advances in Artificial Intelligence (AI), particularly the rise of deep learning, are revolutionizing data collection and analysis in many aspects of the Earth Sciences, including paleontology. Rapid digital transformation of paleontological information enables automation of various paleontological tasks using AI technology. One such task is identifying and classifying skeletal grains at different levels of the Linnaean taxonomic hierarchy, both macro- and microscopically. Unfortunately, fossil classification remains largely untouched by AI due to the expertise and time required to generate high-quality, large-scale labeled training data. This task is particularly challenging when part of the compositional analysis of limestone, where three-dimensional objects (i.e., skeletal grains) are observed in a two-dimensional cross-section. It is compounded by the imbalanced data and limited number of images available, approximately four orders smaller than the number necessary to train deep neural networks. Recent efforts to classify such fossil images using deep learning returned a single output/label classification, limiting the information output from the model, and hindering its further applications to replicate paleontologist-level identification. Here, we couple a multi-head deep convolutional neural network architecture called TaxonNet that adopts the Branch Convolutional Neural Networks and advanced image augmentation to overcome these barriers. This model uses prior knowledge of hierarchical category relationships to output multiple hierarchical taxonomic predictions from a single petrographic image. In this study, we developed and evaluated four different strategies of hierarchical classification: (i) forward (coarse to fine); (ii) reverse (fine to coarse); (iii) parallel (coarse and fine simultaneously), and (iv) hybrid. The forward and reverse strategies yield the most accurate prediction of ~95 % accuracy for coarse-level and fine-level recognitions across major fossil types in carbonate petrographic images, including algae, mollusks, and corals. These results represent an important step toward applying AI to advance fossil grain classification and exhibit a promising real-world geological application of a hierarchical network in paleontology, sedimentary petrography, and other image-based geological analysis requiring hierarchical, multilabel output.

Feature ranking for semi-supervised learning

The data used for analysis are becoming increasingly complex along several directions: high dimensionality, number of examples and availability of labels for the examples. This poses a variety of challenges for the existing machine learning methods, related to analyzing datasets with a large number of examples that are described in a high-dimensional space, where not all examples have labels provided. For example, when investigating the toxicity of chemical compounds, there are many compounds available that can be described with information-rich high-dimensional representations, but not all of the compounds have information on their toxicity. To address these challenges, we propose methods for semi-supervised learning (SSL) of feature rankings. The feature rankings are learned in the context of classification and regression, as well as in the context of structured output prediction (multi-label classification, MLC, hierarchical multi-label classification, HMLC and multi-target regression, MTR) tasks. This is the first work that treats the task of feature ranking uniformly across various tasks of semi-supervised structured output prediction. To the best of our knowledge, it is also the first work on SSL of feature rankings for the tasks of HMLC and MTR. More specifically, we propose two approaches—based on predictive clustering tree ensembles and the Relief family of algorithms—and evaluate their performance across 38 benchmark datasets. The extensive evaluation reveals that rankings based on Random Forest ensembles perform the best for classification tasks (incl. MLC and HMLC tasks) and are the fastest for all tasks, while ensembles based on extremely randomized trees work best for the regression tasks. Semi-supervised feature rankings outperform their supervised counterparts across the majority of datasets for all of the different tasks, showing the benefit of using unlabeled in addition to labeled data.

Feature extraction with spectral clustering for gene function prediction using hierarchical multi-label classification

Gene annotation addresses the problem of predicting unknown associations between gene and functions (e.g., biological processes) of a specific organism. Despite recent advances, the cost and time demanded by annotation procedures that rely largely on in vivo biological experiments remain prohibitively high. This paper presents a novel in silico approach for to the annotation problem that combines cluster analysis and hierarchical multi-label classification (HMC). The approach uses spectral clustering to extract new features from the gene co-expression network (GCN) and enrich the prediction task. HMC is used to build multiple estimators that consider the hierarchical structure of gene functions. The proposed approach is applied to a case study on Zea mays, one of the most dominant and productive crops in the world. The results illustrate how in silico approaches are key to reduce the time and costs of gene annotation. More specifically, they highlight the importance of: (1) building new features that represent the structure of gene relationships in GCNs to annotate genes; and (2) taking into account the structure of biological processes to obtain consistent predictions.

Hierarchical multi-label classification based on LSTM network and Bayesian decision theory for LncRNA function prediction

Growing evidence shows that long noncoding RNAs (lncRNAs) play an important role in cellular biological processes at multiple levels, such as gene imprinting, immune response, and genetic regulation, and are closely related to diseases because of their complex and precise control. However, most functions of lncRNAs remain undiscovered. Current computational methods for exploring lncRNA functions can avoid high-throughput experiments, but they usually focus on the construction of similarity networks and ignore the certain directed acyclic graph (DAG) formed by gene ontology annotations. In this paper, we view the function annotation work as a hierarchical multilabel classification problem and design a method HLSTMBD for classification with DAG-structured labels. With the help of a mathematical model based on Bayesian decision theory, the HLSTMBD algorithm is implemented with the long-short term memory network and a hierarchical constraint method DAGLabel. Compared with other state-of-the-art algorithms, the results on GOA-lncRNA datasets show that the proposed method can efficiently and accurately complete the label prediction work.

OWGC-HMC: An Online Web Genre Classification Model Based on Hierarchical Multilabel Classification

Web genre plays an important role in focused crawling, web link analysis, and contextual advertising. In this paper, web genre is defined as the functional purpose and the information type contained in the website. The intelligent classification of web genre can predict the content and functional type of website. However, there are several critical challenges to solve the web genre classification problem: lack of web genre classification dataset and efficient web genre classification mechanism. To improve web genre classification performance, we crawled Chinese websites of different web genres and converted crawled data into a hierarchical multilabel classification dataset. A website knowledge graph is constructed based on the relationship of website and meta tag features. Using entity features extracted from the knowledge graph, we propose an online web genre classification model based on hierarchical multilabel classification (OWGC-HMC) to mine the functional purpose of the corresponding website. Experimental results show that our OWGC-HMC model can mine hierarchical multilabel structure of web genre and outperform other web genre classification methods.