Hierarchical Text Research Articles

HGBL: A Fine Granular Hierarchical Multi-Label Text Classification Model

Hierarchical multi-label text classification is vital for natural language processing (NLP). However, existing research rarely makes full use of the interaction between labels and text features that are crucial to hierarchical multi-label text classification. To address this issue, a novel model named hierarchy-guided BiLSTM guided contrastive learning classification (HGBL) is proposed, which successfully enhances the interaction between labels and text features by incorporating global context and embedding the idea of contrastive learning into this model. During modeling, Graphormer is adopted to model the dependencies between labels, and the bidirectional recurrent network (BiLSTM) is used to integrate global context including label features. Afterwards, the contrastive learning module embeds hierarchical awareness into the fine-tuned bidirectional encoder representations from transformers (BERT) by training the value of the loss. Experimental results on NYT, WOS and RCV1-V2 datasets show that HGBL exhibits significant competitive advantages compared with 19 competitors in terms of several indicators and can be used effectively for hierarchical multi-label text classification problems.

Mutually-Guided Hierarchical Multi-Modal Feature Learning for Referring Image Segmentation

Referring image segmentation aims to locate and segment the target region based on a given textual expression query. The primary challenge is to understand semantics from visual and textual modalities and achieve alignment and matching. Prior works have attempted to address this challenge by leveraging separately pretrained unimodal models to extract global visual and textual features and perform straightforward fusion to establish cross-modal semantic associations. However, these methods often concentrate solely on the global semantics, disregarding the hierarchical semantics of expression and image and struggling with complex and open real scenarios, thus failing to capture critical cross-modal information. To address these limitations, this article introduces an innovative mutually-guided hierarchical multi-modal feature learning scheme. By leveraging the guidance of global visual features, the model mines hierarchical text features from different stages of the text encoder. Simultaneously, the guidance of global textual features is leveraged to aggregate multi-scale visual features. This mutually guided hierarchical feature learning effectively addresses the semantically inaccurate cause by free-form text and naturally occurring scale variations. Furthermore, a Segment Detail Refinement (SDR) module is designed to enhance the model’s spatial detail awareness through attention mapping of low-level visual features and cross-modal features. To evaluate the effectiveness of the proposed approach, extensive experiments are conducted on three widely used referring image object segmentation datasets. The results demonstrate the superiority of the presented method in accurately locating and segmenting objects in images.

Hierarchical multi-label text classification of tourism resources using a label-aware dual graph attention network

In the era of big data, classifying online tourism resource information can facilitate the matching of user needs with tourism resources and enhance the efficiency of tourism resource integration. However, most research in this field has concentrated on a simple classification problem with a single level of single labelling. In this paper, a Hierarchical Label-Aware Tourism-Informed Dual Graph Attention Network (HLT-DGAT) is proposed for the complex multi-level and multi-label classification presented by online textual information about Chinese tourism resources. This model integrates domain knowledge into a pre-trained language model and employs attention mechanisms to transform the text representation into the label-based representation. Subsequently, the model utilizes dual Graph Attention Network (GAT), with one component capturing vertical information and the other capturing horizontal information within the label hierarchy. The model's performance is validated on two commonly used public datasets as well as on a manually curated Chinese tourism resource dataset, which consists of online textual overviews of Chinese tourism resources above 3A level. Experimental results indicate that HLT-DGAT demonstrates superiority in threshold-based and area-under-curve evaluation metrics. Specifically, the AU(PRC‾) reaches 64.5 % on the Chinese tourism resource dataset with enforced leaf nodes, which is 3 % higher than the optimal corresponding metric of the baseline model. Furthermore, ablation studies show that (1) integrating domain knowledge, (2) combining local information, (3) considering label dependencies within the same level of label hierarchy, and (4) merging dynamic reconstruction can enhance overall model performance.

Prompt tuning discriminative language models for hierarchical text classification

Prompt tuning discriminative language models for hierarchical text classification

Retrieval-style In-context Learning for Few-shot Hierarchical Text Classification

Abstract Hierarchical text classification (HTC) is an important task with broad applications, and few-shot HTC has gained increasing interest recently. While in-context learning (ICL) with large language models (LLMs) has achieved significant success in few-shot learning, it is not as effective for HTC because of the expansive hierarchical label sets and extremely ambiguous labels. In this work, we introduce the first ICL-based framework with LLM for few-shot HTC. We exploit a retrieval database to identify relevant demonstrations, and an iterative policy to manage multi-layer hierarchical labels. Particularly, we equip the retrieval database with HTC label-aware representations for the input texts, which is achieved by continual training on a pretrained language model with masked language modeling (MLM), layer-wise classification (CLS, specifically for HTC), and a novel divergent contrastive learning (DCL, mainly for adjacent semantically similar labels) objective. Experimental results on three benchmark datasets demonstrate superior performance of our method, and we can achieve state-of-the-art results in few-shot HTC.

BERT-BiLSTM model for hierarchical Arabic text classification

BERT-BiLSTM model for hierarchical Arabic text classification

Формальная модель оценки образа персонажа художественного произведения (на материале романа Дж. Оруэлла «1984»)

The aim of the research is to develop and test a formal model for evaluating the image of a character in a fictional work based on the corpus approach. The scientific originality lies in the fact that for the first time such an analysis of the character of a fictional work is carried out on the basis of the corpus approach. In the course of the work, a balanced corpus of the original text of the novel “1984” by G. Orwell was formed, which includes the marking of hierarchical relationships between the tokens. A number of search queries to the corpus were carried out using the author’s software “Balanced Linguistic Corpus Generator and Corpus Manager”. With the help of this tool, a frequency list of proper names from the novel was obtained and lists of tokens characterizing its main characters were output. The data obtained is extracted during the operation of a special algorithm built into the corpus manager, which selects from the text significant character traits and related lexical units. Thus, a model for evaluating the image of a character is built as a formal description of a system of interconnected hierarchical text elements. The results of the study were analyzed and compared with the results of mental interpretation. Thus, the characters’ profiles were compiled in terms of a formal description of their actions and those objects to which their actions were directed.

Hierarchy-Aware and Label Balanced Model for Hierarchical Text Classification

Hierarchical text classification, where labels can be modeled as a hierarchical structure, is a special multi-label text classification sub-task. Current methods mainly improve model performance by modeling label dependencies. The model HGCLR with excellent performance obtains hierarchy-aware text representations through contrastive learning, but the degree of hierarchical awareness is insufficient. To address this problem, a new multi-label negative supervision method is proposed to drive text representations of samples with more different labels farther away. In addition, to tackle the problem of label imbalance in hierarchical text classification, asymmetric loss is employed to compute the classification loss, so that the model focuses on learning from difficult samples and the contributions of positive and negative labels to the loss function tend to be balanced. In summary, based on HGCLR, by adding multi-label negative supervision and replacing the classification loss with asymmetric loss, we propose a hierarchy-aware and label balanced model (HALB) for hierarchical text classification. Experimental results demonstrate that HALB outperforms several classical models for hierarchical text classification. HALB achieves the best results on both Micro-F1 and Macro-F1 and obtains average improvements of 0.51% and 1.28% respectively on four datasets compared with the strongest baseline model HGCLR.

Query-Adaptive Late Fusion for Hierarchical Fine-Grained Video-Text Retrieval.

Recently, a hierarchical fine-grained fusion mechanism has been proved effective in cross-modal retrieval between videos and texts. Generally, the hierarchical fine-grained semantic representations (video-text semantic matching is decomposed into three levels including global-event representation matching, action-relation representation matching, and local-entity representation matching) to be fused can work well by themselves for the query. However, in real-world scenarios and applications, existing methods failed to adaptively estimate the effectiveness of multiple levels of the semantic representations for a given query in advance of multilevel fusion, resulting in a worse performance than expected. As a result, it is extremely essential to identify the effectiveness of hierarchical semantic representations in a query-adaptive manner. To this end, this article proposes an effective query-adaptive multilevel fusion (QAMF) model based on manipulating multiple similarity scores between the hierarchical visual and text representations. First, we decompose video-side and text-side representations into hierarchical semantic representations consisting of global-event level, action-relation level, and local-entity level, respectively. Then, the multilevel representation of the video-text pair is aligned to calculate the similarity score for each level. Meanwhile, the sorted similarity score curves of the good semantic representation are different from the inferior ones, which exhibit a "cliff" shape and gradually decline (see Fig. fig1 as an example). Finally, we leverage the Gaussian decay function to fit the tail of the score curve and calculate the area under the normalized sorted similarity curve as the indicator of semantic representation effectiveness, namely, the area of good semantic representation is small, and vice versa. Extensive experiments on three public benchmark video-text datasets have demonstrated that our method consistently outperforms the state-of-the-art (SoTA). A simple demo of QAMF will soon be publicly available on our homepage: https://github.com/Lab-ANT.

Data Sorting Influence on Short Text Manual Labeling Quality for Hierarchical Classification

The precise categorization of brief texts holds significant importance in various applications within the ever-changing realm of artificial intelligence (AI) and natural language processing (NLP). Short texts are everywhere in the digital world, from social media updates to customer reviews and feedback. Nevertheless, short texts’ limited length and context pose unique challenges for accurate classification. This research article delves into the influence of data sorting methods on the quality of manual labeling in hierarchical classification, with a particular focus on short texts. The study is set against the backdrop of the increasing reliance on manual labeling in AI and NLP, highlighting its significance in the accuracy of hierarchical text classification. Methodologically, the study integrates AI, notably zero-shot learning, with human annotation processes to examine the efficacy of various data-sorting strategies. The results demonstrate how different sorting approaches impact the accuracy and consistency of manual labeling, a critical aspect of creating high-quality datasets for NLP applications. The study’s findings reveal a significant time efficiency improvement in terms of labeling, where ordered manual labeling required 760 min per 1000 samples, compared to 800 min for traditional manual labeling, illustrating the practical benefits of optimized data sorting strategies. Comparatively, ordered manual labeling achieved the highest mean accuracy rates across all hierarchical levels, with figures reaching up to 99% for segments, 95% for families, 92% for classes, and 90% for bricks, underscoring the efficiency of structured data sorting. It offers valuable insights and practical guidelines for improving labeling quality in hierarchical classification tasks, thereby advancing the precision of text analysis in AI-driven research. This abstract encapsulates the article’s background, methods, results, and conclusions, providing a comprehensive yet succinct study overview.

Hierarchical Text Classification and Its Foundations: A Review of Current Research

While collections of documents are often annotated with hierarchically structured concepts, the benefits of these structures are rarely taken into account by classification techniques. Within this context, hierarchical text classification methods are devised to take advantage of the labels’ organization to boost classification performance. In this work, we aim to deliver an updated overview of the current research in this domain. We begin by defining the task and framing it within the broader text classification area, examining important shared concepts such as text representation. Then, we dive into details regarding the specific task, providing a high-level description of its traditional approaches. We then summarize recently proposed methods, highlighting their main contributions. We also provide statistics for the most commonly used datasets and describe the benefits of using evaluation metrics tailored to hierarchical settings. Finally, a selection of recent proposals is benchmarked against non-hierarchical baselines on five public domain-specific datasets. These datasets, along with our code, are made available for future research.

Adaptive micro- and macro-knowledge incorporation for hierarchical text classification

Hierarchical text classification (HTC) aims to classify a text into multiple categories organized in a hierarchical structure. The state-of-the-art HTC methods usually employ graph networks, where label graphs are constructed and label representation is learned to interact with text representations for classification. In general, label graphs are built on the intrinsic label hierarchy, label semantic similarity, or label co-occurrence. Such graphs have been proven to be effective, but they only exploit knowledge from training data or simple label descriptions, without considering the vast external knowledge in the open sources. Actually, external knowledge from open sources could bring in complementary information to enhance the label graph’s representation power. Motivated by the above considerations, we explore the use of external knowledge for improving HTC in this paper. We categorize knowledge into micro-knowledge and macro-knowledge, which are defined as the fundamental concepts related to a single class label and the correlations among class labels, respectively. For tailor-made incorporation of the two types of knowledge into representation learning and classification, we propose Adaptive Micro- and Macro-Knowledge Incorporation for Hierarchical Text Classification (AMKI-HTC) model in this paper. The micro-knowledge incorporation helps capture class-relevant keywords in the text and hence produce discriminative representations, while the macro-knowledge incorporation improves the accuracy of label graphs. Finally, a confidence maximization fusion strategy is developed for adaptive aggregation of multi-view features. Extensive experiments on three benchmark HTC datasets demonstrate that AMKI-HTC consistently outperforms state-of-the-art models.

Hierarchical text classification with multi-label contrastive learning and KNN

Given the complicated label hierarchy, hierarchical text classification (HTC) has emerged as a challenging subtask in the realm of multi-label text classification. Existing methods enhance the quality of text representations by contrastive learning, but this supervised contrastive learning is designed for single-label setting and has two main limitations. On one hand, sample pairs with completely identical labels which should be treated as positive pairs are ignored. On the other hand, a simple pair is deemed as an absolutely positive or negative pair, which lacks consideration about the situation where sample pairs share some labels while having labels unique to each sample. Therefore, we propose a method combining multi-label contrastive learning with KNN (MLCL-KNN) for HTC. The proposed multi-label contrastive learning method can make text representations of sample pairs having more shared labels closer and separate those with no labels in common. During inference, we employ KNN to retrieve several neighbor samples and regard their labels as additional prediction, which is interpolated into the model output to further improve the performance of MLCL-KNN. Compared with the strongest baseline, MLCL-KNN achieves average improvements of 0.31%, 0.76%, 0.83%, and 0.43% on Micro-F1, Macro-F1, accuracy, and HiF respectively, which demonstrates its effectiveness.

A new sentence embedding framework for the education and professional training domain with application to hierarchical multi-label text classification

In recent years, Natural Language Processing (NLP) has made significant advances through advanced general language embeddings, allowing breakthroughs in NLP tasks such as semantic similarity and text classification. However, complexity increases with hierarchical multi-label classification (HMC), where a single entity can belong to several hierarchically organized classes. In such complex situations, applied on specific-domain texts, such as the Education and professional training domain, general language embedding models often inadequately represent the unique terminologies and contextual nuances of a specialized domain. To tackle this problem, we present HMCCCProbT, a novel hierarchical multi-label text classification approach. This innovative framework chains multiple classifiers, where each individual classifier is built using a novel sentence-embedding method BERTEPro based on existing Transformer models, whose pre-training has been extended on education and professional training texts, before being fine-tuned on several NLP tasks. Each individual classifier is responsible for the predictions of a given hierarchical level and propagates local probability predictions augmented with the input feature vectors to the classifier in charge of the subsequent level. HMCCCProbT tackles issues of model scalability and semantic interpretation, offering a powerful solution to the challenges of domain-specific hierarchical multi-label classification. Experiments over three domain-specific textual HMC datasets indicate the effectiveness of HMCCCProbT to compare favorably to state-of-the-art HMC algorithms in terms of classification accuracy and also the ability of BERTEPro to obtain better probability predictions, well suited to HMCCCProbT, than three other vector representation techniques.

Knowledge Graph-Based Hierarchical Text Semantic Representation

Document representation is the basis of language modeling. Its goal is to turn natural language text that flows into a structured form that can be stored and processed by a computer. The bag-of-words model is used by most of the text-representation methods that are currently available. And yet, they do not consider how phrases are used in the text, which hurts the performance of tasks that use natural language processing later on. Representing the meaning of text by phrases is a promising area of future research, but it is hard to do well because phrases are organized in a hierarchy and mining efficiency is low. In this paper, we put forward a method called hierarchical text semantic representation using the knowledge graph (HTSRKG), which uses syntactic structure features to find hierarchical phrases and knowledge graphs to improve how phrases are evaluated. First, we use CKY and PCFG to build the syntax tree sentence by sentence. Second, we walk through the parse tree using the hierarchical routing process to obtain the mixed phrase semantics in passages. Finally, the introduction of the knowledge graph improves the efficiency of text semantic extraction and the accuracy of text representation. This gives us a solid foundation for tasks involving natural language processing that come after. Extensive testing on actual datasets shows that HTSRKG surpasses baseline approaches with respect to text semantic representation, and the results of a recent benchmarking study support this.

Neuroanatomical location of lung cancer brain metastases in 234 patients with a focus on cancer subtyping and biomarkers.

Brain metastases are frequent in neuropathology practices; however, the literature on their distribution is frequently derived from imaging studies. This work examined metastases of lung cancer to the brain through the lens of pathology specimens. All brain surgical pathology cases accessioned from 2011-2020 were retrieved from a regional laboratory. Specimens were classified by neuroanatomical location, diagnostic category, and diagnosis with a hierarchical free text string-matching algorithm. All reports classified as probable metastasis per algorithm were reviewed by a pathologist. Lung biomarkers and selected immunostains were retrieved with text parsing and reviewed. Among 4,625 cases of brain surgical resection specimens, 854 were classified as probable metastasis by the algorithm. On report review, 538/854 cases were confirmed as metastasis with a known primary site. The 538 cases were from 511 patients and 234/511 patients had lung primaries. Small cell lung cancer lesions were most frequently found in the cerebellum (17/30). Lesions from lung adenocarcinoma (59/164) and non-small cell carcinoma-not otherwise specified (NSCLC-NOS) (15/34) were most commonly found in the frontal lobe. Squamous cell carcinoma lesions were most commonly found in the frontal and occipital lobes (8/27). 72/234 cases were reported as NSCLC-NOS and could be further subclassified using immunostaining (41/72). Lung biomarker data were retrieved in ~38% of cases. PD-L1 positivity was dependent on neuroanatomical distribution (p = 0.04); other examined biomarkers were not. The distribution of lung tumours metastatic to the brain is dependent on the lung cancer subtype (p<0.001). The reporting of histologic subtype could be further optimized in the local environment.

Does the Order Matter? A Random Generative Way to Learn Label Hierarchy for Hierarchical Text Classification

Does the Order Matter? A Random Generative Way to Learn Label Hierarchy for Hierarchical Text Classification

An improved multi-stage framework for large-scale hierarchical text classification problems using a modified feature hashing and bi-filtering strategy

The classification of large-scale textual dataset is associated with a huge number of instances and millions of features which must be discriminated between large numbers of categories. The task requires the utilization of a defined hierarchy structure and tools that automatically classify instances within the hierarchy known as Large Scale Hierarchical Text Classification (LSHTC). Predicting the labels of instances by the employed classifiers is challenging due to the high number of features. Furthermore, the existing Dimensional Reduction (DR) approaches in cooperation with the LSHTC framework are still quite inefficient. In such a problem, an effective Hierarchical Dimensional Reduction approach can be advantageous in improving the performance of the LSHTC. Therefore, in this paper, we enhance the performance of LSHTC by proposing a Multi-stage Hierarchical Dimensional Reduction (MHDR) approach based on Modified Feature Hashing (MFH) and Hierarchical Bi-Filtering (HBF) method. In addition to alleviating bad collision and result discrepancy, experimental results show that the proposed approach has achieve the best performance in terms of micro-f1 and macro-f1 by recording average scores of 58.47% and 54.77% using TD-SVM, and average scores of 51.14% and 48.70% using TD-LR, respectively. The method also achieved 11% speed-up than the approaches compared.

Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentation.

The Segment Anything Model (SAM), a profound vision foundation model pretrained on a large-scale dataset, breaks the boundaries of general segmentation and sparks various downstream applications. This paper introduces Hi-SAM, a unified model leveraging SAM for hierarchical text segmentation. Hi-SAM excels in segmentation across four hierarchies, including pixel-level text, word, text-line, and paragraph, while realizing layout analysis as well. Specifically, we first turn SAM into a high-quality pixel-level text segmentation (TS) model through a parameter-efficient fine-tuning approach. We use this TS model to iteratively generate the pixel-level text labels in a semi-automatical manner, unifying labels across the four text hierarchies in the HierText dataset. Subsequently, with these complete labels, we launch the end-to-end trainable Hi-SAM based on the TS architecture with a customized hierarchical mask decoder. During inference, Hi-SAM offers both automatic mask generation (AMG) mode and promptable segmentation (PS) mode. In the AMG mode, Hi-SAM segments pixel-level text foreground masks initially, then samples foreground points for hierarchical text mask generation and achieves layout analysis in passing. As for the PS mode, Hi-SAM provides word, text-line, and paragraph masks with a single point click. Experimental results show the state-of-the-art performance of our TS model: 84.86% fgIOU on Total-Text and 88.96% fgIOU on TextSeg for pixel-level text segmentation. Moreover, compared to the previous specialist for joint hierarchical detection and layout analysis on HierText, Hi-SAM achieves significant improvements: 4.73% PQ and 5.39% F1 on the text-line level, 5.49% PQ and 7.39% F1 on the paragraph level layout analysis, requiring 20× fewer training epochs. The code is available at Hi-SAM.

Neuroanatomical location of brain metastases from solid tumours based on pathology: An analysis of 511 patients with a comparison to the provided clinical history.

Brain metastases are a frequent occurrence in neuropathology practices. The literature on their neuroanatomical location is frequently derived from radiological analyses. This work examines brain metastases through the lens of pathology specimens. All brain surgical pathology reports for cases accessioned 2011-2020 were retrieved from a laboratory. Specimens were classified by neuroanatomical location, diagnosis and diagnostic category with a hierarchical free text string-matching algorithm (HFTSMA) and also subsequently audited. All reports classified as probable metastasis were reviewed by a pathologist. The provided history was compared to the final categorization by a pathologist. The cohort had 4,625 cases. The HFTSMA identified 854 cases (including metastases from a definite primary, metastases from primary not known and improperly classified cases). 514/854 cases had one definite primary site per algorithm and on report review 538/854 cases were confirmed as such. The 538 cases originated from 511 patients. Primaries from breast, gynecologic tract, and gastrointestinal tract not otherwise specified were most frequently found in the cerebellum. Kidney metastases were most frequently found in the occipital lobe. Lung, metastatic melanoma and colorectal primaries were most commonly found in the frontal lobe. The provided clinical history predicted the primary in 206 cases (40.3%), was discordant in 17 cases (3.3%) and non-contributory in 280 cases (54.8%). The observed distribution of the metastatic tumours in the brain is dependent on the primary site. In the majority (54.8%) of cases, the provided clinical history was non-contributory; this suggests surgeon-pathologist communication may have the potential for optimization.