Text Graph Research Articles

Improving text classification via computing category correlation matrix from text graph

In text classification task, models have shown remarkable accuracy across various datasets. However, confusion often arises when certain categories within the dataset are too similar, causing misclassification of certain samples. This paper proposes an improved method for this problem, through the creation of a three-layer text graph for the corpus, which is used to calculate the Category Correlation Matrix (CCM). Additionally, this paper introduces category-adaptive contrastive learning for text embedding from the encoder, enhancing the model’s ability to distinguish between samples in confusable categories that are easily confused. Soft labels are generated using this matrix to guide the classifier, preventing the model from becoming overconfident with one-hot vectors. The efficacy of this approach was demonstrated through experimental evaluations on three text encoders and six different datasets.

Clustering Integrated Fusion Model Based on the Maximum Matching Problem

The maximum-weight perfect matching inverse issue in graph theory and text clustering are the two primary topics of this study. We suggest a novel approach to text clustering that makes use of self-encoders and BERT embeddings for feature extraction in order to increase clustering accuracy. According to experimental results, our technique enhances the clustering results greatly and performs well on numerous short text datasets. In the context of graph theory, we examine the unit paradigm inverse issue of maximum-weight perfect matching with value constraints and provide a robust polynomial-time method for its solution. In addition to effectively solving the maximum-weight perfect matching inverse issue, our technique can also produce the best weight vector configuration scheme for real-world uses. In conclusion, our work has advanced the domains of text clustering and graph theory significantly, offering fresh approaches and theoretical underpinnings for future investigations.

Machine learning based software effort estimation using development-centric features for crowdsourcing platform

Multi-label text classification is a method for categorizing textual data based on features extracted from the original textual information. When it comes to modelling text structural properties, Graph Convolutional Network (GCN) has demonstrated outstanding performance. However, most existing graph-based models do not model the structure of a single text unit and do not consider the sequence information in each document (e.g., word order). To resolve these issues and fully utilize the text’s structural and sequential details, a text classification model called Sequential GCN with Multi-Head Attention (SGCN-MHA) is proposed in this paper. For each text, a separate text graph is constructed in which nodes are the words of the text, and the edges between nodes corresponding to the word relations. Then the GCN is used to extract the structural feature. To enable the word nodes in the document graph to hold contextual information, the BiLSTM is also applied to learn the sequential feature for each graph. Finally, the Multi-Head Attention mechanism is adopted to interact with these two features and then aggregate them to get access to critical information in the text. The efficiency of our approach has been tested on two standard datasets, including comparative and ablation experiments.

Multimodal learning for temporal relation extraction in clinical texts.

This study focuses on refining temporal relation extraction within medical documents by introducing an innovative bimodal architecture. The overarching goal is to enhance our understanding of narrative processes in the medical domain, particularly through the analysis of extensive reports and notes concerning patient experiences. Our approach involves the development of a bimodal architecture that seamlessly integrates information from both text documents and knowledge graphs. This integration serves to infuse common knowledge about events into the temporal relation extraction process. Rigorous testing was conducted on diverse clinical datasets, emulating real-world scenarios where the extraction of temporal relationships is paramount. The performance of our proposed bimodal architecture was thoroughly evaluated across multiple clinical datasets. Comparative analyses demonstrated its superiority over existing methods reliant solely on textual information for temporal relation extraction. Notably, the model showcased its effectiveness even in scenarios where not provided with additional information. The amalgamation of textual data and knowledge graph information in our bimodal architecture signifies a notable advancement in the field of temporal relation extraction. This approach addresses the critical need for a more profound understanding of narrative processes in medical contexts. In conclusion, our study introduces a pioneering bimodal architecture that harnesses the synergy of text and knowledge graph data, exhibiting superior performance in temporal relation extraction from medical documents. This advancement holds significant promise for improving the comprehension of patients' healthcare journeys and enhancing the overall effectiveness of extracting temporal relationships in complex medical narratives.

Narratives from GPT-derived networks of news and a link to financial markets dislocations

AbstractWe introduce a novel framework to study the dynamics of news narratives, by leveraging GPT3.5 advanced text analysis capabilities and graph theory. In particular, we focus on a corpus of economic articles from The Wall Street Journal and dynamically extract the main topics of discussion over time, in a completely systematic and scalable fashion. As a simple application of the suggested approach, we show how the structure of such topics of discussion has a statistically significant relationship with the contemporaneous state of financial markets, which can be used to construct an investment strategy or monitor financial risks. Our work is based on the intrinsic ability of GPT models to track the context of sentences within a document, thanks to which we can accurately extract a ranking of the most important entities discussed within each article, and evaluate their entity-specific sentiments. Then, we create a graph for each week of data, in which nodes are the entities retrieved and edges are built from the co-occurrence of such entities within articles. Graph centrality measures are computed over time to track the most representative keywords of topics of discussion, which result in an accurate summary view of the evolution of economic narratives. Fuzzy community detection is finally used to cluster linked entities into a more detailed representation of topics. Such groups of entities are mapped to the related journal articles, which are in turn summarised to reach a highly nuanced and interpretable view of the topics discussed within each week. Linking the features of these topics to the relevant financial market time series, we find that high fragmentation within our networks’ communities relates to moments of financial markets dislocations (i.e. dates with unusually high volatility across asset classes). This result should thus motivate stronger effort within financial research to move beyond ubiquitous sentiment analysis of news and delve deeper into broader and more holistic studies of textual data.

The effect of augmented reality applied to learning process with different learning styles in structural engineering education

PurposeThe purpose of this study is to scrutinize the influence of individual learning styles on the effectiveness of augmented reality (AR)-based learning in structural engineering. There has been a lack of research examining the correlation between learning efficiency and learning style, particularly in the context of quantitatively assessing the efficacy of AR in structural engineering education.Design/methodology/approachUsing Kolb’s experiential learning theory (ELT), a model that emphasizes learning through experience, students from the construction management department are assigned four learning styles (converging, assimilating, diverging and accommodating). Performance data were gathered, appraised, and compared through the three dimensions from the Knowledge, Attitude and Practices (KAP) survey model across four categories of Kolb’s learning styles in both text-graph (TG)-based and AR-based learning settings.FindingsThe findings indicate that AR-based materials positively impact structural engineering education by enhancing overall learning performance more than TG-based materials. It is also found that the learning style has a profound influence on learning effectiveness, with AR technology markedly improving the information retrieval processes, particularly for converging and assimilating learners, then diverging learners, with a less significant impact on accommodating learners.Originality/valueThese results corroborate prior research analyzing learners' outcomes with hypermedia and informational learning systems. It was found that learners with an “abstract” approach (convergers and assimilators) outperform those with a “concrete” approach (divergers and accommodators). This research emphasizes the importance of considering learning styles before integrating technologies into civil engineering education, thereby assisting software developers and educational institutions in creating more effective teaching materials tailored to specific learning styles.

Heterogeneous text graph for comprehensive multilingual sentiment analysis: capturing short- and long-distance semantics.

Multilingual sentiment analysis (MSA) involves the task of comprehending people's opinions, sentiments, and emotions in multilingual written texts. This task has garnered considerable attention due to its importance in extracting insights for decision-making across diverse fields such as marketing, finance, and politics. Several studies have explored MSA using deep learning methods. Nonetheless, a majority of these studies depend on sequential-based approaches, which focus on capturing short-distance semantics within adjacent word sequences, but they overlook long-distance semantics, which can provide more profound insights for analysis. In this work, we propose an approach for multilingual sentiment analysis, namely MSA-GCN, leveraging a graph convolutional network to effectively capture both short- and long-distance semantics. MSA-GCN involves the comprehensive modeling of the multilingual sentiment analysis corpus through a unified heterogeneous text graph. Subsequently, a slightly deep graph convolutional network is employed to acquire predictive representations for all nodes by encouraging the transfer learning across languages. Extensive experiments are carried out on various language combinations using different benchmark datasets to assess the efficiency of the proposed approach. These datasets include Multilingual Amazon Reviews Corpus (MARC), Internet Movie Database (IMDB), Allociné, and Muchocine. The achieved results reveal that MSA-GCN significantly outperformed all baseline models in almost all datasets with a p-value < 0.05 based on student t-test. In addition, such approach shows prominent results in a variety of language combinations, revealing the robustness of the approach against language variation.

Graph-based Text Classification by Contrastive Learning with Text-level Graph Augmentation

Text Classification (TC) is a fundamental task in the information retrieval community. Nowadays, the mainstay TC methods are built on the deep neural networks, which can learn much more discriminative text features than the traditional shallow learning methods. Among existing deep TC methods, the ones based on Graph Neural Network (GNN) have attracted more attention due to the superior performance. Technically, the GNN-based TC methods mainly transform the full training dataset to a graph of texts; however, they often neglect the dependency between words, so as to miss potential semantic information of texts, which may be significant to exactly represent them. To solve the aforementioned problem, we generate graphs of words instead, so as to capture the dependency information of words. Specifically, each text is translated into a graph of words, where neighboring words are linked. We learn the node features of words by a GNN-like procedure and then aggregate them as the graph feature to represent the current text. To further improve the text representations, we suggest a contrastive learning regularization term. Specifically, we generate two augmented text graphs for each original text graph, we constrain the representations of the two augmented graphs from the same text close and the ones from different texts far away. We propose various techniques to generate the augmented graphs. Upon those ideas, we develop a novel deep TC model, namely Text-level Graph Networks with Contrastive Learning (TGN cl ). We conduct a number of experiments to evaluate the proposed TGN cl model. The empirical results demonstrate that TGN cl can outperform the existing state-of-the-art TC models.

TG468: a text graph convolutional network for predicting clinical response to immune checkpoint inhibitor therapy.

Enhancing cancer treatment efficacy remains a significant challenge in human health. Immunotherapy has witnessed considerable success in recent years as a treatment for tumors. However, due to the heterogeneity of diseases, only a fraction of patients exhibit a positive response to immune checkpoint inhibitor (ICI) therapy. Various single-gene-based biomarkers and tumor mutational burden (TMB) have been proposed for predicting clinical responses to ICI; however, their predictive ability is limited. We propose the utilization of the Text Graph Convolutional Network (GCN) method to comprehensively assess the impact of multiple genes, aiming to improve the predictive capability for ICI response. We developed TG468, a Text GCN model framing drug response prediction as a text classification task. By combining natural language processing (NLP) and graph neural network techniques, TG468 effectively handles sparse and high-dimensional exome sequencing data. As a result, TG468 can distinguish survival time for patients who received ICI therapy and outperforms single gene biomarkers, TMB and some classical machine learning models. Additionally, TG468's prediction results facilitate the identification of immune status differences among specific patient types in the Cancer Genome Atlas dataset, providing a rationale for the model's predictions. Our approach represents a pioneering use of a GCN model to analyze exome data in patients undergoing ICI therapy and offers inspiration for future research using NLP technology to analyze exome sequencing data.

Zero-sample text classification algorithm based on BERT and graph convolutional neural network

Abstract In this study, we undertake a comprehensive examination of zero-shot text classification and its associated implications. We propose the adoption of the BERT model as a method for text feature representation. Subsequently, we utilize the Pointwise Mutual Information (PMI) metric to adjust the weight values within a graph convolutional neural network, thereby facilitating the construction of a text graph. Additionally, we incorporate an attention mechanism to transform this text graph, enabling it to represent the output labels of zero-shot text classification effectively. The experimental environment is set up, and the comparison and ablation experiments of the text classification model based on BERT and graph convolutional neural network with the baseline models are carried out in several different types of datasets, and the parameter settings of λ are adjusted according to the experimental results, and the convergence of the BERT model is compared to test the robustness of the model performance and the classification effect. When λ was set to 0.60, the model achieved the best results in each dataset. When the task is set to 5-way-5-shot, the convergence rate of the model for the Snippets dataset using the penultimate layer of features can reach 74%-80% of the training accuracy at the 5,000th step. The training accuracy gradually flattens out in the first 10,000 steps, and the model achieves classification accuracy in all four learning scenarios, with good stability.

Mind Reasoning Manners: Enhancing Type Perception for Generalized Zero-Shot Logical Reasoning Over Text.

Logical reasoning task involves diverse types of complex reasoning over text, based on the form of multiple-choice question answering (MCQA). Given the context, question and a set of options as the input, previous methods achieve superior performances on the full-data setting. However, the current benchmark dataset has the ideal assumption that the reasoning type distribution on the train split is close to the test split, which is inconsistent with many real application scenarios. To address it, there remain two problems to be studied: 1) how is the zero-shot capability of the models (train on seen types and test on unseen types)? and 2) how to enhance the perception of reasoning types for the models? For problem 1, we propose a new benchmark for generalized zero-shot logical reasoning, named ZsLR. It includes six splits based on the three type sampling strategies. For problem 2, a type-aware model TaCo is proposed. It utilizes the heuristic input reconstruction and builds a text graph with a global node. Incorporating graph reasoning and contrastive learning, TaCo can improve the type perception in the global representation. Extensive experiments on both the zero-shot and full-data settings prove the superiority of TaCo over the state-of-the-art (SOTA) methods. Also, we experiment and verify the generalization capability of TaCo on other logical reasoning dataset.

Synergistic Detection of Multimodal Fake News Leveraging TextGCN and Vision Transformer

In today's digital age, the rapid spread of fake news is a pressing concern. Fake news, whether intentional or inadvertent, manipulates public sentiment and threatens the integrity of online information. To address this, effective detection and prevention methods are vital. Detecting and addressing this multimodal fake news is an intricate challenge as, unlike traditional news articles that predominantly rely on textual content, multimodal fake news leverages the persuasive power of visual elements, making its identification a formidable task. Manipulated images can significantly sway individuals' perceptions and beliefs, making the detection of such deceptive content complex. Our research introduces an innovative approach to multimodal fake news identification by presenting a fusion-based methodology that harnesses the capabilities of Text Graph Convolutional Neural Networks (TextGCN) and Vision Transformers (ViT) to effectively utilise both text and image modalities. The proposed Methodology starts with preprocessing textual content using TextGCN, allowing for the capture of intricate structural dependencies among words and phrases. Simultaneously, visual features are extracted from associated images using ViT. Through a fusion mechanism, these modalities seamlessly integrate, yielding superior embeddings. The primary contributions encompass an in-depth exploration of multimodal fake news detection through a fusion-based approach. What sets our approach apart from existing techniques is its integration of graph-based feature extraction through TextGCN. While previous methods predominantly rely on text or image features, our approach harnesses the additional semantic information and intricate relationships within a graph structure, in addition to image embeddings. This enables our method to capture more comprehensive understanding of the data, resulting in increased accuracy and reliability. Our experiments demonstrate the exceptional performance of our fusion-based approach, which leverages multiple modalities and incorporates graph-based representations and semantic relationships. This method outperformed single modalities of text or image, achieving an impressive accuracy of 94.17% using a neural network after fusion. By seamlessly integrating graph-based representations and semantic relationships, our fusion-based technique represents a significant stride in addressing the challenges posed by multimodal fake news.

Transformer and Graph Convolutional Network for Text Classification

Graph convolutional network (GCN) is an effective tool for feature clustering. However, in the text classification task, the traditional TextGCN (GCN for Text Classification) ignores the context word order of the text. In addition, TextGCN constructs the text graph only according to the context relationship, so it is difficult for the word nodes to learn an effective semantic representation. Based on this, this paper proposes a text classification method that combines Transformer and GCN. To improve the semantic accuracy of word node features, we add a part of speech (POS) to the word-document graph and build edges between words based on POS. In the layer-to-layer of GCN, the Transformer is used to extract the contextual and sequential information of the text. We conducted the experiment on five representative datasets. The results show that our method can effectively improve the accuracy of text classification and is better than the comparison method.

TGCN-Bert Emoji Prediction in Information Systems Using TCN and GCN Fusing Features Based on BERT

In recent studies, graph convolutional neural networks (GCNs) have been used to solve different natural language processing (NLP) tasks. However, few researches apply graph convolutional networks to short text classification. Emoji prediction, as a complex sentiment analysis task, has received even less attention. In this work, the authors propose TGCN-Bert which combines pre-trained BERT temporal convolutional networks (TCNs) and graph convolutional networks for short text classification and emoji prediction. They initialize the nodes with the help of BERT and define the edges in text graph based on the term frequency-inverse document frequency (TF-IDF) and positive point-wise mutual information (PPMI). They employ the model for emoji prediction task, and a metric based on emoji clustering is developed to better measure the validity of emoji prediction results. To validate the performance of TGCN-Bert, they compare it with other GCN variants on short text classification datasets and emoji prediction datasets; experiments show that TGCN-Bert achieves better performance.

Concept and dependencies enhanced graph convolutional networks for short text classification

Short text classification task is a special kind of text classification task in that the text to be classified is generally short, typically generating a sparse text representation that lacks rich semantic information. Given this shortcoming, scholars worldwide have explored improved short text classification methods based on deep learning. However, existing methods cannot effectively use concept knowledge and long-distance word dependencies. Therefore, based on graph neural networks from the perspective of text composition, we propose concept and dependencies enhanced graph convolutional networks for short text classification. First, the co-occurrence relationship between words is obtained by sliding window, the inclusion relationship between documents and words is obtained by TF-IDF, long-distance word dependencies is obtained by Stanford CoreNLP, and the association relationship between concepts in the concept graph with entities in the text is obtained through Microsoft Concept Graph. Then, a text graph is constructed for an entire text corpus based on the four relationships. Finally, the text graph is input into graph convolutional neural networks, and the category of each document node is predicted after two layers of convolution. Experimental results demonstrate that our proposed method overall best on multiple classical English text classification datasets.

When Unstructured Big Text Corpus Meets Text Network Analysis: Social Reality Conceptualization and Visualization Graph of Big Interview Data of Heavy Drug Addicts of Skid Row.

Relying on user-generated content narrating individual experiences and personalized contextualization of location-specific realities, this study introduced a novel methodological approach and analysis tool that can aid health informatics in understanding the social reality of people with a substance-use disorder in Skid Row, Los Angeles. The study also highlighted analysis possibilities for big unstructured interview text corpus using InfraNodus, a text network analysis tool. InfraNodus, which is a text graph analysis tool, identifies pathways for meaning circulation within unstructured interview data and has the potential to classify topical clusters and generate contextualized analysis results for big narrative textual datasets. Using InfraNodus, we analyzed a 1,103,528-word unstructured interview transcript from 315 interview sessions with people with a substance-use disorder, who narrated their respective social realities. Challenging the overgeneralization of onlookers, the conceptualization process identified topical clusters and pathways for meaning circulation within the narrative data, generating unbiased contextualized meaning for the collective social reality. Our endeavors in this research, along with our methodological setting and selection, might contribute to the methodological efforts of health informatics or the conceptualization and visualization needs of any big text corpus.

Chinese text classification by combining Chinese-BERTology-wwm and GCN.

Text classification is an important and classic application in natural language processing (NLP). Recent studies have shown that graph neural networks (GNNs) are effective in tasks with rich structural relationships and serve as effective transductive learning approaches. Text representation learning methods based on large-scale pretraining can learn implicit but rich semantic information from text. However, few studies have comprehensively utilized the contextual semantic and structural information for Chinese text classification. Moreover, the existing GNN methods for text classification did not consider the applicability of their graph construction methods to long or short texts. In this work, we propose Chinese-BERTology-wwm-GCN, a framework that combines Chinese bidirectional encoder representations from transformers (BERT) series models with whole word masking (Chinese-BERTology-wwm) and the graph convolutional network (GCN) for Chinese text classification. When building text graph, we use documents and words as nodes to construct a heterogeneous graph for the entire corpus. Specifically, we use the term frequency-inverse document frequency (TF-IDF) to construct the word-document edge weights. For long text corpora, we propose an improved pointwise mutual information (PMI*) measure for words according to their word co-occurrence distances to represent the weights of word-word edges. For short text corpora, the co-occurrence information between words is often limited. Therefore, we utilize cosine similarity to represent the word-word edge weights. During the training stage, we effectively combine the cross-entropy and hinge losses and use them to jointly train Chinese-BERTology-wwm and GCN. Experiments show that our proposed framework significantly outperforms the baselines on three Chinese benchmark datasets and achieves good performance even with few labeled training sets.

Examining Care Planning Efficiency and Clinical Decision Support Adoption in a System Tailoring to Nurses' Graph Literacy: National, Web-Based Randomized Controlled Trial.

The proliferation of health care data in electronic health records (EHRs) is fueling the need for clinical decision support (CDS) that ensures accuracy and reduces cognitive processing and documentation burden. The CDS format can play a key role in achieving the desired outcomes. Building on our laboratory-based pilot study with 60 registered nurses (RNs) from 1 Midwest US metropolitan area indicating the importance of graph literacy (GL), we conducted a fully powered, innovative, national, and web-based randomized controlled trial with 203 RNs. This study aimed to compare care planning time (CPT) and the adoption of evidence-based CDS recommendations by RNs randomly assigned to 1 of 4 CDS format groups: text only (TO), text+table (TT), text+graph (TG), and tailored (based on the RN's GL score). We hypothesized that the tailored CDS group will have faster CPT (primary) and higher adoption rates (secondary) than the 3 nontailored CDS groups. Eligible RNs employed in an adult hospital unit within the past 2 years were recruited randomly from 10 State Board of Nursing lists representing the 5 regions of the United States (Northeast, Southeast, Midwest, Southwest, and West) to participate in a randomized controlled trial. RNs were randomly assigned to 1 of 4 CDS format groups-TO, TT, TG, and tailored (based on the RN's GL score)-and interacted with the intervention on their PCs. Regression analysis was performed to estimate the effect of tailoring and the association between CPT and RN characteristics. The differences between the tailored (n=46) and nontailored (TO, n=55; TT, n=54; and TG, n=48) CDS groups were not significant for either the CPT or the CDS adoption rate. RNs with low GL had longer CPT interacting with the TG CDS format than the TO CDS format (P=.01). The CPT in the TG CDS format was associated with age (P=.02), GL (P=.02), and comfort with EHRs (P=.047). Comfort with EHRs was also associated with CPT in the TT CDS format (P<.001). Although tailoring based on GL did not improve CPT or adoption, the study reinforced previous pilot findings that low GL is associated with longer CPT when graphs were included in care planning CDS. Higher GL, younger age, and comfort with EHRs were associated with shorter CPT. These findings are robust based on our new innovative testing strategy in which a diverse national sample of RN participants (randomly derived from 10 State Board of Nursing lists) interacted on the web with the intervention on their PCs. Future studies applying our innovative methodology are recommended to cost-effectively enhance the understanding of how the RN's GL, combined with additional factors, can inform the development of efficient CDS for care planning and other EHR components before use in practice.

Tibetan text classification based on graph convolutional network

In this study of Tibetan text classification based on graph convolutional networks, the text graph is first constructed for the Tibetan news corpus based on word-word relations and word-document relations using the constructive relations of Tibetan words and documents, then initialized using the unique thermal representations of words and documents. The embeddings of words and documents are learned using graph convolutional networks under the supervision of the training set of documents, and finally, the text classification problem is transformed into a node classification problem. The graph convolutional network achieves an accuracy of 68.45 % on the Tibetan text classification task, which is higher than other baseline models. In this paper, two comparative experiments are designed to illustrate the effectiveness of graph convolutional networks on Tibetan text classification, which helps people to solve the problem of confusing information in Tibetan texts.

Maximal-Semantics-Augmented BertGCN for Text Classification

Text classification is an important research work in the field of natural language processing (NLP), and many methods of machine learning and deep learning are widely used in this work. In this paper, we propose a method namely Maximal-Semantics-Augmented BertGCN (MSABertGCN) based on BertGCN that further improves the results of text categorization tasks. In this work, the extended semantic information of text is utilized more effectively by means of text semantic enhancement and graph nodes enhancement while preserving the original text features. Four datasets commonly used in the fields of text classification, namely R8, R52, Ohsumed and MR, were used to verify the validity of the method we proposed. Experimental results show that compared with BertGCN and other baselines, the proposed method MSABertGCN has varying degrees of improvement in the accuracy with respect to the R8, R52, Ohsumed and MR datasets.