Graph-based Text Research Articles

TTG-Text: A Graph-Based Text Representation Framework Enhanced by Typical Testors for Improved Classification

Recent advancements in graph-based text representation, particularly with embedding models and transformers such as BERT, have shown significant potential for enhancing natural language processing (NLP) tasks. However, challenges related to data sparsity and limited interpretability remain, especially when working with small or imbalanced datasets. This paper introduces TTG-Text, a novel framework that strengthens graph-based text representation by integrating typical testors—a symbolic feature selection technique that refines feature importance while reducing dimensionality. Unlike traditional TF-IDF weighting, TTG-Text leverages typical testors to enhance feature relevance within text graphs, resulting in improved model interpretability and performance, particularly for smaller datasets. Our evaluation on a text classification task using a graph convolutional network (GCN) demonstrates that TTG-Text achieves a 95% accuracy rate, surpassing conventional methods and BERT with fewer required training epochs. By combining symbolic algorithms with graph-based models, this hybrid approach offers a more interpretable, efficient, and high-performing solution for complex NLP tasks.

A Text Generation Method Based on a Multimodal Knowledge Graph for Fault Diagnosis of Consumer Electronics

As consumer electronics evolve towards greater intelligence, their automation and complexity also increase, making it difficult for users to diagnose faults when they occur. To address the problem where users, relying solely on their own knowledge, struggle to diagnose faults in consumer electronics promptly and accurately, we propose a multimodal knowledge graph-based text generation method. Our method begins by using deep learning models like the Residual Network (ResNet) and Bidirectional Encoder Representations from Transformers (BERT) to extract features from user-provided fault information, which can include images, text, audio, and even olfactory data. These multimodal features are then combined to form a comprehensive representation. The fused features are fed into a graph convolutional network (GCN) for fault inference, identifying potential fault nodes in the electronics. These fault nodes are subsequently fed into a pre-constructed knowledge graph to determine the final diagnosis. Finally, this information is processed through the Bias-term Fine-tuning (BitFit) enhanced Chinese Pre-trained Transformer (CPT) model, which generates the final fault diagnosis text for the user. The experimental results show that our proposed method achieves a 4.4% improvement over baseline methods, reaching a fault diagnosis accuracy of 98.4%. Our approach effectively leverages multimodal fault information, addressing the challenges users face in diagnosing faults through the integration of graph convolutional network and knowledge graph technologies.

Software bug prediction using graph neural networks and graph-based text representations

While open-source communities keep growing at an impressive pace, the corresponding platforms where software engineers share their work, deliberate about software-related issues, point out undetected bugs and suggest potential features that could be incorporated, usually suffer from information overload. This paper proposes a hybrid approach that builds on the strengths of word embeddings techniques, graph-based representation of textual data and graph neural networks for the prediction of software bugs. Existing approaches aim to improve each of these components individually, thus neglecting structural or semantical underlying information. On the contrary, the approach presented in this paper aims to leverage both types of information by representing each text as a graph and utilizing Graph Attention Networks (GATs). The evaluation carried out in this paper demonstrates an overall improvement for a variety of metrics (i.e., accuracy, precision and recall), in comparison with a plethora of graph-based machine learning models. The experiments took place using four datasets of short-text GitHub and Jira issues, which are publicly available on the kaggle.com and zenodo.com platforms respectively.

Evaluating Graph Attention Networks as an Alternative to Transformers for ABSA Task in Low-Resource Languages

Opinions toward subjects and products hold immense relevance in business to guide decision-making processes. However, due to the increase in user-generated content, manual analysis is unrealistic. Techniques such as Sentiment Analysis are paramount to understanding and quantifying human emotion expressed in text data. Aspect-Based Sentiment Analysis aims to extract aspects from an opinionated text while identifying their underlying sentiment. Graph-based text representations have been shown to bring benefits to this task, as they explicitly represent structural relationships in text. While studies have demonstrated the effectiveness of this representation for Aspect-based Sentiment Analysis using Graph Neural Networks in English, there is only sparse evidence of improvement using these techniques for low-resource languages such as Portuguese. We develop a straightforward Graph Attention Network model for the Aspect-Based Sentiment Analysis task in Brazilian Portuguese. The proposed approach achieves a Balanced Accuracy score of 0.74, yielding competitive results and ranking third place in the ABSAPT competition. Furthermore, by leveraging sparse graph connections our model is less computationally demanding than a Transformer architecture in terms of training and inference.

Graph-based Text Classification by Contrastive Learning with Text-level Graph Augmentation

Text Classification (TC) is a fundamental task in the information retrieval community. Nowadays, the mainstay TC methods are built on the deep neural networks, which can learn much more discriminative text features than the traditional shallow learning methods. Among existing deep TC methods, the ones based on Graph Neural Network (GNN) have attracted more attention due to the superior performance. Technically, the GNN-based TC methods mainly transform the full training dataset to a graph of texts; however, they often neglect the dependency between words, so as to miss potential semantic information of texts, which may be significant to exactly represent them. To solve the aforementioned problem, we generate graphs of words instead, so as to capture the dependency information of words. Specifically, each text is translated into a graph of words, where neighboring words are linked. We learn the node features of words by a GNN-like procedure and then aggregate them as the graph feature to represent the current text. To further improve the text representations, we suggest a contrastive learning regularization term. Specifically, we generate two augmented text graphs for each original text graph, we constrain the representations of the two augmented graphs from the same text close and the ones from different texts far away. We propose various techniques to generate the augmented graphs. Upon those ideas, we develop a novel deep TC model, namely Text-level Graph Networks with Contrastive Learning (TGN cl ). We conduct a number of experiments to evaluate the proposed TGN cl model. The empirical results demonstrate that TGN cl can outperform the existing state-of-the-art TC models.

Development of Text Summarization Method based on Graph Theory and Malatya Centrality Algorithm

With the advancement of the internet, humanity has gained easy access to a plethora of information. However, to access accurate content, numerous texts and sources must be read. These texts often contain repetitive words and sentences. The abundance of information renders reading texts in their entirety inefficient in terms of time and makes finding suitable content challenging. To overcome these difficulties, various methods have been developed in research on automatic summarization. In the literature, there are numerous methods developed for different purposes in text summarization. Nevertheless, text summarization can generally be divided into two distinct categories: extractive and abstractive summarization. Abstractive algorithms tend to create new sentences by learning from the text. However, this approach prolongs the working process due to the learning phase and the generated sentences may not possess absolute accuracy. On the other hand, extractive methods, if unable to generate new sentences, have the ability to provide faster and completely accurate summaries by selecting sentences that already exist in the text. For these reasons, in our study, the aim is to perform text summarization using graph theory and the Malatya Centrality Algorithm. The Malatya Centrality Algorithm offers a polynomial approach to solving Vertex Cover Problems and is regarded as an effective solution method. It is believed that the Malatya Centrality Algorithm will contribute to graph-based text summarization. The implementation has been developed using the Python programming language, and the obtained results have been evaluated.

Intelligent information extraction from government on-site inspection reports of construction projects: A graph-based text mining approach

Government inspection reports detail unsafe acts and conditions that arise on construction sites, especially front-line managers’ non-compliance practices, which are hardly identified during self-inspections. Such information serves as a valuable learning source for better construction management. However, non-compliance issue records in inspection reports are typically stored in unstructured text formats, making data analysis challenging. In response, an intelligent text mining framework integrating graph analysis and visualization is presented. The proposed framework comprises data collection and preprocessing and three levels of text analysis: word, sentence, and document. The main tasks of the word-level analysis include (1) extracting keywords using KeyBERT and (2) identifying non-compliance issue types based on community detection in a keyword co-occurrence graph. The sentence-level analysis is performed to automatically classify text data from inspection reports by determining the degree of similarity between texts and communities and assigning the most similar community to each text. The document-level analysis aims to identify the interrelations between various non-compliance issues through association rule mining and a community interaction network. The framework is validated by a total of 6,153 text data featuring non-compliance issues from 322 government on-site inspection reports in Shanghai, China. The results demonstrate that the critical word-level features of non-compliance issues can be accurately identified using KeyBert, which outperforms other state-of-the-art methods. Our approach can also automate the development of a data-driven taxonomy for non-compliance issues and the classification of the corresponding records, requiring less manual intervention than conventional text classification models.

Hierarchical graph-based text classification framework with contextual node embedding and BERT-based dynamic fusion

We propose a novel hierarchical graph-based text classification framework that leverages the power of contextual node embedding and BERT-based dynamic fusion to capture the complex relationships between the nodes in the hierarchical graph and generate a more accurate classification of text. The framework consists of seven stages: Linguistic Feature Extraction, Hierarchical Node Construction with Domain-Specific Knowledge, Contextual Node Embedding, Multi-Level Graph Learning, Dynamic Text Sequential Feature Interaction, Attention-Based Graph Learning, and Dynamic Fusion with BERT. The first stage, Linguistic Feature Extraction, extracts the linguistic features of the text, including part-of-speech tags, dependency parsing, and named entities. The second stage constructs a hierarchical graph based on the domain-specific knowledge, which is used to capture the relationships between nodes in the graph. The third stage, Contextual Node Embedding, generates a vector representation for each node in the hierarchical graph, which captures its local context information, linguistic features, and domain-specific knowledge. The fourth stage, Multi-Level Graph Learning, uses a graph convolutional neural network to learn the hierarchical structure of the graph and extract the features of the nodes in the graph. The fifth stage, Dynamic Text Sequential Feature Interaction, captures the sequential information of the text and generates dynamic features for each node. The sixth stage, Attention-Based Graph earning, uses an attention mechanism to capture the important features of the nodes in the graph. Finally, the seventh stage, Dynamic Fusion with BERT, combines the output from the previous stages with the output from a pre-trained BERT model to obtain the final integrated vector representation of the text. This approach leverages the strengths of both the proposed framework and BERT, allowing for better performance on the classification task. The proposed framework was evaluated on several benchmark datasets and compared to state-of-the-art methods, demonstrating significant improvements in classification accuracy.

GABSA-PT: Graph Neural Networks for Aspect-level Sentiment Analysis in Portuguese Language

Aspect-based Sentiment Analysis is a Natural Language Processing task that aims to extract aspects from an opinionated text and identify the sentiment expressed by the opinion holder towards these aspects. Graph-based text representation has been shown to bring benefits to this task. While studies have demonstrated the effectiveness of this representation for ABSA using Graph Neural Networks in English, more work currently needs to be done evaluating this methodology for Portuguese language. In this article, we adapt a GNN known as DualGCN for sentiment classification using Brazilian Portuguese data. The model created using this approach achieved satisfactory results with a balanced accuracy of 75%

Impact of Similarity Measures in Graph-based Automatic Text Summarization of Konkani Texts

Automatic text summarization is a popular area in Natural Language Processing and Machine Learning. In this work, we adopt a graph-based text summarization approach, using PageRank algorithm, for automatically summarizing Konkani text documents. Konkani is an Indo-Aryan language spoken primarily in the state of Goa, which is on the west coast of India. It is a low-resource language with limited language processing tools. Such tools are readily available in other popular languages of choice for automatic text summarization, like English. The Konkani language dataset used for this purpose is based on Konkani folktales. We examine the impact of various language-independent and language-dependent similarity measures on the construction of the graph. The language-dependent similarity measures use pre-trained fastText word embeddings. A fully connected undirected graph is constructed for each document with the sentences represented as the graph's vertices. The vertices are connected to each other based on how strongly they are related to one another. Thereafter, PageRank algorithm is used for ranking the scores of the vertices. The top-ranking sentences are used to generate the summary. ROUGE toolkit was used for evaluating the quality of these system-generated summaries, and the performance was evaluated against human generated “gold-standard” abstracts and also compared with baselines and benchmark systems. The experimental results show that language-independent similarity measures performed well compared to language-dependent similarity measures despite not using language-specific tools, such as stop-words list, stemming, and word embeddings.

Deep learning, graph-based text representation and classification: a survey, perspectives and challenges

Deep learning, graph-based text representation and classification: a survey, perspectives and challenges

Graph-Based Text Summarization and Its Application on COVID-19 Twitter Data

Large volumes of structured and semi-structured data are being generated every day. Processing this large amount of data and extracting important information is a challenging task. The goal of an automatic text summarization is to preserve the key information and the overall meaning of the article to be summarized. In this paper, a graph-based approach is followed to generate an extractive summary, where sentences of the article are considered as vertices, and weighted edges are introduced based on the cosine similarities among the vertices. A possible subset of maximal independent sets of vertices of the graph is identified with the assumption that adjacent vertices provide sentences with similar information. The degree centrality and clustering coefficient of the vertices are used to compute the score of each of the maximal independent sets. The set with the highest score provides the final summary of the article. The proposed method is evaluated using the benchmark BBC News data to demonstrate its effectiveness and is applied to the COVID-19 Twitter data to express its applicability in topic modeling. Both the application and comparative study with other methods illustrate the efficacy of the proposed methodology.

Multiplex network embedding for implicit sentiment analysis

As one of the hot research directions in natural language processing, sentiment analysis has received continuous and extensive attention. Different from explicit sentiment words indicating sentiment polarity, implicit sentiment analysis is a more challenging problem due to the lack of sentiment words, which makes it inadequate to use traditional sentiment analysis method to judge the polarity of implicit sentiment. This paper takes sentiment analysis as a special sign link prediction problem, which is different from traditional text-based method. In particular, by performing the word graph-based text level information embedding and heterogeneous social network information embedding (i.e. user social relationship network embedding, and user-entity sentiment network embedding), the proposed scheme learns the highly nonlinear representations of network nodes, explores early fusion method to combine the strength of these two types of embedding modeling, optimizes all parameters simultaneously and creates enhanced context representations, leading to better capture of implicit sentiment polarity. The proposed method has been examined on real-world dataset, for implicit sentiment link prediction task. The experimental results demonstrate that the proposed method outperforms state-of-the-art schemes, including LINE, node2vec, and SDNE, by 20.2%, 19.8%, and 14.0%, respectively, on accuracy, and achieves at least 14% gains on AUROC. For sentiment analysis accuracy, the proposed method achieves AUROC of 80.6% and accuracy of 78.3%, which is at least 31% better than other models. This work can provide useful guidance on the implicit sentiment analysis.

Text document clustering using statistical integrated graph based sentence sensitivity ranking algorithm

The proposed methodology employs a novel statistical integrated graph-based sentence sensitivity ranking algorithm for text document clustering. Clustering of documents is a task of grouping a document automatically into a list of meaningful clusters; in order for the documents inside a group to share the same topic. In this paper, first, a novel integrated graph-based methodology using the sentence sensitivity ranking is proposed to extract keyphrases from the documents. In the standard statistical approach, keyphrases are extracted on the basis of the sentence sensitivity ranking; and in the graph-based method, the candidate keyphrases are automatically created as graphs by applying the sentence sensitivity ranking. With the aid of the top listed keyphrases, the documents clustering are carried out by implementing the proposed sentence sensitivity ranking algorithm. The simulation results reveal that the proposed graph-based text document clustering using statistical integrated graph-based sentence sensitivity ranking algorithm obtained the best results for clustering the text documents.

Graph-Based Text Representation and Matching: A Review of the State of the Art and Future Challenges

Graph-based text representation is one of the important preprocessing steps in data and text mining, Natural Language Processing (NLP), and information retrieval approaches. The graph-based methods focus on how to represent text documents in the shape of a graph to exploit the best features of their characteristics. This study reviews and lists the advantages and disadvantages of such methods employed or developed in graph-based text representations. The literature shows that some of the proposed graph-based methods suffer from a lack of representing texts in certain situations. Currently, several techniques are commonly used in graph-based text representation. However, there are still some weaknesses and shortages in these techniques and tools that significantly affect the success of graph representation and graph matching. In this review, we conduct an inclusive survey of the state of the art in graph-based text representation and learning. We provide a formal description of the problem of graph-based text representation and introduce some basic concepts. More significantly, this study proposes a new taxonomy of graph-based text representation, categorizing the existing studies based on representation characteristics and scheme techniques. In terms of the representation scheme taxonomy, we introduce four main types of conceptual graph schemes and summarize the challenges faced in each scheme. The main issues of graph representation, such as research topics and the sub-taxonomy of graph models for web documents, are introduced and categorized. This research also covers some tasks of understanding natural language processing (NLP) that depend on different types of graph structures. In addition, the graph matching taxonomy implements three main categories based on the matching approach, including structural-, semantic-, and similarity-based approaches. Moreover, a deep comparison of these approaches is discussed and reported in terms of methods and tools, the concepts of matching and locality, and the application domains that use these tools. Finally, the paper recommends seven promising future study directions in the graph-based text representation field. These recommendation points are summarized and highlighted as open problems and challenges of graph-based text representation and learning to facilitate and fill the research gaps for scientific researchers in this field.

Leveraging deep graph-based text representation for sentiment polarity applications

Over the last few years, machine learning over graph structures has manifested a significant enhancement in text mining applications such as event detection, opinion mining, and news recommendation. One of the primary challenges in this regard is structuring a graph that encodes and encompasses the features of textual data for the effective machine learning algorithm. Besides, exploration and exploiting of semantic relations is regarded as a principal step in text mining applications. However, most of the traditional text mining methods perform somewhat poor in terms of employing such relations. In this paper, we propose a sentence-level graph-based text representation which includes stop words to consider semantic and term relations. Then, we employ a representation learning approach on the combined graphs of sentences to extract the latent and continuous features of the documents. Eventually, the learned features of the documents are fed into a deep neural network for the sentiment classification task. The experimental results demonstrate that the proposed method substantially outperforms the related sentiment analysis approaches based on several benchmark datasets. Furthermore, our method can be generalized on different datasets without any dependency on pre-trained word embeddings.

SRL-ESA-TextSum: A text summarization approach based on semantic role labeling and explicit semantic analysis

Automatic text summarization attempts to provide an effective solution to today’s unprecedented growth of textual data. This paper proposes an innovative graph-based text summarization framework for generic single and multi document summarization. The summarizer benefits from two well-established text semantic representation techniques; Semantic Role Labelling (SRL) and Explicit Semantic Analysis (ESA) as well as the constantly evolving collective human knowledge in Wikipedia. The SRL is used to achieve sentence semantic parsing whose word tokens are represented as a vector of weighted Wikipedia concepts using ESA method. The essence of the developed framework is to construct a unique concept graph representation underpinned by semantic role-based multi-node (under sentence level) vertices for summarization. We have empirically evaluated the summarization system using the standard publicly available dataset from Document Understanding Conference 2002 (DUC 2002). Experimental results indicate that the proposed summarizer outperforms all state-of-the-art related comparators in the single document summarization based on the ROUGE-1 and ROUGE-2 measures, while also ranking second in the ROUGE-1 and ROUGE-SU4 scores for the multi-document summarization. On the other hand, the testing also demonstrates the scalability of the system, i.e., varying the evaluation data size is shown to have little impact on the summarizer performance, particularly for the single document summarization task. In a nutshell, the findings demonstrate the power of the role-based and vectorial semantic representation when combined with the crowd-sourced knowledge base in Wikipedia.

Graph-Based Text Representation: A Survey of Current Approaches

Lately, we have seen the problem of sparsity data has increased due to the increase in the amount of available documentation on the Internet, to take care of this issue need to choose the best strategy for the representation of the content. In recent years, scientists have been switched to the representation of the content graphically. Because the results of previous studies proved that the represented data as graphs reduce the problem of sparse data. So this study aims to review the sorts of graphs used to represent the content of documents. Were the exploratory outcomes recommended that our methodologies are superior to other methodologies in each of the synthetic global data sets and the real.

Research Trends on Graph-Based Text Mining

Since text mining has been assumed to apply for unformatted text (document), it is necessary to represent text with simplified models. One of the most commonly used models is the vector space model, in which text is represented as a bag of words. Recently, many researches tried to apply a graph-based text model for representing semantic relationships between words. In this paper, we surveyed research trends of graph-based text representation models for text mining. We summarized the models, their features and forecasted further researches.

텍스트 마이닝을 위한 그래프 기반 텍스트 표현 모델의 연구 동향

Text Mining is a research area of retrieving high quality hidden information such as patterns, trends, or distributions through analyzing unformatted text. Basically, since text mining assumes an unstructured text, it needs to be represented as a simple text model for analyzing it. So far, most frequently used model is VSM(Vector Space Model), in which a text is represented as a bag of words. However, recently much researches tried to apply a graph-based text model for representing semantic relationships between words. In this paper, we survey research trends of graph-based text representation models for text mining. Additionally, we also discuss about future models of graph-based text mining.