Graph Construction Research Articles

The effect of the level of data presentation on visual analysts' decisions.

Single-case design (SCD) is frequently utilized in research and applied settings to evaluate the effect of an intervention over time. Once collected, single-case data are typically graphed and analyzed visually in both research and practice contexts. Despite the ubiquity of visual analysis in SCD, this analytic framework has often been critiqued due to findings of limited reliability across visual analysts. Recent research has identified that the way a graph is constructed may contribute to the limitations of visual analysis. The present study sought to evaluate the effect of visually representing multiple measurement occasions as a single data point (e.g., combining measurements taken daily into a weekly composite data point) on visual analysts' decisions regarding the magnitude of an intervention effect. Eleven participants viewed identical data sets, plotted to show different numbers of measurement occasions combined as a single data point, and provided ratings regarding the magnitude of intervention effect depicted within the graph. Results indicated a significant main effect, with data sets with higher levels of data combination being rated as demonstrating significantly larger intervention effects. The results of the study provide additional support for standardization of data presentation and graph construction within SCD in both research and practice contexts. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Multi-Site Wind Speed Prediction Based on Graph Embedding and Cyclic Graph Isomorphism Network (GIN-GRU)

Accurate and reliable wind speed prediction is conducive to improving the power generation efficiency of electrical systems. Due to the lack of adequate consideration of spatial feature extraction, the existing wind speed prediction models have certain limitations in capturing the rich neighborhood information of multiple sites. To address the previously mentioned constraints, our study introduces a graph isomorphism-based gated recurrent unit (GIN-GRU). Initially, the model utilizes a hybrid mechanism of random forest and principal component analysis (PCA-RF) to discuss the feature data from different sites. This process not only preserves the primary features but also extracts critical information by performing dimensionality reduction on the residual features. Subsequently, the model constructs graph networks by integrating graph embedding techniques with the Mahalanobis distance metric to synthesize the correlation information among features from multiple sites. This approach effectively consolidates the interrelated feature data and captures the complex interactions across multiple sites. Ultimately, the graph isomorphism network (GIN) delves into the intrinsic relationships within the graph networks and the gated recurrent unit (GRU) integrates these relationships with temporal correlations to address the challenges of wind speed prediction effectively. The experiments conducted on wind farm datasets for offshore California in 2019 have demonstrated that the proposed model has higher prediction accuracy compared to the comparative model such as CNN-LSTM and GAT-LSTM. Specifically, by modifying the network layers, we achieved higher precision, with the mean square error (MSE) and root mean square error (RMSE) of wind speed at a height of 10 m being 0.8457 m/s and 0.9196 m/s, respectively.

TransGINmer: Identifying viral sequences from metagenomes with self-attention and Graph Isomorphism Network

Viruses, abundant across diverse environments, play pivotal roles in microbial ecosystems and impact human health. Traditional virus studies are limited by their reliance on culture cultivation, which has been mitigated by metagenomics. It obtains nucleotide sequences of all microorganisms from the environment samples through the next-generation sequencing technology. This advancement prompts the need for efficient viral identification methods. To identify viruses accurately and quickly, We propose TransGINmer, a novel deep learning model to identify viral sequences directly from metagenomes. It encodes sequences by a k-mer frequency embedding model, constructs graphs from significant codon token correlations, and classifies them using graph isomorphism neural networks. In comparative tests against some SOTA methods DeepVirFinder, VirSorter2 and PhaMer on the testing dataset, the Amazon River dataset, the Sharon dataset and the CAMI Strain dataset, TransGINmer demonstrates superior accuracy, sensitivity, specificity, and AUC values, showcasing its potential as a robust tool for viral identification from metagenomes. TransGINmer is freely available at Github (https://github.com/xizhilangcc/TransGINmer).

Fd-CasBGRel: A Joint Entity–Relationship Extraction Model for Aquatic Disease Domains

Entity–relationship extraction plays a pivotal role in the construction of domain knowledge graphs. For the aquatic disease domain, however, this relationship extraction is a formidable task because of overlapping relationships, data specialization, limited feature fusion, and imbalanced data samples, which significantly weaken the extraction’s performance. To tackle these challenges, this study leverages published books and aquatic disease websites as data sources to compile a text corpus, establish datasets, and then propose the Fd-CasBGRel model specifically tailored to the aquatic disease domain. The model uses the Casrel cascading binary tagging framework to address relationship overlap; utilizes task fine-tuning for better performance on aquatic disease data; trains on specialized aquatic disease corpora to improve adaptability; and integrates the BRC feature fusion module—which incorporates self-attention mechanisms, BiLSTM, relative position encoding, and conditional layer normalization—to leverage entity position and context for enhanced fusion. Further, it replaces the traditional cross-entropy loss function with the GHM loss function to mitigate category imbalance issues. The experimental results indicate that the F1 score of the Fd-CasBGRel on the aquatic disease dataset reached 84.71%, significantly outperforming several benchmark models. This model effectively addresses the challenges of ternary extraction’s low performance caused by high data specialization, insufficient feature integration, and data imbalances. The model achieved the highest F1 score of 86.52% on the overlapping relationship category dataset, demonstrating its robust capability in extracting overlapping data. Furthermore, We also conducted comparative experiments on the publicly available dataset WebNLG, and the model in this paper obtained the best performance metrics compared to the rest of the comparative models, indicating that the model has good generalization ability.

Three-dimensional vascular graph construction from depth information of blood vessel centerlines in optical coherence tomography angiography

Optical coherence tomography angiography (OCTA) is a noninvasive imaging technique for visualizing high-resolution volumetric vascular networks. Although OCTA has been widely employed in vascular network analysis, most studies have been limited to the analysis of two-dimensional (2D) en-face projection images because leveraging the full potential of OCTA’s three-dimensional (3D) information has been challenging due to projection artifacts beneath blood vessels. In this study, we propose a semi-automatic method for constructing a 3D vascular graph from 3D OCTA images without relying on data-driven learning strategies such as deep learning. The proposed method estimates the depth information of blood vessel centerlines in 2D en-face images and constructs a 3D vascular graph by integrating the depth estimation results for all vessel centerline segments. We demonstrate the effectiveness of the proposed method through experiments conducted on both simulated datasets and real datasets acquired from the dorsal dermis of mice.

Text summarization based on semantic graphs: an abstract meaning representation graph-to-text deep learning approach

Nowadays, due to the constantly growing amount of textual information, automatic text summarization constitutes an important research area in natural language processing. In this work, we present a novel framework that combines semantic graph representations along with deep learning predictions to generate abstractive summaries of single documents, in an effort to utilize a semantic representation of the unstructured textual content in a machine-readable, structured, and concise manner. The overall framework is based on a well defined methodology for performing semantic graph parsing, graph construction, graph transformations for machine learning models and deep learning predictions. The employed semantic graph representation focuses on using the model of abstract meaning representation. Several combinations of graph construction and graph transformation methods are investigated to specify the most efficient of them for the machine learning models. Additionally, a range of deep learning architectures is examined, including a sequence-to-sequence attentive network, reinforcement learning, transformer-based architectures, and pre-trained neural language models. In this direction, a semantic graph representation of an original text is extracted, and then the present framework formulates the problem as a graph-to-summary learning problem to predict a summary of an original text. To the best of our knowledge, this formulation of graph-to-summary predictions in abstractive text summarization, without other intermediate steps in the machine learning phase, has not been presented in the relevant literature. Another important contribution is an introduction of a measure for assessing the factual consistency of the generated summaries in an effort to provide a qualitative evaluation. To assess the framework, an extensive experimental procedure is presented that uses popular datasets to evaluate key aspects of the proposed approach. The obtained results exhibit promising performance, validating the robustness of the proposed framework.

Multi-Level Attention with 2D Table-Filling for Joint Entity-Relation Extraction

Joint entity-relation extraction is a fundamental task in the construction of large-scale knowledge graphs. This task relies not only on the semantics of the text span but also on its intricate connections, including classification and structural details that most previous models overlook. In this paper, we propose the incorporation of this information into the learning process. Specifically, we design a novel two-dimensional word-pair tagging method to define the task of entity and relation extraction. This allows type markers to focus on text tokens, gathering information for their corresponding spans. Additionally, we introduce a multi-level attention neural network to enhance its capacity to perceive structure-aware features. Our experiments show that our approach can overcome the limitations of earlier tagging methods and yield more accurate results. We evaluate our model using three different datasets: SciERC, ADE, and CoNLL04. Our model demonstrates competitive performance compared to the state-of-the-art, surpassing other approaches across the majority of evaluated metrics.

LLM-TIKG: Threat intelligence knowledge graph construction utilizing large language model

Open-source threat intelligence is often unstructured and cannot be directly applied to the next detection and defense. By constructing a knowledge graph through open-source threat intelligence, we can better apply this information to intrusion detection. However, the current methods for constructing knowledge graphs face limitations due to the domain-specific attributes of entities and the analysis of lengthy texts, and they require large amounts of labeled data. Furthermore, there is a lack of authoritative open-source annotated threat intelligence datasets, which require significant manual effort. Moreover, it is noteworthy that current research often neglects the textual descriptions of attack behaviors, resulting in the loss of vital information to understand intricate cyber threats. To address these issues, we propose LLM-TIKG that applies the large language model to construct a knowledge graph from unstructured open-source threat intelligence. The few-shot learning capability of GPT is leveraged to achieve data annotation and augmentation, thereby creating the datasets for fine-tuning a smaller language model (7B). Using the fine-tuned model, we perform topic classification on the collected reports, extract entities and relationships, and extract TTPs from the attack description. This process results in the construction of a threat intelligence knowledge graph, enabling automated and universal analysis of textualized threat intelligence. The experimental results demonstrate improved performance in both named entity recognition and TTP classification, achieving the precision of 87.88% and 96.53%, respectively.

A two-stage importance-aware subgraph convolutional network based on multi-source sensors for cross-domain fault diagnosis

Graph convolutional networks (GCNs) as the emerging neural networks have shown great success in Prognostics and Health Management because they can not only extract node features but can also mine relationship between nodes in the graph data. However, the most existing GCNs-based methods are still limited by graph quality, variable working conditions, and limited data, making them difficult to obtain remarkable performance. Therefore, it is proposed in this paper a two stage importance-aware subgraph convolutional network based on multi-source sensors named I2SGCN to address the above-mentioned limitations. In the real-world scenarios, it is found that the diagnostic performance of the most existing GCNs is commonly bounded by the graph quality because it is hard to get high quality through a single sensor. Therefore, we leveraged multi-source sensors to construct graphs that contain more fault-based information of mechanical equipment. Then, we discovered that unsupervised domain adaptation (UDA) methods only use single stage to achieve cross-domain fault diagnosis and ignore more refined feature extraction, which can make the representations contained in the features inadequate. Hence, it is proposed the two-stage fault diagnosis in the whole framework to achieve UDA. In the first stage, the multiple-instance learning is adopted to obtain the importance factor of each sensor towards preliminary fault diagnosis. In the second stage, it is proposed I2SGCN to achieve refined cross-domain fault diagnosis. Moreover, we observed that deficient and limited data may cause label bias and biased training, leading to reduced generalization capacity of the proposed method. Therefore, we constructed the feature-based graph and importance-based graph to jointly mine more effective relationship and then presented a subgraph learning strategy, which not only enriches sufficient and complementary features but also regularizes the training. Comprehensive experiments conducted on four case studies demonstrate the effectiveness and superiority of the proposed method for cross-domain fault diagnosis, which outperforms the state-of-the art methods.

Towards discovery: an end-to-end system for uncovering novel biomedical relations.

Biomedical relation extraction is an ongoing challenge within the natural language processing community. Its application is important for understanding scientific biomedical literature, with many use cases, such as drug discovery, precision medicine, disease diagnosis, treatment optimization and biomedical knowledge graph construction. Therefore, the development of a tool capable of effectively addressing this task holds the potential to improve knowledge discovery by automating the extraction of relations from research manuscripts. The first track in the BioCreative VIII competition extended the scope of this challenge by introducing the detection of novel relations within the literature. This paper describes that our participation system initially focused on jointly extracting and classifying novel relations between biomedical entities. We then describe our subsequent advancement to an end-to-end model. Specifically, we enhanced our initial system by incorporating it into a cascading pipeline that includes a tagger and linker module. This integration enables the comprehensive extraction of relations and classification of their novelty directly from raw text. Our experiments yielded promising results, and our tagger module managed to attain state-of-the-art named entity recognition performance, with a micro F1-score of 90.24, while our end-to-end system achieved a competitive novelty F1-score of 24.59. The code to run our system is publicly available at https://github.com/ieeta-pt/BioNExt. Database URL: https://github.com/ieeta-pt/BioNExt.

PatchSkip: A lightweight technique for effectively alleviating over-smoothing in vision transformers

Recently vision transformers (ViTs) have encountered the over-smoothing problem, which reduces their capacity by mapping input patches into a similar latent representation. Existing methods introduce regularization terms to alleviate over-smoothing but often increase computational costs. To address this, this paper proposes PatchSkip, a novel and flexible dropout variant, alleviating the over-smoothing problem of ViTs in a lightweight manner. Specifically, PatchSkip draws inspiration from the fact that a similar over-smoothing problem in GNNs is primarily caused by static adjacent matrices leading to solitary message passing mode between nodes. PatchSkip constructs graphs with patch embeddings and analyzes the adjacent matrices in ViTs. By randomly selecting specific patch embeddings to bypass transformer blocks, PatchSkip is proved to generate various adjacent matrices and acts as a multi-mode message passing engine, providing diverse modes of message passing between patches. The effectiveness of PatchSkip in preventing over-smoothing is demonstrated through theoretical proofs and empirical visualizations. Furthermore, PatchSkip is evaluated on various datasets and backbones, showing significant performance improvements while reducing computational costs. For example, when trained on Tiny-ImageNet from scratch, PatchSkip improves the performance of the vanilla CrossViT by 3.85% while reducing computational costs by over 20%.

A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automation

AbstractMaintenance manuals are crucial information sources for maintenance and repair. Prior studies explored factual knowledge extraction from textual documents. However, maintenance knowledge in manuals is more task‐centric rather than factual knowledge and often documented in an unstructured Portable Document Format (PDF), posing challenges for knowledge extraction. Addressing this, this research develops effective methods to extract task‐centric maintenance knowledge from unstructured PDF manuals. A new Task‐centric Knowledge Graph (TCKG) schema centralized on maintenance task components (MTCs) is proposed to address the need for structured knowledge representation. A method (Heterogeneous Graph‐based Method, HGM) for knowledge extraction is then proposed, which is enhanced by incorporating visual and spatial information. In the experiments, the proposed HGM exhibits robust performance in the knowledge extraction process, surpassing the baseline Graph‐based Interaction Model with a Tracker (GIT) method in MTCs extraction by 13.3%, and the baseline Translate Embedding (TransE) method in MTCs' relation extraction by 3.8%. A series of ablation studies also prove that including visual and spatial information through the proposed method can improve the relation extraction performance by over 10%. This research supplies valuable insights for future developments in information extraction from maintenance manuals.

Towards digital-twin-enabled facility management: the natural language processing model for managing facilities in buildings

ABSTRACT Facility management is an essential part of the building which encompasses multiple service areas and processes to ensuring optimum functionality of the facility. Digital twin (DT) has been increasingly introduced to the FM domain to facilitate FM tasks such as decision-making. Despite the advancements in DT-enabled FM, there still exists a problem in the disconnections between DT academics and FM practitioners. The emergence of knowledge management can act as a bridge between academics and FM practitioners. To mine knowledge in the DT-enabled FM industry for a better communication between different stakeholders and timely decisions in the case of emergency, this study proposes a natural language processing (NLP)-based approach. A detailed classification of FM-related entities is first presented, which divides the FM into five categories. The NLP model is used to extract entities and knowledge from a large volume of text data. The results indicate that an NLP-based approach can accurately extract FM-related knowledge. The findings of this study not only provide a basis for automatic DT-enabled FM-related knowledge extraction but also support the downstream tasks such as knowledge graph construction and FM-related emergency decision-making.

Automatic construction and optimization method of enterprise data asset knowledge graph based on graph attention network

Traditional knowledge graph construction methods often rely on a large amount of human intervention and specialized knowledge, which seems inefficient and error-prone when dealing with massive, multi-source, and heterogeneous data assets of enterprises. To address this issue, a Multi Neighborhood Awareness Network (MNAN) model was proposed, which combines a multi language pre training model based on Bidirectional Encoder Representations from Transformers (BERT) and a Graph Convolutional Neural Network (GCN). By introducing an improved attention mechanism to deeply explore the neighborhood characteristics of entities, the accuracy and efficiency of entity matching are enhanced. In addition, to improve the integrity of the knowledge graph, a dynamic graph attention network (RDGAT) model based on graph attention network fusion relationship features is proposed. Through dynamic attention mechanism and relationship fusion strategy, deep learning of relationship features is carried out to optimize the completion performance of the knowledge graph spectrum. The performance test results show that the Hits@1 metric of the multi-neighborhood perceptual network model is close to 90% as the proportion of aligned entity seeds increases to 50%; the average inverse rank of the dynamic graph attention network on the test dataset improves by 6.3%. The experimental results show that the research-proposed knowledge graph fusion and complementation model exhibits significant effectiveness in automatically identifying and integrating large-scale, multi-source heterogeneous data at the enterprise level, and can significantly improve the automation level and accuracy of knowledge graph construction. The research method has achieved significant results in the data asset knowledge graph of petroleum enterprises and other fields, providing strong support for applications such as intelligent search, recommendation systems, and semantic analysis.

Knowledge graph construction for heart failure using large language models with prompt engineering.

Constructing an accurate and comprehensive knowledge graph of specific diseases is critical for practical clinical disease diagnosis and treatment, reasoning and decision support, rehabilitation, and health management. For knowledge graph construction tasks (such as named entity recognition, relation extraction), classical BERT-based methods require a large amount of training data to ensure model performance. However, real-world medical annotation data, especially disease-specific annotation samples, are very limited. In addition, existing models do not perform well in recognizing out-of-distribution entities and relations that are not seen in the training phase. In this study, we present a novel and practical pipeline for constructing a heart failure knowledge graph using large language models and medical expert refinement. We apply prompt engineering to the three phases of schema design: schema design, information extraction, and knowledge completion. The best performance is achieved by designing task-specific prompt templates combined with the TwoStepChat approach. Experiments on two datasets show that the TwoStepChat method outperforms the Vanillia prompt and outperforms the fine-tuned BERT-based baselines. Moreover, our method saves 65% of the time compared to manual annotation and is better suited to extract the out-of-distribution information in the real world.

TVGCN: Time-varying graph convolutional networks for multivariate and multifeature spatiotemporal series prediction.

Spatiotemporal (ST) graph modeling has garnered increasing attention recently. Most existing methods rely on a predefined graph structure or construct a single learnable graph throughout training. However, it is challenging to use a predefined graph structure to capture dynamic ST changes effectively due to evolving node relationships over time. Furthermore, these methods typically utilize only the original data, neglecting external temporal factors. Therefore, we put forward a novel time-varying graph convolutional network model that integrates external factors for multifeature ST series prediction. Firstly, we construct a time-varying adjacency matrix using attention to capture dynamic spatial relationships among nodes. The graph structure adapts over time during training, validation, and testing phases. Then, we model temporal dependence by dilated causal convolution, leveraging gated activation unit and residual connection. Notably, the prediction accuracy is enhanced through the incorporation of embedding absolute time and the fusion of multifeature. This model has been applied to three real-world multifeature datasets, achieving state-of-the-art performance in all cases. Experiments show that the method has high accuracy and robustness when applied to multifeature and multivariate ST series problems.

Graph neural networks for text classification: a survey

Text Classification is the most essential and fundamental problem in Natural Language Processing. While numerous recent text classification models applied the sequential deep learning technique, graph neural network-based models can directly deal with complex structured text data and exploit global information. Many real text classification applications can be naturally cast into a graph, which captures words, documents, and corpus global features. In this survey, we bring the coverage of methods up to 2023, including corpus-level and document-level graph neural networks. We discuss each of these methods in detail, dealing with the graph construction mechanisms and the graph-based learning process. As well as the technological survey, we look at issues behind and future directions addressed in text classification using graph neural networks. We also cover datasets, evaluation metrics, and experiment design and present a summary of published performance on the publicly available benchmarks. Note that we present a comprehensive comparison between different techniques and identify the pros and cons of various evaluation metrics in this survey.

Named Entity Recognition for Chinese Texts on Marine Coral Reef Ecosystems Based on the BERT-BiGRU-Att-CRF Model

In addressing the challenges of non-standardization and limited annotation resources in Chinese marine domain texts, particularly with complex entities like long and nested entities in coral reef ecosystem-related texts, existing Named Entity Recognition (NER) methods often fail to capture deep semantic features, leading to inefficiencies and inaccuracies. This study introduces a deep learning model that integrates Bidirectional Encoder Representations from Transformers (BERT), Bidirectional Gated Recurrent Units (BiGRU), and Conditional Random Fields (CRF), enhanced by an attention mechanism, to improve the recognition of complex entity structures. The model utilizes BERT to capture context-relevant character vectors, employs BiGRU to extract global semantic features, incorporates an attention mechanism to focus on key information, and uses CRF to produce optimized label sequences. We constructed a specialized coral reef ecosystem corpus to evaluate the model’s performance through a series of experiments. The results demonstrated that our model achieved an F1 score of 86.54%, significantly outperforming existing methods. The contributions of this research are threefold: (1) We designed an efficient named entity recognition framework for marine domain texts, improving the recognition of long and nested entities. (2) By introducing the attention mechanism, we enhanced the model’s ability to recognize complex entity structures in coral reef ecosystem texts. (3) This work offers new tools and perspectives for marine domain knowledge graph construction and study, laying a foundation for future research. These advancements propel the development of marine domain text analysis technology and provide valuable references for related research fields.

Evaluation of Impact of UJ-Math Graphing Tool (UJ-MaGT) on Secondary School Students’ Graphing Ability

In recent years, there has been renewed call and effort by Nigerian government to encourage the enrollment of pupils into science, technology, engineering and mathematics; (STEM) subjects. Graph construction is a veritable and indispensable skill required by anyone that engages in a STEM subject. Yet, data from the West African senior school certificate exams (WASSCE) – physics practical – chief examiner’s annual report has revealed serious deficiency in graph construction by secondary school leavers. In an attempt to solve this problem, an app called UJ-Math Graphing Tool (UJ-MaGT) was developed with the aim of incorporating its algorithm in a scientific calculator. The technology was evaluated among 903 senior secondary school students within Jos. Pre-test and post-test were conducted and achievement scores were obtained by grading the test scripts in accordance with West African Examination Council’s (WAEC) marking scheme. Analysis of variance (ANOVA) was carried out using SPSS software. Results show that there was no statistical difference (P>0.05) amongst the groups before the intervention, the null hypothesis “There is no significant difference in achievement score and gap before intervention of UJ-Math Graphing Tool (UJ-MaGT) in graph plotting” was retained. Furthermore, the null hypothesis “There is no significant difference in achievement score and gap after intervention of UJ-Math Graphing Tool (UJ-MaGT) in graph plotting” was rejected because there exist a statistical significant difference (P<0.05) amongst the groups after the intervention. It is therefore concluded that the technological intervention has significant impact on the performance of the students in graph construction.

Deep Learning-Based Detection of Android Malware using Graph Convolutional Networks (GCN)

The study is centered around identifying Android malware using deep learning methods through Graph Neural Networks (GNNs) and Graph Convolutional Networks (GCNs). With Android being widely used worldwide ensuring the security of released applications poses a challenge. Conventional malware detection techniques, like dynamic analysis have limitations in recognizing new malware types leading to a shift towards machine learning and deep learning solutions. The research introduces a malware detection system that employs GNNs particularly focusing on GCNs to analyze the relationships within an applications code by transforming APK files into graph formats. The system follows stages including data gathering, feature extraction, graph construction, model training and implementation. By concentrating on function call graphs the system proves effective in identifying software surpassing traditional machine learning methods in terms of accuracy, precision, recall and F1 score. The GCN based model shows enhancements over approaches with an accuracy rate of 95% compared to 89%, for traditional machine learning models. This progress highlights the potential of learning techniques in bolstering Android security. The system excels not in identifying software but also proves versatile, for different uses like screening apps, in stores and functioning as a standalone antivirus program.