Graph Representation Research Articles

Unsupervised multi-view graph representation learning with dual weight-net

Unsupervised multi-view graph representation learning (UMGRL) aims to capture the complex relationships in the multi-view graph without human annotations, so it has been widely applied in real-world applications. However, existing UMGRL methods still face the issues as follows: (i) Previous UMGRL methods tend to overlook the importance of nodes with different influences and the importance of graphs with different relationships, so that they may lose discriminative information in nodes with large influences and graphs with important relationships. (ii) Previous UMGRL methods generally ignore the heterophilic edges in the multi-view graph to possibly introduce noise from different classes into node representations. To address these issues, we propose a novel bi-level optimization UMGRL framework with dual weight-net. Specifically, the lower-level optimizes the parameters of encoders to obtain node representations of different graphs, while the upper-level optimizes the parameters of the dual weight-net to adaptively and dynamically capture the importance of node level, graph level, and edge level, thus obtaining discriminative fused representations for downstream tasks. Moreover, theoretical analysis demonstrates that the proposed method shows a better generalization ability on downstream tasks, compared to previous UMGRL methods. Extensive experimental results verify the effectiveness of the proposed method on public datasets, in terms of different downstream tasks, compared to numerous comparison methods.

Estimating the Wind Power by Using K-Nearest Neighbors (KNN) and Python

In recent years the production and distribution of renewable energy has become a popular trend in power generation, and renewable energy sources such as hydro, solar, and wind energy are examples of natural and clean energy sources. Among the renewable energy sources, wind energy source is one of the trending and crucial energy sources that has outstanding electricity power generation. This research investigates an estimation model of wind power to address the real-life issues in renewable energy power generation by estimating an accurate amount of wind power production by applying the machine learning (ML) K-Nearest Neighbors (KNN) technique integrated with Python programming. To achieve the expected outcome for this research, it is expected to collect data from wind energy, wind speed forecasts, and all the relevant information that supports the research in analyzing the technique. There are various meteorological methods to evaluate the performance of wind power generation, but in this research, the methodology is to employ machine learning techniques such as KNN integrated with Python coding to achieve estimated wind energy output. Python is a software program and is diversely used for coding various algorithmic formulas to obtain results. In this research, Python is best suited for investigating the performance of the KNN technique for estimating the most accurate result for wind power generation. Here, the report presents the methods of applying the KNN technique and displaying the outcome on screen by a graph representation. The outcome of the research showed that the KNN is a more reliable technique to make an accurate prediction on wind power generation. A similar approach can be implemented in various renewable energy data to estimate an accurate energy output.

PyJedAI: A Library with Resolution-Related Structures and Procedures for Products

This work presents an open-source Python library, named pyJedAI, which provides functionalities supporting the creation of algorithms related to product entity resolution. Building over existing state-of-the-art resolution algorithms, the tool offers a plethora of important tasks required for processing product data collections. It can be easily used by researchers and practitioners for creating algorithms analyzing products, such as real-time ad bidding, sponsored search, or pricing determination. In essence, it allows users to easily import product data from the possible sources, compare products in order to detect either similar or identical products, generate a graph representation using the products and desired relationships, and either visualize or export the outcome in various forms. Our experimental evaluation on data from well-known online retailers illustrates high accuracy and low execution time for the supported tasks. To the best of our knowledge, this is the first Python package to focus on product entities and provide this range of product entity resolution functionalities. History: Accepted by Ted Ralphs, Area Editor for Software Tools. Funding: This was partially funded by the EU project STELAR (Horizon Europe) [Grant 101070122]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.0410 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2023.0410 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Code and Data Repository for pyJedAI: A Library with Resolution-Related Structures and Procedures for Products

This work presents an open-source Python library, named pyJedAI, which provides functionalities supporting the creation of algorithms related to Product Entity Resolution. Building over existing state-of-the-art resolution algorithms (Papadakis et al. 2021a), the tool offers a plethora of important tasks required for processing product data collections. It be can easily used by researchers and practitioners for creating algorithms analyzing products, such as real-time ads bidding, sponsored search, or pricing determination. In essence, it allows to easily import product data from the possible sources, compare products in order to detect either similar or identical products, generate a graph representation using the products and desired relationships, and either visualize or export the outcome in various forms. Our experimental evaluation on data from well-known online retailers illustrates high accuracy and low execution time for the supported tasks. To the best of our knowledge this is the first Python package to focus on product entities and provide this range of Product Entity Resolution functionalities.

Reinforcement Learning-Based Network Dismantling by Targeting Maximum-Degree Nodes in the Giant Connected Component

Tackling the intricacies of network dismantling in complex systems poses significant challenges. This task has relevance across various practical domains, yet traditional approaches focus primarily on singular metrics, such as the number of nodes in the Giant Connected Component (GCC) or the average pairwise connectivity. In contrast, we propose a unique metric that concurrently targets nodes with the highest degree and reduces the GCC size. Given the NP-hard nature of optimizing this metric, we introduce MaxShot, an innovative end-to-end solution that leverages graph representation learning and reinforcement learning. Through comprehensive evaluations on both synthetic and real-world datasets, our method consistently outperforms leading benchmarks in accuracy and efficiency. These results highlight MaxShot’s potential as a superior approach to effectively addressing the network dismantling problem.

Generation and Editing of 2D Shapes using a Branched Representation

In this paper we propose a new planar shape representation, the medial branch graph representation (MBGR) which allows to easily generate, vary and edit all-new sorts of parametric shapes. Each MBGR shape is described by a collection of connected control circles organized as a graph. Each pair of connected circles generates a branch of the shape. The boundaries of the branches are designed using cubic Bézier curves which enable shape representation in a compact SVG format. Each circle is associated with a regularity parameter locally controlling the smoothness of the branch connections. In this way, we can manage shapes having both sharp corners and smooth boundaries. To illustrate the potential of the MBGR representation we have created an online facility ( https://mbgrg.github.io/mbgrg/ ) where the user can create/edit MBGR shapes automatically or interactively.

Survey of source code vulnerability analysis based on deep learning

Amidst the rapid development of the software industry and the burgeoning open-source culture, vulnerability detection within the software security domain has emerged as an ever-expanding area of focus. In recent years, the rapid advancement of artificial intelligence, particularly the notable progress in deep learning for pattern recognition and natural language processing, has catalyzed a surge in research endeavors exploring the integration of deep learning for the enhancement of vulnerability detection techniques. In this paper, we investigate contemporary deep learning-based source code analysis methods, with a concentrated emphasis on those pertaining to static code vulnerability detection. We categorize these methods based on various representations of source code employed during the preprocessing stage, including token-based and graph-based representations of source code, and further subdivided based on the types of deep learning algorithms or graph representations employed. We summarize the basic processes of model training and vulnerability detection under these different representation formats. Furthermore, we explore the limitations inherent in current approaches and provide insights into future trends and challenges for research in this field.

An Explainable Graph Neural Framework to Identify Cancer-Associated Intratumoral Microbial Communities.

Microbes are extensively present among various cancer tissues and play critical roles in carcinogenesis and treatment responses. However, the underlying relationships between intratumoral microbes and tumors remain poorly understood. Here, a MIcrobial Cancer-association Analysis using a Heterogeneous graph transformer (MICAH) to identify intratumoral cancer-associated microbial communities is presented. MICAH integrates metabolic and phylogenetic relationships among microbes into a heterogeneous graph representation. It uses a graph transformer to holistically capture relationships between intratumoral microbes and cancer tissues, which improves the explainability of the associations between identified microbial communities and cancers. MICAH is applied to intratumoral bacterial data across 5 cancer types and 5 fungi datasets, and its generalizability and reproducibility are demonstrated. After experimentally testing a representative observation using a mouse model of tumor-microbe-immune interactions, a result consistent with MICAH's identified relationship is observed. Source tracking analysis reveals that the primary known contributor to a cancer-associated microbial community is the organs affected by the type of cancer. Overall, this graph neural network framework refines the number of microbes that can be used for follow-up experimental validation from thousands to tens, thereby helping to accelerate the understanding of the relationship between tumors and intratumoral microbiomes.

A framework armed with node dynamics for predicting technology convergence

In the rapidly evolving landscape of industrial and societal progress, technology convergence plays a pivotal role. This dynamic process is usually characterized by the emergence of new nodes and new links. With the long-term and recent interests in predicting technology convergence, link prediction has become the primary approach on the basis of large-scale patent data. Though, the problem of node dynamics is still not addressed in the literature. For this purpose, this paper presents a technology convergence prediction framework with three core modules as follows. (1) A candidate node set is introduced during the network construction phase, mimicking the generation of newly-emerging nodes. (2) An inductive graph representation learning approach is deployed to generate feature vectors for newly-emerging nodes as well as existing ones. (3) The evaluation criteria are revised to shift from the predictable range to the actual predicted range, which can provide a more realistic assessment of predictive performance. Finally, experimental results on the domain of cancer drug development validate the feasibility and effectiveness of our framework in capturing the dynamics of technology convergence, especially concerning the relationships of newly emerged nodes and links. This study provides valuable insights into technology convergence dynamics and points to future research and applications.

Analyzing the impact of conversation structure on predicting persuasive comments online

The topic of persuasion in online conversations has social, political and security implications; as a consequence, the problem of predicting persuasive comments in online discussions is receiving increasing attention in the literature. Following recent advancements in graph neural networks, we analyze the impact of conversation structure in predicting persuasive comments in online discussions. We evaluate the performance of artificial intelligence models receiving as input graphs constructed on the top of online conversations sourced from the “Change My View” Reddit channel. We experiment with different graph architectures and compare the performance on graph neural networks, as structure-based models, and dense neural networks as baseline models. Experiments are conducted on two tasks: (1) persuasive comment detection, aiming to predict which comments are persuasive, and (2) influence prediction, aiming to predict which users are persuasive. The experimental results show that the role of the conversation structure in predicting persuasiveness is strongly dependent on its graph representation given as input to the graph neural network. In particular, a graph structure linking only comments belonging to the same speaker in the conversation achieves the best performance in both tasks. This structure outperforms both the baseline model, which does not consider any structural information, and structures linking different speakers’ comments with each other. Specifically, the F1 score of the best performing model is 0.58, which represents an improvement of 5.45% over the baseline model (F1 score of 0.55) and 7.41% over the model linking different speakers’ comments (F1 score of 0.54).

A novel Soft Actor–Critic framework with disjunctive graph embedding and autoencoder mechanism for Job Shop Scheduling Problems

The Job-Shop Scheduling Problem (JSSP) is a well-established and classic NP-hard combinatorial optimization issue. The quality of its scheduling scheme directly affects the operational efficiency of manufacturing systems. Priority Dispatching Rules (PDRs) are often utilized to address JSSP in real-world contexts, but the process of creating effective PDRs can be daunting and time-consuming. It also necessitates comprehensive domain knowledge, typically resulting in mediocre performance. In this paper, we introduce a novel reinforcement learning (RL) model called Disjunctive Graph Embedding with Autoencoder Mechanism for Job Shop Scheduling Problems (DGEAM-JSSP), designed to automate PDRs learning. Our proposed model confronts the issue using a Graph Neural Network (GNN) to learn node features that encapsulate the spatial structure of the JSSP graph representation. The ensuing policy network is size-agnostic, enabling effective generalization on larger-scale instances. Additionally, we employ a transformer encoder, incorporating parallel encoding and a self-attention mechanism, to successfully recognize long-term dependencies among operations in large-scale scheduling problems. We also implemented an end-to-end training approach using the Soft Actor–Critic (SAC) algorithm to instruct the two modules. Computational experiment results reveal that, with a single training, our agent successfully learns a superior dispatching policy, surpassing PDRs and state-of-the-art RL frameworks specifically tailored for each JSSP instance size in solution quality, as well as OR-Tools in execution speed. Moreover, results from random and benchmark instances illustrate that the uniquely-modeled learned policies have impressive generalization performance on real-world instances and significantly larger-scale scenarios involving up to 2000 operations.

Masked contrastive graph representation learning for age estimation

Age estimation of face images is a crucial task with various practical applications in areas such as video surveillance and Internet access control. While deep learning-based age estimation methods like Convolutional Neural Networks (CNN), Multilayer Perceptrons (MLP), and Transformers have shown remarkable performance, they have limitations when modeling complex or irregular objects, and often contain redundant information. Although existing graph neural networks (GNNs) can alleviate this issue, they only considers the structural information of the image and ignores the semantic features, thus the feature representation capability of graph nodes is limited. To address these issues, this paper proposes a novel Masked Contrastive Graph Representation Learning (MCGRL) method for accurate age estimation of face images. Our approach leverages CNN to extract semantic features of the image, which are then partitioned into patches that serve as nodes in the graph. We use a masked graph convolutional network (GCN) to derive image-based node representations that capture rich structural information. Finally, we incorporate multiple losses to explore the complementary relationship between structural information and semantic features, which improves the feature representation capability of GCN. Experimental results on real-world face image datasets demonstrate the superiority of our proposed method over other state-of-the-art age estimation approaches. Our code is available at https://github.com/yuntaoshou/MCGRL

KnotResolver: tracking self-intersecting filaments in microscopy using directed graphs.

Quantification of microscopy time series of in vitro reconstituted motor-driven microtubule transport in "gliding assays" is typically performed using computational object tracking tools. However, these are limited to non-intersecting and rod-like filaments. Here, we describe a novel computational image-analysis pipeline, KnotResolver, to track image time series of highly curved self-intersecting looped filaments (knots) by resolving cross-overs. The code integrates filament segmentation and cross-over or "knot" identification based on directed graph representation, where nodes represent cross-overs and edges represent the path connecting them. The graphs are mapped back to contours and the distance to a reference minimized. The accuracy of contour detection is sub-pixel with a robustness to noise. We demonstrate the utility of KnotResolver by automatically quantifying "flagella-like" curvature dynamics and wave-like oscillations of clamped microtubules in a "gliding assay." The MATLAB-based source code is released as OpenSource and is available at https://github.com/CyCelsLab/MTKnotResolver.

Graph Neural Network-Based Drug Gene Interactions of Wnt/β-Catenin Pathway in Bone Formation.

Introduction The Wnt/β-catenin pathway is crucial for bone formation and remodeling, regulating osteoblast differentiation, bone remodeling, and skeletal homeostasis. Dysregulation of the Wnt/β-catenin pathway is linked to bone-related diseases like osteoporosis, osteoarthritis, and osteosarcoma. The strategies to modulate this pathway include Wnt agonists, inhibitors, and small molecules. Graph neural networks (GNNs) have shown potential in understanding drug-gene interactions, providing accurate predictions, identifying novel drug-target pairs, and enabling personalized drug therapy. So we aim to predict GNN-based drug-gene interactions of Wnt/β-catenin pathway in bone formation. Methodology The drug-gene interactions of Wnt signaling were annotated and preprocessed using Cytoscape, a powerful tool for building drug-gene interactions. Data was imported, nodes representing drugs and genes were created, and edges represented their interactions. GNNswere used to prepare data for nodes, genes, and drugs. GNNsare designed to operate on graph-structured data, capable of learning complex relationships between the nodes. The architecture consists of several steps: graph representation, message passing, node representation update, graph-level readout, and prediction or output. A data representation system is a GNNwith an Adam optimizer, 100 epochs, a learning rate of 0.001, and entropy loss. Results The network has 108 nodes, 134 edges, and 2.444 neighbors, with a diameter of 4, radius of 2, and characteristic path length of 2.635. It lacks clustering, sparse connectivity, wide connection variation, and moderate centralization. The GNN model's drug-gene interactions demonstrate high precision, recall, F1 score, and accuracy, with a high sensitivity to true-positives and low false-negatives. Conclusion The study employs a GNN model to predict drug-gene interactions in the Wnt/β-catenin pathway, demonstrating high precision and accuracy, but further research is needed.

Complex Graph Analysis and Representation Learning: Problems, Techniques, and Applications

Complex Graph Analysis and Representation Learning: Problems, Techniques, and Applications

AutoGMap: Learning to Map Large-Scale Sparse Graphs on Memristive Crossbars.

The sparse representation of graphs has shown great potential for accelerating the computation of graph applications (e.g., social networks and knowledge graphs) on traditional computing architectures (CPU, GPU, or TPU). But, the exploration of large-scale sparse graph computing on processing-in-memory (PIM) platforms (typically with memristive crossbars) is still in its infancy. To implement the computation or storage of large-scale or batch graphs on memristive crossbars, a natural assumption is that a large-scale crossbar is demanded, but with low utilization. Some recent works question this assumption; to avoid the waste of storage and computational resource, the fixed-size or progressively scheduled "block partition" schemes are proposed. However, these methods are coarse-grained or static and are not effectively sparsity-aware. This work proposes the dynamic sparsity-aware mapping scheme generating method that models the problem with a sequential decision-making model, and optimizes it by reinforcement learning (RL) algorithm (REINFORCE). Our generating model [long short-term memory (LSTM), combined with the dynamic-fill scheme] generates remarkable mapping performance on the small-scale graph/matrix data (complete mapping costs 43% area of the original matrix) and two large-scale matrix data (costing 22.5% area on qh882 and 17.1% area on qh1484). Our method may be extended to sparse graph computing on other PIM architectures, not limited to the memristive device-based platforms.

Multi-Domain Based Dynamic Graph Representation Learning for EEG Emotion Recognition.

Graph neural networks (GNNs) have demonstrated efficient processing of graph-structured data, making them a promising method for electroencephalogram (EEG) emotion recognition. However, due to dynamic functional connectivity and nonlinear relationships between brain regions, representing EEG as graph data remains a great challenge. To solve this problem, we proposed a multi-domain based graph representation learning (MD 2GRL) framework to model EEG signals as graph data. Specifically, MD 2GRL leverages gated recurrent units (GRU) and power spectral density (PSD) to construct node features of two subgraphs. Subsequently, the self-attention mechanism is adopted to learn the similarity matrix between nodes and fuse it with the intrinsic spatial matrix of EEG to compute the corresponding adjacency matrix. In addition, we introduced a learnable soft thresholding operator to sparsify the adjacency matrix to reduce noise in the graph structure. In the downstream task, we designed a dual-branch GNN and incorporated spatial asymmetry for graph coarsening. We conducted experiments using the publicly available datasets SEED and DEAP, separately for subject-dependent and subject-independent, to evaluate the performance of our model in emotion classification. Experimental results demonstrated that our method achieved state-of-the-art (SOTA) classification performance in both subject-dependent and subject-independent experiments. Furthermore, the visualization analysis of the learned graph structure reveals EEG channel connections that are significantly related to emotion and suppress irrelevant noise. These findings are consistent with established neuroscience research and demonstrate the potential of our approach in comprehending the neural underpinnings of emotion.

A brain structure learning-guided multi-view graph representation learning for brain network analysis.

Resting-state brain networks represent the interconnectivity of different brain regions during rest. Utilizing brain network analysis methods to model these networks can enhance our understanding of how different brain regions collaborate and communicate without explicit external stimuli. However, analyzing resting-state brain networks faces challenges due to high heterogeneity and noise correlation between subjects. This study proposes a brain structure learning-guided multi-view graph representation learning method to address the limitations of current brain network analysis and improve the diagnostic accuracy (ACC) of mental disorders. We first used multiple thresholds to generate different sparse levels of brain networks. Subsequently, we introduced graph pooling to optimize the brain network representation by reducing noise edges and data inconsistency, thereby providing more reliable input for subsequent graph convolutional networks (GCNs). Following this, we designed a multi-view GCN to comprehensively capture the complexity and variability of brain structure. Finally, we employed an attention-based adaptive module to adjust the contributions of different views, facilitating their fusion. Considering that the Smith atlas offers superior characterization of resting-state brain networks, we utilized the Smith atlas to construct the graph network. Experiments on two mental disorder datasets, the Autism Brain Imaging Data Exchange (ABIDE) dataset and the Mexican Cocaine Use Disorders (SUDMEX CONN) dataset, show that our model outperforms the state-of-the-art methods, achieving nearly 75% ACC and 70% area under the receiver operating characteristic curve (AUC) on both datasets. These findings demonstrate that our method of combining multi-view graph learning and brain structure learning can effectively capture crucial structural information in brain networks while facilitating the acquisition of feature information from diverse perspectives, thereby improving the performance of brain network analysis.

Affinity Uncertainty-Based Hard Negative Mining in Graph Contrastive Learning.

Hard negative mining has shown effective in enhancing self-supervised contrastive learning (CL) on diverse data types, including graph CL (GCL). The existing hardness-aware CL methods typically treat negative instances that are most similar to the anchor instance as hard negatives, which helps improve the CL performance, especially on image data. However, this approach often fails to identify the hard negatives but leads to many false negatives on graph data. This is mainly due to that the learned graph representations are not sufficiently discriminative due to oversmooth representations and/or non-independent and identically distributed (non-i.i.d.) issues in graph data. To tackle this problem, this article proposes a novel approach that builds a discriminative model on collective affinity information (i.e., two sets of pairwise affinities between the negative instances and the anchor instance) to mine hard negatives in GCL. In particular, the proposed approach evaluates how confident/uncertain the discriminative model is about the affinity of each negative instance to an anchor instance to determine its hardness weight relative to the anchor instance. This uncertainty information is then incorporated into the existing GCL loss functions via a weighting term to enhance their performance. The enhanced GCL is theoretically grounded that the resulting GCL loss is equivalent to a triplet loss with an adaptive margin being exponentially proportional to the learned uncertainty of each negative instance. Extensive experiments on ten graph datasets show that our approach does the following: 1) consistently enhances different state-of-the-art (SOTA) GCL methods in both graph and node classification tasks and 2) significantly improves their robustness against adversarial attacks. Code is available at https://github.com/mala-lab/AUGCL.

Graph Representation Learning Based on Specific Subgraphs for Biomedical Interaction Prediction.

Discovering the novel associations of biomedical entities is of great significance and can facilitate not only the identification of network biomarkers of disease but also the search for putative drug targets.Graph representation learning (GRL) has incredible potential to efficiently predict the interactions from biomedical networks by modeling the robust representation for each node.> However, the current GRL-based methods learn the representation of nodes by aggregating the features of their neighbors with equal weights. Furthermore, they also fail to identify which features of higher-order neighbors are integrated into the representation of the central node. In this work, we propose a novel graph representation learning framework: a multi-order graph neural network based on reconstructed specific subgraphs (MGRS) for biomedical interaction prediction. In the MGRS, we apply the multi-order graph aggregation module (MOGA) to learn the wide-view representation by integrating the multi-hop neighbor features. Besides, we propose a subgraph selection module (SGSM) to reconstruct the specific subgraph with adaptive edge weights for each node. SGSM can clearly explore the dependency of the node representation on the neighbor features and learn the subgraph-based representation based on the reconstructed weighted subgraphs. Extensive experimental results on four public biomedical networks demonstrate that the MGRS performs better and is more robust than the latest baselines.