non-Euclidean Data Research Articles

Graph neural networks for multi-view learning: a taxonomic review

With the explosive growth of user-generated content, multi-view learning has become a rapidly growing direction in pattern recognition and data analysis areas. Due to the significant application value of multi-view learning, there has been a continuous emergence of research based on machine learning methods and traditional deep learning paradigms. The core challenge in multi-view learning lies in harnessing both consistent and complementary information to forge a unified, comprehensive representation. However, many multi-view learning tasks are based on graph-structured data, making existing methods unable to effectively mine the information contained in the input multiple data sources. Recently, graph neural networks (GNN) techniques have been widely utilized to deal with non-Euclidean data, such as graphs or manifolds. Thus, it is essential to combine the advantages of the powerful learning capability of GNN models and multi-view data. In this paper, we aim to provide a comprehensive survey of recent research works on GNN-based multi-view learning. In detail, we first provide a taxonomy of GNN-based multi-view learning methods according to the input form of models: multi-relation, multi-attribute and mixed. Then, we introduce the applications of multi-view learning, including recommendation systems, computer vision and so on. Moreover, several public datasets and open-source codes are introduced for implementation. Finally, we analyze the challenges of applying GNN models on various multi-view learning tasks and state new future directions in this field.

A Systematic Review on Graph Neural Network-based Methods for Stock Market Forecasting

Financial technology (FinTech) is a field that uses artificial intelligence to automate financial services. One area of FinTech is stock analysis, which aims to predict future stock prices to develop investment strategies that maximize profits. Traditional methods of stock market prediction, such as time series analysis and machine learning, struggle to handle the non-linear, chaotic, and sudden changes in stock data and may not consider the interdependence between stocks. Recently, graph neural networks (GNNs) have been used in stock market forecasting to improve prediction accuracy by incorporating the interconnectedness of the market. GNNs can process non-Euclidean data in the form of a knowledge graph. However, financial knowledge graphs can have dynamic and complex interactions, which can be challenging for graph modeling technologies. This work presents a systematic review of graph-based approaches for stock market forecasting. This review covers different types of stock analysis tasks (classification, regression, and stock recommendation), a generalized framework for solving these tasks, and a review of various features, datasets, graph models, and evaluation metrics used in the stock market. The results of various studies are analyzed, and future directions for research are highlighted.

Spatial–Temporal-Correlation-Constrained Dynamic Graph Convolutional Network for Traffic Flow Forecasting

Accurate traffic flow prediction in road networks is essential for intelligent transportation systems (ITS). Since traffic data are collected from the road network with spatial topological and time series sequences, the traffic flow prediction is regarded as a spatial–temporal prediction task. With the powerful ability to model the non-Euclidean data, the graph convolutional network (GCN)-based models have become the mainstream framework for traffic forecasting. However, existing GCN-based models either use the manually predefined graph structure to capture the spatial features, ignoring the heterogeneity of road networks, or simply perform 1-D convolution with fixed kernel to capture the temporal dependencies of traffic data, resulting in insufficient long-term temporal feature extraction. To solve those issues, a spatial–temporal correlation constrained dynamic graph convolutional network (STC-DGCN) is proposed for traffic flow forecasting. In STC-DGCN, a spatial–temporal embedding encoder module (STEM) is first constructed to encode the dynamic spatial relationships for road networks at different time steps. Then, a temporal feature encoder module with heterogeneous time series correlation modeling (TFE-HCM) and a spatial feature encoder module with dynamic multi-graph modeling (SFE-DCM) are designed to generate dynamic graph structures for effectively capturing the dynamic spatial and temporal correlations. Finally, a spatial–temporal feature fusion module based on a gating fusion mechanism (STM-GM) is proposed to effectively learn and leverage the inherent spatial–temporal relationships for traffic flow forecasting. Experimental results from three real-world traffic flow datasets demonstrate the superior performance of the proposed STC-DGCN compared with state-of-the-art traffic flow forecasting models.

Diversifying Collaborative Filtering via Graph Spreading Network and Selective Sampling.

Graph neural network (GNN) is a robust model for processing non-Euclidean data, such as graphs, by extracting structural information and learning high-level representations. GNN has achieved state-of-the-art recommendation performance on collaborative filtering (CF) for accuracy. Nevertheless, the diversity of the recommendations has not received good attention. Existing work using GNN for recommendation suffers from the accuracy-diversity dilemma, where slightly increases diversity while accuracy drops significantly. Furthermore, GNN-based recommendation models lack the flexibility to adapt to different scenarios' demands concerning the accuracy-diversity ratio of their recommendation lists. In this work, we endeavor to address the above problems from the perspective of aggregate diversity, which modifies the propagation rule and develops a new sampling strategy. We propose graph spreading network (GSN), a novel model that leverages only neighborhood aggregation for CF. Specifically, GSN learns user and item embeddings by propagating them over the graph structure, utilizing both diversity-oriented and accuracy-oriented aggregations. The final representations are obtained by taking the weighted sum of the embeddings learned at all layers. We also present a new sampling strategy that selects potentially accurate and diverse items as negative samples to assist model training. GSN effectively addresses the accuracy-diversity dilemma and achieves improved diversity while maintaining accuracy with the help of a selective sampler. Moreover, a hyper-parameter in GSN allows for adjustment of the accuracy-diversity ratio of recommendation lists to satisfy the diverse demands. Compared to the state-of-the-art model, GSN improved R @20 by 1.62%, N @20 by 0.67%, G @20 by 3.59%, and E @20 by 4.15% on average over three real-world datasets, verifying the effectiveness of our proposed model in diversifying overall collaborative recommendations.

Bootstrapped Edge Count Tests for Nonparametric Two-Sample Inference Under Heterogeneity

Nonparametric two-sample testing is a classical problem in inferential statistics. While modern two-sample tests, such as the edge count test and its variants, can handle multivariate and non-Euclidean data, contemporary gargantuan datasets often exhibit heterogeneity due to the presence of latent subpopulations. Direct application of these tests, without regulating for such heterogeneity, may lead to incorrect statistical decisions. We develop a new nonparametric testing procedure that accurately detects differences between the two samples in the presence of unknown heterogeneity in the data generation process. Our framework handles this latent heterogeneity through a composite null that entertains the possibility that the two samples arise from a mixture distribution with identical component distributions but with possibly different mixing weights. In this regime, we study the asymptotic behavior of weighted edge count test statistic and show that it can be effectively recalibrated to detect arbitrary deviations from the composite null. For practical implementation we propose a Bootstrapped Weighted Edge Count test which involves a bootstrap-based calibration procedure that can be easily implemented across a wide range of heterogeneous regimes. A comprehensive simulation study and an application to detecting aberrant user behaviors in online games demonstrates the excellent non-asymptotic performance of the proposed test. Supplementary materials for this article are available online.

Application of geometric deep learning in disease-gene relationship prediction

The correlation between diseases and genetic factors represents a pivotal challenge in the biomedical field, often conceptualized as a task of link prediction. Conventional methods employed for prediction, such as statistical models and various machine learning algorithms, frequently fall short in terms of accuracy when confronted with the intricacies of biological network data. These methods also tend to inadequately represent the complex relationships inherent in such data. In contrast, recent advances in geometric deep learning have introduced a powerful tool within artificial intelligence, particularly adept at processing non-Euclidean data structures. This study delves into the potential of leveraging geometric deep learning techniques to enhance the prediction of disease-gene associations. Initially, we conduct a thorough review of existing research related to link prediction, encompassing both traditional approaches and contemporary methods grounded in deep learning. Subsequently, we propose a geometric deep learning framework, incorporating Graph Convolutional Networks (GCN) and Graph Auto-Encoder (GAE), to develop and assess our predictive model. The results of our experiments demonstrate that the proposed geometric deep learning model surpasses conventional techniques in accurately predicting disease-gene associations. In conclusion, we evaluate the implications of our findings, discuss their practical applications in the biomedical domain, and suggest possible avenues for future research.

Conv-RGNN: An efficient Convolutional Residual Graph Neural Network for ECG classification

Background and objective:Electrocardiogram (ECG) analysis is crucial in diagnosing cardiovascular diseases (CVDs). It is important to consider both temporal and spatial features in ECG analysis to improve automated CVDs diagnosis. Significant progress has been made in automated CVDs diagnosis based on ECG with the continuous development of deep learning. Current most researches often treat 12-lead ECG signals as synchronous sequences in Euclidean space, focusing primarily on extracting temporal features while overlooking the spatial relationships among the 12-lead. However, the spatial distribution of 12-lead ECG electrodes can be more naturally represented using non-Euclidean data structures, which makes the relationships among leads more consistent with their intrinsic characteristics. Methods:This study proposes an innovative method, Convolutional Residual Graph Neural Network (Conv-RGNN), for ECG classification. The first step is to segment the 12-lead ECG into twelve single-lead ECG, which are then mapped to nodes in a graph that captures the relationships between the different leads through spatial connections, resulting in the 12-lead ECG graph. The graph is then used as input for Conv-RGNN. A convolutional neural network with a position attention mechanism is used to extract temporal sequence information and selectively integrate contextual information to enhance semantic features at different positions. The spatial features of the 12-lead ECG graph are extracted using the residual graph neural network. Results:The experimental results indicate that Conv-RGNN is highly competitive in two multi-label datasets and one single-label dataset, demonstrating exceptional parameter efficiency, inference speed, model performance, and robustness. Conclusion:The Conv-RGNN proposed in this paper offer a promising and feasible approach for intelligent diagnosis in resource-constrained environments.

Graph Neural Networks in Point Clouds: A Survey

With the advancement of 3D sensing technologies, point clouds are gradually becoming the main type of data representation in applications such as autonomous driving, robotics, and augmented reality. Nevertheless, the irregularity inherent in point clouds presents numerous challenges for traditional deep learning frameworks. Graph neural networks (GNNs) have demonstrated their tremendous potential in processing graph-structured data and are widely applied in various domains including social media data analysis, molecular structure calculation, and computer vision. GNNs, with their capability to handle non-Euclidean data, offer a novel approach for addressing these challenges. Additionally, drawing inspiration from the achievements of transformers in natural language processing, graph transformers have propelled models towards global awareness, overcoming the limitations of local aggregation mechanisms inherent in early GNN architectures. This paper provides a comprehensive review of GNNs and graph-based methods in point cloud applications, adopting a task-oriented perspective to analyze this field. We categorize GNN methods for point clouds based on fundamental tasks, such as segmentation, classification, object detection, registration, and other related tasks. For each category, we summarize the existing mainstream methods, conduct a comprehensive analysis of their performance on various datasets, and discuss the development trends and future prospects of graph-based methods.

Graph Adaptive Attention Network with Cross-Entropy.

Non-Euclidean data, such as social networks and citation relationships between documents, have node and structural information. The Graph Convolutional Network (GCN) can automatically learn node features and association information between nodes. The core ideology of the Graph Convolutional Network is to aggregate node information by using edge information, thereby generating a new node feature. In updating node features, there are two core influencing factors. One is the number of neighboring nodes of the central node; the other is the contribution of the neighboring nodes to the central node. Due to the previous GCN methods not simultaneously considering the numbers and different contributions of neighboring nodes to the central node, we design the adaptive attention mechanism (AAM). To further enhance the representational capability of the model, we utilize Multi-Head Graph Convolution (MHGC). Finally, we adopt the cross-entropy (CE) loss function to describe the difference between the predicted results of node categories and the ground truth (GT). Combined with backpropagation, this ultimately achieves accurate node classification. Based on the AAM, MHGC, and CE, we contrive the novel Graph Adaptive Attention Network (GAAN). The experiments show that classification accuracy achieves outstanding performances on Cora, Citeseer, and Pubmed datasets.

Kernel Angle Dependence Measures in Metric Spaces

Measuring and testing dependence between data in separable metric spaces is of great importance in modern statistics. Most existing work relied on the distance between random variables, which inevitably required the moment conditions to guarantee the distance is well-defined. Based on the geometry element “angle”, we develop a novel class of nonlinear dependence measures for data in metric space that can avoid such conditions. Specifically, by making use of the reproducing kernel Hilbert space equipped with Gaussian measure, we introduce kernel angle covariances that can be applied to various types of data, including low dimensional vector, high dimensional vectors, non-Euclidean data like symmetric positive definite matrices, and compositional data. We estimate kernel angle covariances based on U-statistic and establish the corresponding independence tests via gamma approximation. Our kernel angle independence tests, imposing no-moment conditions on kernels, are robust with heavy-tailed random variables. We conduct comprehensive simulation studies and apply our proposed methods to a facial recognition task. Our kernel angle covariances-based tests show remarkable performances in dealing with image data. All the codes and proofs are included in the supplementary materials.

A Survey of Computationally Efficient Graph Neural Networks for Reconfigurable Systems

Graph neural networks (GNNs) are powerful models capable of managing intricate connections in non-Euclidean data, such as social networks, physical systems, chemical structures, and communication networks. Despite their effectiveness, the large-scale and complex nature of graph data demand substantial computational resources and high performance during both training and inference stages, presenting significant challenges, particularly in the context of embedded systems. Recent studies on GNNs have investigated both software and hardware solutions to enhance computational efficiency. Earlier studies on deep neural networks (DNNs) have indicated that methods like reconfigurable hardware and quantization are beneficial in addressing these issues. Unlike DNN research, studies on efficient computational methods for GNNs are less developed and require more exploration. This survey reviews the latest developments in quantization and FPGA-based acceleration for GNNs, showcasing the capabilities of reconfigurable systems (often FPGAs) to offer customized solutions in environments marked by significant sparsity and the necessity for dynamic load management. It also emphasizes the role of quantization in reducing both computational and memory demands through the use of fixed-point arithmetic and streamlined vector formats. This paper concentrates on low-power, resource-limited devices over general hardware accelerators and reviews research applicable to embedded systems. Additionally, it provides a detailed discussion of potential research gaps, foundational knowledge, obstacles, and prospective future directions.

Integrating self-organizing feature map with graph convolutional network for enhanced superpixel segmentation and feature extraction in non-Euclidean data structure

Integrating self-organizing feature map with graph convolutional network for enhanced superpixel segmentation and feature extraction in non-Euclidean data structure

Intelligent identification of power grid parameters based on dynamic weighting

Transmission line parameter identification plays a crucial role in power systems. However, existing methods for identifying transmission line parameters face several challenges: (1) when dealing with non-Euclidean data such as power grid data, a considerable amount of noise is introduced, (2) inability to treat the power grid as a global system, limiting calculations to individual transmission lines and overlooking inter-branch correlations, (3) high sensitivity to data contamination. To address these issues, this paper proposes a multi-task noisy graph attention network (MNGAN) for power grid parameter identification, which leverages the spatial structure of the power grid and considers the inter-branch correlations. By computing the homogeneity and average degree of the graph data, we introduce an adaptive attention mechanism to the model. This attention mechanism dynamically adjusts the model’s focus on key information, mitigating the impact of data contamination and improving parameter identification accuracy. Additionally, to achieve simultaneous recognition of multiple branch parameters and enhance model training speed and accuracy, we incorporate a multi-task loss function based on homoscedastic uncertainty. Experimental results demonstrate the superiority of our proposed model over other machine learning and deep learning methods, providing more accurate predictions of branch parameters.

Generalized predictive analysis of reactions in paper devices via graph neural networks

Microfluidic technology facilitates high-throughput generation of time series data for biological and medical studies. Deep learning enables accurate, predictive analysis and proactive decision-making based on autonomous recognition of intricate pattern hidden in series. In this work, we first devised a paper-based microfluidic system for portable nucleic acid amplification test with economic energy consumption. Then, we employed Graph Neural Network (GNN), distinguished by its non-Euclidean data structure tailored for deep learning, with spatio-temporal attention mechanism to perform near-sensor predictive analysis of the on-chip reaction. Our findings demonstrated that the novel GNN model can provide accurate predictions of positive outcomes at the early stages of the reaction using less than one-third of the total reaction time. Then, the deep learning model trained by on-chip data was subsequently applied to more than 900 clinical plots. Generalization of the GNN model was successfully validated across different detection methods, diverse types of datasets and time series with variable length. Accuracy, sensitivity and specificity of the predictive approach were 96.5 %, 94.3 % and 99.0 % by utilizing the early half of reaction information. Finally, we compared the GNN model with various deep learning models. Despite differences in the prediction of negative samples among various models were minute, GNN obviously offered overall superior performance. This work ignites a cutting-edge application of deep learning in point-of-care and near-sensor tests. By harnessing the power of body area networks and edge/fog computing, our approach unlocks promising possibilities in diverse fields like healthcare and instrument science.

A Lightweight Method for Graph Neural Networks Based on Knowledge Distillation and Graph Contrastive Learning

Graph neural networks (GNNs) are crucial tools for processing non-Euclidean data. However, due to scalability issues caused by the dependency and topology of graph data, deploying GNNs in practical applications is challenging. Some methods aim to address this issue by transferring GNN knowledge to MLPs through knowledge distillation. However, distilled MLPs cannot directly capture graph structure information and rely only on node features, resulting in poor performance and sensitivity to noise. To solve this problem, we propose a lightweight optimization method for GNNs that combines graph contrastive learning and variable-temperature knowledge distillation. First, we use graph contrastive learning to capture graph structural representations, enriching the input information for the MLP. Then, we transfer GNN knowledge to the MLP using variable temperature knowledge distillation. Additionally, we enhance both node content and structural features before inputting them into the MLP, thus improving its performance and stability. Extensive experiments on seven datasets show that the proposed KDGCL model outperforms baseline models in both transductive and inductive settings; in particular, the KDGCL model achieves an average improvement of 1.63% in transductive settings and 0.8% in inductive settings when compared to baseline models. Furthermore, KDGCL maintains parameter efficiency and inference speed, making it competitive in terms of performance.

Graph Neural Networks for Recommendation Leveraging Multimodal Information

Recommender systems act as filtering algorithms to provide users with items that might meet their interests according to the expressed preferences and items' characteristics. As of today, the collaborative filtering paradigm, along with deep learning techniques to learn high-quality users' and items' representations, constitute the de facto standard for personalized recommendation, showing remarkable recommendation accuracy performance. Nevertheless, recommendation remains a highly-challenging task. Among the most debated open issues in the community, this thesis considers two algorithmic and conceptual ones, namely: (i) the inexplicable nature of users' preferences, especially when they come in the form of implicit feedback; (ii) the effective exploitation of the collaborative information in the designing and training of recommendation models. In domains such as fashion, food, and media content recommendation, the shallow item's profile can be enhanced through the multimodal characteristics describing items [Malitesta et al., 2023]. Driven by these assumptions, in the first part of this thesis, we apply multimodal deep learning strategies for multimedia recommendation; the scope is to study and design recommendation algorithms based upon the principles of multimodality to possibly match each item's characteristic to the implicit preference expressed by the user [Deldjoo et al., 2022], thus addressing the (i) issue. Recent collaborative filtering approaches profile users and items through embedding vectors in the latent space. However, such models disregard structural properties naturally encoded into the user-item interaction data. Indeed, recommendation datasets are easily describable under the topology of a bipartite and undirected graph, with users and items being the graph nodes connected at multiple distance hops. In this respect, the application of graph neural networks , recent deep learning techniques specifically tailored to learn from non-euclidean data, can provide a refined representation of users and items to mine near- and long-distance relationships in the user-item graphs [Anelli et al., 2023b]. Indeed, this is one possible solution to exploit the collaborative information, which is effectively propagated within the user-item graph, addressing the (ii) issue. Conclusively, this thesis aims to match the two families of recommendation strategies by leveraging graph neural networks and multimodal information data [Anelli et al., 2022]. In doing so, other numerous micro-aspects within the two macro-areas (introduced above) are examined. Indeed, the thesis is a systematic compendium of careful analyses regarding, among others, reproducibility, novel evaluation dimensions [Anelli et al., 2023a], and tasks/scenarios complementary to recommendation. Awarded by : Politecnico di Bari, Bari, Italy on 30 January 2024. Supervised by : Tommaso Di Noia. Available at : https://hdl.handle.net/11589/264941.

A model for early rumor detection base on topic-derived domain compensation and multi-user association

In the era of rapid Internet development, social networking sites have become breeding grounds for rumors. Detecting and addressing these rumors early is crucial to mitigate the potentially devastating consequences they may bring. This paper presents an innovative model for early rumor detection to tackle the challenge posed by sparse data in the early stages of rumor outbreaks. Firstly, a content-based method for data compensation is proposed, considering the feature correlation between the original and derived topics. The method effectively matches derived topics based on the content features of the topics, thereby enriching the original domain and solving the challenge of data sparsity. Secondly, a method for constructing the network structure is proposed to tackle the issue of user association within the early topic domain. This method introduces a quantification approach to evaluate the association relationship among multiple users. By incorporating game theory to quantify user emotion, the method effectively transforms the topic domain into a multi-user relationship network with a non-Euclidean structure. Finally, leveraging the capabilities of Graph Convolutional Networks (GCN) to process non-euclidean data, a novel early rumor detection model, namely DDCA-GCN, is proposed. It combines the strategies of topic-derived domain compensation and the principle of multi-user correlation. It can effectively identify early rumors and simultaneously comprehensively analyze users’ responses. The experimental results successfully demonstrate the feasibility of incorporating derived topic elements for early rumor detection. This validation underscores the effectiveness of leveraging derived topic domains to bolster the accuracy and timeliness of rumor detection.

Hypergraph Neural Architecture Search

In recent years, Hypergraph Neural Networks (HGNNs) have achieved considerable success by manually designing architectures, which are capable of extracting effective patterns with high-order interactions from non-Euclidean data. However, such mechanism is extremely inefficient, demanding tremendous human efforts to tune diverse model parameters. In this paper, we propose a novel Hypergraph Neural Architecture Search (HyperNAS) to automatically design the optimal HGNNs. The proposed model constructs a search space suitable for hypergraphs, and derives hypergraph architectures through differentiable search strategies. A hypergraph structure-aware distance criterion is introduced as a guideline for obtaining an optimal hypergraph architecture via the leave-one-out method. Experimental results for node classification on benchmark Cora, Citeseer, Pubmed citation networks and hypergraph datasets show that HyperNAS outperforms existing HGNNs models and graph NAS methods.

Neural Gaussian Similarity Modeling for Differential Graph Structure Learning

Graph Structure Learning (GSL) has demonstrated considerable potential in the analysis of graph-unknown non-Euclidean data across a wide range of domains. However, constructing an end-to-end graph structure learning model poses a challenge due to the impediment of gradient flow caused by the nearest neighbor sampling strategy. In this paper, we construct a differential graph structure learning model by replacing the non-differentiable nearest neighbor sampling with a differentiable sampling using the reparameterization trick. Under this framework, we argue that the act of sampling nearest neighbors may not invariably be essential, particularly in instances where node features exhibit a significant degree of similarity. To alleviate this issue, the bell-shaped Gaussian Similarity (GauSim) modeling is proposed to sample non-nearest neighbors. To adaptively model the similarity, we further propose Neural Gaussian Similarity (NeuralGauSim) with learnable parameters featuring flexible sampling behaviors. In addition, we develop a scalable method by transferring the large-scale graph to the transition graph to significantly reduce the complexity. Experimental results demonstrate the effectiveness of the proposed methods.

Dynamic Spiking Graph Neural Networks

The integration of Spiking Neural Networks (SNNs) and Graph Neural Networks (GNNs) is gradually attracting attention due to the low power consumption and high efficiency in processing the non-Euclidean data represented by graphs. However, as a common problem, dynamic graph representation learning faces challenges such as high complexity and large memory overheads. Current work often uses SNNs instead of Recurrent Neural Networks (RNNs) by using binary features instead of continuous ones for efficient training, which overlooks graph structure information and leads to the loss of details during propagation. Additionally, optimizing dynamic spiking models typically requires the propagation of information across time steps, which increases memory requirements. To address these challenges, we present a framework named Dynamic Spiking Graph Neural Networks (Dy-SIGN). To mitigate the information loss problem, Dy-SIGN propagates early-layer information directly to the last layer for information compensation. To accommodate the memory requirements, we apply the implicit differentiation on the equilibrium state, which does not rely on the exact reverse of the forward computation. While traditional implicit differentiation methods are usually used for static situations, Dy-SIGN extends it to the dynamic graph setting. Extensive experiments on three large-scale real-world dynamic graph datasets validate the effectiveness of Dy-SIGN on dynamic node classification tasks with lower computational costs.