Higher-order Relations Research Articles

Dynamic Spatio-Temporal Residual Hypergraph Convolutional Networks for Traffic Flow Forecasting

In recent years, more and more scholars have applied graph neural networks in combination with other modules to the field of traffic flow forecasting and achieved outstanding results. Most of these graph-based methods describe pairwise relationships between two objects, but in real transportation networks, the relationships between objects are often of a high order. To effectively learn the higher-order relationships between objects, this paper proposes a dynamic spatio-temporal residual hypergraph convolutional network for traffic forecasting (Res-DSTHGCN). In this paper, we combine dynamic spatio-temporal graph convolution and dynamic spatio-temporal residual hypergraph convolution to capture more global spatio-temporal features than models with only spatio-temporal graph convolution or only spatio-temporal hypergraph convolution. Meanwhile, the hypergraph convolutional network enhanced by residual connectivity improves the oversmoothing problem that may occur in the traditional hypergraph convolutional network (HGCN) along with the continuous stacking of layers. It is proved that the prediction accuracy of the method proposed in this paper is improved by conducting experiments for comparison with other baselines.

A Multi-Level Location-Aware Approach for Session-Based News Recommendation

Recently, personalized news recommendation systems have been widely used, which can achieve personalized news recommendations based on people’s different preferences, optimize the reading experience, and alleviate the problem of information overload. Among them, session-based news recommendation has gradually become a research hotspot as it can recommend news without requiring users to log in or when their reading history is difficult to obtain. The key to session-based news recommendation is to use short-term interaction data to learn user preferences. Existing models often focus on mining news content information in sessions and do not fully utilize geolocation information related to news and sessions, and there is also a certain inconsistency between their training objective and model evaluation metric, leading to suboptimal model recommendation performance. In order to fully utilize geolocation information, this paper proposes a multi-level location-aware approach for session-based news recommendation (MLA4SNR). Firstly, a news-location heterogeneous graph is constructed, and a graph element-wise attention network is proposed to mine high-order relationships between news and location. Secondly, a session feature extraction network based on Transformer is proposed to extract session features. Then, a session-location heterogeneous graph is constructed, and a graph element-wise attention network is used to mine high-order relationships between sessions and locations. Finally, a loss function based on the NDCG is used to train the model. Experimental results on a real news dataset show that MLA4SNR outperforms the baselines significantly.

Heterogeneous entity representation for medicinal synergy prediction.

Forecasting the synergistic effects of drug combinations facilitates drug discovery and development, especially regarding cancer therapeutics. While numerous computational methods have emerged, most of them fall short in fully modeling the relationships among clinical entities including drugs, cell lines, and diseases, which hampers their ability to generalize to drug combinations involving unseen drugs. These relationships are complex and multidimensional, requiring sophisticated modeling to capture nuanced interplay that can significantly influence therapeutic efficacy. We present a novel deep hypergraph learning method named Heterogeneous Entity Representation for MEdicinal Synergy prediction (HERMES) to predict the synergistic effects of anti-cancer drugs. Heterogeneous data sources, including drug chemical structures, gene expression profiles, and disease clinical semantics, are integrated into hypergraph neural networks equipped with a gated residual mechanism to enhance high-order relationship modeling. HERMES demonstrates state-of-the-art performance on two benchmark datasets, significantly outperforming existing methods in predicting the synergistic effects of drug combinations, particularly in cases involving unseen drugs. The source code is available at https://github.com/Christina327/HERMES. Supplementary data are available at Bioinformatics online.

GRESS: Grouping Belief-Based Deep Contrastive Subspace Clustering.

The self-expressive coefficient plays a crucial role in the self-expressiveness-based subspace clustering method. To enhance the precision of the self-expressive coefficient, we propose a novel deep subspace clustering method, named grouping belief-based deep contrastive subspace clustering (GRESS), which integrates the clustering information and higher-order relationship into the coefficient matrix. Specifically, we develop a deep contrastive subspace clustering module to enhance the learning of both self-expressive coefficients and cluster representations simultaneously. This approach enables the derivation of relatively noiseless self-expressive similarities and cluster-based similarities. To enable interaction between these two types of similarities, we propose a unique grouping belief-based affinity refinement module. This module leverages grouping belief to uncover the higher-order relationships within the similarity matrix, and integrates the well-designed noisy similarity suppression and similarity increment regularization to eliminate redundant connections while complete absent information. Extensive experimental results on four benchmark datasets validate the superiority of our proposed method GRESS over several state-of-the-art methods.

Deep self-representation learning with hyper-laplacian regularization for brain imaging genetics association analysis.

Brain imaging genetics aims to explore the association between genetic factors such as single nucleotide polymorphisms (SNPs) and brain imaging quantitative traits (QTs). However, most existing methods do not consider the nonlinear correlations between genotypic and phenotypic data, as well as potential higher-order relationships among subjects when identifying bi-multivariate associations. In this paper, a novel method called deep hyper-Laplacian regularized self-representation learning based structured association analysis (DHRSAA) is proposed which can learn genotype-phenotype associations and obtain relevant biomarkers. Specifically, a deep neural network is used first to explore the nonlinear relationships among samples. Secondly, self-representation learning based on hyper-Laplacian regularization is utilized to reconstruct the original data. In particular, the introduction of hyper-Laplacian regularization ensures the local structure of the high-dimensional spatial embedding and explores the higher-order relationships among the samples. Moreover, the structural regularization term in the association analysis uncovers chain relationships among SNPs and graphical relationships among imaging QTs, thus making the obtained markers more interpretable and enhancing the biological significance of the method. The performance of the proposed method is validated on real neuroimaging genetics data. Experimental results show that DHRSAA displays better canonical correlation coefficients and recognizes clearer canonical weight patterns compared to several state-of-the-art methods, which suggests that the proposed DHRSAA achieves better performance and identifies disease-related biomarkers.

Hypergraph-Based Organizers’ Behavior Scheduling Optimization: High-Order Relationships Discovery

Hypergraph-Based Organizers’ Behavior Scheduling Optimization: High-Order Relationships Discovery

Adaptive Neural Message Passing for Inductive Learning on Hypergraphs.

Graphs are the most ubiquitous data structures for representing relational datasets and performing inferences in them. They model, however, only pairwise relations between nodes and are not designed for encoding the higher-order relations. This drawback is mitigated by hypergraphs, in which an edge can connect an arbitrary number of nodes. Most hypergraph learning approaches convert the hypergraph structure to that of a graph and then deploy existing geometric deep learning methods. This transformation leads to information loss, and sub-optimal exploitation of the hypergraph's expressive power. We present HyperMSG, a novel hypergraph learning framework that uses a modular two-level neural message passing strategy to accurately and efficiently propagate information within each hyperedge and across the hyperedges. HyperMSG adapts to the data and task by learning an attention weight associated with each node's degree centrality. Such a mechanism quantifies both local and global importance of a node, capturing the structural properties of a hypergraph. HyperMSG is inductive, allowing inference on previously unseen nodes. Further, it is robust and outperforms state-of-the-art hypergraph learning methods on a wide range of tasks and datasets. Finally, we demonstrate the effectiveness of HyperMSG in learning multimodal relations through detailed experimentation on a challenging multimedia dataset.

Investigating Hypernode Classification of Complex Systems Based on High-order Graph Neural Networks

Investigating latent interactions beyond direct connections is essential for analyzing complex networks. However, traditional graph structures often fail to capture complex relationships, especially in the high-order interactions among multiple individuals. To address this issue, we extend the graph isomorphism network (GIN) framework to hypergraphs, treat nodes as self-hyperedges, and propose the self-hypergraph isomorphism network (SHGIN). Meanwhile, the hypergraph Weisfeiler–Lehman (WL) test is also proposed to distinguish different isomorphisms of hypergraphs and improve the representation power of hypergraph neural networks. Extensive experiments on co-authorship and co-citation networks demonstrate the effectiveness of SHGIN. The results indicate that our model displays superior hypernode classification accuracy compared to traditional graph neural networks in semi-supervised learning (SSL). Furthermore, it surpasses existing hypergraph neural network models in co-authorship datasets, highlighting its effectiveness in capturing high-order relationships in complex networks.

Advancing explainable MOOC recommendation systems: a morphological operations-based framework on partially ordered neutrosophic fuzzy hypergraphs

Recommendation systems constitute an integral part of nearly all digital service platforms. However, the common assumption in most recommendation systems in the literature is that similar users will be interested in similar items. This assumption holds only sometimes due to the inherent inhomogeneity of user-item interactions. To address this challenge, we introduce a novel recommendation system that leverages partially ordered neutrosophic hypergraphs to model higher-order relationships among users and items. The partial ordering of nodes enables the system to develop efficient top-N recommendations with very high Normalized Discounted Cumulative Gain (NDCG). Our approach incorporates the morphological operation of dilation, applied to user clusters obtained through fuzzy spectral clustering of the hypergraph, to generate the requisite number of recommendations. Explanations for recommendations are obtained through morphological erosion applied on the dual of the embedded hypergraph. Through rigorous testing in educational and e-commerce domains, it has been proved that our method outperforms state-of-the-art techniques and demonstrates excellent performance for various evaluation parameters. The NDCG value, a measure of ranking quality, surpasses 0.10, and the Hit Ratio (HR) consistently falls within the range of 0.25 to 0.30. The Root Mean Square Error (RMSE) values are minimal, reaching as low as 0.4. These results collectively position our algorithm as a good choice for generating recommendations with proper explanations, making it a promising solution for real-world applications.

Multi-relational multi-view clustering and its applications in cancer subtype identification

Cancer subtype identification aims to partition the cancer patients into different subgroups with distinct clinical phenotypes, which is important for accurate diagnosis and treatment planning. The recent surge in multi-omics data has spurred research into integrative subtype identification, and multi-view clustering is widely used for identifying the underlying potential subtypes in an unsupervised manner. However, most existing approaches only consider the single-level relations within each view and cannot fully explore the high-order relations across views. In this paper, we propose a new Multi-Relational Multi-View Clustering (MRMVC) method to address these issues, which treats multi-omics data as different views, and thoroughly explores multi-level intra-view and inter-view relations for subtype identification. It fully learns the pairwise similarity between samples based on (1) intra-view global relations that encourage the intra-class cohesion, (2) intra-view local relations that promote the inter-class separability, and (3) inter-view high-order relations that align the multiple graphs, enabling the discovery of intrinsic similarities and enhancing the clustering performance. Experiments on generic datasets and multi-omics cancer datasets illustrate the efficacy and superiority of the proposed method in clustering and identifying more distinct cancer subtypes.

Node Classification Method Based on Hierarchical Hypergraph Neural Network.

Hypergraph neural networks have gained widespread attention due to their effectiveness in handling graph-structured data with complex relationships and multi-dimensional interactions. However, existing hypergraph neural network models mainly rely on planar message-passing mechanisms, which have limitations: (i) low efficiency in encoding long-distance information; (ii) underutilization of high-order neighborhood features, aggregating information only on the edges of the original graph. This paper proposes an innovative hierarchical hypergraph neural network (HCHG) to address these issues. The HCHG combines the high-order relationship-capturing capability of hypergraphs, uses the Louvain community detection algorithm to identify community structures within the network, and constructs hypergraphs layer by layer. In the bottom-level hypergraph, the model establishes high-order relationships through direct neighbor nodes, while in the top-level hypergraph, it captures global relationships between aggregated communities. Through three hierarchical message-passing mechanisms, the HCHG effectively integrates local and global information, enhancing the multi-resolution representation ability of node representations and significantly improving performance in node classification tasks. In addition, the model performs excellently in handling 3D multi-view datasets. Such datasets can be created by capturing 3D shapes and geometric features through sensors or by manual modeling, providing extensive application scenarios for analyzing three-dimensional shapes and complex geometric structures. Theoretical analysis and experimental results show that the HCHG outperforms traditional hypergraph neural networks in complex networks.

Physics-enhanced data-driven turbulence model for flow around submerged bodies

Data-driven turbulence modeling is regarded as an effective approach to enhance the predictive performance of Reynolds-Averaged Navier−Stokes (RANS) models. However, the stability and accuracy of such methods still require further improvement. Therefore, we propose a novel modified model based on a combined neural network. Within this framework, we construct a fully connected neural network to predict the eddy viscosity coefficient in the RANS equations, thereby achieving an implicit solution for the linear part of the Reynolds stress. Additionally, we introduce a tensor basis neural network to predict the higher-order eddy viscosity relationships between unclosed quantities and analytical quantities. Furthermore, the model incorporates pressure gradients and turbulent kinetic energy gradients as inputs, fully considering the influence of pressure gradients and strong non-equilibrium effects in turbulence, thereby enhancing the physical foundation of the model. To evaluate the performance of the constructed model in different-dimensional flow fields with higher Reynolds numbers, we conduct validation analyzes on a two-dimensional NACA0012 model and a three-dimensional SUBOFF model without appendages. The results indicate that the modified model consistently outperforms the RANS model, particularly in predicting the velocity field near the leading edge of the NACA0012 hydrofoil. The predictions of the modified model are closer to those of large eddy simulation (LES) results. Specifically, the root mean square error (RMSE) of the interpolated and extrapolated modified models is reduced by 49.2% and 33.3%, respectively, compared to the RMSE of the RANS model. When applied to three-dimensional isolated SUBOFF flows, the modified model’s predictions for the average velocity field and pressure coefficients at both ends of the SUBOFF are in high agreement with LES results, but the prediction accuracy in the middle section is slightly insufficient. Nevertheless, the prediction errors of the interpolated and extrapolated modified models are reduced by 69.5% and 63.6%, respectively, compared to the baseline RANS model, which fully demonstrates the high accuracy of the modified model in predicting the overall pressure distribution of the SUBOFF. The methods developed in this study provide valuable insights into high-precision intelligent modeling for two-dimensional and three-dimensional flows.

Hypergraph Neural Network for Multimodal Depression Recognition

Deep learning-based approaches for automatic depression recognition offer advantages of low cost and high efficiency. However, depression symptoms are challenging to detect and vary significantly between individuals. Traditional deep learning methods often struggle to capture and model these nuanced features effectively, leading to lower recognition accuracy. This paper introduces a novel multimodal depression recognition method, HYNMDR, which utilizes hypergraphs to represent the complex, high-order relationships among patients with depression. HYNMDR comprises two primary components: a temporal embedding module and a hypergraph classification module. The temporal embedding module employs a temporal convolutional network and a negative sampling loss function based on Euclidean distance to extract feature embeddings from unimodal and cross-modal long-time series data. To capture the unique ways in which depression may manifest in certain feature elements, the hypergraph classification module introduces a threshold segmentation-based hyperedge construction method. This method is the first attempt to apply hypergraph neural networks to multimodal depression recognition. Experimental evaluations on the DAIC-WOZ and E-DAIC datasets demonstrate that HYNMDR outperforms existing methods in automatic depression monitoring, achieving an F1 score of 91.1% and an accuracy of 94.0%.

High-Order Local Clustering on Hypergraphs

Graphs are a commonly used model in data mining to represent complex relationships, with nodes representing entities and edges representing relationships. However, graphs have limitations in modeling high-order relationships. In contrast, hypergraphs offer a more versatile representation, allowing edges to join any number of nodes. This capability empowers hypergraphs to model multiple relationships and capture high-order information present in real-world applications. We focus on the problem of local clustering in hypergraphs, which computes a cluster near a given seed node. Although extensively explored in the context of graphs, this problem has received less attention for hypergraphs. Current methods often directly extend graph-based local clustering to hypergraphs, overlooking their inherent high-order features and resulting in low-quality local clusters. To address this, we propose an effective hypergraph local clustering model. This model introduces a novel conductance measurement that leverages the high-order properties of hypergraphs to assess cluster quality. Based on this new definition of hypergraph conductance, we propose a greedy algorithm to find local clusters in real time. Experimental evaluations and case studies on real-world datasets demonstrate the effectiveness of the proposed methods.

Dynamic Spatio-Temporal Hypergraph Convolutional Network for Traffic Flow Forecasting

Graph convolutional networks (GCN) are an important research method for intelligent transportation systems (ITS), but they also face the challenge of how to describe the complex spatio-temporal relationships between traffic objects (nodes) more effectively. Although most predictive models are designed based on graph convolutional structures and have achieved effective results, they have certain limitations in describing the high-order relationships between real data. The emergence of hypergraphs breaks this limitation. A dynamic spatio-temporal hypergraph convolutional network (DSTHGCN) model is proposed in this paper. It models the dynamic characteristics of traffic flow graph nodes and the hyperedge features of hypergraphs simultaneously, achieving collaborative convolution between graph convolution and hypergraph convolution (HGCN). On this basis, a hyperedge outlier removal mechanism (HOR) is introduced during the process of node information propagation to hyper-edges, effectively removing outliers and optimizing the hypergraph structure while reducing complexity. Through in-depth experimental analysis on real-world datasets, this method has better performance compared to other methods.

Deep Learning Model for Quality Assessment of Urinary Bladder Ultrasound Images using Multi-scale and Higher-order Processing.

Autonomous Ultrasound Image Quality Assessment (US-IQA) is a promising tool to aid the interpretation by practicing sonographers and to enable the future robotization of ultrasound procedures. However, autonomous US-IQA has several challenges. Ultrasound images contain many spurious artifacts, such as noise due to handheld probe positioning, errors in the selection of probe parameters and patient respiration during the procedure. Further, these images are highly variable in appearance with respect to the individual patient's physiology. We propose to use a deep Convolutional Neural Network (CNN), USQNet, which utilizes a Multi-scale and Local-to-Global Second-order Pooling (MS-L2GSoP) classifier to conduct the sonographer-like assessment of image quality. This classifier first extracts features at multiple scales to encode the inter-patient anatomical variations, similar to a sonographer's understanding of anatomy. Then, it uses second-order pooling in the intermediate layers (local) and at the end of the network (global) to exploit the second-order statistical dependency of multi-scale structural and multi-region textural features. The L2GSoP will capture the higher-order relationships between different spatial locations and provide the seed for correlating local patches, much like a sonographer prioritizes regions across the image. We experimentally validated the USQNet for a new dataset of the human urinary bladder ultrasound images. The validation involved first with the subjective assessment by experienced radiologists' annotation, and then with state-of-the-art CNN networks for US-IQA and its ablated counterparts. The results demonstrate that USQNet achieves a remarkable accuracy of 92.4% and outperforms the SOTA models by 3 - 14% while requiring comparable computation time.

Dual semi-supervised hypergraph regular multi-view NMF with anchor graph embedding

Graph regularized nonnegative matrix factorization (GNMF) has been widely used in multi-view clustering tasks due to its good clustering properties. However, it uses a simple graph to describe the complex data relationship of multiple views and tries to obtain a consistent low-dimensional representation, which undoubtedly brings challenges to its clustering performance. In addition, clustering a large amount of high-dimensional data from multiple views undoubtedly faces a huge computational burden. In order to effectively improve the performance and efficiency of GNMF based multi-view clustering algorithm, this paper proposes a dual semi-supervised hypergraph regular multi-view clustering method with anchor graph embedding (DSSHMNMFAE). Specifically, DSSHMNMFAE develops a new anchor selection method to generate anchors and the anchor bipartite graph is constructed to embed the matrix factorization process. DSSHMNMFAE constructs a hypergraph to effectively learn the high-order relationship between data from multiple views. In order to perform semi-supervised learning more efficiently, DSSHMNMFAE integrates pairwise constraint information and label constraint information into the clustering process as dual label information. In addition, DSSHMNMFAE considers the learning of both consistency information and complementarity information, and adopts adaptive measures to distinguish the contributions of different views. We use the alternating iterative algorithm to optimize the objective function of DSSHMNMFAE. The experimental results on eight real datasets show that the performance of DSSHMNMFAE is comparable to other algorithms.

VAE*: A Novel Variational Autoencoder via Revisiting Positive and Negative Samples for Top- N Recommendation

Due to the easy access, implicit feedback is often used for recommender systems. Compared with point-wise learning and pair-wise learning methods, list-wise rank learning methods have superior performance for top- \(N\) recommendation. Recent solutions, especially the list-wise methods, simply treat all interacted items of a user as equally important positives and annotate all no-interaction items of a user as negatives. For the list-wise approaches, we argue that this annotation scheme of implicit feedback is over-simplified due to the sparsity and missing fine-grained labels of the feedback data. To overcome this issue, we revisit the so-called positive and negative samples. First, considering the loss function of list-wise ranking, we analyze the impact of false positives and negatives theoretically. Second, based on the observation, we propose a self-adjusting credibility weight mechanism to re-weigh the positive samples and exploit the higher-order relation based on item–item matrix to sample the critical negative samples. In order to prevent the introduction of noise, we design a pruning strategy for critical negatives. Besides, to combine the reconstruction loss function for the positive samples and critical negative samples, we develop a simple yet effective VAEs framework with linear structure, which abandons the complex non-linear structure. Extensive experiments are conducted on six public real-world datasets. The results demonstrate that, our VAE* outperforms other VAE-based models by a large margin. Besides, we also verify the effect of denoising positives and exploring critical negatives by ablation study.

Integrating higher-order relations for enhanced twitter bot detection

AbstractBot detection is a critical task in preserving the integrity of social networks and mitigating online disinformation. Despite advances in graph-based methods for detecting bots, these models often rely on follow relations, assuming that users sharing similar characteristics are more likely to connect. However, these methods often fail to address complex behavioral patterns indicative of coordinated bot activities, allowing bot developers to avoid detection more easily. We investigate the potential of integrating behavioral and higher-order relations, focusing on Retweet, Co-Retweet (where two users retweet the same tweet) and Co-Hashtag (where two users frequently use the same hashtag), and compare these against conventional follower and following relations. We further explore the process of relation creation, particularly highlighting the Co-Hashtag relation’s robustness against data collection flaws and its ability to mitigate shortcomings in the collection of datasets. Our experiments not only contribute to the ongoing efforts to address challenges in bot detection, but also open up new avenues for exploring how complex network patterns can be employed for social network analysis.

Learning accurate neighborhood- and self-information for higher-order relation prediction in Heterogeneous Information Networks

Heterogeneous Information Networks (HINs) are commonly employed to model complex real-world scenarios with diverse node and edge types. However, due to constraints in data collection and processing, constructed networks often lack certain relations. Consequently, various methods have emerged, particularly recently, leveraging heterogeneous graph neural networks (HGNNs) to predict missing relations. Nevertheless, these methods primarily focus on pairwise relations between two nodes. Real-world interactions, however, often involve multiple nodes and diverse types, extending beyond simple pairwise relations. For instance, academic collaboration networks may entail interactions among authors, papers, and conferences simultaneously. Despite their prevalence, higher-order relations are often overlooked. While HGNNs are effective at learning network structures, they may suffer from over-smoothing, resulting in similar representations for nodes and their neighbors. The learned inaccurate proximity among nodes impedes the discernment of higher-order relations. Furthermore, observed edges among a target group of nodes can provide valuable evidence for predicting higher-order relations. To address these challenges, we propose a novel model called Accurate Neighborhood- and Self-information Enhanced Heterogeneous Graph Neural Network (ANSE-HGN). Building upon HGNNs to encode network structure and attributes, we introduce a relation-based neighborhood encoder to capture information within multi-hop neighborhoods in heterogeneous higher-order relations. This enables the calculation of accurate proximity among target groups of nodes, thereby enhancing prediction accuracy. Additionally, we leverage self-information from observed higher-order relations as an auxiliary loss to reinforce the learning process. Extensive experiments on four real-world datasets demonstrate the superiority of our proposed method in higher-order relation prediction tasks.