Spatial context-enhanced temporal knowledge graph reasoning

Mining semantic relations between concepts underlies many fundamental tasks including natural language processing, web mining, information retrieval, and web search. In order to describe the semantic relation between concepts, in this paper, the problem of automatically generating spatial temporal relation graph (STRG) of semantic relation between concepts is studied. The spatial temporal relation graph of semantic relation between concepts includes relation words, relation sentences, relation factor, relation graph, faceted feature, temporal feature, and spatial feature. The proposed method can automatically generate the spatial temporal relation graph (STRG) of semantic relation between concepts, which is different from the manually generated annotation repository such as WordNet and Wikipedia. Moreover, the proposed method does not need any prior knowledge such as ontology or the hierarchical knowledge base such as WordNet. Empirical experiments on real dataset show that the proposed algorithm is effective and accurate.

Knowledge Graph (KG) reasoning is a crucial task that discovers potential and unknown knowledge based on the existing knowledge. Temporal Knowledge Graph (TKG) reasoning is more challenging than KG reasoning because the additional temporal information needs to be handled. Previous TKG reasoning methods restrict the search space to avoid huge computational consumption, resulting in a decrease in accuracy. In order to improve the accuracy and efficiency of TKG reasoning, a model CMPH (Combination Model of Paths and History) is proposed, which consists of a path memory network and a history memory network. The former finds the paths in advance by a TKG path search algorithm and learns to memorize the recurrent pattern for reasoning, which prevents path search at inference stage. The latter adopts efficient encoder-decoder architecture to learn the features of historical events in TKG, which can avoid tackling a large number of structural dependencies and increase the reasoning accuracy. To take the advantages of these two types of memory networks, a gate component is designed to integrate them for better performance. Extensive experiments on four real-world datasets demonstrate that the proposed model obtains substantial performance and efficiency improvement for the TKG reasoning tasks. Especially, it achieves up to 8.6% and 11.8% improvements in MRR and hit@1 respectively, and up to 21 times speedup at inference stage comparing to the state-of-the-art baseline.

Knowledge graphs’ reasoning is of great significance for the further development of artificial intelligence and information retrieval, especially for reasoning over temporal knowledge graphs. The rotation-based method has been shown to be effective at modeling entities and relations on a knowledge graph. However, due to the lack of temporal information representation capability, existing approaches can only model partial relational patterns and they cannot handle temporal combination reasoning. In this regard, we propose HTTR: Householder Transformation-based Temporal knowledge graph Reasoning, which focuses on the characteristics of relations that evolve over time. HTTR first fuses the relation and temporal information in the knowledge graph, then uses the Householder transformation to obtain an orthogonal matrix about the fused information, and finally defines the orthogonal matrix as the rotation of the head-entity to the tail-entity and calculates the similarity between the rotated vector and the vector representation of the tail entity. In addition, we compare three methods for fusing relational and temporal information. We allow other fusion methods to replace the current one as long as the dimensionality satisfies the requirements. We show that HTTR is able to outperform state-of-the-art methods in temporal knowledge graph reasoning tasks and has the ability to learn and infer all of the four relational patterns over time: symmetric reasoning, antisymmetric reasoning, inversion reasoning, and temporal combination reasoning.

CDRGN-SDE: Cross-Dimensional Recurrent Graph Network with neural Stochastic Differential Equation for temporal knowledge graph embedding

Multiscale spatial temporal attention graph convolution network for skeleton-based anomaly behavior detection

In skeleton-based action recognition, graph convolutional networks (GCNs) have achieved remarkable success. However, there are two shortcomings of current GCN-based methods. Firstly, the computation cost is pretty heavy, typically over 15 GFLOPs for one action sample. Some recent works even reach ~100 GFLOPs. Secondly, the receptive fields of both spatial graph and temporal graph are inflexible. Although recent works introduce incremental adaptive modules to enhance the expressiveness of spatial graph, their efficiency is still limited by regular GCN structures. In this paper, we propose a shift graph convolutional network (ShiftGCN) to overcome both shortcomings. ShiftGCN is composed of novel shift graph operations and lightweight point-wise convolutions, where the shift graph operations provide flexible receptive fields for both spatial graph and temporal graph. To further boost the efficiency, we introduce four techniques and build a more lightweight skeleton-based action recognition model named ShiftGCN++. ShiftGCN++ is an extremely computation-efficient model, which is designed for low-power and low-cost devices with very limited computing power. On three datasets for skeleton-based action recognition, ShiftGCN notably exceeds the state-of-the-art methods with over 10× less FLOPs and 4× practical speedup. ShiftGCN++ further boosts the efficiency of ShiftGCN, which achieves comparable performance with 6× less FLOPs and 2× practical speedup.

Action recognition with skeleton data is attracting more attention in computer vision. Recently, graph convolutional networks (GCNs), which model the human body skeletons as spatiotemporal graphs, have obtained remarkable performance. However, the computational complexity of GCN-based methods are pretty heavy, typically over 15 GFLOPs for one action sample. Recent works even reach about 100 GFLOPs. Another shortcoming is that the receptive fields of both spatial graph and temporal graph are inflexible. Although some works enhance the expressiveness of spatial graph by introducing incremental adaptive modules, their performance is still limited by regular GCN structures. In this paper, we propose a novel shift graph convolutional network (Shift-GCN) to overcome both shortcomings. Instead of using heavy regular graph convolutions, our Shift-GCN is composed of novel shift graph operations and lightweight point-wise convolutions, where the shift graph operations provide flexible receptive fields for both spatial graph and temporal graph. On three datasets for skeleton-based action recognition, the proposed Shift-GCN notably exceeds the state-of-the-art methods with more than 10 times less computational complexity.

Skeleton-based action recognition approaches usually construct the skeleton sequence as spatial-temporal graphs and perform graph convolution on these graphs to extract discriminative features. However, due to the fixed topology shared among different poses and the lack of direct long-range temporal dependencies, it is not trivial to learn the robust spatial-temporal feature. Therefore, we present a spatial-temporal adaptive graph convolutional network (STA-GCN) to learn adaptive spatial and temporal topologies and effectively aggregate features for skeleton-based action recognition. The proposed network is composed of spatial adaptive graph convolution (SA-GC) and temporal adaptive graph convolution (TA-GC) with an adaptive topology encoder. The SA-GC can extract the spatial feature for each pose with the spatial adaptive topology, while the TA-GC can learn the temporal feature by modeling the direct long-range temporal dependencies adaptively. On three large-scale skeleton action recognition datasets: NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton, the STA-GCN outperforms the existing state-of-the-art methods. The code is available at https://github.com/hang-rui/STA-GCN.KeywordsAction recognitionAdaptive topologyGraph convolution

In this paper, we propose a new method for detecting abnormal human behavior based on skeleton features using self-attention augment graph convolution. The skeleton data have been proved to be robust to the complex background, illumination changes, and dynamic camera scenes and are naturally constructed as a graph in non-Euclidean space. Particularly, the establishment of spatial temporal graph convolutional networks (ST-GCN) can effectively learn the spatio-temporal relationships of Non-Euclidean Structure Data. However, it only operates on local neighborhood nodes and thereby lacks global information. We propose a novel spatial temporal self-attention augmented graph convolutional networks (SAA-Graph) by combining improved spatial graph convolution operator with a modified transformer self-attention operator to capture both local and global information of the joints. The spatial self-attention augmented module is used to understand the intra-frame relationships between human body parts. As far as we know, we are the first group to utilize self-attention for video anomaly detection tasks by enhancing spatial temporal graph convolution. Moreover, to validate the proposed model, we performed extensive experiments on two large-scale publicly standard datasets (i.e., ShanghaiTech Campus and CUHK Avenue datasets) which reveal the state-of-art performance for our proposed approach when compared to existing skeleton-based methods and graph convolution methods.

Temporal Knowledge Graph (TKG) representation learning aims to project entities and relations from high-dimensional spaces into low-dimensional ones while preserving dynamic relational characteristics. However, many existing methods primarily focus on single-time-stamp Knowledge Graphs, neglecting the importance of time in capturing the evolving relationships within TKGs. To address this limitation, we introduce TAR-TKG (Temporal-Aware Representation for Temporal Knowledge Graph), a novel framework that consists of three core modules. The first module, the Temporal Dynamics Steering Module, enhances dynamic temporal features by employing a multi-time-awareness network to capture changes between time stamps, thereby improving the understanding of temporal data evolution. The second module, the Cross-Time Domain Gating Module, applies cross-time domain graph convolution and adaptive gating to learn relationships between different time stamps, facilitating the integration of information across multiple time spans to improve the accuracy of temporal reasoning. The third module, the Temporal Adaptive Relation Perception Module, combines temporal embeddings, causal reasoning, and multi-modal relation fusion to enhance the model’s ability to perceive temporal relationships, particularly in managing causal dependencies and complex time-based interactions. Experimental results demonstrate that TAR-TKG outperforms existing baseline methods on three real-world datasets, proving its effectiveness in capturing dynamic relationship evolution and improving temporal reasoning within TKGs.

Historical Trends and Normalizing Flow for One-shot Temporal Knowledge Graph Reasoning

Multi-hop temporal knowledge graph reasoning with temporal path rules guidance

This paper investigates cross-lingual temporal knowledge graph reasoning problem, which aims to facilitate reasoning on Temporal Knowledge Graphs (TKGs) in low-resource languages by transfering knowledge from TKGs in high-resource ones. The cross-lingual distillation ability across TKGs becomes increasingly crucial, in light of the unsatisfying performance of existing reasoning methods on those severely incomplete TKGs, especially in low-resource languages. However, it poses tremendous challenges in two aspects. First, the cross-lingual alignments, which serve as bridges for knowledge transfer, are usually too scarce to transfer sufficient knowledge between two TKGs. Second, temporal knowledge discrepancy of the aligned entities, especially when alignments are unreliable, can mislead the knowledge distillation process. We correspondingly propose a mutually-paced knowledge distillation model MP-KD, where a teacher network trained on a source TKG can guide the training of a student network on target TKGs with an alignment module. Concretely, to deal with the scarcity issue, MP-KD generates pseudo alignments between TKGs based on the temporal information extracted by our representation module. To maximize the efficacy of knowledge transfer and control the noise caused by the temporal knowledge discrepancy, we enhance MP-KD with a temporal cross-lingual attention mechanism to dynamically estimate the alignment strength. The two procedures are mutually paced along with model training. Extensive experiments on twelve cross-lingual TKG transfer tasks in the EventKG benchmark demonstrate the effectiveness of the proposed MP-KD method.

Dealing with spatial and temporal knowledge is an indispensable part of almost all aspects of human activities. The qualitative approach to spatial and temporal reasoning (QSTR) provides a promising framework for spatial and temporal knowledge representation and reasoning. QSTR typically represents spatial/temporal knowledge in terms of qualitative relations (e.g., to the east of, after), and reasons with the knowledge by solving qualitative constraints. When formulating a qualitative constraint satisfaction problem (CSP), it is usually assumed that each variable could be “here, there and everywhere.” Practical applications e.g. urban planning, however, often require a variable taking values from a certain finite subset of the universe, i.e. require it to be 'here or there'. This paper extends the classic framework of qualitative constraint satisfaction by allowing variables taking values from finite domains. The computational complexity of this extended consistency problem is examined for five most important qualitative calculi, viz. Point Algebra, Interval Algebra, Cardinal Relation Algebra, RCC-5, and RCC-8. We show that the extended consistency problem remains in NP, but when only basic constraints are considered, the extended consistency problem for each calculus except Point Algebra is already NP-hard.

Multi-hop path reasoning over sparse temporal knowledge graphs based on path completion and reward shaping