Multiscale Graphs Research Articles

Research on predicting heat loads based on extracting temporal and spatial features of multiple buildings using data-driven methods

Accurate heat load prediction is the key to ensure the stable operation of thermal system and effective planning of thermal resources. However, current heat load forecasting methods tend to treat individual buildings as isolated entities, ignoring the temporal and spatial correlation between buildings. In this study, we propose a data-driven model based on spatiotemporal coupling to predict short-term heat load. Firstly, the spatial and temporal correlation and the correlation among features of various buildings are analyzed by using the autocorrelation function and Spearman correlation coefficient. Secondly, synchronous wavelet transform is used to eliminate high frequency noise of heat load. Secondly, the spatial and temporal characteristics of building heat load are extracted by using the improved Informer model which is concerned with multi-scale graphs. The experimental results show that the proposed model has better predictive performance than the baseline model. It provides a reference for the accurate regulation of thermal system.

A multi-scale graph convolutional network with contrastive-learning enhanced self-attention pooling for intelligent fault diagnosis of gearbox

Recently, the emerging graph convolutional network (GCN) has been applied into fault diagnosis with the aim of providing additional fault features through topological information. However, there are some limitations with these methods. First, the interactions between multi-frequency scales are ignored in existing studies, while they mainly focus on constructing graphs through the relationship between channels/instances. Second, the constructed graph cannot well reflect the topology of noisy samples and lacks robust hierarchical representation learning capability, and the learned graphs have limited interpretability. Hence, a Multi-Scale GCN with Contrastive-learning enhanced Self-attention Pooling (MSGCN-CSP) method is proposed for intelligent fault diagnosis of gearbox. Time–frequency distributions are converted into multi-scale graphs to extract fault features through topological relationships between multi-frequencies. Contrastive-learning is used to implement graph pooling, which enables hierarchical representation learning. Experimental results on two gearbox datasets illustrate that the proposed method offers competitive diagnostic performance and provides good interpretability in establishing GCN.

Appearance and geometry transformer for facial expression recognition in the wild

Facial expression recognition (FER) in the wild is considered to be one of the most challenging tasks in computer vision because it requires identifying the categories of expressions presented by the human face under adverse conditions, such as dynamic poses, extreme illuminations, and partial occlusion. In this paper, we propose a model called the appearance and geometry transformer (AGT), which is a self-attention-based deep neural network, to improve the accuracy of recognition of FER in the wild. The AGT is designed to simultaneously perform feature extraction and fusion on heterogeneous data, including images and graphs, by using two transformer pathways. A pre-trained DeiT model is introduced to the image transformer pathway to explore the most discriminative facial regions. A multi-scale graph transformer network consisting of a self-attention module based on several multi-scale graphs is presented in the graph transformer pathway to adaptively extract features from the facial graph. A distillation mechanism is employed in the AGT to improve training in case of a limited number of training samples. We evaluated the proposed method on two benchmark databases: FERplus and RAF-DB. The results of comprehensive experiments on these two databases show that the proposed model can achieve comparable results to state-of-the-art methods of FER in the wild.

Multiscale spatial temporal attention graph convolution network for skeleton-based anomaly behavior detection

Anomaly behavior detection plays a significant role in emergencies such as robbery. Although a lot of works have been proposed to deal with this problem, the performance in real applications is still relatively low. Here, to detect abnormal human behavior in videos, we propose a multiscale spatial temporal attention graph convolution network (MSTA-GCN) to capture and cluster the features of the human skeleton. First, based on the human skeleton graph, a multiscale spatial temporal attention graph convolution block (MSTA-GCB) is built which contains multiscale graphs in temporal and spatial dimensions. MSTA-GCB can simulate the motion relations of human body components at different scales where each scale corresponds to different granularity of annotation levels on the human skeleton. Then, static, globally-learned and attention-based adjacency matrices in the graph convolution module are proposed to capture hierarchical representation. Finally, extensive experiments are carried out on the ShanghaiTech Campus and CUHK Avenue datasets, the final results of the frame-level AUC/EER are 0.759/0.311 and 0.876/0.192, respectively. Moreover, the frame-level AUC is 0.768 for the human-related ShanghaiTech subset. These results show that our MSTA-GCN outperforms most of methods in video anomaly detection and we have obtained a new state-of-the-art performance in skeleton-based anomaly behavior detection.

An Adaptively Attention-Driven Cascade Part-Based Graph Embedding Framework for UAV Object Re-Identification

With the rapid development of unmanned aerial vehicles (UAVs), object re-identification (Re-ID) based on the UAV platforms has attracted increasing attention, and several excellent achievements have been shown in the traditional scenarios. However, object Re-ID in aerial imagery acquired from the UAVs is still a challenging task, which is mainly due to the reason that variable locations and diverse viewpoints in UAVs platform are always resulting in more appearance ambiguities among the intra-objects and inter-objects. To address the above issues, in this paper, we proposed an adaptively attention-driven cascade part-based graph embedding framework (AAD-CPGE) for UAV object Re-ID. The AAD-CPGE aims to optimally fuse node features and their topological characteristics on the multi-scale structured graphs of parts-based objects, and then adaptively learn the most correlated information for improving the object Re-ID performance. Specifically, we first executed GCNs on the parts-based cascade node feature graphs and topological feature graphs for acquiring multi-scale structured-graph feature representations. After that, we designed a self-attention-based module for adaptive node and topological features fusion on the constructed hierarchical parts-based graphs. Finally, these learning hybrid graph-structured features with the most correlation discriminative capability were applied for object Re-ID. Several experimental verifications on three widely used UAVs-based benchmark datasets were carried out, and comparison with some state-of-the-art object Re-ID approaches validated the effectiveness and benefits of our proposed AAD-CPGE Re-ID framework.

Graph embedding via multi-scale graph representations

Graph embedding provides an effective way to encode graph nodes as continuous vector representations in a low-dimensional space. The high-order proximities based graph embedding methods can preserve global structures, but the high-order proximities based objectives typically imply non-convex optimization and high computational complexity. In contrast, the low-order proximities based graph embedding methods evade the definition and optimization of complex high-order proximities based objectives, but cannot capture global structures. Furthermore, numerous graph embedding methods that only consider the proximity relationships among nodes ignore to capture global structure from the perspective of subgraphs. Motivated by this, we propose a novel graph embedding framework via multi-scale graph representations, named MSGE. MSGE first determines multiple subgraphs of different scales based on random walk. At each scale, MSGE generates a new graph by creating edges among the subgraphs which are treated as supernodes. Thus, the generated multi-scale graphs can approximate rich global structure of the original graph. MSGE then employs any existing graph embedding method as a black box to learn subgraph (i.e., supernode) embeddings on each generated graph. Subsequently, a first-order proximity based embedding fusion method is devised to yield node embeddings of the original graph via the learned multi-scale subgraph embeddings. Finally, we apply MSGE on four classical graph embedding methods and extensive experimental results demonstrate that our framework can generate embeddings of better quality and significantly outperform the original graph embedding methods on visualization, node classification and community detection tasks.

Multiscale characterization and representation of variability in ceramic matrix composites

Low density, high strength, and high creep and oxidation resistance properties of ceramic matrix composites (CMCs) make them an ideal choice for use in extreme environments in space and military applications. This paper presents a detailed characterization study of structural and manufacturing flaws in Carbon fiber Silicon-Carbide-Nitride matrix (C/SiNC) CMCs at different length-scales. Energy-dispersive spectroscopy (EDS) is used for the chemical characterization of the material’s elemental constituents. High-resolution multiscale graphs obtained from scanning electron microscope (SEM) and confocal laser scanning microscope (LSM) are used to characterize the distribution and morphology of defects at different length scales. This is followed by the classification and quantification of the common manufacturing defects. An image processing algorithm based on the image segmentation process is developed to quantify the variability of various scale-dependent architectural parameters. Finally, a three-dimensional stochastic representative volume element (SRVE) generation algorithm is developed to provide precise representations of material textures at multiple length scales. The developed algorithm accurately accounts for material features and flaws based on a range of multiscale structural and defects characterization results.

Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction.

We propose a multiscale spatio-temporal graph neural network (MST-GNN) to predict the future 3D skeleton-based human poses in an action-category-agnostic manner. The core of MST-GNN is a multiscale spatio-temporal graph that explicitly models the relations in motions at various spatial and temporal scales. Different from many previous hierarchical structures, our multiscale spatio-temporal graph is built in a data-adaptive fashion, which captures nonphysical, yet motion-based relations. The key module of MST-GNN is a multiscale spatio-temporal graph computational unit (MST-GCU) based on the trainable graph structure. MST-GCU embeds underlying features at individual scales and then fuses features across scales to obtain a comprehensive representation. The overall architecture of MST-GNN follows an encoder-decoder framework, where the encoder consists of a sequence of MST-GCUs to learn the spatial and temporal features of motions, and the decoder uses a graph-based attention gate recurrent unit (GA-GRU) to generate future poses. Extensive experiments are conducted to show that the proposed MST-GNN outperforms state-of-the-art methods in both short and long-term motion prediction on the datasets of Human 3.6M, CMU Mocap and 3DPW, where MST-GNN outperforms previous works by 5.33% and 3.67% of mean angle errors in average for short-term and long-term prediction on Human 3.6M, and by 11.84% and 4.71% of mean angle errors for short-term and long-term prediction on CMU Mocap, and by 1.13% of mean angle errors on 3DPW in average, respectively. We further investigate the learned multiscale graphs for interpretability.

Persistent spectral graph.

Persistent homology is constrained to purely topological persistence, while multiscale graphs account only for geometric information. This work introduces persistent spectral theory to create a unified low-dimensional multiscale paradigm for revealing topological persistence and extracting geometric shapes from high-dimensional datasets. For a point-cloud dataset, a filtration procedure is used to generate a sequence of chain complexes and associated families of simplicial complexes and chains, from which we construct persistent combinatorial Laplacian matrices. We show that a full set of topological persistence can be completely recovered from the harmonic persistent spectra, that is, the spectra that have zero eigenvalues, of the persistent combinatorial Laplacian matrices. However, non-harmonic spectra of the Laplacian matrices induced by the filtration offer another powerful tool for data analysis, modeling, and prediction. In this work, fullerene stability is predicted by using both harmonic spectra and non-harmonic persistent spectra, while the latter spectra are successfully devised to analyze the structure of fullerenes and model protein flexibility, which cannot be straightforwardly extracted from the current persistent homology. The proposed method is found to provide excellent predictions of the protein B-factors for which current popular biophysical models break down.

A Deep Multiscale Fusion Method via Low-Rank Sparse Decomposition for Object Saliency Detection Based on Urban Data in Optical Remote Sensing Images

The urban data provides a wealth of information that can support the life and work for people. In this work, we research the object saliency detection in optical remote sensing images, which is conducive to the interpretation of urban scenes. Saliency detection selects the regions with important information in the remote sensing images, which severely imitates the human visual system. It plays a powerful role in other image processing. It has successfully made great achievements in change detection, object tracking, temperature reversal, and other tasks. The traditional method has some disadvantages such as poor robustness and high computational complexity. Therefore, this paper proposes a deep multiscale fusion method via low-rank sparse decomposition for object saliency detection in optical remote sensing images. First, we execute multiscale segmentation for remote sensing images. Then, we calculate the saliency value, and the proposal region is generated. The superpixel blocks of the remaining proposal regions of the segmentation map are input into the convolutional neural network. By extracting the depth feature, the saliency value is calculated and the proposal regions are updated. The feature transformation matrix is obtained based on the gradient descent method, and the high-level semantic prior knowledge is obtained by using the convolutional neural network. The process is iterated continuously to obtain the saliency map at each scale. The low-rank sparse decomposition of the transformed matrix is carried out by robust principal component analysis. Finally, the weight cellular automata method is utilized to fuse the multiscale saliency graphs and the saliency map calculated according to the sparse noise obtained by decomposition. Meanwhile, the object priors knowledge can filter most of the background information, reduce unnecessary depth feature extraction, and meaningfully improve the saliency detection rate. The experiment results show that the proposed method can effectively improve the detection effect compared to other deep learning methods.

A Multi-Scale Approach for Graph Link Prediction

Deep models can be made scale-invariant when trained with multi-scale information. Images can be easily made multi-scale, given their grid-like structures. Extending this to generic graphs poses major challenges. For example, in link prediction tasks, inputs are represented as graphs consisting of nodes and edges. Currently, the state-of-the-art model for link prediction uses supervised heuristic learning, which learns graph structure features centered on two target nodes. It then learns graph neural networks to predict the existence of links based on graph structure features. Thus, the performance of link prediction models highly depends on graph structure features. In this work, we propose a novel node aggregation method that can transform the enclosing subgraph into different scales and preserve the relationship between two target nodes for link prediction. A theory for analyzing the information loss during the re-scaling procedure is also provided. Graphs in different scales can provide scale-invariant information, which enables graph neural networks to learn invariant features and improve link prediction performance. Our experimental results on 14 datasets from different areas demonstrate that our proposed method outperforms the state-of-the-art methods by employing multi-scale graphs without additional parameters.

HybridVis: An adaptive hybrid-scale visualization of multivariate graphs

Existing network visualizations support hierarchical exploration, which rely on user interactions to create and modify graph hierarchies based on the patterns in the data attributes. It will take a relatively long time for users to identify the impact of different attributes on the cluster structure. To address this problem, this paper proposes a visual analytical approach, called HybridVis, creating an interactive layout to reveal clusters of obvious characteristics on one or more attributes at different scales. HybridVis can help people gain social insight and better understand the roles of attributes within a cluster. First, an approximate optimal graph hierarchy based on an energy model is created, considering both data attributes and relationships among data items. Then a layout algorithm and a level-dependent perceptual view for multi-scale graphs are proposed to show the attribute-driven graph hierarchy. Several views, which interact with each other, are designed in HybridVis, including a graphical view of the relationships among clusters; a cluster tree revealing the cluster scales and the details of attributes on parallel coordinates augmented with histograms and interactions. From the meaningful and globally approximate optimal abstraction, users can navigate a large multivariate graph with an overview+detail to explore and rapidly find the potential correlations between the graph structure and the attributes of data items. Finally, experiments using two real world data sets are performed to demonstrate the effectiveness of our methods.

Representing and encoding plant architecture: A review

A plant is made up of components of various types and shapes. The geometrical and topological organisation of these components defines the plant architecture. Before the early 1970's, botanical drawings were the only means to represent plant architecture. In the past two decades, high-performance computers have become available for plant growth analysis and simulation, triggering the development of various formal representations and notations of plant architecture (strings of characters, axial trees, tree graphs, multiscale graphs, linked lists of records, object-oriented representations, matrices, fractals, sets of digitised points, etc.). In this paper, we review the main representations of plant architecture and make explicit their common structure and discrepancies. The apparent heterogeneity of these representations makes it difficult to collect plant architecture information in a generic format to allow multiple uses. However, the collection of plant architecture data is an increasingly important issue, which is also particularly time-consuming. At the end of this review, we suggest that a task of primary importance for the plant-modelling community is to define common data formats and tools in order to create standard plant architecture database systems that may be shared by research teams.