Long-range Spatial Interactions Research Articles

TCNet: Continuous Sign Language Recognition from Trajectories and Correlated Regions

A key challenge in continuous sign language recognition (CSLR) is to efficiently capture long-range spatial interactions over time from the video input. To address this challenge, we propose TCNet, a hybrid network that effectively models spatio-temporal information from Trajectories and Correlated regions. TCNet's trajectory module transforms frames into aligned trajectories composed of continuous visual tokens. This facilitates extracting region trajectory patterns. In addition, for a query token, self-attention is learned along the trajectory. As such, our network can also focus on fine-grained spatio-temporal patterns, such as finger movement, of a region in motion. TCNet's correlation module utilizes a novel dynamic attention mechanism that filters out irrelevant frame regions. Additionally, it assigns dynamic key-value tokens from correlated regions to each query. Both innovations significantly reduce the computation cost and memory. We perform experiments on four large-scale datasets: PHOENIX14, PHOENIX14-T, CSL, and CSL-Daily. Our results demonstrate that TCNet consistently achieves state-of-the-art performance. For example, we improve over the previous state-of-the-art by 1.5\% and 1.0\% word error rate on PHOENIX14 and PHOENIX14-T, respectively. Code is available at https://github.com/hotfinda/TCNet

Cross-Scope Spatial-Spectral Information Aggregation for Hyperspectral Image Super-Resolution.

Hyperspectral image super-resolution has attained widespread prominence to enhance the spatial resolution of hyperspectral images. However, convolution-based methods have encountered challenges in harnessing the global spatial-spectral information. The prevailing transformer-based methods have not adequately captured the long-range dependencies in both spectral and spatial dimensions. To alleviate this issue, we propose a novel cross-scope spatial-spectral Transformer (CST) to efficiently investigate long-range spatial and spectral similarities for single hyperspectral image super-resolution. Specifically, we devise cross-attention mechanisms in spatial and spectral dimensions to comprehensively model the long-range spatial-spectral characteristics. By integrating global information into the rectangle-window self-attention, we first design a cross-scope spatial self-attention to facilitate long-range spatial interactions. Then, by leveraging appropriately characteristic spatial-spectral features, we construct a cross-scope spectral self-attention to effectively capture the intrinsic correlations among global spectral bands. Finally, we elaborate a concise feed-forward neural network to enhance the feature representation capacity in the Transformer structure. Extensive experiments over three hyperspectral datasets demonstrate that the proposed CST is superior to other state-of-the-art methods both quantitatively and visually. The code is available at https://github.com/Tomchenshi/CST.git.

PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention.

Recently, transformer architectures have shown superior performance compared to their CNN counterparts in many computer vision tasks. The self-attention mechanism enables transformer networks to connect visual dependencies over short as well as long distances, thus generating a large, sometimes even a global receptive field. In this paper, we propose our Parallel Local-Global Vision Transformer (PLG-ViT), a general backbone model that fuses local window self-attention with global self-attention. By merging these local and global features, short- and long-range spatial interactions can be effectively and efficiently represented without the need for costly computational operations such as shifted windows. In a comprehensive evaluation, we demonstrate that our PLG-ViT outperforms CNN-based as well as state-of-the-art transformer-based architectures in image classification and in complex downstream tasks such as object detection, instance segmentation, and semantic segmentation. In particular, our PLG-ViT models outperformed similarly sized networks like ConvNeXt and Swin Transformer, achieving Top-1 accuracy values of 83.4%, 84.0%, and 84.5% on ImageNet-1K with 27M, 52M, and 91M parameters, respectively.

SFRNet: Fine-Grained Oriented Object Recognition via Separate Feature Refinement

Fine-grained oriented object recognition (FGO <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> R) is a practical need for intellectually interpreting remote sensing images. It aims at realizing fine-grained classification and precise localization with oriented bounding boxes, simultaneously. Our considerations for the task are general but decisive: (i) the extraction of subtle differences carries a big weight in differentiating fine-grained classes, and (ii) oriented localization prefers rotation-sensitive features. In this article, we propose a network with separate feature refinement (SFRNet), in which two transformer-based branches are designed to perform function-specific feature refinement for fine-grained classification and oriented localization, separately. To highlight the discriminative information advantageous to fine-grained classification, we propose a spatial and channel transformer (SC-Former) to capture both the long-range spatial interactions and the key correlations hidden in the feature channels. Besides, we design a Multi-RoI loss (MRL) following the protocol of deep metric learning to enhance the separability of fine-grained classes further. For oriented localization, we integrate the oriented response convolution with the transformer structure (namely, OR-Former) to assist in encoding rotation information during regression. Extensive experimental results validate the effectiveness and robustness of our SFRNet. Without bells and whistles, our SFRNet achieves state-of-the-art performance on the large-scale FAIR1M datasets (FAIR1M-1.0 and FAIR1M-2.0). Code will be available at https://github.com/Ranchosky/SFRNet.

Effective-dimension theory of critical phenomena above upper critical dimensions

Phase transitions and critical phenomena are among the most intriguing phenomena in nature and the renormalization-group theory for them is one of the greatest achievements of theoretical physics. However, the predictions of the theory above an upper critical dimension d c seriously disagree with reality. In addition to its fundamental significance, the problem is also of practical importance because both complex classical systems with long-range spatial or temporal interactions and quantum phase transitions with long-range interactions can substantially lower d c . The extant scenarios built on a dangerous irrelevant variable (DIV) to resolve the problem introduce two sets of critical exponents and even two sets of scaling laws whose origin is obscure. Here, we consider the DIV from a different perspective and clearly unveil the origin of the two sets of exponents and hence the intrinsic inconsistency in those scenarios. We then develop an effective-dimension theory in which critical fluctuations and system volume are fixed at an effective dimension by the DIV. This enables us to account for all the extant results consistently. A novel asymptotic finite-size scaling behavior for a correlation function together with a new anomalous dimension and its associated scaling law is also predicted. We also apply the theory to quantum phase transitions.

SPLNet: A sequence-to-one learning network with time-variant structure for regional wind speed prediction

Although regional wind speed prediction is of great value in the development of wind power, it is rarely studied in the past years. To fill this gap, we introduce a deep learning method for multi-step regional wind speed prediction, which aims to model the spatiotemporal dynamics of wind speed in a given region and make predictions across time steps. There are several important challenges for it: (1) how to simultaneously build long-term temporal dependencies and long-range spatial interactions of wind speed; (2) how to capture the fluctuant, intermittent, and chaotic nature of wind speed. In this paper, we propose a novel framework named Sequence-to-one Predictive Learning Net (SPLNet) for multi-step regional wind speed prediction. To effectively utilize historical information and build the long-term temporal dependencies, SPLNet regards multi-step prediction as multi-time sequence-to-one (Seq2one) learning, where each prediction is made based on a previous historical sequence. Besides, a new spatiotemporal dynamics attention (STD-Atten) unit is developed as the key component of SPLNet, which can realize Seq2one learning and capture long-range spatial dependencies. Moreover, to better model the non-stationary and violent nature of wind speed, SPLNet adopts a time-variant structure, where different STD-Atten units are used for predicting wind speed at different time steps. To stabilize model training and enhance prediction performance, a step-separated training procedure is put forward. Comprehensive experiments have been conducted on three real-world datasets, where regional wind speed is illustrated as images. Experimental results demonstrate the effectiveness and superiority of the proposed method over state-of-the-art techniques.

Deep Residual Involution Network for Hyperspectral Image Classification

Convolutional neural networks (CNNs) have achieved great results in hyperspectral image (HSI) classification in recent years. However, convolution kernels are reused among different spatial locations, known as spatial-agnostic or weight-sharing kernels. Furthermore, the preference of spatial compactness in convolution (typically, 3×3 kernel size) constrains the receptive field and the ability to capture long-range spatial interactions. To mitigate the above two issues, in this article, we combine a novel operation called involution with residual learning and develop a new deep residual involution network (DRIN) for HSI classification. The proposed DRIN could model long-range spatial interactions well by adopting enlarged involution kernels and realize feature learning in a fairly lightweight manner. Moreover, the vast and dynamic involution kernels are distinct over different spatial positions, which could prioritize the informative visual patterns in the spatial domain according to the spectral information of the target pixel. The proposed DRIN achieves better classification results when compared with both traditional machine learning-based and convolution-based methods on four HSI datasets. Especially in comparison with the convolutional baseline model, i.e., deep residual network (DRN), our involution-powered DRIN model increases the overall classification accuracy by 0.5%, 1.3%, 0.4%, and 2.3% on the University of Pavia, the University of Houston, the Salinas Valley, and the recently released HyRANK HSI benchmark datasets, respectively, demonstrating the potential of involution for HSI classification.

Inverting the variable fractional order in a variable-order space-fractional diffusion equation with variable diffusivity: analysis and simulation

Abstract Variable-order space-fractional diffusion equations provide very competitive modeling capabilities of challenging phenomena, including anomalously superdiffusive transport of solutes in heterogeneous porous media, long-range spatial interactions and other applications, as well as eliminating the nonphysical boundary layers of the solutions to their constant-order analogues. In this paper, we prove the uniqueness of determining the variable fractional order of the homogeneous Dirichlet boundary-value problem of the one-sided linear variable-order space-fractional diffusion equation with some observed values of the unknown solutions near the boundary of the spatial domain. We base on the analysis to develop a spectral-Galerkin Levenberg–Marquardt method and a finite difference Levenberg–Marquardt method to numerically invert the variable order. We carry out numerical experiments to investigate the numerical performance of these methods.

High-order time-stepping methods for two-dimensional Riesz fractional nonlinear reaction–diffusion equations

Anomalous diffusion and long-range spatial interactions in anisotropic media could be captured by considering the Riesz fractional derivatives rather than the classical Laplacian. While analytical solutions of the resulting fractional reaction–diffusion models may not be available, the numerical methods for approximating them are challenging. In this paper, fourth-order L-stable (ETDRK04) and A-stable (ETDRK22) linearly implicit predictor-corrector type time-stepping methods are presented. These methods are implemented on two-dimensional Riesz fractional nonlinear reaction–diffusion equations with smooth and non-smooth initial data. Three types of nonlinear reaction–diffusion models are considered: Allen–Cahn equation with cubic nonlinearity, Fisher’s equation with quadratic nonlinearity, and Enzyme Kinetics equation with rational nonlinearity. Fourth-order temporal convergence rate of the methods is proved analytically and computed numerically. Profiles of the numerical solutions corresponding to different orders and rates of diffusion are included. Computational efficiency of the ETDRK04 and ETDRK22 methods over the well known Cox–Matthews ETDRK4 is presented. The superiority of the provided methods, in terms of computational accuracy, efficiency, and reliability, is demonstrated through the numerical experiments.

Fast finite difference methods for space-time fractional partial differential equations in three space dimensions with nonlocal boundary conditions

Fractional partial differential equations (FPDEs) provide very competitive tools to model challenging phenomena involving anomalous diffusion or long-range memory and spatial interactions. However, numerical methods for space-time FPDEs generate dense stiffness matrices and involve numerical solutions at all the previous time steps, and so have O(N2+MN) memory requirement and O(MN3+M2N) computational complexity where N and M are the numbers of spatial unknowns and time steps, respectively.We develop and analyze a finite difference method (FDM) for space-time FPDEs in three space dimensions with a combination of Dirichlet and fractional Neumann boundary conditions, in which a shifted Grünwald discretization is used in space and an L1 discretization is used in time so the derived FDM has virtually first-order accuracy. We then derive different fast FDMs by carefully analyzing the structure of the stiffness matrix of numerical discretization as well as the coupling in the time direction. The resulting fast FDMs have an almost linear computational complexity and linear storage with respect to the number of spatial unknowns as well as time steps. Numerical experiments are presented to demonstrate the utility of the methods.

SEMANTIC BUILDING FAÇADE SEGMENTATION FROM AIRBORNE OBLIQUE IMAGES

Abstract. With the introduction of airborne oblique camera systems and the improvement of photogrammetric techniques, high-resolution 2D and 3D data can be acquired in urban areas. This high-resolution data allows us to perform detailed investigations on building roofs and façades which can contribute to LoD3 city modeling. Normally, façade segmentation is achieved from terrestrial views. In this paper, we address the problem from aerial views by using high resolution oblique aerial images as the data source in urban areas. In addition to traditional image features, such as RGB and SIFT, normal vector and planarity are also extracted from dense matching point clouds. Then, these 3D geometrical features are projected back to 2D space to assist façade interpretation. Random forest is trained and applied to label façade pixels. Fully connected conditional random field (CRF), capturing long-range spatial interactions, is used as a post-processing to refine our classification results. Its pairwise potential is defined by a linear combination of Gaussian kernels and the CRF model is efficiently solved by mean field approximation. Experiments show that 3D features can significantly improve classification results. Also, fully connected CRF performs well in correcting noisy pixels.

Spatially Correlated Gene Expression in Bacterial Groups: The Role of Lineage History, Spatial Gradients, and Cell-Cell Interactions.

Gene expression levels in clonal bacterial groups have been found to be spatially correlated. These correlations can partly be explained by the shared lineage history of nearby cells, although they could also arise from local cell-cell interactions. Here, we present a quantitative framework that allows us to disentangle the contributions of lineage history, long-range spatial gradients, and local cell-cell interactions to spatial correlations in gene expression. We study pathways involved in toxin production, SOS stress response, and metabolism in Escherichia coli microcolonies and find for all pathways that shared lineage history is the main cause of spatial correlations in gene expression levels. However, long-range spatial gradients and local cell-cell interactions also contributed to spatial correlations in SOS response, amino acid biosynthesis, and overall metabolic activity. Together, our data show that the phenotype of a cell is influenced by its lineage history and population context, raising the question of whether bacteria can arrange their activities in space to perform functions they cannot achieve alone.

A fast finite difference method for distributed-order space-fractional partial differential equations on convex domains

Fractional partial differential equations (PDEs) provide a powerful and flexible tool for modeling challenging phenomena including anomalous diffusion processes and long-range spatial interactions, which cannot be modeled accurately by classical second-order diffusion equations. However, numerical methods for space-fractional PDEs usually generate dense or full stiffness matrices, for which a direct solver requires O(N3) computations per time step and O(N2) memory, where N is the number of unknowns. The significant computational work and memory requirement of the numerical methods makes a realistic numerical modeling of three-dimensional space-fractional diffusion equations computationally intractable.Fast numerical methods were previously developed for space-fractional PDEs on multidimensional rectangular domains, without resorting to lossy compression, but rather, via the exploration of the tensor-product form of the Toeplitz-like decompositions of the stiffness matrices. In this paper we develop a fast finite difference method for distributed-order space-fractional PDEs on a general convex domain in multiple space dimensions. The fast method has an optimal order storage requirement and almost linear computational complexity, without any lossy compression. Numerical experiments show the utility of the method.

Incorporating Secondary Structural Features into Sequence Information for Predicting Protein Structural Class

Knowledge of structural classes is applied in numerous important predictive tasks that address structural and functional features of proteins, although the prediction accuracy of the protein structural classes is not high. In this study, 45 different features were rationally designed to model the differences between protein structural classes, among which, 30 of them reflect the combined protein sequence information. In terms of correlation function, the protein sequence can be converted to a digital signal sequence, from which we can generate 20 discrete Fourier spectrum numbers. According to the segments of amino with different characteristics occurring in protein sequences, the frequencies of the 10 kinds of segments of amino acid (motifs) in protein are calculated. Other features include the secondary structural information :10 features were proposed to model the strong adjacent correlations in the secondary structural elements and capture the long-range spatial interactions between secondary structures, other 5 features were designed to differentiate α/β from α+β classes , which is a major problem of the existing algorithm. The methods were proposed based on a large set of low-identity sequences for which secondary structure is predicted from their sequence (based on PSI-PRED). By means of this method, the overall prediction accuracy of four benchmark datasets were all improved. Especially for the dataset FC699, 25PDB and D1189 which are 1.26%, 1% and 0.85% higher than the best previous method respectively.

Quantum criticality in spin chains with non-Ohmic dissipation

We investigate the critical behavior of a spin chain coupled to bosonic baths characterized by a spectral density proportional to $\omega^s$, with $s>1$. Varying $s$ changes the effective dimension $d_\text{eff} = d + z$ of the system, where $z$ is the dynamical critical exponent and the number of spatial dimensions $d$ is set to one. We consider two extreme cases of clock models, namely Ising-like and U(1)-symmetric ones, and find the critical exponents using Monte Carlo methods. The dynamical critical exponent and the anomalous scaling dimension $\eta$ are independent of the order parameter symmetry for all values of $s$. The dynamical critical exponent varies continuously from $z \approx 2$ for $s=1$ to $z=1$ for $s=2$, and the anomalous scaling dimension evolves correspondingly from $\eta \gtrsim 0$ to $\eta = 1/4$. The latter exponent values are readily understood from the effective dimensionality of the system being $d_\text{eff} \approx 3$ for $s=1$, while for $s=2$ the anomalous dimension takes the well-known exact value for the 2D Ising and XY models, since then $d_{\rm{eff}}=2$. A noteworthy feature is, however, that $z$ approaches unity and $\eta$ approaches 1/4 for values of $s < 2$, while naive scaling would predict the dissipation to become irrelevant for $s=2$. Instead, we find that $z=1,\eta=1/4$ for $s \approx 1.75$ for both Ising-like and U(1) order parameter symmetry. These results lead us to conjecture that for all site-dissipative $Z_q$ chains, these two exponents are related by the scaling relation $z = \text{max} {(2-\eta)/s, 1}$. We also connect our results to quantum criticality in nondissipative spin chains with long-range spatial interactions.

Surround suppression and facilitation in the fovea: Very long-range spatial interactions in contrast perception

Surround modulation of perceived contrast has been almost exclusively studied in short-range conditions, i.e., in situations where a tiny gap, at most, separates center from surround. Existing long-range studies suggest that suppression extends to 12-cycle distance, whereas facilitation of perceived contrast is suggested to arise from visual field regions enclosing the center. In V1 neurons, however, long-range surround modulation involves both suppression and facilitation. Thus, we investigated short- and long-range surround modulation by measuring the perceived contrast of a center in the presence of a surround either near (0.3 cycles, 0.1 degree) or far (19.8 cycles, 6.6 degrees) from the center. This study demonstrates that in addition to the well-known suppression, surround modulation involves remarkably long-range facilitation of perceived contrast. At low center contrasts, the long-range facilitation was stronger than the long-range suppression, whereas at high center contrast we found mainly long-range suppression. Because the current models of perceived contrast could not account for our data, we considered our results in the context of models developed for surround modulation in V1 neurons. However, neither mechanistic nor phenomenological models proved satisfactory. Moreover, with the current knowledge, it seems that straightforward pooling of V1 neurons' responses cannot account for surround modulation of perceived contrast.

Visual Perceptual Organization Deficits in Alzheimer’s Dementia

Background: Deficits in visual processing are a prominent feature of Alzheimer’s disease (AD), yet the cognitive and neuropathological mechanisms underlying these deficits are poorly understood. In the current study, we examined the hypothesis that perceptual organization is specifically impaired in AD compared to other dementias. Methods: We examined perceptual organization with a psychophysically well-controlled measure of contour integration in patients with AD (n = 17), vascular dementia (n = 5), frontotemporal dementia (n = 5) and mild cognitive impairment (n = 10) and 11 age-matched healthy controls. Results: Patients with AD differed significantly in their ability to detect contours comprised of Gabor elements as detection relied increasingly on long-range spatial interactions. Impairments in contour integration were particularly pronounced in AD patients with atrophy and gliosis of white matter in the occipital lobe. Deficits in perceptual organization were not found in patients with other dementias and participants with mild cognitive impairment. Conclusions: These results suggest that a subgroup of AD patients is characterized by a specific deficit in visual perceptual organization, which might reflect the impaired functional integrity of occipital cortico-cortical pathways.

Ordered droplets in quantum magnets with long-range interactions

A defect coupling to the square of the order parameter in a nearly quantum-critical magnet can nucleate an ordered droplet while the bulk system is in the paramagnetic phase. We study the influence of long-range spatial interactions of the form r - ( d + σ ) on the droplet formation. To this end, we solve a Landau–Ginzburg–Wilson free energy in saddle-point approximation. The long-range interaction causes the droplet to develop an energetically unfavorable power-law tail. However, for σ > 0 , the free energy contribution of this tail is subleading in the limit of large droplets; and the droplet formation is controlled by the defect bulk. Thus, for large defects, long-range interactions do not hinder the formation of droplets.

Local defect in a magnet with long-range interactions

We investigate a single defect coupling to the square of the order parameter in a nearly critical magnet with long-range spatial interactions of the form $r^{-(d+\sigma)}$, focusing on magnetic droplets nucleated at the defect while the bulk system is in the paramagnetic phase. Because of the long-range interaction, the droplet develops a power-law tail which is energetically unfavorable. However, as long as $\sigma>0$, the tail contribution to the droplet free energy is subleading in the limit of large droplets; and the free energy becomes identical to the case of short-range interactions. We also study the droplet quantum dynamics with and without dissipation; and we discuss the consequences of our results for defects in itinerant quantum ferromagnets.

Short- and long-range spatial interactions: A redefinition

First level short- and long-range spatial interactions are considered to be processed in the primary visual cortex. In psychophysics, they are measured with two kinds of stimuli, Gabor patches and lines/points. Each has its own short- and long-range definitions. We show that first, in terms of visual angle separation, the two definitions do not correspond to identical scales of interactions and second, that Gabor data can be matched to the lines/points definition by properly considering the observed effects. As a consequence, three regimes of spatial interaction are present: a case where overlapping of stimuli is present, and two others for spatially separated stimuli which we define as the short- and long-range regimes. Both types of stimuli show compatible lateral interactions and, we think, permit the measurement of the same underlying mechanisms.