Non-local Information Research Articles

Pseudoinverse graph convolutional networks

Graph Convolutional Networks (GCNs) have proven to be successful tools for semi-supervised classification on graph-based datasets. We propose a new GCN variant whose three-part filter space is targeted at dense graphs. Our examples include graphs generated from 3D point clouds with an increased focus on non-local information, as well as hypergraphs based on categorical data of real-world problems. These graphs differ from the common sparse benchmark graphs in terms of the spectral properties of their graph Laplacian. Most notably we observe large eigengaps, which are unfavorable for popular existing GCN architectures. Our method overcomes these issues by utilizing the pseudoinverse of the Laplacian. Another key ingredient is a low-rank approximation of the convolutional matrix, ensuring computational efficiency and increasing accuracy at the same time. We outline how the necessary eigeninformation can be computed efficiently in each applications and discuss the appropriate choice of the only metaparameter, the approximation rank. We finally showcase our method’s performance regarding runtime and accuracy in various experiments with real-world datasets.

Diffusion geometry of multiplex and interdependent systems.

Complex networks are characterized by latent geometries induced by their topology or by the dynamics on them. In the latter case, different network-driven processes induce distinct geometric features that can be captured by adequate metrics. Random walks, a proxy for a broad spectrum of processes, from simple contagion to metastable synchronization and consensus, have been recently used, Domenico [Phys. Rev. Lett. 118, 168301 (2017)PRLTAO0031-900710.1103/PhysRevLett.118.168301] to define the class of diffusion geometries and pinpoint the functional mesoscale organization of complex networks from a genuine geometric perspective. Here we first extend this class to families of distinct random walk dynamics-including local and nonlocal information-on multilayer networks-a paradigm for biological, neural, social, transportation, and financial systems-overcoming limitations such as the presence of isolated nodes and disconnected components, typical of real-world networks. We then characterize the multilayer diffusion geometry of synthetic and empirical systems, highlighting the role played by different random search dynamics in shaping the geometric features of the corresponding diffusion manifolds.

Automatic segmentation of plantar thermograms using adaptive C means technique

Diabetic foot ulcer (DFU) is one of the major concern of diabetes and it is rapidly increasing, in worst case scenario this may lead to amputation. The DFU can be avoided by the early detection and proper diagnosis. Many of the studies carried out highlights that, thermography is the most useful technique to measure the changes in the temperature of plantar surface and alerts to indicate the risk associated with DFU. The distribution of temperature does not have a fixed pattern across the patients, hence it makes the difficulty in measuring the appropriate changes. This gap will provide a scope to improve the analysis technique so as to measure the plantar surface temperature effectively and identify any abnormal changes. In this paper, the segmentation algorithm namely adaptive C means (ACM) for the image segmentation is discussed. ACM is based on the spatial information and this method includes the two stage implementation. In the first stage, nonlocal spatial information is added and in the second stage, spatial shape information is used in order to refine the constraint of local spatial. Outcome of the proposed method shows that ACM is very much effective and it outperforms the other existing methods.

The sociospatial factors of death: Analyzing effects of geospatially-distributed variables in a Bayesian mortality model for Hong Kong.

Human mortality is in part a function of multiple socioeconomic factors that differ both spatially and temporally. Adjusting for other covariates, the human lifespan is positively associated with household wealth. However, the extent to which mortality in a geographical region is a function of socioeconomic factors in both that region and its neighbors is unclear. There is also little information on the temporal components of this relationship. Using the districts of Hong Kong over multiple census years as a case study, we demonstrate that there are differences in how wealth indicator variables are associated with longevity in (a) areas that are affluent but neighbored by socially deprived districts versus (b) wealthy areas surrounded by similarly wealthy districts. We also show that the inclusion of spatially-distributed variables reduces uncertainty in mortality rate predictions in each census year when compared with a baseline model. Our results suggest that geographic mortality models should incorporate nonlocal information (e.g., spatial neighbors) to lower the variance of their mortality estimates, and point to a more in-depth analysis of sociospatial spillover effects on mortality rates.

Non-monotonic temperature evolution of nonlocal structure–dynamics correlation in CuZr glass-forming liquids* *Project supported by the National Natural Science Foundation of China (Grant Nos. 52031016 and 51631003).

The structure–dynamics correlations in a nonlocal manner were investigated in CuZr metallic glass-forming liquids via classical molecular dynamics simulations. A spatial coarse-graining approach was employed to incorporate the nonlocal structural information of given structural order parameters in the structure–dynamics relationship. It is found that the correlation between structure order parameters and dynamics increases with increasing coarse-graining length and has a characteristic length scale. Moreover, the characteristic correlation length exhibits a non-monotonic temperature evolution as temperature approaches glass transition temperature, which is not sensitive to the considered structure order parameters. Our results unveil a striking change in the structure–dynamics correlation, which involves no fitting theoretical interpretation. These findings provide new insight into the structure–dynamics correlation in glass transition.

Lightweight Image Restoration Network for Strong Noise Removal in Nuclear Radiation Scenes.

In order to remove the strong noise with complex shapes and high density in nuclear radiation scenes, a lightweight network composed of a Noise Learning Unit (NLU) and Texture Learning Unit (TLU) was designed. The NLU is bilinearly composed of a Multi-scale Kernel Module (MKM) and a Residual Module (RM), which learn non-local information and high-level features, respectively. Both the MKM and RM have receptive field blocks and attention blocks to enlarge receptive fields and enhance features. The TLU is at the bottom of the NLU and learns textures through an independent loss. The entire network adopts a Mish activation function and asymmetric convolutions to improve the overall performance. Compared with 12 denoising methods on our nuclear radiation dataset, the proposed method has the fewest model parameters, the highest quantitative metrics, and the best perceptual satisfaction, indicating its high denoising efficiency and rich texture retention.

Unsupervised Deep Learning Network with Self-Attention Mechanism for Non-Rigid Registration of 3D Brain MR Images

In non-rigid registration for medical imaging analysis, computation is complicated, and the high accuracy and robustness needed for registration are difficult to obtain. Recently, many studies have been conducted for nonrigid registration via unsupervised learning networks. This study proposes a method to improve the performance of this unsupervised learning network approach, through the use of a self-attention mechanism. In this paper, the self-attention mechanism is combined with deep learning networks to identify information of higher importance, among large amounts of data, and thereby solve specific tasks. Furthermore, the proposed method extracts both local and non-local information so that the network can create feature vectors with more information. As a result, the limitation of the existing network is addressed: alignment based solely on the entire silhouette of the brain is mitigated in favor of a network which also learns to perform registration of the parts of the brain that have internal structural characteristics. To the best of our knowledge, this is the first such utilization of the attention mechanism in this unsupervised learning network for non-rigid registration. The proposed attention network performs registration that takes into account the overall characteristics of the data, thus yielding more accurate matching results than those of the existing methods. In particular, matching is achieved with especially high accuracy in the gray matter and cortical ventricle areas, since these areas contain many of the structural features of the brain. The experiment was performed on 3D magnetic resonance images of the brains of 50 people. The measured average dice similarity coefficient after registration was 70.40%, which is an improvement of 17.48% compared to that before registration. This improvement indicates that application of the attention block can further improve the performance by an additional 8.5%, as relative to that without attention block. Ultimately, through implementation of non-rigid registration via the attention block method, the internal structure and overall shape of the brain can be addressed, without additional data input. Additionally, attention blocks have the advantage of being able to easily connect to existing networks without a significant computational overhead. Furthermore, by producing an attention map, the area of the brain around which registration was more performed can be visualized. This approach can be used for non-rigid registration with various types of medical imaging data.

Knowledge-based transfer fuzzy clustering with non-local spatial information for surface roughness measurement

Surface roughness is an important quality indicator of machining processing, which is often needed as a feedback. To meet the requirement of on-line roughness measurement, the machine vision-based measurement method has emerged in recent decades. In this paper, a new surface roughness measurement method is developed. It measures the surface roughness by evaluating the aliasing degree of the images reflected by the machined surface. To analyze the aliasing degree more accurately, a knowledge-based transfer fuzzy clustering with non-local spatial information (KTFCM_NLS) is proposed. The KTFCM_NLS incorporates both the auxiliary knowledge and non-local spatial information into the fuzzy c-means (FCM) algorithm. It can preserve more image details, and is robust to image noise. The results show that the proposed algorithm performs better than other FCM algorithms, and the corresponding index is strongly related to surface roughness. The average relative error of the proposed method is only 5.95% on the testing samples.

Global voxel transformer networks for augmented microscopy

Advances in deep learning have led to remarkable success in augmented microscopy, enabling us to obtain high-quality microscope images without using expensive microscopy hardware and sample preparation techniques. Current deep learning models for augmented microscopy are mostly U-Net-based neural networks, thus sharing certain drawbacks that limit the performance. In particular, U-Nets are composed of local operators only and lack dynamic non-local information aggregation. In this work, we introduce global voxel transformer networks (GVTNets), a deep learning tool for augmented microscopy that overcomes intrinsic limitations of the current U-Net-based models and achieves improved performance. GVTNets are built on global voxel transformer operators, which are able to aggregate global information, as opposed to local operators like convolutions. We apply the proposed methods on existing datasets for three different augmented microscopy tasks under various settings. Computational augmentation of microscopic images aims at reducing the need to chemically label or stain cells to extract information. The popular U-Net model often employed for these tasks uses mostly local information. A new method for augmenting microscopic images is presented that allows for global information to be used at each step of the process.

Multiobjective fuzzy clustering with multiple spatial information for Noisy color image segmentation

Clustering method is a widely used and effective technique in color image segmentation. In general, traditional clustering-based image segmentation algorithms consider only one objective function and the segmentation performance is easily influenced by the noise in the image. Therefore, utilizing many clustering criteria and the neighborhood statistic information of pixels are more effective to improve the segmentation performance. In this paper, we put forward a multiobjective fuzzy clustering algorithm with multiple spatial information (MFCMSI) for noisy color image segmentation. Firstly, two conflicting fitness functions including the local spatial information with a circular neighborhood window and the non-local spatial information with circular search and similarity windows are designed to improve the noise-resistant ability and segmentation performance. In order to optimize these two fitness functions, the variable-length and cluster-center-based encoding strategy and some efficient evolutionary operations are utilized in each generation. Finally, a cluster validity index with multiple spatial information is constructed to select the best solution from the final non-dominated solution set of the last generation. Aiming to improve the robustness of MFCMSI, the ensemble strategy is introduced into MFCMSI and an ensemble version of MFCMSI is presented. Experimental results show that MFCMSI and its ensemble version behave well in evolving the number of segments and obtaining the satisfactory segmentation performance.

Non-Local Aggregation for RGB-D Semantic Segmentation

Exploiting both RGB (2D appearance) and Depth (3D geometry) information can improve the performance of semantic segmentation. However, due to the inherent difference between the RGB and Depth information, it remains a challenging problem in how to integrate RGB-D features effectively. In this letter, to address this issue, we propose a Non-local Aggregation Network (NANet), with a well-designed Multi-modality Non-local Aggregation Module (MNAM), to better exploit the non-local context of RGB-D features at multi-stage. Compared with most existing RGB-D semantic segmentation schemes, which only exploit local RGB-D features, the MNAM enables the aggregation of non-local RGB-D information along both spatial and channel dimensions. The proposed NANet achieves comparable performances with state-of-the-art methods on popular RGB-D benchmarks, NYUDv2 and SUN-RGBD.

Contextualized Graph Attention Network for Recommendation with Item Knowledge Graph

Graph neural networks (GNN) have recently been applied to exploit knowledge graph (KG) for recommendation. Existing GNN-based methods explicitly model the dependency between an entity and its local graph context in KG (i.e., the set of its first-order neighbors), but may not be effective in capturing its non-local graph context (i.e., the set of most related high-order neighbors). In this paper, we propose a novel recommendation framework, named Contextualized Graph Attention Network (CGAT), which can explicitly exploit both local and non-local graph context information of an entity in KG. More specifically, CGAT captures the local context information by a user-specific graph attention mechanism, considering a user's personalized preferences on entities. In addition, CGAT employs a biased random walk sampling process to extract the non-local context of an entity, and utilizes a Recurrent Neural Network (RNN) to model the dependency between the entity and its non-local contextual entities. To capture the user's personalized preferences on items, an item-specific attention mechanism is also developed to model the dependency between a target item and the contextual items extracted from the user's historical behaviors. We compared CGAT with state-of-the-art KG-based recommendation methods on real datasets, and the experimental results demonstrate the effectiveness of CGAT.

Nonlocal Low-Rank and Prior Image-Based Reconstruction in a Wavelet Tight Frame Using Limited-Angle Projection Data

Computed tomography (CT) reconstruction for limited-angle projection data is an ill-posed inverse problem, that often produces artifacts near the edges of an image. In this study, a hybrid minimization model based on a nonlocal low-rank approximation and a prior image in a wavelet tight framework is proposed to improve reconstructions from limited-angle projections. Low-frequency wavelet coefficients of the reconstructed image were estimated using a nonlocal low-rank approximation and the $l_{2}$ norm minimization was applied to the difference between the high-frequency components of a prior image and the reconstructed image. In addition, the alternative direction method of multipliers (ADMM) was used for alternately minimization to solve two regularization terms which produce the most parameters. Experimental results demonstrated that the proposed algorithm offers several advantages over conventional iterative reconstruction techniques, including faster convergence, suppression of limited-angle artifacts, noise reduction, and the preservation of edges and other image details. This study represents the first time that nonlocal low-rank prior information has been applied to limited-angle CT.

Road Detection Network Based on Anti-Disturbance and Variable-Scale Spatial Context Detector

Road detection plays a critical role in the application of smart transportation. The performance of the mainstream methods such as PSPNet (Pyramid Scene Parsing Network), DeepLab V3, FCRN (Fully Convolutional Residual Networks) still suffers from uncertain disturbances of surface abrasion buildings, pedestrians, and variation of illumination like tree-shadow. The extracted features are vulnerable to extra-disturbance, and non-local spatial-context information has not been fully utilized. In this paper, a detector based on anti-disturbance and variable-scale spatial context features (AVD) is proposed: the training of the multi-layer features of the detector is always taken under the imposing of fake-feature-disturbance from an independent generator, which is trained to exacerbate the detector errors and the mistakes of feature discriminator. The detector is prepared to be immune from the fake-feature-disturbance, and the discriminator is trained to distinguish the differences between the non-interference features and disturbing features. We also designed a novel variable-scale spatial context module to enhance the richness performance of the extracted features. And a soft connection link is bridged between the low and high feature layers. The detection experiments on the Munich road dataset and urban road dataset show that AVD is better than all the mainstream above methods. Our method increases the accuracy by 3% on the Munich remote sensing dataset and 0.4% on the urban road dataset. Our code and datasets are available at https://github.com/Ding-Q/AVD for download.

Infrared Small Maritime Target Detection Based on Integrated Target Saliency Measure

Robust and effective detection of a small target in an infrared maritime image is a key technology of maritime target search and tracking applications. Infrared small target detection is a challenging task due to the factors such as dim small targets and various complex backgrounds caused by sun glitters and strong waves. In this article, the integrated target saliency measure (ITSM) based on local and nonlocal spatial information is proposed to improve target detection performance. We combine local heterogeneity property of targets and nonlocal self-correlation property of background with targets' sparsity to separate real targets from background clutters. First, local heterogeneity calculation based on cross-window standard deviation (CSD) is proposed to extract candidate targets preliminarily, which enhances the local intensity difference between small targets and neighboring background. Meanwhile, low-rank representation (LRR) is applied to background prediction and removal, which is followed by adaptive threshold segmentation to enhance target saliency. Finally, we integrate the results obtained from the two steps mentioned above to further enhance targets and suppress background clutters. Then, real targets are extracted by an iterative threshold on the integrated map so as to generate the seed map, in addition, the target expansion strategy is exploited to keep full target areas. Experimental results on six datasets show that the proposed method outperforms the state-of-the-art methods in terms of robustness and target detection accuracy.

Current Frame Priors Assisted Neural Network for Intra Prediction

Intra prediction is the key technology to reduce spatial redundancy in the modern video coding standard. Recently, deep learning based methods that directly generate the intra prediction by neural network achieve superior performance than traditional directional based intra prediction. However, these methods lack the ability to handle complex blocks which contain mixed directional textures or recurrent patterns since they only use the neighboring reference samples of the current one. The other intermediate information denoted as reference priors in this paper generated during the coding process is not exploited. In this paper, a Current Frame Priors assisted Neural Network (CFPNN) is presented to improve the intra prediction efficiency. Specifically, we utilize the local contextual information provided by the neighboring multiple references as the primary inference source. In addition to the neighboring references, we additionally use the other two reference priors within the current frame – the predictor searched by intra block copy (IntraBC) and the corresponding residual component. The IntraBC predictor provides useful nonlocal information to help generate more accurate prediction for complex blocks together with neighboring local information. While the residual component contains unique information that reflects the characteristics of the block to some extent is utilized to reduce the noise contained in the reconstructed reference samples. Moreover, we investigate the best way to integrate the proposed method into the codec. Experimental results demonstrate that compared to HEVC, our proposed CFPNN achieves an average of 4.1% BD-rate reduction for the luma component under the All Intra configuration.

Self-Supervised Ultrasound to MRI Fetal Brain Image Synthesis.

Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for second-trimester anomaly screening, for which ultrasound (US) is employed. Although expert sonographers are adept at reading US images, MR images which closely resemble anatomical images are much easier for non-experts to interpret. Thus in this article we propose to generate MR-like images directly from clinical US images. In medical image analysis such a capability is potentially useful as well, for instance for automatic US-MRI registration and fusion. The proposed model is end-to-end trainable and self-supervised without any external annotations. Specifically, based on an assumption that the US and MRI data share a similar anatomical latent space, we first utilise a network to extract the shared latent features, which are then used for MRI synthesis. Since paired data is unavailable for our study (and rare in practice), pixel-level constraints are infeasible to apply. We instead propose to enforce the distributions to be statistically indistinguishable, by adversarial learning in both the image domain and feature space. To regularise the anatomical structures between US and MRI during synthesis, we further propose an adversarial structural constraint. A new cross-modal attention technique is proposed to utilise non-local spatial information, by encouraging multi-modal knowledge fusion and propagation. We extend the approach to consider the case where 3D auxiliary information (e.g., 3D neighbours and a 3D location index) from volumetric data is also available, and show that this improves image synthesis. The proposed approach is evaluated quantitatively and qualitatively with comparison to real fetal MR images and other approaches to synthesis, demonstrating its feasibility of synthesising realistic MR images.

Unsupervised medical images denoising via graph attention dual adversarial network

A lot of natural images denoising methods have been proposed, however, there are dual primary challenges for medical images denoising: 1) paired datasets are scarce and 2) medical images are often three-dimensional. In this paper,we propose an U-net and graph attention network based end-to-end unsupervised deep residual generative adversarial network (GAT-URGAN). We add residual and attention blocks for excavating non-local detail information. In addition, for three-dimensional structure, we introduce graph attention layers to learn global structural detail information of inter-slices. To handle unpaired training data, we utilize deep network prior and introduce dual adversarial loss to constrain the generation of noise. We perform extensive experiments on Brainweb MRI datasets, which are divided into single noise images and mixed noise images, and accomplish reformative consequences compared to recent state-of-the-art denoising approaches.

Nonlocal Tensor-Based Sparse Hyperspectral Unmixing

Sparse unmixing is an important technique for analyzing and processing hyperspectral images (HSIs). Simultaneously exploiting spatial correlation and sparsity improves substantially abundance estimation accuracy. In this article, we propose to exploit nonlocal spatial information in the HSI for the sparse unmixing problem. Specifically, we first group similar patches in the HSI, and then unmix each group by imposing simultaneous a low-rank constraint and joint sparsity in the corresponding third-order abundance tensor. To this end, we build an unmixing model with a mixed regularization term consisting of the sum of the weighted tensor trace norm and the weighted tensor <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{2,1}$ </tex-math></inline-formula> -norm of the abundance tensor. The proposed model is solved under the alternating direction method of multipliers framework. We term the developed algorithm as the nonlocal tensor-based sparse unmixing algorithm. The effectiveness of the proposed algorithm is illustrated in experiments with both simulated and real hyperspectral data sets.

Deep Reinforcement Learning for Joint Channel Selection and Power Control in D2D Networks

Device-to-device (D2D) technology, which allows direct communications between proximal devices, is widely acknowledged as a promising candidate to alleviate the mobile traffic explosion problem. In this paper, we consider an overlay D2D network, in which multiple D2D pairs coexist on several orthogonal spectrum bands, i.e., channels. Due to spectrum scarcity, the number of D2D pairs is typically more than that of available channels, and thus multiple D2D pairs may use a single channel simultaneously. This may lead to severe co-channel interference and degrade network performance. To deal with this issue, we formulate a joint channel selection and power control optimization problem, with the aim to maximize the weighted-sum-rate (WSR) of the D2D network. Unfortunately, this problem is non-convex and NP-hard. To solve this problem, we first adopt the state-of-art fractional programming (FP) technique and develop an FP-based algorithm to obtain a near-optimal solution. However, the FP-based algorithm requires instantaneous global channel state information (CSI) for centralized processing, resulting in poor scalability and prohibitively high signalling overheads. Therefore, we further propose a distributed deep reinforcement learning (DRL)-based scheme, with which D2D pairs can autonomously optimize channel selection and transmit power by only exploiting local information and outdated nonlocal information. Compared with the FP-based algorithm, the DRL-based scheme can achieve better scalability and reduce signalling overheads significantly. Simulation results demonstrate that even without instantaneous global CSI, the performance of the DRL-based scheme can approach closely to that of the FP-based algorithm.