Long-range Spatial Dependencies Research Articles

Unsupervised MR-to-CT Synthesis Using Structure-Constrained CycleGAN.

Synthesizing a CT image from an available MR image has recently emerged as a key goal in radiotherapy treatment planning for cancer patients. CycleGANs have achieved promising results on unsupervised MR-to-CT image synthesis; however, because they have no direct constraints between input and synthetic images, cycleGANs do not guarantee structural consistency between these two images. This means that anatomical geometry can be shifted in the synthetic CT images, clearly a highly undesirable outcome in the given application. In this paper, we propose a structure-constrained cycleGAN for unsupervised MR-to-CT synthesis by defining an extra structure-consistency loss based on the modality independent neighborhood descriptor. We also utilize a spectral normalization technique to stabilize the training process and a self-attention module to model the long-range spatial dependencies in the synthetic images. Results on unpaired brain and abdomen MR-to-CT image synthesis show that our method produces better synthetic CT images in both accuracy and visual quality as compared to other unsupervised synthesis methods. We also show that an approximate affine pre-registration for unpaired training data can improve synthesis results.

Stochastic simulation of streamflow and spatial extremes: a continuous, wavelet-based approach

Abstract. Stochastically generated streamflow time series are used for various water management and hazard estimation applications. They provide realizations of plausible but as yet unobserved streamflow time series with the same temporal and distributional characteristics as the observed data. However, the representation of non-stationarities and spatial dependence among sites remains a challenge in stochastic modeling. We investigate whether the use of frequency-domain instead of time-domain models allows for the joint simulation of realistic, continuous streamflow time series at daily resolution and spatial extremes at multiple sites. To do so, we propose the stochastic simulation approach called Phase Randomization Simulation using wavelets (PRSim.wave) which combines an empirical spatio-temporal model based on the wavelet transform and phase randomization with the flexible four-parameter kappa distribution. The approach consists of five steps: (1) derivation of random phases, (2) fitting of the kappa distribution, (3) wavelet transform, (4) inverse wavelet transform, and (5) transformation to kappa distribution. We apply and evaluate PRSim.wave on a large set of 671 catchments in the contiguous United States. We show that this approach allows for the generation of realistic time series at multiple sites exhibiting short- and long-range dependence, non-stationarities, and unobserved extreme events. Our evaluation results strongly suggest that the flexible, continuous simulation approach is potentially valuable for a diverse range of water management applications where the reproduction of spatial dependencies is of interest. Examples include the development of regional water management plans, the estimation of regional flood or drought risk, or the estimation of regional hydropower potential. Highlights. Stochastic simulation of continuous streamflow time series using an empirical, wavelet-based, spatio-temporal model in combination with the parametric kappa distribution. Generation of stochastic time series at multiple sites showing temporal short- and long-range dependence, non-stationarities, and spatial dependence in extreme events. Implementation of PRSim.wave in R package PRSim: Stochastic Simulation of Streamflow Time Series using Phase Randomization.

A generalized approach based on convolutional neural networks for large area cropland mapping at very high resolution

Timely and accurate delineation of the cropland extent over large area is crucial for operational agriculture monitoring and is also beneficial to address food security issues. Existing global datasets associated with cropland are limited by insufficient spatial resolution to properly represent areas with small parcel size distributions, and their less-than-ideal accuracies hamper application at regional and local scales. Diverse very high spatial resolution (VHSR) satellite systems are now available, offering sub-meter to five-meter resolution (e.g. Gaofen-1, Gaofen-2, and ZiYuan-3), and hence enabling explicit extraction of cropland areas from heterogeneous and fragmented landscapes. This study presented a generalized methodology for operational cropland mapping at very high resolution using a deep convolutional neural network to automatically learn the robust and discriminative features. Specifically, we slightly modified the pyramid scene parsing network (PSPNet) and combined deep long-range features with shadow local features to provide predictions with high level of detail. We demonstrated the modified PSPNet (MPSPNet) over four province-wide study areas (Heilongjiang, Hebei, Zhejiang and Guangdong) with diverse agrosystems across China from north to south using multi-source very high spatial resolution satellite images (mainly Gaofen-1 supplemented with Gaofen-2 and ZiYuan-3), with the overall accuracies ranging from 89.99% to 92.31%. Moreover, we compared MPSPNet with other CNN models and investigated the its behavior by visualizing the learned features on different layers, indicating that combining low and high level features for final classification was an efficient and accurate strategy for cropland mapping because the former capture edge information related to object boundaries and the latter could learn long-range spatial dependencies that helped recognize croplands. The temporal transfer and spatial transfer assessments from the respects of qualitative and quantitative corroborated the robust generalizability of the proposed method. And the contrast to the traditional object-based classification method also demonstrated the advantages and strong generalization capabilities of MPSPNet in extracting cropland using VHSR remote sensed images. We compared our results with the current cropland maps generated from FROM-GLC10, which further verified the effectiveness of the proposed approach for large-scale cropland mapping at very high resolution.

Self-Attention ConvLSTM for Spatiotemporal Prediction

Spatiotemporal prediction is challenging due to the complex dynamic motion and appearance changes. Existing work concentrates on embedding additional cells into the standard ConvLSTM to memorize spatial appearances during the prediction. These models always rely on the convolution layers to capture the spatial dependence, which are local and inefficient. However, long-range spatial dependencies are significant for spatial applications. To extract spatial features with both global and local dependencies, we introduce the self-attention mechanism into ConvLSTM. Specifically, a novel self-attention memory (SAM) is proposed to memorize features with long-range dependencies in terms of spatial and temporal domains. Based on the self-attention, SAM can produce features by aggregating features across all positions of both the input itself and memory features with pair-wise similarity scores. Moreover, the additional memory is updated by a gating mechanism on aggregated features and an established highway with the memory of the previous time step. Therefore, through SAM, we can extract features with long-range spatiotemporal dependencies. Furthermore, we embed the SAM into a standard ConvLSTM to construct a self-attention ConvLSTM (SA-ConvLSTM) for the spatiotemporal prediction. In experiments, we apply the SA-ConvLSTM to perform frame prediction on the MovingMNIST and KTH datasets and traffic flow prediction on the TexiBJ dataset. Our SA-ConvLSTM achieves state-of-the-art results on both datasets with fewer parameters and higher time efficiency than previous state-of-the-art method.

Deep Reinforcement Learning Based Weather Monitoring Systemusing Arduino for Smart Environment

Weather forecasting is an essential predictive challenge that has depended primarily on model-based methods. Collection of data about the different weather parameters is needed a smart environment. Recent developments in machine learning (ML) made possible to collect the data. The data from input sensors is then read by Arduino, which acts as server. The sensors collect the data of various environmental parameters and provide it to Arduino, which act as a base station. It then transmits the data using WIFI and the processed data will be displayed on laptop through accessing the server that is on the receiver side. In this paper, new directions are explored with forecasting weather as a data intensive challenge that involves inferences across space and time. Machine Learningmakes predictions through a unique hybrid approach that combines discriminatively trained predictive models and a deep neural network. The Deep Learning algorithm utilized here is Value-Based – Temporal Difference Algorithm. This in turnmodels the joint statistics of a set of weather-related variables. It is shown that the base model can be enhanced with spatial interpolation that uses learned long-range spatial dependencies. An efficient learning and inference procedure is also devised, that allows for large scale optimization of the model parameters. The methods are evaluated with experiments on real-world meteorological data that highlight the promise of the approach.

Non-Local Context Encoder: Robust Biomedical Image Segmentation against Adversarial Attacks

Recent progress in biomedical image segmentation based on deep convolutional neural networks (CNNs) has drawn much attention. However, its vulnerability towards adversarial samples cannot be overlooked. This paper is the first one that discovers that all the CNN-based state-of-the-art biomedical image segmentation models are sensitive to adversarial perturbations. This limits the deployment of these methods in safety-critical biomedical fields. In this paper, we discover that global spatial dependencies and global contextual information in a biomedical image can be exploited to defend against adversarial attacks. To this end, non-local context encoder (NLCE) is proposed to model short- and long-range spatial dependencies and encode global contexts for strengthening feature activations by channel-wise attention. The NLCE modules enhance the robustness and accuracy of the non-local context encoding network (NLCEN), which learns robust enhanced pyramid feature representations with NLCE modules, and then integrates the information across different levels. Experiments on both lung and skin lesion segmentation datasets have demonstrated that NLCEN outperforms any other state-of-the-art biomedical image segmentation methods against adversarial attacks. In addition, NLCE modules can be applied to improve the robustness of other CNN-based biomedical image segmentation methods.

DeepSTN+: Context-Aware Spatial-Temporal Neural Network for Crowd Flow Prediction in Metropolis

Crowd flow prediction is of great importance in a wide range of applications from urban planning, traffic control to public safety. It aims to predict the inflow (the traffic of crowds entering a region in a given time interval) and outflow (the traffic of crowds leaving a region for other places) of each region in the city with knowing the historical flow data. In this paper, we propose DeepSTN+, a deep learning-based convolutional model, to predict crowd flows in the metropolis. First, DeepSTN+ employs the ConvPlus structure to model the longrange spatial dependence among crowd flows in different regions. Further, PoI distributions and time factor are combined to express the effect of location attributes to introduce prior knowledge of the crowd movements. Finally, we propose an effective fusion mechanism to stabilize the training process, which further improves the performance. Extensive experimental results based on two real-life datasets demonstrate the superiority of our model, i.e., DeepSTN+ reduces the error of the crowd flow prediction by approximately 8%∼13% compared with the state-of-the-art baselines.

Object Scene Flow

This work investigates the estimation of dense three-dimensional motion fields, commonly referred to as scene flow. While great progress has been made in recent years, large displacements and adverse imaging conditions as observed in natural outdoor environments are still very challenging for current approaches to reconstruction and motion estimation. In this paper, we propose a unified random field model which reasons jointly about 3D scene flow as well as the location, shape and motion of vehicles in the observed scene. We formulate the problem as the task of decomposing the scene into a small number of rigidly moving objects sharing the same motion parameters. Thus, our formulation effectively introduces long-range spatial dependencies which commonly employed local rigidity priors are lacking. Our inference algorithm then estimates the association of image segments and object hypotheses together with their three-dimensional shape and motion. We demonstrate the potential of the proposed approach by introducing a novel challenging scene flow benchmark which allows for a thorough comparison of the proposed scene flow approach with respect to various baseline models. In contrast to previous benchmarks, our evaluation is the first to provide stereo and optical flow ground truth for dynamic real-world urban scenes at large scale. Our experiments reveal that rigid motion segmentation can be utilized as an effective regularizer for the scene flow problem, improving upon existing two-frame scene flow methods. At the same time, our method yields plausible object segmentations without requiring an explicitly trained recognition model for a specific object class.

A Bayesian spatial model for neuroimaging data based on biologically informed basis functions

The dominant approach to neuroimaging data analysis employs the voxel as the unit of computation. While convenient, voxels lack biological meaning and their size is arbitrarily determined by the resolution of the image. Here, we propose a multivariate spatial model in which neuroimaging data are characterised as a linearly weighted combination of multiscale basis functions which map onto underlying brain nuclei or networks or nuclei. In this model, the elementary building blocks are derived to reflect the functional anatomy of the brain during the resting state. This model is estimated using a Bayesian framework which accurately quantifies uncertainty and automatically finds the most accurate and parsimonious combination of basis functions describing the data. We demonstrate the utility of this framework by predicting quantitative SPECT images of striatal dopamine function and we compare a variety of basis sets including generic isotropic functions, anatomical representations of the striatum derived from structural MRI, and two different soft functional parcellations of the striatum derived from resting-state fMRI (rfMRI). We found that a combination of ∼50 multiscale functional basis functions accurately represented the striatal dopamine activity, and that functional basis functions derived from an advanced parcellation technique known as Instantaneous Connectivity Parcellation (ICP) provided the most parsimonious models of dopamine function. Importantly, functional basis functions derived from resting fMRI were more accurate than both structural and generic basis sets in representing dopamine function in the striatum for a fixed model order. We demonstrate the translational validity of our framework by constructing classification models for discriminating parkinsonian disorders and their subtypes. Here, we show that ICP approach is the only basis set that performs well across all comparisons and performs better overall than the classical voxel-based approach. This spatial model constitutes an elegant alternative to voxel-based approaches in neuroimaging studies; not only are their atoms biologically informed, they are also adaptive to high resolutions, represent high dimensions efficiently, and capture long-range spatial dependencies, which are important and challenging objectives for neuroimaging data.

Asymptotic analysis of simultaneous damages in spatial Boolean models

A notion of the positive spatial association is introduced in this paper to analyze spatial dependence of Boolean models with the focus on estimating the long-range spatial dependence. The explicit tail estimates for probabilities of simultaneous damage to two distant spatial regions are obtained using the regular variation method, and the long-range spatial covariance for the Boolean models with heavy-tailed grains is shown to decay at the power-law rate that is smaller than the tail decay rate of grains. Examples and applications to spatial reliability modeling are also discussed.

A multi-scale probabilistic network model for detection, synthesis and compression in mammographic image analysis

We develop a probabilistic network model over image spaces and demonstrate its broad utility in mammographic image analysis, particularly with respect to computer-aided diagnosis. The model employs a multi-scale pyramid decomposition to factor images across scale and a network of tree-structured hidden variables to capture long-range spatial dependencies. This factoring makes the computation of the density functions local and tractable. The result is a hierarchical mixture of conditional probabilities, similar to a hidden Markov model on a tree. The model parameters are found with maximum likelihood estimation using the expectation-maximization algorithm. The utility of the model is demonstrated for three applications: (1) detection of mammographic masses for computer-aided diagnosis; (2) qualitative assessment of model structure through mammographic synthesis; and (3) compression of mammographic regions of interest.

Numerical Method for Conditional Simulation of Levy Random Fields

Stochastic simulations of subsurface heterogeneity require accurate statistical models for spatial fluctuations. Incremental values in subsurface properties were shown previously to be approximated accurately by Levy distributions in the center and in the start of the tails of the distribution. New simulation methods utilizing these observations have been developed. Multivariate Levy distributions are used to model the multipoint joint probability density. Explicit bounds on the simulated variables prevent nonphysical extreme values and introduce a cutoff in the tails of the distribution of increments. Long-range spatial dependence is introduced through off-diagonal terms in the Levy association matrix, which is decomposed to yield a maximum likelihood type estimate at unobserved locations. This procedure reduces to a known interpolation formula developed for Gaussian fractal fields in the situation of two control points. The conditional density is not univariate Levy and is not available in closed form, but can be constructed numerically. Sequential simulation algorithms utilizing the numerically constructed conditional density successfully reproduce the desired statistical properties in simulations.