Spatial Priors Research Articles

CS2DIPs: Unsupervised HSI Super-Resolution Using Coupled Spatial and Spectral DIPs.

In recent years, fusing high spatial resolution multispectral images (HR-MSIs) and low spatial resolution hyperspectral images (LR-HSIs) has become a widely used approach for hyperspectral image super-resolution (HSI-SR). Various unsupervised HSI-SR methods based on deep image prior (DIP) have gained wide popularity thanks to no pre-training requirement. However, DIP-based methods often demonstrate mediocre performance in extracting latent information from the data. To resolve this performance deficiency, we propose a coupled spatial and spectral deep image priors (CS2DIPs) method for the fusion of an HR-MSI and an LR-HSI into an HR-HSI. Specifically, we integrate the nonnegative matrix-vector tensor factorization (NMVTF) into the DIP framework to jointly learn the abundance tensor and spectral feature matrix. The two coupled DIPs are designed to capture essential spatial and spectral features in parallel from the observed HR-MSI and LR-HSI, respectively, which are then used to guide the generation of the abundance tensor and spectral signature matrix for the fusion of the HSI-SR by mode-3 tensor product, meanwhile taking some inherent physical constraints into account. Free from any training data, the proposed CS2DIPs can effectively capture rich spatial and spectral information. As a result, it exhibits much superior performance and convergence speed over most existing DIP-based methods. Extensive experiments are provided to demonstrate its state-of-the-art overall performance including comparison with benchmark peer methods.

Improved MRF Reconstruction via Structure-Preserved Graph Embedding Framework.

Highly undersampled schemes in magnetic resonance fingerprinting (MRF) typically lead to aliasing artifacts in reconstructed images, thereby reducing quantitative imaging accuracy. Existing studies mainly focus on improving the reconstruction quality by incorporating temporal or spatial data priors. However, these methods seldom exploit the underlying MRF data structure driven by imaging physics and usually suffer from high computational complexity due to the high-dimensional nature of MRF data. In addition, data priors constructed in a pixel-wise manner struggle to incorporate non-local and non-linear correlations. To address these issues, we introduce a novel MRF reconstruction framework based on the graph embedding framework, exploiting non-linear and non-local redundancies in MRF data. Our work remodels MRF data and parameter maps as graph nodes, redefining the MRF reconstruction problem as a structure-preserved graph embedding problem. Furthermore, we propose a novel scheme for accurately estimating the underlying graph structure, demonstrating that the parameter nodes inherently form a low-dimensional representation of the high-dimensional MRF data nodes. The reconstruction framework is then built by preserving the intrinsic graph structure between MRF data nodes and parameter nodes and extended to exploiting the globality of graph structure. Our approach integrates the MRF data recovery and parameter map estimation into a single optimization problem, facilitating reconstructions geared toward quantitative accuracy. Moreover, by introducing graph representation, our methods substantially reduce the computational complexity, with the computational cost showing a minimal increase as the data acquisition length grows. Experiments show that the proposed method can reconstruct high-quality MRF data and multiple parameter maps within reduced computational time.

DERnet: a deep neural network for end-to-end reconstruction in magnetic particle imaging

Objective. Magnetic particle imaging (MPI) shows potential for contributing to biomedical research and clinical practice. However, MPI images are effectively affected by noise in the signal as its reconstruction is an ill-posed inverse problem. Thus, effective reconstruction method is required to reduce the impact of the noise while mapping signals to MPI images. Traditional methods rely on the hand-crafted data-consistency (DC) term and regularization term based on spatial priors to achieve noise-reducing and reconstruction. While these methods alleviate the ill-posedness and reduce noise effects, they may be difficult to fully capture spatial features. Approach. In this study, we propose a deep neural network for end-to-end reconstruction (DERnet) in MPI that emulates the DC term and regularization term using the feature mapping subnetwork and post-processing subnetwork, respectively, but in a data-driven manner. By doing so, DERnet can better capture signal and spatial features without relying on hand-crafted priors and strategies, thereby effectively reducing noise interference and achieving superior reconstruction quality. Main results. Our data-driven method outperforms the state-of-the-art algorithms with an improvement of 0.9–8.8 dB in terms of peak signal-to-noise ratio under various noise levels. The result demonstrates the advantages of our approach in suppressing noise interference. Furthermore, DERnet can be employed for measured data reconstruction with improved fidelity and reduced noise. In conclusion, our proposed method offers performance benefits in reducing noise interference and enhancing reconstruction quality by effectively capturing signal and spatial features. Significance. DERnet is a promising candidate method to improve MPI reconstruction performance and facilitate its more in-depth biomedical application.

SSPNet: Scale and spatial priors guided generalizable and interpretable pedestrian attribute recognition

Global feature based Pedestrian Attribute Recognition (PAR) models are often poorly localized when using Grad-CAM for attribute response analysis, which has a significant impact on the interpretability, generalizability and performance. Previous researches have attempted to improve generalization and interpretation through meticulous model design, yet they often have neglected or underutilized effective prior information crucial for PAR. To this end, a novel Scale and Spatial Priors Guided Network (SSPNet) is proposed for PAR, which is mainly composed of the Adaptive Feature Scale Selection (AFSS) and Prior Location Extraction (PLE) modules. The AFSS module learns to provide reasonable scale prior information for different attribute groups, allowing the model to focus on different levels of feature maps with varying semantic granularity. The PLE module reveals potential attribute spatial prior information, which avoids unnecessary attention on irrelevant areas and lowers the risk of model over-fitting. More specifically, the scale prior in AFSS is adaptively learned from different layers of feature pyramid with maximum accuracy, while the spatial priors in PLE can be revealed from part feature with different granularity (such as image blocks, human pose keypoint and sparse sampling points). Besides, a novel IoU based attribute localization metric is proposed for Weakly-supervised Pedestrian Attribute Localization (WPAL) based on the improved Grad-CAM for attribute response mask. The experimental results on the intra-dataset and cross-dataset evaluations demonstrate the effectiveness of our proposed method in terms of mean accuracy (mA). Furthermore, it also achieves superior performance on the PCS dataset for attribute localization in terms of IoU. Code will be released at https://github.com/guotengg/SSPNet.

Neural Adaptive Loop Filtering for Video Coding: Exploring Multi-Hypothesis Sample Refinement

Adaptive loop filtering (ALF) is extensively investigated for lossy video coding to mitigate compression noise. Numerous learning-based ALFs have emerged recently and improved the coding efficiency significantly through the use of complexity-intensive, large-scale models trained on excessive samples, making it impractical for real-life applications. By contrast, lightweight, small-scale ALF models cannot promise convincing performance and model generalization. In principle, the ALF estimates the sample distortion for restoration. Instead of directly approximating the distortion as in existing solutions, we reformulate it as a Multi-hypothesis Sample Refinement (MSR) problem. To this end, we first generate multiple distortion hypotheses through a deep neural network (DNN) model. Then, these hypotheses are linearly superimposed to approximate the final distortion through the minimization of mean square error (MMSE) between the filtered reconstruction and its original, uncompressed input. Finally, the linear superimposition coefficients are explicitly signaled in the compressed bitstream. As seen, the superimposition coefficients inherently generalize the MSR to various content. And using DNNs to generate distortion hypotheses essentially models the spatial priors of a local block and its underlying compression error distribution. As a result, the MSR using a small-scale convolutional neural network (CNN) model with only 5k parameters and 5 KMACs/pixel achieves 4.35% (Intra) and 2.49% (Inter) BD-Rate (Bjontegaard Delta Rate) gains over the AV1 anchor. Ablation studies further demonstrate that the MSR can be generalized to diverse small-scale network structures, different standards (e.g., H.265/HEVC and H.266/VVC), and diverse video content (e.g., screen videos).

Bayesian spatiotemporal modelling for disease mapping: an application to preeclampsia and gestational diabetes in Florida, United States.

Morbidities generally show patterns of concentration that vary by space and time. Disease mapping models are useful in estimating the spatiotemporal patterns of disease risks and are therefore pivotal for effective disease surveillance, resource allocation, and the development of prevention strategies. This study considers six spatiotemporal Bayesian hierarchical models based on two spatial conditional autoregressive priors. It could serve as a guideline on the development and application of Bayesian hierarchical models to assess the emerging risk trends, risk clustering, and spatial inequality trends, with estimation of covariables' effects on the interested disease risk. The method is applied to the Florida Birth Record data between 2006 and 2015 to study two cardiovascular risk factors: preeclampsia and gestational diabetes. High-risk clusters were detected in North Central Florida for preeclampsia and in Central Florida for gestational diabetes. While the adjusted disease trend was stable, spatial inequality peaked in 2011-2012 for both diseases. Exposure to PM2.5 at first or/and second trimester increased the risk of preeclampsia and gestational diabetes, but the magnitude is less severe compared to previous studies. In conclusion, this study underscores the significance of selecting appropriate disease mapping models in estimating the intricate spatiotemporal patterns of disease risk and suggests the importance of localized interventions to reduce health disparities. The result also identified an opportunity to study potential risk factors of preeclampsia, as the spike of risk in North Central Florida cannot be explained by current covariables.

Estimation of fMRI responses related to epileptic discharges using Bayesian hierarchical modeling.

Simultaneous electroencephalography-functional MRI (EEG-fMRI) is a unique and noninvasive method for epilepsy presurgical evaluation. When selecting voxels by null-hypothesis tests, the conventional analysis may overestimate fMRI response amplitudes related to interictal epileptic discharges (IEDs), especially when IEDs are rare. We aimed to estimate fMRI response amplitudes represented by blood oxygen level dependent (BOLD) percentage changes related to IEDs using a hierarchical model. It involves the local and distributed hemodynamic response homogeneity to regularize estimations. Bayesian inference was applied to fit the model. Eighty-two epilepsy patients who underwent EEG-fMRI and subsequent surgery were included in this study. A conventional voxel-wise general linear model was compared to the hierarchical model on estimated fMRI response amplitudes and on the concordance between the highest response cluster and the surgical cavity. The voxel-wise model overestimated fMRI responses compared to the hierarchical model, evidenced by a practically and statistically significant difference between the estimated BOLD percentage changes. Only the hierarchical model differentiated brief and long-lasting IEDs with significantly different BOLD percentage changes. Overall, the hierarchical model outperformed the voxel-wise model on presurgical evaluation, measured by higher prediction performance. When compared with a previous study, the hierarchical model showed higher performance metric values, but the same or lower sensitivity. Our results demonstrated the capability of the hierarchical model of providing more physiologically reasonable and more accurate estimations of fMRI response amplitudes induced by IEDs. To enhance the sensitivity of EEG-fMRI for presurgical evaluation, it may be necessary to incorporate more appropriate spatial priors and bespoke decision strategies.

Vision-Guided Attention-Enhanced Network for Predicting Microvascular Invasion in Hepatocellular Carcinoma.

As the most common malignant tumor worldwide, hepatocellular carcinoma (HCC) has a high rate of death and recurrence, and microvascular invasion (MVI) is considered to be an independent risk factor affecting its early recurrence and poor survival rate. Accurate preoperative prediction of MVI is of great significance for the formulation of individualized treatment plans and long-term prognosis assessment for HCC patients. However, as the mechanism of MVI is still unclear, existing studies use deep learning methods to directly train CT or MR images, with limited predictive performance and lack of explanation. We map the pathological "7-point" baseline sampling method used to confirm the diagnosis of MVI onto MR images, propose a vision-guided attention-enhanced network to improve the prediction performance of MVI, and validate the prediction on the corresponding pathological images reliability of the results. Specifically, we design a learnable online class activation map (CAM) to guide the network to focus on high-incidence regions of MVI guided by an extended tumor mask. Further, an attention-enhanced module is proposed to force the network to learn image regions that can explain the MVI results. The generated attention maps capture long-distance dependencies and can be used as spatial priors for MVI to promote the learning of vision-guided module. The experimental results on the constructed multi-center dataset show that the proposed algorithm achieves the state-of-the-art compared to other models.

Towards Real-Time Panoptic Narrative Grounding by an End-to-End Grounding Network

Panoptic Narrative Grounding (PNG) is an emerging cross-modal grounding task, which locates the target regions of an image corresponding to the text description. Existing approaches for PNG are mainly based on a two-stage paradigm, which is computationally expensive. In this paper, we propose a one-stage network for real-time PNG, termed End-to-End Panoptic Narrative Grounding network (EPNG), which directly generates masks for referents. Specifically, we propose two innovative designs, i.e., Locality-Perceptive Attention (LPA) and a bidirectional Semantic Alignment Loss (SAL), to properly handle the many-to-many relationship between textual expressions and visual objects. LPA embeds the local spatial priors into attention modeling, i.e., a pixel may belong to multiple masks at different scales, thereby improving segmentation. To help understand the complex semantic relationships, SAL proposes a bidirectional contrastive objective to regularize the semantic consistency inter modalities. Extensive experiments on the PNG benchmark dataset demonstrate the effectiveness and efficiency of our method. Compared to the single-stage baseline, our method achieves a significant improvement of up to 9.4% accuracy. More importantly, our EPNG is 10 times faster than the two-stage model. Meanwhile, the generalization ability of EPNG is also validated by zero-shot experiments on other grounding tasks. The source codes and trained models for all our experiments are publicly available at https://github.com/Mr-Neko/EPNG.git.

A deep unrolling pansharpening method based on spectral consistency and double spatial priors

ABSTRACT In recent years, deep learning (DL) pansharpening methods have become the most popular solution for acquiring high-resolution multispectral (HRMS) images. However, most DL-based methods design the task as black-box network architecture, ignoring the physical meaning of individual network modules. Such a network structure is difficult to extract sufficient spatial details or lacks interpretability. To address this problem, we propose a neural network pansharpening method with interpretable deep spatial detail injection based on the assumption of spectral consistency and double spatial detail priors in HRMS images. First, we assume that the HRMS image consists of a spectral and spatial component together. Spectral and partial spatial details are obtained by upsampling the multispectral (MS) images. Next, a variational model is constructed for image fusion from the LRMS image and the panchromatic (PAN) image to the HRMS image.The alternating direction multiplier method (ADMM) algorithm is then used to optimally solve the variational model to obtain the spatial detail that needs to be injected into the MS image. Finally, the model optimization solving process is expanded into a corresponding neural network structure using the idea of deep unrolling. Each network module corresponds to the solving step of the iterative algorithm. Our proposed pansharpening method is therefore interpretable in terms of spatial detail injection. In our experiments, we find that the proposed method achieves an optimal balance between spectral and spatial quality, and has a strong generalization capability over different datasets, demonstrating the superiority of the proposed pansharpening method.

Toward Efficient Process Monitoring Using Spatiotemporal PCA

Principal component analysis (PCA) has shown its high efficiency in process monitoring (PM). However, most of the existing PCA-based PM approaches only consider the spatial prior and ignore the temporal prior. Therefore, in this brief, we propose a novel PM framework using spatiotemporal PCA (STPCA), which incorporates both spatial and temporal priors. Technically, the spatial prior is integrated to preserve the cause-effect relationship of process variables, and the temporal prior is embedded to maintain the geometric structure of process samples. Moreover, an efficient optimization algorithm is developed using the alternating direction method of multipliers (ADMM) in a symmetric Gauss-Seidel (sGS) manner. Finally, the improved monitoring performance is verified on the benchmark Tennessee Eastman (TE) process. In particular, compared with PCA, the fault detection rate of fault IDV(20) is increased by 9.88%. This suggests that the proposed framework is promising for PM.

Assessing dynamic brain activity during verbal associative learning using MEG/fMRI co-processing

BackgroundThe abilities of individual neuroimaging methods to resolve spatial and temporal contributions of brain regions during cognitive processes are limited. Co-processing of functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) may overcome some of the limitations by utilizing Multiple Sparse Priors (MSP) in a Bayesian framework that takes advantage of the temporal resolution of MEG and spatial resolution of fMRI. Methods24 healthy participants were recruited to perform a paired-associate verbal learning task during fMRI and MEG scans. FMRI data were processed within Group ICA fMRI Toolbox. Independent components (ICs) were temporally sorted by task time series (|r|>0.30 threshold identified task-related ICs). Task-positive (“generate”) ICs were retained as spatial priors for MEG analyses. MEG data were processed by an event-related potential (ERP) approach and with a theta power approach. MEG source reconstructions were constrained within the task-positive ICs for both ERP and theta-power approaches. ResultsFor fMRI, five networks were identified as task-related. Four ICs underlying active generation spanned bilateral parietal, orbitofrontal, medial frontal and superior temporal regions, and occipital lobe. FMRI-constrained MEG source reconstructions using the ERP approach yielded early visual cortex activity followed by left inferior frontal gyrus (IFG) and orbito-frontal cortex (OFC) recruitment to coalesce in the left inferior temporal lobe. For the theta approach, MEG source reconstructions showed a progression of activity from bilateral temporal areas to left OFC and middle temporal gyrus, followed by right IFG. DiscussionMSP analyses informed by fMRI produced more focused regional activity than reconstructions without priors suggesting this approach may result in identifying more relevant semantic information during active generation. Constraining MEG source reconstruction to fMRI priors during active generation indicates fronto-temporal and fronto-parietal networks are interconnected across time and space.

Pansharpening With Spatial Hessian Non-Convex Sparse and Spectral Gradient Low Rank Priors.

To get the high resolution multi-spectral (HRMS) images by the fusion of low resolution multi-spectral (LRMS) and panchromatic (PAN) images, an effectively pansharpening model with spatial Hessian non-convex sparse and spectral gradient low rank priors (PSHNSSGLR) is proposed in this paper. In particularly, from the statistical aspect of view, the spatial Hessian hyper-Laplacian non-convex sparse prior is developed to model the spatial Hessian consistency between HRMS and PAN. More importantly, it is recently the first work for pansharpening modeling with the spatial Hessian hyper-Laplacian non-convex sparse prior. Meanwhile, the spectral gradient low rank prior on HRMS is further developed for spectral feature preservation. Then, the alternating direction method of multipliers (ADMM) approach is applied for optimizing the proposed PSHNSSGLR model. Afterwards, many fusion experiments demonstrate the capability and superiority of PSHNSSGLR.

Unmixing Guided Unsupervised Network for RGB Spectral Super-Resolution.

Spectral super-resolution has attracted research attention recently, which aims to generate hyperspectral images from RGB images. However, most of the existing spectral super-resolution algorithms work in a supervised manner, requiring pairwise data for training, which is difficult to obtain. In this paper, we propose an Unmixing Guided Unsupervised Network (UnGUN), which does not require pairwise imagery to achieve unsupervised spectral super-resolution. In addition, UnGUN utilizes arbitrary other hyperspectral imagery as the guidance image to guide the reconstruction of spectral information. The UnGUN mainly includes three branches: two unmixing branches and a reconstruction branch. Hyperspectral unmixing branch and RGB unmixing branch decompose the guidance and RGB images into corresponding endmembers and abundances respectively, from which the spectral and spatial priors are extracted. Meanwhile, the reconstruction branch integrates the above spectral-spatial priors to generate a coarse hyperspectral image and then refined it. Besides, we design a discriminator to ensure that the distribution of generated image is close to the guidance hyperspectral imagery, so that the reconstructed image follows the characteristics of a real hyperspectral image. The major contribution is that we develop an unsupervised framework based on spectral unmixing, which realizes spectral super-resolution without paired hyperspectral-RGB images. Experiments demonstrate the superiority of UnGUN when compared with some SOTA methods.

Underwater Image Enhancement via Piecewise Color Correction and Dual Prior Optimized Contrast Enhancement

Due to the absorption and scattering of light, underwater captured images often face serious quality degradation issues. In this letter, we propose to cope with the aforementioned issues via piecewise color correction and dual prior optimized contrast enhancement. Specifically, we first present the piecewise color correction method using the maximum mean and two gain factors to correct the color cast of each color channel. Then, we propose a dual prior optimized contrast enhancement method, which relies on the spatial and texture priors to decompose the base layer and detail layer of the V channel in HSV color space. Meanwhile, we employ different enhancement strategies in different layers to enhance the contrast and texture detail of underwater images. Our extensive experiments on several benchmark datasets show that our method outperforms eleven compared state-of-the-art methods. Moreover, our method has good generalization capability for fog and low-light images. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Li-Chongyi/PCDE</uri> .

Multi-model order spatially constrained ICA reveals highly replicable group differences and consistent predictive results from resting data: A large N fMRI schizophrenia study

Brain functional networks identified from resting functional magnetic resonance imaging (fMRI) data have the potential to reveal biomarkers for brain disorders, but studies of complex mental illnesses such as schizophrenia (SZ) often yield mixed results across replication studies. This is likely due in part to the complexity of the disorder, the short data acquisition time, and the limited ability of the approaches for brain imaging data mining. Therefore, the use of analytic approaches which can both capture individual variability while offering comparability across analyses is highly preferred. Fully blind data-driven approaches such as independent component analysis (ICA) are hard to compare across studies, and approaches that use fixed atlas-based regions can have limited sensitivity to individual sensitivity. By contrast, spatially constrained ICA (scICA) provides a hybrid, fully automated solution that can incorporate spatial network priors while also adapting to new subjects. However, scICA has thus far only been used with a single spatial scale (ICA dimensionality, i.e., ICA model order). In this work, we present an approach using multi-objective optimization scICA with reference algorithm (MOO-ICAR) to extract subject-specific intrinsic connectivity networks (ICNs) from fMRI data at multiple spatial scales, which also enables us to study interactions across spatial scales. We evaluate this approach using a large N (N > 1,600) study of schizophrenia divided into separate validation and replication sets. A multi-scale ICN template was estimated and labeled, then used as input into scICA which was computed on an individual subject level. We then performed a subsequent analysis of multiscale functional network connectivity (msFNC) to evaluate the patient data, including group differences and classification. Results showed highly consistent group differences in msFNC in regions including cerebellum, thalamus, and motor/auditory networks. Importantly, multiple msFNC pairs linking different spatial scales were implicated. The classification model built on the msFNC features obtained up to 85% F1 score, 83% precision, and 88% recall, indicating the strength of the proposed framework in detecting group differences between schizophrenia and the control group. Finally, we evaluated the relationship of the identified patterns to positive symptoms and found consistent results across datasets. The results verified the robustness of our framework in evaluating brain functional connectivity of schizophrenia at multiple spatial scales, implicated consistent and replicable brain networks, and highlighted a promising approach for leveraging resting fMRI data for brain biomarker development.

TransMIN: Transformer-Guided Multi-Interaction Network for Remote Sensing Object Detection

Remote sensing (RS) object detectors based on convolutional neural networks (CNNs) are hard to model global context dependencies. Transformer-based detectors can overcome this problem via global pairwise interactions, but more comprehensive information interactions are not systemically investigated to boost the detection performance. In view of this issue, we propose a transformer-guided multi-interaction network (TransMIN), which uses ResNet50-feature pyramid network (FPN) as the backbone for remote sensing object detection (RSOD). Specifically, we implement local–global feature interactions (LGFIs) by combining convolution and transformer in the residual blocks of ResNet50 to learn complementary features. We implement cross-view feature interactions (CVFIs) via transformers in the pyramid layers of FPN to capture the correlation between reference features (spatial edge priors and channel statistics) and pyramid features. This enhances edge information and suppresses background interference. In the detection head, we adopt a task-interactive sample assigner (TISA) by considering the interactions of classification and localization losses to obtain high-quality predictions. Experiments on two benchmark datasets demonstrate the superior detection performance of TransMIN over state-of-the-art methods.

Hyperspectral Image Classification With Contrastive Graph Convolutional Network

Recently, graph convolutional network (GCN) has been widely used in hyperspectral image (HSI) classification due to its satisfactory performance. However, the number of labeled pixels is very limited in HSI, and thus, the available supervision information is usually insufficient, which will inevitably degrade the representation ability of most existing GCN-based methods. To enhance the feature representation ability, in this article, a GCN model with contrastive learning is proposed to explore the supervision signals contained in both spectral information and spatial relations, which is termed contrastive GCN (ConGCN), for HSI classification. First, in order to mine sufficient supervision signals from spectral information, a semisupervised contrastive loss function is utilized to maximize the agreement between different views of the same node or the nodes from the same land cover category. Second, to extract the precious yet implicit spatial relations in HSI, a graph generative loss function is leveraged to explore supplementary supervision signals contained in the graph topology. In addition, an adaptive graph augmentation technique is designed to flexibly incorporate the spectral–spatial priors of HSI, which helps facilitate the subsequent contrastive representation learning. The extensive experimental results on six typical benchmark datasets firmly demonstrate the effectiveness of the proposed ConGCN in both qualitative and quantitative aspects.

Joint spatial structural sparsity constraint and spectral low-rank approximation for snapshot compressive spectral imaging reconstruction

Snapshot compressive spectral imaging (SCI) is a potential technology in the field of hyperspectral imaging (HSI), where multi-frame spectral images are compressed into a single snapshot measurement and collected by a 2D detector. Existing reconstruction algorithms of SCI systems do not make full use of the redundancy of hyperspectral data, artifacts and blur degrade the quality of the reconstruction target image. In this paper, an efficient algorithm, named as spatial structural sparsity and spectral low-rank priors (SSS-SLR), is proposed based on two inherent priors of hyperspectral images. Specifically, the spatial structural sparsity and spectral low-rank priori are simultaneously integrated into the SCI image restoration process to establish a variational optimization model, which can be solved via an alternating minimization algorithm. Extensive reconstruction results of hyperspectral data cube from both synthetic and real datasets demonstrate that the proposed method significantly outperforms the canonical algorithms.

Interpretive JIVE: Connections with CCA and an application to brain connectivity.

Joint and Individual Variation Explained (JIVE) is a model that decomposes multiple datasets obtained on the same subjects into shared structure, structure unique to each dataset, and noise. JIVE is an important tool for multimodal data integration in neuroimaging. The two most common algorithms are R.JIVE, an iterative approach, and AJIVE, which uses principal angle analysis. The joint structure in JIVE is defined by shared subspaces, but interpreting these subspaces can be challenging. In this paper, we reinterpret AJIVE as a canonical correlation analysis of principal component scores. This reformulation, which we call CJIVE, (1) provides an intuitive view of AJIVE; (2) uses a permutation test for the number of joint components; (3) can be used to predict subject scores for out-of-sample observations; and (4) is computationally fast. We conduct simulation studies that show CJIVE and AJIVE are accurate when the total signal ranks are correctly specified but, generally inaccurate when the total ranks are too large. CJIVE and AJIVE can still extract joint signal even when the joint signal variance is relatively small. JIVE methods are applied to integrate functional connectivity (resting-state fMRI) and structural connectivity (diffusion MRI) from the Human Connectome Project. Surprisingly, the edges with largest loadings in the joint component in functional connectivity do not coincide with the same edges in the structural connectivity, indicating more complex patterns than assumed in spatial priors. Using these loadings, we accurately predict joint subject scores in new participants. We also find joint scores are associated with fluid intelligence, highlighting the potential for JIVE to reveal important shared structure.