Reconstruction Constraints Research Articles

Tensor Dictionary Learning with an Enhanced Sparsity Constraint for Sparse-View Spectral CT Reconstruction

Spectral computed tomography (CT) can divide collected photons into multi-energy channels and gain multi-channel projections synchronously by using photon-counting detectors. However, reconstructed images usually contain severe noise due to the limited number of photons in the corresponding energy channel. Tensor dictionary learning (TDL)-based methods have achieved better performance, but usually lose image edge information and details, especially from an under-sampling dataset. To address this problem, this paper proposes a method termed TDL with an enhanced sparsity constraint for spectral CT reconstruction. The proposed algorithm inherits the superiority of TDL by exploring the correlation of spectral CT images. Moreover, the method designs a regularization using the L0-norm of the image gradient to constrain images and the difference between images and a prior image in each energy channel simultaneously, further improving the ability to preserve edge information and subtle image details. The split-Bregman algorithm has been applied to address the proposed objective minimization model. Several numerical simulations and realistic preclinical mice are studied to assess the effectiveness of the proposed algorithm. The results demonstrate that the proposed method improves the quality of spectral CT images in terms of noise elimination, edge preservation, and image detail recovery compared to the several existing better methods.

Real-Time Super-Resolution ISAR Imaging Using Unsupervised Learning

Compressive sensing (CS) enables high-resolution inverse synthetic aperture radar (ISAR) imaging with limited measurements. However, these methods reconstruct images via iterative optimization, resulting in a high computational load. Recently, convolutional neural networks (CNNs) have been used to perform super-resolution ISAR imaging in real time, where high-resolution images are necessarily used as ground truth. However, the desired high-resolution images are not reliable in practice. This letter presents an unsupervised CNN-based framework for super-resolution ISAR imaging. The well-trained CNN can directly produce high-resolution ISAR images in real time. Moreover, the network is trained in an unsupervised manner, which is suitable for practical applications. Furthermore, a pseudo <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{0}$ </tex-math></inline-formula> -norm has been used as the sparse constraint for the exact image reconstruction. The proposed approach has been used to process the real ISAR data, and the experimental results are convincing.

Learning Enriched Features for Image Super Resolution

In recent years, significant progress has been made in image super-resolution (SR) methods based on convolutional neural networks. However, most of them do not fully utilize multi-scale feature correspondence in the image SR process, resulting in blurred and artifact detail restoration, especially for SR tasks with larger scaling factors (i.e. <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times 4$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times 8$ </tex-math></inline-formula> ). A multi-scale feature-enhanced SR network (MFENet) is proposed to solve the problems mentioned above. Specifically, the super-resolution method is enhanced by utilizing multi-scale feature correspondence in two ways. On the one hand, we propose an effective multi-scale non-local attention module and it can fully exploit a large number of low-level features at different scales of a single low-resolution (LR) feature map by exhaustively evaluating the correlations in local feature, in-scale non-local feature, and cross-scale non-local feature. On the other hand, the mapping from high-resolution (HR) to LR images is explored by a feedback branch we introduced, that provides an additional constraint for images reconstruction. It can penalize incorrect SR predictions to achieve better SR performance. Experimental results on 5 benchmark datasets show that our method outperforms state-of-the-art methods in terms of both subjective visual quality and objective quantitative metrics.

Semi-Supervised SAR Target Detection Based on an Improved Faster R-CNN

In the remote sensing image processing field, the synthetic aperture radar (SAR) target-detection methods based on convolutional neural networks (CNNs) have gained remarkable performance relying on large-scale labeled data. However, it is hard to obtain many labeled SAR images. Semi-supervised learning is an effective way to address the issue of limited labels on SAR images because it uses unlabeled data. In this paper, we propose an improved faster regions with CNN features (R-CNN) method, with a decoding module and a domain-adaptation module called FDDA, for semi-supervised SAR target detection. In FDDA, the decoding module is adopted to reconstruct all the labeled and unlabeled samples. In this way, a large number of unlabeled SAR images can be utilized to help structure the latent space and learn the representative features of the SAR images, devoting attention to performance promotion. Moreover, the domain-adaptation module is further introduced to utilize the unlabeled SAR images to promote the discriminability of features with the assistance of the abundantly labeled optical remote sensing (ORS) images. Specifically, the transferable features between the ORS images and SAR images are learned to reduce the domain discrepancy via the mean embedding matching, and the knowledge of ORS images is transferred to the SAR images for target detection. Ultimately, the joint optimization of the detection loss, reconstruction, and domain adaptation constraints leads to the promising performance of the FDDA. The experimental results on the measured SAR image datasets and the ORS images dataset indicate that our method achieves superior SAR target detection performance with limited labeled SAR images.

The Role of the Seismically Slow Central‐East Atlantic Anomaly in the Genesis of the Canary and Madeira Volcanic Provinces

AbstractThe Canary and Madeira provinces in the Central‐East Atlantic Ocean are characterized by an irregular spatio‐temporal distribution of volcanism along the hotspot tracks, and several alternative scenarios have been suggested to explain it. Here, we combine results from seismic tomography, shear‐wave splitting and gravity along with plate reconstruction constraints to investigate the mantle structure and dynamics beneath those provinces. We find that the Central‐East Atlantic Anomaly (CEAA), which rises from the core‐mantle boundary and stalls in the topmost lower mantle, is the deep source of distinct upper‐mantle upwellings beneath the region. The upwellings detach intermittently from the top of the CEAA and appear to be at different evolutionary stages. We argue that the accumulation of plume material in the topmost lower mantle can play a key role in governing the first‐order spatio‐temporal irregularities in the distribution of hotspot volcanism.

Multi-energy reconstructions, central electron temperature measurements, and early detection of the birth and growth of runaway electrons using a versatile soft x-ray pinhole camera at MST.

A multi-energy soft x-ray pinhole camera has been designed, built, and deployed at the Madison Symmetric Torus to aid the study of particle and thermal transport, as well as MHD stability physics. This novel imaging diagnostic technique employs a pixelated x-ray detector in which the lower energy threshold for photon detection can be adjusted independently on each pixel. The detector of choice is a PILATUS3 100K with a 450 μm thick silicon sensor and nearly 100 000 pixels sensitive to photon energies between 1.6 and 30 keV. An ensemble of cubic spline smoothing functions has been applied to the line-integrated data for each time-frame and energy-range, obtaining a reduced standard-deviation when compared to that dominated by photon-noise. The multi-energy local emissivity profiles are obtained from a 1D matrix-based Abel-inversion procedure. Central values of Te can be obtained by modeling the slope of the continuum radiation from ratios of the inverted radial emissivity profiles over multiple energy ranges with no a priori assumptions of plasma profiles, magnetic field reconstruction constraints, high-density limitations, or need of shot-to-shot reproducibility. In tokamak plasmas, a novel application has recently been tested for early detection, 1D imaging, and study of the birth, exponential growth, and saturation of runaway electrons at energies comparable to 100 × Te,0; thus, early results are also presented.

Unravelling constraints of urban housing reconstruction

The Nepal's National Reconstruction Authority (NRA), which was established for reconstruction and recovery of damaged infrastructure by 2015 Gorkha Earthquake, achieved only 43% progress in the Kathmandu Valley in comparison to 75% in the Rural Municipalities by April 2020. The broader objective of this study is to unravel reconstruction constraints in urban housing. In addition to collecting information from the secondary sources, this study conducted primary surveys and focused interviews. This study found that the problem of land and access to finance are the structural problems. Hence, the Government of Nepal (GoN) will need to integrate reconstruction activities with a long-term urban renewal strategy. Other non-structural issues can be addressed by NRA itself.

Complex wavefront sensing based on coherent diffraction imaging using vortex modulation

Phase retrieval seeks to reconstruct the phase from the measured intensity, which is an ill-posed problem. A phase retrieval problem can be solved with physical constraints by modulating the investigated complex wavefront. Orbital angular momentum has been recently employed as a type of reliable modulation. The topological charge l is robust during propagation when there is atmospheric turbulence. In this work, topological modulation is used to solve the phase retrieval problem. Topological modulation offers an effective dynamic range of intensity constraints for reconstruction. The maximum intensity value of the spectrum is reduced by a factor of 173 under topological modulation when l is 50. The phase is iteratively reconstructed without a priori knowledge. The stagnation problem during the iteration can be avoided using multiple topological modulations.

Dynamic nanoimaging of extended objects via hard X-ray multiple-shot coherent diffraction with projection illumination optics

The quest for understanding the structural mechanisms of material properties and biological cell functions has led to the active development of coherent diffraction imaging (CDI) and its variants in the hard X-ray regime. Herein, we propose multiple-shot CDI, a full-field CDI technique dedicated to the visualisation of local nanostructural dynamics in extended objects at a spatio-temporal resolution beyond that of current instrumentation limitations. Multiple-shot CDI reconstructs a “movie” of local dynamics from time-evolving diffraction patterns, which is compatible with a robust scanning variant, ptychography. We developed projection illumination optics to produce a probe with a well-defined illumination area and a phase retrieval algorithm, establishing a spatio-temporal smoothness constraint for the reliable reconstruction of dynamic images. The numerical simulations and proof-of-concept experiment using synchrotron hard X-rays demonstrated the capability of visualising a dynamic nanostructured object at a frame rate of 10 Hz or higher.

Semantically Derived Geometric Constraints for MVS Reconstruction of Textureless Areas

Conventional multi-view stereo (MVS) approaches based on photo-consistency measures are generally robust, yet often fail in calculating valid depth pixel estimates in low textured areas of the scene. In this study, a novel approach is proposed to tackle this challenge by leveraging semantic priors into a PatchMatch-based MVS in order to increase confidence and support depth and normal map estimation. Semantic class labels on image pixels are used to impose class-specific geometric constraints during multiview stereo, optimising the depth estimation on weakly supported, textureless areas, commonly present in urban scenarios of building facades, indoor scenes, or aerial datasets. Detecting dominant shapes, e.g., planes, with RANSAC, an adjusted cost function is introduced that combines and weighs both photometric and semantic scores propagating, thus, more accurate depth estimates. Being adaptive, it fills in apparent information gaps and smoothing local roughness in problematic regions while at the same time preserves important details. Experiments on benchmark and custom datasets demonstrate the effectiveness of the presented approach.

Latent discriminative representation learning for speaker recognition

Extracting discriminative speaker-specific representations from speech signals and transforming them into fixed length vectors are key steps in speaker identification and verification systems. In this study, we propose a latent discriminative representation learning method for speaker recognition. We mean that the learned representations in this study are not only discriminative but also relevant. Specifically, we introduce an additional speaker embedded lookup table to explore the relevance between different utterances from the same speaker. Moreover, a reconstruction constraint intended to learn a linear mapping matrix is introduced to make representation discriminative. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods based on the Apollo dataset used in the Fearless Steps Challenge in INTERSPEECH2019 and the TIMIT dataset.

Cross-Scene Hyperspectral Image Classification With Discriminative Cooperative Alignment

Cross-scene classification is one of the major challenges for hyperspectral image (HSI) classification, especially for target scenes without label samples. Most traditional domain adaptive methods learn a domain invariant subspace to reduce statistical shift while ignoring the fact that there may not exist a shared subspace when marginal distributions of source and target domains are very different. In addition, it is important for HSI classification to preserve discriminant information in the original space. To solve this issue, discriminative cooperative alignment (DCA) of subspace and distribution is proposed to cooperatively reduce the geometric and statistical shift. In the proposed framework, both geometrical and statistical alignments are considered to learn subspaces of the two domains with preserving discrimination information. Furthermore, a reconstruction constraint is imposed to enhance the robustness of subspace projection. Experimental results on three cross-scene HSI data sets demonstrate that the proposed DCA is significantly better than some state-of-the-art domain-adaptive approaches.

On the use of low-dimensional temporal subspace constraints to reduce reconstruction time and improve image quality of accelerated 4D-MRI

Background and purposeThe purpose of this work is to investigate the use of low-dimensional temporal subspace constraints for 4D-MRI reconstruction from accelerated data in the context of MR-guided online adaptive radiation therapy (MRgOART). Materials and methodsSubspace basis functions are derived directly from the accelerated golden angle radial stack-of-stars 4D-MRI data. The reconstruction times, image quality, and motion estimates are investigated as a function of the number of subspace coefficients and compared with a conventional frame-by-frame reconstruction. These experiments were performed in five patients with four 4D-MRI scans per patient on a 1.5T MR-Linac. ResultsIf two or three subspace coefficients are used, the iterative reconstruction time is reduced by 32% and 18%, respectively, compared to conventional parallel imaging with compressed sensing reconstructions. No significant difference was found between motion estimates made with the subspace-constrained reconstructions (p > 0.08). Qualitative improvements in image quality included reduction in apparent noise and reductions in streaking artifacts from the radial k-space coverage. ConclusionIncorporating subspace constraints for accelerated 4D-MRI reconstruction reduces noise and residual undersampling artifacts in the images while reducing computation time, making it a strong candidate for use in clinical MRgOART workflows.

Anomaly Detection With Bidirectional Consistency in Videos.

The core component of most anomaly detectors is a self-supervised model, tasked with modeling patterns included in training samples and detecting unexpected patterns as the anomalies in testing samples. To cope with normal patterns, this model is typically trained with reconstruction constraints. However, the model has the risk of overfitting to training samples and being sensitive to hard normal patterns in the inference phase, which results in irregular responses at normal frames. To address this problem, we formulate anomaly detection as a mutual supervision problem. Due to collaborative training, the complementary information of mutual learning can alleviate the aforementioned problem. Based on this motivation, a SIamese generative network (SIGnet), including two subnetworks with the same architecture, is proposed to simultaneously model the patterns of the forward and backward frames. During training, in addition to traditional constraints on improving the reconstruction performance, a bidirectional consistency loss based on the forward and backward views is designed as the regularization term to improve the generalization ability of the model. Moreover, we introduce a consistency-based evaluation criterion to achieve stable scores at the normal frames, which will benefit detecting anomalies with fluctuant scores in the inference phase. The results on several challenging benchmark data sets demonstrate the effectiveness of our proposed method.

Robust SAR Automatic Target Recognition Via Adversarial Learning

The traditional denoising methods in noise robust synthetic aperture radar (SAR) automatic target recognition research are independent of the recognition model, which limits the robust recognition performance. In this article, we present a robust SAR automatic target recognition method via adversarial learning, which could integrate data denoising, feature extraction, and classification into a unified framework for joint learning. Different from the common recognition methods of directly inputting the SAR data into the classifiers, we add a dual-generative-adversarial-network (GAN) model between the SAR data and the classifier for data translation from a noise-polluted style to a relatively clean style to reduce the noise from SAR data. In order to ensure the target information in the SAR data can be retained during the data style translation, reconstruction constraint and label constraint are also used in the dual-GAN model. Then, the more reliable transferred SAR data are fed into the classifier. The parameters of the dual-GAN and classifier are learned through joint optimization in our method. Thus, the data separability is guaranteed in the process of denoising and feature extraction, which greatly improves the recognition performance of the method. In addition, our method can be easily extended to a semisupervised method by using different objective functions for labeled and unlabeled training data, which is more suitable for practical application. Experimental results on MSTAR dataset and Gotcha dataset show that our method can get the encouraging performance in the case of low signal-to-noise ratio and small labeled data size.

Prospects and constraints of post-cyclone housing reconstruction in Vanuatu drawing from the experience of tropical cyclone Harold

Vanuatu is one of the countries in the world most at risk from natural hazards. This Pacific island country is frequently struck by ferocious tropical cyclones, such as Cyclone Harold in April 2020, causing massive devastation to the housing sector; nearly 21,000 houses were destroyed and damaged by the cyclone. Drawing from the literature and communications with local stakeholders, five main thematic aspects were analysed: nature of the impact of cyclones on housing; key challenges for post-cyclone housing reconstruction in Vanuatu; cyclone-resistant construction approaches; post-Harold housing reconstruction initiatives; and key opportunities. Of particular significance is the dilemma posed by traditional versus ‘modern’ approaches to design and construction for post-disaster reconstruction. There are many guidelines available for cyclone-resistant housing, but they face barriers to dissemination and application, and whether they are necessarily appropriate in the cultural context of Vanuatu is examined. Vanuatu faces a difficult situation in the aftermath of Cyclone Harold in the midst of the COVID-19 pandemic, and it can be expected that the reconstruction will be a protracted process.

Stacked Enhanced Auto-Encoder for Data-Driven Soft Sensing of Quality Variable

Data-driven soft sensors have been widely used in industrial processes. Traditional soft sensors are mostly shallow networks, which cannot easily describe the complicated process data patterns. In this article, a new deep learning approach is proposed for soft sensors, which is based on the stacked enhanced auto-encoder (SEAE). The original stacked auto-encoder (SAE) learns the hierarchical features of the raw observed input data with unsupervised layerwise pretraining. In each layer, the features are learned from its low-level ones with an AE by minimizing the reconstruction error for them. This usually causes accumulation of information loss from the lowest layer to the highest layer. Hence, the learned features may not well represent the raw input data patterns. The new SEAE network is designed by adding the constraint of additional reconstruction for the raw input data at each layer. In this way, the learned features at each layer are a good representation for the raw input data. The effectiveness of the proposed SEAE-based soft sensor method is validated on an industrial sulfur recovery unit.

Robust Adaptive Linear Discriminant Analysis with Bidirectional Reconstruction Constraint

Linear discriminant analysis (LDA) is a well-known supervised method for dimensionality reduction in which the global structure of data can be preserved. The classical LDA is sensitive to the noises, and the projection direction of LDA cannot preserve the main energy. This article proposes a novel feature extraction model with l 2,1 norm constraint based on LDA, termed as RALDA. This model preserves within-class local structure in the latent subspace according to the label information. To reduce information loss, it learns a projection matrix and an inverse projection matrix simultaneously. By introducing an implicit variable and matrix norm transformation, the alternating direction multiple method with updating variables is designed to solve the RALDA model. Moreover, both computational complexity and weak convergence property of the proposed algorithm are investigated. The experimental results on several public databases have demonstrated the effectiveness of our proposed method.

Unsupervised Video Summarization via Relation-Aware Assignment Learning

We address the problem of unsupervised video summarization that automatically selects key video clips. Most state-of-the-art approaches suffer from two issues: (1) they model video clips without explicitly exploiting their relations, and (2) they learn soft importance scores over all the video clips to generate the summary representation. However, a meaningful video summary should be inferred by taking the relation-aware context of the original video into consideration, and directly selecting a subset of clips with a hard assignment. In this paper, we propose to exploit clip-clip relations to learn relation-aware hard assignments for selecting key clips in an unsupervised manner. First, we consider the clips as graph nodes to construct an assignment-learning graph. Then, we utilize the magnitude of the node features to generate hard assignments as the summary selection. Finally, we optimize the whole framework via a proposed multi-task loss including a reconstruction constraint, and a contrastive constraint. Extensive experimental results on three popular benchmarks demonstrate the favourable performance of our approach.

Sparse graph based self-supervised hashing for scalable image retrieval

In recent years, learning-based image hashing techniques have elicited wide interest among researchers because they can be applied in high-dimensional data such as videos and images. Supervised hashing techniques can achieve a satisfactory retrieval performance, but are excessively reliant on label information, resulting in the appearance of unsupervised hashing methods. However, existing unsupervised hashing methods often fail to obtain effective similarity information from the training set; this causes a large degradation in their retrieval performance. Some pseudo label-based methods partially alleviate this problem, but are sensitive to the pre-defined number of categories. Therefore, in this paper, we mainly discuss a solution to the above-mentioned problems regarding image hashing. We propose a sparse graph based self-supervised hashing method in which a sparse graph is constructed, which not only circumvents the requirement of a predefined number of categories as compared with pseudo label-based methods, but also significantly reduces the memory demand compared with a dense graph-based method. In addition, we also exploit a self-supervised reconstruction constraint to further preserve the semantic information. These items are combined in a linear manner and optimized using an iterative strategy. Four representative datasets, including two single-label and two multi-label datasets, are employed to evaluate our method. The results, based on multiple metrics, show that our method can outperform other state-of-the-art methods.