High-resolution Image Reconstruction Research Articles

High-resolution Holographic Image Reconstruction Based on Deep Learning

High-resolution Holographic Image Reconstruction Based on Deep Learning

Visual State Space Model for Image Deraining with Symmetrical Scanning

Image deraining aims to mitigate the adverse effects of rain streaks on image quality. Recently, the advent of convolutional neural networks (CNNs) and Vision Transformers (ViTs) has catalyzed substantial advancements in this field. However, these methods fail to effectively balance model efficiency and image deraining performance. In this paper, we propose an effective, locally enhanced visual state space model for image deraining, called DerainMamba. Specifically, we introduce a global-aware state space model to better capture long-range dependencies with linear complexity. In contrast to existing methods that utilize fixed unidirectional scan mechanisms, we propose a direction-aware symmetrical scanning module to enhance the feature capture of rain streak direction. Furthermore, we integrate a local-aware mixture of experts into our framework to mitigate local pixel forgetting, thereby enhancing the overall quality of high-resolution image reconstruction. Experimental results validate that the proposed method surpasses state-of-the-art approaches on six benchmark datasets.

Wide-field, high-resolution reconstruction in computational multi-aperture miniscope using a Fourier neural network

Traditional fluorescence microscopy is constrained by inherent trade-offs among resolution, field of view, and system complexity. To navigate these challenges, we introduce a simple and low-cost computational multi-aperture miniature microscope, utilizing a microlens array for single-shot wide-field, high-resolution imaging. Addressing the challenges posed by extensive view multiplexing and non-local, shift-variant aberrations in this device, we present SV-FourierNet, a multi-channel Fourier neural network. SV-FourierNet facilitates high-resolution image reconstruction across the entire imaging field through its learned global receptive field. We establish a close relationship between the physical spatially varying point-spread functions and the network’s learned effective receptive field. This ensures that SV-FourierNet has effectively encapsulated the spatially varying aberrations in our system and learned a physically meaningful function for image reconstruction. Training of SV-FourierNet is conducted entirely on a physics-based simulator. We showcase wide-field, high-resolution video reconstructions on colonies of freely moving C. elegans and imaging of a mouse brain section. Our computational multi-aperture miniature microscope, augmented with SV-FourierNet, represents a major advancement in computational microscopy and may find broad applications in biomedical research and other fields requiring compact microscopy solutions.

Highly Efficient Fluorescent Probe to Visualize Protein Interactions at the Superresolution.

Superresolution microscopy (SR microscopy) of protein-protein interactions (PPIs) occurring in subcellular structures is essential for understanding cellular functions. However, a powerful and useful technology for SR microscopy of PPIs remains elusive. Here, we develop a highly efficient photoconvertible fluorescent probe, named split-Dendra2, for SR microscopy of PPIs in the cell. We found that split-Dendra2 enables a highly efficient detection of PPIs, making it possible to perform SR microscopy of PPIs with high spatial resolution and high image reconstruction fidelity. We demonstrate the utility of split-Dendra2 by visualizing PPIs occurring in small subcellular structures at the superresolution, such as clathrin-coated pits and focal adhesions, which cannot be visualized by the existing tools. Split-Dendra2 offers a powerful and useful tool that greatly expands the possibility of SR microscopy and can contribute to revealing the function of PPIs at the nanoscale resolution.

Super-resolution reconstruction of single image for latent features

AbstractSingle-image super-resolution (SISR) typically focuses on restoring various degraded low-resolution (LR) images to a single high-resolution (HR) image. However, during SISR tasks, it is often challenging for models to simultaneously maintain high quality and rapid sampling while preserving diversity in details and texture features. This challenge can lead to issues such as model collapse, lack of rich details and texture features in the reconstructed HR images, and excessive time consumption for model sampling. To address these problems, this paper proposes a Latent Feature-oriented Diffusion Probability Model (LDDPM). First, we designed a conditional encoder capable of effectively encoding LR images, reducing the solution space for model image reconstruction and thereby improving the quality of the reconstructed images. We then employed a normalized flow and multimodal adversarial training, learning from complex multimodal distributions, to model the denoising distribution. Doing so boosts the generative modeling capabilities within a minimal number of sampling steps. Experimental comparisons of our proposed model with existing SISR methods on mainstream datasets demonstrate that our model reconstructs more realistic HR images and achieves better performance on multiple evaluation metrics, providing a fresh perspective for tackling SISR tasks.

MPIGAN: An end-to-end deep based generative framework for high-resolution magnetic particle imaging reconstruction.

Magnetic particle imaging (MPI) is a recently developed, non-invasive in vivo imaging technique to map the spatial distribution of superparamagnetic iron oxide nanoparticles (SPIONs) in animal tissues with high sensitivity and speed. It is a challenge to reconstruct images directly from the received signals of MPI device due to the complex physical behavior of the nanoparticles. System matrix and X-space are two commonly used MPI reconstruction methods, where the former is extremely time-consuming and the latter usually produces blurry images. Currently, we proposed an end-to-end machine learning framework to reconstruct high-resolution MPI images from 1-D voltage signals directly and efficiently. The proposed framework, which we termed "MPIGAN", was trained on a large MPI simulation dataset containing 291597 pairs of high-resolution 2-D phantom images and each image's corresponding voltage signals, so that it was able to accurately capture the nonlinear relationship between the spatial distribution of SPIONs and the received voltage signal, and realized high-resolution MPI image reconstruction. Experiment results showed that, MPIGAN exhibited remarkable abilities in high-resolution MPI image reconstruction. MPIGAN outperformed the traditional methods of system matrix and X-space in recovering the fine-scale structure of magnetic nanoparticles' spatial distribution and achieving enhanced reconstruction performance in both visual effects and quantitative assessments. Moreover, even when the received signals were severely contaminated with noise, MPIGAN could still generate high-quality MPI images. Our study provides a promising AI solution for end-to-end, efficient, and high-resolution magnetic particle imaging reconstruction.

New Vacuum Solar Telescope Achieves Narrowband Infrared Solar Imaging Observation at He i 10830 Å

The near-infrared imaging channel constitutes a crucial component of the multichannel high-resolution imaging system of the New Vacuum Solar Telescope (NVST). We have successfully achieved high-resolution, narrowband imaging of the chromosphere using He i 10830 Å triplet within this channel, which significantly enhances the imaging observation capabilities of NVST. This paper provides a concise overview of the optical system associated with the near-infrared imaging channel, detailing data processing procedures and presenting several observed images. Leveraging a high-resolution image reconstruction algorithm, we were able to generate a narrowband image near the diffraction limit at 10830 Å with a temporal resolution of less than 10 s.

Learning hybrid dynamic transformers for underwater image super-resolution

Underwater image super-resolution is vital for enhancing the clarity and detail of underwater imagery, enabling improved analysis, navigation, and exploration in underwater environments where visual quality is typically degraded due to factors like water turbidity and light attenuation. In this paper, we propose an effective hybrid dynamic Transformer (called HDT-Net) for underwater image super-resolution, leveraging a collaborative exploration of both local and global information aggregation to help image restoration. Firstly, we introduce a dynamic local self-attention to adaptively capture important spatial details in degraded underwater images by employing dynamic weighting. Secondly, considering that visual transformers tend to introduce irrelevant information when modeling the global context, thereby interfering with the reconstruction of high-resolution images, we design a sparse non-local self-attention to more accurately compute self-similarity by setting a top-k threshold. Finally, we integrate these two self-attention mechanisms into the hybrid dynamic transformer module, constituting the primary feature extraction unit of our proposed method. Quantitative and qualitative analyses on benchmark datasets demonstrate that our approach achieves superior performance compared to previous CNN and Transformer models.

Quantitative susceptibility mapping through model-based deep image prior (MoDIP)

The data-driven approach of supervised learning methods has limited applicability in solving dipole inversion in Quantitative Susceptibility Mapping (QSM) with varying scan parameters across different objects. To address this generalization issue in supervised QSM methods, we propose a novel training-free model-based unsupervised method called MoDIP (Model-based Deep Image Prior). MoDIP comprises a small, untrained network and a Data Fidelity Optimization (DFO) module. The network converges to an interim state, acting as an implicit prior for image regularization, while the optimization process enforces the physical model of QSM dipole inversion. Experimental results demonstrate MoDIP's excellent generalizability in solving QSM dipole inversion across different scan parameters. It exhibits robustness against pathological brain QSM, achieving over 32 % accuracy improvement than supervised deep learning methods. It is also 33 % more computationally efficient and runs 4 times faster than conventional DIP-based approaches, enabling 3D high-resolution image reconstruction in under 4.5 min.

RTEN-SR: A reference-based texture enhancement network for single image super-resolution

Most current super-resolution (SR) reconstruction methods suffer from edge blurring and insufficient detail reconstruction. To avoid these problems, this paper proposes a reference-based texture enhancement network for single image SR (RTEN-SR). Firstly, a preliminary reconstruction module (PRM) is constructed to learn the initial reconstructed high-resolution (HR) image features. Then, a multi-scale texture enhancement module (MTEM) based on the idea of texture transfer is designed to further supplement and enhance the texture details for the initial reconstructed HR images. To improve the feature learning ability of network, an efficient mixed attention module (EMAM) is constructed by combining channel attention and spatial attention. Meanwhile, the EMAM is integrated into the above two modules to enhance the important image features in the channel and spatial dimensions. Experimental results demonstrate that the proposed method can achieve better performance than state-of-the-art methods, effectively improving the objective indexes and visual results of reference-based image super-resolution (RefSR). For example, the proposed method outperforms the best RefSR method among the comparison methods by 0.38 dB in PSNR metric and 0.128 in SSIM metric.

LMD: Faster Image Reconstruction with Latent Masking Diffusion

As a class of fruitful approaches, diffusion probabilistic models (DPMs) have shown excellent advantages in high-resolution image reconstruction. On the other hand, masked autoencoders (MAEs), as popular self-supervised vision learners, have demonstrated simpler and more effective image reconstruction and transfer capabilities on downstream tasks. However, they all require extremely high training costs, either due to inherent high temporal-dependence (i.e., excessively long diffusion steps) or due to artificially low spatial-dependence (i.e., human-formulated high mask ratio, such as 0.75). To the end, this paper presents LMD, a faster image reconstruction framework with Latent Masking Diffusion. First, we propose to project and reconstruct images in latent space through a pre-trained variational autoencoder, which is theoretically more efficient than in the pixel-based space. Then, we combine the advantages of MAEs and DPMs to design a progressive masking diffusion model, which gradually increases the masking proportion by three different schedulers and reconstructs the latent features from simple to difficult, without sequentially performing denoising diffusion as in DPMs or using fixed high masking ratio as in MAEs, so as to alleviate the high training time-consumption predicament. Our approach allows for learning high-capacity models and accelerate their training (by 3x or more) and barely reduces the original accuracy. Inference speed in downstream tasks also significantly outperforms the previous approaches.

An Improved NLCS Algorithm Based on Series Reversion and Elliptical Model Using Geosynchronous Spaceborne–Airborne UHF UWB Bistatic SAR for Oceanic Scene Imaging

Geosynchronous spaceborne–airborne (GEO-SA) ultra-high-frequency ultra-wideband bistatic synthetic aperture radar (UHF UWB BiSAR) provides high-precision images for marine and polar environments, which are pivotal in glacier monitoring and sea ice thickness measurement for polar ocean mapping and navigation. Contrasting with traditional high-frequency BiSAR, it faces unique challenges, such as the considerable spatial variability, significant range–azimuth coupling, and vast volumes of echo data, which impede high-resolution image reconstruction. This paper presents an improved bistatic nonlinear chirp scaling (NLCS) algorithm for imaging oceanic scenes with GEO-SA UHF UWB BiSAR. This methodology extends the two-dimensional (2-D) spectrum up to the sixth order via the method of series reversion (MSR) to meet accuracy demands and then employs an elliptical model to elucidate the alterations in the azimuth frequency modulation (FM) rate mismatch. Initially, the imaging geometry and signal model are introduced, and then a separation of bistatic slant ranges based on the configuration is proposed. In addition, during range processing, after eliminating linear range cell migration (RCM), the derivation process for the sixth-order 2-D spectrum is detailed and an improved filter is applied to correct the high-order RCM. Finally, during azimuth processing, the causes of the FM rate mismatch are analyzed, a cubic perturbation function derived from the elliptical model is used for FM rate equalization, and a unified sixth-order filter is applied to complete the azimuth compression. Experimental results with point targets and natural oceanic scenes validate the outstanding efficacy of the proposed NLCS algorithm, particularly in imaging quality enhancements for GEO-SA UHF UWB BiSAR.

Deep-learning-based super-resolution for accelerating chemical exchange saturation transfer MRI.

Chemical exchange saturation transfer (CEST) MRI is a molecular imaging tool that provides physiological information about tissues, making it an invaluable tool for disease diagnosis and guided treatment. Its clinical application requires the acquisition of high-resolution images capable of accurately identifying subtle regional changes in vivo, while simultaneously maintaining a high level of spectral resolution. However, the acquisition of such high-resolution images is time consuming, presenting a challenge for practical implementation in clinical settings. Among several techniques that have been explored to reduce the acquisition time in MRI, deep-learning-based super-resolution (DLSR) is a promising approach to address this problem due to its adaptability to any acquisition sequence and hardware. However, its translation to CEST MRI has been hindered by the lack of the large CEST datasets required for network development. Thus, we aim to develop a DLSR method, named DLSR-CEST, to reduce the acquisition time for CEST MRI by reconstructing high-resolution images from fast low-resolution acquisitions. This is achieved by first pretraining the DLSR-CEST on human brain T1w and T2w images to initialize the weights of the network and then training the network on very small human and mouse brain CEST datasetsto fine-tune the weights. Using the trained DLSR-CEST network, the reconstructed CEST source images exhibited improved spatial resolution in both peak signal-to-noise ratio and structural similarity index measure metrics at all downsampling factors (2-8). Moreover, amide CEST and relayed nuclear Overhauser effect maps extrapolated from the DLSR-CEST source images exhibited high spatial resolution and low normalized root mean square error, indicating a negligible loss in Z-spectrum information. Therefore, our DLSR-CEST demonstrated a robust reconstruction of high-resolution CEST source images from fast low-resolution acquisitions, thereby improving the spatial resolution and preserving most Z-spectrum information.

High resolution imaging of acoustic orbital angular momentum wave based on orthogonal matching tracking

The orbital angular momentum (OAM) wave has shown great potential for improving radar imaging and underwater communication performance due to its helical wavefront phase and infinite orthogonal modes. However, there are currently no known applications of this technology in underwater imaging. In this paper, we employed acoustic OAM wave for underwater imaging and established transceiver signal models using the uniform circular array. We concurrently achieved two-dimensional imaging of azimuth and elevation angles, which differs from radar imaging. We proposed a matching process for the echo signal in the modal domain, the OAM wave beam image's sidelobe decreased by 7.9 dB in the elevation direction and 6.1 dB in the azimuth direction compared to the plane wave, with the mainlobe decreased by 0.2° in the elevation direction and 0.4° in the azimuth direction. Furthermore, this paper introduced OAM wave high-resolution image reconstruction based on the orthogonal matching pursuit (OMP) algorithm. Finally, we implemented broadband acoustic OAM wave for underwater imaging and introduced an image reconstruction method based on the modal domain OMP algorithm. Simulation results demonstrate that the use of OAM wave in underwater imaging is feasible, and the proposed scheme can achieve high-resolution imaging.

The Construction of a Lattice Image and Dislocation Analysis in High-Resolution Characterizations Based on Diffraction Extinctions.

This paper introduces a method for high-resolution lattice image reconstruction and dislocation analysis based on diffraction extinction. The approach primarily involves locating extinction spots in the Fourier transform spectrum (reciprocal space) and constructing corresponding diffraction wave functions. By the coherent combination of diffraction and transmission waves, the lattice image of the extinction planes is reconstructed. This lattice image is then used for dislocation localization, enabling the observation and analysis of crystal planes that exhibit electron diffraction extinction effects and atomic jump arrangements during high-resolution transmission electron microscopy (HRTEM) characterization. Furthermore, due to the method's effectiveness in localizing dislocations, it offers a unique advantage when analyzing high-resolution images with relatively poor quality. The feasibility of this method is theoretically demonstrated in this paper. Additionally, the method was successfully applied to observed edge dislocations, such as 1/6[211-], 1/6[2-11-], and 1/2[01-1], which are not easily observable in conventional HRTEM characterization processes, in electro-deposited Cu thin films. The Burgers vectors were determined. Moreover, this paper also attempted to observe screw dislocations that are challenging to observe in high-resolution transmission electron microscopy. By shifting a pair of diffraction extinction spots and superimposing the reconstructed images before and after the shift, screw dislocations with a Burgers vector of 1/2[011-] were successfully observed in electro-deposited Cu thin films.

Single image super-resolution based on Bendlets analysis and structural dictionary learning

High-resolution biological tissue slice images provide opportunities for more precise observation and analysis of histopathological features. These images typically consist of enclosed contours, and Bendlets transform is an effective means to approximate such structures. In this study, we leverage Bendlet transform to incorporate multi-level image data, thereby establishing a multi-scale pyramid feature set. This approach aids in achieving a sparser representation of image textures and structures. We apply Bendlets transform to extract multi-frequency wavelet subband features and formulate structured dictionaries for both high and low resolutions based on these features. After creating these dictionaries, we compute sparse representation coefficients using the low-resolution dictionary and subsequently integrate them with the high-resolution dictionary to generate high-resolution subbands. Ultimately, through the process of inverse wavelet transformation, we completed the reconstruction of high-resolution images. This method not only significantly enhances the restoration of image details and clarity but also effectively preserves the overall structural characteristics of the original images.

Shadow Detection and Reconstruction of High-Resolution Remote Sensing Images in Mountainous and Hilly Environments

Shadow Detection and Reconstruction of High-Resolution Remote Sensing Images in Mountainous and Hilly Environments

Computed Tomography of Transcatheter Aortic Valve Replacement-Related Complications

Transcatheter aortic valve replacement (TAVR) has revolutionized the treatment of severe aortic stenosis, with an impressive reduction in mortality and complication rates over the last decade, becoming a safer alternative in patients at high/moderate risk for surgical treatment. Complications are rare but can be life-threatening; thus, cardiac imagers must be familiar with their imaging findings. TAVR-related complications often require multimodality imaging, although ECG-gated cardiac computed tomography (CCT) is the technique of choice, given its high spatial resolution and multiplanar image reconstruction. This review will discuss the imaging findings of the most important post-TAVR complications and novel preventive measures.

Feature Pyramid Fusion Network for Hyperspectral Pansharpening.

Hyperspectral (HS) pansharpening aims at fusing an observed HS image with a panchromatic (PAN) image, to produce an image with the high spectral resolution of the former and the high spatial resolution of the latter. Most of the existing convolutional neural networks (CNNs)-based pansharpening methods reconstruct the desired high-resolution image from the encoded low-resolution (LR) representation. However, the encoded LR representation captures semantic information of the image and is inadequate in reconstructing fine details. How to effectively extract high-resolution and LR representations for high-resolution image reconstruction is the main objective of this article. In this article, we propose a feature pyramid fusion network (FPFNet) for pansharpening, which permits the network to extract multiresolution representations from PAN and HS images in two branches. The PAN branch starts from the high-resolution stream that maintains the spatial resolution of the PAN image and gradually adds LR streams in parallel. The structure of the HS branch remains highly consistent with that of the PAN branch, but starts with the LR stream and gradually adds high-resolution streams. The representations with corresponding resolutions of PAN and HS branches are fused and gradually upsampled in a coarse to fine manner to reconstruct the high-resolution HS image. Experimental results on three datasets demonstrate the significant superiority of the proposed FPFNet over the state-of-the-art methods in terms of both qualitative and quantitative comparisons.

A High-Resolution Image Reconstruction Method of Lung Nodules With B-Spline-Based Whale Optimization Algorithm (B-WOA) for Electrical Impedance Tomography

A High-Resolution Image Reconstruction Method of Lung Nodules With B-Spline-Based Whale Optimization Algorithm (B-WOA) for Electrical Impedance Tomography