Improve Reconstruction Quality Research Articles

DFUSNN: zero-shot dual-domain fusion unsupervised neural network for parallel MRI reconstruction

Objective. Recently, deep learning models have been used to reconstruct parallel magnetic resonance (MR) images from undersampled k-space data. However, most existing approaches depend on large databases of fully sampled MR data for training, which can be challenging or sometimes infeasible to acquire in certain scenarios. The goal is to develop an effective alternative for improved reconstruction quality that does not rely on external training datasets. Approach. We introduce a novel zero-shot dual-domain fusion unsupervised neural network (DFUSNN) for parallel MR imaging reconstruction without any external training datasets. We employ the Noise2Noise (N2N) network for the reconstruction in the k-space domain, integrate phase and coil sensitivity smoothness priors into the k-space N2N network, and use an early stopping criterion to prevent overfitting. Additionally, we propose a dual-domain fusion method based on Bayesian optimization to enhance reconstruction quality efficiently. Results. Simulation experiments conducted on three datasets with different undersampling patterns showed that the DFUSNN outperforms all other competing unsupervised methods and the one-shot Hankel-k-space generative model (HKGM). The DFUSNN also achieves comparable results to the supervised Deep-SLR method. Significance. The novel DFUSNN model offers a viable solution for reconstructing high-quality MR images without the need for external training datasets, thereby overcoming a major hurdle in scenarios where acquiring fully sampled MR data is difficult.

Lp-norm relaxation approach for 3D measurement via instantaneous light field of ethylene premixed flames

Optical imaging technology based on flame spontaneous radiation is a promising tool for combustion diagnosis. However, the existing radiation thermometry technology faces the problems of low angular resolution and low efficiency in solving radiative transfer. Therefore, a novel optical imaging model is proposed, which combines light field imaging theory with equation-solving integral equation method based on radiation distribution factor to effectively decouple the radiative properties related to transfer process and the temperature-related source terms. Aiming at the problem that traditional method may cause over-smooth effect, Lp-norm regularization based on non-negative constraint is established. Results show that Lp-norm can improve reconstruction quality and robustness, but p value needs to be reasonably selected according to noise level and flame inhomogeneity. Experimental study of ethylene/air-premixed flames shows that the methodology can realize 3D measurement (900 ∼ 1550 K). Compared with laser absorption spectroscopy and thermocouple, error is better than 3 % at height above burner 15 mm.

Low-Light Face Super-resolution via Illumination, Structure, and Texture Associated Representation

Human face captured at night or in dimly lit environments has become a common practice, accompanied by complex low-light and low-resolution degradations. However, the existing face super-resolution (FSR) technologies and derived cascaded schemes are inadequate to recover credible textures. In this paper, we propose a novel approach that decomposes the restoration task into face structural fidelity maintaining and texture consistency learning. The former aims to enhance the quality of face images while improving the structural fidelity, while the latter focuses on eliminating perturbations and artifacts caused by low-light degradation and reconstruction. Based on this, we develop a novel low-light low-resolution face super-resolution framework. Our method consists of two steps: an illumination correction face super-resolution network (IC-FSRNet) for lighting the face and recovering the structural information, and a detail enhancement model (DENet) for improving facial details, thus making them more visually appealing and easier to analyze. As the relighted regions could provide complementary information to boost face super-resolution and vice versa, we introduce the mutual learning to harness the informative components from relighted regions and reconstruction, and achieve the iterative refinement. In addition, DENet equipped with diffusion probabilistic model is built to further improve face image visual quality. Experiments demonstrate that the proposed joint optimization framework achieves significant improvements in reconstruction quality and perceptual quality over existing two-stage sequential solutions. Code is available at https://github.com/wcy-cs/IC-FSRDENet.

Alternating projection combined with fast gradient projection (FGP-AP) method for intensity-only measurement optical diffraction tomography in LED array microscopy.

Optical diffraction tomography (ODT) is a powerful label-free measurement tool that can quantitatively image the three-dimensional (3D) refractive index (RI) distribution of samples. However, the inherent "missing cone problem," limited illumination angles, and dependence on intensity-only measurements in a simplified imaging setup can all lead to insufficient information mapping in the Fourier domain, affecting 3D reconstruction results. In this paper, we propose the alternating projection combined with the fast gradient projection (FGP-AP) method to compensate for the above problem, which effectively reconstructs the 3D RI distribution of samples using intensity-only images captured from LED array microscopy. The FGP-AP method employs the alternating projection (AP) algorithm for gradient descent and the fast gradient projection (FGP) algorithm for regularization constraints. This approach is equivalent to incorporating prior knowledge of sample non-negativity and smoothness into the 3D reconstruction process. Simulations demonstrate that the FGP-AP method improves reconstruction quality compared to the original AP method, particularly in the presence of noise. Experimental results, obtained from mouse kidney cells and label-free blood cells, further affirm the superior 3D imaging efficacy of the FGP-AP method.

Image Deblurring Using Feedback Mechanism and Dual Gated Attention Network

Recently, image deblurring task driven by the encoder-decoder network has made a tremendous amount of progress. However, these encoder-decoder-based networks still have two disadvantages: (1) due to the lack of feedback mechanism in the decoder design, the reconstruction results of existing networks are still sub-optimal; (2) these networks introduce multiple modules, such as the self-attention mechanism, to improve the performance, which also increases the computational burden. To overcome these issues, this paper proposes a novel feedback-mechanism-based encoder-decoder network (namely, FMNet) that is equipped with two key components: (1) the feedback-mechanism-based decoder and (2) the dual gated attention module. To improve reconstruction quality, the feedback-mechanism-based decoder is proposed to leverage the feedback information via the feedback attention module, which adaptively selects useful features in the feedback path. To decrease the computational cost, an efficient dual gated attention module is proposed to perform the attention mechanism in the frequency domain twice, which improves deblurring performance while reducing the computational cost by avoiding redundant convolutions and feature channels. The superiority of FMNet in terms of both deblurring performance and computational efficiency is demonstrated via comparisons with state-of-the-art methods on multiple public datasets.

Compensating unknown speed of sound in learned fast 3D limited-view photoacoustic tomography

Real-time applications in three-dimensional photoacoustic tomography from planar sensors rely on fast reconstruction algorithms that assume the speed of sound (SoS) in the tissue is homogeneous. Moreover, the reconstruction quality depends on the correct choice for the constant SoS. In this study, we discuss the possibility of ameliorating the problem of unknown or heterogeneous SoS distributions by using learned reconstruction methods. This can be done by modelling the uncertainties in the training data. In addition, a correction term can be included in the learned reconstruction method. We investigate the influence of both and while a learned correction component can improve reconstruction quality further, we show that a careful choice of uncertainties in the training data is the primary factor to overcome unknown SoS. We support our findings with simulated and in vivo measurements in 3D.

Spatially‐Coded Fourier Ptychography: Flexible and Detachable Coded Thin Films for Quantitative Phase Imaging with Uniform Phase Transfer Characteristics

AbstractFourier ptychography (FP) is an enabling imaging technique that produces high‐resolution complex‐valued images with extended field coverages. However, when FP images a phase object with any specific spatial frequency, the captured images contain only constant values, rendering the recovery of the corresponding linear phase ramp impossible. This challenge is not unique to FP but also affects other common microscopy techniques — a rather counterintuitive outcome given their widespread use in phase imaging. The underlying issue originates from the non‐uniform phase transfer characteristic inherent in microscope systems, which impedes the conversion of object wavefields into discernible intensity variations. To address this challenge, spatially‐coded Fourier ptychography (scFP) is presented for true quantitative phase imaging. In scFP, a flexible and detachable coded thin film is attached atop the image sensor in a regular FP setup. The spatial modulation of this thin film ensures a uniform phase response across the entire synthetic bandwidth. It improves reconstruction quality, corrects refractive index underestimation issues prevalent in conventional FP, and adds a new dimension of measurement diversity in spatial domain. The development of scFP is expected to catalyze new research directions and applications for phase imaging, emphasizing the need for true quantitative accuracy with uniform frequency response.

Joint multi-contrast CT for edge illumination X-ray phase contrast imaging using split Barzilai-Borwein steps.

Edge illumination (EI) is an X-ray imaging technique that, in addition to conventional absorption contrast, provides refraction and scatter contrast. It relies on an absorption mask in front of the sample that splits the X-ray beam into beamlets, which hits a second absorption mask positioned in front of the detector. The sample mask is then shifted in multiple steps with respect to the detector mask, thereby measuring an illumination curve per detector element. The width, position, and area of this curve estimated with and without the sample in the beam is then compared, which ultimately provides absorption, refraction, and scatter contrast for each detector pixel. From the obtained contrast sinograms, three contrast tomograms can be computed. In summary, conventional EI relies on a two-stage process comprised of a computational and time intensive contrast retrieval process, followed by tomographic reconstruction. In this work, a novel joint reconstruction method is proposed, which utilizes a combined forward model to reconstruct the three contrasts simultaneously, without the need for an intermediate contrast retrieval step. Compared to the state-of-the-art, this approach reduces reconstruction times, as the retrieval step is skipped and allows a much more flexible acquisition scheme, as there is no need to sample a full illumination curve at each projection angle. The proposed method is shown to improve reconstruction quality on subsampled datasets, enabling the reconstruction of three contrasts from single-shot datasets.

Learning-Assisted Fast Determination of Regularization Parameter in Constrained Image Reconstruction.

To leverage machine learning (ML) for fast selection of optimal regularization parameter in constrained image reconstruction. Constrained image reconstruction is often formulated as a regularization problem and selecting a good regularization parameter value is an essential step. We solved this problem using an ML-based approach by leveraging the finding that for a specific constrained reconstruction problem defined for a fixed class of image functions, the optimal regularization parameter value is weakly subject-dependent and the dependence can be captured using few experimental data. The proposed method has four key steps: a) solution of a given constrained reconstruction problem for a few (say, 3) pre-selected regularization parameter values, b) extraction of multiple approximated quality metrics from the initial reconstructions, c) predicting the true quality metrics values from the approximated values using pre-trained neural networks, and d) determination of the optimal regularization parameter by fusing the predicted quality metrics. The effectiveness of the proposed method was demonstrated in two constrained reconstruction problems. Compared with L-curve-based method, the proposed method determined the regularization parameters much faster and produced substantially improved reconstructions. Our method also outperformed state-of-the-art learning-based methods when trained with limited experimental data. This paper demonstrates the feasibility and improved reconstruction quality by using machine learning to determine the regularization parameter in constrained reconstruction. The proposed method substantially reduces the computational burden of the traditional methods (e.g., L-curve) or relaxes the requirement of large training data by modern learning-based methods, thus enhancing the practical utility of constrained reconstruction.

NeuralRecon: Real-Time Coherent 3D Scene Reconstruction from Monocular Video.

We present a novel framework named NeuralRecon for real-time 3D scene reconstruction from a monocular video. Unlike previous methods that estimate single-view depth maps separately on each key-frame and fuse them later, we propose to directly reconstruct local surfaces represented as sparse TSDF volumes for each video fragment sequentially by a neural network. A learning-based TSDF fusion module based on gated recurrent units is used to guide the network to fuse features from previous fragments. This design allows the network to capture local smoothness prior and global shape prior of 3D surfaces when sequentially reconstructing the surfaces, resulting in accurate, coherent, and real-time surface reconstruction. The fused features can also be used to predict semantic labels, allowing our method to reconstruct and segment the 3D scene simultaneously. Furthermore, we purpose an efficient self-supervised fine-tuning scheme that refines scene geometry based on input images through differentiable volume rendering. This fine-tuning scheme improves reconstruction quality on the fine-tuned scenes as well as the generalization to similar test scenes. The experiments on ScanNet, 7-Scenes and Replica datasets show that our system outperforms state-of-the-art methods in terms of both accuracy and speed.

Laplacian Projection Based Global Physical Prior Smoke Reconstruction.

We present a novel framework for reconstructing fluid dynamics in real-life scenarios. Our approach leverages sparse view images and incorporates physical priors across long series of frames, resulting in reconstructed fluids with enhanced physical consistency. Unlike previous methods, we utilize a differentiable fluid simulator (DFS) and a differentiable renderer (DR) to exploit global physical priors, reducing reconstruction errors without the need for manual regularization coefficients. We introduce divergence-free Laplacian eigenfunctions (div-free LE) as velocity bases, improving computational efficiency and memory usage. By employing gradient-related strategies, we achieve better convergence and superior results. Extensive experiments demonstrate the effectiveness of our method, showcasing improved reconstruction quality and computational efficiency compared to existing approaches. We validate our approach using both synthetic and real data, highlighting its practical potential.

Cross-domain Denoising for Low-dose Multi-frame Spiral Computed Tomography.

Computed tomography (CT) has been used worldwide as a non-invasive test to assist in diagnosis. However, the ionizing nature of X-ray exposure raises concerns about potential health risks such as cancer. The desire for lower radiation doses has driven researchers to improve reconstruction quality. Although previous studies on low-dose computed tomography (LDCT) denoising have demonstrated the effectiveness of learning-based methods, most were developed on the simulated data. However, the real-world scenario differs significantly from the simulation domain, especially when using the multi-slice spiral scanner geometry. This paper proposes a two-stage method for the commercially available multi-slice spiral CT scanners that better exploits the complete reconstruction pipeline for LDCT denoising across different domains. Our approach makes good use of the high redundancy of multi-slice projections and the volumetric reconstructions while leveraging the over-smoothing issue in conventional cascaded frameworks caused by aggressive denoising. The dedicated design also provides a more explicit interpretation of the data flow. Extensive experiments on various datasets showed that the proposed method could remove up to 70% of noise without compromised spatial resolution, while subjective evaluations by two experienced radiologists further supported its superior performance against state-of-the-art methods in clinical practice. Code is available at https://github.com/YCL92/TMD-LDCT.

An unrolled half-quadratic approach for sparse signal recovery in spectroscopy

This paper addresses the problem of sparse signal reconstruction frequently arising in analytical chemistry. Spectroscopic restoration is an important task that was tackled using two distinct approaches. First, model-based iterative methods were deployed. These methods rely on a strong theoretical background. However they face practical obstacles in terms of hyperparameter tuning. Conversely, approaches based on deep neural networks are fast and easy to implement. Nonetheless, they are often described as black-box tools, and their stability/robustness are ongoing research topics. In this paper, we propose a deep neural network based on unrolling a Majorization-Minimization (MM) Half-Quadratic (HQ) algorithm. This allows us to build interpretable layers mirroring iterations, making it possible to learn automatically data-driven hyperparameters such as regularization and stepsizes. Furthermore, we propose a dictionary of custom activation functions derived from potentials used in the original variational model. This interpretation of activations can be useful for analyzing the stability of neural networks. The efficiency of our method in comparison to iterative and learning-based methods is showcased through various experiments conducted on realistic mass spectrometry (MS) databases with various blur kernels and noise levels. All experiments show an improved reconstruction quality of chemical signals, while maintaining a low execution time at the test phase. All the results are reproducible, through the code shared on a repository available at https://github.com/GHARBIMouna/Unrolled-Half-Quadratic.

Physics constrained unsupervised deep learning for rapid, high resolution scanning coherent diffraction reconstruction

By circumventing the resolution limitations of optics, coherent diffractive imaging (CDI) and ptychography are making their way into scientific fields ranging from X-ray imaging to astronomy. Yet, the need for time consuming iterative phase recovery hampers real-time imaging. While supervised deep learning strategies have increased reconstruction speed, they sacrifice image quality. Furthermore, these methods’ demand for extensive labeled training data is experimentally burdensome. Here, we propose an unsupervised physics-informed neural network reconstruction method, PtychoPINN, that retains the factor of 100-to-1000 speedup of deep learning-based reconstruction while improving reconstruction quality by combining the diffraction forward map with real-space constraints from overlapping measurements. In particular, PtychoPINN gains a factor of 4 in linear resolution and an 8 dB improvement in PSNR while also accruing improvements in generalizability and robustness. This blend of performance and computational efficiency offers exciting prospects for high-resolution real-time imaging in high-throughput environments such as X-ray free electron lasers (XFELs) and diffraction-limited light sources.

CamP: Camera Preconditioning for Neural Radiance Fields

Neural Radiance Fields (NeRF) can be optimized to obtain high-fidelity 3D scene reconstructions of objects and large-scale scenes. However, NeRFs require accurate camera parameters as input --- inaccurate camera parameters result in blurry renderings. Extrinsic and intrinsic camera parameters are usually estimated using Structure-from-Motion (SfM) methods as a pre-processing step to NeRF, but these techniques rarely yield perfect estimates. Thus, prior works have proposed jointly optimizing camera parameters alongside a NeRF, but these methods are prone to local minima in challenging settings. In this work, we analyze how different camera parameterizations affect this joint optimization problem, and observe that standard parameterizations exhibit large differences in magnitude with respect to small perturbations, which can lead to an ill-conditioned optimization problem. We propose using a proxy problem to compute a whitening transform that eliminates the correlation between camera parameters and normalizes their effects, and we propose to use this transform as a preconditioner for the camera parameters during joint optimization. Our preconditioned camera optimization significantly improves reconstruction quality on scenes from the Mip-NeRF 360 dataset: we reduce error rates (RMSE) by 67% compared to state-of-the-art NeRF approaches that do not optimize for cameras like Zip-NeRF, and by 29% relative to state-of-the-art joint optimization approaches using the camera parameterization of SCNeRF. Our approach is easy to implement, does not significantly increase runtime, can be applied to a wide variety of camera parameterizations, and can straightforwardly be incorporated into other NeRF-like models.

A compressive sensing approach for inferring cognitive representations with reverse correlation.

Uncovering cognitive representations is an elusive goal that is increasingly pursued using the reverse correlation method, wherein human subjects make judgments about ambiguous stimuli. Employing reverse correlation often entails collecting thousands of stimulus-response pairs, which severely limits the breadth of studies that are feasible using the method. Current techniques to improve efficiency bias the outcome. Here we show that this methodological barrier can be diminished using compressive sensing, an advanced signal processing technique designed to improve sampling efficiency. Simulations are performed to demonstrate that compressive sensing can improve the accuracy of reconstructed cognitive representations and dramatically reduce the required number of stimulus-response pairs. Additionally, compressive sensing is used on human subject data from a previous reverse correlation study, demonstrating a dramatic improvement in reconstruction quality. This work concludes by outlining the potential of compressive sensing to improve representation reconstruction throughout the fields of psychology, neuroscience, and beyond.

Multi-derivative method for phase extraction without knowing carrier frequencies in off-axis quantitative phase imaging.

We propose a multi-derivative method to reconstruct the phase of transparent objects in off-axis quantitative phase imaging (QPI). By numerically computing first-, second-, and third-order derivatives of the interferogram, we demonstrate that one can extract the quantitative phase information in a straightforward way, without prior knowledge of the carrier frequencies or Fourier transform. In contrast to existing advanced derivative methods, our approach markedly streamlines the alignment and retrieval processes, all without requiring any special prerequisites. This enhancement seamlessly translates into improved reconstruction quality. Furthermore, when compared to cutting-edge Fourier-division-based methods, our technique distinctly accelerates the phase retrieval speed. We verified our method using white-light diffraction phase microscopy and laser off-axis QPI, and the results indicate that our method can allow a fast, high-quality retrieval with frame rates up to 41.6 fps for one- megapixel interferograms on a regular computer.

Multi-Scale CNN-Transformer Dual Network for Hyperspectral Compressive Snapshot Reconstruction

Coded aperture snapshot spectral imaging (CASSI) is a new imaging mode that captures the spectral characteristics of materials in real scenes. It encodes three-dimensional spatial–spectral data into two-dimensional snapshot measurements, and then recovers the original hyperspectral image (HSI) through a reconstruction algorithm. Hyperspectral data have multi-scale coupling correlations in both spatial and spectral dimensions. Designing a network architecture that effectively represents this coupling correlation is crucial for enhancing reconstruction quality. Although the convolutional neural network (CNN) can effectively represent local details, it cannot capture long-range correlation well. The Transformer excels at representing long-range correlation within the local window, but there are also issues of over-smoothing and loss of details. In order to cope with these problems, this paper proposes a dual-branch CNN-Transformer complementary module (DualCT). Its CNN branch mainly focuses on learning the spatial details of hyperspectral images, and the Transformer branch captures the global correlation between spectral bands. These two branches are linked through bidirectional interactions to promote the effective fusion of spatial–spectral features of the two branches. By utilizing characteristics of CASSI imaging, the residual mask attention is also designed and encapsulated in the DualCT module to refine the fused features. Furthermore, by using the DualCT module as a basic component, a multi-scale encoding and decoding model is designed to capture multi-scale spatial–spectral features of hyperspectral images and achieve end-to-end reconstruction. Experiments show that the proposed network can effectively improve reconstruction quality, and ablation experiments also verify the effectiveness of our network design.

Runtime optimization of acquisition trajectories for x-ray computed tomography with a robotic sample holder

Tomographic imaging systems are expected to work with a wide range of samples that house complex structures and challenging material compositions, which can influence image quality in a bad way. Complex samples increase total measurement duration and may introduce beam-hardening artifacts that lead to poor reconstruction image quality. This work presents an online trajectory optimization method for an x-ray computed tomography system with a robotic sample holder. The proposed method reduces measurement time and increases reconstruction image quality by generating an optimized spherical trajectory for the given sample without prior knowledge. The trajectory is generated successively at runtime based on intermediate sample measurements. We present experimental results with the robotic sample holder where two sample measurements using an optimized spherical trajectory achieve improved reconstruction quality compared to a conventional spherical trajectory. Our results demonstrate the ability of our system to increase reconstruction image quality and avoid artifacts at runtime when no prior information about the sample is provided.

Fast physic-informed mixer architecture for color Lensfree holographic reconstruction

Accurate color image reconstruction from multi-wavelength holograms is crucial in biomedical imaging applications. Current data-driven deep learning methods have made significant progress with promising reconstruction performance. Especially, untrained neural network methods have been developed to overcome data acquisition and the generalization issue. However, improving reconstruction quality requires more iterations, which can lead to longer convergence time. To address this challenge, we proposed an efficient complex-valued attention mixer (ECA-Mixer) architecture for fast and accurate physic-informed color holographic reconstruction. Our architecture consists of three core modules, an encoder, a nonlinear transformer and a decoder, each combining efficient attention mechanisms and Mixer layers for channel feature extraction and spatial information transformation. To preserve high-frequency information, we also introduced the use of 2D Haar wavelet and its inverse transform for encoding and decoding features. We then validate our method with extensive experiments on simulated and experimental samples and achieve state-of-art color reconstruction results in terms of computation time and image quality. Besides, we also demonstrate that our proposed lightweight network is capable of imaging large-size wide-field samples at a higher resolution in just a few minutes. The code is available and will be uploaded later at https://github.com/DeepPhysicVision/ECA-Mixer.git.