Improve Reconstruction Quality Research Articles

NeuralRecon: Real-Time Coherent 3D Scene Reconstruction From Monocular Video.

We present a novel framework named NeuralRecon for real-time 3D scene reconstruction from a monocular video. Unlike previous methods that estimate single-view depth maps separately on each key-frame and fuse them later, we propose to directly reconstruct local surfaces represented as sparse TSDF volumes for each video fragment sequentially by a neural network. A learning-based TSDF fusion module based on gated recurrent units is used to guide the network to fuse features from previous fragments. This design allows the network to capture local smoothness prior and global shape prior of 3D surfaces when sequentially reconstructing the surfaces, resulting in accurate, coherent, and real-time surface reconstruction. The fused features can also be used to predict semantic labels, allowing our method to reconstruct and segment the 3D scene simultaneously. Furthermore, we purpose an efficient self-supervised fine-tuning scheme that refines scene geometry based on input images through differentiable volume rendering. This fine-tuning scheme improves reconstruction quality on the fine-tuned scenes, as well as the generalization to similar test scenes. The experiments on ScanNet, 7-Scenes and Replica datasets show that our system outperforms state-of-the-art methods in terms of both accuracy and speed.

Residual trio feature network for efficient super-resolution

Deep learning-based approaches have demonstrated impressive performance in single-image super-resolution (SISR). Efficient super-resolution compromises the reconstructed image’s quality to have fewer parameters and Flops. Ensured efficiency in image reconstruction and improved reconstruction quality of the model are significant challenges. This paper proposes a trio branch module (TBM) based on structural reparameterization. TBM achieves equivalence transformation through structural reparameterization operations, which use a complex network structure in the training phase and convert it to a more lightweight structure in the inference, achieving efficient inference while maintaining accuracy. Based on the TBM, we further design a lightweight version of the enhanced spatial attention mini (ESA-mini) and the residual trio feature block (RTFB). Moreover, the multiple RTFBs are combined to construct the residual trio network (RTFN). Finally, we introduce a localized contrast loss for better applicability to the super-resolution task, which enhances the reconstruction quality of the super-resolution model. Experiments show that the RTFN framework proposed in this paper outperforms other state-of-the-art efficient super-resolution methods in terms of inference speed and reconstruction quality.

Optimizing Parameters for Enhanced Iterative Image Reconstruction Using Extended Power Divergence

In this paper, we propose a method for optimizing the parameter values in iterative reconstruction algorithms that include adjustable parameters in order to optimize the reconstruction performance. Specifically, we focus on the power divergence-based expectation-maximization algorithm, which includes two power indices as adjustable parameters. Through numerical and physical experiments, we demonstrate that optimizing the evaluation function based on the extended power-divergence and weighted extended power-divergence measures yields high-quality image reconstruction. Notably, the optimal parameter values derived from the proposed method produce reconstruction results comparable to those obtained using the true image, even when using distance functions based on differences between forward projection data and measured projection data, as verified by numerical experiments. These results suggest that the proposed method effectively improves reconstruction quality without the need for machine-learning techniques in parameter selection. Our findings also indicate that this approach is useful for enhancing the performance of iterative reconstruction algorithms, especially in medical imaging, where high-accuracy reconstruction under noisy conditions is required.

ICe: interior cevian initialization for enhanced reconstruction methods

This paper introduces two novel self-induced basis strategies, interior cevian initialization for orthogonal matching pursuit (ICe) and its l2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$l_2$$\\end{document}-sense optimized version, optimized interior cevian initialization (OICe). These methods address the complexity and convergence issues of the previously proposed self-inspired bases (SIBS) approach. Both algorithms enhance sparse image representation by generating adaptive auxiliary atoms tailored to the image, thereby improving reconstruction quality and computational efficiency. Experimental results show that ICe achieves up to a 35.6% improvement in Normalized Mean Squared Error (NMSE) and a 16.4% increase in Structural Similarity Index (SSIM) compared to conventional OMP. OICe, as the l2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$l_2$$\\end{document}-sense optimized version of ICe, delivers even greater performance, with improvements of up to 40% in NMSE and 18.5% in SSIM. Additionally, both ICe and OICe significantly reduce bits per pixel (BPP) for image encoding, outperforming SIBS when paired with state-of-the-art dictionaries such as KSVD, MOD, RBDL, and ODL. These results highlight the effectiveness of ICe and OICe in addressing key challenges in sparse representation, offering enhanced accuracy and efficiency across a variety of applications.

Cross-Domain Denoising for Low-Dose Multi-Frame Spiral Computed Tomography.

Computed tomography (CT) has been used worldwide as a non-invasive test to assist in diagnosis. However, the ionizing nature of X-ray exposure raises concerns about potential health risks such as cancer. The desire for lower radiation doses has driven researchers to improve reconstruction quality. Although previous studies on low-dose computed tomography (LDCT) denoising have demonstrated the effectiveness of learning-based methods, most were developed on the simulated data. However, the real-world scenario differs significantly from the simulation domain, especially when using the multi-slice spiral scanner geometry. This paper proposes a two-stage method for the commercially available multi-slice spiral CT scanners that better exploits the complete reconstruction pipeline for LDCT denoising across different domains. Our approach makes good use of the high redundancy of multi-slice projections and the volumetric reconstructions while leveraging the over-smoothing issue in conventional cascaded frameworks caused by aggressive denoising. The dedicated design also provides a more explicit interpretation of the data flow. Extensive experiments on various datasets showed that the proposed method could remove up to 70% of noise without compromised spatial resolution, while subjective evaluations by two experienced radiologists further supported its superior performance against state-of-the-art methods in clinical practice. Code is available at https://github.com/YCL92/TMD-LDCT.

WTDUN: Wavelet Tree-Structured Sampling and Deep Unfolding Network for Image Compressed Sensing

Deep unfolding networks have gained increasing attention in the field of compressed sensing (CS) owing to their theoretical interpretability and superior reconstruction performance. However, most existing deep unfolding methods often face the following issues: 1) they learn directly from single-channel images, leading to a simple feature representation that does not fully capture complex features; and 2) they treat various image components uniformly, ignoring the characteristics of different components. To address these issues, we propose a novel wavelet-domain deep unfolding framework named WTDUN, which operates directly on the multi-scale wavelet subbands. Our method utilizes the intrinsic sparsity and multi-scale structure of wavelet coefficients to achieve a tree-structured sampling and reconstruction, effectively capturing and highlighting the most important features within images. Specifically, the design of tree-structured reconstruction aims to capture the inter-dependencies among the multi-scale subbands, enabling the identification of both fine and coarse features, which can lead to a marked improvement in reconstruction quality. Furthermore, a wavelet domain adaptive sampling method is proposed to greatly improve the sampling capability, which is realized by assigning measurements to each wavelet subband based on its importance. Unlike pure deep learning methods that treat all components uniformly, our method introduces a targeted focus on important subbands, considering their energy and sparsity. This targeted strategy lets us capture key information more efficiently while discarding less important information, resulting in a more effective and detailed reconstruction. Extensive experimental results on various datasets validate the superior performance of our proposed method.

TaNSR:Efficient 3D Reconstruction with Tetrahedral Difference and Feature Aggregation

AbstractNeural surface reconstruction methods have demonstrated their ability to recover 3D surfaces from multiple images. However, current approaches struggle to rapidly achieve high‐fidelity surface reconstructions. In this work, we propose TaNSR, which inherits the speed advantages of multi‐resolution hash encodings and extends its representation capabilities. To reduce training time, we propose an efficient numerical gradient computation method that significantly reduces additional memory access overhead. To further improve reconstruction quality and expedite training, we propose a feature aggregation strategy in volume rendering. Building on this, we introduce an adaptively weighted aggregation function to ensure the network can accurately reconstruct the surface of objects and recover more geometric details. Experiments on multiple datasets indicate that TaNSR significantly reduces training time while achieving better reconstruction accuracy compared to state‐of‐the‐art nerual implicit methods.

Accelerating multi-coil MR image reconstruction using weak supervision.

Deep-learning-based MR image reconstruction in settings where large fully sampled dataset collection is infeasible requires methods that effectively use both under-sampled and fully sampled datasets. This paper evaluates a weakly supervised, multi-coil, physics-guided approach to MR image reconstruction, leveraging both dataset types, to improve both the quality and robustness of reconstruction. A physics-guided end-to-end variational network (VarNet) is pretrained in a self-supervised manner using a 4 under-sampled dataset following the self-supervised learning via data undersampling (SSDU) methodology. The pre-trained weights are transferred to another VarNet, which is fine-tuned using a smaller, fully sampled dataset by optimizing multi-scale structural similarity (MS-SSIM) loss in image space. The proposed methodology is compared with fully self-supervised and fully supervised training. Reconstruction quality improvements in SSIM, PSNR, and NRMSE when abundant training data is available (the high-data regime), and enhanced robustness when training data is scarce (the low-data regime) are demonstrated using weak supervision for knee and brain MR image reconstructions at 8 and 10 acceleration, respectively. Multi-coil physics-guided MR image reconstruction using both under-sampled and fully sampled datasets is achievable with transfer learning and fine-tuning. This methodology can provide improved reconstruction quality in the high-data regime and improved robustness in the low-data regime at high acceleration rates.

AIDM-CT: software for cone-beam X-ray tomography and deep-learning-based analysis

ABSTRACT Non-destructive testing (NDT) with high-resolution cone-beam X-ray computed tomography (XCT) plays a crucial role in revealing the 3D distribution and morphology of defects within an object. It’s necessary to generalise an intelligent and customised XCT-based NDT protocol, providing solutions for reconstruction quality improvement and complex defect quantitative analysis for users. We analysed the key techniques of XCT and developed the systematic software for data acquisition, reconstruction, intelligent super-resolution and segmentation in our advanced imaging and data mining laboratory (AIDM-CT). The experimental results demonstrated the software efficiently achieves the control of XCT scanning, artefact correction algorithms related to cone-beam geometrics, beam hardening, ring and radial artefacts, deep-learning based slice super-resolution and feature segmentation, and quantitative analysis. The software is programmed by Python language to provide the friendly multi-function graphical user interface (GUI), in which the programming of reconstruction and corrections combined with Computer Unified Device Architecture (CUDA) acceleration to guarantee the high efficiency of XCT task. Particularly, the multi-functional super resolution module (MF-GAN) is proposed to optimise the quality of CT image and the improved U-Net segmentation module (CBAM U-Net) is also developed to fulfill the high-precision quantitative non-destructive testing of homogeneous and variform microdefects in our AIDM-CT software.

REXPACO ASDI: joint unmixing and deconvolution of the circumstellar environment by angular and spectral differential imaging

ABSTRACT Angular and spectral differential imaging is an observational technique of choice to investigate the immediate vicinity of stars. By leveraging the relative angular motion and spectral scaling between on-axis and off-axis sources, post-processing techniques can separate residual star light from light emitted by surrounding objects such as circumstellar discs or point-like objects. This paper introduces a new algorithm that jointly unmixes these components and deconvolves disc images. The proposed algorithm is based on a statistical model of the residual star light, accounting for its spatial and spectral correlations. These correlations are crucial yet remain inadequately modelled by existing reconstruction algorithms. We employ dedicated shrinkage techniques to estimate the large number of parameters of our correlation model in a data-driven fashion. We show that the resulting separable model of the spatial and spectral covariances captures very accurately the star light, enabling its efficient suppression. We apply our method to data sets from the Very Large Telescope/Spectro-Polarimetry High-contrast Exoplanet REsearch instrument and compare its performance with standard algorithms (median subtraction, PCA, PACO). We demonstrate that considering the multiple correlations within the data significantly improves reconstruction quality, resulting in better preservation of both disc morphology and photometry. With its unique joint spectral modelling, the proposed algorithm can reconstruct discs with circular symmetry (e.g. rings, spirals) at intensities one million times fainter than the star, without needing additional reference data sets free from off-axis objects.

Joint Deconvolution of Astronomical Images in the Presence of Poisson Noise

We present a new framework for joint likelihood deconvolution (Jolideco) of a set of astronomical observations of the same sky region in the presence of Poisson noise. The observations may be obtained from different instruments with different resolution, and different point-spread functions (PSFs). Jolideco reconstructs a single flux image by optimizing the posterior distribution based on the joint Poisson likelihood of all observations under one of several prior distributions. Most notably, we employ a patch-based image prior that is parameterized via a Gaussian mixture model, which we train on high-signal-to-noise astronomical images, including data from the JWST and the GLEAM radio survey. This prior favors correlation structures among the reconstructed pixel intensities that are characteristic of those observed in the training images. It is, however, not informative for the mean or scale of the reconstruction. By applying the method to simulated data, we show that the combination of multiple observations and the patch-based prior leads to much improved reconstruction quality in many different source scenarios and signal-to-noise regimes. We demonstrate that with the patch prior Jolideco yields superior reconstruction quality relative to alternative standard methods such as the Richardson–Lucy method. We illustrate the results of Jolideco applied to example data from the Chandra X-ray Observatory and the Fermi Gamma-ray Space Telescope. By comparing the measured width of a counts-based and the corresponding Jolideco flux profile of an X-ray filament in SNR 1E 0102.2–7219, we find the deconvolved width of 0.″58 ± 0.″02 to be consistent with the theoretical expectation derived from the known width of the PSF.

Space-time reconstruction for lensless imaging using implicit neural representations

Many computational imaging inverse problems are challenged by noise, model mismatch, and other imperfections that decrease reconstruction quality. For data taken sequentially in time, instead of reconstructing each frame independently, space-time algorithms simultaneously reconstruct multiple frames, thereby taking advantage of temporal redundancy through space-time priors. This helps with denoising and provides improved reconstruction quality, but often requires significant computational and memory resources. Designing effective but flexible temporal priors is also challenging. Here, we propose using an implicit neural representation to model dynamics and act as a computationally tractable and flexible space-time prior. We demonstrate this approach on video captured with a lensless imager, DiffuserCam, and show improved reconstruction results and robustness to noise compared to frame-by-frame methods.

Robust hyperspectral reconstruction via a multi-channel clustering compressive sensing approach

Wavelength-coded spectral imaging represents a fusion of spectral imaging and compressed sensing, offering advantages such as reduced storage requirements and straightforward miniaturization. This approach employs optical filters for hyperspectral reconstruction. However, issues like insufficient energy and excessive noise arise in low-light detection, leading to a notable degradation of image reconstruction quality. This paper presents a robust segmented reconstruction algorithm named MCC-WSCI (Multi-Channel Clustering-based Wavelength-Coded Spectral Imaging), designed to enhance the method's tolerance to noise. This algorithm can accurately classify and process compressed images obtained by wavelength-coded spectral imaging systems, thus improving reconstruction quality significantly. Numerical simulations and experiments in low-light scenarios are carried out to verify the proposed method. Results show that the MCC-WSCI method is robust to different noise and sampling rates and outperforms other state-of-the-art compressed sensing reconstruction methods in terms of reconstructed spatial resolution and spectral resolution. The proposed method provides effective experimental robustness to wavelength-coded spectral imaging with a natural algorithmic extension, paving the way for its application in remote sensing.

Deep-learning-based image reconstruction with limited data: generating synthetic raw data using deep learning.

Deep learning has shown great promise for fast reconstruction of accelerated MRI acquisitions by learning from large amounts of raw data. However, raw data is not always available in sufficient quantities. This study investigates synthetic data generation to complement small datasets and improve reconstruction quality. An adversarial auto-encoder was trained to generate phase and coil sensitivity maps from magnitude images, which were combined into synthetic raw data. On a fourfold accelerated MR reconstruction task, deep-learning-based reconstruction networks were trained with varying amounts of training data (20 to 160 scans). Test set performance was compared between baseline experiments and experiments that incorporated synthetic training data. Training with synthetic raw data showed decreasing reconstruction errors with increasing amounts of training data, but importantly this was magnitude-only data, rather than real raw data. For small training sets, training with synthetic data decreased the mean absolute error (MAE) by up to 7.5%, whereas for larger training sets the MAE increased by up to 2.6%. Synthetic raw data generation improved reconstruction quality in scenarios with limited training data. A major advantage of synthetic data generation is that it allows for the reuse of magnitude-only datasets, which are more readily available than raw datasets.

NN-VVC: A Hybrid Learned-Conventional Video Codec Targeting Humans and Machines

Advancements in artificial intelligence have significantly increased the use of images and videos in machine analysis algorithms, predominantly neural networks. However, the traditional methods of compressing, storing and transmitting media have been optimized for human viewers rather than machines. Current research in coding images and videos for machine analysis has evolved in two distinct paths. The first is characterized by End-to-End (E2E) learned codes, which show promising results in image coding but have yet to match the performance of leading Conventional Video Codecs (CVC) and suffer from a lack of interoperability. The second path optimizes CVC, such as the Versatile Video Coding (VVC) standard, for machine-oriented reconstruction. Although CVC-based approaches enjoy widespread hardware and software compatibility and interoperability, they often fall short in machine task performance, especially at lower bitrates. This paper proposes a novel hybrid codec for machines named NN-VVC, which combines the advantages of an E2E-learned image codec and a CVC to achieve high performance in both image and video coding for machines. Our experiments show that the proposed system achieved up to − 43.20% and − 26.8% Bjøntegaard Delta rate reduction over VVC for image and video data, respectively, when evaluated on multiple different datasets and machine vision tasks according to the common test conditions designed by the VCM study group in MPEG standardization activities. Furthermore, to improve reconstruction quality, we introduce a human-focused branch into our codec, enhancing the visual appeal of reconstructions intended for human supervision of the machine-oriented main branch.

AutoSamp: Autoencoding k-space Sampling via Variational Information Maximization for 3D MRI.

Accelerated MRI protocols routinely involve a predefined sampling pattern that undersamples the k-space. Finding an optimal pattern can enhance the reconstruction quality, however this optimization is a challenging task. To address this challenge, we introduce a novel deep learning framework, AutoSamp, based on variational information maximization that enables joint optimization of sampling pattern and reconstruction of MRI scans. We represent the encoder as a non-uniform Fast Fourier Transform that allows continuous optimization of k-space sample locations on a non-Cartesian plane, and the decoder as a deep reconstruction network. Experiments on public 3D acquired MRI datasets show improved reconstruction quality of the proposed AutoSamp method over the prevailing variable density and variable density Poisson disc sampling for both compressed sensing and deep learning reconstructions. We demonstrate that our data-driven sampling optimization method achieves 4.4dB, 2.0dB, 0.75dB, 0.7dB PSNR improvements over reconstruction with Poisson Disc masks for acceleration factors of R = 5, 10, 15, 25, respectively. Prospectively accelerated acquisitions with 3D FSE sequences using our optimized sampling patterns exhibit improved image quality and sharpness. Furthermore, we analyze the characteristics of the learned sampling patterns with respect to changes in acceleration factor, measurement noise, underlying anatomy, and coil sensitivities. We show that all these factors contribute to the optimization result by affecting the sampling density, k-space coverage and point spread functions of the learned sampling patterns.

ERT image reconstruction using marker region segmentation method

Abstract Inspired by image region segmentation method, a marking region segmentation iterative method is proposed to reconstruct sparse binary images of electrical resistance tomography. The grayscale matrix of the iteration process is mapped to another linear space for segmentation processing, adding the watershed thresholding of marking regions. In the iteration process for estimating conductivity distribution, the target regions are separated to avoid excessive segmentation effects. By applying this method in conjunction with the Landweber iterative model to solve the inverse problem of resistivity tomography imaging, more accurate binary images can be obtained, and the method exhibits superior convergence properties in comparison to the Landweber algorithm. To verify the reconstruction effectiveness of LW-TDIS method, numerical simulations and static experiments are conducted for comparison with three other methods. The results demonstrate that the proposed method effectively reduces reconstruction artifacts, improves reconstruction quality, and achieves better reconstruction performance.

Split-and-Fit: Learning B-Reps via Structure-Aware Voronoi Partitioning

We introduce a novel method for acquiring boundary representations (B-Reps) of 3D CAD models which involves a two-step process: it first applies a spatial partitioning , referred to as the "split", followed by a "fit" operation to derive a single primitive within each partition. Specifically, our partitioning aims to produce the classical Voronoi diagram of the set of ground-truth (GT) B-Rep primitives. In contrast to prior B-Rep constructions which were bottom-up, either via direct primitive fitting or point clustering, our Split-and-Fit approach is top-down and structure-aware , since a Voronoi partition explicitly reveals both the number of and the connections between the primitives. We design a neural network to predict the Voronoi diagram from an input point cloud or distance field via a binary classification. We show that our network, coined NVD-Net for neural Voronoi diagrams, can effectively learn Voronoi partitions for CAD models from training data and exhibits superior generalization capabilities. Extensive experiments and evaluation demonstrate that the resulting B-Reps, consisting of parametric surfaces, curves, and vertices, are more plausible than those obtained by existing alternatives, with significant improvements in reconstruction quality. Code will be released on https://github.com/yilinliu77/NVDNet.

Learning-Assisted Fast Determination of Regularization Parameter in Constrained Image Reconstruction.

To leverage machine learning (ML) for fast selection of optimal regularization parameter in constrained image reconstruction. Constrained image reconstruction is often formulated as a regularization problem and selecting a good regularization parameter value is an essential step. We solved this problem using an ML-based approach by leveraging the finding that for a specific constrained reconstruction problem defined for a fixed class of image functions, the optimal regularization parameter value is weakly subject-dependent and the dependence can be captured using few experimental data. The proposed method has four key steps: a) solution of a given constrained reconstruction problem for a few (say, 3) pre-selected regularization parameter values, b) extraction of multiple approximated quality metrics from the initial reconstructions, c) predicting the true quality metrics values from the approximated values using pre-trained neural networks, and d) determination of the optimal regularization parameter by fusing the predicted quality metrics. The effectiveness of the proposed method was demonstrated in two constrained reconstruction problems. Compared with L-curve-based method, the proposed method determined the regularization parameters much faster and produced substantially improved reconstructions. Our method also outperformed state-of-the-art learning-based methods when trained with limited experimental data. This paper demonstrates the feasibility and improved reconstruction quality by using machine learning to determine the regularization parameter in constrained reconstruction. The proposed method substantially reduces the computational burden of the traditional methods (e.g., L-curve) or relaxes the requirement of large training data by modern learning-based methods, thus enhancing the practical utility of constrained reconstruction.

Multiphoton simultaneous multislice imaging.

To develop multiphoton excitation techniques for simultaneous multislice (SMS) imaging and evaluate their performance and specific absorption rate (SAR) benefit. To improve multiphoton SMS reconstruction quality with a novel CAIPIRINHA (controlled aliasing in parallel imaging results in higher acceleration) design. When a conventional single-slice RF field is applied together with an oscillating gradient field, the two can combine to generate multiphoton excitation at multiple discrete spatial locations. Because the conventional RF is reused at multiple spatial locations, multiphoton excitation offers reduced SAR for SMS applications. CAIPIRINHA shifts are often used to improve parallel-imaging acceleration. Interestingly, CAIPIRINHA-type shifts can be obtained for multiphoton SMS by updating the oscillating gradient phase at every phase encode. In this work, both a gradient-echo and a spin-echo sequence with multiphoton CAIPIRINHA-SMS excitation pulses are implemented for in vivo human imaging at 3 T. For three slices, multiphoton SMS provides a 51% reduction in SAR compared with conventional superposition SMS, whereas for five slices, SAR is reduced by 66%. Multiphoton SMS outperforms PINS (power independent of number of slices) and MultiPINS in terms of SAR reduction especially when the pulse duration is short, slices are thin, and/or the slice spacing is large. A custom CAIPIRINHA phase-encoding design for multiphoton SMS significantly improves reconstruction quality. Multiphoton SMS excitation can be obtained by combining conventional single-slice RF pulses with an oscillating gradient and offers significant SAR benefits compared with conventional superposition SMS. A novel CAIPIRINHA design allows higher multiband factors for multiphoton SMS imaging.