Improve Reconstruction Accuracy Research Articles

Group sparse-based Taylor expansion method for liver pharmacokinetic parameters imaging of dynamic fluorescence molecular tomography

Objective. Pharmacokinetic parametric images obtained through dynamic fluorescence molecular tomography (DFMT) has ability of capturing dynamic changes in fluorescence concentration, thereby providing three-dimensional metabolic information for applications in biological research and drug development. However, data processing of DFMT is time-consuming, involves a vast amount of data, and the problem itself is ill-posed, which significantly limits the application of pharmacokinetic parametric images reconstruction. In this study, group sparse-based Taylor expansion method is proposed to address these problems. Approach. Firstly, Taylor expansion framework is introduced to reduce time and computational cost. Secondly, group sparsity based on structural prior is introduced to improve reconstruction accuracy. Thirdly, alternating iterative solution based on accelerated gradient descent algorithm is introduced to solve the problem. Main results. Numerical simulation and in vivo experimental results demonstrate that, in comparison to existing methods, the proposed approach significantly enhances reconstruction speed without a degradation of quality, particularly when confronted with background fluorescence interference from other organs. Significance. Our research greatly reduces time and computational cost, providing strong support for real-time monitoring of liver metabolism.

ZH3: Quadratic Zonal Harmonics

Spherical Harmonics (SH) have been used widely to represent lighting in games and film. While the quadratic (SH3) and higher order spherical harmonics represent irradiance well, they are expensive to store and evaluate, requiring 27 coefficients per sample. Linear SH (SH2), requiring only 12 coefficients, are sometimes used, but they do not represent irradiance signals accurately and can have challenges with negative reconstruction. We introduce a new representation (ZH3) that augments linear SH with just the zonal coefficient of quadratic SH, yielding significant visual improvement with just 15 coefficients, and discuss how solving for a luminance zonal axis can significantly improve reconstruction accuracy and reduce color artifacts. We also discuss how, rather than storing the ZH3 coefficients explicitly, we can hallucinate them from the linear SH, improving reconstruction accuracy over linear SH at minimal extra cost.

Bayesian model error method for the passive inverse scattering problem

This paper focuses on the passive inverse scattering problem, which uses passive measurements corresponding to randomly distributed incident sources to recover the shape of the sound-soft obstacle from a Bayesian perspective. Due to the unpredictability and randomness of incident sources, the classical Bayesian inversion framework may be unable to capture the likelihood involving the passive forward model for this inverse problem. We present the Bayesian model error method (BMEM), a novel passive imaging technique, to overcome this difficulty. The cross-correlations and the Helmholtz–Kirchhoff identity are specifically used to build an approximate active scattering model. This approximate model and the model error that it produces can be combined effectively by the suggested BMEM. The well-posedness of the posterior measure in the BMEM is proved. To further estimate the model error, an online scheme is utilized in conjunction with a preconditioned Crank–Nicolson Markov Chain Monte Carlo method to numerically approximate the posterior. Numerical experiments illustrate the effectiveness of the proposed method and also show that the online evaluation of model error can significantly improve reconstruction accuracy.

Spectral reconstruction of fundus images using retinex-based semantic spectral separation transformer, applied for retinal oximetry

Multispectral imaging, a non-invasive technique to measure retinal oxygen saturation levels, is limited due to its time-consuming and expensive nature. In this paper, we present R3ST (Retinex-based Semantic Spectral Separation Transformer), an innovative end-to-end method for RGB fundus image spectral reconstruction. Our proposed model leverages RGB images to reconstruct multispectral images, making retinal oximetry more accessible for diagnostics.Existing spectral reconstruction methods face challenges in achieving high accuracy and generalizing across camera sources, affecting diagnostic results. R3ST addresses these issues through an unsupervised semantic spectral separation module that strives to encourage the separate reconstruction of pixels with different semantics as much as possible. Additionally, we introduce an oxygen saturation loss to improve reconstruction accuracy within the relevant regions. To overcome the limitations of generalization properties, our model incorporates a self-supervised Retinex reflectance extraction module. This simulates various imaging systems and extracts reflectance information from fundus images for reconstruction.Extensive experiments demonstrate R3ST’s superior performance in spectral reconstruction accuracy, making retinal oximetry a more viable, cost-effective diagnostic option that provides essential eye health and systemic health information. The code of R3ST is available at https://github.com/telesc0pe/R3ST.

A noise robust image reconstruction using slice aware cycle interpolator network for parallel imaging in MRI.

Reducing Magnetic resonance imaging (MRI) scan time has been an important issue for clinical applications. In order to reduce MRI scan time, imaging acceleration was made possible by undersampling k-space data. This is achieved by leveraging additional spatial information from multiple, independent receiver coils, thereby reducing the number of sampled k-spacelines. The aim of this study is to develop a deep-learning method for parallel imaging with a reduced number of auto-calibration signals (ACS) lines in noisyenvironments. A cycle interpolator network is developed for robust reconstruction of parallel MRI with a small number of ACS lines in noisy environments. The network estimates missing (unsampled) lines of each coil data, and these estimated missing lines are then utilized to re-estimate the sampled k-space lines. In addition, a slice aware reconstruction technique is developed for noise-robust reconstruction while reducing the number of ACS lines. We conducted an evaluation study using retrospectively subsampled data obtained from three healthy volunteers at 3T MRI, involving three different slice thicknesses (1.5, 3.0, and 4.5 mm) and three different image contrasts (T1w, T2w, and FLAIR). Despite the challenges posed by substantial noise in cases with a limited number of ACS lines and thinner slices, the slice aware cycle interpolator network reconstructs the enhanced parallel images. It outperforms RAKI, effectively eliminating aliasing artifacts. Moreover, the proposed network outperforms GRAPPA and demonstrates the ability to successfully reconstruct brain images even under severe noisyconditions. The slice aware cycle interpolator network has the potential to improve reconstruction accuracy for a reduced number of ACS lines in noisyenvironments.

LNMVSNet: A Low-Noise Multi-View Stereo Depth Inference Method for 3D Reconstruction.

With the widespread adoption of modern RGB cameras, an abundance of RGB images is available everywhere. Therefore, multi-view stereo (MVS) 3D reconstruction has been extensively applied across various fields because of its cost-effectiveness and accessibility, which involves multi-view depth estimation and stereo matching algorithms. However, MVS tasks face noise challenges because of natural multiplicative noise and negative gain in algorithms, which reduce the quality and accuracy of the generated models and depth maps. Traditional MVS methods often struggle with noise, relying on assumptions that do not always hold true under real-world conditions, while deep learning-based MVS approaches tend to suffer from high noise sensitivity. To overcome these challenges, we introduce LNMVSNet, a deep learning network designed to enhance local feature attention and fuse features across different scales, aiming for low-noise, high-precision MVS 3D reconstruction. Through extensive evaluation of multiple benchmark datasets, LNMVSNet has demonstrated its superior performance, showcasing its ability to improve reconstruction accuracy and completeness, especially in the recovery of fine details and clear feature delineation. This advancement brings hope for the widespread application of MVS, ranging from precise industrial part inspection to the creation of immersive virtual environments.

Unsupervised spectral reconstruction from RGB images under two lighting conditions.

Unsupervised spectral reconstruction (SR) aims to recover the hyperspectral image (HSI) from corresponding RGB images without annotations. Existing SR methods achieve it from a single RGB image, hindered by the significant spectral distortion. Although several deep learning-based methods increase the SR accuracy by adding RGB images, their networks are always designed for other image recovery tasks, leaving huge room for improvement. To overcome this problem, we propose a novel, to our knowledge, approach that reconstructs the HSI from a pair of RGB images captured under two illuminations, significantly improving reconstruction accuracy. Specifically, an SR iterative model based on two illuminations is constructed at first. By unfolding the proximal gradient algorithm solving this SR model, an interpretable unsupervised deep network is proposed. All the modules in the proposed network have precise physical meanings, which enable our network to have superior performance and good generalization capability. Experimental results on two public datasets and our real-world images show the proposed method significantly improves both visually and quantitatively as compared with state-of-the-art methods.

MiPhDUO: microwave imaging via physics-informed deep unrolled optimization

Microwave imaging (MWI) is a non-invasive technique that can identify unknown scatterer objects’ features while offering advantages such as low cost and portable devices with respect to other imaging methods. However, MWI faces challenges in solving the underlying inverse scattering problem, which involves recovering target properties from its scattered fields. Existing methods include linearized and non-linear optimization approaches, but they have limitations respectively in terms of range of validity and computational complexity (in view of the possible occurrence of ‘false solutions’). In recent years, learning-based approaches have emerged as they can allow real-time imaging but usually lack generalizability and a direct connection to the underlying physics. This paper proposes a physics-informed approach that combines convolutional neural networks with physics-based calculations. It is based on a few cascaded operations, making use of the gradient of the relevant cost function, and successively improving the estimation of the unknown target. The proposed approach is assessed using simulated as well as experimental Fresnel data. The results show that the integration of physics with deep learning can contribute to improve reconstruction accuracy, generalizability, and computational efficiency in MWI.

Neurobiologically realistic neural network enables cross-scale modeling of neural dynamics

Fundamental principles underlying computation in multi-scale brain networks illustrate how multiple brain areas and their coordinated activity give rise to complex cognitive functions. Whereas brain activity has been studied at the micro- to meso-scale to reveal the connections between the dynamical patterns and the behaviors, investigations of neural population dynamics are mainly limited to single-scale analysis. Our goal is to develop a cross-scale dynamical model for the collective activity of neuronal populations. Here we introduce a bio-inspired deep learning approach, termed NeuroBondGraph Network (NBGNet), to capture cross-scale dynamics that can infer and map the neural data from multiple scales. Our model not only exhibits more than an 11-fold improvement in reconstruction accuracy, but also predicts synchronous neural activity and preserves correlated low-dimensional latent dynamics. We also show that the NBGNet robustly predicts held-out data across a long time scale (2 weeks) without retraining. We further validate the effective connectivity defined from our model by demonstrating that neural connectivity during motor behaviour agrees with the established neuroanatomical hierarchy of motor control in the literature. The NBGNet approach opens the door to revealing a comprehensive understanding of brain computation, where network mechanisms of multi-scale activity are critical.

Data and measurement mechanism integrated imaging method for electrical capacitance tomography

This study presents a new imaging paradigm for overcoming the challenges limiting the improvement of the imaging quality in the electrical capacitance technique. The new imaging model enables the integration of measurement physics, sparsity-induced prior and physics-informed multi-fidelity learning prior (PIMFLP), as well as the synergy between data-driven and measurement physics modeling paradigms. The transformed L1 norm is used to model the PIMFLP, and the maximum correntropy criterion is used as a data misfit term to restrict the adverse impact of noises or outliers. The PIMFLP is learned from data and characterizes the structural details of imaging targets. The half-quadratic optimization method is developed to overcome computational challenges in solving the novel imaging model. A new physics-informed multi-fidelity learning method is developed to predict PIMFLP by synergizing deep convolutional neural network with measurement physics, and a new bilevel optimization model solved by a new nested algorithm that merges the genetic algorithm and the split Bregman method is proposed for training. Comparisons with other popular reconstruction algorithms confirm that the new imaging method leads to the improved reconstruction accuracy and noise immunity, and opens up new possibilities for unlocking the potential of the measurement technology.

Seismic data reconstruction based on a multicascade self-guided network

Due to inherent limitations in data acquisition, seismic data reconstruction is an important procedure to recover missing data or improve observation density. Many conventional methods exist to solve the reconstruction task. Reconstruction is challenging, especially in the case of complex seismic data. Recently, convolutional neural networks (CNN) have been applied in seismic data processing. In most cases, the architectures of these CNN-based methods are relatively simple, without sufficient feature interaction, limiting their performance. To improve reconstruction results, a multicascade self-guided network (MSG-Net) is presented. In general, MSG-Net is inspired by the self-guided scheme, and a multicascade architecture is designed to extract informative features within the analyzed seismic data at different resolutions. Following this, a parallel spatial attention module is used to further refine and enhance the primary features, thereby improving reconstruction accuracy. To test and verify the new approach, a training data set is generated, based on the synthetic records obtained by forward modeling methods. Experimental results demonstrate that MSG-Net is a promising approach for performing seismic data interpolation.

Compressive Perception Image Reconstruction Technology for Basic Mixed Sparse Basis in Metal Surface Detection

<p>Applying Compressed Sensing (CS) technology to robot vision image transmission, an effective method for image reconstruction in robot imaging is proposed to improve the accuracy of reconstruction. Reconstructing images using a mixed sparse representation of DCT and circularly symmetric contour wave transform, the basic algorithm used is the Smoothed Projection Landweber (SPL) algorithm, which optimizes the coefficients under different sparse transformations by incorporating hard thresholding and binary thresholding methods for different sparse bases during iterations. The experiment shows that compared with single sparse base image reconstruction, the proposed reconstruction method has improved reconstruction accuracy.</p> <p> </p>

MsDC-DEQ-Net: Deep Equilibrium Model (DEQ) with Multiscale Dilated Convolution for Image Compressive Sensing (CS)

Compressive sensing (CS) is a technique that enables the recovery of sparse signals using fewer measurements than traditional sampling methods. To address the computational challenges of CS reconstruction, our objective is to develop an interpretable and concise neural network model for reconstructing natural images using CS. We achieve this by mapping one step of the iterative shrinkage thresholding algorithm (ISTA) to a deep network block, representing one iteration of ISTA. To enhance learning ability and incorporate structural diversity, we integrate aggregated residual transformations (ResNeXt) and squeeze-and-excitation mechanisms into the ISTA block. This block serves as a deep equilibrium layer connected to a semi-tensor product network for convenient sampling and providing an initial reconstruction. The resulting model, called MsDC-DEQ-Net, exhibits competitive performance compared to state-of-the-art network-based methods. It significantly reduces storage requirements compared to deep unrolling methods, using only one iteration block instead of multiple iterations. Unlike deep unrolling models, MsDC-DEQ-Net can be iteratively used, gradually improving reconstruction accuracy while considering computation tradeoffs. Additionally, the model benefits from multiscale dilated convolutions, further enhancing performance.

Supervised-unsupervised combined transformer for spectral compressive imaging reconstruction

To solve the low spatial and/or temporal resolution problem which the conventional hyperspectral cameras often suffer from, spectral compressive imaging systems (SCI) have attracted more attention recently. Recovering a hyperspectral image (HSI) from its corresponding 2D coded image is an ill-posed inverse problem, and learning accurate prior from HSI and 2D coded image is essential to solve this inverse problem. Existing methods only use supervised networks that focus on learning generalized prior from training datasets, or only use unsupervised networks that focus on learning specific prior from 2D coded image, resulting in the inability to learn both generalized and specific priors. Also, when learning the priors, existing methods cannot simultaneously give consideration to both global and local scales, as well as both spatial and spectral dimensions. To cope with this problem, in this paper, we propose a Supervised-Unsupervised Combined Transformer Network (SUCTNet) composed by a supervised Spatio-spectral Transformer network (SSTNet) and an Unsupervised Multi-level Feature Refinement network (UMFRNet). Specifically, we first develop the SSTNet to learn generalized prior and obtain a preliminary HSI. In SSTNet, the proposed spatial encoding and spectral decoding network architecture enables it to simultaneously consider both spatial and spectral dimensions, and a proposed Global and Local Multi head Self Attention block (GL-MSA) enables it simultaneously to consider both global and local scales. Then, the preliminary HSI is fed into the proposed UMFRNet to learn specific prior and obtain the target HSI. In UMFRNet, a proposed multi-level feature refinement mechanism and the physical imaging model of SCI are used to improve reconstruction accuracy and generalization performance. Extensive experiments show that our method significantly outperforms state-of-the-art (SOTA) methods on simulated and real datasets. Codes will be available at https://github.com/Vzhouhan/SUCTNet.

Particle image velocimetry combining unsupervised learning and optical flow model

Particle Image Velocimetry (PIV) is a multi-point, transient, contactless flow field measurement technique which can obtain accurate data information for the flow field. Particle image velocimetry technology based on deep learning has been widely used, with higher measurement accuracy and testing efficiency than traditional algorithms. Algorithms based on supervised learning need real data for network training. However, obtaining real flow field data is difficult, so unsupervised learning methods are more suitable for network training. The LiteFlowNet3 network has only four layers of flow inference, leading to limited reconstruction accuracy, especially for complex flow field. This paper adds one layer of flow inference based on the original network, while increasing the last layer of the loss. The network is trained using an unsupervised learning method, focusing on improving reconstruction accuracy for real-flow field training. The experimental results show that the improved model can reduce the average endpoint error by up to 30.89 % and 13.33 % compared to the original model and the unsupervised LiteFlowNet. In addition, by building a two-dimensional PIV system to simulate the advection and vortex field, real particle image pairs were obtained and put into the trained model for testing, proving the feasibility for obtaining velocity fields in real flow field.

Autonomous Internet of Things (IoT) Data Reduction Based on Adaptive Threshold

With the development of intelligent IoT applications, vast amounts of data are generated by various volume sensors. These sensor data need to be reduced at the sensor and then reconstructed later to save bandwidth and energy. As the reduced data increase, the reconstructed data become less accurate. Usually, the trade-off between reduction rate and reconstruction accuracy is controlled by the reduction threshold, which is calculated by experiments based on historical data. Considering the dynamic nature of IoT, a fixed threshold cannot balance the reduction rate with the reconstruction accuracy adaptively. Aiming to dynamically balance the reduction rate with the reconstruction accuracy, an autonomous IoT data reduction method based on an adaptive threshold is proposed. During data reduction, concept drift detection is performed to capture IoT dynamic changes and trigger threshold adjustment. During data reconstruction, a data trend is added to improve reconstruction accuracy. The effectiveness of the proposed method is demonstrated by comparing the proposed method with the basic Kalman filtering algorithm, LMS algorithm, and PIP algorithm on stationary and nonstationary datasets. Compared with not applying the adaptive threshold, on average, there is an 11.7% improvement in accuracy for the same reduction rate or a 17.3% improvement in reduction rate for the same accuracy.

Reconstruction of incomplete flow fields based on unsupervised learning

The loss of flow field data is a common challenge in experimental techniques and data transmission processes. Traditional methods often rely on another complete or high-fidelity dataset to compensate for missing information regarding fluid characteristics. However, obtaining such complete data is not always feasible. In this study, we propose an unsupervised learning approach that efficiently, swiftly, and cost-effectively reconstructs the complete flow field by leveraging the inherent properties of the flow field itself, without the need for high-fidelity data integration. Our proposed methods primarily encompass three techniques: Fourier basis function interpolation, the Autoregressive Integrated Moving Average model (ARIMA), and AMF (ARIMA-Median Filter). The first two methods are founded on the temporal evolution characteristics of the flow field and reconstruct the missing fluid information by extrapolating from the available data. The AMF method, building upon the ARIMA reconstruction outcomes, employs a median filtering approach to eliminate noise and outliers introduced by the lack of spatial correlation. All three methods perform well, with the AMF method exhibiting superior performance, particularly in cases involving scatter missing data with higher absence rates. While ARIMA and AMF entail longer computational times, they offer stability and are less prone to overfitting during the reconstruction process, rendering them more suitable for reconstructing flow fields with varying degrees of missing data. The Fourier basis function interpolation method, with its shorter runtime, is better suited for swiftly reconstructing flow fields with lower rates of missing data. However, it is worth noting that improper parameter selection for this method can lead to overfitting. Additionally, the non-stationarity of the flow field has an impact on reconstruction accuracy, with regions exhibiting stronger non-stationarity resulting in more noticeable reconstruction errors. For the ARIMA method, which relies on temporal correlation for reconstruction, the missing structures have a relatively minor effect. However, in the case of flow fields with block missing data, the improvement in reconstruction accuracy when using median filtering compared to ARIMA is not substantial.

TFF_aDCNN: A Pre-Trained Base Model for Intelligent Wideband Spectrum Sensing

In the beyond 5 G (B5G)/6 G era, to achieve ultra-dense and ultra-large-capacity intelligent connection of all things, an intelligent wideband spectrum sensing technology is particularly important. However, in an extremely wide frequency range, it is still a challenge to achieve high-precision and high-reconstruction-capability wideband spectrum sensing (WSS) under a very low SNR. We propose a Time-Frequency-Fused adjustable Deep Convolutional Neural Network (TFF_aDCNN). Meanwhile, a novel TFF_aDCNN-based sensing framework is also proposed. In this framework, we can obtain a pre-trained base model with a single distribution by training TFF_aDCNN. Then, for the sensing task in the actual environment, we use the base model for transfer learning, so that a newly trained sensing model can be obtained very quickly (i.e. fine-tuned model). In the TFF_aDCNN, we design a main network and an adjustable auxiliary network, where the former learns complex and abstract signal features, while the latter assists the main network in learning different data distribution patterns during the training process and regulates the focus direction of the main network during the perception process. Simulation results show that TFF_aDCNN can significantly reduce hardware cost and improve reconstruction accuracy and reconstruction capability, when compared with SOMP and SwSOMP-based WSS algorithms, single-dimensional deep learning spectrum sensing method, and deep learning-based WSS (DLWSS), especially at very low SNRs.

Fast deep learning reconstruction techniques for preclinical magnetic resonance fingerprinting.

We propose a deep learning (DL) model and a hyperparameter optimization strategy to reconstruct T1 and T2 maps acquired with the magnetic resonance fingerprinting (MRF) methodology. We applied two different MRF sequence routines to acquire images of ex vivo rat brain phantoms using a 7-T preclinical scanner. Subsequently, the DL model was trained using experimental data, completely excluding the use of any theoretical MRI signal simulator. The best combination of the DL parameters was implemented by an automatic hyperparameter optimization strategy, whose key aspect is to include all the parameters to the fit, allowing the simultaneous optimization of the neural network architecture, the structure of the DL model, and the supervised learning algorithm. By comparing the reconstruction performances of the DL technique with those achieved from the traditional dictionary-based method on an independent dataset, the DL approach was shown to reduce the mean percentage relative error by a factor of 3 for T1 and by a factor of 2 for T2 , and to improve the computational time by at least a factor of 37. Furthermore, the proposed DL method enables maintaining comparable reconstruction performance, even with a lower number of MRF images and a reduced k-space sampling percentage, with respect to the dictionary-based method. Our results suggest that the proposed DL methodology may offer an improvement in reconstruction accuracy, as well as speeding up MRF for preclinical, and in prospective clinical, investigations.

PhySR: Physics-informed deep super-resolution for spatiotemporal data

High-fidelity simulation of complex physical systems is exorbitantly expensive and inaccessible across spatiotemporal scales. Recently, there has been an increasing interest in leveraging deep learning to augment scientific data based on the coarse-grained simulations, which is of cheap computational expense and retains satisfactory solution accuracy. However, the major existing work focuses on data-driven approaches which rely on rich training datasets and lack sufficient physical constraints. To this end, we propose a novel and efficient spatiotemporal super-resolution framework via physics-informed learning (i.e., PhySR), inspired by the independence between temporal and spatial derivatives in partial differential equations (PDEs). The general principle is to leverage the temporal interpolation for flow estimation, and then introduce convolutional-recurrent neural networks for learning temporal refinement. Furthermore, we employ the stacked residual blocks with wide activation and sub-pixel layers with pixelshuffle for spatial reconstruction, where feature extraction is conducted in a low-resolution latent space. Moreover, we consider hard imposition of boundary conditions in the network to improve reconstruction accuracy. Results demonstrate the superior effectiveness and efficiency of the proposed method, which outperforms existing baseline algorithms, based on extensive numerical experiments.