Network For Image Reconstruction Research Articles

Self-supervised Natural Image Reconstruction and Large-scale Semantic Classification from Brain Activity

Reconstructing natural images and decoding their semantic category from fMRI brain recordings is challenging. Acquiring sufficient pairs of images and their corresponding fMRI responses, which span the huge space of natural images, is prohibitive. We present a novel self-supervised approach that goes well beyond the scarce paired data, for achieving both: (i) state-of-the art fMRI-to-image reconstruction, and (ii) first-ever large-scale semantic classification from fMRI responses. By imposing cycle consistency between a pair of deep neural networks (from image-to-fMRI & from fMRI-to-image), we train our image reconstruction network on a large number of “unpaired” natural images (images without fMRI recordings) from many novel semantic categories. This enables to adapt our reconstruction network to a very rich semantic coverage without requiring any explicit semantic supervision. Specifically, we find that combining our self-supervised training with high-level perceptual losses, gives rise to new reconstruction & classification capabilities. In particular, this perceptual training enables to classify well fMRIs of never-before-seen semantic classes, without requiring any class labels during training. This gives rise to: (i) Unprecedented image-reconstruction from fMRI of never-before-seen images (evaluated by image metrics and human testing), and (ii) Large-scale semantic classification of categories that were never-before-seen during network training. Such large-scale (1000-way) semantic classification from fMRI recordings has never been demonstrated before. Finally, we provide evidence for the biological consistency of our learned model.

Proxy-Bridged Image Reconstruction Network for Anomaly Detection in Medical Images.

Anomaly detection in medical images refers to the identification of abnormal images with only normal images in the training set. Most existing methods solve this problem with a self-reconstruction framework, which tends to learn an identity mapping and reduces the sensitivity to anomalies. To mitigate this problem, in this paper, we propose a novel Proxy-bridged Image Reconstruction Network (ProxyAno) for anomaly detection in medical images. Specifically, we use an intermediate proxy to bridge the input image and the reconstructed image. We study different proxy types, and we find that the superpixel-image (SI) is the best one. We set all pixels' intensities within each superpixel as their average intensity, and denote this image as SI. The proposed ProxyAno consists of two modules, a Proxy Extraction Module and an Image Reconstruction Module. In the Proxy Extraction Module, a memory is introduced to memorize the feature correspondence for normal image to its corresponding SI, while the memorized correspondence does not apply to the abnormal images, which leads to the information loss for abnormal image and facilitates the anomaly detection. In the Image Reconstruction Module, we map an SI to its reconstructed image. Further, we crop a patch from the image and paste it on the normal SI to mimic the anomalies, and enforce the network to reconstruct the normal image even with the pseudo abnormal SI. In this way, our network enlarges the reconstruction error for anomalies. Extensive experiments on brain MR images, retinal OCT images and retinal fundus images verify the effectiveness of our method for both image-level and pixel-level anomaly detection.

A model-based deep network for limited-angle computed tomography image reconstruction

In this paper, we propose a variational model for the limited-angle computed tomography (CT) image reconstruction and convert it into an end-to-end deep network. We first use the penalty method to solve the model and divide it into three iterative subproblems, where the first subproblem completes the sinograms, the second refines the CT images, and the last merges the outputs of the first two subproblems. In each iteration, we use the convolutional neural networks (CNNs) to approximate the solutions of the first two subproblems and, thus, obtain an end-to-end deep network for the limited-angle CT image reconstruction. Our network tackles both the sinograms and the CT images, and can simultaneously suppress the artifacts caused by the incomplete data and recover fine structural information in the CT images. Experimental results show that our method outperforms the related methods for the limited-angle CT image reconstruction.

Cross-domain heterogeneous residual network for single image super-resolution

Single image super-resolution is an ill-posed problem, whose purpose is to acquire a high-resolution image from its degraded observation. Existing deep learning-based methods are compromised on their performance and speed due to the heavy design (i.e., huge model size) of networks. In this paper, we propose a novel high-performance cross-domain heterogeneous residual network for super-resolved image reconstruction. Our network models heterogeneous residuals between different feature layers by hierarchical residual learning. In outer residual learning, dual-domain enhancement modules extract the frequency-domain information to reinforce the space-domain features of network mapping. In middle residual learning, wide-activated residual-in-residual dense blocks are constructed by concatenating the outputs from previous blocks as the inputs into all subsequent blocks for better parameter efficacy. In inner residual learning, wide-activated residual attention blocks are introduced to capture direction- and location-aware feature maps. The proposed method was evaluated on four benchmark datasets, indicating that it can construct the high-quality super-resolved images and achieve the state-of-the-art performance. Code and pre-trained models are available at https://github.com/zhangyongqin/HRN.

Electrical resistance tomography image reconstruction based on one-dimensional multi-branch convolutional neural network combined with attention mechanism

Electrical resistance tomography (ERT) is an important branch of process tomography (PT), which has been developed for decades. Image reconstruction is a critical step in ERT, where the object of reconstruction is the conductivity distribution of the measured field. Traditional algorithms cannot accurately establish the mapping between the measured voltage and conductivity distribution. With the development of machine learning, the convolutional neural network (CNN) has become a new image reconstruction method. Specific results have been achieved in ERT image reconstruction using CNNs. This study proposes a one-dimensional multi-branch convolutional neural network (1D-MBCNN) for ERT image reconstruction, which could retain the 1D spatial structure of the measured voltage and adaptively and efficiently extract feature information. COMSOL software and the PyTorch framework are used to build the dataset and train the neural network model, respectively. The advantages of the multi-branch structure and the effectiveness of the attention mechanism in ERT image reconstruction are verified by RIE and CC. We also evaluated the practicality of this method in the ERT system. Based on the results of different experiments, the method proposed in this paper has good imaging accuracy, noise resistance, generalization ability, and robustness.

An optimal control framework for joint-channel parallel MRI reconstruction without coil sensitivities

GoalThis work aims at developing a novel calibration-free fast parallel MRI (pMRI) reconstruction method incorporate with discrete-time optimal control framework. The reconstruction model is designed to learn a regularization that combines channels and extracts features by leveraging the information sharing among channels of multi-coil images. We propose to recover both magnitude and phase information by taking advantage of structured convolutional networks in image and Fourier spaces. MethodsWe develop a novel variational model with a learnable objective function that integrates an adaptive multi-coil image combination operator and effective image regularization in the image and Fourier spaces. We cast the reconstruction network as a structured discrete-time optimal control system, resulting in an optimal control formulation of parameter training where the parameters of the objective function play the role of control variables. We demonstrate that the Lagrangian method for solving the control problem is equivalent to back-propagation, ensuring the local convergence of the training algorithm. ResultsWe conduct a large number of numerical experiments of the proposed method with comparisons to several state-of-the-art pMRI reconstruction networks on real pMRI datasets. The numerical results demonstrate the promising performance of the proposed method evidently. ConclusionThe proposed method provides a general deep network design and training framework for efficient joint-channel pMRI reconstruction. SignificanceBy learning multi-coil image combination operator and performing regularizations in both image domain and k-space domain, the proposed method achieves a highly efficient image reconstruction network for pMRI.

Super-resolution image reconstruction network based on dual-channel pixel attention mechanism

In the field of image processing, image super-resolution reconstruction technology based on deep learning is becoming more and more mature. Relying on stacking convolutional layers to expand the depth of the reconstruction network can improve the similarity of the reconstructed image, but as the depth of the convolutional layer continues increase, the learning rate of the reconstruction network will also become lower. In this paper, the dual-channel pixel attention mechanism PA combined with the ESPCN network model can not only retains the high reconstruction rate of ESPCN, but also improves the image reconstruction effect. The experimental results show that the network model proposed in this paper is better than SRCNN, FSRCNN, ESPCN for single image reconstruction, and the peak signal-to-noise ratio (PSNR) and structural similarity ratio (SSIM) are significantly improved.

Edge-Conditioned Feature Transform Network for Hyperspectral and Multispectral Image Fusion

Despite recent advances achieved by deep learning techniques in the fusion of low-spatial-resolution hyperspectral image (LR-HSI) and high-spatial-resolution multispectral image (HR-MSI), it remains a challenge to reconstruct the high-spatial-resolution HSI (HR-HSI) with more accurate spatial details and less spectral distortions, since the low-level structure information such as sharp edges tends to be weakened or lost as the network depth grows. To tackle this issue, we creatively propose an edge-conditioned feature transform network (EC-FTN) in this article, which is mainly composed of three parts, namely, feature extraction network (FEN), feature fusion and transformation network (FFTN), and image reconstruction network (IRN). First, two computationally efficient FENs with 3-D convolutions and reshaping layers are employed to extract the joint spectral-spatial features of input images. Then, the FFTN conditioned on the edge map prior can fuse and transform the features adaptively, in which a fusion node and several cascaded feature modulation modules (FMMs) equipped with feature-wise modulation layers are constructed. Specifically, the edge map is generated via transfer learning, i.e., by applying the Sobel operator to feature maps of the red-green-blue (RGB) version of HR-MSI resulting from the pretrained VGG16 model without extra training. Finally, the desired HR-HSI is recovered from the transformed features through IRN. Furthermore, we elaborately design a weighted combinatorial loss function consisting of mean absolute error, image gradient difference, and spectral angle terms to guide the training. Experiments on both ground-based and remotely sensed datasets demonstrate that our EC-FTN outperforms state-of-the-art methods in visual and quantitive evaluations, as well as in fine details reconstruction.

Conductivity Prediction and Image Reconstruction of Complex-Valued Multi-Frequency Electrical Capacitance Tomography Based on Deep Neural Network

Complex-valued multi-frequency electrical capacitance tomography (CVMF-ECT) is a contactless and radiation-free imaging method for both industrial and medical applications. It can measure the complex multi-frequency capacitance data, that is, the complex capacitance spectra, which contains more information than single-frequency measurement for conductive samples. In this article, to investigate the potential applications of CVMF-ECT, a new conductivity prediction network and an image reconstruction network that can effectively utilize the spectra data are proposed. The conductivity prediction network can predict the phantom background and the conductivity level of water phase to determine the linearization point for imaging, regardless of the conductivity/permittivity distribution of the phantom. If the linearization point is unavailable, the image reconstruction network can be used to reconstruct the distribution at different conductivity level and different background with the reference of air-filled sensor measurement. The feasibility of these two networks has been proved by numerical simulations and practical experiments. This article provides a new perspective to determine the distribution and conductivity simultaneously without contact and shows the potential of the complex multi-frequency capacitance data.

Hybrid Convolutional Neural Network (hCNN) for Image Reconstruction in Near-Infrared Optical Tomography.

Near-infrared optical tomography (NIROT), a promising imaging modality for early detection of oxygenation in the brain of preterm infants, requires data acquisition at the tissue surface and thus an image reconstruction adaptable to cephalometric variations and surface topologies. Widely used model-based reconstruction methods come with the drawback of huge computational cost. Neural networks move this computational load to an offline training phase, allowing much faster reconstruction. Our aim is a data-driven volumetric image reconstruction that generalises well to different surfaces, increases reconstruction speed, localisation accuracy and image quality. We propose a hybrid convolutional neural network (hCNN) based on the well-known V-net architecture to learn inclusion localisation and absorption coefficients of heterogenous arbitrary shapes via a joint cost function. We achieved an average reconstruction time of 30.45s, a time reduction of 89% and inclusion detection with an average Dice score of 0.61. The CNN is flexible to surface topologies and compares well in quantitative metrics with the traditional model-based (MB) approach and state-of-the-art neuronal networks for NIROT. The proposed hCNN was successfully trained, validated and tested on in-silico data, excels MB methods in localisation accuracy and provides a remarkable increase in reconstruction speed.

Conditional Generative Adversarial Networks for 2D core grayscale image reconstruction from pore parameters

Digital cores are of great significance for reservoir structure simulation, oil and gas exploration and development. Most existing digital core reconstruction methods only generate binary cores with complicated implementation processes, among other problems. To address these problems, this study proposed a combination of core pore parameters and conditional generative adversarial network (CGAN) to realize the 2D reconstruction of core grayscale images from only pore parameters (namely, text-to-image synthesis). The current text-to-image synthesis approaches still have many difficulties in generating fine images, but the technologies of image-to-image generation have improved drastically in recent years. Therefore, the proposed method involves two stages to avoid the difficulty of directly generating core grayscale images from pore parameters. In stage I, we preprocessed core sample images to obtain binary-grayscale image pairs, and then used the CGAN to learn the mapping from core binary images to real sample images. At the same time, the pores in the binary images were segmented and extracted to construct the pore component library. In stage II, on the basis of the given pore parameters, the corresponding pores were randomly extracted from the pore component library to generate binary images, and then the generated binary images were used as input for the trained CGAN model to produce core grayscale images. The experimental results showed that the core grayscale images reconstructed by the proposed method meet the pore conditions and reflect the basic characteristics of real cores. • The reconstruction by our method is based on pore parameters rather than images. • The reconstruction is decomposed into two stages: “pore parameters to binary core” and “binary core to grayscale core”. • The method has a general validity and can reconstruct different types of cores according to the research requirements.

Spatial and Temporal Regularized Iterative Deep Neural Network for Non-Stationary Image Reconstruction of Electrical Impedance Tomography

Electrical Impedance Tomography (EIT) is a promising technique for monitoring complex dynamics. However, due to the ill-posedness and non-linearity of EIT, its spatial resolution is low, especially when the target dynamics are too fast to be observed under the ’stationary’ condition. The traditional ’non-stationary’ image reconstruction methods for EIT, like Kalman filter, suffers from low spatial resolution and high computational complexity. To achieve fast and accurate ’non-stationary’ image reconstruction of EIT, a spatial-temporal regularized learned gradient descent (STLGD) algorithm is proposed. A novel iterative neural network is constructed for numerically implementing the proposed algorithm. A two-dimensional stochastic data interpolation method is proposed for generating a large scale ’non-stationary’ dataset using ’stationary’ experiments, which is crucial from the numerical implementation of the learning based methods. A series of experimental tests with a water tank model are conducted. Ablation and augmentation tests are conducted to prove the optimization of the proposed network. The comparison between the proposed method and the baseline methods shows that the STLGD can automatically learn the spatial-temporal correlations among the targets from the training dataset, and achieve the fast and accurate conductivity image reconstructions which are much beyond the traditional methods.

High-Fidelity Shape Reconstruction for Electrical Impedance Tomography of Multi-Phase Conductivity via Deep Discrete Representation

As a noninvasive imaging technique, electrical impedance tomography (EIT) owns many potential industrial and biomedical applications. However, due to the nonlinearity and ill-posedness of its inverse problem, EIT suffers from low spatial resolution. To achieve the high-fidelity shape reconstruction of multiphase conductivity, a novel deep neural network is proposed. The network is constructed in the manifold-learning framework and implemented in a three-stage strategy. In the first stage, a deep discrete representation of the target conductivity image is learned from the training image set. In the second stage, the inverse operator, mapping from the boundary voltage data to the latent variables, is learned from the training voltage-to-latent-variables data pairs. In the third stage, the learned inverse operator is connected with the deep discrete decoders to infer the target conductivity image from the given voltage data. To improve the conductivity distinguishability of the proposed image reconstruction network, a conductivity image normalization approach is proposed according to the nonlinear response characteristic of EIT voltage data to the target conductivity changes. The proposed method is evaluated with a series of numerical and experimental tests, and is compared with several baseline methods. The results proved that the proposed network can provide high-fidelity reconstructions of multiphase conductivity distributions, achieving <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.05~\pm ~0.04$ </tex-math></inline-formula> area error (AE), <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.13~\pm ~0.08$ </tex-math></inline-formula> ringing (RNG) factor, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.30~\pm ~0.10$ </tex-math></inline-formula> relative root mean square error (RRMSE), and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.96~\pm ~0.03$ </tex-math></inline-formula> structural similarity (SSIM). Ablation experiments proved that the proposed three-stage manifold-learning strategy can help our network to avoid blurry reconstructions and achieve stabilized results, and also proved the efficiency of the proposed conductivity image normalization approach.

Single-Pass Object-Adaptive Data Undersampling and Reconstruction for MRI

There is much recent interest in techniques to accelerate the data acquisition process in MRI by acquiring limited measurements. Often sophisticated reconstruction algorithms are deployed to maintain high image quality in such settings. In this work, we propose a data-driven sampler using a convolutional neural network, MNet, to provide object-specific sampling patterns adaptive to each scanned object. The network observes very limited low-frequency k-space data for each object and rapidly predicts the desired undersampling pattern in one go that achieves high image reconstruction quality. We propose an accompanying alternating-type training framework with a mask-backward procedure that efficiently generates training labels for the sampler network and jointly trains an image reconstruction network. Experimental results on the fastMRI knee dataset demonstrate the ability of the proposed learned undersampling network to generate object-specific masks at fourfold and eightfold acceleration that achieve superior image reconstruction performance than several existing schemes. The source code for the proposed joint sampling and reconstruction learning framework is available at https://github.com/zhishenhuang/mri.

Adversarial example defense based on image reconstruction.

The rapid development of deep neural networks (DNN) has promoted the widespread application of image recognition, natural language processing, and autonomous driving. However, DNN is vulnerable to adversarial examples, such as an input sample with imperceptible perturbation which can easily invalidate the DNN and even deliberately modify the classification results. Therefore, this article proposes a preprocessing defense framework based on image compression reconstruction to achieve adversarial example defense. Firstly, the defense framework performs pixel depth compression on the input image based on the sensitivity of the adversarial example to eliminate adversarial perturbations. Secondly, we use the super-resolution image reconstruction network to restore the image quality and then map the adversarial example to the clean image. Therefore, there is no need to modify the network structure of the classifier model, and it can be easily combined with other defense methods. Finally, we evaluate the algorithm with MNIST, Fashion-MNIST, and CIFAR-10 datasets; the experimental results show that our approach outperforms current techniques in the task of defending against adversarial example attacks.

Deep Learning Reconstruction Enables Highly Accelerated Biparametric MR Imaging of the Prostate.

Early diagnosis and treatment of prostate cancer (PCa) can be curative; however, prostate-specific antigen is a suboptimal screening test for clinically significant PCa. While prostate magnetic resonance imaging (MRI) has demonstrated value for the diagnosis of PCa, the acquisition time is too long for a first-line screening modality. To accelerate prostate MRI exams, utilizing a variational network (VN) for image reconstruction. Retrospective. One hundred and thirteen subjects (train/val/test: 70/13/30) undergoing prostate MRI. 3.0 T; a T2 turbo spin echo (TSE) T2-weighted image (T2WI) sequence in axial and coronal planes, and axial echo-planar diffusion-weighted imaging (DWI). Four abdominal radiologists evaluated the image quality of VN reconstructions of retrospectively under-sampled biparametric MRIs (bp-MRI), and standard bp-MRI reconstructions for 20 test subjects (studies). The studies included axial and coronal T2WI, DWI B50 seconds/mm2 and B1000 seconds/mm (4-fold T2WI, 3-fold DWI), all of which were evaluated separately for image quality on a Likert scale (1: non-diagnostic to 5: excellent quality). In another 10 test subjects, three readers graded lesions on bp-MRI-which additionally included calculated B1500 seconds/mm2 , and apparent diffusion coefficient map-according to the Prostate Imaging Reporting and Data System (PI-RADS v2.1), for both VN and standard reconstructions. Accuracy of PI-RADS ≥3 for clinically significant cancer was computed. Projected scan time of the retrospectively under-sampled biparametric exam was also computed. One-sided Wilcoxon signed-rank test was used for comparison of image quality. Sensitivity, specificity, positive predictive value, and negative predictive value were calculated for lesion detection and grading. Generalized estimating equation with cluster effect was used to compare differences between standard and VN bp-MRI. A P-value of <0.05 was considered statistically significant. Three of four readers rated no significant difference for overall quality between the standard and VN axial T2WI (Reader 1: 4.00 ± 0.56 (Standard), 3.90 ± 0.64 (VN) P=0.33; Reader 2: 4.35 ± 0.74 (Standard), 3.80 ± 0.89 (VN) P=0.003; Reader 3: 4.60 ± 0.50 (Standard), 4.55 ± 0.60 (VN) P=0.39; Reader 4: 3.65 ± 0.99 (Standard), 3.60 ± 1.00 (VN) P=0.38). All four readers rated no significant difference for overall quality between standard and VN DWI B1000 seconds/mm2 (Reader 1: 2.25 ± 0.62 (Standard), 2.45 ± 0.75 (VN) P=0.96; Reader 2: 3.60 ± 0.92 (Standard), 3.55 ± 0.82 (VN) P=0.40; Reader 3: 3.85 ± 0.72 (Standard), 3.55 ± 0.89 (VN) P=0.07; Reader 4: 4.70 ± 0.76 (Standard); 4.60 ± 0.73 (VN) P=0.17) and three of four readers rated no significant difference for overall quality between standard and VN DWI B50 seconds/mm2 (Reader 1: 3.20 ± 0.70 (Standard), 3.40 ± 0.75 (VN) P=0.98; Reader 2: 2.85 ± 0.81 (Standard), 3.00 ± 0.79 (VN) P=0.93; Reader 3: 4.45 ± 0.72 (Standard), 4.05 ± 0.69 (VN) P=0.02; Reader 4: 4.50 ± 0.69 (Standard), 4.45 ± 0.76 (VN) P=0.50). In the lesion evaluation study, there was no significant difference in the number of PI-RADS ≥3 lesions identified on standard vs. VN bp-MRI (P=0.92, 0.59, 0.87) with similar sensitivity and specificity for clinically significant cancer. The average scan time of the standard clinical biparametric exam was 11.8minutes, and this was projected to be 3.2minutes for the accelerated exam. Diagnostic accelerated biparametric prostate MRI exams can be performed using deep learning methods in <4 minutes, potentially enabling rapid screening prostate MRI. 3 TECHNICAL EFFICACY: Stage 5.

Attention augmented residual network for tomato disease detection and classification

Deep learning techniques help agronomists efficiently identify, analyze, and monitor tomato health. CNN (convolutional neural network) locality constraint and existing small train sample adversely influenced disease recognition performance. To alleviate these challenges, we proposed a discriminative feature learning attention augmented residual (AAR) network. The AAR network contains a stacked pre-activated residual block that learns deep coarse level features with locality context, whereas the attention block captures salient feature sets while maintaining the global relationship in data points, attention features augment the learning of the residual block. We used conditional variational generative adversarial network (CVGAN) image reconstruction network and augmentation techniques to enlarge the training sample size and improve feature distribution. We conducted several experiments to demonstrate the AAR network performance.The AAR network performed 97.04% accuracy without data generation and augmentation, 98.91% with data generation and augmentation, and 99.03% trained with data augmentation, which consistently improved tomato disease recognition and visualization effectiveness in both cases by learning salient features than deep and wide CNN baseline networks and other related works. Therefore, the AAR network can be a good candidate for improved tomato disease detection and classification task.

MB-DAMPNet: a novel multi-branch denoising-based approximate message passing algorithm via deep neural network for image reconstruction

Compressive sensing has gained great attention due to its effectiveness in solving linear inverse problems. However, how to further improve the accuracy of compressed image inversion while ensuring or even accelerating the speed is still a major challenge. To tackle this problem, we present a novel multi-branch denoising-based approximate message passing algorithm via deep neural network, dubbed MB-DAMPNet. It mainly consists of three components, i.e. sampling subnet, initial recovery subnet, and deep recovery subnet, which are optimized jointly. The sampling subnet is constructed to obtain the compressed measurements, the initial recovery subnet is employed to generate the reconstructed image by inverse transformation, while the deep recovery subnet is designed to refine the reconstructed results obtained by the former, so as to improve the image accuracy. Moreover, the matrix multiplication in the network is all designed as matrix convolution which can be learned automatically, so that the input image of the MB-DAMPNet can be of different scales, which improves the flexibility and applicability of the network. In addition, all parameters in the network are learned end-to-end instead of fixed or hand-crafted. The numerical results validate that our method significantly outperforms other state-of-the-art methods in image reconstruction accuracy.

Domain knowledge augmentation of parallel MR image reconstruction using deep learning.

A deep learning (DL) method for accelerated magnetic resonance (MR) imaging is presented that incorporates domain knowledge of parallel MR imaging to augment the DL networks for accurate and stable image reconstruction. The proposed DL method employs a novel loss function consisting of a combination of mean absolute error, structural similarity, and sobel edge loss. The DL model takes both original measurements and images reconstructed by the parallel imaging method as inputs to the network. The accuracy of the proposed method was evaluated using two anatomical regions and six MRI contrasts and was compared with state-of-the-art parallel imaging and deep learning methods. The proposed method significantly outperformed the other methods for all the six different contrasts in terms of structural similarity, peak signal to noise ratio, and normalized mean squared error. The out-of-sample performance of the proposed method was assessed for a truly "unseen" case in a volunteer scan. The method produced images without any artificial features, often occurring in the DL image reconstruction methods. A stability analysis was performed by adding perturbations to the input, which demonstrated that the proposed method is robust and stable with respect to small structural changes, and different undersampling ratios. Comprehensive validation on large datasets demonstrated that incorporation of domain knowledge sufficiently regularizes the DL based image reconstruction and produces accurate and stable image enhancement.

Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data

Optical coherence tomography (OCT) is a widely used non-invasive biomedical imaging modality that can rapidly provide volumetric images of samples. Here, we present a deep learning-based image reconstruction framework that can generate swept-source OCT (SS-OCT) images using undersampled spectral data, without any spatial aliasing artifacts. This neural network-based image reconstruction does not require any hardware changes to the optical setup and can be easily integrated with existing swept-source or spectral-domain OCT systems to reduce the amount of raw spectral data to be acquired. To show the efficacy of this framework, we trained and blindly tested a deep neural network using mouse embryo samples imaged by an SS-OCT system. Using 2-fold undersampled spectral data (i.e., 640 spectral points per A-line), the trained neural network can blindly reconstruct 512 A-lines in 0.59 ms using multiple graphics-processing units (GPUs), removing spatial aliasing artifacts due to spectral undersampling, also presenting a very good match to the images of the same samples, reconstructed using the full spectral OCT data (i.e., 1280 spectral points per A-line). We also successfully demonstrate that this framework can be further extended to process 3× undersampled spectral data per A-line, with some performance degradation in the reconstructed image quality compared to 2× spectral undersampling. Furthermore, an A-line-optimized undersampling method is presented by jointly optimizing the spectral sampling locations and the corresponding image reconstruction network, which improved the overall imaging performance using less spectral data points per A-line compared to 2× or 3× spectral undersampling results. This deep learning-enabled image reconstruction approach can be broadly used in various forms of spectral-domain OCT systems, helping to increase their imaging speed without sacrificing image resolution and signal-to-noise ratio.