Photoacoustic Image Reconstruction Research Articles

Comparing Deep Learning Frameworks for Photoacoustic Tomography Image Reconstruction

Conventional reconstruction methods for photoacoustic images are not suitable for the scenario of sparse sensing and geometrical limitation. To overcome these challenges and enhance the quality of reconstruction, several learning-based methods have recently been introduced for photoacoustic tomography reconstruction. The goal of this study is to compare and systematically evaluate the recently proposed learning-based methods and modified networks for photoacoustic image reconstruction. Specifically, learning-based post-processing methods and model-based learned iterative reconstruction methods are investigated. In addition to comparing the differences inherently brought by the models, we also study the impact of different inputs on the reconstruction effect. Our results demonstrate that the reconstruction performance mainly stems from the effective amount of information carried by the input. The inherent difference of the models based on the learning-based post-processing method does not provide a significant difference in photoacoustic image reconstruction. Furthermore, the results indicate that the model-based learned iterative reconstruction method outperforms all other learning-based post-processing methods in terms of generalizability and robustness.

Generalized spatial coherence reconstruction for photoacoustic computed tomography.

.Significance: Coherence, a fundamental property of waves and fields, plays a key role in photoacoustic image reconstruction. Previously, techniques such as short-lag spatial coherence (SLSC) and filtered delay, multiply, and sum (FDMAS) have utilized spatial coherence to improve the reconstructed resolution and contrast with respect to delay-and-sum (DAS). While SLSC uses spatial coherence directly as the imaging contrast, FDMAS employs spatial coherence implicitly. Despite being more robust against noise, both techniques have their own drawbacks: SLSC does not preserve a relative signal magnitude, and FDMAS shows a reduced contrast-to-noise ratio.Aim: To overcome these limitations, our aim is to develop a beamforming algorithm—generalized spatial coherence (GSC)—that unifies SLSC and FDMAS into a single equation and outperforms both beamformers.Approach: We demonstrated the application of GSC in photoacoustic computed tomography (PACT) through simulation and experiments and compared it to previous beamformers: DAS, FDMAS, and SLSC.Results: GSC outperforms the imaging metrics of previous state-of-the-art coherence-based beamformers in both simulation and experiments.Conclusions: GSC is an innovative reconstruction algorithm for PACT, which combines the strengths of FDMAS and SLSC expanding PACT’s applications.

Photoacoustic tomography reconstruction using lag-based delay multiply and sum with a coherence factor improves in vivo ovarian cancer diagnosis.

Ovarian cancer is the fifth most common cause of death due to cancer, and it is the deadliest of all gynecological cancers. Diagnosing ovarian cancer via conventional photoacoustic delay-and-sum beamforming (DAS) presents several challenges, such as poor image resolution and low lesion to background tissue contrast. To address these concerns, we propose an improved beamformer named lag-based delay multiply and sum combined with coherence factor (DMAS-LAG-CF). Simulations and phantom experiments demonstrate that compared with the conventional DAS, the proposed algorithm can provide 1.39 times better resolution and 10.77 dB higher contrast. For patient data, similar performance on contrast ratios has been observed. However, since the diagnostic accuracy between cancer and benign/normal groups is a significant measure, we have extracted photoacoustic histogram features of mean, kurtosis and skewness. DMAS-LAG-CF can improve cancer diagnosis with an AUC of 0.91 for distinguishing malignant vs. benign ovarian lesions when mean and skewness are used as features.

Spatiotemporal Coherence Weighting for In Vivo Cardiac Photoacoustic Image Beamformation.

Photoacoustic (PA) image reconstruction generally utilizes delay-and-sum (DAS) beamforming of received acoustic waves from tissue irradiated with optical illumination. However, nonadaptive DAS reconstructed cardiac PA images exhibit temporally varying noise which causes reduced myocardial PA signal specificity, making image interpretation difficult. Adaptive beamforming algorithms such as minimum variance (MV) with coherence factor (CF) weighting have been previously reported to improve the DAS image quality. In this article, we report on an adaptive beamforming algorithm by extending CF weighting to the temporal domain for preclinical cardiac PA imaging (PAI). The proposed spatiotemporal coherence factor (STCF) considers multiple temporally adjacent image acquisition events during beamforming and cancels out signals with low spatial coherence and temporal coherence, resulting in higher background noise cancellation while preserving the main features of interest (myocardial wall) in the resultant PA images. STCF has been validated using the numerical simulations and in vivo ECG and respiratory-signal-gated cardiac PAI in healthy murine hearts. The numerical simulation results demonstrate that STCF weighting outperforms DAS and MV beamforming with and without CF weighting under different levels of inherent contrast, acoustic attenuation, optical scattering, and signal-to-noise (SNR) of channel data. Performance improvement is attributed to higher sidelobe reduction (at least 5 dB) and SNR improvement (at least 10 dB). Improved myocardial signal specificity and higher signal rejection in the left ventricular chamber and acoustic gel region are observed with STCF in cardiac PAI.

Photo-acoustic tomographic image reconstruction from reduced data using physically inspired regularization

We propose a model-based image reconstruction method for photoacoustic tomography (PAT) involving a novel form of regularization and demonstrate its ability to recover good quality images from significantly reduced size datasets. The regularization is constructed to suit the physical structure of typical PAT images. We construct it by combining second-order derivatives and intensity into a non-convex form to exploit a structural property of PAT images that we observe: in PAT images, high intensities and high second-order derivatives are jointly sparse. The specific form of regularization constructed here is a modification of the form proposed for fluorescence image restoration. This regularization is combined with a data fidelity cost, and the required image is obtained as the minimizer of this cost. As this regularization is non-convex, the efficiency of the minimization method is crucial in obtaining artifact-free reconstructions. We develop a custom minimization method for efficiently handling this non-convex minimization problem. Further, as non-convex minimization requires a large number of iterations and the PAT forward model in the data-fidelity term has to be applied in the iterations, we propose a computational structure for efficient implementation of the forward model with reduced memory requirements. We evaluate the proposed method on both simulated and real measured data sets and compare them with a recent reconstruction method that is based on a well-known fast iterative shrinkage threshold algorithm (FISTA).

Non-local means improves total-variation constrained photoacoustic image reconstruction.

Photoacoustic/Optoacoustic tomography aims to reconstruct maps of the initial pressure rise induced by the absorption of light pulses in tissue. This reconstruction is an ill-conditioned and under-determined problem, when the data acquisition protocol involves limited detection positions. The aim of the work is to develop an inversion method which integrates denoising procedure within the iterative model-based reconstruction to improve quantitative performance of optoacoustic imaging. Among the model-based schemes, total-variation (TV) constrained reconstruction scheme is a popular approach. In this work, a two-step approach was proposed for improving the TV constrained optoacoustic inversion by adding a non-local means based filtering step within each TV iteration. Compared to TV-based reconstruction, inclusion of this non-local means step resulted in signal-to-noise ratio improvement of 2.5 dB in the reconstructed optoacoustic images.

3-D photoacoustic tomography reconstruction with deep learning for vasculature imaging

Photoacoustic tomography (PAT) is a hybrid imaging modality capable of acquiring high contrast and resolution images of optical absorption at depths greater than traditional optical imaging techniques. Practical considerations with instrumentation limit the number of acoustic sensors and their “view” of the imaging target, which result in image reconstruction artifacts degrading image quality. Iterative reconstruction methods can be used to improve image quality but are computationally expensive, especially for 3-D PAT over large imaging volumes. Deep learning has emerged as an efficient alternative and is capable of achieving state-of-the-art performance. In this work, we compare the 3-D fully dense UNet convolutional neural network with the widely used time reversal reconstruction method. Simulated acoustic data was generated using the K-wave MATLAB toolbox (225 detectors, planar geometry). A public database containing 50 patient's whole-lung CT scans was used to create training (n = 40) and testing (n = 10) data. The training and testing datasets are comprised of 500 and 50 lung vasculature volumes sampled from their respective whole-lung CT scans. Using the structural similarity index, the proposed deep learning method (0.87 ± 0.04) is shown to be superior to time reversal (0.38 ± 0.11).

Cross-sectional photoacoustic tomography image reconstruction with a multi-curve integration model

Background and objectiveIn acoustic inversion of photoacoustic tomography (PAT), an imaging model that precisely describes both the ultrasonic wave propagation and the detector properties is of crucial importance. Inspired by the multi-stripe integration model in clinical X-ray computed tomography systems, in this work, we introduce the Multi-Curve-Integration-based acoustic inversion for cross-sectional Photoacoustic Tomography (MCI-PAT). MethodsWe assumed that in cross-sectional PAT system, the three-dimensional (3-D) wave propagation problem could be reduced to a two-dimensional (2-D) problem in a limited, yet sufficient field of view. Under such condition, the MCI-PAT imaging model is generated by integrating several circular acoustic curves, the centers of which are points evenly distributed on the finite-size ultrasonic transducer surface. In this way, the spatial detector response is taken into account, while its computational burden does not largely increase because the integration process is performed only on a 2-D plane. ResultsAs proven by simulation, phantom and in vivo small animal experiments, the MCI-PAT method leads to promising improvement in PAT image quality. Comparing to traditional imaging models that considered only a single acoustic curve, our proposed method successfully improved the visibility of small structures and achieved evident enhancement on signal-to-noise ratio. ConclusionsThe performance of the MCI-PAT method demonstrates that for cross-sectional PAT, a 2-D simplification of the propagation of multiple photoacoustic waves is feasible. Due to its simplicity, our method can be used as an add-on to current system models considering only a single acoustic curve.

A New Deep Learning Network for Mitigating Limited-view and Under-sampling Artifacts in Ring-shaped Photoacoustic Tomography

Photoacoustic tomography (PAT) is a hybrid technique for high-resolution imaging of optical absorption in tissue. Among various transducer arrays proposed for PAT, the ring-shaped transducer array is widely used in cross-sectional imaging applications. However, due to the high fabrication cost, most ring-shaped transducer arrays have a sparse transducer arrangement, which leads to limited-view problems and under-sampling artifacts. To address these issues, we paired conventional PAT reconstruction with deep learning, which recently achieved a breakthrough in image processing and tomographic reconstruction. In this study, we designed a convolutional neural network (CNN) called a ring-array deep learning network (RADL-net), which can eliminate limited-view and under-sampling artifacts in PAT images. The method was validated on a three-quarter ring transducer array using numerical simulation, phantom imaging, and in vivo imaging. Our results indicate that the proposed RADL-net significantly improves the quality of reconstructed images on a three-quarter ring transducer array. The method is also superior to the conventional compressed sensing (CS) algorithm.

High-quality photoacoustic image reconstruction based on deep convolutional neural network: towards intra-operative photoacoustic imaging

The use of intra-operative imaging system as an intervention solution to provide more accurate localization of complicated structures has become a necessity during the neurosurgery. However, due to the limitations of conventional imaging systems, high-quality real-time intra-operative imaging remains as a challenging problem. Meanwhile, photoacoustic imaging has appeared so promising to provide images of crucial structures such as blood vessels and microvasculature of tumors. To achieve high-quality photoacoustic images of vessels regarding the artifacts caused by the incomplete data, we proposed an approach based on the combination of time-reversal (TR) and deep learning methods. The proposed method applies a TR method in the first layer of the network which is followed by the convolutional neural network with weights adjusted to a set of simulated training data for the other layers to estimate artifact-free photoacoustic images. It was evaluated using a generated synthetic database of vessels. The mean of signal to noise ratio (SNR), peak SNR, structural similarity index, and edge preservation index for the test data were reached 14.6 dB, 35.3 dB, 0.97 and 0.90, respectively. As our results proved, by using the lower number of detectors and consequently the lower data acquisition time, our approach outperforms the TR algorithm in all criteria in a computational time compatible with clinical use.

Photoacoustic imaging through a cortical bone replica with anisotropic elasticity

Photoacoustic (PA) imaging is an emerging modality, which combines the high optical absorption contrast of biological chromophores with centimeter imaging depths and sub-millimeter resolution of ultrasonic (US) waves. However, PA imaging through cortical bone remains an unmet challenge. Cortical bone is an anisotropic medium, which is not accurately modeled using existing PA image reconstruction methods. In this Letter, we address the PA source localization problem for imaging through a cortical bone-mimicking layer. Our approach accounts for both refraction and elastic anisotropy to accurately reconstruct US and PA images in the presence of a cortical bone replica. We demonstrate our technique using a PA and US experiment, where we image a 700 μm diameter target beneath a cortical bone-mimicking plate. Pulse-echo US experiment is used to estimate the wavespeed in each layer and create an anatomical image of the bone replica, and the PA source is reconstructed using the wavespeed model defined with US. We compute the thickness of the plate with less than 1% error, whereas isotropic assumptions overestimate the thickness by 20% or more. Incorporating both refraction and anisotropy accurately localizes the target with PA and US at the true depth, whereas isotropic assumptions blur the lateral dimension and mislocate the target depth by 1.5–4 mm.

Pre-migration: A General Extension for Photoacoustic Imaging Reconstruction

In recent years, photoacoustic image reconstruction has received extensive attention. Various reconstruction methods, such as back-projection, frequency domain reconstruction, time reversal, and model-based reconstructions have been developed. Although these methods are based on different propagation theories, they have relatively simple implementations when reconstructing images on homogenous media. However, in cases of heterogeneous layered media, like photoacoustic transcranial imaging, the propagation models have to be modified to accommodate various acoustic effects at layer interface, which complicates the reconstruction process. In this article, we propose an algorithm extension called pre-migration to first convert the reconstruction problems that apply to homogeneous media. From there, the sources can be rebuilt again using classical reconstruction methods. Pre-migration does not require classical reconstructions to be modified. It only preadjusts the sensor position via wave extrapolation. It also simplifies a focused transducer model as a point-like sensor. Based on simulation and experiment results, pre-migration can be integrated into almost all classical reconstruction algorithms and successfully solves reconstruction problems when imaging through heterogeneous media. In our experiments, it takes less than 20% of the total computational time during the entire reconstruction process.

Multiple Delay and Sum With Enveloping Beamforming Algorithm for Photoacoustic Imaging.

Delay and Sum (DAS) is one of the most common beamforming algorithms for photoacoustic imaging (PAI) reconstruction. Based on calculating beamformed signal with simple delaying and summing, DAS can function in a quick response and is quite suitable for real-time PAI. However, high sidelobes and intense artifacts may appear when using DAS due to summing with unnecessary data. In this paper, a beamforming algorithm called Multiple Delay and Sum with Enveloping (multi-DASE) is introduced to solve this problem. Compared to DAS, the multi-DASE algorithm calculates not only the initial value of the beamformed signal but also the complete N-shaped photoacoustic signal for each pixel. Through computer simulation, a phantom experiment and experiment on human finger joint, the multi-DASE algorithm is compared with other beamforming methods in removing artifacts by evaluating the quality of the reconstructed images. Furthermore, by rearranging the calculation sequences, the multi-DASE algorithm can be computing in parallel using GPU acceleration to meet the needs of real-time clinical application.

Image enhancement of photoacoustic imaging for early endometrial cancer detection by employing a filtered delay multiply and sum beamforming algorithm

In endometrial cancer, patients in early stages have a 91% 5-year survival chance. By contrast, patients in advanced stages have only 20% survival chance. Therefore, early diagnosis of endometrial cancer is very important. Photoacoustic imaging is able to distinguish benign from malignant tumors. However, the images acquired through photoacoustic imaging contain inherent artifacts, caused by imperfect reconstruction algorithms. In this paper, to improve the said images, a filtered delay-multiply-and-sum (F-DMAS) algorithm, which was proven to have an increased dynamic range and better quality of B-mode images was employed in the reconstruction process. First, the images of two blood vessel phantom experiments, acquired through photoacoustic imaging, were reconstructed by employing the F-DMAS algorithm. The results show the lateral resolutions of the system improving from 2.22 mm (with traditional photoacoustic imaging reconstruction algorithms) to 1.47 mm. Next, images of a pig uterus, filled with intralipid-20% emulsion, were also reconstructed by employing the F-DMAS algorithm. The average signal-to-noise ratio increased from 11.14 dB (with traditional photoacoustic imaging reconstruction algorithms) to 64.90 dB. In order to improve the continuity of the 3D PA image, F-DMAS with an adaptive coefficient was discussed to find the best balance between the signal-to-noise ratio and continuity. In conclusion, this paper demonstrates that the use of F-DMAS algorithms in the reconstruction of the images acquired through photoacoustic imaging in clinical investigations would improve the detection rate of early endometrial cancer.

Photoacoustic tomography with direction dependent data: an exact series reconstruction approach

Photoacoustic image reconstruction often assumes that the restriction of the acoustic pressure on the detection surface is given. However, commonly used detectors often have a certain directivity and frequency dependence, in which case the measured data are more accurately described as a linear combination of the acoustic pressure and its normal derivative on the detection surface. In this paper, we consider the inverse source problem for data that are a combination of the acoustic pressure and its normal derivative. For the special case of a spherical detection geometry we derive exact frequency domain reconstruction formulas. We present numerical results showing the robustness and validity of the derived formulas. Moreover, we compare several different combinations of the pressure and its normal derivative showing that used measurement model significantly affects the recovered initial pressure.

Rapid Three-Dimensional Photoacoustic Imaging Reconstruction for Irregularly Layered Heterogeneous Media.

Photoacoustic imaging (PAI) is susceptible to speed of sound (SOS) differences in heterogeneous media which greatly reduce the resolutions and qualities of the imaging results. Several reconstruction methods have been reported to adapt for heterogenous media, but they are limited by specific deficiencies such as efficiency, accuracy, and model limitation problems. Among them, the plane wave model based on wavefield reconstruction is the most efficient and promising one for high-efficiency three-dimensional PAI. However, the classic plane wave model only suits for planar layered media, severely limiting its applications in practice. To this end, we modify the plane wave model to apply for irregularly layered heterogeneous media and propose a corresponding wavefield extrapolation to reconstruct photoacoustic image. This method employs split-step Fourier to compensate the SOS differences, extrapolates wavefields and reconstructs the image depth by depth. Furthermore, a floating discretization strategy is introduced to control and balance the efficiency and accuracy with a hyperparameter. The simulation and experiment results demonstrate that the proposed method can reconstruct the image with an equivalent resolution to time reversal's and even have higher efficiency and robustness. To reconstruct a three-dimensional image with 50×50×600 pixels, the proposed method takes only 5.5 seconds using a laptop loaded with Intel(R) Core (TM) i7-8550U CPU @1.8GHz.

Sparsity-based photoacoustic image reconstruction with a linear array transducer and direct measurement of the forward model (Erratum)

.The erratum corrects an error in the article, “Sparsity-based photoacoustic image reconstruction with a linear array transducer and direct measurement of the forward model,” by R. Shang et al.

GPU-accelerated Double-stage Delay-multiply-and-sum Algorithm for Fast Photoacoustic Tomography Using LED Excitation and Linear Arrays.

Double-stage delay-multiply-and-sum (DS-DMAS) is an algorithm proposed for photoacoustic image reconstruction. The DS-DMAS algorithm offers a higher contrast than conventional delay-and-sum and delay-multiply and-sum but at the expense of higher computational complexity. Here, we utilized a compute unified device architecture (CUDA) graphics processing unit (GPU) parallel computation approach to address the high complexity of the DS-DMAS for photoacoustic image reconstruction generated from a commercial light-emitting diode (LED)-based photoacoustic scanner. In comparison with a single-threaded central processing unit (CPU), the GPU approach increased speeds by nearly 140-fold for 1024 × 1024 pixel image; there was no decrease in accuracy. The proposed implementation makes it possible to reconstruct photoacoustic images with frame rates of 250, 125, and 83.3 when the images are 64 × 64, 128 × 128, and 256 × 256, respectively. Thus, DS-DMAS can be efficiently used in clinical devices when coupled with CUDA GPU parallel computation.

Accelerated Photoacoustic Tomography Reconstruction via Recurrent Inference Machines.

Accelerated photoacoustic tomography (PAT) reconstruction is important for real-time photoacoustic imaging (PAI) applications. PAT requires a reconstruction algorithm to reconstruct the detected photoacoustic signal in order to obtain the detected image of the tissue, which is usually an inverse problem. Different from the typical method for solving the inverse problems that defines a model and chooses an inference procedure, we propose to use the Recurrent Inference Machines (RIM) as a framework for PAT reconstruction. Our model performs an accelerated iterative reconstruction, and directly learns to solve the inverse problem in PAT using the information from a forward model that is based on k-space methods. As shown in experiments, our method achieves faster high-resolution PAT reconstruction, and outperforms another method based on deep neural network in some respects.

Review on photoacoustic tomographic image reconstruction for acoustically heterogeneous tissues

Acoustic properties of biological tissues usually vary inhomogeneously in space. Tissues with different chemical composition often have different acoustic properties. The assumption of acoustic homogeneity may lead to blurred details, misalignment of targets and artifacts in the reconstructed photoacoustic tomography (PAT) images. This paper summarizes the main solutions to PAT imaging of acoustically heterogeneous tissues, including the variable sound speed and acoustic attenuation. The advantages and limits of the methods are discussed and the possible future development is prospected.