Real-time Imaging Applications Research Articles

An adaptive parameter decoupling algorithm-based image reconstruction model (ADAIR) for rapid golden-angle radial DCE-MRI

Objective. The acceleration of magnetic resonance imaging (MRI) acquisition is crucial for both clinical and research applications, particularly in dynamic MRI. Existing compressed sensing (CS) methods, despite being effective for fast imaging, face limitations such as the need for incoherent sampling and residual noise, which restrict their practical use for rapid MRI.Approach. To overcome these challenges, we propose a novel image reconstruction framework that integrates the MRI physical model with a flexible, self-adjusting, decoupling data-driven model. We validated this method through experiments using both simulated andin vivodynamic contrast-enhanced MRI datasets.Main results. The experimental results demonstrate that the proposed framework achieves high spatial and temporal resolution reconstructions. Additionally, when compared to state-of-the-art image reconstruction approaches, our method significantly enhances acceleration capabilities, enabling sparse and rapid imaging with high resolution.Significance. Our proposed framework offers a promising solution for real-time imaging and image-guided radiation therapy applications by providing superior performance in achieving high spatial and temporal resolution reconstructions, thus addressing the limitations of existing CS schemes.

Integrating Deep Learning and Homomorphic Encryption for Secure Image Transmission

This paper introduces a novel approach to securing medical image transmission through the integration of deep learning techniques into cryptographic processes. Leveraging the capabilities of Backpropagation (BP), Convolutional Neural Networks (CNN), Residual Networks (ResNet), and Generative Adversarial Network (GAN), our method aims to enhance the privacy and security of medical images in real-time applications like telemedicine. The proposed system focuses on optimizing performance metrics including Peak Signal-to-Noise Ratio (PSNR), Root Mean Square Error (RMSE), Structural Similarity Index Measure (SSIM), Mean Average Precision (MAP), and encryption speed. Through experimental evaluation, our approach demonstrates promising results in terms of encryption efficiency and preservation of image quality. By addressing the critical need for secure transmission methods in healthcare, this research contributes to advancing the field of medical image cryptography and lays the groundwork for further exploration in deep learning-based security solutions for healthcare data.

Parallel illumination for depletion microscopy through acousto-optic spatial light modulation

Several types of super-resolution microscopy techniques, such as Stimulated Emission Depletion (STED), Reversible Saturable Optical Fluorescence Transitions (RESOLFT) or Switching Laser Mode (SLAM) microscopies, employ Laguerre-Gaussian beams (also called vortex or doughnut beams) to obtain fluorescence information within a sub-wavelength region of the specimen under observation, thus breaking the diffraction limit and producing images of greatly improved quality. However, in general, these techniques operate on a point-by-point basis, so scanning the sample in order to build a full, meaningful image, takes time. Parallelization of the illumination is the only way to make these microscopy techniques more suitable for real-time live cell imaging applications. Here, we demonstrate the parallel production of arbitrary arrays of Gaussian and Laguerre-Gaussian lasers foci suitable for super-resolution microscopy, together with the possibility to fast scan through the sample, by means of acousto-optic spatial light modulation, a technique that we have pioneered in the past in several other fields. Parallelized illumination with both Gaussian and doughnut beams will be then used to acquire super-resolution images.

Lightweight super-resolution multimode fiber imaging with regularized linear regression

Super-resolution multimode fiber imaging provides the means to image samples quickly with compact and flexible setups finding many applications from biology and medicine to material science and nanolithography. Typically, fiber-based imaging systems suffer from low spatial resolution and long measurement times. State-of-the-art computational approaches can achieve fast super-resolution imaging through a multimode fiber probe but currently rely on either per-sample optimised priors or large data sets with subsequent long training and image reconstruction times. This unfortunately hinders any real-time imaging applications. Here we present an ultimately fast non-iterative algorithm for compressive image reconstruction through a multimode fiber. The proposed approach helps to avoid many constraints by determining the prior of the target distribution from a simulated set and solving the under-determined inverse matrix problem with a mathematical closed-form solution. We have demonstrated theoretical and experimental evidence for enhanced image quality and sub-diffraction spatial resolution of the multimode fiber optical system.

Approximate bilateral filters for real-time and low-energy imaging applications on FPGAs

Bilateral filtering is an image processing technique commonly adopted as intermediate step of several computer vision tasks. Opposite to the conventional image filtering, which is based on convolving the input pixels with a static kernel, the bilateral filtering computes its weights on the fly according to the current pixel values and some tuning parameters. Such additional elaborations involve nonlinear weighted averaging operations, which make difficult the deployment of bilateral filtering within existing vision technologies based on real-time and low-energy hardware architectures. This paper presents a new approximation strategy that aims to improve the energy efficiency of circuits implementing the bilateral filtering function, while preserving their real-time performances and elaboration accuracy. In contrast to the state-of-the-art, the proposed technique allows the filtering action to be on the fly adapted to both the current pixel values and to the tuning parameters, thus avoiding any architectural modification or tables update. When hardware implemented within the Xilinx Zynq XC7Z020 FPGA device, a 5 × 5 filter based on the proposed method processes 237.6 Mega pixels per second and consumes just 0.92 nJ per pixel, thus improving the energy efficiency by up to 2.8 times over the competitors. The impact of the proposed approximation on three different imaging applications has been also evaluated. Experiments demonstrate reasonable accuracy penalties over the accurate counterparts.

Direct Conjugation of Gallium-(III)-Corroles to Short Interfering RNA(siRNA) Providing Real-Time siRNA Imaging and Gene Silencing.

Discovering new modifications for oligonucleotide therapeutics is essential for expanding its application to new targets and diseases. In this project, we focus on conjugating metaled ligands to short interfering RNAs (siRNAs) to investigate robust and simple conjugation methods for adding new properties such as real-time imaging to the siRNA. Here we report the chemical synthesis of novel Ga-(III)-corroles for their direct conjugation to siRNAs. Ga-(III)-corrole-siRNAs showed promising results when evaluated for gene silencing and live cell imaging. The knockdown activity of the firefly luciferase reporter gene was measured to evaluate gene silencing activity. Gene silencing studies from two 5'-Ga-(III)-labeled-siRNAs exhibited dose-dependent knockdown with IC50s of 812.7 and 451.4 pM, which is comparable to wild-type (IC50=439.7 pM) in the absence of red light, and IC50s of 562.9 and 354.5 pM, which is also comparable to wild-type (IC50=337.4 pM), in the presence of red light. In addition, imaging studies with Ga-(III)-corrole-modified siRNAs showed intense fluorescence in HeLa cells, highlighting that the Ga-(III)-corrole modification is an effective fluorophore for siRNA tracing and imaging. Moreover, the photodynamic activity of free base corrole vs the Ga-(III)-corrole was evaluated. Results show an increase of light cytotoxicity of the corrole ligand upon the addition of Ga-(III); however, no phototoxicity was observed when Ga-(III) ligands were linked to siRNA. In conclusion, Ga-(III)-corrole-siRNAs show promising results for applications in simultaneous real-time imaging and gene silencing.

A high-performance terahertz detector based on a graded channel GaN high-electron-mobility transistor with a recessed gate

A novel recessed gate graded AlGaN high-electron-mobility transistor (RGGA HEMT) for high-sensitivity terahertz (THz) detectors is reported, which exhibits significant potential for high-sensitivity THz detectors, and consequently. The simulation results indicate the exceptional field-effect factor of the proposed RGGA HEMT, which is largely attributed to the three-dimensional electron gas generated by the graded AlGaN barrier layer and the subsequent enhancement of channel conductance. The responsivity and noise equivalent power reach 58mA/W and 43pW/Hz respectively at 227GHz by utilizing a technical computer-aided design platform. As compared to conventional structure GaN HEMT experimental data, the current responsivity is improved by 2.64 times, and the NEP is reduced by 43.4%. The unprecedented improvement in detection performance renders the RGGA HEMT a highly promising approach for future THz real-time single-pixel imaging applications at room temperature.

Thermal imaging study of Sm2+ doped SrB4O7 based on time-resolved technology.

Thermal imaging materials with high sensitivity and the ability to reflect real-time temperature play an important role in research areas such as biotechnology and electronic engineering. However, the temperature sensitivity and temporal resolution of the current materials are not suitable for the complicated detection situation. In this paper, we introduce a thermal imaging material - SrB4O7:5%Sm2+ - with high temperature sensitivity. Furthermore, by applying a time resolving technique based on an intensified charge-coupled device, the sensitivity and temporal resolution are greatly promoted. The good temperature sensitivity (9.67% K-1 at 533 K), the high spatial resolution (2.7 μm) and the fast detection time (<1 s) suggest its considerable potential for real-time thermal imaging applications. The results of temperature distribution on a printed circuit board show that the as-prepared material will be greatly beneficial for thermal imaging applications.

An adaptive multi-frame parallel iterative method for accelerating real-time magnetic particle imaging reconstruction

Objective. Real-time reconstruction of magnetic particle imaging (MPI) shows promising clinical applications. However, prevalent reconstruction methods are mainly based on serial iteration, which causes large delay in real-time reconstruction. In order to achieve lower latency in real-time MPI reconstruction, we propose a parallel method for accelerating the speed of reconstruction methods. Approach. The proposed method, named adaptive multi-frame parallel iterative method (AMPIM), enables the processing of multi-frame signals to multi-frame MPI images in parallel. To facilitate parallel computing, we further propose an acceleration strategy for parallel computation to improve the computational efficiency of our AMPIM. Main results. OpenMPIData was used to evaluate our AMPIM, and the results show that our AMPIM improves the reconstruction frame rate per second of real-time MPI reconstruction by two orders of magnitude compared to prevalent iterative algorithms including the Kaczmarz algorithm, the conjugate gradient normal residual algorithm, and the alternating direction method of multipliers algorithm. The reconstructed image using AMPIM has high contrast-to-noise with reducing artifacts. Significance. The AMPIM can parallelly optimize least squares problems with multiple right-hand sides by exploiting the dimension of the right-hand side. AMPIM has great potential for application in real-time MPI imaging with high imaging frame rate.

Single-frame reconstruction by fractional Fourier-transform domain filtering in off-axis digital holographic microscopy

We propose a single-frame zero-order-eliminated reconstruction method by fractional Fourier transform filtering for an off-axis digital hologram. The filtering in the fractional Fourier transform domain of the hologram can effectively improve the reconstruction resolution, but it is required to remove its zero-order term. With the zero-order-term elimination of the Laplacian hologram, the higher reconstruction resolution of a single-frame hologram is achieved by zero-padding the hologram and choosing the optimal option of the fractional-order number. The results demonstrate that the resolutions of reconstructed amplitude and phase images are obviously improved. It will have a promising application in real-time imaging for biological cells and moving objects.

A Red-Emitting Fluorescence Probe for Rapid Detecting Exogenous and Endogenous Peroxynitrite in Living Cells with High Sensitivity and Selectivity

Peroxynitrite (ONOO−) has been revealed to play crucial roles in many physiological and pathological processes, and many diseases were proven to be associated with its misregulated production. The development of fluorescent probes meets the need for tracking ONOO− and gives a better understanding of its diverse mechanisms. In this work, a red-emitting fluorescent probe BP-ONOO was synthesized via functionalization of the rhodol-like fluorophore with a reactive site of hydrazide. The probe BP-ONOO exhibited high sensitivity, excellent selectivity, and short response time (less than 4 s) towards ONOO− under neutral or weak alkaline conditions. These attractive properties favor its application in real-time imaging of ONOO− in living cells, and the probe has been successfully applied for imaging the concentration levels of ONOO− in RAW 264.7 macrophage cells under drug stimulation.

Advancements in waveguide architectures using high-performance silica-on-silicon platform - INVITED

Novel applications in optical coherence tomography (OCT) and LiDAR systems have become possible due to performance characteristics of a state-of-the-art silica-on-silicon planar lightwave circuit (PLC) platform. We have achieved ultra-low propagation losses of &lt;0.009 dB/cm with unmatched phase control in a polarization-insensitive way, enabling a range of real-time advanced vision and imaging applications utilizing k-clocks and analog frequency sampling architectures.

Dual Image Cryptosystem Using Henon Map and Discrete Fourier Transform

This paper introduces an efficient image cryptography system. The proposed image cryptography system is based on employing the two-dimensional (2D) chaotic henon map (CHM) in the Discrete Fourier Transform (DFT). The proposed DFT-based CHM image cryptography has two procedures which are the encryption and decryption procedures. In the proposed DFT-based CHM image cryptography, the confusion is employed using the CHM while the diffusion is realized using the DFT. So, the proposed DFT-based CHM image cryptography achieves both confusion and diffusion characteristics. The encryption procedure starts by applying the DFT on the image then the DFT transformed image is scrambled using the CHM and the inverse DFT is applied to get the finally encrypted image. The decryption procedure follows the inverse procedure of encryption. The proposed DFT-based CHM image cryptography system is examined using a set of security tests like statistical tests, entropy tests, differential tests, and sensitivity tests. The obtained results confirm and ensure the superiority of the proposed DFT-based CHM image cryptography system. These outcomes encourage the employment of the proposed DFT-based CHM image cryptography system in real-time image and video applications.

Solution-Processed Hybrid Europium (II) Iodide Scintillator for Sensitive X-Ray Detection.

Lead halide perovskite nanocrystals have recently demonstrated great potential as x-ray scintillators, yet they still suffer toxicity issues, inferior light yield (LY) caused by severe self-absorption. Nontoxic bivalent europium ions (Eu2+) with intrinsically efficient and self-absorption-free d-f transition are a prospective replacement for the toxic Pb2+. Here, we demonstrated solution-processed organic-inorganic hybrid halide BA10EuI12 (BA denotes C4H9NH4+) single crystals for the first time. BA10EuI12 was crystallized in a monoclinic space group of P21/c, with photoactive sites of [EuI6]4- octahedra isolated by BA+ cations, which exhibited high photoluminescence quantum yield of 72.5% and large Stokes shift of 97 nm. These properties enable an appreciable LY value of 79.6% of LYSO (equivalent to ~27,000 photons per MeV) for BA10EuI12. Moreover, BA10EuI12 shows a short excited-state lifetime (151 ns) due to the parity-allowed d-f transition, which boosts the potential of BA10EuI12 for use in real-time dynamic imaging and computer tomography applications. In addition, BA10EuI12 demonstrates a decent linear scintillation response ranging from 9.21 μGyair s-1 to 145 μGyair s-1 and a detection limit as low as 5.83 nGyair s-1. The x-ray imaging measurement was performed using BA10EuI12 polystyrene (PS) composite film as a scintillation screen, which exhibited clear images of objects under x-ray irradiation. The spatial resolution was determined to be 8.95 lp mm-1 at modulation transfer function = 0.2 for BA10EuI12/PS composite scintillation screen. We anticipate that this work will stimulate the exploration of d-f transition lanthanide metal halides for sensitive x-ray scintillators.

Real-Time 3D Microwave Medical Imaging With Enhanced Variational Born Iterative Method.

In this paper, we present a new variational Born iterative method (VBIM) for real-time microwave imaging (MWI) applications. The S-parameter volume integral equation and waveport vector Green's function are implemented to utilize the measured signal of the MWI system. Meanwhile, the real and imaginary separation (RIS) approach is used at each iterative step to simultaneously reconstruct the dielectric permittivity and conductivity of unknown objects. Compared with the Born iterative method and distorted Born iterative method, VBIM requires less computational time to reach the convergence threshold. The graphics processing unit based acceleration technique is implemented for real-time imaging. To demonstrate the efficiency and accuracy of this VBIM-RIS method, synthetic analysis of a complex multi-layer spherical phantom is first conducted. Then, the algorithm is tested with measured data using our new MWI system prototype. Finally, a synthetic brain-tumor phantom model under a thermal therapy procedure is monitored to exemplify the real-time imaging with about 5 seconds per reconstruction frame.

GPU-accelerated image registration algorithm in ophthalmic optical coherence tomography.

Limited to the power of the light source in ophthalmic optical coherence tomography (OCT), the signal-to-noise ratio (SNR) of the reconstructed images is usually lower than OCT used in other fields. As a result, improvement of the SNR is required. The traditional method is averaging several images at the same lateral position. However, the image registration average costs too much time, which limits its real-time imaging application. In response to this problem, graphics processing unit (GPU)-side kernel functions are applied to accelerate the reconstruction of the OCT signals in this paper. The SNR of the images reconstructed from different numbers of A-scans and B-scans were compared. The results demonstrated that: 1) There is no need to realize the axial registration with every A-scan. The number of the A-scans used to realize axial registration is suitable to set as ∼25, when the A-line speed was set as ∼12.5kHz. 2) On the basis of ensuring the quality of the reconstructed images, the GPU can achieve 43× speedup compared with CPU.

Exploring Multiple-Incidence Information in Deep Learning Schemes for Inverse Scattering Problems

Recently, many successes have been witnessed in the field of inverse scattering problems (ISPs) with deep learning schemes (DLSs). However, most of the studies focus on the spatial characteristics of interested targets, and little attentions are paid to the information from multiple incidences that can be treated as temporal information measured in a time series. In this work, a multiple-incidence framework (MIF) is introduced by incorporating the information from multiple incidences into an extra dimension of data space, helping to enrich physical knowledge in DLSs. A 3-D convolutional neural network (CNN) structure is further introduced to explore cross-frame information of the data space, which enables simultaneous utilizations of multiple-incidence information. The proposed MIF can be easily adapted into previous DLSs in the literatures, where two widely used DLSs including back-propagation scheme (BPS) and dominant current scheme (DCS) are used to demonstrate the advantages of the proposed framework. It is shown by extensive numerical and experimental examples that MIF enables consistent improvements for BPS and DCS. It is expected that the proposed MIF will also find its applications in real-time dynamic microwave imaging and multiple-frequency imaging by effectively using physical knowledge from temporal and frequency measurements, respectively.

Terahertz PHASR Scanner with 2 kHz, 100 picosecond Time-Domain Trace Acquisition Rate and an Extended Field-of-View Based on a Heliostat Design.

Recently, we introduced a Portable HAndheld Spectral Reflection (PHASR) Scanner to allow THz Time-Domain Spectroscopic (THz-TDS) imaging in clinical and industrial settings using a fiber-coupled and alignment-free telecentric beam steering design. The key limitations of the version 1.0 of the PHASR Scanner were its field-of-view and speed of time-domain trace acquisition. In this paper, we address these limitations by introducing a heliostat geometry for beam scanning to achieve an extended field-of-view, and by reconfiguring the Asynchronous OPtical Sampling (ASOPS) system to perform Electronically Controlled OPtical Sampling (ECOPS) measurements. The former change improved the deflection range of the beam, while also drastically reducing the coupling of the two scanning axes, the combination of which resulted in a larger than four-fold increase in the FOV area. The latter change significantly improves the acquisition speed and frequency domain performance simultaneously by improving measurement efficiency. To accomplish this, we characterized the non-linear time-axis sampling behavior of the electro-mechanical system in the ECOPS mode. We proposed methods to model and correct the non-linear time-axis distortions and tested the performance of the high-speed ECOPS trace acquisition. Therefore, here we introduce the PHASR Scanner version 2.0, which is capable of imaging a 40×27 mm2 FOV with 2000 traces per second over a 100 picosecond TDS range. This new scanner represents a significant leap towards translating the THz-TDS technology from the lab bench to the bedside for real-time clinical imaging applications.

Deep-Learning-Based Electrical Noise Removal Enables High Spectral Optoacoustic Contrast in Deep Tissue

Image contrast in multispectral optoacoustic tomography (MSOT) can be severely reduced by electrical noise and interference in the acquired optoacoustic signals. Previously employed signal processing techniques have proven insufficient to remove the effects of electrical noise because they typically rely on simplified models and fail to capture complex characteristics of signal and noise. Moreover, they often involve time-consuming processing steps that are unsuited for real-time imaging applications. In this work, we develop and demonstrate a discriminative deep learning approach to separate electrical noise from optoacoustic signals prior to image reconstruction. The proposed deep learning algorithm is based on two key features. First, it learns spatiotemporal correlations in both noise and signal by using the entire optoacoustic sinogram as input. Second, it employs training on a large dataset of experimentally acquired pure noise and synthetic optoacoustic signals. We validated the ability of the trained model to accurately remove electrical noise on synthetic data and on optoacoustic images of a phantom and the human breast. We demonstrate significant enhancements of morphological and spectral optoacoustic images reaching 19% higher blood vessel contrast and localized spectral contrast at depths of more than 2 cm for images acquired in vivo. We discuss how the proposed denoising framework is applicable to clinical multispectral optoacoustic tomography and suitable for real-time operation.

A fast blind zero-shot denoiser

Image noise is a common problem in light microscopy. This is particularly true in real-time live-cell imaging applications in which long-term cell viability necessitates low-light conditions. Modern denoisers are typically trained on a representative dataset, sometimes consisting of just unpaired noisy shots. However, when data are acquired in real time to track dynamic cellular processes, it is not always practical nor economical to generate these training sets. Recently, denoisers have emerged that allow us to denoise single images without a training set or knowledge about the underlying noise. But such methods are currently too slow to be integrated into imaging pipelines that require rapid, real-time hardware feedback. Here we present Noise2Fast, which can overcome these limitations. Noise2Fast uses a novel downsampling technique we refer to as ‘chequerboard downsampling’. This allows us to train on a discrete 4-image training set, while convergence can be monitored using the original noisy image. We show that Noise2Fast is faster than all similar methods with only a small drop in accuracy compared to the gold standard. We integrate Noise2Fast into real-time multi-modal imaging applications and demonstrate its broad applicability to diverse imaging and analysis pipelines.