Imaging Scenarios Research Articles

Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusion.

Differentiable 3D-Gaussian splatting (GS) is emerging as a prominent technique in computer vision and graphics for reconstructing 3D scenes. GS represents a scene as a set of 3D Gaussians with varying opacities and employs a computationally efficient splatting operation along with analytical derivatives to compute the 3D Gaussian parameters given scene images captured from various viewpoints. Unfortunately, capturing surround view (360° viewpoint) images is impossible or impractical in many real-world imaging scenarios, including underwater imaging, rooms inside a building, and autonomous navigation. In these restricted baseline imaging scenarios, the GS algorithm suffers from a well-known 'missing cone' problem, which results in poor reconstruction along the depth axis. In this paper, we demonstrate that using transient data (from sonars) allows us to address the missing cone problem by sampling high-frequency data along the depth axis. We extend the Gaussian splatting algorithms for two commonly used sonars and propose fusion algorithms that simultaneously utilize RGB camera data and sonar data. Through simulations, emulations, and hardware experiments across various imaging scenarios, we show that the proposed fusion algorithms lead to significantly better novel view synthesis (5dB improvement in PSNR) and 3D geometry reconstruction (60% lower Chamfer distance).

Prior Knowledge-guided Triple-Domain Transformer-GAN for Direct PET Reconstruction from Low-Count Sinograms.

To obtain high-quality positron emission tomography (PET) images while minimizing radiation exposure, numerous methods have been dedicated to acquiring standard-count PET (SPET) from low-count PET (LPET). However, current methods have failed to take full advantage of the different emphasized information from multiple domains, i.e., the sinogram, image, and frequency domains, resulting in the loss of crucial details. Meanwhile, they overlook the unique inner-structure of the sinograms, thereby failing to fully capture its structural characteristics and relationships. To alleviate these problems, in this paper, we proposed a prior knowledge-guided transformer-GAN that unites triple domains of sinogram, image, and frequency to directly reconstruct SPET images from LPET sinograms, namely PK-TriDo. Our PK-TriDo consists of a Sinogram Inner-Structure-based Denoising Transformer (SISD-Former) to denoise the input LPET sinogram, a Frequency-adapted Image Reconstruction Transformer (FaIR-Former) to reconstruct high-quality SPET images from the denoised sinograms guided by the image domain prior knowledge, and an Adversarial Network (AdvNet) to further enhance the reconstruction quality via adversarial training. Specifically tailored for the PET imaging mechanism, we injected a sinogram embedding module that partitions the sinograms by rows and columns to obtain 1D sequences of angles and distances to faithfully preserve the inner-structure of the sinograms. Moreover, to mitigate high-frequency distortions and enhance reconstruction details, we integrated global-local frequency parsers (GLFPs) into FaIR-Former to calibrate the distributions and proportions of different frequency bands, thus compelling the network to preserve high-frequency details. Evaluations on three datasets with different dose levels and imaging scenarios demonstrated that our PK-TriDo outperforms the state-of-the-art methods.

Influence mechanism of the temporal duration of laser irradiation on photoacoustic technique: a review.

In the photoacoustic (PA) technique, the laser irradiation in the time domain (i.e., laser pulse duration) governs the characteristics of PA imaging-it plays a crucial role in the optical-acoustic interaction, the generation of PA signals, and the PA imaging performance. We aim to provide a comprehensive analysis of the impact of laser pulse duration on various aspects of PA imaging, encompassing the signal-to-noise ratio, the spatial resolution of PA imaging, the acoustic frequency spectrum of the acoustic wave, the initiation of specific physical phenomena, and the photothermal-PA (PT-PA) interaction/conversion. By surveying and reviewing the state-of-the-art investigations, we discuss the effects of laser pulse duration on the generation of PA signals in the context of biomedical PA imaging with respect to the aforementioned aspects. First, we discuss the impact of laser pulse duration on the PA signal amplitude and its correlation with the lateral resolution of PA imaging. Subsequently, the relationship between the axial resolution of PA imaging and the laser pulse duration is analyzed with consideration of the acoustic frequency spectrum. Furthermore, we examine the manipulation of the pulse duration to trigger physical phenomena and its relevant applications. In addition, we elaborate on the tuning of the pulse duration to manipulate the conversion process and ratio from the PT to PA effect. We contribute to the understanding of the physical mechanisms governing pulse-width-dependent PA techniques. By gaining insight into the mechanism behind the influence of the laser pulse, we can trigger the pulse-with-dependent physical phenomena for specific PA applications, enhance PA imaging performance in biomedical imaging scenarios, and modulate PT-PA conversion by tuning the pulse duration precisely.

ABUS tumor segmentation via decouple contrastive knowledge distillation

Objective. In recent years, deep learning-based methods have become the mainstream for medical image segmentation. Accurate segmentation of automated breast ultrasound (ABUS) tumor plays an essential role in computer-aided diagnosis. Existing deep learning models typically require a large number of computations and parameters. Approach. Aiming at this problem, we propose a novel knowledge distillation method for ABUS tumor segmentation. The tumor or non-tumor regions from different cases tend to have similar representations in the feature space. Based on this, we propose to decouple features into positive (tumor) and negative (non-tumor) pairs and design a decoupled contrastive learning method. The contrastive loss is utilized to force the student network to mimic the tumor or non-tumor features of the teacher network. In addition, we designed a ranking loss function based on ranking the distance metric in the feature space to address the problem of hard-negative mining in medical image segmentation. Main results. The effectiveness of our knowledge distillation method is evaluated on the private ABUS dataset and a public hippocampus dataset. The experimental results demonstrate that our proposed method achieves state-of-the-art performance in ABUS tumor segmentation. Notably, after distilling knowledge from the teacher network (3D U-Net), the Dice similarity coefficient (DSC) of the student network (small 3D U-Net) is improved by 7%. Moreover, the DSC of the student network (3D HR-Net) reaches 0.780, which is very close to that of the teacher network, while their parameters are only 6.8% and 12.1% of 3D U-Net, respectively. Significance. This research introduces a novel knowledge distillation method for ABUS tumor segmentation, significantly reducing computational demands while achieving state-of-the-art performance. The method promises enhanced accuracy and feasibility for computer-aided diagnosis in diverse imaging scenarios.

Enhancing the Performance of a Large Aperture Ultrasound System (LAUS): A Combined Approach of Simulation and Measurement for Transmitter-Receiver Optimization.

The Large Aperture Ultrasound System (LAUS) developed at BAM is known for its ability to penetrate thick objects, especially concrete structures commonly used in nuclear waste storage and other applications in civil engineering. Although the current system effectively penetrates up to ~9 m, further optimization is imperative to enhance the safety and integrity of disposal structures for radioactive or toxic waste. This study focuses on enhancing the system's efficiency by optimizing the transducer spacing, ensuring that resolution is not compromised. An array of twelve horizontal shear wave transducers was used to find a balance between penetration depth and resolution. Systematic adjustments of the spacing between transmitter and receiver units were undertaken based on target depth ranges of known reflectors at depth ranges from 5 m to 10 m. The trade-offs between resolution and artifact generation were meticulously assessed. This comprehensive study employs a dual approach using both simulations and measurements to investigate the performance of transducer units spaced at 10 cm, 20 cm, 30 cm, and 40 cm. We found that for depths up to 5 m, a spacing of 10 cm for LAUS transducer units provided the best resolution as confirmed by both simulations and measurements. This optimal distance is particularly effective in achieving clear reflections and a satisfactory signal-to-noise ratio (SNR) in imaging scenarios with materials such as thick concrete structures. However, when targeting depths greater than 10 m, we recommend increasing the distance between the transducers to 20 cm. This increased spacing improves the SNR in comparison to other spacings, as seen in the simulation of a 10 m deep backwall. Our results emphasize the critical role of transducer spacing in achieving the desired SNR and resolution, especially in the context of depth imaging requirements for LAUS applications. In addition to the transducer spacing, different distances between individual sets of measurement positions were tested. Overall, keeping the minimal possible distance between measurement position offsets provides the best imaging results at greater depths. The proposed optimizations for the LAUS in this study are primarily relevant to applications on massive nuclear structures for nuclear waste management. This research highlights the need for better LAUS efficiency in applications such as sealing structures, laying the foundation for future technological advances in this field.

Training Universal Deep-Learning Networks for Electromagnetic Medical Imaging Using a Large Database of Randomized Objects.

Deep learning has become a powerful tool for solving inverse problems in electromagnetic medical imaging. However, contemporary deep-learning-based approaches are susceptible to inaccuracies stemming from inadequate training datasets, primarily consisting of signals generated from simplified and homogeneous imaging scenarios. This paper introduces a novel methodology to construct an expansive and diverse database encompassing domains featuring randomly shaped structures with electrical properties representative of healthy and abnormal tissues. The core objective of this database is to enable the training of universal deep-learning techniques for permittivity profile reconstruction in complex electromagnetic medical imaging domains. The constructed database contains 25,000 unique objects created by superimposing from 6 to 24 randomly sized ellipses and polygons with varying electrical attributes. Introducing randomness in the database enhances training, allowing the neural network to achieve universality while reducing the risk of overfitting. The representative signals in the database are generated using an array of antennas that irradiate the imaging domain and capture scattered signals. A custom-designed U-net is trained by using those signals to generate the permittivity profile of the defined imaging domain. To assess the database and confirm the universality of the trained network, three distinct testing datasets with diverse objects are imaged using the designed U-net. Quantitative assessments of the generated images show promising results, with structural similarity scores consistently exceeding 0.84, normalized root mean square errors remaining below 14%, and peak signal-to-noise ratios exceeding 33 dB. These results demonstrate the practicality of the constructed database for training deep learning networks that have generalization capabilities in solving inverse problems in medical imaging without the need for additional physical assistant algorithms.

A Critical Assessment of Generative Models for Synthetic Data Augmentation on Limited Pneumonia X-ray Data.

In medical imaging, deep learning models serve as invaluable tools for expediting diagnoses and aiding specialized medical professionals in making clinical decisions. However, effectively training deep learning models typically necessitates substantial quantities of high-quality data, a resource often lacking in numerous medical imaging scenarios. One way to overcome this deficiency is to artificially generate such images. Therefore, in this comparative study we train five generative models to artificially increase the amount of available data in such a scenario. This synthetic data approach is evaluated on a a downstream classification task, predicting four causes for pneumonia as well as healthy cases on 1082 chest X-ray images. Quantitative and medical assessments show that a Generative Adversarial Network (GAN)-based approach significantly outperforms more recent diffusion-based approaches on this limited dataset with better image quality and pathological plausibility. We show that better image quality surprisingly does not translate to improved classification performance by evaluating five different classification models and varying the amount of additional training data. Class-specific metrics like precision, recall, and F1-score show a substantial improvement by using synthetic images, emphasizing the data rebalancing effect of less frequent classes. However, overall performance does not improve for most models and configurations, except for a DreamBooth approach which shows a +0.52 improvement in overall accuracy. The large variance of performance impact in this study suggests a careful consideration of utilizing generative models for limited data scenarios, especially with an unexpected negative correlation between image quality and downstream classification improvement.

On nonparaxial single-pixel imaging of semitransparent objects using flat diffractive optics

High numerical apertures can result in distortions appearing in a single-shot image, rendering the acquisition of usable images challenging, if not outright impossible. However, in the realm of single-pixel imaging, various strategies can be employed to effectively inspect objects with an excellent resolution, contrast and brightness. Recent advancements in flat photonic components have facilitated the development of compact nonparaxial imaging systems, which show great promise, particularly in the THz range of wavelengths. These innovations hold the potential to advance fields such as communication, material inspection and spectroscopy. In this study, we delve into the imaging of semi-transparent objects with varying levels of detail. Furthermore, we introduce a nonparaxial design for a flat hyperbolical lens and evaluate its performance in these imaging scenarios, comparing it to structured illumination techniques involving Airy, Bessel, and common thin lens configurations. We present findings regarding potential improvements in imaging attributable to the nonparaxial hyperbolical lens.

Understanding Breast Cancer Detection: A Comprehensive Survey

The potentialities of improving the penetration of millimeter waves for breast cancer imaging are here explored. Methods: A field shaping technique based on a convex optimization method is proposed, capable of increasing the field level inside a breast-emulating stratification. Results: The theoretical results are numerically validated via the design and simulation of two circularly polarized antennas. The experimental validation of the designed antennas, using tissue-mimicking phantoms, is provided, being in good agreement with the theoretical predictions. Conclusion: The possibility of focusing, within a lossy medium, the electromagnetic power at millimeter-wave frequencies is demonstrated. Significance: Field shaping can be a key for using millimeter waves for breast cancer detection. Index Terms—Breast Cancer, Microwave Imaging, Millimeter Waves, Radial Line Slot Arrays, Near-Field Optimization. A field focusing technique based on a convex optimization method is proposed, capable of increasing the field level inside a breast-emulating stratification. A focusing technique based on convex optimization has been used here to increase the penetration in breast cancer imaging scenarios at millimeter waves. Significant enhancements of the field level inside a breast model have been achieved by employing focused apertures. The key challenge in cancer detection is how to classify tumors into malignant or benign machine learning techniques can dramatically improves the accuracy of diagnosis. As the methods itself show that it’s much complicated in designing and implementation. The data visualization process itself is a complicated process which can generate false results if the image has noise, so the result won’t be correct. It usually contains two major steps but the further steps are much similar to the above methods ,which makes it further more complicated as it including converting patched images to real images which makes the result inaccurate in diagnosis.

A new digital image shifting method by considering the geometry of camera sensors

Digital image shifting occurs when the pixels needs to be moved, the algorithm behind this is interpolation. Current digital image shifting algorithm utilizing interpolation has a limitation: it treats all images as the same type of matrix and does not consider how the images are formed this could cause problems in some sensitive imaging scenarios such as microcopy and precision optical metrology. The goal of our research is to propose a new approach to implement the digital image shifting method. We based our new method off of digital camera sensors, specifically the CMOS sensor. When light (a continuous signal) hits the CMOS sensor, the sensor transforms the light into a discrete signal and its energy into electric energy. Then, this discrete signal then generates the pixels of the image. Our algorithm is similar: The signal is convoluted with the rectangular function, generating a continuous signal, and then we multiply the discrete dirac comb function to generate a discrete signal. After that, we only have to manipulate the discrete dirac comb function to obtain the desired result. Our conducted simulations on the computer proves our method correct, hopefully more experiments can be done in the industrial setting.

Unveiling the future of breast cancer assessment: a critical review on generative adversarial networks in elastography ultrasound.

Elastography Ultrasound provides elasticity information of the tissues, which is crucial for understanding the density and texture, allowing for the diagnosis of different medical conditions such as fibrosis and cancer. In the current medical imaging scenario, elastograms for B-mode Ultrasound are restricted to well-equipped hospitals, making the modality unavailable for pocket ultrasound. To highlight the recent progress in elastogram synthesis, this article performs a critical review of generative adversarial network (GAN) methodology for elastogram generation from B-mode Ultrasound images. Along with a brief overview of cutting-edge medical image synthesis, the article highlights the contribution of the GAN framework in light of its impact and thoroughly analyzes the results to validate whether the existing challenges have been effectively addressed. Specifically, This article highlights that GANs can successfully generate accurate elastograms for deep-seated breast tumors (without having artifacts) and improve diagnostic effectiveness for pocket US. Furthermore, the results of the GAN framework are thoroughly analyzed by considering the quantitative metrics, visual evaluations, and cancer diagnostic accuracy. Finally, essential unaddressed challenges that lie at the intersection of elastography and GANs are presented, and a few future directions are shared for the elastogram synthesis research.

Comparative Analysis of Pulse Sequence Scenarios of Nuclear Magnetic Resonance Imaging of Human Brain

Comparative Analysis of Pulse Sequence Scenarios of Nuclear Magnetic Resonance Imaging of Human Brain

Mitigating AC and DC Interference in Multi-ToF-Camera Environments.

Multi-camera interference (MCI) is an important challenge faced by continuous-wave time-of-flight (C-ToF) cameras. In the presence of other cameras, a C-ToF camera may receive light from other cameras' sources, resulting in potentially large depth errors. We propose stochastic exposure coding (SEC), a novel approach to mitigate MCI. In SEC, the camera integration time is divided into multiple time slots. Each camera is turned on during a slot with an optimal probability to avoid interference while maintaining high signal-to-noise ratio (SNR). The proposed approach has the following benefits. First, SEC can filter out both the AC and DC components of interfering signals effectively, which simultaneously achieves high SNR and mitigates depth errors. Second, time-slotting in SEC enables 3D imaging without saturation in the high photon flux regime. Third, the energy savings due to camera turning on during only a fraction of integration time can be utilized to amplify the source peak power, which increases the robustness of SEC to ambient light. Lastly, SEC can be implemented without modifying the C-ToF camera's coding functions, and thus, can be used with a wide range of cameras with minimal changes. We demonstrate the performance benefits of SEC with thorough theoretical analysis, simulations and real experiments, across a wide range of imaging scenarios.

On evaluation of image quality in nonparaxial single-pixel imaging

High numerical apertures lead to an appearance of distortions in a single-shot image, which make obtaining images troublesome if not impossible. These obstacles can be overcome in single-pixel imaging, where different strategies lead to inspection of objects with a good resolution, contrast and brightness. Recent advances in flat photonic elements have enabled the creation of compact nonparaxial imaging systems, which are especially promising in the THz range of wavelengths, bringing advances to such fields as communication, material inspection and spectroscopy. In this work, we dive into the problematics of single-pixel imaging: we introduce an object sample, which we use to investigate the resolution, contrast and brightness of the classical two-lens imaging setup. We evaluate the nonparaxial imaging of the sample and report that the conditions for the best contrast and the best brightness are decoupled in nonparaxial single pixel imaging. To overcome this hurdle, we use two integral image quality assessment techniques from computational imaging theory and estimate the quality of the image in a virtual numerical THz imaging scenario. The localized mean square error metric did not cause additional constraints to the quality of the image, whereas the global mean square error has restricted the range of possible imaging setups. Thus, the computational integral image quality assessment techniques back up the main claim of this study that in the single-pixel imaging the resolution is decoupled from the image brightness.

Improved Methods for Fourier-Based Microwave Imaging.

Fourier-based imaging has been widely adopted for microwave imaging thanks to its efficiency in terms of computational complexity without compromising image resolution. Together with other backpropagation imaging algorithms like delay-and-sum (DAS), they are based on a far-field approach to the electromagnetic expression relating to fields and sources. To improve the accuracy of these techniques, this contribution presents a modified version of the well-known Fourier-based algorithm by taking into account the field radiated by the Tx/Rx antennas of the microwave imaging system. The impact on the imaged targets is discussed, providing a quantitative and qualitative analysis. The performance of the proposed method for subsampled microwave imaging scenarios is compared against other well-known aliasing mitigation methods.

A GPU-Based, Real-Time Dealiasing Framework for High-Frame-Rate Vector Doppler Imaging.

Vector Doppler is well regarded as a potential way of deriving flow vectors to intuitively visualize complex flow profiles, especially when it is implemented at high frame rates. However, this technique's performance is known to suffer from aliasing artifacts. There is a dire need to devise real-time dealiasing solutions for vector Doppler. In this paper, we present a new methodological framework for achieving aliasing-resistant flow vector estimation at real-time throughput from precalculated Doppler frequencies. Our framework comprises a series of compute kernels that have synergized: 1) an extended least squares vector Doppler (ELS-VD) algorithm, 2) single-instruction, multiple-thread (SIMT) processing principles, and 3) implementation on a graphical processing unit (GPU). Results show that this new framework, when executed on an RTX-2080 GPU, can effectively generate aliasing-free flow vector maps using high-frame-rate imaging datasets acquired from multiple transmit-receive angle pairs in a carotid phantom imaging scenario. Over the entire cardiac cycle, the frame processing time for aliasing-resistant vector estimation was measured to be less than 16 ms, which corresponds to a minimum processing throughput of 62.5 fps. In a human femoral bifurcation imaging trial with fast flow (150 cm/s), our framework was found to be effective in resolving two-cycle aliasing artifacts at a minimum throughput of 53 fps. The framework's processing throughput was generally in the real-time range for practical combinations of ELS-VD algorithmic parameters. Overall, this work represents the first demonstration of real-time, GPU-based aliasing-resistant vector flow imaging using vector Doppler estimation principles.

Design and implementation of student work management system in the context of deep learning

Abstract This paper aims to analyze and design the system’s functions by student work management requirements, focusing on the business and user modules. The security testing algorithm based on the deep learning model verifies the system’s operation, analyzes the adversarial sample attack in image and text scenarios, and uses a black-box-white-box attack algorithm and defense algorithm for system security testing. Perform environment testing, compatibility testing, data response performance testing, and instance model evaluation for implementing the student work management system. Less than 500ms is the response time, over 400 requests are processed per second, and the timeout response rate is below 5%. The security evaluation coefficient was 66.982 for Example Model I and 74.628 for Example Model II, showing the system has good loadability and security.

Dynamic and rapid deep synthesis of chemical exchange saturation transfer and semisolid magnetization transfer MRI signals

Model-driven analysis of biophysical phenomena is gaining increased attention and utility for medical imaging applications. In magnetic resonance imaging (MRI), the availability of well-established models for describing the relations between the nuclear magnetization, tissue properties, and the externally applied magnetic fields has enabled the prediction of image contrast and served as a powerful tool for designing the imaging protocols that are now routinely used in the clinic. Recently, various advanced imaging techniques have relied on these models for image reconstruction, quantitative tissue parameter extraction, and automatic optimization of acquisition protocols. In molecular MRI, however, the increased complexity of the imaging scenario, where the signals from various chemical compounds and multiple proton pools must be accounted for, results in exceedingly long model simulation times, severely hindering the progress of this approach and its dissemination for various clinical applications. Here, we show that a deep-learning-based system can capture the nonlinear relations embedded in the molecular MRI Bloch–McConnell model, enabling a rapid and accurate generation of biologically realistic synthetic data. The applicability of this simulated data for in-silico, in-vitro, and in-vivo imaging applications is then demonstrated for chemical exchange saturation transfer (CEST) and semisolid macromolecule magnetization transfer (MT) analysis and quantification. The proposed approach yielded 63–99% acceleration in data synthesis time while retaining excellent agreement with the ground truth (Pearson’s r > 0.99, p < 0.0001, normalized root mean square error < 3%).

Assessing appropriate responses to ACR urologic imaging scenarios using ChatGPT and Bard

Artificial intelligence (AI) has recently become a trending tool and topic regarding productivity especially with publicly available free services such as ChatGPT and Bard. In this report, we investigate if two widely available chatbots chatGPT and Bard, are able to show consistent accurate responses for the best imaging modality for urologic clinical situations and if they are in line with American College of Radiology (ACR) Appropriateness Criteria (AC). All clinical scenarios provided by the ACR were inputted into ChatGPT and Bard with result compared to the ACR AC and recorded. Both chatbots had an appropriate imaging modality rate of of 62% and no significant difference in proportion of correct imaging modality was found overall between the two services (p>0.05). The results of our study found that both ChatGPT and Bard are similar in their ability to suggest the most appropriate imaging modality in a variety of urologic scenarios based on ACR AC criteria. Nonetheless, both chatbots lack consistent accuracy and further development is necessary for implementation in clinical settings. For proper use of these AI services in clinical decision making, further developments are needed to improve the workflow of physicians.

A Fast Object Detection-Based Framework for Via Modeling on PCB X-Ray CT Images

For successful printed circuit board (PCB) reverse engineering (RE), the resulting device must retain the physical characteristics and functionality of the original. Although the applications of RE are within the discretion of the executing party, establishing a viable, non-destructive framework for analysis is vital for any stakeholder in the PCB industry. A widely regarded approach in PCB RE uses non-destructive x-ray computed tomography (CT) to produce three-dimensional volumes with several slices of data corresponding to multi-layered PCBs. However, the noise sources specific to x-ray CT and variability from designers hampers the thorough acquisition of features necessary for successful RE. This article investigates a deep learning approach as a successor to the current state-of-the-art for detecting vias on PCB x-ray CT images; vias are a key building block of PCB designs. During RE, vias offer an understanding of the PCB’s electrical connections across multiple layers. Our method is an improvement on an earlier iteration which demonstrates significantly faster runtime with quality of results comparable to or better than the current state-of-the-art, unsupervised iterative Hough-based method. Compared with the Hough-based method, the current framework is 4.5 times faster for the discrete image scenario and 24.1 times faster for the volumetric image scenario. The upgrades to the prior deep learning version include faster feature-based detection for real-world usability and adaptive post-processing methods to improve the quality of detections.