Volumetric Datasets Research Articles

Automated 3-D Ultrasound Elastography of the Breast: An In Vivo Validation Study

Studies have indicated that adding 2-D quasi-static elastography to B-mode ultrasound imaging improved the specificity for malignant lesion detection, as malignant lesions are often stiffer (increased strain ratio) compared with benign lesions. This method is limited by its user dependency and so unsuitable for breast screening. To overcome this limitation, we implemented quasi-static elastography in an automated breast volume scanner (ABVS), which is an operator-independent 3-D ultrasound system and is especially useful for screening women with dense breasts. The study aim was to investigate if 3-D quasi-static elastography implemented in a clinically used ABVS can discriminate between benign and malignant breast lesions. Volumetric breast ultrasound radiofrequency data sets of 82 patients were acquired before and after automated transducer lifting. Lesions were annotated and strain was calculated using an in-house-developed strain algorithm. Two strain ratio types were calculated per lesion: using axial and maximal principal strain (i.e., strain in dominant direction). Forty-four lesions were detected: 9 carcinomas, 23 cysts and 12 other benign lesions. A significant difference was found between malignant (median: 1.7, range: [1.0-3.2]) and benign (1.0, [0.6-1.9]) using maximal principal strain ratios. Axial strain ratio did not reveal a significant difference between benign (0.6, [-12.7 to 4.9]) and malignant lesions (0.8, [-3.5 to 5.1]). Three-dimensional strain imaging was successfully implemented on a clinically used ABVS to obtain, visualize and analyze in vivo strain images in three dimensions. Results revealed that maximal principal strain ratios are significantly increased in malignant compared with benign lesions.

Reconstructing higher-resolution four-dimensional time-varying volumetric data

We have witnessed substantial growth in super-resolution research within the computer vision community. Unlike previous works that mainly focus on the super-resolution synthesis of images, videos, or single volumes, our research is dedicated to the super-resolution synthesis of time-varying volumetric data, which are generated from scientific simulations and are crucial for domain scientists to understand and analyse complex scientific phenomena. Compared to previous works, our research presents a greater challenge: the time-varying volumetric data have higher dimensions, making it more difficult to synthesise super-resolution that maintains good spatio-temporal consistency while achieving high visual quality. To tackle this challenge, we introduce a new GAN-based network called SSR-DoubleUNetGAN, which includes novel network architecture and loss functions, allowing for accurate synthesis of spatial super-resolution for time-varying volumetric data with relatively fast training time. Our method can be applied in the context of in-situ visualisation to aid domain scientists in analysing more time-varying volumetric data more efficiently. In addition, it can be used in the compression-decompression pipeline to recover the super-resolution time-varying volumetric data from their low-resolution counterpart. To demonstrate its effectiveness, we applied various time-varying volumetric datasets from different scientific simulations to it. To demonstrate its advantages, we compared it qualitatively and quantitatively with five state-of-the-art super-resolution techniques, namely SSR-TVD, Tricubic, SRResNet, Cubic, and Linear. Furthermore, we conducted an ablation study to validate its important modules. The experimental results show that our method outperforms the compared state-of-the-art techniques.

CoordNet: Data Generation and Visualization Generation for Time-Varying Volumes via a Coordinate-Based Neural Network.

Although deep learning has demonstrated its capability in solving diverse scientific visualization problems, it still lacks generalization power across different tasks. To address this challenge, we propose CoordNet, a single coordinate-based framework that tackles various tasks relevant to time-varying volumetric data visualization without modifying the network architecture. The core idea of our approach is to decompose diverse task inputs and outputs into a unified representation (i.e., coordinates and values) and learn a function from coordinates to their corresponding values. We achieve this goal using a residual block-based implicit neural representation architecture with periodic activation functions. We evaluate CoordNet on data generation (i.e., temporal super-resolution and spatial super-resolution) and visualization generation (i.e., view synthesis and ambient occlusion prediction) tasks using time-varying volumetric data sets of various characteristics. The experimental results indicate that CoordNet achieves better quantitative and qualitative results than the state-of-the-art approaches across all the evaluated tasks. Source code and pre-trained models are available at https://github.com/stevenhan1991/CoordNet.

Conditional generative adversarial networks for stripe artefact removal in high-resolution X-ray tomography

Tomographic imaging supports a great number of medical and material science applications. The collected projection data usually has different types of imaging artefacts and noise. Various image pre-processing and reconstruction methods are used to obtain volumetric datasets of high quality for further analysis. In order to minimise reconstruction artefacts, one can apply either filtering and/or data completion/inpainting techniques which can recover the data. Deep learning (DL) methods to remove artefacts and noise have been successfully applied in the past. In this paper, we present a novel approach based on conditional generative adversarial networks (cGANs) to remove stripe artefacts. The novelty of the presented technique is in how the training data for DL is extracted from the same tomographic dataset that needs recovery. We also provide new deterministic stripe detection and inpainting algorithms to support the development. The presented methods are compared with other stripe removal algorithms and applied to 3D and 4D high-resolution X-ray data collected at Diamond Light Source synchrotron, UK. The proposed DL method delivers reconstructed images with minimised ring artefacts while being a parameter-free approach. A similar DL strategy can also be applied to remove other types of artefacts in images.

Four-dimensional particle tracking velocimetry measurements of unsteady three-dimensional vortex onset and progression for 5415 straight ahead, static drift, and pure sway

Large-scale towed system four-dimensional particle tracking velocimetry (4DPTV) implementation and measurements are presented for towing tank tests for a naval combatant 5415 ship model for straight ahead, static drift β = 10°, and pure sway βmax = 10° conditions. The results, for the first time, provide instantaneous volumetric flow field data around the ship model including near the hull surface, and a complete description of the sonar dome 3D unsteady vortex separation onset and progression due to the 4DPTV larger measurement volume and higher data rates in comparison with previous results using tomographic particle image velocimetry (TPIV); thereby, providing the essential data required for the assessment of current hybrid Reynolds-averaged Navier–Stokes/large eddy simulation turbulence modeling capabilities and guidance for the necessary modeling improvements. The 4DPTV system is summarized and compared with the previous TPIV, including the camera calibration procedures, trigger systems, and synchronization with the sway motion. Analysis procedures, including data reduction procedures and the 3D vortex core detection and voxel labeling techniques, are described. The identification of the complex vortex separations and breakdown is aided using complementary detached eddy simulations. The vortex–vortex interaction process is visualized from instantaneous volumetric datasets. The pros and cons of the 4DPTV vs TPIV, the comparison between the second- and third-generation vortex visualization technique, and the statistical convergence error analysis of sonar dome port vortex for β = 10° are discussed. Plans for increased spatial resolution of the 4DPTV system and additional data reduction techniques for detailed turbulence analysis are also discussed.

Virtual Reality Simulation for the Middle Cranial Fossa Approach: A Validation Study.

Virtual reality (VR) surgical rehearsal is an educational tool that exists in a safe environment. Validation is necessary to establish the educational value of this platform. The middle cranial fossa (MCF) is ideal for simulation because trainees have limited exposure to this approach and it has considerable complication risk. Our objectives were to assess the face, content, and construct validities of an MCF VR simulation, as well as the change in performance across serial simulations. Using high-resolution volumetric data sets of human cadavers, the authors generated a high-fidelity visual and haptic rendering of the MCF approach using CardinalSim software. Trainees from Neurosurgery and Otolaryngology-Head and Neck Surgery at two Canadian academic centers performed MCF dissections on this VR platform. Randomization was used to assess the effect of enhanced VR interaction. Likert scales were used to assess the face and content validities. Performance metrics and pre- and postsimulation test scores were evaluated. Construct validity was evaluated by examining the effect of the training level on simulation performance. Twenty trainees were enrolled. Face and content validities were achieved in all domains. Construct validity, however, was not demonstrated. Postsimulation test scores were significantly higher than presimulation test scores (P < .001). Trainees demonstrated statistically significant improvement in the time to complete dissections (P < .001), internal auditory canal skeletonization (P < .001), completeness of the anterior petrosectomy (P < .001), and reduced number of injuries to critical structures (P = .001). This MCF VR simulation created using CardinalSim demonstrated face and content validities. Construct validity was not established because no trainee included in the study had previous MCF approach experience, which further emphasizes the importance of simulation. When used as a formative educational adjunct in both Neurosurgery and Otolaryngology-Head and Neck Surgery, this simulation has the potential to enhance understanding of the complex anatomic relationships of critical neurovascular structures.

A transfer function optimization using visual saliency for region of interest-based direct volume rendering

Direct volume rendering (DVR) helps data interpretation by allowing users to focus attention on specific regions in a volume that are of most interest. The effective generation of these regions of interest (ROIs)-based DVR images, however, still remains a major challenge. It involved the intricate process of assigning appropriate optical parameters (opacity and/or color) to the ROIs and other regions in the volume using transfer function (TF) specification. Achieving an optimal TF often requires multiple iterations of a trial-and-error process, starting from scratch in each case, which is time-consuming and labor-intensive. To address this issue, there have been various TF optimization approaches, which could automatically fine-tune an initial TF toward an optimal solution satisfying a pre-defined objective metric. Although effective, the previous objective metrics exhibit two inherent limitations: (i) they were only capable of optimizing ROIs along an individual viewing ray of the volume, and the relationship with neighboring ROIs (as context) from other rays could not be controlled; (ii) the TF parameters to be optimized were restricted to opacity (i.e., the observability of ROIs) and did not include color which is also important to enhance the visual distinction of ROIs. In this work, we propose a new TF optimization approach by introducing an objective metric using visual saliency. Visual saliency is a biologically-inspired measure to aid the identification of ROIs considered important relative to other regions within an image according to the human visual system. Our novel adoption of visual saliency enables the TF optimization to take into account all ROIs in a volume (with its rendered image) implicitly. In addition, both opacity and color parameters can be applied to the TF optimization. Given the user-assigned target visual saliency of ROIs, our approach can produce more controllable and complete ROIs-based DVR images. We outline our approach by applications to a variety of volumetric datasets and highlight its advantages in comparison to the dominant TF optimization approaches that use visibility-based objective metrics.

A Comparative Measurement Study of Point Cloud-Based Volumetric Video Codecs

Volumetric video is fully three-dimensional and immersive, which shows great potential in various application scenarios. However, the naive delivery of volumetric video over the Internet requires a huge amount of bandwidth and compute resources. A volumetric video should be compressed by a specific codec before transmission. In spite of the existence of a few pioneering efforts on prototype design, there is very limited work to investigate the efficiency and practicality of existing volumetric video codecs. In this paper, we conduct a comparative measurement study on five representative point cloud-based volumetric video codecs, including 3 conventional codecs (e.g., Draco, G-PCC, V-PCC) and 2 neural-based codecs (e.g., PU-GCN <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$+$</tex-math> </inline-formula> , MPU <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$+$</tex-math> </inline-formula> ), on six real volumetric video datasets. We investigate the applicability of the above codecs in different application scenarios, and examine how the features of point cloud (e.g., quality, texture, geometric and scene complexity) affect the quality of compressed volumetric video from multiple perspectives. In addition, we also study the impact of users’ viewing behaviors on the coding and delivery efficiency with Quality of Experience (QoE) metrics. Our results shed a number of important insights, which provide useful guidelines for optimizing the design of future volumetric video streaming systems.

Differential evolution-based neural architecture search for brain vessel segmentation

Brain vasculature analysis is critical in developing novel treatment targets for neurodegenerative diseases. Such an accurate analysis cannot be performed manually but requires a semi-automated or fully-automated approach. Deep learning methods have recently proven indispensable for the automated segmentation and analysis of medical images. However, optimizing a deep learning network architecture is another challenge. Manually selecting deep learning network architectures and tuning their hyper-parameters requires a lot of expertise and effort. To solve this problem, neural architecture search (NAS) approaches that explore more efficient network architectures with high segmentation performance have been proposed in the literature. This study introduces differential evolution-based NAS approaches in which a novel search space is proposed for brain vessel segmentation. We select two architectures that are frequently used for medical image segmentation, i.e. U-Net and Attention U-Net, as baselines for NAS optimizations. The conventional differential evolution and the opposition-based differential evolution with novel search space are employed as search methods in NAS. Furthermore, we perform ablation studies and evaluate the effects of specific loss functions, model pruning, threshold selection and generalization performance on the proposed models. The experiments are conducted on two datasets providing 335 single-channel 8-bit gray-scale images. These datasets are a public volumetric cerebrovascular system dataset (vesseINN) and our own dataset called KUVESG. The proposed NAS approaches, namely UNAS-Net and Attention UNAS-Net architectures, yield better segmentation performance in terms of different segmentation metrics. More specifically, UNAS-Net with differential evolution reveals high dice score/sensitivity values of 79.57/81.48, respectively. Moreover, they provide shorter inference times by a factor of 9.15 than the baseline methods.

Multi-physics data registration for the improvement of Additive Manufacturing process control

The work presented in this paper is to tend towards a goal of developing a methodology and associated software module to establish cross-correlation between real time process monitoring data and characterization data. To implement this methodology, a number of steel specimen were built following a design of experiment approach. During the manufacturing, all monitoring data was recorded in log files with a precise timestamp, the data collected being then temporal signals. Subsequently, these parts were inspected by high resolution X-ray tomography, and the defects present in the part were characterized, resulting in 3D volumetric dataset. The software module developed on CIVA platform then enables the registration of these multi-physics data.

Reduced cingulate gyrus volume in Cavalier King Charles Spaniels with syringomyelia and neuropathic pain revealed by voxel-based morphometry: a pilot study.

Pathomorphological alterations of the central nervous system in dogs, such as syringomyelia and Chiari-like malformation, can cause cranial and cervical hyperesthesia and neuropathic pain. The long-term activity of the pain network can induce functional alteration and eventually even morphological changes in the pain network. This may happen especially in the prefrontal and cingulate cortex, where atrophy of the gray matter (GM) was observed in humans with chronic pain, irrespective of the nature of the pain syndrome. We tested the hypothesis that Cavalier King Charles Spaniels (CKCS) with Chiari-like malformation and associated syringomyelia (SM) and pain show cerebral morphological differences compared to animals without signs of syringomyelia and pain. Volumetric datasets of 28 different brain structures were analyzed in a retrospective manner, including voxel-based morphometry, using magnetic resonance imaging data obtained from 41 dogs. Volumetric analyses revealed a decrease in GM volumes in the cingulate gyrus (CG) in CKCS with SM and chronic pain when normalized to brain volume. This finding was supported by voxel-based morphometry, which showed a cluster of significance within the CG. GM atrophy in the CG is associated with chronic pain and thus may serve as an objective readout parameter for the diagnosis or treatment of canine pain syndromes.

Tightly Coupled, Graph-Based DVL/IMU Fusion and Decoupled Mapping for SLAM-Centric Maritime Infrastructure Inspection

In this article, we address the problem of simultaneous localization and mapping (SLAM)-centric maritime infrastructure inspection [using unmanned surface vehicles (USVs)] via novel approaches in tightly-coupled, graph-based DVL/IMU fusion and decoupled mapping. As our first contribution, we formalize the preintegration of linear velocity measurements, obtained by a Doppler velocity log (DVL), in combination with angular velocity measurements, obtained by an inertial measurement unit (IMU), as binary factors encoding relative position. To evaluate state estimation improvements imparted by DVL/IMU fusion, we implement our proposed factor within a state-of-the-art, graph-based lidar-visual-inertial (LVI) SLAM system as our second contribution. Accuracy and robustness improvements are demonstrated in simulation by comparing maximum <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a posteriori</i> pose estimates with and without DVL/IMU fusion against ground truth poses. As our third contribution, we propose a map generation framework for downstream inspection applications decoupled from SLAM. In our framework, volumetric data (captured by sonar, lidar, etc.) is transformed into a common world coordinate frame using extrinsic calibrations and SLAM pose estimates as input. Our framework operates over the complete set of raw volumetric data, whereas SLAM systems (both online and offline) typically operate over a subset of down-sampled volumetric data. To address the processing of additional volumetric data, we present innovations in refined pose correction and staged filtering for user-controlled denoising. We experimentally evaluate our map generation framework against the LVI SLAM system adopted for this study using real-world data and demonstrate improvements to map quality metrics important to inspection.

Mapping internal strain fields of fused filament fabrication metal filled polylactic acid structure using digital volume correlation.

With the advancement in Fused Filament Fabrication (FFF), its application is increasing widely across different industries such as aeronautical, biomedical, robotics, etc. The internal structure is becoming more complex and intricate with varying materials of reinforcement which are used to improve mechanical properties. Current measurement techniques like Digital Image Correlation (DIC) are non-destructive testing methods that do not provide enough information on the behaviour of internal microstructure for anisotropic FFF materials. Digital Volume Correlation (DVC) is non-destructive testing technique which provides full field internal 3D deformation and strain fields. Copper particle filled PLA samples manufactured using FFF method with 20, 40, 60 and 80 infill percentages were loaded in tension inside Micro-CT. X-rays were passed through the sample to get a volumetric dataset for different loadings. Using DVC method on the dataset, internal displacement and strain fields were generated for 20, 40, 60 and 80 infill percentage FFF sample.

Three-dimensional morphology of an ultrafine Al-Si eutectic produced via laser rapid solidification

Al-Si alloys processed by laser rapid solidification yield eutectic microstructures with ultrafine and interconnected fibers. Such fibrous structures have long been thought to bear resemblance to those formed in impurity-doped alloys upon conventional casting. Here, we show that any similarity is purely superficial. By harnessing high-throughput characterization and computer vision techniques, we perform a three-dimensional analysis of the branching behavior of the ultrafine eutectic and compare it against an impurity-modified eutectic as well as a random fractal (as a benchmark). Differences in the branching statistics point to different microstructural origins of the impurity- and quench-modified eutectic. Our quantitative approach is not limited to the data presented here but can be used to extract abstract information from other volumetric datasets, without customization.

Ultra-Widefield OCT Angiography.

Optical Coherence Tomography Angiography (OCTA), a functional extension of OCT, has the potential to replace most invasive fluorescein angiography (FA) exams in ophthalmology. So far, OCTA's field of view is however still lacking behind fluorescence fundus photography techniques. This is problematic, because many retinal diseases manifest at an early stage by changes of the peripheral retinal capillary network. It is therefore desirable to expand OCTA's field of view to match that of ultra-widefield fundus cameras. We present a custom developed clinical high-speed swept-source OCT (SS-OCT) system operating at an acquisition rate 8-16 times faster than today's state-of-the-art commercially available OCTA devices. Its speed allows us to capture ultra-wide fields of view of up to 90 degrees with an unprecedented sampling density and hence extraordinary resolution by merging two single shot scans with 60 degrees in diameter. To further enhance the visual appearance of the angiograms, we developed for the first time a three-dimensional deep learning based algorithm for denoising volumetric OCTA data sets. We showcase its imaging performance and clinical usability by presenting images of patients suffering from diabetic retinopathy.

Self-supervised denoising of Nyquist-sampled volumetric images via deep learning.

Deep learning has demonstrated excellent performance enhancing noisy or degraded biomedical images. However, many of these models require access to a noise-free version of the images to provide supervision during training, which limits their utility. Here, we develop an algorithm (noise2Nyquist) that leverages the fact that Nyquist sampling provides guarantees about the maximum difference between adjacent slices in a volumetric image, which allows denoising to be performed without access to clean images. We aim to show that our method is more broadly applicable and more effective than other self-supervised denoising algorithms on real biomedical images, and provides comparable performance to algorithms that need clean images during training. We first provide a theoretical analysis of noise2Nyquist and an upper bound for denoising error based on sampling rate. We go on to demonstrate its effectiveness in denoising in a simulated example as well as real fluorescence confocal microscopy, computed tomography, and optical coherence tomography images. We find that our method has better denoising performance than existing self-supervised methods and is applicable to datasets where clean versions are not available. Our method resulted in peak signal to noise ratio (PSNR) within 1dB and structural similarity (SSIM) index within 0.02 of supervised methods. On medical images, it outperforms existing self-supervised methods by an average of 3dB in PSNR and 0.1 in SSIM. noise2Nyquist can be used to denoise any volumetric dataset sampled at at least the Nyquist rate making it useful for a wide variety of existing datasets.

A Web-Based Synchronized Architecture for Collaborative Dynamic Diagnosis and Therapy Planning

Clinical treatment delivery often involves multiple medical professionals, where collaborative teamwork is crucial to ensure the quality of diagnosis and therapeutic decision making. With the development of Internet of Medical Things (IoMT) and its applications in mobile health (mHealth), healthcare services can be delivered to remote users. However, a challenging situation arises when the data are volumetric and dynamic, it will be difficult to achieve real-time performance in information streaming and data dynamic rendering and synchronization over Internet, as is the case with cardiac procedures, where both the anatomy and dynamic function characteristics of the organ must be considered. To address this issue, we have developed new algorithms and a web-based collaborative software architecture to display dynamic volumetric data in a web browser to enable dynamic data sharing with remote healthcare professionals. We illustrate our system using the diagnosis and treatment planning of an atrial septal defect (ASD) repair as an example to evaluate the capabilities of our platform. In this example, a series of three-dimensional (3D) ultrasound images are registered and fused with the high-quality magnetic resonance (MR) cardiac data to provide complementary information; virtual septal occluder patches are modeled and integrated into the beating heart views to facilitate the planning of the procedure to occlude the ASD, and feedback is streamed amongst all participating professionals. Our algorithm runs on a Node.js server with WebSocket protocol to synchronize dynamic heart rendering, so all the connected users can observe the same four-dimensional (4D) display of the dual-modality heart data during the process of examining cardiac anatomy, performing functional analysis, and sharing treatment strategies across distributed geographic locations. The presented computational models and software architecture create a new vehicle for collaborative exploration and manipulation of dynamic volumetric medical datasets.

KD-INR: Time-Varying Volumetric Data Compression via Knowledge Distillation-Based Implicit Neural Representation.

Traditional deep learning algorithms assume that all data is available during training, which presents challenges when handling large-scale time-varying data. To address this issue, we propose a data reduction pipeline called knowledge distillation-based implicit neural representation (KD-INR) for compressing large-scale time-varying data. The approach consists of two stages: spatial compression and model aggregation. In the first stage, each time step is compressed using an implicit neural representation with bottleneck layers and features of interest preservation-based sampling. In the second stage, we utilize an offline knowledge distillation algorithm to extract knowledge from the trained models and aggregate it into a single model. We evaluated our approach on a variety of time-varying volumetric data sets. Both quantitative and qualitative results, such as PSNR, LPIPS, and rendered images, demonstrate that KD-INR surpasses the state-of-the-art approaches, including learning-based (i.e., CoordNet, NeurComp, and SIREN) and lossy compression (i.e., SZ3, ZFP, and TTHRESH) methods, at various compression ratios ranging from hundreds to ten thousand.

Automated Learning for Deformable Medical Image Registration by Jointly Optimizing Network Architectures and Objective Functions.

Deformable image registration plays a critical role in various tasks of medical image analysis. A successful registration algorithm, either derived from conventional energy optimization or deep networks, requires tremendous efforts from computer experts to well design registration energy or to carefully tune network architectures with respect to medical data available for a given registration task/scenario. This paper proposes an automated learning registration algorithm (AutoReg) that cooperatively optimizes both architectures and their corresponding training objectives, enabling non-computer experts to conveniently find off-the-shelf registration algorithms for various registration scenarios. Specifically, we establish a triple-level framework to embrace the searching for both network architectures and objectives with a cooperating optimization. Extensive experiments on multiple volumetric datasets and various registration scenarios demonstrate that AutoReg can automatically learn an optimal deep registration network for given volumes and achieve state-of-the-art performance. The automatically learned network also improves computational efficiency over the mainstream UNet architecture from 0.558 to 0.270 seconds for a volume pair on the same configuration.

Quantitative Comparisons of Volumetric Datasets From Experiments and Computational Models

Quantitative Comparisons of Volumetric Datasets From Experiments and Computational Models