Image Pairs Research Articles

Taking measurement in every direction: Implicit scene representation for accurately estimating target dimensions under monocular endoscope

Background and objectives:In endoscopy, measurement of target size can assist medical diagnosis. However, limited operating space, low image quality, and irregular target shape pose great challenges to traditional vision-based measurement methods. Methods:In this paper, we propose a novel approach to measure irregular target size under monocular endoscope using image rendering. Firstly synthesize virtual poses on the same main optical axis as known camera poses, and use implicit neural representation module that considers brightness and target boundaries to render images corresponding to virtual poses. Then, Swin-Unet and rotating calipers are utilized to obtain maximum pixel length of the target in image pairs with the same main optical axis. Finally, the similarity triangle relationship of the endoscopic imaging model is used to measure the size of the target. Results:The evaluation is conducted using renal stone fragments of patients which are placed in the kidney model and the isolated porcine kidney. The mean error of measurement is 0.12 mm. Conclusions:The approached method can automatically measure object size within narrow body cavities in any visible direction. It improves the effectiveness and accuracy of measurement in limited endoscopic space.

Expanding Access to 3D Technology in Plastic Surgery of the Breast: Validation of the iPhone Against the Vectra H2.

The iPhone (Apple Inc, Cupertino, California) contains a high-fidelity 3D-scanner and is widely distributed in the United States. Presently, 3D analysis of the breast necessitates ownership of cost-prohibitive cameras and software packages such as the Vectra system. We compared the accuracy of 3D photos of the breast obtained with the iPhone X 3D scanner against the Canfield Vectra H2 (Canfield Scientific Inc. Parsippany, NJ) in an effort to expand access to 3D technology in plastic surgery. Twenty breasts (n=20) were 3D-photographed with iPhone X and the Vectra H2 and compared with color map analysis and by measuring distances across the model between key anatomical landmarks. These distances included sternal notch to nipple (SN-N), mid-chest to nipple (M-N), nipple to mid-inframammary fold (N-IMF), and inframammary fold width (IMF). Statistical tests included the Bland-Altman Plot analysis. When comparing absolute differences in distances between key anatomical landmarks, the average discrepancy in measurements between iPhone and Vectra image pairs were the following: SN-N: 0.94mm, M-N: 0.70mm, N-IMF 0.81mm, and IMF 0.96mm. Colormap analysis demonstrated an average error of 1.53mm, mean of 0.53mm, and standard deviation of ±1.81mm. Bland-Altman Plot revealed a mean difference of 0.13mm and an agreement interval between -1.90 and 2.17mm. The iPhone is capable of capturing 3D-photographs with a high level of fidelity when compared to Vectra. 3D-scans obtained with the iPhone may be useful for planning nipple position, measuring the breast footprint, choosing implants, and performing other functions using 3D technology that are typically performed using the more expensive systems.

A novel CatractNetDetect deep learning model for effective cataract classification through data fusion of fundus images

Cataracts are common eye disorders characterized by the clouding of the lens, preventing light from passing through and impairing vision. Various factors, including changes in the lens’s hydration or alterations in its proteins, may contribute to their development. Regular eye examinations conducted by an ophthalmologist or optometrist are imperative for detecting cataracts and other ocular conditions early on. Manual checks by caregivers pose several problems, including subjectivity, human error, and a lack of expertise. Biomedical fusion involves combining or linking various characteristics specific to certain diseases from different medical imaging resources. The primary objectives of this approach in disease classification are to reduce the error rate and increase the number of retrieved features. The aim of this study is to evaluate the outcomes associated with fusing visual features related to left and right eye cataract characteristics. Additionally, we investigate the impact of limited variability in deep learning models, specifically in the classification of cataract fundus versus normal fundus images. To address this issue, this study introduces CataractNetDetect, an innovative multi-label deep learning classification system that fuses feature representations from pairs of fundus images (e.g., left and right eyes) for the automatic diagnosis of various ocular disorders. Our focus is on achieving improved performance by stacking discriminative deep feature representations to combine two fundus images into a unified feature representation. Several deep learning architectures are utilized as feature descriptors, including ResNet-50, DenseNet-121, and Inception-V3, enhancing the resilience and quality of representations. Fine-tuning of these DL architectures is conducted using the ImageNet dataset, followed by an integrated stacking approach combining ResNet-50, DenseNet-121, and Inception-V3 models. The model is trained on the publicly available ODIR-5k dataset, which includes 5000 left/right eye images depicting eight different ocular conditions, ranging from healthy states to uncommon ailments such as cataracts, glaucoma, age-related macular degeneration (AMD), diabetes, hypertension, and myopia abnormalities. Moreover, extensive preprocessing of the images is performed, including data augmentation, noise reduction, contrast enhancement, scaling, and circular border cropping. The CataractNetDetect system demonstrates F1-scores, AUC, and maximum validation scores of 98.0%, 97.9%, and 100%, respectively. This ensemble-based model distinguishes itself by surpassing the performance of conventional established methodologies, including ResNet-50, DenseNet-121, and Inception-V3, thereby underscoring its efficacy in diagnostic applications.

The Effect of Attentional Bias on Emotions in Patients with Breast Cancer.

Attentional bias may influence the emotional experiences of breast cancer patients, both positively and negatively. This study aimed to investigate attentional bias in breast cancer patients and its impact on their emotions. Thirty-eight breast cancer patients completed a modified dot-probe task and the Depression Anxiety and Stress Scale to assess attentional bias and emotional states. Attentional bias was measured by analyzing response times to different stimuli in the modified dot-probe task. Emotional stimuli included 80 pairs of facial images depicting sad-neutral, fearful-neutral, happy-neutral, and neutral-neutral expressions. Attentional bias components were observed at stimulus presentation durations of 300ms and 1500ms. Differences in emotional responses among breast cancer patients with varying attentional biases were compared. Breast cancer patients exhibited attentional avoidance of sad and happy stimuli at 300ms. Further analysis revealed that patients who exhibited attentional avoidance of sad stimuli at 300ms reported higher levels of anxiety and stress. Those with attentional avoidance of fearful stimuli at 1500ms reported increased anxiety, while individuals showing attentional avoidance of happy stimuli or difficulty disengaging from happy stimuli at 1500ms reported higher levels of depression and stress. Breast cancer patients demonstrated an attentional bias toward emotional stimuli, particularly avoidance of sad and happy stimuli in 300ms. Different components of attentional bias were associated with distinct negative emotional outcomes.

Adaptive similarity-guided self-merging network for few-shot semantic segmentation

Few-shot Semantic Segmentation (FSS) attempts to segment the new category with only a few labeled samples, presenting a significant challenge. Existing approaches primarily focus on leveraging category information from the support set to identify objects of the new category in the query image. However, these models often struggle when confronted with substantial differences between paired images. To address issues stemming from scenario differences and intra-class diversity, this paper proposes an adaptive similarity-guided self-merging network. Firstly, style differences of multi-level features are introduced to alleviate the network's sensitivity to scenario variations and learn an adaptive weight for the K-shot scheme. Secondly, a feature-mask bi-aggregation module is designed to learn an enhanced feature and an initial mask for the query image. Within this module, dynamic correlations cover all the spatial locations, providing global information crucial for feature and mask aggregation. Subsequently, a self-merging module is proposed to alleviate prototype bias. It merges a self-prototype derived from the initial mask with an adaptive weighted support prototype obtained from K support images. Finally, the target object is segmented using the enhanced feature and merging prototype, and segmentation results are further refined by predictions of base categories and an adjustment factor derived from multi-level style differences. The proposed method achieves 69.1% (1-shot) and 72.3% (5-shot) mIoU on the PASCAL-5i dataset, and 47.4% (1-shot) and 52.1% (5-shot) mIoU on the COCO-20i dataset. These results demonstrate state-of-the-art segmentation performance compared to mainstream methods.© 2017 Elsevier Inc. All rights reserved.

NCNet: Deformable medical image registration network based on neighborhood cross-attention combined with multi-resolution constraints

Objective. Existing registration networks based on cross-attention design usually divide the image pairs to be registered into patches for input. The division and merging operations of a series of patches are difficult to maintain the topology of the deformation field and reduce the interpretability of the network. Therefore, our goal is to develop a new network architecture based on a cross-attention mechanism combined with a multi-resolution strategy to improve the accuracy and interpretability of medical image registration. Approach. We propose a new deformable image registration network NCNet based on neighborhood cross-attention combined with multi-resolution strategy. The network structure mainly consists of a multi-resolution feature encoder, a multi-head neighborhood cross-attention module and a registration decoder. The hierarchical feature extraction capability of our encoder is improved by introducing large kernel parallel convolution blocks; the cross-attention module based on neighborhood calculation is used to reduce the impact on the topology of the deformation field and double normalization is used to reduce its computational complexity. Main result. We performed atlas-based registration and inter-subject registration tasks on the public 3D brain magnetic resonance imaging datasets LPBA40 and IXI respectively. Compared with the popular VoxelMorph method, our method improves the average DSC value by 7.9% and 3.6% on LPBA40 and IXI. Compared with the popular TransMorph method, our method improves the average DSC value by 4.9% and 1.3% on LPBA40 and IXI. Significance. We proved the advantages of the neighborhood attention calculation method compared to the window attention calculation method based on partitioning patches, and analyzed the impact of the pyramid feature encoder and double normalization on network performance. This has made a valuable contribution to promoting the further development of medical image registration methods.

(Invited) Designing Quantum Dots for Multiplexed Deep-Tissue Imaging in the NIR-II

The combination of bright, tunable semiconductor quantum dots (QDs) emitting in the shortwave infrared or second near infrared window (SWIR/NIR-II, 1000 – 1600 nm) and InGaAs camera-equipped preclinical imagers opens the door to non-invasive, multiplexed, deep tissue imaging in live mice. While imaging in the NIR-I is like imaging through fog due to highly scattering tissue, imaging in the SWIR/NIR-II is more like imaging through pond water, enabling deeper imaging with much improved resolution. We synthesized multiple PbS/CdS core/shell QDs with emission wavelengths spanning 1000-1600 nm. These particles have been encapsulated in lipid-PEG micelles and up to three colors (so far) dosed into nude mice for imaging on a Photon Etc IR VIVO imager equipped with a hyperspectral Bragg grating and 6 bandpass filters, enabling demixing of the discrete colors. We have obtained exceptionally clear images of lymphatic drainage and the vasculature system, including visualization of lymphatic valve opening in videos. Interestingly, following intravascular injection, the liver and spleen are very visible and distinct, enabling the non-invasive visualization of a key nanomedicine drug delivery parameter (i.e., are the nanoparticles accumulating in the liver or the spleen?). Image processing investigations are expanding our ability to demix colors as well as to determine the average depth of the contrast agent, based on the wavelength-dependent attenuation of the photoluminescence based on water absorption. In addition to the photophysical advantages of using QDs for NIR-II, elemental analysis of blood and tissue samples enables quantitative biodistribution and pharmacokinetic studies. Ongoing work involves the application of this imaging approach to assess the impact nanomaterial surface properties and route of delivery on their pharmacokinetics and the use of deep tissue paired agent imaging for non-invasive “optical biopsy” of orthotopic tumors. Current data are already visually stunning, and ongoing imaging highlights new opportunities for research with every experiment! Figure 1

Children and young people’s unaccompanied mobility: the role of the built and social environments in an unequal Latin American Metropolis

ABSTRACT Unaccompanied trips are highly beneficial for children, yet their practice is difficult in unequal Latin American metropoles. This article investigates how the built and social environments influence children and young people’s trips in Santiago de Chile. It does so by reporting the findings of a stated preference questionnaire in which children and young people were invited to choose the preferred environment to walk unaccompanied in between pairs of images illustrating different social and physical attributes. The findings suggest that Santiago’s inequalities are mirrored in the environments through which children and young people prefer to walk unaccompanied. Attributes that enhance the streetscape (e.g., well-kept footpaths, grass), provide recreation and natural surveillance positively affect children’s willingness to make unaccompanied trips in wealthy areas. By contrast, those attributes are less significant for children and young people from other parts of the city, where other features have a greater impact (e.g., strangers drinking alcohol, off-licences, vehicles parked on the footpath). Furthermore, young girls are greatly discouraged from travelling unaccompanied in most circumstances, in both wealthy and not-wealthy areas. Public policies should prioritise non-wealthy neighbourhoods, improve the built environment, and strengthen local communities and local familiarity to create safer spaces that encourage unaccompanied trips.

Robot-Assisted 2D Fluoroscopic Needle Placement-A Phantom Study.

To evaluate the targeting accuracy of a novel robot-assisted guidance technique relying on one pair of 2D C-arm images. In total, 160 punctures were carried out semi-automatically by using a novel robotic device. The needle's paths were planned based on one pair of 2D fluoroscopic images from different angles. Conically shaped aluminum tips inside a gelatin-filled plexiglass phantom served as targets. The accuracy of the needle placement was assessed by taking control CTs and measuring the Euclidean distance (ED) and normal distance (ND) between the needle and the target point. In addition, the procedural time per needle placement was evaluated. The accomplished mean NDs at the target for the 45°, 60°, 75° and 90° angles were 1.86 mm (SD ± 0.19), 2.68 mm (SD ± 0.18), 2.19 mm (SD ± 0.18) and 1.86 mm (SD ± 0.18), respectively. The corresponding mean EDs were 2.32 mm (SD ± 0.16), 2.68 mm (SD ± 0.18), 2.65 mm (SD ± 0.16) and 2.44 mm (SD ± 0.15). The mean duration of the total procedure, including image acquisition, trajectory planning and placement of four needles sequentially, was 12.7 min. Robotic guidance based on two 2D fluoroscopy images allows for the precise placement of needle-like instruments at the first attempt without the need for using an invasive dynamic reference frame. This novel approach seems to be a valuable tool for the precise targeting of various anatomical structures that can be identified in fluoroscopic images.

Dataset of Registered Hematoxylin–Eosin and Ki67 Histopathological Image Pairs Complemented by a Registration Algorithm

In this work, we describe a dataset suitable for analyzing the extent to which hematoxylin–eosin (HE)-stained tissue contains information about the expression of Ki67 in immunohistochemistry staining. The dataset provides images of corresponding pairs of HE and Ki67 stainings and is complemented by algorithms for computing the Ki67 index. We introduce a dataset of high-resolution histological images of testicular seminoma tissue. The dataset comprises digitized histology slides from 77 conventional testicular seminoma patients, obtained via surgical resection. For each patient, two physically adjacent tissue sections are stained: one with hematoxylin and eosin, and one with Ki67 immunohistochemistry staining. This results in a total of 154 high-resolution images. The images are provided in PNG format, facilitating ease of use for image analysis compared to the original scanner output formats. Each image contains enough tissue to generate thousands of non-overlapping 224 × 224 pixel patches. This shows the potential to generate more than 50,000 pairs of patches, one with HE staining and a corresponding Ki67 patch that depicts a very similar part of the tissue. Finally, we present the results of applying a ResNet neural network for the classification of HE patches into categories according to their Ki67 label.

Forward and backward blocking in statistical learning.

Prediction errors have a prominent role in many forms of learning. For example, in reinforcement learning, agents learn by updating the association between states and outcomes as a function of the prediction error elicited by the event. One paradigm often used to study error-driven learning is blocking. In forward blocking, participants are first presented with stimulus A, followed by outcome X (A→X). In the second phase, A and B are presented together, followed by X (AB→X). Here, A→X blocks the formation of B→X, given that X is already fully predicted by A. In backward blocking, the order of phases is reversed. Here, the association between B and X that is formed during the first learning phase of AB→X is weakened when participants learn exclusively A→X in the second phase. The present study asked the question whether forward and backward blocking occur during visual statistical learning, i.e., the incidental learning of the statistical structure of the environment. In a series of studies, using both forward and backward blocking, we observed statistical learning of temporal associations among pairs of images. While we found no forward blocking, we observed backward blocking, thereby suggesting a retrospective revaluation process in statistical learning and supporting a functional similarity between statistical learning and reinforcement learning.

A cross-modal high-resolution image generation approach based on cloud-terminal collaboration for low-altitude intelligent network

The advancement of digitization and automation in Low Altitude Intelligent Networking (LAIN) is constrained by limited computational resources and the absence of a dedicated modal transformation mechanism, affecting the performance of latency-sensitive missions. This study addresses these challenges by proposing a Downscaling Reconstruction Multi-scale Locally Focused Generative Adversarial Network (DR-MFGAN) with Federated Learning (FL). This integration employs wavelet transform downscaling and zero-shot residual learning techniques to create noise-suppressed image pairs, ultimately facilitating high-quality image reconstruction. The core network structure is composed of multidimensional residual blocks and generative confrontation network, and feature extraction is further enhanced through cross channel attention mechanism. Finally, distributed training based on Federated Learning ensures the training effectiveness of nodes with small data volumes.Experimental results demonstrate significant improvements: an 18.18% reduction in Mean Squared Error (MSE), a 33.52% increase in Peak Signal to Noise Ratio (PSNR), and a 39.54% improvement in Learned Perceptual Image Patch Similarity (LPIPS). The edge terminal can provide high-resolution imagery with limited data, achieving precise cross-modal transformations. This approach enhances LAIN capabilities, addressing computational and transformation challenges to support critical latency-sensitive missions.

Ground-to-air aircraft infrared image deblurring based on imaging degradation simulation

The issue of infrared image deblurring has been a significant concern. However, in some specific scenes, the current mainstream deblurring algorithms based on optimization or deep learning fail to provide satisfactory results. Aiming to address the ineffectiveness of deep learning methods due to the low-cost datasets' unavailability for specific scenes, we innovatively propose a relatively simple full-chain imaging degradation simulation method using ground-to-air aircraft infrared imaging scene as an example, which considers the effects of blur and noise caused by the atmosphere, imaging system, target motion and detector. Through this method, we could generate abundant blur-clear image pairs by altering various parameters. To enhance the neural network’s generalization ability and the deblurring performance in the specific scenes, we employ a two-step approach: pretraining on the public GoPro dataset and subsequent finetuning on the simulation dataset. After testing on the simulation dataset and some real-world images, we have discovered the importance of selecting a pretraining dataset that closely matches the scene degradation mode. Additionally, regardless of whether the model is pre-trained on the UIRD or GoPro dataset, there are significant enhancements in the deblurring effect following finetuning with our constructed simulation dataset. In summary, compared to the traditional deconvolution methods and the methods trained on a general dataset, our approach not only exhibits superior deblurring capabilities but also effectively mitigates noise and prevents the occurrence of artifactual textures such as ringing artifact.

ICNoduleNet: Enhancing Pulmonary Nodule Detection Performance on Sharp Kernel CT Imaging.

Thoracic computed tomography (CT) currently plays the primary role in pulmonary nodule detection, where the reconstruction kernel significantly impacts performance in computer-aided pulmonary nodule detectors. The issue of kernel selection affecting performance has been overlooked in pulmonary nodule detection. This paper first introduces a novel pulmonary nodule detection dataset named Reconstruction Kernel Imaging for Pulmonary Nodule Detection (RKPN) for quantifying algorithm differences between the two imaging types. The dataset contains pairs of images taken from the same patient on the same date, featuring both smooth (B31f) and sharp kernel (B60f) reconstructions. All other imaging parameters and pulmonary nodule labels remain entirely consistent across these pairs. Extensive quantification reveals mainstream detectors perform better on smooth kernel imaging than on sharp kernel imaging. To address suboptimal detection on the sharp kernel imaging, we further propose an image conversion-based pulmonary nodule detector called ICNoduleNet. A lightweight 3D slice-channel converter (LSCC) module is introduced to convert sharp kernel images into smooth kernel images, which can sufficiently learn inter-slice and inter-channel feature information while avoiding introducing excessive parameters. We conduct thorough experiments that validate the effectiveness of ICNoduleNet, it takes sharp kernel images as input and can achieve comparable or even superior detection performance to the baseline that uses the smooth kernel images. The evaluation shows promising results and proves the effectiveness of ICNoduleNet.

Pain Experience and Image Quality with Curved versus Standard Compression for Breast Cancer Screening Mammography: A Randomized Controlled Trial.

Background A curve-shaped compression paddle could reduce the pain experienced by some women at breast cancer screening. Purpose To compare curved and standard compression systems in terms of pain experience and image quality in mammography screening. Materials and Methods In this randomized controlled trial conducted between October 2021 and February 2022, participants screened at three screening sites in the Netherlands were randomized to either a curved-paddle or sham-paddle group. The sham paddle was a standard paddle that was presented as a new paddle. At a standard screening examination, one additional image was acquired with a curved or sham paddle. Pain was measured on a numerical rating scale (range, 0-10). Participants provided a pain score after compression with the standard and test paddles, resulting in two scores per participant. Differences in pain scores were compared between groups using analysis of covariance, adjusting for pain score after standard-paddle compression. Two radiographers and two radiologists performed unblinded paired comparisons of curved-paddle vs standard-paddle images, using standard image quality criteria (radiographers evaluated 1246 image pairs using 12 criteria; radiologists evaluated 320 image pairs using six criteria). The one-sample Wilcoxon signed-rank test was used to determine if there was a significant preference for either paddle. Results In total, 2499 female participants (mean age, 61.6 years ± 7.1 [SD]) were studied; 1250 in the curved-paddle group and 1249 in the sham-paddle group. The mean pain score decreased by an additional 0.19 points in the curved-paddle group compared with the sham-paddle group (95% CI: 0.09, 0.28; P < .001). In terms of image quality, the observers showed no preference or a preference for the standard paddle. Decreased image contrast (range Bonferroni-corrected P values: P < .001 to P > .99) and visibility of structures were the main concerns for curved-paddle images. Conclusion The use of the curved paddle resulted in a minimal pain reduction during mammography breast compression but image quality was reduced. © RSNA, 2024 Supplemental material is available for this article.

Weakly supervised low-dose computed tomography denoising based on generative adversarial networks.

Low-dose computed tomography (LDCT) is a diagnostic imaging technique designed to minimize radiation exposure to the patient. However, this reduction in radiation may compromise computed tomography (CT) image quality, adversely impacting clinical diagnoses. Various advanced LDCT methods have emerged to mitigate this challenge, relying on well-matched LDCT and normal-dose CT (NDCT) image pairs for training. Nevertheless, these methods often face difficulties in distinguishing image details from nonuniformly distributed noise, limiting their denoising efficacy. Additionally, acquiring suitably paired datasets in the medical domain poses challenges, further constraining their applicability. Hence, the objective of this study was to develop an innovative denoising framework for LDCT images employing unpaired data. In this paper, we propose a LDCT denoising network (DNCNN) that alleviates the need for aligning LDCT and NDCT images. Our approach employs generative adversarial networks (GANs) to learn and model the noise present in LDCT images, establishing a mapping from the pseudo-LDCT to the actual NDCT domain without the need for paired CT images. Within the domain of weakly supervised methods, our proposed model exhibited superior objective metrics on the simulated dataset when compared to CycleGAN and selective kernel-based cycle-consistent GAN (SKFCycleGAN): the peak signal-to-noise ratio (PSNR) was 43.9441, the structural similarity index measure (SSIM) was 0.9660, and the visual information fidelity (VIF) was 0.7707. In the clinical dataset, we conducted a visual effect analysis by observing various tissues through different observation windows. Our proposed method achieved a no-reference structural sharpness (NRSS) value of 0.6171, which was closest to that of the NDCT images (NRSS =0.6049), demonstrating its superiority over other denoising techniques in preserving details, maintaining structural integrity, and enhancing edge contrast. Through extensive experiments on both simulated and clinical datasets, we demonstrated the superior efficacy of our proposed method in terms of denoising quality and quantity. Our method exhibits superiority over both supervised techniques, including block-matching and 3D filtering (BM3D), residual encoder-decoder convolutional neural network (RED-CNN), and Wasserstein generative adversarial network-VGG (WGAN-VGG), and over weakly supervised approaches, including CycleGAN and SKFCycleGAN.

CBCT-based synthetic CT image generation using a diffusion model for CBCT-guided lung radiotherapy.

Although cone beam computed tomography (CBCT) has lower resolution compared to planning CTs (pCT), its lower dose, higher high-contrast resolution, and shorter scanning time support its widespread use in clinical applications, especially in ensuring accurate patient positioning during the image-guided radiation therapy (IGRT) process. While CBCT is critical to IGRT, CBCT image quality can be compromised by severe stripe and scattering artifacts. Tumor movement secondary to respiratory motion also decreases CBCT resolution. In order to improve the image quality of CBCT, we propose a Lung Diffusion Model (L-DM) framework. Our proposed algorithm is based on a conditional diffusion model trained on pCT and deformed CBCT (dCBCT) image pairs to synthesize lung CT images from dCBCT images and benefit CBCT-based radiotherapy. dCBCT images were used as the constraint for the L-DM. The image quality and Hounsfield unit (HU) values of the synthetic CTs (sCT) images generated by the proposed L-DM were compared to three selected mainstream generation models. We verified our model in both an institutional lung cancer dataset and a selected public dataset. Our L-DM showed significant improvement in the four metrics of mean absolute error (MAE), peak signal-to-noise ratio (PSNR), normalized cross-correlation (NCC), and structural similarity index measure (SSIM). In our institutional dataset, our proposed L-DM decreased the MAE from 101.47 to 37.87HU and increased the PSNR from 24.97 to 29.89 dB, the NCC from 0.81 to 0.97, and the SSIM from 0.80 to 0.93. In the public dataset, our proposed L-DM decreased the MAE from 173.65 to 58.95HU, while increasing the PSNR, NCC, and SSIM from 13.07 to 24.05 dB, 0.68 to 0.94, and 0.41 to 0.88, respectively. The proposed L-DM significantly improved sCT image quality compared to the pre-correction CBCT andthree mainstream generative models. Our model can benefit CBCT-based IGRT and other potential clinical applications as it increases the HU accuracy and decreases the artifacts from input CBCT images.

Cone-beam computed tomography imaging and three-dimensional analysis of midfoot joints during non-weightbearing and weightbearing in 11 healthy feet.

Studies report that Lisfranc injury is more common than thought. Several imaging methods for assessing the stability of Lisfranc injury have been described but many are impossible to standardize and not accurate enough. To present a three-dimensional (3D) method for analyzing the changes in the joint space width of the midfoot joint and the joints of the medial part of the Lisfranc complex in healthy individuals. Non-weightbearing and weightbearing cone-beam computed tomography (CBCT) images of 11 healthy feet were acquired and analyzed with 3D software. The mean range of joint space width changes of each joint was computed from the changes in individual image pairs. 3D analysis software was used to analyze the medial part of the Lisfranc complex. In this sample of healthy feet, the changes in the joint spaces in the medial part of Lisfranc complex, calculated with 3D analysis software, was less than 0.6 mm. The distance between bones increased or decreased, depending on which part of the joint surface the measurements were taken. In this study, we present a 3D analysis method to evaluate midfoot joint space width changes. Our analysis revealed that in healthy feet there are only minimal changes in the joint space width between weightbearing and non-weightbearing indicating minimal movement of the midtarsal joints. The 3D analysis of weightbearing CBCT data provides a promising tool for analyzing the small midfoot joints in a variety of conditions.

Adversarial EM for variational deep learning: Application to semi-supervised image quality enhancement in low-dose PET and low-dose CT

In positron emission tomography (PET) and X-ray computed tomography (CT), reducing radiation dose can cause significant degradation in image quality. For image quality enhancement in low-dose PET and CT, we propose a novel theoretical adversarial and variational deep neural network (DNN) framework relying on expectation maximization (EM) based learning, termed adversarial EM (AdvEM). AdvEM proposes an encoder–decoder architecture with a multiscale latent space, and generalized-Gaussian models enabling datum-specific robust statistical modeling in latent space and image space. The model robustness is further enhanced by including adversarial learning in the training protocol. Unlike typical variational-DNN learning, AdvEM proposes latent-space sampling from the posterior distribution, and uses a Metropolis–Hastings scheme. Unlike existing schemes for PET or CT image enhancement which train using pairs of low-dose images with their corresponding normal-dose versions, we propose a semi-supervised AdvEM (ssAdvEM) framework that enables learning using a small number of normal-dose images. AdvEM and ssAdvEM enable per-pixel uncertainty estimates for their outputs. Empirical analyses on real-world PET and CT data involving many baselines, out-of-distribution data, and ablation studies show the benefits of the proposed framework.

A comparative study of principal component analysis and kernel principal component analysis for photogrammetric shape-based turbine blade damage analysis

Photogrammetry is popular as a non-contact full-field data acquisition technique for machine condition monitoring purposes. Two popular implementations thereof are Digital Image Correlation (DIC) and 3D Point Tracking (3DPT). Since these two approaches require surface preparation in the form of applying speckle patterns or discrete markers, their use is negated in several industrial applications, such as the online condition monitoring and/or assessment of horizontal axis wind turbines. A shape-based photogrammetric approach that focuses on analysing changes in the form of a rotating blade’s boundary contour is a viable alternative that does not require any surface preparation. 3D shapes of a blade at a particular instance can be extracted from a pair of stereoscopic images of the blade. Shape Principal Component Descriptors (SPCDs) determined from applying Principal Component Analysis (PCA) to Fourier coefficients of chain-code shape signatures of the blade contour can be determined. These can be regarded as indicators of the form of the shape, and as the blade rotates, variations in the SPCDs can be investigated to better understand the dynamics of the blades. This paper focuses on the application and performance evaluation of data reduction and classification techniques for a novel shape-based photogrammetry condition monitoring technique. Investigations are conducted to show that applying data reduction methods to raw time domain SPCDs can result in successful classification of differently damaged blades. Post-processing strategies are developed to ensure a more robust shape based condition monitoring tool for analysing turbine blades. Kernel Principal Component Analysis (KPCA) is implemented and it is shown that it out-performs the conventional PCA in terms of classifying different blade damage modes. The feasibility of using multi-domain statistical features as feature vectors to which PCA or KPCA is applied for classification purposes is also investigated. Results indicating how well differently damaged blades can be distinguished are provided, and it is clearly illustrated that multi-domain statistical features are more robust to noise contamination in the signals compared to using the raw SPCDs data.