Simulated Images Research Articles

Enhancing Inter-AUV Perception: Adaptive 6-DOF Pose Estimation with Synthetic Images for AUV Swarm Sensing

The capabilities of AUV mutual perception and localization are crucial for the development of AUV swarm systems. We propose the AUV6D model, a synthetic image-based approach to enhance inter-AUV perception through 6D pose estimation. Due to the challenge of acquiring accurate 6D pose data, a dataset of simulated underwater images with precise pose labels was generated using Unity3D. Mask-CycleGAN technology was introduced to transform these simulated images into realistic synthetic images, addressing the scarcity of available underwater data. Furthermore, the Color Intermediate Domain Mapping strategy is proposed to ensure alignment across different image styles at pixel and feature levels, enhancing the adaptability of the pose estimation model. Additionally, the Salient Keypoint Vector Voting Mechanism was developed to improve the accuracy and robustness of underwater pose estimation, enabling precise localization even in the presence of occlusions. The experimental results demonstrated that our AUV6D model achieved millimeter-level localization precision and pose estimation errors within five degrees, showing exceptional performance in complex underwater environments. Navigation experiments with two AUVs further verified the model’s reliability for mutual 6D pose estimation. This research provides substantial technical support for more complex and precise collaborative operations for AUV swarms in the future.

Topologically frustrated structures in inkjet printed chiral nematic liquid crystal droplets - experiments and simulations.

Director field alignment in inkjet printed droplets of chiral nematic liquid crystalline materials is investigated using both experiments and numerical simulations. Experimental investigations are performed by depositing droplets of varying sizes and pitches on homeotropic alignment layers. The competition between the bulk behaviour of the chiral nematic liquid crystal and the boundary conditions imposed by the droplet surface leads to the formation of a range of possible internal director configurations. Numerical investigations are performed using a free energy minimisation approach, and the resultant simulated polarising optical microscope images are found to agree well with experimental observations.

Bilateral Proxy Federated Domain Generalization for Privacy-preserving Medical Image Diagnosis.

Contemporary domain generalization methods have demonstrated effectiveness in aiding the generalized diagnosis of medical images with multi-source data by joint optimization. However, the centralized training paradigm employed by these approaches becomes infeasible when data are non-shared across domains due to the high privacy of medical data. Despite attempts by existing federated domain generalization methods to address this issue, the simultaneous attainment of strict privacy protection and a satisfactory level of generalization ability on out-of-distribution data remains a persistent challenge. In this paper, to tackle this challenging problem, we propose a novel approach called the Bilateral Proxy Framework (BPF). The BPF leverages the client-side proxies to facilitate the strict privacy-preserving communications with the server and ensure smoother and more stable convergences of local models through mutual distillation. Meanwhile, the server-side proxy adopts a distance-based strategy and a parameter moving average scheme, which enhances the stability and robustness of the global model, particularly by averting abrupt parameter changes that could result in fluctuations or overfitting. Through these advancements, our framework strives to enhance the generalization capability of the global model, enabling more accurate and reliable medical image diagnosis in federated settings. The effectiveness of our method is demonstrated with superior performance over state-of-the-arts on both simulated and real-world distribution medical image diagnosis tasks.

Quaternary phase-field simulation of the effect of additives on the structure of polyvinylidene fluoride microporous membranes

This study builds upon the existing phase-field model of the non-solvent-induced phase separation (NIPS) method, simulating factors that influence membrane formation in a ternary system. A polyvinylidene fluoride (PVDF) membrane system is utilized, and a fourth substance, an additive, is introduced based on the original mathematical model of the ternary system. The interaction parameters of the additives within the system are calculated using the Krevelen method. The simulation is configured to set the values of additive mobility and loss coefficients of the concentration gradient, thereby simulating the effect of additives in the quaternary system on the evolution of the micropore membrane structure. Finally, simulated images of different concentrations of additives with various types are compared with experimental results of microporous structure and porosity formation. This comparison serves to validate the model.

Ray-Tracing-Assisted SAR Image Simulation under Range Doppler Imaging Geometry

In order to achieve an effective balance between SAR image simulation fidelity and efficiency, we proposed a ray-tracing-assisted SAR image simulation method under range doppler (RD) imaging geometry. This method utilizes the spatial traversal mode of RD imaging geometry to transmit discrete electromagnetic (EM) waves into the SAR radiation area and follows the Nyquist sampling law to set the density of transmitted EM waves to effectively identify the beam radiation area. The ray-tracing algorithm is used to obtain the backscatter amplitude and real-time slant range of the transmitted EM wave, which can effectively record the multiple backscattering among the components of the distributed target so that the backscattering subfields of each component can be correlated. According to the RD condition equation, the backscattering amplitude is assigned to the corresponding range gate, and the three-dimensional (3D) target is mapped into the two-dimensional (2D) SAR slant-range coordinate system, and the SAR target simulated image is directly obtained. Finally, the simulation images of the proposed method are compared qualitatively and quantitatively with those obtained by commercial simulation software, and the effectiveness of the proposed method is verified.

Jacobi-Fourier phase masks as ophthalmic elements to correct presbyopia.

Investigations into the correction of presbyopia have considered lens design, clinical implications and the development of objective metrics such as the visual Strehl ratio. This study investigated the Jacobi-Fourier phase mask as an ophthalmic element in the correction of presbyopia. The goal was to develop a contact or intraocular lens whose performance was largely insensitive to changes in pupil diameter. Numerical simulations based on Fourier optics were performed to evaluate three different Jacobi-Fourier polynomials, with the aim of providing a range of clear vision (1 Dioptre (D)). Performance was evaluated for three pupil sizes (6, 4 and 2 mm), while polychromatic images were simulated using three different wavelengths (656.3, 587.6 and 486.1 nm). The Neural Transfer function was included in the simulation. To validate the method and results, we used the Visual Strehl combined objective metric (VSCombined) currently used in visual optics. This metric gives more weight to the phase transfer function and is more suitable for non-symmetrical phase functions. Numerical validation showed the suitability of the Jacobi-Fourier phase masks for extending the range of clear vision of presbyopic eyes, providing a visual acuity of at least 0.10 logMAR (6/7.5 Snellen) at all distances between 1 and 6 m. The results show a range of clear vision of 1D was not affected by changes in pupil size, an increase in retinal image contrast accompanied by image artefact reduction by increasing the radial order of the Jacobi-Fourier phase mask and a reduction of wavelength dependence of the retinal images. These results are supported by simulated images and the objective criterion VSCombined. The use of Jacobi-Fourier phase masks as ophthalmic elements for presbyopic correction show promising results, with a good range of clear vision andreduced dependence on pupil size and chromatic aberration.

Clinical performance of deep learning-enhanced ultrafast whole-body scintigraphy in patients with suspected malignancy

BackgroundTo evaluate the clinical performance of two deep learning methods, one utilizing real clinical pairs and the other utilizing simulated datasets, in enhancing image quality for two-dimensional (2D) fast whole-body scintigraphy (WBS).MethodsA total of 83 patients with suspected bone metastasis were retrospectively enrolled. All patients underwent single-photon emission computed tomography (SPECT) WBS at speeds of 20 cm/min (1x), 40 cm/min (2x), and 60 cm/min (3x). Two deep learning models were developed to generate high-quality images from real and simulated fast scans, designated 2x-real and 3x-real (images from real fast data) and 2x-simu and 3x-simu (images from simulated fast data), respectively. A 5-point Likert scale was used to evaluate the image quality of each acquisition. Accuracy, sensitivity, specificity, and the area under the curve (AUC) were used to evaluate diagnostic efficacy. Learned perceptual image patch similarity (LPIPS) and the Fréchet inception distance (FID) were used to assess image quality. Additionally, the count-level consistency of WBS was compared between the two models.ResultsSubjective assessments revealed that the 1x images had the highest general image quality (Likert score: 4.40 ± 0.45). The 2x-real, 2x-simu and 3x-real, 3x-simu images demonstrated significantly better quality than the 2x and 3x images (Likert scores: 3.46 ± 0.47, 3.79 ± 0.55 vs. 2.92 ± 0.41, P < 0.0001; 2.69 ± 0.40, 2.61 ± 0.41 vs. 1.36 ± 0.51, P < 0.0001), respectively. Notably, the quality of the 2x-real images was inferior to that of the 2x-simu images (Likert scores: 3.46 ± 0.47 vs. 3.79 ± 0.55, P = 0.001). The diagnostic efficacy for the 2x-real and 2x-simu images was indistinguishable from that of the 1x images (accuracy: 81.2%, 80.7% vs. 84.3%; sensitivity: 77.27%, 77.27% vs. 87.18%; specificity: 87.18%, 84.63% vs. 87.18%. All P > 0.05), whereas the diagnostic efficacy for the 3x-real and 3x-simu was better than that for the 3x images (accuracy: 65.1%, 66.35% vs. 59.0%; sensitivity: 63.64%, 63.64% vs. 64.71%; specificity: 66.67%, 69.23% vs. 55.1%. All P < 0.05). Objectively, both the real and simulated models achieved significantly enhanced image quality from the accelerated scans in the 2x and 3x groups (FID: 0.15 ± 0.18, 0.18 ± 0.18 vs. 0.47 ± 0.34; 0.19 ± 0.23, 0.20 ± 0.22 vs. 0.98 ± 0.59. LPIPS: 0.17 ± 0.05, 0.16 ± 0.04 vs. 0.19 ± 0.05; 0.18 ± 0.05, 0.19 ± 0.05 vs. 0.23 ± 0.04. All P < 0.05). The count-level consistency with the 1x images was excellent for all four sets of model-generated images (P < 0.0001).ConclusionsUltrafast 2x speed (real and simulated) images achieved comparable diagnostic value to that of standardly acquired images, but the simulation algorithm does not necessarily reflect real data.

Ellipse-fitting in Mock Images of TNG50 Barred Galaxies

Recent studies have utilized the TNG50 simulation to explore barred galaxy morphology. The ellipse-fitting method is commonly used to assess properties of the isophotes in the central region of the disk. This work adapts the ellipse-fitting method to simulated images from TNG50, dealing with the issue of excessively pronounced ellipticities in central regions, whether in mass distribution maps or simulated radiative transfer images. To solve this problem, we introduce synthetic realism in the form of convolution with point spread functions, correcting the misbehavior in central ellipticities. These improvements simplify the application of ellipse-fitting to barred galaxies in the TNG50 simulation, enabling more accurate analysis of properties such as bar length and ellipticity, which would otherwise be more difficult to measure accurately. Thus, we conclude that when measuring shapes of simulated barred galaxies, one should apply realistic smoothing, otherwise the inner ellipticities will not be comparable to observations.

U²PNet: An Unsupervised Underwater Image-Restoration Network Using Polarization.

This article presents U 2PNet, a novel unsupervised underwater image restoration network using polarization for improving signal-to-noise ratio and image quality in underwater imaging environments. Traditional methods for underwater image restoration using polarization require specific cues or pairs of underwater polarization datasets, which limit their practical applications. Our proposed method requires only one mosaicked polarized image of the scene and does not require datasets for pretraining or specific cues. We design two subnetworks (T-net and B textsubscript ∞ -net) to accurately estimate the transmission map and background light, and unique nonreference loss functions to ensure effective restoration. Our experiments are based on an indoor polarization simulated dataset and a real polarization image dataset constructed from our underwater robotic platform equipped with polarization cameras. Experiment results demonstrate that our proposed method achieves state-of-the-art performance on both simulated and real underwater polarization images. The code and datasets will be available at https://github.com/polwork/U-2Pnet.

Maximizing Learning in Caring for Older Patients Through a Multi-Specialty Simulation Approach

Background: Simulation-based learning has been utilized in medical education and studied to enhance its educational impact since the 1960s (Hallinger, &amp; Wang, 2020). However, there is a lack of multispecialty simulations in the literature (Fisher, &amp; Walker, 2014; Age UK. 2023; NHS England, 2021; Romero-Ortuno, Stuck, &amp; Masud, 2022; Keijsers, et al., 2016). We developed and delivered simulations on Orthopaedics and Geriatric topics. Simulation mannequins, role players, imaging, and simulated clinical documentation were incorporated into scenarios. We evaluated the effectiveness of this approach on students’ knowledge and confidence when caring for older patients. Methods: Fourth-year medical students at the University of Nottingham received simulation-based teaching during their Geriatrics placement at Queens Medical Centre. Their knowledge and confidence levels were assessed before and after the simulations. We utilized six knowledge-based and six confidence-level questions mapped to their learning outcomes for hip fractures, pressure ulcers, and discharge planning stations. In addition, we asked students to provide scenario-specific feedback and their thoughts on whether the simulation workshop was pitched at the right level. Results: Students’ knowledge and confidence levels improved significantly following the simulation workshop. About 75% of the students displayed enhanced results in the knowledge-based category. In terms of their confidence level, there was improvement seen across all simulation stations, with most learners feeling four times more confident when comparing proportions pre and post-simulation. Furthermore, 99% of the students thought the simulations had positively impacted their learning. Conclusion: Our findings demonstrate the effectiveness of the multispecialty simulation approach in undergraduate geriatrics teaching. Given significant improvements in students’ knowledge, confidence levels, and positive feedback, we aim to continue delivering this multispecialty simulation-based teaching to our students. To measure the long-term efficacy of this approach, we can perhaps re-evaluate students learning after a month to assess the efficacy of this simulation workshop.

Chromatic cues for the sign of defocus in the peripheral retina.

Detecting optical defocus at the retina is crucial for accurate accommodation and emmetropization. However, the optical characteristics of ocular defocus are not fully understood. To bridge this knowledge gap, we simulated polychromatic retinal image quality by considering both the monochromatic wavefront aberrations and chromatic aberrations of the eye, both in the fovea and the periphery (nasal visual field). Our study revealed two main findings: (1) chromatic and monochromatic aberrations interact to provide a signal to the retina (chromatic optical anisotropy) to discern positive from negative defocus and (2) that chromatic optical anisotropy exhibited notable differences among refractive error groups (myopes, emmetropes and hyperopes). These findings could enhance our understanding of the underlying mechanisms of defocus detection and their subsequent implications for myopia control therapies. Further research is needed to explore the retinal architecture's ability to utilize the optical signals identified in this study.

Cross-Dimensional Attention Fusion Network for Simulated Single Image Super-Resolution

Cross-Dimensional Attention Fusion Network for Simulated Single Image Super-Resolution

Circular Polarization of Simulated Images of Black Holes

Models of the resolved Event Horizon Telescope (EHT) sources Sgr A* and M87* are constrained by observations at multiple wavelengths, resolutions, polarizations, and time cadences. In this paper, we compare unresolved circular polarization (CP) measurements to a library of models, where each model is characterized by a distribution of CP over time. In the library, we vary the spin of the black hole, the magnetic field strength at the horizon (i.e., both SANE and magnetically arrested disk or MAD models), the observer inclination, a parameter for the maximum ion–electron temperature ratio assuming a thermal plasma, and the direction of the magnetic field dipole moment. We find that Atacama Large Millimeter/submillimeter Array (ALMA) observations of Sgr A* are inconsistent with all edge-on (i = 90°) models. Restricting attention to the MAD models favored by earlier EHT studies of Sgr A*, we find that only models with magnetic dipole moment pointing away from the observer are consistent with ALMA data. We also note that in 26 of the 27 passing MAD models, the accretion flow rotates clockwise on the sky. We provide a table of the means and standard deviations of the CP distributions for all model parameters, along with their trends.

Curtaining artifacts generation on synthetic FIB-SEM data via Generative Adversarial Networks

FIB-SEM imaging stands out as an advanced method for capturing nanoscale structures. Throughout the image acquisition process, various artifacts emerge, including curtaining and charging artifacts. Effectively addressing these artifacts requires specialized algorithms tailored to their unique characteristics. Consequently, the development of algorithms demands simulated images used as benchmarks for validation. Simulating FIB-SEM images is a complex task, prompting the exploration of generative models as an alternative for simulation. We have adapted generative models to encompass curtaining artifacts, a feature challenging to replicate through conventional simulations. The resulting images demonstrate comparability with synthetically generated counterparts.

SAR Image Despeckling Based on Denoising Diffusion Probabilistic Model and Swin Transformer

The speckle noise inherent in synthetic aperture radar (SAR) imaging has long posed a challenge for SAR data processing, significantly affecting image interpretation and recognition. Recently, deep learning-based SAR speckle removal algorithms have shown promising results. However, most existing algorithms rely on convolutional neural networks (CNN), which may struggle to effectively capture global image information and lead to texture loss. Besides, due to the different characteristics of optical images and synthetic aperture radar (SAR) images, the results of training with simulated SAR data may bring instability to the real-world SAR data denoising. To address these limitations, we propose an innovative approach that integrates swin transformer blocks into the prediction noise network of the denoising diffusion probabilistic model (DDPM). By harnessing DDPM’s robust generative capabilities and the Swin Transformer’s proficiency in extracting global features, our approach aims to suppress speckle while preserving image details and enhancing authenticity. Additionally, we employ a post-processing strategy known as pixel-shuffle down-sampling (PD) refinement to mitigate the adverse effects of training data and the training process, which rely on spatially uncorrelated noise, thereby improving its adaptability to real-world SAR image despeckling scenarios. We conducted experiments using both simulated SAR image datasets and real SAR image datasets, evaluating our algorithm from subjective and objective perspectives. The visual results demonstrate significant improvements in noise suppression and image detail restoration. The objective results demonstrate that our method obtains state-of-the-art performance, which outperforms the second-best method by an average peak signal-to-noise ratio (PSNR) of 0.93 dB and Structural Similarity Index (SSIM) of 0.03, affirming the effectiveness of our approach.

Transformer for low concentration image denoising in magnetic particle imaging.

Objective.Magnetic particle imaging (MPI) is an emerging tracer-basedin vivoimaging technology. The use of MPI at low superparamagnetic iron oxide nanoparticle concentrations has the potential to be a promising area of clinical application due to the inherent safety for humans. However, low tracer concentrations reduce the signal-to-noise ratio of the magnetization signal, leading to severe noise artifacts in the reconstructed MPI images. Hardware improvements have high complexity, while traditional methods lack robustness to different noise levels, making it difficult to improve the quality of low concentration MPI images.Approach.Here, we propose a novel deep learning method for MPI image denoising and quality enhancing based on a sparse lightweight transformer model. The proposed residual-local transformer structure reduces model complexity to avoid overfitting, in which an information retention block facilitates feature extraction capabilities for the image details. Besides, we design a noisy concentration dataset to train our model. Then, we evaluate our method with both simulated and real MPI image data.Main results.Simulation experiment results show that our method can achieve the best performance compared with the existing deep learning methods for MPI image denoising. More importantly, our method is effectively performed on the real MPI image of samples with an Fe concentration down to 67μgFeml-1.Significance.Our method provides great potential for obtaining high quality MPI images at low concentrations.

Distributed Harmonization: Federated Clustered Batch Effect Adjustment and Generalization.

Independent and identically distributed (i.i.d.) data is essential to many data analysis and modeling techniques. In the medical domain, collecting data from multiple sites or institutions is a common strategy that guarantees sufficient clinical diversity, determined by the decentralized nature of medical data. However, data from various sites are easily biased by the local environment or facilities, thereby violating the i.i.d. rule. A common strategy is to harmonize the site bias while retaining important biological information. The COMBAT is among the most popular harmonization approaches and has recently been extended to handle distributed sites. However, when faced with situations involving newly joined sites in training or evaluating data from unknown/unseen sites, COMBAT lacks compatibility and requires retraining with data from all the sites. The retraining leads to significant computational and logistic overhead that is usually prohibitive. In this work, we develop a novel Cluster ComBat harmonization algorithm, which leverages cluster patterns of the data in different sites and greatly advances the usability of COMBAT harmonization. We use extensive simulation and real medical imaging data from ADNI to demonstrate the superiority of the proposed approach. Our codes are provided in https://github.com/illidanlab/distributed-cluster-harmonization.

Cross-modal Transfer Learning Based on an Improved CycleGAN Model for Accurate Kidney Segmentation in Ultrasound Images

ObjectiveDeep-learning algorithms have been widely applied in the field of automatic kidney ultrasound (US) image segmentation. However, obtaining a large number of accurate kidney labels clinically is very difficult and time-consuming. To solve this problem, we have proposed an efficient cross-modal transfer learning method to improve the performance of the segmentation network on a limited labeled kidney US dataset. MethodsWe aim to implement an improved image-to-image translation network called Seg-CycleGAN to generate accurate annotated kidney US data from labeled abdomen computed tomography images. The Seg-CycleGAN framework primarily consists of two structures: (i) a standard CycleGAN network to visually simulate kidney US from a publicly available labeled abdomen computed tomography dataset; (ii) and a segmentation network to ensure accurate kidney anatomical structures in US images. Based on the large number of simulated kidney US images and small number of real annotated kidney US images, we then aimed to employ a fine-tuning strategy to obtain better segmentation results. ResultsTo validate the effectiveness of the proposed method, we tested this method on both normal and abnormal kidney US images. The experimental results showed that the proposed method achieved a segmentation accuracy of 0.8548 in dice similarity coefficient on all testing datasets and 0.7622 on the abnormal testing dataset. ConclusionsCompared with existing data augmentation and transfer learning methods, the proposed method improved the accuracy and generalization of the kidney US image segmentation network on a limited number of training datasets. It therefore has the potential to significantly reduce annotation costs in clinical settings.

Depth estimation from monocular endoscopy using simulation and image transfer approach

Obtaining accurate distance or depth information in endoscopy is crucial for the effective utilization of navigation systems. However, due to space constraints, incorporating depth cameras into endoscopic systems is often impractical. Our goal is to estimate depth images directly from endoscopic images using deep learning. This study presents a three-step methodology for training a depth-estimation network model. Initially, simulated endoscopy images and corresponding depth maps are generated using Unity based on a colon surface model obtained from segmented computed tomography colonography data. Subsequently, a cycle generative adversarial network model is employed to enhance the realism of the simulated endoscopy images. Finally, a deep learning model is trained using the synthesized endoscopy images and depth maps to estimate depths accurately. The performance of the proposed approach is evaluated and compared against prior studies utilizing unsupervised training methods. The results demonstrate the superior precision of the proposed technique in estimating depth images within endoscopy. The proposed depth estimation method holds promise for advancing the field by enabling enhanced navigation, improved lesion marking capabilities, and ultimately leading to better clinical outcomes.

Elastic reweighted sparsity regularized sparse unmixing for hyperspectral image analysis

Sparse unmixing is a crucial component in the analysis of hyperspectral images because of its ability to sparsely estimate abundances with a spectral library of considerable scale. The existing sparse unmixing techniques tend to employ the sparsity from a single perspective, i.e., ℓ1 or ℓ2,1. In this paper, we accurately estimate abundances from element-wise and channel-wise sparsity perspectives, which forms a new elastic reweighted sparse regularized sparse unmixing method, termed ElSpaSU. The proposed ElSpaSU method leverages the abundance matrix simultaneously using ℓ1 and ℓ2,1 sparsity regularizations. To further promote sparsity in the abundances, the proposed ElSpaSU incorporates two types of iterative reweighting weights. We utilize the alternating direction method of multipliers (ADMM) framework to ensure guaranteed convergence in solving the optimization problem. Experimental outcomes from simulated and real hyperspectral images highlight the remarkable efficacy of the suggested ElSpaSU in contrast to competing approaches.