Low-quality Images Research Articles

Transformer with Hybrid Attention Mechanism for Stereo Endoscopic Video Super Resolution

With stereo cameras becoming widely used in invasive surgery systems, stereo endoscopic images provide important depth information for delicate surgical tasks. However, the small size of sensors and their limited lighting conditions lead to low-quality and low-resolution endoscopic images and videos. In this paper, we propose a stereo endoscopic video super-resolution method using transformer with a hybrid attention mechanism named HA-VSR. Stereo video SR aims to reconstruct high-resolution (HR) images from corresponding low-resolution (LR) videos. In our method, the stereo correspondence and temporal correspondence are incorporated into the HA-VSR model. Specifically, the Swin transformer architecture is utilized in proposed framework with hybrid attention mechanisms. The parallel attention mechanism is utilized by using the symmetry and consistency of left and right images, and the temporal attention mechanism is utilized by using the consistency of consecutive frames. Detailed quantitative evaluation and experiments on two datasets show the proposed model achieves advanced SR reconstruction performance, showing that the proposed stereo VSR framework outperforms alternative approaches.

Intraoperative Ultrasonography in Neurosurgery: Experience of an Institution

Abstract Objective Intraoperative ultrasound (IOUS) is an imaging method that can be used in various neurosurgical procedures. It assists in the removal of brain tumors, hematomas, in the insertion of ventricular catheters, and in spinal and peripheral nerve surgeries. The original studies using IOUS were performed with devices that produced lower-quality images, generating little interest in their use. The introduction of ultrasound devices that yielded higher-quality images associated with a relative low cost and ease of use rendered IOUS attractive. Thus, we started using it in multiple neurosurgical procedures, studying its practicality, efficacy and limitations. Materials and methods A retrospective case study on the use of IOUS in neurosurgery from August 2014 to December 2020 at a single institution, evaluating its practicality, efficacy and limitations. Results A total of 127 IOUS scans were performed in 112 patients aged 19 to 83 (mean: 53.8) years: 106 scans of the brain, 4 of the spinal cord, and 2 of the peripheral nerves. Brain tumors were the majority, with 86 cases (67.8%). The IOUS was unsatisfactory in 9 cases (7.1%), and there were no cases of infection related to the IOUS. Conclusion The IOUS is a dynamic, safe, and practical exam. It can be performed in a few minutes and repeated several times during the surgical procedure. It enables surgeons to observe the pulsation of tissues and vessels, the displacement of lumps in liquid collections, and to monitor the drainage of cysts and collections, in vivid, interactive and real-time images.

A Dual Convolutional Neural Network with Attention Mechanism for Thermal Infrared Image Enhancement

In industrial applications, thermal infrared images, which are commonly used, often suffer from issues such as low contrast and blurred details. Traditional image enhancement algorithms are limited in their effectiveness in improving the visual quality of thermal infrared images due to the specific nature of the application. Therefore, we propose a dual Convolutional Neural Network (CNN) combined with an attention mechanism to address the challenges of enhancing low-quality thermal infrared images and improving their visual quality. Firstly, we employ two parallel sub-networks to extract both global and local features. In one sub-network, we utilize a sparse mechanism incorporating dilated convolutions, while the other sub-network employs Feature Attention (FA) blocks based on channel attention and pixel attention. This architecture significantly enhances the feature extraction capability. The use of attention mechanisms allows the network to filter out irrelevant background information, enabling more flexible feature extraction. Finally, through a simple yet effective fusion block, we thoroughly integrate the extracted features to achieve an optimal fusion strategy, ensuring the highest quality enhancement of the final image. Extensive experiments on benchmark datasets and real images demonstrate that our proposed method outperforms other state-of-the-art models in terms of objective evaluation metrics and subjective assessments. The generated images also exhibit superior visual quality.

End-to-end memory-efficient reconstruction for cone beamCT.

Cone beam computed tomography (CBCT) plays an important role in many medical fields nowadays. Unfortunately, the potential of this imaging modality is hampered by lower image quality compared to the conventional CT, and producing accurate reconstructions remains challenging. A lot of recent research has been directed towards reconstruction methods relying on deep learning, which have shown great promise for various imaging modalities. However, practical application of deep learning to CBCT reconstruction is complicated by several issues, such as exceedingly high memory costs of deep learning methods when working with fully 3D data. Additionally, deep learning methods proposed in the literature are often trained and evaluated only on data from a specific region of interest, thus raising concerns about possible lack of generalization to other regions. In this work, we aim to address these limitations and propose LIRE: a learned invertible primal-dual iterative scheme for CBCTreconstruction. LIRE is a learned invertible primal-dual iterative scheme for CBCT reconstruction, wherein we employ a U-Net architecture in each primal block and a residual convolutional neural network (CNN) architecture in each dual block. Memory requirements of the network are substantially reduced while preserving its expressive power through a combination of invertible residual primal-dual blocks and patch-wise computations inside each of the blocks during both forward and backward pass. These techniques enable us to train on data with isotropic 2 mm voxel spacing, clinically-relevant projection count and detector panel resolution on current hardware with 24 GBvideo random access memory (VRAM). Two LIRE models for small and for large field-of-view (FoV)setting were trained and validated on a set of 260 + 22 thorax CT scans and tested using a set of 142 thorax CT scans plus an out-of-distribution dataset of 79 head and neck CT scans. For both settings, our method surpasses the classical methods and the deep learning baselines on both test sets. On the thorax CT set, our method achieves peak signal-to-noise ratio (PSNR) of 33.84 ± 2.28 for the small FoV setting and 35.14 ± 2.69 for the large FoV setting; U-Net baseline achieves PSNR of 33.08 ± 1.75 and 34.29 ± 2.71 respectively. On the head and neck CT set, our method achieves PSNR of 39.35 ± 1.75 for the small FoV setting and 41.21 ± 1.41 for the large FoV setting; U-Net baseline achieves PSNR of 33.08 ± 1.75 and 34.29 ± 2.71 respectively. Additionally, we demonstrate that LIRE can be finetuned to reconstruct high-resolution CBCT data with the same geometry but 1 mm voxel spacing and higher detector panel resolution, where it outperforms the U-Net baseline aswell. Learned invertible primal-dual schemes with additional memory optimizations can be trained to reconstruct CBCT volumes directly from the projection data with clinically-relevant geometry and resolution. Such methods can offer better reconstruction quality and generalization compared to classical deep learning baselines.

Ultrasound Plane-Wave Compressed Sensing Reconstruction Method Using Intra-frame and Inter-frame Joint Multi-hypothesis and Reference Frame Multi-hypothesis Predictions

Ultrasound plane-wave imaging has the advantage of high frame rate in addition to high data volume. High data sampling rates and large amounts of data storage can become bottlenecks in ultrasound system design. Although compressed sensing technology can help reduce the burden of sampling and transmission, it achieves relatively low image quality because of its reliance solely on signal sparsity. Therefore, we proposed reconstructing the ultrasound signal by applying additional prior knowledge, such as plane-wave imaging and its echo characteristics. Inspired by multi-hypothesis prediction methods in video compression coding, the plane-wave multi-hypothesis prediction compressed sensing reconstruction method was proposed to improve the accuracy of reconstructions. We applied multi-hypothesis prediction and residual reconstruction on the plane wave to enhance the quality of reconstruction and correct predicted values. Also, to acquire high-quality hypotheses, two hypothesis acquisition schemes were evaluated, constructing search windows on both preceding and subsequent frames as well as the reference frame. Compared with traditional reconstruction methods that rely on sparsity, multi-hypothesis prediction compressed sensing methods can reduce signal reconstruction errors and significantly eliminate image artifacts. Furthermore, by using improved hypotheses, signal reconstruction and image quality can be enhanced, resulting in higher contrast. Comparative simulation experimental results based on the publicly available Plane-Wave Imaging Challenge in Medical Ultrasound (PICMUS) and acoustic radiation force imaging data sets demonstrate that the proposed method outperforms other methods in both reconstruction errors and image quality. This helps to reduce the complexity of sampling and transmission of the ultrasound system.

DP-FishNet: Dual-path Pyramid Vision Transformer-based underwater fish detection network

The detection of underwater fish targets is critical for ecological monitoring and marine biodiversity research. However, underwater fish detection is typically constrained by problems including low image quality and variable underwater surroundings. On behalf of further improving the underwater fish detection accuracy in complex underwater environments, this paper proposes a dual-path (DP) Pyramid Vision Transformer (PVT) feature extraction network named DP-FishNet. The backbone network DP-PVT composed from the PVT Network is made up of two feature extraction paths. The first represents the Vision Transformer path, which extracts global features to enhance the distinction between the foreground and background of underwater images. The second is the convolutional neural network path, which enhances the accuracy of detecting small targets by extracting local features. Additionally, to more effectively utilize the feature information extracted by the network, this paper provides a promising solution to employ the content-aware reassembly of features (Carafe) in the feature pyramid network (FPN). The seesaw loss is utilized as a classification loss to address the problem of unbalanced samples caused by the gap in the number of fish populations. According to the experimental findings, the AP and AP50 of the DP-FishNet are 76.0% and 95.2%, respectively. In comparison to currently available advanced two-stage detection algorithms, the quantity of computation and parameters is reduced by approximately 40%. DP-FishNet strengthens the ability to extract global and local features from underwater images and enhances feature reuse. DP-FishNet can be utilized to detect fish targets in actual and complicated underwater habitats.

PTS-LSTM: Temperature prediction for fused filament fabrication using thermal image time series

In-situ thermography for Fuse Filament Fabrication (FFF) processes reveals the dynamic thermal behavior during printing. The data collected are thermal image time series. Their infrared (IR) intensity is visual evidence of heat-affected zone (HAZ) temperatures, which can be leveraged to train deep learning models, e.g., Long Short-Term Memory (LSTM), for real-time temperature prediction and process monitoring. Nonetheless, the data collection method and printing path may pose challenges for data modeling. Typically, the IR camera has a fixed position while the HAZ moves per the predetermined printing path. Consequently, the HAZ shifts in images and the features extracted from these thermal images show a “periodic” behavior over time. Such periodic patterns do not reflect any useful information about HAZ temperatures. Instead, they are noise hiding and interrupting the true temperature information, thus must be removed before using the data to train an LSTM model for temperature prediction. This study integrates a time series model, i.e., ARIMA, with Stacked LSTM to build a pTS-LSTM model that eliminates noisy patterns and predicts temperatures during FFF printing. The case study results show the outperformance of pTS-LSTM over conventional LSTM and classic Recurrent Neural Network models. pTS-LSTM is demonstrated to be promising for in-situ process monitoring with low-quality thermal images. In FFF practices, pTS-LSTM will be a preferred option over the commonly used deep learning models for thermal-image-based temperature prediction.

MS-CFNet: a multi-scale clinical studying-based and feature extraction-guided network for breast fibroadenoma segmentation in ultrasonography.

Segmenting breast tumors in ultrasonography is challenging due to the low image quality and presence of artifacts. Radiologists' studying and diagnosis skills are integrated with artificial intelligence to establish a clinical learning-based deep learning network in order to robustly extract and delineate features of breast fibroadenoma. The spatial local feature contrast (SLFC) module captures overall tumor contours, while the channel recursive gated attention (CRGA) module enhances edge perception through high-dimensional information interaction. Additionally, full-scale feature fusion and enhanced deep supervision are applied to improve model stability and performance. To achieve smoother boundaries, we introduce a new loss function (cosh-smooth) that penalizes and finely tunes tumor edges. Our dataset comprises 1016 clinical ultrasound images of breast fibroadenoma with labeled masks, alongside a publicly available dataset of 246 ones. Segmentation performance is evaluated using the Dice similarity coefficient (DSC) and mean intersection over union (MIOU). Extensive experiments demonstrate that our proposed MS-CFNet outperforms state-of-the-art methods. Compared to TransUNet as a baseline model, MS-CFNet improves by 1.47% in DSC and 2.56% in MIOU. The promising result of MS-CFNet is attributed to the integration of radiologists' clinical diagnosis procedure and the bionic mindset, enhancing the network's ability to recognize and segment breast fibroadenomas effectively.

Threshold-Based BRISQUE-Assisted Deep Learning for Enhancing Crack Detection in Concrete Structures.

Automated visual inspection has made significant advancements in the detection of cracks on the surfaces of concrete structures. However, low-quality images significantly affect the classification performance of convolutional neural networks (CNNs). Therefore, it is essential to evaluate the suitability of image datasets used in deep learning models, like Visual Geometry Group 16 (VGG16), for accurate crack detection. This study explores the sensitivity of the BRISQUE method to different types of image degradations, such as Gaussian noise and Gaussian blur. By evaluating the performance of the VGG16 model on these degraded datasets with varying levels of noise and blur, a correlation is established between image degradation and BRISQUE scores. The results demonstrate that images with lower BRISQUE scores achieve higher accuracy, F1 score, and Matthew's correlation coefficient (MCC) in crack classification. The study proposes the implementation of a BRISQUE score threshold (BT) to optimise training and testing times, leading to reduced computational costs. These findings have significant implications for enhancing accuracy and reliability in automated visual inspection systems for crack detection and structural health monitoring (SHM).

Deep Learning-Based T2-Weighted MR Image Quality Assessment and Its Impact on Prostate Cancer Detection Rates.

Image quality evaluation of prostate MRI is important for successful implementation of MRI into localized prostate cancer diagnosis. To examine the impact of image quality on prostate cancer detection using an in-house previously developed artificial intelligence (AI) algorithm. Retrospective. 615 consecutive patients (median age 67 [interquartile range [IQR]: 61-71] years) with elevated serum PSA (median PSA 6.6 [IQR: 4.6-9.8] ng/mL) prior to prostate biopsy. 3.0T/T2-weighted turbo-spin-echo MRI, high b-value echo-planar diffusion-weighted imaging, and gradient recalled echo dynamic contrast-enhanced. Scans were prospectively evaluated during clinical readout using PI-RADSv2.1 by one genitourinary radiologist with 17 years of experience. For each patient, T2-weighted images (T2WIs) were classified as high-quality or low-quality based on evaluation of both general distortions (eg, motion, distortion, noise, and aliasing) and perceptual distortions (eg, obscured delineation of prostatic capsule, prostatic zones, and excess rectal gas) by a previously developed in-house AI algorithm. Patients with PI-RADS category 1 underwent 12-core ultrasound-guided systematic biopsy while those with PI-RADS category 2-5 underwent combined systematic and targeted biopsies. Patient-level cancer detection rates (CDRs) were calculated for clinically significant prostate cancer (csPCa, International Society of Urological Pathology Grade Group ≥2) by each biopsy method and compared between high- and low-quality images in each PI-RADS category. Fisher's exact test. Bootstrap 95% confidence intervals (CI). A P value <0.05 was considered statistically significant. 385 (63%) T2WIs were classified as high-quality and 230 (37%) as low-quality by AI. Targeted biopsy with high-quality T2WIs resulted in significantly higher clinically significant CDR than low-quality images for PI-RADS category 4 lesions (52% [95% CI: 43-61] vs. 32% [95% CI: 22-42]). For combined biopsy, there was no significant difference in patient-level CDRs for PI-RADS 4 between high- and low-quality T2WIs (56% [95% CI: 47-64] vs. 44% [95% CI: 34-55]; P = 0.09). Higher quality T2WIs were associated with better targeted biopsy clinically significant cancer detection performance for PI-RADS 4 lesions. Combined biopsy might be needed when T2WI is lower quality. 2 TECHNICAL EFFICACY: Stage 1.

Deep learning-based algorithm for postoperative glioblastoma MRI segmentation: a promising new tool for tumor burden assessment

ObjectiveClinical and surgical decisions for glioblastoma patients depend on a tumor imaging-based evaluation. Artificial Intelligence (AI) can be applied to magnetic resonance imaging (MRI) assessment to support clinical practice, surgery planning and prognostic predictions. In a real-world context, the current obstacles for AI are low-quality imaging and postoperative reliability. The aim of this study is to train an automatic algorithm for glioblastoma segmentation on a clinical MRI dataset and to obtain reliable results both pre- and post-operatively.MethodsThe dataset used for this study comprises 237 (71 preoperative and 166 postoperative) MRIs from 71 patients affected by a histologically confirmed Grade IV Glioma. The implemented U-Net architecture was trained by transfer learning to perform the segmentation task on postoperative MRIs. The training was carried out first on BraTS2021 dataset for preoperative segmentation. Performance is evaluated using DICE score (DS) and Hausdorff 95% (H95).ResultsIn preoperative scenario, overall DS is 91.09 (± 0.60) and H95 is 8.35 (± 1.12), considering tumor core, enhancing tumor and whole tumor (ET and edema). In postoperative context, overall DS is 72.31 (± 2.88) and H95 is 23.43 (± 7.24), considering resection cavity (RC), gross tumor volume (GTV) and whole tumor (WT). Remarkably, the RC segmentation obtained a mean DS of 63.52 (± 8.90) in postoperative MRIs.ConclusionsThe performances achieved by the algorithm are consistent with previous literature for both pre-operative and post-operative glioblastoma’s MRI evaluation. Through the proposed algorithm, it is possible to reduce the impact of low-quality images and missing sequences.

Application of mask images of contrast-enhanced MR angiography to detect carotid intraplaque hemorrhage in patients with moderate to severe symptomatic and asymptomatic carotid stenosis

PurposeCarotid intraplaque hemorrhage (IPH) on MRI predicts stroke. Magnetization-prepared rapid acquisition gradient (MP-RAGE) is widely used to detect IPH. CE-MRA is used routinely to assess stenosis. Initial studies indicated that IPH can be identified on mask images of CE-MRA, while Time-of-Flight (TOF) images were reported to have high specificity but lower sensitivity. We investigated the diagnostic accuracy of detecting IPH on mask images of CE-MRA and TOF. MethodsThirty-six patients with ≥ 50% stenosis enrolled in the ongoing 2nd European Carotid Surgery Trial underwent carotid MRI. A 5-point quality score was used. Inter-observer agreement between two independent readers was determined. The sensitivity and specificity of IPH detection on mask MRA and TOF were calculated with MP-RAGE as a reference standard. ResultsOf the 36 patients included in the current analysis, 66/72 carotid arteries could be scored. The inter-observer agreements for identifying IPH on MP-RAGE, mask, and TOF were outstanding (κ: 0.93, 0.96, and 0.85). The image quality of mask (1.42 ± 0.66) and TOF (2.42 ± 0.66) was significantly lower than MP-RAGE (3.47 ± 0.61). When T1w images were used to delineate the outer carotid wall, very high specificities (>95%) of IPH detection on mask and TOF images were found, while the sensitivity was high for mask images (>81%) and poor for TOF (50–60%). Without these images, the specificity was still high (>97%), while the sensitivity reduced to 62–71%. ConclusionDespite the lower image quality, routinely acquired mask images from CE-MRA, but not TOF, can be used as an alternative to MP-RAGE images to visualize IPH.

Image denoising and deringing for fourier single-pixel imaging based on upgraded weighted nuclear norm minimization

Single-pixel imaging techniques have been extensively studied because of the low requirements for the detector resolution and the low equipment cost. However, a large amount of noise and ringing commonly occur in the reconstructed images via previous single-pixel imaging methods at ultra-low sampling rates, which limits the real applications. In this paper, a novel Fourier single-pixel imaging method is proposed to reconstruct high-quality single-pixel imaging images at ultra-low sampling rates, which involves the stages of Fourier single-pixel imaging, block matching, upgraded weight nuclear norm minimization, and aggregation. To decrease the computational burden while important image details, block matching and aggregation are employed for the blocks divided from the low-quality image reconstructed by Fourier single-pixel imaging. To adapt the proposed upgraded weight nuclear norm minimization to different contaminated images, an adaptive scheme for the iterative regularization parameter is proposed to update the traditional weight nuclear norm minimization. Experimental results indicate that the proposed Fourier single-pixel imaging method based on upgraded weight nuclear norm minimization exhibits the capacity to reliably and effectively reconstruct high-quality images for single-pixel imaging at low sampling rates, even at the sampling rate of 1 %. Experimental results also demonstrate that the proposed method performs better image reconstruction than some existing single-pixel imaging.

Image Processing Algorithm Design of Agricultural Internet of Things Platform Based on Big Data Analysis

Abstract To solve the problems of low signal to noise ratio (SNR), poor processing effect, and long processing times for traditional image processing algorithms, an image processing algorithm for the agricultural Internet of Things platform based on big data analysis was designed. The big data analysis method was used for agricultural low-quality image recognition, and the Internet of Things platform uses sparse representation and a complete dictionary of low-quality image denoising processing with an improved histogram equalization method to enhance the image denoising result and get a high-quality image. Through this, the image processing algorithm design of the agricultural Internet of Things platform was completed. Experimental results show that the algorithm has high SNR and a good image processing effect, and the image processing time is always below 0.7 s, which meets research expectations.

A Prior-Knowledge-Based Generative Adversarial Network for Unsupervised Satellite Cloud Image Restoration

High-quality satellite cloud images are of great significance for weather diagnosis and prediction. However, many of these images are often degraded due to relative motion, atmospheric turbulence, instrument noise, and other factors. In the satellite imaging process, the degradation also cannot be completely corrected. Therefore, it is necessary to further improve the satellite cloud image quality for real applications. In this study, we propose an unsupervised image restoration model with a two-stage network, in which the first stage, named the Prior-Knowledge-based Generative Adversarial Network (PKKernelGAN), aims to learn the blur kernel, and the second stage, named the Zero-Shot Deep Residual Network (ZSResNet), aims to improve the image quality. In PKKernelGAN, we propose a satellite cloud imaging loss function, which is a novel objective function that brings optimization of a generative model into the prior-knowledge domain. In ZSResNet, we build a dataset which contains the original satellite cloud image as high-quality images (HQ) paired with low-quality images (LQ) generated by the blur kernel learning from PKKernelGAN. The above innovations lead to a more efficient local structure in satellite cloud image restoration. The original dataset of our experiment is from the Sunflower 8 satellite provided by the Japan Meteorological Agency. This dataset is divided into training and testing sets to train and test PKKernelGAN. Then, ZSResNet is trained by the “LQ–HQ” image pairs generated by PKKernelGAN. Compared with other supervised and unsupervised deep learning models for image restoration, our model has a better performance. Extensive experiments have demonstrated that our proposed model can achieve better performance on different datasets.

A Lightweight Diabetic Retinopathy Detection Model Using a Deep-Learning Technique.

Diabetic retinopathy (DR) is a severe complication of diabetes. It affects a large portion of the population of the Kingdom of Saudi Arabia. Existing systems assist clinicians in treating DR patients. However, these systems entail significantly high computational costs. In addition, dataset imbalances may lead existing DR detection systems to produce false positive outcomes. Therefore, the author intended to develop a lightweight deep-learning (DL)-based DR-severity grading system that could be used with limited computational resources. The proposed model followed an image pre-processing approach to overcome the noise and artifacts found in fundus images. A feature extraction process using the You Only Look Once (Yolo) V7 technique was suggested. It was used to provide feature sets. The author employed a tailored quantum marine predator algorithm (QMPA) for selecting appropriate features. A hyperparameter-optimized MobileNet V3 model was utilized for predicting severity levels using images. The author generalized the proposed model using the APTOS and EyePacs datasets. The APTOS dataset contained 5590 fundus images, whereas the EyePacs dataset included 35,100 images. The outcome of the comparative analysis revealed that the proposed model achieved an accuracy of 98.0 and 98.4 and an F1 Score of 93.7 and 93.1 in the APTOS and EyePacs datasets, respectively. In terms of computational complexity, the proposed DR model required fewer parameters, fewer floating-point operations (FLOPs), a lower learning rate, and less training time to learn the key patterns of the fundus images. The lightweight nature of the proposed model can allow healthcare centers to serve patients in remote locations. The proposed model can be implemented as a mobile application to support clinicians in treating DR patients. In the future, the author will focus on improving the proposed model's efficiency to detect DR from low-quality fundus images.

CIPS-3D++: End-to-End Real-Time High-Resolution 3D-Aware GANs for GAN Inversion and Stylization.

Style-based GANs achieve state-of-the-art results for generating high-quality images, but lack explicit and precise control over camera poses. Recently proposed NeRF-based GANs have made great progress towards 3D-aware image generation. However, the methods either rely on convolution operators which are not rotationally invariant, or utilize complex yet suboptimal training procedures to integrate both NeRF and CNN sub-structures, yielding un-robust, low-quality images with a large computational burden. On top of our open-source CIPS-3D framework 1https://github.com/PeterouZh/CIPS-3D, this paper presents an upgraded version called CIPS-3D++, aiming at high-robust, high-resolution and high-efficiency 3D-aware GANs. On the one hand, our basic model CIPS-3D, encapsulated in a style-based architecture, features a shallow NeRF-based 3D shape encoder as well as a deep MLP-based 2D image decoder, achieving robust image generation/editing with rotation-invariance. On the other hand, our proposed CIPS-3D++, inheriting the rotational invariance of CIPS-3D, together with geometric regularization and upsampling operations, encourages high-resolution high-quality image generation/editing with great computational efficiency. Trained on raw single-view images, without any bells and whistles, CIPS-3D++ sets new records for 3D-aware image synthesis, with an impressive FID of 3.2 on FFHQ at the 1024×1024 resolution. In the meantime, CIPS-3D++ runs efficiently and enjoys a low GPU memory footprint so that it can be trained end-to-end on high-resolution images directly, in contrast to previous alternate/progressive methods. Based on the infrastructure of CIPS-3D++, we propose a 3D-aware GAN inversion algorithm named FlipInversion, which can reconstruct the 3D object from a single-view image. We also provide a 3D-aware stylization method for real images based on CIPS-3D++ and FlipInversion. In addition, we analyze the problem of mirror symmetry suffered in training, and solve it by introducing an auxiliary discriminator for the NeRF network. Overall, CIPS-3D++ provides a strong base model that can serve as a testbed for transferring GAN-based image editing methods from 2D to 3D. Our open-source project as well as accompanying demo videos can be found online 2https://github.com/PeterouZh/CIPS-3Dplusplus.

Pseudo CT Images Synthesized from CBCT Based on Unsupervised DDPM in Cervical Cancer Radiotherapy

The low image quality and inaccurate HU value of cone beam computed tomography (CBCT) restrict its further application in cervical cancer radiotherapy. A new unsupervised based denoising diffusion probabilistic model (DDPM) was proposed to synthesize pseudo-CT images from CBCT. CBCT and CT images of 120 patients with cervical cancer were selected. The proposed DDPM network with condition and iterative mechanism was used for data training and testing between two image domains. In the training process, the model first obtained coarse pseudo-CT images with Gaussian noise through a diffusion process. Then, with the real CT images as the training target, the noise images were nonlinearly mapped to the domain of the CT images through the inverse diffusion process, and the fine pseudo-CT images were obtained. We repeated the above steps and gradually increased the number of noise diffusion. When the image difference value is less than the threshold, the training of the model is terminated and the pseudo-CT images are output. In the testing stage, each pseudo-CT generated was compared against the real CT image of the same patient based on the metrics of peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and normalized mean absolute error (NMAE). In terms of anatomical verification, the PSNR (dB), SSIM (%) and NCC values between the pseudo-CT images obtained based on DDPM and the real CT images were presented as mean (standard deviation), which were 31.92(0.46), 86.69(4.55) and 0.0106(0.002) respectively. Compared with CBCT, the accuracy of the three metric values has been improved 15.6%, 14.2% and 23.3% respectively. For the metric values of the pseudo-CT images obtained based on the U-Net and CycleGAN models, the results synthesized based on the proposed model were paired with T-tests, the p values were all less than 0.05, and the differences were statistically significant. The pseudo-CT images obtained based on the DDPM were close to the real CT images in anatomy. The pseudo-CT images synthesized by the proposed DDPM network have good application prospects in cervical cancer radiotherapy.

Machine learning super-resolution of laboratory CT images in all-solid-state batteries using synchrotron radiation CT as training data

High-performance all-solid-state lithium-ion batteries require observation, control, and optimization of the electrode structure. X-ray computational tomography (CT) is an effective nondestructive method for observing the electrode structure in three dimensions. However, the limited availability of synchrotron radiation CT, which offers high-resolution imaging with a high signal-to-noise ratio, makes it difficult to conduct experiments and restricts the use of X-ray CT in battery development. Conversely, laboratory CT systems are widely available, but they use X-rays emitted from a metal target, resulting in lower image quality and resolution compared with synchrotron radiation CT. This study explores a method for achieving comparable resolution in laboratory CT images of all-solid-state batteries to that of synchrotron radiation CT. Our method involves using the synchrotron radiation CT images as training data for machine learning super-resolution. The results demonstrate that, by employing an appropriate machine learning algorithm and activation function, along with a sufficiently deep network, the image quality of laboratory CT becomes equivalent to that of synchrotron radiation CT.

Feasibility of artificial intelligence-driven interfractional monitoring of organ changes by mega-voltage computed tomography in intensity-modulated radiotherapy of prostate cancer.

High-dose radiotherapy (RT) for localized prostate cancer requires careful consideration of target position changes and adjacent organs-at-risk (OARs), such as the rectum and bladder. Therefore, daily monitoring of target position and OAR changes is crucial in minimizing interfractional dosimetric uncertainties. For efficient monitoring of the internal condition of patients, we assessed the feasibility of an auto-segmentation of OARs on the daily acquired images, such as megavoltage computed tomography (MVCT), via a commercial artificial intelligence (AI)-based solution in this study. We collected MVCT images weekly during the entire course of RT for 100 prostate cancer patients treated with the helical TomoTherapy system. Based on the manually contoured body outline, the bladder including prostate area, and rectal balloon regions for the 100 MVCT images, we trained the commercially available fully convolutional (FC)-DenseNet model and tested its auto-contouring performance. Based on the optimally determined hyperparameters, the FC-DenseNet model successfully auto-contoured all regions of interest showing high dice similarity coefficient (DSC) over 0.8 and a small mean surface distance (MSD) within 1.43 mm in reference to the manually contoured data. With this well-trained AI model, we have efficiently monitored the patient's internal condition through six MVCT scans, analyzing DSC, MSD, centroid, and volume differences. We have verified the feasibility of utilizing a commercial AI-based model for auto-segmentation with low-quality daily MVCT images. In the future, we will establish a fast and accurate auto-segmentation and internal organ monitoring system for efficiently determining the time for adaptive replanning.