U-Net Structure Research Articles

Melt pool super solution reconstruction based on dual path deep learning for laser powder bed fusion monitoring

Melt pool monitoring in laser powder bed fusion (L-PBF) is an important foundation for process control and melting dynamics research. However, due to hardware limitations and the intense metallurgical phenomena during the melting process, the quality of the melt pool monitoring images cannot be guaranteed. This paper proposes a deep learning-based melt pool image super-resolution (SR) reconstruction method based on the particularity of melt pool images. It is an innovative dual-path structure. The first branch utilizes residual-in-residual structures and the efficient channel attention-Net attention mechanism to achieve the adaptive feature extraction of high-frequency information, thereby capturing effective melt pool boundary information. The second branch uses the U-Net structure to address the low utilization of overall characteristics of the melt pool in the chain-based SR network. In addition to the universal peak signal-to-noise ratio and structural similarity index measure metrics, the reconstruction accuracy of melt pool contour features is used to measure model performance, which intuitively reflects the significance of this work for melt pool monitoring. The results demonstrate that the proposed SR reconstruction method enhances the resolution and clarity of melt pool images. Furthermore, SR reconstruction of the melt pool effectively reduces errors in extracting melt pool features. This method provides a network paradigm for high-precision L-PBF monitoring. It integrates important boundary information and overall morphology features of the melt pool through a dual path structure, thereby achieving reliable SR reconstruction. This research will contribute to low-cost in situ monitoring of L-PBF and subsequent process control.

Learnable Resized and Laplacian-Filtered U-Net: Better Road Marking Extraction and Classification on Sparse-Point-Cloud-Derived Imagery

High-definition (HD) maps for autonomous driving rely on data from mobile mapping systems (MMS), but the high cost of MMS sensors has led researchers to explore cheaper alternatives like low-cost LiDAR sensors. While cost effective, these sensors produce sparser point clouds, leading to poor feature representation and degraded performance in deep learning techniques, such as convolutional neural networks (CNN), for tasks like road marking extraction and classification, which are essential for HD map generation. Examining common image segmentation workflows and the structure of U-Net, a CNN, reveals a source of performance loss in the succession of resizing operations, which further diminishes the already poorly represented features. Addressing this, we propose improving U-Net’s ability to extract and classify road markings from sparse-point-cloud-derived images by introducing a learnable resizer (LR) at the input stage and learnable resizer blocks (LRBs) throughout the network, thereby mitigating feature and localization degradation from resizing operations in the deep learning framework. Additionally, we incorporate Laplacian filters (LFs) to better manage activations along feature boundaries. Our analysis demonstrates significant improvements, with F1-scores increasing from below 20% to above 75%, showing the effectiveness of our approach in improving road marking extraction and classification from sparse-point-cloud-derived imagery.

Seismic Random Noise Attenuation Using DARE U-Net

Seismic data processing plays a pivotal role in extracting valuable subsurface information for various geophysical applications. However, seismic records often suffer from inherent random noise, which obscures meaningful geological features and reduces the reliability of interpretations. In recent years, deep learning methodologies have shown promising results in performing noise attenuation tasks on seismic data. In this research, we propose modifications to the standard U-Net structure by integrating dense and residual connections, which serve as the foundation of our approach named the dense and residual (DARE U-Net) network. Dense connections enhance the receptive field and ensure that information from different scales is considered during the denoising process. Our model implements local residual connections between layers within the encoder, which allows earlier layers to directly connect with deep layers. This promotes the flow of information, allowing the network to utilize filtered and unfiltered input. The combined network mechanisms preserve the spatial information loss during the contraction process so that the decoder can locate the features more accurately by retaining the high-resolution features, enabling precise location in seismic image denoising. We evaluate this adapted architecture by applying synthetic and real data sets and calculating the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). The effectiveness of this method is well noted.

Hybrid Space Calibrated 3D Network of Diffractive Hyperspectral Optical Imaging Sensor.

Diffractive multispectral optical imaging plays an essential role in optical sensing, which typically suffers from the image blurring problem caused by the spatially variant point spread function. Here, we propose a novel high-quality and efficient hybrid space calibrated 3D network "HSC3D" for spatially variant diffractive multispectral imaging that utilizes the 3D U-Net structure combined with space calibration modules of magnification and rotation effects to achieve high-accuracy eight-channel multispectral restoration. The algorithm combines the advantages of the space calibrated module and U-Net architecture with 3D convolutional layers to improve the image quality of diffractive multispectral imaging without the requirements of complex equipment modifications and large amounts of data. A diffractive multispectral imaging system is established by designing and manufacturing one diffractive lens and four refractive lenses, whose monochromatic aberration is carefully corrected to improve imaging quality. The mean peak signal-to-noise ratio and mean structural similarity index of the reconstructed multispectral images are improved by 3.33 dB and 0.08, respectively, presenting obviously improved image quality compared with a typical Unrolled Network algorithm. The new algorithm with high space calibrated ability and imaging quality has great application potential in diffraction lens spectroscopy and paves a new method for complex practical diffractive multispectral image sensing.

Self-supervised monocular depth estimation based on improved dense network and wavelet decomposition

Traditional self-supervised monocular depth estimation models do not adequately extract and fuse shallow features, which can easily lead to problems such as missed detection of small objects and blurred object edges. To address the above problems, this paper proposes a self-supervised monocular depth estimation model based on improved dense network and wavelet decomposition. The model follows the structure of U-net, in which the encoder adopts an improved dense connection method to improve the feature extraction and fusion capabilities of the encoder; at the same time, a detail enhancement module is added to the jump connection to further refine and integrate the multi-scale features output by the encoder; finally, the wavelet decomposition method is introduced in the decoder, forcing the decoder to pay more attention to high-frequency information and achieve refined processing of image edges. Experimental results show that the depth estimation model proposed in this paper has a stronger ability to capture small object features, and the edges of the generated depth map are clearer and more accurate.

Bearing fault diagnostic framework under unknown working conditions based on condition-guided diffusion model

Fault diagnosis is of vital importance in ensuring the safety of smart manufacturing. Current diagnostic methodologies require data spanning various working conditions. However, industrial settings offer scarce bearing failure data, leading to failure or degradation of performance of traditional intelligent diagnostic methods. Obtaining a complete sample of industrial environments is intensive and unrealistic. Acquiring a comprehensive sample of industrial environments is both resource-intensive and impractical. To handle this situation, an unknown condition diagnosis framework based on diffusion model (DiffUCD) is proposed, effectively integrating the generation capability of the diffusion model and learning from the condition-guided information (CGI), Specifically, signals under limited working conditions are gradually convert to noise through a forward noising process. Then, CGDiffusion reconstructs signals from the noise by a reverse denoising process. In addition, a condition-guided embedding UNet (CGE-UNet) structure is designed to extract CGI for noise level prediction during the reverse process. Moreover, an Unsupervised Clustering Filter (UCFilter) is constructed to select the qualified signals after generation. Signals under unknown working condition can be generated with specialized CGI. Ultimately, extensive experiments and validations on two public bearing datasets (SDUST and PU) are carried out, which validate the effectiveness of our method compared with the state-of-the-art baselines and the hyperparameter analysis confirms the advances of DiffUCD.

F.sh: A 3D Recurrent Residual Attention U-Net for Automated Multiple Sclerosis Lesion Segmentation

Multiple sclerosis (MS) is an autoimmune disease affecting the central nervous system, characterized by lesions in the brain and spinal cord. Accurate detection and localization of these lesions on MRI scans is crucial for diagnosis and monitoring disease progression. Manual segmentation is time-consuming and prone to inter-rater variability. This study proposes F.sh (3DR2AUNet), a novel deep learning architecture for automated MS lesion segmentation. F.sh combines 3D recurrent residual blocks, attention gates, and the U-Net structure to effectively capture lesion features. The model was trained and evaluated using a comprehensive approach, including patch-based preprocessing, data augmentation, and a composite loss function combining Binary Cross-Entropy and 3D Dice Loss. Experimental results demonstrate the superior performance of F.sh compared to baseline methods, achieving a Dice score of 0.92. The proposed approach has the potential to assist radiologists in the accurate and efficient assessment of MS lesion burden.

Metric learning guided sinogram denoising for cone beam CT enhancement.

Cone beam computed tomography (CBCT) is a widely available modality, but its clinical utility has been limited by low detail conspicuity and quantitative accuracy. Convenient post-reconstruction denoising is subject to back projected patterned residual, but joint denoise-reconstruction is typically computationally expensive and complex. In this study, we develop and evaluate a novel Metric-learning guided wavelet transform reconstruction (MEGATRON) approach to enhance image domain quality with projection-domain processing. Projection domain based processing has the benefit of being simple, efficient, and compatible with various reconstruction toolkit and vendor platforms. However, they also typically show inferior performance in the final reconstructed image, because the denoising goals in projection and image domains do not necessarily align. Motivated by these observations, this work aims to translate the demand for quality enhancement from the quantitative image domain to the more easily operable projection domain. Specifically, the proposed paradigm consists of a metric learning module and a denoising network module. Via metric learning, enhancement objectives on the wavelet encoded sinogram domain data are defined to reflect post-reconstruction image discrepancy. The denoising network maps measured cone-beam projection to its enhanced version, driven by the learnt objective. In doing so, the denoiser operates in the convenient sinogram to sinogram fashion but reflects improvement in reconstructed image as the final goal. Implementation-wise, metric learning was formalized as optimizing the weighted fitting of wavelet subbands, and a res-Unet, which is a Unet structure with residual blocks, was used for denoising. To access quantitative reference, cone-beam projections were simulated using the X-ray based Cancer Imaging Simulation Toolkit (XCIST). In both learning modules, a data set of 123 human thoraxes, which was from Open-Source Imaging Consortium (OSIC) Pulmonary Fibrosis Progression challenge, was used. Reconstructed CBCT thoracic images were compared against ground truth FB and performance was assessed in root mean square error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM). MEGATRON achieved RMSE in HU value, PSNR, and SSIM were 30.97±4.25, 37.45±1.78, and 93.23±1.62, respectively. These values are on par with reported results from sophisticated physics-driven CBCT enhancement, demonstrating promise and utility of the proposed MEGATRON method. We have demonstrated that incorporating the proposed metric learning into sinogram denoising introduces awareness of reconstruction goal and improves final quantitative performance. The proposed approach is compatible with a wide range of denoiser network structures and reconstruction modules, to suit customized need or further improve performance.

S2VSNet: Single stage V-shaped network for image deraining & dehazing

Producing high-quality, noise-free images from noisy or hazy inputs relies on essential tasks such as single image deraining and dehazing. In many advanced multi-stage networks, there is often an imbalance in contextual information, leading to increased complexity. To address these challenges, we propose a simplified method inspired by a U-Net structure, resulting in the “Single-Stage V-Shaped Network” (S2VSNet), capable of handling both deraining and dehazing tasks. A key innovation in our approach is the introduction of a Feature Fusion Module (FFM), which facilitates the sharing of information across multiple scales and hierarchical layers within the encoder-decoder structure. As the network progresses towards deeper layers, the FFM gradually integrates insights from higher levels, ensuring that spatial details are preserved while contextual feature maps are balanced. This integration enhances the image processing capability, producing noise-free, high-quality outputs. To maintain efficiency and reduce system complexity, we replaced or removed several non-essential non-linear activation functions, opting instead for simple multiplication operations. Additionally, we introduced a “Multi-Head Attention Integrated Module” (MHAIM) as an intermediary layer between encoder-decoder levels. This module addresses the limited receptive fields of traditional Convolutional Neural Networks (CNNs), allowing for the capture of more comprehensive feature-map information. Our focus on deraining and dehazing led to extensive experiments on a wide range of synthetic and real-world datasets. To further validate the robustness of our network, we implemented S2VSNet on a low-end edge device, achieving deraining in 2.46 seconds.

MLFEU-NET: A Multi-scale Low-level Feature Enhancement Unet for breast lesions segmentation in ultrasound images

Breast lesions constantly threaten the health of females. Segmentation methods of breast lesions are important for clinical diagnosis, and neural networks have been widely used and played a significant role in this field. However, false detections and miss detections still exist due to the variability in the shape and location of breast lesions, artifacts and noise in breast ultrasound (BUS) images, and structural defects in conventional segmentation networks. In this paper, a multi-scale low-level feature enhancement Unet (MLFEU-net) structure is presented, consisting of the U-net structure, low-level feature enhancement block (LFEB), and a parallel multiscale feature fusion (PMFF) module. Specifically, LFEB enhances the detail information during shallow downsampling and further feature selection. Meanwhile, in the neck of Unet, PMFF module uses different scales of dilation convolution to provide different sizes of sensory fields, and to efficiently merge shallow and deep features filtered, leads to more accurate segmentation results. To evaluate the MLFEU-net’s segmentation ability, five quantitative metrics were used on two breast ultrasound datasets, and it was compared with several advanced segmentation techniques. The results illustrate that our approach surpasses other methods in performance. Moreover, robustness experiments validate the effectiveness of our approach in achieving robust segmentation of breast lesions.

MS-UNet: Multi-Scale Nested UNet for Medical Image Segmentation with Few Training Data Based on an ELoss and Adaptive Denoising Method

Traditional U-shape segmentation models can achieve excellent performance with an elegant structure. However, the single-layer decoder structure of U-Net or SwinUnet is too “thin” to exploit enough information, resulting in large semantic differences between the encoder and decoder parts. Things get worse in the field of medical image processing, where annotated data are more difficult to obtain than other tasks. Based on this observation, we propose a U-like model named MS-UNet with a plug-and-play adaptive denoising module and ELoss for the medical image segmentation task in this study. Instead of the single-layer U-Net decoder structure used in Swin-UNet and TransUNet, we specifically designed a multi-scale nested decoder based on the Swin Transformer for U-Net. The proposed multi-scale nested decoder structure allows for the feature mapping between the decoder and encoder to be semantically closer, thus enabling the network to learn more detailed features. In addition, ELoss could improve the attention of the model to the segmentation edges, and the plug-and-play adaptive denoising module could prevent the model from learning the wrong features without losing detailed information. The experimental results show that MS-UNet could effectively improve network performance with more efficient feature learning capability and exhibit more advanced performance, especially in the extreme case with a small amount of training data. Furthermore, the proposed ELoss and denoising module not only significantly enhance the segmentation performance of MS-UNet but can also be applied individually to other models.

CBAM-RIUnet: Breast Tumor Segmentation With Enhanced Breast Ultrasound and Test-Time Augmentation.

This study addresses the challenge of precise breast tumor segmentation in ultrasound images, crucial for effective Computer-Aided Diagnosis (CAD) in breast cancer. We introduce CBAM-RIUnet, a deep learning (DL) model for automated breast tumor segmentation in breast ultrasound (BUS) images. The model, featuring an efficient convolutional block attention module residual inception Unet, outperforms existing models, particularly excelling in Dice and IoU scores. CBAM-RIUnet follows the Unet structure with a residual inception depth-wise separable convolution, and incorporates a convolutional block attention module (CBAM) to eliminate irrelevant features and focus on the region of interest. Evaluation under three scenarios, including enhanced breast ultrasound (EBUS) and test-time augmentation (TTA), demonstrates impressive results. CBAM-RIUnet achieves Dice and IoU scores of 89.38% and 88.71%, respectively, showcasing significant improvements compared to state-of-the-art DL techniques. In conclusion, CBAM-RIUnet presents a highly effective and simplified DL model for breast tumor segmentation in BUS imaging.

Development and evaluation of a deep learning model for automatic segmentation of non-perfusion area in fundus fluorescein angiography

Diabetic retinopathy (DR) is the most prevalent cause of preventable vision loss worldwide, imposing a significant economic and medical burden on society today, of which early identification is the cornerstones of the management. The diagnosis and severity grading of DR rely on scales based on clinical visualized features, but lack detailed quantitative parameters. Retinal non-perfusion area (NPA) is a pathogenic characteristic of DR that symbolizes retinal hypoxia conditions, and was found to be intimately associated with disease progression, prognosis, and management. However, the practical value of NPA is constrained since it appears on fundus fluorescein angiography (FFA) as distributed, irregularly shaped, darker plaques that are challenging to measure manually. In this study, we propose a deep learning-based method, NPA-Net, for accurate and automatic segmentation of NPAs from FFA images acquired in clinical practice. NPA-Net uses the U-net structure as the basic backbone, which has an encoder-decoder model structure. To enhance the recognition performance of the model for NPA, we adaptively incorporate multi-scale features and contextual information in feature learning and design three modules: Adaptive Encoder Feature Fusion (AEFF) module, Multilayer Deep Supervised Loss, and Atrous Spatial Pyramid Pooling (ASPP) module, which enhance the recognition ability of the model for NPAs of different sizes from different perspectives. We conducted extensive experiments on a clinical dataset with 163 eyes with NPAs manually annotated by ophthalmologists, and NPA-Net achieved better segmentation performance compared to other existing methods with an area under the receiver operating characteristic curve (AUC) of 0.9752, accuracy of 0.9431, sensitivity of 0.8794, specificity of 0.9459, IOU of 0.3876 and Dice of 0.5686. This new automatic segmentation model is useful for identifying NPA in clinical practice, generating quantitative parameters that can be useful for further research as well as guiding DR detection, grading severity, treatment planning, and prognosis.

Early Breast Cancer Detection Using Artificial Intelligence Techniques Based on Advanced Image Processing Tools

The early detection of breast cancer is essential for improving treatment outcomes, and recent advancements in artificial intelligence (AI), combined with image processing techniques, have shown great potential in enhancing diagnostic accuracy. This study explores the effects of various image processing methods and AI models on the performance of early breast cancer diagnostic systems. By focusing on techniques such as Wiener filtering and total variation filtering, we aim to improve image quality and diagnostic precision. The novelty of this study lies in the comprehensive evaluation of these techniques across multiple medical imaging datasets, including a DCE-MRI dataset for breast-tumor image segmentation and classification (BreastDM) and the Breast Ultrasound Image (BUSI), Mammographic Image Analysis Society (MIAS), Breast Cancer Histopathological Image (BreakHis), and Digital Database for Screening Mammography (DDSM) datasets. The integration of advanced AI models, such as the vision transformer (ViT) and the U-KAN model—a U-Net structure combined with Kolmogorov–Arnold Networks (KANs)—is another key aspect, offering new insights into the efficacy of these approaches in different imaging contexts. Experiments revealed that Wiener filtering significantly improved image quality, achieving a peak signal-to-noise ratio (PSNR) of 23.06 dB and a structural similarity index measure (SSIM) of 0.79 using the BreastDM dataset and a PSNR of 20.09 dB with an SSIM of 0.35 using the BUSI dataset. When combined filtering techniques were applied, the results varied, with the MIAS dataset showing a decrease in SSIM and an increase in the mean squared error (MSE), while the BUSI dataset exhibited enhanced perceptual quality and structural preservation. The vision transformer (ViT) framework excelled in processing complex image data, particularly with the BreastDM and BUSI datasets. Notably, the Wiener filter using the BreastDM dataset resulted in an accuracy of 96.9% and a recall of 96.7%, while the combined filtering approach further enhanced these metrics to 99.3% accuracy and 98.3% recall. In the BUSI dataset, the Wiener filter achieved an accuracy of 98.0% and a specificity of 98.5%. Additionally, the U-KAN model demonstrated superior performance in breast cancer lesion segmentation, outperforming traditional models like U-Net and U-Net++ across datasets, with an accuracy of 93.3% and a sensitivity of 97.4% in the BUSI dataset. These findings highlight the importance of dataset-specific preprocessing techniques and the potential of advanced AI models like ViT and U-KAN to significantly improve the accuracy of early breast cancer diagnostics.

Phase unwrapping of SAR interferogram from modified U-net via training data simulation and network structure optimization

Phase unwrapping is the process of retrieving the true phase values from observed wrapped phases by adding the correct multiples of 2π. This process is crucial in synthetic aperture radar (SAR) interferometry, and numerous studies have aimed to enhance its performance. This study explored phase unwrapping using a modified U-Net regression model by optimizing both the network structure and training data. For network structure optimization, the study first compared model performance based on the size ratio of the lowest feature maps, determined by the number of pooling layers, to the size of convolutional kernels. A multi-kernel U-Net structure was developed to ensure robustness against variations in phase noise and gradient, compared to a standard single-kernel U-Net. Regarding the training data, data augmentation was implemented to address imbalances and better represent the local noise characteristics found in actual SAR interferograms. The training data was simulated to include local noise effects based on coherence measurements from real SAR data, as well as simple noise used for benchmarking the unwrapping performance with different training datasets. The results indicated that when the convolutional kernel size is smaller than the feature map size at the lowest layer, increasing the number of pooling layers leads to improvements in unwrapping performance. Conversely, performance decreased when the feature map size at the lowest layers was smaller than the convolutional kernel size. Specifically, the single-kernel U-Net with six pooling layers and the multi-kernel U-Net with five pooling layers exhibited the best unwrapping performance. Considering both simulated and real synthetic aperture radar interferogram data, the mean absolute errors for the single- and multi-kernel U-Net trained with simple noise were approximately 0.235 and 0.254, respectively. In contrast, for models trained with locally variable noise simulation data, the MAEs dropped to about 0.033 and 0.032, showing an improvement by approximately eightfold over models trained with simple noise. For real synthetic aperture radar interferograms, the mean absolute errors were 0.542 (single-kernel U-Net trained using simple noise), 0.592 (multi-kernel U-Net trained using simple noise), 0.542 (single-kernel U-Net trained using local noise), and 0.445 (proposed), respectively, underscoring the significant impact of training data on unwrapping performance. The study also evaluated the performance of the statistical-cost, network-flow algorithm for phase unwrapping (SNAPHU), obtaining mean absolute errors of about 0.043 for simulation data and 0.861 for real SAR data. Consequently, the multi-kernel model trained with locally different noise simulation data demonstrated roughly twice the performance compared to the traditional phase unwrapping method.

Semantic Segmentation of Urban Remote Sensing Images Based on Deep Learning

In the realm of urban planning and environmental evaluation, the delineation and categorization of land types are pivotal. This study introduces a convolutional neural network-based image semantic segmentation approach to delineate parcel data in remote sensing imagery. The initial phase involved a comparative analysis of various CNN architectures. ResNet and VGG serve as the foundational networks for training, followed by a comparative assessment of the experimental outcomes. Subsequently, the VGG+U-Net model, which demonstrated superior efficacy, was chosen as the primary network. Enhancements to this model were made by integrating attention mechanisms. Specifically, three distinct attention mechanisms—spatial, SE, and channel—were incorporated into the VGG+U-Net framework, and various loss functions were evaluated and selected. The impact of these attention mechanisms, in conjunction with different loss functions, was scrutinized. This study proposes a novel network model, designated VGG+U-Net+Channel, that leverages the VGG architecture as the backbone network in conjunction with the U-Net structure and augments it with the channel attention mechanism to refine the model’s performance. This refinement resulted in a 1.14% enhancement in the network’s overall precision and marked improvements in MPA and MioU. A comparative analysis of the detection capabilities between the enhanced and original models was conducted, including a pixel count for each category to ascertain the extent of various semantic information. The experimental validation confirms the viability and efficacy of the proposed methodology.

Soldering Defect Segmentation Method for PCB on Improved UNet

Despite being indispensable devices in the electronic manufacturing industry, printed circuit boards (PCBs) may develop various soldering defects in the production process, which seriously affect the product’s quality. Due to the substantial background interference in the soldering defect image and the small and irregular shapes of the defects, the accurate segmentation of soldering defects is a challenging task. To address this issue, a method to improve the encoder–decoder network structure of UNet is proposed for PCB soldering defect segmentation. To enhance the feature extraction capabilities of the encoder and focus more on deeper features, VGG16 is employed as the network encoder. Moreover, a hybrid attention module called the DHAM, which combines channel attention and dynamic spatial attention, is proposed to reduce the background interference in images and direct the model’s focus more toward defect areas. Additionally, based on GSConv, the RGSM is introduced and applied in the decoder to enhance the model’s feature fusion capabilities and improve the segmentation accuracy. The experiments demonstrate that the proposed method can effectively improve the segmentation accuracy for PCB soldering defects, achieving an mIoU of 81.74% and mPA of 87.33%, while maintaining a relatively low number of model parameters at only 22.13 M and achieving an FPS of 30.16, thus meeting the real-time detection speed requirements.

Low-light enhancement method with dual branch feature fusion and learnable regularized attention

Restricted by the lighting conditions, the images captured at night tend to suffer from color aberration, noise, and other unfavorable factors, making it difficult for subsequent vision-based applications. To solve this problem, we propose a two-stage size-controllable low-light enhancement method, named Dual Fusion Enhancement Net (DFEN). The whole algorithm is built on a double U-Net structure, implementing brightness adjustment and detail revision respectively. A dual branch feature fusion module is adopted to enhance its ability of feature extraction and aggregation. We also design a learnable regularized attention module to balance the enhancement effect on different regions. Besides, we introduce a cosine training strategy to smooth the transition of the training target from the brightness adjustment stage to the detail revision stage during the training process. The proposed DFEN is tested on several low-light datasets, and the experimental results demonstrate that the algorithm achieves superior enhancement results with the similar parameters. It is worth noting that the lightest DFEN model reaches 11 FPS for image size of 1224×1024 in an RTX 3090 GPU.Graphical

Head and neck tumor segmentation from [18F]F-FDG PET/CT images based on 3D diffusion model

Objective. Head and neck (H&N) cancers are among the most prevalent types of cancer worldwide, and [18F]F-FDG PET/CT is widely used for H&N cancer management. Recently, the diffusion model has demonstrated remarkable performance in various image-generation tasks. In this work, we proposed a 3D diffusion model to accurately perform H&N tumor segmentation from 3D PET and CT volumes. Approach. The 3D diffusion model was developed considering the 3D nature of PET and CT images acquired. During the reverse process, the model utilized a 3D UNet structure and took the concatenation of 3D PET, CT, and Gaussian noise volumes as the network input to generate the tumor mask. Experiments based on the HECKTOR challenge dataset were conducted to evaluate the effectiveness of the proposed diffusion model. Several state-of-the-art techniques based on U-Net and Transformer structures were adopted as the reference methods. Benefits of employing both PET and CT as the network input, as well as further extending the diffusion model from 2D to 3D, were investigated based on various quantitative metrics and qualitative results. Main results. Results showed that the proposed 3D diffusion model could generate more accurate segmentation results compared with other methods (mean Dice of 0.739 compared to less than 0.726 for other methods). Compared to the diffusion model in 2D form, the proposed 3D model yielded superior results (mean Dice of 0.739 compared to 0.669). Our experiments also highlighted the advantage of utilizing dual-modality PET and CT data over only single-modality data for H&N tumor segmentation (with mean Dice less than 0.570). Significance. This work demonstrated the effectiveness of the proposed 3D diffusion model in generating more accurate H&N tumor segmentation masks compared to the other reference methods.

Deep learning for quantifying spatial patterning and formation process of early differentiated human-induced pluripotent stem cells with micropattern images.

Micropatterning is reliable method for quantifying pluripotency of human-induced pluripotent stem cells (hiPSCs) that differentiate to form a spatial pattern of sorted, ordered and nonoverlapped three germ layers on the micropattern. In this study, we propose a deep learning method to quantify spatial patterning of the germ layers in the early differentiation stage of hiPSCs using micropattern images. We propose decoding and encoding U-net structures learning labelled Hoechst (DNA-stained) hiPSC regions with corresponding Hoechst and bright-field micropattern images to segment hiPSCs on Hoechst or bright-field images. We also propose a U-net structure to extract extraembryonic regions on a micropattern, and an algorithm to compares intensities of the fluorescence images staining respective germ-layer cells and extract their regions. The proposed method thus can quantify the pluripotency of a hiPSC line with spatial patterning including cell numbers, areas and distributions of germ-layer and extraembryonic cells on a micropattern, and reveal the formation process of hiPSCs and germ layers in the early differentiation stage by segmenting live-cell bright-field images. In our assay, the cell-number accuracy achieved 86% and 85%, and the cell region accuracy 89% and 81% for segmenting Hoechst and bright-field micropattern images, respectively. Applications to micropattern images of multiple hiPSC lines, micropattern sizes, groups of markers, living and fixed cells show the proposed method can be expected to be a useful protocol and tool to quantify pluripotency of a new hiPSC line before providing it to the scientific community.