U-Net Structure Research Articles

Research on a multi‐scale degradation fusion network in all‐in‐one image restoration

AbstractImage restoration aims to recover high‐quality clean images from degraded low‐quality ones. Deep learning‐based approaches have been a focal point in the field of image restoration. However, most methods focus solely on a single type of degradation and may not extend well to real‐world scenarios with unknown degradation. For this purpose, the present study introduces an all‐in‐one image restoration approach by designing a multi‐scale feature fusion UNet structure (MdfUNet). In summary, the proposed method exhibits two significant advantages. For starters, it implicitly fuses degradation information across multiple scales, enabling the network to extract rich hierarchical features and enhancing its generalization ability towards unknown degradations. Secondly, MdfUnet possesses strong image reconstruction capabilities. It utilizes a simple non‐linear feature optimizer to enhance skip connections, providing rich feature representations for the image reconstruction process, and ultimately generating high‐quality restored images. Extensive experimental results show the proposed method outperforms multiple baselines on deraining, dehazing, and denoising datasets.

Lung X-ray image segmentation based on improved Unet deep learning network algorithm with GSConv module

In this paper, we propose an optimised image segmentation method by structurally innovating and improving the traditional Unet model and integrating the latest GSConv module. In our experiments, we integrate the GSConv module into the encoder and decoder parts of U-Net to take advantage of its excellent feature extraction and information transfer capabilities. In comparing the training process of the two models of Unet and GSConv Unet, it is found that GSConv Unet has faster convergence speed and better generalisation ability, and finally shows higher segmentation accuracy and iou values in the test part. From the segmentation results, GSConv Unet delineates the lung region more accurately and meticulously compared to Unet, providing an effective idea for lung X-ray image segmentation tasks. This research is of great significance, which not only improves the effectiveness of the image segmentation task but also brings new technological breakthroughs in the field of medical imaging. By introducing the GSConv module and optimising the Unet structure, we have successfully improved the precision and efficiency of lung X-ray image segmentation, providing doctors with a more reliable and accurate diagnostic tool.

An Efficient Transformer–CNN Network for Document Image Binarization

Color image binarization plays a pivotal role in image preprocessing work and significantly impacts subsequent tasks, particularly for text recognition. This paper concentrates on document image binarization (DIB), which aims to separate an image into a foreground (text) and background (non-text content). We thoroughly analyze conventional and deep-learning-based approaches and conclude that prevailing DIB methods leverage deep learning technology. Furthermore, we explore the receptive fields of pre- and post-network training to underscore the Transformer model’s advantages. Subsequently, we introduce a lightweight model based on the U-Net structure and enhanced with the MobileViT module to capture global information features in document images better. Given its adeptness at learning both local and global features, our proposed model demonstrates competitive performance on two standard datasets (DIBCO2012 and DIBCO2017) and good robustness on the DIBCO2019 dataset. Notably, our proposed method presents a straightforward end-to-end model devoid of additional image preprocessing or post-processing, eschewing the use of ensemble models. Moreover, its parameter count is less than one-eighth of the model, which achieves the best results on most DIBCO datasets. Finally, two sets of ablation experiments are conducted to verify the effectiveness of the proposed binarization model.

Global Precipitation Nowcasting of Integrated Multi-satellitE Retrievals for GPM: A U-Net Convolutional LSTM Architecture

Abstract This paper presents a deep supervised learning architecture for 30-min global precipitation nowcasts with a 4-h lead time. The architecture follows a U-Net structure with convolutional long short-term memory (ConvLSTM) cells empowered by ConvLSTM-based skip connections to reduce information loss due to the pooling operation. The training uses data from the Integrated Multi-satellitE Retrievals for GPM (IMERG) and a few key drivers of precipitation from the Global Forecast System (GFS). The impacts of different training loss functions, including the mean-squared error (regression) and the focal loss (classification), on the quality of precipitation nowcasts are studied. The results indicate that the regression network performs well in capturing light precipitation (<1.6 mm h−1), while the classification network can outperform the regression counterpart for nowcasting of high-intensity precipitation (>8 mm h−1), in terms of the critical success index (CSI). It is uncovered that including the forecast variables can improve precipitation nowcasting, especially at longer lead times in both networks. Taking IMERG as a relative reference, a multiscale analysis, in terms of fractions skill score (FSS), shows that the nowcasting machine remains skillful for precipitation rate above 1 mm h−1 at the resolution of 10 km compared to 50 km for GFS. For precipitation rates greater than 4 mm h−1, only the classification network remains FSS skillful on scales greater than 50 km within a 2-h lead time. Significance Statement This study presents a deep neural network architecture for global precipitation nowcasting with a 4-h lead time, using sequences of past satellite precipitation data and simulations from a numerical weather prediction model. The results show that the nowcasting machine can improve short-term predictions of high-intensity global precipitation. The research outcomes will enable us to expand our understanding of how modern artificial intelligence can improve the predictability of extreme weather and benefit flood early warning systems for saving lives and properties.

Efficiency-Accuracy Trade-Off in Light Field Estimation with Cost Volume Construction and Aggregation.

The Rich spatial and angular information in light field images enables accurate depth estimation, which is a crucial aspect of environmental perception. However, the abundance of light field information also leads to high computational costs and memory pressure. Typically, selectively pruning some light field information can significantly improve computational efficiency but at the expense of reduced depth estimation accuracy in the pruned model, especially in low-texture regions and occluded areas where angular diversity is reduced. In this study, we propose a lightweight disparity estimation model that balances speed and accuracy and enhances depth estimation accuracy in textureless regions. We combined cost matching methods based on absolute difference and correlation to construct cost volumes, improving both accuracy and robustness. Additionally, we developed a multi-scale disparity cost fusion architecture, employing 3D convolutions and a UNet-like structure to handle matching costs at different depth scales. This method effectively integrates information across scales, utilizing the UNet structure for efficient fusion and completion of cost volumes, thus yielding more precise depth maps. Extensive testing shows that our method achieves computational efficiency on par with the most efficient existing methods, yet with double the accuracy. Moreover, our approach achieves comparable accuracy to the current highest-accuracy methods but with an order of magnitude improvement in computational performance.

Short-Term and Imminent Rainfall Prediction Model Based on ConvLSTM and SmaAT-UNet.

Short-term precipitation forecasting methods are mainly divided into statistical forecasting, numerical model-based forecasting, and radar image extrapolation techniques. The two methods based on statistical prediction and numerical model have the disadvantages of being unstable and generating large errors. Therefore, this study proposes the use of deep learning for radar image extrapolation for precipitation forecasting, in particular by developing algorithms for ConvLSTM and SmaAT-UNet. The ConvLSTM model is a fusion of a CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory network), which solves the challenge of processing spatial sequence data, which is a task that traditional LSTM models cannot accomplish. At the same time, SmaAT-UNet enhances the traditional UNet structure by incorporating the CBAM (Convolutional Block Attention Module) attention mechanism and replacing the standard convolutional layer with depthwise separable convolution. This innovative approach aims to improve the efficiency and accuracy of short-term precipitation forecasting by improving feature extraction and data processing techniques. Evaluation and analysis of experimental data show that both models exhibit good predictive ability, with the SmaAT-UNet model outperforming ConvLSTM in terms of accuracy. The results show that the performance indicators of precipitation prediction, especially detection probability (POD) and the Critical Success index (CSI), show a downward trend with the extension of the prediction time. This trend highlights the inherent challenges of maintaining predictive accuracy over longer periods of time and highlights the superior performance and resilience of the SmaAT-UNet model under these conditions. Compared with the statistical forecasting method and numerical model forecasting method, its accuracy in short-term rainfall forecasting is improved.

A Dual-Branch Self-Boosting Network Based on Noise2Noise for Unsupervised Image Denoising

While unsupervised denoising models have shown progress in recent years, their noise reduction capabilities still lag behind those of supervised denoising models. This limitation can be attributed to the lack of effective constraints during training, which only utilizes noisy images and hinders further performance improvements In this work, we propose a novel dual-branch self-boosting network called DBSNet, which offers a straightforward and effective approach to image denoising. By leveraging task-dependent features, we exploit the intrinsic relationships between the two branches to enhance the effectiveness of our proposed model. Initially, we extend the classic Noise2Noise (N2N) architecture by adding a new branch for noise component prediction to the existing single-branch network designed for content prediction. This expansion creates a dual-branch structure, enabling us to simultaneously decompose a given noisy image into its content (clean) and noise components. This enhancement allows us to establish stronger constraint conditions and construct more powerful loss functions to guide the training process. Furthermore, we replace the UNet structure in the N2N network with the proven DnCNN (Denoising Convolutional Neural Network) sequential network architecture, which enhances the nonlinear mapping capabilities of the DBSNet. This modification enables our dual-branch network to effectively map a noisy image to its content (clean) and noise components simultaneously. To further improve the stability and effectiveness of training, and consequently enhance the denoising performance, we introduce a feedback mechanism where the network’s outputs, i.e., content and noise components, are fed back into the dual-branch network. This results in an enhanced loss function that ensures our model possesses excellent decomposition ability and further boosts the denoising performance. Extensive experiments conducted on both synthetic and real-world images demonstrate that the proposed DBSNet outperforms the unsupervised N2N denoising model as well as mainstream supervised models trained with supervised methods. Moreover, the evaluation results on real-world noisy images highlight the desirable generalization ability of DBSNet for practical denoising applications.

Enhancement of the Consistency and Connectivity of the Ant-tracking Algorithm via 3D U-Net with Dual-Threshold Iteration

Abstract The ant-tracking algorithm is commonly used to extract faults in geological structures. However, obtaining 3D ant-tracking data requires the calculation of various volume attributes. To obtain satisfactory data, it is necessary to iterate through the parameters of these attributes to achieve reasonable continuity. Moreover, due to the numerous parameters involved, the algorithm can produce different outputs with each execution. In this study, we aimed to enhance the performance of the ant-tracking algorithm by combining it with a basic U-Net structure. The input and corresponding labels to the model are "cubic shaped" 3D data segmented from the original 3D seismic volume to facilitate cross-validation with distinct regions. We used the label data as the single ant-tracking result to minimize the operator’s bias by executing the ant-tracking with several different parameters and executors and then taking the average. An evaluative comparison of three different loss functions (MAE, RMSE, and MSE) was conducted to identify the optimal function for training the model. Across five out of the six metrics, MSE function demonstrated predominant performance, leading to its adoption. Apart from this, a significant number of misinterpreted faults led us to propose the post-processing algorithm named "Dual-Threshold Iteration." It was initially used to extract fine blood vessels branching out from large vessels in medical image segmentation and adapted in our work to ensure a high level of continuity while ignoring worthless noise. Comparison with the F1 score and the number of 3D-connected components confirmed that the proposed method could generate reduced bias and smoothly connected fault structures.

Magnetic Resonance Images Segmentation of Multifidus based on Dense-unet and Superpixel.

Lumbar disc herniation (LDH) is a common clinical condition causing lower back and leg pain. Accurate segmentation of the lumbar discs is crucial for assessing and diagnosing LDH. Magnetic resonance imaging (MRI) can reveal the condition of articular cartilage. However, manual segmentation of MRI images is burdensome for physicians and needs to be more efficient. In this study, we propose a method that combines UNet and superpixel segmentation to address the problem of loss of detailed information in the feature extraction phase, leading to poor segmentation results at object edges. The aim is to provide a reproducible solution for diagnosing patients with lumbar disc herniation. We suggest using the network structure of UNet. Firstly, dense blocks are inserted into the UNet network, and training is performed using the Swish activation function. The Dense-UNet model extracts semantic features from the images and obtains rough semantic segmentation results. Then, an adaptive-scale superpixel segmentation algorithm is applied to segment the input images into superpixel images. Finally, high-level abstract semantic features are fused with the detailed information of the superpixels to obtain edge-optimized semantic segmentation results. Evaluation of a private dataset of multifidus muscles in magnetic resonance images demonstrates that compared to other segmentation algorithms, this algorithm exhibits better semantic segmentation performance in detailed areas such as object edges. Compared to UNet, it achieves a 9.5% improvement in the Dice Similarity Coefficient (DSC) and an 11.3% improvement in the Jaccard Index (JAC). The experimental results indicate that this algorithm improves segmentation performance while reducing computational complexity.

Sample-imbalanced wafer map defects classification based on auxiliary classifier denoising diffusion probability model

This paper research on the classification task of wafer map defects under imbalanced sample. To improve the number of imbalanced wafer maps, a denoising diffusion probabilistic model with auxiliary classifier is proposed to generate new wafer maps. The developed model has an U-net structure with multiple inputs and outputs, taking wafer images, noise degree and category features as inputs, and outputs predicted noise and prediction category. When performing data augmentation, the trained model gradually removes prediction noise from the initial random noise and obtains new wafers. Finally, the balanced wafer defect maps are classified using a residual neural network. The proposed auxiliary classifier denoising diffusion probabilistic model and residual neural networks (ACDDPM-ResNet) method has validated on the MIR-WM811K dataset and MixedWM38 dataset, and the defect classification results of the imbalanced wafer map are remarkable improved after data augmentation. In addition, the paper also discusses and analyzes the influence of the maximum noise addition steps and the data augmentation size on the accuracy of wafer classification. It is further validated that the proposed wafer map classification method based on data augmentation can solve the classification problem caused by sample imbalance.

Segmentation of bone surface from ultrasound using a lightweight network UBS-Net

Objective. Ultrasound-assisted orthopaedic navigation held promise due to its non-ionizing feature, portability, low cost, and real-time performance. To facilitate the applications, it was critical to have accurate and real-time bone surface segmentation. Nevertheless, the imaging artifacts and low signal-to-noise ratios in the tomographical B-mode ultrasound (B-US) images created substantial challenges in bone surface detection. In this study, we presented an end-to-end lightweight US bone segmentation network (UBS-Net) for bone surface detection. Approach. We presented an end-to-end lightweight UBS-Net for bone surface detection, using the U-Net structure as the base framework and a level set loss function for improved sensitivity to bone surface detectability. A dual attention (DA) mechanism was introduced at the end of the encoder, which considered both position and channel information to obtain the correlation between the position and channel dimensions of the feature map, where axial attention (AA) replaced the traditional self-attention (SA) mechanism in the position attention module for better computational efficiency. The position attention and channel attention (CA) were combined with a two-class fusion module for the DA map. The decoding module finally completed the bone surface detection. Main Results. As a result, a frame rate of 21 frames per second (fps) in detection were achieved. It outperformed the state-of-the-art method with higher segmentation accuracy (Dice similarity coefficient: 88.76% versus 87.22%) when applied the retrospective ultrasound (US) data from 11 volunteers. Significance. The proposed UBS-Net for bone surface detection in ultrasound achieved outstanding accuracy and real-time performance. The new method out-performed the state-of-the-art methods. It had potential in US-guided orthopaedic surgery applications.

Design and evaluation of a deep learning-based automatic segmentation of maxillary and mandibular substructures using a 3D U-Net

BackgroundCurrent segmentation approaches for radiation treatment planning in head and neck cancer patients (HNCP) typically consider the entire mandible as an organ at risk, whereas segmentation of the maxilla remains uncommon. Accurate risk assessment for osteoradionecrosis (ORN) or implant-based dental rehabilitation after radiation therapy may require a nuanced analysis of dose distribution in specific mandibular and maxillary segments. Manual segmentation is time-consuming and inconsistent, and there is no definition of jaw subsections. Materials and methodsThe mandible and maxilla were divided into 12 substructures. The model was developed from 82 computed tomography (CT) scans of HNCP and adopts an encoder-decoder three-dimensional (3D) U-Net structure. The efficiency and accuracy of the automated method were compared against manual segmentation on an additional set of 20 independent CT scans. The evaluation metrics used were the Dice similarity coefficient (DSC), 95% Hausdorff distance (HD95), and surface DSC (sDSC). ResultsAutomated segmentations were performed in a median of 86 s, compared to manual segmentations, which took a median of 53.5 min. The median DSC per substructure ranged from 0.81 to 0.91, and the median HD95 ranged from 1.61 to 4.22. The number of artifacts did not affect these scores. The maxillary substructures showed lower metrics than the mandibular substructures. ConclusionsThe jaw substructure segmentation demonstrated high accuracy, time efficiency, and promising results in CT scans with and without metal artifacts. This novel model could provide further investigation into dose relationships with ORN or dental implant failure in normal tissue complication prediction models.

Multi-scale nested UNet with transformer for colorectal polyp segmentation.

Polyp detection and localization are essential tasks for colonoscopy. U-shape network based convolutional neural networks have achieved remarkable segmentation performance for biomedical images, but lack of long-range dependencies modeling limits their receptivefields. Our goal was to develop and test a novel architecture for polyp segmentation, which takes advantage of learning local information with long-range dependenciesmodeling. A novel architecture combining with multi-scale nested UNet structure integrated transformer for polyp segmentation was developed. The proposed network takes advantage of both CNN and transformer to extract distinct feature information. The transformer layer is embedded between the encoder and decoder of a U-shape net to learn explicit global context and long-range semantic information. To address the challenging of variant polyp sizes, a MSFF unit was proposed to fuse features with multipleresolution. Four public datasets and one in-house dataset were used to train and test the model performance. Ablation study was also conducted to verify each component of the model. For dataset Kvasir-SEG and CVC-ClinicDB, the proposed model achieved mean dice score of 0.942 and 0.950 respectively, which were more accurate than the other methods. To show the generalization of different methods, we processed two cross dataset validations, the proposed model achieved the highest mean dice score. The results demonstrate that the proposed network has powerful learning and generalization capability, significantly improving segmentation accuracy and outperforming state-of-the-artmethods. The proposed model produced more accurate polyp segmentation than current methods on four different public and one in-house datasets. Its capability of polyps segmentation in different sizes shows the potential clinicalapplication.

Multi-head attention ResUnet with sequential sliding windows for sea surface height anomaly field forecast: A regional study in North Atlantic Ocean

Accurate and efficient prediction of Sea surface height anomaly (SSHA) field is very important for operational marine monitoring and engineering. It is also a vital indicator of better understanding global climate changes and ocean dynamics. Many traditional SSHA forecasting methods focused primarily on single grid-point based predictions, and some classical Recurrent Neural Network/Long-Short-Term-Memory based machine learning approaches can cost heavily computational consumption, also the SSHA prediction by only employing the observations in single locations would lead to large potential uncertainties. This study proposed a novel purely Convolutional neural network (CNN) based Unet structure with Multi-head attention mechanism and Residual CNN blocks to accomplish daily SSHA variabilities prediction with higher accuracy and better computational efficiency. Specifically, the satellite altimetry observations aggregated by Data Unification and Altimeter Combination System (DUACS) from a sub-region with very high occurring frequently of storm surges and mesoscale/small-scale eddies in North Atlantic Ocean (NAO) was employed to verify the practicability and functionality of the proposed model. In addition, several existed deep learning approaches were adopted to implement a better comparison. Experimental results in this study demonstrated that the proposed methodology can achieve superior prediction performance amongst the four deep learning methods, and specifically, presenting significant superiorities on the aspect of computational costs compared to recurrent-based neural networks. Very high accurate SSHA predictions with averaged RMSE as 0.018 m and Correlation coefficient as 0.99 for 1 day ahead forecasting, and corresponded 0.107 m, 0.80 for 24days ahead forecasting was obtained by the proposed approach. It took about less than 2 minutes to fulfil a 5-year SSHA field forecasts together with optimal model training process, Additional forecasting experiments based on different seasonal SSHA sequential patterns reveal that the proposed purely CNN-based technique show pretty well generalization capability. This study provides an alternative promising-prospect on the application of purely CNN-based deep learning methods in sea surface variabilities forecasting.

Abstract 2273: Development of a prognostic biomarker for ovarian cancer based on a high-resolution digital pathology platform for spatial statistical analysis of platelets and blood vessels

Abstract Purpose: A third of ovarian cancer patients have elevated platelet counts that are associated with poor prognosis. However, the exact mechanism by which platelets preferentially extravasate into tumor microenvironment is unknown, partly owing to lack of effective tools that can accurately segment and measure platelet counts found in tumor tissues. In this study, we developed a high-resolution digital pathology (DP) platform with whole tissue confocal imaging and deep learning model that can be used to automatically segment out platelets and blood vessels to perform spatial statistics (SA) of platelets with high precision and consistency. We utilized our SADP platform to study an interaction between tumor secreted SDF-1 and CXCR4 on platelets which initiates platelet chemotaxis towards tumor microenvironment. Methods: A deep learning algorithm to accurately segment out platelets was developed based on the U-Net structure using our original datasets from the murine ovarian cancer model. Tumor-bearing mice were injected with 5 mg/kg of Plerixafor, a CXCR4 inhibitor, daily for 4 weeks. Then, tumor nodules were resected, stained with CD41a and CD31 antibodies, and imaged using a whole slide confocal microscopy. We performed manual annotation of these images to generate the mask dataset and performed further data augmentation to increase the diversity of datasets. We trained our model using 85% of these images and performed validation using the remaining 15%. Results: Using our SADP platform, the validation dataset achieved pixel-wise accuracy of 98.8%. A receiver operating characteristic (ROC) curve demonstrated an area under the ROC curve (AUC) of 0.99 with high sensitivity and specificity. Intersection over union (IoU) score of 0.765 was obtained. A statistically significant (p=0.0467), 7.03% reduction in the platelet density was shown in the Plerixafor-treated mouse ovarian tumor tissues (n=8) compared to the control tumor tissues (n=9). We also found that platelets are spatially concentrated near blood vessels in the control tumor tissues compared to the Plerixafor-treated tumor tissues (p=0.0377). Conclusions: We developed a SADP platform to segment and quantitate tens of thousands of platelets from whole slide mouse ovarian cancer tissues automatically with high accuracy and consistency and incorporated spatial statistics to understand their spatial relationships and patterns that would not be apparent in non-spatial analyses. Applying the SADP platform, we demonstrated that treatment with Plerixafor leads to overall reduction in platelet infiltration within ovarian murine tumors. Our ongoing effort is to utilize our SADP platform on human tumor tissue images from a whole slide digital scanner to further characterize spatial distribution of platelets as a potential prognostic marker for ovarian cancer. Citation Format: Ju Young Ahn, Wendolyn Carlos-Alcalde, Matthew Vasquez, Min Soon Cho, Vahid Afshar-Kharghan, Stephen T. Wong. Development of a prognostic biomarker for ovarian cancer based on a high-resolution digital pathology platform for spatial statistical analysis of platelets and blood vessels [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2273.

SSTU: Swin-Spectral Transformer U-Net for hyperspectral whole slide image reconstruction

Whole Slide Imaging and Hyperspectral Microscopic Imaging provide great quality data with high spatial and spectral resolution for histopathology. Existing Hyperspectral Whole Slide Imaging systems combine the advantages of the techniques above, thus providing rich information for pathological diagnosis. However, it cannot avoid the problems of slow acquisition speed and mass data storage demand. Inspired by the spectral reconstruction task in computer vision and remote sensing, the Swin-Spectral Transformer U-Net (SSTU) has been developed to reconstruct Hyperspectral Whole Slide images (HWSis) from multiple Hyperspectral Microscopic images (HMis) of small Field of View and Whole Slide images (WSis). The Swin-Spectral Transformer (SST) module in SSTU takes full advantage of Transformer in extracting global attention. Firstly, Swin Transformer is exploited in space domain, which overcomes the high computation cost in Vision Transformer structures, while it maintains the spatial features extracted from WSis. Furthermore, Spectral Transformer is exploited to collect the long-range spectral features in HMis. Combined with the multi-scale encoder-bottleneck-decoder structure of U-Net, SSTU network is formed by sequential and symmetric residual connections of SSTs, which reconstructs a selected area of HWSi from coarse to fine. Qualitative and quantitative experiments prove the performance of SSTU in HWSi reconstruction task superior to other state-of-the-art spectral reconstruction methods.

PVI-Net: Point–Voxel–Image Fusion for Semantic Segmentation of Point Clouds in Large-Scale Autonomous Driving Scenarios

In this study, we introduce a novel framework for the semantic segmentation of point clouds in autonomous driving scenarios, termed PVI-Net. This framework uniquely integrates three different data perspectives—point clouds, voxels, and distance maps—executing feature extraction through three parallel branches. Throughout this process, we ingeniously design a point cloud–voxel cross-attention mechanism and a multi-perspective feature fusion strategy for point images. These strategies facilitate information interaction across different feature dimensions of perspectives, thereby optimizing the fusion of information from various viewpoints and significantly enhancing the overall performance of the model. The network employs a U-Net structure and residual connections, effectively merging and encoding information to improve the precision and efficiency of semantic segmentation. We validated the performance of PVI-Net on the SemanticKITTI and nuScenes datasets. The results demonstrate that PVI-Net surpasses most of the previous methods in various performance metrics.

Enhanced Tropical Cyclone Precipitation Prediction in the Northwest Pacific Using Deep Learning Models and Ensemble Techniques

This study focuses on optimizing precipitation forecast induced by tropical cyclones (TCs) in the Northwest Pacific region, with lead times ranging from 6 to 72 h. The research employs deep learning models, such as U-Net, UNet3+, SE-Net, and SE-UNet3+, which utilize precipitation forecast data from the Global Forecast System (GFS) and real-time GFS environmental background data using a U-Net structure. To comprehensively make use of the precipitation forecasts from these models, we additionally use probabilistic matching (PM) and simple averaging (AVR) in rainfall prediction. The precipitation data from the Global Precipitation Measurement (GPM) Mission serves as the rainfall observation. The results demonstrate that the root mean squared errors (RMSEs) of U-Net, UNet3+, SE-UNet, SE-UNet3+, AVR, and PM are lowered by 8.7%, 10.1%, 9.7%, 10.0%, 11.4%, and 11.5%, respectively, when compared with the RMSE of the GFS TC precipitation forecasts, while the mean absolute errors are reduced by 9.6%, 11.3%, 9.0%, 12.0%, 12.8%, and 13.0%, respectively. Furthermore, the neural network model improves the precipitation threat scores (TSs). On average, the TSs of U-Net, UNet3+, SE-UNet, SE-UNet3+, AVR, and PM are raised by 12.8%, 21.3%, 19.3%, 20.7%, 22.5%, and 22.9%, respectively, compared with the GFS model. Notably, AVR and PM outperform all other individual models, with PM’s performance slightly better than AVR’s. The most important feature variables in optimizing TC precipitation forecast in the Northwest Pacific region based on the UNet-based neural network include GFS precipitation forecast data, land and sea masks, latitudinal winds at 500 hPa, and vertical winds at 500 hPa.

Deep Upscale U-Net for automatic tongue segmentation.

In a treatment or diagnosis related to oral health conditions such as oral cancer and oropharyngeal cancer, an investigation of tongue's movements is a major part. In an automatic measurement of such movement, it must first start with a task of tongue segmentation. This paper proposes a solution of tongue segmentation based on a decoder-encoder CNN-based structure i.e., U-Net. However, it could suffer from a problem of feature loss in deep layers. This paper proposes a Deep Upscale U-Net (DU-UNET). An additional up-sampling of the feature map from a contracting path is concatenated to an upper layer of an expansive path, based on an original U-Net structure. The segmentation model is constructed by training DU-UNET on the two publicly available datasets, and transferred to the self-collected dataset of tongue images with five tongue postures which were recorded at a far distance from a camera under a real-world scenario. The proposed DU-UNET outperforms the other existing methods in our literature reviews, with accuracy of 99.2%, mean IoU of 97.8%, Dice score of 96.8%, and Jaccard score of 96.8%.

Face photo-line drawings synthesis based on local extraction preserving generative adversarial networks

ABSTRACT Facial photo-to-sketch synthesis is crucial for entertainment and criminal investigations, yet challenges persist, including local detail blurring and identity feature loss. To mitigate these problems in face sketch synthesis, We propose a technique for synthesizing photographic sketches of faces using generative adversarial networks that focus on image localization to extract and preserve image identity features – namely LEPGAN. Our method introduces an image local extraction module, an attention mechanism in the generator, and an extended U-Net structure to enhance critical image features and preserve identity information. Multi-scale perceptual and local loss functions further enhance synthetic image quality. LEPGAN outperforms existing methods, producing images with crisper facial details that closely align with human visual perception, effectively preserving unique identity characteristics. Our approach represents a significant advancement in face sketch synthesis for various applications.