U-shaped Convolutional Network Research Articles

RAGE-Net: Enhanced retinal vessel segmentation U-shaped network using Gabor convolution

Extracting vessel morphology from fundus images is pivotal in acquiring pathological insights and enabling early diagnosis of retinal disorders. Manual segmentation of retinal vessels requires a high degree of expertise and is notably time-intensive. Although existing deep learning techniques for retinal vessel segmentation predominantly hinge on U-shaped convolutional neural networks, significant headway has been made, complexities persist in delineating faint, low-contrast vessels amidst noisy backgrounds. To confront these challenges, we propose an innovative U-shaped convolutional neural network fortified with oriented priors, labeled as the Receptive Field Aggregating Gabor Enhance Network (RAGE-Net). We revamp the conventional U-shaped convolutional network with a foundation in Gabor wavelet and Gabor convolutional network, introducing a Gabor Matching Enhance Architecture (GMEA) amalgamated into the U-shaped convolutional network. This architecture comprises two distinct modules. Initially, a Dual-scale Gabor Enhance Module (DGEB) is introduced to bolster vessel continuity and effectively fortify delicate vessels by integrating oriented feature enhancement through Gabor convolution. Subsequently, a Receptive Field Pyramid Module (RPM) is proposed to supplant the escalation of scale count in the Gabor filter bank for vessel alignment, also serving as feature fusion to enhance the network's comprehensive vessel discernment. In comparison to U-Net, our model boasts fewer parameters and surpasses in terms of sensitivity, accuracy, and F1 score on the DRIVE dataset by 3.58%, 0.34%, and 2.29%, respectively. Our model showcases stellar performance across three public datasets: DRIVE, STARE, and CHASE_DB1, with sensitivity values of 0.8172, 0.8126, and 0.8540 and accuracy figures of 0.9708, 0.9725, and 0.9757, respectively. The analysis on three datasets reveals that our model demonstrates distinct strengths in AUC and Se as a result of integrating GMEA, which comprises RPM and DGEB, into U-shaped networks. However, it does not reduce overall accuracy to improve the model's ability to perceive weak contrast and small blood vessels. While maintaining superior Acc, our model also demonstrates advancements in Sp and F1 score, indicating a balanced progression in multiple evaluation metrics.

Comparative Analysis of Transformer Integration in U-net Networks for Enhanced Medical Image Segmentation

Transformer is popular in Natural Language Processing (NLP) and is a cornerstone of large models. Transformer has been used by researchers to address the limitations of Convolutional Neural Networks (CNNs) in medical picture segmentation models. Through an extensive literature review and case studies, this paper comparatively analyzes the performance of different models in this field, summarizes different methods of integrating transformers into U-net, and points out existing gaps and challenges. Research has found that the Transformer model can significantly improve the accuracy and efficiency of medical image analysis. The paper discusses the advantages, disadvantages, innovations, performance, and complexity of various models in detail, and shows how to enhance performance by integrating the Transformer structure into the U-net network. In particular, the paper also analyzes the advantages of Transformers that are most suitable for integration into the encoder part and highlights the balance that needs to be made between improving performance and computational cost. The conclusion shows that although there is no perfect model, optimal performance and efficiency can be achieved by selecting different combinations of Transformer and U-net according to the actual situation. It can be seen from the networks’ performance that the mixed use of a U-shaped convolutional network and Transformer module has good development prospects and high research significance.

NSNet: An N-Shaped Convolutional Neural Network with Multi-Scale Information for Image Denoising

Deep learning models with convolutional operators have received widespread attention for their good image denoising performance. However, since the convolutional operation prefers to extract local features, the extracted features may lose some global information, such as texture, structure, and color characteristics, when the object in the image is large. To address this issue, this paper proposes an N-shaped convolutional neural network with the ability to extract multi-scale features to capture more useful information and alleviate the problem of global information loss. The proposed network has two main parts: a multi-scale input layer and a multi-scale feature extraction layer. The former uses a two-dimensional Haar wavelet to create an image pyramid, which contains the corrupted image’s high- and low-frequency components at different scales. The latter uses a U-shaped convolutional network to extract features at different scales from this image pyramid. The method sets the mean-squared error as the loss function and uses the residual learning strategy to learn the image noise directly. Compared with some existing image denoising methods, the proposed method shows good performance in gray and color image denoising, especially in textures and contours.

A Super-resolution Guided Network for Improving Automated Thyroid Nodule Segmentation

Background and Objective: A thyroid nodule is an abnormal lump that grows in the thyroid gland, which is the early symptom of thyroid cancer. In order to diagnose and treat thyroid cancer at the earliest stage, it is desired to characterize the nodule accurately. Ultrasound thyroid nodules segmentation is a challenging task due to the speckle noise, intensity heterogeneity, low contrast and low resolution. In this paper, we propose a novel framework to improve the accuracy of thyroid nodules segmentation. Methods: Different from previous work, a super-resolution reconstruction network is firstly constructed to upscale the resolution of the input ultrasound image. After that, our proposed N-shape network is utilized to perform the segmentation task. The guidance of super-resolution reconstruction network can make the high-frequency information of the input thyroid ultrasound image richer and more comprehensive than the original image. Our N-shape network consists of several atrous spatial pyramid pooling blocks, a multi-scale input layer, a U-shape convolutional network with attention blocks and a proposed parallel atrous convolution(PAC) module. These modules are conducive to capture context information at multiple scales so that semantic features can be fully utilized for lesion segmentation. Especially, our proposed PAC module is beneficial to further improve the segmentation by extracting high-level semantic features from different receptive fields. We use the UTNI-2021 dataset for model training, validating and testing. Results: The experimental results show that our proposed method achieve a Dice value of 91.9%, a mIoU value of 87.0%, a Precision value of 88.0%, a Recall value 83.7% and a F1-score value of 84.3%, which outperforms most state-of-the-art methods. Conclusions: Our method achieves the best performance on the UTNI-2021 dataset and provides a new way of ultrasound image segmentation. We believe that our method can provide doctors with reliable auxiliary diagnosis information in clinical practice.

N-Net: A novel dense fully convolutional neural network for thyroid nodule segmentation.

Medical image segmentation is an essential component of computer-aided diagnosis (CAD) systems. Thyroid nodule segmentation using ultrasound images is a necessary step for the early diagnosis of thyroid diseases. An encoder-decoder based deep convolutional neural network (DCNN), like U-Net architecture and its variants, has been extensively used to deal with medical image segmentation tasks. In this article, we propose a novel N-shape dense fully convolutional neural network for medical image segmentation, referred to as N-Net. The proposed framework is composed of three major components: a multi-scale input layer, an attention guidance module, and an innovative stackable dilated convolution (SDC) block. First, we apply the multi-scale input layer to construct an image pyramid, which achieves multi-level receiver field sizes and obtains rich feature representation. After that, the U-shape convolutional network is employed as the backbone structure. Moreover, we use the attention guidance module to filter the features before several skip connections, which can transfer structural information from previous feature maps to the following layers. This module can also remove noise and reduce the negative impact of the background. Finally, we propose a stackable dilated convolution (SDC) block, which is able to capture deep semantic features that may be lost in bilinear upsampling. We have evaluated the proposed N-Net framework on a thyroid nodule ultrasound image dataset (called the TNUI-2021 dataset) and the DDTI publicly available dataset. The experimental results show that our N-Net model outperforms several state-of-the-art methods in the thyroid nodule segmentation tasks.

Scene terrain classification for autonomous vehicle navigation based on semantic segmentation method

Autonomous transportation is a new paradigm of an Industry 5.0 cyber-physical system that provides a lot of opportunities in smart logistics applications. The safety and reliability of deep learning-driven systems are still a question under research. The safety of an autonomous guided vehicle is dependent on the proper selection of sensors and the transmission of reflex data. Several academics worked on sensor-based difficulties by developing a sensor correction system and fine-tuning algorithms to regulate the system’s efficiency and precision. In this paper, the introduction of vision sensor and its scene terrain classification using a deep learning algorithm is performed with proposed datasets during sensor failure conditions. The proposed classification technique is to identify the mobile robot obstacle and obstacle-free path for smart logistic vehicle application. To analyze the information from the acquired image datasets, the proposed classification algorithm employs segmentation techniques. The analysis of proposed dataset is validated with U-shaped convolutional network (U-Net) architecture and region-based convolutional neural network (Mask R-CNN) architecture model. Based on the results, the selection of 1400 raw image datasets is trained and validated using semantic segmentation classifier models. For various terrain dataset clusters, the Mask R-CNN classifier model method has the highest model accuracy of 93%, that is, 23% higher than the U-Net classifier model algorithm, which has the lowest model accuracy nearly 70%. As a result, the suggested Mask R-CNN technique has a significant potential of being used in autonomous vehicle applications.

Segmentation and recognition of breast ultrasound images based on an expanded U-Net.

This paper establishes a fully automatic real-time image segmentation and recognition system for breast ultrasound intervention robots. It adopts the basic architecture of a U-shaped convolutional network (U-Net), analyses the actual application scenarios of semantic segmentation of breast ultrasound images, and adds dropout layers to the U-Net architecture to reduce the redundancy in texture details and prevent overfitting. The main innovation of this paper is proposing an expanded training approach to obtain an expanded of U-Net. The output map of the expanded U-Net can retain texture details and edge features of breast tumours. Using the grey-level probability labels to train the U-Net is faster than using ordinary labels. The average Dice coefficient (standard deviation) and the average IOU coefficient (standard deviation) are 90.5% (±0.02) and 82.7% (±0.02), respectively, when using the expanded training approach. The Dice coefficient of the expanded U-Net is 7.6 larger than that of a general U-Net, and the IOU coefficient of the expanded U-Net is 11 larger than that of the general U-Net. The context of breast ultrasound images can be extracted, and texture details and edge features of tumours can be retained by the expanded U-Net. Using an expanded U-Net can quickly and automatically achieve precise segmentation and multi-class recognition of breast ultrasound images.

Multiscale fused network with additive channel–spatial attention for image segmentation

Class imbalance is a challenge in image segmentation. Deep learning has demonstrated potential advantages as a promising approach in solving such challenging problems because of its powerful ability to extract features. To treat class imbalance in medical image segmentation effectively, in this study, deep learning technology is employed. Further, a new approach, referred to as multiscale fused network with additive channel–spatial attention (MSF-ACSA), is developed. In particular, using the U-shape convolutional network as the structure of the main body network, an additive channel–spatial attention (ACSA) module, which exploits high-level features to guide low-level features for optimal responses, is designed. Therefore, the proposed MSF-ACSA can availably focus on regions of interest while adaptively recalibrating the channel significance of feature maps. Simultaneously, a multiscale fusion with deep supervision module is introduced to incorporate fine-grained details with coarse-grained semantics from diverse scale features. Furthermore, a focal weighted Tversky loss function is proposed to mitigate class imbalance issues. Numerous experimental results indicate that the proposed MSF-ACSA achieves the best performance when compared with state-of-the-art approaches for medical image segmentation.

Dual-Stage U-Shape Convolutional Network for Esophageal Tissue Segmentation in OCT Images

Automatic segmentation is the crucial step for esophageal optical coherence tomography (OCT) image processing, which is able to highlight diagnosis-related tissue layers and provide characteristics such as shape and thickness for esophageal disease diagnosis. This study proposes a dual-stage framework using a specifically designed encoder-decoder network configuration for accurate and reliable esophageal layer segmentation, which is named as the dual-stage U-shape convolutional network (D-UCN). The proposed approach utilized one UCN to locate the target tissue region, which is followed by another UCN with similar architecture to achieve the final segmentation. In this way, the proposed strategy effectively solves the problems encountered in our previous studies, such as disturbance from neighboring diagnostically unrelated tissues, probe protection sheaths from the imaging equipment and the inevitable speckle noise. Experimental results on esophageal OCT B-scans from C57BL mice demonstrated that the proposed dual-stage framework achieved performance comparable to manual segmentation. The effectiveness and advantages of the dual-stage strategy are also confirmed in comparison with graph theory dynamic program (GTDP) and U-Net.

Detection of laser-induced optical defects based on image segmentation

A number of vision-based methods for detecting laser-induced defects on optical components have been implemented to replace the time-consuming manual inspection. While deep-learning-based methods have achieved state-of-the-art performances in many visual recognition tasks, their success often hinges on the availability of a large number of labeled training sets. In this paper, we propose a surface defect detection method based on image segmentation with a U-shaped convolutional network (U-Net). The designed network was trained on paired sets of online and offline images of optics from a large laser facility. We show in our experimental evaluation that our approach can accurately locate laser-induced defects on the optics in real time. The main advantage of the proposed method is that the network can be trained end to end on small samples, without the requirement for manual labeling or manual feature extraction. The approach can be applied to the daily inspection and maintenance of optical components in large laser facilities.

Joint Optic Disc and Cup Segmentation Based on Multi-Label Deep Network and Polar Transformation.

Glaucoma is a chronic eye disease that leads to irreversible vision loss. The cup to disc ratio (CDR) plays an important role in the screening and diagnosis of glaucoma. Thus, the accurate and automatic segmentation of optic disc (OD) and optic cup (OC) from fundus images is a fundamental task. Most existing methods segment them separately, and rely on hand-crafted visual feature from fundus images. In this paper, we propose a deep learning architecture, named M-Net, which solves the OD and OC segmentation jointly in a one-stage multi-label system. The proposed M-Net mainly consists of multi-scale input layer, U-shape convolutional network, side-output layer, and multi-label loss function. The multi-scale input layer constructs an image pyramid to achieve multiple level receptive field sizes. The U-shape convolutional network is employed as the main body network structure to learn the rich hierarchical representation, while the side-output layer acts as an early classifier that produces a companion local prediction map for different scale layers. Finally, a multi-label loss function is proposed to generate the final segmentation map. For improving the segmentation performance further, we also introduce the polar transformation, which provides the representation of the original image in the polar coordinate system. The experiments show that our M-Net system achieves state-of-the-art OD and OC segmentation result on ORIGA data set. Simultaneously, the proposed method also obtains the satisfactory glaucoma screening performances with calculated CDR value on both ORIGA and SCES datasets.