Standard U-Net Research Articles

A novel approach for semantic segmentation of automatic road network extractions from remote sensing images by modified UNet

Accurate and up-to-date road maps are crucial for numerous applications such as urban planning, automatic vehicle navigation systems, and traffic monitoring systems. However, even in the high resolutions remote sensing images, the background and roads look similar due to the occlusion of trees and buildings, and it is difficult to accurately segment the road network from complex background images. In this research paper, an algorithm based on deep learning was proposed to segment road networks from remote sensing images. This semantic segmentation algorithm was developed with a modified UNet. Because of the lower availability of remote sensing images for semantic segmentation, the data augmentation method was used. Initially, the semantic segmentation network was trained by a large number of training samples using traditional UNet architecture. After then, the number of training samples is reduced gradually, and measures the performance of a traditional UNet model. This basic UNet model gives better results in the form of accuracy, IOU, DICE score, and visualization of the image for the 362 training samples. The idea here is to simply extract road data from remote sensing images. As a result, unlike traditional UNet, there is no need for a deeper neural network encoder-decoder structure. Hence, the number of convolutional layers in the modified UNet is lower than that in the standard UNet. Therefore, the complexity of the deep learning architecture and the training time required by the road network model was reduced. The model performance measured by the intersection over union (IOU) was 93.71% and the average segmentation time of a single image was 0.28 sec. The results showed that the modified UNet could efficiently segment road networks from remote sensing images with identical backgrounds. It can be used under various situations.

STAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentation.

Acquisition of high quality manual annotations is vital for the development of segmentation algorithms. However, to create them we require a substantial amount of expert time and knowledge. Large numbers of labels are required to train convolutional neural networks due to the vast number of parameters that must be learned in the optimisation process. Here, we develop the STAMP algorithm to allow the simultaneous training and pruning of a UNet architecture for medical image segmentation with targeted channelwise dropout to make the network robust to the pruning. We demonstrate the technique across segmentation tasks and imaging modalities. It is then shown that, through online pruning, we are able to train networks to have much higher performance than the equivalent standard UNet models while reducing their size by more than 85% in terms of parameters. This has the potential to allow networks to be directly trained on datasets where very low numbers of labels are available.

An uncertainty-aware deep learning architecture with outlier mitigation for prostate gland segmentation in radiotherapy treatment planning.

Task automation is essential for efficient and consistent image segmentation in radiation oncology. We report on a deep learning architecture, comprising a U-Net and a variational autoencoder (VAE) for automatic contouring of the prostate gland incorporating interobserver variation for radiotherapy treatment planning. The U-Net/VAE generates an ensemble set of segmentations for each image CT slice. A novel outlier mitigation (OM) technique was implemented to enhance the model segmentation accuracy. The primary source dataset (source_prim) consisted of 19 200 CT slices (from 300 patient planning CT image datasets) with manually contoured prostate glands. A smaller secondary source dataset (source_sec) comprised 640 CT slices (from 10 patient CT datasets), where prostate glands were segmented by 5 independent physicians on each dataset to account for interobserver variability. Data augmentation via random rotation (<5 degrees), cropping, and horizontal flipping was applied to each dataset to increase sample size by a factor of 100. A probabilistic hierarchical U-Net with VAE was implemented and pretrained using the augmented source_prim dataset for 30 epochs. Model parameters of the U-Net/VAE were fine-tuned using the augmented source_sec dataset for 100 epochs. After the first round of training, outlier contours in the training dataset were automatically detected and replaced by the most accurate contours (based on Dice similarity coefficient, DSC) generated by the model. The U-Net/OM-VAE was retrained using the revised training dataset. Metrics for comparison included DSC, Hausdorff distance (HD, mm), normalized cross-correlation (NCC) coefficient, and center-of-mass (COM) distance (mm). Results for U-Net/OM-VAE with outliers replaced in the training dataset versus U-Net/VAE without OM were as follows: DSC=0.82±0.01 versus 0.80±0.02 (p=0.019), HD=9.18±1.22 versus 10.18±1.35mm (p=0.043), NCC=0.59±0.07 versus 0.62±0.06, and COM=3.36±0.81 versus 4.77±0.96mm over the average of 15 contours. For the average of 15 highest accuracy contours, values were as follows: DSC=0.90±0.02 versus 0.85±0.02, HD=5.47±0.02 versus 7.54±1.36mm, and COM=1.03±0.58 versus 1.46±0.68mm (p<0.03 for all metrics). Results for the U-Net/OM-VAE with outliers removed were as follows: DSC=0.78±0.01, HD=10.65±1.95mm, NCC=0.46±0.10, COM=4.17±0.79mm for the average of 15 contours, and DSC=0.88±0.02, HD=7.00±1.17mm, COM=1.58±0.63mm for the average of 15 highest accuracy contours. All metrics for U-Net/VAE trained on the source_prim and source_sec datasets via pretraining, followed by fine-tuning, show statistically significant improvement over that trained on the source_sec dataset only. Finally, all metrics for U-Net/VAE with or without OM showed statistically significant improvement over those for the standard U-Net. A VAE combined with a hierarchical U-Net and an OM strategy (U-Net/OM-VAE) demonstrates promise toward capturing interobserver variability and produces accurate prostate auto-contours for radiotherapy planning. The availability of multiple contours for each CT slice enables clinicians to determine trade-offs in selecting the "best fitting" contour on each CT slice. Mitigation of outlier contours in the training dataset improves prediction accuracy, but one must be wary of reduction in variability in the training dataset.

Using high-resolution imagery and deep learning to classify land-use following deforestation: a case study in Ethiopia

ABSTRACT National-scale assessments of post-deforestation land-use are crucial for decreasing deforestation and forest degradation-related emissions. In this research, we assess the potential of different satellite data modalities (single-date, multi-date, multi-resolution, and an ensemble of multi-sensor images) for classifying land-use following deforestation in Ethiopia using the U-Net deep neural network architecture enhanced with attention. We performed the analysis on satellite image data retrieved across Ethiopia from freely available Landsat-8, Sentinel-2 and Planet-NICFI satellite data. The experiments aimed at an analysis of (a) single-date images from individual sensors to account for the differences in spatial resolution between image sensors in detecting land-uses, (b) ensembles of multiple images from different sensors (Planet-NICFI/Sentinel-2/Landsat-8) with different spatial resolutions, (c) the use of multi-date data to account for the contribution of temporal information in detecting land-uses, and, finally, (d) the identification of regional differences in terms of land-use following deforestation in Ethiopia. We hypothesize that choosing the right satellite imagery (sensor) type is crucial for the task. Based on a comprehensive visually interpreted reference dataset of 11 types of post-deforestation land-uses, we find that either detailed spatial patterns (single-date Planet-NICFI) or detailed temporal patterns (multi-date Sentinel-2, Landsat-8) are required for identifying land-use following deforestation, while medium-resolution single-date imagery is not sufficient to achieve high classification accuracy. We also find that adding soft-attention to the standard U-Net improved the classification accuracy, especially for small-scale land-uses. The models and products presented in this work can be used as a powerful data resource for governmental and forest monitoring agencies to design and monitor deforestation mitigation measures and data-driven land-use policy.

Weighted average ensemble-based semantic segmentation in biological electron microscopy images

Semantic segmentation of electron microscopy images using deep learning methods is a valuable tool for the detailed analysis of organelles and cell structures. However, these methods require a large amount of labeled ground truth data that is often unavailable. To address this limitation, we present a weighted average ensemble model that can automatically segment biological structures in electron microscopy images when trained with only a small dataset. Thus, we exploit the fact that a combination of diverse base-learners is able to outperform one single segmentation model. Our experiments with seven different biological electron microscopy datasets demonstrate quantitative and qualitative improvements. We show that the Grad-CAM method can be used to interpret and verify the prediction of our model. Compared with a standard U-Net, the performance of our method is superior for all tested datasets. Furthermore, our model leverages a limited number of labeled training data to segment the electron microscopy images and therefore has a high potential for automated biological applications.

ConvUNeXt: An efficient convolution neural network for medical image segmentation

Recently, ConvNeXts constructing from standard ConvNet modules has produced competitive performance in various image applications. In this paper, an efficient model based on the classical UNet, which can achieve promising results with a low number of parameters, is proposed for medical image segmentation. Inspired by ConvNeXt, the designed model is called ConvUNeXt and towards reduction in the amount of parameters while retaining outstanding segmentation superiority. Specifically, we firstly improved the convolution block of UNet by using large convolution kernels and depth-wise separable convolution to considerably decrease the number of parameters; then residual connections in both encoder and decoder are added and pooling is abandoned via adopting convolution for down-sampling; during skip connection, a lightweight attention mechanism is designed to filter out noise in low-level semantic information and suppress irrelevant features, so that the network can pay more attention to the target area. Compared to the standard UNet, our model has 20% fewer parameters, meanwhile, experimental results on different datasets show that it exhibits superior segmentation performance when the amount of data is scarce or sufficient. Code will be available at https://github.com/1914669687/ConvUNeXt.

CEL-Unet: a novel CNN architecture for 3D Segmentation of Knee Bones affected by Severe Osteoarthritis for PSI-Based Surgical Planning.

Unet architectures are promising deep learning networks exploited to perform the automatic segmentation of bone CT images, in line with their ability to deal with pathological deformations and size-varying anatomies. However, bone degeneration, like the development of irregular osteophytes as well as mineral density alterations might interfere with this automated process and demand extensive manual refinement. The aim of this work is to implement an innovative Unet variant, the CEL-Unet, to improve the femur and tibia segmentation outcomes in osteoarthritic knee joints. In this network the decoding path is split into a region and contour-aware branch to increase the prediction reliability in such pathological conditions. The comparison between the segmentation results achieved with a standard Unet and its novel variant (CEL-Unet) was performed as follows: the Unet was trained with 5 different loss functions: Dice Loss, Focal Loss, Exponential Logarithmic Loss, Double Cross Entropy Loss and Distanced Cross Entropy loss. The CEL-Unet was instead trained with two loss functions, one for each of the network outputs, namely Mask and Edge, yielding the so-called Combined Edge Loss (CEL) function. A set of 259 knee CT scans was used to train the model and test segmentation performance. The CEL-Unet outperformed all other Unet-based models, reaching the highest Jaccard values of about 0.97 and 0.96 on femur and tibia, respectively. Clinical Relevance- With the increasing rate of Total Knee Arthoplasty deep learning-based methods can achieve fast accurate and automatic 3D segmentation of the knee joint bones to enhance new costumized pre-operative planning.

Retinal Vessel Segmentation Based on the Anam-Net Model

Accurate segmentation of retinal blood vessels can help ophthalmologists diagnose eye-related diseases such as diabetes and hypertension. The task of segmentation of the vessels comes with a number of challenges. Some of the challenges are due to haemorrhages and microaneurysms in fundus imaging, while others are due to the central vessel reflex and low contrast. Encoder-decoder networks have recently achieved excellent performance in retinal vascular segmentation at the trade-off of increased computational complexity. In this work, we use the Anam-Net model to accurately segment retinal vessels at a low computational cost. The Anam-Net model consists of a lightweight convolutional neural network (CNN) along with bottleneck layers in the encoder and decoder stages. Compared to the standard U-Net model and the R2U-Net model, the Anam-Net model has 6.9 times and 10.9 times fewer parameters. We evaluated the Anam-Net model on three open-access datasets: DRIVE, STARE, and CHASE_DB. The results show that the Anam-Net model achieves better segmentation accuracy compared to several state-of-the-art methods. For the DRIVE, STARE, and CHASE DB datasets, the model achieved {sensitivity and accuracy} of {0.8601, 0.9660}, {0.8697, 0.9728}, and {0.8553, 0.9746}, respectively. On the DRIVE, STARE, and CHASE_DB datasets, we also conduct cross-training experiments. The outcome of this experiment demonstrates the generalizability and robustness of the Anam-Net model.

Attention-aware 3D U-Net convolutional neural network for knowledge-based planning 3D dose distribution prediction of head-and-neck cancer.

PurposeDeep learning–based knowledge‐based planning (KBP) methods have been introduced for radiotherapy dose distribution prediction to reduce the planning time and maintain consistent high‐quality plans. This paper presents a novel KBP model using an attention‐gating mechanism and a three‐dimensional (3D) U‐Net for intensity‐modulated radiation therapy (IMRT) 3D dose distribution prediction in head‐and‐neck cancer.MethodsA total of 340 head‐and‐neck cancer plans, representing the OpenKBP—2020 AAPM Grand Challenge data set, were used in this study. All patients were treated with the IMRT technique and a dose prescription of 70 Gy. The data set was randomly divided into 64%/16%/20% as training/validation/testing cohorts. An attention‐gated 3D U‐Net architecture model was developed to predict full 3D dose distribution. The developed model was trained using the mean‐squared error loss function, Adam optimization algorithm, a learning rate of 0.001, 120 epochs, and batch size of 4. In addition, a baseline U‐Net model was also similarly trained for comparison. The model performance was evaluated on the testing data set by comparing the generated dose distributions against the ground‐truth dose distributions using dose statistics and clinical dosimetric indices. Its performance was also compared to the baseline model and the reported results of other deep learning‐based dose prediction models.ResultsThe proposed attention‐gated 3D U‐Net model showed high capability in accurately predicting 3D dose distributions that closely replicated the ground‐truth dose distributions of 68 plans in the test set. The average value of the mean absolute dose error was 2.972 ± 1.220 Gy (vs. 2.920 ± 1.476 Gy for a baseline U‐Net) in the brainstem, 4.243 ± 1.791 Gy (vs. 4.530 ± 2.295 Gy for a baseline U‐Net) in the left parotid, 4.622 ± 1.975 Gy (vs. 4.223 ± 1.816 Gy for a baseline U‐Net) in the right parotid, 3.346 ± 1.198 Gy (vs. 2.958 ± 0.888 Gy for a baseline U‐Net) in the spinal cord, 6.582 ± 3.748 Gy (vs. 5.114 ± 2.098 Gy for a baseline U‐Net) in the esophagus, 4.756 ± 1.560 Gy (vs. 4.992 ± 2.030 Gy for a baseline U‐Net) in the mandible, 4.501 ± 1.784 Gy (vs. 4.925 ± 2.347 Gy for a baseline U‐Net) in the larynx, 2.494 ± 0.953 Gy (vs. 2.648 ± 1.247 Gy for a baseline U‐Net) in the PTV_70, and 2.432 ± 2.272 Gy (vs. 2.811 ± 2.896 Gy for a baseline U‐Net) in the body contour. The average difference in predicting the D 99 value for the targets (PTV_70, PTV_63, and PTV_56) was 2.50 ± 1.77 Gy. For the organs at risk, the average difference in predicting the Dmax (brainstem, spinal cord, and mandible) and Dmean (left parotid, right parotid, esophagus, and larynx) values was 1.43 ± 1.01 and 2.44 ± 1.73 Gy, respectively. The average value of the homogeneity index was 7.99 ± 1.45 for the predicted plans versus 5.74 ± 2.95 for the ground‐truth plans, whereas the average value of the conformity index was 0.63 ± 0.17 for the predicted plans versus 0.89 ± 0.19 for the ground‐truth plans. The proposed model needs less than 5 s to predict a full 3D dose distribution of 64 × 64 × 64 voxels for a new patient that is sufficient for real‐time applications.ConclusionsThe attention‐gated 3D U‐Net model demonstrated a capability in predicting accurate 3D dose distributions for head‐and‐neck IMRT plans with consistent quality. The prediction performance of the proposed model was overall superior to a baseline standard U‐Net model, and it was also competitive to the performance of the best state‐of‐the‐art dose prediction method reported in the literature. The proposed model could be used to obtain dose distributions for decision‐making before planning, quality assurance of planning, and guiding‐automated planning for improved plan consistency, quality, and planning efficiency.

Automatic coastline extraction through enhanced sea-land segmentation by modifying Standard U-Net

Sea-land segmentation (SLS) is an essential remote sensing task for various coastal and environmental studies such as coastline extraction, coastal erosion, coastal area monitoring, and ship or iceberg detection. This study aims at improving the SLS performance by modifying the Standard U-Net (SUN) model and developing an automatic coastline extraction framework. SUN generally has an acceptable performance in many applications. However, better SLS outputs are needed for reliable coastline extraction. In our proposed framework, we firstly analyzed three different input images, including Red-Green-Blue (RGB), Normalized Difference Water Index (NDWI), and Near-Infrared (NIR) images. Secondly, we modified the SUN architecture to improve the segmentation results. The main modifications are using different loss functions and two fusion methods for RGB and NIR images. The segmentation results were then passed into the subsequent automatic coastline extraction pipeline based on morphological operations and pixel connectivity analysis. The training and testing steps were accomplished utilizing a benchmark dataset of China’s coastal areas. Moreover, another dataset consisting of a time series of Landsat-8 imagery from the southern Caspian Sea coastlines was collected to evaluate coastline extraction efficiency. The results indicate that the proposed modifications could effectively enhance the performance of the SUN, with the most significant improvement to the Intersection over Union (IoU) score being as high as 1.68% and 8.95% in China and Caspian Sea datasets, respectively, while outperforming other state-of-the-art models including FC-DenseNet and DeepLabV3+.

Joint Dense Residual and Recurrent Attention Network for DCE-MRI Breast Tumor Segmentation.

Breast cancer detection largely relies on imaging characteristics and the ability of clinicians to easily and quickly identify potential lesions. Magnetic resonance imaging (MRI) of breast tumors has recently shown great promise for enabling the automatic identification of breast tumors. Nevertheless, state-of-the-art MRI-based algorithms utilizing deep learning techniques are still limited in their ability to accurately separate tumor and healthy tissue. Therefore, in the current work, we propose an automatic and accurate two-stage U-Net-based segmentation framework for breast tumor detection using dynamic contrast-enhanced MRI (DCE-MRI). This framework was evaluated using T2-weighted MRI data from 160 breast tumor cases, and its performance was compared with that of the standard U-Net model. In the first stage of the proposed framework, a refined U-Net model was utilized to automatically delineate a breast region of interest (ROI) from the surrounding healthy tissue. Importantly, this automatic segmentation step reduced the impact of the background chest tissue on breast tumors' identification. For the second stage, we employed an improved U-Net model that combined a dense residual module based on dilated convolution with a recurrent attention module. This model was used to accurately and automatically segment the tumor tissue from healthy tissue in the breast ROI derived in the previous step. Overall, compared to the U-Net model, the proposed technique exhibited increases in the Dice similarity coefficient, Jaccard similarity, positive predictive value, sensitivity, and Hausdorff distance of 3%, 3%, 3%, 2%, and 16.2, respectively. The proposed model may in the future aid in the clinical diagnosis of breast cancer lesions and help guide individualized patient treatment.

State-of-the-art retinal vessel segmentation with minimalistic models

The segmentation of retinal vasculature from eye fundus images is a fundamental task in retinal image analysis. Over recent years, increasingly complex approaches based on sophisticated Convolutional Neural Network architectures have been pushing performance on well-established benchmark datasets. In this paper, we take a step back and analyze the real need of such complexity. We first compile and review the performance of 20 different techniques on some popular databases, and we demonstrate that a minimalistic version of a standard U-Net with several orders of magnitude less parameters, carefully trained and rigorously evaluated, closely approximates the performance of current best techniques. We then show that a cascaded extension (W-Net) reaches outstanding performance on several popular datasets, still using orders of magnitude less learnable weights than any previously published work. Furthermore, we provide the most comprehensive cross-dataset performance analysis to date, involving up to 10 different databases. Our analysis demonstrates that the retinal vessel segmentation is far from solved when considering test images that differ substantially from the training data, and that this task represents an ideal scenario for the exploration of domain adaptation techniques. In this context, we experiment with a simple self-labeling strategy that enables moderate enhancement of cross-dataset performance, indicating that there is still much room for improvement in this area. Finally, we test our approach on Artery/Vein and vessel segmentation from OCTA imaging problems, where we again achieve results well-aligned with the state-of-the-art, at a fraction of the model complexity available in recent literature. Code to reproduce the results in this paper is released.

Feasibility of the soft attention-based models for automatic segmentation of OCT kidney images.

Clinically, optical coherence tomography (OCT) has been utilized to obtain the images of the kidney's proximal convoluted tubules (PCTs), which can be used to quantify the morphometric parameters such as tubular density and diameter. Such parameters are useful for evaluating the status of the donor kidney for transplant. Quantifying PCTs from OCT images by human readers is a time-consuming and tedious process. Despite the fact that conventional deep learning models such as conventional neural networks (CNNs) have achieved great success in the automatic segmentation of kidney OCT images, gaps remain regarding the segmentation accuracy and reliability. Attention-based deep learning model has benefits over regular CNNs as it is intended to focus on the relevant part of the image and extract features for those regions. This paper aims at developing an Attention-based UNET model for automatic image analysis, pattern recognition, and segmentation of kidney OCT images. We evaluated five methods including the Residual-Attention-UNET, Attention-UNET, standard UNET, Residual UNET, and fully convolutional neural network using 14403 OCT images from 169 transplant kidneys for training and testing. Our results show that Residual-Attention-UNET outperformed the other four methods in segmentation by showing the highest values of all the six metrics including dice score (0.81 ± 0.01), intersection over union (IOU, 0.83 ± 0.02), specificity (0.84 ± 0.02), recall (0.82 ± 0.03), precision (0.81 ± 0.01), and accuracy (0.98 ± 0.08). Our results also show that the performance of the Residual-Attention-UNET is equivalent to the human manual segmentation (dice score = 0.84 ± 0.05). Residual-Attention-UNET and Attention-UNET also demonstrated good performance when trained on a small dataset (3456 images) whereas the performance of the other three methods dropped dramatically. In conclusion, our results suggested that the soft Attention-based models and specifically Residual-Attention-UNET are powerful and reliable methods for tubule lumen identification and segmentation and can help clinical evaluation of transplant kidney viability as fast and accurate as possible.

Automatic Left Ventricle Segmentation from Short-Axis Cardiac MRI Images Based on Fully Convolutional Neural Network.

Background: Left ventricle (LV) segmentation using a cardiac magnetic resonance imaging (MRI) dataset is critical for evaluating global and regional cardiac functions and diagnosing cardiovascular diseases. LV clinical metrics such as LV volume, LV mass and ejection fraction (EF) are frequently extracted based on the LV segmentation from short-axis MRI images. Manual segmentation to assess such functions is tedious and time-consuming for medical experts to diagnose cardiac pathologies. Therefore, a fully automated LV segmentation technique is required to assist medical experts in working more efficiently. Method: This paper proposes a fully convolutional network (FCN) architecture for automatic LV segmentation from short-axis MRI images. Several experiments were conducted in the training phase to compare the performance of the network and the U-Net model with various hyper-parameters, including optimization algorithms, epochs, learning rate, and mini-batch size. In addition, a class weighting method was introduced to avoid having a high imbalance of pixels in the classes of image’s labels since the number of background pixels was significantly higher than the number of LV and myocardium pixels. Furthermore, effective image conversion with pixel normalization was applied to obtain exact features representing target organs (LV and myocardium). The segmentation models were trained and tested on a public dataset, namely the evaluation of myocardial infarction from the delayed-enhancement cardiac MRI (EMIDEC) dataset. Results: The dice metric, Jaccard index, sensitivity, and specificity were used to evaluate the network’s performance, with values of 0.93, 0.87, 0.98, and 0.94, respectively. Based on the experimental results, the proposed network outperforms the standard U-Net model and is an advanced fully automated method in terms of segmentation performance. Conclusion: This proposed method is applicable in clinical practice for doctors to diagnose cardiac diseases from short-axis MRI images.

A Modular System Based on U-Net for Automatic Building Extraction from very high-resolution satellite images

Recently, convolutional neural networks have grown in popularity in a variety of fields, such as computer vision and audio and text processing. This importance is due to the performance of this type of neural network in the state of the art, and in a wide variety of disciplines. However, the use of convolutional neural networks has not been widely used for remote sensing applications until recently. In this paper, we propose a CNN-based system capable of efficiently extracting buildings from very high-resolution satellite images, by combining the performances of the two architectures; U-Net and VGG19, which is obtained by putting two blocks in parallel based mainly on U-Net: The first block is a standard U-Net, and the second is designed by replacing the contraction path of standard U-Net with the pre-trained weights of VGG19.

Multi-Scale ConvLSTM Attention-Based Brain Tumor Segmentation

In computer vision, there are various machine learning algorithms that have proven to be very effective. Con-volutional Neural Networks (CNNs) are a kind of deep learning algorithms that became mostly used in image processing with a remarkable success rate compared to conventional machine learning algorithms. CNNs are widely used in different computer vision fields, especially in the medical domain. In this study, we perform a semantic brain tumor segmentation using a novel deep learning architecture we called multi-scale ConvLSTM Attention Neural Network, that resides in Convolutional Long-Short-Term-Memory (ConvLSTM) and Attention units with the use of multiple feature extraction blocks such as Inception, Squeeze-Excitation and Residual Network block. The use of such blocks separately is known to boost the performance of the model, in our case we show that their combination has also a beneficial effect on the accuracy. Experimental results show that our model performs brain tumor segmentation effectively compared to standard U-Net, Attention U-net and Fully Connected Network (FCN), with 79.78 Dice score using our method compared to 78.61, 73.65 and 72.89 using Attention U-net, standard U-net and FCN respectively.

TransNorm: Transformer Provides a Strong Spatial Normalization Mechanism for a Deep Segmentation Model

In the past few years, convolutional neural networks (CNNs), particularly U-Net, have been the prevailing technique in the medical image processing era. Specifically, the U-Net model, as well as its alternatives, have successfully managed to address a wide variety of medical image segmentation tasks. However, these architectures are intrinsically imperfect as they fail to exhibit long-range interactions and spatial dependencies leading to a severe performance drop in the segmentation of medical images with variable shapes and structures. Transformers, preliminary proposed for sequence-to-sequence prediction, have arisen as surrogate architectures to precisely model global information assisted by the self-attention mechanism. Despite being feasibly designed, utilizing a pure Transformer for image segmentation purposes can result in limited localization capacity stemming from inadequate low-level features. Thus, a line of research strives to design robust variants of Transformer-based U-Net. In this paper, we propose Trans-Norm, a novel deep segmentation framework which concomitantly consolidates a Transformer module into both encoder and skip-connections of the standard U-Net. We argue that the expedient design of skip-connections can be crucial for accurate segmentation as it can assist feature fusion between the expanding and contracting paths. In this respect, we derive a Spatial Normalization mechanism from the Transformer module to adaptively recalibrate the skip connection path. Extensive experiments across three typical tasks for medical image segmentation demonstrate the effectiveness of TransNorm. The codes and trained models are publicly available at github.

An Improved Kidney Tumor Prediction Using Deep Convolutional Neural Network-Restricted Boltzmann Machine Technique in Medical Image Segmentation

Automatic medical image segmentation has become increasingly important as contemporary medical imaging has become more widely available and used. Existing image segmentation solutions however lack the necessary functionality for simple medical image segmentation pipeline design. Pipelines that have already been deployed are frequently standalone software that has been optimised for a certain public data collection. As a result, the open-source python module deep-Convolutional neural network-Restricted Boltzmann Machine (deep CNNRBM) was introduced in this research work. The goal of Deep CNN-purpose RBMs is to have an easy-touse API that allows for the rapid creation of medical image segmentation transmission lines that include data augmentation, metrics, data I/O pre-processing, patch wise analysis, a library of pre-built deep neural networks, and fully automated assessment. Similarly, comprehensive pipeline customisation is possible because of strong configurability and many open interfaces. The dataset of Kidney tumor Segmentation challenge 2019 (KiTS19) acquired a strong predictor with respect to the standard 3D U-net model after cross-validation using deep CNNRBM. To that purpose, deep CNN-RBM, an expressive deep learning medical image segmentation architecture is introduced. The CNN sub-model captures frame-level spatial features automatically while the RBM submodel fuses spatial data over time to learn higher-level semantics in kidney tumor prediction. A neural network recognises medical picture segmentation, which is initiated using RBM to second-order collected data and then fine-tuned using back propagation to be more differential. According to the simulation outcome, the proposed deep CNN-RBM produced good classification results on the kidney tumour segmentation dataset.

Graph neural networks for laminar flow prediction around random two-dimensional shapes

In recent years, the domain of fast flow field prediction has been vastly dominated by pixel-based convolutional neural networks. Yet, the recent advent of graph convolutional neural networks (GCNNs) has attracted considerable attention in the computational fluid dynamics (CFD) community. In this contribution, we proposed a GCNN structure as a surrogate model for laminar flow prediction around two-dimensional (2D) obstacles. Unlike traditional convolution on image pixels, the graph convolution can be directly applied on body-fitted triangular meshes, hence yielding an easy coupling with CFD solvers. The proposed GCNN model is trained over a dataset composed of CFD-computed laminar flows around 2000 random 2D shapes. Accuracy levels are assessed on reconstructed velocity and pressure fields around out-of-training obstacles and are compared with that of standard U-net architectures, especially in the boundary layer area.

Randomly connected neural networks for self-supervised monocular depth estimation

ABSTRACT A plethora of deep learning-based methods have been proposed for self-supervised monocular depth estimation. The majority of these models utilise a U-Net-based architecture for disparity estimation. However, this architecture may not be optimal, and as shown in self-supervised approaches, replacing standard U-Net encoder with more complex architectures like the ResNet encoder achieves superior performance. In monocular depth estimation, the success of the method is attributed to the model architecture design capable of extracting high order features from a single image and the loss function for enforcing a strong feature distribution. In this work, a novel randomly connected encoder-decoder architecture has been designed for self-supervised monocular depth estimation. To enable efficient search in the connection space, the ‘cascaded random search’ approach is introduced for the generation of random network architectures. We introduce a new U-Net topology capable of utilising the semantic information of feature maps. For high quality image reconstructions, a loss function has been proposed, which efficiently extends perceptual and adversarial loss to multiple scales. We conduct performance evaluation on two surgical datasets, including comparisons to state-of-the-art self-supervised depth estimation methods. The performance evaluation analysis verifies the superiority of our randomly connected network architecture.