Neural Network For Semantic Segmentation Research Articles

An artificial intelligence algorithm for automated blastocyst morphometric parameters demonstrates a positive association with implantation potential

Blastocyst selection is primarily based on morphological scoring systems and morphokinetic data. These methods involve subjective grading and time-consuming techniques. Artificial intelligence allows for objective and quick blastocyst selection. In this study, 608 blastocysts were selected for transfer using morphokinetics and Gardner criteria. Retrospectively, morphometric parameters of blastocyst size, inner cell mass (ICM) size, ICM-to-blastocyst size ratio, and ICM shape were automatically measured by a semantic segmentation neural network model. The model was trained on 1506 videos with 102 videos for validation with no overlap between the ICM and trophectoderm models. Univariable logistic analysis found blastocyst size and ICM-to-blastocyst size ratio to be significantly associated with implantation potential. Multivariable regression analysis, adjusted for woman age, found blastocyst size to be significantly associated with implantation potential. The odds of implantation increased by 1.74 for embryos with a blastocyst size greater than the mean (147 ± 19.1 μm). The performance of the algorithm was represented by an area under the curve of 0.70 (p < 0.01). In conclusion, this study supports the association of a large blastocyst size with higher implantation potential and suggests that automatically measured blastocyst morphometrics can be used as a precise, consistent, and time-saving tool for improving blastocyst selection.

Corrigendum to CrackDenseLinkNet: a deep convolutional neural network for semantic segmentation of cracks on concrete surface images

Corrigendum to CrackDenseLinkNet: a deep convolutional neural network for semantic segmentation of cracks on concrete surface images

Enhancing identification of digital rock images using super-resolution deep neural network

Identification of pore and minerals from digital rock images plays an important role in characterization of oil and gas reservoirs. Traditional methods usually require high costs of time and labor. With the rapid development of artificial intelligence, deep learning methods have been applied to geosciences. Among them, semantic segmentation neural networks are used to identify rock images, but traditional semantic segmentation networks require high-resolution input images to improve performance and can only generate masks with the same resolutions as the input images, which again requires high-performance equipment and increases computational and equipment costs. In this paper, we propose an approach to enhancing identification of digital rock images with super-resolution neural network (SSRN) which enables low-resolution input images to generate high-resolution representations. We use SSRN to train the 10-classification thin-section images, and compare it with traditional deep learning methods. It indicates that SSRN can effectively improve the performance of the generated high-resolution masks.

Domain‐adapted driving scene understanding with uncertainty‐aware and diversified generative adversarial networks

AbstractAutonomous vehicles are required to operate in an uncertain environment. Recent advances in computational intelligence techniques make it possible to understand driving scenes in various environments by using a semantic segmentation neural network, which assigns a class label to each pixel. It requires massive pixel‐level labelled data to optimise the network. However, it is challenging to collect sufficient data and labels in the real world. An alternative solution is to obtain synthetic dense pixel‐level labelled data from a driving simulator. Although the use of synthetic data is a promising way to alleviate the labelling problem, models trained with virtual data cannot generalise well to realistic data due to the domain shift. To fill this gap, the authors propose a novel uncertainty‐aware generative ensemble method. In particular, ensembles are obtained from different optimisation objectives, training iterations, and network initialisation so that they are complementary to each other to produce reliable predictions. Moreover, an uncertainty‐aware ensemble scheme is developed to derive fused prediction by considering the uncertainty from ensembles. Such a design can make better use of the strengths of ensembles to enhance adapted segmentation performance. Experimental results demonstrate the effectiveness of our method on three large‐scale datasets.

CrackDenseLinkNet: a deep convolutional neural network for semantic segmentation of cracks on concrete surface images

Cracks are the defects formed by cyclic loading, fatigue, shrinkage, creep, and so on. In addition, they represent the deterioration of the structures over some time. Therefore, it is essential to detect and classify them according to the condition grade at the early stages to prevent the collapse of structures. Deep learning-based semantic segmentation convolutional neural network (CNN) has millions of learnable parameters. However, depending on the complexity of the CNN, it takes hours to days to train the network fully. In this study, an encoder network DenseNet and modified LinkNet with five upsampling blocks were used as a decoder network. The proposed network is referred to as the “CrackDenseLinkNet” in this work. CrackDenseLinkNet has 19.15 million trainable parameters, although the input image size is 512 × 512 and has a deeper encoder. CrackDenseLinkNet and four other state-of-the-art (SOTA) methods were evaluated on three public and one private datasets. The proposed CNN, CrackDenseLinkNet, outperformed the best SOTA method, CrackSegNet, by 2.2% of F1-score on average across the four datasets. Lastly, a crack profile analysis demonstrated that the CrackDenseLinkNet has lesser variance in relative errors for the crack width, length, and area categories against the ground-truth data. The code and datasets can be downloaded at https://github.com/preethamam/CrackDenseLinkNet-DeepLearning-CrackSegmentation.

SEGMENTATION OF PICTURES CONTAINING BLADE EDGE OF A GAS TURBINE ENGINE

The article describes common techniques for semantic segmentation pictures containing edges of gas turbine engines blades for detecting left and right borders for further using in forming trajectory algorithms with direct metal deposition. For analysis such metrics, as pixel accuracy, mean pixel accuracy, intersection over union, frequency weighed intersection over union are used. Classic method of computer vision with threshold filters, border segmentation neural network method, fully convoluted neural network for semantic segmentation are focused on. The classic method of computer vision process image by several sequential applied filters: translate RBG to HSL, select lightness layer, threshold for this layer, morphological transformation, select top and bottom pixels in blade edge. This method gave 95,18 % pixel accuracy and 65,19 % intersection over union. Several architectures neural network for edge’s border segmentation, such as DexiNed, RCF, PiDiNet were compared. PiDiNet gave the best result: this architecture gave 96,37 % pixel accuracy and 77,57 % intersection over union. The last method in this research was fully convoluted neural network. 75 combinations of encoders and decoders architectures were trained and tested. The represented encoders were ResNet34, ResNet50, ResNet101, VGG11, VGG16, VGG19, InceptionResNetV2, InceptionV4, Efficientnet-b0, Efficientnet-b4, Efficientnet-b7, Xception. The represented decoders architectures were Unet, Unet++, MAnet, Linknet, PSPNet, FPN, DeepLabV3, DeepLabV3+, PAN. Fully convoluted neural network method gave the best result. The most accurate combination was Unet-InceptionResNetV2 model with 99,22 % pixel accuracy and 97,25 % intersection over union metric. The best method for semantic segmentation pictures contain blade edges was chosen.

Application of optical flow algorithm for drift correction in electron microscopy images.

Transmission electron microscopy (TEM) image drift correction has been effectively addressed using diverse approaches, including the cross correlation algorithm (CC) and other strategies. However, most of the strategies fall short of achieving sufficient accuracy or cannot strike a balance between time consumption and accuracy. The present study proposes a TEM image drift correction strategy that enhances accuracy without any additional time consumption. Unlike the CC algorithm that matches pixels one by one, our approach involves the extraction of multiple feature points from the first TEM image and then uses the Lucas-Kanade (LK) optical flow algorithm to calculate the optical field of these feature points in the subsequent TEM images. The LK algorithm is used to calculate the instantaneous velocity of these feature points, which can help track the movement of the TEM image series. In addition, a high-precision sub-pixel level correction strategy by the utilization of linear interpolation during the correction process is developed in this work. Experimental results confirm that this strategy offers superior accuracy in comparison with the CC algorithm and also is insensitive to the size of the image. Furthermore, we offer a semantic segmentation neural network for electron microscope image pre-processing, thereby expanding the applicability of our methodology.

Mutually reinforcing non-local neural networks for semantic segmentation

The ability to capture pixels' long-distance interdependence is beneficial to semantic segmentation. In addition, semantic segmentation requires the effective use of pixel-to-pixel similarity in the channel direction to enhance pixel regions. Asymmetric Non-local Neural Networks (ANNet) combine multi-scale spatial pyramidal pooling modules and Non-local blocks to reduce model parameters without sacrificing performance. However, ANNet does not consider pixel similarity in the channel direction in the feature map, so its segmentation effect is not ideal. This article proposes a Mutually Reinforcing Non-local Neural Networks (MRNNet) to improve ANNet. MRNNet consists specifically of the channel enhancement regions module (CERM), and the position-enhanced pixels module (PEPM). In contrast to Asymmetric Fusion Non-local Block (AFNB) in ANNet, CERM does not combine the feature maps of the high and low stages, but rather utilizes the auxiliary loss function of ANNet. Calculating the similarity between feature maps in channel direction improves the category representation of feature maps in the channel aspect and reduces matrix multiplication computation. PEPM enhances pixels in the spatial direction of the feature map by calculating the similarity between pixels in the spatial direction of the feature map. Experiments reveal that our segmentation accuracy for cityscapes test data reaches 81.9%. Compared to ANNet, the model's parameters are reduced by 11.35 (M). Given ten different pictures with a size of 2048 × 1024, the average reasoning time of MRNNet is 0.103(s) faster than that of the ANNet model.

Pruning Points Detection of Sweet Pepper Plants Using 3D Point Clouds and Semantic Segmentation Neural Network.

Automation in agriculture can save labor and raise productivity. Our research aims to have robots prune sweet pepper plants automatically in smart farms. In previous research, we studied detecting plant parts by a semantic segmentation neural network. Additionally, in this research, we detect the pruning points of leaves in 3D space by using 3D point clouds. Robot arms can move to these positions and cut the leaves. We proposed a method to create 3D point clouds of sweet peppers by applying semantic segmentation neural networks, the ICP algorithm, and ORB-SLAM3, a visual SLAM application with a LiDAR camera. This 3D point cloud consists of plant parts that have been recognized by the neural network. We also present a method to detect the leaf pruning points in 2D images and 3D space by using 3D point clouds. Furthermore, the PCL library was used to visualize the 3D point clouds and the pruning points. Many experiments are conducted to show the method's stability and correctness.

320 Deep-Learning Cortical Thickness Analysis as Predictor for Shunt Surgery Effectiveness in Possible iNPH Patients

INTRODUCTION: While optimal treatment for iNPH is quite known, diagnosis remains a topic of debate, with different procedures suggested as potential predictors for shunt surgery effectiveness. Various tests, biomarkers, and neuroimaging techniques can be used; however, the diagnostic accuracy is below optimal when performed with neuroimaging techniques alone and not always cost-effective when achieved with CSF biomarkers. Quantitative computational methods for cortical thickness analysis have become available in the last decade, and they only require regularly available T1-weighted brain MR images. METHODS: We analyzed 294 total patients referred to our clinic from January 2015 until December 2021. After exclusion criteria, the final sample consisted of 98 possible iNPH patients: 64 were diagnosed as probable iNPH and underwent VPS surgery, while 34 did not receive a surgical indication after diagnostic procedures. We investigated differences in cortical thickness for all possible iNPH patients using a public deep-learning based neuroimaging pipeline composed by an advanced semantic segmentation neural network architecture and automated surface-based cortical thickness analysis. RESULTS: In 38 total patients with a negative CSFTT or not responsive to shunt surgery, a significant localized cortical thinning was seen in the lateral surface of the frontal and parietal lobes (bilateral superior and middle frontal gyrus, bilateral pars opercularis, bilateral superior parietal gyrus, bilateral supramarginal gyrus, left inferior parietal gyrus, bilateral precuneus). CONCLUSIONS: Preoperative cortical thickness is a feasible analysis that might be useful in defining the optimal therapeutic plan for possible iNPH patients. A large multicentric study is encouraged to elucidate specific patterns and ratios of cortical thickness that might be used as potential predictors for shunt surgery effectiveness.

Aggregated micropatch-based deep learning neural network for ultrasonic diagnosis of cirrhosis

Despite the advancements in the diagnosis of early-stage cirrhosis, the accuracy in the diagnosis using ultrasound is still challenging owing to the presence of various image artifacts, which results in poor visual quality of the textural and lower-frequency components. In this study, we propose an end-to-end multistep network called CirrhosisNet that includes two transfer-learned convolutional neural networks for semantic segmentation and classification tasks. It uses a uniquely designed image, called an aggregated micropatch (AMP), as an input image to the classification network, thereby assessing whether the liver is in a cirrhotic stage. With a prototype AMP image, we synthesized a bunch of AMP images while retaining the textural features. This synthesis significantly increases the number of insufficient cirrhosis-labeled images, thereby circumventing overfitting issues and optimizing network performance. Furthermore, the synthesized AMP images contained unique textural patterns, mostly generated on the boundaries between adjacent micropatches (μ-patches) during their aggregation. These newly created boundary patterns provide rich information regarding the texture features of the ultrasound image, thereby making cirrhosis diagnosis more accurate and sensitive. The experimental results demonstrated that our proposed AMP image synthesis is extremely effective in expanding the dataset of cirrhosis images, thus diagnosing liver cirrhosis with considerably high accuracy. We achieved an accuracy of 99.95 %, a sensitivity of 100 %, and a specificity of 99.9 % on the Samsung Medical Center dataset using 8 × 8 pixels-sized μ-patches. The proposed approach provides an effective solution to deep-learning models with limited-training data, such as medical imaging tasks.

A Self-Configuring Deep Learning Network for Segmentation of Temporal Bone Anatomy in Cone-Beam CT Imaging.

Preoperative planning for otologic or neurotologic procedures often requires manual segmentation of relevant structures, which can be tedious and time-consuming. Automated methods for segmenting multiple geometrically complex structures can not only streamline preoperative planning but also augment minimally invasive and/or robot-assisted procedures in this space. This study evaluates a state-of-the-art deep learning pipeline for semantic segmentation of temporal bone anatomy. A descriptive study of a segmentation network. Academic institution. A total of 15 high-resolution cone-beam temporal bone computed tomography (CT) data sets were included in this study. All images were co-registered, with relevant anatomical structures (eg, ossicles, inner ear, facial nerve, chorda tympani, bony labyrinth) manually segmented. Predicted segmentations from no new U-Net (nnU-Net), an open-source 3-dimensional semantic segmentation neural network, were compared against ground-truth segmentations using modified Hausdorff distances (mHD) and Dice scores. Fivefold cross-validation with nnU-Net between predicted and ground-truth labels were as follows: malleus (mHD: 0.044 ± 0.024 mm, dice: 0.914 ± 0.035), incus (mHD: 0.051 ± 0.027 mm, dice: 0.916 ± 0.034), stapes (mHD: 0.147 ± 0.113 mm, dice: 0.560 ± 0.106), bony labyrinth (mHD: 0.038 ± 0.031 mm, dice: 0.952 ± 0.017), and facial nerve (mHD: 0.139 ± 0.072 mm, dice: 0.862 ± 0.039). Comparison against atlas-based segmentation propagation showed significantly higher Dice scores for all structures (p < .05). Using an open-source deep learning pipeline, we demonstrate consistently submillimeter accuracy for semantic CT segmentation of temporal bone anatomy compared to hand-segmented labels. This pipeline has the potential to greatly improve preoperative planning workflows for a variety of otologic and neurotologic procedures and augment existing image guidance and robot-assisted systems for the temporal bone.

A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery.

The current study presents a multi-task end-to-end deep learning model for real-time blood accumulation detection and tools semantic segmentation from a laparoscopic surgery video. Intraoperative bleeding is one of the most problematic aspects of laparoscopic surgery. It is challenging to control and limits the visibility of the surgical site. Consequently, prompt treatment is required to avoid undesirable outcomes. This system exploits a shared backbone based on the encoder of the U-Net architecture and two separate branches to classify the blood accumulation event and output the segmentation map, respectively. Our main contribution is an efficient multi-task approach that achieved satisfactory results during the test on surgical videos, although trained with only RGB images and no other additional information. The proposed multi-tasking convolutional neural network did not employ any pre- or postprocessing step. It achieved a Dice Score equal to 81.89% for the semantic segmentation task and an accuracy of 90.63% for the event detection task. The results demonstrated that the concurrent tasks were properly combined since the common backbone extracted features proved beneficial for tool segmentation and event detection. Indeed, active bleeding usually happens when one of the instruments closes or interacts with anatomical tissues, and it decreases when the aspirator begins to remove the accumulated blood. Even if different aspects of the presented methodology could be improved, this work represents a preliminary attempt toward an end-to-end multi-task deep learning model for real-time video understanding.

Semantic segmentation of surface defects of smooth parts based on deep convolutional neural networks

Machine vision plays an increasingly important role in industrial product quality detection. During processing, scratches, dents and other defects are inevitable on the surface of a smooth part. Although surface defects do not affect the overall performance of the product, their existence is unacceptable when a perfect product is required. The surface defect detection method based on machine vision and deep convolutional neural networks overcomes, to a certain extent, the problem of low detection efficiency, high false detection and missing detection rates in the traditional detection method. In this paper, a multistream semantic segmentation neural network is proposed to identify defects on smooth parts. Taking a seatbelt buckle as an example, the scratch and crush defects on the surface are classified. The network takes DeepLabV3+ as the framework and three types of image stream as the input of the network. In the backbone feature extraction network, the Xception structure is improved to MobilenetV2 and the convolutional block attention module (CBAM) is introduced into the decoding network, which improves the operational efficiency and accuracy. Compared with other classical networks, this network demonstrates good performance in the image dataset of the seatbelt buckle and realises fast and accurate semantic segmentation and classification of surface defects. The evaluation results of the network model have been significantly improved.

An Improved U-Net Model for Infrared Image Segmentation of Wind Turbine Blade

Infrared image segmentation of the wind turbine blade is an important link of wind turbine condition monitoring. Nevertheless, there remain plenty of challenges when the traditional semantic segmentation networks are applied directly to the infrared image segmentation of wind turbine blade, including the blurred boundaries caused by similar thermal radiation of background and blade, the irregularity of edge identification caused by uneven heating, and the hub misidentified as blade. To address these issues, a semantic segmentation neural network based on the basic U-Net is improved for infrared image segmentation of the wind turbine blade. The hierarchical-split depthwise separable convolution block is integrated into the constructed network to have a higher segmentation accuracy. Also, the fusion of convolution layer and batch normalization layer is performed in inference to get a better speed–accuracy tradeoff. The experimental results show that the proposed approach outperforms all the comparing methods, which further demonstrates that the trained network can achieve superior runtime performance and accurate segmentation with excellent anti-interference performance.

Semantic segmentation of water bodies in very high-resolution satellite and aerial images

This study evaluates the performance of convolutional neural networks for semantic segmentation of water bodies in very high-resolution satellite and aerial images from multiple sensors with particular focus on flood emergency response applications. Different model architectures (U-Net and DeepLab-V3+) are combined with encoder backbones (MobileNet-V3, ResNet-50 and EfficientNet-B4) and tested for their ability to delineate inundated areas under varying environmental conditions and data availability scenarios. An unprecedented reference dataset of 1120 globally sampled images with quality checked binary water masks is introduced and used to train, validate and test the models for water body segmentation. Furthermore, independent test datasets are developed to test the generalization ability of the trained models across regions, sensors (IKONOS, GeoEye-1, WorldView-2, WorldView-3 and four different airborne camera systems) and tasks (normal water and flood water segmentation). Results indicate that across all tested scenarios a U-Net model with Mobilenet-V3 backbone pre-trained on ImageNet performs best. While using R-G-B image bands performs well, adding the near infrared band (if available) slightly improves prediction results. Similarly, adding slope information from an independent digital elevation model increases accuracies. Train-time augmentation and contrast enhancement could improve transferability across sensors and in particular between satellite and aerial images. Moreover, adding noisy training data from freely available online resources could further improve performance with minimal annotation effort.

Segment-then-Segment: Context-Preserving Crop-Based Segmentation for Large Biomedical Images.

Medical images are often of huge size, which presents a challenge in terms of memory requirements when training machine learning models. Commonly, the images are downsampled to overcome this challenge, but this leads to a loss of information. We present a general approach for training semantic segmentation neural networks on much smaller input sizes called Segment-then-Segment. To reduce the input size, we use image crops instead of downscaling. One neural network performs the initial segmentation on a downscaled image. This segmentation is then used to take the most salient crops of the full-resolution image with the surrounding context. Each crop is segmented using a second specially trained neural network. The segmentation masks of each crop are joined to form the final output image. We evaluate our approach on multiple medical image modalities (microscopy, colonoscopy, and CT) and show that this approach greatly improves segmentation performance with small network input sizes when compared to baseline models trained on downscaled images, especially in terms of pixel-wise recall.

Semantic segmentation of superficial layer in intracoronary optical coherence tomography based on cropping-merging and deep learning

Manual labelling and qualitative analysis of lesion tissues in vessel optical coherence tomography (OCT) images are time-consuming and laborious for cardiovascular specialists. To beat these issues, a semantic segmentation method was presented based on deep learning on the OCT image analysis of the human vessel to reduce the diagnosis pressure of cardiovascular doctors. An outer border of ROI was segmented based on the level-set method to obtain the visible superficial layer containing useful information. And then, cropping square patches from the ROI with its patch center pixel inside the ROI and using patches as the input data. To reuse the preceding layers' feature maps, we used the dense block to replace the normal convolution layer, and simultaneously, employed a skip-connection from m the down-sampling path to the up-sampling path to keep the spatial information. With the advantage of SegNet on semantic segmentation, a dense-block-SegNet (DBSegNet) is constructed to complete the pixel-level segmentation. Training and testing were executed on 7 datasets (22, 210 images) to assess the model. Tenfold cross-validation method was implemented to measure the classification outcomes and the semantic segmentation capacity of our model. In deep learning experiment, sensitivities for calcified, fibrous and lipid plaques were 91.81 ± 3.60 %, 92.81 ± 2.72 % and 91.78 ± 1.62 %, respectively. A 3-D volume was created to insert each prediction slice along the depth axis to compute the maximum type number of each pixel in order to identify the final type of each pixel. Post-processing was used to refine the classification findings in order to eliminate classification errors. Semantic segmentation neural network based on the cropped input data and feature reusing was a feasible approach for the vessel tissue pixel-classification of IVOCT image. The proposed method has the potential to become a useful tool for specialists in analysing lesion kinds and determining clinical lesion characteristics (e.g., distribution position, angle and depth).

A Texture Integrated Deep Neural Network for Semantic Segmentation of Urban Meshes

3-D geo-information is essential for many urban related applications. Point cloud and mesh are two common representations of the 3-D urban surface. Compared to point cloud data, mesh possesses indispensable advantages, such as high-resolution image texture and sharp geometry representation. Semantic segmentation, as an important way to obtain 3-D geo-information, however, is mainly performed on the point cloud data. Due to the complex geometry representation and lack of efficient utilizing of image texture information, the semantic segmentation of the mesh is still a challenging task for urban 3-D geo-information acquisition. In this article, we propose a texture and geometry integrated deep learning method for the mesh segmentation task. A novel texture convolution module is introduced to capture image texture features. The texture features are concatenate with nontexture features on a point cloud that represents by the center of gravity (COG) of the mesh triangles. A hierarchical deep network is employed to segment the COG point cloud. Our experimental results show that the proposed network significantly improves the accuracy with the introduced texture convolution module (1.9% for overall accuracy and 4.0% for average F1 score). It also compares with other state-of-the-art methods on the public SUM-Helsinki dataset and achieves considerable results.

DWin-HRFormer: A High-Resolution Transformer Model With Directional Windows for Semantic Segmentation of Urban Construction Land

In this paper, a deep neural network for semantic segmentation of high-resolution remote sensing images is proposed for urban construction land classification. The network follows a high-resolution network architecture. Specifically, a directional self-attention on the paths of different resolutions is proposed, aiming to correct the directional bias caused by the attention of strip windows during the model learning, while also reducing the computational complexity, and allowing the model to improve both the accuracy and speed. At the end of the network, a distributed alignment module with spatial information is constructed to train additional learnable parameters, to adjust the biased decision boundaries through a two-stage learning strategy, and alleviate the problem of accuracy degradation due to the unbalanced training data. We tested the proposed method and compared it with the current state-of-the-art semantic segmentation methods on the Luojia-FGLC and WHDLD datasets, and the proposed one obtained the best performance. We also verified the effectiveness of each component of the network through ablation experiments. The code and model will be available at https://github.com/Zhzhyd/DWin-HRFormer.