Pixel-wise Image Segmentation Research Articles

DeepNet model empowered cuckoo search algorithm for the effective identification of lung cancer nodules.

Globally, lung cancer is a highly harmful type of cancer. An efficient diagnosis system can enable pathologists to recognize the type and nature of lung nodules and the mode of therapy to increase the patient's chance of survival. Hence, implementing an automatic and reliable system to segment lung nodules from a computed tomography (CT) image is useful in the medical industry. This study develops a novel fully convolutional deep neural network (hereafter called DeepNet) model for segmenting lung nodules from CT scans. This model includes an encoder/decoder network that achieves pixel-wise image segmentation. The encoder network exploits a Visual Geometry Group (VGG-19) model as a base architecture, while the decoder network exploits 16 upsampling and deconvolution modules. The encoder used in this model has a very flexible structural design that can be modified and trained for any resolution based on the size of input scans. The decoder network upsamples and maps the low-resolution attributes of the encoder. Thus, there is a considerable drop in the number of variables used for the learning process as the network recycles the pooling indices of the encoder for segmentation. The Thresholding method and the cuckoo search algorithm determines the most useful features when categorizing cancer nodules. The effectiveness of the intended DeepNet model is cautiously assessed on the real-world database known as The Cancer Imaging Archive (TCIA) dataset and its effectiveness is demonstrated by comparing its representation with some other modern segmentation models in terms of selected performance measures. The empirical analysis reveals that DeepNet significantly outperforms other prevalent segmentation algorithms with 0.962 ± 0.023% of volume error, 0.968 ± 0.011 of dice similarity coefficient, 0.856 ± 0.011 of Jaccard similarity index, and 0.045 ± 0.005s average processing time.

Multi-scale triple-attention network for pixelwise crack segmentation

Currently, intelligent crack detection is of great value for the maintenance of infrastructure, of which the most significant kind in China is roads. For pavement defects, the pavement can be repaired and maintained in a timely manner with an accurate defect detection task, which significantly reduces the occurrence of hazards. However, the detection of pavement defects remains a great challenge owing to many difficulties, for example, complex backgrounds, microdefects, various defect shapes and sizes, class imbalance issues, etc. Recently, deep learning has demonstrated its superior performance on pixelwise image segmentation, but some issues still exist on demanding pixelwise image segmentation, for instance, limited receptive field, insufficiency processing of local features, information loss issue generated by pooling operations, etc. Based on all of the above issues, a multiscale triple-attention network, named MST-Net, is proposed for end-to-end pixelwise crack detection. First, a multiscale input strategy is applied to the proposed segmentation network to capture more context information. Meanwhile, it can capably reduce the effect of the information loss issue generated by pooling operations. Second, to realize effective feature representation of local features, an additive attention fusion (AAF) block is proposed to guide feature learning to capture both global and local contexts. In addition, faced with the crack detection task with class imbalance issues, a triple attention (TA) block is proposed to detect spatial, channel and pixel attention information to suppress the background and useless information, which is conducive to the characterization of microcracks. Finally, aiming at the limited receptive field, a multiscale feature aggregation unit is proposed for feature fusion to increase the detection ability of multiscale defects. To better guide network training, a deep supervision mechanism is also introduced to speed up the convergence of the proposed segmentation model and improve the performance of defect segmentation. The related evaluation and detection experiments are carried out on three public datasets on crack segmentation, and the comparison experiments with the mainstream segmentation models show that the proposed segmentation network achieves excellent performance on pixelwise crack detection.

On exploring weakly supervised domain adaptation strategies for semantic segmentation using synthetic data

Pixel-wise image segmentation is key for many Computer Vision applications. The training of deep neural networks for this task has expensive pixel-level annotation requirements, thus, motivating a growing interest on synthetic data to provide unlimited data and its annotations. In this paper, we focus on the generation and application of synthetic data as representative training corpuses for semantic segmentation of urban scenes. First, we propose a synthetic data generation protocol, which identifies key features affecting performance and provides datasets with variable complexity. Second, we adapt two popular weakly supervised domain adaptation approaches (combined training, fine-tuning) to employ synthetic and real data. Moreover, we analyze several backbone models, real/synthetic datasets and their proportions when combined. Third, we propose a new curriculum learning strategy to employ several synthetic and real datasets. Our major findings suggest the high performance impact of pace and order of synthetic and real data presentation, achieving state of the art results for well-known models. The results by training with the proposed dataset outperform popular alternatives, thus demonstrating the effectiveness of the proposed protocol. Our code and dataset are available at http://www-vpu.eps.uam.es/publications/WSDA_semantic/

Deep learning approach for delamination identification using animation of Lamb waves

The complex design of composite structures makes it very difficult for conventional visual inspection techniques to detect defects in these materials. Therefore, numerous types of nondestructive testing and structural health monitoring techniques have been developed for damage identification in composite structures, among which ultrasonic guided waves, particularly Lamb waves, have gained much popularity. In recent years, apart from piezoelectric pointwise measurements, laser Doppler vibrometry has been used for full wavefield measurements of propagating Lamb waves. Damage imaging can be done with animations of Lamb waves interacting with defects. In this work, two end-to-end deep learning-based models of many-to-one sequence prediction were developed to perform pixel-wise image segmentation. A large dataset of full wavefield images resulting from the interaction with delamination of random size, shape, and location was utilised to train the proposed deep learning models. The main goal of this work is to investigate the feasibility of implementing deep learning methods for delamination identification in composite laminates by only utilising the animations of Lamb waves. The elaborated models worked well on the numerically generated test data and have also shown good performance on the real-world experimental data. These methods can enable the automation of delamination identification and to create damage maps without the intervention of the user.

Segmentation of breast tumors using cutting-edge semantic segmentation models

ABSTRACT Early detection of breast cancer is the most important area of mammography research at the moment. It is critical to use computer-aided diagnosis to screen for and prevent breast cancer. In this study, the effectiveness of cutting-edge deep segmentation models for mammography in the detection of breast tumors was investigated. A medical images dataset was compiled and annotated at Lady Reading Hospital, one of the largest teaching hospitals in Pakistan in collaboration with the local health specialists, radiologists, and technologists. A comparison was made between the performance of the segmentation techniques used, and the model that performed the best in detecting tumors and normal breast regions was selected. The evaluation metrics, such as the mean IoU, pixel accuracy, and an in-depth experimental evaluation were used as performance parameters. This investigation determined how well semantic segmentation techniques were performed based on two datasets (cityscapes and mammograms) in this study. The global Dilation 10 semantic segmentation model outperformed the other three semantic segmentation models with a pixel accuracy of 92.98 percent in comparison tests. This paper demonstrates the efficacy of pixel-wise image segmentation techniques and their superiority to other techniques by outperforming other current state-of-the-art automatic image segmentation models.

Exploratory Analysis on Pixelwise Image Segmentation Metrics with an Application in Proximal Sensing

A considerable number of metrics can be used to evaluate the performance of machine learning algorithms. While much work is dedicated to the study and improvement of data quality and models’ performance, much less research is focused on the study of these evaluation metrics, their intrinsic relationship, the interplay of the influence among the metrics, the models, the data, and the environments and conditions in which they are to be applied. While some works have been conducted on general machine learning tasks such as classification, fewer efforts have been dedicated to more complex problems such as object detection and image segmentation, in which the evaluation of performance can vary drastically depending on the objectives and domains of application. Working in an agricultural context, specifically on the problem of the automatic detection of plants in proximal sensing images, we studied twelve evaluation metrics that we used to evaluate three image segmentation models recently presented in the literature. After a unified presentation of these metrics, we carried out an exploratory analysis of their relationships using a correlation analysis, a clustering of variables, and two factorial analyses (namely principal component analysis and multiple factorial analysis). We distinguished three groups of highly linked metrics and, through visual inspection of the representative images of each group, identified the aspects of segmentation that each group evaluates. The aim of this exploratory analysis was to provide some clues to practitioners for understanding and choosing the metrics that are most relevant to their agricultural task.

Workflow for high-resolution phase segmentation of cement clinker from combined BSE image and EDX spectral data.

Burning of clinker is the most influencing step of cement quality during the production process. Appropriate characterisation for quality control and decision-making is therefore the critical point to maintain a stable production but also for the development of alternative cements. Scanning electron microscopy (SEM) in combination with energy dispersive X-ray spectroscopy (EDX) delivers spatially resolved phase and chemical information for cement clinker. This data can be used to quantify phase fractions and chemical composition of identified phases. The contribution aims to provide an overview of phase fraction quantification by semi-automatic phase segmentation using high-resolution backscattered electron (BSE) images and lower-resolved EDX element maps. Therefore, a tool for image analysis was developed that uses state-of-the-art algorithms for pixel-wise image segmentation and labelling in combination with a decision tree that allows searching for specific clinker phases. Results show that this tool can be applied to segment sub-micron scale clinker phases and to get a quantification of all phase fractions. In addition, statistical evaluation of the data is implemented within the tool to reveal whether the imaged area is representative for all clinker phases.

Full wavefield processing by using FCN for delamination detection

A novel full wavefield processing method by using fully convolutional neural networks is presented. The full wavefield of propagating Lamb waves in the fibre-reinforced composite plate was simulated by the parallel spectral element method. It resembles a full wavefield measurements acquired on a surface of the plate by the scanning laser Doppler vibrometer. The aim of the proposed technique is an identification of delamination location, size and shape. It is achieved by pixel-wise image segmentation by using the end-to-end approach. It is possible because of the large dataset of Lamb wave propagation patterns resulting from interaction with delaminations of random location, size and shape. It is demonstrated that the proposed method, tested on numerical data, is performing better than conventional adaptive wavenumber filtering method which was developed in previous work. Moreover, it enables better automation of delamination identification so that the damage map can be created without user intervention. The method was also tested on experimental data acquired on the surface of the specimen in which delamination was artificially created by a Teflon insert. The obtained results with the deep learning approach show its capability to predict the delamination in the numerically generated dataset with high accuracy compared to the conventional damage detection approach. Furthermore, the deep learning model shows the ability to generalize to a further experiential set.

Task Decomposition and Synchronization for Semantic Biomedical Image Segmentation

Semantic segmentation is essentially important to biomedical image analysis. Many recent works mainly focus on integrating the Fully Convolutional Network (FCN) architecture with sophisticated convolution implementation and deep supervision. In this paper, we propose to decompose the single segmentation task into three subsequent sub-tasks, including (1) pixel-wise image segmentation, (2) prediction of the class labels of the objects within the image, and (3) classification of the scene the image belonging to. While these three sub-tasks are trained to optimize their individual loss functions of different perceptual levels, we propose to let them interact by the task-task context ensemble. Moreover, we propose a novel sync-regularization to penalize the deviation between the outputs of the pixel-wise segmentation and the class prediction tasks. These effective regularizations help FCN utilize context information comprehensively and attain accurate semantic segmentation, even though the number of the images for training may be limited in many biomedical applications. We have successfully applied our framework to three diverse 2D/3D medical image datasets, including Robotic Scene Segmentation Challenge 18 (ROBOT18), Brain Tumor Segmentation Challenge 18 (BRATS18), and Retinal Fundus Glaucoma Challenge (REFUGE18). We have achieved top-tier performance in all three challenges.

Deep learning in the built environment: automatic detection of rooftop solar panels using Convolutional Neural Networks

Mapping the location and size of solar installations in urban areas can be a valuable input for policymakers and for investing in distributed energy infrastructures. Machine Learning techniques, combined with satellite and aerial imagery, allow to overcome the limitations of surveys and sparse databases in providing this mapping at large scale. In this paper we apply a supervised method based on convolutional neural networks to delineate rooftop solar panels and to detect their sizes by means of pixel-wise image segmentation. As input to the algorithm, we rely on high resolution aerial photos provided by the Swiss Federal Office of Topography. We explore different data augmentation and we vary network parameters in order to maximize model performance. Preliminary results show that we are able to automatically detect in test images the area of a set of solar panels at pixel level with an accuracy of about 0.94 and an Intersection over Union index of up to 0.64. The scalability of the trained model allows to predict the existing solar panels deployment at the Swiss national scale. The correlation with local environmental and socio-economic variables would allow to extract predictive models to foster future adoption of solar technology in urban areas.

3D Façade Labeling over Complex Scenarios: A Case Study Using Convolutional Neural Network and Structure-From-Motion

Urban environments are regions in which spectral variability and spatial variability are extremely high, with a huge range of shapes and sizes, and they also demand high resolution images for applications involving their study. Due to the fact that these environments can grow even more over time, applications related to their monitoring tend to turn to autonomous intelligent systems, which together with remote sensing data could help or even predict daily life situations. The task of mapping cities by autonomous operators was usually carried out by aerial optical images due to its scale and resolution; however new scientific questions have arisen, and this has led research into a new era of highly-detailed data extraction. For many years, using artificial neural models to solve complex problems such as automatic image classification was commonplace, owing much of their popularity to their ability to adapt to complex situations without needing human intervention. In spite of that, their popularity declined in the mid-2000s, mostly due to the complex and time-consuming nature of their methods and workflows. However, newer neural network architectures have brought back the interest in their application for autonomous classifiers, especially for image classification purposes. Convolutional Neural Networks (CNN) have been a trend for pixel-wise image segmentation, showing flexibility when detecting and classifying any kind of object, even in situations where humans failed to perceive differences, such as in city scenarios. In this paper, we aim to explore and experiment with state-of-the-art technologies to semantically label 3D urban models over complex scenarios. To achieve these goals, we split the problem into two main processing lines: first, how to correctly label the façade features in the 2D domain, where a supervised CNN is used to segment ground-based façade images into six feature classes, roof, window, wall, door, balcony and shop; second, a Structure-from-Motion (SfM) and Multi-View-Stereo (MVS) workflow is used to extract the geometry of the façade, wherein the segmented images in the previous stage are then used to label the generated mesh by a “reverse” ray-tracing technique. This paper demonstrates that the proposed methodology is robust in complex scenarios. The façade feature inferences have reached up to 93% accuracy over most of the datasets used. Although it still presents some deficiencies in unknown architectural styles and needs some improvements to be made regarding 3D-labeling, we present a consistent and simple methodology to handle the problem.

Permutation-Invariant Consensus over Crowdsourced Labels

This paper introduces a novel crowdsourcing consensus model and inference algorithm — which we call PICA (Permutation-Invariant Crowdsourcing Aggregation) — that is designed to recover the ground-truth labels of a dataset while being invariant to the class permutations enacted by the different annotators. This is particularly useful for settings in which annotators may have systematic confusions about the meanings of different classes, as well as clustering problems (e.g., dense pixel-wise image segmentation) in which the names/numbers assigned to each cluster have no inherent meaning.The PICA model is constructed by endowing each annotator with a doubly-stochastic matrix (DSM), which models the probabilities that an annotator will perceive one class and transcribe it into another. We conduct simulations and experiments to show the advantage of PICA compared to two baselines (Majority Vote, and an "unpermutation" heuristic) for three different clustering/labeling tasks. We also explore the conditions under which PICA provides better inference accuracy compared to a simpler but related model based on right-stochastic matrices. Finally, we show that PICA can be used to crowdsource responses for dense image segmentation tasks, and provide a proof-of-concept that aggregating responses in this way could improve the accuracy of this labor-intensive task.

Skin lesion segmentation using deep convolution networks guided by local unsupervised learning

Automatic localization of skin lesions within dermoscopy images is a crucial step toward developing a decision support system for skin cancer detection. However, segmentation of the lesion image can be challenging, as these images possess various artifacts distorting the uniformity of the lesion area. Recently, deep convolution learning-based techniques have drawn great attention for pixel-wise image segmentation. These deep networks produce coarse segmentation, and convolutional filters and pooling layers result in segmentation of a skin lesion at a lower resolution than the original skin image. To overcome these drawbacks, we have proposed a superpixel-based fine-tuning strategy to effectively utilize the characteristics of the skin image pixels to accurately extract the border of the lesion. Our proposed approach not only learns a global map for skin lesions, but also acquires the local contextual information, such as lesion boundary. It can, therefore, accurately segment lesions within a given skin image, even in the presence of fuzzy boundaries and complex textures. To evaluate the performance of our proposed method, experiments have been conducted using the 2016 International Symposium on Biomedical Imaging dataset, and these experiments suggest the effectiveness of the proposed method.

Multispectral image segmentation using localized spectral binarization

This paper proposes a new feature extraction method for multispectral image segmentation based in Localized Spectral Binarization (LSB). In contrast with the standard image operation, which is applied with traditional image, LSB is computed on a single pixel with numerous bands. The proposed algorithm calculates differences of spectra locally on same pixel’s coordinate in different bands of the multispectral image. The difference value is converted into binary by breaking the difference values into two directions, which are the positive and negative value then the differences are thresholded to form a binary codeword. A binomial factor is assigned to these codewords to form another unique value. These values are then grouped to construct the LSB feature image where is used in the image segmentation. LANDSAT multispectral images are used in the experiment to evaluate the segmentation and classification accuracy of the proposed LSB in terms of pixel-wise image segmentation. The result shows that LSB feature outperforms the spectral feature.

English

A wide variety of image processing applications require segmentation and classification ofimages. The problem becomes complex when the images are obtained in an uncontrolledenvironment with a non-uniform illumination. The selection of suitable features is a critical partof an image segmentation and classification process, where the basic objective is to identify theimage regions that are homogeneous but dissimilar to all spatially adjacent regions. This paperproposes an automatic method for the classification of a terrain using image features such asintensity, texture, and edge. The textural features are calculated using statistics of geometricalattributes of connected regions in a sequence of binary images obtained from a texture image.A pixel-wise image segmentation scheme using a multi-resolution pyramid is used to correct thesegmentation process so as to get homogeneous image regions. Localisation of texture boundariesis done using a refined-edge map obtained by convolution, thinning, thresholding, and linking.The individual regions are classified using a database generated from the features extracted fromknown samples of the actual terrain. The algorithm is used to classify airborne images of a terrainobtained from the sensor mounted on an aerial reconnaissance platform and the results arepresented.