Pixel-based Segmentation Research Articles

Enhanced WGAN Model for Diagnosing Laryngeal Carcinoma

This study modifies the U-Net architecture for pixel-based segmentation to automatically classify lesions in laryngeal endoscopic images. The advanced U-Net incorporates five-level encoders and decoders, with an autoencoder layer to derive latent vectors representing the image characteristics. To enhance performance, a WGAN was implemented to address common issues such as mode collapse and gradient explosion found in traditional GANs. The dataset consisted of 8171 images labeled with polygons in seven colors. Evaluation metrics, including the F1 score and intersection over union, revealed that benign tumors were detected with lower accuracy compared to other lesions, while cancers achieved notably high accuracy. The model demonstrated an overall accuracy rate of 99%. This enhanced U-Net model shows strong potential in improving cancer detection, reducing diagnostic errors, and enhancing early diagnosis in medical applications.

Revolutionizing urban mapping: deep learning and data fusion strategies for accurate building footprint segmentation

In the dynamic urban landscape, understanding the distribution of buildings is paramount. Extracting and delineating building footprints from high-resolution images, captured by aerial platforms or satellites, is essential but challenging to accomplish manually, due to the abundance of high-resolution data. Automation becomes imperative, yet it introduces complexities related to handling diverse data sources and the computational demands of advanced algorithms. The innovative solution proposed in this paper addresses some intricate challenges occurring when integrating deep learning and data fusion on Earth Observed imagery. By merging RGB orthophotos with Digital Surface Models, deriving from the same aerial high-resolution surveys, an integrated consistent four-band dataset is generated. This unified approach, focused on the extraction of height information through stereoscopy utilizing a singular source, facilitates enhanced pixel-to-pixel data fusion. Employing DeepLabv3 algorithms, a state-of-the-art semantic segmentation network for multi-scale context, pixel-based segmentation on the integrated dataset was performed, excelling in capturing intricate details, particularly when enhanced by the additional height information deriving from the Digital Surface Models acquired over urban landscapes. Evaluation over a 21 km2 area in Turin, Italy, featuring diverse building frameworks, showcases how the proposed approach leads towards superior accuracy levels and building boundary refinement. Notably, the methodology discussed in the present article, significantly reduces training time compared to conventional approaches like U-Net, overcoming inherent challenges in high-resolution data automation. By establishing the effectiveness of leveraging DeepLabv3 algorithms on an integrated dataset for precise building footprint segmentation, the present contribution holds promise for applications in 3D modelling, Change detection and urban planning. An approach favouring the application of deep learning strategies on integrated high-resolution datasets can then guide decision-making processes facilitating urban management tasks.

CF-Loss: Clinically-relevant feature optimised loss function for retinal multi-class vessel segmentation and vascular feature measurement

Characterising clinically-relevant vascular features, such as vessel density and fractal dimension, can benefit biomarker discovery and disease diagnosis for both ophthalmic and systemic diseases. In this work, we explicitly encode vascular features into an end-to-end loss function for multi-class vessel segmentation, categorising pixels into artery, vein, uncertain pixels, and background. This clinically-relevant feature optimised loss function (CF-Loss) regulates networks to segment accurate multi-class vessel maps that produce precise vascular features. Our experiments first verify that CF-Loss significantly improves both multi-class vessel segmentation and vascular feature estimation, with two standard segmentation networks, on three publicly available datasets. We reveal that pixel-based segmentation performance is not always positively correlated with accuracy of vascular features, thus highlighting the importance of optimising vascular features directly via CF-Loss. Finally, we show that improved vascular features from CF-Loss, as biomarkers, can yield quantitative improvements in the prediction of ischaemic stroke, a real-world clinical downstream task. The code is available at https://github.com/rmaphoh/feature-loss.

Cascading Machine Learning to Monitor Volcanic Thermal Activity Using Orbital Infrared Data: From Detection to Quantitative Evaluation

Several satellite missions are currently available to provide thermal infrared data at different spatial resolutions and revisit time. Furthermore, new missions are planned thus enabling to keep a nearly continuous ‘eye’ on thermal volcanic activity around the world. This massive volume of data requires the development of artificial intelligence (AI) techniques for the automatic processing of satellite data in order to extract significant information about volcano conditions in a short time. Here, we propose a robust machine learning approach to accurately detect, recognize and quantify high-temperature volcanic features using Sentinel-2 MultiSpectral Instrument (S2-MSI) imagery. We use the entire archive of high spatial resolution satellite data containing more than 6000 S2-MSI scenes at ten different volcanoes around the world. Combining a ‘top-down’ cascading architecture, two different machine learning models, a scene classifier (SqueezeNet) and a pixel-based segmentation model (random forest), we achieved a very high accuracy, namely 95%. These results show that the cascading approach can be applied in near-real time to any available satellite image, providing a full description of the scene, with an important contribution to the monitoring, mapping and characterization of volcanic thermal features.

Integrating object-based and pixel-based segmentation for building footprint extraction from satellite images

Accurately delineating building footprints from optical satellite imagery presents a formidable challenge, particularly in urban settings characterized by intricate and diverse structures. Consequently, enhancing the utility of these images for geospatial data updates demands meticulous refinement. Machine learning algorithms have made notable contributions in this context, yet the pursuit of precision remains an ongoing challenge. This paper aims to enhance the accuracy of building footprint extraction through the integration of object-based and pixel-based segmentation techniques. Additionally, it evaluates the performance of machine learning methodologies, specifically LightGBM, XGBoost, and Neural Network (NN) approaches. The model's evaluation employed low spectral resolution optical images, widely accessible and cost-effective for acquisition. The study's outcomes demonstrate a substantial enhancement in extraction accuracy compared to extant literature. The proposed methodology attains an overall accuracy of 99.39%, an F1 measurement of 0.9935, and a Cohen Kappa index of 0.9870. Thus, the proposed approach signifies a noteworthy advancement over existing techniques for building footprint extraction from high-resolution optical imagery.

Osteoporosis Analysis using Evolutionary Algorithm

Abstract—Already, a greater number of techniques and algorithms are evolving. but the existing system was lacking somewhere and had limitations in its implementation. At present, existing studies or research are done only for local optima. Local optima were the main problem in our study. The objective of the system revolves around the combination of natural and fused evolutionary algorithms with clinical expertise data to result in global optimal early detection and better accuracy of osteoporosis to get the best fitness score for the improvement of patient. We are collecting the sample data and diagnosing the patient with X-ray techniques. The X-ray image will be in noisy form. To eliminate noise in the image, we are doing the denoising and deblurring processes. After that, enhance the image, and pixel-based segmentation is done in this x-ray image. From the sample data, we are extracting features. . So, we can obtain the result from the performance parameters, i.e., whether the patient has osteoporosis or not. Through this process, we get the 97.93 best accuracy compared to other studies. Keywords— Bone Mass Density (BMD), feature extraction, evolutionary algorithm - genetic algorithm, Deep learning modules, Jaccard index

Breast Cancer Segmentation from Ultrasound Images Using ResNext-based U-Net Model

Breast cancer is a type of cancer caused by the uncontrolled growth and proliferation of cells in the breast tissue. Differentiating between benign and malignant tumors is critical in the detection and treatment of breast cancer. Traditional methods of cancer detection by manual analysis of radiological images are time-consuming and error-prone due to human factors. Modern approaches based on image classifier deep learning models provide significant results in disease detection, but are not suitable for clinical use due to their black-box structure. This paper presents a semantic segmentation method for breast cancer detection from ultrasound images. First, an ultrasound image of any resolution is divided into 256×256 pixel patches by passing it through an image cropping function. These patches are sequentially numbered and given as input to the model. Features are extracted from the 256×256 pixel patches with pre-trained ResNext models placed in the encoder network of the U-Net model. These features are processed in the default decoder network of the U-Net model and estimated at the output with three different pixel values: benign tumor areas (1), malignant tumor areas (2) and background areas (0). The prediction masks obtained at the output of the decoder network are combined sequentially to obtain the final prediction mask. The proposed method is validated on a publicly available dataset of 780 ultrasound images of female patients. The ResNext-based U-Net model achieved 73.17% intersection over union (IoU) and 83.42% dice coefficient (DC) on the test images. ResNext-based U-Net models perform better than the default U-Net model. Experts could use the proposed pixel-based segmentation method for breast cancer diagnosis and monitoring.

Deep-Learning-Based Segmentation of Extraocular Muscles from Magnetic Resonance Images.

In this study, we investigated the performance of four deep learning frameworks of U-Net, U-NeXt, DeepLabV3+, and ConResNet in multi-class pixel-based segmentation of the extraocular muscles (EOMs) from coronal MRI. Performances of the four models were evaluated and compared with the standard F-measure-based metrics of intersection over union (IoU) and Dice, where the U-Net achieved the highest overall IoU and Dice scores of 0.77 and 0.85, respectively. Centroid distance offset between identified and ground truth EOM centroids was measured where U-Net and DeepLabV3+ achieved low offsets (p > 0.05) of 0.33 mm and 0.35 mm, respectively. Our results also demonstrated that segmentation accuracy varies in spatially different image planes. This study systematically compared factors that impact the variability of segmentation and morphometric accuracy of the deep learning models when applied to segmenting EOMs from MRI.

Towards global scale segmentation with OpenStreetMap and remote sensing

Land Use Land Cover (LULC) segmentation is a famous application of remote sensing in an urban environment. Up-to-date and complete data are of major importance in this field. Although with some success, pixel-based segmentation remains challenging because of class variability. Due to the increasing popularity of crowd-sourcing projects, like OpenStreetMap, the need for user-generated content has also increased, providing a new prospect for LULC segmentation. We propose a deep-learning approach to segment objects in high-resolution imagery by using semantic crowdsource information. Due to satellite imagery and crowdsource database complexity, deep learning frameworks perform a significant role. This integration reduces computation and labor costs. Our methods are based on a fully convolutional neural network (CNN) that has been adapted for multi-source data processing. We discuss the use of data augmentation techniques and improvements to the training pipeline. We applied semantic (U-Net) and instance segmentation (Mask R-CNN) methods and, Mask R–CNN showed a significantly higher segmentation accuracy from both qualitative and quantitative viewpoints. The conducted methods reach 91% and 96% overall accuracy in building segmentation and 90% in road segmentation, demonstrating OSM and remote sensing complementarity and potential for city sensing applications.

Cell segmentation and representation with shape priors

Cell segmentation is a fundamental problem of computational biology, for which convolutional neural networks yield the best results nowadays. This field is expanding rapidly, and in the recent years, shape-constrained segmentation models emerged as strong competitors to traditional, pixel-based segmentation methods for instance segmentation. These methods predict the parameters of the underlying shape model, so choosing the right shape representation is critical for the success of the segmentation. In this study, we introduce two new representation-based deep learning segmentation methods after a quantitative comparison of the most important shape descriptors in the literature. Our networks are based on Fourier coefficients and statistical shape models, both of which have proven to be reliable tools for cell shape modelling. Our results indicate that the methods are competitive alternatives to the most widely used baseline deep learning algorithms, especially when the number of parameters for the underlying shape model are low or the cells to be segmented have irregular morphologies.

Aeroponic systems design: considerations and challenges

Controlled Environment Agriculture (CEA) holds promise as a way to intensify current agricultural production systems while limiting pressures on land, water, and energy resources. However, its use has not yet been widely adopted, in part because the engineering design considerations and associated challenges are not well known. This is even more apparent for aeroponics, where the additional cost and complexities in controlling atomization have yet to establish an advantage in scale over simpler hydroponic systems To shed light on these considerations and challenges, an instrumented aeroponic system was prototyped with the goal of creating a quantitative model of growth for various species of leafy greens. As the first consideration, pressure swirl atomizers were paired with a diaphragm-type pressure tank to supply the necessary pressures needed for effective atomization. Secondly, nutrient solution was mixed on-demand from Reverse Osmosis (RO) water and concentrated nutrient stock then pumped into the pressure tank using a positive displacement pump. A bamboo-based substrate that allowed both germination and extended vegetative growth was supported on a stainless steel mesh and PVC frame acting as a grow tray. Finally, a camera microservice platform was developed to quantify plant growth using a computer vision pixel-based segmentation method.

Using superpixel- or pixel-based segmentation for efficient green roof digital image classification and rapid estimation of plant species cover

Green roofs are “nature-based solutions” that provide numerous ecosystem services in the context of urban green infrastructures. Plant species diversity and the associated vegetation communities, in strong interactions with green roof substrate, play a central role in the green infrastructure functioning. In order to better understand the influence of vegetation in relation with the co-benefits provided by green roofs as well as to select suitable species for these usually harsh environments, it is essential to be able to achieve accurate and long-term monitoring of plant communities. In this short communication, two free plugins recently developed for the open-source image analysis software Fiji (a distribution of the freely available ImageJ platform, initially dedicated to biological image analysis) were investigated for their capacity to rapidly and efficiently perform supervised machine-learning for the classification of green roof vegetation photographs, with the aim of estimating individual plant species abundance. Two simple methods are thus described using the Trainable Weka Pixel Segmentation (Arganda-Carreras et al., 2017) or the Trainable Superpixel Segmentation (Salinas Colina et al., 2018), which allowed for rapid, efficient and reproducible classification and estimation of multispecies colonized green roof regardless the color or shape similarities among species or ground cover materials. Finally, recommendations are made for the use of the Trainable Superpixel Segmentation which is particularly convenient for quick and efficient green roof image analysis.

Unsupervised domain adaptation for clinician pose estimation and instance segmentation in the operating room.

The fine-grained localization of clinicians in the operating room (OR) is a key component to design the new generation of OR support systems. Computer vision models for person pixel-based segmentation and body-keypoints detection are needed to better understand the clinical activities and the spatial layout of the OR. This is challenging, not only because OR images are very different from traditional vision datasets, but also because data and annotations are hard to collect and generate in the OR due to privacy concerns. To address these concerns, we first study how joint person pose estimation and instance segmentation can be performed on low resolutions images with downsampling factors from 1x to 12x. Second, to address the domain shift and the lack of annotations, we propose a novel unsupervised domain adaptation method, called AdaptOR, to adapt a model from an in-the-wild labeled source domain to a statistically different unlabeled target domain. We propose to exploit explicit geometric constraints on the different augmentations of the unlabeled target domain image to generate accurate pseudo labels and use these pseudo labels to train the model on high- and low-resolution OR images in a self-training framework. Furthermore, we propose disentangled feature normalization to handle the statistically different source and target domain data. Extensive experimental results with detailed ablation studies on the two OR datasets MVOR+ and TUM-OR-test show the effectiveness of our approach against strongly constructed baselines, especially on the low-resolution privacy-preserving OR images. Finally, we show the generality of our method as a semi-supervised learning (SSL) method on the large-scale COCO dataset, where we achieve comparable results with as few as 1% of labeled supervision against a model trained with 100% labeled supervision. Code is available at https://github.com/CAMMA-public/HPE-AdaptOR.

Multi-modal temporal attention models for crop mapping from satellite time series

Optical and radar satellite time series are synergetic: optical images contain rich spectral information, while C-band radar captures useful geometrical information and is immune to cloud cover. Motivated by the recent success of temporal attention-based methods across multiple crop mapping tasks, we propose to investigate how these models can be adapted to operate on several modalities. We implement and evaluate multiple fusion schemes, including a novel approach and simple adjustments to the training procedure, significantly improving performance and efficiency with little added complexity. We show that most fusion schemes have advantages and drawbacks, making them relevant for specific settings. We then evaluate the benefit of multimodality across several tasks: parcel classification, pixel-based segmentation, and panoptic parcel segmentation. We show that by leveraging both optical and radar time series, multimodal temporal attention-based models can outmatch single-modality models in terms of performance and resilience to cloud cover. To conduct these experiments, we augment the PASTIS dataset (Garnot and Landrieu, 2021a) with spatially aligned radar image time series. The resulting dataset, PASTIS-R, constitutes the first large-scale, multimodal, and open-access satellite time series dataset with semantic and instance annotations. (Dataset available at: https://zenodo.org/record/5735646)

Diagnosis of Breast Cancer by K-Mean Clustering and Otsu Thresholding Segmentation Methods

Breast cancer has increased decidedly among women. But with early diagnosis, a positive response to treatment can be given. Researchers are conducting various studies in imaging methods to detect the disease early and accurately. In this study, 9 cancerous images taken from the TCİA image data bank were detected by K-mean clustering and the Otsu threshold method. Performance metrics were evaluated by comparing them with marked reference images (ground truth) by the radiologist. For the clustering process, TPR (True Positive Rate) 0.89, FPR (False Positive Rate) 0.14, similarity 0.67, accuracy 0.87, sensitivity 0.89, exact hit ratio 0.86, specificity 0.87, F Score 0.87 were found, respectively. For Otsu, TPR (True Positive Rate) 0.84, FPR (False Positive Rate) 0.12, similarity 0.73, accuracy 0.84, sensitivity 0.84, exact hit 0.86, specificity 0.87, F Score 0.84 were calculated. The aim of this study is to determine the tumor boundaries more accurately and to use them in imaging devices in the field of health with pixel-based segmentation. As a result, both methods were successful can be used in the field and close to each other.

Metaheuristic Algorithms Applied to Color Image Segmentation on HSV Space.

In this research, we propose an unsupervised method for segmentation and edge extraction of color images on the HSV space. This approach is composed of two different phases in which are applied two metaheuristic algorithms, respectively the Firefly (FA) and the Artificial Bee Colony (ABC) algorithms. In the first phase, we performed a pixel-based segmentation on each color channel, applying the FA algorithm and the Gaussian Mixture Model. The FA algorithm automatically detects the number of clusters, given by histogram maxima of each single-band image. The detected maxima define the initial means for the parameter estimation of the GMM. Applying the Bayes’ rule, the posterior probabilities of the GMM can be used for assigning pixels to clusters. After processing each color channel, we recombined the segmented components in the final multichannel image. A further reduction in the resultant cluster colors is obtained using the inner product as a similarity index. In the second phase, once we have assigned all pixels to the corresponding classes of the HSV space, we carry out the second step with a region-based segmentation applied to the corresponding grayscale image. For this purpose, the bioinspired Artificial Bee Colony algorithm is performed for edge extraction.

Feature selection using competitive coevolution of bio-inspired algorithms for the diagnosis of pulmonary emphysema

A Computer-Aided Diagnosis (CAD) system to assist a radiologist for diagnosing pulmonary emphysema from chest Computed Tomography (CT) slices has been developed. The lung tissues are segmented from the chest CT slices using Spatial Fuzzy C-Means (SFCM) clustering algorithm and the Regions of Interest (ROIs) are extracted using pixel-based segmentation. The ROIs considered for this work are pulmonary emphysematous lesions namely, centrilobular emphysema, paraseptal emphysema and sub-pleural bullae. The extracted ROIs are then validated and labelled by an expert radiologist. From each ROI, features with respect to shape, texture and run-length are extracted. A competitive coevolution model is proposed for Feature Selection (FS). The model makes use of two bio-inspired algorithms namely, Spider Monkey Optimization (SMO) algorithm and Paddy Field Algorithm (PFA) as its building blocks. FS is performed as a wrapper approach, using the bio-inspired algorithms namely, SMO and PFA with the accuracy of the Support Vector Machine (SVM) classifier as the fitness function. Ten-fold cross validation technique is used in training the SVM classifier using the selected features. The model is tested using two datasets: Real-time emphysema dataset and CT emphysema database (CTED) dataset. The accuracy, precision, recall and specificity obtained for both the datasets are (81.95%, 93.74%), (72.92%, 90.61%), (72.92%, 90.61%), (86.46%, 95.3%), which are better compared to the performance of SMO and PFA algorithms applied individually for FS and the CAD system without performing FS.

Dual Crisscross Attention Module for Road Extraction from Remote Sensing Images.

Traditional pixel-based semantic segmentation methods for road extraction take each pixel as the recognition unit. Therefore, they are constrained by the restricted receptive field, in which pixels do not receive global road information. These phenomena greatly affect the accuracy of road extraction. To improve the limited receptive field, a non-local neural network is generated to let each pixel receive global information. However, its spatial complexity is enormous, and this method will lead to considerable information redundancy in road extraction. To optimize the spatial complexity, the Crisscross Network (CCNet), with a crisscross shaped attention area, is applied. The key aspect of CCNet is the Crisscross Attention (CCA) module. Compared with non-local neural networks, CCNet can let each pixel only perceive the correlation information from horizontal and vertical directions. However, when using CCNet in road extraction of remote sensing (RS) images, the directionality of its attention area is insufficient, which is restricted to the horizontal and vertical direction. Due to the recurrent mechanism, the similarity of some pixel pairs in oblique directions cannot be calculated correctly and will be intensely dilated. To address the above problems, we propose a special attention module called the Dual Crisscross Attention (DCCA) module for road extraction, which consists of the CCA module, Rotated Crisscross Attention (RCCA) module and Self-adaptive Attention Fusion (SAF) module. The DCCA module is embedded into the Dual Crisscross Network (DCNet). In the CCA module and RCCA module, the similarities of pixel pairs are represented by an energy map. In order to remove the influence from the heterogeneous part, a heterogeneous filter function (HFF) is used to filter the energy map. Then the SAF module can distribute the weights of the CCA module and RCCA module according to the actual road shape. The DCCA module output is the fusion of the CCA module and RCCA module with the help of the SAF module, which can let pixels perceive local information and eight-direction non-local information. The geometric information of roads improves the accuracy of road extraction. The experimental results show that DCNet with the DCCA module improves the road IOU by 4.66% compared to CCNet with a single CCA module and 3.47% compared to CCNet with a single RCCA module.

Threshold segmentation based multi-layer analysis for detecting diabetic retinopathy using convolution neural network

Diabetic retinopathy (DR) syndrome affects the vision of the eyes by damaging the blood vessels. Fore-hand detection and prevention of this syndrome are most significant as it results in vision blindness. Diagnosis and procedural analysis of this syndrome with modern healthcare science and technology are aided through artificial intelligence and processing units. In this article, a threshold segmentation based DR detection method is introduced. This method is keen is classifying the foreground and background of the input retinal image and processing through pixel-based segmentation. The process of assessing the layers is augmented using a two-layer convolutional neural network (CNN) that mitigates the false positives during classification. This process is sequential in determining the precise detection of the infected region of the retina. Besides, the segment-based CNN (S-CNN) handles the flaw in diagnosis through two-hidden layers for differentiating the threshold and normalized conditions based on classification. The proposed method is reliable in achieving better accuracy of detection, sensitivity, and true positives.

A New Mask R-CNN-Based Method for Improved Landslide Detection

This article presents a novel method of landslide detection by exploiting the Mask R-CNN capability of identifying an object layout by using a pixel-based segmentation, along with transfer learning used to train the proposed model. A data set of 160 elements is created containing landslide and nonlandslide images. The proposed method consists of three steps: augmenting training image samples to increase the volume of the training data; fine-tuning with limited image samples; and performance evaluation of the algorithm in terms of precision, recall, and F1 measure, on the considered landslide images, by adopting ResNet-50 and 101 as backbone models. The experimental results are quite encouraging as the proposed method achieves precision equals to 1.00, recall 0.93, and F1 measure 0.97, when ResNet-101 is used as backbone model, and with a low number of landslide photographs used as training samples. The proposed algorithm can be potentially useful for land-use planners and policymakers of hilly areas where intermittent slope deformations necessitate landslide detection as a prerequisite before planning.