Evaluation Of Segmentation Quality Research Articles

Individual Tree Segmentation Quality Evaluation Using Deep Learning Models LiDAR Based

Individual Tree Segmentation Quality Evaluation Using Deep Learning Models LiDAR Based

A benchmark study of convolutional neural networks in fully automatic segmentation of aortic root.

Recent clinical studies have suggested that introducing 3D patient-specific aortic root models into the pre-operative assessment procedure of transcatheter aortic valve replacement (TAVR) would reduce the incident rate of peri-operative complications. Tradition manual segmentation is labor-intensive and low-efficient, which cannot meet the clinical demands of processing large data volumes. Recent developments in machine learning provided a viable way for accurate and efficient medical image segmentation for 3D patient-specific models automatically. This study quantitively evaluated the auto segmentation quality and efficiency of the four popular segmentation-dedicated three-dimensional (3D) convolutional neural network (CNN) architectures, including 3D UNet, VNet, 3D Res-UNet and SegResNet. All the CNNs were implemented in PyTorch platform, and low-dose CTA image sets of 98 anonymized patients were retrospectively selected from the database for training and testing of the CNNs. The results showed that despite all four 3D CNNs having similar recall, Dice similarity coefficient (DSC), and Jaccard index on the segmentation of the aortic root, the Hausdorff distance (HD) of the segmentation results from 3D Res-UNet is 8.56 ± 2.28, which is only 9.8% higher than that of VNet, but 25.5% and 86.4% lower than that of 3D UNet and SegResNet, respectively. In addition, 3D Res-UNet and VNet also performed better in the 3D deviation location of interest analysis focusing on the aortic valve and the bottom of the aortic root. Although 3D Res-UNet and VNet are evenly matched in the aspect of classical segmentation quality evaluation metrics and 3D deviation location of interest analysis, 3D Res-UNet is the most efficient CNN architecture with an average segmentation time of 0.10 ± 0.04 s, which is 91.2%, 95.3% and 64.3% faster than 3D UNet, VNet and SegResNet, respectively. The results from this study suggested that 3D Res-UNet is a suitable candidate for accurate and fast automatic aortic root segmentation for pre-operative assessment of TAVR.

Feature Contrastive Learning for No-Reference Segmentation Quality Evaluation

No-reference segmentation quality evaluation aims to evaluate the quality of image segmentation without any reference image during the application process. It usually depends on certain quality criteria to describe a good segmentation with some prior knowledge. Therefore, there is a need for a precise description of the objects in the segmentation and an integration of the representation in the evaluation process. In this paper, from the perspective of understanding the semantic relationship between the original image and the segmentation results, we propose a feature contrastive learning method. This method can enhance the performance of no-reference segmentation quality evaluations and be applied in semantic segmentation scenarios. By learning the pixel-level similarity between the original image and the segmentation result, a contrastive learning step is performed in the feature space. In addition, a class activation map (CAM) is used to guide the evaluation, making the score more consistent with the human visual judgement. Experiments were conducted on the PASCAL VOC2012 dataset, with segmentation results obtained by state-of-the-art (SoA) segmentation methods. We adopted two meta-measure criteria to validate the efficiency of the proposed method. Compared with other no-reference evaluation methods, our method achieves a higher accuracy which is comparable to the supervised evaluation methods and partly even exceeds them.

SECURE: A Segmentation Quality Evaluation Metric on SEM Images for Reverse Engineering on Integrated Circuits

SECURE: A Segmentation Quality Evaluation Metric on SEM Images for Reverse Engineering on Integrated Circuits

Benchmarking of deep learning algorithms for 3D instance segmentation of confocal image datasets.

Segmenting three-dimensional (3D) microscopy images is essential for understanding phenomena like morphogenesis, cell division, cellular growth, and genetic expression patterns. Recently, deep learning (DL) pipelines have been developed, which claim to provide high accuracy segmentation of cellular images and are increasingly considered as the state of the art for image segmentation problems. However, it remains difficult to define their relative performances as the concurrent diversity and lack of uniform evaluation strategies makes it difficult to know how their results compare. In this paper, we first made an inventory of the available DL methods for 3D cell segmentation. We next implemented and quantitatively compared a number of representative DL pipelines, alongside a highly efficient non-DL method named MARS. The DL methods were trained on a common dataset of 3D cellular confocal microscopy images. Their segmentation accuracies were also tested in the presence of different image artifacts. A specific method for segmentation quality evaluation was adopted, which isolates segmentation errors due to under- or oversegmentation. This is complemented with a 3D visualization strategy for interactive exploration of segmentation quality. Our analysis shows that the DL pipelines have different levels of accuracy. Two of them, which are end-to-end 3D and were originally designed for cell boundary detection, show high performance and offer clear advantages in terms of adaptability to new data.

Image segmentation evaluation: a survey of methods

Image segmentation is a prerequisite for image processing. There are many methods for image segmentation, and as a result, a great number of methods for evaluating segmentation results have also been proposed. How to effectively evaluate the quality of image segmentation is very important. In this paper, the existing image segmentation quality evaluation methods are summarized, mainly including unsupervised methods and supervised methods. Based on hot issues, the application of metrics in natural, medical and remote sensing image evaluation is further outlined. In addition, an experimental comparison for some methods were carried out and the effectiveness of these methods was ranked. At the same time, the effectiveness of classical metrics for remote sensing and medical image evaluation is also verified.

Deadlift Recognition and Application based on Multiple Modalities using Recurrent Neural Network

Multiple Sclerosis (MS) is a chronic, often disabling, autoimmune disease affecting the central nervous system and characterized by demyelination and neuropathic alterations. Magnetic Resonance (MR) images plays a pivotal role in the diagnosis and the screening of MS. MR images identify and localize demyelinating lesions (or plaques) and possible associated atrophic lesions whose MR aspect is in relation with the evolution of the disease. We propose a novel MS lesions segmentation method for MR images, based on Convolutional Neural Networks (CNNs) and partial self-supervision and studied the pros and cons of using self-supervision for the current segmentation task. Investigating the transferability by freezing the firsts convolutional layers, we discovered that improvements are obtained when the CNN is retrained from the first layers. We believe such results suggest that MRI segmentation is a singular task needing high level analysis from the very first stages of the vision process, as opposed to vision tasks aimed at day-to-day life such as face recognition or traffic sign classification. The evaluation of segmentation quality has been performed on full image size binary maps assembled from predictions on image patches from an unseen database.

An improved thermal exchange optimization based GLCM for multi-level image segmentation

The gray-level co-occurrence matrix (GLCM) can obtain the pixel matrix of the image, and selecting multiple thresholds for the matrix can obtain better segmentation results. However, as the number of threshold increases, the computational complexity of the algorithm will also increase. In order to solve this problem, this paper proposes a multi-threshold image segmentation method based on thermal exchange optimization (TEO) algorithm, and take a novel diagonal class entropy (DCE) as the fitness function. We improve TEO algorithm by using two strategic methods of Levy flight (LF) and opposition-based learning (OBL). In order to verify the segmentation ability of the proposed algorithm, color natural images, satellite images and Berkeley images are taken as experimental objects to analyze the segmentation result graph and image segmentation quality evaluation indexes. Experimental results show that the GLCM-ITEO algorithm has good segmentation capability, less CPU time.

Segmentation of 3D FIB-SEM data with pore-back effect

Digital rock physics is used for the investigation of oil and gas reservoirs. It involves various mathematical simulations on a digital representation of a rock sample, which is usually obtained with imaging techniques. Focused ion beam scanning electron microscopy (FIB-SEM) tomography provides high-resolution images of sequential layers of a sample, and segmentation of these images is a key stage in the construction of 3D digital rock. Conventional segmentation methods are not applicable for FIB-SEM images due to specific artifacts such as the pore-back effect. We propose a new segmentation algorithm that relies on the marker-controlled watershed, variance filter and morphological operations. The results are validated with the use of manually labelled ground truth data. Furthermore, we develop a new metric for evaluation of segmentation quality. This metric is based on analysis of segmented regions and, in the case of porous media, provides more reliable evaluation than pixel-wise measures.

Hierarchical Segmentation Evaluation of Region-Based Image Hierarchy

Multiscale image segmentation plays an important role in object-based image analysis (OBIA) applications, and the evaluation of segmentation quality is a hot topic for OBIA community. Recently, with the increasing use of multiscale and hierarchical strategy in OBIA works, region-based image hierarchies have attracted increasing attentions. A region-based image hierarchy (i.e., binary partition tree and scale-sets hierarchy) is a tree-like structure, and it can be used to record multiscale segmentation results. Although many methods have been employed to evaluate multiscale segmentation, they cannot be applied to evaluate a region-based image hierarchy directly. In this study, a hierarchical segmentation evaluation approach was proposed to evaluate the upper-bound accuracy of a region-based image hierarchy. In our study, an image hierarchy is implemented using a region merging process, and organized as a scale-indexed binary partition tree; the best matching segment of each reference polygon is then selected by scanning whole hierarchy; and finally, the similarity between reference polygons and corresponding best matching segments are used to evaluate the overall performance of the hierarchy. Two high spatial resolution images (GaoFen-2 and QuickBird) and three region merging criteria were used to evaluate the effectiveness of the proposed approach. Moreover, the proposed method was compared with traditional multiscale evaluation strategies. The experimental results have demonstrated the effectiveness of the proposed approach, as well as the advantages compared to existing approaches.

EVALUATION METHODS OF IMAGE SEGMENTATION QUALITY

Context. The basic methods of quantitative evaluation of image segmentation quality are explored. They are used to select segmentation algorithms for specific image classes. The object of the study is cytological and histological images that are used in diagnosing the pathological processes in oncology. The subject of the study is quantitative methods for segmentation algorithms’ quality evaluation. Objective. The purpose of the work is to introduce the Gromov-Fr chet metric and develop a metric-based method for quantitative evaluation of segmentation quality for image segmentation algorithms’ comparison. Method. The quantitative evaluation criteria, which are based on comparison with etalon image and without the comparison with etalon image, are analyzed. The algorithms for measuring the distances between images based on the Fr chet, Hausdorff, and Gromov-Hausdorff metrics are analyzed. To calculate the distance between the contours of images, the Gromov-Fr chet distance was introduced. The condition of identity, symmetry and triangle is proved, and it is shown that the Gromov-Fr chet distance is a metric. The metric-based method of quantitative evaluation of segmentation quality is developed. It is based on the use of the Gromov-Hausdorff and Gromov-Fr chet metrics. The method is based on the algorithms for non-convex-into-convex polygon transformation, weighted chord algorithm, and algorithms for calculating the Fr chet and Hausdorff distances. To calculate the Hausdorff distance between convex regions, the Atalah’s algorithm was used. The Thierry and Manillo algorithm was used to find the discrete Fr chet distance. These algorithms have the lowest computational complexity among their class of algorithms. Results. The Gromov-Fr chet metric was introduced and the metric-based method of quantitative evaluation of segmentation quality was developed. Conclusions. The conducted experiments on the basis of cytological images confirmed the performance of software for evaluation the distances between images. The developed method showed a high accuracy of estimation the distances between images. The developed software module was used in intelligence systems for diagnosing the breast precancerous and cancerous conditions. The software can be used in various software systems of computer vision. Promising areas for further research are search for new metrics to evaluate the distances between images

HOSUR: A Novel Measure for Evaluation of Image Segmentation Quality

Image segmentation is one of the vital tasks in image processing. Nevertheless, no universally accepted quality measure for evaluating the performance of various segmentation algorithms or even different parameterizations of the same algorithm exists. In this paper, we propose a new segmentation evaluation measure, based on the fusion of HOG and SURF features. We call it the HOSUR. HOSUR exploits the local shape and corner information to evaluate the similarity between a given segmentation and its respective ground truth. It thus belongs to the category of supervised evaluation measures. Experimental results show accuracy of up to 85%

A Novel Unsupervised Segmentation Quality Evaluation Method for Remote Sensing Images.

The segmentation of a high spatial resolution remote sensing image is a critical step in geographic object-based image analysis (GEOBIA). Evaluating the performance of segmentation without ground truth data, i.e., unsupervised evaluation, is important for the comparison of segmentation algorithms and the automatic selection of optimal parameters. This unsupervised strategy currently faces several challenges in practice, such as difficulties in designing effective indicators and limitations of the spectral values in the feature representation. This study proposes a novel unsupervised evaluation method to quantitatively measure the quality of segmentation results to overcome these problems. In this method, multiple spectral and spatial features of images are first extracted simultaneously and then integrated into a feature set to improve the quality of the feature representation of ground objects. The indicators designed for spatial stratified heterogeneity and spatial autocorrelation are included to estimate the properties of the segments in this integrated feature set. These two indicators are then combined into a global assessment metric as the final quality score. The trade-offs of the combined indicators are accounted for using a strategy based on the Mahalanobis distance, which can be exhibited geometrically. The method is tested on two segmentation algorithms and three testing images. The proposed method is compared with two existing unsupervised methods and a supervised method to confirm its capabilities. Through comparison and visual analysis, the results verified the effectiveness of the proposed method and demonstrated the reliability and improvements of this method with respect to other methods.

Fast Evaluation of Segmentation Quality with Parallel Computing

In digital image processing and computer vision, a fairly frequent task is the performance comparison of different algorithms on enormous image databases. This task is usually time-consuming and tedious, such that any kind of tool to simplify this work is welcome. To achieve an efficient and more practical handling of a normally tedious evaluation, we implemented the automatic detection system, with the help of MATLAB®’s Parallel Computing Toolbox™. The key parts of the system have been parallelized to achieve simultaneous execution and analysis of segmentation algorithms on the one hand and the evaluation of detection accuracy for the nonforested regions, such as a study case, on the other hand. As a positive side effect, CPU usage was reduced and processing time was significantly decreased by 68.54% compared to sequential processing (i.e., executing the system with each algorithm one by one).

Evaluation of Segmentation Quality via Adaptive Composition of Reference Segmentations.

Evaluating image segmentation quality is a critical step for generating desirable segmented output and comparing performance of algorithms, among others. However, automatic evaluation of segmented results is inherently challenging since image segmentation is an ill-posed problem. This paper presents a framework to evaluate segmentation quality using multiple labeled segmentations which are considered as references. For a segmentation to be evaluated, we adaptively compose a reference segmentation using multiple labeled segmentations, which locally matches the input segments while preserving structural consistency. The quality of a given segmentation is then measured by its distance to the composed reference. A new dataset of 200 images, where each one has 6 to 15 labeled segmentations, is developed for performance evaluation of image segmentation. Furthermore, to quantitatively compare the proposed segmentation evaluation algorithm with the state-of-the-art methods, a benchmark segmentation evaluation dataset is proposed. Extensive experiments are carried out to validate the proposed segmentation evaluation framework.

Region Based Exemplar References for Image Segmentation Evaluation

Quantitative evaluation of image segmentation quality is usually based on comparing a segmentation with multiple reference segmentations. Instead of holistically comparing with each reference, we propose a region based evaluation framework, where an exemplar reference is adaptively constructed and applied to a generally defined evaluation measure. As examples, we implement three well-known evaluation measures and present an efficient scheme to compute each measure. Extensive experiments on the benchmark databases show that the proposed evaluation framework can improve the evaluation precision of existing measures.

Extended surface distance for local evaluation of 3D medical image segmentations

The evaluation of 3D medical image segmentation quality requires a reliable detailed comparison of a reference segmentation with an automatic segmentation. It should be able to measure the quality accurately and, thus, to reveal problematic regions. While several (global) measures, providing a single quality value, are available, the only widely used local measure is the Surface Distance (i.e., point-to-surface distance). This measure, however, has significant drawbacks such as asymmetry and underestimation in distant and differently formed regions. Other available measures have limited suitability for 3D medical segmentation evaluation. We present a more reliable distance measure for assessing and analyzing local differences between automatic and reference (i.e., ground truth) 3D segmentations. We identify and overcome Surface Distance drawbacks, esp. in regions with larger dissimilarities. We evaluated our approach on four real medical image datasets. The results indicate that our measure provides more accurate local distance values.

Segmentation quality evaluation using region-based precision and recall measures for remote sensing images

Segmentation of remote sensing images is a critical step in geographic object-based image analysis. Evaluating the performance of segmentation algorithms is essential to identify effective segmentation methods and optimize their parameters. In this study, we propose region-based precision and recall measures and use them to compare two image partitions for the purpose of evaluating segmentation quality. The two measures are calculated based on region overlapping and presented as a point or a curve in a precision–recall space, which can indicate segmentation quality in both geometric and arithmetic respects. Furthermore, the precision and recall measures are combined by using four different methods. We examine and compare the effectiveness of the combined indicators through geometric illustration, in an effort to reveal segmentation quality clearly and capture the trade-off between the two measures. In the experiments, we adopted the multiresolution segmentation (MRS) method for evaluation. The proposed measures are compared with four existing discrepancy measures to further confirm their capabilities. Finally, we suggest using a combination of the region-based precision–recall curve and the F-measure for supervised segmentation evaluation.

Segmentation of HeLa cells in phase-contrast images with and without DAPI stained cell nuclei

Microscopy is an essential tool for the observation of microbial effectors in host cells. Manual evaluation of the acquired image data is elaborate and error prone. Thus, in order to increase objectivity and reproducibility of results, an automated evaluation of the experiment is performed. The present study evaluates methods to delineate HeLa cells depicted in phase-contrast (PC) micrographs, with and without the additional use of the DAPI stained cell nuclei. For initial cell detection a variance filter is applied to the PC micrographs. Based on this foreground cells are segmented with an intensity based k-means clustering. In a second step, the foreground areas are refined to match the cell boarders by region growing on the background. These foreground areas are used to study the performance of cell separation methods, namely seeded watershed (SW) and graph cuts (GC). Either the maxima of the distance transform of the foreground regions in the PC images (“dist”) or alternatively regions of the cell nuclei (“nuc”) obtained from the DAPI channel are utilized to initialize the cell separation methods. For the detailed evaluation of segmentation quality the cells were categorized to four classes, namely isolated cells, touching cells, overlapping and overlaying cells. With regards the rating of the obtained performance using the Jaccard overlap measure, an intra-observer variance was determined as the median of performance values of all cells with p=0.79. The performance using seeds of type “dist” resulted in pSW(dist)=0.59 for watershed transform and pGC(dist)=0.60 for graph cuts. Switching seeds to type “nuc” performance improved to pSW(nuc)=0.72 for watershed transform and pGC(nuc)=0.69 for graph cuts.

A Function for Quality Evaluation of Retinal Vessel Segmentations

Retinal blood vessel assessment plays an important role in the diagnosis of ophthalmic pathologies. The use of digital images for this purpose enables the application of a computerized approach and has fostered the development of multiple methods for automated vascular tree segmentation. Metrics based on contingency tables for binary classification have been widely used for evaluating the performance of these algorithms. Metrics from this family are based on the measurement of a success or failure rate in the detected pixels, obtained by means of pixel-to-pixel comparison between the automated segmentation and a manually-labeled reference image. Therefore, vessel pixels are not considered as a part of a vascular structure with specific features. This paper contributes a function for the evaluation of global quality in retinal vessel segmentations. This function is based on the characterization of vascular structures as connected segments with measurable area and length. Thus, its design is meant to be sensitive to anatomical vascularity features. Comparison of results between the proposed function and other general quality evaluation functions shows that this proposal renders a high matching degree with human quality perception. Therefore, it can be used to enhance quality evaluation in retinal vessel segmentations, supplementing the existing functions. On the other hand, from a general point of view, the applied concept of measuring descriptive properties may be used to design specialized functions aimed at segmentation quality evaluation in other complex structures.