Segmentation Quality Measures Research Articles

Panoptic quality should be avoided as a metric for assessing cell nuclei segmentation and classification in digital pathology

Panoptic Quality (PQ), designed for the task of “Panoptic Segmentation” (PS), has been used in several digital pathology challenges and publications on cell nucleus instance segmentation and classification (ISC) since its introduction in 2019. Its purpose is to encompass the detection and the segmentation aspects of the task in a single measure, so that algorithms can be ranked according to their overall performance. A careful analysis of the properties of the metric, its application to ISC and the characteristics of nucleus ISC datasets, shows that is not suitable for this purpose and should be avoided. Through a theoretical analysis we demonstrate that PS and ISC, despite their similarities, have some fundamental differences that make PQ unsuitable. We also show that the use of the Intersection over Union as a matching rule and as a segmentation quality measure within PQ is not adapted for such small objects as nuclei. We illustrate these findings with examples taken from the NuCLS and MoNuSAC datasets. The code for replicating our results is available on GitHub (https://github.com/adfoucart/panoptic-quality-suppl).

(Retracted) Hybrid whale optimization algorithm-Levy flight approach for multilevel thresholding image segmentation

Determining the optimal threshold for multilevel image segmentation is a stimulating task. Image segmentation has many applications, such as content-based image retrieval, medical imaging, object detection, object recognition, machine vision, etc. In multilevel thresholding, multiple thresholds are used to segment complex images, and the images are segmented into multiple levels to extract meaningful information for further image analysis. Where threshold number is small, p-tile, Otsu, moment preserving, and entropy thresholding methods attain good accuracy. These classical thresholding methods are time-consuming, computation expensive, and unable to produce good segmentation accuracy with increased threshold numbers. To overcome these problems, the classical thresholding methods are utilized as objective functions with nature-inspired metaheuristic algorithms such as whale optimization algorithm (WOA), simplified swarm optimization (SSO), sine cosine algorithm (SCA), BAT algorithm, WOA-thresholding, and black widow optimization (BWO) to determine optimal multiple thresholds. Nature-inspired metaheuristic algorithms are widely used to search for an optimal solution for global optimization problems. WOA is widely used to find the optimal solution in search space. WOA suffers from entrapment into local optima due to premature convergence behavior. We introduce hybridization of WOA with Levy flight trajectory and named as hybrid whale optimization algorithm-Levy flight (HWOAL), which is utilized to find optimal multiple thresholds for multilevel image segmentation. Levy flight trajectory is utilized to increase diversity in the swarm population. The efficacy of HWOAL is tested on 23 benchmark optimization functions (F1 to F23) and compared with WOA, SSO, SCA, and BAT algorithms. Experiment arms that HWOAL is efficient and makes WOA faster, enhances the ability of exploitation and exploration phase, and can avoid getting stuck into local optima. The segmentation performance of the proposed HWOAL method has been compared with other algorithms, such as WOA, SSO, SCA, BAT, and other two recent segmentation algorithms, such as WOA-TH and BWO on several benchmark images (BSD 300). The experimental result is carried in 30 trials and analyzed based on objective fitness value, optimal multilevel threshold values, segmentation quality measures, such as mean square error, peak signal-to-noise ratio, structural similarity index, average difference, and computation time to compute optimal multilevel threshold values. The experiment conducted by the authors shows that the proposed HWOAL method is efficient, produces better fitness value, and segmentation metrics for multilevel image segmentation than other algorithms.

Optimising and comparing source-extraction tools using objective segmentation quality criteria

Context. With the growth of the scale, depth, and resolution of astronomical imaging surveys, there is increased need for highly accurate automated detection and extraction of astronomical sources from images. This also means there is a need for objective quality criteria, and automated methods to optimise parameter settings for these software tools. Aims. We present a comparison of several tools developed to perform this task: namely SExtractor, ProFound, NoiseChisel, and MTObjects. In particular, we focus on evaluating performance in situations that present challenges for detection. For example, faint and diffuse galaxies; extended structures, such as streams; and objects close to bright sources. Furthermore, we develop an automated method to optimise the parameters for the above tools. Methods. We present four different objective segmentation quality measures, based on precision, recall, and a new measure for the correctly identified area of sources. Bayesian optimisation is used to find optimal parameter settings for each of the four tools when applied to simulated data, for which a ground truth is known. After training, the tools are tested on similar simulated data in order to provide a performance baseline. We then qualitatively assess tool performance on real astronomical images from two different surveys. Results. We determine that when area is disregarded, all four tools are capable of broadly similar levels of detection completeness, while only NoiseChisel and MTObjects are capable of locating the faint outskirts of objects. MTObjects achieves the highest scores on all tests for all four quality measures, whilst SExtractor obtains the highest speeds. No tool has sufficient speed and accuracy to be well suited to large-scale automated segmentation in its current form.

Synergistic use of particle swarm optimization, artificial neural network, and extreme gradient boosting algorithms for urban LULC mapping from WorldView-3 images

Geographic object-based image analysis (GEOBIA) has emerged as an effective and evolving paradigm for analyzing very high resolution (VHR) images as it demonstrates preeminence over the traditional pixel-wise methods and enables the utilization of diverse spectral, geometrical, and textural information to for image classification. Among feature selection (FS) methods, metaheuristic FS techniques have recently demonstrated effective performance in the dimensionality reduction of GEOBIA features. In this study, an artificial neural network (ANN) was integrated with particle swarm optimization (PSO) to enhance the learning process and more effectively determine the most significant features and their importance using WorldView-3 (WV-3) satellite data. First, multi-resolution image segmentation parameters were tuned using Taguchi optimization technique and unsupervised segmentation quality measure. Second, the proposed ANN–PSO was compared with PSO under 100 iterations. The ANN–PSO integration achieved lower root mean square error (RMSE) in all the iterations. Third, state-of-the-art extreme gradient boosting (Xgboost) image classifier was used to derive the land use/land cover (LULC) map of the first study area and assess the transferability of the selected features on the second and third regions. The Xgboost classifier obtained 91.68%, 89.54%, and 89.33% overall accuracies for the first, second, and third sites, respectively. ANN contributed to an intelligent approach for identifying which features are more likely to be relevant and discriminate the land cover types. The proposed integrated FS is a promising approach and an efficient tool for determining significant features and enhancing the detection of urban LULC classes from WV-3 data.

Size Projection Algorithm: Optimal Thresholding Value Selection for Image Segmentation of Electrical Impedance Tomography

Thresholding is an efficient step to extract quantitative information since the potential artefacts are often introduced by the point-spread effect of tomographic imaging. The thresholding value was previously selected only relying on engineering experience or histogram of tomographic image, which often presents a great challenge to determine an accurate thresholding value for various applications. As the tomographic image features often do not provide sufficient information to choose the best thresholding value, the information implicit in the measured boundary data is introduced into the thresholding process in this paper. A projection error, the relative difference between the computed boundary data of current segmentation and the measured boundary data, is proposed as a quantitative measure of such image segmentation quality. Then, a new multistep image segmentation process, called size projection algorithm (SPA), is proposed to automatically determine an optimal thresholding value by minimising the projection error. Results of simulation and experiment demonstrate the improved performance of the SPA-based tomographic image segmentation. An application of size projection algorithm for gas-water two-phase flow visualisation is also reported in this paper.

A potential field segmentation based method for tumor segmentation on multi-parametric MRI of glioma cancer patients

BackgroundAccurate segmentation of brain tumors is vital for the gross tumor volume (GTV) definition in radiotherapy. Functional MR images like apparent diffusion constant (ADC) and fractional anisotropy (FA) images can provide more comprehensive information for sensitive detection of the GTV. We synthesize anatomical and functional MRI for accurate and semi-automatic segmentation of GTVs and improvement of clinical efficiency.MethodsFour MR image sets including T1-weighted contrast-enhanced (T1C), T2-weighted (T2), apparent diffusion constant (ADC) and fractional anisotropy (FA) images of 5 glioma patients were acquired and registered. A new potential field segmentation (PFS) method was proposed based on the concept of potential field in physics. For T1C, T2 and ADC images, global potential field segmentation (global-PFS) was used on user defined region of interest (ROI) for rough segmentation and then morphologically processed for accurate delineation of the GTV. For FA images, white matter (WM) was removed using local potential field segmentation (local-PFS), and then tumor extent was delineated with region growing and morphological methods. The individual segmentations of multi-parametric images were ensembled into a fused segmentation, considered as final GTV. GTVs were compared with manually delineated ground truth and evaluated with segmentation quality measure (Q), Dice’s similarity coefficient (DSC) and Sensitivity and Specificity.ResultsExperimental study with the five patients’ data and new method showed that, the mean values of Q, DSC, Sensitivity and Specificity were 0.80 (±0.07), 0.88 (±0.04), 0.92 (±0.01) and 0.88 (±0.05) respectively. The global-PFS used on ROIs of T1C, T2 and ADC images can avoid interferences from skull and other non-tumor areas. Similarity to local-PFS on FA images, it can also reduce the time complexity as compared with the global-PFS on whole image sets.ConclusionsEfficient and semi-automatic segmentation of the GTV can be achieved with the new method. Combination of anatomical and functional MR images has the potential to provide new methods and ideas for target definition in radiotherapy.

Large-scale urban mapping using integrated geographic object-based image analysis and artificial bee colony optimization from worldview-3 data

ABSTRACT Geographic object-based image analysis (GEOBIA) has demonstrated strong capability compared with pixel-based algorithms for urban characterization studies. In this study, new solutions to feature selection (FS) and image segmentation optimization are investigated in the GEOBIA domain. First, the combination of Taguchi-based optimization technique and F-score segmentation quality measures was adopted to optimize the parameters of multiresolution segmentation (MRS) and determine the optimum multiscale combinations of MRS parameters. Second, artificial bee colony (ABC) FS was integrated to select the most relevant features. Third, random forest (RF) classification algorithm was utilized to extract multiscale urban land use/land cover (LULC) classes from geographically wide images obtained from two WorldView-3 image datasets. The proposed method was developed in the first study area and later applied to the second study area for validation. Results of image segmentation optimization indicated that scales 40 and 80 were the best for classification. The result of FS through ABC outperformed those of other FS techniques, including support vector machine with recursive feature elimination (SVM-REF), variable selection using RF, Boruta, genetic algorithm, correlation-based FS, and chi-square, with an overall accuracy (OA) of 88.46%. Among the 100 examined features, only 25 were significant. The RF classification results showed a kappa coefficient (κ) of 0.84 in the first study area. The transferability and scalability of the best-performing features based on ABC FS were evaluated in the second study area, which covered a geographically wide scene of 162. The results for the second study area obtained an OA of 86.78% and a κ of 0.82. The proposed integrated method is an efficient and promising technique for high-quality LULC mapping of geographically wide areas.

Bayesian QuickNAT: Model uncertainty in deep whole-brain segmentation for structure-wise quality control

We introduce Bayesian QuickNAT for the automated quality control of whole-brain segmentation on MRI T1 scans. Next to the Bayesian fully convolutional neural network, we also present inherent measures of segmentation uncertainty that allow for quality control per brain structure. For estimating model uncertainty, we follow a Bayesian approach, wherein, Monte Carlo (MC) samples from the posterior distribution are generated by keeping the dropout layers active at test time. Entropy over the MC samples provides a voxel-wise model uncertainty map, whereas expectation over the MC predictions provides the final segmentation. Next to voxel-wise uncertainty, we introduce four metrics to quantify structure-wise uncertainty in segmentation for quality control. We report experiments on four out-of-sample datasets comprising of diverse age range, pathology and imaging artifacts. The proposed structure-wise uncertainty metrics are highly correlated with the Dice score estimated with manual annotation and therefore present an inherent measure of segmentation quality. In particular, the intersection over union over all the MC samples is a suitable proxy for the Dice score. In addition to quality control at scan-level, we propose to incorporate the structure-wise uncertainty as a measure of confidence to do reliable group analysis on large data repositories. We envisage that the introduced uncertainty metrics would help assess the fidelity of automated deep learning based segmentation methods for large-scale population studies, as they enable automated quality control and group analyses in processing large data repositories.

A thresholding method based on interval-valued intuitionistic fuzzy sets: an application to image segmentation

This paper proposes a new fuzzy approach for the segmentation of images. L-interval-valued intuitionistic fuzzy sets (IVIFSs) are constructed from two L-fuzzy sets that corresponds to the foreground (object) and the background of an image. Here, L denotes the number of gray levels in the image. The length of the membership interval of IVIFS quantifies the influence of the ignorance in the construction of the membership function. Threshold for an image is chosen by finding an IVIFS with least entropy. Contributions also include a comparative study with ten other image segmentation techniques. The results obtained by each method have been systematically evaluated using well-known measures for judging the segmentation quality. The proposed method has globally shown better results in all these segmentation quality measures. Experiments also show that the results acquired from the proposed method are highly correlated to the ground truth images.

Improving Graph-Based Image Segmentation Using Nonlinear Color Similarity Metrics

We present a new segmentation method called weighted Felzenszwalb and Huttenlocher (WFH), an improved version of the well-known graph-based segmentation method, Felzenszwalb and Huttenlocher (FH). Our algorithm uses a nonlinear discrimination function based on polynomial Mahalanobis Distance (PMD) as the color similarity metric. Two empirical validation experiments were performed using as a golden standard ground truths (GTs) from a publicly available source, the Berkeley dataset, and an objective segmentation quality measure, the Rand dissimilarity index. In the first experiment the results were compared against the original FH method. In the second, WFH was compared against several well-known segmentation methods. In both cases, WFH presented significant better similarity results when compared with the golden standard and segmentation results presented a reduction of over-segmented regions.

Materials-based 3D segmentation of unknown objects from dual-energy computed tomography imagery in baggage security screening

We present a novel technique for the 3D segmentation of unknown objects from cluttered dual-energy Computed Tomography (CT) data obtained in the baggage security-screening domain. Initial materials-based coarse segmentations, generated using the Dual-Energy Index (DEI), are refined by partitioning at automatically detected regions. Partitioning is guided by a novel random forest based quality metric, trained to recognise high-quality, single-object segments. A second novel segmentation quality measure is presented for quantifying the quality of full segmentations based on the random forest metric of the constituent parts and the error in the number of objects segmented. In a comparative evaluation between the proposed approach and three state-of-the-art volumetric segmentation techniques designed for single-energy CT data (two region-growing [1,2] and one graph-based [3]) our method is shown to outperform both region-growing methods in terms of segmentation quality and speed. Although the graph-based approach generates more accurate partitions, it is characterised by high processing times and is significantly outperformed by the proposed method in this regard. The observations made in this study indicate that the proposed segmentation technique is well-suited to the baggage security-screening domain, where the demand for computational efficiency is paramount to maximise throughput.

Image Segmentation Algorithm for Images having Asymmetrically Distributed Image Regions

Image segmentation is one of the most important prerequisite for image analysis. This paper addresses the problem of model based image segmentation using mixture of Pearsonian Type I Distribution. Here the whole image is characterized by a mixture of K-components Type I Personian Distribution. The Pearsonian Type I Distribution is capable of portraying the asymmetric nature of image regions more close to the reality. The model parameters estimated by EM Algorithm. The initialization of model parameters is done through the integrating the histogram method, K-means algorithm and moment of method of estimators. The Image Segmentation algorithm is developed using component maximum likelihood. The proposed algorithm is evolved by conducting experiments with 5 images taken from Berkeley image data set. The Experiments revealed that this algorithm performs better than that of Gaussian mixture model with respect to image segmentation quality measures such as PRI, VOC and GCE for some images taken in sky and on earth. .

Reconstruction of White Matter Tracts via Repeated Deterministic Streamline Tracking – Initial Experience

Diffusion Tensor Imaging (DTI) and fiber tractography are established methods to reconstruct major white matter tracts in the human brain in-vivo. Particularly in the context of neurosurgical procedures, reliable information about the course of fiber bundles is important to minimize postoperative deficits while maximizing the tumor resection volume. Since routinely used deterministic streamline tractography approaches often underestimate the spatial extent of white matter tracts, a novel approach to improve fiber segmentation is presented here, considering clinical time constraints. Therefore, fiber tracking visualization is enhanced with statistical information from multiple tracking applications to determine uncertainty in reconstruction based on clinical DTI data. After initial deterministic fiber tracking and centerline calculation, new seed regions are generated along the result’s midline. Tracking is applied to all new seed regions afterwards, varying in number and applied offset. The number of fibers passing each voxel is computed to model different levels of fiber bundle membership. Experimental results using an artificial data set of an anatomical software phantom are presented, using the Dice Similarity Coefficient (DSC) as a measure of segmentation quality. Different parameter combinations were classified to be superior to others providing significantly improved results with DSCs of 81.02%±4.12%, 81.32%±4.22% and 80.99%±3.81% for different levels of added noise in comparison to the deterministic fiber tracking procedure using the two-ROI approach with average DSCs of 65.08%±5.31%, 64.73%±6.02% and 65.91%±6.42%. Whole brain tractography based on the seed volume generated by the calculated seeds delivers average DSCs of 67.12%±0.86%, 75.10%±0.28% and 72.91%±0.15%, original whole brain tractography delivers DSCs of 67.16%, 75.03% and 75.54%, using initial ROIs as combined include regions, which is clearly improved by the repeated fiber tractography method.

Unsupervised colour image segmentation by low-level perceptual grouping

This paper proposes a new unsupervised approach for colour image segmentation. A hierarchy of image partitions is created on the basis of a function that merges spatially connected regions according to primary perceptual criteria. Likewise, a global function that measures the goodness of each defined partition is used to choose the best low-level perceptual grouping in the hierarchy. Contributions also include a comparative study with five unsupervised colour image segmentation techniques. These techniques have been frequently used as a reference in other comparisons. The results obtained by each method have been systematically evaluated using four well-known unsupervised measures for judging the segmentation quality. Our methodology has globally shown the best performance, obtaining better results in three out of four of these segmentation quality measures. Experiments will also show that our proposal finds low-level perceptual solutions that are highly correlated with the ones provided by humans.

Segmentation of Cereal Plant Images Using Level Set Methods – A Comparative Study

In this paper we evaluate the quality of segmentation of plant images achieved by different level set methods commonly used in the literature. The plants of study are narrow-leaf cereal plants at different growth stages and the segmentation quality measure was considered to be the boundary, leaf tips, and axils. The results show that region-based level set methods can perform the segmentation of plants with high accuracy when the plants are at either early or mature stages of growth. The results also show that contour based level set algorithms are not applicable to the segmentation of narrow leaf plants because the front being computed does not advance to the high curvature features, such as sharp tips and axils. A typical image of a mature plant has isolated regions from the interlacing of leaves. Only region-based methods can perform the segmentation with good accuracy. Level set methods are sensitive to initialization and parameter selection.

Simultaneous Estimation of Chords and Musical Context From Audio

Chord labels provide a concise description of musical harmony. In pop and jazz music, a sequence of chord labels is often the only written record of a song, and forms the basis of so-called lead sheets. We devise a fully automatic method to simultaneously estimate from an audio waveform the chord sequence including bass notes, the metric positions of chords, and the key. The core of the method is a six-layered dynamic Bayesian network, in which the four hidden source layers jointly model metric position, key, chord, and bass pitch class, while the two observed layers model low-level audio features corresponding to bass and treble tonal content. Using 109 different chords our method provides substantially more harmonic detail than previous approaches while maintaining a high level of accuracy. We show that with 71% correctly classified chords our method significantly exceeds the state of the art when tested against manually annotated ground truth transcriptions on the 176 audio tracks from the MIREX 2008 Chord Detection Task. We introduce a measure of segmentation quality and show that bass and meter modeling are especially beneficial for obtaining the correct level of granularity.

Spatiotemporal Region Enhancement and Merging for Unsupervized Object Segmentation

This paper proposes an unsupervized offline video object segmentation method that introduces a number of improvements to existing work in the area. It consists of the following steps. The initial segmentation utilizes object color and motion variance to more accurately classify image pixels in the first frame. Histogram-based merging is then employed to reduce oversegmentation of the first frame. During object tracking, segmentation quality measures based on object color and motion contrast are taken. These measures are then used to enhance video objects through selective pixel reclassification. After object enhancement, cumulative histogram-based merging, occlusion handling, and island detection are used to help group regions into meaningful objects. Compared to two reference methods, greater success and improved accuracy in segmenting video objects are first demonstrated by subjectively examining selected frames from a set of standard video sequences. Objective results are obtained through the use of a set of measures that aim at evaluating the accuracy of object boundaries and temporal stability through the use of color, motion, and histograms.