Interactive Segmentation Method Research Articles

Image Co-Segmentation via Examples Guidance

Given a collection of images which contains objects from the same category, the co-segmentation methods aim at simultaneously segmenting such common objects in each image. Most of existing co-segmentation approaches rely on comput-ing similarities inter-regions representing foregrounds in these images. However, region similarity measurement is challenging due to the large appearance variations among objects in the same category. In addition, for real-world images which have cluttered backgrounds, the existing co-segmentation approaches miss sufficient robustness to extract the common object from the background. In this paper, we propose a new co-segmentation method which takes advantage of the reliable segmentation of few selected images, in order to guide the segmentation of the remaining images in the collection. A random sample of images is first selected from the image collection. Then, the selected images are segmented using an interactive segmentation method. These segmentation results are used to construct positive/negative samples of the targeted common object and background regions respectively. Finally, these samples are propagated to the remain-ing images in the collection through computing both local and global consistency. The experiments on the iCoseg and MSRC datasets demonstrate the performance and robustness of the proposed method.

Marker-based watershed transform method for fully automatic mandibular segmentation from CBCT images.

To propose a reliable and practical method for automatically segmenting the mandible from CBCT images. The marker-based watershed transform is a region-growing approach that dilates or "floods" predefined markers onto a height map whose ridges denote object boundaries. We applied this method to segment the mandible from the rest of the CBCT image. The height map was generated to enhance the sharp decreases of intensity at the mandible/tissue border and suppress noise by computing the intensity gradient image of the CBCT itself. Two sets of markers, "mandible" and "background" were automatically placed inside and outside the mandible, respectively in a novel image using image registration. The watershed transform flooded the gradient image by dilating the markers simultaneously until colliding at watershed lines, estimating the mandible boundary. CBCT images of 20 adolescent subjects were chosen as test cases. Segmentation accuracy of the proposed method was evaluated by measuring overlap (Dice similarity coefficient) and boundary agreement against a well-accepted interactive segmentation method described in the literature. The Dice similarity coefficient was 0.97 ± 0.01 (mean ± SD), indicating almost complete overlap between the automatically and the interactively segmented mandibles. Boundary deviations were predominantly under 1 mm for most of the mandibular surfaces. The errors were mostly from bones around partially erupted wisdom teeth, the condyles and the dental enamels, which had minimal impact on the overall morphology of the mandible. The marker-based watershed transform method produces segmentation accuracy comparable to the well-accepted interactive segmentation approach.

A fully convolutional two-stream fusion network for interactive image segmentation

In this paper, we propose a novel fully convolutional two-stream fusion network (FCTSFN) for interactiveimage segmentation. The proposed network includes two sub-networks: a two-stream late fusion network (TSLFN) that predicts the foreground at a reduced resolution, and a multi-scale refining network (MSRN) that refines the foreground at full resolution. The TSLFN includes two distinct deep streams followed by a fusion network. The intuition is that, since user interactions are more direct information on foreground/background than the image itself, the two-stream structure of the TSLFN reduces the number of layers between the pure user interaction features and the network output, allowing the user interactions to have a more direct impact on the segmentation result. The MSRN fuses the features from different layers of TSLFN with different scales, in order to seek the local to global information on the foreground to refine the segmentation result at full resolution. We conduct comprehensive experiments on four benchmark datasets. The results show that the proposed network achieves competitive performance compared to current state-of-the-art interactive image segmentation methods. 1

DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation.

Accurate medical image segmentation is essential for diagnosis, surgical planning and many other applications. Convolutional Neural Networks (CNNs) have become the state-of-the-art automatic segmentation methods. However, fully automatic results may still need to be refined to become accurate and robust enough for clinical use. We propose a deep learning-based interactive segmentation method to improve the results obtained by an automatic CNN and to reduce user interactions during refinement for higher accuracy. We use one CNN to obtain an initial automatic segmentation, on which user interactions are added to indicate mis-segmentations. Another CNN takes as input the user interactions with the initial segmentation and gives a refined result. We propose to combine user interactions with CNNs through geodesic distance transforms, and propose a resolution-preserving network that gives a better dense prediction. In addition, we integrate user interactions as hard constraints into a back-propagatable Conditional Random Field. We validated the proposed framework in the context of 2D placenta segmentation from fetal MRI and 3D brain tumor segmentation from FLAIR images. Experimental results show our method achieves a large improvement from automatic CNNs, and obtains comparable and even higher accuracy with fewer user interventions and less time compared with traditional interactive methods.

Global graph diffusion for interactive object extraction

Many interactive image segmentation methods design energy functions based on the local relationship of neighboring pixels, which is insufficient at capturing the abundant information of an image, and these methods are susceptible to many problems, such as sensitivity to seeds and under-segmentation of boundaries. To solve these problems, this paper explores utilizing the global relationship for interactive image segmentation. To effectively obtain the global information of the image, we first propose a robust affinity diffusion (RAD) method to propagate the local affinity graph. Compared with the existing diffusion-based approaches, the advantage of RAD is that it can converge to an effective limit value, which makes the diffusion process more computationally efficient and easier to control. The segmentation model is then constructed based on this convergent global graph. To efficiently utilize the global information, the energy function is designed by the multiplication of the global affinity matrix and a prior probability vector. The use of global information can significantly improve the segmentation performance. Experiments on challenging data sets demonstrate that RAD can obtain better results than state-of-the-art methods.

Geodesic registration for interactive atlas-based segmentation using learned multi-scale anatomical manifolds

Atlas-based segmentation is often used to segment medical image regions. For intensity-normalized data, the quality of these segmentations is highly dependent on the similarity between the atlas and the target under the used registration method. We propose a geodesic registration method for interactive atlas-based segmentation using empirical multi-scale anatomical manifolds. The method utilizes unlabeled images together with the labeled atlases to learn empirical anatomical manifolds. These manifolds are defined on distinct scales and regions and are used to propagate the labeling information from the atlases to the target along anatomical geodesics. The resulting competing segmentations from the different manifolds are then ranked according to an image-based similarity measure. We used image volumes acquired using magnetic resonance imaging from 36 subjects. The performance of the method was evaluated using a liver segmentation task. The result was then compared to the corresponding performance of direct segmentation using Dice Index statistics. The method shows a significant improvement in liver segmentation performance between the proposed method and direct segmentation. Furthermore, the standard deviation in performance decreased significantly. Using competing complementary manifolds defined over a hierarchy of region of interests gives an additional improvement in segmentation performance compared to the single manifold segmentation.

Tap and Shoot Segmentation

We present a new segmentation method that leverages latent photographic information available at the moment of taking pictures. Photography on a portable device is often done by tapping to focus before shooting the picture. This tap-and-shoot interaction for photography not only specifies the region of interest but also yields useful focus/defocus cues for image segmentation. However, most of the previous interactive segmentation methods address the problem of image segmentation in a post-processing scenario without considering the action of taking pictures. We propose a learning-based approach to this new tap-and-shoot scenario of interactive segmentation. The experimental results on various datasets show that, by training a deep convolutional network to integrate the selection and focus/defocus cues, our method can achieve higher segmentation accuracy in comparison with existing interactive segmentation methods.

Interactive segmentation of texture image based on active contour model with local inverse difference moment feature

Texture image segmentation is a challenging problem in image processing field due to wide variability of characterizing textures and a lack of proper contour information. In this paper, an effective presentation is found with which different textures can be represented with some corresponding distributions generated by feature values of local inverse difference moment (LIDM). By the analysis of local statistical information in gray level co-occurrence matrix (GLCM), we found that similar textures can be characterized with similar distributions. In this way, an interactive segmentation method is presented to achieve the segmentation of texture image based on GLCM with an optimizing model. Our scheme can be narrated separately as follows, firstly, a proper Gaussian kernel is selected to discriminate two classes of textures by analyzing LIDM feature distributions, which are obtained from two different local regions marked manually in the initial texture image. Secondly, the LIDM feature map can be constructed by computing LIDM feature values of image patches with the proper Gaussian kernel, and the center of these image patches may traverse the whole image domain. Finally, the texture image segmentation is implemented based on an improved optimizing model with local binary fitting and local extremum regularizing. In order to validate the performance of our proposed method, two kinds of experiments about discriminative feature map construction and texture image segmentation are carried out to demonstrate its well performance, and more experiments on real texture images are also conducted.

Diffusive likelihood for interactive image segmentation

The performance of conventional interactive image segmentation methods is strongly affected by seed quantity and position, and it is difficult for them to maintain global data coherence due to the bias that is caused by limited interactions. Furthermore, the pixel-level relationships in these methods are too local to capture long-range connectivity cues, which often causes them to obtain under-segmented results. To solve these problems, this paper proposes an interactive segmentation method that is based on likelihood diffusion and perceptual learning. The diffusive likelihood strategy is proposed for accurately estimating the prior label probability from limited user inputs. Superpixel-level grouping cues are utilized to enforce continuity during the segmentation process. The geometrical adjacency and long-range grouping cues are fused in the proposed framework to ensure that the segmentation results maintain proximity and continuity. The final results can be obtained by applying a joint optimization technique to solve a pair of sub-module functions. Experiments on the Berkeley segmentation data set and the Microsoft GrabCut database demonstrate that the proposed method outperforms state-of-the-art methods.

ТЕМАТИЧНЕ СЕГМЕНТУВАННЯ ЗАШУМЛЕНОГО ОПТИКО-ЕЛЕКТРОННОГО ЗОБРАЖЕННЯ РОЙОВИМ МЕТОДОМ

Проведено стислий відомих автоматичних та інтерактивних методів сегментування зображень, їхпереваги та недоліки. Встановлено, що для сегментування оптико-електронного зображення, що отриманоз бортової системи спостереження, в умовах впливу шумів доцільно використовувати інтерактивні методи сегментування, взагалі та удосконалений еволюційний метод сегментування оптико-електронного зображення, що заснований на інтегруванні мурашиного та ройового методів, зокрема. Проведено сегментування оптико-електронного зображення в умовах впливу адитивного білого гаусового шуму при різних значеннях середньоквадратичного відхилення та візуальна оцінка якості сегментування. Проведено порівнянняудосконаленого еволюційного методу, що заснований на інтегруванні мурашиного та ройового методів звідомим методом Канні в умовах впливу адитивного білого гаусового шуму. Встановлено, що в умовах впливу адитивного білого гаусового шуму удосконалений еволюційний метод сегментування, що заснований наінтегруванні мурашиного та ройового методів, забезпечує виграш у значенні інформаційного показника –відстані Кульбака-Лейбнера від 5% до 15%.

Structured sampling and fast reconstruction of smooth graph signals

This work concerns sampling of smooth signals on arbitrary graphs. We first study a structured sampling strategy for such smooth graph signals that consists of a random selection of few pre-defined groups of nodes. The number of groups to sample to stably embed the set of k-bandlimited signals is driven by a quantity called the group graph cumulative coherence. For some optimised sampling distributions, we show that sampling O(k log(k)) groups is always sufficient to stably embed the set of k-bandlimited signals but that this number can be smaller – down to O(log(k)) – depending on the structure of the groups of nodes. Fast methods to approximate these sampling distributions are detailed. Second, we consider k-bandlimited signals that are nearly piecewise constant over pre-defined groups of nodes. We show that it is possible to speed up the reconstruction of such signals by reducing drastically the dimension of the vectors to reconstruct. When combined with the proposed structured sampling procedure, we prove that the method provides stable and accurate reconstruction of the original signal. Finally, we present numerical experiments that illustrate our theoretical results and, as an example, show how to combine these methods for interactive object segmentation in an image using superpixels.

Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning.

Convolutional neural networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they have not demonstrated sufficiently accurate and robust results for clinical use. In addition, they are limited by the lack of image-specific adaptation and the lack of generalizability to previously unseen object classes (a.k.a. zero-shot learning). To address these problems, we propose a novel deep learning-based interactive segmentation framework by incorporating CNNs into a bounding box and scribble-based segmentation pipeline. We propose image-specific fine tuning to make a CNN model adaptive to a specific test image, which can be either unsupervised (without additional user interactions) or supervised (with additional scribbles). We also propose a weighted loss function considering network and interaction-based uncertainty for the fine tuning. We applied this framework to two applications: 2-D segmentation of multiple organs from fetal magnetic resonance (MR) slices, where only two types of these organs were annotated for training and 3-D segmentation of brain tumor core (excluding edema) and whole brain tumor (including edema) from different MR sequences, where only the tumor core in one MR sequence was annotated for training. Experimental results show that: 1) our model is more robust to segment previously unseen objects than state-of-the-art CNNs; 2) image-specific fine tuning with the proposed weighted loss function significantly improves segmentation accuracy; and 3) our method leads to accurate results with fewer user interactions and less user time than traditional interactive segmentation methods.

Interactive segmentation: a scalable superpixel-based method

This paper addresses the problem of interactive multiclass segmentation of images. We propose a fast and efficient new interactive segmentation method called superpixel α fusion (SαF). From a few strokes drawn by a user over an image, this method extracts relevant semantic objects. To get a fast calculation and an accurate segmentation, SαF uses superpixel oversegmentation and support vector machine classification. We compare SαF with competing algorithms by evaluating its performances on reference benchmarks. We also suggest four new datasets to evaluate the scalability of interactive segmentation methods, using images from some thousand to several million pixels. We conclude with two applications of SαF.

Graph Cutting Tumor Images

A new proposed method of fully automatic processing frameworks is based on graph-cut active contour algorithms. This paper addresses the problem of segmenting a liver and tumor regions from the abdominal CT images. A predicate is defined for measuring the evidence for a boundary between two regions using a Graph-based representation of the image. The algorithm is applied to image segmentation using two different kinds of local neighborhoods in constructing the graph. Liver and Hepatic Tumor Segmentation can be automatically processed by the graph-cut based method. This system has concentrated on finding a fast and interactive segmentation method for liver and tumor segmentation. In preprocessing stage, the CT image process with mean shift filter and statistical thresholding method for reducing processing area with improving detections rate. Second stage is liver segmentation; the liver region has been segmented using the algorithm of the proposed method. The next stage tumor segmentation also followed the same steps. Finally the liver and tumor regions are separately segmented from the computer tomography image.

Interactive image segmentation with a regression based ensemble learning paradigm

To achieve fine segmentation of complex natural images, people often resort to an interactive segmentation paradigm, since fully automatic methods often fail to obtain a result consistent with the ground truth. However, when the foreground and background share some similar areas in color, the fine segmentation result of conventional interactive methods usually relies on the increase of manual labels. This paper presents a novel interactive image segmentation method via a regression-based ensemble model with semi-supervised learning. The task is formulated as a non-linear problem integrating two complementary spline regressors and strengthening the robustness of each regressor via semi-supervised learning. First, two spline regressors with a complementary nature are constructed based on multivariate adaptive regression splines (MARS) and smooth thin plate spline regression (TPSR). Then, a regressor boosting method based on a clustering hypothesis and semi-supervised learning is proposed to assist the training of MARS and TPSR by using the region segmentation information contained in unlabeled pixels. Next, a support vector regression (SVR) based decision fusion model is adopted to integrate the results of MARS and TPSR. Finally, the GraphCut is introduced and combined with the SVR ensemble results to achieve image segmentation. Extensive experimental results on benchmark datasets of BSDS500 and Pascal VOC have demonstrated the effectiveness of our method, and the comparison with experiment results has validated that the proposed method is comparable with the state-of-the-art methods for interactive natural image segmentation.

GrabCut image segmentation algorithm based on structure tensor

This paper attempts to present an interactive color natural images segmentation method. This method extracts the feature of images by using the nonlinear compact structure tensor (NCST) and then uses GrabCut method to obtain the segmentation. This method not only realizes the non-parametric fusion of texture information and color information, but also improves the efficiency of the calculation. Then, the improved GrabCut algorithm is used to evaluate the foreground target segmentation. In order to calculate the simplicity and efficiency, this paper also extends the Gaussian mixture model (GMM) constructed base on the GrabCut to the tensor space, and uses the Kullback-Leibler (KL) divergence instead of the usual Riemannian geometry. Lastly, an iteration convergence criterion is proposed to reduce the time of the iteration of GrabCut algorithm dramatically with satisfied segmentation accuracy. After conducting a large number of experiments on synthetic texture images and natural images, the results demonstrate that this method has a more accurate segmentation effect.

Reconstruction of three-dimensional grain structure in polycrystalline iron via an interactive segmentation method

Using a total of 297 segmented sections, we reconstructed the three-dimensional (3D) structure of pure iron and obtained the largest dataset of 16254 3D complete grains reported to date. The mean values of equivalent sphere radius and face number of pure iron were observed to be consistent with those of Monte Carlo simulated grains, phase-field simulated grains, Ti-alloy grains, and Ni-based super alloy grains. In this work, by finding a balance between automatic methods and manual refinement, we developed an interactive segmentation method to segment serial sections accurately in the reconstruction of the 3D microstructure; this approach can save time as well as substantially eliminate errors. The segmentation process comprises four operations: image preprocessing, breakpoint detection based on mathematical morphology analysis, optimized automatic connection of the breakpoints, and manual refinement by artificial evaluation.

Robust interactive image segmentation using structure-aware labeling

Interactive image segmentation has remained an active research topic in image processing and graphics, since the user intention can be incorporated to enhance the performance. It can be employed to mobile devices which now allow user interaction as an input, enabling various applications. Most interactive segmentation methods assume that the initial labels are correctly and carefully assigned to some parts of regions to segment. Inaccurate labels, such as foreground labels in background regions for example, lead to incorrect segments, even by a small number of inaccurate labels, which is not appropriate for practical usage such as mobile application. In this paper, we present an interactive segmentation method that is robust to inaccurate initial labels (scribbles). To address this problem, we propose a structure-aware labeling method using occurrence and co-occurrence probability (OCP) of color values for each initial label in a unified framework. Occurrence probability captures a global distribution of all color values within each label, while co-occurrence one encodes a local distribution of color values around the label. We show that nonlocal regularization together with the OCP enables robust image segmentation to inaccurately assigned labels and alleviates a small-cut problem. We analyze theoretic relations of our approach to other segmentation methods. Intensive experiments with synthetic and manual labels show that our approach outperforms the state of the art.

An Interactive Image Segmentation Method in Hand Gesture Recognition.

In order to improve the recognition rate of hand gestures a new interactive image segmentation method for hand gesture recognition is presented, and popular methods, e.g., Graph cut, Random walker, Interactive image segmentation using geodesic star convexity, are studied in this article. The Gaussian Mixture Model was employed for image modelling and the iteration of Expectation Maximum algorithm learns the parameters of Gaussian Mixture Model. We apply a Gibbs random field to the image segmentation and minimize the Gibbs Energy using Min-cut theorem to find the optimal segmentation. The segmentation result of our method is tested on an image dataset and compared with other methods by estimating the region accuracy and boundary accuracy. Finally five kinds of hand gestures in different backgrounds are tested on our experimental platform, and the sparse representation algorithm is used, proving that the segmentation of hand gesture images helps to improve the recognition accuracy.

An Improved Random Walker with Bayes Model for Volumetric Medical Image Segmentation.

Random walk (RW) method has been widely used to segment the organ in the volumetric medical image. However, it leads to a very large-scale graph due to a number of nodes equal to a voxel number and inaccurate segmentation because of the unavailability of appropriate initial seed point setting. In addition, the classical RW algorithm was designed for a user to mark a few pixels with an arbitrary number of labels, regardless of the intensity and shape information of the organ. Hence, we propose a prior knowledge-based Bayes random walk framework to segment the volumetric medical image in a slice-by-slice manner. Our strategy is to employ the previous segmented slice to obtain the shape and intensity knowledge of the target organ for the adjacent slice. According to the prior knowledge, the object/background seed points can be dynamically updated for the adjacent slice by combining the narrow band threshold (NBT) method and the organ model with a Gaussian process. Finally, a high-quality image segmentation result can be automatically achieved using Bayes RW algorithm. Comparing our method with conventional RW and state-of-the-art interactive segmentation methods, our results show an improvement in the accuracy for liver segmentation (p < 0.001).