Interactive Image Segmentation Algorithm Research Articles

Cartoon copyright recognition method based on character personality action

Aiming at the problem of cartoon piracy and plagiarism, this paper proposes a method of cartoon copyright recognition based on character personality actions. This method can be used to compare the original cartoon actions with the action characteristics of pirated or copied cartoons to identify whether there is piracy or plagiarism. Firstly, an image preprocessing scheme for character extraction is designed. GrabCut interactive image segmentation algorithm was used to obtain cartoon characters, and then binarization and morphological processing were performed on the results. Secondly, a feature extraction scheme based on character profile, moving character and character pose is designed. By extracting the perimeter and area of the character contour, the length-to-width ratio of the smallest rectangle and the inclination angle of the contour, the character contour features are obtained. The three-dimensional coordinates are established by the central point position of the cartoon character in the two-dimensional image and the change of the character's zoom in and out, and the character's motion angle characteristics are calculated. By skeletonizing a character to obtain the pose characteristics, and using deburring operation to remove redundant branches, then extract the skeleton joint angle information. Finally, feature fusion is performed on the extracted features. The experimental results show that the proposed method breaks the limitation of the conventional single feature based recognition, and can better extract the character features including contour feature, motion feature and pose feature through multi-feature based extraction, so as to protect the cartoon copyright.

Parallel Processing of Image Databases for Accelerated Morphological Analysis

Nuclear forensics relies on different signatures to identify the source of nuclear material. Such signatures include crystalline structure, chemical composition, and particle morphology. One way to quantify morphology in electron microscope imagery is through image segmentation, where pixels are assigned to several partitions (or groups) that correspond to particles, grains, and other objects of interest within the image. Once pixels are assigned to segments, it is relatively straightforward to quantify other quantities of interest, such as grain size, circularity, etc. However, the range and diversity of microscope images make it difficult to obtain an accurate segmentation automatically. The accuracy of segmentation can be improved through supervised learning, but this requires many images to be manually segmented. Another way to improve the accuracy is to use interactive segmentation. Interactive segmentation requires a human to provide image-specific user input to improve performance. However, the amount of user input (effort) is generally far less than is required for supervised learning. In this paper, we investigate several parallelization strategies to automatically explore the user input parameter space of interactive segmentation algorithms across a large number of images. Specifically, we investigate four different parallelization mechanisms in a high-performance computing (HPC) environment and use the Amdahl fraction to evaluate efficiency on multiple processor cores across multiple nodes. Ultimately, the parallelization strategy that was most efficient utilized the message passing interface integrated with the segmentation and quantification code. This strategy had an Amdahl fraction of 0.985 and a performance of about 0.251 s/image. These results indicate that the parameter space of interactive segmentation algorithms can be efficiently explored using HPC. This opens the door to future work where user input is reduced and where interactive image segmentation algorithms are automatically applied to large image sets.

A Novel Interactive Image Segmentation Algorithm Based on Maximization of Submodular Function

In this paper, an efficient interactive image segmentation method based on maximization of submodular function under user’s scribble constraint is presented. The problem of interactive image segmentation is formulated as the maximum entropy rate under user’s constraints. The objective function is submodular, and we solve the constrained submodular function maximization by incorporating a new data structure and some aggregating rules into the greedy algorithm. The main steps of our algorithm are as follows. First, the pixels scribbled by the user are clustered separately as target foreground and background clusters. Second, in the process of greedy algorithm, unscribbled pixels are aggregated to the corresponding target cluster according to the proposed aggregating rules. Finally, the segmentation result is presented by the two target clusters. The experiments and comparisons on three standard benchmarks show that our method has good performance. Our method is straightforward and efficient, and the time complexity of our method is between linear and polynomial. Furthermore, we analyze the influence of different scribbles, and propose some optimal scribble strategies.

Design and Implementation of Print Advertisement Based on Computer Image Interactive Virtualization

This study aimed to further enhance and enrich users’ visual experience, provide users with a more infectious image sense, and further promote the development of multidimensional, virtualized, and interactive graphic advertising design applications. This study presents a design idea of print advertisement based on computer image interactive virtual technology. Based on the in-depth analysis of the multidimensional expression of plane image interactive virtualization, this idea carries out interactive virtual design with the help of interactive virtual image segmentation algorithm and plane advertising image element optimization technology. Through the test of image interactive edge extraction function on MATLAB platform, the simulation results show that the edge width of single side extraction results with different gray levels of the same width and different widths of the same gray level are 9 pixels, the gray values are 5a0, 110, and 200, respectively, and the edge width corresponds to 3, 9, and 15 pixels, respectively. The results show that the system can effectively enhance the image definition and obtain better print advertising design effect.

ScribbleNet: Efficient interactive annotation of urban city scenes for semantic segmentation

Annotation is a crucial first step in the semantic segmentation of urban images that facilitates the development of autonomous navigation systems. However, annotating complex urban images is time-consuming and challenging. It requires significant human effort making it expensive and error-prone. To reduce human effort during annotation, multiple images need to be annotated in a short time-span. In this paper, we introduce ScribbleNet, an interactive image segmentation algorithm to address this issue. Our approach provides users with a pre-segmented image that iteratively improves the segmentation using scribble as an annotation input. This method is based on conditional inference and exploits the learnt correlations in a deep neural network (DNN). ScribbleNet can: (1) work with urban city scenes captured in unseen environments, (2) annotate new classes not present in the training set, and (3) correct several labels at once. We compare this method with other interactive segmentation approaches on multiple datasets such as CityScapes, BDD, Mapillary Vistas, KITTI, and IDD. ScribbleNet reduces the annotation time of an image by up to 14.7 × over manual annotation and up to 5.4× over the current approaches. The algorithm is integrated into the publicly available LabelMe image annotation tool and will be released as an open-source software.

Effects of Different Superpixel Algorithms on Interactive Segmentations

Semi-automated segmentation or more commonly known as interactive image segmentation is an algorithm that extracts a region of interest (ROI) from an image based on the input information from the user. The said algorithm will be repetitively fed with such input information until required region of interest is successfully segmented. To accelerate this segmentation procedure as well as enhancing the result, pre-processing steps can be applied. The application of superpixel is an example of such pre-processing step. Superpixel can be defined as a collection of pixels that share common features such as texture and colours. Though employed as pre-processing step in many interactive segmentation algorithms, to date, no study has been conducted to assess the effects of such incorporations on the segmentation algorithms. Thus, this study aims to address this issue. In this study, five different types of superpixels ranging from watershed, density, graph, clustering and energy optimization categories are evaluated. The superpixels generated by these five algorithms will be used on two interactive image segmentation algorithms: i) Maximal Similarity based Region Merging (MSRM) and ii) Graph-Based Manifold Ranking (GBMR) with single and multiple strokes on various images from the Berkeley image dataset. The result of testing had shown that MSRM achieved better result compared to GBMR in both single and multiple input strokes using SEEDS superpixel algorithm. This study summary concluded that at different superpixel algorithms produced different results and that it is not possible to single out one particular superpixel algorithm that can work well for all the interactive segmentation algorithms. As such, the key to achieving a decent segmentation result lies in choosing the right superpixel algorithms for a given interactive segmentation algorithm.

Geodesic Models With Convexity Shape Prior.

The minimal geodesic models established upon the eikonal equation framework are capable of finding suitable solutions in various image segmentation scenarios. Existing geodesic-based segmentation approaches usually exploit image features in conjunction with geometric regularization terms, such as euclidean curve length or curvature-penalized length, for computing geodesic curves. In this paper, we take into account a more complicated problem: finding curvature-penalized geodesic paths with a convexity shape prior. We establish new geodesic models relying on the strategy of orientation-lifting, by which a planar curve can be mapped to an high-dimensional orientation-dependent space. The convexity shape prior serves as a constraint for the construction of local geodesic metrics encoding a particular curvature constraint. Then the geodesic distances and the corresponding closed geodesic paths in the orientation-lifted space can be efficiently computed through state-of-the-art Hamiltonian fast marching method. In addition, we apply the proposed geodesic models to the active contours, leading to efficient interactive image segmentation algorithms that preserve the advantages of convexity shape prior and curvature penalization.

FAST INTERACTIVE REGIONAL PATTERN MERGING FOR GENERIC TISSUE SEGMENTATION IN HISTOPATHOLOGY IMAGES

The image segmentation of histopathological tissue images has always been a challenge due to the overlapping of tissue color distributions, the complexity of extracellular texture and the large image size. In this paper, we introduce a new region-merging algorithm, namely, the Regional Pattern Merging (RPM) for interactive color image segmentation and annotation, by efficiently retrieving and applying the user’s prior knowledge of stroke-based interaction. Low-level color/texture features of each region are used to compose a regional pattern adapted to differentiating a foreground object from the background scene. This iterative region-merging is based on a modified Region Adjacency Graph (RAG) model built from initial segmented results of the mean shift to speed up the merging process. The foreground region of interest (ROI) is segmented by the reduction of the background region and discrimination of uncertain regions. We then compare our method against state-of-the-art interactive image segmentation algorithms in both natural images and histological images. Taking into account the homogeneity of both color and texture, the resulting semi-supervised classification and interactive segmentation capture histological structures more completely than other intensity or color-based methods. Experimental results show that the merging of the RAG model runs in a linear time according to the number of graph edges, which is essentially faster than both traditional graph-based and region-based methods.

Adaptive fusion with multi-scale features for interactive image segmentation

Multi-scale features are usually utilized to improve the performance of interactive image segmentation, however, they have varying leverages over the result of segmentation, for example, thinner segmentation results could be achieved by pixel-level features, but sensitive to image noise, and superpixel-level features could provide the semantic perception of the object, but easily lead to over-segmentations. Therefore, we propose an interactive image segmentation algorithm by adaptive fusion with multi-scale features (AFMSF). It intends on combining the multi-scale information adaptively for the segmentation via learning the influence coefficients of multi-scale features. First, multi-scale superpixel layers are generated by controlling the size of superpixels. Based on features of this multi-scale information, the similarity matrices and label priors with pixel-superpixel levels are then obtained. A fusion with diffusion strategy is designed to build the energy function by combining these multi-cues. Finally, the influence coefficient of each scale and the labeling are updated with each other until convergence. The algorithm we proposed is robust to diverse circumstances of objects, the experimental results on public interactive image segmentation datasets Graz, LHI, and MSRC validate the superior performance of the proposed method.

Global Manifold Learning for Interactive Image Segmentation

This paper presents an interactive image segmen-tation algorithm, in which the segmentation problem is formulated as a global manifold learning process. Based on the principle that the label of each element depends on the influence of all the elements in the image, we extend the conventional local neighborhood or the long range regional relationships to the global relationships over the whole image. A probabilistic framework is established to measure the global effect of each element on all other elements. Based on two different manifold learning styles, the semi-global manifold learning (SGML) and the fully-global manifold learning (FGML) algorithms are proposed to capture the global geometry structure of the data manifold. SGML learns the intrinsic effects of each element separately, equivalent to a matrix diffusion process on an affinity graph. FGML learns the intrinsic effects of all elements together, equivalent to a label pair diffusion process on a higher-order tensor product graph. The global manifold learning helps to overcome the low contrast, weak boundary and texture problems. Extensive experiments on three public interactive segmentation datasets demonstrate the superior performance of the proposed algorithms both in accuracy and efficiency.

Feasibility of Automatic Seed Generation Applied to Cardiac MRI Image Analysis

We present a method of using interactive image segmentation algorithms to reduce specific image segmentation problems to the task of finding small sets of pixels identifying the regions of interest. To this end, we empirically show the feasibility of automatically generating seeds for GrowCut, a popular interactive image segmentation algorithm. The principal contribution of our paper is the proposal of a method for automating the seed generation method for the task of whole-heart segmentation of MRI scans, which achieves competitive unsupervised results (0.76 Dice on the MMWHS dataset). Moreover, we show that segmentation performance is robust to seeds with imperfect precision, suggesting that GrowCut-like algorithms can be applied to medical imaging tasks with little modeling effort.

SwipeCut: Interactive Segmentation via Seed Grouping

Interactive image segmentation algorithms rely on the user to provide annotations as the guidance. When the task of interactive segmentation is performed on a small touchscreen device, the requirement of providing precise annotations could be cumbersome to the user. We design a new interaction mechanism that actively queries seeds for guiding the user to label. Our method enforces sparsity and diversity criteria on the selection of query seeds, and at each round of interaction, the user is only presented with a small number of informative query seeds that are far apart from each other. Therefore, the user merely has to swipe through on the ROI-relevant query seeds for checking which ones of the query seeds are inside the region of interest (ROI). This kind of interaction should be easy since those gestures are commonly used on a touchscreen. As a result, we are able to derive a user-friendly interaction mechanism for annotation on small touchscreen devices. The performance of our algorithm is evaluated on six publicly available datasets. The evaluation results show that our algorithm achieves high segmentation accuracy, with short computational time and less user feedback.

RETRACTED: Virtual scene dynamic interactive image segmentation technology based on OpenGL module

A virtual scene dynamic interactive image segmentation technology based on OpenGL sky module is proposed to improve the performance of image dynamic interactive recognition algorithm and the recognition effect of image segmentation. First, an automatic dynamic interactive image segmentation algorithm is proposed, a probability segmentation model of Gauss mixture fracture based on multi-scale crack enhancement filter is established, and expectation maximization algorithm is used to determine the parameters of GMM (Gaussian Mixture Model) and the underlying segmentation; second, the drawing function of OpenGL is used to draw the graph, and the drawing method needs to be calculated. That is to say, a mathematical model is established, which converts the graph into the calculation of vertex coordinates for drawing. It uses an extended cost function to guide dynamic interactive image segmentation using a set of cracks from the population to be segmented. Finally, the validity of the algorithm is verified by experimental simulation.

Rapid extraction of the hottest or coldest regions of medical thermographic images.

Early detection of breast tumors, feet pre-ulcers diagnosing in diabetic patients, and identifying the location of pain in patients are essential to physicians. Hot or cold regions in medical thermographic images have potential to be suspicious. Hence extracting the hottest or coldest regions in the body thermographic images is an important task. Lazy snapping is an interactive image cutout algorithm that can be applied to extract the hottest or coldest regions in the body thermographic images quickly with easy detailed adjustment. The most important advantage of this technique is that it can provide the results for physicians in real time readily. In other words, it is a good interactive image segmentation algorithm since it has two basic characteristics: (1) the algorithm produces intuitive segmentation that reflects the user intent with given a certain user input and (2) the algorithm is efficient enough to provide instant visual feedback. Comparing to other methods used by the authors for segmentation of breast thermograms such as K-means, fuzzy c-means, level set, and mean shift algorithms, lazy snapping was more user-friendly and could provide instant visual feedback. In this study, twelve test cases were presented and by applying lazy snapping algorithm, the hottest or coldest regions were extracted from the corresponding body thermographic images. The time taken to see the results varied from 7 to 30s for these twelve cases. It was concluded that lazy snapping was much faster than other methods applied by the authors such as K-means, fuzzy c-means, level set, and mean shift algorithms for segmentation. Graphical abstract Time taken to implement lazy snapping algorithm to extract suspicious regions in different presented thermograms (in seconds). In this study, ten test cases are presented that by applying lazy snapping algorithm, the hottest or coldest regions were extracted from the corresponding body thermographic images. The time taken to see the results varied from 7 to 30s for the ten cases. It concludes lazy snapping is much faster than other methods applied by the authors.

An interactive image segmentation method for lithological boundary detection: A rapid mapping tool for geologists

Large volumes of images are collected by geoscientists using remote sensing platforms. Manual analysis of these images is a time consuming task and there is a need for fast and robust image interpretation tools. In particular the reliable mapping of lithological boundaries is a critical step for geological interpretation. In this contribution we developed an interactive image segmentation algorithm that harnesses the geologist's input and exploits automated image analysis to provide a practical tool for lithology boundary detection, using photographic images of rock surfaces.In the proposed method, the user is expected to draw rough markings to indicate the locations of different geological units in the image. Image segmentation is performed by segmenting regions based on their homogeneity in colour. This results in a high density of segmented regions which are then iteratively merged based on the colour of different geological units and the user input. Finally, a post-processing step allows the user to edit the boundaries.An experiment was conducted using photographic rock surface images collected by a UAV and a handheld digital camera. The proposed technique was applied to detect lithology boundaries. It was found that the proposed method reduced the interpretation time by a factor of four relative to manual segmentation, while achieving more than 96% similarity in boundary detection. As a result the proposed method has the potential to provide practical support for interpreting large volume of complex geological images.

A rapid multi-source shortest path algorithm for interactive image segmentation

This paper addresses the problem of performing multiple objects, interactive image segmentation. Given a small number of pixels (seeds) with predefined labels, we can quickly and accurately determine the closest seed from each unlabeled pixel. By assigning each pixel to the label same as its closest seed, a rapid image segmentation result can be obtained. Since the shortest distance is considered in this paper which directs our attention of segmentation into path planning problem, Dijkstra occurs to mind. Modifying the classical single-source algorithm, a simple yet rapid multi-source Dijkstra (MSD) algorithm is put forward. From both theoretical and experimental aspects, the proposed algorithm performs quite well in resisting noise, and preserving the objects details. Moreover, under the situation of multiple sources, instead of performing Dijkstra several times to obtain the distance from each pixel to each seed and choose the closest seed, the proposed multi-source image segmentation algorithm could determine the closest seed by running Dijkstra only once. Its efficiency, which will not be affected by the number of initial seed settings, maintains the same as the Dijkstra.

A semantic model for general purpose content-based image retrieval systems

The ‘semantic gap’ is the main challenge in content-based image retrieval. To overcome this challenge, we propose a semantic-based model to retrieve images efficiently with considering user-interested concepts. In this model, an interactive image segmentation algorithm is carried out on the query image to extract the user-interested regions. To recognize the image objects from regions, a neural network classifier is used in this model. In order to have a general-purpose system, no priori assumptions should be made regarding the nature of images in extracting features. So a large number of features should be extracted from all aspect of the image. The high dimensional feature space, not only increases the complexity and required memory, but also may reduce the efficiency and accuracy. Hence, the ant colony optimization algorithm is employed to eliminate irrelevant and redundant features. To find the most similar images to the query image, the similarity between images is measured based on their semantic objects which are defined according to a predefined ontology.

Interactive color image segmentation via iterative evidential labeling

We develop an interactive color image segmentation method in this paper. This method makes use of the conception of Markov random fields (MRFs) and D–S evidence theory to obtain segmentation results by considering both likelihood information and priori information under Bayesian framework. The method first uses expectation maximization (EM) algorithm to estimate the parameter of the user input regions, and the Bayesian information criterion (BIC) is used for model selection. Then the beliefs of each pixel are assigned by a predefined scheme. The result is obtained by iteratively fusion of the pixel likelihood information and the pixel contextual information until convergence. The method is initially designed for two-label segmentation, however it can be easily generalized to multi-label segmentation. Experimental results show that the proposed method is comparable to other prevalent interactive image segmentation algorithms in most cases of two-label segmentation task, both qualitatively and quantitatively.

Improved Selective Segmentation Model Using One Level-Set

Variational segmentation models have proven to be extremely efficient for segmenting and tracking boundaries and in most cases all of the boundaries in an image. Such models are for global segmentation. For a large class of image segmentation tasks where only one object is required to be extracted automatically, global models cannot deliver the solution and we need selective segmentation techniques. In this paper, we propose a novel, variational and single level-set function for the selective segmentation task. The model is much faster to implement than the previously dual level set model by Rada-Chen [29] by having the same efficiency and reliability. In comparing with interactive image segmentation algorithm of Nguyen-Cai-Zhang-Zheng method [2], our model shows some improvement in some cases. Several new ideas are incorporated in this new work: i) the distance function is only needed optionally and its inclusion does not affect the result; ii) an adaptive parameter is introduced in the edge detection function; iii) a new area-based fitting term is added to enhance the model's reliability (different from the idea of minimizing the area of the object). We develop an additive operator splitting method for solving the resulting Euler-Lagrange equation. Test results show that the new model finds the desired local boundaries successfully in various challenging cases and indeed it is not much dependent on the prior information of markers or the distance function based on them. More importantly, the new model gives an overall improvement over the previous models and can be recommended for selective segmentation.

Interactive segmentation algorithm optimized by multi-threshold with application in medical images

Interactive image segmentation methods usually ask users to mark much more initial seeds or more than one interaction when they are used for medical image segmentation with fuzzy boundaries.This paper presented an optimized interactive image segmentation algorithm with multi-threshold technology.Based on GorwCut algorithm put forward by Vladimir,the optimized algorithm introduced image gray histogram with more than one threshold values to generate initial seeds template automatically and then used improved cellular automaton iterative algorithm to realize image segmentation.The algorithm simplified the user interactive operations and improved the segmentation accuracy.In applications,the algorithm was used to test on 100 plaque and liver image segmentations respectively,of which the results show that the optimized algorithm is of good performance.