Over-segmented Regions Research Articles

Robust Face Alignment Under Occlusion via Regional Predictive Power Estimation

Face alignment has been well studied in recent years, however, when a face alignment model is applied on facial images with heavy partial occlusion, the performance deteriorates significantly. In this paper, instead of training an occlusion-aware model with visibility annotation, we address this issue via a model adaptation scheme that uses the result of a local regression forest (RF) voting method. In the proposed scheme, the consistency of the votes of the local RF in each of several oversegmented regions is used to determine the reliability of predicting the location of the facial landmarks. The latter is what we call regional predictive power (RPP). Subsequently, we adapt a holistic voting method (cascaded pose regression based on random ferns) by putting weights on the votes of each fern according to the RPP of the regions used in the fern tests. The proposed method shows superior performance over existing face alignment models in the most challenging data sets (COFW and 300-W). Moreover, it can also estimate with high accuracy (72.4% overlap ratio) which image areas belong to the face or nonface objects, on the heavily occluded images of the COFW data set, without explicit occlusion modeling.

Weakly-supervised region annotation for understanding scene images

Scene image understanding has drawn much attention for its intriguing applications in the past years. In this paper, we propose a unified probabilistic graphical model called Topic-based Coherent Region Annotation (TCRA) for weakly-supervised scene region annotation. The multiscale over-segmented regions within a scene image are considered as the "words" of our topic model, which impose neighborhood contextual constraints on topic level through spatial MRF modeling, and incorporate an annotation reasoning mechanism for learning and inferring region labels automatically. Mean field variational inference is provided for model learning. The proposed TCRA has the following two main advantages for understanding natural scene images. First, spatial information of multiscale over-segmented regions is explicitly modeled to obtain coherent region annotations. Second, only image-level labels are needed for automatically inferring the label of every region within the scene. This is particularly helpful in reducing human burden on manually labeling pixel-level semantics in the scene understanding research. Thus, given a scene image that has no textual prior, the regions in it can be automatically labeled using the learned TCRA model. The experimental results conducted on three benchmarks consisting of the MSRCORID image dataset, the UIUC Events image dataset and the SIFT FLOW dataset show that the proposed model outperforms the recent state-of-the-art methods.

SAR Image Segmentation Based on Closeness Degree Cut and Minimum Description Length Criterion

In this paper, a new spectral clustering algorithm is proposed for data clustering and SAR image segmentation. There are two main contributions in this paper. First, a new SAR image segmentation scheme based on the closeness degree cut (CDCut) model is proposed. The closeness degree cut is an improved form and an interpretation from fuzzy mathematics of normalized cut by taking the local information of each node into consideration. The second contribution is the development of the minimum description length criterion for determining the number of clustering in SAR image segmentation. The whole process of SAR image segmentation is composed of three steps. Firstly, the watershed algorithm is used to obtain the over-segmented image, which preserves the discontinuity characteristics of the image. Secondly, a graph is formed using each over-segmented region as a node and the spectral clustering based on the closeness degree cut is applied to the graph. Finally, the minimum description length criterion which takes into account the statistical properties of the speckle noise is used to determine the clustering number. Experimental results with simulated and real-world SAR images demonstrate that the proposed method is effective for SAR image segmentation and provides comparable or better results than the classical graph cut based methods.

Online glocal transfer for automatic figure-ground segmentation.

This paper addresses the problem of automatic figure-ground segmentation, which aims at automatically segmenting out all foreground objects from background. The underlying idea of this approach is to transfer segmentation masks of globally and locally (glocally) similar exemplars into the query image. For this purpose, we propose a novel high-level image representation method named as object-oriented descriptor. Using this descriptor, a set of exemplar images glocally similar to the query image is retrieved. Then, using over-segmented regions of these retrieved exemplars, a discriminative classifier is learned on-the-fly and subsequently used to predict foreground probability for the query image. Finally, the optimal segmentation is obtained by combining the online prediction with typical energy optimization of Markov random field. The proposed approach has been extensively evaluated on three datasets, including Pascal VOC 2010, VOC 2011 segmentation challenges, and iCoseg dataset. Experiments show that the proposed approach outperforms state-of-the-art methods and has the potential to segment large-scale images containing unknown objects, which never appear in the exemplar images.

A global energy optimization framework for 2.1D sketch extraction from monocular images

The 2.1D sketch is a layered image representation, which assigns a partial depth ordering of over-segmented regions in a monocular image. This paper presents a global optimization framework for inferring the 2.1D sketch from a monocular image. Our method only uses over-segmented image regions (i.e., superpixels) as input, without any information of objects in the image, since (1) segmenting objects in images is a difficult problem on its own and (2) the objective of our proposed method is to be generic as an initial module useful for downstream high-level vision tasks. This paper formulates the inference of the 2.1D sketch using a global energy optimization framework. The proposed energy function consists of two components: (1) one is defined based on the local partial ordering relations (i.e., figure-ground) between two adjacent over-segmented regions, which captures the marginal information of the global partial depth ordering and (2) the other is defined based on the same depth layer relations among all the over-segmented regions, which groups regions of the same object to account for the over-segmentation issues. A hybrid evolution algorithm is utilized to minimize the global energy function efficiently. In experiments, we evaluated our method on a test data set containing 100 diverse real images from Berkeley segmentation data set (BSDS500) with the annotated ground truth. Experimental results show that our method can infer the 2.1D sketch with high accuracy.

Sub-scene segmentation using constraints based on Gestalt principles

In this paper, an unsupervised sub-scene segmentation method is proposed. It emphasizes on generating more integrated and semantically consistent regions instead of homogeneous but detailed over-segmented regions usually produced by conventional segmentation methods. Several properties of sub-scenes are explored such as proximity grouping, area of influence, similarity and harmony based on psychological principles. These properties are formulated into constraints that are used directly in the proposed sub-scene segmentation. A self-determined approach is conducted to get the optimal segmentation result based on the characteristics of each image in an unsupervised manner. The proposed method is evaluated over three datasets. For quantitative evaluation, the performance of the proposed method is on par with state-of-the-art unsupervised segmentation methods; for qualitative evaluation, the proposed method handles various sub-scenes well, and produces neater results. The sub-scenes segmented by the proposed method are generally consistent with natural scene categories.

Segmentation of tumor ultrasound image via region-based Ncut method

To segment the tumor region precisely is a prerequisite for ultrasound navigation and treatment. In this paper, a normalized cut method to segment tumor ultrasound image is proposed by means of simple linear iterative clustering for pre-segmentation procedure. The first step, we use simple linear iterative clustering algorithm to divide the image into a number of homogeneous over-segmented regions. Then, these regions are regarded as nodes, and a similarity matrix is constructed by comparing the histograms of each two regions. Finally, we apply the Ncut method to merging the over-segmented regions, then the image segmentation process is completed. The results show that the proposed segmentation scheme handles the strong speckle noise, low contrast, and weak edges well in ultrasound image. Our method has high segmentation precision and computation efficiency than the pixel-based Ncut method.

Real-time color image segmentation based on mean shift algorithm using an FPGA

Image segmentation is one of the most important tasks in the image processing, and mean shift algorithm is often used for color image segmentation because of its high quality. The computational cost of the mean shift algorithm, however, is high, and it is difficult to realize its real time processing on microprocessors, though many techniques for reducing the cost have been researched. In this paper, we describe an FPGA system for the image segmentation based on the mean shift algorithm. In the image segmentation based on the mean shift algorithm, the image is once over-segmented, and then the small regions are merged considering the similarity between the over-segmented regions to obtain better segmentation. In our system, the mean shift filter is accelerated using a cache memory which can access to all pixels in a w s × w s pixel window at arbitrary position. This cache memory allows us to process w s × w s pixels in parallel every clock cycle. The region merging is also accelerated by not strictly managing the list structures used for the merging. This loose management causes the redundant and out-of-date data into the list structures, but it makes the pointer dereferences unnecessary, and the overhead by those data can be hidden by pipeline processing. The performance for 768 × 512 pixel images is fast enough for real-time applications.

A Fast Image Segmentation Based on Path-based Similarity and Dual Super-pixel Sets

Path-based clustering is a recently developed clustering approach that has delivered impressive results in quite a few challenging tasks. However, its extremely high computational complexity limits the application to image segmentation. In this paper, we propose a fast path-based spectral clustering method by defining a dual region-based graphical model for similarity computation. Our method is significantly faster than path-based clustering for considering the over-segmented regions generated from the mean shift algorithm as graph nodes, whose number is much less than that of the image pixels. Besides, taking over-segmented regions as nodes may reduce the sensitivity to noise and outliers.Furthermore, the graphical model combined double segmentations in a principle manner to avoid inappropriate partition which often occurs in a single region-based graphical model. We have performed experiments under both unsupervised and semi-supervised settings, and compared our method with some other methods as well. Experimental results show that our method consistently outperforms other methods due to its great accuracy and lower computational complexity.

해양환경에서 고속 영역 병합 알고리즘을 이용한 물표 탐지 기법

In this paper, we present a method to detect an object such as ship, rock and buoy from sea IR image for the safety navigation. To this end, we do the image smoothing first and the apply watershed algorithm to segment image into subregions. Since watershed algorithm almost always produces over-segmented regions, it requires posterior merging process to get meaningful segmented regions. We propose an efficient merger algorithm that requires only two times of direct access to the pixels regardless of the number of regions. Also by analyzing IR image obtained from sea environments, we could find out that most horizontal edge come out from object regions. For the given input IR image we extract horizontal edge and eliminate isolated edges produced from background and noises by adopting morphological operator. Among the segmented regions, the regions that have horizontal edges are extracted as final results. Experimental results show the adequacy of the proposed method.

On multi-view video segmentation for object-based coding

A novel scheme for multi-view segmentation and tracking is proposed aiming to acquire perceptually consistent results for object-based coding. Firstly, a classic image segmentation technique is employed to perform initial segmentation to divide the whole image into spatially homogeneous regions. Secondly, the motion information is extracted based on frame differences and the disparity information is derived by employing a classic disparity estimation technique. Thirdly, a novel scheme is proposed to perform merging of the initial segmentation results based on both motion and disparity information to remove over-segmented regions and extract perceptually consistent semantic objects. Finally, a contour-based tracking algorithm is proposed to implement accurate and robust object tracking along both temporal and view directions. Experiments are conducted and the results demonstrate that the proposed scheme is effective and, compared with the existing technique, it can acquire more perceptually consistent results.

Make Your Picture More Attractive – An Interactive System for Image Blur

Image blurring is a very attractive feature of photography. Due to hardware constraint, it is nearly impossible for compact DC to produce such image with blurred background. In this paper, we propose a system to simulate background blurring function, which consists of two main steps. In foreground extraction step, firstly, we select meanshift based segmentation method to get over-segmented regions. Then we extract both texture and color features for each region. Furthermore, based on input stroke information, the foreground centroids are clustered and all regions are classified into foreground and background considering both feature value and coordinate value. In background blurring step, to make the simulation result more natural, we calculate the distance of each pixels in the background to the foreground based on distance transform and give different blurring effect according to the distance. The experimental results show that the system can produce stratifying result and gain relatively high subjective evaluation.

A Resegmentation Approach for Detecting Rectangular Objects in High-Resolution Imagery

Image segmentation covers techniques for splitting one image into its components as homogeneous regions. This letter presents a resegmentation approach applied to urban images. Resegmentation represents the set of adjustments from a previous segmentation in which the elements are small regions with a high degree of spectral similarity (a condition known as oversegmentation). The focus of this letter is the house roofs, which are assumed to have a rectangular shape. These regions are merged according to an objective function, which, in the technique presented here, maximizes the rectangularity. With oversegmentation, we create a graph known as a region adjacency graph (RAG) that relates border elements. The main contribution of this letter is a technique, which works with the RAG, to maximize the objective function in a relaxationlike approach that splits and merges oversegmented regions until they form a meaningful object. The results showed that the method was able to detect rectangles according to user-defined parameters, such as the maximum level of the graph depth and the minimum degree of rectangularity for objects of interest.

Liver Image Segmentation Using Improved Watershed Method

This paper introduces an improved watershed algorithm for liver image segmentation. Medical images have complicated structure and the soft tissues have deformation sometimes. To exactly conduct the following image registration or surgery navigation, the image segmentation must identify the changes quickly and accurately. Watershed algorithm has fast speed and good edge location for complex structure, but it is sensitive to noise and has the over-segmentation problem. In this paper, pre-processing and post-processing methods are proposed during watershed segmentation procedure. According to the thresholds of region area and gray difference between adjacent regions, the image noise is reduced at pre-processing stage and the over-segmented regions are merged at post-processing part. Through the experiment of two similar liver images, we can see the segmented images have clear outline and the difference of two images can be identified obviously.

Unsupervised image segmentation evaluation and refinement using a multi-scale approach

In this study, a multi-scale approach is used to improve the segmentation of a high spatial resolution (30 cm) color infrared image of a residential area. First, a series of 25 image segmentations are performed in Definiens Professional 5 using different scale parameters. The optimal image segmentation is identified using an unsupervised evaluation method of segmentation quality that takes into account global intra-segment and inter-segment heterogeneity measures (weighted variance and Moran’s I, respectively). Once the optimal segmentation is determined, under-segmented and over-segmented regions in this segmentation are identified using local heterogeneity measures (variance and Local Moran’s I). The under- and over-segmented regions are refined by (1) further segmenting under-segmented regions at finer scales, and (2) merging over-segmented regions with spectrally similar neighbors. This process leads to the creation of several segmentations consisting of segments generated at three different segmentation scales. Comparison of single- and multi-scale segmentations shows that identifying and refining under- and over-segmented regions using local statistics can improve global segmentation results.

Image segmentation via mean shift and loopy belief propagation

This paper presents a novel approach that can quickly and effectively partition images based on fully exploiting the spatially coherent property. We propose an algorithm named iterative loopy belief propagation (iLBP) to integrate the homogenous regions and prove its convergence. The image is first segmented by mean shift (MS) algorithm to form over-segmented regions that preserve the desirable edges and spatially coherent parts. The segmented regions are then represented by region adjacent graph (RAG). Motivated by k-means algorithm, the iLBP algorithm is applied to perform the minimization of the cost function to integrate the over-segmented parts to get the final segmentation result. The image clustering based on the segmented regions instead of the image pixels reduces the number of basic image entities and enhances the image segmentation quality. Comparing the segmentation result with some existing algorithms, the proposed algorithm shows a better performance based on the evaluation criteria of entropy especially on complex scene images.