Graph Cut Algorithm Research Articles

Multiple Human Identification and Cosegmentation: A Human-Oriented CRF Approach With Poselets

Localizing, identifying, and extracting humans with consistent appearance jointly from a personal photo stream is an important problem and has wide applications. The strong variations in foreground and background and irregularly occurring foreground humans make this realistic problem challenging. Inspired by advancements in object detection, scene understanding, and image cosegmentation, we explore explicit constraints to label and segment human objects rather than other nonhuman objects and “stuff.” We refer to such a problem as multiple human identification and cosegmentation (MHIC). To identify specific human subjects, we propose an efficient human instance detector by combining an extended color line model with a poselet-based human detector. Moreover, to capture high-level human shape information, a novel soft shape cue is proposed. It is initialized by the human detector, then further enhanced through a generalized geodesic distance transform, and finally refined with a joint bilateral filter. We also propose to capture the rich feature context around each pixel by using an adaptive cross-region data structure, which gives a higher discriminative power than a single pixel-based estimation. The high-level object cues from the detector and the shape are then integrated with the low-level pixel cues and midlevel contour cues into a principled conditional random field (CRF) framework, which can be efficiently solved by using fast graph cut algorithms. We evaluate our method over a newly created NTU-MHIC human dataset, which contains 351 images with manually annotated groundtruth segmentation. Both visual and quantitative results demonstrate that our method achieves state-of-the-art performance for the MHIC task.

Robust natural image segmentation by using spatially constrained multivariate mixed Student’s t-distribution and TV flow edge

In this paper, a new method of spatially constrained energy function and total variation flow edge (TVFE) model is proposed for natural image segmentation. As the segmentation process of natural image is sensitive to corrupted noise, the Markov random field (MRF) based spatially constrained information is combined with Graph cut model. Secondly, a total variation flow edge is extracted by calculating the accumulated gradient difference from a nonlinear diffusion filter image. Thirdly, to improve the segmentation performance, the spatially constrained region term and the TVFE edge term are combined together. As the optimization solution of the proposed model is NP hard problem, and then an Expectation Maximum and Multilayer graph cut (EM2GC) algorithm is presented. Lastly, testing experiments are carried out on some synthetic images and real natural scene images, which demonstrate the superiority of our proposed method, such as the effectiveness, robustness, and high accuracy.

A graph theoretic approach for computing 3D+time biventricular cardiac strain from tagged MRI data

Tagged magnetic resonance imaging (tMRI) is a well-established method for evaluating regional mechanical function of the heart. Many techniques have been developed to compute 2D or 3D cardiac deformation and strain from tMRI images. In this paper, we present a new method for measuring 3D plus time biventricular myocardial strain from tMRI data. The method is composed of two parts. First, we use a Gabor filter bank to extract tag points along tag lines. Second, each tag point is classified to one of a set of indexed reference tag lines using a point classification with graph cuts (PCGC) algorithm and a motion compensation technique. 3D biventricular deformation and strain is computed at each image time frame from the classified tag points using a previously published finite difference method. The strain computation is fully automatic after myocardial contours are defined near end-diastole and end-systole. An in-vivo dataset composed of 30 human imaging studies with a range of pathologies was used for validation. Strains computed with the PCGC method with no manual corrections were compared to strains computed from both manually placed tag points and a manually-corrected unwrapped phase method. A typical cardiac imaging study with 10 short-axis slices and 6 long-axis slices required 30 min for contouring followed by 44 min of automated processing. The results demonstrate that the proposed method can reconstruct accurate 3D plus time cardiac strain maps with minimal user intervention.

Fast background subtraction for moving cameras based on nonparametric models

In this paper, a fast background subtraction algorithm for freely moving cameras is presented. A nonparametric sample consensus model is employed as the appearance background model. The as-similar-as-possible warping technique, which obtains multiple homographies for different regions of the frame, is introduced to robustly estimate and compensate the camera motion between the consecutive frames. Unlike previous methods, our algorithm does not need any preprocess step for computing the dense optical flow or point trajectories. Instead, a superpixel-based seeded region growing scheme is proposed to extend the motion cue based on the sparse optical flow to the entire image. Then, a superpixel-based temporal coherent Markov random field optimization framework is built on the raw segmentations from the background model and the motion cue, and the final background/foreground labels are obtained using the graph-cut algorithm. Extensive experimental evaluations show that our algorithm achieves satisfactory accuracy, while being much faster than the state-of-the-art competing methods.

Graph‐based approach to scene text localisation and tracking in videos

A text localisation and tracking method is presented that finds text-regions in videos and assigns unique IDs to their trajectories. For the goal, a graph-based framework that can work with existing text detection methods is developed. To be precise, graphs are built where vertices are image-level text detection results and edges represent the correspondence scores of the vertices. From these graphs, text-region trajectories by using the graph-cut algorithm are extracted. This approach allows considering false positives and misses, as well as their patch-based tracking results at the same time, and text trajectories are reliably extracted. Finally, the results are refined by interpolating misses and filtering out false positives. The proposed method is submitted to the International Conference on Document Analysis and Recognition 2015 robust reading competition (video text localisation) and the method showed the best performance in terms of CLEAR MOT metrics and was ranked third place according to VACE metrics among the seven participating methods.

Clothes Co-Parsing Via Joint Image Segmentation and Labeling With Application to Clothing Retrieval

This paper aims at developing an integrated system for clothing co-parsing (CCP), in order to jointly parse a set of clothing images (unsegmented but annotated with tags) into semantic configurations. A novel data-driven system consisting of two phases of inference is proposed. The first phase, referred as “image cosegmentation,” iterates to extract consistent regions on images and jointly refines the regions over all images by employing the exemplar-SVM technique [1]. In the second phase (i.e., “region colabeling”), we construct a multiimage graphical model by taking the segmented regions as vertices, and incorporating several contexts of clothing configuration (e.g., item locations and mutual interactions). The joint label assignment can be solved using the efficient Graph Cuts algorithm. In addition to evaluate our framework on the Fashionista dataset [2], we construct a dataset called the SYSU-Clothes dataset consisting of 2098 high-resolution street fashion photos to demonstrate the performance of our system. We achieve 90.29%/88.23% segmentation accuracy and 65.52%/63.89% recognition rate on the Fashionista and the SYSU-Clothes datasets, respectively, which are superior compared with the previous methods. Furthermore, we apply our method on a challenging task, i.e., cross-domain clothing retrieval: given user photo depicting a clothing image, retrieving the same clothing items from online shopping stores based on the fine-grained parsing results.

Automatic Abdominal Aortic Aneurysm segmentation in MR images

Abdominal Aortic Aneurism is a disease related to a weakening in the aortic wall that can cause a break in the aorta and the death. The detection of an unusual dilatation of a section of the aorta is an indicative of this disease. However, it is difficult to diagnose because it is necessary image diagnosis using computed tomography or magnetic resonance. An automatic diagnosis system would allow to analyze abdominal magnetic resonance images and to warn doctors if any anomaly is detected. We focus our research in magnetic resonance images because of the absence of ionizing radiation. Although there are proposals to identify this disease in magnetic resonance images, they need an intervention from clinicians to be precise and some of them are computationally hard. In this paper we develop a novel approach to analyze magnetic resonance abdominal images and detect the lumen and the aortic wall. The method combines different algorithms in two stages to improve the detection and the segmentation so it can be applied to similar problems with other type of images or structures. In a first stage, we use a spatial fuzzy C-means algorithm with morphological image analysis to detect and segment the lumen; and subsequently, in a second stage, we apply a graph cut algorithm to segment the aortic wall. The obtained results in the analyzed images are pretty successful obtaining an average of 79% of overlapping between the automatic segmentation provided by our method and the aortic wall identified by a medical specialist. The main impact of the proposed method is that it works in a completely automatic way with a low computational cost, which is of great significance for any expert and intelligent system.

A Novel Approach for Serial Crime Detection with the Consideration of Class Imbalance Problem

Objective: The main objective of this research is to reduce the burden of crime investigators by identifying the series of crimes happening at different places. And also, this work aims to reduce the investigation time by grouping similar crimes happened in different places based on its behavior with the consideration of the class imbalance problem. Methods: In this research, Majority Weighted Class Oversampling (MWCS) method is introduced which overcomes the problem of class imbalance problem. It is introduced to handle class imbalance problem by identifying the hard to learn information which is named as minority class samples from the major class samples. And also in this work, the Incremental Clustering (IC) is introduced which can handle the insertion and deletion operations where the existing methodology called graph cut clustering algorithm cant handle these problems. The proposed methodologies deal with the class imbalance problems effectively and also the modification processes over the partitioned graphs are supported well than the existing researches. Results: The methods used in this work namely MWCS and IC are used to detect the series of crimes by identifying the similarity relationship exists among the crimes happened in different places. The experimental tests conducted were proves that the proposed methodology can leads to well detection of serial residential crimes than the existing methodologies. The experimental results of this work prove that the proposed methodology is improved in terms of all performance metrics called jaccard index, mantel index and the journey distance time. Conclusion: The findings demonstrate that serial residential crimes are identified by clustering them effectively using the methodologies called MWCs and IC and it has high possibility of detection of crimes than the existing methodology.

Carotid Artery Wall Segmentation in Multispectral MRI by Coupled Optimal Surface Graph Cuts.

We present a new three-dimensional coupled optimal surface graph-cut algorithm to segment the wall of the carotid artery bifurcation from Magnetic Resonance (MR) images. The method combines the search for both inner and outer borders into a single graph cut and uses cost functions that integrate information from multiple sequences. Our approach requires manual localization of only three seed points indicating the start and end points of the segmentation in the internal, external, and common carotid artery. We performed a quantitative validation using images of 57 carotid arteries. Dice overlap of 0.86 ± 0.06 for the complete vessel and 0.89 ± 0.05 for the lumen compared to manual annotation were obtained. Reproducibility tests were performed in 60 scans acquired with an interval of 15 ± 9 days, showing good agreement between baseline and follow-up segmentations with intraclass correlations of 0.96 and 0.74 for the lumen and complete vessel volumes respectively.

Superpixel-Based Segmentation for 3D Prostate MR Images.

This paper proposes a method for segmenting the prostate on magnetic resonance (MR) images. A superpixel-based 3D graph cut algorithm is proposed to obtain the prostate surface. Instead of pixels, superpixels are considered as the basic processing units to construct a 3D superpixel-based graph. The superpixels are labeled as the prostate or background by minimizing an energy function using graph cut based on the 3D superpixel-based graph. To construct the energy function, we proposed a superpixel-based shape data term, an appearance data term, and two superpixel-based smoothness terms. The proposed superpixel-based terms provide the effectiveness and robustness for the segmentation of the prostate. The segmentation result of graph cuts is used as an initialization of a 3D active contour model to overcome the drawback of the graph cut. The result of 3D active contour model is then used to update the shape model and appearance model of the graph cut. Iterations of the 3D graph cut and 3D active contour model have the ability to jump out of local minima and obtain a smooth prostate surface. On our 43 MR volumes, the proposed method yields a mean Dice ratio of 89.3 ±1.9%. On PROMISE12 test data set, our method was ranked at the second place; the mean Dice ratio and standard deviation is 87.0±3.2%. The experimental results show that the proposed method outperforms several state-of-the-art prostate MRI segmentation methods.

Distribution Matching with the Bhattacharyya Similarity: A Bound Optimization Framework.

We present efficient graph cut algorithms for three problems: (1) finding a region in an image, so that the histogram (or distribution) of an image feature within the region most closely matches a given model; (2) co-segmentation of image pairs and (3) interactive image segmentation with a user-provided bounding box. Each algorithm seeks the optimum of a global cost function based on the Bhattacharyya measure, a convenient alternative to other matching measures such as the Kullback-Leibler divergence. Our functionals are not directly amenable to graph cut optimization as they contain non-linear functions of fractional terms, which make the ensuing optimization problems challenging. We first derive a family of parametric bounds of the Bhattacharyya measure by introducing an auxiliary labeling. Then, we show that these bounds are auxiliary functions of the Bhattacharyya measure, a result which allows us to solve each problem efficiently via graph cuts. We show that the proposed optimization procedures converge within very few graph cut iterations. Comprehensive and various experiments, including quantitative and comparative evaluations over two databases, demonstrate the advantages of the proposed algorithms over related works in regard to optimality, computational load, accuracy and flexibility.

Hill Climbing Approach for Text Binarization from Videos

Objectives: To develop presents an efficient algorithm for binarization of text from videos. However, it is a challenging and difficult task in image processing, due to their complicated background and non uniform character size. Method/Analysis: This study focuses a novel approach for binarization of text region from videos based on Hill climbing Algorithm. Symmetric filtering and graph cut algorithms are used to refine the obtained clusters. Finally, an optimal clustering selection algorithm is applied to obtain the text region. Findings: The experimental results show that the proposed text binarization technique is robust in text detection with various character size and complicated background. Data sets are taken from the You Tube Video Text (YVT) harvested from YouTube. Application: This proposed binarization technique is applicable to automated licence plate recognition system, especially to identify Indian vehicles licence plates.

Learning-Based Object Identification and Segmentation Using Dual-Energy CT Images for Security.

In recent years, baggage screening at airports has included the use of dual-energy X-ray computed tomography (DECT), an advanced technology for nondestructive evaluation. The main challenge remains to reliably find and identify threat objects in the bag from DECT data. This task is particularly hard due to the wide variety of objects, the high clutter, and the presence of metal, which causes streaks and shading in the scanner images. Image noise and artifacts are generally much more severe than in medical CT and can lead to splitting of objects and inaccurate object labeling. The conventional approach performs object segmentation and material identification in two decoupled processes. Dual-energy information is typically not used for the segmentation, and object localization is not explicitly used to stabilize the material parameter estimates. We propose a novel learning-based framework for joint segmentation and identification of objects directly from volumetric DECT images, which is robust to streaks, noise and variability due to clutter. We focus on segmenting and identifying a small set of objects of interest with characteristics that are learned from training images, and consider everything else as background. We include data weighting to mitigate metal artifacts and incorporate an object boundary field to reduce object splitting. The overall formulation is posed as a multilabel discrete optimization problem and solved using an efficient graph-cut algorithm. We test the method on real data and show its potential for producing accurate labels of the objects of interest without splits in the presence of metal and clutter.

Integrating Geometrical Context for Semantic Labeling of Indoor Scenes using RGBD Images

Inexpensive structured light sensors can capture rich information from indoor scenes, and scene labeling problems provide a compelling opportunity to make use of this information. In this paper we present a novel conditional random field (CRF) model to effectively utilize depth information for semantic labeling of indoor scenes. At the core of the model, we propose a novel and efficient plane detection algorithm which is robust to erroneous depth maps. Our CRF formulation defines local, pairwise and higher order interactions between image pixels. At the local level, we propose a novel scheme to combine energies derived from appearance, depth and geometry-based cues. The proposed local energy also encodes the location of each object class by considering the approximate geometry of a scene. For the pairwise interactions, we learn a boundary measure which defines the spatial discontinuity of object classes across an image. To model higher-order interactions, the proposed energy treats smooth surfaces as cliques and encourages all the pixels on a surface to take the same label. We show that the proposed higher-order energies can be decomposed into pairwise sub-modular energies and efficient inference can be made using the graph-cuts algorithm. We follow a systematic approach which uses structured learning to fine-tune the model parameters. We rigorously test our approach on SUN3D and both versions of the NYU-Depth database. Experimental results show that our work achieves superior performance to state-of-the-art scene labeling techniques.

Ground-Based Cloud Detection Using Automatic Graph Cut

Ground-based cloud detection plays an essential role in meteorological research, and object segmentation techniques have recently been introduced to solve this issue. As a kind of object segmentation technique, interactive graph cut has emerged as a very powerful tool due to its effective segmentation ability. However, it requires users to provide labels for certain pixels as “object” or “background,” which inevitably prohibits automatic cloud detection in large-scale applications. In this letter, we focus on the issue of automatic cloud detection and propose a novel algorithm named as automatic graph cut. We treat clouds as a special kind of object and eliminate human labeling by two procedures. First, we adaptively compute the thresholds for each cloud image which automatically label some pixels as “cloud” or “clear sky” with high confidence. Then, those labeled pixels serve as hard constraint seeds for the following graph cut algorithm. The experimental results show that the proposed algorithm not only achieves better results than the state-of-the-art cloud detection algorithms but also achieves comparable results with the interactive segmentation algorithm.

Vegetation Height Estimation Near Power transmission poles Via satellite Stereo Images using 3D Depth Estimation Algorithms

Abstract. Monitoring vegetation encroachment under overhead high voltage power line is a challenging problem for electricity distribution companies. Absence of proper monitoring could result in damage to the power lines and consequently cause blackout. This will affect electric power supply to industries, businesses, and daily life. Therefore, to avoid the blackouts, it is mandatory to monitor the vegetation/trees near power transmission lines. Unfortunately, the existing approaches are more time consuming and expensive. In this paper, we have proposed a novel approach to monitor the vegetation/trees near or under the power transmission poles using satellite stereo images, which were acquired using Pleiades satellites. The 3D depth of vegetation has been measured near power transmission lines using stereo algorithms. The area of interest scanned by Pleiades satellite sensors is 100 square kilometer. Our dataset covers power transmission poles in a state called Sabah in East Malaysia, encompassing a total of 52 poles in the area of 100 km. We have compared the results of Pleiades satellite stereo images using dynamic programming and Graph-Cut algorithms, consequently comparing satellites’ imaging sensors and Depth-estimation Algorithms. Our results show that Graph-Cut Algorithm performs better than dynamic programming (DP) in terms of accuracy and speed.

SAR-based change detection using hypothesis testing and Markov random field modelling

Abstract. The objective of this study is to automatically detect changed areas caused by natural disasters from bi-temporal co-registered and calibrated TerraSAR-X data. The technique in this paper consists of two steps: Firstly, an automatic coarse detection step is applied based on a statistical hypothesis test for initializing the classification. The original analytical formula as proposed in the constant false alarm rate (CFAR) edge detector is reviewed and rewritten in a compact form of the incomplete beta function, which is a builtin routine in commercial scientific software such as MATLAB and IDL. Secondly, a post-classification step is introduced to optimize the noisy classification result in the previous step. Generally, an optimization problem can be formulated as a Markov random field (MRF) on which the quality of a classification is measured by an energy function. The optimal classification based on the MRF is related to the lowest energy value. Previous studies provide methods for the optimization problem using MRFs, such as the iterated conditional modes (ICM) algorithm. Recently, a novel algorithm was presented based on graph-cut theory. This method transforms a MRF to an equivalent graph and solves the optimization problem by a max-flow/min-cut algorithm on the graph. In this study this graph-cut algorithm is applied iteratively to improve the coarse classification. At each iteration the parameters of the energy function for the current classification are set by the logarithmic probability density function (PDF). The relevant parameters are estimated by the method of logarithmic cumulants (MoLC). Experiments are performed using two flood events in Germany and Australia in 2011 and a forest fire on La Palma in 2009 using pre- and post-event TerraSAR-X data. The results show convincing coarse classifications and considerable improvement by the graph-cut post-classification step.

Detection of fallen trees in ALS point clouds using a Normalized Cut approach trained by simulation

Downed dead wood is regarded as an important part of forest ecosystems from an ecological perspective, which drives the need for investigating its spatial distribution. Based on several studies, Airborne Laser Scanning (ALS) has proven to be a valuable remote sensing technique for obtaining such information. This paper describes a unified approach to the detection of fallen trees from ALS point clouds based on merging short segments into whole stems using the Normalized Cut algorithm. We introduce a new method of defining the segment similarity function for the clustering procedure, where the attribute weights are learned from labeled data. Based on a relationship between Normalized Cut’s similarity function and a class of regression models, we show how to learn the similarity function by training a classifier. Furthermore, we propose using an appearance-based stopping criterion for the graph cut algorithm as an alternative to the standard Normalized Cut threshold approach. We set up a virtual fallen tree generation scheme to simulate complex forest scenarios with multiple overlapping fallen stems. This simulated data is then used as a basis to learn both the similarity function and the stopping criterion for Normalized Cut. We evaluate our approach on 5 plots from the strictly protected mixed mountain forest within the Bavarian Forest National Park using reference data obtained via a manual field inventory. The experimental results show that our method is able to detect up to 90% of fallen stems in plots having 30–40% overstory cover with a correctness exceeding 80%, even in quite complex forest scenes. Moreover, the performance for feature weights trained on simulated data is competitive with the case when the weights are calculated using a grid search on the test data, which indicates that the learned similarity function and stopping criterion can generalize well on new plots.

JF-cut: a parallel graph cut approach for large-scale image and video.

Graph cut has proven to be an effective scheme to solve a wide variety of segmentation problems in vision and graphics community. The main limitation of conventional graph-cut implementations is that they can hardly handle large images or videos because of high computational complexity. Even though there are some parallelization solutions, they commonly suffer from the problems of low parallelism (on CPU) or low convergence speed (on GPU). In this paper, we present a novel graph-cut algorithm that leverages a parallelized jump flooding technique and an heuristic push-relabel scheme to enhance the graph-cut process, namely, back-and-forth relabel, convergence detection, and block-wise push-relabel. The entire process is parallelizable on GPU, and outperforms the existing GPU-based implementations in terms of global convergence, information propagation, and performance. We design an intuitive user interface for specifying interested regions in cases of occlusions when handling video sequences. Experiments on a variety of data sets, including images (up to 15 K × 10 K), videos (up to 2.5 K × 1.5 K × 50), and volumetric data, achieve high-quality results and a maximum 40-fold (139-fold) speedup over conventional GPU (CPU-)-based approaches.

시공간 영상분할을 이용한 이동 및 이동 중 정지물체 검출

본 논문에서는 이동 카메라 환경에서 이동 및 이동 중 정지물체를 검출하기 위한 방법을 제안한다. 이동 중에 일시적으로 정지한 물체는 검출 결과의 응용관점에서 볼 때 이동물체의 검출만큼이나 중요한데, 기존의 이동물체 검출 방법들은 이들을 배경과 구분하지 못하는 한계를 갖는다. 이러한 문제를 해결하기 위해 제안하는 방법에서는 이동 가능성 큐, 위치 가능성 큐, 그리고 색 분포 유사성 큐를 정의하여 이동물체 검출 및 지속적인 추적에 이용한다. 그래프 컷 알고리즘은 세 개의 큐를 결합하여 시공간 영상분할을 수행함으로써 이동 및 이동 중 정지물체를 검출한다. 제안하는 방법은 이동물체 뿐 아니라 이동 중 정지물체에 대해서도 검출이 가능함을 실험을 통해 증명하였다. This paper proposes a method for detection of objects temporally stop moving in video sequences taken by a moving camera. Even though the consequence of missed detection of those objects could be catastrophic in terms of application level requirements, not much attention has been paid in conventional approaches. In the proposed method, we introduce cues for consistent detection and tracking of objects: motion potential, position potential, and color distribution similarity. Integration of the three cues in the graph-cut algorithm makes possible to detect objects that temporally stop moving and are newly appearing. Experiment results prove that the proposed method can not only detect moving objects but also track objects stop moving.