Markov Random Field Framework Research Articles

Three‐Dimensional Multiphase Segmentation of X‐Ray CT Data of Porous Materials Using a Bayesian Markov Random Field Framework

Advancements in noninvasive imaging methods such as X‐ray computed tomography (CT) have led to a recent surge of applications in porous media research with objectives ranging from theoretical aspects of pore‐scale fluid and interfacial dynamics to practical applications such as enhanced oil recovery and advanced contaminant remediation. While substantial efforts and resources have been devoted to advance CT technology, microscale analysis, and fluid dynamics simulations, the development of efficient and stable three‐dimensional multiphase image segmentation methods applicable to large data sets is lacking. To eliminate the need for wet–dry or dual‐energy scans, image alignment, and subtraction analysis, commonly applied in X‐ray micro‐CT, a segmentation method based on a Bayesian Markov random field (MRF) framework amenable to true three‐dimensional multiphase processing was developed and evaluated. Furthermore, several heuristic and deterministic combinatorial optimization schemes required to solve the labeling problem of the MRF image model were implemented and tested for computational efficiency and their impact on segmentation results. Test results for three grayscale data sets consisting of dry glass beads, partially saturated glass beads, and partially saturated crushed tuff obtained with synchrotron X‐ray micro‐CT demonstrate great potential of the MRF image model for three‐dimensional multiphase segmentation. While our results are promising and the developed algorithm is stable and computationally more efficient than other commonly applied porous media segmentation models, further potential improvements exist for fully automated operation.

Similarity-based appearance-prior for fitting a subdivision mesh in gene expression images.

Automated segmentation of multi-part anatomical objects in images is a challenging task. In this paper, we propose a similarity-based appearance-prior to fit a compartmental geometric atlas of the mouse brain in gene expression images. A subdivision mesh which is used to model the geometry is deformed using a Markov random field (MRF) framework. The proposed appearance-prior is computed as a function of the similarity between local patches at corresponding atlas locations from two images. In addition, we introduce a similarity-saliency score to select the mesh points that are relevant for the computation of the proposed prior. Our method significantly improves the accuracy of the atlas fitting, especially in the regions that are influenced by the selected similarity-salient points, and outperforms the previous subdivision mesh fitting methods for gene expression images.

Model-based adaptive spatial sampling for occurrence map construction

In many environmental management problems, the construction of occurrence maps of species of interest is a prerequisite to their effective management. However, the construction of occurrence maps is a challenging problem because observations are often costly to obtain (thus incomplete) and noisy (thus imperfect). It is therefore critical to develop tools for designing efficient spatial sampling strategies and for addressing data uncertainty. Adaptive sampling strategies are known to be more efficient than non-adaptive strategies. Here, we develop a model-based adaptive spatial sampling method for the construction of occurrence maps. We apply the method to estimate the occurrence of one of the world's worst invasive species, the red imported fire ant, in and around the city of Brisbane, Australia. Our contribution is threefold: (i) a model of uncertainty about invasion maps using the classical image analysis probabilistic framework of Hidden Markov Random Fields (HMRF), (ii) an original exact method for optimal spatial sampling with HMRF and approximate solution algorithms for this problem, both in the static and adaptive sampling cases, (iii) an empirical evaluation of these methods on simulated problems inspired by the fire ants case study. Our analysis demonstrates that the adaptive strategy can lead to substantial improvement in occurrence mapping.

Nonparametric Scene Parsing via Label Transfer

While there has been a lot of recent work on object recognition and image understanding, the focus has been on carefully establishing mathematical models for images, scenes, and objects. In this paper, we propose a novel, nonparametric approach for object recognition and scene parsing using a new technology we name label transfer. For an input image, our system first retrieves its nearest neighbors from a large database containing fully annotated images. Then, the system establishes dense correspondences between the input image and each of the nearest neighbors using the dense SIFT flow algorithm [28], which aligns two images based on local image structures. Finally, based on the dense scene correspondences obtained from SIFT flow, our system warps the existing annotations and integrates multiple cues in a Markov random field framework to segment and recognize the query image. Promising experimental results have been achieved by our nonparametric scene parsing system on challenging databases. Compared to existing object recognition approaches that require training classifiers or appearance models for each object category, our system is easy to implement, has few parameters, and embeds contextual information naturally in the retrieval/alignment procedure.

Occlusion-Aware Motion Layer Extraction Under Large Interframe Motions

Extracting motion layers from videos is an important task for video representation, analysis, and compression. For videos with large interframe motions, motion layer extraction is challenging in two respects: the estimation of large disparity motions and the awareness of large occluded regions. In this paper, we propose an effective method for motion layer extraction under large disparity motions. To robustly estimate large displacement motions, we have developed an efficient voting-based method that estimates planar homographies from sparse feature matches. To handle occlusions, we first integrate color and motion consistency into a Markov random field framework to achieve per-pixel assignment with occlusion detection. Then, we perform motion-color segmentation and an earth mover's distance-based comparison to determine motion labels for occluded pixels. Experimental results show that our proposed method achieves good performance in automatically extracting multiple moving objects under large disparity motions while maintaining a low computational cost.

REGISTRATION OF BRAIN RESECTION MRI WITH INTENSITY AND LOCATION PRIORS.

Images with missing correspondences are difficult to align using standard registration methods due to the assumption that the same features appear in both images. To address this problem in brain resection images, we have recently proposed an algorithm in which the registration process is aided by an indicator map that is simultaneously estimated to distinguish between missing and valid tissue. We now extend our method to include both intensity and location information for the missing data. We introduce a prior on the indicator map using a Markov random field (MRF) framework to incorporate map smoothness and spatial knowledge of the missing correspondences. The parameters for the indicator map prior are automatically estimated along with the transformation and indicator map. The new method improves both segmentation and registration accuracy as demonstrated using synthetic and real patient data.

Unsupervised 4D myocardium segmentation with a Markov Random Field based deformable model

A stochastic deformable model is proposed for the segmentation of the myocardium in Magnetic Resonance Imaging. The segmentation is posed as a probabilistic optimization problem in which the optimal time-dependent surface is obtained for the myocardium of the heart in a discrete space of locations built upon simple geometric assumptions. For this purpose, first, the left ventricle is detected by a set of image analysis tools gathered from the literature. Then, the segmentation solution is obtained by the Maximization of the Posterior Marginals for the myocardium location in a Markov Random Field framework which optimally integrates temporal-spatial smoothness with intensity and gradient related features in an unsupervised way by the Maximum Likelihood estimation of the parameters of the field. This scheme provides a flexible and robust segmentation method which has been able to generate results comparable to manually segmented images for some derived cardiac function parameters in a set of 43 patients affected in different degrees by an Acute Myocardial Infarction.

Abnormality detection using low-level co-occurring events

We propose in this paper a method for behavior modeling and abnormal events detection which uses low-level features. In conventional object-based approaches, objects are identified, classified, and tracked to locate those with suspicious behavior. We proceed directly with event characterization and behavior modeling using low-level features. We first learn statistics about co-occurring events in a spatio-temporal volume in order to build the normal behavior model, called the co-occurrence matrix. The notion of co-occurring events is defined using mutual information between motion labels sequences. Then, in the second phase, the co-occurrence matrix is used as a potential function in a Markov random field framework to describe, as the video streams in, the probability of observing new volumes of activity. The co-occurrence matrix is thus used for detecting moving objects whose behavior differs from the ones observed during the training phase. Interestingly, the Markov random field distribution implicitly accounts for speed, direction, as well as the average size of the objects without any higher-level intervention. Furthermore, when the spatio-temporal volume is sufficiently large, the co-occurrence distribution contains the average normal path followed by moving objects. Our method has been tested on various indoor and outdoor videos representing various challenges.

A Low-Complexity Algorithm for Static Background Estimation from Cluttered Image Sequences in Surveillance Contexts

For the purposes of foreground estimation, the true background model is unavailable in many practical circumstances and needs to be estimated from cluttered image sequences. We propose a sequential technique for static background estimation in such conditions, with low computational and memory requirements. Image sequences are analysed on a block-by-block basis. For each block location a representative set is maintained which contains distinct blocks obtained along its temporal line. The background estimation is carried out in a Markov Random Field framework, where the optimal labelling solution is computed using iterated conditional modes. The clique potentials are computed based on the combined frequency response of the candidate block and its neighbourhood. It is assumed that the most appropriate block results in the smoothest response, indirectly enforcing the spatial continuity of structures within a scene. Experiments on real-life surveillance videos demonstrate that the proposed method obtains considerably better background estimates (both qualitatively and quantitatively) than median filtering and the recently proposed "intervals of stable intensity" method. Further experiments on the Wallflower dataset suggest that the combination of the proposed method with a foreground segmentation algorithm results in improved foreground segmentation.

Scale selection for anisotropic diffusion filter by Markov random field model

The selection of stopping time (i.e., scale) significantly affects the performance of anisotropic diffusion filter for image denoising. This paper designs a Markov random field (MRF) scale selection model, which selects scales for image segments, then the denoised image is the composition of segments at their optimal scales in the scale space. Firstly, statistics-based scale selection criteria are proposed for image segments. Then we design a scale selection energy function in the MRF framework by considering the scale coherence between neighboring segments. A segment-based noise estimation algorithm is also developed to estimate the noise statistics efficiently. Experiments show that the performance of MRF scale selection model is much better than the previous global scale selection schemes. Combined with this scale selection model, the anisotropic diffusion filter is comparable to or even outperform the state-of-the-art denoising methods in performance.

Building detection from high-resolution PolSAR data at the rectangle level by combining region and edge information

We propose a new approach at the rectangle feature level to extract buildings from high-resolution polarimetric synthetic aperture radar (PolSAR) data, using both region-based and edge-based information. The first step employs low-level detectors to provide raw region and edge information of the scene. In the second step, the rectangle features are initially extracted from the edge detection results, and further optimized to best fit the rough region-based building detection results. In the last step, a novel Markov random field (MRF) framework for rectangles is proposed, in which the data energy term of rectangles is defined from the region information while the smoothness term is defined according to the contextual prior knowledge about the buildings. Under this framework, the building rectangles are identified from the optimized rectangle candidates by minimizing the total energy. The effectiveness of the proposed method is verified using the real fully PolSAR data.

PolSAR Data Segmentation by Combining Tensor Space Cluster Analysis and Markovian Framework

We present a new segmentation method for the fully polarimetric synthetic aperture radar (PolSAR) data by coupling the cluster analysis in the tensor space and the Markov random field (MRF) framework. The PolSAR data are usually obtained as a set of 3 × 3 Hermitian positive definite polarimetric covariance matrices, which do not form a Euclidean space. If we regard each matrix as a tensor, the PolSAR data space can be represented as a Riemannian manifold. First, the mean shift algorithm is extended to the manifold to cluster such tensors. Then, under the MRF framework, the data energy term is defined by the memberships of all tensors in all the clusters, and the smoothness energy term is defined according to the cluster overlap rates. These parameters regarding the cluster analysis are computed under the Riemannian framework. The total energy is minimized using a graph-cut-based optimization to achieve the segmentation results. The effectiveness of the proposed method is verified using real fully PolSAR data and synthetic images.

Deformed Lattice Detection in Real-World Images Using Mean-Shift Belief Propagation

We propose a novel and robust computational framework for automatic detection of deformed 2D wallpaper patterns in real-world images. The theory of 2D crystallographic groups provides a sound and natural correspondence between the underlying lattice of a deformed wallpaper pattern and a degree-4 graphical model. We start the discovery process with unsupervised clustering of interest points and voting for consistent lattice unit proposals. The proposed lattice basis vectors and pattern element contribute to the pairwise compatibility and joint compatibility (observation model) functions in a Markov Random Field (MRF). Thus, we formulate the 2D lattice detection as a spatial, multitarget tracking problem, solved within an MRF framework using a novel and efficient Mean-Shift Belief Propagation (MSBP) method. Iterative detection and growth of the deformed lattice are interleaved with regularized thin-plate spline (TPS) warping, which rectifies the current deformed lattice into a regular one to ensure stability of the MRF model in the next round of lattice recovery. We provide quantitative comparisons of our proposed method with existing algorithms on a diverse set of 261 real-world photos to demonstrate significant advances in accuracy and speed over the state of the art in automatic discovery of regularity in real images.

Cascade Markov random fields for stroke extraction of Chinese characters

Extracting perceptually meaningful strokes plays an essential role in modeling structures of handwritten Chinese characters for accurate character recognition. This paper proposes a cascade Markov random field (MRF) model that combines both bottom-up (BU) and top-down (TD) processes for stroke extraction. In the low-level stroke segmentation process, we use a BU MRF model with smoothness prior to segment the character skeleton into directional substrokes based on self-organization of pixel-based directional features. In the high-level stroke extraction process, the segmented substrokes are sent to a TD MRF-based character model that, in turn, feeds back to guide the merging of corresponding substrokes to produce reliable candidate strokes for character recognition. The merit of the cascade MRF model is due to its ability to encode the local statistical dependencies of neighboring stroke components as well as prior knowledge of Chinese character structures. Encouraging stroke extraction and character recognition results confirm the effectiveness of our method, which integrates both BU/TD vision processing streams within the unified MRF framework.

Linear Neighborhood Propagation and Its Applications

In this paper, a novel graph-based transductive classification approach, called Linear Neighborhood Propagation, is proposed. The basic idea is to predict the label of a data point according to its neighbors in a linear way. This method can be cast into a second-order intrinsic Gaussian Markov random field framework. Its result corresponds to a solution to an approximate inhomogeneous biharmonic equation with Dirichlet boundary conditions. Different from existing approaches, our approach provides a novel graph structure construction method by introducing multiple-wise edges instead of pairwise edges, and presents an effective scheme to estimate the weights for such multiple-wise edges. To the best of our knowledge, these two contributions are novel for semi-supervised classification. The experimental results on image segmentation and transductive classification demonstrate the effectiveness and efficiency of the proposed approach.

Use of Markov Random Fields for automatic cloud/shadow detection on high resolution optical images

In this study, we propose an automatic detection algorithm for cloud/shadow on remote sensing optical images. It is based on physical properties of clouds and shadows, namely for a cloud and its associated shadow: both are connex objects of similar shape and area, and they are related by their relative locations. We show that these properties can be formalized using Markov Random Field (MRF) framework at two levels: one MRF over the pixel graph for connexity modelling, and one MRF over the graph of objects (clouds and shadows) for their relationship modelling. Then, we show that, practically, having performed an image pre-processing step (channel inter-calibration) specific to cloud detection, the local optimization of the proposed MRF models leads to a rather simple image processing algorithm involving only six parameters. Using a 39 image database, performance is shown and discussed, in particular in comparison with the Marked Point Process approach.

A Model-Based Approach to Gene Clustering with Missing Observation Reconstruction in a Markov Random Field Framework

The different measurement techniques that interrogate biological systems provide means for monitoring the behavior of virtually all cell components at different scales and from complementary angles. However, data generated in these experiments are difficult to interpret. A first difficulty arises from high-dimensionality and inherent noise of such data. Organizing them into meaningful groups is then highly desirable to improve our knowledge of biological mechanisms. A more accurate picture can be obtained when accounting for dependencies between components (e.g., genes) under study. A second difficulty arises from the fact that biological experiments often produce missing values. When it is not ignored, the latter issue has been solved by imputing the expression matrix prior to applying traditional analysis methods. Although helpful, this practice can lead to unsound results. We propose in this paper a statistical methodology that integrates individual dependencies in a missing data framework. More explicitly, we present a clustering algorithm dealing with incomplete data in a Hidden Markov Random Field context. This tackles the missing value issue in a probabilistic framework and still allows us to reconstruct missing observations a posteriori without imposing any pre-processing of the data. Experiments on synthetic data validate the gain in using our method, and analysis of real biological data shows its potential to extract biological knowledge.

Markov Random Field-Based Statistical Character Structure Modeling for Handwritten Chinese Character Recognition

This paper proposes a statistical-structural character modeling method based on Markov random fields (MRFs) for handwritten Chinese character recognition (HCCR). The stroke relationships of a Chinese character reflect its structure, which can be statistically represented by the neighborhood system and clique potentials within the MRF framework. Based on the prior knowledge of character structures, we design the neighborhood system that accounts for the most important stroke relationships. We penalize the structurally mismatched stroke relationships with MRFs using the prior clique potentials, and derive the likelihood clique potentials from Gaussian mixture models, which encode the large variations of stroke relationships statistically. In the proposed HCCR system, we use the single-site likelihood clique potentials to extract many candidate strokes from character images, and use the pairsite clique potentials to determine the best structural match between the input candidate strokes and the MRF-based character models by relaxation labeling. The experiments on the KAIST character database demonstrate that MRFs can statistically model character structures, and work well in the HCCR system.

A quadratic programming approach to image labelling

Image labelling tasks are usually formulated within the framework of discrete Markov random fields where the optimal labels are recovered by extremising a discrete energy function. The authors present an alternative continuous relaxation approach to image labelling, which makes use of a quadratic cost function over the class labels. The cost function to be minimised is convex and its discrete version is equivalent up to a constant additive factor to the target function used in discrete MRF approaches. Moreover, its corresponding Hessian matrix is given by the graph Laplacian of the adjacency matrix. Therefore the optimisation of the cost function is governed by the pairwise interactions between pixels in the local neighbourhood. This leads to a sparse Hessian matrix for which the global minimum of the continuous relaxation problem can be efficiently found by solving a system of linear equations using the Cholesky factorisation. The authors elaborate on the links between the method and other techniques elsewhere in the literature and provide results on synthetic and real-world imagery. The authors also provide a comparison with competing approaches.

Multitarget Initiation, Tracking and Termination Using Bayesian Monte Carlo Methods

In this paper, we present an online approach for joint initiation/termination and tracking for multiple targets with multiple sensors using sequential Monte Carlo (SMC) methods. There are several main contributions in the paper. The first contribution is the extension of the deterministic initiation and termination method proposed by the authors' previous publications to a full SMC context in which track initiation/termination are executed with sampling methods. In effect, the dimensions of the particles are variable. In addition, we also integrate a Markov random field (MRF) motion model with the framework to enable efficient and accurate tracking for interacting targets and to avoid potential track coalescence problems. With the employment of multiple sensors, a centralized tracking strategy is adopted, where the observations from all active sensors are fused together for target initiation/termination and tracking and a set of global tracks is maintained. Intra-and inter-sensor clusters are constructed, comprised of closely spaced observations either in time for single sensors or from distinct sensors at a single time, that can increase the reliability when proposing new tracks for initiation. Computer simulations demonstrate that the proposed approach is robust in joint initiation/termination and tracking of multiple manoeuvring targets even when the environment is hostile with high-clutter rates and low target detection probabilities. The integration of the MRF framework into the proposed methods improves robustness in handling close target interactions when the observation noise is high.