Multiscale Markov Random Field Research Articles

Multimodal Fusion of Mobility Demand Data and Remote Sensing Imagery for Urban Land-Use and Land-Cover Mapping

This paper aims at exploring the potentiality of the multimodal fusion of remote sensing imagery with information coming from mobility demand data in the framework of land-use mapping in urban areas. After a discussion on the function of mobility demand data, a probabilistic fusion framework is developed to take advantage of remote sensing and transport data, and their joint use for urban land-use and land-cover applications in urban and surrounding areas. Two different methods are proposed within this framework, the first based on pixelwise probabilistic decision fusion and the second on the combination with a region-based multiscale Markov random field. The experimental validation is conducted on a case study associated with the city of Genoa, Italy.

Regional multiscale Markov random field for remote sensing image classification

According to public and industrial interests on mobile graphics, a preliminary implementation regarding 3D feature visualization system on cell phone was performed using M3G API, one of the de-facto standards for mobile 3D graphic API. Through this experiment, it is revealed that scene graph structure and 3D mobile file format supported from this API is useful one for 3D geo-modeling and rendering in mobile environment. It is necessary that 3D mobile graphic standards can be considered as one component of current mobile GIS services standards to provide value-added 3D GIS contents.

Markov random fields and facial landmarks for handling uncontrolled images of face sketch synthesis

Face sketch synthesis has drawn great attention in many computer vision applications such as law enforcement and digital entertainment. The majority of existing face sketch synthesis techniques are exemplar-based techniques, where a set of training photo–sketch pairs are first divided into patches. For an input photo patch from the face to be synthesized, k similar photo patches are found from the training set. The corresponding sketch patch of the best match is then selected to be synthesized. In such techniques, a multiscale Markov random fields (MRF) model is utilized for synthesizing a sketch using candidate sketch patches; having observed that techniques tend to fail with face photos acquired in uncontrolled imaging conditions like pose and lighting variations. For example, some structures along the lower part of the face sketch contour get lost due to ignoring the global face shape information and illumination changes. In this paper, we propose a reliable face sketch synthesis method based on MRF model and facial landmarks, called MRF-FL that can maintain further structures with uncontrolled face photos. Besides matching the input photo with training photo, the input photo and training sketch are also matched based on the facial landmarks so as to enhance face sketch structures around the lower part of face sketch contour. Experimental results showed that the proposed MRF-FL achieves superior performance compared with recent face sketch synthesis methods on CUHK and AR face sketch databases.

Normalized filter pool for prior modeling of nature images

Markov random field (MRF), as one of special undirected graphs, is widely used in modeling priors of natural images. Targeting to learn better prior models from a given database, we explore the natural image statistics at different scales and build normalized filter pool, a kind of high-order MRF, for prior learning of nature images. The main contribution of the proposed model is that we construct a multi-scale MRF model through constraining the norms of filters in kernel space and integrate all the filtering responses in a unified framework. We formulate both learning and inference as constrained optimization problems and solve them using augmented Lagrange method. The experiment results demonstrate that the normalization of filters at different scales helps to achieve fast convergence in learning stage and obtain superior performance in image restoration, e.g., image denoising and image inpainting.

Lip segmentation under MAP-MRF framework with automatic selection of local observation scale and number of segments.

This paper addresses the problem of segmenting lip region from frontal human face image. Supposing each pixel of the target image has an optimal local scale from the segmentation viewpoint, we treat the lip segmentation problem as a combination of observation scale selection and observed data classification. Accordingly, we propose a hierarchical multiscale Markov random field (MRF) model to represent the membership map of each input pixel to a specific segment and local-scale map simultaneously. Subsequently, lip segmentation can be formulated as an optimal problem in the maximum a posteriori (MAP)-MRF framework. Then, we present a rival-penalized iterative algorithm to implement the segmentation, which is independent of the number of predefined segments. The proposed method mainly features two aspects: 1) its performance is independent of the predefined number of segments, and 2) it takes into account the local optimal observation scale for each pixel. Finally, we conduct the experiments on four benchmark databases, i.e. AR, CVL, GTAV, and VidTIMIT. Experimental results show that the proposed method is robust to the segment number that changes with a speaker's appearance, and can enhance the segmentation accuracy by taking advantage of the local optimal observation scale information.

Global Optimized Multiscale Tobacco Leaves Inspection through Graph Cut

We present a novel multiscale methodology for automatic machine vision application aiming at detecting the size ratio of tobacco leave, and the inspection data will be feedback to adjust running parameters of packaging systems. Firstly, the image is represented by a multiscale Markov Random Field(MRF) model, namely, hidden Markov tree(HMT), which models inter and intra scale dependices of wavelet coefficients. Secondly, according to convex optimization theorem, the energy on hidden Markov tree model is reformulated as a convex version in terms of pseudo marginals. Finally we give the tobacco target segmentation results of the inspection system, which appears to be of better segmentation quality compared to that of the conventional nonconvex energy function.

Face Photo-Sketch Synthesis and Recognition

In this paper, we propose a novel face photo-sketch synthesis and recognition method using a multiscale Markov Random Fields (MRF) model. Our system has three components: 1) given a face photo, synthesizing a sketch drawing; 2) given a face sketch drawing, synthesizing a photo; and 3) searching for face photos in the database based on a query sketch drawn by an artist. It has useful applications for both digital entertainment and law enforcement. We assume that faces to be studied are in a frontal pose, with normal lighting and neutral expression, and have no occlusions. To synthesize sketch/photo images, the face region is divided into overlapping patches for learning. The size of the patches decides the scale of local face structures to be learned. From a training set which contains photo-sketch pairs, the joint photo-sketch model is learned at multiple scales using a multiscale MRF model. By transforming a face photo to a sketch (or transforming a sketch to a photo), the difference between photos and sketches is significantly reduced, thus allowing effective matching between the two in face sketch recognition. After the photo-sketch transformation, in principle, most of the proposed face photo recognition approaches can be applied to face sketch recognition in a straightforward way. Extensive experiments are conducted on a face sketch database including 606 faces, which can be downloaded from our Web site (http://mmlab.ie.cuhk.edu.hk/facesketch.html).

Markov Random Field Modeling of the Spatial Distribution of Proteins on Cell Membranes

Cell membranes display a range of receptors that bind ligands and activate signaling pathways. Signaling is characterized by dramatic changes in membrane molecular topography, including the co-clustering of receptors with signaling molecules and the segregation of other signaling molecules away from receptors. Electron microscopy of immunogold-labeled membranes is a critical technique to generate topographical information at the 5-10 nm resolution needed to understand how signaling complexes assemble and function. However, due to experimental limitations, only two molecular species can usually be labeled at a time. A formidable challenge is to integrate experimental data across multiple experiments where there are from 10 to 100 different proteins and lipids of interest and only the positions of two species can be observed simultaneously. As a solution, we propose the use of Markov random field (MRF) modeling to reconstruct the distribution of multiple cell membrane constituents from pair-wise data sets. MRFs are a powerful mathematical formalism for modeling correlations between states associated with neighboring sites in spatial lattices. The presence or absence of a protein of a specific type at a point on the cell membrane is a state. Since only two protein types can be observed, i.e., those bound to particles, and the rest cannot be observed, the problem is one of deducing the conditional distribution of a MRF with unobservable (hidden) states. Here, we develop a multiscale MRF model and use mathematical programming techniques to infer the conditional distribution of a MRF for proteins of three types from observations showing the spatial relationships between only two types. Application to synthesized data shows that the spatial distributions of three proteins can be reliably estimated. Application to experimental data provides the first maps of the spatial relationship between groups of three different signaling molecules. The work is an important step toward a more complete understanding of membrane spatial organization and dynamics during signaling.

3-D Depth Reconstruction from a Single Still Image

We consider the task of 3-d depth estimation from a single still image. We take a supervised learning approach to this problem, in which we begin by collecting a training set of monocular images (of unstructured indoor and outdoor environments which include forests, sidewalks, trees, buildings, etc.) and their corresponding ground-truth depthmaps. Then, we apply supervised learning to predict the value of the depthmap as a function of the image. Depth estimation is a challenging problem, since local features alone are insufficient to estimate depth at a point, and one needs to consider the global context of the image. Our model uses a hierarchical, multiscale Markov Random Field (MRF) that incorporates multiscale local- and global-image features, and models the depths and the relation between depths at different points in the image. We show that, even on unstructured scenes, our algorithm is frequently able to recover fairly accurate depthmaps. We further propose a model that incorporates both monocular cues and stereo (triangulation) cues, to obtain significantly more accurate depth estimates than is possible using either monocular or stereo cues alone.

Anomaly subspace detection based on a multi-scale Markov random field model

In this paper we introduce a multi-scale Gaussian Markov random field (GMRF) model and a corresponding anomaly subspace detection algorithm. Natural clutter images, often appear to have several periodical patterns of various period lengths. In such cases, the GMRF model may not sufficiently describe the clutter image. The proposed model is based on a multi-scale wavelet representation of the image, independent component analysis, and modeling each independent component as a GMRF. Anomaly detection is subsequently carried out by applying a matched subspace detector to the innovations process generated by the presumed model. The robustness of the proposed approach is demonstrated with application to automatic target detection in synthetic and real imagery. A quantitative performance analysis and experimental results demonstrate the advantage of the proposed method in comparison to competing methods.

Error concealment for video transmission with dual multiscale markov random field modeling

A novel error concealment algorithm based on a stochastic modeling approach is proposed as a post-processing tool at the decoder side for recovering the lost information incurred during the transmission of encoded digital video bitstreams. In our proposed scheme, both the spatial and the temporal contextual features in video signals are separately modeled using the multiscale Markov random field (MMRF). The lost information is then estimated using maximum a posteriori (MAP) probabilistic approach based on the spatial and temporal MMRF models; hence, a unified MMRF-MAP framework. To preserve the high frequency information (in particular, the edges) of the damaged video frames through iterative optimization, a new adaptive potential function is also introduced in this paper. Comparing to the existing MRF-based schemes and other traditional concealment algorithms, the proposed dual MMRF (DMMRF) modeling method offers significant improvement on both objective peak signal-to-noise ratio (PSNR) measurement and subjective visual quality of restored video sequence.

Segmentation of textured images using a multiresolution Gaussian autoregressive model

We present a new algorithm for segmentation of textured images using a multiresolution Bayesian approach. The new algorithm uses a multiresolution Gaussian autoregressive (MGAR) model for the pyramid representation of the observed image, and assumes a multiscale Markov random field model for the class label pyramid. The models used in this paper incorporate correlations between different levels of both the observed image pyramid and the class label pyramid. The criterion used for segmentation is the minimization of the expected value of the number of misclassified nodes in the multiresolution lattice. The estimate which satisfies this criterion is referred to as the "multiresolution maximization of the posterior marginals" (MMPM) estimate, and is a natural extension of the single-resolution "maximization of the posterior marginals" (MPM) estimate. Previous multiresolution segmentation techniques have been based on the maximum a posterior (MAP) estimation criterion, which has been shown to be less appropriate for segmentation than the MPM criterion. It is assumed that the number of distinct textures in the observed image is known. The parameters of the MGAR model-the means, prediction coefficients, and prediction error variances of the different textures-are unknown. A modified version of the expectation-maximization (EM) algorithm is used to estimate these parameters. The parameters of the Gibbs distribution for the label pyramid are assumed to be known. Experimental results demonstrating the performance of the algorithm are presented.

Direct incremental model-based image motion segmentation for video analysis

Dynamic analysis of image sequences is an important task in object-oriented video applications. It often relies on the segmentation of each image of the sequence into region entities of apparent homogeneous motion. In this paper, we present an original motion segmentation algorithm based on 2D polynomial motion models, a multiresolution robust estimator to compute these motion models, and appropriate local observations supplying both motion relevant information and their reliability. Motion segmentation is formulated as a contextual statistical labeling problem exploiting multiscale Markov random field (MRF) models. One of its main features is that it avoids time consuming alternate iterations between motion model estimation and spatial support identification. An original detection step allows us to estimate and to update the number of required motion models, and thus to handle the appearance of new objects. Numerous experiments performed with real indoor and outdoor image sequences demonstrate the efficiency of the method.

Multiscale representations of Markov random fields

Recently, a framework for multiscale stochastic modeling was introduced based on coarse-to-fine scale-recursive dynamics defined on trees. This model class has some attractive characteristics which lead to extremely efficient, statistically optimal signal and image processing algorithms. The authors show that this model class is also quite rich. In particular, they describe how 1-D Markov processes and 2-D Markov random fields (MRFs) can be represented within this framework. The recursive structure of 1-D Markov processes makes them simple to analyze, and generally leads to computationally efficient algorithms for statistical inference. On the other hand, 2-D MRFs are well known to be very difficult to analyze due to their noncausal structure, and thus their use typically leads to computationally intensive algorithms for smoothing and parameter identification. In contrast, their multiscale representations are based on scale-recursive models and thus lead naturally to scale-recursive algorithms, which can be substantially more efficient computationally than those associated with MRF models. In 1-D, the multiscale representation is a generalization of the midpoint deflection construction of Brownian motion. The representation of 2-D MRFs is based on a further generalization to a midline deflection construction. The exact representations of 2-D MRFs are used to motivate a class of multiscale approximate MRF models based on one-dimensional wavelet transforms. They demonstrate the use of these latter models in the context of texture representation and, in particular, they show how they can be used as approximations for or alternatives to well-known MRF texture models. >