Coupled Markov Random Fields Research Articles

Discontinuity preserving phase unwrapping algorithm for simulated InSAR based on MRF model using line process

Phase unwrapping (PU) is one of the important procedures in measuring digital elevation model (DEM) from interferometric synthetic aperture radar (InSAR) images. It is known that the existence of phase discontinuity poses challenges to the traditional PU algorithms. In this study, the authors propose a coupled Markov random fields (MRF) called line process (LP) for PU, abbreviated as the LP-MRF method. First, an 8-neighbourhood line process is used to detect discontinuous phase surfaces while regularising continuous phase surface. Then, a truncated quadratic is utilised in the LP model for optimisation convenience. Finally, the proposed method on four types of simulated experimental data is tested, which yields that the proposed method has better performance than some of the commonly used methods in handling sharp phase discontinuities while regularising continuous phase surfaces.

Change Detection of Polarimetric SAR Images Based on the Integration of Improved Watershed and MRF Segmentation Approaches

Land cover and land use change detection is one of the most important applications of polarimetric synthetic aperture radar (PolSAR) data. In a pixel-based method, the speckle effects and the lack of consideration for spatial information produce false alarms. Therefore, an object-based image analysis has been used to overcome these weaknesses. This paper proposes a novel hybrid image segmentation algorithm for improving the accuracy of land cover change detection. This method consists of three steps. First, the segmentation of two PolSAR images by integrating segmentation techniques, namely, region-based improved watershed and an edge-based coupled Markov random field (MRF). Second, the selection of the optimal ratios of polarimetric features based on a genetic algorithm and Jefferies–Matusita distance criteria. Third, the binary classification of image objects using the ratio of mean pixel values of the corresponding image objects. Compared with the conventional watershed and multiresolution segmentation methods, the improved watershed reduces the speckle effects in PolSAR images and avoids the oversegmentation problem. The use of coupled MRF detects the most accurate edge positions and improves the hybrid segmentation approach. The proposed change detection method, as applied on unmanned aerial vehicle synthetic aperture full polarimetric images, demonstrates an overall accuracy of 90.14% and a kappa coefficient of 0.80. Compared with the pixel-based change detection method based on a support vector machine classifier for the selected subspace of ratios of polarimetric features, the proposed method improved the overall accuracy by 8.72% and the kappa coefficient by 0.18.

A Coupled Hidden Markov Random Field model for simultaneous face clustering and tracking in videos

Face clustering and face tracking are two areas of active research in automatic facial video processing. They, however, have long been studied separately, despite the inherent link between them. In this paper, we propose to perform simultaneous face clustering and face tracking from real world videos. The motivation for the proposed research is that face clustering and face tracking can provide useful information and constraints to each other, thus can bootstrap and improve the performances of each other. To this end, we introduce a Coupled Hidden Markov Random Field (CHMRF) to simultaneously model face clustering, face tracking, and their interactions. We provide an effective algorithm based on constrained clustering and optimal tracking for the joint optimization of cluster labels and face tracking. We demonstrate significant improvements over state-of-the-art results in face clustering and tracking on several videos.

Joint estimation of tissue types and linear attenuation coefficients for photon counting CT.

Newly developed spectral computed tomography (CT) such as photon counting detector CT enables more accurate tissue-type identification through material decomposition technique. Many iterative reconstruction methods, including those developed for spectral CT, however, employ a regularization term whose penalty transition is designed using pixel value of CT image itself. Similarly, the tissue-type identification methods are then applied after reconstruction; thus, it is impossible to take into account probability distribution obtained from projection likelihood. The purpose of this work is to develop comprehensive image reconstruction and tissue-type identification algorithm which improves quality of both reconstructed image and tissue-type map. The authors propose a new framework to jointly perform image reconstruction, material decomposition, and tissue-type identification for photon counting detector CT by applying maximum a posteriori estimation with voxel-based latent variables for the tissue types. The latent variables are treated using a voxel-based coupled Markov random field to describe the continuity and discontinuity of human organs and a set of Gaussian distributions to incorporate the statistical relation between the tissue types and their attenuation characteristics. The performance of the proposed method is quantitatively compared to that of filtered backprojection and a quadratic penalized likelihood method by 100 noise realization. Results showed a superior trade-off between image noise and resolution to current reconstruction methods. The standard deviation (SD) and bias of reconstructed image were improved from quadratic penalized likelihood method: bias, -0.9 vs -0.1 Hounsfield unit (HU); SD, 46.8 vs 27.4 HU. The accuracy of tissue-type identification was also improved from quadratic penalized likelihood method: 80.1% vs 86.9%. The proposed method makes it possible not only to identify tissue types more accurately but also to reconstruct CT images with decreased noise and enhanced sharpness owing to the information about the tissue types.

VLSI implementation of coupled MRF model using pulse‐coupled phase oscillators

Efficient pixel-parallel image processing using a pulse-coupled phase oscillator model and its very large scale integration (VLSI) implementation is proposed. A processing unit that corresponds to a pixel of an image transmits spike pulses to other units, and updates its own analogue state value at timing when spikes come from other units. From a VLSI implementation point of view, this mechanism is suitable for very low-power operation because analogue buffers are unnecessary for data transmission. On the basis of this model, a VLSI image processor chip that performs a coupled Markov random field model for image region segmentation is designed and fabricated. A very low-power VLSI design has been achieved by the combination of an analogue oscillator and digital coupling function generator circuits with time-domain computation. The processing performance of the fabricated oscillator-based image processor chip using a 0.25 μm CMOS process has achieved 43.2 GOPS or 656 GOPS/W. Experiments using the fabricated chip have shown successful image region segmentation in one- and two-dimensional images.

A Region-based Coupled MRF Model for Coarse Image Region Segmentation toward its VLSI Implementation

Coupled Markov random field (MRF) models have been proposed so far for visual image processing. These are classified into boundary- and region-based models, where hidden variables that determine the interaction between the units corresponding to image pixels are given by line and label processes, respectively. In this paper, we have investigated a region-based coupled MRF model with phase variables as the hidden variables, and have modified the model for applications to coarse image-region segmentation by replacing some nonlinear functions in the update equations of the intensity and label processes with piecewise linear (PWL) functions. Using PWL functions facilitates VLSI implementation of coupled MRF models, and also makes their performance improved. We have verified that the modified region-based MRF model is superior to the resistive-fuse network, which is one of the boundary-based MRF models. We also propose an improvement of the modified region-based MRF model by introducing a new parameter, and evaluate the model performance in coarse image region segmentation.

Tracking Multiple Visual Targets via Particle-Based Belief Propagation

Multiple-target tracking in video (MTTV) presents a technical challenge in video surveillance applications. In this paper, we formulate the MTTV problem using dynamic Markov network (DMN) techniques. Our model consists of three coupled Markov random fields: 1) a field for the joint state of the multitarget; 2) a binary random process for the existence of each individual target; and 3) a binary random process for the occlusion of each dual adjacent target. To make the inference tractable, we introduce two robust functions that eliminate the two binary processes. We then propose a novel belief propagation (BP) algorithm called particle-based BP and embed it into a Markov chain Monte Carlo approach to obtain the maximum a posteriori estimation in the DMN. With a stratified sampler, we incorporate the information obtained from a learned bottom-up detector (e.g., support-vector-machine-based classifier) and the motion model of the target into the message propagation. Other low-level visual cues such as motion and shape can be easily incorporated into our framework to obtain better tracking results. We have performed extensive experimental verification, and the results suggest that our method is comparable to the state-of-art multitarget tracking methods in all the cases we tested.

Adaptive Segmentation of Textured Images by Using the Coupled Markov Random Field Model

Although simple and efficient, traditional feature-based texture segmentation methods usually suffer from the intrinsical less inaccuracy, which is mainly caused by the oversimplified assumption that each textured subimage used to estimate a feature is homogeneous. To solve this problem, an adaptive segmentation algorithm based on the coupled Markov random field (CMRF) model is proposed in this paper. The CMRF model has two mutually dependent components: one models the observed image to estimate features, and the other models the labeling to achieve segmentation. When calculating the feature of each pixel, the homogeneity of the subimage is ensured by using only the pixels currently labeled as the same pattern. With the acquired features, the labeling is obtained through solving a maximum a posteriori problem. In our adaptive approach, the feature set and the labeling are mutually dependent on each other, and therefore are alternately optimized by using a simulated annealing scheme. With the gradual improvement of features' accuracy, the labeling is able to locate the exact boundary of each texture pattern adaptively. The proposed algorithm is compared with a simple MRF model based method in segmentation of Brodatz texture mosaics and real scene images. The satisfying experimental results demonstrate that the proposed approach can differentiate textured images more accurately.

Sequential stratified sampling belief propagation for multiple targets tracking

Rather than the difficulties of highly non-linear and non-Gaussian observation process and the state distribution in single target tracking, the presence of a large, varying number of targets and their interactions place more challenge on visual tracking. To overcome these difficulties, we formulate multiple targets tracking problem in a dynamic Markov network which consists of three coupled Markov random fields that model the following: a field for joint state of multi-target, one binary process for existence of individual target, and another binary process for occlusion of dual adjacent targets. By introducing two robust functions, we eliminate the two binary processes, and then apply a novel version of belief propagation called sequential stratified sampling belief propagation algorithm to obtain the maximum a posteriori (MAP) estimation in the dynamic Markov network. By using stratified sampler, we incorporate bottom-up information provided by a learned detector (e.g. SVM classifier) and belief information for the messages updating. Other low-level visual cues (e.g. color and shape) can be easily incorporated in our multi-target tracking model to obtain better tracking results. Experimental results suggest that our method is comparable to the state-of-the-art multiple targets tracking methods in several test cases.

Guest editors' introduction to the special section on graphical models in computer vision

THE last 10 years have witnessed rapid growth in the popularity of graphical models, most notably Bayesian networks, as a tool for representing, learning, and computing complex probability distributions. Graphical models provide an explicit representation of the statistical dependencies between the components of a complex probability model, effectively marrying probability theory and graph theory. As Jordan puts it in [2], graphical models are “a natural tool for dealing with two problems that occur throughout applied mathematics and engineering—uncertainty and complexity—and, in particular, they are playing an increasingly important role in the design and analysis of machine learning algorithms.” Graphical models provide powerful computational support for the Bayesian approach to computer vision, which has become a standard framework for addressing vision problems. Many familiar tools from the vision literature, such as Markov random fields, hidden Markov models, and the Kalman filter, are instances of graphical models. More importantly, the graphical models formalism makes it possible to generalize these tools and develop novel statistical representations and associated algorithms for inference and learning. The history of graphical models in computer vision follows closely that of graphical models in general. Research by Pearl [3] and Lauritzen [4] in the late 1980s played a seminal role in introducing this formalism to areas of AI and statistical learning. Not long after, the formalism spread to fields such as statistics, systems engineering, information theory, pattern recognition, and, among others, computer vision. One of the earliest occurrences of graphical models in the vision literature was a paper by Binford et al. [1]. The paper described the use of Bayesian inference in a hierarchical probability model to match 3D object models to groupings of curves in a single image. The following year marked the publication of Pearl’s influential book [3] on graphical models. Since then, many technical papers have been published in IEEE journals and conference proceedings that address different aspects and applications of graphical models in computer vision. Our goal in organizing this special section was to demonstrate the breadth of applicability of the graphical models formalism to vision problems. Our call for papers in February 2002 produced 16 submissions. After a careful review process, we selected six papers for publication, including five regular papers, and one short paper. These papers reflect the state-of-the-art in the use of graphical models in vision problems that range from low-level image understanding to high-level scene interpretation. We believe these papers will appeal both to vision researchers who are actively engaged in the use of graphical models and machine learning researchers looking for a challenging application domain. The first paper in this section is “Stereo Matching Using Belief Propagation” by J. Sun, N.-N. Zheng, and H.-Y. Shum. The authors describe a new stereo algorithm based on loopy belief propagation, a powerful inference technique for complex graphical models in which exact inference is intractable. They formulate the dense stereo matching problem as MAP estimation on coupled Markov random fields and obtain promising results on standard test data sets. One of the benefits of this formulation, as the authors demonstrate, is the ease with which it can be extended to handle multiview stereo matching. In their paper “Statistical Cue Integration of DAG Deformable Models” S.K. Goldenstein, C. Vogler, and D. Metaxas describe a scheme for combining different sources of information into estimates of the parameters of a deformable model. They use a DAG representation of the interdependencies between the nodes in a deformable model. This framework supports the efficient integration of information from edges and other cues using the machinery of affine arithmetic and the propagation of uncertainties. They present experimental results for a face tracking application. Y. Song, L. Goncalves, and P. Perona describe, in their paper “Unsupervised Learning of Human Motion,” a method for learning probabilistic models of human motion from video sequences in cluttered scenes. Two key advantages of their method are its unsupervised nature, which can mitigate the need for tedious hand labeling of data, and the utilization of graphical model constraints to reduce the search space when fitting a human figure model. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 25, NO. 7, JULY 2003 785

Stereo matching using belief propagation

In this paper, we formulate the stereo matching problem as a Markov network and solve it using Bayesian belief propagation. The stereo Markov network consists of three coupled Markov random fields that model the following: a smooth field for depth/disparity, a line process for depth discontinuity, and a binary process for occlusion. After eliminating the line process and the binary process by introducing two robust functions, we apply the belief propagation algorithm to obtain the maximum a posteriori (MAP) estimation in the Markov network. Other low-level visual cues (e.g., image segmentation) can also be easily incorporated in our stereo model to obtain better stereo results. Experiments demonstrate that our methods are comparable to the state-of-the-art stereo algorithms for many test cases.

Coupled Markov random field models with phases as line field and region field

AbstractWe investigate a boundary‐based and a region‐based coupled Markov random field model, both of which are useful in image restorations of gray‐level images. In the conventional boundary‐based and the conventional region‐based coupled Markov random field models, both a line field and a region field take only two discrete states, 0 and 1. In the boundary‐based coupled random field model, existence and nonexistence of the edge at each nearest‐neighbor pair of pixels are denoted by 1 and 0, respectively. In the region‐based coupled random field model, some different regions at each pixel are labeled by discrete numbers. We propose a boundary‐based coupled Markov random field model with continuous line field and a region‐based coupled Markov random field model with continuous segmentation field from a standpoint of a plane rotator model in statistical mechanics. The iterative algorithms for image restoration are constructed by using a mean‐field approximation. We investigate how the proposed models produce a better quality of restored images. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(7): 24–33, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10364

Fast Fully Data-Driven Image Restoration by means of Edge-Preserving Regularization

The fully data driven deconvolution of noisy images is a highly ill-posed problem, where the image, the blur and the noise parameters have to be simultaneously estimated from the data alone. Our approach is to exploit the information related to the image intensity edges both to improve the solution and to significantly reduce the computational costs. To detect reliable intensity edges, the image is modeled through a coupled Markov Random Field with an explicit, binary and constrained line process. Following a fully Bayesian approach, the solution should be given by the joint maximization of a distribution of the image field, the data, the blur and model parameters. A first, significant reduction in computational complexity is obtained by decomposing this joint maximization into a sequence of Maximum a posteriori and/or Maximum Likelihood estimations, to be performed alternately and iteratively. The presence of an explicit and binary line field is then exploited to reduce the computational cost of the usually very expensive model parameter estimation step. On this basis, we derive efficient and fast algorithms along with procedures which are feasible and effective for real-time applications, where the real-time requirements are not too strict. Indeed, the structure of these algorithms are intrinsically parallel, and thus suitable for implementation on high-performance machines, or on specialized hardware and allows the computation time to be greatly reduced. The experimental results show that the method allows one to obtain good blur estimates even in the presence of noise, without any need for smoothness assumptions on the blur coefficients, which would polarize the solution towards often unrealistic uniform blurs.

Bayesian estimation for homogeneous and inhomogeneous Gaussian random fields

This paper investigates Bayesian estimation for Gaussian Markov random fields. In particular, a new class of compound model is proposed which describes the observed intensities using an inhomogeneous model and the degree of spatial variation described by a second random field. The coupled Markov random fields are used as prior distributions, and combined with Gaussian noise models to produce posterior distributions on which estimation is based. All model parameters are estimated, in a fully Bayesian setting, using the Metropolis-Hasting algorithm. The full posterior estimation procedures are illustrated and compared using various artificial examples. For these examples the inhomogeneous model performs very favorably when compared to the homogeneous model, allowing differential degrees of smoothing and varying local textures.

Reconstruction and Boundary Detection of Range and Intensity Images Using Multiscale MRF Representations

The basic difficulty encountered in filtering-based multiscale boundary detection methods is the elimination of noise and insignificant edges without distorting the shape of boundaries. These methods remove noise and unnecessary detail by blurring the input image at different scales, which results in the loss of positional accuracy at the image discontinuities. In this paper, a nonlinear multiscale boundary detection method which prevents the conflict between the detection and localization goals is introduced. The method uses multiscale representations of coupled Markov random fields and applies a stochastic regularization scheme based on the Bayesian approach. This allows the robust integration of boundary information extracted at multiple scales simultaneously. The scheme is applicable to intensity and range images as well as to sparse data and eliminates the dependency on edge operator size which is the main difficulty in filtering-based multiscale techniques.

Cooperative integration of stereopsis and optic flow computation

A cooperative integration of stereopsis and optic flow computation is presented. Central to our approach is the modeling of the visual processes as a sequence of coupled Markov random fields by definition of suitable interprocess interactions based on some natural constraints. The integration makes each of the individual processes better constrained and more reliable. Further, as a result of the integration, it becomes possible to obtain accurately the discontinuities in both the flow and the disparity fields along with the regions of stereo occlusion. Results on both noisy synthetic image data and real images are presented.

Physical Modeling and Combination of Range and Intensity Edge Data

We present a method for semantic labelling of edges and reconstruction of range data by fusion of registered range and intensity images. An initial set of edge labels is derived using a physical model of object geometry and shading. A final edge classification and range reconstruction are obtained using Bayesian estimation within coupled Markov random fields employing constraints of surface smoothness and edge continuity. The approach is demonstrated on synthetic and real source data, obtained from an active laser rangefinder.

Distributed propagation of a-priori constraints in a Bayesian network of Markov random fields

In this paper, Bayesian networks of Markov random fields (BN-MRFs) are proposed as a technique for representing and applying apriori knowledge at different abstraction levels inside a distributed image processing framework. It is shown that this approach, thanks to the common probabilistic basis of the two techniques, is able to combine in a natural way causal inference properties at different abstraction levels as provided by Bayesian networks with optimisation criteria usually applied to find the best configuration for an MRF. Examples of two-level BN-MRFs are given, where each node uses a coupled Markov random field which has to solve a coupled restoration and segmentation problem. Experiments are concerned with expert-driven registered segmentation and tracking of regions from image sequences.

Integration of stereo vision and optical flow by using an energy-minimization approach

A cooperative method is proposed in which image intensity (brightness) and optical-flow information are integrated into a single stereo technique by modeling the input data as coupled Markov random fields (MRF’s). The Bayesian probabilistic estimation method and the MRF–Gibbs equivalence theory are used to integrate the optical flow and the gray-level intensity information to obtain an energy function that will explicitly represent the depth discontinuity and occlusion constraints on the solution. This energy function involves the similarity in intensity (or edge orientation) and the optical flow between corresponding sites of the left and right images as well as the smoothness constraint on the disparity solution. If a simple MRF is used to model the data, the energy function will yield a poor disparity by smoothing across object boundaries, particularly when occluding objects are present. We use optical-flow information to indicate object boundaries (depth discontinuities) and occluded regions, in order to improve the disparity solution in occluded regions. A stochastic relaxation algorithm (simulated annealing) is used to find a favorable disparity solution by minimization of the energy equation.

Integration of vision modules and labeling of surface discontinuities

It is assumed that a major goal of the early vision modules and their integration is to deliver a cartoon of the discontinuities in the scene and to label them in terms of their physical origin. The output of each of the vision modules is noisy, possibly sparse, and sometimes not unique. The authors suggest the use of a coupled Markov random field (MRF) at the output of each module (image cues)-stereo, motion, color, and texture-to achieve two goals: first, to counteract the noise and fill in sparse data, and secondly, to integrate the image within each MRF to find the module discontinuities and align them with the intensity edges. The authors outline a theory of how to label the discontinuities in terms of depth, orientation, albedo, illumination, and specular discontinuities. They present labeling results using a simple linear classifier operating on the output of the MRF associated with each vision module and coupled to the image data. The classifier has been trained on a small set of a mixture of synthetic and real data.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>