Markov Random Field Energy Function Research Articles

Efficient Seamline Network Generation for Large-Scale Orthoimage Mosaicking

Seamline network generation from multiple aerial images with overlapping regions is a key issue for creating seamless and large-scale digital orthophoto maps (DOMs). In this letter, an efficient algorithm is proposed that can find adequate networks from hundreds of aerial images in several minutes. It can also be ensured that the seamline passes through weakly textured regions as much as possible. The proposed algorithm consists of two steps. First, the regions overlapped by the images are triangulated into a mesh. Second, a labeling <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l$ </tex-math></inline-formula> is computed that assigns an image <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{i}$ </tex-math></inline-formula> as a texture for each triangle using a Markov random field energy function. Triangulating the overlapping area of the images can greatly improve the efficiency of the seamline network generation, and by designing a smoothing term with a divergence difference between images, the seamline network passes through the weakly textured area as much as possible. In addition, we also use a hierarchical optimization strategy to improve efficiency in large-scale scenes. The proposed method can be easily combined with other data information (e.g., a digital surface model), or other auxiliary methods (e.g., building segmentation), to generate high-quality seamline networks. Experimental results on numerous challenging datasets show that the proposed algorithm can achieve high-quality seamline networks efficiently. Moreover, it outperforms several existing methods and software.

A Novel OpenMVS-Based Texture Reconstruction Method Based on the Fully Automatic Plane Segmentation for 3D Mesh Models

The Markov Random Field (MRF) energy function, constructed by existing OpenMVS-based 3D texture reconstruction algorithms, considers only the image label of the adjacent triangle face for the smoothness term and ignores the planar-structure information of the model. As a result, the generated texture charts results have too many fragments, leading to a serious local miscut and color discontinuity between texture charts. This paper fully utilizes the planar structure information of the mesh model and the visual information of the 3D triangle face on the image and proposes an improved, faster, and high-quality texture chart generation method based on the texture chart generation algorithm of the OpenMVS. This methodology of the proposed approach is as follows: (1) The visual quality on different visual images of each triangle face is scored using the visual information of the triangle face on each image in the mesh model. (2) A fully automatic Variational Shape Approximation (VSA) plane segmentation algorithm is used to segment the blocked 3D mesh models. The proposed fully automatic VSA-based plane segmentation algorithm is suitable for multi-threaded parallel processing, which solves the VSA framework needed to manually set the number of planes and the low computational efficiency in a large scene model. (3) The visual quality of the triangle face on different visual images is used as the data term, and the image label of adjective triangle and result of plane segmentation are utilized as the smoothness term to construct the MRF energy function. (4) An image label is assigned to each triangle by the minimizing energy function. A texture chart is generated by clustering the topologically-adjacent triangle faces with the same image label, and the jagged boundaries of the texture chart are smoothed. Three sets of data of different types were used for quantitative and qualitative evaluation. Compared with the original OpenMVS texture chart generation method, the experiments show that the proposed approach significantly reduces the number of texture charts, significantly improves miscuts and color differences between texture charts, and highly boosts the efficiency of VSA plane segmentation algorithm and OpenMVS texture reconstruction.

Non-Rigid Multi-Modal 3D Medical Image Registration Based on Foveated Modality Independent Neighborhood Descriptor.

The non-rigid multi-modal three-dimensional (3D) medical image registration is highly challenging due to the difficulty in the construction of similarity measure and the solution of non-rigid transformation parameters. A novel structural representation based registration method is proposed to address these problems. Firstly, an improved modality independent neighborhood descriptor (MIND) that is based on the foveated nonlocal self-similarity is designed for the effective structural representations of 3D medical images to transform multi-modal image registration into mono-modal one. The sum of absolute differences between structural representations is computed as the similarity measure. Subsequently, the foveated MIND based spatial constraint is introduced into the Markov random field (MRF) optimization to reduce the number of transformation parameters and restrict the calculation of the energy function in the image region involving non-rigid deformation. Finally, the accurate and efficient 3D medical image registration is realized by minimizing the similarity measure based MRF energy function. Extensive experiments on 3D positron emission tomography (PET), computed tomography (CT), T1, T2, and (proton density) PD weighted magnetic resonance (MR) images with synthetic deformation demonstrate that the proposed method has higher computational efficiency and registration accuracy in terms of target registration error (TRE) than the registration methods that are based on the hybrid L-BFGS-B and cat swarm optimization (HLCSO), the sum of squared differences on entropy images, the MIND, and the self-similarity context (SSC) descriptor, except that it provides slightly bigger TRE than the HLCSO for CT-PET image registration. Experiments on real MR and ultrasound images with unknown deformation have also be done to demonstrate the practicality and superiority of the proposed method.

Spectral–spatial classification of hyperspectral images by algebraic multigrid based multiscale information fusion

ABSTRACT In this work, we present a novel spectral-spatial classification framework of hyperspectral images (HSIs) by integrating the techniques of algebraic multigrid (AMG), hierarchical segmentation (HSEG) and Markov random field (MRF). The proposed framework manifests two main contributions. First, an effective HSI segmentation method is developed by combining the AMG-based marker selection approach and the conventional HSEG algorithm to construct a set of unsupervised segmentation maps in multiple scales. To improve the computational efficiency, the fast Fish Markov selector (FMS) algorithm is exploited for feature selection before image segmentation. Second, an improved MRF energy function is proposed for multiscale information fusion (MIF) by considering both spatial and inter-scale contextual information. Experiments were performed using two airborne HSIs to evaluate the performance of the proposed framework in comparison with several popular classification methods. The experimental results demonstrated that the proposed framework can provide superior performance in terms of both qualitative and quantitative analysis.

A contextual-based segmentation of compact PolSAR images using Markov Random Field (MRF) model

ABSTRACTAs the first major step in each object-oriented feature extraction approach, segmentation plays an essential role as a preliminary step towards further and higher levels of image processing. The primary objective of this paper is to illustrate the potential of Polarimetric Synthetic Aperture Radar (PolSAR) features extracted from Compact Polarimetry (CP) SAR data for image segmentation using Markov Random Field (MRF). The proposed method takes advantage of both spectral and spatial information to segment the CP SAR data. In the first step of the proposed method, k-means clustering was applied to over-segment the image using the appropriate features optimally selected using Genetic Algorithm (GA). As a similarity criterion in each cluster, a probabilistic distance was used for an agglomerative hierarchical merging of small clusters into an appropriate number of larger clusters. In the agglomerative clustering approach, the estimation of the appropriate number of clusters using the data log-likelihood algorithm differs depending on the distance criterion used in the algorithm. In particular, the Wishart Chernoff distance which is independent of samples (pixels) tends to provide a higher appropriate number of clusters compared to the Wishart test statistic distance. This is because the Wishart Chernoff distance preserves detailed data information corresponding to small clusters. The probabilistic distance used in this study is Wishart Chernoff distance which evaluates the similarity of clusters by measuring the distance between their complex Wishart probability density functions. The output of this step, as the initial segmentation of the image, is applied to a Markov Random Field model to improve the final segmentation using vicinity information. The method combines Wishart clustering and enhanced initial clusters in order to access the posterior MRF energy function. The contextual image classifier adopts the Iterated Conditional Mode (ICM) approach to converge to a local minimum and represent a good trade-off between segmentation accuracy and computation burden. The results showed that the PolSAR features extracted from CP mode can provide an acceptable overall accuracy in segmentation when compared to the full polarimetry (FP) and Dual Polarimetry (DP) data. Moreover, the results indicated that the proposed algorithm is superior to the existing image segmentation techniques in terms of segmentation accuracy.

Interactive stereo image segmentation via adaptive prior selection

Interactive stereo image segmentation (i.e., cutting out objects from stereo pairs with limited user assistance) is an important research topic in computer vision. Given a pair of images, users mark a few foreground/background pixels, based on which prior models are formulated for labeling unknown pixels. Note that color priors might not help if the marked foreground and background have similar colors. However, integrating multiple types of priors, e.g., color and disparity in segmenting stereo pairs, is not trivial. This is because differing pairs of images and even differing pixels in the same image might require different proportions of the priors. Besides, disparities of natural images are too noisy to be directly used. This paper presents a method that can adaptively determine the proportion of the priors (color or disparity) for each pixel. Specifically speaking, the segmentation problem is defined in the framework of MRF (Markov Random Field). We formulate an MRF energy function which is composed of clues from the two types of priors, as well as neighborhood smoothness and stereo correspondence constraints. The weights of the color and disparity priors at each pixel are treated as variables which are optimized together with the label (foreground or background) of the pixel. In order to overcome the noise problem, the weight of the disparity prior is controlled by a confidence value learned from data. The energy function is optimized by using multi-label graph cut. Experimental results show that our method performs well.

BUILDING ROOF BOUNDARY EXTRACTION FROM LiDAR AND IMAGE DATA BASED ON MARKOV RANDOM FIELD

Abstract. In this paper a method for automatic extraction of building roof boundaries is proposed, which combines LiDAR data and highresolution aerial images. The proposed method is based on three steps. In the first step aboveground objects are extracted from LiDAR data. Initially a filtering algorithm is used to process the original LiDAR data for getting ground and non-ground points. Then, a region-growing procedure and the convex hull algorithm are sequentially used to extract polylines that represent aboveground objects from the non-ground point cloud. The second step consists in extracting corresponding LiDAR-derived aboveground objects from a high-resolution aerial image. In order to avoid searching for the interest objects over the whole image, the LiDAR-derived aboveground objects’ polylines are photogrammetrically projected onto the image space and rectangular bounding boxes (sub-images) that enclose projected polylines are generated. Each sub-image is processed for extracting the polyline that represents the interest aboveground object within the selected sub-image. Last step consists in identifying polylines that represent building roof boundaries. We use the Markov Random Field (MRF) model for modelling building roof characteristics and spatial configurations. Polylines that represent building roof boundaries are found by optimizing the resulting MRF energy function using the Genetic Algorithm. Experimental results are presented and discussed in this paper.

A Kernel PCA Shape Prior and Edge Based MRF Image Segmentation

We introduce both shape prior and edge information to Markov random field (MRF) to segment target of interest in images. Kernel Principal component analysis (PCA) is performed on a set of training shapes to obtain statistical shape representation. Edges are extracted directly from images. Both of them are added to the MRF energy function and the integrated energy function is minimized by graph cuts. An alignment procedure is presented to deal with variations between the target object and shape templates. Edge information makes the influence of inaccurate shape alignment not too severe, and brings result smoother. The experiments indicate that shape and edge play important roles for complete and robust foreground segmentation.

PAIRWISE-SVM FOR ON-BOARD URBAN ROAD LIDAR CLASSIFICATION

The common method of LiDAR classifications is Markov random fields (MRF). Based on construction of MRF energy function, spectral and directional features are extracted for on-board urban point clouds. The MRF energy function is consisted of unary and pairwise potentials. The unary terms are computed by SVM classifictaion. The initial labeling is mainly processed through geometrical shapes. The pairwise potential is estimated by Naïve Bayes. From training data, the probability of adjacent objects is computed by prior knowledge. The final labeling method is reweighted message-passing to minimization the energy function. The MRF model is difficult to process the large-scale misclassification. We propose a super-voxel clustering method for over-segment and grouping segment for large objects. Trees, poles ground, and building are classified in this paper. The experimental results show that this method improves the accuracy of classification and speed of computation.

PAIRWISE-SVM FOR ON-BOARD URBAN ROAD LIDAR CLASSIFICATION

Abstract. The common method of LiDAR classifications is Markov random fields (MRF). Based on construction of MRF energy function, spectral and directional features are extracted for on-board urban point clouds. The MRF energy function is consisted of unary and pairwise potentials. The unary terms are computed by SVM classifictaion. The initial labeling is mainly processed through geometrical shapes. The pairwise potential is estimated by Naïve Bayes. From training data, the probability of adjacent objects is computed by prior knowledge. The final labeling method is reweighted message-passing to minimization the energy function. The MRF model is difficult to process the large-scale misclassification. We propose a super-voxel clustering method for over-segment and grouping segment for large objects. Trees, poles ground, and building are classified in this paper. The experimental results show that this method improves the accuracy of classification and speed of computation.

Reveal Difference in Synthetic Aperture Radar Images Implementing Fuzzy Clustering Along With Improved MRF Energy Function and Wavelet Denoising Technique

In this paper we put forward a novel method to reveal difference in synthetic aperture radar (SAR) images. In this approach we classify the changed and unchanged region by the help of the fuzzy c-means (FCM) clustering along with the use of an Improved Markov random field (MRF) as energy function. It is important to deal with speckle noise that is found in SAR images. So we use Improved MRF energy function along with FCM and a Wavelet denoising technique to reduce the speckle noise found in the SAR images. In this we use two methodologies for the detection of change in synthetic aperture radar (SAR) images. First, we apply the Wavelet Bayesian denoising technique to reduce the speckle noise. Then the image fusion technique is introduced to generate a difference image by using complementary information from a mean-ratio image and a log-ratio image. Later with the help of FCM and the improved MRF we detect the variation in the SAR images. The main advantage of the proposed method is its dominance in reducing speckle noises and its computational simplicity.

Progressive completion of a panoramic image

This paper proposes an exemplar-based modified priority belief propagation (MP-BP) method to complete a stitched panorama. The result is an image with a rectangular boundary in which the missing area is filled by a visually plausible background that copies the appearance of the source region. Global optimization is usually preferable to greedy algorithms for image completion. Komodakis (IEEE Trans. Image Process. 16 (11): 2649---2661, 2007) proposed a priority BP method, which is expressed in the form of a discrete global optimization problem with an MRF energy function. However, this priority BP method cannot be directly applied to stitched panoramas completing problem because there are very few useful messages. Instead, a progressive way to expand the boundary is proposed. The main contributions of this study are: (1) it defines a progressive way in which to apply a Markov Random Field (MRF) model to complete panoramas; (2) it defines a priority term that integrates the concepts of "confidence" and "breadth first", to guide the filling order; and (3) a restricted source region and clustered candidate patches are used, to alleviate computation complexity. A number of examples of real stitched panoramas demonstrate the effectiveness of this algorithm. The results compare favorably with those obtained using existing techniques.

Multichannel DEM reconstruction method based on Markov random fields for bistatic SAR

It appears very interesting and particularly useful to perform DEM reconstruction with the config- uration of multichannel bistatic SAR (BiSAR) because of its considerable advantages. In typical processing flow of multichannel interferometric synthetic aperture radar (InSAR), techniques based on statistical methods or iterative methods does not work very well with limits of specific BiSAR configurations. Moreover, the tradeoff between the correctness and precision of the reconstruction of different baselines needs to be overcome as well. Based on the advantages of multichannel BiSAR configuration, basic idea is the joint utilization of amplitude data, iterative reconstruction results and ML results to help the optimization of the total MRF energy function. Thus, a multichannel DEM reconstruction method based on Markov Random Fields (MRFs) for BiSAR, which concerns much about discontinuities and energy functions, is proposed to achieve not only correct but also more precise results. Afterward, the strategy to determine the weight parameter and processing flow is presented. Finally, simulated and real data experiments are given to validate the proposed method.

Glaucoma progression detection using nonlocal Markov random field prior.

Glaucoma is neurodegenerative disease characterized by distinctive changes in the optic nerve head and visual field. Without treatment, glaucoma can lead to permanent blindness. Therefore, monitoring glaucoma progression is important to detect uncontrolled disease and the possible need for therapy advancement. In this context, three-dimensional (3-D) spectral domain optical coherence tomography (SD-OCT) has been commonly used in the diagnosis and management of glaucoma patients. We present a new framework for detection of glaucoma progression using 3-D SD-OCT images. In contrast to previous works that use the retinal nerve fiber layer thickness measurement provided by commercially available instruments, we consider the whole 3-D volume for change detection. To account for the spatial voxel dependency, we propose the use of the Markov random field (MRF) model as a prior for the change detection map. In order to improve the robustness of the proposed approach, a nonlocal strategy was adopted to define the MRF energy function. To accommodate the presence of false-positive detection, we used a fuzzy logic approach to classify a 3-D SD-OCT image into a "non-progressing" or "progressing" glaucoma class. We compared the diagnostic performance of the proposed framework to the existing methods of progression detection.

An Efficient MRF Embedded Level Set Method for Image Segmentation

This paper presents a fast and robust level set method for image segmentation. To enhance the robustness against noise, we embed a Markov random field (MRF) energy function to the conventional level set energy function. This MRF energy function builds the correlation of a pixel with its neighbors and encourages them to fall into the same region. To obtain a fast implementation of the MRF embedded level set model, we explore algebraic multigrid (AMG) and sparse field method (SFM) to increase the time step and decrease the computation domain, respectively. Both AMG and SFM can be conducted in a parallel fashion, which facilitates the processing of our method for big image databases. By comparing the proposed fast and robust level set method with the standard level set method and its popular variants on noisy synthetic images, synthetic aperture radar (SAR) images, medical images, and natural images, we comprehensively demonstrate the new method is robust against various kinds of noises. In particular, the new level set method can segment an image of size 500 × 500 within 3 s on MATLAB R2010b installed in a computer with 3.30-GHz CPU and 4-GB memory.

SVM-MRF segmentation of colorectal NBI endoscopic images.

In this paper we investigate a method for segmentation of colorectal Narrow Band Imaging (NBI) endoscopic images with Support Vector Machine (SVM) and Markov Random Field (MRF). SVM classifiers recognize each square patch of an NBI image and output posterior probabilities that represent how likely the given patch falls into a certain label. To prevent the spatial inconsistency between adjacent patches and encourage segmented regions to have smoother shapes, MRF is introduced by using the posterior outputs of SVMs as a unary term of MRF energy function. Segmentation results of 1191 NBI images are evaluated in experiments in which SVMs were trained with 480 trimmed NBI images and the MRF energy was minimized by an α - β swap Graph Cut.

Fuzzy Clustering With a Modified MRF Energy Function for Change Detection in Synthetic Aperture Radar Images

In this paper, we put forward a novel approach for change detection in synthetic aperture radar (SAR) images. The approach classifies changed and unchanged regions by fuzzy c-means (FCM) clustering with a novel Markov random field (MRF) energy function. In order to reduce the effect of speckle noise, a novel form of the MRF energy function with an additional term is established to modify the membership of each pixel. In addition, the degree of modification is determined by the relationship of the neighborhood pixels. The specific form of the additional term is contingent upon different situations, and it is established ultimately by utilizing the least-square method. There are two aspects to our contributions. First, in order to reduce the effect of speckle noise, the proposed approach focuses on modifying the membership instead of modifying the objective function. It is computationally simple in all the steps involved. Its objective function can just return to the original form of FCM, which leads to its consuming less time than that of some obviously recently improved FCM algorithms. Second, the proposed approach modifies the membership of each pixel according to a novel form of the MRF energy function through which the neighbors of each pixel, as well as their relationship, are concerned. Theoretical analysis and experimental results on real SAR datasets show that the proposed approach can detect the real changes as well as mitigate the effect of speckle noises. Theoretical analysis and experiments also demonstrate its low time complexity.

Clustering based on enhanced α-expansion move

The exemplar-based data clustering problem can be formulated as minimizing an energy function defined on a Markov random field (MRF). However, most algorithms for optimizing the MRF energy function cannot be directly applied to the task of clustering, as the problem has a high-order energy function. In this paper, we first show that the high-order energy function for the clustering problem can be simplified as a pairwise energy function with the metric property, and consequently it can be optimized by the α-expansion move algorithm based on graph cut. Then, the original expansion move algorithm is improved in the following two aspects: 1) Instead of solving a minimal s-t graph cut problem, we show that there is an explicit and interpretable solution for minimizing the energy function in the clustering problem. Based on this interpretation, a fast α-expansion move algorithm is proposed, which is much more efficient than the graph-cut-based algorithm. 2) The fast α-expansion move algorithm is further improved by extending its move space so that a larger energy value reduction can be achieved in each iteration. Experiments on benchmark data sets show that the enhanced expansion move algorithm has a better performance, compared to other state-of-the-art exemplar-based clustering algorithms.

A Semisupervised Context-Sensitive Change Detection Technique via Gaussian Process

In this letter, we propose a semisupervised context-sensitive technique for change detection in high-resolution multitemporal remote sensing images. This is achieved by analyzing the posterior probability of probabilistic Gaussian process (GP) classifier within a Markov random field (MRF) model. In particular, the method consists of two steps: 1) A semisupervised initialization exploits both labeled and unlabeled data based on a probabilistic GP classifier, and 2) an MRF regularization aims at refining the posterior probability by employing the spatial context information. In particular, both edge information and high-order potential are utilized in MRF energy function formulation. Experimental results obtained on real remote sensing multitemporal imagery data sets confirm the effectiveness of the proposed approach.