Pairwise Conditional Random Field Research Articles

Efficient JPEG image steganography using pairwise conditional random field model

Recent research shows that the security performance of JPEG image steganography can be improved by developing an accurate statistical model to characterize the correlations between Discrete Cosine Transform (DCT) coefficients. In this paper, a novel JPEG steganographic scheme called J-CRF is proposed to characterize the correlations between DCT coefficients using a pairwise Conditional Random Field (CRF) model, in which the correlated DCT coefficient pairs are modeled as bivariate Gaussian random variables. Under the framework of pairwise CRF, the proposed scheme formulates the problem of minimizing the statistical detectability within the Gaussian image model as the minimization of the sum of a series of unary and pairwise potentials, both of which are utilized to measure the local detectability of embedding messages in the DCT coefficient pairs. All the potentials associated with statistical detectability are formalized as the KL divergence between the statistical distributions of the cover and the stego. Experimental results demonstrate that the proposed J-CRF can outperform other SOTA JPEG steganographic schemes, e.g., J-UNIWARD, J-MiPOD, and JEC-RL, in resisting the detection of various advanced steganalyzers. In addition, the performance of J-CRF could be further boosted by reformulating it as the weighted sum of unary and pairwise potentials, called J-CRFW.

CRFalign: A Sequence-Structure Alignment of Proteins Based on a Combination of HMM-HMM Comparison and Conditional Random Fields.

Sequence–structure alignment for protein sequences is an important task for the template-based modeling of 3D structures of proteins. Building a reliable sequence–structure alignment is a challenging problem, especially for remote homologue target proteins. We built a method of sequence–structure alignment called CRFalign, which improves upon a base alignment model based on HMM-HMM comparison by employing pairwise conditional random fields in combination with nonlinear scoring functions of structural and sequence features. Nonlinear scoring part is implemented by a set of gradient boosted regression trees. In addition to sequence profile features, various position-dependent structural features are employed including secondary structures and solvent accessibilities. Training is performed on reference alignments at superfamily levels or twilight zone chosen from the SABmark benchmark set. We found that CRFalign method produces relative improvement in terms of average alignment accuracies for validation sets of SABmark benchmark. We also tested CRFalign on 51 sequence–structure pairs involving 15 FM target domains of CASP14, where we could see that CRFalign leads to an improvement in average modeling accuracies in these hard targets (TM-CRFalign ) compared with that of HHalign (TM-HHalign ) and also that of MRFalign (TM-MRFalign ). CRFalign was incorporated to our template search framework called CRFpred and was tested for a random target set of 300 target proteins consisting of Easy, Medium and Hard sets which showed a reasonable template search performance.

Building Footprint Generation by Integrating Convolution Neural Network With Feature Pairwise Conditional Random Field (FPCRF)

Building footprint maps are vital to many remote sensing applications, such as 3D building modeling, urban planning, and disaster management. Due to the complexity of buildings, the accurate and reliable generation of the building footprint from remote sensing imagery is still a challenging task. In this work, an end-to-end building footprint generation approach that integrates convolution neural network (CNN) and graph model is proposed. CNN serves as the feature extractor, while the graph model can take spatial correlation into consideration. Moreover, we propose to implement the feature pairwise conditional random field (FPCRF) as a graph model to preserve sharp boundaries and fine-grained segmentation. Experiments are conducted on four different datasets: (1) Planetscope satellite imagery of the cities of Munich, Paris, Rome, and Zurich; (2) ISPRS benchmark data from the city of Potsdam, (3) Dstl Kaggle dataset; and (4) Inria Aerial Image Labeling data of Austin, Chicago, Kitsap County, Western Tyrol, and Vienna. It is found that the proposed end-to-end building footprint generation framework with the FPCRF as the graph model can further improve the accuracy of building footprint generation by using only CNN, which is the current state-of-the-art.

Spatial–Spectral Fusion Based on Conditional Random Fields for the Fine Classification of Crops in UAV-Borne Hyperspectral Remote Sensing Imagery

The fine classification of crops is critical for food security and agricultural management. There are many different species of crops, some of which have similar spectral curves. As a result, the precise classification of crops is a difficult task. Although the classification methods that incorporate spatial information can reduce the noise and improve the classification accuracy, to a certain extent, the problem is far from solved. Therefore, in this paper, the method of spatial–spectral fusion based on conditional random fields (SSF-CRF) for the fine classification of crops in UAV-borne hyperspectral remote sensing imagery is presented. The proposed method designs suitable potential functions in a pairwise conditional random field model, fusing the spectral and spatial features to reduce the spectral variation within the homogenous regions and accurately identify the crops. The experiments on hyperspectral datasets of the cities of Hanchuan and Honghu in China showed that, compared with the traditional methods, the proposed classification method can effectively improve the classification accuracy, protect the edges and shapes of the features, and relieve excessive smoothing, while retaining detailed information. This method has important significance for the fine classification of crops in hyperspectral remote sensing imagery.

Efficient Graph Cut Optimization for Full CRFs with Quantized Edges.

Fully connected pairwise Conditional Random Fields (Full-CRF) with Gaussian edge weights can achieve superior results compared to sparsely connected CRFs. However, traditional methods for Full-CRFs are too expensive. Previous work develops efficient approximate optimization based on mean field inference, which is a local optimization method and can be far from the optimum. We propose efficient and effective optimization based on graph cuts for Full-CRFs with quantized edge weights. To quantize edge weights, we partition the image into superpixels and assume that the weight of an edge between any two pixels depends only on the superpixels these pixels belong to. Our quantized edge CRF is an approximation to the Gaussian edge CRF, and gets closer to it as superpixel size decreases. Being an approximation, our model offers an intuition about the regularization properties of the Guassian edge Full-CRF. For efficient inference, we first consider the two-label case and develop an approximate method based on transforming the original problem into a smaller domain. Then we handle multi-label CRF by showing how to implement expansion moves. In both binary and multi-label cases, our solutions have significantly lower energy compared to that of mean field inference. We also show the effectiveness of our approach on semantic segmentation task.

Color image splicing localization algorithm by quaternion fully convolutional networks and superpixel-enhanced pairwise conditional random field.

Recently, fully convolutional network (FCN) has been successfully used to locate spliced regions in synthesized images. However, all the existing FCN-based algorithms use real-valued FCN to process each channel separately. As a consequence, they fail to capture the inherent correlation between color channels and the integrity of three channels. So, in this paper, quaternion fully convolutional network (QFCN) is proposed to generalize FCN to quaternion domain by replacing real-valued conventional blocks in FCN with quaternion conventional blocks. In addition, a new color image splicing localization algorithm is proposed by combining QFCNs and superpixel (SP)-enhanced pairwise conditional random field (CRF). QFCNs consider three different versions (QFCN32, QFCN16, and QFCN8) with different up-sampling layers. The SP-enhanced pairwise CRF is used to refine the results of QFCNs. Experimental results on three publicly available datasets demonstrate that the proposed algorithm outperforms the existing algorithms including some conventional algorithms and some deep learning-based algorithms.

Depth Reconstruction from Single Images Using a Convolutional Neural Network and a Condition Random Field Model.

This paper presents an effective approach for depth reconstruction from a single image through the incorporation of semantic information and local details from the image. A unified framework for depth acquisition is constructed by joining a deep Convolutional Neural Network (CNN) and a continuous pairwise Conditional Random Field (CRF) model. Semantic information and relative depth trends of local regions inside the image are integrated into the framework. A deep CNN network is firstly used to automatically learn a hierarchical feature representation of the image. To get more local details in the image, the relative depth trends of local regions are incorporated into the network. Combined with semantic information of the image, a continuous pairwise CRF is then established and is used as the loss function of the unified model. Experiments on real scenes demonstrate that the proposed approach is effective and that the approach obtains satisfactory results.

Revisiting Deep Structured Models for Pixel-Level Labeling with Gradient-Based Inference

Semantic segmentation and other pixel-level labeling tasks have made significant progress recently due to the deep learning paradigm. Many state-of-the-art structured prediction methods also include a random field model with a hand-crafted Gaussian potential to model spatial priors and label consistencies and feature-based image conditioning. These random field models with image conditioning typically require computationally demanding filtering techniques during inference. In this paper, we present a new inference and learning framework which can learn arbitrary pairwise conditional random field (CRF) potentials. Both standard spatial and high-dimensional bilateral kernels are considered. In addition, we introduce a new type of potential function which is image-dependent like the bilateral kernel, but an order of magnitude faster to compute since only spatial convolutions are employed. It is empirically demonstrated that such learned potentials can improve segmentation accuracy and that certain label-class interactions are indeed better modeled by a non-Gaussian potential. Our framework is evaluated on several public benchmarks for semantic segmentation with improved performance compared to previous state-of-the-art CNN+CRF models.

Multiple human 3D pose estimation from multiview images

Multiple human 3D pose estimation is a challenging task. It is mainly because of large variations in the scale and pose of humans, fast motions, multiple persons in the scene, and arbitrary number of visible body parts due to occlusion or truncation. Some of these ambiguities can be resolved by using multiview images. This is due to the fact that more evidences of body parts would be available in multiple views. In this work, a novel method for multiple human 3D pose estimation using evidences in multiview images is proposed. The proposed method utilizes a fully connected pairwise conditional random field that contains two types of pairwise terms. The first pairwise term encodes the spatial dependencies among human body joints based on an articulated human body configuration. The second pairwise term is based on the output of a 2D deep part detector. An approximate inference is then performed using the loopy belief propagation algorithm. The proposed method is evaluated on the Campus, Shelf, Utrecht Multi-Person Motion benchmark, Human3.6M, KTH Football II, and MPII Cooking datasets. Experimental results indicate that the proposed method achieves substantial improvements over the existing state-of-the-art methods in terms of the probability of correct pose and the mean per joint position error performance measures.

Supervised training and contextually guided salient object detection

Existing salient object detection research generally focused on designing diverse saliency features and integrating them heuristically. In this paper, a novel salient object detection method is proposed by employing supervised training and contextual modeling. Gradient boosting decision trees are explored to aggregate features on segmented regions using a supervised training manner. Feature representation of hierarchically segmented regions is exploited to capture salient objects at different levels so as to extract discriminative features better. A region-level pairwise conditional random field (CRF) method is constructed to further boost the accuracy of saliency estimation as well as to improve the perceptual consistency of saliency maps. Experimental results demonstrate that the proposed method could achieve state-of-the-art performance over all public datasets. The F-measure is improved by 3.9%, 13.0%, 4.3% on the MSRA-B, DUT-OMRON and HKU-IS dataset respectively, and the mean absolute error (MAE) is reduced by 31.6%, 26.4% and 21.2% respectively on these three datasets.

Exploiting Large Image Sets for Road Scene Parsing

There is an increasing interest in exploiting multiple images for scene understanding, with great progress in areas such as cosegmentation and video segmentation. Jointly analyzing the images in a large set offers the opportunity to exploit a greater source of information than when considering a single image on its own. However, this also yields challenges since, to effectively exploit all the available information, the resulting methods need to consider not just local connections, but efficiently analyze similarity between all pairs of pixels within and across all the images. In this paper, we propose to model an image set as a fully connected pairwise Conditional Random Field (CRF) defined over the image pixels, or superpixels, with Gaussian edge potentials. We show that this lets us co-label the images of a large set efficiently, thus yielding increased accuracy at no additional computational cost compared to sequential labeling of the images. Furthermore, we extend our framework to incorporate temporal dependence, thus effectively encompassing video segmentation as a special case of our approach, as well as to modeling label dependence over larger image regions. Our experimental evaluation demonstrates that our framework lets us handle over 10 000 images in a matter of seconds.

Adaptivity of conditional random field based outdoor point cloud classification

In this paper we present how adaptable learned models of graphical models are and how they can be used for classification tasks of 3D laser point clouds with different distributions and density. In order to model the contextual information we use a pair-wise conditional random field and an adaptive graph down-sampling method based on voxel grids. As feature we apply the rotation invariant histogram-of-oriented-residuals operator to describe the local point cloud distribution. We validate the approach with data collected from different laser range finders with varying point cloud distribution and density. Our experiments imply, that conditional random field models learned from one dataset can be applied to another dataset without a significant loss of precision.

Mapping of Agricultural Crops from Single High-Resolution Multispectral Images—Data-Driven Smoothing vs. Parcel-Based Smoothing

Mapping agricultural crops is an important application of remote sensing. However, in many cases it is based either on hyperspectral imagery or on multitemporal coverage, both of which are difficult to scale up to large-scale deployment at high spatial resolution. In the present paper, we evaluate the possibility of crop classification based on single images from very high-resolution (VHR) satellite sensors. The main objective of this work is to expose performance difference between state-of-the-art parcel-based smoothing and purely data-driven conditional random field (CRF) smoothing, which is yet unknown. To fulfill this objective, we perform extensive tests with four different classification methods (Support Vector Machines, Random Forest, Gaussian Mixtures, and Maximum Likelihood) to compute the pixel-wise data term; and we also test two different definitions of the pairwise smoothness term. We have performed a detailed evaluation on different multispectral VHR images (Ikonos, QuickBird, Kompsat-2). The main finding of this study is that pairwise CRF smoothing comes close to the state-of-the-art parcel-based method that requires parcel boundaries (average difference ≈ 2.5%). Our results indicate that a single multispectral (R, G, B, NIR) image is enough to reach satisfactory classification accuracy for six crop classes (corn, pasture, rice, sugar beet, wheat, and tomato) in Mediterranean climate. Overall, it appears that crop mapping using only one-shot VHR imagery taken at the right time may be a viable alternative, especially since high-resolution multitemporal or hyperspectral coverage as well as parcel boundaries are in practice often not available.

Detail-Preserving Smoothing Classifier Based on Conditional Random Fields for High Spatial Resolution Remote Sensing Imagery

In the field of high spatial resolution (HSR) remote sensing imagery classification, object-oriented classification and conditional random field (CRF) approaches are widely used due to their ability to incorporate the spatial contextual information. However, the selection of the optimal segmentation scale in object-oriented classification is not an easy task, and some pairwise CRF models always show an oversmooth performance. In this paper, a detail-preserving smoothing classifier based on conditional random fields (DPSCRF) for HSR imagery is proposed to apply the object-oriented strategy in the CRF classification framework, thus integrating the merits of both approaches to consider the spatial contextual information and preserve the detail information in the classification. The DPSCRF model defines suitable potential functions based on the CRF model for HSR image classification, which comprise the spatial smoothing and local class label cost terms. Both terms favor spatial smoothing in a local neighborhood to consider the spatial information. In addition, the local class label cost also considers the different label information of neighboring pixels at each iterative step in the classification to preserve the detail information. In order to deal with the spectral variability of HSR imagery, a segmentation prior is used by the object-oriented processing strategy. This models the probability of each pixel based on the segmentation regions obtained by the connected-component labeling algorithm. The experimental results with three HSR images demonstrate that the proposed classification algorithm shows a competitive performance in both the quantitative and the qualitative evaluation when compared to the other state-of-the-art classification algorithms.

A Hybrid Object-Oriented Conditional Random Field Classification Framework for High Spatial Resolution Remote Sensing Imagery

High spatial resolution (HSR) remote sensing imagery provides abundant geometric and detailed information, which is important for classification. In order to make full use of the spatial contextual information, object-oriented classification and pairwise conditional random fields (CRFs) are widely used. However, the segmentation scale choice is a challenging problem in object-oriented classification, and the classification result of pairwise CRF always has an oversmooth appearance. In this paper, a hybrid object-oriented CRF classification framework for HSR imagery, namely, CRF $+$ OO, is proposed to address these problems by integrating object-oriented classification and CRF classification. In CRF $+$ OO, a probabilistic pixel classification is first performed, and then, the classification results of two CRF models with different potential functions are used to obtain the segmentation map by a connected-component labeling algorithm. As a result, an object-level classification fusion scheme can be used, which integrates the object-oriented classifications using a majority voting strategy at the object level to obtain the final classification result. The experimental results using two multispectral HSR images (QuickBird and IKONOS) and a hyperspectral HSR image (HYDICE) demonstrate that the proposed classification framework has a competitive quantitative and qualitative performance for HSR image classification when compared with other state-of-the-art classification algorithms.

CRF-based models of protein surfaces improve protein-protein interaction site predictions.

BackgroundThe identification of protein-protein interaction sites is a computationally challenging task and important for understanding the biology of protein complexes. There is a rich literature in this field. A broad class of approaches assign to each candidate residue a real-valued score that measures how likely it is that the residue belongs to the interface. The prediction is obtained by thresholding this score.Some probabilistic models classify the residues on the basis of the posterior probabilities. In this paper, we introduce pairwise conditional random fields (pCRFs) in which edges are not restricted to the backbone as in the case of linear-chain CRFs utilized by Li et al. (2007). In fact, any 3D-neighborhood relation can be modeled. On grounds of a generalized Viterbi inference algorithm and a piecewise training process for pCRFs, we demonstrate how to utilize pCRFs to enhance a given residue-wise score-based protein-protein interface predictor on the surface of the protein under study. The features of the pCRF are solely based on the interface predictions scores of the predictor the performance of which shall be improved.ResultsWe performed three sets of experiments with synthetic scores assigned to the surface residues of proteins taken from the data set PlaneDimers compiled by Zellner et al. (2011), from the list published by Keskin et al. (2004) and from the very recent data set due to Cukuroglu et al. (2014). That way we demonstrated that our pCRF-based enhancer is effective given the interface residue score distribution and the non-interface residue score are unimodal.Moreover, the pCRF-based enhancer is also successfully applicable, if the distributions are only unimodal over a certain sub-domain. The improvement is then restricted to that domain. Thus we were able to improve the prediction of the PresCont server devised by Zellner et al. (2011) on PlaneDimers.ConclusionsOur results strongly suggest that pCRFs form a methodological framework to improve residue-wise score-based protein-protein interface predictors given the scores are appropriately distributed. A prototypical implementation of our method is accessible at http://ppicrf.informatik.uni-goettingen.de/index.html.

An Optimization-based Framework to Learn Conditional Random Fields for Multi-label Classification.

This paper studies multi-label classification problem in which data instances are associated with multiple, possibly high-dimensional, label vectors. This problem is especially challenging when labels are dependent and one cannot decompose the problem into a set of independent classification problems. To address the problem and properly represent label dependencies we propose and study a pairwise conditional random Field (CRF) model. We develop a new approach for learning the structure and parameters of the CRF from data. The approach maximizes the pseudo likelihood of observed labels and relies on the fast proximal gradient descend for learning the structure and limited memory BFGS for learning the parameters of the model. Empirical results on several datasets show that our approach outperforms several multi-label classification baselines, including recently published state-of-the-art methods.

Quantification of Relative Movement Uncertainty Based on Laser Scan Matching and Graphical Model for Mobile Robot

摘要: 提出激光束匹配问题中移动机器人相对运动估计误差的分析方法. 以概率图模型建模激光点几何特征, 构建链式成双条件随机场推理激光点关联的边缘概率. 通过对关联配置空间的节点和状态限定, 紧凑采样激光束匹配不确定性概率分布, 并映射于相对运动估计空间, 推理相对运动估计误差. 实验验证了算法的有效性. 关键词: 激光束匹配 / 不确定性估计 / 条件随机场 / 图模型

Integrating Motion and Segmentation for Road Scene Labeling

Structure from motion (SfM) and appearance-based segmentation have played an important role in the interpretation of road scenes. The integration of these approaches can lead to good performance during interpretation since the relation between 3D spatial structure and 2D semantic segmentation can be taken into account. This paper presents a new integration framework using an SfM module and a bag of textons method for road scene labeling. By using a multiband image, which consists of a near-infrared and a visible color image, we can generate better discriminative textons than those generated by using only a color image. Our SfM module can accurately estimate the ego motion of the vehicle and reconstruct a 3D structure of the road scene. The bag of textons is computed over local rectangular regions: its size depends on the distance of the textons. Therefore, the 3D bag of textons method can help to effectively recognize the objects of a road scene because it considers the object's 3D structure. For solving the labeling problem, we employ a pairwise conditional random field (CRF) model. The unary potential of the CRF model is affected by SfM results, and the pairwise potential is optimized by the multiband image intensity. Experimental results show that the proposed method can effectively classify the objects in a 2D road scene with 3D structures. The proposed system can revolutionize 3D scene understanding systems used for vehicle environment perception.