Pairwise Pixel Research Articles

BoNuS: Boundary Mining for Nuclei Segmentation With Partial Point Labels.

Nuclei segmentation is a fundamental prerequisite in the digital pathology workflow. The development of automated methods for nuclei segmentation enables quantitative analysis of the wide existence and large variances in nuclei morphometry in histopathology images. However, manual annotation of tens of thousands of nuclei is tedious and time-consuming, which requires significant amount of human effort and domain-specific expertise. To alleviate this problem, in this paper, we propose a weakly-supervised nuclei segmentation method that only requires partial point labels of nuclei. Specifically, we propose a novel boundary mining framework for nuclei segmentation, named BoNuS, which simultaneously learns nuclei interior and boundary information from the point labels. To achieve this goal, we propose a novel boundary mining loss, which guides the model to learn the boundary information by exploring the pairwise pixel affinity in a multiple-instance learning manner. Then, we consider a more challenging problem, i.e., partial point label, where we propose a nuclei detection module with curriculum learning to detect the missing nuclei with prior morphological knowledge. The proposed method is validated on three public datasets, MoNuSeg, CPM, and CoNIC datasets. Experimental results demonstrate the superior performance of our method to the state-of-the-art weakly-supervised nuclei segmentation methods. Code: https://github.com/hust-linyi/bonus.

Adaptive hypergraph superpixels

Superpixel segmentation, which amounts to partitioning an image into a number of superpixels each of which is a set of pixels sharing common visual meanings, requires specific needs for different computer vision tasks. Graph based methods, as a kind of popular superpixel segmentation method, regard an image as a weighted graph whose nodes correspond to pixels of the image, and partition all pixels into superpixels according to the similarity between pixels over various feature spaces. Despite their improvement of the performance of segmentation, these methods ignore high-order relationship between them incurred from either locally neighboring pixels or structured layout of the image. Moreover, they measure the similarity of pairwise pixels using Gaussian kernel where a robust radius parameter is difficult to find for pixels which exhibit multiple features (e.g., texture, color, brightness). In this paper, we propose an adaptive hypergraph superpixel segmentation (AHS) of intensity images for solving both issues. AHS constructs a hypergraph by building the hyperedges with an adaptive neighborhood scheme, which explores an intrinsic relationship of pixels. Afterwards, AHS encodes the relationship between pairwise pixels using characteristics of current two pixels as well as their neighboring pixels defined by hyperedges. Essentially, AHS models the relationship of pairwise pixels in a high-order group fashion while graph based methods evaluate it in a one-vs-one fashion. Experiments on four datasets demonstrate that AHS achieves higher or comparable performance compared with state-of-the-art methods.

Spectral–Spatial Feature Extraction With Dual Graph Autoencoder for Hyperspectral Image Clustering

Autoencoder (AE) is an unsupervised neural network framework for efficient and effective feature extraction. Most AE-based methods do not consider spatial information and band correlations for hyperspectral image (HSI) analysis. In addition, graph-based AE methods often learn discriminative representations with the assumption that connected samples share the same label and they cannot directly embed the geometric structure into feature extraction. To address these issues, in this paper, we propose a dual graph autoencoder (DGAE) to learn discriminative representations for HSIs. Utilizing the relationships of pair-wise pixels within homogenous regions and pair-wise spectral bands, DGAE first constructs the superpixel-based similarity graph with spatial information and band-based similarity graph to characterize the geometric structures of HSIs. With the developed dual graph convolution, more discriminative feature representations are learnt from the hidden layer via the encoder-decoder structure of DGAE. The main advantage of DGAE is that it fully exploits both the geometric structures of pixels with spatial information and spectral bands to promote nonlinear feature extraction of HSIs. Experiments on HSI datasets show the superiority of the proposed DGAE over the state-of-the-art methods. The source code of DGAE is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ZhangYongshan/DGAE</uri> .

Novel Automatic Approach for Land Cover Change Detection by Using VHR Remote Sensing Images

Many land cover change detection (LCCD) approaches applied on very high resolution (VHR) remote sensing images utilize spatial information by using a regular window or strict mathematical model. However, regular shape or strict models cannot fit the various shapes and sizes of the ground targets. In this article, a novel LCCD approach without the parameter is proposed to detect land cover change with VHR remote sensing images. First, an adaptive spatial-context extraction algorithm is applied to explore contextual information around a pixel. Second, the change magnitude between pairwise pixels is quantitatively measured by computing the band-to-band distance which is defined by the pairwise adaptive regions around the corresponding pixels. Finally, after the generation of a change magnitude image (CMI), a binary threshold method called double-window flexible pace search (DFPS) is adopted to divide CMI into a binary change detection map. The performance of the proposed approach is verified by comparing it with five state-of-the-art methods with three pairs of VHR images. The comparisons demonstrated that the proposed approach achieved the improved detected results comparing with state-of-the-art LCCD methods. The code of the proposed approach is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/TongfeiLiu/ASEA-CD</uri> .

Early crack detection using modified spectral clustering method assisted with FE analysis for distress anticipation in cement-based composites

The present work reports an efficient way of capturing real-time crack propagation in concrete structures. The modified spectral analysis based algorithm and finite element modeling (FEM) were utilised for crack detection and quantitative analysis of crack propagation. Crack propagation was captured in cement-based composite (CBC) containing saw dust and M20 grade concrete under compressive loading using a simple and inexpensive 8-megapixel mobile phone camera. The randomly selected images showing crack initiation and propagation in CBCs demonstrated the crack capturing capability of developed algorithm. A measure of oriented energy was provided at crack edges to develop a similarity spatial relationship among the pairwise pixels. FE modelling was used for distress anticipation, by analysing stresses during the compressive test in constituents of CBCs. FE modeling jointly with the developed algorithm, can provide real-time inputs from the crack-prone areas and useful in early crack detection of concrete structures for preventive support and management.

Affinity Space Adaptation for Semantic Segmentation Across Domains.

Semantic segmentation with dense pixel-wise annotation has achieved excellent performance thanks to deep learning. However, the generalization of semantic segmentation in the wild remains challenging. In this paper, we address the problem of unsupervised domain adaptation (UDA) in semantic segmentation. Motivated by the fact that source and target domain have invariant semantic structures, we propose to exploit such invariance across domains by leveraging co-occurring patterns between pairwise pixels in the output of structured semantic segmentation. This is different from most existing approaches that attempt to adapt domains based on individual pixel-wise information in image, feature, or output level. Specifically, we perform domain adaptation on the affinity relationship between adjacent pixels termed affinity space of source and target domain. To this end, we develop two affinity space adaptation strategies: affinity space cleaning and adversarial affinity space alignment. Extensive experiments demonstrate that the proposed method achieves superior performance against some state-of-the-art methods on several challenging benchmarks for semantic segmentation across domains. The code is available at https://github.com/idealwei/ASANet.

Local Histogram-Based Analysis for Detecting Land Cover Change Using VHR Remote Sensing Images

The majority of the change detection (CD) methods consider spatial information by using a regular window or strict mathematical model. Moreover, these methods use the spectra directly to measure the change magnitude between bitemporal images. To solve this problem, local histogram-based analysis (LHBA) is proposed for detecting a land cover change in this letter. This new approach aims to inhibit the pseudo change by defining the local histogram trend (LHT) in an adaptive manner instead of using spectral values to measure change magnitude directly. In the proposed approach, the spatial information around each pixel is first exploited by defining an adaptive local histogram. The LHT distance between the pairwise local histograms is then developed to measure the change magnitude between the pairwise pixels of bitemporal images. Finally, the change magnitude image is generated, and a binary CD is achieved by a threshold method. Experiments based on two pairs of very high-resolution remote sensing images, which refer to land use change and landslides events, demonstrate the advantages and performance of the proposed approach.

Enhanced pairwise IPVO-based reversible data hiding scheme using rhombus context

Pixel-Value-Ordering (PVO) based reversible data hiding schemes provide high-fidelity stego-images with moderate embedding capacity. In this paper, a novel reversible data hiding scheme based on enhanced pairwise pixel value ordering is proposed to increase embedding capacity. The proposed scheme divides a host image into non-overlapping blocks of three pixels in zig-zag order and categorizes the blocks into two categories namely absolute smooth and probably complex based on the rhombus mean for each pixel of the block. In the case of absolute smooth, the pairwise embedding is done in two-pass using two different predictors. In the pass-1, enhanced pairwise improved PVO method is used to embed the secret data in the block. The proposed method decreases the minimum valued pixels and increases the maximum valued pixels for embedding the secret data. In the pass-2, secret data is hidden using a novel recovery based pairwise embedding strategy, where the value of the first pixel according to the sorted mean sequence will be either increased or remains unchanged while the value of the last pixel will be either decreased or remains unchanged. Additionally, the medium pixel is also used to embed the secret data by modifying its value in any direction using prediction error expansion. Therefore, the pass-2 embedding is done in such a way that it becomes the complement of pass-1 embedding so that stego-image quality can be maintained with additional embedded data. In case of probable complex blocks, the secret data is embedded in pixel wise manner to local complexity of each pixel. The use of rhombus mean in block categorization allows embedding in each and every pixel of the host image while ensuring its reversibility. Thus, the proposed scheme significantly increases the embedding capacity. The experimental results show that the proposed scheme has almost doubled on the embedding capacity than the previous PVO-based reversible data hiding schemes. Further, it has higher PSNR at high embedding capacity.

Robust random walk for leaf segmentation

In this study, the authors focus on the task of leaf segmentation under different imaging conditions (e.g. backgrounds and shadows). A new method - robust random walk (RW) is proposed to propagate the prior of user's specified pixels. Specifically, they first employ RWs to take the relationship of pairwise pixels into consideration. A superpixel-consistent constraint is added to make the edges of segmentation smooth. Owing to the effect of illumination, some parts of a leaf surface are brighter than others and it may further harm the subsequent label propagation. To address this problem, they learn a common subspace by taking into account the illumination of local and non-local pixels. By doing so, it has good adaptability to process noise interfering and non-uniform illumination. In addition, since RW only considers the pairwise relationship of pixels, it will be sensitive to the specified and connected pixels. Thus, they further employ a log-likelihood ratio to predict the probability of a pixel belonging to the background and use it to guide the label propagation. Based on the proposed method, they can obtain a smoothed and robust leaf segmentation. Experimental results on unconstrained leaf images demonstrate the efficiency of their algorithm.

Novel Adaptive Histogram Trend Similarity Approach for Land Cover Change Detection by Using Bitemporal Very-High-Resolution Remote Sensing Images

Detecting land cover change through very-high-resolution (VHR) remote sensing images is helpful in supporting urban sustainable development, natural disaster evaluation, and environmental assessment. However, the intraclass spectral variance in VHR remote sensing images is usually larger than that of median-low remote sensing images. Furthermore, the bitemporal images are usually acquired under different atmospheric conditions, sun height, soil moisture, and other factors. Consequently, in practical applications, many pseudo changes are presented in the detected map. In this paper, an adaptive histogram trend (AHT) similarity approach is promoted to quantitatively measure the magnitude between the corresponding pixels in bitemporal images in terms of change semantic. In the proposed approach, to reduce the phenological effect on the bitemporal images of land cover change detection (LCCD), we first define the quantitative description of AHT. Second, the change magnitudes between pairwise pixels are quantitatively measured by an improved bin-to-bin (B2B) distance between the corresponding AHTs. Then, the change magnitudes between two entire bitemporal images are measured AHT-by-AHT. Finally, binary threshold methods, such as the Otsu method or the double-window flexible pace search (DFPS) method, are used to divide the change magnitude image into binary change detection maps and obtain the final change detection map. The performance of the AHT-based LCCD approach is verified by four pairs of VHR remote-sensing images that correspond to two types of real land cover change cases. The detected results based on the four pairs of bitemporal VHR images outperformed the compared state-of-the-art LCCD methods.

Subspace Clustering Constrained Sparse NMF for Hyperspectral Unmixing

As one of the most important information of hyperspectral images (HSI), spatial information is usually simulated with the similarity among pixels to enhance the unmixing performance of nonnegative matrix factorization (NMF). Nevertheless, the similarity is generally calculated based on the Euclidean distance between pairwise pixels, which is sensitive to noise and fails in capturing subspace information of hyperspectral data. In addition, it is independent of the NMF framework. In this article, we propose a novel unmixing method called subspace clustering constrained sparse NMF (SC-NMF) for hyperspectral unmixing to more accurately extract endmembers and correspond abundances. First, the nonnegative subspace clustering is embedded into the NMF framework to learn a similar graph, which takes full advantage of the characteristics of the reconstructed data itself to extract the spatial correlation of pixels for unmixing. It is noteworthy that the similar graph and NMF will be simultaneously updated. Second, to mitigate the influence of noise in HSI, only the $k$ largest values are retained in each self-expression vector. Finally, we use the idea of subspace clustering to extract endmembers by linearly combining of all pixels in spectral subspace, aiming at giving a reasonable physical significance to the endmembers. We evaluate the proposed SC-NMF on both synthetic and real hyperspectral data, and experimental results demonstrate that the proposed method is effective and superior by comparing with the state-of-the-art methods.

3D human pose estimation from range images with depth difference and geodesic distance

Depth difference, as a popularly used feature for characterizing pairwise pixels of range images, fails to precisely capture skeleton joints when human body possesses a wild and complicated articulation. As the geodesic distance of pairwise pixels is able to present a global connected property and adjacent pixels often belong to the same body component, we propose an effective and efficient framework for pose estimation from range images. Firstly, all the pixels of a range image are grouped into superpixels using an improved Simple Linear Iterative Clustering algorithm. Secondly, those superpixels are labelled as the components of a human body using the hybrid feature. Thirdly, componentwise cluster feature extraction is undertaken on skeleton joints of body components with K-means clustering algorithm. Finally, the feature points of each component are then stacked as a compact representation of human poses and mapped to the skeleton joints of a human body. Experimental results demonstrate that the proposed framework outperforms several state-of-the-art pose estimation methods.

A Novel Framework to Localize Moving Targets in Video Surveillance Systems via Spectral Clustering

Accurately localizing moving targets in each individual frame of video clips captured via an ordinary surveillance system is still challenging nowadays, provided the fact that many serious problems, including sudden illumination changes, partial or full obstacles, rigid or non-rigid targets’ shape transformations, etc., are still hard to be tackled at the current stage. In this study, a new semi-supervised offline incremental-learning framework via spectral clustering is introduced to solve the above moving target localization problem. The framework is composed of two steps. First, a computer-user interaction step is enabled on the first frame of a video clip, in order to allow the ending user to delineate a region-of-interested enclosing the interested target. Positive and negative samples are automatically determined via a stratified random sampling strategy therein. Second, a spatially-weighted metric-based measure is defined to reveal the similarity between pair-wise pixels. This similarity measure is then determined via a supervised spectral clustering technique algorithmically. The newly introduced framework is evaluated using a database composed of over 1000 frames, with comparisons towards 6 other well-established approaches for moving targets localization. Promising outcomes are demonstrated and the superiority of the new framework is suggested from the statistical point of view.

Pure endmember extraction using robust kernel archetypoid analysis for hyperspectral imagery

A robust kernel archetypoid analysis (RKADA) method is proposed to extract pure endmembers from hyperspectral imagery (HSI). The RKADA assumes that each pixel is a sparse linear mixture of all endmembers and each endmember corresponds to a real pixel in the image scene. First, it improves the re8gular archetypal analysis with a new binary sparse constraint, and the adoption of the kernel function constructs the principal convex hull in an infinite Hilbert space and enlarges the divergences between pairwise pixels. Second, the RKADA transfers the pure endmember extraction problem into an optimization problem by minimizing residual errors with the Huber loss function. The Huber loss function reduces the effects from big noises and outliers in the convergence procedure of RKADA and enhances the robustness of the optimization function. Third, the random kernel sinks for fast kernel matrix approximation and the two-stage algorithm for optimizing initial pure endmembers are utilized to improve its computational efficiency in realistic implementations of RKADA, respectively. The optimization equation of RKADA is solved by using the block coordinate descend scheme and the desired pure endmembers are finally obtained. Six state-of-the-art pure endmember extraction methods are employed to make comparisons with the RKADA on both synthetic and real Cuprite HSI datasets, including three geometrical algorithms vertex component analysis (VCA), alternative volume maximization (AVMAX) and orthogonal subspace projection (OSP), and three matrix factorization algorithms the preconditioning for successive projection algorithm (PreSPA), hierarchical clustering based on rank-two nonnegative matrix factorization (H2NMF) and self-dictionary multiple measurement vector (SDMMV). Experimental results show that the RKADA outperforms all the six methods in terms of spectral angle distance (SAD) and root-mean-square-error (RMSE). Moreover, the RKADA has short computational times in offline operations and shows significant improvement in identifying pure endmembers for ground objects with smaller spectrum differences. Therefore, the RKADA could be an alternative for pure endmember extraction from hyperspectral images.

A Probabilistic Weighted Archetypal Analysis Method with Earth Mover’s Distance for Endmember Extraction from Hyperspectral Imagery

A Probabilistic Weighted Archetypal Analysis method with Earth Mover’s Distance (PWAA-EMD) is proposed to extract endmembers from hyperspectral imagery (HSI). The PWAA-EMD first utilizes the EMD dissimilarity matrix to weight the coefficient matrix in the regular Archetypal Analysis (AA). The EMD metric considers manifold structures of spectral signatures in the HSI data and could better quantify the dissimilarity features among pairwise pixels. Second, the PWAA-EMD adopts the Bayesian framework and formulates the improved AA into a probabilistic inference problem by maximizing a joint posterior density. Third, the optimization problem is solved by the iterative multiplicative update scheme, with a careful initialization from the two-stage algorithm and the proper endmembers are finally obtained. The synthetic and real Cuprite Hyperspectral datasets are utilized to verify the performance of PWAA-EMD and five popular methods are implemented to make comparisons. The results show that PWAA-EMD surpasses all the five methods in the average results of spectral angle distance (SAD) and root-mean-square-error (RMSE). Especially, the PWAA-EMD obtains more accurate estimation than AA in almost all the classes of endmembers including two similar ones. Therefore, the PWAA-EMD could be an alternative choice for endmember extraction on the hyperspectral data.

A discriminant sparse representation graph-based semi-supervised learning for hyperspectral image classification

The classification of hyperspectral image with a paucity of labeled samples is a challenging task. In this paper, we present a discriminant sparse representation (DSR) graph for semi-supervised learning (SSL) to address this problem. For graph-based methods, how to construct a graph among the pixels is the key to a successful classification. Our graph construction method contains two steps. Sparse representation (SR) method is first employed to estimate the probability matrix of the pairwise pixels belonging to the same class, and then this probability matrix is integrated into the SR graph, which can be obtained by solving an l1 optimization problem, to form a DSR graph. Experiments on Hyperion and AVIRIS hyperspectral data show that our proposed method outperforms state of the art.

Towards binary robust fast features using the comparison of pixel blocks

Binary descriptors have become popular in many vision-based applications, as a fast and efficient replacement of floating point, heavy counterparts. They achieve a short computation time and low memory footprint due to many simplifications. Consequently, their robustness against a variety of image transformations is lowered, since they rely on pairwise pixel intensity comparisons. This observation has led to the emergence of techniques performing tests on intensities of predefined pixel regions. These approaches, despite a visible improvement in the quality of the obtained results, suffer from a long computation time, and their patch partitioning strategies produce long binary strings requiring the use of salient bit detection techniques. In this paper, a novel binary descriptor is proposed to address these shortcomings. The approach selects image patches around a keypoint, divides them into a small number of pixel blocks and performs binary tests on gradients which are determined for the blocks. The size of each patch depends on the keypoint’s scale. The robustness and distinctiveness of the descriptor are evaluated according to five demanding image benchmarks. The experimental results show that the proposed approach is faster to compute, produces a short binary string and offers a better performance than state-of-the-art binary and floating point descriptors.

Nonlocal Markovian models for image denoising

Currently, the state-of-the art methods for image denoising are patch-based approaches. Redundant information present in nonlocal regions (patches) of the image is considered for better image modeling, resulting in an improved quality of filtering. In this respect, nonlocal Markov random field (MRF) models are proposed by redefining the energy functions of classical MRF models to adopt a nonlocal approach. With the new energy functions, the pairwise pixel interaction is weighted according to the similarities between the patches corresponding to each pair. Also, a maximum pseudolikelihood estimation of the spatial dependency parameter (β) for these models is presented here. For evaluating this proposal, these models are used as an a priori model in a maximum a posteriori estimation to denoise additive white Gaussian noise in images. Finally, results display a notable improvement in both quantitative and qualitative terms in comparison with the local MRFs.

Fast Minimax Path-Based Joint Depth Interpolation

We propose a fast minimax path-based depth interpolation method. The algorithm computes for each target pixel varying contributions from reliable depth seeds, and weighted averaging is used to interpolate missing depths. Compared with state-of-the-art joint geodesic upsampling method which selects the $K$ nearest seeds to interpolate missing depths with $O(Kn)$ complexity, our method does not need to limit the number of seeds to $K$ and reduces the computational complexity to $O(n)$ . In addition, the minimax path chooses a path with the smallest maximum immediate pairwise pixel difference on it, so it tends to preserve sharp depth discontinuities better. In contrast to the results of previous depth upsampling algorithms, our approach can provide accurate depths with fewer artifacts.

WITHDRAWN: Joint depth map interpolation using minimax paths

We propose a minimax path-based depth interpolation method. The algorithm computes for each target pixel varying contributions from reliable depth seeds, and weighted averaging is used to interpolate missing depths. Compared with state-of-the-art joint geodesic upsampling method which selects the K nearest seeds to interpolate missing depths with O(Kn) complexity, our method does not need to limit the number of seeds to K and reduces the computational complexity to O(n). In addition, the minimax path chooses a path with the smallest maximum immediate pairwise pixel difference on it, so it tends to preserve sharp depth discontinuities better. In contrast to the results of previous depth upsampling algorithms, our approach provides accurate depths with fewer artifacts.