Pixels In Local Neighborhood Research Articles

Compared-neighborhood based image dehazing for improved visibility

Image hazing is the degradation of photographic quality due to light attenuation by mist or suspended particles. This paper presents solutions to two shortcomings of existing haze-removal techniques. One shortcoming is that pixels often get zeroed out in the dehazing process, which suppresses edges and features. The other is that the assumption of homogeneity of features and properties in input images during dehazing reduces the resolution of features and textures. A solution that considers how feature and edge visibility, which are primarily disrupted by noise, is provided. This noise is responsible for the lack of distinction between local and global pixel neighborhoods. An attenuation coefficient that helps to minimize pixel distortion is proposed. This coefficient is sensitive to local relative pixel intensity and prevents the pixels from being zeroed out in the context of certain local or global neighborhoods. The proposed technique is implemented via the existing dual-stream network based on a CNN with the block-greedy algorithm. The qualitative and quantitative evaluation based on 117 images shows 75% improvement in haze density ζ, 90% increase in edge visibility e, and 150% improvement in the peak-signal-to-noise ratio (PNR), and 95% increase in structural similarity index measure (SSIM) compared to the original hazed image. These show a remarkable improvement compared to the existing state-of-the-art methods. The shortcoming is a need for color improvement, which can be studied further in future studies.

Local neighborhood gradient pattern: A feature descriptor for content based image retrieval

Local feature descriptors are efficient encoders for capturing repeated local patterns in many of the computer vision applications. Majority of such descriptors consider only limited local neighborhood pixels to encode a pattern. One of the major issues while considering more number of neighborhood pixels is that it increases the dimensionality of the feature descriptor. The proposed descriptor addresses these issues by describing an effective encoding pattern with optimal feature vector length. In this paper, we have proposed Local Neighborhood Gradient Pattern (LNGP) for Content-Based Image Retrieval (CBIR) in which the relationship between a set of neighbours and the centre pixel is considered to obtain a compact 8-bit pattern in the respective pixel position. The relationship of the gradient information of immediate, next-immediate, and diagonal neighbours with the centre pixel is considered for pattern formation, and thus the local information based on pixels in three directions are captured. The experiments are conducted on benchmarked image retrieval datasets such as Wang’s 1K, Corel 5K, Corel 10K, Salzburg (Stex), MIT-Vistex, AT & T, and FEI datasets and it is observed that the proposed descriptor yields average precision of 71.88%, 54.57%, 40.66%, 71.85%, 86.12%, 82.54%, and 68.54% respectively in the mentioned datasets. The comparative analysis of the recent descriptors indicates that the proposed descriptor performs efficiently in CBIR applications.

Deep Stacked Autoencoder-Based Automatic Emotion Recognition Using an Efficient Hybrid Local Texture Descriptor

Extracting an effective facial feature representation is the critical task for an automatic expression recognition system. Local Binary Pattern (LBP) is known to be a popular texture feature for facial expression recognition. However, only a few approaches utilize the relationship between local neighborhood pixels itself. This paper presents a Hybrid Local Texture Descriptor (HLTD) which is derived from the logical fusion of Local Neighborhood XNOR Patterns (LNXP) and LBP to investigate the potential of positional pixel relationship in automatic emotion recognition. The LNXP encodes texture information based on two nearest vertical and/or horizontal neighboring pixel of the current pixel whereas LBP encodes the center pixel relationship of the neighboring pixel. After logical feature fusion, the Deep Stacked Autoencoder (DSA) is established on the CK+, MMI and KDEF-dyn dataset and the results show that the proposed HLTD based approach outperforms many of the state of art methods with an average recognition rate of 97.5% for CK+, 94.1% for MMI and 88.5% for KDEF.

Nonlocal Feature Selection Encoder–Decoder Network for Accurate InSAR Phase Filtering

Accurate interferometric phase filtering is an essential step in InSAR data processing. The existing deep learning-based phase-filtering methods were developed based on local neighboring pixels and only use local phase information. The idea of nonlocal processing has been proven to be very effective for improving the accuracy of interferometric phase filtering. In this paper, we propose a deep convolutional neural network-based nonlocal InSAR filtering method via a nonlocal phase filtering network (NL-PFNet) based on the encoder–decoder structure and nonlocal feature selection strategy. Thanks to the powerful phase feature extraction ability of the encoder–decoder structure and the utilization of nonlocal phase information, NL-PFNet can predict an accurately filtered interferometric phase after training using a large number of interferometric phase images with different noise levels. Experiments on both simulated and real InSAR data show that the proposed method significantly outperforms three traditional well-established methods and another deep learning-based method. Compared with the InSAR-BM3D filter and another deep learning-based method, the mean square error of the proposed method is 25% and 11% lower when processing simulated data, respectively, and when processing the real Sentinel-1 interferometric phase, the no-reference evaluation metric Q of the proposed method is 25% and 9% higher, respectively. In addition, the running time of the proposed method is tens of times less than that of the traditional filtering methods.

Color Context Binary Pattern Using Progressive Bit Correction for Image Classification

Local binary pattern (LBP) is sensitive to noise. Noise-resistant LBP (NRLBP) addresses this problem by thresholding local neighboring pixels into three-valued states (i.e., 0, 1 and uncertain bits) and recovering uncertain bits via an error-correction mechanism. In this paper, we extend NRLBP to deal with color images and propose a robust color image descriptor called Color context binary pattern (CCBP). In CCBP, we employ scale context and neighbor context to progressively correct the encoded bits. First, we encode intra-channel local neighboring pixels into three-valued states in scale space and use majority voting to correct all states across scales. Then, we compute inter-channel color feature distances and correct the uncertain bits via neighboring bit propagation. Finally, we construct the image descriptor by concatenating all histograms based on the corrected binary codes. Experiments on four benchmark databases demonstrate the robustness of CCBP for color image classification under very low signal-to-noise ratio levels.

Dual attention per-pixel filter network for spatially varying image deblurring

Spatially varying motion deblurring has recently witnessed substantial progress due to the development of deep neural network. However, most existing CNN-based methods involve two major shortcomings: (1) The CNN weights are space-sharing, and these methods thus ignore the properties of complex spatially variant blurs which vary from pixel to pixel in natural blurry images. (2) Stacked convolution layers with a large kernel or recurrent neural networks (RNNs) cannot capture the global contextual dependence of features, they thus cannot exploit the relationship between different blur pixels at a distance. To solve these problems, we propose a new dual attention per-pixel filter network (DAPFN). First, we develop a multiscale per-pixel filter network (MSPFN) to learn a specific deblurring mapping for different blur pixels, which predicts the per-pixel spatially adaptive convolution kernel for each blur pixel in the input blurry image of different scales and restores the clean pixel by performing channel-wise spatially adaptive convolution with the local neighborhood pixels. Second, we develop a dual attention enhanced residual network (DAERN) to capture the global contextual dependence of the blurry images, which introduces a dual attention (DA) module consisting of the spatial self-attention module (SSA) and channel self-attention module (CSA). The fusion of the two attention modules helps to further improve the deblurring performance. Third, we propose a new receptive field selection (RFS) block to learn the nonlinear characteristics of spatially variant blurs, which enables the adaptive fusing of the features with different receptive fields and effectively enhances the network nonlinear representation ability. The experimental results on GOPRO dataset indicate that the average PSNR and SSIM of the proposed method reached 31.8455 and 0.9231, respectively. The results of extensive experiments pertaining to spatially varying image deblurring demonstrate that the proposed method outperforms the state-of-the-art image deblurring methods.

An efficient automatic facial expression recognition using local neighborhood feature fusion

In computer vision, several feature extraction methods have been developed to differentiate the variations of facial expressions. But the effect of the relationship among the neighboring pixel is not considered in the existing texture encoding based method. This paper exploits the method to analyze the association among the adjacent pixels using feature fusion technique. For efficient texture representation, the proposed approach combines the Local Binary Pattern (LBP) with the Local Neighborhood Encoded Pattern (LNEP). The LBP feature encodes the relationship of adjacent pixels with respect to the central pixel whereas LNEP represents the relationship among the two closest local neighboring pixels of the current pixel. After concatenating LBP with LNEP, the most relevant features are selected using chi-square statistical analysis and classified using multiclass Support Vector Machine (SVM). Experimental findings show that the proposed hybrid feature performed better than an individual feature and it achieves an average recognition accuracy of 97.86% and 97.11% on CK+ and MMI dataset, respectively. The effectiveness of the reduced hybrid feature is also evaluated under a noisy environment and the results show better performance in such conditions.

Statistical image-based material decomposition for triple-energy computed tomography using total variation regularization.

Triple-energy computed tomography (TECT) can obtain x-ray attenuation measurements at three energy spectra, thereby allowing identification of different material compositions with same or very similar attenuation coefficients. This ability is known as material decomposition, which can decompose TECT images into different basis material image. However, the basis material image would be severely degraded when material decomposition is directly performed on the noisy TECT measurements using a matrix inversion method. To achieve high quality basis material image, we present a statistical image-based material decomposition method for TECT, which uses the penalized weighted least-squares (PWLS) criteria with total variation (TV) regularization (PWLS-TV). The weighted least-squares term involves the noise statistical properties of the material decomposition process, and the TV regularization penalizes differences between local neighboring pixels in a decomposed image, thereby contributing to improving the quality of the basis material image. Subsequently, an alternating optimization method is used to minimize the objective function. The performance of PWLS-TV is quantitatively evaluated using digital and mouse thorax phantoms. The experimental results show that PWLS-TV material decomposition method can greatly improve the quality of decomposed basis material image compared to the quality of images obtained using the competing methods in terms of suppressing noise and preserving edge and fine structure details. The PWLS-TV method can simultaneously perform noise reduction and material decomposition in one iterative step, and it results in a considerable improvement of basis material image quality.

Hyperspectral classification using deep fusion spectral–spatial features

The deep convolution neural network (CNN) has been of great interest in hyperspectral image classification recently. Current CNN-based approaches adopt a two- or three-dimensional convolution network to extract spectral–spatial features from local pixel neighborhoods. High computation cost and overfitting problems should be handled carefully for these large-scale networks. A compact one-dimensional (1-D) CNN-based method is proposed to explore the joint spectral–spatial features by employing the pixel coordinates as the spatial information. Furthermore, two kinds of CNN architecture for spectral–spatial feature fusion are compared. To be specific, the shallow fusion model (SF-CNN) concatenates the spatial information to the spectral features before they are fed into the final fully connected layer, whereas the deep fusion model (DF-CNN) combines the spectral and the spatial information in an early stage and extracts the high-level spectral-spatial features by the 1-D convolution layers. The experimental results demonstrate that the DF-CNN method provides competitive classification results to state-of-the-art methods. Moreover, attributed to the concise spatial information and the effective feature fusion structure, the proposed method is economical in terms of computation cost when compared with the current CNN-based spectral-spatial methods.

A Novel Hyperspectral Image Classification Pattern Using Random Patches Convolution and Local Covariance

Today, more and more deep learning frameworks are being applied to hyperspectral image classification tasks and have achieved great results. However, such approaches are still hampered by long training times. Traditional spectral–spatial hyperspectral image classification only utilizes spectral features at the pixel level, without considering the correlation between local spectral signatures. Our article has tested a novel hyperspectral image classification pattern, using random-patches convolution and local covariance (RPCC). The RPCC is an effective two-branch method that, on the one hand, obtains a specified number of convolution kernels from the image space through a random strategy and, on the other hand, constructs a covariance matrix between different spectral bands by clustering local neighboring pixels. In our method, the spatial features come from multi-scale and multi-level convolutional layers. The spectral features represent the correlations between different bands. We use the support vector machine as well as spectral and spatial fusion matrices to obtain classification results. Through experiments, RPCC is tested with five excellent methods on three public data-sets. Quantitative and qualitative evaluation indicators indicate that the accuracy of our RPCC method can match or exceed the current state-of-the-art methods.

Structure descriptor based on just noticeable difference for texture image classification.

Local binary pattern (LBP) and its derivates have been widely used in texture classification. However, LBP-based methods are sensitive to noise, and some structure information represented by non-uniform patterns is lost due to the combination of these patterns. In this paper, a new local structure descriptor based on just noticeable difference (JND) for texture classification is proposed by exploring the spatial and relative intensity correlations among local neighborhood pixels. First, a JND map of the image is computed, and then we attempt to model the correlations among local neighborhood pixels by comparing the absolute differences in intensity between the central pixel and its neighbors with the corresponding JND threshold. A new visual pattern (JNDVP) is designed using modeled correlations to describe image structure. Next, considering that image contrast makes important contributions to structure description, contrast is employed as a weighting factor for JNDVP histogram creation to represent structural and contrast information in a single representation. Finally, the nearest neighborhood classifier is employed for texture classification. Results on two texture image databases demonstrate that the proposed structure descriptor is rotation invariant and more robust to noise than LBP. Moreover, texture classification based on JNDVP outperforms LBP-based methods.

Joint Local Block Grouping with Noise-Adjusted Principal Component Analysis for Hyperspectral Remote-Sensing Imagery Sparse Unmixing

Spatial regularized sparse unmixing has been proved as an effective spectral unmixing technique, combining spatial information and standard spectral signatures known in advance into the traditional spectral unmixing model in the form of sparse regression. In a spatial regularized sparse unmixing model, spatial consideration acts as an important role and develops from local neighborhood pixels to global structures. However, incorporating spatial relationships will increase the computational complexity, and it is inevitable that some negative influences obtained by inaccurate estimated abundances’ spatial correlations will reduce the accuracy of the algorithms. To obtain a more reliable and efficient spatial regularized sparse unmixing results, a joint local block grouping with noise-adjusted principal component analysis for hyperspectral remote-sensing imagery sparse unmixing is proposed in this paper. In this work, local block grouping is first utilized to gather and classify abundant spatial information in local blocks, and noise-adjusted principal component analysis is used to compress these series of classified local blocks and select the most significant ones. Then the representative spatial correlations are drawn and replace the traditional spatial regularization in the spatial regularized sparse unmixing method. Compared with total variation-based and non-local means-based sparse unmixing algorithms, the proposed approach can yield comparable experimental results with three simulated hyperspectral data cubes and two real hyperspectral remote-sensing images.

Hyperspectral and LiDAR Data Fusion Classification Using Superpixel Segmentation-Based Local Pixel Neighborhood Preserving Embedding

A new method of superpixel segmentation-based local pixel neighborhood preserving embedding (SSLPNPE) is proposed for the fusion of hyperspectral and light detection and ranging (LiDAR) data based on the extinction profiles (EPs), superpixel segmentation and local pixel neighborhood preserving embedding (LPNPE). A new workflow is proposed to calibrate the Goddard’s LiDAR, hyperspectral and thermal (G-LiHT) data, which allows our method to be applied to actual data. Specifically, EP features are extracted from both sources. Then, the derived features of each source are fused by the SSLPNPE. Using the labeled samples, the final label assignment is produced by a classifier. For the open standard experimental data and the actual data, experimental results prove that the proposed method is fast and effective in hyperspectral and LiDAR data fusion.

R-theta local neighborhood pattern for unconstrained facial image recognition and retrieval

In this paper R-Theta Local Neighborhood Pattern (RTLNP) is proposed for facial image retrieval. RTLNP exploits relationships amongst the pixels in local neighborhood of the reference pixel at different angular and radial widths. The proposed encoding scheme divides the local neighborhood into sectors of equal angular width. These sectors are again divided into subsectors of two radial widths. Average grayscales values of these two subsectors are encoded to generate the micropatterns. Performance of the proposed descriptor has been evaluated and results are compared with the state of the art descriptors e.g. LBP, LTP, CSLBP, CSLTP, Sobel-LBP, LTCoP, LMeP, LDP, LTrP, MBLBP, BRINT and SLBP. The most challenging facial constrained and unconstrained databases, namely; AT&T, CARIA-Face-V5-Cropped, LFW, and Color FERET have been used for showing the efficiency of the proposed descriptor. Proposed descriptor is also tested on near infrared (NIR) face databases; CASIA NIR-VIS 2.0 and PolyU-NIRFD to explore its potential with respect to NIR facial images. Better retrieval rates of RTLNP as compared to the existing state of the art descriptors show the effectiveness of the descriptor

Hyperspectral anomalous change detection based on joint sparse representation

Anomalous change detection aims at finding small but unusual changes from the unchanged or generally changed background in multi-temporal hyperspectral remote sensing images. It is important to model the spectral variations of background so as to highlight the anomalous changes. In this paper, we proposed a hyperspectral anomalous change detection method based on joint sparse representation. A background dictionary is constructed by the randomly selected pixels in the stacked multi-temporal images. The local neighborhood pixels surrounding the test pixel are presented by joint sparse representation with the background dictionary. Thus, the change tendencies in the local background are modeled by the active dictionary bases. The difference of separate reconstruction coefficients of the test pixel with the active bases will reflect the probability to be anomalously changed. Three detectors, which are coefficient difference, Mahalanobis distance of coefficient difference and multi-temporal residual analysis, are proposed to measure the change intensity. Two experiments with the datasets of “Viareggio 2013 Trial” and one Hyperion indicate that the proposed method obtains better performances than the comparative methods.

Developing a bio-inspired multi-gene genetic programming based intelligent estimator to reduce speckle noise from ultrasound images

The speckle noise commonly occurs in ultrasound imaging based applications. Due to the multiplicative nature, speckle noise deteriorates the visual quality of ultrasound images. This affects the performance of radiologists and practitioners for disease diagnosis and/or patient treatment. The current study proposes a bio-inspired multi-gene genetic programming (MGGP) based intelligent estimator to reduce the speckle noise from ultrasound images. The proposed MGGP approach is based on the parallel framework of multiple genes and has effectively utilized the evolutionary learning capabilities to develop an intelligent estimator, by exploiting the useful statistical features extracted from local neighboring pixels. The performance of the proposed novel approach is evaluated on ultrasound images of common carotid artery corrupted with different noise levels. Further, the robust performance was validated on several diverse types of ultrasound images of Breast Cyst, Kidney Cancer, Liver, Liver Cyst, and Fetal Head. The proposed bio-inspired approach showed superior denoising performance over existing approaches. The proposed intelligent estimator is capable of removing speckle noise effectively while preserving the fine lines and edges. During evolution, the MGGP framework automatically selects the useful statistical features and primitive functions from a wider solution space to develop the intelligent estimator. Further, the proposed approach does not require image-dependent optimal threshold values, as conventional speckle denoising approaches required.

Gaussian-Hermite moment-based depth estimation from single still image for stereo vision

Depth information of objects plays a significant role in image-based rendering. Traditional depth estimation techniques use different visual cues including the disparity, motion, geometry, and defocus of objects. This paper presents a novel approach of focus cue-based depth estimation for still images using the Gaussian-Hermite moments (GHMs) of local neighboring pixels. The GHMs are chosen due to their superior reconstruction ability and invariance properties to intensity and geometric distortions of objects as compared to other moments. Since depths of local neighboring pixels are significantly correlated, the Laplacian matting is employed to obtain final depth map from the moment-based focus map. Experiments are conducted on images of indoor and outdoor scenes having objects with varying natures of resolution, edge, occlusion, and blur contents. Experimental results reveal that the depth estimated from GHMs can provide anaglyph images with stereo quality better than that provided by existing methods using traditional visual cues.

Improving the Spatial Resolution of Land Surface Phenology by Fusing Medium- and Coarse-Resolution Inputs

Satellite-derived land surface phenology (LSP) serves as a valuable input source for many environmental applications such as land cover classifications and global change studies. Commonly, LSP is derived from coarse-resolution (CR) sensors due to their well-suited temporal resolution. However, LSP is increasingly demanded at medium resolution (MR), but inferring LSP directly from MR imagery remains a challenging task (e.g., due to acquisition frequency). As such, we present a methodology that directly predicts MR LSP on the basis of the respective CR LSP and MR reflectance imagery. The approach considers information from the local pixel neighborhood at both resolutions by utilizing several prediction proxies, including spectral distance and multiscale heterogeneity metrics. The prediction performs well with simulated data $(R^{2} = 0.84)$ , and the approach substantially reduces noise. The size of the smallest reliably predicted object coincides with the effective CR pixel size (i.e., field-of-view). Nevertheless, even subpixel objects can be reliably predicted provided that pure CR pixels are located within the search radius. The application to real MODIS LSP and Landsat reflectance well preserves the phenological landscape composition, and the spatial refinement is especially striking in heterogeneous agricultural areas, where, for example, the circular shape of center pivot irrigation schemes is successfully restored at MR.

Regularized set-to-set distance metric learning for hyperspectral image classification

Combining the spectral and spatial information, we propose a regularized set-to-set distance metric learning method (RSSDML) for the hyperspectral image (HSI) classification. It first performs a local pixel neighborhood preserving embedding to reduce the dimensionality and meanwhile to preserve the local similarity structures of HSI, and then puts each target spectral pixel and its spatial neighbors into a set, and measures the distance between different sets to reveal the overall differences of different target spectral pixels. In the computation of the set-to-set distance, a regularization strategy is used to differentiate individual pixels in a pixel set and to improve the set-based metric relations. Exploiting both the correlations between neighboring pixels in a pixel set and the similarities between different pixel sets, the proposed RSSDML dramatically improves traditional point-based and set-based metric learning methods and provides better classification results than some state-of-the-art spatial-spectral classifiers on two benchmark hyperspectral data sets.

Completed local similarity pattern for color image recognition

In this paper, we propose a simple yet effective local color image descriptor, completed local similarity pattern (CLSP), for face recognition. We represent the color image as the co-occurrence of its image pixel color quantization information and the local color image textural information. Specifically, CLSP consists of two complementary components: color label and local similarity pattern. Expressed by soft color label for every image pixel, we adopt the k-means clustering method and the soft-assignment coding method to summarize and encode the image pixel color information globally ignoring local neighboring pixels. While based on the similarity of color information between central pixel and its neighbors, local similarity pattern (LSP) is used to encode the local spatial textural feature of the color image ignoring their color value. Therefore, LSP has the merits of robustness and compactness of coding based feature extraction method. The joint distribution (2-D histogram), which unifies the color label and LSP, is used for a given image representation. Moreover, seven different color spaces are selected and fused to compensate each other for color image feature extraction. Experimental results on GeorgiaTech, AR, NUST-RWFR and LFW face database show that CLSP outperforms state-of-the-art color image feature extraction methods.