Neighboring Pixels Research Articles

Detection, Attribution and Localization of GAN Generated Images

Recent advances in Generative Adversarial Networks (GANs) have led to the creation of realistic-looking digital images that pose a major challenge to their detection by humans or computers. GANs are used in a wide range of tasks, from modifying small attributes of an image (StarGAN [14]), transferring attributes between image pairs (CycleGAN [92]), as well as generating entirely new images (ProGAN [37], StyleGAN [38], SPADE/GauGAN [65]). In this paper, we propose a novel approach to detect, attribute and localize GAN generated images that combines image features with deep learning methods. For every image, co-occurrence matrices are computed on neighborhood pixels of RGB channels in different directions (horizontal, vertical and diagonal). A deep learning network is then trained on these features to detect, attribute and localize these GAN generated/manipulated images. A large scale evaluation of our approach on 5 GAN datasets comprising over 2.76 million images (ProGAN, StarGAN, CycleGAN, StyleGAN and SPADE/GauGAN) shows promising results in detecting GAN generated images.

Prediction of Interpolants in Zero Diluted Images

This paper provides an algorithmic procedure to predict interpolants of zero diluted images. Given a digital image, one can zero dilute it by right adjoining a column consisting of ‘0s’ to every column except the last column and inserting a row consisting of ‘0s’ below every row except the last row. This yields a new image with a size (2W-1)×(2H-1), where W is the width and H is the height of the original image. Another way of zero diluting an image is by right adjoining a column consisting of ‘0s’ to every column and inserting a row consisting of ‘0s’ below every row. This yields a new image with a size (2W)×(2H), where W is the width and H is the height of the original image. Alternatively, subsampling of an image is carried out by forcing pixel values in the alternate columns and rows to zero. Thus, the size of the subsampled image is reduced to half of the size of the original image. This means 75% of the information in the original image is lost in the subsampled image. On the other hand, zero dilution of an image does not cause loss of information but increases the possibility of predicting more information. The question that arises here is whether it is possible to predict more pixel values, which are called interpolants so that the reconstructed image is an enhanced version of the original image in resolution. In this paper, two novel interpolant prediction techniques, which are reliable and computationally efficient, are discussed. They are (i) interpolant prediction using neighborhood pixel value averaging and (ii) interpolant prediction using extended morphological filtering. These techniques can be applied to predict interpolants in a subsampled image also.

Imagery Network Fine Registration by Reference Point Cloud Data Based on the Tie Points and Planes.

Cameras and laser scanners are complementary tools for a 2D/3D information generation. Systematic and random errors cause the misalignment of the multi-sensor imagery and point cloud data. In this paper, a novel feature-based approach is proposed for imagery and point cloud fine registration. The tie points and its two neighbor pixels are matched in the overlap images, which are intersected in the object space to create the differential tie plane. A preprocessing is applied to the corresponding tie points and non-robust ones are removed. Initial coarse Exterior Orientation Parameters (EOPs), Interior Orientation Parameters (IOPs), and Additional Parameters (APs) are used to transform tie plane points to the object space. Then, the nearest points of the point cloud data to the transformed tie plane points are estimated. These estimated points are used to calculate Directional Vectors (DV) of the differential planes. As a constraint equation along with the collinearity equation, each object space tie point is forced to be located on the point cloud differential plane. Two different indoor and outdoor experimental data are used to assess the proposed approach. Achieved results show about 2.5 pixels errors on checkpoints. Such results demonstrated the robustness and practicality of the proposed approach.

Efficient Image Segmentation of Natural Images with Noise using Energy Curved Based on Electromagnetism Optimization Algorithm

The method of digital image processing is an important step for extraction features from images, especially details from the medical image by dividing the image into various regions. In the last two decades, many significant image segmentation schemes modeled for different applications like medical to weather forecasting by images from the satellite. From available methods, multilevel thresholding of pixel values using the histogram of an image is an exceedingly important method developed for vide range of applications. Otsu’s method is the primer multilevel thresholding method based on the maximizing the inter-class variance based on the histogram. In Otsu’s method, the pixels of an image are grouped into different regions based on threshold levels calculated on the histogram of the image. All the pixels are segmented into two groups based on threshold level in the case of bi-level thresholding. The effectiveness of segmentation is depends on threshold levels, efficiency can be enhanced by computing optimized threshold level, Electromagnetism Optimization (EMO) technique used for this purpose in this paper. The techniques based on histograms of image are not holding the spatial particulars of contextual information for computing the optimized threshold levels, to nullify this shortcoming Energy Curve is used as an alternative of histogram, which is restore the special relation of pixels with its neighbor pixels. To examine the results several benchmark images considered, the final results shows that the method with Otsu’s technique with EMO on energy curve is giving better results as compared with methods based on the image histogram.

Semisupervised Discriminative Random Field for Hyperspectral Image Classification

The integration of spectral and spatial information is crucial in remotely sensed hyperspectral image classification. Some available approaches extract spatial features before classification, while other techniques include spatial information as a spatial regularizer. Due to the model complexity, these methods generally assume that a given pixel is only dependent on itself and its neighboring pixels. However, it is known that there are strong interactions among the pixels in different areas of the image, and as a result, this general assumption can limit the classifier performance. In this work, we develop a new semi-supervised discriminative random field (DRF) technique for spatial-spectral classification of hyperspectral images, which constructs the complete model using the whole image. Furthermore, to cope with the unlabeled samples in the model, a new expectation maximization (EM) based algorithm is introduced to optimize the learning of DRF parameters, where the missing labels of these samples serve as the latent variable. As compared to other conventional learning approaches such as maximum a posteriori marginal (MPM) labeling, the proposed approach can obtain better optimization that leads to improved classification performance. Our experimental results, conducted using real hyperspectral datasets, indicate that the newly developed DRF algorithm exhibits very good performance.

Center Attention Network for Hyperspectral Image Classification

Classification is one of the most important research topics in hyperspectral image (HSI) analyses and applications. Although convolutional neural networks (CNNs) have been widely introduced into the study of HSI classification with appreciable performance, the misclassification problem of the pixels on the boundary of adjacent land covers is still significant due to the interfering neighboring pixels whose categories are different from the target pixel. To address this challenge, in this article, we propose a center attention network for HSI classification. The proposed method simultaneously captures spectral-spatial features of the target pixel and its neighboring pixels for classification. Specifically, the method adopts a center attention module (CAM) that pays more attention to the features which are more correlated with the target pixel, that is, the central pixel of the sample, and then sums up the weighted features to generate more relevant and discriminative features. In this way, our method has a high potential for improving the performance of HSI classification. In addition, the CAM greatly reduces the number of parameters in the network via weighted sum of the spectral-spatial features, thus improving the computing efficiency while still maintaining classification accuracy. We evaluate the proposed method on three public datasets, and the experimental results demonstrate the superiority of our method on accuracy and efficiency compared with several state-of-the-art methods.

A New Hyperchaotic Image Encryption Algorithm Based on Stochastic Signals

Image encryption is often used to protect private images during transmission on a public channel. A high dimensional chaotic map has a greater secret key space, better ergodicity and dynamic property than a low-dimensional chaotic system. A seven-dimensional (7D) hyperchaotic map is used to produce chaotic sequences. Given secret keys and SHA-512 function are employed to generate initial values for iteration. Many stochastic signals are injected into one of the variables during iteration to transfer trajectory and increase dynamic behavior of a chaotic system. Three matrices are constructed with generated chaotic sequences. Permutation is performed on the basis of a control matrix. It keeps the pixel far from its neighboring pixels. Cycle shift is executed during bit-level permutation. Characteristic values of scrambling image are calculated and temporary values are achieved successfully. Two dynamic values are also applied during the diffusion process. Experimental results display the effects of the proposed algorithm. Security analysis reveals that the proposed method has some special advantages.

NeighborLoss: A Loss Function Considering Spatial Correlation for Semantic Segmentation of Remote Sensing Image

House segmentation of remote sensing image based on deep learning has become the main segmentation method because it can automatically extract features. However, the accuracy of image segmentation is affected not only by the network model, but also by the loss function, but the existing loss functions, except Binary Cross Entropy, are designed to deal with imbalanced dataset, no new research on improving Binary Cross Entropy for balanced dataset, and all loss function treat each pixel in isolation, without considering the spatial correlation between pixel and its neighbor pixels. To solve this problem, a new loss function, named NeighborLoss function, is proposed. Firstly, the deep learning network is used to get the prediction results of each pixel. According to whether the prediction results of the eight neighboring pixels of each pixel are consistent with the each pixel prediction, different weights are given to each pixel. Finally, the weighted average value of cross entropy of all pixels in the batch is taken as the final loss function value. We use the main deep learning semantic segmentation networks SegNet, PSPNet, UNET ++, MUNet with both NeighborLoss and cross entropy loss respectively to extract houses on the open data set named WHU dataset for remote sensing. The results show that compared with cross entropy loss functions, the MIoU, Precision, Recall, and Accuracy of NeighborLoss function are improved. From the predicted graph, the NeighborLoss function is more accurate to extract the edge of the house, especially in the corner of the house. NeighborLoss function is a more effective loss function for remote sensing image segmentation.

Focus Measurement in Color Space for Shape From Focus Systems

Shape from Focus (SFF) has been studied extensively in computer vision for 3D shape and depth recovery. The first stage in SFF methods is to compute the focus value of every pixel by converting the colored images into gray scale and then apply the focus measure operator. Converting colored values in the images into gray scale values may lead to imprecise mapping of pixels with different colored values onto the same gray scale value, this affects the overall accuracy of the system. In a colored image, the focused pixels maintain a considerable color difference from their neighboring pixels as compared to the defocused ones, which are blended into their neighborhood. This article presents an alternative method to measure the degree of focus by directly processing colored images. The color differences of the neighbor pixels with respect to the central pixel are obtained and summed together, this is followed by calculating their spread. The sum and the spread are combined to measure the degree of focus of the pixel in consideration. The proposed focus measure is then used for shape recovery of various simulated and real objects and is compared with previous techniques. The comparison results show the proposed method has the highest correlation and smallest RMSE values confirming the effectiveness of using color images for shape recovery.

Stack Attention-Pruning Aggregates Multiscale Graph Convolution Networks for Hyperspectral Remote Sensing Image Classification

The hyperspectral remote sensing images are classified by traditional neural networks methods can achieve promising performance, but only operate on regular square regions with fixed. This will lead to between neighborhood pixels have limitations in achieve long-distances joint interaction modeling and cross-spacetime information flow for capturing complex spatial-temporal dependencies. Meanwhile ignoring importance detail information and improved utilization of irrelevant information. In the work, we propose a stack attention-pruning multiscale aggregates graph convolution framework (SAP-MAGACN). The framework can automatically learn and selectively attend to the relevant subspace structure by stack attention-pruning module, can effectively disentangle the complex space structure of remote sensing images and capture the rich structural semantics. Meanwhile a refine graph of neighborhood pixels are constructures. Then we adopt the aggregation manner for multiscale graph convolution of pixels nodes in different neighborhood for effective long-range joint interaction modeling. Finally, we leverage dense cross-spacetime edges to completion propagation of multiscale spatial-temporal information, and gradually produce the discriminative embedded features and effectively distinguish the categories of boundary pixels. The experimental results shown the propose SAP-MAGCN outperformance all others state-of-the-art methods on Indian Pines and Salinas public benchmark datasets. Such as the OA, AA and Kappa of our propose SAP-MAGCN frameworks is 96.75%, 95.73% and 97.33%, respectively, on Indian Pines datasets.

An Adaptive Change Threshold Selection Method Based on Land Cover Posterior Probability and Spatial Neighborhood Information

Change threshold selection (CTS) plays an important role in land cover change detection. The traditional CTS methods are mainly proposed by using the information contained in grayscale histogram distributions or pixel neighborhoods. However, land cover is highly spatially heterogeneous, and changes in different land cover types are characterized by different magnitudes. Unfortunately, few CTS studies have considered the effects of both land cover type and spatial heterogeneity on CTS, potentially leading to false alarms or missed alarms. To address this challenge, we propose an adaptive CTS method based on land cover posterior probability and spatial neighborhood information (LCSN). First, the posterior probability of the change magnitude in each land cover type is calculated according to a Bayesian criterion to integrate the land cover type information. Second, the posterior probability is calculated using a bilateral filtering method to construct the spatial surface based on the land cover type and spatial neighborhood information. Finally, the degree of difference between the spatial surface and the change magnitude map is taken as the final threshold. The proposed LCSN method is verified with Landsat 8-Operational Land Imager (OLI) images and IKONOS images. The experimental results show that the LCSN method is effective in reducing the pseudo changes and identifying changes in land cover types with low grayscale values in the corresponding change magnitude maps.

Bilateral Joint-Sparse Regression for Hyperspectral Unmixing

Sparse hyperspectral unmixing has been a hot topic in recent years. Joint sparsity assumes that each pixel in a small neighborhood of hyperspectral images (HSIs) is composed of the same endmembers, which results in a few nonzero rows in the abundance matrix. Recall that a plethora of unmixing algorithms transform a 3-D HSI into a 2-D matrix with vertical priority. The transformation makes matrix computation easier. It is, however, hard to maintain the horizontal spatial information in HSIs in many cases. To make further use of the spatial information of HSIs, in this article, we propose a bilateral joint-sparse structure for hyperspectral unmixing in an attempt to exploit the local joint sparsity of the abundance matrix in both the vertical and horizontal directions. In particular, we introduce a permutation matrix to realize the bilateral joint-sparse representation and there is no need to construct the matrix explicitly. Moreover, we propose to simultaneously impose the bilateral joint-sparse structure and low rankness on the abundance and develop a new algorithm named bilateral joint-sparse and low-rank unmixing . The proposed algorithm is based on the alternating direction method of multipliers framework and employs a reweighting strategy. The convergence analysis of the proposed algorithm is investigated. Simulated and real-data experiments show the effectiveness of the proposed algorithm.

Spectral-Spatial Constrained Nonnegative Matrix Factorization for Spectral Mixture Analysis of Hyperspectral Images

Hyperspectral spectral mixture analysis (SMA), which intends to decompose mixed pixels into a collection of endmembers weighted by their corresponding fraction abundances, has been successfully used to tackle mixed-pixel problem in hyperspectral remote sensing applications. As an approach of decomposing a high-dimensional data matrix into the multiplication of two nonnegative matrices, nonnegative matrix factorization (NMF) has shown its advantages and been widely applied to SMA. Unfortunately, most of the NMF-based unmixing methods can easily lead to an unsuitable solution, due to inadequate mining of spatial and spectral information and the influence of outliers and noise. To overcome such limitations, a spatial constraint over abundance and a spectral constraint over endmembers are imposed over NMF-based unmixing model for spectral-spatial constrained unmixing. Specifically, a spatial neighborhood preserving constraint is proposed to preserve the local geometric structure of the hyperspectral data by assuming that pixels in a spatial neighborhood generally fall into a low-dimensional manifold, while a minimum spectral distance constraint is formulated to optimize endmember spectra as compact as possible. Furthermore, to handle non-Gaussian noises or outliers, an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm based loss function is ultimately adopted for the proposed spectral-spatial constrained nonnegative matrix factorization model and a projected gradient based optimization algorithm is designed for optimization. Experimental results over both synthetic and real-world datasets demonstrate that the proposed spatial and spectral constraints can certainly improve the performance of NMF-based unmixing and outperform state-of-the-art NMF-based unmixing algorithms.

Pattern-adaptive error diffusion algorithm for improved phase-only hologram generation

The bidirectional error diffusion (BERD) algorithm is free from random phase modulation that introduces speckle noise on the reconstructed images, compared with other computer-generated phase-only hologram (POH) approaches. During the POH generation process, the amplitudes of all pixels are traditionally set to one for diffusing the errors to their neighborhood of unprocessed pixels. In this paper, we reveal that the reconstruction quality depends on the uniform amplitude value for different object pattern. The pattern-adaptive BERD (PA-BERD) algorithm is proposed for high-quality holographic reconstruction. The optimized amplitude value can be acquired for each object pattern and each propagation distance. The PA-BERD-based POHs have shown higher reconstruction quality than traditional BERD-based POHs in simulations as well as optical experiments.

An Improved Method for Phase Triangulation Algorithm Based on the Coherence Matrix Eigen-Decomposition in Time-Series SAR Interferometry

Time-series SAR interferometry, which combines permanent scatterers (PSs) and distributed scatterers (DSs), has been strongly developed in recent years. Unlike PS, DS corresponds to a natural target whose neighboring pixels share similar reflectivity values. The selection of DS is relevant to the goodness-of-fit value, the estimation of which is based on all possible combined interferometric phases and fails to avoid the adverse effect of low-quality phases. This paper used eigen-decomposition of coherence matrix that was constructed based on the identified homogeneous pixels to perform phase optimization, and then only the interferometric phases with low noise and clear fringes are utilized to measure the goodness-of-fit. 30 Sentinel-1A images were applied to test the improved method of land subsidence monitoring in Beijing, China. The deformation results of different methods were cross verified and the area statistics of the study area were carried out. The results show that the maximum subsidence monitored by the improved method is located in Jinzhan Town with a maximum rate of −109 mm/yr. This improved method extracts more measurement points with accuracy ensured, which is proved to be an effective way to provide the high spatial density of deformation measurements. The land subsidence in this area is mainly caused by the excessive exploitation of groundwater resources. This research provides support for the prevention and control work of relevant departments.

High Precision Error Prediction Algorithm Based on Ridge Regression Predictor for Reversible Data Hiding

An efficient predictor is crucial for high embedding capacity and low image distortion. In this letter, a ridge regression-based high precision error prediction algorithm for reversible data hiding is proposed. The ridge regression is a penalized least-square algorithm, which solves the overfitting problem of the least-square method. Reversible data hiding based on ridge regression predictor minimizes the residual sum of squares between predicted and target pixels subject to the constraint expressed in terms of the L2-norm. Compared to a least-square-based predictor, the ridge regression-based predictor can obtain more small prediction errors, proving that the proposed method has a higher accuracy. In addition, the eight neighbor pixels of the target pixels and their two different combinations are selected as training and support sets, respectively. This selection scheme further improves the prediction accuracy. Experimental results show that the proposed method outperforms state-of-the-art adaptive reversible data hiding in terms of prediction accuracy and embedding performance.

Moving Object Detection Algorithm based on Improved Visual Background Extractor

Aiming at the problems that the ViBe algorithm causes error detection when the camera jitter, ghosts appear when processing moving targets, and the detected moving targets are incomplete, an improved ViBe algorithm based on motion compensation is proposed. In the background modeling stage, the motion compensation method based on KLT is used to obtain background model to enhance the robustness of the algorithm to the dynamic background. In the foreground detection stage, combined the background model of the current pixel with 8 neighbor pixel background model, by the double discriminant algorithm to eliminate ghost areas caused by detecting the real background points as foreground points, introduce OTSU algorithm to obtain the optimal threshold for foreground detection. In the post processing stage, filter the connected components for segmentation masks and update models to limit the wrong background points scattered in the foreground.Finally, the connected domain is repaired by morphological opening operation. The results show that the algorithm can effectively eliminate the interference caused by camera jitter, and suppress the ghost phenomenon to obtain more accurate foreground images.

A Label Similarity Probability Filter for Hyperspectral Image Postclassification

This article presents a label similarity probability filter (LSPF) to make hyperspectral image postclassification. The LSPF is inspired by the first law of geography and proposes a class label probability function to quantify the probability of both centered and its neighboring pixels belonging to the same class. It first classifies the hyperspectral data using the regular support vector machine classifier. Then, it binarizes the posterior classification result to obtain the binary label maps of each class. After that, it traverses all spatial windows centered by each pixel and calculates the cumulative probability of all pixels in each class. Finally, the cumulative probabilities are used to make reclassification to obtain the refined classification map. The experiments on Indian Pines, Pavia University, and ZY1-02D Yellow River Estuary data show that LSPF greatly improves the classification accuracy of spectral signatures and outperforms other state-of-the-art spectral-spatial methods.

Adjacency effect correction of optical satellite image with sub-meter spatial resolution

The adjacency effect, the contribution of the neighboring pixels to the radiance of the line of sight pixel, is caused by the Rayleigh scattering of atmospheric molecules and Mie scattering of aerosol particles. The adjacency effect will cause the reflectance of each pixel in the apparent reflectance satellite image to be between the real reflectance and the average background reflectance, reducing the accuracy of the surface reflectance inversion. Therefore, it is very important to remove the adjacency effect to improve the accuracy of retrieving the surface reflectance from satellite images. The most critical issue of the adjacency effect is to accurately calculate the weight of the contribution of each background pixel to the adjacency effect. The weight value of the contribution of each background pixel to the adjacency effect mainly depends on the spatial distance between the target pixel and the background pixel, the difference in reflectance between the target pixel and the background pixel, and the optical thickness of atmospheric molecules and the optical thickness of aerosol. At present, the commonly used weight function for calculating the weight value considers only the influence of optical thickness and spatial distance on the weight value. These weight functions are applied to a relatively uniform surface. However, when these weight functions are applied to an inhomogeneous surface, they will greatly reduce the accuracy of the adjacency effect correction. The combination of ground features in satellite images with the sub-meter spatial resolution is complex, so the influence of the difference in reflectance between the target pixel and the background pixel on the adjacency effect must be considered. The adaptive atmospheric correction algorithm proposed in this paper can adjust the weight value of the contribution of background pixels to the adjacency effect according to the spatial distance between the target pixel and the background pixel, the difference in reflectance between the target pixel and the background pixel, and the difference between the atmospheric molecules’ optical thickness and aerosol optical thickness. The adaptive atmospheric correction algorithm is used to correct the adjacency effect on GF-2 panchromatic satellite images. The results show that the adaptive atmospheric correction algorithm can effectively remove the adjacency effect in sub-meter spatial resolution optical satellite images, improve both the accuracy of quantitative study and the satellite image quality.

An Adaptive Phase Optimization Algorithm for Distributed Scatterer Phase History Retrieval

The multitemporal interferometric synthetic aperture radar (InSAR) technique based on distributed scatterers (DSs) has been widely applied in high-precision deformation measurements, which compensates for the drawback that the persistent scatterer InSAR technique does not obtain sufficient monitoring points, especially in rural areas. Considering that DS pixels are susceptible to various decorrelation factors, it is necessary to retrieve the optimal phase series by phase optimization algorithms (POAs). However, conventional POAs rely on a sample covariance matrix or complex coherence matrix (CCM) derived by spatially averaging statistically homogeneous pixel neighborhoods, which may blur and destroy phase information, especially in dense fringe areas. To overcome this limitation, an adaptive POA is proposed in this article. The adaptive POA artificially constructs a superior CCM by the filtered interferometric phase, which is derived through spatial adaptive filtering approach fusion of principal phase component estimation and fast nonlocal means filtering, and an accurate coherence matrix determined via coherence estimation bias correction. Moreover, the modified eigen-decomposition-based maximum-likelihood-estimator of the interferometric phase (EMI) with coherence-power-weighting is proposed to further improve the estimation precision and computational efficiency. The estimated CCM is then processed with the modified coherence-power-weighted EMI algorithm, and the optimal phase history is retrieved. The experimental results validated against both simulated and Sentinel-1A data demonstrate the superior optimization performance and robustness of the adaptive POA over traditional POAs.