Neighborhood Reconstruction Research Articles

Fast Discriminant Analysis With Adaptive Reconstruction Structure Preserving.

Neighborhood reconstruction methods have been widely applied to feature engineering. Existing reconstruction-based discriminant analysis methods normally project high-dimensional data into a low-dimensional space while preserving the reconstruction relationships among samples. However, there are three limitations: 1) the reconstruction coefficients are learned based on the collaborative representation of all sample pairs, which requires the training time to be the cube of the number of samples; 2) these coefficients are learned in the original space, ignoring the interference of the noise and redundant features; and 3) there is a reconstruction relationship between heterogeneous samples; this will enlarge the similarity of heterogeneous samples in the subspace. In this article, we propose a fast and adaptive discriminant neighborhood projection model to tackle the above drawbacks. First, the local manifold structure is captured by bipartite graphs in which each sample is reconstructed by anchor points derived from the same class as that sample; this can avoid the reconstruction between heterogeneous samples. Second, the number of anchor points is far less than the number of samples; this strategy can reduce the time complexity substantially. Third, anchor points and reconstruction coefficients of bipartite graphs are updated adaptively in the process of dimensionality reduction, which can enhance the quality of bipartite graphs and extract discriminative features simultaneously. An iterative algorithm is designed to solve this model. Extensive results on toy data and benchmark datasets show the effectiveness and superiority of our model.

ScHolography: a computational method for single-cell spatial neighborhood reconstruction and analysis

Spatial transcriptomics has transformed our ability to study tissue complexity. However, it remains challenging to accurately dissect tissue organization at single-cell resolution. Here we introduce scHolography, a machine learning-based method designed to reconstruct single-cell spatial neighborhoods and facilitate 3D tissue visualization using spatial and single-cell RNA sequencing data. scHolography employs a high-dimensional transcriptome-to-space projection that infers spatial relationships among cells, defining spatial neighborhoods and enhancing analyses of cell–cell communication. When applied to both human and mouse datasets, scHolography enables quantitative assessments of spatial cell neighborhoods, cell–cell interactions, and tumor-immune microenvironment. Together, scHolography offers a robust computational framework for elucidating 3D tissue organization and analyzing spatial dynamics at the cellular level.

HGDO: An oversampling technique based on hypergraph recognition and Gaussian distribution

The synthetic minority oversampling technique (SMOTE) is the most prevalent solution in class imbalance learning. While SMOTE and its variant methods handle imbalanced data well in most cases, they fail to take full advantage of the structural information in the overall data, which leads to the propagation of noise. Some existing SMOTE variants remove noisy samples by adding an undersampling process. However, due to the complexity of the data distribution, it is difficult to accurately identify real noise samples, leading to lower modeling quality. To this end, we propose an oversampling technique based on hypergraph identification and Gaussian distribution (HGDO). First, neighborhood reconstruction is performed for each sample depending on the sparse representation to build a hypergraph model, and outlier and noisy samples are filtered according to this model. Then, the weight of each retained minority class sample is determined through the distribution relationship of hyperedges and vertices. Finally, new samples are generated based on the Laplacian matrix and Gaussian distribution to balance the dataset. The comprehensive experimental analysis demonstrates the superiority of HGDO over some popular SMOTE variants.

A knowledge-driven cooperative scatter search algorithm with reinforcement learning for the distributed blocking flow shop scheduling problem

The distributed flow shop scheduling problem has become one of the key problems related to the high efficiency impacted factor in the manufacturing industry due to its typical scenarios in real-world industrial applications. In this paper, a knowledge-driven cooperative scatter search (KCSS) is proposed to address the distributed blocking flow shop scheduling problem (DBFSP) to minimize the makespan. The scatter search (SS) is adopted as the basic optimization framework in KCSS. The neighborhood perturbation operator and the Q-learning algorithm are combined to select the appropriate perturbation operator in the search process. Firstly, considering the complexity of distributed scenarios, five search operators are used to construct a disturbance strategy pool. Secondly, the Q-learning algorithm dynamically chooses disturbance strategies to enhance exploration ability and search efficiency. Afterward, a local search method based on neighborhood reconstruction is proposed to perturb the currently found optimal solution to strengthen the ability of KCSS to develop in local areas. In addition, the path relinking mechanism is introduced into the subset combination method to guarantee the diversity of solutions in the optimization process. Finally, the performance of the KCSS algorithm is verified on the benchmark set, and the experimental results demonstrate the robustness and effectiveness of the KCSS algorithm. In addition, 518 of the best-known solutions out of 720 benchmark instances are updated.

Joint Dynamic Manifold and Discriminant Information Learning for Feature Extraction.

Neighborhood reconstruction is a good recipe to learn the local manifold structure. Representation-based discriminant analysis methods normally learn the reconstruction relationship between each sample and all the other samples. However, reconstruction graphs constructed in these methods have three limitations: 1) they cannot guarantee the local sparsity of reconstruction coefficients; 2) heterogeneous samples may own nonzero coefficients; and 3) they learn the manifold information prior to the process of dimensionality reduction. Due to the existence of noise and redundant features in the original space, the prelearned manifold structure may be inaccurate. Accordingly, the performance of dimensionality reduction would be affected. In this article, we propose a joint model to simultaneously learn the affinity relationship, reconstruction relationship, and projection matrix. In this model, we actively assign neighbors for each sample and learn the inter-reconstruction coefficients between each sample and their neighbors with the same label information in the process of dimensionality reduction. Specifically, a sparse constraint is employed to ensure the sparsity of neighbors and reconstruction coefficients. The whitening constraint is imposed on the projection matrix to remove the relevance between features. An iterative algorithm is proposed to solve this method. Extensive experiments on toy data and public datasets show the superiority of the proposed method.

Robust and Fast Normal Mollification via Consistent Neighborhood Reconstruction for Unorganized Point Clouds.

This paper introduces a robust normal estimation method for point cloud data that can handle both smooth and sharp features. Our method is based on the inclusion of neighborhood recognition into the normal mollification process in the neighborhood of the current point: First, the point cloud surfaces are assigned normals via a normal estimator of robust location (NERL), which guarantees the reliability of the smooth region normals, and then a robust feature point recognition method is proposed to identify points around sharp features accurately. Furthermore, Gaussian maps and clustering are adopted for feature points to seek a rough isotropic neighborhood for the first-stage normal mollification. In order to further deal with non-uniform sampling or various complex scenes efficiently, the second-stage normal mollification based on residual is proposed. The proposed method was experimentally validated on synthetic and real-world datasets and compared to state-of-the-art methods.

Classification via Structure-Preserved Hypergraph Convolution Network for Hyperspectral Image

Graph convolutional network (GCN) as a combination of deep learning and graph learning has gained increasing attention in hyperspectral image (HSI) classification. However, most GCN methods consider the simple point-to-point structure between two pixels rather than the high-order structure of multiple pixels, which is contradict with the real feature distribution of ground object. And the nonlinear property of HSI also brings challenge for precise structural representation in GCN. To tackle these problems, this work proposes a structure preserved hypergraph convolution network (SPHGCN). It first builds a multiple neighborhood reconstruction (MNR) model to reveal the essential resemblance of multiple pixels in nonlinear spectral feature space. With the high-order structure, SPHGCN designs the hypergraph convolution operation for irregular feature aggregation among similar pixels from different regions, which achieves more discriminative features from multiple pixel nodes. Meanwhile, a structure preservation layer is built to optimize the distribution of convolutional features under the guidance of high-order structure. Moreover, SPHGCN integrates local regular convolution and irregular hypergraph convolution to learn the structured semantic feature of HSI. This strategy breaks the boundary restriction in traditional convolution and aggregates semantic feature across different image patches. Experiments on three HSI data sets indicate that SPHGCN outperforms a few state-of-the-art methods for HSI classification.

Fast neighborhood reconstruction with adaptive weights learning

Neighborhood reconstruction is one of the hot topics in data analysis, as it has a powerful ability to capture the local manifold information of data. However, one common drawback of existing discriminant neighborhood reconstruction methods is that they partition the reconstruction weights learning and feature extraction into two separate steps; due to the existence of noisy and redundant features, this affects the learning performance. In addition to this, each sample is expressed by the linear combination of all other samples, and the sparsity of the reconstruction coefficients is usually constrained by the sparse norm or regularization term. This will not only cause heterogeneous samples to own unnecessary weights, but also increases the time complexity. To address these issues, in this paper, we propose a Fast and Adaptive Neighborhood Reconstruction (FANR) model, where each sample is expressed by representative points from the same class. By replacing the fully connected graph with a bipartite graph, the time complexity can be reduced from the original O(n3) to O(n), while the adverse impact of heterogeneous samples can also be avoided. The weights and representative points of the bipartite graph in each class are auto-updated in the ideal subspace to eliminate the influence of noisy and redundant features. Furthermore, to reduce the sensitivity of the model to the reduced dimension, PCA is introduced to hold the main energy of the original data. An iterative algorithm is developed to solve the model effectively. The results of experiments conduced on toy data and benchmark datasets demonstrate the efficiency and superiority of our proposed model.

An enhanced decomposition-based multi-objective evolutionary algorithm with a self-organizing collaborative scheme

The multi-objective evolutionary algorithm based on decomposition (MOEA/D) decomposes a multi-objective optimization problem (MOP) into multiple single-objective subproblems using an aggregation function and optimizes them together using a collaborative approach. MOEA/D exhibits extraordinary optimization ability in solving MOPs. However, the algorithm’s performance is hampered by the single evolutionary operator, fixed control parameters, and fixed neighborhood architecture used in primal MOEA/D. To obtain the balance between convergence and diversity, a self-organizing collaborative scheme (SCS) is designed to achieve the best combination of differential evolution (DE) operator, control parameters, and neighborhood size in this paper. The SCS is integrated into MOEA/D to form the MOEA/D-SCS. In MOEA/D-SCS, firstly, the evolutionary process is separated into three stages: early, middle, and late, and different indicators are utilized to screen elite and inferior solutions for the early and late stages. Secondly, different evolutionary strategies are developed for the three stages and the various types of solutions to help boost the algorithm’s efficiency. Then, to enhance convergence and diversity in different evolutionary stages, a neighborhood reconstruction strategy is proposed to assign different neighborhood sizes according to the neighborhood information. Finally, the capability of MOEA/D-SCS was tested using three standard test suites, ZDT, DTLZ, and UF, and a MOP of sewage treatment. The simulation results demonstrate that the proposed MOEA/D-SCS can achieve better optimization results and show stronger optimization ability in most cases compared to state-of-the-art evolutionary algorithms, such as MOEA/D-DE, MOEA/D-STM, MOEA/D-FRRMAB, MOEA/D-DRA, NSGA-II, and dMOPSO, especially in problems with multiple peaks, complex Pareto fronts, and practical engineering optimization problems.

On the feasibility of crawling-based attacks against recommender systems1

Nowadays, online services, like e-commerce or streaming services, provide a personalized user experience through recommender systems. Recommender systems are built upon a vast amount of data about users/items acquired by the services. Such knowledge represents an invaluable resource. However, commonly, part of this knowledge is public and can be easily accessed via the Internet. Unfortunately, that same knowledge can be leveraged by competitors or malicious users. The literature offers a large number of works concerning attacks on recommender systems, but most of them assume that the attacker can easily access the full rating matrix. In practice, this is never the case. The only way to access the rating matrix is by gathering the ratings (e.g., reviews) by crawling the service’s website. Crawling a website has a cost in terms of time and resources. What is more, the targeted website can employ defensive measures to detect automatic scraping. In this paper, we assess the impact of a series of attacks on recommender systems. Our analysis aims to set up the most realistic scenarios considering both the possibilities and the potential attacker’s limitations. In particular, we assess the impact of different crawling approaches when attacking a recommendation service. From the collected information, we mount various profile injection attacks. We measure the value of the collected knowledge through the identification of the most similar user/item. Our empirical results show that while crawling can indeed bring knowledge to the attacker (up to 65% of neighborhood reconstruction on a mid-size dataset and up to 90% on a small-size dataset), this will not be enough to mount a successful shilling attack in practice.

Spatial-Spectral Extraction for Hyperspectral Anomaly Detection

A novel hyperspectral anomaly detection method is proposed in this letter. This method is motivated by two important observations. First, there are hundreds of bands in a hyperspectral image (HSI), among which redundant bands and some noisy bands could be eliminated to increase the discriminability from the spectral domain. Meanwhile, considering the small amount of the anomalies, they may be neglected in the spatial clustering process, and the anomalies can be highlighted by eliminating the clustered result from the original HSI. In this way, the spatial-spectral extraction is proposed to detect the anomalies in the HSI. First, an iterative optimal neighborhood reconstruction (IONR) method is utilized to select the bands while preserving their divergence. Second, the spatial clustering is achieved by minimizing the intradistance in any given category. Finally, the spatial clustered HSI is eliminated from the band selected HSI and acts as the direct input for the detection process. Experimental results and data analysis have demonstrated the effectiveness of the proposed method.

Joint neighborhood preserving and projected clustering for feature extraction

Neighborhood reconstruction is proved effective for dimensionality reduction because of the preservation of manifold structure. Conventional neighborhood preserving embedding (NPE) method first learns the affinity relationship or reconstruction relationship in the original space, and then learns the projection matrix to preserve the learned local information in low-dimensional space. However, the pre-learned manifold information may be inaccurate due to the noises and irrelevant features in real-world data. The performance of dimensionality reduction would be influenced as well. Besides, NPE and its variants only aim to preserve the local reconstruction relationship but ignore the fuzzy membership relationship between samples and cluster prototypes. To address these issues, we propose an adaptive neighborhood preserving discriminant projection model, where sparse reconstruction coefficients are updated in the process of dimensionality reduction to eliminate the influence of noises and irrelevant features. Meanwhile, we also learn the fuzzy membership relationships between data points and cluster prototypes to gather the samples belonging to the same class together in low-dimensional space. Neighborhood reconstruction learning and clustering are seamlessly connected in the learned subspace. To solve this model, an iterative algorithm is developed. The experimental results of recognition accuracy show the superiorities of the proposed methods over the state-of-the-arts.

Linear neighborhood reconstruction constrained latent subspace discovery for incomplete multi-view clustering

Linear neighborhood reconstruction constrained latent subspace discovery for incomplete multi-view clustering

ATPGNN: Reconstruction of Neighborhood in Graph Neural Networks With Attention-Based Topological Patterns

Graph Neural Networks (GNNs) have been applied in many fields of semi-supervised node classification for non-Euclidean data. However, some GNNs cannot make good use of positive information brought by nodes which are far away from each central node for aggregation operations. These remote nodes with positive information can enhance the representation of the central node. Some GNNs also ignore rich structure information around each central node's surroundings or entire network. Besides, most of GNNs have a fixed architecture and cannot change their components to adapt to different tasks. In this article, we propose a semi-supervised learning platform ATPGNN with three variable components to overcome the above shortcomings. This novel model can fully adapt to different tasks by changing its components and support inductive learning. The key idea is that we first create a high-order topology graph, which is from similarity of node structure information. Specifically, we reconstruct the relationships between nodes in a potential space obtained by network embedding in graph. Second, we introduce graph representation learning methods to extract representation information of remote nodes on the high-order topology graph. Third, we use some network embedding methods to get graph structure information of each node. Finally, we combine the representation information of remote nodes, graph structure information and feature for each node by attention mechanism, and apply them to learning node representation in graph. Extensive experiments on real attributed networks demonstrate the superiority of the proposed model against traditional GNNs.

Graph exponentiation and neighborhood reconstruction

Graph exponentiation and neighborhood reconstruction

Hyperspectral Band Selection via Optimal Neighborhood Reconstruction

Band selection is one of the most important technique in the reduction of hyperspectral image (HSI). Different from traditional feature selection problem, an important characteristic of it is that there is usually strong correlation between neighboring bands, that is, bands with close indexes. Aiming to fully exploit this prior information, a novel band selection method called optimal neighborhood reconstruction (ONR) is proposed. In ONR, band selection is considered as a combinatorial optimization problem. It evaluates a band combination by assessing its ability to reconstruct the original data, and applies a noise reducer to minimize the influence of noisy bands. Instead of using some approximate algorithms, ONR exploits a recurrence relation that underlies the optimization target to obtain the optimal solution in an efficient way. Besides, we develop a parameter selection approach to automatically determine the parameter of ONR, ensuring it is adaptable to different data sets. In experiments, ONR is compared with some state-of-the-art methods on six HSI data sets. The results demonstrate that ONR is more effective and robust than the others in most of the cases.

Joint Label Prediction Based Semi-Supervised Adaptive Concept Factorization for Robust Data Representation

Constrained Concept Factorization (CCF) yields the enhanced representation ability over CF by incorporating label information as additional constraints, but it cannot classify and group unlabeled data appropriately. Minimizing the difference between the original data and its reconstruction directly can enable CCF to model a small noisy perturbation, but is not robust to gross sparse errors. Besides, CCF cannot preserve the manifold structures in new representation space explicitly, especially in an adaptive manner. In this paper, we propose a joint label prediction based Robust Semi-Supervised Adaptive Concept Factorization (RS2ACF) framework. To obtain robust representation, RS2ACF relaxes the factorization to make it simultaneously stable to small entrywise noise and robust to sparse errors. To enrich prior knowledge to enhance the discrimination, RS2ACF clearly uses class information of labeled data and more importantly propagates it to unlabeled data by jointly learning an explicit label indicator for unlabeled data. By the label indicator, RS2ACF can ensure the unlabeled data of the same predicted label to be mapped into the same class in feature space. Besides, RS2ACF incorporates the joint neighborhood reconstruction error over the new representations and predicted labels of both labeled and unlabeled data, so the manifold structures can be preserved explicitly and adaptively in the representation space and label space at the same time. Owing to the adaptive manner, the tricky process of determining the neighborhood size or kernel width can be avoided. Extensive results on public databases verify that our RS2ACF can deliver state-of-the-art data representation, compared with other related methods.

Unsupervised Nonnegative Adaptive Feature Extraction for Data Representation

In this paper, we propose a novel unsupervised Nonnegative Adaptive Feature Extraction (NAFE) algorithm for data representation and classification. The formulation of NAFE integrates the sparsity constrained nonnegative matrix factorization (NMF), representation learning, and adaptive reconstruction weight learning into a unified model. Specifically, NAFE performs feature and weight learning over the new robust representations of NMF for more accurate measure and representation. For nonnegative adaptive feature extraction, our NAFE first utilizes the sparsity constrained NMF to obtain the new and robust representations of the original data. To preserve the manifold structures of the learnt new representations, we also incorporate a neighborhood reconstruction error over the weight matrix for joint minimization. Note that to further improve the representation power, the weights are jointly shared in the new low-dimensional nonnegative representation space, low-dimensional nonlinear manifold space, and low-dimensional projective subspace, i.e., local neighborhood information is clearly preserved in different feature spaces so that informative representations and features can be jointly obtained. To enable NAFE to extract features from new data, we also include a feature approximation error by a linear projection so that the learnt extractor can obtain features from new data efficiently. Extensive simulations show that our formulation can deliver state-of-the-art results on several public databases for feature extraction and classification, compared with several related methods.

Robust point cloud normal estimation via neighborhood reconstruction

This article presents a novel method for estimating normals on unorganized point clouds that preserves sharp features. Many existing methods are unable to reliably estimate normals for points around sharp features since the neighborhood employed for the normal estimation would enclose points belonging to different surface patches across the sharp feature. To address this challenging issue, a neighborhood reconstruction-based normal estimation method is developed to find a proper neighborhood for points around sharp features. A robust statistics-based method is proposed to identify points that near sharp features and classify the points into two categories: edge points and non-edge points. Two specific neighborhood reconstruction strategies are designed for these two types of points to generate a neighborhood clear of sharp features. The normal of the current point can thus be reliably estimated by principal component analysis using the generated isotropic neighborhood. Numerous case studies have been carried out to compare the reliability and robustness of the proposed method against various existing methods. The experiment results show that the proposed approach performs better than the state-of-the-art methods most of the time and offers an ideal compromise between precision, speed, and robustness.

Semi-supervised local multi-manifold Isomap by linear embedding for feature extraction

In this paper, we mainly propose a semi-supervised local multi-manifold Isomap learning framework by linear embedding, termed SSMM-Isomap, that can apply the labeled and unlabeled training samples to perform the joint learning of neighborhood preserving local nonlinear manifold features and a linear feature extractor. The formulation of SSMM-Isomap aims at minimizing pairwise distances of intra-class points in the same manifold and maximizing the distances over different manifolds. To enhance the performance of nonlinear manifold feature learning, we also incorporate the neighborhood reconstruction error to preserve local topology structures between both labeled and unlabeled samples. To enable our SSMM-Isomap to extract local manifold features from the outside new data, we also add a feature approximation error that correlates manifold features with embedded features by the jointly learnt feature extractor. Thus, the learnt linear extractor can extract the local manifold features from the new data efficiently by direct embedding. To optimize the proposed objective function, two effective schemes are presented, i.e., Scaling by MAjorizing a Complicated Function and Eigen-decomposition. Notice that the comparison of the proposed two solvers is also described. We mainly evaluate SSMM-Isomap for manifold feature learning, data clustering and classification. Extensive simulation results verify the effectiveness of our SSMM-Isomap algorithm, compared with other related feature learning techniques.