Neighboring Samples Research Articles

Exploiting the similarity of dissimilarities for biomedical applications and enhanced machine learning.

The "similarity of dissimilarities" is an emerging paradigm in biomedical science with significant implications for protein function prediction, machine learning (ML), and personalized medicine. In protein function prediction, recognizing dissimilarities alongside similarities provides a more detailed understanding of evolutionary processes, allowing for a deeper exploration of regions that influence biological functionality. For ML models, incorporating dissimilarity measures helps avoid misleading results caused by highly correlated or similar data, addressing confounding issues like the Doppelgänger Effect. This leads to more accurate insights and a stronger understanding of complex biological systems. In the realm of personalized AI and precision medicine, the importance of dissimilarities is paramount. Personalized AI builds local models for each sample by identifying a network of neighboring samples. However, if the neighboring samples are too similar, it becomes difficult to identify factors critical to disease onset for the individual, limiting the effectiveness of personalized interventions or treatments. This paper discusses the "similarity of dissimilarities" concept, using protein function prediction, ML, and personalized AI as key examples. Integrating this approach into an analysis allows for the design of better, more meaningful experiments and the development of smarter validation methods, ensuring that the models learn in a meaningful way.

A meta-learning imbalanced classification framework via boundary enhancement strategy with Bayes imbalance impact index.

For imbalanced classification problem, algorithm-level methods can effectively avoid the information loss and noise introduction of data-level methods. However, the differences in the characteristics of the datasets, such as imbalance ratio, data dimension, and sample distribution, make it difficult to determine the optimal parameters of the algorithm-level methods, which leads to low universality. This paper proposes a meta-learning imbalanced classification framework via boundary enhancement strategy with Bayes imbalance impact index. First, the meta-learning idea is introduced to build multiple comprehensive training tasks. Each task contains a support set and a query set, which are randomly sampled from the original dataset. The support set is used to train the meta-classifier, and the meta-classifier's performance on the query set serves as an optimization object. This approach enables the meta-classifier to enhance itself based on the experiences gained from multiple training tasks, which makes it quickly adapt in the fine-tuning stage and exhibit excellent performance on the test set. Second, a boundary enhancement strategy with Bayes imbalance impact index (IBI3) is designed. The IBI3 value is used to identify boundary samples. On this basis, the feature interpolation among the boundary samples and their neighbor samples is applied to enhance the classification boundary, effectively alleviating the meta-classifier decision-making bias caused by data imbalance in the fine-tuning stage. Experimental results on 38 imbalanced public datasets with diverse characteristics show that the proposed method outperforms 24 typical imbalanced classification methods in F-measure and G-mean without parameters adjustment, which proves that the proposed method has greater universality.

Migration and Segregated Spaces: Analysis of Qualitative Sources Such as Wikipedia Using Artificial Intelligence

The scientific literature on residential segregation in large metropolitan areas highlights various explanatory factors, including economic, social, political, landscape, and cultural elements related to both migrant and local populations. This paper contrasts the impact of these factors individually, such as the immigrant rate and neighborhood segregation. To achieve this, a machine learning analysis was conducted on a sample of neighborhoods in the main Spanish metropolitan areas (Madrid and Barcelona), using a database created from a combination of official statistical sources and textual sources, such as Wikipedia. These texts were transformed into indexes using Natural Language Processing (NLP) and other artificial intelligence algorithms capable of interpreting images and converting them into indexes. The results indicate that the factors influencing immigrant concentration and segregation differ significantly, with crucial roles played by the urban landscape, population size, and geographic origin. While land prices showed a relationship with immigrant concentration, their effect on segregation was mediated by factors such as overcrowding, social support networks, and landscape degradation. The novel application of AI and big data, particularly through ChatGPT and Google Street View, has enhanced model predictability, contributing to the scientific literature on segregated spaces.

Negative Selection Algorithm for Unsupervised Anomaly Detection

In this work, we present a modification of the well-known Negative Selection Algorithm (NSA), inspired by the process of T-cell generation in the immune system. The approach employs spherical detectors and was initially developed in the context of semi-supervised anomaly detection. The novelty of this work lies in proposing an adapted version of the NSA for unsupervised anomaly detection. The goal is to develop a method that can be applied to datasets that may not only represent self-data but also contain a small percentage of anomalies, which must be detected without prior knowledge of their locations. The proposed unsupervised algorithm leverages neighborhood sampling and ensemble methods to enhance its performance. We conducted comparative tests with 11 other algorithms across 17 datasets with varying characteristics. The results demonstrate that the proposed algorithm is competitive. The proposed algorithm performs well across multiple metrics, including accuracy, AUC, precision, recall, F1 score, Cohen’s kappa, and Matthews correlation coefficient. It consistently ranks among the top algorithms for recall, indicating its effectiveness in scenarios where detecting all existing anomalies is critical, even at the expense of some increase in false positives. Further research is possible and may focus on exploring normalization procedures, improving threshold automation, and extending the method for more detailed anomaly confidence assessments.

Multi-level network Lasso for multi-task personalized learning

We propose the multi-level network Lasso, which aims to overcome the key limitations of existing personalized learning methods, such as ignoring sample homogeneity or heterogeneity, and over-parametrization. Multi-level network Lasso learns both sample-common model and sample-specific model, that are succinct and interpretable in the sense that model parameters are shared across neighboring samples based on only a subset of relevant features. To apply personalized learning in multi-task scenarios, we further extend the multi-level network Lasso for multi-task personalized learning by learning underlying task groups in the feature subspace. Additionally, we investigate a family of the multi-level network Lasso based on the ℓp quasi-norm (0<p<1), that helps prevent over-penalization on large group outliers. An alternating algorithm is developed to efficiently solve the proposed optimization problem. Experimental results on synthetic and real-world datasets demonstrate the effectiveness of the proposed method.

BalancerGNN: Balancer Graph Neural Networks for imbalanced datasets: A case study on fraud detection

Fraud detection for imbalanced datasets is challenging due to machine learning models inclination to learn the majority class. Imbalance in fraud detection datasets affects how graphs are built, an important step in many Graph Neural Networks (GNNs). In this paper, we introduce our BalancerGNN framework to tackle with imbalanced datasets and show its effectiveness on fraud detection. Our framework has three major components: (i) node construction with feature representations, (ii) graph construction using balanced neighbor sampling, and (iii) GNN training using balanced training batches leveraging a custom loss function with multiple components. For node construction, we have introduced (i) Graph-based Variable Clustering (GVC) to optimize feature selection and remove redundancies by analyzing multi-collinearity and (ii) Encoder-Decoder based Dimensionality Reduction (EDDR) using transformer-based techniques to reduce feature dimensions while keeping important information intact about textual embeddings. Our experiments on Medicare, Equifax, IEEE, and auto insurance fraud datasets highlight the importance of node construction with features representations. BalancerGNN trained with balanced batches consistently outperforms other methods, showing strong abilities in identifying fraud cases, with sensitivity rates ranging from 72.87% to 81.23% across datasets while balancing specificity. Additionally, BalancerGNN achieves impressive accuracy rates, ranging from 73.99% to 94.28%. These outcomes underscore the crucial role of graph representation and neighbor sampling techniques in optimizing BalancerGNN for fraud detection models in real-world applications.

Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced Training

Although GNNs have achieved success in semi-supervised graph learning tasks, common GNNs suffer from expensive message passing during each epoch and the exponentially growing receptive field occupying too much memory, especially on large graphs. Neighbor sampling techniques can reduce GNNs’ memory footprints, but they encounter either redundant computation or incomplete edges. Some simplified GNNs decouple graph convolutions and feature transformations to reduce computation in training. However, only a part of them can scale to large graphs without neighbor sampling techniques, which can be concluded as decoupled GNNs. Nevertheless, they either only utilize the last convolution output or simply add multi-hop features with uniform weights, which limits their expressiveness. In this paper, we refine the pipeline of decoupled GNNs and propose Scalable and Adaptive Graph Neural Networks (SAGN), which effectively leverages multi-hop information with a scalable attention mechanism. Moreover, we generalize the input of decoupled GNNs to view another classical technique, label propagation, as a special case of decoupled GNNs and propose decoupled label trick (DecLT) to incorporate label information into decoupled GNNs. Furthermore, by incorporating self-training technique, we further propose the Self-Label-Enhanced (SLE) training framework, leveraging pseudo labels to simultaneously augment the training set and improve label propagation. Extensive experiments show that SAGN outperforms other baselines, and that DecLT and SLE can consistently and significantly improve all types of models on semi-supervised node classification tasks. Many top-ranked models on Open Graph Benchmark (OGB) leaderboard adopt our methods as the main backbone.

Fault Detection Strategy of Partial Least Squares Based on Temporal Neighborhood Difference

ABSTRACTAiming at the difficulty of detecting time‐lag faults in dynamic processes, a fault detection strategy based on time neighborhood difference (TND) is proposed, and it is introduced into the partial least squares (PLS) method to suggest the PLS‐TND fault detection method. The TND method takes the mean to the multibatch training set to obtain a baseline training set, and it constructs the mean squared Euclidean distance (MSED) statistic by calculating the average distance between the sample's first k‐moments neighborhood samples and samples at the same moment in the baseline training set. The TND method can help the PLS method to effectively detect time‐lag faults and significantly improve the fault detection capability of PLS by measuring the overall positional difference between the temporal neighborhood sample set of the sample and its temporal neighborhood sample set in the baseline training set. The PLS‐TND method is compared with some classical fault detection methods through a numerical simulation process and a Continuous Stirred Tank Reactor (CSTR) system design fault detection experiment. The PLS‐TND method gives the best performance of fault detection.

ANNE: Adaptive Nearest Neighbours and Eigenvector-based sample selection for robust learning with noisy labels

An important stage of most state-of-the-art (SOTA) noisy-label learning methods consists of a sample selection procedure that classifies samples from the noisy-label training set into noisy-label or clean-label subsets. The process of sample selection typically consists of one of the two approaches: loss-based sampling, where high-loss samples are considered to have noisy labels, or feature-based sampling, where samples from the same class tend to cluster together in the feature space and noisy-label samples are identified as anomalies within those clusters. Empirically, loss-based sampling is robust to a wide range of noise rates, while feature-based sampling tends to work effectively in particular scenarios, e.g., the filtering of noisy instances via their eigenvectors (FINE) sampling exhibits greater robustness in scenarios with low noise rates, and the K nearest neighbour (KNN) sampling mitigates better high noise-rate problems. This paper introduces the Adaptive Nearest Neighbours and Eigenvector-based (ANNE) sample selection methodology, a novel approach that integrates loss-based sampling with the feature-based sampling methods FINE and Adaptive KNN to optimize performance across a wide range of noise rate scenarios. ANNE achieves this integration by first partitioning the training set into high-loss and low-loss sub-groups using loss-based sampling. Subsequently, within the low-loss subset, sample selection is performed using FINE, while the high-loss subset employs Adaptive KNN for effective sample selection. We integrate ANNE into the noisy-label learning state of the art (SOTA) method SSR+, and test it on CIFAR-10/-100 (with symmetric, asymmetric and instance-dependent noise), Webvision and ANIMAL-10, where our method shows better accuracy than the SOTA in most experiments, with a competitive training time. The code is available at https://github.com/filipe-research/anne.

Design of A Linear Array Holographic Subsurface Radar System

Abstract This work introduces a new holographic subsurface radar (HSR) system designed with a cross-arranged linear array antenna. In non-destructive applications, holographic subsurface radar utilizes continuous wave, high-density spatial sampling, and a wide synthetic aperture to achieve high azimuth resolution. However, this results in a low data acquisition speed, which limits the detection efficiency. In this work, we designed a cross-arranged linear array antenna, where the transmitting- and receiving-antennas are layered on two parallel baselines. In this way, the space between the neighboring sampling points can still meet the spatial sampling criterion. The use of the electric-switching antenna array reduces 2-dimensional mechanical scanning to a 1-dimensional scan, significantly improving data acquisition speed while maintaining sufficient spatial sampling density. The prototype of the system was also detailed in the paper. The experiments show that the newly designed HSR system can improve detection efficiency by more than 10 times while maintaining imaging resolution.

Robust multi-label classification via data reconstruction by neighborhood samples augmentation

Robust multi-label classification via data reconstruction by neighborhood samples augmentation

A novel ensemble over-sampling approach based Chebyshev inequality for imbalanced multi-label data

With the development of intelligent technology, data exhibits characteristics of multi-label and imbalanced distribution, which lead to the degradation of classification model performance. Therefore, addressing multi-label class imbalance has become a hot research topic. Nowadays, over-sampling approaches aim to generate a superset of the original dataset to deal with imbalanced data. However, traditional over-sampling methods only employ the central data point and its nearest neighbor samples to synthesize samples without considering the impact of data distribution. To address these issues, in this paper, we propose an ensemble multi-label over-sampling algorithm (MLCIO) based on Chebyshev inequality and a group optimization strategy. Firstly, to generate more representative and diverse samples, with the seed sample serving as the sphere’s center, Chebyshev inequality is utilized to ensure that synthetic samples fall within its m times the standard deviation. Secondly, a group optimization ranking weighting approach is employed to obtain more reliable and stable label information. Finally, comparative experiments are conducted on 11 imbalanced datasets from various domains using different evaluation metrics. The results demonstrate that our proposal achieves better performance than other approaches.

DAPLSR: Data Augmentation Partial Least Squares Regression Model via Manifold Optimization

Traditional Partial Least Squares Regression (PLSR) models frequently underperform when handling data characterized by uneven categories. To address the issue, this paper proposes a Data Augmentation Partial Least Squares Regression (DAPLSR) model via manifold optimization. The DAPLSR model introduces the Synthetic Minority Over-sampling Technique (SMOTE) to increase the number of samples and utilizes the Value Difference Metric (VDM) to select the nearest neighbor samples that closely resemble the original samples for generating synthetic samples. In solving the model, in order to obtain a more accurate numerical solution for PLSR, this paper proposes a manifold optimization method that uses the geometric properties of the constraint space to improve model degradation and optimization. Comprehensive experiments show that the proposed DAPLSR model achieves superior classification performance and outstanding evaluation metrics on various datasets, significantly outperforming existing methods.

Impacts of urban block form on carbon and pollutant emissions from urban life in China from the perspective of regional differences

Air pollution emissions (PE) and carbon emissions (CE) pose significant challenges to achieving the Sustainable Development Goals (SDGs) globally. The intensity of urban block development and spatial form can influence the relationship between PE and CE. This study analyzed 11,228 neighborhood samples from various climate zones across China using spatial statistics and an optimized random forest model to examine the impact of block spatial form on PE and CE. The findings reveal that: (1) The PE–CE correlation in non-first-tier city blocks in southern China is stronger than in those in northern China. The correlation is strongest in urban neighborhoods located in temperate climates. Additionally, the PE–CE correlation is weakest in Beijing and Shanghai. (2) The variation in explanatory power of different driving factors is more pronounced for CE than for PE, with PR, NDVI, and AH emerging as the most significant factors. (3) The synergy between PE and CE is strongest when BD is in the 20 %-30 % range. Similarly, the synergy is strongest when PR is in the 2–3 range. (4) BD in the 40 %-60 % range is most effective in reducing PE and CE, with 40 %-50 % range favoring CE reduction and 50 %-60 % range favoring PE reduction.

Label-noise learning via uncertainty-aware neighborhood sample selection

Existing deep learning methods often require a large amount of high-quality labeled data. Yet, the presence of noisy labels in the real-world training data seriously affects the generalization ability of the model. Sample selection techniques, the current dominant approach to mitigating the effects of noisy labels on models, use the consistency of sample predictions and observed labels to make clean selections. However, these methods rely heavily on the accuracy of the sample predictions and inevitably suffer when the model predictions are unstable. To address these issues, we propose an uncertainty-aware neighborhood sample selection method. Especially, it calibrates for sample prediction by neighbor prediction and reassigns model attention to the selected samples based on sample uncertainty. By alleviating the influence of prediction bias on sample selection and avoiding the occurrence of prediction bias, our proposed method achieves excellent performance in extensive experiments. In particular, we achieved an average of 5% improvement in asymmetric noise scenarios.

Label-Only Membership Inference Attack Based on Model Explanation

It is well known that machine learning models (e.g., image recognition) can unintentionally leak information about the training set. Conventional membership inference relies on posterior vectors, and this task becomes extremely difficult when the posterior is masked. However, current label-only membership inference attacks require a large number of queries during the generation of adversarial samples, and thus incorrect inference generates a large number of invalid queries. Therefore, we introduce a label-only membership inference attack based on model explanations. It can transform a label-only attack into a traditional membership inference attack by observing neighborhood consistency and perform fine-grained membership inference for vulnerable samples. We use feature attribution to simplify the high-dimensional neighborhood sampling process, quickly identify decision boundaries and recover a posteriori vectors. It also compares different privacy risks faced by different samples through finding vulnerable samples. The method is validated on CIFAR-10, CIFAR-100 and MNIST datasets. The results show that membership attributes can be identified even using a simple sampling method. Furthermore, vulnerable samples expose the model to greater privacy risks.

MNN: Mixed nearest-neighbors for self-supervised learning

In contrastive self-supervised learning, positive samples are typically drawn from the same image but in different augmented views, resulting in a limited source of positive samples. An effective way to alleviate this problem is to incorporate the relationship between samples, which involves including the top-K nearest neighbors of positive samples. However, the issue of false neighbors (i.e., neighbors that do not belong to the same category as the positive sample) is a significant yet often overlooked challenge. In this paper, we present a simple self-supervised learning framework called Mixed Nearest-Neighbors (MNN) for Self-Supervised Learning. The primary objective of the MNN is to enhance the robustness and accuracy of the model by strategically incorporating synthesized neighboring samples. Specifically, our proposed method utilizes image mixture to mitigate the effects of noise introduced by false neighbors, while adopting an intuitive weighting strategy that effectively integrates these synthesized neighbors into the model. The linear evaluation of MNN on CIFAR-10, CIFAR,100, STL-10, and Tiny ImageNet shows an improvement of 0.41% to 1.96% over the previous state-of-the-art (SOTA) methods and 1.82% to 8.8% over the Mean Shift (MSF) method. Furthermore, the proposed components can be seamlessly integrated into other self-supervised learning algorithms to enhance their performance. Code is available at https://github.com/pc-cp/MNN.

Fund transfer fraud detection: Analyzing irregular transactions and customer relationships with self-attention and graph neural networks

This paper presents a method for identifying fraudulent fund transfers using real bank data, analyzing customer information, transactional activities, and customer relationships. The preprocessing step transforms high-dimensional, irregular transaction time series into regular time series, then further compresses them in a latent space using a self-attention-based autoencoder. To address the scarcity of fraudulent data samples and mitigate training issues caused by data imbalance, various deep generative models, including the conditional variational autoencoder and Wasserstein generative adversarial network, are applied to generate additional fraudulent raw data and augment fraud samples in latent space. The reparameterization trick is integrated into the encoder–decoder structure to boost the model’s generative capabilities. Additionally, a Graph Neural Network (GNN) is used to model customer relationships. The proposed approach utilizes end-to-end learning, integrating the autoencoder’s reconstruction loss, KL divergence loss (when reparameterization trick is applied), and classification loss for fraud detection. To optimize computational resources, neighborhood sampling for GNN is combined with mini-batch training for the autoencoder, improving both training efficiency and model reliability. Comprehensive experiments demonstrate the effectiveness of the proposed fusion network, highlighting the importance of each component and preprocessing step. For example, the areas under the precision–recall curves for fraud detection show notable improvements in our model. For suspicious transactions identified by the bank’s rules, other models range from 0.66% to 22.15%, while our model reached 27%. For non-suspicious transactions, other models range from 2.53% to 22.00%, with our model achieving 22.90%. This model has potential for wider applications in anomaly detection, particularly in datasets with irregular time series and complex customer relationships.

Scattering matrix similarity metric optimization for improved defect characterisation based on dynamic graph attention networks

Ultrasonic scattering matrices contain rich defect information and have great potential for characterising small crack-like defects. However, experimentally measured scattering matrices often exhibit some level of distortions compared to those of the idealised defects, posing challenges for accurate defect characterisation. In this paper, defect characterisation was performed by adopting a nearest neighbour approach based on a scattering matrix database of reference defects, and the test data were contaminated by coherent measurement noise of varying amplitudes. The performance of different similarity metrics on characterisation accuracy was studied, including the Euclidean similarity, cosine similarity, Pearson correlation coefficient, and the structural similarity index. Based on a comprehensive analysis of the strengths and weaknesses of different similarity metrics, we propose a defect characterisation framework by constructing similarity graphs and leveraging advanced graph neural networks. Within the proposed approach, multiple metrics were adopted to quantify the similarity between the scattering matrices of different defects, and an improved dynamic graph attention network was developed based on a customised neighbour sampling strategy to learn the optimal metric from the graph-structured data. Experimental results show that compared to the conventional approach which adopted a globally optimal similarity metric, the proposed method can reduce the root mean squared error for the length and angle predictions by 60.5% and 67.1%, respectively.

A Lightweight Machine-Learning Method for Cloud Removal in Remote Sensing Images Constrained by Conditional Information

Reconstructing cloud-covered regions in remote sensing (RS) images holds great promise for continuous ground object monitoring. A novel lightweight machine-learning method for cloud removal constrained by conditional information (SMLP-CR) is proposed. SMLP-CR constructs a multilayer perceptron with a presingle-connection layer (SMLP) based on multisource conditional information. The method employs multi-scale mean filtering and local neighborhood sampling to gain spatial information while also taking into account multi-spectral and multi-temporal information as well as pixel similarity. Meanwhile, the feature importance from the SMLP provides a selection order for conditional information—homologous images are prioritized over images from the same season as the restoration image, and images with close temporal distances rank last. The results of comparative experiments indicate that SMLP-CR shows apparent advantages in terms of visual naturalness, texture continuity, and quantitative metrics. Moreover, compared with popular deep-learning methods, SMLP-CR samples locally around cloud pixels instead of requiring a large cloud-free training area, so the samples show stronger correlations with the missing data, which demonstrates universality and superiority.