Sample Reweighting Research Articles

ECC-PolypDet: Enhanced CenterNet With Contrastive Learning for Automatic Polyp Detection.

Accurate polyp detection is critical for early colorectal cancer diagnosis. Although remarkable progress has been achieved in recent years, the complex colon environment and concealed polyps with unclear boundaries still pose severe challenges in this area. Existing methods either involve computationally expensive context aggregation or lack prior modeling of polyps, resulting in poor performance in challenging cases. In this paper, we propose the Enhanced CenterNet with Contrastive Learning (ECC-PolypDet), a two-stage training & end-to-end inference framework that leverages images and bounding box annotations to train a general model and fine-tune it based on the inference score to obtain a final robust model. Specifically, we conduct Box-assisted Contrastive Learning (BCL) during training to minimize the intra-class difference and maximize the inter-class difference between foreground polyps and backgrounds, enabling our model to capture concealed polyps. Moreover, to enhance the recognition of small polyps, we design the Semantic Flow-guided Feature Pyramid Network (SFFPN) to aggregate multi-scale features and the Heatmap Propagation (HP) module to boost the model's attention on polyp targets. In the fine-tuning stage, we introduce the IoU-guided Sample Re-weighting (ISR) mechanism to prioritize hard samples by adaptively adjusting the loss weight for each sample during fine-tuning. Extensive experiments on six large-scale colonoscopy datasets demonstrate the superiority of our model compared with previous state-of-the-art detectors.

Toward Bias-Agnostic Recommender Systems: A Universal Generative Framework

User behavior data, such as ratings and clicks, has been widely used to build personalizing models for recommender systems. However, many unflattering factors (e.g., popularity, ranking position, users’ selection) significantly affect the performance of the learned recommendation model. Most existing work on unbiased recommendation addressed these biases from sample granularity (e.g., sample reweighting, data augmentation) or from the perspective of representation learning (e.g., bias-modeling). However, these methods are usually designed for a specific bias, lacking the universal capability to handle complex situations where multiple biases co-exist. Besides, rare work frees itself from laborious and sophisticated debiasing configurations (e.g., propensity scores, imputed values, or user behavior-generating process). Towards this research gap, in this article, we propose a universal G enerative framework for B ias D isentanglement termed as GBD , constantly generating calibration perturbations for the intermediate representations during training to keep them from being affected by the bias. Specifically, a bias-identifier that tries to retrieve the bias-related information from the representations is first introduced. Subsequently, the calibration perturbations are generated to significantly deteriorate the bias-identifier’s performance, making the bias gradually disentangled from the calibrated representations. Therefore, without relying on notorious debiasing configurations, a bias-agnostic model is obtained under the guidance of the bias identifier. We further present its universality by subsuming the representative biases and their mixture under the proposed framework. Finally, extensive experiments on the real-world, synthetic, and semi-synthetic datasets have demonstrated the superiority of the proposed approach against a wide range of recommendation debiasing methods. The code is available at https://github.com/Zhidan-Wang/GBD .

A Novel Air Target Intention Recognition Method Based on Sample Reweighting and Attention-Bi-GRU

A Novel Air Target Intention Recognition Method Based on Sample Reweighting and Attention-Bi-GRU

Geotechnical reliability‐based design optimization updating under changing design scenarios

AbstractIn the context of geotechnical uncertainty characterization, the reliability‐based design optimization (RBDO) provides an effective means to achieve the most balanced design outcome between the targeted safety and the risk‐based cost planning. As a common practice, the geotechnical RBDO is conducted on the premise of prescribed statistical information of those uncertain parameters. Nevertheless, the uncertain geotechnical parameters are usually subject to amendment resulting from further site investigation. In other words, changing design scenarios must be properly handled in advanced geotechnical RBDO. Thus, in this study, a novel reliability updating method using a sample reweighting technique that requires only a single simulation run is developed for RBDO. Simultaneously, the RBDO is readily extendable to RBDO updating under changing design scenarios while avoiding any further re‐evaluation of the performance functions. The proposed RBDO updating method allows for the consideration of a continuous design space encompassing an infinite number of feasible designs, whereas the existing simulation‐based method can only consider a finite number of feasible designs in a discrete design space. Moreover, the efficiency and flexibility of the proposed RBDO updating method are illustrated through a standard RBDO test problem and a practical rock slope design example, where the changing design scenarios of different means, standard deviations, correlation coefficient, and distributions are respectively studied.

Learning to Holistically Detect Bridges From Large-Size VHR Remote Sensing Imagery.

Bridge detection in remote sensing images (RSIs) plays a crucial role in various applications, but it poses unique challenges compared to the detection of other objects. In RSIs, bridges exhibit considerable variations in terms of their spatial scales and aspect ratios. Therefore, to ensure the visibility and integrity of bridges, it is essential to perform holistic bridge detection in large-size very-high-resolution (VHR) RSIs. However, the lack of datasets with large-size VHR RSIs limits the deep learning algorithms' performance on bridge detection. Due to the limitation of GPU memory in tackling large-size images, deep learning-based object detection methods commonly adopt the cropping strategy, which inevitably results in label fragmentation and discontinuous prediction. To ameliorate the scarcity of datasets, this paper proposes a large-scale dataset named GLH-Bridge comprising 6,000 VHR RSIs sampled from diverse geographic locations across the globe. These images encompass a wide range of sizes, varying from 2,048 × 2,048 to 16,384 × 16,384pixels, and collectively feature 59,737 bridges. These bridges span diverse backgrounds, and each of them has been manually annotated, using both an oriented bounding box (OBB) and a horizontal bounding box (HBB). Furthermore, we present an efficient network for holistic bridge detection (HBD-Net) in large-size RSIs. The HBD-Net presents a separate detector-based feature fusion (SDFF) architecture and is optimized via a shape-sensitive sample re-weighting (SSRW) strategy. The SDFF architecture performs inter-layer feature fusion (IFF) to incorporate multi-scale context in the dynamic image pyramid (DIP) of the large-size image, and the SSRW strategy is employed to ensure an equitable balance in the regression weight of bridges with various aspect ratios. Based on the proposed GLH-Bridge dataset, we establish a bridge detection benchmark including the OBB and HBB tasks, and validate the effectiveness of the proposed HBD-Net. Additionally, cross-dataset generalization experiments on two publicly available datasets illustrate the strong generalization capability of the GLH-Bridge dataset. The dataset and source code will be released at https://luo-z13.github.io/GLH-Bridge-page/.

Gradient-aware learning for joint biases: Label noise and class imbalance

Data biases such as class imbalance and label noise always exist in large-scale datasets in real-world. These problems bring huge challenges to deep learning methods. Some previous works adopted loss re-weighting, sample re-weighting, or data-dependent regularization to mitigate the influence of these training biases. But these methods usually pay more attention to class imbalance problem when both the class imbalance and label noise exist in training set simultaneously. These methods may overfit noisy labels, which leads to a great degradation in performance. In this paper, we propose a gradient-aware learning method for the combination of the two biases. During the training process, we update only a part of crucial parameters regularly and rectify the update direction of the rest redundant parameters. This update rule is conducted both in the encoder and classifier of the deep network to decouple label noise and class imbalance implicitly. The experimental results verify the effectiveness of the proposed method on synthetic and real-world data biases.

Improving Load Forecasting Performance via Sample Reweighting

Load forecasting techniques have been well developed and almost all typical models are trying to minimize the error over all equally weighted data samples. However, treating all data equally in the training process is unreasonable and even harmful to model generalization. This letter investigates the issue of data inequality which is crucial but is overlooked in the field of load forecasting. To this end, we propose a novel load forecasting method via sample reweighting to tackle the data inequality issue. The proposed method determines the weights of samples using their gradients and influence functions in the process of training. Case studies on a real-world load dataset using linear models, artificial neural networks, and long short term memory in both deterministic and probabilistic situations all reveal that assigning different weights to various samples can improve forecasting accuracy.

Stable Learning via Sparse Variable Independence

The problem of covariate-shift generalization has attracted intensive research attention. Previous stable learning algorithms employ sample reweighting schemes to decorrelate the covariates when there is no explicit domain information about training data. However, with finite samples, it is difficult to achieve the desirable weights that ensure perfect independence to get rid of the unstable variables. Besides, decorrelating within stable variables may bring about high variance of learned models because of the over-reduced effective sample size. A tremendous sample size is required for these algorithms to work. In this paper, with theoretical justification, we propose SVI (Sparse Variable Independence) for the covariate-shift generalization problem. We introduce sparsity constraint to compensate for the imperfectness of sample reweighting under the finite-sample setting in previous methods. Furthermore, we organically combine independence-based sample reweighting and sparsity-based variable selection in an iterative way to avoid decorrelating within stable variables, increasing the effective sample size to alleviate variance inflation. Experiments on both synthetic and real-world datasets demonstrate the improvement of covariate-shift generalization performance brought by SVI.

Accuracy improvement of laser-induced breakdown spectroscopy coal analysis by hybrid transfer learning.

Laser-induced breakdown spectroscopy (LIBS) has been applied in coal analysis for advantages such as real-time online analysis. Fine-tuning is a transfer learning method that has been utilized in LIBS to improve accuracy in the target domain with a limited training set by introducing a model trained on a different but related source domain. This research proposed a hybrid transfer learning method (HTr-LIBS) to further enhance the performance of LIBS coal analysis by combining fine-tuning with sample reweighting. A neural network was pre-trained on the source domain and target domain training set. The sample weights of the source domain were iteratively adjusted according to the prediction errors. The pre-trained neural network with optimal sample weights was then fine-tuned using the target domain training set. The proposed method significantly improved the analytical accuracy compared to direct modeling using small training sets. When the training set size increased to 19, the R2P of direct modeling for ash content and volatile matter content were 0.8105 and 0.9440, respectively. HTr-LIBS increased the R2P for ash content and volatile matter content to 0.9029 and 0.9627, respectively. The improvements were more significant and stable than fine-tuning of the source domain model without sample reweighting. The introduction of target domain data during pre-training and the iterative adjustment of sample weights both contributed to the improvements.

MSFR-GCN:A Multi-scale Feature Reconstruction Graph Convolutional Network for EEG Emotion and Cognition Recognition.

Graph Convolutional Network (GCN) excels at EEG recognition by capturing brain connections, but previous studies neglect the important EEG feature itself. In this study, we propose MSFR-GCN, a multi-scale feature reconstruction GCN for recognizing emotion and cognition tasks. Specifically, MSFR-GCN includes the MSFR and feature-pool characteristically, with the MSFR consisting of two sub-modules, multi-scale Squeeze-and-Excitation (MSSE) and multi-scale sample re-weighting (MSSR). MSSE assigns weights to channels and frequency bands based on their separate statistical information, while MSSR assigns sample weights based on combined channel and frequency information. The feature-pool, which pools across the feature dimension, is applied after GCN to retain EEG channel information. The MSFR-GCN achieves excellent results in emotion recognition when first tested on two public datasets, SEED and SEED-IV. Than the MSFR-GCN is tested on our self-collected Emotion and Cognition EEG dataset (ECED) for both emotion and cognition classification tasks. The results show MSFR-GCN's good performance in emotion and cognition classification tasks and reveal the implicit relationship between the two, which may provide aid in the rehabilitation of people with cognitive impairments from an emotional perspective.

Class-aware sample reweighting optimal transport for multi-source domain adaptation

Multi-Source Domain Adaptation (MSDA) techniques have attracted widespread attention due to their availability to transfer knowledge from multiple source domains to the unlabeled target domain. Optimal transport (OT) has recently been utilized to measure the distance between distributions in virtue of its robustness. This paper proposes a novel OT-based Class-Aware Sample Reweighting (CASR) method to achieve sample-level fine-grained alignment between multi-source and target. Technically, the class-aware sampling strategy ensures class-level conditional alignment during transport by explicitly selecting samples from each domain. Besides, the sample-reweighting module is designed to allocate specific mass to each transmitted sample, which considers the classification reliability and the spatial information correlation to obtain the alignment priority between target and multi-source and further optimize the transport plan. Extensive experiments conducted on several benchmarks show that CASR presents significant advantages compared with other MSDA methods, and the visualization analysis further demonstrates the effectiveness of each proposed module.

Multi-Scale Interactive Network With Artery/Vein Discriminator for Retinal Vessel Classification.

Automatic classification of retinal arteries and veins plays an important role in assisting clinicians to diagnosis cardiovascular and eye-related diseases. However, due to the high degree of anatomical variation across the population, and the presence of inconsistent labels by the subjective judgment of annotators in available training data, most of existing methods generally suffer from blood vessel discontinuity and arteriovenous confusion, the artery/vein (A/V) classification task still faces great challenges. In this work, we propose a multi-scale interactive network with A/V discriminator for retinal artery and vein recognition, which can reduce the arteriovenous confusion and alleviate the disturbance of noisy label. A multi-scale interaction (MI) module is designed in encoder for realizing the cross-space multi-scale features interaction of fundus images, effectively integrate high-level and low-level context information. In particular, we also design an ingenious A/V discriminator (AVD) that utilizes the independent and shared information between arteries and veins, and combine with topology loss, to further strengthen the learning ability of model to resolve the arteriovenous confusion. In addition, we adopt a sample re-weighting (SW) strategy to effectively alleviate the disturbance from data labeling errors. The proposed model is verified on three publicly available fundus image datasets (AV-DRIVE, HRF, LES-AV) and a private dataset. We achieve the accuracy of 97.47%, 96.91%, 97.79%, and 98.18% respectively on these four datasets. Extensive experimental results demonstrate that our method achieves competitive performance compared with state-of-the-art methods for A/V classification. To address the problem of training data scarcity, we publicly release 100 fundus images with A/V annotations to promote relevant research in the community.

Imprecise subset simulation

The objective of this work is to quantify the uncertainty in probability of failure estimates resulting from incomplete knowledge of the probability distributions for the input random variables. We propose a framework that couples the widely used Subset simulation (SuS) with Bayesian/information theoretic multi-model inference. The process starts with data used to infer probability distributions for the model inputs. Often such data sets are small. Multi-model inference is used to assess uncertainty associated with the model-form and parameters of these random variables in the form of model probabilities and the associated joint parameter probability densities. A sampling procedure is used to construct a set of equally probable candidate probability distributions and an optimal importance sampling distribution is determined analytically from this set. Subset simulation is then performed using this optimal sampling density and the resulting conditional probabilities are re-weighted using importance sampling. The result of this process are empirical probability distributions of failure probabilities that provide direct estimates of the uncertainty in failure probability estimates that result from inference on small data sets. The method is demonstrated to be both computationally efficient – requiring only a single subset simulation and nominal cost of sample re-weighting – and to provide reasonable estimates of the uncertainty in failure probabilities.

Simulation-based inference with approximately correct parameters via maximum entropy

Inferring the input parameters of simulators from observations is a crucial challenge with applications from epidemiology to molecular dynamics. Here we show a simple approach in the regime of sparse data and approximately correct models, which is common when trying to use an existing model to infer latent variables with observed data. This approach is based on the principle of maximum entropy (MaxEnt) and provably makes the smallest change in the latent joint distribution to fit new data. This method requires no likelihood or model derivatives and its fit is insensitive to prior strength, removing the need to balance observed data fit with prior belief. The method requires the ansatz that data is fit in expectation, which is true in some settings and may be reasonable in all settings with few data points. The method is based on sample reweighting, so its asymptotic run time is independent of prior distribution dimension. We demonstrate this MaxEnt approach and compare with other likelihood-free inference methods across three systems: a point particle moving in a gravitational field, a compartmental model of epidemic spread and molecular dynamics simulation of a protein.

Grassmannian graph-attentional landmark selection for domain adaptation

Domain adaptation aims to leverage information from the source domain to improve the classification performance in the target domain. It mainly utilizes two schemes: sample reweighting and feature matching. While the first scheme allocates different weights to individual samples, the second scheme matches the feature of two domains using global structural statistics. The two schemes are complementary with each other, which are expected to jointly work for robust domain adaptation. Several methods combine the two schemes, but the underlying relationship of samples is insufficiently analyzed due to the neglect of the hierarchy of samples and the geometric properties between samples. To better combine the advantages of the two schemes, we propose a Grassmannian graph-attentional landmark selection (GGLS) framework for domain adaptation. GGLS presents a landmark selection scheme using attention-induced neighbors of the graphical structure of samples and performs distribution adaptation and knowledge adaptation over Grassmann manifold. the former treats the landmarks of each sample differently, and the latter avoids feature distortion and achieves better geometric properties. Experimental results on different real-world cross-domain visual recognition tasks demonstrate that GGLS provides better classification accuracies compared with state-of-the-art domain adaptation methods.

Meta Learning for Blind Image Quality Assessment Via Adaptive Sample Re-Weighting

Meta Learning for Blind Image Quality Assessment Via Adaptive Sample Re-Weighting

A framework for feature selection through boosting

As dimensions of datasets in predictive modelling continue to grow, feature selection becomes increasingly practical. Datasets with complex feature interactions and high levels of redundancy still present a challenge to existing feature selection methods. We propose a novel framework for feature selection that relies on boosting, or sample re-weighting, to select sets of informative features in classification problems. The method uses as its basis the feature rankings derived from fast and scalable tree-boosting models, such as XGBoost. We compare the proposed method to standard feature selection algorithms on 9 benchmark datasets. We show that the proposed approach reaches higher accuracies with fewer features on most of the tested datasets, and that the selected features have lower redundancy.

Preventing Overfitting via Sample Reweighting for Recommender System Incremental Update (Student Abstract)

Incremental update of recommender system models using only newly arrived data may easily cause the model to overfit to the current data. To address this issue without relying on historical data, we propose a sample reweighting method based on prediction performance of previous model on current data. The proposed method effectively alleviates the problem of overfitting and improves the performance of incremental update.

Balanced Open Set Domain Adaptation via Centroid Alignment

Open Set Domain Adaptation (OSDA) is a challenging domain adaptation setting which allows the existence of unknown classes on the target domain. Although existing OSDA methods are good at classifying samples of known classes, they ignore the classification ability for the unknown samples, making them unbalanced OSDA methods. To alleviate this problem, we propose a balanced OSDA methods which could recognize the unknown samples while maintain high classification performance for the known samples. Specifically, to reduce the domain gaps, we first project the features to a hyperspherical latent space. In this space, we propose to bound the centroid deviation angles to not only increase the intra-class compactness but also enlarge the inter-class margins. With the bounded centroid deviation angles, we employ the statistical Extreme Value Theory to recognize the unknown samples that are misclassified into known classes. In addition, to learn better centroids, we propose an improved centroid update strategy based on sample reweighting and adaptive update rate to cooperate with centroid alignment. Experimental results on three OSDA benchmarks verify that our method can significantly outperform the compared methods and reduce the proportion of the unknown samples being misclassified into known classes.

PREMERE: Meta-Reweighting via Self-Ensembling for Point-of-Interest Recommendation

Point-of-interest (POI) recommendation has become an important research topic in these days. The user check-in history used as the input to POI recommendation is very imbalanced and noisy because of sparse and missing check-ins. Although sample reweighting is commonly adopted for addressing this challenge with the input data, its fixed weighting scheme is often inappropriate to deal with different characteristics of users or POIs. Thus, in this paper, we propose PREMERE, an adaptive weighting scheme based on meta-learning. Because meta-data is typically required by meta-learning but is inherently hard to obtain in POI recommendation, we self-generate the meta-data via self-ensembling. Furthermore, the meta-model architecture is extended to deal with the scarcity of check-ins. Thorough experiments show that replacing a weighting scheme with PREMERE boosts the performance of the state-of-the-art recommender algorithms by 2.36–26.9% on three benchmark datasets.