Hardness Of Samples Research Articles

Masked Modeling-Based Ultrasound Image Classification via Self-Supervised Learning.

Recently, deep learning-based methods have emerged as the preferred approach for ultrasound data analysis. However, these methods often require large-scale annotated datasets for training deep models, which are not readily available in practical scenarios. Additionally, the presence of speckle noise and other imaging artifacts can introduce numerous hard examples for ultrasound data classification. In this paper, drawing inspiration from self-supervised learning techniques, we present a pre-training method based on mask modeling specifically designed for ultrasound data. Our study investigates three different mask modeling strategies: random masking, vertical masking, and horizontal masking. By employing these strategies, our pre-training approach aims to predict the masked portion of the ultrasound images. Notably, our method does not rely on externally labeled data, allowing us to extract representative features without the need for human annotation. Consequently, we can leverage unlabeled datasets for pre-training. Furthermore, to address the challenges posed by hard samples in ultrasound data, we propose a novel hard sample mining strategy. To evaluate the effectiveness of our proposed method, we conduct experiments on two datasets. The experimental results demonstrate that our approach outperforms other state-of-the-art methods in ultrasound image classification. This indicates the superiority of our pre-training method and its ability to extract discriminative features from ultrasound data, even in the presence of hard examples.

Improving Human Activity Recognition With Wearable Sensors Through BEE: Leveraging Early Exit and Gradient Boosting.

Early-exiting has recently provided an ideal solution for accelerating activity inference by attaching internal classifiers to deep neural networks. It allows easy activity samples to be predicted at shallower layers, without executing deeper layers, hence leading to notable adaptiveness in terms of accuracy-speed trade-off under varying resource demands. However, prior most works typically optimize all the classifiers equally on all types of activity data. As a result, deeper classifiers will only see hard samples during test phase, which renders the model suboptimal due to the training-test data distribution mismatch. Such issue has been rarely explored in the context of activity recognition. In this paper, to close the gap, we propose to organize all these classifiers as a dynamic-depth network and jointly optimize them in a similar gradient-boosting manner. Specifically, a gradient-rescaling is employed to bound the gradients of parameters at different depths, that makes such training procedure more stable. Particularly, we perform a prediction reweighting to emphasize current deep classifier while weakening the ensemble of its previous classifiers, so as to relieve the shortage of training data at deeper classifiers. Comprehensive experiments on multiple HAR benchmarks including UCI-HAR, PAMAP2, UniMiB-SHAR, and USC-HAD verify that it is state-of-the-art in accuracy and speed. A real implementation is measured on an ARM-based mobile device.

Relation-Preserving Feature Embedding for Unsupervised Person Re-Identification

Some unsupervised approaches have been proposed recently for the person re-identification (ReID) problem since annotations of samples across cameras are time-consuming. However, most of these methods focus on the appearance content of the sample itself, and thus seldom take the structure relations among samples into account when learning the feature representation, which would provide a valuable guide for learning the representations of the samples. Thus hard samples may not be well solved due to the limited or even misleading information of the sample itself. To address this issue, in this paper, we propose a Relation-Preserving Feature Embedding (RPE) model that leverages structure relations among samples to boost the performance of the unsupervised person ReID methods without requiring any sample annotations. RPE aims at integrating the sample content and the neighborhood structure relations among samples into the learning of feature embeddings by combining the advantages of the autoencoder and graph autoencoder. Specifically, a relation and content information fusion (RCIF) module is proposed to dynamically merge the information from both perspectives of content and relation levels for feature embedding learning. Also, due to the lack of the identity labels of samples, we adopt an adaptive optimization strategy to update the affinity relations among samples instead of the reconstruction of the whole affinity matrix for optimizing the RPE model, which is more suitable for the unsupervised ReID task. Rigorous experiments on three widely-used large-scale benchmarks for person ReID demonstrate the superiority of the proposed method over current state-of-the-art unsupervised methods.

MixIR: Mixing Input and Representations for Contrastive Learning.

Recently, contrastive learning has shown significant progress in learning visual representations from unlabeled data. The core idea is training the backbone to be invariant to different augmentations of an instance. While most methods only maximize the feature similarity between two augmented data, we further generate more challenging training samples and force the model to keep predicting aggregated representation on these hard samples. In this article, we propose MixIR, a mixture-based approach upon the traditional Siamese network. On the one hand, we input two augmented images of an instance to the backbone and obtain the aggregated representation by performing an elementwise maximum of two features. On the other hand, we take the mixture of these augmented images as input and expect the model prediction to be close to the aggregated representation. In this way, the model could access more variant data samples of an instance and keep predicting invariant representations for them. Thus, the learned model is more discriminative compared with previous contrastive learning methods. Extensive experiments on large-scale datasets show that MixIR steadily improves the baseline and achieves competitive results with state-of-the-art methods. Our code is available at https://github.com/happytianhao/MixIR.

A Practical Contrastive Learning Framework for Single-Image Super-Resolution.

Contrastive learning has achieved remarkable success on various high-level tasks, but there are fewer contrastive learning-based methods proposed for low-level tasks. It is challenging to adopt vanilla contrastive learning technologies proposed for high-level visual tasks to low-level image restoration problems straightly. Because the acquired high-level global visual representations are insufficient for low-level tasks requiring rich texture and context information. In this article, we investigate the contrastive learning-based single-image super-resolution (SISR) from two perspectives: positive and negative sample construction and feature embedding. The existing methods take naive sample construction approaches (e.g., considering the low-quality input as a negative sample and the ground truth as a positive sample) and adopt a prior model (e.g., pretrained very deep convolutional networks proposed by visual geometry group (VGG) model) to obtain the feature embedding. To this end, we propose a practical contrastive learning framework for SISR (PCL-SR). We involve the generation of many informative positive and hard negative samples in frequency space. Instead of utilizing an additional pretrained network, we design a simple but effective embedding network inherited from the discriminator network, which is more task-friendly. Compared with the existing benchmark methods, we retrain them by our proposed PCL-SR framework and achieve superior performance. Extensive experiments have been conducted to show the effectiveness and technical contributions of our proposed PCL-SR thorough ablation studies. The code and resulting models will be released via https://github.com/Aitical/PCL-SISR.

Binary Similarity Few-Shot Object Detection With Modeling of Hard Negative Samples

Binary Similarity Few-Shot Object Detection With Modeling of Hard Negative Samples

Experimental study on the strain energy evolution mechanism of thick and hard sandstone roof in Xinjiang Mining Area

For understanding the mechanical performance and strain energy evolution mechanism of thick hard roof sandstone samples, a sequence of uniaxial compression trials with acoustic emission (AE) monitoring were carried out. The results indicate: (1) The stress-strain curve of the thick hard roof sandstone specimens exhibits distinct stage characteristics. Based on the evolvement of instantaneous axial stiffness, it is separated into Fracture Closure Phase, Elastic Deformation Phase, Steady Fracture Expansion Phase, Unsteady Fracture Expansion Phase, and Post-Peak Phase. (2) The AE energy and cumulative count curves of the thick hard roof sandstone specimens also exhibit significant stage characteristics and can be mutually corroborated with the stage division of the stress-strain curve. (3) Based on the energy conservation principle, the evolution of strain energy density in the thick hard roof sandstone specimens under uniaxial compression loading was analyzed, and plastic strain energy increment was employed to study the stage characteristics of strain energy dissipation. (4) A damage constitutive model for the thick hard roof sandstone specimens was constructed, considering the characteristics of strain energy dissipation. This model effectively describes the stress-strain relationship among the samples, which undergo strain hardening, strain softening, and sudden destruction.

AdaptiveClick: Click-Aware Transformer With Adaptive Focal Loss for Interactive Image Segmentation.

Interactive image segmentation (IIS) has emerged as a promising technique for decreasing annotation time. Substantial progress has been made in pre-and post-processing for IIS, but the critical issue of interaction ambiguity, notably hindering segmentation quality, has been under-researched. To address this, we introduce AdaptiveClick - a click-aware transformer incorporating an adaptive focal loss (AFL) that tackles annotation inconsistencies with tools for mask-and pixel-level ambiguity resolution. To the best of our knowledge, AdaptiveClick is the first transformer-based, mask-adaptive segmentation framework for IIS. The key ingredient of our method is the click-aware mask-adaptive transformer decoder (CAMD), which enhances the interaction between click and image features. Additionally, AdaptiveClick enables pixel-adaptive differentiation of hard and easy samples in the decision space, independent of their varying distributions. This is primarily achieved by optimizing a generalized AFL with a theoretical guarantee, where two adaptive coefficients control the ratio of gradient values for hard and easy pixels. Our analysis reveals that the commonly used Focal and BCE losses can be considered special cases of the proposed AFL. With a plain ViT backbone, extensive experimental results on nine datasets demonstrate the superiority of AdaptiveClick compared to state-of-the-art methods. The source code is publicly available at https://github.com/lab206/AdaptiveClick.

Multisource Soft Labeling and Hard Negative Sampling for Retrieval Distractor Ranking

Multisource Soft Labeling and Hard Negative Sampling for Retrieval Distractor Ranking

DMMG: Dual Min-Max Games for Self-Supervised Skeleton-Based Action Recognition.

In this work, we propose a new Dual Min-Max Games (DMMG) based self-supervised skeleton action recognition method by augmenting unlabeled data in a contrastive learning framework. Our DMMG consists of a viewpoint variation min-max game and an edge perturbation min-max game. These two min-max games adopt an adversarial paradigm to perform data augmentation on the skeleton sequences and graph-structured body joints, respectively. Our viewpoint variation min-max game focuses on constructing various hard contrastive pairs by generating skeleton sequences from various viewpoints. These hard contrastive pairs help our model learn representative action features, thus facilitating model transfer to downstream tasks. Moreover, our edge perturbation min-max game specializes in building diverse hard contrastive samples through perturbing connectivity strength among graph-based body joints. The connectivity-strength varying contrastive pairs enable the model to capture minimal sufficient information of different actions, such as representative gestures for an action while preventing the model from overfitting. By fully exploiting the proposed DMMG, we can generate sufficient challenging contrastive pairs and thus achieve discriminative action feature representations from unlabeled skeleton data in a self-supervised manner. Extensive experiments demonstrate that our method achieves superior results under various evaluation protocols on widely-used NTU-RGB+D, NTU120-RGB+D and PKU-MMD datasets.

Graph Reasoning With Supervised Contrastive Learning for Legal Judgment Prediction.

Given the fact descriptions of legal cases, the legal judgment prediction (LJP) problem aims to determine three judgment tasks of law articles, charges, and the term of penalty. Most existing studies have considered task dependencies while neglecting the prior dependencies of labels among different tasks. Therefore, how to make better use of the information on the relation dependencies among tasks and labels becomes a crucial issue. To this end, we transform the text classification problem into a node classification framework based on graph reasoning and supervised contrastive learning (SCL) techniques, named GraSCL. Specifically, we first design a graph reasoning network to model the potential dependency structures and facilitate relational learning under various graph topologies. Then, we introduce the SCL method for the LJP task to further leverage the label relation on the graph. To accommodate the node classification settings, we extend the traditional SCL method to novel variants for SCL at the node level, which allows the GraSCL framework to be trained efficiently even with small batches. Furthermore, to recognize the importance of hard negative samples in contrastive learning, we introduce a simple yet effective technique called online hard negative mining (OHNM) to enhance our SCL approach. This technique complements our SCL method and enables us to control the number and complexity of negative samples, leading to further improvements in the model's performance. Finally, extensive experiments are conducted on two well-known benchmarks, demonstrating the effectiveness and rationality of our proposed SCL approach as compared to the state-of-the-art competitors.

DiscrimLoss: A Universal Loss for Hard Samples and Incorrect Samples Discrimination

DiscrimLoss: A Universal Loss for Hard Samples and Incorrect Samples Discrimination

ReCurricularFace: Revisiting CurricularFace for Hard Sample Mining

ReCurricularFace: Revisiting CurricularFace for Hard Sample Mining

Meta Pairwise Relationship Distillation for Unsupervised Person Re-Identification.

Unsupervised person re-identification (Re-ID) is challenging due to the lack of ground-truth labels. Most existing methods rely on pseudo labels estimated via iterative clustering and thus are highly susceptible to performance penalties incurred by the inaccurate estimated number of clusters. Alternatively, we utilize the sample pairs with pairwise pseudo labels to guide the feature learning to avoid the dilemma of determining cluster numbers. In this article, we propose a meta pairwise relationship distillation (MPRD) method that incorporates a graph convolutional network (GCN) to provide high-fidelity pairwise relationships to supervise the model training. A small amount of metadata with very-confidence pairwise relationships and the unlabeled pairs with the provided pseudo pairwise relationships participate in the GCN training. Besides, we introduce a hard sample deduction (HSD) module to timely mine the sample pairs with error-prone pairwise pseudo labels to mitigate the misled optimization by noisy labels. Furthermore, since the features of each positive pair represent the same person, we design a positive pair alignment (PPA) module to reduce the redundant information in each feature, which is achieved by minimizing the difference between each positive pair's feature distributions. Extensive experiments on the Market-1501, DukeMTMC-reID, and MSMT17 datasets show that our method outperforms the state-of-the-art unsupervised methods.

GeoDTR+: Toward Generic Cross-View Geolocalization via Geometric Disentanglement.

Cross-View Geo-Localization (CVGL) estimates the location of a ground image by matching it to a geo-tagged aerial image in a database. Recent works achieve outstanding progress on CVGL benchmarks. However, existing methods still suffer from poor performance in cross-area evaluation, in which the training and testing data are captured from completely distinct areas. We attribute this deficiency to the lack of ability to extract the geometric layout of visual features and models' overfitting to low-level details. Our preliminary work[1] introduced a Geometric Layout Extractor (GLE) to capture the geometric layout from input features. However, the previous GLE does not fully exploit information in the input feature. In this work, we propose GeoDTR+ with an enhanced GLE module that better models the correlations among visual features. To fully explore the LS techniques from our preliminary work, we further propose Contrastive Hard Samples Generation (CHSG) to facilitate model training. Extensive experiments show that GeoDTR+ achieves state-of-the-art (SOTA) results in cross-area evaluation on CVUSA[2], CVACT[3], and VIGOR[4] by a large margin ( 16.44%, 22.71%, and 13.66% without polar transformation) while keeping the same-area performance comparable to existing SOTA. Moreover, we provide detailed analyses of GeoDTR+. Our code will be available at https://gitlab.com/vail-uvm/geodtr_plus.

Prototypical Bidirectional Adaptation and Learning for Cross-Domain Semantic Segmentation

Cross-domain semantic segmentation, which aims to address the distribution shift while adapting from a labeled source domain to an unlabeled target domain, has achieved great progress in recent years. However, most existing work adopts a source-to-target adaptation path, which often suffers from clear class mismatching or class imbalance issues. We design PBAL, a prototypical bidirectional adaptation and learning technique that introduces bidirectional prototype learning and prototypical self-training for optimal inter-domain alignment and adaptation. We perform bidirectional alignments in a complementary and cooperative manner which balances both dominant and tail categories as well as easy and hard samples effectively. In addition, We derive prototypes efficiently from a source-trained classifier, which enables class-aware adaptation as well as synchronous prototype updating and network optimization. Further, we re-examine self-training and introduce prototypical contrast above it which greatly improves inter-domain alignment by promoting better intra-class compactness and inter-class separability in the feature space. Extensive experiments over two widely studied benchmarks show that the proposed PBAL achieves superior domain adaptation performance as compared with the state-of-the-art.

Gradient-Guided Network With Fourier Enhancement for Glioma Segmentation in Multimodal 3D MRI.

Glioma segmentation is a crucial task in computer-aided diagnosis, requiring precise discrimination between lesions and normal tissue at the pixel level. Popular methods neglect crucial edge information, leading to inaccurate contour delineation. Moreover, global information has been proven beneficial for segmentation. The feature representations extracted by convolution neural networks often struggle with local-related information owing to the limited receptive fields. To address these issues, we propose a novel edge-aware segmentation network that incorporates a dual-path gradient-guided training strategy with Fourier edge-enhancement for precise glioma segmentation, a.k.a. GFNet. First, we introduce a Dual-path Gradient-guided Training strategy (DGT) based on a Siamese network guiding the optimizing direction of one path by the gradient from the other path. DGT pays attention to the indistinguishable pixels with large weight-updating gradient, such as the pixels near the boundary, to guide the network training, addressing hard samples. Second, to further perceive the edge information, we derive a Fourier Edge-enhancement Module (FEM) to augment feature edges with high-frequency representations from the spectral domain, providing global information and edge details. Extensive experiments on public glioma segmentation datasets, BraTS2020 and Medical Segmentation Decathlon (MSD) glioma and prostate segmentation, demonstrate that GFNet achieves competitive performance compared to other state-of-the-art methods, both qualitatively and quantitatively.

Spatial–Spectral Graph Contrastive Clustering With Hard Sample Mining for Hyperspectral Images

Spatial–Spectral Graph Contrastive Clustering With Hard Sample Mining for Hyperspectral Images

MIMR: Modality-Invariance Modeling and Refinement for unsupervised visible-infrared person re-identification

Compared to visible modality person re-identification that handles only the intra-modality discrepancy, visible-infrared person re-identification (VI-ReID) aims to achieve the match between visible modality and infrared modality. However, supervised VI-ReID methods are limited in flexibility due to their dependence on labeled data. Although several unsupervised VI-ReID methods have been developed, they usually ignore the impact of hard samples and noisy labels. In this paper, we propose the modality-invariance modeling and refinement (MIMR) framework for unsupervised VI-ReID. For the first issue, we first conduct cross-modality image translation, then enable the encoder to extract modality-invariant feature representation by aligning translated versions with original samples, and finally use cross-modality clustering and hard sample contrastive loss to handle the hard samples. For the second issue, we believe that ambiguous samples are usually assigned noisy labels, so we design ambiguity-oriented pseudo label refinement (APLR), which evaluates ambiguity from both the sample itself and the corresponding translated version, rather than discards hard samples as existing methods. Extensive experiments demonstrate that MIMR achieves superior performance compared to state-of-the-art unsupervised methods and even surpasses early supervised methods.

FAUC-S: Deep AUC maximization by focusing on hard samples

Deep AUC maximization (DAM) is a popular method to deal with complex imbalanced classification problems. It learns a deep neural network by minimizing a surrogate AUC loss. The mostly used surrogates are the pairwise square loss and its variants. However, they are usually used in regression problems and are sensitive to noise samples. The easy samples suffer larger losses than those without correct classification, which is unreasonable in optimization problems. In this paper, we propose a focus AUC loss based on samples (FAUC-S) by constructing a differentiable weight function that identifies hard and easy samples, which makes easy samples have small losses, and hard samples have large losses. It is more reasonable than the traditional AUC loss while having the same advantages on large-scale datasets. In experiments, we use an end-to-end compositional DAM to investigate the effect of different weight functions on the AUC loss and compare it with other methods. Experimental results on several benchmark datasets demonstrate that FAUC-S achieves superior performance in terms of AUC compared to existing state-of-the-art methods.