Hardness Of Samples Research Articles

IfCMD: A Novel Method for Radar Target Detection under Complex Clutter Backgrounds

Traditional radar target detectors, which are model-driven, often suffer remarkable performance degradation in complex clutter environments due to the weakness in modeling the unpredictable clutter. Deep learning (DL) methods, which are data-driven, have been introduced into the field of radar target detection (RTD) since their intrinsic non-linear feature extraction ability can enhance the separability between targets and the clutter. However, existing DL-based detectors are unattractive since they require a large amount of independent and identically distributed (i.i.d.) training samples of target tasks and fail to be generalized to the other new tasks. Given this issue, incorporating the strategy of meta-learning, we reformulate the RTD task as a few-shot classification problem and develop the Inter-frame Contrastive Learning-Based Meta Detector (IfCMD) to generalize to the new task efficiently with only a few samples. Moreover, to further separate targets from the clutter, we equip our model with Siamese architecture and introduce the supervised contrastive loss into the proposed model to explore hard negative samples, which have the targets overwhelmed by the clutter in the Doppler domain. Experimental results on simulated data demonstrate competitive detection performance for moving targets and superior generalization ability for new tasks of the proposed method.

Metric learning with adversarial hard negative samples for tag recommendation

Metric learning with adversarial hard negative samples for tag recommendation

Star Formation Rates in [Ne V] 3426 Å Selected Active Galactic Nuclei: Evidence for a Decrease along the Main Sequence?

Studying the behavior along the galaxy main sequence is key in furthering our understanding of the possible connection between active galactic nucleus (AGN) activity and star formation. We select a sample of 1215 AGN from the catalog of Sloan Digital Sky Survey galaxy properties from the Portsmouth group by detection of the high-ionization [Ne v] 3426 Å emission line. Our sample extends from 1040 to 1042.5 erg s−1 in [Ne v] luminosity in a redshift range z = 0.17 to 0.57. We compare the specific star formation rates (sSFRs; SFR scaled by galaxy mass) obtained from the corrected [O ii] and Hα luminosities, and the spectral energy distribution (SED)–determined values from Portsmouth. We find that the emission-line-based sSFR values are unreliable for the [Ne v] sample due to the AGN contribution, and proceed with the SED sSFRs for our study of the main sequence. We find evidence for a decrease in sSFR along the main sequence in the [Ne v] sample, which is consistent with results from the hard X-ray Burst Alert Telescope AGN sample, which extends to lower redshifts than our [Ne v] sample. Although we do not find evidence that the concurrent AGN activity is suppressing star formation, our results are consistent with a lower gas fraction in the host galaxies of the AGN as compared to that of the star-forming galaxies. If the evacuation of gas, and therefore suppression of star formation, is due to AGN activity, it must have occurred in a previous epoch.

Learning From Human Educational Wisdom: A Student-Centered Knowledge Distillation Method.

Existing studies on knowledge distillation typically focus on teacher-centered methods, in which the teacher network is trained according to its own standards before transferring the learned knowledge to a student one. However, due to differences in network structure between the teacher and the student, the knowledge learned by the former may not be desired by the latter. Inspired by human educational wisdom, this paper proposes a Student-Centered Distillation (SCD) method that enables the teacher network to adjust its knowledge transfer according to the student network's needs. We implemented SCD based on various human educational wisdom, e.g., the teacher network identified and learned the knowledge desired by the student network on the validation set, and then transferred it to the latter through the training set. To address the problems of current deficiency knowledge, hard sample learning and knowledge forgetting faced by a student network in the learning process, we introduce and improve Proportional-Integral-Derivative (PID) algorithms from automation fields to make them effective in identifying the current knowledge required by the student network. Furthermore, we propose a curriculum learning-based fuzzy strategy and apply it to the proposed PID control algorithm, such that the student network in SCD can actively pay attention to the learning of challenging samples after with certain knowledge. The overall performance of SCD is verified in multiple tasks by comparing it with state-of-the-art ones. Experimental results show that our student-centered distillation method outperforms existing teacher-centered ones.

Progressive supervised pedestrian detection algorithm for green edge–cloud computing

In this paper, we present a novel supervised pedestrian detection algorithm tailored for Green Edge–Cloud Computing (GECC), aimed at addressing the challenges associated with pre-trained models in real-world settings. These models often exhibit performance degradation due to the divergence between training datasets and complex real-world scenarios. To bridge this gap, our proposed algorithm employs a progressive occlusion-aware iterative training strategy that significantly enhances the representation of occluded pedestrians, a prevailing issue in dense urban environments. Additionally, we introduce a K-stratified hard sampling strategy, which accelerates the training process and substantially lowers the energy consumption required for model retraining. We also explore a refined non-maximal value suppression algorithm that employs neighboring centroids separation, improving the precision of pedestrian detection within crowded scenes. Through rigorous experiments on established pedestrian detection datasets, we demonstrate that our algorithm not only elevates pedestrian detection accuracy in high-density situations but also markedly reduces the model training duration.

Microsyringe-based slug-flow microextraction for rapid and accurate determination of antibiotics in highly saline seawater

BackgroundExtensive use of antibiotics leads to widespread environmental pollution, endangering ecosystems, and human health. It is particularly concerning, posing global threats requiring urgent attention and action. In this regard, the shift to mass spectrometry in determining antibiotics is highly desirable. Significant progress has been made in analyzing and optimizing the sensitivity of high-salt samples. However, the persistence of cumbersome operational procedures presents a significant challenge to this shift. Thus, the persistence of complex operational procedures needs to be addressed. ResultsIn this study, a rapid and direct method for determining antibiotics in highly saline environmental water samples using microsyringe-based slug-flow microextraction (MSFME)-droplet spray ionization (DSI) mass spectrometry (MS) has been described. The proposed method successfully detected clarithromycin, ofloxacin, and sulfadimidine in seawater within a linear range of 1–1200 ng mL−1, with low limits of detection of 0.19 ng mL−1, 0.17 ng mL−1, and 0.20 ng mL−1, respectively (Signal/Noise = 3). Additionally, spiked real seawater samples of all three antibiotics demonstrated satisfactory recoveries (95.1–107.5%) and precision (RSD≤8.8%). The MSFME–treated high-salt sample (3.5 wt%) showed a mass spectral response intensity 4–5 orders of magnitude higher than the untreated medium-salt sample (0.35 wt%). Furthermore, exploration of the applicability of MSFME showed that it is suitable not only for high-salinity (3.5 wt%) samples but also for salt-free or low-salt and hard water samples rich in calcium and magnesium ions. SignificanceComparisons with other methods, complex laboratory setups for sample processing are now simplified to a single step, completing the entire process, including desalination and detection, MSFME–DSI–MS provides faster results in less than 1 min while maintaining sensitivity comparable to that of other detection methods. In conclusion, this advancement provides an exceptionally simplified protocol for the rapid, highly sensitive, and quantitative determination of antibiotics in environmental water samples.

Hospital readmission prediction with hybrid‐sampling and self‐paced balance learning

SummaryHospital readmission prediction is defined as an evaluation task to model the historical medical data to predict whether patients will be readmitted after discharge. In the past few years, many feasible and effective prediction methods have been proposed, however, most of them neglect the imbalanced distribution of medical data, which causes great difficulties in modeling. Thus, we proposed a new hospital readmission prediction method, which utilizes hybrid‐sampling and self‐paced balance learning strategies to solve the class‐imbalance problem. To be specifically, we first employ an interference negative sample deletion strategy to reduce the probability of important majority class samples being deleted. Then, we design a hard positive sample generation strategy to generate more positive samples. Meanwhile, we also introduce a self‐paced balance factor during the oversampling process to improve the similarity between newly generated minority class samples and hard positive samples. Finally, we perform the experiments on six real‐world readmission datasets to indicate the superiority of the proposed method.

CoRTEx: contrastive learning for representing terms via explanations with applications on constructing biomedical knowledge graphs.

Biomedical Knowledge Graphs play a pivotal role in various biomedical research domains. Concurrently, term clustering emerges as a crucial step in constructing these knowledge graphs, aiming to identify synonymous terms. Due to a lack of knowledge, previous contrastive learning models trained with Unified Medical Language System (UMLS) synonyms struggle at clustering difficult terms and do not generalize well beyond UMLS terms. In this work, we leverage the world knowledge from large language models (LLMs) and propose Contrastive Learning for Representing Terms via Explanations (CoRTEx) to enhance term representation and significantly improves term clustering. The model training involves generating explanations for a cleaned subset of UMLS terms using ChatGPT. We employ contrastive learning, considering term and explanation embeddings simultaneously, and progressively introduce hard negative samples. Additionally, a ChatGPT-assisted BIRCH algorithm is designed for efficient clustering of a new ontology. We established a clustering test set and a hard negative test set, where our model consistently achieves the highest F1 score. With CoRTEx embeddings and the modified BIRCH algorithm, we grouped 35 580 932 terms from the Biomedical Informatics Ontology System (BIOS) into 22 104 559 clusters with O(N) queries to ChatGPT. Case studies highlight the model's efficacy in handling challenging samples, aided by information from explanations. By aligning terms to their explanations, CoRTEx demonstrates superior accuracy over benchmark models and robustness beyond its training set, and it is suitable for clustering terms for large-scale biomedical ontologies.

An end-to-end multi-scale airway segmentation framework based on pulmonary CT image

Objective. Automatic and accurate airway segmentation is necessary for lung disease diagnosis. The complex tree-like structures leads to gaps in the different generations of the airway tree, and thus airway segmentation is also considered to be a multi-scale problem. In recent years, convolutional neural networks have facilitated the development of medical image segmentation. In particular, 2D CNNs and 3D CNNs can extract different scale features. Hence, we propose a two-stage and 2D + 3D framework for multi-scale airway tree segmentation. Approach. In stage 1, we use a 2D full airway SegNet(2D FA-SegNet) to segment the complete airway tree. Multi-scale atros spatial pyramid and Atros Residual Skip connection modules are inserted to extract different scales feature. We designed a hard sample selection strategy to increase the proportion of intrapulmonary airway samples in stage 2. 3D airway RefineNet (3D ARNet) as stage 2 takes the results of stage 1 as a priori information. Spatial information extracted by 3D convolutional kernel compensates for the loss of in 2D FA-SegNet. Furthermore, we added false positive losses and false negative losses to improve the segmentation performance of airway branches within the lungs. Main results. We performed data enhancement on the publicly available dataset of ISICDM 2020 Challenge 3, and on which evaluated our method. Comprehensive experiments show that the proposed method has the highest dice similarity coefficient (DSC) of 0.931, and IoU of 0.871 for the whole airway tree and DSC of 0.699, and IoU of 0.543 for the intrapulmonary bronchi tree. In addition, 3D ARNet proposed in this paper cascaded with other state-of-the-art methods to increase detected tree length rate by up to 46.33% and detected tree branch rate by up to 42.97%. Significance. The quantitative and qualitative evaluation results show that our proposed method performs well in segmenting the airway at different scales.

Active Dynamic Weighting for multi-domain adaptation

Multi-source unsupervised domain adaptation aims to transfer knowledge from multiple labeled source domains to an unlabeled target domain. Existing methods either seek a mixture of distributions across various domains or combine multiple single-source models for weighted fusion in the decision process, with little insight into the distributional discrepancy between different source domains and the target domain. Considering the discrepancies in global and local feature distributions between different domains and the complexity of obtaining category boundaries across domains, this paper proposes a novel Active Dynamic Weighting (ADW) for multi-source domain adaptation. Specifically, to effectively utilize the locally advantageous features in the source domains, ADW designs a multi-source dynamic adjustment mechanism during the training process to dynamically control the degree of feature alignment between each source and target domain in the training batch. In addition, to ensure the cross-domain categories can be distinguished, ADW devises a dynamic boundary loss to guide the model to focus on the hard samples near the decision boundary, which enhances the clarity of the decision boundary and improves the model’s classification ability. Meanwhile, ADW applies active learning to multi-source unsupervised domain adaptation for the first time, guided by dynamic boundary loss, proposes an efficient importance sampling strategy to select target domain hard samples to annotate at a minimal annotation budget, integrates it into the training process, and further refines the domain alignment at the category level. Experiments on various benchmark datasets consistently demonstrate the superiority of our method.

DCSN: Focusing on hard samples mining in small-sample fault diagnosis of marine engine

Fault samples of marine engine are extremely scarce, and there are unavoidably some hard samples with small inter-class differences, which pose a serious challenge to fault diagnosis of marine engines. This paper proposes a deep metric learning method, namely deep concentric Siamese network (DCSN), to apply strong forces to hard samples towards their corresponding correct distribution areas under small-sample conditions. First, DCSN is committed to learn discriminative information from limited fault samples through a carefully designed metric learning strategy. Then, DCSN distinguishes hard samples using inner and outer boundaries, and applies strong forces to them, making the deep model more focusing on the correct classification of hard samples. Third, DCSN shrinks the distribution area of intra-class samples, which improves intra-class compactness and inter-class separability. Finally, the experimental results on the marine engine fault dataset show that the proposed DCSN yields higher diagnostic performance compared to the considered competitive methods.

Transfer learning method for specific emitter identification based on pseudo‐labelling and meta‐learning

AbstractSpecific emitter identification (SEI) represents a prominent research direction within the electronic countermeasures domain aimed at discerning carrier identity attributes by analysing subtle radar characteristics. At present, most established SEI techniques assume that both the source and target domain (TD) data adhere to the same distribution. However, this assumption is invalidated by semantic drift which frequently occurs between TD and source domain (SD) samples owing to variations in the collection environment, equipment, and other factors. Considering the aforementioned challenges, this article introduces a transfer learning approach for SEI to leverage pseudo‐label integration within the framework of meta‐learning. This approach employs the bispectral perimeter integral for extracting emitter signal features to construct a feature extractor and basic learner based on CNN13. To label and filter the TD samples, the proposed approach utilises the multiple pseudo‐label serial filtering mechanism, which comprises positive and negative pseudo‐labelling strategies, label uncertainty prediction methods, and hard sample filtering strategies. Ultimately, to address algorithmic real‐time requirements, the labelled TD samples are integrated into the feature extractor and learner of the SD through meta‐learning to facilitate the transfer of TD features to the SD training model. Experimental validation conducted on a real radar dataset demonstrated that the proposed algorithm significantly enhances identification accuracy, exhibiting an improvement from approximately 50% to approximately 90%. Furthermore, the algorithm exhibits a short runtime and robust adaptability, effectively catering to the demands of practical scenarios.

MSCLK: Multi-scale fully separable convolution neural network with large kernels for early diagnosis of Alzheimer’s disease

Alzheimer’s disease (AD) is identified as a central nervous system disease that exhibits irreversible degeneration, while mild cognitive impairment (MCI) is viewed as the preliminary stage of AD, and its pathogenesis is notably intricate. MCI contains two stages: early MCI (EMCI), and late MCI (LMCI). EMCI diagnosis can prevent EMCI from progressing to LMCI, and then to AD. Therefore, accurate diagnosis of EMCI/LMCI is crucial for developing the early intervention and treatment strategies of AD. Currently, most existing EMCI/LMCI diagnostic methods use single modality images, while different modality images carry different complementary information that helps for accurate diagnosis of EMCI/LMCI, and the lesion area is usually not limited to a single brain area, which involves multiple regions. In this case, conventional convolution operations cannot be able to accurately extract the pathological features of AD. In this work, we propose a novel Multi-scale fully Separable Convolution neural network with Large Kernels (MSCLK) method to diagnose early Alzheimer’s disease with structural Magnetic Resonance Imaging (sMRI) images. MSCLK mainly consists of the multi-scale 3D fully separable convolution modules and the deep metric learning module. The multi-scale convolution that contains both small and large kernels is used to effectively capture the discrimination features of different scale acceptance domains. 3D fully separable convolution is used to reduce parameters and overfitting. The deep metric learning is used to learn hard samples that are similar but belong to different classes. We also propose a variant method of MSCLK (called MSCLK-Fusion MRI and PET, MSCLK-FMP) by adding the pixel-level fusion module and feature-level fusion module into the MSCLK framework to integrate the sMRI image and the Positron Emission Computed Tomography (PET) image for further improving the accuracy of EMCI vs. LMCI classification task. The pixel-level fusion is used to achieve early pixel-level fusion of sMRI and PET images, and the feature-level fusion is used to achieve high-dimensional feature-level fusion of sMRI and PET images. Experimental results on the ADNI database show that the performance of our MSCLK and MSCLK-FMP are superior to other state-of-the-art methods. The accuracy of MSCLK achieves 98.89%, 95.97%, 96.39% and 98.76% for AD vs. EMCI, AD vs. LMCI, EMCI vs. NC and LMCI vs. NC classification tasks, respectively, and MSCLK-FMP achieves 93.93% for EMCI vs. LMCI classification task, indicating that MSCLK/MSCLK-FMP can be effectively used for diagnosing MCI patients. Moreover, our MSCLK-FMP is capable of pinpointing key brain areas involved in the pathological progression of MCI, such as the Temporal_Inf, the Hippocampus, the Precuneus, the Precentral, and the Thalamus. These findings contribute to uncovering the early onset of AD pathogenesis.

TODO-Net: Temporally Observed Domain Contrastive Network for 3-D Early Action Prediction.

Early action prediction aiming to recognize which classes the actions belong to before they are fully conveyed is a very challenging task, owing to the insufficient discrimination information caused by the domain gaps among different temporally observed domains. Most of the existing approaches focus on using fully observed temporal domains to "guide" the partially observed domains while ignoring the discrepancies between the harder low-observed temporal domains and the easier highly observed temporal domains. The recognition models tend to learn the easier samples from the highly observed temporal domains and may lead to significant performance drops on low-observed temporal domains. Therefore, in this article, we propose a novel temporally observed domain contrastive network, namely, TODO-Net, to explicitly mine the discrimination information from the hard actions samples from the low-observed temporal domains by mitigating the domain gaps among various temporally observed domains for 3-D early action prediction. More specifically, the proposed TODO-Net is able to mine the relationship between the low-observed sequences and all the highly observed sequences belonging to the same action category to boost the recognition performance of the hard samples with fewer observed frames. We also introduce a temporal domain conditioned supervised contrastive (TD-conditioned SupCon) learning scheme to empower our TODO-Net with the ability to minimize the gaps between the temporal domains within the same action categories, meanwhile pushing apart the temporal domains belonging to different action classes. We conduct extensive experiments on two public 3-D skeleton-based activity datasets, and the results show the efficacy of the proposed TODO-Net.

Methods to Enable Spatial Transcriptomics of Bone Tissues.

Understanding the relationship between the cells and their location within each tissue is critical to uncover the biological processes associated with normal development and disease pathology. Spatial transcriptomics is a powerful method that enables the analysis of the whole transcriptome within tissue samples, thus providing information about the cellular gene expression and the histological context in which the cells reside. While this method has been extensively utilized for many soft tissues, its application for the analyses of hard tissues such as bone has been challenging. The major challenge resides in the inability to preserve good quality RNA and tissue morphology while processing the hard tissue samples for sectioning. Therefore, a method is described here to process freshly obtained bone tissue samples to effectively generate spatial transcriptomics data. The method allows for the decalcification of the samples, granting successful tissue sections with preserved morphological details while avoiding RNA degradation. In addition, detailed guidelines are provided for samples that were previously paraffin-embedded, without demineralization, such assamples collected from tissue banks. Using these guidelines, high-quality spatial transcriptomics data generated from tissue bank samples of primary tumor and lung metastasis of bone osteosarcoma are shown.

Learning With Noisy Labels Over Imbalanced Subpopulations.

Learning with noisy labels (LNL) has attracted significant attention from the research community. Many recent LNL methods rely on the assumption that clean samples tend to have a "small loss." However, this assumption often fails to generalize to some real-world cases with imbalanced subpopulations, that is, training subpopulations that vary in sample size or recognition difficulty. Therefore, recent LNL methods face the risk of misclassifying those "informative" samples (e.g., hard samples or samples in the tail subpopulations) into noisy samples, leading to poor generalization performance. To address this issue, we propose a novel LNL method to deal with noisy labels and imbalanced subpopulations simultaneously. It first leverages sample correlation to estimate samples' clean probabilities for label correction and then utilizes corrected labels for distributionally robust optimization (DRO) to further improve the robustness. Specifically, in contrast to previous works using classification loss as the selection criterion, we introduce a feature-based metric that takes the sample correlation into account for estimating samples' clean probabilities. Then, we refurbish the noisy labels using the estimated clean probabilities and the pseudo-labels from the model's predictions. With refurbished labels, we use DRO to train the model to be robust to subpopulation imbalance. Extensive experiments on a wide range of benchmarks demonstrate that our technique can consistently improve state-of-the-art (SOTA) robust learning paradigms against noisy labels, especially when encountering imbalanced subpopulations. We provide our code in https://github.com/chenmc1996/LNL-IS.

Penalizing the Hard Example But Not Too Much: A Strong Baseline for Fine-Grained Visual Classification.

Though significant progress has been achieved on fine-grained visual classification (FGVC), severe overfitting still hinders model generalization. A recent study shows that hard samples in the training set can be easily fit, but most existing FGVC methods fail to classify some hard examples in the test set. The reason is that the model overfits those hard examples in the training set, but does not learn to generalize to unseen examples in the test set. In this article, we propose a moderate hard example modulation (MHEM) strategy to properly modulate the hard examples. MHEM encourages the model to not overfit hard examples and offers better generalization and discrimination. First, we introduce three conditions and formulate a general form of a modulated loss function. Second, we instantiate the loss function and provide a strong baseline for FGVC, where the performance of a naive backbone can be boosted and be comparable with recent methods. Moreover, we demonstrate that our baseline can be readily incorporated into the existing methods and empower these methods to be more discriminative. Equipped with our strong baseline, we achieve consistent improvements on three typical FGVC datasets, i.e., CUB-200-2011, Stanford Cars, and FGVC-Aircraft. We hope the idea of moderate hard example modulation will inspire future research work toward more effective fine-grained visual recognition.

BAFNet: Bottleneck Attention Based Fusion Network for Sleep Apnea Detection.

Sleep apnea (SA) is a common sleep-related breathing disorder that tends to induce a series of complications, such as pediatric intracranial hypertension, psoriasis, and even sudden death. Therefore, early diagnosis and treatment can effectively prevent malignant complications SA incurs. Portable monitoring (PM) is a widely used tool for people to monitor their sleep conditions outside of hospitals. In this study, we focus on SA detection based on single-lead electrocardiogram (ECG) signals which are easily collected by PM. We propose a bottleneck attention based fusion network named BAFNet, which mainly includes five parts of RRI (R-R intervals) stream network, RPA (R-peak amplitudes) stream network, global query generation, feature fusion, and classifier. To learn the feature representation of RRI/RPA segments, fully convolutional networks (FCN) with cross-learning are proposed. Meanwhile, to control the information flow between RRI and RPA networks, a global query generation with bottleneck attention is proposed. To further improve the SA detection performance, a hard sample scheme with k-means clustering is employed. Experiment results show that BAFNet can achieve competitive results, which are superior to the state-of-the-art SA detection methods. It means that BAFNet has great potential to be applied in the home sleep apnea test (HSAT) for sleep condition monitoring.

Drone-captured vehicle re-identification via perspective mask segmentation and hard sample learning

Drone-captured vehicle re-identification via perspective mask segmentation and hard sample learning

Rumor detection based on Attention Graph Adversarial Dual Contrast Learning.

It is becoming harder to tell rumors from non-rumors as social media becomes a key news source, which invites malicious manipulation that could do harm to the public's health or cause financial loss. When faced with situations when the session structure of comment sections is deliberately disrupted, traditional models do not handle them adequately. In order to do this, we provide a novel rumor detection architecture that combines dual comparison learning, adversarial training, and attention filters. We suggest the attention filter module to achieve the filtering of some dangerous comments as well as the filtering of some useless comments, allowing the nodes to enter the GAT graph neural network with greater structural information. The adversarial training module (ADV) simulates the occurrence of malicious comments through perturbation, giving the comments some defense against malicious comments. It also serves as a hard negative sample to aid double contrast learning (DCL), which aims to learn the differences between various comments, and incorporates the final loss in the form of a loss function to strengthen the model. According to experimental findings, our AGAD (Attention Graph Adversarial Dual Contrast Learning) model outperforms other cutting-edge algorithms on a number of rumor detection tasks. The code is available at https://github.com/icezhangGG/AGAD.git.