Meaningful Samples Research Articles

Semi-supervised comparative learning compensation method for chemical gas sensor drift.

The gradual and unpredictable variation in chemo-sensory signal responses when exposed to the same analyte under identical conditions, commonly referred to as sensor drift, has long been recognized as one of the most serious challenges faced by chemical sensors. The traditional drift compensation method is both labor-intensive and expensive, as it requires frequent collection and labeling of gas samples for recalibration. Introducing a small number of meaningful drift calibration samples can be an attractive strategy to reduce the computational load and improve the performance of the updated classifier. However, under the influence of drift, new challenges arise due to the difference in the distribution of source and target domain data. This paper proposes a novel algorithm framework called semi-supervised contrastive learning drift compensation (SSCLDC). The framework automatically extracts high-level abstract features based on a multilayer perceptron to better represent the structure of the source data. In addition, to address the issue of data distribution differences caused by drift between the source and target domains. We add a small number of reference sample pairs into the training for semi-supervised learning. Combining a contrastive loss function that can represent the matching degree of paired samples effectively overcomes the problem of sensor drift. The Kennard-Stone sequential algorithm is used to select the representative reference sample from the set of candidate reference samples. Experiments conducted on a widely used long-term chemical gas sensor drift dataset demonstrate that the proposed method outperforms several classic drift compensation techniques, highlighting its effectiveness and practical applicability.

The Chandra Source Catalog Release 2 Series

The Chandra Source Catalog (CSC) is a virtual X-ray astrophysics facility that enables both detailed individual source studies and statistical studies of large samples of X-ray sources detected in Advanced CCD Imaging Spectrometer and High Resolution Camera-I imaging observations obtained by the Chandra X-ray Observatory. The catalog provides carefully curated, high-quality, and uniformly calibrated and analyzed tabulated positional, spatial, photometric, spectral, and temporal source properties, as well as science-ready X-ray data products. The latter includes multiple types of source- and field-based FITS format products that can be used as a basis for further research, significantly simplifying follow-up analysis of scientifically meaningful source samples. We discuss in detail the algorithms used for the CSC Release 2 Series, including CSC 2.0, which includes 317,167 unique X-ray sources on the sky identified in observations released publicly through the end of 2014, and CSC 2.1, which adds Chandra data released through the end of 2021 and expands the catalog to 407,806 sources. Besides adding more recent observations, the CSC Release 2 Series includes multiple algorithmic enhancements that provide significant improvements over earlier releases. The compact source sensitivity limit for most observations is ∼5 photons over most of the field of view, which is ∼2× fainter than Release 1, achieved by coadding observations and using an optimized source detection approach. A Bayesian X-ray aperture photometry code produces robust fluxes even in crowded fields and for low-count sources. The current release, CSC 2.1, is tied to the Gaia-CRF3 astrometric reference frame for the best sky positions for catalog sources.

Leak detection and localization of fluid-filled pipeline using accelerometer pairs and mode separation method

Acoustic methods are commonly used to detect and locate leaks in pipelines, but they often overlook important multimodal characteristics of acoustic waves, limiting their applicability. Based on propagation theories regarding F(1,1) mode signal extraction, a new and easy-to-use method for leak detection and localization in pressurized pipelines is proposed. The F(1,1) mode signals are obtained using accelerometer pairs and mode separation method, which are then used to train detection models and to estimate the time difference of arrival (TDOA) for leak localization. Mode separation empowers the data-driven models with physically meaningful samples, aiming to broaden their applicability. The mode consistency between TDOA and velocity is achieved to ensure accurate leak localization. Experiments in buried water-filled ductile iron pipe (experiment A) and in fuel-filled iron pipe (experiment B) are conducted and the longest test distance reaches 72 m. The results validate that the employment of mode separation notably improves current leak detection and localization methods which primarily rely on original signals. The recognition rates rise about 3 %∼11 %, and the false alarm rates reduce about 1 %∼2 %, even when confronted with pipe junction interferences. Also, the leak localization errors are overall lower, with absolute errors below 2 m and relative errors less than 5 % of the test segment length for most cases.

A Hybrid Approach for Damage Detection and Localization on a Thin Metallic Plate

This paper illustrates a hybrid method for identifying structural damage that includes concepts from Physics-Based approaches within a Data-Driven framework. The lessons learned from the use of damage identification methods based on the analysis of modal curvature allows us to develop effective feature engineering in the development of Machine Learning (ML) algorithms. Moreover, the training of the algorithms exploits data provided from a population of damage cases generated by Finite Element (FE) analysis. The above data-analysis pipeline is applied to the identification of damage in a rectangular metallic plate for which measured modal data are available from a dedicated experimental campaign in CNR-INM laboratory using accelerometers. The damage is modelled as a stiffness reduction over a small area being also representative of material thinning due to corrosion. To have meaningful samples of the training database, the modal convergence of the FE model to the real structure is guaranteed by a structural optimization process. Experimentally identified noise, representative of real-life applications, is then added to the FE results before algorithm training. Damage existence and position are determined by a Novelty Detection approach and a Regression Neural Network, respectively. First, damage is identified on new test cases generated by the same FE procedure used for training. Secondly, the trained algorithms are applied to the experimental dataset. Sensitivity analysis on several parameters (number of samples for training, damage severity and noise levels) is numerically carried out to understand the applicability limits of the present methodology.

Curriculum-guided dynamic division strategy for graph contrastive learning

Contrastive learning is a commonly used framework in the field of graph self-supervised learning, where models are trained by bringing positive samples closer together and pushing negative samples apart. Most existing graph contrastive learning models divide all nodes into positive and negative samples, which leads to the selection of some meaningless samples and reduces the model’s performance. Additionally, there is a significant disparity in the ratio between positive and negative samples, with an excessive number of negative samples introducing noise. Therefore, we propose a novel dynamic sampling strategy that selects more meaningful samples from the perspectives of structure and features and we incorporate an iteration-based sample selection process into the model training to enhance its performance. Furthermore, we introduce a curriculum learning training method based on the principle of starting from easy to difficult. Sample training for each iteration is treated as a task, enabling the rapid capture of relevant and meaningful sample information. Extensive experiments have been conducted to validate the superior performance of our model across nine real-world datasets.

Deformable Dynamic Sampling and Dynamic Predictable Mask Mining for Image Inpainting.

Existing image inpainting methods often produce artifacts that are caused by using vanilla convolution layers as building blocks that treat all image regions equally and generate holes at random locations with equal probability. This design does not differentiate the missing regions and valid regions in inference and does not consider the predictability of missing regions in training. To address these issues, we propose a deformable dynamic sampling (DDS) mechanism which is built on deformable convolutions (DCs), and a constraint is proposed to avoid the deformably sampled elements falling into the corrupted regions. Furthermore, to select both valid sample locations and suitable kernels dynamically, we equip DCs with content-aware dynamic kernel selection (DKS). In addition, to further encourage the DDS mechanism to find meaningful sampling locations, we propose to train the inpainting model with mined predictable regions as holes. During training, we jointly train a mask generator with the inpainting network to generate hole masks dynamically for each training sample. Thus, the mask generator can find large yet predictable missing regions as a better alternative to random masks. Extensive experiments demonstrate the advantages of our method over state-of-the-art methods qualitatively and quantitatively.

Fully spherical 3D datasets on sedimentary particles: Fast measurement and evaluation

AbstractRecently it became increasingly evident that the statistical distributions of size and shape descriptors of sedimentary particles reveal crucial information on their evolution and may even carry the fingerprints of their provenance as fragments. However, to unlock this trove of information, measurement of traditional geophysical shape descriptors (mostly detectable on 2D projections) is not sufficient; fully spherical 3D imaging and mathematical algorithms suitable to extract new types of inherently 3D shape descriptors are necessary. Available 3D imaging technologies force users to choose either speed or full sphericity. Only partial morphological information can be extracted in the absence of the latter (e.g., LIDAR imaging). In the case of fully spherical imaging, speed was proved to be prohibitive for obtaining meaningful statistical samples, and inherently 3D shape descriptors were not extracted. Here we present a new method by complementing a commercial, portable 3D scanner with simple hardware to quickly obtain fully spherical 3D datasets from large collections of sedimentary particles. We also present software for the automated extraction of 3D shapes and automated measurement of inherently 3D-shape properties. This technique allows for examining large samples without the need for transportation or storage of the samples, and it may also facilitate the collaboration of geographically distant research groups. We validated our software on a large sample of pebbles by comparing previously hand-measured parameters with the results of automated shape analysis. We also tested our hardware and software tools on a large pebble sample in Kawakawa Bay, New Zealand.

Integrating spatial and ecological information into comprehensive biodiversity monitoring on agricultural land

Biodiversity loss on agricultural land is a major concern. Comprehensive monitoring is needed to quantify the ongoing changes and assess the effectiveness of agri-environmental measures. However, current approaches to monitoring biodiversity on agricultural land are limited in their ability to capture the complex pattern of species and habitats. Using a real-world example of plant and habitat monitoring on Swiss agricultural land, we show how meaningful and efficient sampling can be achieved at the relevant scales. The multi-stage sampling design of this approach uses unequal probability sampling in combination with intermediate small-scale habitat sampling to ensure broad representation of regions, landscape types, and plant species. To achieve broad coverage of temporary agri-environmental measures, the baseline survey on permanent plots is complemented by dynamic sampling of these specific areas. Sampling efficiency and practicality are ensured at all stages of sampling through modern sampling techniques, such as unequal probability sampling with fixed sample size, self-weighting, spatial spreading, balancing on additional information, and stratified balancing. In this way, the samples are well distributed across ecological and geographic space. Despite the high complexity of the sampling design, simple estimators are provided. The effects of stratified balancing and clustering of samples are demonstrated in Monte Carlo simulations using modelled habitat data. A power analysis based on actual survey data is also presented. Overall, the study could serve as a useful example for improving future biodiversity monitoring networks on agricultural land at multiple scales.

Data Augmentation in High Dimensional Low Sample Size Setting Using a Geometry-Based Variational Autoencoder.

In this paper, we propose a new method to perform data augmentation in a reliable way in the High Dimensional Low Sample Size (HDLSS) setting using a geometry-based variational autoencoder (VAE). Our approach combines the proposal of 1) a new VAE model, the latent space of which is modeled as a Riemannian manifold and which combines both Riemannian metric learning and normalizing flows and 2) a new generation scheme which produces more meaningful samples especially in the context of small data sets. The method is tested through a wide experimental study where its robustness to data sets, classifiers and training samples size is stressed. It is also validated on a medical imaging classification task on the challenging ADNI database where a small number of 3D brain magnetic resonance images (MRIs) are considered and augmented using the proposed VAE framework. In each case, the proposed method allows for a significant and reliable gain in the classification metrics. For instance, balanced accuracy jumps from 66.3% to 74.3% for a state-of-the-art convolutional neural network classifier trained with 50 MRIs of cognitively normal (CN) and 50 Alzheimer disease (AD) patients and from 77.7% to 86.3% when trained with 243 CN and 210 AD while improving greatly sensitivity and specificity metrics.

A method for measuring meaningful physiological variables in fish blood without surgical cannulation

Gaining meaningful blood samples from water-breathing fish is a significant challenge. Two main methods typically used are grab ‘n’ stab and surgical cannulation. Both methods have benefits, but also significant limitations under various scenarios. Here we present a method of blood sampling laboratory fish involving gradual induction of anaesthesia within their home tank, avoiding physical struggling associated with capture, followed by rapid transfer to a gill irrigation system to maintain artificial ventilation via adequate gill water flow and then followed by sampling the caudal vasculature. This method negates many blood chemistry disturbances associated with grab ‘n’ stab (i.e., low pH and oxygen, elevated lactate, CO2 and stress hormones) and generates results that are directly comparable to cannulated fish under a wide range of experimentally-induced acid–base scenarios (acidosis and alkalosis). Crucially this method was successful in achieving accurate acid–base blood measurements from fish ten times smaller than are typically suitable for cannulation. This opens opportunities not previously possible for studies that relate to basic physiology, sustainable aquaculture, ecotoxicology, conservation, and climate change.

Cross-Modal Data Augmentation for Tasks of Different Modalities

Data augmentation has become one of the keys to alleviating the over-fitting of models on training data and improving the generalization capabilities on testing data. Most existing data augmentation methods only focus on one modality, which is incapable when facing multiple data modalities. Some prior works try to interpolate with random coefficients in the latent space to generate new samples, which can generically work for any data modality. However, these works ignore the extra information conveyed by multimodality data. In fact, the extra information in one modality can provide semantic directions to generate more meaningful samples in another modality. This paper proposes Cross-modal Data Augmentation (CMDA), a simple yet effective data augmentation method to alleviate the over-fitting issue and improve the generalization performance. We evaluate CMDA on unsupervised and supervised tasks of different modalities, on which CMDA consistently and significantly outperforms baselines. For instance, CMDA improves the unsupervised anomaly detection baseline in vision modality from the AUROC <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$76.46\%, 73.07\%$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$64.36\%$</tex-math></inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$83.25\%, 76.22\%$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$70.57\%$</tex-math></inline-formula> on three different datasets, respectively. Besides, extensive experiments demonstrate that CMDA is applicable to various neural network architectures. Furthermore, prior methods that interpolate in the latent space need to work with downstream tasks to construct the latent space. In contrast, CMDA can work with or without downstream tasks, which makes the applicability of CMDA more extensive. Our source code is publicly available for non-commercial or research use at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Anfeather/CMDA</uri> .

Triple Generative Adversarial Networks.

We propose a unified game-theoretical framework to perform classification and conditional image generation given limited supervision. It is formulated as a three-player minimax game consisting of a generator, a classifier and a discriminator, and therefore is referred to as Triple Generative Adversarial Network (Triple-GAN). The generator and the classifier characterize the conditional distributions between images and labels to perform conditional generation and classification, respectively. The discriminator solely focuses on identifying fake image-label pairs. Theoretically, the three-player formulation guarantees consistency. Namely, under a nonparametric assumption, the unique equilibrium of the game is that the distributions characterized by the generator and the classifier converge to the data distribution. As a byproduct of the three-player formulation, Triple-GAN is flexible to incorporate different semi-supervised classifiers and GAN architectures. We evaluate Triple-GAN in two challenging settings, namely, semi-supervised learning and the extremely low data regime. In both settings, Triple-GAN can achieve excellent classification results and generate meaningful samples in a specific class simultaneously. In particular, using a commonly adopted 13-layer CNN classifier, Triple-GAN outperforms extensive semi-supervised learning methods substantially on several benchmarks no matter data augmentation is applied or not.

A deep learning generative model approach for image synthesis of plant leaves.

A well-known drawback to the implementation of Convolutional Neural Networks (CNNs) for image-recognition is the intensive annotation effort for large enough training dataset, that can become prohibitive in several applications. In this study we focus on applications in the agricultural domain and we implement Deep Learning (DL) techniques for the automatic generation of meaningful synthetic images of plant leaves, which can be used as a virtually unlimited dataset to train or validate specialized CNN models or other image-recognition algorithms. Following an approach based on DL generative models, we introduce a Leaf-to-Leaf Translation (L2L) algorithm, able to produce collections of novel synthetic images in two steps: first, a residual variational autoencoder architecture is used to generate novel synthetic leaf skeletons geometry, starting from binarized skeletons obtained from real leaf images. Second, a translation via Pix2pix framework based on conditional generator adversarial networks (cGANs) reproduces the color distribution of the leaf surface, by preserving the underneath venation pattern and leaf shape. The L2L algorithm generates synthetic images of leaves with meaningful and realistic appearance, indicating that it can significantly contribute to expand a small dataset of real images. The performance was assessed qualitatively and quantitatively, by employing a DL anomaly detection strategy which quantifies the anomaly degree of synthetic leaves with respect to real samples. Finally, as an illustrative example, the proposed L2L algorithm was used for generating a set of synthetic images of healthy end diseased cucumber leaves aimed at training a CNN model for automatic detection of disease symptoms. Generative DL approaches have the potential to be a new paradigm to provide low-cost meaningful synthetic samples. Our focus was to dispose of synthetic leaves images for smart agriculture applications but, more in general, they can serve for all computer-aided applications which require the representation of vegetation. The present L2L approach represents a step towards this goal, being able to generate synthetic samples with a relevant qualitative and quantitative resemblance to real leaves.

Performance analysis of control allocation using data‐driven integral quadratic constraints

AbstractA new method is presented for evaluating the performance of a nonlinear control allocation system within a linear control loop. To that end, a worst‐case gain analysis problem is formulated that can be readily solved by means of well‐established methods from robustness analysis using integral quadratic constraints (IQCs). It exploits the fact that control allocation systems are in general memoryless mappings that can be bounded by IQCs. A data‐driven approach is used to find an optimal bound of the input/output mapping of the control allocation. Additionally, an iterative procedure based on local IQCs is introduced to determine meaningful sampling limits for less conservative yet accurate results. The effectiveness of the proposed data‐driven performance analysis is shown at the example of an actively controlled flexible wing in a wind tunnel.

Dual-discriminator adversarial framework for data-free quantization

Thanks to the potential to address the privacy and security issues, data-free quantization that generates samples based on the prior information in the model has recently been widely investigated. However, existing methods failed to adequately utilize the prior information and thus cannot fully restore the real-data characteristics and provide effective supervision to the quantized model, resulting in poor performance. In this paper, we propose Dual-Discriminator Adversarial Quantization (DDAQ), a novel data-free quantization framework with an adversarial learning style that enables effective sample generation and learning of the quantized model. Specifically, we employ a generator to produce meaningful and diverse samples directed by two discriminators, aiming to facilitate the matching of the batch normalization (BN) distribution and maximizing the discrepancy between the full-precision model and the quantized model, respectively. Moreover, inspired by mixed-precision quantization, i.e., the importance of each layer is different, we introduce layer importance prior to both discriminators, allowing us to make better use of the information in the model. Subsequently, the quantized model is trained with the generated samples under the supervision of the full-precision model. We evaluate DDAQ on various network structures for different vision tasks, including image classification and object detection, and the experimental results show that DDAQ outperforms all baseline methods with good generality.

Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated Label Mixing

The Mixup scheme suggests mixing a pair of samples to create an augmented training sample and has gained considerable attention recently for improving the generalizability of neural networks. A straightforward and widely used extension of Mixup is to combine with regional dropout-like methods: removing random patches from a sample and replacing it with the features from another sample. Albeit their simplicity and effectiveness, these methods are prone to create harmful samples due to their randomness. To address this issue, 'maximum saliency' strategies were recently proposed: they select only the most informative features to prevent such a phenomenon. However, they now suffer from lack of sample diversification as they always deterministically select regions with maximum saliency, injecting bias into the augmented data. In this paper, we present, a novel, yet simple Mixup-variant that captures the best of both worlds. Our idea is two-fold. By stochastically sampling the features and ‘grafting’ them onto another sample, our method effectively generates diverse yet meaningful samples. Its second ingredient is to produce the label of the grafted sample by mixing the labels in a saliency-calibrated fashion, which rectifies supervision misguidance introduced by the random sampling procedure. Our experiments under CIFAR, Tiny-ImageNet, and ImageNet datasets show that our scheme outperforms the current state-of-the-art augmentation strategies not only in terms of classification accuracy, but is also superior in coping under stress conditions such as data corruption and object occlusion.

InsCLR: Improving Instance Retrieval with Self-Supervision

This work aims at improving instance retrieval with self-supervision. We find that fine-tuning using the recently developed self-supervised learning (SSL) methods, such as SimCLR and MoCo, fails to improve the performance of instance retrieval. In this work, we identify that the learnt representations for instance retrieval should be invariant to large variations in viewpoint and background etc., whereas self-augmented positives applied by the current SSL methods can not provide strong enough signals for learning robust instance-level representations. To overcome this problem, we propose InsCLR, a new SSL method that builds on the instance-level contrast, to learn the intra-class invariance by dynamically mining meaningful pseudo positive samples from both mini-batches and a memory bank during training. Extensive experiments demonstrate that InsCLR achieves similar or even better performance than the state-of-the-art SSL methods on instance retrieval. Code is available at https://github.com/zeludeng/insclr.

AutoGCL: Automated Graph Contrastive Learning via Learnable View Generators

Contrastive learning has been widely applied to graph representation learning, where the view generators play a vital role in generating effective contrastive samples. Most of the existing contrastive learning methods employ pre-defined view generation methods, e.g., node drop or edge perturbation, which usually cannot adapt to input data or preserve the original semantic structures well. To address this issue, we propose a novel framework named Automated Graph Contrastive Learning (AutoGCL) in this paper. Specifically, AutoGCL employs a set of learnable graph view generators orchestrated by an auto augmentation strategy, where every graph view generator learns a probability distribution of graphs conditioned by the input. While the graph view generators in AutoGCL preserve the most representative structures of the original graph in generation of every contrastive sample, the auto augmentation learns policies to introduce adequate augmentation variances in the whole contrastive learning procedure. Furthermore, AutoGCL adopts a joint training strategy to train the learnable view generators, the graph encoder, and the classifier in an end-to-end manner, resulting in topological heterogeneity yet semantic similarity in the generation of contrastive samples. Extensive experiments on semi-supervised learning, unsupervised learning, and transfer learning demonstrate the superiority of our AutoGCL framework over the state-of-the-arts in graph contrastive learning. In addition, the visualization results further confirm that the learnable view generators can deliver more compact and semantically meaningful contrastive samples compared against the existing view generation methods. Our code is available at https://github.com/Somedaywilldo/AutoGCL.

Negative Insurance Claim Generation Using Distance Pooling on Positive Diagnosis-Procedure Bipartite Graphs

Negative samples in health and medical insurance domain refer to fraudulent or erroneous insurance claims that may include inconsistent diagnosis-procedure relations with respect to a medical coding system. Unfortunately, only a few datasets are publicly available for research in health insurance domain, yet none reports any negative claims. However, negative claims are essential not only to develop new machine learning approaches but also to test and validate automated artificial intelligence systems deployed by insurance providers. In this study, we introduce a synthetic negative claim generation procedure based on the bipartite graph representations of positive claims. Our empirical results demonstrate promising outcomes that will improve the development and evaluation processes of machine learning approaches in healthcare, where negative samples are required, but not available. Moreover, the proposed scheme can be applied to other domains, where bipartite graph representations are meaningful and negative samples are lacking.

Effective active learning in digital pathology: A case study in tumor infiltrating lymphocytes

Background and Objective:Deep learning methods have demonstrated remarkable performance in pathology image analysis, but they require a large amount of annotated training data from expert pathologists. The aim of this study is to minimize the data annotation need in these analyses.Methods:Active learning (AL) is an iterative approach to training deep learning models. It was used in our context with a Tumor Infiltrating Lymphocytes (TIL) classification task to minimize annotation. State-of-the-art AL methods were evaluated with the TIL application and we have proposed and evaluated a more efficient and effective AL acquisition method. The proposed method uses data grouping based on imaging features and model prediction uncertainty to select meaningful training samples (image patches).Results:An experimental evaluation with a collection of cancer tissue images shows that: (i) Our approach reduces the number of patches required to attain a given AUC as compared to other approaches, and (ii) our optimization (subpooling) leads to AL execution time improvement of about 2.12×.Conclusions: This strategy enabled TIL based deep learning analyses using smaller annotation demand. We expect this approach may be used to build other analyses in digital pathology with fewer training samples.