Selection Of Training Samples Research Articles

Semantically redundant training data removal and deep model classification performance: A study with chest X-rays

Deep learning (DL) has demonstrated its innate capacity to independently learn hierarchical features from complex and multi-dimensional data. A common understanding is that its performance scales up with the amount of training data. However, the data must also exhibit variety to enable improved learning. In medical imaging data, semantic redundancy, which is the presence of similar or repetitive information, can occur due to the presence of multiple images that have highly similar presentations for the disease of interest. Also, the common use of augmentation methods to generate variety in DL training could limit performance when indiscriminately applied to such data. We hypothesize that semantic redundancy would therefore tend to lower performance and limit generalizability to unseen data and question its impact on classifier performance even with large data. We propose an entropy-based sample scoring approach to identify and remove semantically redundant training data and demonstrate using the publicly available NIH chest X-ray dataset that the model trained on the resulting informative subset of training data significantly outperforms the model trained on the full training set, during both internal (recall: 0.7164 vs 0.6597, p<0.05) and external testing (recall: 0.3185 vs 0.2589, p<0.05). Our findings emphasize the importance of information-oriented training sample selection as opposed to the conventional practice of using all available training data.

Estimation of canopy height based on multi-source remote sensing data using forest structure aided sample selection

ABSTRACT Forest canopy height data are crucial for estimating forest carbon storage and assessing forest ecology. By utilizing satellite imagery, canopy height data obtained from airborne or spaceborne LiDAR have been expanded from footprint and plot levels to spatially continuous elevation mapping of forests. However, current research suggests that estimating forest canopy height without forest type data presents a challenge in how to effectively integrate multi-source LiDAR data and ensure the samples adequately represent various forest types for higher estimation accuracy. Therefore, this study proposes a forest canopy height estimation method that considers forest structure and integrates multi-source LiDAR data to overcome the challenge. First, a stratified sampling method based on forest structure (SSMFS) was proposed to select training samples and enhance their representativeness. Second, we combined GEDI and ATL08 data to create a multi-source spaceborne LiDAR dataset, enhancing geographic coverage and increasing canopy height samples. Third, the spaceborne LiDAR-based canopy height estimation model incorporates previously unconsidered canopy openness features and uses SSMFS to select training samples. Finally, we improved spaceborne LiDAR canopy height accuracy by creating a residual correction model that adjusts for differences between airborne scanner (ALS) and spaceborne LiDAR estimates. This study, conducted in Zhangwu County, achieved an accuracy of R2 = 0.71, MAE = 1.20 m, and RMSE = 1.71 m. These results show a 51.06% increase in R2, a 26.38% decrease in MAE, and a 24.00% decrease in RMSE compared to recent research. In summary, this study profoundly amplifies predictive accuracy, providing a clear advantage in the delineation of regional forest canopy maps.

Improving Semantic Segmentation via Efficient Self-Training.

Starting from the seminal work of Fully Convolutional Networks (FCN), there has been significant progress on semantic segmentation. However, deep learning models often require large amounts of pixelwise annotations to train accurate and robust models. Given the prohibitively expensive annotation cost of segmentation masks, we introduce a self-training framework in this paper to leverage pseudo labels generated from unlabeled data. In order to handle the data imbalance problem of semantic segmentation, we propose a centroid sampling strategy to uniformly select training samples from every class within each epoch. We also introduce a fast training schedule to alleviate the computational burden. This enables us to explore the usage of large amounts of pseudo labels. Our Centroid Sampling based Self-Training framework (CSST) achieves state-of-the-art results on Cityscapes and CamVid datasets. On PASCAL VOC 2012 test set, our models trained with the original train set even outperform the same models trained on the much bigger augmented train set. This indicates the effectiveness of CSST when there are fewer annotations. We also demonstrate promising few-shot generalization capability from Cityscapes to BDD100K and from Cityscapes to Mapillary datasets.

A novel framework based on two-stage multi-view feature optimization and improved support vector data description for aeroengine bearing early fault detection

Early fault detection is crucial to avoid catastrophic flight accidents caused by unplanned downtime of equipment. Aimed at the shortcomings of feature selection and early fault detection, this paper proposes a method combining two-stage multi-view feature optimization and improved support vector data description for aeroengine bearing early fault detection. First, a comprehensive evaluation index is constructed via combining monotonicity, correlation, robustness index and entropy weight method to be used for initial screening sensitive degradation features from multi-view candidate degradation feature. After that, the hierarchical clustering algorithm and local preserving projection algorithm is introduced to realize sensitive degradation features clustering and dimensionality reduction. On this basis, the Euclidean distance sample reduction and the sparrow search algorithm is utilized to complete the training sample selection and model parameter optimization of different clusters and further use the verification samples to filter the optimal features and model. Finally, based on the optimal degradation features and models, a fusion performance degradation index is built to achieve intelligent detection of early fault. Comparisons with some existing fault detection methods, the proposed method improves degradation trend sensibility and demonstrates potential of good early fault detection engineering applications.

Robust Adaptive Radar Beamforming Based on Iterative Training Sample Selection Using Kurtosis of Generalized Inner Product Statistics

Robust Adaptive Radar Beamforming Based on Iterative Training Sample Selection Using Kurtosis of Generalized Inner Product Statistics

Adaptive Data-Driven Deep-Learning Surrogate Model for Frontal Polymerization in Dicyclopentadiene.

Frontal polymerization (FP) is a self-sustaining curing process that enables rapid and energy-efficient manufacturing of thermoset polymers and composites. Computational methods conventionally used to simulate the FP process are time-consuming, and repeating simulations are required for sensitivity analysis, uncertainty quantification, or optimization of the manufacturing process. In this work, we develop an adaptive surrogate deep-learning model for FP of dicyclopentadiene (DCPD), which predicts the evolution of temperature and degree of cure orders of magnitude faster than the finite-element method (FEM). The adaptive algorithm provides a strategy to select training samples efficiently and save computational costs by reducing the redundancy of FEM-based training samples. The adaptive algorithm calculates the residual error of the FP governing equations using automatic differentiation of the deep neural network. A probability density function expressed in terms of the residual error is used to select training samples from the Sobol sequence space. The temperature and degree of cure evolution of each training sample are obtained by a 2D FEM simulation. The adaptive method is more efficient and has a better prediction accuracy than the random sampling method. With the well-trained surrogate neural network, the FP characteristics (front speed, shape, and temperature) can be extracted quickly from the predicted temperature and degree-of-cure fields.

Time-dependent reliability analysis of structural systems based on parallel active learning Kriging model

Time-dependent reliability analysis quantifies the failures of structural systems due to time-dependent uncertainties, such as material degradation and dynamic loads. The active learning Kriging model methods are widely used in structural reliability analysis to replace extensive time-consuming finite element simulations. However, they can only update one training sample and one failure mode per iteration, which limits their application to time-dependent, parallel computing, and multiple failure modes problems. In this study, we propose a new parallel active learning Kriging model for time-dependent reliability analysis, which can update multiple training samples and multiple failure modes per iteration. It includes the following strategies: (1) a novel parallel learning function is proposed, which combines the correlation function and U learning function to allow for the selection of multiple training samples per iteration; (2) an adaptive adjustment strategy for the number of parallel samples is proposed, which takes into account the prediction probability of parallel samples; (3) the proposed parallel learning function is integrated into time-dependent reliability analysis with multiple failure modes, enabling simultaneous updates of multiple training samples and failure modes, thus greatly reducing the number of iterations and computational time; and (4) a new stopping criterion is proposed to improve the efficiency of the estimation of failure probability. The proposed method can be applied to series or parallel time-dependent structural systems with multiple failure modes. We demonstrate the effectiveness of the proposed method through three examples, and the proposed method can achieve a balance between the computational time and function calls while maintaining a high level of accuracy in the estimation of time-dependent failure probability.

A learnable population filter for dynamic multi-objective optimization

In this paper, a novel learnable population filter is proposed to solve the dynamic multi-objective problems (DMOPs), whose main idea is to pick out potential valuable individuals in the changing situation via the trained filter to facilitate the population convergence in a new environment. In particular, by selecting training samples from three different optional pools, rich information in the previous searching experiences can be sufficiently used to cope with the dynamic behaviors in DMOPs, thereby handling the imperfect history data availability caused by the environmental uncertainty to some extents. Then, the screened population in the new environment, which is regarded to have reliable quality that benefits accelerating the evolution, is used to initialize the static optimizer. In addition, partial random solutions are employed as supplement to the initial population to further enhance the diversity. It is shown from the benchmark evaluations that the proposed method outperforms some other popular algorithms on both convergence and diversity by seamlessly combining the mainstream strategies in dealing with the dynamic behaviors, which is a competitive method that conforms to the novel evolutionary transfer optimization (ETO) trend. Moreover, effectiveness of the core components is validated through extensive ablation studies, which brings a new sight in developing the learnable DMOP solvers.

A Review of Pattern Recognition and Machine Learning

This article aims to provide a concise overview of diverse methodologies employed at different stages of a pattern recognition system, highlighting contemporary research challenges and applications in this dynamic field. The integration of machine learning techniques has played a pivotal role in converging pattern recognition frameworks in academic literature. The process relies heavily on supervised or unsupervised categorization methods to achieve its objectives, with a notable focus on statistical approaches. More recently, there is a growing emphasis on incorporating neural network methodologies and insights from statistical learning theory. Designing an effective recognition system necessitates careful consideration of various factors, including pattern representation, pattern class definition, feature extraction, sensing environment, feature selection, classifier learning and design, cluster analysis, test and training sample selection, and performance assessment.

Fishing risky behavior recognition based on adaptive transformer, reinforcement learning and stochastic configuration networks

The fishing behavior in a high-voltage environment may lead to electric shock accidents. This paper proposes a fishing risky behavior recognition method based on adaptive Transformer, reinforcement learning and stochastic configuration networks (SCNs). Firstly, the fishing behavior image sample training set is evaluated using the actor network based on linear entropy in the reinforcement learning module to select suitable training samples that possess abundant straight-line elements. Subsequently, the selected training samples are fed into a multi-scale spatial deep convolution to extract continuous spatial features of elongated objects. The deformable Transformer network, enhanced with adaptive encoding and decoding layers through SCNs, is used to obtain the position and detection results of the fishing rod. The trained Transformer model is evaluated by a defined credibility evaluation metric for acquiring rewards to update the training sample set iteratively. Then, an adaptive adjustment mechanism is constructed for the encoding and decoding layers of the adaptive Transformer network to establish a library of adaptive Transformer models with different encoding and decoding layer levels to accommodate the feature requirements of training samples in various scenarios. Finally, the blending learning method for combining the detection results from the model library is integrated with human body object detection and pose estimation methods, and a predefined expert system is utilized for logic reasoning on fishing risky behavior. Experimental results demonstrate the effectiveness of the proposed method in this paper.

A multi-surrogate multi-tasking genetic algorithm with an adaptive training sample selection strategy for expensive optimization problems

Multi-surrogate assisted evolutionary algorithms, utilizing inexpensive global and local surrogate models to assist evolutionary search, have demonstrated a remarkable ability to solve expensive optimization problems (EOPs). The selected training samples of the local surrogate model used in the current algorithms may not be updated continuously, which could bring about a waste of computing resources. This paper proposes a multi-surrogate multi-tasking genetic algorithm with an adaptive training sample selection strategy (MS-MTGAwA), in which the local surrogate model is updated adaptively based on information from established local models to enhance the capability in exploiting the optimal solution. The adaptive training sample selection strategy (ATS) applies optimal points found in historical local models as training samples. In addition, MS-MTGAwA inherits the optimization framework of a multi-tasking genetic algorithm. The radial basis function model is chosen as the modeling basis of the global and local surrogate models. A set of common benchmark functions whose dimensions vary from 10 to 100 and the tension/compression spring design problem are adopted to validate the performance of the proposed algorithm. Based on the average best fitness value over thirty independent runs on the benchmark functions, MS-MTGAwA ranks first in Friedman’s test compared to five state-of-the-art algorithms including S-JADE, GORS-SSLPSO, SAHO, SAMSO, and MS-MTO. The proposed ATS can be expected to be applied in any algorithms with local surrogate to make full use of historical information of the local model to improve the prediction accuracy of the next local model.If this paper is accepted, MATLAB codes associated with this paper will be uploaded tohttps://github.com/Zhongbo-Hu/Prediction-Evolutionary-Algorithm-HOMEPAGE.

CLFR-Det: Cross-level feature refinement detector for tiny-ship detection in SAR images

Ship detection in synthetic aperture radar (SAR) images is an important and active topic, due to the characteristics of SAR images involving all-time and all-weather imaging. However, complex backgrounds caused by speckle noise and inshore land, coupled with information deficiency of tiny ships, pose a great challenge for tiny-ship detection in SAR images. To tackle this problem, we present a cross-level feature refinement detector (CLFR-Det) that utilizes features reflecting different levels and distinct semantics (classification and localization). To enrich the semantic information of tiny ships, our approach incorporates a cross-level modulated deformable convolution to aggregate features from relevant positions across multi-level feature maps. This is supported by a spatially-informed multi-scale feature refinement mechanism that combines the features for classification and those for localization. We implement a uniform IoU-weighted adaptive training sample selection method for equitably distributing the impact of positive samples from targets of various sizes during the training process. A generalized IoU loss between ground-truth and preliminary bounding box is further proposed to supervise the learning process of the CLFR-Det, with uncertainty weights incorporated to dynamically depict the levels of disparate losses, enabling adequate training across different tasks. Also, we construct a novel tiny SAR ship detection dataset to comprehensively evaluate the effectiveness of our system, in conjunction with the use of publicly available SSDD and HRSID datasets. Experimental investigations demonstrate that CLFR-Det generally surpasses state-of-the-art performance for multi-scale ship detection, particularly for the detection of tiny ships.

GAN-assisted Road Segmentation from Satellite Imagery

Geo-information extraction from satellite imagery has become crucial to carry out large-scale ground surveys in a short amount of time. With the increasing number of commercial satellites launched into orbit in recent years, high-resolution RGB color remote sensing imagery has attracted a lot of attention. However, because of the high cost of image acquisition and even more complicated annotation procedures, there are limited high-resolution satellite datasets available. Compared to close-range imagery datasets, existing satellite datasets have a much lower number of images and cover only a few scenarios (cities, background environments, etc. ). They may not be sufficient for training robust learning models that fit all environmental conditions or be representative enough for training regional models that optimize for local scenarios. Instead of collecting and annotating more data, using synthetic images could be another solution to boost the performance of a model. This study proposes a GAN-assisted training scheme for road segmentation from high-resolution RGB color satellite images, which includes three critical components: a) synthetic training sample generation, b) synthetic training sample selection, and c) assisted training strategy. Apart from the GeoPalette and cSinGAN image generators introduced in our prior work, this paper in detail explains how to generate new training pairs using OpenStreetMap (OSM) and introduces a new set of evaluation metrics for selecting synthetic training pairs from a pool of generated samples. We conduct extensive quantitative and qualitative experiments to compare different image generators and training strategies. Our experiments on the downstream road segmentation task show that 1) our proposed metrics are more aligned with the trained model performance compared to commonly used GAN evaluation metrics such as the Fréchet inception distance (FID); and 2) by using synthetic data with the best training strategy, the model performance, mean Intersection over Union (mean IoU), is improved from 60.92% to 64.44%, when 1,000 real training pairs are available for learning, which reaches a similar level of performance as a model that is standard-trained with 4,000 real images (64.59%), i.e. , enabling a 4-fold reduction in real dataset size.

A fully automatic framework for sub-pixel mapping of thermokarst lakes using Sentinel-2 images

Mapping and monitoring thermokarst lakes are crucial to understanding the impact of climate change on permafrost regions and quantifying permafrost-related carbon emissions. Several automatic methods based on remote sensing images have been developed for thermokarst lake mapping. However, mixed pixels containing both land and water characteristics in the lakeshore zones pose a significant challenge to the accuracy of these methods. Furthermore, few approaches were able to fully automate the identification of thermokarst lakes without the manual training sample selection or parameter tuning. In this study, we present a fully automatic framework for thermokarst lake mapping using moderate-resolution Sentinel-2 images. The proposed method combines multidimensional hierarchical clustering and sub-pixel mapping (SPM) based on the radial basis function (RBF) interpolation and Markov random field (MRF) (referred to as RBF-then-MRF SPM), so as to achieve thermokarst lake mapping at a spatial resolution of 3.3 m. We apply the proposed method to two representative thermokarst lake distribution regions in the Northern Hemisphere and achieve a mean Kappa coefficient of 0.89 and 0.99, and a mean Quality of 89.86% and 96.60% on the central Tibetan Plateau and the northern Seward Peninsula, respectively. The results demonstrate that the proposed method significantly improves the accuracy of mixed pixel extraction, and the automatic thermokarst lake mapping is applicable to diverse permafrost regions.

Efficient and precise cell counting for RNAi screening of Orientia tsutsugamushi infection using deep learning techniques

Acquiring fluorescent scrub typhus images obtained through RNA interference screening for the analysis of 60 different human genes and 18 control genes poses challenges due to nonuniform or clumped cells and variations in image quality, rendering image processing (IP) counting inadequate. This study addresses three key questions concerning the application of deep learning methods to this dataset. Firstly, it explores the potential for object detection (OD) models to replace instance segmentation (IS) models in cell counting, striking a balance between accuracy and computational efficiency. Object detection models, including Faster R-CNN, You Only Look Once (YOLO), and Adaptive Training Sample Selection (ATSS) with reduced backbone sizes, outperform the instance segmentation model (Mask Region-Based Convolutional Neural Network: Mask R-CNN, Cascade Mask-RCNN) with both deep and shallow backbones. Notably, ATSS with Resnet-50 achieves an impressive mean average precision of 0.884 in just 33.1 milliseconds. Secondly, reducing the feature extractor size enhances cell counting efficiency, with OD models featuring reduced backbones demonstrating improved performance and faster processing. Finally, deep learning, especially OD models with shallow backbones, outperforms IP methods in both absolute and relative cell counting. This study demonstrates the potential for OD models to replace IS models, the efficiency gains achieved through the reduction of feature extractors, and the superiority of DL over IP in cell counting tasks.

Class3Dp: A supervised classifier of vegetation species from point clouds

Recognizing the species composition of an ecosystem is essential for conservation and land management. This study presents the software Class3Dp, a supervised classifier of vegetation species for coloured point clouds. Class3Dp is run through a graphical user interface (GUI) that allows for the selection of training samples from RGB or MS (multispectral) clouds and their classification based on geometric, spectral and neighbourhood features, along with different machine learning methods, obtaining the point cloud classified according to the classes (species) introduced. A case study is shown where a classification of ground and vegetation is carried out, obtaining an overall accuracy (OA) of 0.94 in the RGB classification and 0.95 in the MS. Points classified as vegetation were re-classified in the species Anthyllis cytisoides L., Chamaerops humilis L., Cistus monspeliensis L., Pistacia lentiscus L. and Quercus coccifera L., obtaining an OA of 0.86 in the RGB classification and 0.87 in the MS.

Change detection in VHR Remote Sensing Images by automatic sample selection and progressive classification

ABSTRACT This paper proposes an automatic method for land cover change detection in very-high-spatial-resolution optical remote sensing images based on the automatic selection of training samples using expectation-maximization (EM) and an extreme learning machine classifier, which has two key characteristics: (1) combining the advantages of supervised and unsupervised methods with progressive mask classification. (2) training samples of three classes (changed, unchanged, and unknown) were automatically selected by K-Means and EM, and then further refined by the likelihood function. The method was validated by one dataset of SuperView-1 imagery with a spatial resolution of 2.0 m, two datasets of TripleSat-2 imagery with a spatial resolution of 3.2 m, and one open dataset of Zi-Yuan-3 imagery with a spatial resolution of 5.8 m, and the results were compared with that of three unsupervised methods (iterative slow feature analysis, multivariate alteration detection, and adaptive object-oriented spatial-contextual extraction algorithm), two deep learning methods (convolutional-wavelet neural networks and dual-domain networks), and three supervised classifiers (support vector machine, random forest, and Naive Bayes), showing the effectiveness of this method in decrease of false-positive rates and increase of change detection accuracy. The average and maximum of the accuracy metric F1 score of our method are 0.6842 and 0.8708, respectively; the average F1 score of the unsupervised, deep learning, and supervised methods are 0.5049, 0.4943, and 0.633, respectively.

Comparing Three Methods of Selecting Training Samples in Supervised Classification of Multispectral Remote Sensing Images.

Selecting training samples is crucial in remote sensing image classification. In this paper, we selected three images-Sentinel-2, GF-1, and Landsat 8-and employed three methods for selecting training samples: grouping selection, entropy-based selection, and direct selection. We then used the selected training samples to train three supervised classification models-random forest (RF), support-vector machine (SVM), and k-nearest neighbor (KNN)-and evaluated the classification results of the three images. According to the experimental results, the three classification models performed similarly. Compared with the entropy-based method, the grouping selection method achieved higher classification accuracy using fewer samples. In addition, the grouping selection method outperformed the direct selection method with the same number of samples. Therefore, the grouping selection method performed the best. When using the grouping selection method, the image classification accuracy increased with the increase in the number of samples within a certain sample size range.

Prediction of Au-Polymetallic Deposits Based on Spatial Multi-Layer Information Fusion by Random Forest Model in the Central Kunlun Area of Xinjiang, China

In recent years, there has been a growing emphasis on combining intelligent prospecting algorithms, such as random forest, with extensive geological and mineral data for the purpose of quantitatively predicting exploration geochemistry. This approach holds significant importance for enhancing the accuracy of target delineation. The central Kunlun area in Xinjiang possesses highly favorable ore-forming geological conditions, offering excellent prospects for mineral exploration. However, the depletion of shallow deposits coupled with a decade-long gap in geological exploration have presented increasing challenges in the quest to discover substantial metal resources. Consequently, there is now a severe shortage of reserve assets in the region, prompting an urgent need for the implementation of new theories, methods, and technologies in mineral resource investigation and evaluation efforts. The researchers used geological and regional geochemical data to construct a random forest metallogenic discriminant model for predicting the mineralization of gold polymetallic minerals in the central Kunlun area of Xinjiang and delineating the metallogenic target area. Two different sampling methods were compared to quantitatively predict gold polymetallic mineral resources. The results indicate that the selected training samples offer higher prediction accuracy and reliability by fully capturing the complex information of the original data. The random forest model using select training samples has valuable applications in metallogenic prospect prediction and potential division due to its ability to consider the actual exploration cost and identify small areas with high potential and a high proportion of ore. This study significantly improves prediction accuracy, reduces exploration risk, and expands the use of machine learning algorithms in mathematical geology in the central Kunlun area of Xinjiang.

Multi-Scale Recursive Identification of Urban Functional Areas Based on Multi-Source Data

The study of urban functional area identification is of great significance for urban function cognition, spatial planning, and economic development. In the identification of urban functional areas, most studies considered only a single data source and a single division scale, the research results have problems such as low update frequency or incomplete information in a single data set, and overfitting or underfitting in a single spatial resolution. Aiming at the above problems, this paper proposes a multi-scale recursive recognition method based on interactive validation for urban functional areas using taxi trajectory data and point of interest (POI) data as the main data sources. First, the dynamic time warping (DTW) algorithm generates a time series similarity matrix, a CA-RFM model combining the clustering algorithm and random forest model is constructed. The model extracts significant feature regions as inputs through a K-medoid clustering algorithm, which are imported into the random forest model for urban functional zone (UFZ) identification. Then, to overcome the shortcomings of a single scale in expressing urban structural characteristics, a recursive model of different levels of urban road networks is established to classify multi-scale functional areas. Finally, cross-validation using the CA-RFM model and POI quantitative identification method obtains the final identification results of urban functional areas. This paper selects Shenzhen as the study area, the results show that the combination of clustering algorithm and random forest model greatly reduces the error of manual selection of training samples. In addition, the study demonstrates the superiority of the proposed method in two aspects, namely, faster delineation and improved accuracy in urban functional area identification.