Selection Of Training Samples Research Articles

Automated identification of earthen berms in Western US rangelands from LiDAR‐based digital elevation models

AbstractEarthworks such as earthen berms have been constructed across the western US since the late 1800s to mitigate erosion in landscapes where water is both the dominant driver of erosion and the limiting resource for biota. Berms alter hydrologic, geomorphic and ecologic processes by intercepting runoff and altering patterns of water availability in the landscape. Understanding site‐specific changes in process dynamics requires accurate mapping of berm locations and knowledge of their condition. This paper presents an automated, object‐based framework for identifying earthen berms from 1 m LiDAR‐derived digital elevation models in the western US rangelands. Geomorphon, a computer vision tool, was used to classify landforms and identify berm‐like landforms, including summits and ridges. Ten geomorphic and geometric attributes associated with each potential berm object were used to develop a machine‐learning model for distinguishing berms from natural summits and ridges. The model was trained and applied to independent test sites to identify and map berms. The mapped berms were compared with manually identified reference berms for accuracy assessment. The identification results achieved 79% to 87% recall, 82% to 92% precision and 81% to 89% F‐measure. We also explored the influence of training sample selection on model performance and conducted an analysis of attribute relative importance. The automated framework has the potential to be scaled up to larger areas in semi‐arid environments.

Continual compression model for online continual learning

Task-Free Continual Learning (TFCL) presents a notably demanding but realistic ongoing learning concept, aiming to address catastrophic forgetting in sequential learning systems. In this paper, we tackle catastrophic forgetting by introducing an innovative dynamic expansion framework designed to adaptively enhance the model’s capacity for novel data learning while also remembering the information learnt in the past, by using a minimal-size processing architecture. Our proposed framework incorporates three key mechanisms to mitigate model’ forgetting: (1) by employing a Maximum Mean Discrepancy (MMD)-based expansion mechanism that assesses the disparity between previously acquired knowledge and that from the new training data, serving as a signal for the model’s architecture expansion; (2) a component discarding mechanism that eliminates components characterized by redundant information, thereby optimizing the model size while fostering knowledge diversity; (3) a novel training sample selection strategy that leads to the diversity of the training data for each task. We conduct a series of TFCL experiments that demonstrate the superiority of the proposed framework over all baselines while utilizing fewer components than alternative dynamic expansion models. The results on the Split Mini ImageNet dataset, after splitting the original dataset into multiple tasks, are improved by more than 2% when compared to the closest baseline.

Self-supervised change detection of heterogeneous images based on difference algorithms

ABSTRACT The presence of heterogeneous image disparities often leads to inferior quality in the generated difference images during change detection. This paper proposes a self-supervised change detection of heterogeneous images based on a difference algorithm. Firstly, a combination of phase consistency and a simplified pulse-coupled neural network (PC-SPCNN) is used to fuse the heterogeneous images, and the result is used to compute the difference image (DI). The new DI generation method can generate the standard and exponential difference images. Secondly, the hierarchical FCM clustering algorithm is improved to extract stable and correct self-supervised samples by difference images so that the clustering process is not overly dependent on thresholds. Then, the support vector machine classifier is trained based on the heterogeneous images, the fused images, and self-supervised sample sets, and the information from the fused images is utilized to increase the feature dimension for better detection of changes. Finally, the support vector machine classifier automatically detects whether the intermediate pixels are changed and produces the change detection results. The experimental results confirm the improvements made by the proposed method in difference image extraction, training sample selection, and clustering algorithm, and the stability of the method exceeds that of the state-of-the-art change detection methods.

A novel method combining deep learning with the Kennard-Stone algorithm for training dataset selection for image-based rice seed variety identification.

Different varieties of rice vary in planting time, stress resistance, and other characteristics. With advances in rice-breeding technology, the number of rice varieties has increased significantly, making variety identification crucial for both trading and planting. This study collected RGB images of 20 hybrid rice seed varieties. An enhanced deep super-resolution network (EDSR) was employed to enhance image resolution, and a variety classification model utilizing the high-resolution dataset demonstrated superior performance to that of the model using the low-resolution dataset. A novel training sample selection methodology was introduced integrating deep learning with the Kennard-Stone (KS) algorithm. Convolutional neural networks (CNN) and autoencoders served as supervised and unsupervised feature extractors, respectively. The extracted feature vectors were subsequently processed by the KS algorithm to select training samples. The proposed methodologies exhibited superior performance over the random selection approach in rice variety classification, with an approximately 10.08% improvement in overall classification accuracy. Furthermore, the impact of noise on the proposed methodology was investigated by introducing noise to the images, and the proposed methodologies maintained superior performance relative to the random selection approach on the noisy image dataset. The experimental results indicate that both supervised and unsupervised learning models performed effectively as feature extractors, and the deep learning framework significantly influenced the selection of training set samples. This study presents a novel approach for training sample selection in classification tasks and suggests the potential for extending the proposed method to image datasets and other types of datasets. Further exploration of this potential is warranted. © 2024 Society of Chemical Industry.

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

One of the focal points in the field of intelligent transportation is the intelligent control of traffic signals (TS), aimed at enhancing the efficiency of urban road networks through specific algorithms. Deep Reinforcement Learning (DRL) algorithms have become mainstream, yet they suffer from inefficient training sample selection, leading to slow convergence. Additionally, enhancing model robustness is crucial for adapting to diverse traffic conditions. Hence, this paper proposes an enhanced method for traffic signal control (TSC) based on DRL. This approach utilizes dueling network and double q-learning to alleviate the overestimation issue of DRL. Additionally, it introduces a priority sampling mechanism to enhance the utilization efficiency of samples in memory. Moreover, noise parameters are integrated into the neural network model during training to bolster its robustness. By representing high-dimensional real-time traffic information as matrices, and employing a phase-cycled action space to guide the decision-making of intelligent agents. Additionally, utilizing a reward function that closely mirrors real-world scenarios to guide model training. Experimental results demonstrate faster convergence and optimal performance in metrics such as queue length and waiting time. Testing experiments further validate the method's robustness across different traffic flow scenarios.

TL-YOLO: Foreign-Object Detection on Power Transmission Line Based on Improved Yolov8

Foreign objects on power transmission lines carry a significant risk of triggering large-scale power interruptions which may have serious consequences for daily life if they are not detected and handled in time. To accurately detect foreign objects on power transmission lines, this paper proposes a TL-Yolo method based on the Yolov8 framework. Firstly, we design a full-dimensional dynamic convolution (ODConv) module as a backbone network to enhance the feature extraction capability, thus retaining richer semantic content and important visual features. Secondly, we present a feature fusion framework combining a weighted bidirectional feature pyramid network (BiFPN) and multiscale attention (MSA) module to mitigate the degradation effect of multiscale feature representation in the fusion process, and efficiently capture the high-level feature information and the core visual elements. Thirdly, we utilize a lightweight GSConv cross-stage partial network (GSCSP) to facilitate efficient cross-level feature fusion, significantly reducing the complexity and computation of the model. Finally, we employ the adaptive training sample selection (ATSS) strategy to balance the positive and negative samples, and dynamically adjust the selection process of the training samples according to the current state and performance of the model, thus effectively reducing the object misdetection and omission. The experimental results show that the average detection accuracy of the TL-Yolo method reaches 91.30%, which is 4.20% higher than that of the Yolov8 method. Meanwhile, the precision and recall metrics of our method are 4.64% and 3.53% higher than those of Yolov8. The visualization results also show the superior detection performance of the TL-Yolo algorithm in real scenes. Compared with the state-of-the-art methods, our method achieves higher accuracy and speed in the detection of foreign objects on power transmission lines.

Semantically redundant training data removal and deep model classification performance: A study with chest X-rays

Deep learning (DL) has demonstrated its innate capacity to independently learn hierarchical features from complex and multi-dimensional data. A common understanding is that its performance scales up with the amount of training data. However, the data must also exhibit variety to enable improved learning. In medical imaging data, semantic redundancy, which is the presence of similar or repetitive information, can occur due to the presence of multiple images that have highly similar presentations for the disease of interest. Also, the common use of augmentation methods to generate variety in DL training could limit performance when indiscriminately applied to such data. We hypothesize that semantic redundancy would therefore tend to lower performance and limit generalizability to unseen data and question its impact on classifier performance even with large data. We propose an entropy-based sample scoring approach to identify and remove semantically redundant training data and demonstrate using the publicly available NIH chest X-ray dataset that the model trained on the resulting informative subset of training data significantly outperforms the model trained on the full training set, during both internal (recall: 0.7164 vs 0.6597, p<0.05) and external testing (recall: 0.3185 vs 0.2589, p<0.05). Our findings emphasize the importance of information-oriented training sample selection as opposed to the conventional practice of using all available training data.

Estimation of canopy height based on multi-source remote sensing data using forest structure aided sample selection

ABSTRACT Forest canopy height data are crucial for estimating forest carbon storage and assessing forest ecology. By utilizing satellite imagery, canopy height data obtained from airborne or spaceborne LiDAR have been expanded from footprint and plot levels to spatially continuous elevation mapping of forests. However, current research suggests that estimating forest canopy height without forest type data presents a challenge in how to effectively integrate multi-source LiDAR data and ensure the samples adequately represent various forest types for higher estimation accuracy. Therefore, this study proposes a forest canopy height estimation method that considers forest structure and integrates multi-source LiDAR data to overcome the challenge. First, a stratified sampling method based on forest structure (SSMFS) was proposed to select training samples and enhance their representativeness. Second, we combined GEDI and ATL08 data to create a multi-source spaceborne LiDAR dataset, enhancing geographic coverage and increasing canopy height samples. Third, the spaceborne LiDAR-based canopy height estimation model incorporates previously unconsidered canopy openness features and uses SSMFS to select training samples. Finally, we improved spaceborne LiDAR canopy height accuracy by creating a residual correction model that adjusts for differences between airborne scanner (ALS) and spaceborne LiDAR estimates. This study, conducted in Zhangwu County, achieved an accuracy of R2 = 0.71, MAE = 1.20 m, and RMSE = 1.71 m. These results show a 51.06% increase in R2, a 26.38% decrease in MAE, and a 24.00% decrease in RMSE compared to recent research. In summary, this study profoundly amplifies predictive accuracy, providing a clear advantage in the delineation of regional forest canopy maps.

Improving Semantic Segmentation via Efficient Self-Training.

Starting from the seminal work of Fully Convolutional Networks (FCN), there has been significant progress on semantic segmentation. However, deep learning models often require large amounts of pixelwise annotations to train accurate and robust models. Given the prohibitively expensive annotation cost of segmentation masks, we introduce a self-training framework in this paper to leverage pseudo labels generated from unlabeled data. In order to handle the data imbalance problem of semantic segmentation, we propose a centroid sampling strategy to uniformly select training samples from every class within each epoch. We also introduce a fast training schedule to alleviate the computational burden. This enables us to explore the usage of large amounts of pseudo labels. Our Centroid Sampling based Self-Training framework (CSST) achieves state-of-the-art results on Cityscapes and CamVid datasets. On PASCAL VOC 2012 test set, our models trained with the original train set even outperform the same models trained on the much bigger augmented train set. This indicates the effectiveness of CSST when there are fewer annotations. We also demonstrate promising few-shot generalization capability from Cityscapes to BDD100K and from Cityscapes to Mapillary datasets.

A novel framework based on two-stage multi-view feature optimization and improved support vector data description for aeroengine bearing early fault detection

Early fault detection is crucial to avoid catastrophic flight accidents caused by unplanned downtime of equipment. Aimed at the shortcomings of feature selection and early fault detection, this paper proposes a method combining two-stage multi-view feature optimization and improved support vector data description for aeroengine bearing early fault detection. First, a comprehensive evaluation index is constructed via combining monotonicity, correlation, robustness index and entropy weight method to be used for initial screening sensitive degradation features from multi-view candidate degradation feature. After that, the hierarchical clustering algorithm and local preserving projection algorithm is introduced to realize sensitive degradation features clustering and dimensionality reduction. On this basis, the Euclidean distance sample reduction and the sparrow search algorithm is utilized to complete the training sample selection and model parameter optimization of different clusters and further use the verification samples to filter the optimal features and model. Finally, based on the optimal degradation features and models, a fusion performance degradation index is built to achieve intelligent detection of early fault. Comparisons with some existing fault detection methods, the proposed method improves degradation trend sensibility and demonstrates potential of good early fault detection engineering applications.

Robust Adaptive Radar Beamforming Based on Iterative Training Sample Selection Using Kurtosis of Generalized Inner Product Statistics

Robust Adaptive Radar Beamforming Based on Iterative Training Sample Selection Using Kurtosis of Generalized Inner Product Statistics

Adaptive Data-Driven Deep-Learning Surrogate Model for Frontal Polymerization in Dicyclopentadiene.

Frontal polymerization (FP) is a self-sustaining curing process that enables rapid and energy-efficient manufacturing of thermoset polymers and composites. Computational methods conventionally used to simulate the FP process are time-consuming, and repeating simulations are required for sensitivity analysis, uncertainty quantification, or optimization of the manufacturing process. In this work, we develop an adaptive surrogate deep-learning model for FP of dicyclopentadiene (DCPD), which predicts the evolution of temperature and degree of cure orders of magnitude faster than the finite-element method (FEM). The adaptive algorithm provides a strategy to select training samples efficiently and save computational costs by reducing the redundancy of FEM-based training samples. The adaptive algorithm calculates the residual error of the FP governing equations using automatic differentiation of the deep neural network. A probability density function expressed in terms of the residual error is used to select training samples from the Sobol sequence space. The temperature and degree of cure evolution of each training sample are obtained by a 2D FEM simulation. The adaptive method is more efficient and has a better prediction accuracy than the random sampling method. With the well-trained surrogate neural network, the FP characteristics (front speed, shape, and temperature) can be extracted quickly from the predicted temperature and degree-of-cure fields.

Time-dependent reliability analysis of structural systems based on parallel active learning Kriging model

Time-dependent reliability analysis quantifies the failures of structural systems due to time-dependent uncertainties, such as material degradation and dynamic loads. The active learning Kriging model methods are widely used in structural reliability analysis to replace extensive time-consuming finite element simulations. However, they can only update one training sample and one failure mode per iteration, which limits their application to time-dependent, parallel computing, and multiple failure modes problems. In this study, we propose a new parallel active learning Kriging model for time-dependent reliability analysis, which can update multiple training samples and multiple failure modes per iteration. It includes the following strategies: (1) a novel parallel learning function is proposed, which combines the correlation function and U learning function to allow for the selection of multiple training samples per iteration; (2) an adaptive adjustment strategy for the number of parallel samples is proposed, which takes into account the prediction probability of parallel samples; (3) the proposed parallel learning function is integrated into time-dependent reliability analysis with multiple failure modes, enabling simultaneous updates of multiple training samples and failure modes, thus greatly reducing the number of iterations and computational time; and (4) a new stopping criterion is proposed to improve the efficiency of the estimation of failure probability. The proposed method can be applied to series or parallel time-dependent structural systems with multiple failure modes. We demonstrate the effectiveness of the proposed method through three examples, and the proposed method can achieve a balance between the computational time and function calls while maintaining a high level of accuracy in the estimation of time-dependent failure probability.

A learnable population filter for dynamic multi-objective optimization

In this paper, a novel learnable population filter is proposed to solve the dynamic multi-objective problems (DMOPs), whose main idea is to pick out potential valuable individuals in the changing situation via the trained filter to facilitate the population convergence in a new environment. In particular, by selecting training samples from three different optional pools, rich information in the previous searching experiences can be sufficiently used to cope with the dynamic behaviors in DMOPs, thereby handling the imperfect history data availability caused by the environmental uncertainty to some extents. Then, the screened population in the new environment, which is regarded to have reliable quality that benefits accelerating the evolution, is used to initialize the static optimizer. In addition, partial random solutions are employed as supplement to the initial population to further enhance the diversity. It is shown from the benchmark evaluations that the proposed method outperforms some other popular algorithms on both convergence and diversity by seamlessly combining the mainstream strategies in dealing with the dynamic behaviors, which is a competitive method that conforms to the novel evolutionary transfer optimization (ETO) trend. Moreover, effectiveness of the core components is validated through extensive ablation studies, which brings a new sight in developing the learnable DMOP solvers.

A Review of Pattern Recognition and Machine Learning

This article aims to provide a concise overview of diverse methodologies employed at different stages of a pattern recognition system, highlighting contemporary research challenges and applications in this dynamic field. The integration of machine learning techniques has played a pivotal role in converging pattern recognition frameworks in academic literature. The process relies heavily on supervised or unsupervised categorization methods to achieve its objectives, with a notable focus on statistical approaches. More recently, there is a growing emphasis on incorporating neural network methodologies and insights from statistical learning theory. Designing an effective recognition system necessitates careful consideration of various factors, including pattern representation, pattern class definition, feature extraction, sensing environment, feature selection, classifier learning and design, cluster analysis, test and training sample selection, and performance assessment.

Fishing risky behavior recognition based on adaptive transformer, reinforcement learning and stochastic configuration networks

The fishing behavior in a high-voltage environment may lead to electric shock accidents. This paper proposes a fishing risky behavior recognition method based on adaptive Transformer, reinforcement learning and stochastic configuration networks (SCNs). Firstly, the fishing behavior image sample training set is evaluated using the actor network based on linear entropy in the reinforcement learning module to select suitable training samples that possess abundant straight-line elements. Subsequently, the selected training samples are fed into a multi-scale spatial deep convolution to extract continuous spatial features of elongated objects. The deformable Transformer network, enhanced with adaptive encoding and decoding layers through SCNs, is used to obtain the position and detection results of the fishing rod. The trained Transformer model is evaluated by a defined credibility evaluation metric for acquiring rewards to update the training sample set iteratively. Then, an adaptive adjustment mechanism is constructed for the encoding and decoding layers of the adaptive Transformer network to establish a library of adaptive Transformer models with different encoding and decoding layer levels to accommodate the feature requirements of training samples in various scenarios. Finally, the blending learning method for combining the detection results from the model library is integrated with human body object detection and pose estimation methods, and a predefined expert system is utilized for logic reasoning on fishing risky behavior. Experimental results demonstrate the effectiveness of the proposed method in this paper.

A lightweight grape detection model in natural environments based on an enhanced YOLOv8 framework.

Grapefruit and stem detection play a crucial role in automated grape harvesting. However, the dense arrangement of fruits in vineyards and the similarity in color between grape stems and branches pose challenges, often leading to missed or false detections in most existing models. Furthermore, these models' substantial parameters and computational demands result in slow detection speeds and difficulty deploying them on mobile devices. Therefore, we propose a lightweight TiGra-YOLOv8 model based on YOLOv8n. Initially, we integrated the Attentional Scale Fusion (ASF) module into the Neck, enhancing the network's ability to extract grape features in dense orchards. Subsequently, we employed Adaptive Training Sample Selection (ATSS) as the label-matching strategy to improve the quality of positive samples and address the challenge of detecting grape stems with similar colors. We then utilized the Weighted Interpolation of Sequential Evidence for Intersection over Union (Wise-IoU) loss function to overcome the limitations of CIoU, which does not consider the geometric attributes of targets, thereby enhancing detection efficiency. Finally, the model's size was reduced through channel pruning. The results indicate that the TiGra-YOLOv8 model's mAP(0.5) increased by 3.33% compared to YOLOv8n, with a 7.49% improvement in detection speed (FPS), a 52.19% reduction in parameter count, and a 51.72% decrease in computational demand, while also reducing the model size by 45.76%. The TiGra-YOLOv8 model not only improves the detection accuracy for dense and challenging targets but also reduces model parameters and speeds up detection, offering significant benefits for grape detection.

A multi-surrogate multi-tasking genetic algorithm with an adaptive training sample selection strategy for expensive optimization problems

Multi-surrogate assisted evolutionary algorithms, utilizing inexpensive global and local surrogate models to assist evolutionary search, have demonstrated a remarkable ability to solve expensive optimization problems (EOPs). The selected training samples of the local surrogate model used in the current algorithms may not be updated continuously, which could bring about a waste of computing resources. This paper proposes a multi-surrogate multi-tasking genetic algorithm with an adaptive training sample selection strategy (MS-MTGAwA), in which the local surrogate model is updated adaptively based on information from established local models to enhance the capability in exploiting the optimal solution. The adaptive training sample selection strategy (ATS) applies optimal points found in historical local models as training samples. In addition, MS-MTGAwA inherits the optimization framework of a multi-tasking genetic algorithm. The radial basis function model is chosen as the modeling basis of the global and local surrogate models. A set of common benchmark functions whose dimensions vary from 10 to 100 and the tension/compression spring design problem are adopted to validate the performance of the proposed algorithm. Based on the average best fitness value over thirty independent runs on the benchmark functions, MS-MTGAwA ranks first in Friedman’s test compared to five state-of-the-art algorithms including S-JADE, GORS-SSLPSO, SAHO, SAMSO, and MS-MTO. The proposed ATS can be expected to be applied in any algorithms with local surrogate to make full use of historical information of the local model to improve the prediction accuracy of the next local model.If this paper is accepted, MATLAB codes associated with this paper will be uploaded tohttps://github.com/Zhongbo-Hu/Prediction-Evolutionary-Algorithm-HOMEPAGE.

CLFR-Det: Cross-level feature refinement detector for tiny-ship detection in SAR images

Ship detection in synthetic aperture radar (SAR) images is an important and active topic, due to the characteristics of SAR images involving all-time and all-weather imaging. However, complex backgrounds caused by speckle noise and inshore land, coupled with information deficiency of tiny ships, pose a great challenge for tiny-ship detection in SAR images. To tackle this problem, we present a cross-level feature refinement detector (CLFR-Det) that utilizes features reflecting different levels and distinct semantics (classification and localization). To enrich the semantic information of tiny ships, our approach incorporates a cross-level modulated deformable convolution to aggregate features from relevant positions across multi-level feature maps. This is supported by a spatially-informed multi-scale feature refinement mechanism that combines the features for classification and those for localization. We implement a uniform IoU-weighted adaptive training sample selection method for equitably distributing the impact of positive samples from targets of various sizes during the training process. A generalized IoU loss between ground-truth and preliminary bounding box is further proposed to supervise the learning process of the CLFR-Det, with uncertainty weights incorporated to dynamically depict the levels of disparate losses, enabling adequate training across different tasks. Also, we construct a novel tiny SAR ship detection dataset to comprehensively evaluate the effectiveness of our system, in conjunction with the use of publicly available SSDD and HRSID datasets. Experimental investigations demonstrate that CLFR-Det generally surpasses state-of-the-art performance for multi-scale ship detection, particularly for the detection of tiny ships.

GAN-assisted Road Segmentation from Satellite Imagery

Geo-information extraction from satellite imagery has become crucial to carry out large-scale ground surveys in a short amount of time. With the increasing number of commercial satellites launched into orbit in recent years, high-resolution RGB color remote sensing imagery has attracted a lot of attention. However, because of the high cost of image acquisition and even more complicated annotation procedures, there are limited high-resolution satellite datasets available. Compared to close-range imagery datasets, existing satellite datasets have a much lower number of images and cover only a few scenarios (cities, background environments, etc. ). They may not be sufficient for training robust learning models that fit all environmental conditions or be representative enough for training regional models that optimize for local scenarios. Instead of collecting and annotating more data, using synthetic images could be another solution to boost the performance of a model. This study proposes a GAN-assisted training scheme for road segmentation from high-resolution RGB color satellite images, which includes three critical components: a) synthetic training sample generation, b) synthetic training sample selection, and c) assisted training strategy. Apart from the GeoPalette and cSinGAN image generators introduced in our prior work, this paper in detail explains how to generate new training pairs using OpenStreetMap (OSM) and introduces a new set of evaluation metrics for selecting synthetic training pairs from a pool of generated samples. We conduct extensive quantitative and qualitative experiments to compare different image generators and training strategies. Our experiments on the downstream road segmentation task show that 1) our proposed metrics are more aligned with the trained model performance compared to commonly used GAN evaluation metrics such as the Fréchet inception distance (FID); and 2) by using synthetic data with the best training strategy, the model performance, mean Intersection over Union (mean IoU), is improved from 60.92% to 64.44%, when 1,000 real training pairs are available for learning, which reaches a similar level of performance as a model that is standard-trained with 4,000 real images (64.59%), i.e. , enabling a 4-fold reduction in real dataset size.