Training Samples Research Articles

A Self-Distilled Learning to Rank Model for Ad Hoc Retrieval

Learning to rank models are broadly applied in ad hoc retrieval for scoring and sorting documents based on their relevance to textual queries. The generalizability of the trained model in the learning to rank approach, however, can have an impact on the retrieval performance, particularly when data includes noise and outliers, or is incorrectly collected or measured. In this paper, we introduce a Self-Distilled Learning to Rank (SDLR) framework for ad hoc retrieval, and analyze its performance over a range of retrieval datasets and also in the presence of features’ noise. SDLR assigns a confidence weight to each training sample, aiming at reducing the impact of noisy and outlier data in the training process. The confidence weight is approximated based on the feature’s distributions derived from the values observed for the features of the documents labeled for a query in a listwise training sample. SDLR includes a distillation process that facilitates passing on the underlying patterns in assigning confidence weights from the teacher model to the student one. We empirically illustrate that SDLR outperforms state-of-the-art learning to rank models in ad hoc retrieval. We thoroughly investigate the SDLR performance in different settings including when no distillation strategy is applied; when different portion of data are used for training the teacher and the student models, and when both teacher and student models are trained over identical data. We show that SDLR is more effective when training data are split between a teacher and a student model. We also show that SDLR’s performance is robust when data features are noisy.

New Approaches to Extremal Index Estimation

The extremal index is a parameter associated with the extreme value distributions of dependent stationary sequences. Under certain local dependence conditions, exceedances above a specified threshold tend to occur in isolated clusters. The reciprocal of the extremal index can be interpreted as the limiting size of these clusters. Accurately estimating the size of such clusters is crucial for analyzing real data and can significantly influence decision making processes that impact population well being. The paper presents a recent method for the estimation of the extremal index which starts by the estimation of the parameter itself and, only then, to use that estimate in the cluster mean size estimation. The procedure starts with the estimation of a specific proportion by the corresponding relative frequency. Thus, it is very simple, intuitive, it has good statistical properties, and it does not depend on the method used for the mean cluster estimation. The interpretation of the extremal index as a proportion is known, but it has not been used directly as an estimation method. In recent years, various authors have proposed different estimators for the extremal index. This paper applies some of the latest estimation methods for the extremal index to real data and analyses their performance using training and test samples. The results are compared with other well known estimators, for which R packages are available. The results show a better performance of the Proportion estimator, followed by the Gaps estimator, when compared to the other considered index estimators.

Research on Status Assessment and Operation and Maintenance of Electric Vehicle DC Charging Stations Based on XGboost

With the rapid development of electric vehicles, the infrastructure for charging stations is also expanding quickly, and the failure rate of charging piles is increasing. To address the effective operation and maintenance of charging stations, a method based on the XGBoost algorithm for electric vehicle DC charging stations is proposed. An operation and maintenance system is constructed based on state analysis, considering the operational status of the charging stations and users’ charging habits. Factors such as driving and charging habits, road traffic, and charging station equipment are taken into account. The training sample data are established using historical data, online monitoring data, and external environmental data, and the charging station status evaluation model is trained using the XGBoost algorithm. Based on the condition assessment results, a risk assessment model is established in combination with fault parameters. Risk tracking of the charging stations is conducted using the energy not charged (ENC), evaluating the risk level of each station and determining the operation and maintenance order. The optimal operation and maintenance model for DC charging stations, aimed at achieving both economic and reliability goals, is constructed to determine the operation and maintenance schedule for each station. The results of the case study demonstrate that the state evaluation and operation and maintenance strategy can significantly improve the reliability of the system and the overall benefits of operation and maintenance while meeting the required standards.

Enhancing slope stability prediction through integrated PCA-SSA-SVM modeling: a case study of LongLian expressway

China is one of the regions most frequently affected by landslides, which have significant socio-economic impacts. Traditional slope stability analysis methods, such as the limit equilibrium method, limit analysis method, and finite element method, often face limitations due to computational complexity and the need for extensive soil property data. This study proposes a novel approach that combines Principal Component Analysis (PCA), Sparrow Search Algorithm (SSA), and Support Vector Machine (SVM) to improve the accuracy of slope stability prediction. PCA effectively reduces data dimensionality while retaining critical information. SSA optimizes SVM parameters, addressing the limitations of traditional optimization methods. The integrated PCA-SSA-SVM model was applied to a dataset of 257 slope stability samples and validated using five-fold cross-validation to ensure the model’s generalization capability. The results show that the model exhibits superior performance in prediction accuracy, precision, recall, and F1-score, with the test set achieving an accuracy of 84.6%, a recall of 84.7%, a precision of 83.1%, and an F1-score of 84.6%. The model’s robustness was further validated using slope data from the LongLian Expressway, demonstrating high consistency with the actual stability status. These findings indicate that the PCA-SSA-SVM-based slope stability prediction model has significant potential for practical engineering applications, providing a reliable and efficient tool for slope stability forecasting. Classify the training samples through cross-validation, using the accuracy of cross-validation as the fitness of the sparrow individual. Retain the optimal fitness value and position information.

Adaptive Hybrid Prediction Model for Adapting to Data Distribution Shifts in Machining Quality Prediction

Abstract The accuracy of data-driven intelligent prediction for machining quality relies on the training samples. However, in actual applications, the continuous operation of machining equipment leads to gradual distribution shifts between the process data and the training samples for modeling. The shifts result in a degradation in the performance of predictive model, previous studies have often overlooked this issue. To tackle with the intricate problem, this research proposes a real-time model optimization approach. Firstly, a method for detecting machining data distribution shifts based on the two-sample Kolmogorov-Smirnov (KS) test is proposed. Then, an adaptive hybrid prediction model (AHPM) capable of real-time optimization is developed. This model consists of a deep neural network (DNN) and a broad learning system (BLS). DNN plays a primary role in prediction within the hybrid model with excellent generalization capability. BLS quickly completes optimization prior to DNN with its unique parameter update mechanism to compensate for prediction loss. Experimental results indicate that AHPM achieves the shortest optimization time while maintaining high accuracy, with post-optimization error reduction rates for MSE, MAE, and MAPE all exceeding 10%. In the test of application to actual machining cases, accuracy improved by 8.88% compared to traditional methods without optimization.

High-dimensional asymptotics of denoising autoencoders*

Abstract We address the problem of denoising data from a Gaussian mixture using a two-layer non-linear autoencoder with tied weights and a skip connection. We consider the high-dimensional limit where the number of training samples and the input dimension jointly tend to infinity while the number of hidden units remains bounded. We provide closed-form expressions for the denoising mean-squared test error. Building on this result, we quantitatively characterize the advantage of the considered architecture over the autoencoder without the skip connection that relates closely to principal component analysis. We further show that our results accurately capture the learning curves on a range of real data sets.

What can be learnt with wide convolutional neural networks?*

Abstract Understanding how convolutional neural networks (CNNs) can efficiently learn high-dimensional functions remains a fundamental challenge. A popular belief is that these models harness the local and hierarchical structure of natural data such as images. Yet, we lack a quantitative understanding of how such structure affects performance, for example the rate of decay of the generalisation error with the number of training samples. In this paper, we study infinitely wide deep CNNs in the kernel regime. First, we show that the spectrum of the corresponding kernel inherits the hierarchical structure of the network, and we characterise its asymptotics. Then, we use this result together with generalisation bounds to prove that deep CNNs adapt to the spatial scale of the target function. In particular, we find that if the target function depends on low-dimensional subsets of adjacent input variables then the decay of the error is controlled by the effective dimensionality of these subsets. Conversely, if the target function depends on the full set of input variables then the error decay is controlled by the input dimension. We conclude by computing the generalisation error of a deep CNN trained on the output of another deep CNN with randomly initialised parameters. Interestingly, we find that, despite their hierarchical structure, the functions generated by infinitely wide deep CNNs are too rich to be efficiently learnable in high dimensions.

Structure-Based Training: A Training Method Aimed at Pixel Errors for a Correlation-Coefficient-Based Neural Network.

In research on building a one-shot learning neural network without pre-training using mass data, the limitation on the information obtained from a single training sample downgrades the performance of the network. In order to improve performance and take full advantage of the support set, in this study, we design three kinds of shadow nodes and propose a structure-based training method for a correlation-coefficient-based neural network. This training strategy focuses on branches that are not activated or inactivated as expected. In contrast to existing networks that optimize the parameters using back-propagation, the training method proposed in this paper optimizes the structure of the correlation-coefficient-based network by correcting its pixel errors. For the shadow nodes and training process based on this strategy, the intersection over union (IOU) of a detected target increases by 4.83% in the experiments when using the Fashion-Mnist dataset, increases by 4.02% when using the Omniglot dataset, and increases by 3.89% when using the Cifar-10 dataset. The samples in category "7" wrongly classified as "1" decreased by 27.32% when using the Mnist dataset after training. This training strategy, along with shadow nodes, makes the correlation-coefficient-based network a more practical model and enables the network to develop during the accumulation of reliable samples, thus making it more suitable for simple target detection projects that collect samples over time. Moreover, the shadow nodes and training method proposed in this paper supplement the non-gradient-based parameter-gaining strategy. Additionally, it is a new attempt to explore the imitation of a human's ability to learn a new pattern from a low number of references.

Dynamic Coastal Mapping Using Sentinel-1 and Sentinel-2 Data Through Digital Earth Africa

Abstract. Coastal erosion poses a continuous threat to ecosystems, infrastructure, and property. To address these challenges and mitigate the effects of coastal changes, effective and current monitoring is essential. It is particularly important to monitor coastlines and coastal changes in Africa, where a significant portion of the population resides in coastal regions. While optical satellite imagery has been used for large-scale annual coastlines and change monitoring for Africa, its availability and quality are largely limited by the presence of cloud and cloud shadow. In comparison, using radar satellite observations such as Sentinel-1 data can provide consistent coastal mapping and change detection regardless of cloud presence. This paper outlines a fully automated supervised machine learning workflow using Sentinel-1 data and training samples extracted from Sentinel-2 data. It also explores the performance of the workflow for different coastal morphology types across the African coast. The workflow has proved to perform better and produced results that were visually more consistent with Sentinel-2 data compared to thresholding methods. While challenges exist to distinguish between land and water over smooth sandy beaches and rough near-shore water surfaces, our workflow provides an alternative method for coastal change mapping where optical satellites provide insufficient observations free from clouds. Python code of the proposed methodology has been made publicly available.

Inversion of biophysical parameters of potato based on an active learning pool-based sampling strategy

ABSTRACT Timely estimations of leaf chlorophyll content (LCC) and leaf area index (LAI) can provide critical information for potato field management. We employed a hybrid method that integrated machine learning with a radiative transfer model to estimate potato growth parameters. A look-up table was generated using the PROSAIL model, which was used as an unlabelled sample set. Measurements were taken from a potato field, and the data were labelled according to growth period and variety. Then, training samples for potato LCC, LAI, and canopy chlorophyll content (CCC) were selected from the simulated unlabelled sample set using the Euclidean distance-based diversity algorithm based on different labelled data sets. The training sample size required to accurately estimate the parameters varied considerably depending on the parameter, variety, and growth period, despite using the same labelled data set. Moreover, our results indicate that the growth period has a substantial impact on model accuracy and needs to be considered when constructing the labelled data set. The study results indicate that the hybrid method combined with the radiative transfer model and active learning can effectively select informative training samples from a data pool and improve the accuracy of potato parameter estimation, which provides a valid tool for accurately monitoring crop growth and growth health.

MulCPred: Learning Multi-Modal Concepts for Explainable Pedestrian Action Prediction.

Pedestrian action prediction is crucial for many applications such as autonomous driving. However, state-of-the-art methods lack the explainability needed for trustworthy predictions. In this paper, a novel framework called MulCPred is proposed that explains its predictions based on multi-modal concepts represented by training samples. Previous concept-based methods have limitations, including the following: (1) they cannot be directly applied to multi-modal cases; (2) they lack the locality needed to attend to details in the inputs; (3) they are susceptible to mode collapse. These limitations are tackled accordingly through the following approaches: (1) a linear aggregator to integrate the activation results of the concepts into predictions, which associates concepts of different modalities and provides ante hoc explanations of the relevance between the concepts and the predictions; (2) a channel-wise recalibration module that attends to local spatiotemporal regions, which enables the concepts with locality; (3) a feature regularization loss that encourages the concepts to learn diverse patterns. MulCPred is evaluated on multiple datasets and tasks. Both qualitative and quantitative results demonstrate that MulCPred is promising in improving the explainability of pedestrian action prediction without obvious performance degradation. Moreover, by removing unrecognizable concepts, MulCPred shows improved cross-dataset prediction performance, suggesting its potential for further generalization.

An investigation of the relationship between long bone measurements and stature: Implications for estimating skeletal stature in subadults.

The present study introduces new regression formulae that address several challenges of current subadult stature estimation methods by 1) using a large, contemporary, cross-sectional sample of subadult skeletal remains; 2) generating regression models using both lengths and breadths; 3) utilizing both linear and nonlinear regression models to accommodate the nonlinear shape of long bone growth; and 4) providing usable prediction intervals for estimating stature. Eighteen long bone measurements, stature, and age were collected from computed tomography images for a sample of individuals (n = 990) between birth and 20 years from the United States. The bivariate relationship between long bone measurements and stature was modeled using linear and nonlinear methods on an 80% training sample and evaluated on a 20% testing sample. Equations were generated using pooled-sex samples. Goodness of fit was evaluated using Kolmogorov-Smirnov tests and mean absolute deviation (MAD). Accuracy and precision were quantified using percent testing accuracy and Bland-Altman plots. In total, 38 stature estimation equations were created and evaluated, all achieving testing accuracies greater than 90%. Nonlinear models generated better fits compared to linear counterparts and generally produced smaller MAD (3.65 - 15.90cm). Length models generally performed better than breadth models, and a mixture of linear and nonlinear methods resulted in highest testing accuracies. Model performance was not biased by sex, age, or measurement type. A freely available, online graphical user interface is provided for immediate use of the models by practitioners in forensic anthropology and will be expanded to include bioarchaeological contexts in the future.

Perfusion index as an early predictor of hypotension during advanced oncothoracic operations (pilot study)

Introduction. Current standards of intraoperative monitoring significantly reduce the response time to a decrease in blood pressure, but do not include methods for predicting changes in hemodynamics to avoid the effects of systemic hypoperfusion.The objective was to evaluate the usefulness of determining the perfusion index for predicting the development of hypotension and its earlier correction during advanced oncothoracic operations.Materials and methods. A prospective open-label observational study was conducted, which included 100 patients who underwent elective oncothoracic surgery. Standard monitoring was established, as well as continuous monitoring of the perfusion index. The perfusion index value was recorded immediately before induction. The patients were divided into 2 groups: in the 1st group (n = 37), patients experienced a decrease in blood pressure, requiring vasopressor support; in the 2nd group (n = 63), no vasopressor support was required.Results. A moderate linear relationship was revealed with a correlation coefficient (r) equal to (–0.68), p < 0.001, and a strong non-linear relationship with a coefficient (ρ) equal to (–0.95), p < 0.001. To identify the relationship model, polynomial regression analysis was performed. A prognostic formula for the use of vasopressors was calculated depending on the preoperative level of perfusion index. At the next stage, a classification tree for analyzing the data array was used. The entire d data array (n = 100) is randomly divided into training (n = 70) and test samples (n = 30). It was revealed that hypotension requiring vasopressor support occurred in patients whose initial perfusion index was ≤ 2.85. ROC analysis determined an AUC of 0.952 (95% CI: 0.888–1.016).Conclusion. A low PI value before induction of anesthesia during advanced oncothoracic operations can be used to monitor the functional activity of the autonomic nervous system and predict intraoperative hypotension. Patients with a baseline PI ≤ 2.85 have a higher risk of developing hypotension during advanced oncothoracic surgery compared with those with a baseline PI ≥ 2.85. The effectiveness of the method in patients with concomitant pathologies has not been tested.

Self-supervised image change detection method based on lightweight capsule network

In response to the significant impact of speckle noise on the accuracy of SAR image change detection, the high complexity of existing capsule network based image change detection methods, and the loss of a large amount of original image information in training samples, this paper proposes a self supervised image change detection method based on the Light Capsule Network (SCapsNet). First, the log ratio operator difference graph is generated, and the "pseudo label" of training samples with high confidence is obtained through the maximum inter class variance method and fuzzy C-means clustering method, which lays the foundation for self-supervised learning; Secondly, construct a three channel training sample based on the difference map of two temporal SAR images and logarithmic ratio operators to maximize the preservation of sample information; Then, SCapsNet is designed to extract training sample features through single scale convolution, and a single scale capsule network is used to mine spatial relationships between features; Finally, comparative experiments and ablation experiments were set up and tested on 5 real SAR Datasets. The experimental results show that the The advantage of this method is to improve the operational efficiency of the method while reducing model complexity, obtain stronger robust features, suppress the adverse impact of speckle noise on change detection performance, and improve change detection performance.

Domain adaptation for semantic segmentation of road scenes via two-stage alignment of traffic elements

Unsupervised domain adaptation has been used to reduce the domain shift, which would improve the performance of semantic segmentation on unlabeled real-world data. However, existing methodologies fall short in effectively addressing the domain shift issue prevalent in traffic scenarios, leading to less than satisfactory segmentation results. In this paper, we propose a novel domain adaptation method for semantic segmentation via unsupervised alignment of traffic elements. Firstly, we introduce a two-stage self-training framework that leverages a blended set of training samples to enhance the training process. In the first stage, we leverage generated mixup training samples as inputs within our two-stage self-training framework and have developed corresponding loss functions for both the source and target domains to direct the training process. Then, the alignment modules for dynamic and static traffic elements are designed to achieve accurate matching between the source and the target domain images. The cosine similarity maximization is applied to the alignment of dynamic traffic elements, while the prototype learning is utilized for the static traffic elements. Additionally, we present a new technique for reducing noise in pseudo labels by constructing thresholds that adjust to each class. Meanwhile, we formulate the associated target domain loss function for vacant pseudo label pixels. The experimental results demonstrate that the proposed method is superior to the existing methods on five different domain adaptation tasks, which is more applicable to semantic segmentation of road scenes.

Fault diagnosis of reducers based on digital twins and deep learning

A new method was proposed to address fault diagnosis by applying the digital twin (DT) high-fidelity behavior and the deep learning (DL) data mining capabilities. Subsequently, the proposed fault distribution GAN (FDGAN) was built to map virtual and physical entities for the data from the established test platform. Finally, the MobileViG was employed to validate the model and diagnose faults. The accuracy of the proposed method with training samples of 600 and 800 were 88.4% and 99.5%, respectively. These accuracies surpass those of other methods based on CycleGAN (98.86%), CACGAN (94.92%), ACGAN (86.45%), ML1D-GAN (82.33%), and transfer learning (99.38%). Therefore, with the integration of global connectivity, an innovative network structure, and training methods, FDGAN can effectively address challenges such as network degradation, limited feature extraction in small windows, and insufficient model robustness.

Camera-Based Respiratory Imaging System for Monitoring Infant Thoracoabdominal Patterns of Respiration.

Existing respiratory monitoring techniques primarily focus on respiratory rate measurement, neglecting the potential of using thoracoabdominal patterns of respiration for infant lung health assessment. To bridge this gap, we exploit the unique advantage of spatial redundancy of a camera sensor to analyze the infant thoracoabdominal respiratory motion. Specifically, we propose a camera-based respiratory imaging (CRI) system that utilizes optical flow to construct a spatio-temporal respiratory imager for comparing the infant chest and abdominal respiratory motion, and employs deep learning algorithms to identify infant abdominal, thoracoabdominal synchronous, and thoracoabdominal asynchronous patterns of respiration. To alleviate the challenges posed by limited clinical training data and subject variability, we introduce a novel multiple-expert contrastive learning (MECL) strategy to CRI. It enriches training samples by reversing and pairing different-class data, and promotes the representation consistency of same-class data through multi-expert collaborative optimization. Clinical validation involving 44 infants shows that MECL achieves 70% in sensitivity and 80.21% in specificity, which validates the feasibility of CRI for respiratory pattern recognition. This work investigates a novel video-based approach for assessing the infant thoracoabdominal patterns of respiration, revealing a new value stream of video health monitoring in neonatal care.

Research on the prediction method for fluvial-phase sandbody connectivity based on big data analysis--a case study of Bohai a oilfield

The connectivity of sandbodies is a key constraint to the exploration effectiveness of Bohai A Oilfield. Conventional connectivity studies often use methods such as seismic attribute fusion, while the development of contiguous composite sandbodies in this area makes it challenging to characterize connectivity changes with conventional seismic attributes. Aiming at the above problem in the Bohai A Oilfield, this study proposes a big data analysis method based on the Deep Forest algorithm to predict the sandbody connectivity. Firstly, by compiling the abundant exploration and development sandbodies data in the study area, typical sandbodies with reliable connectivity were selected. Then, sensitive seismic attribute were extracted to obtain training samples. Finally, based on the Deep Forest algorithm, mapping model between attribute combinations and sandbody connectivity was established through machine learning. This method achieves the first quantitative determination of the connectivity for continuous composite sandbodies in the Bohai Oilfield. Compared with conventional connectivity discrimination methods such as high-resolution processing and seismic attribute analysis, this method can combine the sandbody characteristics of the study area in the process of machine learning, and jointly judge connectivity by combining multiple seismic attributes. The study results show that this method has high accuracy and timeliness in predicting connectivity for continuous composite sandbodies. Applied to the Bohai A Oilfield, it successfully identified multiple sandbody connectivity relationships and provided strong support for the subsequent exploration potential assessment and well placement optimization. This method also provides a new idea and method for studying sandbody connectivity under similar complex geological conditions.

DeepLPos: A Comprehensive Hybrid Deep Learning Model for Lying Position Recognition Using a Tactile Sensor Array System

Abstract Unpredictable limb movements or turning motions can significantly disrupt the extraction of respiratory and heart rate signals. Lying position detection plays a vital role in clinical settings, enabling continuous monitoring of patients' sleep conditions and respiratory rates. In this paper, a smart sleeping position recognition system is proposed, which employs a tactile pressure sensor array based on the unique structure of "the double-layer capacitors (EDLC)". The sensor array, comprising 64 rows and 32 columns (2048 nodes), captures four types of healthy lying positions using an 8-bit AD module. Despite challenges arising from limited experimental samples for accurate training, we propose DeepLPos, a hybrid deep learning approach combining Generative Adversarial Networks (GAN) and the YOLO network. To tackle the differentiation challenge between supine and prone positions, we introduce an SPD Conv attention module to enhance the resolution of detailed descriptions in pressure images. The designed model is then pruned to simplify its structure and parameters, ensuring fast and accurate lying position detection. Furthermore, we evaluate the performance of our proposed method on the SLP dataset and demonstrate an impressive accuracy rate of 97.5%. Real-time processing speed reaches 0.069s/frame, demonstrating the efficiency and effectiveness of the developed approach in practical applications.

Research on the prediction of blasting fragmentation in open-pit coal mines based on KPCA-BAS-BP

The blasting block size of open-pit mines is influenced by many factors, and the influencing factors have a very complex nonlinear relationship. Traditional empirical formulas and a single neural network model cannot meet the requirements of modern blasting safety. To improve the prediction accuracy of blasting block size, the measured data of Beskuduk open-pit coal mine is used as training and testing samples. Seven factors including rock tensile strength, rock compressive strength, and blast hole spacing are selected as input variables of the prediction model. The average size of blasting fragmentation X50 is used as the output variable of the prediction model. The kernel principal component analysis (KPCA) is adopted to reduce the dimensionality of the input variables. The beetle antennae search algorithm (BAS) is selected to optimize the parameters of the initial weights and thresholds of the back propagation (BP) neural network. Finally, prediction model of blasting fragmentation in open-pit coal mine based on KPCA-BAS-BP is established. The results show that the average relative error of the model is 1.77%, and the root mean square error is 1.52%. Compared with the unoptimized BP neural network and the BP neural network optimized by the artificial bee colony algorithm (ABC) model, this model has higher prediction accuracy and is more suitable for predicting the blasting block size of open-pit coal mines, it provides a new method for predicting the fragmentation of blasting under the influence of multiple factors, filling the gap in related theoretical research, and has certain practical application value.