Size Of Training Data Research Articles

Using gait videos to automatically assess anxiety.

In recent years, the number of people with anxiety disorders has increased worldwide. Methods for identifying anxiety through objective clues are not yet mature, and the reliability and validity of existing modeling methods have not been tested. The objective of this paper is to propose an automatic anxiety assessment model with good reliability and validity. This study collected 2D gait videos and Generalized Anxiety Disorder (GAD-7) scale data from 150 participants. We extracted static and dynamic time-domain features and frequency-domain features from the gait videos and used various machine learning approaches to build anxiety assessment models. We evaluated the reliability and validity of the models by comparing the influence of factors such as the frequency-domain feature construction method, training data size, time-frequency features, gender, and odd and even frame data on the model. The results show that the number of wavelet decomposition layers has a significant impact on the frequency-domain feature modeling, while the size of the gait training data has little impact on the modeling effect. In this study, the time-frequency features contributed to the modeling, with the dynamic features contributing more than the static features. Our model predicts anxiety significantly better in women than in men (r Male = 0.666, r Female = 0.763, p < 0.001). The best correlation coefficient between the model prediction scores and scale scores for all participants is 0.725 (p < 0.001). The correlation coefficient between the model prediction scores for odd and even frame data is 0.801~0.883 (p < 0.001). This study shows that anxiety assessment based on 2D gait video modeling is reliable and effective. Moreover, we provide a basis for the development of a real-time, convenient and non-invasive automatic anxiety assessment method.

Harnessing heterogeneity in space with statistically guided meta-learning.

Spatial data are ubiquitous, massively collected, and widely used to support critical decision-making in many societal domains, including public health (e.g., COVID-19 pandemic control), agricultural crop monitoring, transportation, etc. While recent advances in machine learning and deep learning offer new promising ways to mine such rich datasets (e.g., satellite imagery, COVID statistics), spatial heterogeneity-an intrinsic characteristic embedded in spatial data-poses a major challenge as data distributions or generative processes often vary across space at different scales, with their spatial extents unknown. Recent studies (e.g., SVANN, spatial ensemble) targeting this difficult problem either require a known space-partitioning as the input, or can only support very limited number of partitions or classes (e.g., two) due to the decrease in training data size and the complexity of analysis. To address these limitations, we propose a model-agnostic framework to automatically transform a deep learning model into a spatial-heterogeneity-aware architecture, where the learning of arbitrary space partitionings is guided by a learning-engaged generalization of multivariate scan statistic and parameters are shared based on spatial relationships. Moreover, we propose a spatial moderator to generalize learned space partitionings to new test regions. Finally, we extend the framework by integrating meta-learning-based training strategies into both spatial transformation and moderation to enhance knowledge sharing and adaptation among different processes. Experiment results on real-world datasets show that the framework can effectively capture flexibly shaped heterogeneous footprints and substantially improve prediction performances.

FGC: GCN-Based Federated Learning Approach for Trust Industrial Service Recommendation

With the development of the Industrial Internet of Things system, the huge amount of devices, services, and continuous data, making it difficult to discover a trusted service in complex scenarios. To better leverage knowledge and historical behavior, recommendation systems are applied. However, the model accuracy closely depends on training data size; there is a great risk of data leaking by collecting from multiple departments. To solve these problems, we propose a graph-convolutional-neural-network-based federated approach, which accurately recommends proper service for participating clients without gathering the raw data. Specifically, each client trains locally and uploads the weights of their model to the server for aggregation. Besides, the potential overlapping services of different clients are leveraged to guide the embedding aggregation and sharing, which, in turn, optimize the local training results. Their sensitive scenarios' embedding is kept locally. Owing to the model aggregation, it also resists the poisoning attack to some degree. In addition, the comprehensive experiments on classic public recommendation datasets evaluate the feasibility, effectiveness, trustworthiness, and potential influences.

Procedural Defect Modeling for Virtual Surface Inspection Environments.

Development of automated visual surface inspection systems heavily depends on the availability of defected product samples. Both inspection hardware configuration and training of defect detection models require diversified, representative and precisely annotated data. Reliable training data of sufficient size is frequently challenging to obtain. Using virtual environments, it is possible to simulate defected products which would serve both for configuration of acquisition hardware as well as for generation of required datasets. In this work, we present parameterized models for adaptable simulation of geometrical defects, based on procedural methods. Presented models are suitable for creating defected products in virtual surface inspection planning environments. As such, they enable inspection planning experts to assess defect visibility for various configurations of acquisition hardware. Finally, the presented method enables pixel-precise annotations alongside image synthesis for the creation of training-ready datasets.

Implementation of Mel-Frequency Cepstral Coefficient as Feature Extraction using K-Nearest Neighbor for Emotion Detection Based on Voice Intonation

Purpose: To determine emotions based on voice intonation by implementing MFCC as a feature extraction method and KNN as an emotion detection method.Design/methodology/approach: In this study, the data used was downloaded from several video podcasts on YouTube. Some of the methods used in this study are pitch shifting for data augmentation, MFCC for feature extraction on audio data, basic statistics for taking the mean, median, min, max, standard deviation for each coefficient, Min max scaler for the normalization process and KNN for the method classification.Findings/result: Because testing is carried out separately for each gender, there are two classification models. In the male model, the highest accuracy was obtained at 88.8% and is included in the good fit model. In the female model, the highest accuracy was obtained at 92.5%, but the model was unable to correctly classify emotions in the new data. This condition is called overfitting. After testing, the cause of this condition was because the pitch shifting augmentation process of one tone in women was unable to solve the problem of the training data size being too small and not containing enough data samples to accurately represent all possible input data values.Originality/value/state of the art: The research data used in this study has never been used in previous studies because the research data is obtained by downloading from Youtube and then processed until the data is ready to be used for research.

Online transportation network cyber-attack detection based on stationary sensor data

Penetration of connected vehicles and crowdsourced mapping applications give rise to security vulnerabilities in transportation networks. Accurate detection of cyber-attacks on transportation networks is critical to minimize impacts on transportation systems. This task is particularly challenging because the impacts of regional cyber-attacks can be invisible on aggregated traffic data, especially when only sensor data is accessible to transportation agencies. We propose an analytical framework that leverages real-time road link sensory data to conduct online data-driven transportation network anomaly detection using non-parametric long short-term memory (LSTM) and parametric Gaussian process model. The online anomaly detection models can continuously update model coefficients as real-time sensory data arrives. We utilize a city-scale microscopic traffic simulation to validate our cyber-attack detecting framework. The cyber-attack detection model achieves a F1 score, which is a harmonic mean of the precision and recall of classifiers, between 84% to 96% considering different initial training data sizes. We compare with major offline models to demonstrate the effectiveness and robustness of online models. In addition, we devised a meta-heuristic method to solve the multi-objective sensor location problem to simultaneously enhance anomaly detection efficiency and maximize traffic information gain. This study demonstrates a systematic approach to address the emerging concerns of cyber-security in transportation networks with minimum requirements for infrastructure upgrades. Our results can help transportation security authorities identify potential cyber-attacks and protect transportation infrastructure from malicious cyber-hackers.

Retention time prediction for small samples based on integrating molecular representations and adaptive network

Retention time (RT) can provide orthogonal information different from that of mass spectrometry and contribute to identifying compounds. Many machine learning methods have been developed and applied to RT prediction. In application, the training data size is usually small in most chromatography systems. To enhance the performance of RT prediction, this study proposes a RT prediction method based on multi-data combinations and adaptive neural network (MDC-ANN). MDC-ANN establishes the RT prediction model for the target chromatographic system through transfer learning and a base deep learning model trained on a big dataset. It selects the optimal molecular representation combination from the multiple input candidates and automatically determines the neural network structure according to the determined input combination. MDC-ANN was compared with two new efficient deep learning methods, three transferring methods and four popular machine learning methods on 14 small datasets and showed advantages in MAE, MedAE, MRE and R2 in most cases. The experiment results illustrated that integrating multiple molecular representations can provide more information, improve the performance of RT prediction and contribute to compound annotation, different chromatographic systems may use different molecular representation combinations to obtain good RT prediction performance. Hence, MDC-ANN which automatically determines the best combination of molecular representations for a specific system is promising for predicting RTs accurately in real applications.

Learning machining stability using a bayesian model

Instabilities in machining can be detrimental. Usually, analytical model-predicted stability charts guide selection of cutting parameters to ensure stable processes. However, since inputs to the model seldom account for the speed-dependent behaviour of the cutting process or the dynamics, models often fail to guide stable cutting parameter selection in real industrial settings. To address this issue, this paper discusses how real experimentally classified stable and unstable cutting data with all its vagaries and uncertainties can instead be used to learn the stability behaviour using a supervised Bayes' learning approach. We expand previously published work to systematically characterize how probability distributions, training data size, and thresholding influence the learning capacity of the Bayesian approach. Prediction accuracies of up to 95% are shown to be possible. We also show how the approach nicely extends itself to a continuous learning process. Results can hence inform further development towards self-optimizing and autonomous machining systems.

Physics-Informed LSTM Network for Flexibility Identification in Evaporative Cooling System

In energy-intensive industrial systems, an evaporative cooling process may introduce operational flexibility. Such flexibility refers to a system’s ability to deviate from its scheduled energy consumption. Identifying the flexibility, and therefore, designing control that ensures efficient and reliable operation presents a great challenge due to the inherently complex dynamics of industrial systems. Recently, machine learning (ML) models have attracted attention for identifying flexibility, due to their ability to model complex nonlinear behavior. This article presents ML-based methods that integrate system dynamics into the ML models (e.g., neural networks) for better adherence to physical constraints. We define and evaluate physics-informed long-short term memory networks (PhyLSTMs) and physics-informed neural networks (PhyNN) for the identification of flexibility in the evaporative cooling process. These physics-informed networks approximate the time-dependent relationship between control input and system response while enforcing the dynamics of the process in the neural network architecture. Our proposed PhyLSTM provides less than 2% system response estimation error, converges in less than half iterations compared to a baseline NN, and accurately estimates the defined flexibility metrics. We include a detailed analysis of the impact of training data size on the performance and optimization of our proposed models.

Generalizable transfer learning of automated tumor segmentation from cervical cancers toward a universal model for uterine malignancies in diffusion-weighted MRI

PurposeTo investigate the generalizability of transfer learning (TL) of automated tumor segmentation from cervical cancers toward a universal model for cervical and uterine malignancies in diffusion-weighted magnetic resonance imaging (DWI).MethodsIn this retrospective multicenter study, we analyzed pelvic DWI data from 169 and 320 patients with cervical and uterine malignancies and divided them into the training (144 and 256) and testing (25 and 64) datasets, respectively. A pretrained model was established using DeepLab V3 + from the cervical cancer dataset, followed by TL experiments adjusting the training data sizes and fine-tuning layers. The model performance was evaluated using the dice similarity coefficient (DSC).ResultsIn predicting tumor segmentation for all cervical and uterine malignancies, TL models improved the DSCs from the pretrained cervical model (DSC 0.43) when adding 5, 13, 26, and 51 uterine cases for training (DSC improved from 0.57, 0.62, 0.68, 0.70, p < 0.001). Following the crossover at adding 128 cases (DSC 0.71), the model trained by combining data from adding all the 256 patients exhibited the highest DSCs for the combined cervical and uterine datasets (DSC 0.81) and cervical only dataset (DSC 0.91).ConclusionsTL may improve the generalizability of automated tumor segmentation of DWI from a specific cancer type toward multiple types of uterine malignancies especially in limited case numbers.

A locally unsupervised hybrid learning method for removing environmental effects under different measurement periods

Environmental effects induce deceptive variability in unlabeled vibration data for structural health monitoring (SHM). Although unsupervised learning is an effective solution to this issue, some new challenges such as the size of training data in different measurement periods and the type of learning between local and global frameworks seriously affect overall performance of this technique. To tackle these issues, we propose a locally unsupervised hybrid learning method based on an innovative discriminative reconstruction-based dictionary learning (DRDL) algorithm. The proposed method initially uses a Gaussian mixture model to provide local information for the DRDL algorithm by clustering entire training data into local subsets. Subsequently, this algorithm computes sub-dictionaries and sparse coefficients to reconstruct local training subsets. Using these subsets, an anomaly detector developed from the Mahalanobis-squared distance is used to determine damage indices for SHM. Real data from two bridges are incorporated to verify the proposed method with some comparisons.

Robust Multi-Channel Far-Field Speaker Verification Under Different In-Domain Data Availability Scenarios

The popularity and application of smart home devices have made far-field speaker verification an urgent need. However, speaker verification performance is unsatisfactory under far-field environments despite its significant improvements enabled by deep neural networks (DNN). In this paper, we summarize our previous work and propose multiple training strategies and models for multi-channel far-field speaker verification with different in-domain data availability scenarios. The experiments are conducted on the FFSVC20 dataset, and we proposed the cross-device and cross-domain trials. We focus on single-channel and multi-channel speaker verification training based on the dataset. For single-channel speaker verification, considering the size of training data and availability of labels, we introduce three training scenarios and given our proposed training methods, including 1) given zero out-of-domain data and few in-domain labeled data; 2) given large-scale out-of-domain labeled data and few in-domain labeled data; 3) given large-scale out-of-domain labeled data and few in-domain unlabeled data. To this end, we propose a meta-learning approach, refined transfer learning methods, and semi-supervised learning for three scenarios, respectively. For multi-channel speaker verification, we first introduce two types of 3 dimension convolution (3D Conv) residual network (ResNet) models proposed in our previous works, including fully 3D ResNet and incorporating 3D Conv with 2D Conv ResNet (3D2D-ResNet). In this paper, we propose channel-wise 3D squeeze-and-excitation ResNet (C3DSE-ResNet) and spatial-wise 3D SE ResNet (S3DSE-ResNet) to further explore the channel dependencies and improve the 3D ConvNet performance. The results show that the proposed strategies and models can significantly boost performance under the far-field scenario.

Predicting band gaps of MOFs on small data by deep transfer learning with data augmentation strategies.

Porphyrin-based MOFs combine the unique photophysical and electrochemical properties of metalloporphyrins with the catalytic efficiency of MOF materials, making them an important candidate for light energy harvesting and conversion. However, accurate prediction of the band gap of porphyrin-based MOFs is hampered by their complex structure-function relationships. Although machine learning (ML) has performed well in predicting the properties of MOFs with large training datasets, such ML applications become challenging when the training data size of the materials is small. In this study, we first constructed a dataset of 202 porphyrin-based MOFs using DFT computations and increased the training data size using two data augmentation strategies. After that, four state-of-the-art neural network models were pre-trained with the recognized open-source database QMOF and fine-tuned with our augmented self-curated datasets. The GCN models predicted the band gaps of the porphyrin-based materials with the lowest RMSE of 0.2767 eV and MAE of 0.1463 eV. In addition, the data augmentation strategy rotation and mirroring effectively decreased the RMSE by 38.51% and MAE by 50.05%. This study demonstrates that, when proper transfer learning and data augmentation strategies are applied, machine learning models can predict the properties of MOFs using small training data.

Computer vision for automatic defect detection of self-pierce rivet joints

Rising costs for raw materials and energy require the development of lightweight construction concepts that are suitable for the load. Especially in the transportation sector, lightweight construction of structural components leads to improvements in resource- and energy-efficiency. The joining process of self-pierce riveting with semi-hollow pierce rivet is often used to join these lightweight materials. To ensure the product properties, the quality of each joint must be guaranteed. Currently, checking the joint quality is done manually through cyclical visual inspections and destructive tests, which costs considerable resources and time. Hence, a timely defect detection during the production process, based on computer vision, can save costs and enables better quality documentation of the rivet joints.In this paper a computer vision approach, based on the object detection algorithm YOLOv5, for automated defect detection of self-pierce riveting joints is presented. To this end, current methods for quality control of rivet joints in automotive production are discussed, as well as existing research on computer vision for defect detection on self-pierce rivet joints. Images from rivet joints on two different sheet metal pairings have been gathered and labelled, containing five different defect classes that can be found in automotive production. The images comprise a 90 ° and 45 ° viewing angle of the rivet joints to compare the algorithm's performance for these angles. To estimate the influence of training data samples on the defect detection performance the algorithm was trained and tested on different training data sizes. Furthermore, the transferability of the learned model from an aluminium sheet metal to an aluminium die cast pairing is assessed. It has been found that computer vision delivers sound results at the task of detecting defects in rivet joints, even with small training data size and hence has the potential to improve quality control in automotive riveting processes.

Face recognition under large age gap using age face generation

Age invariant face recognition (AIFR) is a challenging problem in the area of the face recognition. To handle large age gap for face recognition, we proposed a robust approach based on deep learning for face recognition under a large age gap. The presented approach consists of four important steps. The pre-processing is done for face detection. Age face generation is processed with the help of modified age conditional generative adversarial network (acGAN). Generated age face images are mixed with train dataset and augmentation is applied to increase the size of training data for handling biasness of the deep learning models towards dataset size. A modified residual convolutional neural network is applied for training and testing of face images. The performance has been evaluated using two-fold cross-validation on standard and challenging LAG dataset. The proposed approach achieved the 92.5% recognition accuracy, which is better than the existing face recognition approaches for a large age gap.

A Theoretical Analysis of Using Gradient Data for Sobolev Training in RKHS

Recent works empirically demonstrated that incorporating target derivatives, in addition to the conventional usage of target values, during the training process improves the accuracy of the predictor and data efficiency. Despite the successful application of gradient data in the learning process, very little is understood theoretically about their performance guarantee. In this paper, our goal is to highlight (i) the limitations of gradient data on their performance guarantees, especially in low-data regimes, and (ii) the extent to which the gradients affect the learning rate. Our result implies that in a low-data regime, if the Lipschitz of the target function is below a threshold, gradient data for Sobolev training outperforms the classical training in terms of sample efficiency. For a target function with a large Lipschitz constant, there is a threshold for training data size beyond which the gradient data perform better than conventional training. The convergence behavior of gradient data for Sobolev training is studied, and the learning rate of order O(n−1/2+ϵ) is derived. Experiments are conducted to determine the effect of gradient data in the learning process.

HRNeXt: High-Resolution Context Network for Crowd Pose Estimation

Occlusion handling in crowded scenes is an intractable challenge for human pose estimation. To address this problem, we propose two novel feed-forward network structures named Global Feed-Forward Network (GFFN) and Dynamic Feed-Forward Network (DFFN), which are specifically designed for image-based tasks to capture both local and global contextual information within intermediate features and update feature representations with high adaptability for occlusions. By exploiting the context modeling ability of the proposed GFFN and DFFN, we present a novel backbone network, namely High-Resolution Context Network (HRNeXt), which learns high-resolution representations with abundant contextual information to better estimate poses of occluded human bodies. Compared to state-of-the-art pose estimation networks, our HRNeXt absorbs advantages of convolution operation and attention mechanism, and it is more efficient in terms of training data sizes, network parameters and computational costs. Experimental results show that our HRNeXt significantly outperforms state-of-the-art backbone networks on challenging pose estimation datasets with high occurrence of crowds and occlusions. Code is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ZiyiZhang27/HRNeXt</uri> .

Mordo: Silent Command Recognition Through Lightweight Around-Ear Biosensors

The prevalence of smart devices encourages increasing requirements of wearable human–computer interactions. To improve user acceptance, such interactions require easy-to-manipulate and unobtrusive characteristics. In this article, we, for the first time, propose to recognize silent commands through a lightweight and around-ear biosensing system Mordo that can be easily integrated with earphones, manipulate smart devices, and minimize social awkwardness. In particular, we first determine the empirical principles of constructing commands and experimentally screen the commands based on the around-ear configuration. Second, we select the optimal around-ear sensor configuration according to the single-channel signal-to-noise ratios (SNRs) and classification accuracies. Third, we propose a multistream CNN-LSTM network to learn the spatiotemporal mapping between the around-ear signals and commands. Finally, extensive experiments have been conducted to evaluate the feasibility and stability. The results indicate an averaged accuracy of 89.66% that outperforms other algorithms of similar tasks. The stability tests show that our system presents sufficient stability under command deformations and head motions. We demonstrate the necessity of collecting such scale of data by gradually reducing training data size. We also validate the generalization ability of our method toward other sensing parameters by reducing the spatial and temporal resolutions. The proof-of-concept design will aim the further development of the commercial products for silent command recognition.

Landslide Susceptibility Modeling Using a Deep Random Neural Network

Developing landslide susceptibility modeling is essential for detecting landslide-prone areas. Recently, deep learning theories and methods have been investigated in landslide modeling. However, their generalization is hindered because of the limited size of landslide data. In the present study, a novel deep learning-based landslide susceptibility assessment method named deep random neural network (DRNN) is proposed. In DRNN, a random mechanism is constructed to drop network layers and nodes randomly during landslide modeling. We take the Lushui area (Southwest China) as the case and select 12 landslide conditioning factors to perform landslide modeling. The performance evaluation results show that our method achieves desirable generalization performance (Kappa = 0.829) and outperforms other network models such as the convolution neural network (Kappa = 0.767), deep feedforward neural network (Kappa = 0.731), and Adaboost-based artificial neural network (Kappa = 0.732). Moreover, the robustness test shows the advantage of our DRNN, which is insensitive to variations in training data size. Our method yields an accuracy higher than 85% when the training data size stands at only 10%. The results demonstrate the effectiveness of the proposed landslide modeling method in enhancing generalization. The proposed DRNN produces accurate results in terms of delineating landslide-prone areas and shows promising applications.

ESG Text Classification: An Application of the Prompt-Based Learning Approach

Over the past decade, there is a surging trend to integrate environmental, social, and governance (ESG) criteria into financial decision making. ESG information extracted manually from text sources, such as company statements, press releases, and regulatory disclosures, can be expensive and inconsistent due to human interpretation. In this article, the authors introduce the application of prompt-based learning, a cutting-edge natural language processing (NLP) technology, to classify textual data into ESG and non-ESG categories. In particular, the authors establish a prompt-based ESG classifier, using data from Refinitiv, and benchmark it against a traditional pre-train and fine-tune classifier through statistical test. The authors fine-tune the classifiers on various sizes of training data. The experiment shows that the prompt-based learning approach outperforms the traditional pre-train and fine-tune classifier and can generate promising results when training data are limited.