Distribution Of Training Data Research Articles

Data-assisted training of a physics-informed neural network to predict the separated Reynolds-averaged turbulent flow field around an airfoil under variable angles of attack

AbstractPhysics-informed neural networks are a promising method to yield surrogate models of flow fields. We present a metamodeling technique for variable geometries based on physics-informed neural networks. The method was applied to the DU99W350 airfoil at a Reynolds number of $$1\times 10^{5}$$ 1 × 10 5 . Using our technique, the angle of attack was introduced as an additional input parameter of the network and the model was trained to predict the Reynolds-averaged velocity and pressure fields around the airfoil for arbitrary angles of attack between 10.0 and 17.5$$^{\circ }$$ ∘ . Furthermore, we present an effective method to generate the training points for the parameterized geometry. The model was trained with data from simulations for a limited set of angles of attack. Additionally, satisfaction of the a priori known boundary conditions as well as the Reynolds-averaged Navier–Stokes equations was attained. A sensitivity analysis concerning the Reynolds number, the amount and distribution of training data, and the turbulence model was conducted showing the superiority of the pseudo-Reynolds stress method and the demand for labeled training data in the domain. The trained network was capable of predicting the flow separation progressing with angle of attack on the suction surface and exhibited excellent agreement with numerically simulated results, even in the proximity of the wall for interpolations as well as extrapolations from the labeled data set. Our study demonstrates that physics-informed neural networks can be used to obtain accurate flow field surrogate models of variable geometries.

Preprocessing and Modeling Approach for Gearbox Pitting Severity Prediction under Unseen Operating Conditions and Fault Severities

Gear pitting is a common gearbox failure mode that can lead to unplanned machine downtime, inefficient power transmission and a higher risk of sudden catastrophic failure. Consequently, there is strong incentive to create machine learning models that are capable of detecting and quantifying the severity of gearbox pitting faults. The performance of machine learning models is however highly dependent on the availability of training data and since training data for a wide variety of different operating conditions and fault severities is rarely available in practice, machine learning models must be designed to be robust to unseen operating conditions and fault severities. Furthermore, models should be capable of identifying data outside of the training data distribution and adjusting the confidence in a prediction accordingly. This work presents a strategy for pitting severity estimation in gearboxes under unseen operating conditions and fault severities in response to the PHM North America 2023 Conference Data Challenge. The strategy includes the design of dedicated validation sets for quantifying model performance on unseen data, an investigation into the most appropriate preprocessing methods, and a specialized convolutional neural network with an integrated out of distribution detection model for identifying samples from foreign operating conditions and fault severities. The results show that the best models are capable of some generalization to unseen operating conditions, but the generalization to unseen pitting severities is more challenging.

A Normalization Strategy for Weakly Supervised 3D Hand Pose Estimation

The effectiveness of deep neural network models is intricately tied to the distribution of training data. However, in pose estimation, potential discrepancies in root joint positions and inherent variability in biomechanical features across datasets are often overlooked in current training strategies. To address these challenges, a novel Hand Pose Biomechanical Model (HPBM) is developed. In contrast to the traditional 3D coordinate-encoded pose, it provides a more intuitive depiction of the anatomical characteristics of the hand. Through this model, a data normalization approach is implemented to align the root joint and unify the biomechanical features of training samples. Furthermore, the HPBM facilitates a weakly supervised strategy for dataset expansion, significantly enhancing the data diversity. The proposed normalized method is evaluated on two widely used 3D hand pose estimation datasets, RHD and STB, demonstrating superior performance compared to the models trained without normalized datasets. Utilizing ground truth 2D keypoints as input, a reduction of 45.1% and 43.4% in error is achieved on the STB and RHD datasets, respectively. When leveraging 2D keypoints from MediaPipe, a reduction in error by 11.3% and 14.3% is observed on the STB and RHD datasets.

DF-QSM: Data Fidelity based Hybrid Approach for Improved Quantitative Susceptibility Mapping of the Brain.

Quantitative Susceptibility Mapping (QSM) is an advanced magnetic resonance imaging (MRI) technique to quantify the magnetic susceptibility of the tissue under investigation. Deep learning methods have shown promising results in deconvolving the susceptibility distribution from the measured local field obtained from the MR phase. Although existing deep learning based QSM methods can produce high-quality reconstruction, they are highly biased toward training data distribution with less scope for generalizability. This work proposes a hybrid two-step reconstruction approach to improve deep learning based QSM reconstruction. The susceptibility map prediction obtained from the deep learning methods has been refined in the framework developed in this work to ensure consistency with the measured local field. The developed method was validated on existing deep learning and model-based deep learning methods for susceptibility mapping of the brain. The developed method resulted in improved reconstruction for MRI volumes obtained with different acquisition settings, including deep learning models trained on constrained (limited) data settings.

Assessment of training data on visual acuity of evaluators in plant diseases, carried out through the web tool SiTAV

The present study analyzed the data compiled after the training of the precision and visual accuracy of the evaluators of both genders in plant diseases, using the SiTAV (Visual Acuity Training System) website. The training data were analyzed through the exploratory analysis, with the aid of the statistical program SAS 9.4. With 52,495 training data being stored in the system's MySQL database identified. The tabulation of training data ranked the evaluations in three classes of accuracy of major hits (underestimated, precise and overestimated), where 30.36% (15.938) training were classified as accurate. The distribution of training data between accuracy classes and five hit categories (excellent, optimal, good, fair and poor) indicated that the level of excellent severity obtained 30.04% (15,772) of training data of the six levels of severity, level 1 (0 to 3% of real severity) reached 33.31% (17,486) of training data. It is concluded that the smaller the lesion size of plant diseases, the greater the accuracy of the evaluator.

Domain Adaptation and Generalization of Functional Medical Data: A Systematic Survey of Brain Data

In spite of the excellent capabilities of machine learning algorithms, their performance deteriorates when the distribution of test data differs from the distribution of training data. In medical data research, this problem is exacerbated by its connection to human health, expensive equipment, and meticulous setups. Consequently, achieving domain generalizations (DG) and domain adaptations (DA) under distribution shifts is an essential step in the analysis of medical data. As the first systematic review of DG and DA on functional brain signals, the paper discusses and categorizes various methods, tasks, and datasets in this field. Moreover, it discusses relevant directions for future research.

DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning

Neural predictors have shown great potential in the evaluation process of neural architecture search (NAS). However, current predictor-based approaches overlook the fact that training a predictor necessitates a considerable number of trained neural networks as the labeled training set, which is costly to obtain. Therefore, the critical issue in utilizing predictors for NAS is to train a high-performance predictor using as few trained neural networks as possible. Although some methods attempt to address this problem through unsupervised learning, they often result in inaccurate predictions. We argue that the unsupervised tasks intended for the common graph data are too challenging for neural networks, causing unsupervised training to be susceptible to performance crashes in NAS. To address this issue, we propose a CurricuLum-guided Contrastive Learning framework for neural Predictor (DCLP). Our method simplifies the contrastive task by designing a novel curriculum to enhance the stability of unlabeled training data distribution during contrastive training. Specifically, we propose a scheduler that ranks the training data according to the contrastive difficulty of each data and then inputs them to the contrastive learner in order. This approach concentrates the training data distribution and makes contrastive training more efficient. By using our method, the contrastive learner incrementally learns feature representations via unsupervised data on a smooth learning curve, avoiding performance crashes that may occur with excessively variable training data distributions. We experimentally demonstrate that DCLP has high accuracy and efficiency compared with existing predictors, and shows promising potential to discover superior architectures in various search spaces when combined with search strategies. Our code is available at: https://github.com/Zhengsh123/DCLP.

Navigating Open Set Scenarios for Skeleton-Based Action Recognition

In real-world scenarios, human actions often fall outside the distribution of training data, making it crucial for models to recognize known actions and reject unknown ones. However, using pure skeleton data in such open-set conditions poses challenges due to the lack of visual background cues and the distinct sparse structure of body pose sequences. In this paper, we tackle the unexplored Open-Set Skeleton-based Action Recognition (OS-SAR) task and formalize the benchmark on three skeleton-based datasets. We assess the performance of seven established open-set approaches on our task and identify their limits and critical generalization issues when dealing with skeleton information.To address these challenges, we propose a distance-based cross-modality ensemble method that leverages the cross-modal alignment of skeleton joints, bones, and velocities to achieve superior open-set recognition performance. We refer to the key idea as CrossMax - an approach that utilizes a novel cross-modality mean max discrepancy suppression mechanism to align latent spaces during training and a cross-modality distance-based logits refinement method during testing. CrossMax outperforms existing approaches and consistently yields state-of-the-art results across all datasets and backbones. We will release the benchmark, code, and models to the community.

CASE: Exploiting Intra-class Compactness and Inter-class Separability of Feature Embeddings for Out-of-Distribution Detection

Detecting out-of-distribution (OOD) inputs is critical for reliable machine learning, but deep neural networks often make overconfident predictions, even for OOD inputs that deviate from the distribution of training data. Prior methods relied on the widely used softmax cross-entropy (CE) loss that is adequate for classifying in-distribution (ID) samples but not optimally designed for OOD detection. To address this issue, we propose CASE, a simple and effective OOD detection method by explicitly improving intra-class Compactness And inter-class Separability of feature Embeddings. To enhance the separation between ID and OOD samples, CASE uses a dual-loss framework, which includes a separability loss that maximizes the inter-class Euclidean distance to promote separability among different class centers, along with a compactness loss that minimizes the intra-class Euclidean distance to encourage samples to be close to their class centers. In particular, the class centers are defined as a free optimization parameter of the model and updated by gradient descent, which is simple and further enhances the OOD detection performance. Extensive experiments demonstrate the superiority of CASE, which reduces the average FPR95 by 37.11% and improves the average AUROC by 15.89% compared to the baseline method using a softmax confidence score on the more challenging CIFAR-100 model.

CMDA: Cross-Modal and Domain Adversarial Adaptation for LiDAR-Based 3D Object Detection

Recent LiDAR-based 3D Object Detection (3DOD) methods show promising results, but they often do not generalize well to target domains outside the source (or training) data distribution. To reduce such domain gaps and thus to make 3DOD models more generalizable, we introduce a novel unsupervised domain adaptation (UDA) method, called CMDA, which (i) leverages visual semantic cues from an image modality (i.e., camera images) as an effective semantic bridge to close the domain gap in the cross-modal Bird's Eye View (BEV) representations. Further, (ii) we also introduce a self-training-based learning strategy, wherein a model is adversarially trained to generate domain-invariant features, which disrupt the discrimination of whether a feature instance comes from a source or an unseen target domain. Overall, our CMDA framework guides the 3DOD model to generate highly informative and domain-adaptive features for novel data distributions. In our extensive experiments with large-scale benchmarks, such as nuScenes, Waymo, and KITTI, those mentioned above provide significant performance gains for UDA tasks, achieving state-of-the-art performance.

Symmetric Self-Paced Learning for Domain Generalization

Deep learning methods often suffer performance degradation due to domain shift, where discrepancies exist between training and testing data distributions. Domain generalization mitigates this problem by leveraging information from multiple source domains to enhance model generalization capabilities for unseen domains. However, existing domain generalization methods typically present examples to the model in a random manner, overlooking the potential benefits of structured data presentation. To bridge this gap, we propose a novel learning strategy, Symmetric Self-Paced Learning (SSPL), for domain generalization. SSPL consists of a Symmetric Self-Paced training scheduler and a Gradient-based Difficulty Measure (GDM). Specifically, the proposed training scheduler initially focuses on easy examples, gradually shifting emphasis to harder examples as training progresses. GDM dynamically evaluates example difficulty through the gradient magnitude with respect to the example itself. Experiments across five popular benchmark datasets demonstrate the effectiveness of the proposed learning strategy.

DART: Dual-Modal Adaptive Online Prompting and Knowledge Retention for Test-Time Adaptation

As an up-and-coming area, CLIP-based pre-trained vision-language models can readily facilitate downstream tasks through the zero-shot or few-shot fine-tuning manners. However, they still face critical challenges in test-time generalization due to the shifts between the training and test data distributions, hindering the further improvement of the performance. To address this crucial problem, the latest works have introduced Test-Time Adaptation (TTA) techniques to CLIP which dynamically learn text prompts using only test samples. However, their limited learning capacity due to the overlook of visual modality information, and the underutilization of knowledge in previously seen test samples result in reduced performance. In this paper, we propose a novel Dual-modal Adaptive online prompting and knowledge ReTention method called DART to overcome these challenges. To increase the learning capacity, DART captures knowledge from each test sample by learning class-specific text prompts and instance-level image prompts. Additionally, to fully leverage the knowledge from previously seen test samples, DART utilizes dual-modal knowledge retention prompts to adaptively retain the acquired knowledge, thereby enhancing the predictions on subsequent test samples. Extensive experiments on various large-scale benchmarks demonstrate the effectiveness of our proposed DART against state-of-the-art methods.

Time-series land cover change detection using deep learning-based temporal semantic segmentation

The process of sustainable urban development is accompanied by frequent and complex land cover changes, and thus, clarify accurate information on land cover changes can provide scientific data for urban management. To characterize urban development at an accurate spatiotemporal scale, a change detection model is not only required to provide accurate location (Where) and time (When) of the changes, but also semantic information on the change types (What). Accordingly, this study proposed a deep learning method for temporal semantic segmentation change detection (TSSCD) that obtains information on the where, when, and what of changes simultaneously. TSSCD model bridges the semantic gap between remote sensing time series abrupt changes and land cover changes by learning the month-to-month mapping from spectral information to land cover types. We implemented a temporal semantic segmentation model based on the most classic fully convolutional network, where all two-dimensional convolutions and pooling operations were replaced with one-dimensional. We conducted tests on the TSSCD in several urban study areas, and it consistently exhibited good accuracy. In most cases, it outperformed the BFAST and CCDC algorithms, except when only a single spectral band was used. Simultaneously, we analyzed the minimum data requirements for training a TSSCD. The TSSCD currently faces challenges in achieving strong generalization beyond the training data distribution. Additionally, we observed that change detection for specific land cover types can be achieved through the flexible configuration of TSSCD. Finally, we explored a method for constructing datasets using existing products to minimize data annotation efforts, yielding promising results. However, there is still some gap compared to complete manual annotation. Overall, the TSSCD model provided a novel solution to accurately characterize sustainable urban development at the spatiotemporal scale.

Rolling bearing fault diagnosis based on the fusion of sparse filtering and discriminative domain adaptation method under multi-channel data-driven

Currently, the diagnostic performance of many deep learning algorithms may drop dramatically when the distribution of training data is significantly different from that of the test data. Moreover, the fault diagnosis approaches based on single-channel data may suffer problems such as large precision fluctuation, low reliability, and incomplete expression of fault features. To overcome the above deficiencies, a novel multi-channel data-driven fault recognition method based on the fusion of sparse filtering (SF) and discriminative domain adaptation (MSFDDA) is proposed in this article. Firstly, inspired by attention mechanisms and information fusion methods, a spectrum-based weighted multi-channel data fusion strategy is designed to fully utilize the data collected by sensors to obtain a more comprehensive representation of fault features. Then, the joint probability-based discriminative maximum mean discrepancy algorithm is introduced into the SF method to strengthen the capability of extracting the domain invariant features. Finally, two bearing datasets are employed to verify the validity of the MSFDDA method, which proved to be superior to other current domain adaptation methods.

KLASIFIKASI UNTUK MEMPREDIKSI TINGKAT KELULUSAN MAHASISWA STMIK WIDURI MENGGUNAKAN ALGORITMA NAÏVE BAYES

Student delays in completing their studies are experienced by most higher education institutions, for example at STMIK Widuri. STMIK Widuri must be able to predict student graduation early to prevent graduation that is not on time and maintain a good name and the accreditation assessment that has been obtained. For this reason, this research was conducted to predict the graduation of STMIK Widuri students using the classification method with the Naïve Bayes algorithm. Naïve Bayes is a classification algorithm that uses probability and statistics to predict a class. The dataset used is lecture activities of STMIK Widuri students class of 2021 from 2021-2022 odd to even 2022-2023 academic year and processed using the Rapidminer application. The dataset is processed through the stages of Knowledge Discovery in Database, including selection, pre-processing, transformation, data mining and evaluation stages. From the evaluation results using the confusion matrix on the distribution of training data 50% and data testing 50%, this study resulted in an Accuracy 93,10%, Precision 95,24%, and Recall 90%. In this way, it is hoped that STMIK Widuri can utilize attributes of the data stored in the database to be processed more optimally, for example using existing techniques in data mining.

A Meta-adversarial Framework for Cross-Domain Cold-Start Recommendation

AbstractThe cold-start problem in recommender systems has been facing a great challenge. Cross-domain recommendation can improve the performance of cold-start user recommendations in the target domain by using the rich information of users in the source domain. In cross-domain cold-start recommendation, users in target domain lack sufficient historical behaviors. Existing meta-learning-based methods depend on the feature distribution of training data and limit the adaptability in new tasks. To address these issues, we propose a meta-adversarial framework for cross-domain cold-start recommendation (MAFCDR) . Specifically, we employ a multi-level feature attention mechanism for independently learning the weights of long-term and short-term features to construct preferences of users in source domain. To migrate user representations, we train a meta-adversarial network that utilizes feature embeddings in the source domain as input and enhances the robustness and stability of the model. Then, the personalized bridge function transfers the user preferences in the source domain to the target domain. We build three cross-domain tasks using Amazon dataset and conduct extensive experiments, which demonstrate the effectiveness of the proposed model in cold-start user recommendation.

Bibimbap : Pre-trained models ensemble for Domain Generalization

This paper addresses a machine learning problem often challenged by differences in the distributions of training and real-world data. We propose a framework that addresses the problem of underfitting in the ensembling method using pre-trained models and improves the performance and robustness of deep learning models through ensemble diversity. For the naive weight ensembling framework, we discovered that the ensembled models could not lie in the same loss basin under extreme domain shift conditions, suggesting that a loss barrier may exist. We used a fine-tuning step after the weighted ensemble to address the underfitting problem caused by the loss barrier and stabilize the batch normalization running parameters. We also inferred through qualitative analysis that the diversity of ensemble models affects domain generalization. We validate our method on a large-scale image dataset (ImageNet-1K) and chemical molecule data, which is suitable for testing with domain shift problems due to its data-splitting method.

Enhanced model iteration algorithm with graph neural network for diffuse optical tomography.

Diffuse optical tomography (DOT) employs near-infrared light to reveal the optical parameters of biological tissues. Due to the strong scattering of photons in tissues and the limited surface measurements, DOT reconstruction is severely ill-posed. The Levenberg-Marquardt (LM) is a popular iteration method for DOT, however, it is computationally expensive and its reconstruction accuracy needs improvement. In this study, we propose a neural model based iteration algorithm which combines the graph neural network with Levenberg-Marquardt (GNNLM), which utilizes a graph data structure to represent the finite element mesh. In order to verify the performance of the graph neural network, two GNN variants, namely graph convolutional neural network (GCN) and graph attention neural network (GAT) were employed in the experiments. The results showed that GCNLM performs best in the simulation experiments within the training data distribution. However, GATLM exhibits superior performance in the simulation experiments outside the training data distribution and real experiments with breast-like phantoms. It demonstrated that the GATLM trained with simulation data can generalize well to situations outside the training data distribution without transfer training. This offers the possibility to provide more accurate absorption coefficient distributions in clinical practice.

Exploring biometric domain adaptation in human action recognition models for unconstrained environments

AbstractIn conventional machine learning (ML), a fundamental assumption is that the training and test sets share identical feature distributions, a reasonable premise drawn from the same dataset. However, real-world scenarios often defy this assumption, as data may originate from diverse sources, causing disparities between training and test data distributions. This leads to a domain shift, where variations emerge between the source and target domains. This study delves into human action recognition (HAR) models within an unconstrained, real-world setting, scrutinizing the impact of input data variations related to contextual information and video encoding. The objective is to highlight the intricacies of model performance and interpretability in this context. Additionally, the study explores the domain adaptability of HAR models, specifically focusing on their potential for re-identifying individuals within uncontrolled environments. The experiments involve seven pre-trained backbone models and introduce a novel analytical approach by linking domain-related (HAR) and domain-unrelated (re-identification (re-ID)) tasks. Two key analyses addressing contextual information and encoding strategies reveal that maintaining the same encoding approach during training results in high task correlation while incorporating richer contextual information enhances performance. A notable outcome of this study is the comprehensive evaluation of a novel transformer-based architecture driven by a HAR backbone, which achieves a robust re-ID performance superior to state-of-the-art (SOTA). However, it faces challenges when other encoding schemes are applied, highlighting the role of the HAR classifier in performance variations.

Joint streaming model for backchannel prediction and automatic speech recognition

AbstractIn human conversations, listeners often utilize brief backchannels such as “uh‐huh” or “yeah.” Timely backchannels are crucial to understanding and increasing trust among conversational partners. In human–machine conversation systems, users can engage in natural conversations when a conversational agent generates backchannels like a human listener. We propose a method that simultaneously predicts backchannels and recognizes speech in real time. We use a streaming transformer and adopt multitask learning for concurrent backchannel prediction and speech recognition. The experimental results demonstrate the superior performance of our method compared with previous works while maintaining a similar single‐task speech recognition performance. Owing to the extremely imbalanced training data distribution, the single‐task backchannel prediction model fails to predict any of the backchannel categories, and the proposed multitask approach substantially enhances the backchannel prediction performance. Notably, in the streaming prediction scenario, the performance of backchannel prediction improves by up to 18.7% compared with existing methods.