Quality Of Training Data Research Articles

Clinical information extraction for lower-resource languages and domains with few-shot learning using pretrained language models and prompting

Abstract A vast amount of clinical data are still stored in unstructured text. Automatic extraction of medical information from these data poses several challenges: high costs of clinical expertise, restricted computational resources, strict privacy regulations, and limited interpretability of model predictions. Recent domain adaptation and prompting methods using lightweight masked language models showed promising results with minimal training data and allow for application of well-established interpretability methods. We are first to present a systematic evaluation of advanced domain-adaptation and prompting methods in a lower-resource medical domain task, performing multi-class section classification on German doctor’s letters. We evaluate a variety of models, model sizes (further-pre)training and task settings, and conduct extensive class-wise evaluations supported by Shapley values to validate the quality of small-scale training data and to ensure interpretability of model predictions. We show that in few-shot learning scenarios, a lightweight, domain-adapted pretrained language model, prompted with just 20 shots per section class, outperforms a traditional classification model, by increasing accuracy from $48.6\%$ to $79.1\%$ . By using Shapley values for model selection and training data optimization, we could further increase accuracy up to $84.3\%$ . Our analyses reveal that pretraining of masked language models on general-language data is important to support successful domain-transfer to medical language, so that further-pretraining of general-language models on domain-specific documents can outperform models pretrained on domain-specific data only. Our evaluations show that applying prompting based on general-language pretrained masked language models combined with further-pretraining on medical-domain data achieves significant improvements in accuracy beyond traditional models with minimal training data. Further performance improvements and interpretability of results can be achieved, using interpretability methods such as Shapley values. Our findings highlight the feasibility of deploying powerful machine learning methods in clinical settings and can serve as a process-oriented guideline for lower-resource languages and domains such as clinical information extraction projects.

Developing Generalizable Scoring Functions for Molecular Docking: Challenges and Perspectives.

Structure-based drug discovery methods, such as molecular docking and virtual screening, have become invaluable tools in developing novel drugs. At the core of these methods are Scoring Functions (SFs), which predict the binding affinity between ligands and protein targets. This study aims to review and contextualize the challenges and best practices in training novel scoring functions to improve their accuracy and generalizability in predicting protein-ligand binding affinities. Effective training of scoring functions requires careful attention to the quality of training data and methodologies. We emphasize the need for robust training strategies to produce consistent and generalizable SFs. Key considerations include addressing hidden biases and overfitting in machine-learning models, as well as ensuring the use of high-quality, unbiased datasets for both training and evaluation of SFs. Innovative hybrid methods, combining the advantages of empirical and machine-learning approaches, hold promise for outperforming current scoring functions while displaying greater generalizability and versatility.

Leveraging Unsupervised Task Adaptation and Semi‐Supervised Learning With Semantic‐Enriched Representations for Online Sexism Detection

ABSTRACTOver the past decade, the proliferation of hateful and sexist content targeting women on social media has become a concerning issue, adversely affecting women's lives and freedom of expression. Previous efforts to detect online sexism have utilized monolingual ensemble transformers combined with data augmentation techniques that incorporate related‐domain data, such as hate speech. However, these approaches often struggle to capture the full diversity and complexity of sexism due to limitations in the size and quality of training data. In this study, we introduce a novel sexism detection system that employs in‐domain unlabeled data through unsupervised task‐adaptation techniques and semi‐supervised learning, using an efficient single multilingual transformer model. Additionally, we incorporate a Sentence‐BERT layer to enhance our system with semantically meaningful sentence embeddings. Our proposed system outperforms existing state‐of‐the‐art methods across all tasks and datasets, demonstrating its effectiveness in detecting and addressing sexism in social media text. These results underscore the potential of our approach, providing a foundation for further research and practical applications.

Tennis teaching assistance model based on double chain shared unsupervised action recognition algorithm

The traditional self-supervised methods based on skeleton data frequently categorize the various enhancements of a given sample as positive examples, while the remaining samples are designated as negative examples. This approach results in a significant imbalance in the ratio of positive to negative samples, which in turn constrains the efficacy of samples with identical semantic information. Therefore, to further improve the training quality in tennis teaching and enhance the accuracy of action recognition, a dual chain sharing unsupervised action recognition algorithm has been proposed. This study first designs a skeleton topology data augmentation method based on the physical connections of human joints to obtain advanced semantic embeddings. Then, an improved positive sample expansion strategy is utilized to enhance the diversity and quality of training data. Next, an unsupervised learning mechanism is employed to autonomously learn and recognize complex patterns of tennis movements. To measure the performance of the model, tests were conducted on its accuracy, F1 score, recognition time, and fitting degree. The experimental results showed that the proposed algorithm could reach its optimal state after 23 iterations of training, and its F1 value reached 0.918, with an average accuracy of 92.9 % and an average recognition time of 7.4 s. The research algorithms were superior to the multi-input branch graph convolutional network action recognition algorithms used for comparison, pull-push contrastive loss action recognition algorithms, and multi-granularity anchor contrastive representation learning action recognition algorithms. Its accuracy was leading by 8.6 %-23.5 %, and the recognition time has been reduced by 2.3s-7.4 s. In addition, in terms of the fitting degree of action recognition, compared with other methods, the proposed method had a fitting degree of up to 95.7 %, which was at least 8.1 % higher. This research method can improve the timeliness of feedback from trainers by accurately and quickly recognizing movements, helping coaches develop more targeted training plans, thereby improving the efficiency and quality of tennis teaching.

Batik Pattern Classification Using Machine Learning Approaches

Batik is one of Malaysia traditional textile art form that is well known. By using Artificial Neural Network (ANN), k-Nearest Neighbors (k-NN) and Decision Tree methods, this study aims to classify Kelantan batik designs according to flora, fauna and geometry motifs. 133 images of the three categories were collected from social media and underwent pre-processing techniques. Image augmentation was done to enhance the diversity and quality of available training data for machine learning models. Digital transformation on the images based on colour features was developed to classify the three types of batik motifs. Image embedding was employed which returned a vector representation of each image in a data table in Orange software. The data divided into training and testing dataset, with a ratio of 8:2. The performance of each machine learning techniques were compared. The findings showed that ANN produced an excellent performance as indicated by classification accuracy (CA) of 87.9% for original images and 82.2% using Luminance Gradient image transformation technique. ANN also returns the highest AUC score indicating it is the best in distinguishing the images motif. Results from confusion matrix showed that ANN have the least misclassification for batik classification. Based on the result, it signifies that ANN performs the best classification among the three approaches.

Water Resources’ AI–ML Data Uncertainty Risk and Mitigation Using Data Assimilation

Artificial intelligence (AI), including machine learning (ML) and deep learning (DL), learns by training and is restricted by the amount and quality of training data. Training involves a tradeoff between prediction bias and variance controlled by model complexity. Increased model complexity decreases prediction bias, increases variance, and increases overfitting possibilities. Overfitting is a significantly smaller training prediction error relative to the trained model prediction error for an independent validation set. Uncertain data generate risks for AI–ML because they increase overfitting and limit generalization ability. Specious confidence in predictions from overfit models with limited generalization ability, leading to misguided water resource management, is the uncertainty-related negative consequence. Improved data is the way to improve AI–ML models. With uncertain water resource data sets, like stream discharge, there is no quick way to generate improved data. Data assimilation (DA) provides mitigation for uncertainty risks, describes data- and model-related uncertainty, and propagates uncertainty to results using observation error models. A DA-derived mitigation example is provided using a common-sense baseline, derived from an observation error model, for the confirmation of generalization ability and a threshold identifying overfitting. AI–ML models can also be incorporated into DA to provide additional observations for assimilation or as a forward model for prediction and inverse-style calibration or training. The mitigation of uncertain data risks using DA involves a modified bias–variance tradeoff that focuses on increasing solution variability at the expense of increased model bias. Increased variability portrays data and model uncertainty. Uncertainty propagation produces an ensemble of models and a range of predictions.

Machine Learning for Advanced Emission Monitoring and Reduction Strategies in Fossil Fuel Power Plants

Fossil fuel power plants are a significant contributor to global carbon dioxide (CO2) and nitrogen oxide (NOx) emissions. Accurate monitoring and effective reduction of these emissions are crucial for mitigating climate change. This systematic review examines the current state of research on the application of machine learning techniques in evaluating the emissions from fossil fuel power plants. This review first briefly introduces the continuous emission monitoring (CEM) systems and predictive emission monitoring (PEM) systems that are commonly used in power plants and highlights that machine learning models can significantly improve PEM systems through their capability to process and interpret large datasets intelligently to transform traditional emission monitoring systems by enhancing their precision, effectiveness, and cost-efficiency. Compared to previously published review articles, the key contribution and innovation in this present review is the discussion of machine learning models in CO2/NOx emissions according to the different algorithms used, including their advantages and disadvantages in a systematic way, which aims to help future researchers to develop more effective machine learning models. The most popular machine learning model includes reinforcement learning, a forward neural network, a long short-term memory neural network, and support vector regression. While each model method has its own advantages and disadvantages, we noted that training data quality, as well as the proper selection of model parameters, plays an important role. The challenges and research gaps, such as model transferability, a deep understanding of the physics of CO2/NOx emissions, and the availability of high-quality data for training machine learning models, are identified, and recommendations as well as potential future research directions to address these challenges are proposed and discussed.

Latent Diffusion Models to Enhance the Performance of Visual Defect Segmentation Networks in Steel Surface Inspection.

This paper explores the use of state-of-the-art latent diffusion models, specifically stable diffusion, to generate synthetic images for improving the robustness of visual defect segmentation in manufacturing components. Given the scarcity and imbalance of real-world defect data, synthetic data generation offers a promising solution for training deep learning models. We fine-tuned stable diffusion using the LoRA technique on the NEU-seg dataset and evaluated the impact of different ratios of synthetic to real images on the training set of DeepLabV3+ and FPN segmentation models. Our results demonstrated a significant improvement in mean Intersection over Union (mIoU) when the training dataset was augmented with synthetic images. This study highlights the potential of diffusion models for enhancing the quality and diversity of training data in industrial defect detection, leading to more accurate and reliable segmentation results. The proposed approach achieved improvements of 5.95% and 6.85% in mIoU of defect segmentation on each model over the original dataset.

Robust optical picometrology through data diversity

Topologically structured light contains deeply subwavelength features, such as phase singularities, and the scattering of such light can therefore be sensitive to the geometry or movement of scattering objects at such scales. Indeed, it has been shown recently that single-shot optical measurements can yield positional precision better than 100 pm (less than one five-thousandth of the wavelength λ) via a deep-learning-enabled analysis of scattering patterns. Measurement performance, and the extent to which it can be sustained, are constrained by the quality and depth of neural network training data and the stability of the experimental apparatus. Here, we show that a neural network can be trained through exposure to an extended envelope of instrumental/ambient noise conditions to robustly quantify picometric displacements of a target against orders-of-magnitude larger background fluctuations, to maintain precision and accuracy of 100–150 pm in optical measurements (at λ = 488 nm) of nanowire positional change. This capability opens up a range of application opportunities, for example in the optical study of nanostructural dynamics, stiction, material fatigue, and phase transitions.

Application of Artificial Intelligence in Ophthalmology: An Updated Comprehensive Review.

Artificial intelligence (AI) holds immense promise for transforming ophthalmic care through automated screening, precision diagnostics, and optimized treatment planning. This paper reviews recent advances and challenges in applying AI techniques such as machine learning and deep learning to major eye diseases. In diabetic retinopathy, AI algorithms analyze retinal images to accurately identify lesions, which helps clinicians in ophthalmology practice. Systems like IDx-DR (IDx Technologies Inc, USA) are FDA-approved for autonomous detection of referable diabetic retinopathy. For glaucoma, deep learning models assess optic nerve head morphology in fundus photographs to detect damage. In age-related macular degeneration, AI can quantify drusen and diagnose disease severity from both color fundus and optical coherence tomography images. AI has also been used in screening for retinopathy of prematurity, keratoconus, and dry eye disease. Beyond screening, AI can aid treatment decisions by forecasting disease progression and anti-VEGF response. However, potential limitations such as the quality and diversity of training data, lack of rigorous clinical validation, and challenges in regulatory approval and clinician trust must be addressed for the widespread adoption of AI. Two other significant hurdles include the integration of AI into existing clinical workflows and ensuring transparency in AI decision-making processes. With continued research to address these limitations, AI promises to enable earlier diagnosis, optimized resource allocation, personalized treatment, and improved patient outcomes. Besides, synergistic human-AI systems could set a new standard for evidence-based, precise ophthalmic care.

FBLearn: Decentralized Platform for Federated Learning on Blockchain

In recent years, rapid technological advancements have propelled blockchain and artificial intelligence (AI) into prominent roles within the digital industry, each having unique applications. Blockchain, recognized for its secure and transparent data storage, and AI, a powerful tool for data analysis and decision making, exhibit common features that render them complementary. At the same time, machine learning has become a robust and influential technology, adopted by many companies to address non-trivial technical problems. This adoption is fueled by the vast amounts of data generated and utilized in daily operations. An intriguing intersection of blockchain and AI occurs in the realm of federated learning, a distributed approach allowing multiple parties to collaboratively train a shared model without centralizing data. This paper presents a decentralized platform FBLearn for the implementation of federated learning in blockchain, which enables us to harness the benefits of federated learning without the necessity of exchanging sensitive customer or product data, thereby fostering trustless collaboration. As the decentralized blockchain network is introduced in the distributed model training to replace the centralized server, global model aggregation approaches have to be utilized. This paper investigates several techniques for model aggregation based on the local model average and ensemble using either local or globally distributed validation data for model evaluation. The suggested aggregation approaches are experimentally evaluated based on two use cases of the FBLearn platform: credit risk scoring using a random forest classifier and credit card fraud detection using a logistic regression. The experimental results confirm that the suggested adaptive weight calculation and ensemble techniques based on the quality of local training data enhance the robustness of the global model. The performance evaluation metrics and ROC curves prove that the aggregation strategies successfully isolate the influence of the low-quality models on the final model. The proposed system’s ability to outperform models created with separate datasets underscores its potential to enhance collaborative efforts and to improve the accuracy of the final global model compared to each of the local models. Integrating blockchain and federated learning presents a forward-looking approach to data collaboration while addressing privacy concerns.

Computer-Simulated Virtual Image Datasets to Train Machine Learning Models for Non-Invasive Fish Detection in Recirculating Aquaculture.

Artificial Intelligence (AI) and Machine Learning (ML) can assist producers to better manage recirculating aquaculture systems (RASs). ML is a data-intensive process, and model performance primarily depends on the quality of training data. Relatively higher fish density and water turbidity in intensive RAS culture produce major challenges in acquiring high-quality underwater image data. Additionally, the manual image annotation involved in model training can be subjective, time-consuming, and labor-intensive. Therefore, the presented study aimed to simulate fish schooling behavior for RAS conditions and investigate the feasibility of using computer-simulated virtual images to train a robust fish detection model. Additionally, to expedite the model training and automate the virtual image annotation, a process flow was developed. The 'virtual model' performances were compared with models trained on real-world images and combinations of real and virtual images. The results of the study indicate that the virtual model trained solely with computer-simulated images could not perform satisfactorily (mAP = 62.8%, F1 score = 0.61) to detect fish in a real RAS environment; however, replacing a small number of the virtual images with real images in the training dataset significantly improved the model's performance. The M6 mixed model trained with 630 virtual and 70 real images (virtual-to-real image ratio: 90:10) achieved mAP and F1 scores of 91.8% and 0.87, respectively. Furthermore, the training time cost for the M6 model was seven times shorter than that for the 'real model'. Overall, the virtual simulation approach exhibited great promise in rapidly training a reliable fish detection model for RAS operations.

A generative adversarial learning strategy for spatial inspection of compaction quality

Reliable prediction methods play a crucial role in enhancing the compaction quality and implementing the intelligent compaction (IC). The predictive performance of contemporary data-driven models is significantly influenced by the quantity and quality of training data. The scarcity and high dimensionality of field compaction data may lead to overfitting issues in prediction models. In this study, an innovative methodology, spatial trajectory condition generative adversarial networks (STC-GAN), is proposed to synthesize IC datasets by considering the construction procedures. The dataset expansion is driven both by the non-attentional algorithm and the engineering practice. The augmented dataset is used to train the extreme gradient boosting (XGBoost) for estimating the compaction quality. The model accuracy is consistently enhanced by adopting the STC-GAN augmented datasets. The optimal model is achieved after a 2.5-fold expansion by performing the sensitivity study. The proposed STC-GAN-PSO-XGBoost is a physical-based intelligent model, enhancing the reliability of data augmentation and the compaction evaluation model. This method is beneficial to minimize the amount of field tests, enabling the continuous evaluation of compaction quality for IC.

Accuracy of machine learning in predicting outcomes post-percutaneous coronary intervention: a systematic review.

Recent studies have shown potential in introducing machine learning (ML) algorithms to predict outcomes post-percutaneous coronary intervention (PCI). We aimed to critically appraise current ML models' effectiveness as clinical tools to predict outcomes post-PCI. Searches of four databases were conducted for articles published from the database inception date to 29 May 2021. Studies using ML to predict outcomes post-PCI were included. For individual post-PCI outcomes, measures of diagnostic accuracy were extracted. An adapted checklist comprising existing frameworks for new risk markers, diagnostic accuracy, prognostic tools and ML was used to critically appraise the included studies along the stages of the translational pathway: development, validation, and impact. Quality of training data and methods of dealing with missing data were evaluated. Twelve cohorts from 11 studies were included with a total of 4,943,425 patients. ML models performed with high diagnostic accuracy. However, there are concerns over the development of the ML models. Methods of dealing with missing data were problematic. Four studies did not discuss how missing data were handled. One study removed patients if any of the predictor variable data points were missing. Moreover, at the validation stage, only three studies externally validated the models presented. There could be concerns over the applicability of these models. None of the studies discussed the cost-effectiveness of implementing the models. ML models show promise as a useful clinical adjunct to traditional risk stratification scores in predicting outcomes post-PCI. However, significant challenges need to be addressed before ML can be integrated into clinical practice.

Deep learning the Hurst parameter of linear fractional processes and assessing its reliability

AbstractThis research explores the reliability of deep learning, specifically Long Short‐Term Memory (LSTM) networks, for estimating the Hurst parameter in fractional stochastic processes. The study focuses on three types of processes: fractional Brownian motion (fBm), fractional Ornstein–Uhlenbeck (fOU) process, and linear fractional stable motions (lfsm). The work involves a fast generation of extensive datasets for fBm and fOU to train the LSTM network on a large volume of data in a feasible time. The study analyses the accuracy of the LSTM network's Hurst parameter estimation regarding various performance measures like root mean squared error (RMSE), mean absolute error (MAE), mean relative error (MRE), and quantiles of the absolute and relative errors. It finds that LSTM outperforms the traditional statistical methods in the case of fBm and fOU processes; however, it has limited accuracy on lfsm processes. The research also delves into the implications of training length and valuation sequence length on the LSTM's performance. The methodology is applied to estimating the Hurst parameter in li‐ion battery degradation data and obtaining confidence bounds for the estimation. The study concludes that while deep learning methods show promise in parameter estimation of fractional processes, their effectiveness is contingent on the process type and the quality of training data.

Finding core labels for maximizing generalization of graph neural networks

Graph neural networks (GNNs) have become a popular approach for semi-supervised graph representation learning. GNNs research has generally focused on improving methodological details, whereas less attention has been paid to exploring the importance of labeling the data. However, for semi-supervised learning, the quality of training data is vital. In this paper, we first introduce and elaborate on the problem of training data selection for GNNs. More specifically, focusing on node classification, we aim to select representative nodes from a graph used to train GNNs to achieve the best performance. To solve this problem, we are inspired by the popular lottery ticket hypothesis, typically used for sparse architectures, and we propose the following subset hypothesis for graph data: “There exists a core subset when selecting a fixed-size dataset from the dense training dataset, that can represent the properties of the dataset, and GNNs trained on this core subset can achieve a better graph representation”. Equipped with this subset hypothesis, we present an efficient algorithm to identify the core data in the graph for GNNs. Extensive experiments demonstrate that the selected data (as a training set) can obtain performance improvements across various datasets and GNNs architectures.

Fault Diagnosis Method for Elevator Carriages Based on Temporal Generative Federated Distillation

Abstract Addressing the "data silo" issue among different elevator operating units and the temporal correlations in elevator vibration signals, a novel small-sample fault diagnosis method for elevator carriages based on temporal generative federated distillation is proposed. This method incorporates a temporal generative adversarial network into Federated Distillation via Generative Learning (FedGen). FedGen combines federated learning, knowledge distillation, and generative models to enhance model aggregation efficiency while mitigating data heterogeneity. However, the original generative model struggles to maintain dynamic correlations between signals when extracting temporal features. Therefore, an improved Time Series Generative Adversarial Networks (TimeGAN) model is introduced, substituting the initial logarithmic loss function with a least squares error function, thereby enhancing training stability and data quality. This approach eliminates the need for proxy datasets in knowledge distillation, avoiding the loss of temporal information during central server feature extraction. Simulation results demonstrate that this method enables data sharing while protecting data privacy, and enhances model generalization capabilities.

Leak Event Diagnosis for Power Plants: Generative Anomaly Detection Using Prototypical Networks.

Anomaly detection systems based on artificial intelligence (AI) have demonstrated high performance and efficiency in a wide range of applications such as power plants and smart factories. However, due to the inherent reliance of AI systems on the quality of training data, they still demonstrate poor performance in certain environments. Especially in hazardous facilities with constrained data collection, deploying these systems remains a challenge. In this paper, we propose Generative Anomaly Detection using Prototypical Networks (GAD-PN) designed to detect anomalies using only a limited number of normal samples. GAD-PN is a structure that integrates CycleGAN with Prototypical Networks (PNs), learning from metadata similar to the target environment. This approach enables the collection of data that are difficult to gather in real-world environments by using simulation or demonstration models, thus providing opportunities to learn a variety of environmental parameters under ideal and normal conditions. During the inference phase, PNs can classify normal and leak samples using only a small number of normal data from the target environment by prototypes that represent normal and abnormal features. We also complement the challenge of collecting anomaly data by generating anomaly data from normal data using CycleGAN trained on anomaly features. It can also be adapted to various environments that have similar anomalous scenarios, regardless of differences in environmental parameters. To validate the proposed structure, data were collected specifically targeting pipe leakage scenarios, which are significant problems in environments such as power plants. In addition, acoustic ultrasound signals were collected from the pipe nozzles in three different environments. As a result, the proposed model achieved a leak detection accuracy of over 90% in all environments, even with only a small number of normal data. This performance shows an average improvement of approximately 30% compared with traditional unsupervised learning models trained with a limited dataset.

Analysis of Vina Film Sentiment on Social Media X Using The Naïve Bayes Method

The increasingly rapid development of technology and information, one of which is the internet. Where users can share opinions and discuss various topics or problems around them, namely social media One of the news items that frequently appears as a trending topic on X is the Vina film controversy. However, with the large amount of review data available, it will be difficult to process manually. Therefore, sentiment analysis is needed to see whether people's tendencies toward the Vina film case are positive or negative. The stages carried out were data collection taken via web scrapping with an initial amount of data of 833 and processed through the preprocessing stage, including cleaning, case folding, normalization, stopword removal, tokenization, and stemming, the data became 830. The application of the Naïve Bayes algorithm in this research uses the probability method to classify and predict 664 training data and 166 test data, with the help of the Python library. The accuracy calculation results show quite good performance with TF-IDF weighting producing an accuracy of 78%, precision of 80%, and recall of 90%, f1-score of 84%. Analysis from this research shows that the dominance of negative sentiment is 517 while positive sentiment is 313. The amount and quality of training data play an important role in system quality, where high data quality provides better accuracy in predicting sentiment classes.

GeDa: Improving training data with large language models for Aspect Sentiment Triplet Extraction

Aspect Sentiment Triplet Extraction (ASTE) is a subtask of Aspect-based Sentiment Analysis (ABSA). Recently, ASTE methods have achieved promising results. However, the performance of ASTE models is restricted to both the quantity and the quality of training data. As such, challenges lie in collecting valuable data and selecting targeted data for diversified ASTE model architecture. To this end, we propose a novel General Data-Centric Framework (GeDa), which is capable of improving the training data for ASTE models accurately and efficiently. Specifically, two types of prompts are designed to guide large language models in synthetic candidates synthesizing for ASTE task. Then, the Characteristic-Driven Iterative Strategy is put forward to optimize the interaction between the model and the training data. The data is iteratively selected from the synthetic candidates, aiming to improve the quantity and the quality of training data. With multiple iterations, a targeted training set can be obtained to benefit ASTE model learning. Extensive experiments reveal that ASTE models with GeDa reach a more than 5% increment on average F1 by adding only a small amount of training data.