Train Machine Learning Models Research Articles

Supervised Machine Learning Techniques for Breeding Value Prediction in Horses: An Example Using Gait Visual Scores

Gait scores are widely used in the genetic evaluation of horses. However, the nature of such measurement may limit genetic progress since there is subjectivity in phenotypic information. This study aimed to assess the application of machine learning techniques in the prediction of breeding values for five visual gait scores in Campolina horses: dissociation, comfort, style, regularity, and development. The dataset contained over 5000 phenotypic records with 107,951 horses (14 generations) in the pedigree. A fixed model was used to estimate least-square solutions for fixed effects and adjusted phenotypes. Variance components and breeding values (EBV) were obtained via a multiple-trait model (MTM). Adjusted phenotypes and fixed effects solutions were used to train machine learning models (using the EBV from MTM as target variable): artificial neural network (ANN), random forest regression (RFR) and support vector regression (SVR). To validate the models, the linear regression method was used. Accuracy was comparable across all models (but it was slightly higher for ANN). The highest bias was observed for ANN, followed by MTM. Dispersion varied according to the trait; it was higher for ANN and the lowest for MTM. Machine learning is a feasible alternative to EBV prediction; however, this method will be slightly biased and over-dispersed for young animals.

A multi-stage lithium-ion battery aging dataset using various experimental design methodologies

This dataset encompasses a comprehensive investigation of combined calendar and cycle aging in commercially available lithium-ion battery cells (Samsung INR21700-50E). A total of 279 cells were subjected to 71 distinct aging conditions across two stages. Stage 1 is based on a non-model-based design of experiments (DoE), including full-factorial and Latin hypercube experimental designs, to determine the degradation behavior. Stage 2 employed model-based parameter individual optimal experimental design (pi-OED) to refine specific dependencies, along with a second non-model-based approach for fair comparison of DoE methodologies. While the primary aim was to validate the benefits of optimal experimental design in lithium-ion battery aging studies, this dataset offers extensive utility for various applications. They include training of machine learning models for battery life prediction, calibrating of physics-based or (semi-)empirical models for battery performance and degradation, and numerous other investigations in battery research. Additionally, the dataset has the potential to uncover hidden dependencies and correlations in battery aging mechanisms that were not evident in previous studies, which often relied on pre-existing assumptions and limited experimental designs.

Machine Learning for Advanced Emission Monitoring and Reduction Strategies in Fossil Fuel Power Plants

Fossil fuel power plants are a significant contributor to global carbon dioxide (CO2) and nitrogen oxide (NOx) emissions. Accurate monitoring and effective reduction of these emissions are crucial for mitigating climate change. This systematic review examines the current state of research on the application of machine learning techniques in evaluating the emissions from fossil fuel power plants. This review first briefly introduces the continuous emission monitoring (CEM) systems and predictive emission monitoring (PEM) systems that are commonly used in power plants and highlights that machine learning models can significantly improve PEM systems through their capability to process and interpret large datasets intelligently to transform traditional emission monitoring systems by enhancing their precision, effectiveness, and cost-efficiency. Compared to previously published review articles, the key contribution and innovation in this present review is the discussion of machine learning models in CO2/NOx emissions according to the different algorithms used, including their advantages and disadvantages in a systematic way, which aims to help future researchers to develop more effective machine learning models. The most popular machine learning model includes reinforcement learning, a forward neural network, a long short-term memory neural network, and support vector regression. While each model method has its own advantages and disadvantages, we noted that training data quality, as well as the proper selection of model parameters, plays an important role. The challenges and research gaps, such as model transferability, a deep understanding of the physics of CO2/NOx emissions, and the availability of high-quality data for training machine learning models, are identified, and recommendations as well as potential future research directions to address these challenges are proposed and discussed.

Feature Engineering for Agile Requirement Management Using Semantic Analysis

Efficient management and prioritization of software requirements are critical challenges in agile projects, where requirements constantly evolve due to changing user needs, business goals, and regulatory updates. This paper explores the role of semantic feature extraction in enabling adaptive management strategies. Using the PROMISE Expanded Dataset and the Coquina Dataset, we employed TF-IDF weighted Word2Vec for advanced tokenization and feature extraction. Latent Dirichlet Allocation (LDA) was used to analyze how preprocessing steps like stop word removal impact topic representation, revealing that removing stop words improved topic specificity and coherence. To address class imbalance, Synthetic Minority Over-sampling Technique (SMOTE) was applied, enhancing the model's ability to handle underrepresented classes effectively. Principal Component Analysis (PCA) reduced the dimensionality of TF-IDF weighted Word2Vec embeddings from 100 features to 30, while Analysis of Variance (ANOVA) identified the most significant features for classification. The results obtained identified three features to have p-values below 0.05 as statistically significant, p-value = 0.0000605, p-value = 0.00000000469, and p-value = 0.0024. These extracted features could be used as input to training machine learning models for predicting and managing software requirements adaptively during agile development. With the reduction of ambiguities and sentiments of the user at the requirement phase, the development phase could be undertaken seamlessly with ease.

Interpretable machine learning for understanding compositional and testing condition effects on refractive index, density, dielectric constant, and loss tangent of inorganic melts and glasses

Artificial intelligence (AI) and machine learning (ML) have enabled property-targeted design of glasses. Several machine learning models and open-source tools in the literature allow researchers to predict the optical, physical, mechanical, and electrical properties of glasses as a function of their chemical compositions. However, these properties also depend on testing conditions. In this paper, we train machine learning models by considering composition and wavelength, temperature, and frequency to predict the refractive index, density, and the two electrical properties, i.e., dielectric constant and loss tangent of glasses, respectively. The predictions of trained models are explained using SHAP analysis, revealing that testing conditions, such as wavelength and temperature, interact majorly with network formers while predicting refractive index and density. In the case of electrical properties, network formers and frequency have the highest interactions, followed by network modifiers and intermediates, and hence govern predictions of dielectric constant and loss tangent. Overall, AI/ML models that can predict the properties of glasses as a function of their composition and testing conditions, coupled with SHAP plots, provide a practical tool to develop a range of glasses for application under varying conditions.

Building custom NLP tools to annotate discourse-functional features for second language writing research: A tutorial

The current tutorial paper describes a process of developing a custom natural language processing model with a particular focus on a discourse annotation task. After an overview of recent developments in natural language processing (NLP), the paper discusses the development of the Engagement Analyzer (Eguchi & Kyle, 2023), focusing on corpus annotation, the machine learning model, model training, evaluation, and dissemination. A step-by-step tutorial of this process via the spaCy Python package is provided. The paper highlights the feasibility of developing custom NLP tools to enhance the scalability and replicability of the annotation of context-sensitive linguistic features in L2 writing research.

Predicting Reactivity and Passivation of Solid-State Battery Interfaces.

In this work, we build a computationally inexpensive, data-driven model that utilizes atomistic structure information to predict the reactivity of interfaces between any candidate solid-state electrolyte material and a Li metal anode. This model is trained on data from ab initio molecular dynamics (AIMD) simulations of the time evolution of the solid electrolyte-Li metal interfaces for 67 different materials. Predicting the reactivity of solid-state interfaces with ab initio techniques remains an elusive challenge in materials discovery and informatics, and previous work on predicting interfacial compatibility of solid-state Li-ion electrolytes and Li metal anodes has focused mainly on thermodynamic convex hull calculations. Our framework involves training machine learning models on AIMD data, thereby capturing information on both kinetics and thermodynamics, and then leveraging these models to predict the reactivity of thousands of new candidates in the span of seconds, avoiding the need for additional weeks-long AIMD simulations. We identify over 300 new chemically stable and over 780 passivating solid electrolytes that are predicted to be thermodynamically unfavored. Our results indicate many potential solid-state electrolyte candidates have been incorrectly labeled unstable via purely thermodynamic approaches using density functional theory (DFT) energetics, and that the pool of promising, Li-stable solid-state electrolyte materials may be much larger than previously thought from screening efforts. To showcase the value of our approach, we highlight two borate materials that were identified by our model and confirmed by further AIMD calculations to likely be highly conductive and chemically stable with Li: LiB13C2 and LiB12PC.

Data-driven shear capacity analysis of headed stud in steel-UHPC composite structures

This study employs machine learning (ML) techniques for shear capacity analysis of headed stud in steel-UHPC composite structure. 194 experimental and numerical results of push-out tests are collected and serve as the dataset for ML models training and testing. Six ML algorithms are implemented to train ML models. Comparison analysis is conducted to compare the performance of three empirical formulae and ML models. The results demonstrate that both the Random Forest and eXtreme Gradient Boosting Trees (XGBoost) models exhibit excellent performance, surpassing an R2 value of 97 % on both training and testing datasets. In contrast, the empirical formulae perform less effectively. Besides, the study incorporates the Shapley additive explanations algorithm to ranking the importance of each feature, and carried out parametric analysis to investigate the correlation between each feature and shear capacity using all samples in the collected dataset. Notably, the most influential variables include diameter and ultimate strength of the headed studs, followed by stud height and thickness of UHPC slab. Cover thickness of UHPC layer and steel fiber volume fractions shows little influence on the shear capacity. Furthermore, based on findings of parametric analysis, design recommendations are provided to avoid shear capacity reduction caused by group effect of studs and UHPC damage. Finally, a user-friendly interactive software is developed and provided to facilitate the shear capacity prediction and design of headed studs in steel-UHPC composite structures.

Antibiotic profile classification of Proteus mirabilis using Machine Learning: An Investigation into Multidimensional Radiomics Features

Antimicrobial resistance (AMR) presents a significant threat to global healthcare. Proteus mirabilis causes catheter-associated urinary tract infections (CAUTIs) and exhibits increased antibiotic resistance. Traditional diagnostics still rely on culture-based approaches, which remain time-consuming. Here, we study the use of machine learning (ML) to classify bacterial resistance profiles using straightforward microscopic imaging of P. mirabilis for resistance classification integrated with radiomics feature analysis and ML models.From 150 P. mirabilis strains isolated from catheters of patients diagnosed with a CAUTI, 30 % displayed multidrug resistance using the standardized disk diffusion method, and 60% showed strong biofilm activity in microtiter plate assays. As a more rapid alternative, we introduce wavelet-based and regular microscopy imaging with feature extraction/selection, following image preprocessing steps (image denoising, normalization, and mask creation). These features enable training and testing different ML models with 5-fold cross-validation for P. mirabilis resistance classification. From these models, the Random Forest (RF) algorithm exhibited the highest performance with ACC=0.95, specificity=0.97, sensitivity=0.88, and AUC=0.98 among the other ML algorithms considered in this study for P. mirabilis resistance classification.This successful application of wavelet-based feature Radiomics analysis with RF model represents a crucial step towards a precise, rapid, and cost-effective method to distinguish antibiotic resistant P. mirabilis strains.

Federated Motor Imagery Classification for Privacy-Preserving Brain-Computer Interfaces.

Training an accurate classifier for EEG-based brain-computer interface (BCI) requires EEG data from a large number of users, whereas protecting their data privacy is a critical consideration. Federated learning (FL) is a promising solution to this challenge. This paper proposes Federated classification with local Batch-specific batch normalization and Sharpness-aware minimization (FedBS) for privacy protection in EEG-based motor imagery (MI) classification. FedBS utilizes local batch-specific batch normalization to reduce data discrepancies among different clients, and sharpness-aware minimization optimizer in local training to improve model generalization. Experiments on three public MI datasets using three popular deep learning models demonstrated that FedBS outperformed six state-of-the-art FL approaches. Remarkably, it also outperformed centralized training, which does not consider privacy protection at all. In summary, FedBS protects user EEG data privacy, enabling multiple BCI users to participate in large-scale machine learning model training, which in turn improves the BCI decoding accuracy.

Exploring databases for training models in machine learning in the Fashion industry

Growing interest in applying machine learning (ML) to fashion highlights the importance of using labeled data to develop models, facilitating research replication, and automating the analysis of new data, such as fashion show images available online. Despite this need, few studies, especially in Brazil, methodologically explore the intersection between fashion and AM. This research aims to provide an overview of online databases for training ML models. A systematic review identified 26 articles that use these databases, such as Fashion-MNIST and DeepFashion2. Content analysis revealed that these databases, including Polyvore and Fashion Image Dataset, have diverse applications, highlighting the transformative potential of AM in fashion and encouraging innovations in design, production, and marketing in the fashion industry.

Efficient compressed storage and fast reconstruction of large binary images using chain codes

AbstractLarge binary images are used in many modern applications of image processing. For instance, they serve as inputs or target masks for training machine learning (ML) models in computer vision and image segmentation. Storing large binary images in limited memory and loading them repeatedly on demand, which is common in ML, calls for efficient image encoding and decoding mechanisms. In the paper, we propose an encoding scheme for efficient compressed storage of large binary images based on chain codes, and introduce a new single-pass algorithm for fast parallel reconstruction of raster images from the encoded representation. We use three large real-life binary masks to test the efficiency of the proposed method, which were derived from vector layers of single-class objects – a building cadaster, a woody vegetation landscape feature map, and a road network map. We show that the masks encoded by the proposed method require significantly less storage space than standard lossless compression formats. We further compared the proposed method for mask reconstruction from chain codes with a recent state-of-the-art algorithm, and achieved between $$12\%$$ 12 % and $$33\%$$ 33 % faster reconstruction on test data.

Computer-Simulated Virtual Image Datasets to Train Machine Learning Models for Non-Invasive Fish Detection in Recirculating Aquaculture.

Artificial Intelligence (AI) and Machine Learning (ML) can assist producers to better manage recirculating aquaculture systems (RASs). ML is a data-intensive process, and model performance primarily depends on the quality of training data. Relatively higher fish density and water turbidity in intensive RAS culture produce major challenges in acquiring high-quality underwater image data. Additionally, the manual image annotation involved in model training can be subjective, time-consuming, and labor-intensive. Therefore, the presented study aimed to simulate fish schooling behavior for RAS conditions and investigate the feasibility of using computer-simulated virtual images to train a robust fish detection model. Additionally, to expedite the model training and automate the virtual image annotation, a process flow was developed. The 'virtual model' performances were compared with models trained on real-world images and combinations of real and virtual images. The results of the study indicate that the virtual model trained solely with computer-simulated images could not perform satisfactorily (mAP = 62.8%, F1 score = 0.61) to detect fish in a real RAS environment; however, replacing a small number of the virtual images with real images in the training dataset significantly improved the model's performance. The M6 mixed model trained with 630 virtual and 70 real images (virtual-to-real image ratio: 90:10) achieved mAP and F1 scores of 91.8% and 0.87, respectively. Furthermore, the training time cost for the M6 model was seven times shorter than that for the 'real model'. Overall, the virtual simulation approach exhibited great promise in rapidly training a reliable fish detection model for RAS operations.

Quantitative detection of the width and depth of subsurface defects in curved structures using ultrasonic Rayleigh waves

ABSTRACT Rayleigh waves are highly sensitive to surface and subsurface defects. However, the varying surface profiles of complex structures induce constant changes in the Rayleigh waveform. Furthermore, Rayleigh waves encompass multi-dimensional characteristics of subsurface defects, significantly increasing the difficulty of quantitatively detecting the sizes of subsurface defects in different dimensions. This paper presents a machine learning (ML)-based method for quantitatively detecting subsurface defect depth and width in curved structures using laser-generated Rayleigh waves. An experimentally validated finite element model is first established to construct a dataset of Rayleigh wave signals labeled with multidimensional subsurface defect sizes. The dataset is augmented from 188 to 368 samples using the Mix-up algorithm, with additional experimental signals incorporated. Feature parameters of Rayleigh waves are extracted in both time and frequency domains through waveform analysis and sliding window wavelet transform (SWT). These features are used to train ML models: Gaussian Process (GP), Extreme Gradient Boosting (XGBoost), and Random Forest (RF). Comparative results demonstrate GP achieves the highest prediction accuracy for subsurface defect depth and width on the test set, reaching 98.64% and 97.73%, respectively. The study highlights the integration of experimental and simulated data in training, which reduces costs while enhancing model robustness in practical applications.

The role of radiomics analysis in the assessment of renal nodules on CT

PurposeTo develop a radiomics model for the characterization of renal nodules on CT.MethodsPatients who underwent surgical resection of renal nodules, with preoperative CT (LightSpeed VCT, GE) with contrast agent (Iopamidol 370; 1.5 ml/kg) and availability of a histopathological report, were retrospectively included. Renal lesions were segmented by two radiologists in consensus, in the arterial phase on the axial section with greater diameter. The radiomics analysis was performed with validated software (PyRadiomics on Syngo.via Frontier, Siemens) by applying normalization, resampling (1,1,1), setting bin width (15), and applying padding (2). The extracted features were used for training and testing machine learning models (random forest and support vector machine (end point: clear cell carcinoma, ccRCC).Results45 patients were included (mean age 56 years; 34/45 ccRCC). SVM e RF models obtained an AUC of 0.84 and accuracy of 0.87 and 0.80, respectively, in the testing phase.ConclusionRadiomics analysis is feasible and effective in the characterization of renal lesions on CT.

Practical feature filter strategy to machine learning for small datasets in chemistry

Many potential use cases for machine learning in chemistry and materials science suffer from small dataset sizes, which demands special care for the model design in order to deliver reliable predictions. Hence, feature selection as the key determinant for dataset design is essential here. We propose a practical and efficient feature filter strategy to determine the best input feature candidates. We illustrate this strategy for the prediction of adsorption energies based on a public dataset and sublimation enthalpies using an in-house training dataset. The input of adsorption energies reduces the feature space from 12 dimensions to two and still delivers accurate results. For the sublimation enthalpies, three input configurations are filtered from 14 possible configurations with different dimensions for further productive predictions as being most relevant by using our feature filter strategy. The best extreme gradient boosting regression model possesses a good performance and is evaluated from statistical and theoretical perspectives, reaching a level of accuracy comparable to density functional theory computations and allowing for physical interpretations of the predictions. Overall, the results indicate that the feature filter strategy can help interdisciplinary scientists without rich professional AI knowledge and limited computational resources to establish a reliable small training dataset first, which may make the final machine learning model training easier and more accurate, avoiding time-consuming hyperparameter explorations and improper feature selection.

Machine learning optimization and 4E analysis of a CCHP system integrated into a greenhouse system for carbon dioxide capturing

The world has seen an increase in the popularity of renewable-based energy systems. However, because of their erratic nature, fossil fuels cannot be entirely phased out. Utilizing carbon dioxide in greenhouses close to power plants and generating extra power electricity from dissipated thermal energy are appealing strategies that cause us to have a cleaner environment and affordable power electricity. To exploit the high enthalpy of expelled gases from a gas turbine, the gas turbine was coupled with an absorption refrigerant cycle and an organic Rankine cycle and analyzed. Additionally, a deep artificial neural network method was used to calculate all functions in a rapid optimization and accurate 4E analysis. The carbon dioxide emissions subsided from 0.9183 to 0.41 kg CO2/s, and the optimization time improved 257 times after using the trained machine learning model. An adsorption technique for CO2 separation is used for the proposed system because of its lower energy consumption. The maximum exergy and energy efficiencies were attained at 36.71% and 49.2%, respectively, and parametric studies are being assessed. Due to increases in harvests and efficiencies, the net annual interest has increased by 21.8%, up to 22.5 million dollars.

Enhancing Sports Injury Risk Assessment in Soccer Through Machine Learning and Training Load Analysis.

Sports injuries pose significant challenges in athlete welfare and team dynamics, particularly in high-intensity sports like soccer. This study used machine learning algorithms to assess non-contact injury risk in professional male soccer players from physiological and mechanical load variables. Twenty-five professional male soccer players with a first-time, non-contact muscle injury were included in this study. Recordings of external load (speed, distance, and acceleration/deceleration data) and internal load (heart rate) were obtained during all training sessions and official matches over a 4-year period. Machine learning model training and evaluation features were calculated for each of nine different metrics for a 28-day period prior to the injury and an equal-length baseline epoch. The acute surge in the values of each workload metric was quantified by the deviation of maximum values from the average, while the variations of cumulative workload over the last four weeks preceding injury were also calculated. Seven features were selected by the model as prominent estimators of injury incidence. Three of the features concerned acute load deviations (number of sprints, training load score-incorporating heart rate and muscle load- and time of heart rate at the 90-100% of maximum). The four cumulative load features were (total distance, high speed and sprint running distance and training load score). The accuracy of the muscle injury risk assessment model was 0.78, with a sensitivity of 0.73 and specificity of 0.85. Our model achieved high performance in injury risk detection using a limited number of training load variables. The inclusion, for the first time, of heart rate related variables in an injury risk assessment model highlights the importance of physiological overload as a contributor to muscle injuries in soccer. By identifying the important parameters, coaches may prevent muscle injuries by controlling surges of training load during training and competition.

Enhancing IoT network defense: advanced intrusion detection via ensemble learning techniques

The Internet of Things (IoT) has evolved significantly, automating daily activities by connecting numerous devices. However, this growth has increased cybersecurity threats, compromising data integrity. To address this, intrusion detection systems (IDSs) have been developed, mainly using predefined attack patterns. With rising cyber-attacks, improving IDS effectiveness is crucial, and machine learning is a key solution. This research enhances IDS capabilities by introducing binary attack identification and multiclass attack categorization for IoT traffic, aiming to improve IDS performance. Our framework uses the ‘BoT-IoT’ and ‘TON-IoT’ datasets, which include various IoT network traffic and cyber-attack scenarios, such as DDoS and data infiltration, to train machine learning and ensemble models. Specifically, it combines three machine learning models-decision tree, resilient backpropagation (RProp) multilayer perceptron (MLP), and logistic regression-into ensemble methods like voting and stacking to improve prediction accuracy and reduce detection errors. These ensemble classifiers outperform individual models, demonstrating the benefit of diverse learning techniques. Our framework achieves high accuracy, with 99.99% for binary classification on the BoT-IoT dataset and 97.31% on the ToN-IoT dataset. For multiclass classification, it achieves 99.99% on BoT-IoT and 96.32% on ToN-IoT, significantly enhancing IDS effectiveness against IoT cybersecurity threats.

Fair swarm learning: Improving incentives for collaboration by a fair reward mechanism

Swarm learning is an emerging technique for collaborative machine learning in which several participants train machine learning models without sharing private data. In a standard swarm network, all the nodes in the network receive identical final models regardless of their individual contributions. This mechanism may be deemed unfair from an economic perspective, discouraging organizations with more resources from participating in any collaboration. Here, we present a framework for swarm learning in which nodes receive personalized models based on their contributions. The results of this study demonstrate the efficacy of this approach by showing that all participants experience performance enhancements compared to their local models. However, participants with higher contributions receive better models than those with lower contributions. This fair mechanism results in the highest possible accuracy for the most contributive participant, comparable to the standard swarm learning model. Such incentive structure can motivate resource-rich organizations to engage in collaboration, leading to the development of machine learning models that incorporate data from more resources, which is ultimately beneficial for every party.