Train Machine Learning Models Research Articles

On the Inference of Original Graph Information from Graph Embeddings

Graph embedding converts a graph data into a low dimensional space to preserve the original graph information. However, graph data can be reconstructed by malicious adversaries to train machine learning models from graph embeddings. This paper studies to what extent an adversary (without the original graph data) can recover the original graph data from graph embeddings. To quantify the original graph information leakage from graph embeddings, we develop a deep neural network model InferNet that can be used by adversaries to infer the original graph information from an adversary-accessible graph embedding database. More specifically, we propose the data-free reversed knowledge distillation technique to support the InferNet training even if the original graph dataset is absent. To ensure the performance of InferNet, we design two cycle-consistency loss functions to have an interactive training of InferNet over three series of datasets. To further enhance the performance of InferNet, we provide a joint training algorithm that simultaneously trains the pseudo-sample generator and InferNet, which significantly reduces the storage space. We evaluate the performance of InferNet on three datasets, and the intensive experiments demonstrate that InferNet can infer the original graph information from the graph embedding dataset with high accuracy.

Supervised Machine Learning Techniques for Breeding Value Prediction in Horses: An Example Using Gait Visual Scores

Gait scores are widely used in the genetic evaluation of horses. However, the nature of such measurement may limit genetic progress since there is subjectivity in phenotypic information. This study aimed to assess the application of machine learning techniques in the prediction of breeding values for five visual gait scores in Campolina horses: dissociation, comfort, style, regularity, and development. The dataset contained over 5000 phenotypic records with 107,951 horses (14 generations) in the pedigree. A fixed model was used to estimate least-square solutions for fixed effects and adjusted phenotypes. Variance components and breeding values (EBV) were obtained via a multiple-trait model (MTM). Adjusted phenotypes and fixed effects solutions were used to train machine learning models (using the EBV from MTM as target variable): artificial neural network (ANN), random forest regression (RFR) and support vector regression (SVR). To validate the models, the linear regression method was used. Accuracy was comparable across all models (but it was slightly higher for ANN). The highest bias was observed for ANN, followed by MTM. Dispersion varied according to the trait; it was higher for ANN and the lowest for MTM. Machine learning is a feasible alternative to EBV prediction; however, this method will be slightly biased and over-dispersed for young animals.

A multi-stage lithium-ion battery aging dataset using various experimental design methodologies

This dataset encompasses a comprehensive investigation of combined calendar and cycle aging in commercially available lithium-ion battery cells (Samsung INR21700-50E). A total of 279 cells were subjected to 71 distinct aging conditions across two stages. Stage 1 is based on a non-model-based design of experiments (DoE), including full-factorial and Latin hypercube experimental designs, to determine the degradation behavior. Stage 2 employed model-based parameter individual optimal experimental design (pi-OED) to refine specific dependencies, along with a second non-model-based approach for fair comparison of DoE methodologies. While the primary aim was to validate the benefits of optimal experimental design in lithium-ion battery aging studies, this dataset offers extensive utility for various applications. They include training of machine learning models for battery life prediction, calibrating of physics-based or (semi-)empirical models for battery performance and degradation, and numerous other investigations in battery research. Additionally, the dataset has the potential to uncover hidden dependencies and correlations in battery aging mechanisms that were not evident in previous studies, which often relied on pre-existing assumptions and limited experimental designs.

Machine Learning for Advanced Emission Monitoring and Reduction Strategies in Fossil Fuel Power Plants

Fossil fuel power plants are a significant contributor to global carbon dioxide (CO2) and nitrogen oxide (NOx) emissions. Accurate monitoring and effective reduction of these emissions are crucial for mitigating climate change. This systematic review examines the current state of research on the application of machine learning techniques in evaluating the emissions from fossil fuel power plants. This review first briefly introduces the continuous emission monitoring (CEM) systems and predictive emission monitoring (PEM) systems that are commonly used in power plants and highlights that machine learning models can significantly improve PEM systems through their capability to process and interpret large datasets intelligently to transform traditional emission monitoring systems by enhancing their precision, effectiveness, and cost-efficiency. Compared to previously published review articles, the key contribution and innovation in this present review is the discussion of machine learning models in CO2/NOx emissions according to the different algorithms used, including their advantages and disadvantages in a systematic way, which aims to help future researchers to develop more effective machine learning models. The most popular machine learning model includes reinforcement learning, a forward neural network, a long short-term memory neural network, and support vector regression. While each model method has its own advantages and disadvantages, we noted that training data quality, as well as the proper selection of model parameters, plays an important role. The challenges and research gaps, such as model transferability, a deep understanding of the physics of CO2/NOx emissions, and the availability of high-quality data for training machine learning models, are identified, and recommendations as well as potential future research directions to address these challenges are proposed and discussed.

Feature Engineering for Agile Requirement Management Using Semantic Analysis

Efficient management and prioritization of software requirements are critical challenges in agile projects, where requirements constantly evolve due to changing user needs, business goals, and regulatory updates. This paper explores the role of semantic feature extraction in enabling adaptive management strategies. Using the PROMISE Expanded Dataset and the Coquina Dataset, we employed TF-IDF weighted Word2Vec for advanced tokenization and feature extraction. Latent Dirichlet Allocation (LDA) was used to analyze how preprocessing steps like stop word removal impact topic representation, revealing that removing stop words improved topic specificity and coherence. To address class imbalance, Synthetic Minority Over-sampling Technique (SMOTE) was applied, enhancing the model's ability to handle underrepresented classes effectively. Principal Component Analysis (PCA) reduced the dimensionality of TF-IDF weighted Word2Vec embeddings from 100 features to 30, while Analysis of Variance (ANOVA) identified the most significant features for classification. The results obtained identified three features to have p-values below 0.05 as statistically significant, p-value = 0.0000605, p-value = 0.00000000469, and p-value = 0.0024. These extracted features could be used as input to training machine learning models for predicting and managing software requirements adaptively during agile development. With the reduction of ambiguities and sentiments of the user at the requirement phase, the development phase could be undertaken seamlessly with ease.

Interpretable machine learning for understanding compositional and testing condition effects on refractive index, density, dielectric constant, and loss tangent of inorganic melts and glasses

Artificial intelligence (AI) and machine learning (ML) have enabled property-targeted design of glasses. Several machine learning models and open-source tools in the literature allow researchers to predict the optical, physical, mechanical, and electrical properties of glasses as a function of their chemical compositions. However, these properties also depend on testing conditions. In this paper, we train machine learning models by considering composition and wavelength, temperature, and frequency to predict the refractive index, density, and the two electrical properties, i.e., dielectric constant and loss tangent of glasses, respectively. The predictions of trained models are explained using SHAP analysis, revealing that testing conditions, such as wavelength and temperature, interact majorly with network formers while predicting refractive index and density. In the case of electrical properties, network formers and frequency have the highest interactions, followed by network modifiers and intermediates, and hence govern predictions of dielectric constant and loss tangent. Overall, AI/ML models that can predict the properties of glasses as a function of their composition and testing conditions, coupled with SHAP plots, provide a practical tool to develop a range of glasses for application under varying conditions.

Building custom NLP tools to annotate discourse-functional features for second language writing research: A tutorial

The current tutorial paper describes a process of developing a custom natural language processing model with a particular focus on a discourse annotation task. After an overview of recent developments in natural language processing (NLP), the paper discusses the development of the Engagement Analyzer (Eguchi & Kyle, 2023), focusing on corpus annotation, the machine learning model, model training, evaluation, and dissemination. A step-by-step tutorial of this process via the spaCy Python package is provided. The paper highlights the feasibility of developing custom NLP tools to enhance the scalability and replicability of the annotation of context-sensitive linguistic features in L2 writing research.

Intelligent dielectric method for evaluating some qualitative characteristics of date fruit

This study aimed to develop an intelligent capacitive system to measure the moisture content of date fruit and to recognize fruit characteristics, such as variety, size, and ripeness. A cost-effective and fast non-contact measurement solution using the capacitive method was employed to create a platform with a variable oscillator to measure the dielectric properties of date fruit after harvest. Different date varieties, namely Zahedi, Ghasb, Mazafati and Medjool, representing dry, semi-dry and wet date fruit, respectively, were selected to model and calibrate the proposed system. Samples of date fruit of each variety were selected at three different ripening stages (Khalal, Rutab and Tamr), ranging from high to low moisture content. Additionally, five distinct moisture contents were determined using the oven method. The moisture content of the date fruit samples ranged from 8.6 % to 86.9 % owing to the selection of four varieties, three ripening stages and five stepwise thermal treatments. After acquiring electronic information, 80 % of the dataset was allocated for training purposes, while the remaining 20 % was reserved for evaluating the final regression model. The results showed that of all the trained machine learning models, Support Vector Regression (SVR) had the highest potential for predicting moisture content at the specified frequencies. The SVR model was fine-tuned by fitting 1824 combinations of hyperparameters over 6 folds. The tuned model's prediction for 20 % of the assigned test data resulted in a coefficient of determination of 88 % compared to the actual moisture content, with a Root Mean Square Error (RMSE) of 9.4 %. Furthermore, the dielectric-based system classified the ripening stages using a Multilayer Perceptron (MLP) model, achieving F1 scores of 87 %, 60 % and 68 % for the Khalal, Rutab and Tamr stages, respectively. The MLP regression model also predicted the geometric mean of the date fruit with a coefficient of determination of 0.82 and an RMSE of 3.05 mm.

Predicting Reactivity and Passivation of Solid-State Battery Interfaces.

In this work, we build a computationally inexpensive, data-driven model that utilizes atomistic structure information to predict the reactivity of interfaces between any candidate solid-state electrolyte material and a Li metal anode. This model is trained on data from ab initio molecular dynamics (AIMD) simulations of the time evolution of the solid electrolyte-Li metal interfaces for 67 different materials. Predicting the reactivity of solid-state interfaces with ab initio techniques remains an elusive challenge in materials discovery and informatics, and previous work on predicting interfacial compatibility of solid-state Li-ion electrolytes and Li metal anodes has focused mainly on thermodynamic convex hull calculations. Our framework involves training machine learning models on AIMD data, thereby capturing information on both kinetics and thermodynamics, and then leveraging these models to predict the reactivity of thousands of new candidates in the span of seconds, avoiding the need for additional weeks-long AIMD simulations. We identify over 300 new chemically stable and over 780 passivating solid electrolytes that are predicted to be thermodynamically unfavored. Our results indicate many potential solid-state electrolyte candidates have been incorrectly labeled unstable via purely thermodynamic approaches using density functional theory (DFT) energetics, and that the pool of promising, Li-stable solid-state electrolyte materials may be much larger than previously thought from screening efforts. To showcase the value of our approach, we highlight two borate materials that were identified by our model and confirmed by further AIMD calculations to likely be highly conductive and chemically stable with Li: LiB13C2 and LiB12PC.

Computationally effective machine learning approach for modular thermal energy storage design

This research presents an innovative approach that integrates computational fluid dynamics (CFD) and machine learning (ML) for the design and optimization of thermal energy storage (TES) systems. Heat discharging parametric analyses conducted using CFD serve as the basis for training ML models, including linear regression, K-nearest neighbor (KNN) regression, gradient boost regression (GBR), XGBoost, LightGBM, and neural network (NN). NN emerges as the most suitable for predicting time-dependent variations of concrete and heat transfer fluid (HTF) temperatures. The trained ML models offer an efficient alternative to traditional CFD simulations, enabling the prediction of temperatures in concrete thermal energy storage (CTES) modules under varying inlet conditions, velocities, and time. Leveraging these ML models, the research demonstrates the design of modular CTES cascaded systems with multiple modules in series and parallel configurations, significantly reducing computational cost and time by over 99% compared to full-scale CFD simulations. For instance, in predicting 4-hour time-dependent thermal behavior, CFD takes 97 s per data point and 238,500 s for a single module, compared to ML models’ 16-20 ms per data point and around 290 s per module, indicating their efficiency and scalability in predicting thermal discharge, especially for modular CTES system design and optimization. ML models also demonstrate computational efficiency for designing CTES systems involving multiple modules, taking approximately 765 s - 1047 s for various CTES system configurations, indicating their effectiveness over CFD in predicting thermal discharge for modular CTES systems. The integration of CFD and ML provides a streamlined workflow for designing and optimizing CTES systems, reducing computational efforts, cost, and time. Moreover, this workflow can be updated with additional training data to implement it for unique modular designs with different conditions. Such a generalization of this ML-based approach makes it applicable to a wide range of thermal energy storage designs and geometries, offering a promising avenue for future research and development in the field of thermal energy storage.

Data-driven shear capacity analysis of headed stud in steel-UHPC composite structures

This study employs machine learning (ML) techniques for shear capacity analysis of headed stud in steel-UHPC composite structure. 194 experimental and numerical results of push-out tests are collected and serve as the dataset for ML models training and testing. Six ML algorithms are implemented to train ML models. Comparison analysis is conducted to compare the performance of three empirical formulae and ML models. The results demonstrate that both the Random Forest and eXtreme Gradient Boosting Trees (XGBoost) models exhibit excellent performance, surpassing an R2 value of 97 % on both training and testing datasets. In contrast, the empirical formulae perform less effectively. Besides, the study incorporates the Shapley additive explanations algorithm to ranking the importance of each feature, and carried out parametric analysis to investigate the correlation between each feature and shear capacity using all samples in the collected dataset. Notably, the most influential variables include diameter and ultimate strength of the headed studs, followed by stud height and thickness of UHPC slab. Cover thickness of UHPC layer and steel fiber volume fractions shows little influence on the shear capacity. Furthermore, based on findings of parametric analysis, design recommendations are provided to avoid shear capacity reduction caused by group effect of studs and UHPC damage. Finally, a user-friendly interactive software is developed and provided to facilitate the shear capacity prediction and design of headed studs in steel-UHPC composite structures.

Exploring databases for training models in machine learning in the Fashion industry

Growing interest in applying machine learning (ML) to fashion highlights the importance of using labeled data to develop models, facilitating research replication, and automating the analysis of new data, such as fashion show images available online. Despite this need, few studies, especially in Brazil, methodologically explore the intersection between fashion and AM. This research aims to provide an overview of online databases for training ML models. A systematic review identified 26 articles that use these databases, such as Fashion-MNIST and DeepFashion2. Content analysis revealed that these databases, including Polyvore and Fashion Image Dataset, have diverse applications, highlighting the transformative potential of AM in fashion and encouraging innovations in design, production, and marketing in the fashion industry.

Efficient compressed storage and fast reconstruction of large binary images using chain codes

AbstractLarge binary images are used in many modern applications of image processing. For instance, they serve as inputs or target masks for training machine learning (ML) models in computer vision and image segmentation. Storing large binary images in limited memory and loading them repeatedly on demand, which is common in ML, calls for efficient image encoding and decoding mechanisms. In the paper, we propose an encoding scheme for efficient compressed storage of large binary images based on chain codes, and introduce a new single-pass algorithm for fast parallel reconstruction of raster images from the encoded representation. We use three large real-life binary masks to test the efficiency of the proposed method, which were derived from vector layers of single-class objects – a building cadaster, a woody vegetation landscape feature map, and a road network map. We show that the masks encoded by the proposed method require significantly less storage space than standard lossless compression formats. We further compared the proposed method for mask reconstruction from chain codes with a recent state-of-the-art algorithm, and achieved between $$12\%$$ 12 % and $$33\%$$ 33 % faster reconstruction on test data.

Computer-Simulated Virtual Image Datasets to Train Machine Learning Models for Non-Invasive Fish Detection in Recirculating Aquaculture.

Artificial Intelligence (AI) and Machine Learning (ML) can assist producers to better manage recirculating aquaculture systems (RASs). ML is a data-intensive process, and model performance primarily depends on the quality of training data. Relatively higher fish density and water turbidity in intensive RAS culture produce major challenges in acquiring high-quality underwater image data. Additionally, the manual image annotation involved in model training can be subjective, time-consuming, and labor-intensive. Therefore, the presented study aimed to simulate fish schooling behavior for RAS conditions and investigate the feasibility of using computer-simulated virtual images to train a robust fish detection model. Additionally, to expedite the model training and automate the virtual image annotation, a process flow was developed. The 'virtual model' performances were compared with models trained on real-world images and combinations of real and virtual images. The results of the study indicate that the virtual model trained solely with computer-simulated images could not perform satisfactorily (mAP = 62.8%, F1 score = 0.61) to detect fish in a real RAS environment; however, replacing a small number of the virtual images with real images in the training dataset significantly improved the model's performance. The M6 mixed model trained with 630 virtual and 70 real images (virtual-to-real image ratio: 90:10) achieved mAP and F1 scores of 91.8% and 0.87, respectively. Furthermore, the training time cost for the M6 model was seven times shorter than that for the 'real model'. Overall, the virtual simulation approach exhibited great promise in rapidly training a reliable fish detection model for RAS operations.

Quantitative detection of the width and depth of subsurface defects in curved structures using ultrasonic Rayleigh waves

ABSTRACT Rayleigh waves are highly sensitive to surface and subsurface defects. However, the varying surface profiles of complex structures induce constant changes in the Rayleigh waveform. Furthermore, Rayleigh waves encompass multi-dimensional characteristics of subsurface defects, significantly increasing the difficulty of quantitatively detecting the sizes of subsurface defects in different dimensions. This paper presents a machine learning (ML)-based method for quantitatively detecting subsurface defect depth and width in curved structures using laser-generated Rayleigh waves. An experimentally validated finite element model is first established to construct a dataset of Rayleigh wave signals labeled with multidimensional subsurface defect sizes. The dataset is augmented from 188 to 368 samples using the Mix-up algorithm, with additional experimental signals incorporated. Feature parameters of Rayleigh waves are extracted in both time and frequency domains through waveform analysis and sliding window wavelet transform (SWT). These features are used to train ML models: Gaussian Process (GP), Extreme Gradient Boosting (XGBoost), and Random Forest (RF). Comparative results demonstrate GP achieves the highest prediction accuracy for subsurface defect depth and width on the test set, reaching 98.64% and 97.73%, respectively. The study highlights the integration of experimental and simulated data in training, which reduces costs while enhancing model robustness in practical applications.

Machine Learning Strategies to Tackle Data Challenges in Mass Spectrometry-Based Proteomics.

In computational proteomics, machine learning (ML) has emerged as a vital tool for enhancing data analysis. Despite significant advancements, the diversity of ML model architectures and the complexity of proteomics data present substantial challenges in the effective development and evaluation of these tools. Here, we highlight the necessity for high-quality, comprehensive data sets to train ML models and advocate for the standardization of data to support robust model development. We emphasize the instrumental role of key data sets like ProteomeTools and MassIVE-KB in advancing ML applications in proteomics and discuss the implications of data set size on model performance, highlighting that larger data sets typically yield more accurate models. To address data scarcity, we explore algorithmic strategies such as self-supervised pretraining and multitask learning. Ultimately, we hope that this discussion can serve as a call to action for the proteomics community to collaborate on data standardization and collection efforts, which are crucial for the sustainable advancement and refinement of ML methodologies in the field.

The role of radiomics analysis in the assessment of renal nodules on CT

PurposeTo develop a radiomics model for the characterization of renal nodules on CT.MethodsPatients who underwent surgical resection of renal nodules, with preoperative CT (LightSpeed VCT, GE) with contrast agent (Iopamidol 370; 1.5 ml/kg) and availability of a histopathological report, were retrospectively included. Renal lesions were segmented by two radiologists in consensus, in the arterial phase on the axial section with greater diameter. The radiomics analysis was performed with validated software (PyRadiomics on Syngo.via Frontier, Siemens) by applying normalization, resampling (1,1,1), setting bin width (15), and applying padding (2). The extracted features were used for training and testing machine learning models (random forest and support vector machine (end point: clear cell carcinoma, ccRCC).Results45 patients were included (mean age 56 years; 34/45 ccRCC). SVM e RF models obtained an AUC of 0.84 and accuracy of 0.87 and 0.80, respectively, in the testing phase.ConclusionRadiomics analysis is feasible and effective in the characterization of renal lesions on CT.

Machine learning optimization and 4E analysis of a CCHP system integrated into a greenhouse system for carbon dioxide capturing

The world has seen an increase in the popularity of renewable-based energy systems. However, because of their erratic nature, fossil fuels cannot be entirely phased out. Utilizing carbon dioxide in greenhouses close to power plants and generating extra power electricity from dissipated thermal energy are appealing strategies that cause us to have a cleaner environment and affordable power electricity. To exploit the high enthalpy of expelled gases from a gas turbine, the gas turbine was coupled with an absorption refrigerant cycle and an organic Rankine cycle and analyzed. Additionally, a deep artificial neural network method was used to calculate all functions in a rapid optimization and accurate 4E analysis. The carbon dioxide emissions subsided from 0.9183 to 0.41 kg CO2/s, and the optimization time improved 257 times after using the trained machine learning model. An adsorption technique for CO2 separation is used for the proposed system because of its lower energy consumption. The maximum exergy and energy efficiencies were attained at 36.71 % and 49.2 %, respectively, and parametric studies are being assessed. Due to increases in harvests and efficiencies, the net annual interest has increased by 21.8 %, up to 22.5 million dollars.

Practical feature filter strategy to machine learning for small datasets in chemistry

Many potential use cases for machine learning in chemistry and materials science suffer from small dataset sizes, which demands special care for the model design in order to deliver reliable predictions. Hence, feature selection as the key determinant for dataset design is essential here. We propose a practical and efficient feature filter strategy to determine the best input feature candidates. We illustrate this strategy for the prediction of adsorption energies based on a public dataset and sublimation enthalpies using an in-house training dataset. The input of adsorption energies reduces the feature space from 12 dimensions to two and still delivers accurate results. For the sublimation enthalpies, three input configurations are filtered from 14 possible configurations with different dimensions for further productive predictions as being most relevant by using our feature filter strategy. The best extreme gradient boosting regression model possesses a good performance and is evaluated from statistical and theoretical perspectives, reaching a level of accuracy comparable to density functional theory computations and allowing for physical interpretations of the predictions. Overall, the results indicate that the feature filter strategy can help interdisciplinary scientists without rich professional AI knowledge and limited computational resources to establish a reliable small training dataset first, which may make the final machine learning model training easier and more accurate, avoiding time-consuming hyperparameter explorations and improper feature selection.

Enhancing Sports Injury Risk Assessment in Soccer Through Machine Learning and Training Load Analysis.

Sports injuries pose significant challenges in athlete welfare and team dynamics, particularly in high-intensity sports like soccer. This study used machine learning algorithms to assess non-contact injury risk in professional male soccer players from physiological and mechanical load variables. Twenty-five professional male soccer players with a first-time, non-contact muscle injury were included in this study. Recordings of external load (speed, distance, and acceleration/deceleration data) and internal load (heart rate) were obtained during all training sessions and official matches over a 4-year period. Machine learning model training and evaluation features were calculated for each of nine different metrics for a 28-day period prior to the injury and an equal-length baseline epoch. The acute surge in the values of each workload metric was quantified by the deviation of maximum values from the average, while the variations of cumulative workload over the last four weeks preceding injury were also calculated. Seven features were selected by the model as prominent estimators of injury incidence. Three of the features concerned acute load deviations (number of sprints, training load score-incorporating heart rate and muscle load- and time of heart rate at the 90-100% of maximum). The four cumulative load features were (total distance, high speed and sprint running distance and training load score). The accuracy of the muscle injury risk assessment model was 0.78, with a sensitivity of 0.73 and specificity of 0.85. Our model achieved high performance in injury risk detection using a limited number of training load variables. The inclusion, for the first time, of heart rate related variables in an injury risk assessment model highlights the importance of physiological overload as a contributor to muscle injuries in soccer. By identifying the important parameters, coaches may prevent muscle injuries by controlling surges of training load during training and competition.