Train Machine Learning Research Articles

Deep Learning of a 200-Member Ensemble with a Limited Historical Training to Improve the Prediction of Extreme Precipitation Events

Abstract This study introduces a deep learning (DL) scheme to generate reliable and skillful probabilistic quantitative precipitation forecasts (PQPFs) in a postprocessing framework. Enhanced machine learning model architecture and training mechanisms are proposed to improve the reliability and skill of PQPFs while permitting computationally efficient model fitting using a short training dataset. The methodology is applied to postprocessing of 24-h accumulated PQPFs from an ensemble forecast system recently introduced by the Center for Western Weather and Water Extremes (CW3E) and for lead times from 1 to 6 days. The ensemble system was designed based on a high-resolution version of the Weather Research and Forecasting (WRF) Model, named West-WRF, to produce a 200-member ensemble in near–real time (NRT) over the western United States during the boreal cool seasons to support Forecast-Informdayed Reservoir Operations (FIRO) and studies of prediction of heavy-to-extreme events. Postprocessed PQPFs are compared with those from the raw West-WRF ensemble, the operational Global Ensemble Forecast System version 12 (GEFSv12), and the ensemble from the European Centre for Medium-Range Weather Forecasts (ECMWF). As an additional baseline, we provide PQPF verification metrics from a recently developed neural network postprocessing scheme. The results demonstrate that the skill of postprocessed forecasts significantly outperforms PQPFs and deterministic forecasts from raw ensembles and the recently developed algorithm. The resulting PQPFs broadly improve upon the reliability and skill of baselines in predicting heavy-to-extreme precipitation (e.g., >75 mm) across all lead times while maintaining the spatial structure of the high-resolution raw ensemble.

Feature Engineering for Physics-informed Evaluation of Electromechanical Impedance Measurements for Sandwich Face Layer Debonding Identification

The present research exploits various physically explainable features for the evaluation of electromechanical impedance (EMI) measurements for sandwich face layer debonding identification using a recently published two-step physics- and machine-learning (ML)-based approach. In the previous work, the ML approach used employs a One-Class Support Vector Machine and a K-Nearest Neighbor model for debonding detection and size estimation, respectively. Feature engineering was used to compensate for the simplifications of the finite element (FE)-based physical model and a simple data calibration step was used to adapt to distribution shifts. This enabled to train both ML models exclusively with FE simulation-based synthetic EMI spectra data. The efficacy of the method was demonstrated on real-world EMI spectra measurements of a circular aluminum sandwich panel by a piezoelectric transducer. The considered face layer debonding damage was idealized and stepwise increased by a milling process. In the present work, we explore opportunities for further optimization of our method with respect to data distillation, where we reduce the computational requirements of both training and inference while at the same time preserving essential information. Furthermore, we investigate the usefulness of a separate approach for debonding detection, formulated as an anomaly detection problem. The data preprocessing and ML models are validated by unseen real-world EMI measurement data and benchmarked with the previously published damage evaluation results, considering both reliability and accuracy. The data and the code used in this study are provided in their entirety to enable reproducibility, enhance comprehensibility, and encourage future research.

An Innovative Approach for Enhancing Relay Coordination in Distribution Systems Through Online Adaptive Strategies Utilizing DNN Machine Learning and a Hybrid GA-SQP Framework

The present study addresses the issue of varying fault locations within a distribution system, which leads to fluctuations in short-circuit currents and requires the implementation of adaptive protection strategies for network reliability. This paper presents a novel adaptive protection scheme that specifically considers these fault location variations using directional overcurrent relays (DOCRs). Unlike previous research on adaptive protection, which does not adequately account for fault location variations, this method employs deep neural networks (DNNs) for online fault location detection. In the verification process, the effectiveness of the proposed methodologies was assessed by analyzing the time derivative of a trained machine learning model for fault identification. This approach enables the immediate detection of fault locations within the distribution system and facilitates the transmission of the setting group index to activate preset optimal coordination parameter values for the system relays. Crucially, the proposed method ensures that the coordination constraints remain intact across various adaptive settings, relying on precise fault identification through machine learning. The practical significance of this approach lies in its applicability to real-world systems because the proposed solutions and protective settings can be easily implemented using commercially available relays. To evaluate its effectiveness, the adaptive protection scheme was tested on three distribution networks: IEEE 14-Bus, 15-Bus and 30-Bus. The comparative test results highlight that the proposed method significantly improves the speed of the protection system for distribution networks when compared to existing studies, making it a valuable contribution to enhancing network reliability and performance.

Melanoma image synthesis: a review using generative adversarial networks

Melanoma is a highly malignant skin cancer that may be fatal if not promptly detected and treated. The limited availability of high-quality melanoma images, which are needed for training machine learning models, is one of the obstacles to detecting melanoma. Generative adversarial networks (GANs) have grown in popularity as a strong technique for image synthesis. This research is also targeted at the sustainable development goal (SDG) for health care. In this study, we survey existing GAN-based melanoma image synthesis methods. In this work, we briefly introduce GANs and how they may be used for generating synthetic images. Ensuring healthy lifestyles and promoting well-being for everyone, regardless of age, is the main aim. A comparative study is carried out on how GANs are used in current research to generate melanoma images and how they improve the classification performance of neural networks. Various public and proprietary datasets for training GANs in melanoma image synthesis are also discussed. Lastly, we assess the examined studies' performance using measures like the Frechet Inception distance (FID), Inception score, structural similarity ındex (SSIM), and various classification performance metrics. We compare the evaluated findings and suggest further GAN-based melanoma image-creation research.

Machine Learning-Enabled Acoustic Emission Signal Processing for real-time crack growth monitoring in heterogeneous structures

Heterogeneous structures pose unique challenges for structural health monitoring, demanding advanced techniques for early detection of crack propagation. Acoustic Emission (AE) is a non-destructive testing technique that relies on the detection and analysis of stress-induced high-frequency acoustic waves. It is commonly used for identifying and monitoring the development of cracks, defects, or other structural anomalies. However, raw AE data is often noisy and contains background noise and interference, which make them not suitable for reliable real-time crack monitoring applications. This study integrates AE data with machine learning algorithms to provide a proactive and accurate system for continuous crack propagation monitoring. The performance of the proposed approach for features extraction to derive relevant information from the AE signals, data labeling for training machine learning models, and the deployment of these models for real-time monitoring, are discussed.

Hawk: Accurate and Fast Privacy-Preserving Machine Learning Using Secure Lookup Table Computation

Training machine learning models on data from multiple entities without direct data sharing can unlock applications otherwise hindered by business, legal, or ethical constraints. In this work, we design and implement new privacy-preserving machine learning protocols for logistic regression and neural network models. We adopt a two-server model where data owners secret-share their data between two servers that train and evaluate the model on the joint data. A significant source of inefficiency and inaccuracy in existing methods arises from using Yao’s garbled circuits to compute non-linear activation functions. We propose new methods for computing non-linear functions based on secret-shared lookup tables, offering both computational efficiency and improved accuracy. Beyond introducing leakage-free techniques, we initiate the exploration of relaxed security measures for privacy-preserving machine learning. Instead of claiming that the servers gain no knowledge during the computation, we contend that while some information is revealed about access patterns to lookup tables, it maintains epsilon-dX-privacy. Leveraging this relaxation significantly reduces the computational resources needed for training. We present new cryptographic protocols tailored to this relaxed security paradigm and define and analyze the leakage. Our evaluations show that our logistic regression protocol is up to 9x faster, and the neural network training is up to 688x faster than SecureML. Notably, our neural network achieves an accuracy of 96.6% on MNIST in 15 epochs, outperforming prior benchmarks that capped at 93.4% using the same architecture.

Robust and effective ab initio molecular dynamics simulations on the GPU cloud infrastructure using the Schrödinger Materials Science Suite

Ab initio Born-Oppenheimer molecular dynamics (AIMD) is a valuable method for simulating physico-chemical processes of complex systems, including reactive systems, and for training machine learning models and force fields. Speed and stability issues on traditional hardware preclude routine AIMD simulations for larger systems and longer timescales. We postulate that any practically useful AIMD simulation must generate a trajectory of a minimum 1000 MD steps a day on a moderate cloud resource. In this work, we implement a computing workflow that enables routine calculations at this throughput and demonstrate results for several non-trivial atomistic dynamical systems. In particular, we have employed the GPU implementation of the Quantum ESPRESSO code which we will show increases AIMD productivity compared to the CPU version. In order to take advantage of transient servers (which are more cost and energy effective compared to the stable servers), we have implemented automatic restart/continuation of the AIMD runs within the Schrödinger Materials Science Suite. Finally, to reduce simulation size and thus reduce compute time when modeling surfaces, we have implemented a wall potential constraint. Our benchmarks using several reactive systems (lithium anode surface/solvent interface, hydrogen diffusion in an iron grain boundary) show a significant speed up when running on a GPU-enabled transient server using our updated implementation.

Predictive design of KSnI3-based perovskite solar cells using SCAPS and machine learning model

The commercial application of lead (Pb)-based Perovskite Solar Cells (PSCs) are restrained due to its toxicity. In this work, we have investigated potassium tin iodide (KSnI3) which is a potential candidate for non-lead PSC. The SCAPS-1D simulation is employed to examine the effect of several Hole Transport Layers (HTLs) as follow: Spiro-OMeTAD, CuI, CuSCN, Cu2O, PEDOT: PSS, CdTe, MASnBr3, CFTS on the performance of KSnI3-based PSCs. The impact of layer thickness, doping density (NA) and defect density (Nt) has been analyzed. We have a trained Machine learning model to predict the performance matrices of PSCs. We have obtained a maximum efficiency of 23.08 % at the thickness of the KSnI3 absorber layer of 550 nm and Nt of 1013 cm−3. The ML model predicts the performance matrices of examined PSC with accuracy of ∼ 89 %. Hence, the FTO/WS2/KSnI3/PEDOT: PSS/Au configuration is most suitable to manufacture highly efficient PV devices.

On-Device Neural Network for Object Train and Recognition using Mobile

A neural network is a machine learning (ML) program or model that processes information and recognizes patterns, similar to the human brain. The neural network algorithm operates by training on data to learn and enhance its accuracy. If there is any mistake in the learning process, the machine will react unexpectedly and produce incorrect information. So, whenever a neural network model is developed, it is mandatory to evaluate its performance and ensure its output is accurate. Cameras, touchscreens, internet connectivity, and powerful CPUs have contributed to the popularity of smartphones. Mobile apps are software applications that run on smartphones. ML enhances mobile app functionality by offering features such as voice recognition, image analysis, natural language processing, personalization, and recommendations. Training ML models using mobile apps is challenging due to limited resources, data, and privacy concerns. Object recognition is a neural network-based technique that enables the identification and localization of objects in an image or video. This technique is used in driverless cars, disease identification, industrial inspection, robotics, and more. In this paper, the author introduces a new neural network-based algorithm that utilizes mobile devices for training and recognizing images. Although mobile devices have some technological limitations for training, well-established guidelines for systematically mapping verification and validation techniques have been used in the proposed neural network to ensure performance and correctness in on-device object training and recognition.

Data-driven discovery of novel metal organic frameworks with superior ammonia adsorption capacity

Ammonia (NH3) has been a subject of great interest due to its important roles in diverse technological applications. However, high toxicity and corrosiveness of NH3 has made it an important task to develop an efficient carrier to safely capture NH3 with high capacity. Here, we employ a machine learning (ML) model to discover high-performance metal organic frameworks (MOFs) that will work as efficient NH3 carriers. By constructing databases at two distinct conditions, adsorption and desorption, through Grand Canonical Monte Carlo (GCMC) simulations to train ML models, we identify eight novel MOFs as potentially efficient NH3 carriers through screening the large-scale MOF databases with the trained models and GCMC verification. The identified MOFs exhibit the average NH3 working capacity exceeding 1100 mg/g, and subsequent molecular dynamics simulations demonstrate mechanical stability of the predicted MOFs. Moreover, analyses of the diffusion mechanism within the proposed MOFs underscore the strong dependence of NH₃ gas diffusivity on the structural details of the materials.

Contrastive learning based on hierarchical graph of microstructures through directed energy deposition process to establish process-structure–property relationship via autoencoder

n this study, an innovative algorithm is developed to transform microstructures to hierarchical graphs without manual feature engineering. The hierarchical graph comprises two layers. Initially, pixel data, which includes Euler angles, phase, and position, forms the pixel-wise graphs within individual grains. Following this foundation, the grains serve as nodes, constituting the second layer. Importantly, the hierarchical graph preserves essential measurement data and structural details for training machine learning models. After that, a contrastive learning model based on hierarchical graph is designed to capture representations of microstructures obtained by the electron backscatter diffraction (EBSD) technique. This model can be directly extended for other microscopy techniques, such as Scanning Electron Microscopy (SEM), Energy-Dispersive X-ray spectroscopy (EDX), and Transmission Electron Microscopy (TEM). Using the learned microstructure representations, an autoencoder model for the directed energy deposition (DED) is developed to establish the relationship among process parameters, microstructures, and material properties, completing the process-structure–property cycle quantitatively. The performance of the present model is naturally benchmarked against two other models: a contrastive learning model based on pixel-wise graph (without manual feature engineering) and a contrastive learning model based on grain-wise graph (employing manual feature engineering). The results of the present model highlight the potential in decoding the process-structure–property relationships in the DED process.

Differential Private Federated Learning in Geographically Distributed Public Administration Processes

Public administration frequently deals with geographically scattered personal data between multiple government locations and organizations. As digital technologies advance, public administration is increasingly relying on collaborative intelligence while protecting individual privacy. In this context, federated learning has become known as a potential technique to train machine learning models on private and distributed data while maintaining data privacy. This work looks at the trade-off between privacy assurances and vulnerability to membership inference attacks in differential private federated learning in the context of public administration applications. Real-world data from collaborating organizations, concretely, the payroll data from the Ministry of Education and the public opinion survey data from Asia Foundation in Afghanistan, were used to evaluate the effectiveness of noise injection, a typical defense strategy against membership inference attacks, at different noise levels. The investigation focused on the impact of noise on model performance and selected privacy metrics applicable to public administration data. The findings highlight the importance of a balanced compromise between data privacy and model utility because excessive noise can reduce the accuracy of the model. They also highlight the need for careful consideration of noise levels in differential private federated learning for public administration tasks to provide a well-calibrated balance between data privacy and model utility, contributing toward transparent government practices.

Development of a differential treatment selection model for depression on consolidated and transformed clinical trial datasets

Major depressive disorder (MDD) is the leading cause of disability worldwide, yet treatment selection still proceeds via “trial and error”. Given the varied presentation of MDD and heterogeneity of treatment response, the use of machine learning to understand complex, non-linear relationships in data may be key for treatment personalization. Well-organized, structured data from clinical trials with standardized outcome measures is useful for training machine learning models; however, combining data across trials poses numerous challenges. There is also persistent concern that machine learning models can propagate harmful biases. We have created a methodology for organizing and preprocessing depression clinical trial data such that transformed variables harmonized across disparate datasets can be used as input for feature selection. Using Bayesian optimization, we identified an optimal multi-layer dense neural network that used data from 21 clinical and sociodemographic features as input in order to perform differential treatment benefit prediction. With this combined dataset of 5032 individuals and 6 drugs, we created a differential treatment benefit prediction model. Our model generalized well to the held-out test set and produced similar accuracy metrics in the test and validation set with an AUC of 0.7 when predicting binary remission. To address the potential for bias propagation, we used a bias testing performance metric to evaluate the model for harmful biases related to ethnicity, age, or sex. We present a full pipeline from data preprocessing to model validation that was employed to create the first differential treatment benefit prediction model for MDD containing 6 treatment options.

The use of vibrational spectroscopy and supervised machine learning for chemical identification of plastics ingested by seabirds

Plastic pollution is now ubiquitous in the environment and represents a growing threat to wildlife, who can mistake plastic for food and ingest it. Tackling this problem requires reliable, consistent methods for monitoring plastic pollution ingested by seabirds and other marine fauna, including methods for identifying different types of plastic. This study presents a robust method for the rapid, reliable chemical characterisation of ingested plastics in the 1–50 mm size range using infrared and Raman spectroscopy. We analysed 246 objects ingested by Flesh-footed Shearwaters (Ardenna carneipes) from Lord Howe Island, Australia, and compared the data yielded by each technique: 92 % of ingested objects visually identified as plastic were confirmed by spectroscopy, 98 % of those were low density polymers such as polyethylene, polypropylene, or their copolymers. Ingested plastics exhibit significant spectral evidence of biological contamination compared to other reports, which hinders identification by conventional library searching. Machine learning can be used to identify ingested plastics by their vibrational spectra with up to 93 % accuracy. Overall, we find that infrared is the more effective technique for identifying ingested plastics in this size range, and that appropriately trained machine learning models can be superior to conventional library searching methods for identifying plastics.

Federated Learning for Edge Computing in REST APIs: Investigate the Feasibility of Employing Federated Learning Techniques to Train Machine Learning Models Directly on Edge Devices Interacting with REST APIs, Minimizing Data Transfer and Enhancing Privacy

This research explores the feasibility of using federated learning techniques to train machine learning models directly on edge devices interacting through REST APIs. This article designs an architecture that facilitates federated learning using REST APIs for communication between a central server and edge nodes. The architecture ensures efficient model distribution, update aggregation, and iterative global model refinement. This research implements this system in a simulated environment to evaluate its performance. Initial results show that federated learning significantly reduces data transfer and enhances privacy without compromising model accuracy. The REST API-based communication model effectively coordinates updates between the central server and edge devices, and privacy-preserving techniques maintain data security throughout the process. This study confirms that federated learning is a viable solution for edge computing, addressing key challenges in data privacy and communication efficiency, and opens pathways for scalable and effective distributed learning systems in real-world IoT applications. Federated learning offers a decentralized alternative, enabling local model training on edge devices with only model updates shared with a central server. This approach minimizes data transfer and enhances privacy.

Adaptive loss weighting for machine learning interatomic potentials

Training machine learning interatomic potentials often requires optimizing a loss function composed of three variables: potential energies, forces, and stress. The contribution of each variable to the total loss is typically weighted using fixed coefficients. Identifying these coefficients usually relies on iterative or heuristic methods, which may yield sub-optimal results. To address this issue, we propose an adaptive loss weighting algorithm that automatically adjusts the loss weights of these variables during the training of potentials, dynamically adapting to the characteristics of the training dataset. The comparative analysis of models trained with fixed and adaptive loss weights demonstrates that the adaptive method not only achieves more balanced predictions across the three variables but also improves overall prediction accuracy.

Comparison of SERS spectra of intact and inactivated viruses via machine learning algorithms for the viral disease’s diagnosis application

In this work, Surface-enhanced Raman spectroscopy (SERS) along with machine learning algorithms (MLA) were used to detect and classify the viral particles to assess the possibility of using the spectra of inactivated influenza A viruses for MLA training and spectra database compilation for further study and diagnosis of intact forms of the virus. Viral particles inactivation was performed by formalin, ultraviolet and beta-propiolactone. Support vector method and principal component analysis allowed to classify intact and inactivated viral particles spectra with an accuracy of 80.0–96.7 %. The results obtained suggest that it is not advisable to create a spectral database and train machine learning algorithms for their further application in SERS diagnostics of intact viruses based on the spectra of the inactivated virus particles.

Learning More with Less Data in Manufacturing: The Case of Turning Tool Wear Assessment through Active and Transfer Learning

Monitoring tool wear is key for the optimization of manufacturing processes. To achieve this, machine learning (ML) has provided mechanisms that work adequately on setups that measure the cutting force of a tool through the use of force sensors. However, given the increased focus on sustainability, i.e., in the context of reducing complexity, time and energy consumption required to train ML algorithms on large datasets dictate the use of smaller samples for training. Herein, the concepts of active learning (AL) and transfer learning (TL) are simultaneously studied concerning their ability to meet the aforementioned objective. A method is presented which utilizes AL for training ML models with less data and then it utilizes TL to further reduce the need for training data when ML models are transferred from one industrial case to another. The method is tested and verified upon an industrially relevant scenario to estimate the tool wear during the turning process of two manufacturing companies. The results indicated that through the application of the AL and TL methodologies, in both companies, it was possible to achieve high accuracy during the training of the final model (1 and 0.93 for manufacturing companies B and A, respectively). Additionally, reproducibility of the results has been tested to strengthen the outcomes of this study, resulting in a small standard deviation of 0.031 in the performance metrics used to evaluate the models. Thus, the novelty presented in this paper is the presentation of a straightforward approach to apply AL and TL in the context of tool wear classification to reduce the dependency on large amounts of high-quality data. The results show that the synergetic combination of AL with TL can reduce the need for data required for training ML models for tool wear prediction.

Classification of Partial Discharge Sources in Ultra-High Frequency Using Signal Conditioning Circuit Phase-Resolved Partial Discharges and Machine Learning

This work presents a methodology for the generation and classification of phase-resolved partial discharge (PRPD) patterns based on the use of a printed UHF monopole antenna and signal conditioning circuit to reduce hardware requirements. For this purpose, the envelope detection technique was applied. In addition, test objects such as a hydrogenerator bar, dielectric discs with internal cavities in an oil cell, a potential transformer and tip–tip electrodes immersed in oil were used to generate partial discharge (PD) signals. To detect and classify partial discharges, the standard IEC 60270 (2000) method was used as a reference. After the acquisition of conditioned UHF signals, a digital signal filtering threshold technique was used, and peaks of partial discharge envelope pulses were extracted. Feature selection techniques were used to classify the discharges and choose the best features to train machine learning algorithms, such as multilayer perceptron, support vector machine and decision tree algorithms. Accuracies greater than 84% were met, revealing the classification potential of the methodology proposed in this work.

Development and validation of a clinical breast cancer tool for accurate prediction of recurrence

Given high costs of Oncotype DX (ODX) testing, widely used in recurrence risk assessment for early-stage breast cancer, studies have predicted ODX using quantitative clinicopathologic variables. However, such models have incorporated only small cohorts. Using a cohort of patients from the National Cancer Database (NCDB, n = 53,346), we trained machine learning models to predict low-risk (0-25) or high-risk (26-100) ODX using quantitative estrogen receptor (ER)/progesterone receptor (PR)/Ki-67 status, quantitative ER/PR status alone, and no quantitative features. Models were externally validated on a diverse cohort of 970 patients (median follow-up 55 months) for accuracy in ODX prediction and recurrence. Comparing the area under the receiver operating characteristic curve (AUROC) in a held-out set from NCDB, models incorporating quantitative ER/PR (AUROC 0.78, 95% CI 0.77–0.80) and ER/PR/Ki-67 (AUROC 0.81, 95% CI 0.80–0.83) outperformed the non-quantitative model (AUROC 0.70, 95% CI 0.68–0.72). These results were preserved in the validation cohort, where the ER/PR/Ki-67 model (AUROC 0.87, 95% CI 0.81–0.93, p = 0.009) and the ER/PR model (AUROC 0.86, 95% CI 0.80–0.92, p = 0.031) significantly outperformed the non-quantitative model (AUROC 0.80, 95% CI 0.73–0.87). Using a high-sensitivity rule-out threshold, the non-quantitative, quantitative ER/PR and ER/PR/Ki-67 models identified 35%, 30% and 43% of patients as low-risk in the validation cohort. Of these low-risk patients, fewer than 3% had a recurrence at 5 years. These models may help identify patients who can forgo genomic testing and initiate endocrine therapy alone. An online calculator is provided for further study.