Train Machine Learning Models Research Articles

Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach

Artificial intelligence (AI), and especially its sub-field of Machine Learning (ML), are impacting the daily lives of everyone with their ubiquitous applications. In recent years, AI researchers and practitioners have introduced principles and guidelines to build systems that make reliable and trustworthy decisions. From a practical perspective, conventional ML systems process historical data to extract the features that are consequently used to train ML models that perform the desired task. However, in practice, a fundamental challenge arises when the system needs to be operationalized and deployed to evolve and operate in real-life environments continuously. To address this challenge, Machine Learning Operations (MLOps) have emerged as a potential recipe for standardizing ML solutions in deployment. Although MLOps demonstrated great success in streamlining ML processes, thoroughly defining the specifications of robust MLOps approaches remains of great interest to researchers and practitioners. In this paper, we provide a comprehensive overview of the trustworthiness property of MLOps systems. Specifically, we highlight technical practices to achieve robust MLOps systems. In addition, we survey the existing research approaches that address the robustness aspects of ML systems in production. We also review the tools and software available to build MLOps systems and summarize their support to handle the robustness aspects. Finally, we present the open challenges and propose possible future directions and opportunities within this emerging field. The aim of this paper is to provide researchers and practitioners working on practical AI applications with a comprehensive view to adopt robust ML solutions in production environments.

Annotated interictal discharges in intracranial EEG sleep data and related machine learning detection scheme

Interictal epileptiform discharges (IEDs) such as spikes and sharp waves represent pathological electrophysiological activities occurring in epilepsy patients between seizures. IEDs occur preferentially during non-rapid eye movement (NREM) sleep and are associated with impaired memory and cognition. Despite growing interest, most studies involving IED detections rely on visual annotations or employ simple amplitude threshold approaches. Alternatively, advanced computerized detection methods are not standardized or publicly available. To address this gap, we introduce a novel dataset comprising multichannel intracranial electroencephalography (iEEG) data recorded at two medical centers during overnight sleep with IED annotations performed by expert neurologists. Utilizing these annotations to train machine learning models via a gradient-boosting algorithm, we demonstrate automated IED detection with high precision (94.4%) and sensitivity (94.3%) that can generalize across individuals and surpass performance of a leading commercial software. The dataset featuring multi-channel annotations with sub-second resolution including hippocampus and medial temporal lobe (MTL) regions is made publicly available, together with the detection algorithm, to advance research on detection methodology, epilepsy, sleep, and cognition.

How Users' Personality Traits Predict Sentiment Tendencies of User-Generated Content in Social Media: A Mixed Method of Configuration Analysis and Machine Learning.

Social media content created by users with different personality traits presents various sentiment tendencies, easily leading to irrational public opinion. This study aims to explore the relationships between users' personality traits and sentiment tendencies of user-generated content (UGC). We crawled 18,686 tweets of 1, 215 users from Twitter to figure out the relationships between personality traits and sentiment tendencies. This study utilizes Essays and Sentiment datasets to train machine learning models for the identification of personality traits and sentiment tendencies and then explores the configuration effect of personality traits on sentiment tendency via crisp-set Qualitative Comparative Analysis (csQCA). The findings suggest that (1) one-dimensional personality trait is not a necessary condition for the sentiment tendencies of UGC. (2) There are multiple equivalent configurations that lead to the sentiment tendencies of UGC. The study suggests that the sentiment tendencies pattern of UGC can be discovered via the configurations of various dimensions of personality traits.

Evaluation of artificial intelligence and optical image recognition techniques used in OneIso, an off-axis Winston-Lutz quality assurance phantom.

Single-isocenter multitarget (SIMT) stereotactic-radiosurgery (SRS) has recently emerged as a powerful treatment regimen for intracranial tumors. With high specificity, SIMT SRS allows for rapid, high-dose delivery while maintaining integrity of adjacent healthy tissues and minimizing neurocognitive damage to patients. Highly robust and accurate quality assurance (QA) tests are critical to minimize off-targets and damage to surrounding healthy tissues. We have developed a novel QA phantom, named OneIso, to accurately and precisely measure off-axis accuracy, via off-axis Winston-Lutz (OAWL), to assist SIMT SRS programs. In this study, a comparison of three different quantitative numerical methods were performed for isolating and measuring the position of ball-bearings (BBs) used in the OAWL measurement. The three methods evaluated were: 1) feature extraction technique combined with manual intervention 2) a proprietary software utilizing optical image recognition (OIR) techniques, and 3) a machine learning (ML) model employing convolutional neural networks (CNNs). These methods were used to analyze OAWL datasets gathered from a OneIso phantom deployed on a Varian TrueBeam. The precision of localizing positional BBs within the OneIso QA phantom, analysis speed, and robustness were compared across the methods. Significantly, the trained ML model utilizing CNNs was found to exhibit superior precision, analysis speed, and efficiency compared to the other two methods. These results highlight the benefit in shifting from manual and OIR methods to that of ML techniques. The incorporation of CNNs in automated QA analysis can achieve improved precision, allowing for more rapid and wider adoption of SIMT SRS for treating intracranial metastases while preserving integrity of surrounding healthy tissues.

Dynamic SelectOut and voting-based federated learning for enhanced medical image analysis

Abstract Federated Learning (FL) is a promising technique for training machine learning models on distributed, privacy-aware datasets. Nevertheless, FL faces difficulties with agent/client participation, model performance, and the heterogeneous nature of networked data sources when it comes to distributed healthcare systems. When these agents work together in the system, it is imperative to tackle the complexities of distributed deep learning. We suggest a novel approach that uses a voting mechanism and dynamic SelectOut inside the FL framework to address these problems. Local medical imaging datasets frequently show diversity in distribution and data imbalances. In certain situations, traditional FL techniques like FedProx and Federated Averaging (FedAvg), which depend on data size to weight contributions, might not be the optimal choice. We improve parameter aggregation and client selection unpredictability and increase the model's adaptability to imbalanced and heterogeneous datasets, our proposed FedVoteNet model introduces SelectOut techniques based on Voting methodology. Based on how much their local performance has improved from the last communication cycle, we arbitrarily remove clients. Additionally eliminated are clients whose model weights when combined with the global model adversely affect its performance. After each communication cycle, clients that improve both their local performance and their contribution to the global model are awarded higher voting values. This encourages more significant and effective contributions from clients by providing incentives for them to actively increase the diversity of their training data. We assess our approach on a dataset of medical images, including MRI scans, and find that the FL model performs noticeably better (F1 Score=0.968, Sensitivity=0.977, Specificity=0.945, and AUC=0.950). The voting system and the dynamic SelectOut algorithms improve the convergence of the FL model and successfully handle the difficulties presented by uneven and heterogeneous datasets. To sum up, our proposed approach uses voting and dynamic SelectOut techniques to improve FL performance on a variety of uneven, distributed, and varied datasets. This strategy has a lot of potential to improve FL across a range of applications, especially those that prioritize data privacy, diversity, and performance.

Backdoor Attack Against Dataset Distillation in Natural Language Processing

Dataset distillation has become an important technique for enhancing the efficiency of data when training machine learning models. It finds extensive applications across various fields, including computer vision (CV) and natural language processing (NLP). However, it essentially consists of a deep neural network (DNN) model, which remain susceptible to security and privacy vulnerabilities (e.g., backdoor attacks). Existing studies have primarily focused on optimizing the balance between computational efficiency and model performance, overlooking the accompanying security and privacy risks. This study presents the first backdoor attack targeting NLP models trained on distilled datasets. We introduce malicious triggers into synthetic data during the distillation phase to execute a backdoor attack on downstream models trained with these data. We employ several widely used datasets to assess how different architectures and dataset distillation techniques withstand our attack. The experimental findings reveal that the attack achieves strong performance with a high (above 0.9 and up to 1.0) attack success rate (ASR) in most cases. For backdoor attacks, high attack performance often comes at the cost of reduced model utility. Our attack maintains high ASR while maximizing the preservation of downstream model utility, as evidenced by results showing that the clean test accuracy (CTA) of the backdoored model is very close to that of the clean model. Additionally, we performed comprehensive ablation studies to identify key factors affecting attack performance. We tested our attack method against five defense strategies, including NAD, Neural Cleanse, ONION, SCPD, and RAP. The experimental results show that these defense methods are unable to reduce the attack success rate without compromising the model’s performance on normal tasks. Therefore, these methods cannot effectively defend against our attack.

Classification and prediction of heart disease using Machine Learning models: A promising approach for medical diagnosis

Background: heart disease is one of the leading causes of death worldwide, claiming 17.9 million lives. They are a major public health problem that affects people regardless of age or gender. Objective: This work aims to classify and predict heart disease using Machine Learning (ML) models such as Random Forest (RF), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Naive Bayes (NB), Decision Tree (DT) and Logistic Regression (LR). Methods: We worked with the Cleveland dataset from Kaggle, consisting of 303 patient records and 14 attributes. This research was conducted in different stages, including model understanding, dataset analysis and cleaning, ML model training, and model performance evaluation. Results: The results showed that the RF and KNN models achieved the highest levels of performance and accuracy with 88.52%, surpassing the other models such as SVM, NB, and LR which obtained 86.89% accuracy, and DT with 78.69%. Conclusions: In conclusion, the RF and KNN models stand out over the other models for this type of prediction task.

Machine learning‐assisted performance analysis of organic photovoltaics

AbstractAlthough the power conversion efficiency of organic solar cells (OSCs) has been rapidly improved, there is still a lot of room for designing and developing new materials and their combinations to approach the efficiency limit. In this work, we establish a database of ∼100 bulk heterojunction OSCs composed of representative donors and acceptors reported in the literature, and train machine learning models to identify the efficiency potential of donor‐acceptor combinations. We find that the fully connected neural network achieves a Pearson coefficient of up to 0.88 for predicting the efficiency of OSCs with different combinations of donors and acceptors. We use sure independence screening and sparsifying method with feature analysis to analyze and evaluate the performance of OSCs. To prove the reliability and viability of the predictive model, we introduce the theoretical efficiency limits and confidence tests into the process, which provides a simple but reliable solution to quickly analyze and evaluate the potential of OSC materials and material combinations.

Streamflow simulation at different temporal scales under rating curve uncertainty conditions using machine learning models

ABSTRACT Reducing uncertainty in streamflow simulation is vital for effective water resource management. The impact of uncertainty in model calibration data (discharge), commonly derived from the rating curve, is often overlooked. This study applies the Monte Carlo simulation technique (MCST) to assess uncertainty in the rating curve. Advanced machine learning (ML) models, bidirectional long short-term memory (BiLSTM), and bidirectional gated recurrent units (BiGRUs) were used comparatively to evaluate the propagation of this uncertainty onto streamflow simulation on both daily and monthly temporal scales. Different sets of streamflow data, derived from the fitted curve and its lower and upper uncertainty bands were utilized to train ML models independently. The results show the substantial impact of rating curve uncertainty in streamflow simulations, with the BiGRU model surpassing the BiLSTM model on both scales. As a result, the uncertainty in the rating curve results in an uncertainty of the streamflow of up to 30 and 25% on daily and monthly simulations, respectively. These findings underscore the importance of considering rating curve uncertainty in streamflow simulation to ensure accurate and reliable results. Therefore, streamflow should be treated as an uncertain variable and managed by incorporating rating curve uncertainty in decision-making.

A Green Multi-Attribute Client Selection for Over-The-Air Federated Learning: A Grey-Wolf-Optimizer Approach

Federated Learning (FL) has gained attention across various industries for its capability to train machine learning models without centralizing sensitive data. While this approach offers significant benefits such as privacy preservation and decreased communication overhead, it presents several challenges, including deployment complexity and interoperability issues, particularly in heterogeneous scenarios or resource-constrained environments. Over-the-air (OTA) FL was introduced to tackle these challenges by disseminating model updates without necessitating direct device-to-device connections or centralized servers. However, OTA-FL brought forth limitations associated with heightened energy consumption and network latency. In this paper, we propose a multi-attribute client selection framework employing the grey wolf optimizer (GWO) to strategically control the number of participants in each round and optimize the OTA-FL process while considering accuracy, energy, delay, reliability, and fairness constraints of participating devices. We evaluate the performance of our multi-attribute client selection approach in terms of model loss minimization, convergence time reduction, and energy efficiency. In our experimental evaluation, we assessed and compared the performance of our approach against the existing state-of-the-art methods. Our results demonstrate that the proposed GWO-based client selection outperforms these baselines across various metrics. Specifically, our approach achieves a notable reduction in model loss, accelerates convergence time, and enhances energy efficiency while maintaining high fairness and reliability indicators.

Quality Assessment of MRI-Radiomics-Based Machine Learning Methods in Classification of Brain Tumors: Systematic Review.

Brain tumors present a complex challenge in clinical oncology, where precise diagnosis and classification are pivotal for effective treatment planning. Radiomics, a burgeoning field in neuro-oncology, involves extracting and analyzing numerous quantitative features from medical images. This approach captures subtle spatial and textural information imperceptible to the human eye. However, implementation in clinical practice is still distant, and concerns have been raised regarding the methodological quality of radiomic studies. A systematic literature search was performed to identify original articles focused on the use of radiomics for brain tumors from 2015 based on the inclusion and exclusion criteria. The radiomic features train machine learning models for glioma classification, and data are split into training and testing subsets to validate the model accuracy, reliability, and generalizability. The present study systematically reviews the status of radiomic studies concerning brain tumors, also using the radiomics quality score (RQS) to assess the quality of the methodology used in each study. A systematic search of PubMed identified 300 articles, with 18 studies meeting the inclusion criteria for qualitative synthesis. These studies collectively demonstrate the potential of radiomics-based machine learning models in accurately distinguishing between glioma subtypes and grades. Various imaging modalities, including MRI, PET/CT, and advanced techniques like ASL and DTI, were utilized to extract radiomic features for analysis. Machine learning algorithms such as deep learning networks, support vector machines, random forests, and logistic regression were applied to develop predictive models. The present study indicates high accuracies in glioma classification, outperforming traditional imaging methods and inexperienced radiologists in some cases. Further validation and standardization efforts are warranted to facilitate the clinical integration of radiomics into routine practice, ultimately enhancing glioma management and patient outcomes. Open science practices: Machine learning using MRI radiomic features provides a simple, noninvasive, and cost-effective method for glioma classification, enhancing transparency, reproducibility, and collaboration within the scientific community.

Appraising the porphyry Cu fertility using apatite trace elements: A machine learning method

Apatite chemical composition have been often invoked for appraising the magmatic copper (Cu) fertility, because trace elements in apatite hold important clues for tracing magma composition, oxidation states, and crystallization processes. However, low-dimensional Cu fertility discriminants developed on apatite trace elements suffer from significant limitations and uncertainties in practice. Here, machine learning (ML) models including random forests and support vector machines were trained using high-dimensional apatite trace elements dataset for discriminating ore-bearing magmas from ore-barren magmas. The results suggest that the ML models obtained a higher accuracy (96 %) for identifying the given apatites from ore-bearing samples compared to that of traditional discriminant diagrams (56 %). The feature importance analysis suggests that δEu and Sr are the most significant proxy for distinguishing ore-bearing and ore-barren samples when using machine learning model. In general, apatites from ore-bearing intrusion have higher δEu and Sr concentrations than ore-barren samples. Specifically, the high δEu and Sr concentrations indicate a relatively higher oxidation of parental magmas and rich in water, which can promote sulfate formation and Cu release. Moreover, the higher Sr/Y and δEu values observed in apatite from ore-bearing samples imply that the ore-productive magmas are adakites with high oxygen fugacity. We finally applied the trained ML model to the apatites from the Tampakan district of the Philippines, which provide new insights on Cu fertility of pre-, syn- and post-ore intrusions. The general applicability of this model demonstrates that ML-based discriminants developed on mineral trace element data provide new powerful tools for appraising the porphyry Cu fertility.

VA-Creator—A Virtual Appliance Creator based on adaptive Neural Networks to generate synthetic power consumption patterns

With the advent of the Smart Home domain and the increasingly widespread application of Machine Learning (ML), obtaining power consumption data is becoming more and more important. Collecting real-world energy data using sensors is time consuming, expensive, error-prone and in some situations not possible. Therefore, we present the VA-Creator, a framework to create Virtual Appliances (VAs). These VAs synthesize power consumption patterns (PCPs) based on Neural Networks (NNs) which adapt their architecture to the training data structure to simplify the creation of new VAs. To be able to generate all appliance types available in a typical household we use various kinds of NN, including Multilayer Perceptrons (MLPs), Long Short-Term Memorys (LSTMs) and a specific Generative Adversarial Network (GAN) as well as different ML techniques such as XGBoost, selecting the appropriate technique depending on each appliance’s characteristics. We then compare the results of the ML models against real data and evaluate them by using Dynamic time Warping (DTW) as well as the classification performance of an MLP descriminator as metrics. Additionally, to ensure that the VAs allow to meaningfully train ML models, we use them to generate synthetic data and then train Non intrusive Load Monitoring (NILM) models in an extensive evaluation. The presented evaluation provides evidence that the VA models produce realistic and meaningful results.

Physics-coupled Data-driven Design of High-Temperature Alloys

We present a materials design loop, which streamlines physics-coupled machine learning (ML) surrogate models to discover new alloy chemistries with improved properties. The efficacy is demonstrated by discovering a high-temperature alumina-forming austenitic (AFA) stainless steel with enhanced creep, followed by experimental validation. The ML models have been trained using a well-curated, highly consistent experimental dataset augmented with synthetic microstructural features from a computational thermodynamic approach. We have populated a large number of hypothetical AFA alloys to explore the high-dimensional composition space and have predicted their creep properties by providing the same synthetic input features obtained from the trained ML models. Uncertainties from the ML training were taken as thresholds for truncating predicted results to identify alloys with improved or deteriorated creep. Individual elemental compositions have been determined via probability density distribution analysis from the group of alloys at the top and bottom of the predicted creep values for further virtual and experimental validations. We anticipate that this workflow can be applied to screen desired conditions, such as chemistry and processing parameters, in high-dimensional space through physics-guided data analytics.

Predicting hepatitis C infection with machine learning algorithms: a prospective study

<p>Globally, chronic hepatitis C virus (HCV) infection affects millions of people and leads to a high number of deaths annually. In 2019, the World Health Organization (WHO) recorded around 290,000 deaths related to HCV, a virus transmitted mainly through blood that causes liver damage. The virus has infected more than 169 million people worldwide. This study aims to compare the performance of machine learning (ML) models for HCV detection. ML models such as logistic regression (LR), random forest (RF), decision tree (DT), and catBoost classifier (CATBC) were used. To carry out this task, a dataset of 615 patient records, and 14 variables were used. This research process was carried out in multiple phases, encompassing model understanding, data analysis and cleaning, ML model training, and subsequent model evaluation. The results revealed that the gradient boosting (GB) model stood out by achieving the best performance and highest accuracy, achieving a rate of 94% in HCV detection, this demonstrates outstanding performance compared to the other models such as LR, RF, k-nearest neighbor (KNN), DT, and CATBC, which obtained accuracy rates of 89%, 93%, 85%, 93%, 93%, and 92%, respectively. It can be concluded that the GB model stands out as the best algorithm for this task.</p>

AI Based Hazardous Asteroid Detection

As the need for early detection and mitigation of potential threats from near-Earth objects grows, this study presents a comprehensive approach to predicting hazardous asteroids using machine learning techniques. Given the increasing importance of protecting our planet from potential impact events, accurately classifying and predicting hazardous asteroids is cru- cial. The study proposes a comprehensive system for predicting and mitigating asteroid hazards using advanced data processing, machine learning, and simulation techniques. The proposed study collect real-time and historical asteroid data, process and clean this data, extract relevant features, and train machine learning models to predict potentially hazardous asteroids. Additionally, Monte Carlo simulations are employed to assess the collision risks of these asteroids with Earth. A forecasting module predicts future asteroid behavior based on historical and simulated data. The system also includes an alert mechanism that notifies relevant stakeholders when a potentially hazardous asteroid is detected

Training machine learning models to detect rare inborn errors of metabolism (IEMs) based on GC–MS urinary metabolomics for diseases screening

Training machine learning models to detect rare inborn errors of metabolism (IEMs) based on GC–MS urinary metabolomics for diseases screening

Non-invasive and continuous monitoring of 3D stem cell culture in a bioreactor

Abstract The development and discovery of new drugs and therapy products in regenerative medicine are carried out through multiple stages which involve several strict quality protocols. The use of cell-based model systems is the most relevant alternative to deviate from animal testing. However, the underlying biomanufacturing process in a 3D environment can be prone to failures or unexpected behaviors such as cell growth anomalies or inconsistencies in cell aggregation, which can seriously affect the quality of the result. Therefore, it is necessary to establish a non-invasive and continuous monitoring mechanism that minimizes contamination risks, allows taking early decisions and enables large-scale production in the long term. The results of our embedded machine learning approach show that it is feasible to train machine learning models that can operate on a resourcelimited embedded board with acceptable prediction accuracy for the estimation of the average size of growing stem cell spheroids. Further experiments are needed to investigate the full information potential of the recorded process data.

Advanced Machine Learning Approaches for Multi-Class Lung Disease Classification Using Respiratory Acoustics

This study introduces a novel method for the multi-class classification of lung diseases using respiratory sounds. Lung diseases represent a major global health issue, and early detection is essential for effective treatment and management. Traditional diagnostic techniques often involve specialized equipment and expertise, which can limit accessibility and increase costs. In this research, we propose a machine learning-based approach that utilizes respiratory sounds, which can be easily collected with affordable, non-invasive devices like digital stethoscopes or smartphone apps. Our method involves extracting features from respiratory sound signals and training machine learning models to classify various lung conditions, including pneumonia, asthma, chronic obstructive pulmonary disease (COPD), and bronchiectasis. We validate our approach using realworld respiratory sound datasets and evaluate its performance based on accuracy, sensitivity, and specificity. The results highlight the potential of using respiratory sounds for multi-class lung disease classification, providing an effective solution for early diagnosis and remote monitoring, especially in resource-constrained environments. This research contributes to the evolution of digital healthcare technologies, offering significant potential for enhancing respiratory disease management and improving patient outcomes

Word Sense Disambiguation using NLP

Word Sense Disambiguation (WSD) is a critical task in Natural Language Processing (NLP) aimed at determining the correct meaning of a word based on its context within a text. We categorize WSD techniques into three main paradigms: knowledge-based methods, supervised learning approaches, and neural network-based models. Knowledge-based methods leverage lexical resources like WordNet and other semantic networks to disambiguate word senses by comparing context with predefined sense definitions. These methods often rely on similarity measures and heuristic rules but may struggle with the flexibility and variability of natural language. Supervised learning approaches utilize annotated corpora to train machine learning models that predict word senses. These methods, including decision trees, support vector machines, and ensemble techniques, have shown significant improvements with the advent of large-scale labelled datasets and feature engineering. Keywords: Lexical Semantics, Sense Inventory, Knowledge- based WSD, Contextual Disambiguation