Train Machine Learning Models Research Articles

Federated learning enables privacy-preserving and data-efficient dimension prediction and part qualification across additive manufacturing factories

A crucial part of quality control in additive manufacturing (AM) is the decision to accept or reject parts based on their dimensional accuracy. Machine learning (ML) models can learn nuanced relationships between process parameters and resulting geometries; however, obtaining large quantities of supervised learning data to train ML models incurs non-trivial costs. Relevant measurement data may be available at other factories or stations within the same factory but cannot be pooled together for conventional centralized learning (CL) due to privacy constraints. Here, we propose federated learning (FL) that uses private data and information from distributed sources to train ML models that predict part dimensions and inform part qualification, thereby allowing collaborative model building without compromising privacy. We manufacture and measure 405 parts having three different designs and distribute them across three to 45 factories. The performance of FL is evaluated when factories produce similar parts, parts of different designs, and parts of different qualities. When factories produce similar parts or parts of different designs, FL predicts part geometry within 10 µm of the CL benchmark and within 15 µm of the process capability limit, with as few as nine parts at each factory. FL also outperforms individual learning where factories train on their own data by up to 96% in geometry prediction, thereby providing a win-win paradigm for privacy-preserving collaborative learning. When factories produce parts of different qualities, FL outperforms individual factories only when they produce less than 15 parts locally. We show that FL performance is influenced by two factors: the quantity of data in the federation and the statistical homogeneity of data across participating factories. Overall, this research demonstrates the promise of FL for privacy-preserving, data-efficient quality control and offers key insights to design data federations in scalable AM production.

An integrated computation framework for predicting mechanical performance of single-phase alloys manufactured using laser powder bed fusion: A case study of CoCrFeMnNi high-entropy alloy

Establishing a process-structure-property (PSP) relationship is crucial for understanding and optimizing the mechanical properties of components via laser powder bed fusion (L-PBF) additive manufacturing (AM). However, the parameters pertinent to the PSP relationship are encapsulated in a “black box” with a high dimension, resulting in relying on trial-and-error approaches to elucidate hidden associations becoming impracticable, while the acquisition of microscopic data proves financially burdensome. Thus this paper presents an integrated computation framework based on machine learning (ML) for predicting the tensile yield strengths and post-yield hardening rates of single-phase alloys fabricated through L-PBF, for efficient process optimization purposes. Research herein focuses on single-phase alloys to be built via L-PBF, taking CoCrFeMnNi high-entropy alloy as a study case, and provide an efficient integrated computation solution. The methodology proposed in this paper generates as-built microstructures associated with the varying L-PBF process parameters via cellular automata (CA). Subsequently, the mechanical responses of these microstructures are estimated by employing crystal plasticity finite element (CPFE) simulations. In addition, the approach raised in the paper leverages particle-level microstructure descriptors instead of average feature descriptors for the entire representative volume element (RVE) structure in an ML framework to predict the characteristics of the newly randomly growing grains. This research demonstrate the practicality and instructiveness of the methodology for modeling realistic grains. Then, trained ML model exhibits promising accuracy for the prediction of mechanical properties. This work showcases an integrated computation framework combining advanced materials science and ML techniques for enhancing understanding of the L-PBF process and offering valuable insights into process optimization for L-PBF-built diverse materials.

Review on Sanskrit Sandhi Splitting using Deep Learning Techniques

A procedure called sandhi is used in Sanskrit to join short words (morphemes) to create compound words. A composite word are broken down into their component morphemes by a process known as sandhi splitting. This study focuses on several performance technologies and methodologies used to perform the above operation on Sanskrit sentences. Various approaches were identified for the problem from the literature survey. Initial approaches involved use of Finite State Transducers. Earlier the approaches introduced to increase accuracy include use of mathematical models and various optimality theories. Graph based approaches and parser based techniques were introduced later. With the advancement of deep learning techniques Recurrent Neural Networks, Long-Short Term Memory models and Double decoder models were adopted which involved training machine learning models through neural networks and classifier algorithms. Bidirectional LSTM models with attention mechanism, transformer based models and large language models like BERT were the most recent methodologies adopted and proved to be of higher accuracy and performance.

Privacy-Centric AI: Navigating the Landscape with Federated Learning

Abstract: In the era of big data and privacy concerns, federated learning has emerged as a promising approach to training machine learning models while preserving data privacy. This paper explores the principles and applications of federated learning, highlighting its potential to revolutionize privacy-centric AI. We discuss the methodology, significance, and challenges of federated learning, providing insights into its future directions. By leveraging decentralized data and aggregating model updates, federated learning enables the development of powerful AI models without compromising individual privacy. We present real-world applications and cite relevant studies to demonstrate the transformative impact of federated learning across various domains.

Phishing Website Detection

Phishing websites have emerged as a serious security risk. Phishing is the starting point for many cyberattacks that compromise the confidentiality, integrity, and availability of customer and business data. Decades of effort have gone into developing novel methods for automatically identifying phishing websites. Modern systems aren't very adept at spotting new phishing threats and require a lot of manual feature engineering, even though they can produce better outcomes. Thus, an open problem in this discipline is to identify tactics that can swiftly handle zero-day phishing attempts and automatically recognize phishing websites. The web page that the URL hosts has a plethora of information that can be utilized to assess the maliciousness of the web server. One useful technique for spotting phishing emails is machine learning. Additionally, it does away with the drawbacks of the earlier approach. After a careful analysis of the literature, we proposed a novel approach that combines a machine learning algorithm with feature extraction to identify phishing websites. Using the gathered dataset, this study aims to train deep neural networks and machine learning models to detect phishing websites.

Unmasking The Enigma:Exploring Personality Traits Using Social Media Text

In this paper personality is an important parameter as it differentiates various individuals from another. Predicting personality has many applications in real world. The main objective of this project is to take textual data as input from the user and then run the trained machine learning model on this data to predict his four personality traits which are Introversion vs Extroversion, Sensing vs Intuition, Thinking vs Feeling, Judging vs Perceiving.The main objective is to build an application where users can answer the questions which are processed and analyzed to output his personality traits. The output is a string of four characters where each character determines a personality trait, total of sixteen personality types are possible.The Machine learning model Random Forest Classifier is used to classify the text and output four personality traits. Processing of large textual data is to be done using Natural Language Processing (NLP) techniques with the help of NLTK libraries to process and categorize the data. In order to increase the performance of the model hyper parameter tuning along with cross fold validations is done. Keywords: Random Forest, Natural Language Processing, Text

Detection and Classification of ChatGPT Generated Contents Using Deep Transformer Models

AI advancements, particularly in neural networks, have brought about groundbreaking tools like text generators and chatbots. While these technologies offer tremendous benefits, they also pose serious risks such as privacy breaches, spread of misinformation, and challenges to academic integrity. Previous efforts to distinguish between human and AI-generated text have been limited, especially with models like ChatGPT. To tackle this, we created a dataset containing both human and ChatGPT-generated text, using it to train and test various machine and deep learning models. Your results, particularly the high F1-score and accuracy achieved by the RoBERTa-based custom deep learning model and Distil BERT, indicate promising progress in this area. By establishing a robust baseline for detecting and classifying AI-generated content, your work contributes significantly to mitigating potential misuse of AI-powered text generation tools.

Analyzing the robustness of decentralized horizontal and vertical federated learning architectures in a non-IID scenario

AbstractFederated learning (FL) enables participants to collaboratively train machine and deep learning models while safeguarding data privacy. However, the FL paradigm still has drawbacks that affect its trustworthiness, as malicious participants could launch adversarial attacks against the training process. Previous research has examined the robustness of horizontal FL scenarios under various attacks. However, there is a lack of research evaluating the robustness of decentralized vertical FL and comparing it with horizontal FL architectures affected by adversarial attacks. Therefore, this study proposes three decentralized FL architectures: HoriChain, VertiChain, and VertiComb. These architectures feature different neural networks and training protocols suitable for horizontal and vertical scenarios. Subsequently, a decentralized, privacy-preserving, and federated use case with non-IID data to classify handwritten digits is deployed to assess the performance of the three architectures. Finally, a series of experiments computes and compares the robustness of the proposed architectures when they are affected by different data poisoning methods, including image watermarks and gradient poisoning adversarial attacks. The experiments demonstrate that while specific configurations of both attacks can undermine the classification performance of the architectures, HoriChain is the most robust one.

Revolutionizing AI-Assisted Education with Federated Learning: A Pathway to Distributed, Privacy-Preserving, and Debiased Learning Ecosystems

The majority of current research on the application of artificial intelligence (AI) and machine learning (ML) in science, technology, engineering, and mathematics (STEM) education relies on centralized model training architectures. Typically, this involves pooling data at a centralized location alongside an ML model training module, such as a cloud server. However, this approach necessitates transferring student data across the network, leading to privacy concerns. In this paper, we explore the application of federated learning (FL), a highly recognized distributed ML technique, within the educational ecosystem. We highlight the potential benefits FL offers to students, classrooms, and institutions. Also, we identify a range of technical, logistical, and ethical challenges that impede the sustainable implementation of FL in the education sector. Finally, we discuss a series of open research directions, focusing on nuanced aspects of FL implementation in educational contexts. These directions aim to explore and address the complexities of applying FL in varied educational settings, ensuring its deployment is technologically sound, beneficial, and equitable for all stakeholders involved.

Robust quantum federated learning with noise

Abstract Federated learning is a decentralized machine learning approach where multiple participants collaboratively train machine learning models. With the development of quantum computing, there has been significant potential in the integration of quantum computing and federated learning. However, existing research has demonstrated that, similar to classical federated learning models, quantum federated learning models also face various security threats and privacy leakage issues. This paper proposes a quantum federated learning model based on quantum noise. Adding quantum noise to the model not only addresses privacy leakage, but also enhances the model robustness, effectively resists adversarial attacks. Specifically, extensive numerical simulations are conducted using various datasets to evaluate the effectiveness of the proposed method. The results reveal a more pronounced variation in robust training in high-dimensional datasets compared to low-dimensional datasets. Furthermore, the impact of noise intensity on model robustness is explored. Experimental demonstrate that a small amount of quantum noise does not have a significant impact on accuracy, and as the noise increases, the robustness of the model also improves. Finally, three different types of quantum noise were used for robustness testing in the paper to analyze the impact of quantum noise on the robustness of quantum machine learning models. The abundant experimental results have verified that the noise can improve the security of distributed quantum machine learning.

DeepSP: Deep learning-based spatial properties to predict monoclonal antibody stability

Therapeutic antibody development faces challenges due to high viscosities and aggregation tendencies. The spatial charge map (SCM) and spatial aggregation propensity (SAP) are computational techniques that aid in predicting viscosity and aggregation, respectively. These methods rely on structural data derived from molecular dynamics (MD) simulations, which are computationally demanding. DeepSCM, a deep learning surrogate model based on sequence information to predict SCM, was recently developed to screen high-concentration antibody viscosity. This study further utilized a dataset of 20,530 antibody sequences to train a convolutional neural network deep learning surrogate model called Deep Spatial Properties (DeepSP). DeepSP directly predicts SAP and SCM scores in different domains of antibody variable regions based solely on their sequences without performing MD simulations. The linear correlation coefficient between DeepSP scores and MD-derived scores for 30 properties achieved values between 0.76 and 0.96 with an average of 0.87. DeepSP descriptors were employed as features to build machine learning models to predict the aggregation rate of 21 antibodies, and the performance is similar to the results obtained from the previous study using MD simulations. This result demonstrates that the DeepSP approach significantly reduces the computational time required compared to MD simulations. The DeepSP model enables the rapid generation of 30 structural properties that can also be used as features in other research to train machine learning models for predicting various antibody stability using sequences only. DeepSP is freely available as an online tool via https://deepspwebapp.onrender.com and the codes and parameters are freely available at https://github.com/Lailabcode/DeepSP.

AntiMicro.ai: An Artificial Intelligence powered web application for predicting antibacterial/antifungal susceptibility and constructing personalized machine learning models

Background To train and develop machine learning models on the Pfizer antibacterial and antifungal datasets with subsequent deployment to an interactive Web Application. Methods We utilized R version 4.3.1 to perform descriptive analysis to obtain features/predictors. Python 3.10 libraries NumPy, Pandas, Scikit learn, Pycaret were used to train machine learning models. All these models were scored using area under the curve, recall, precision, F1, Kappa and the Mathews correlation coefficient. The best performing model was then deployed into a web application built on Streamlit after which deployment was done using GitHub and Streamlit cloud. A prototype android app was also developed using GoNative. Results The exploratory data analysis, S Aureus (17.2%) was the most common species however in the sub group analysis of the isolates with genotypic values, K Pneumoniae(48.2%) and E Coli (36.9%) were dominant. Amongst the fungi, Candida albicans (38.3%) and Candida glabrata (15.5%) were dominant. Feature selection was done using Shapley additive explanation plots. Using Extreme Gradient Boosting (XGBoost), we were able to achieve 99% and 97.8% AUC in the prediction of antibacterial and antifungal susceptibility respectively with minimal overfitting. Conclusions Decision tree based methods are a viable options of predicting antibacterial and antifungal drug resistance. When presented in visually appealing modes like web applications and android apps, it can be a useful guide to clinicians on initial treatment while awaiting definitive phenotypic testing. It can also be a surveillance tool that can craft antimicrobial resistance strategies.

Performance Evaluation of a Supervised Machine Learning Pain Classification Model Developed by Neonatal Nurses.

Early-life pain is associated with adverse neurodevelopmental consequences; and current pain assessment practices are discontinuous, inconsistent, and highly dependent on nurses' availability. Furthermore, facial expressions in commonly used pain assessment tools are not associated with brain-based evidence of pain. To develop and validate a machine learning (ML) model to classify pain. In this retrospective validation study, using a human-centered design for Embedded Machine Learning Solutions approach and the Neonatal Facial Coding System (NFCS), 6 experienced neonatal intensive care unit (NICU) nurses labeled data from randomly assigned iCOPEvid (infant Classification Of Pain Expression video) sequences of 49 neonates undergoing heel lance. NFCS is the only observational pain assessment tool associated with brain-based evidence of pain. A standard 70% training and 30% testing split of the data was used to train and test several ML models. NICU nurses' interrater reliability was evaluated, and NICU nurses' area under the receiver operating characteristic curve (AUC) was compared with the ML models' AUC. Nurses weighted mean interrater reliability was 68% (63%-79%) for NFCS tasks, 77.7% (74%-83%) for pain intensity, and 48.6% (15%-59%) for frame and 78.4% (64%-100%) for video pain classification, with AUC of 0.68. The best performing ML model had 97.7% precision, 98% accuracy, 98.5% recall, and AUC of 0.98. The pain classification ML model AUC far exceeded that of NICU nurses for identifying neonatal pain. These findings will inform the development of a continuous, unbiased, brain-based, nurse-in-the-loop Pain Recognition Automated Monitoring System (PRAMS) for neonates and infants.

SchNetPack Hyperparameter Optimization for a More Reliable Top Docking Scores Prediction.

Options to improve the extrapolation power of the neural network designed using the SchNetPack package with respect to top docking scores prediction are presented. It is shown that hyperparameter tuning of the atomistic model representation (in the schnetpack.representation) improves the prediction of the top scoring compounds, which have characteristically a low incidence in randomized data sets for training of machine learning models. The prediction robustness is evaluated according to the mean square error (MSE) and the entropy of the average loss landscape decrease. Admittedly, the improvement of the top scoring compounds' prediction accuracy comes with the penalty of worsening the overall prediction power. It is revealed that the most impactful hyperparameter is the cutoff (5 Å is reported as the optimal choice). Other parameters (e.g., number of radial basis functions, number of interaction layers of the neural network, feature vector size or its batch size) are found to not affect the prediction robustness of the top scoring compounds in any comparable way relative to the cutoff. The MSE of the best docking score prediction (below -13 kcal/mol) improves from ca. 3.5 to 0.9 kcal/mol, while the prediction of less potent compounds (-13 to -11 kcal/mol) shows a lesser improvement, i.e., a decrease of MSE from 1.6 to 1.3 kcal/mol. Additionally, oversampling and undersampling of the training set with respect to the top scoring compounds' abundance is presented. The results indicate that the cutoff choice performs better than over- or undersampling of the training set, with undersampling performing better than oversampling.

Dynamic Playlist Generation using Machine Learning

This study investigates the application of machine learning algorithms for dynamic playlist generation in the context of music recommendation systems. The traditional method of creating playlists frequently depends on immutable standards, like artist or genre, which might not adequately represent the unique and dynamic character of personal preferences. On the other hand, our suggested system makes use of machine learning methods to examine user behavior, preferences, and contextual elements in order to create playlists that are dynamically created and customized to each user’s individual preferences. In order to train machine learning models, the study collects user interaction data, such as listening history, skip patterns, and user feedback. To extract patterns and relationships from the data, a variety of algorithms are used, including hybrid models, content-based filtering, and collaborative filtering. After that, the models are incorporated into a dynamic playlist creation system that can eventually adjust to changing user preferences.Our test findings show how well the suggested strategy works to improve user experience by offering more interesting and customized playlists. With its ability to adjust to changing user preferences and contextual cues, the dynamic playlist generation system provides a smooth and pleasurable music discovery experience. We also talk about possible enhancements, implementation difficulties, and deployment considerations in the real world.This work adds to the ongoing efforts to improve music recommendation systems by demonstrating how machine learning can be used to develop more responsive and intelligent playlist generation systems. The results highlight how crucial customized experiences are in the constantly changing world of digital music consumption. Index Terms—hybrid mode, content-based filtering, intelligent playlist, music

Machine Learning Library in C++

This project aims to develop a comprehensive and versatile machine learning library in C++ tailored to address the diverse needs of developers and researchers in the field. The library encompasses a robust set of core machine learning algorithms, encompassing supervised, unsupervised, and reinforcement learning techniques. Additionally, it incorporates essential data preprocessing tools to streamline data manipulation and feature engineering tasks, along with model evaluation capabilities crucial for assessing algorithm performance. The library's primary focus is on providing a rich suite of machine learning algorithms, This empowers users to effectively prepare data for training machine learning models. Additionally, the library provides tools for data splitting, and model evaluation to ensure reliable and robust model performance assessment.

SYN-GAN: A robust intrusion detection system using GAN-based synthetic data for IoT security

As technological communication progresses, diverse datasets are exchanged across distributed environments using the Internet of Things (IoT). However, the IoT environment is vulnerable to attacking and breaching data privacy or making a robust system worse by providing attack data. To address potential risks of attacks, researchers have been conducting experiments on network intrusion detection systems (NIDS) to mitigate threats effectively. The issue of data imbalance and associated data collection costs persists, hindering the ability of machine learning (ML) models to learn malicious behaviour effectively and consequently impacting the accuracy of network threat detection. Addressing these issues, our study explores the potential of using 100% synthetic data generated via Generative Adversarial Networks (GAN) for training ML models in Network Intrusion Detection Systems (NIDS). This approach reduces the dependency on real-world data significantly, paving the way for a more flexible and ethically convenient model-building process. For the UNSW-NB15 dataset, we achieved an accuracy of 90%, a precision of 91%, a recall of 90%, and an F1 score of 89%. For the NSL-KDD dataset, our results showed an accuracy of 84%, a precision of 85%, a recall of 84%, and an F1 score of 84%. For the BoT-IoT dataset, we attained perfect scores of 100% across all metrics. These outcomes indicate that the values obtained from our analysis demonstrate high performance, yielding comparative or superior results to previous studies. Therefore, our study successfully replicates real-world network intrusion detection data, showing new opportunities for the use of generative data in cyber security.

Multimodal emotion classification using machine learning in immersive and non-immersive virtual reality

Affective computing has been widely used to detect and recognize emotional states. The main goal of this study was to detect emotional states using machine learning algorithms automatically. The experimental procedure involved eliciting emotional states using film clips in an immersive and non-immersive virtual reality setup. The participants’ physiological signals were recorded and analyzed to train machine learning models to recognize users’ emotional states. Furthermore, two subjective ratings emotional scales were provided to rate each emotional film clip. Results showed no significant differences between presenting the stimuli in the two degrees of immersion. Regarding emotion classification, it emerged that for both physiological signals and subjective ratings, user-dependent models have a better performance when compared to user-independent models. We obtained an average accuracy of 69.29 ± 11.41% and 71.00 ± 7.95% for the subjective ratings and physiological signals, respectively. On the other hand, using user-independent models, the accuracy we obtained was 54.0 ± 17.2% and 24.9 ± 4.0%, respectively. We interpreted these data as the result of high inter-subject variability among participants, suggesting the need for user-dependent classification models. In future works, we intend to develop new classification algorithms and transfer them to real-time implementation. This will make it possible to adapt to a virtual reality environment in real-time, according to the user’s emotional state.

Audio Recognition of the Percussion Sounds Generated by a 3D Auto-Drum Machine System via Machine Learning

Audio Recognition of the Percussion Sounds Generated by a 3D Auto-Drum Machine System via Machine Learning

Approach to Detecting Malicious Bots in the Vkontakte Social Network and Assessing Their Parameters

The emergence of new varieties of bots in social networks and the improvement of their capabilities to imitate the natural behavior of real users represent a significant problem in the field of protection of social networks and online communities. This paper proposes a new approach to detecting and assessing the parameters of bots within the social network «VKontakte». The basis of the proposed approach is the creation of datasets using the method of «controlled purchase» of bots, which allows one to assess bots’ characteristics such as price, quality, and speed of action of bots, and using the Turing Test to assess how much users trust bots. In combination with traditional machine learning methods and features extracted from interaction graphs, text messages, and statistical distributions, it becomes possible to not only detect bots accurately but also predict their characteristics. This paper demonstrates that the trained machine learning model, based on the proposed approach, is robust to imbalanced data and can identify most types of bots as it has only a minor correlation with their main characteristics. The proposed approach can be used within the choice of countermeasures for the protection of social networks and for historical analysis, which allows not only to confirm the presence of bots but also to characterize the specifics of the attack.