Machine Learning Framework Research Articles

Decentralized collaborative machine learning for protecting electricity data

In recent years, there has been a noticeable surge in electric power load due to economic development and improved living standards. The growing need for smart power solutions, such as leveraging user electricity data to forecast power peaks and utilizing power data statistics to enhance end-user services, has been on the rise. However, the misuse and unauthorized access of data have prompted stringent regulations to safeguard data integrity. This paper presents a novel decentralized collaborative machine learning framework aimed at predicting peak power loads while protecting the privacy of users’ power data. In this scheme, multiple users engage in collaborative machine learning training within a peer-to-peer network free from a centralized server, with the objective of predicting peak power loads without compromising users’ local data privacy. The proposed approach leverages blockchain technology and advanced cryptographic techniques, including multi-key homomorphic encryption and consistent hashing. Key contributions of this framework include the development of a secure dual-aggregate node aggregation algorithm and the establishment of a verifiable process within a decentralized architecture. Experimental validation has been conducted to assess the feasibility and effectiveness of the proposed scheme, demonstrating its potential to address the challenges associated with predicting peak power loads securely and preserving user data privacy.

Exploring the correlation between DNA methylation and biological age using an interpretable machine learning framework

DNA methylation plays a significant role in regulating transcription and exhibits a systematic change with age. These changes can be used to predict an individual’s age. First, to identify methylation sites associated with biological age; second, to construct a biological age prediction model and preliminarily explore the biological significance of methylation-associated genes using machine learning. A biological age prediction model was constructed using human methylation data through data preprocessing, feature selection procedures, statistical analysis, and machine learning techniques. Subsequently, 15 methylation data sets were subjected to in-depth analysis using SHAP, GO enrichment, and KEGG analysis. XGBoost, LightGBM, and CatBoost identified 15 groups of methylation sites associated with biological age. The cg23995914 locus was identified as the most significant contributor to predicting biological age by calculating SHAP values. Furthermore, GO enrichment and KEGG analyses were employed to initially explore the methylated loci’s biological significance.

Early quality prediction of complex double-walled hollow turbine blades based on improved whale optimization algorithm

Abstract The precision in forming complex double-walled hollow turbine blades significantly influences their cooling efficiency, making the selection of appropriate casting process parameters critical for achieving fine-casting blade formation. However, the high cost associated with real blade casting necessitates strategies to enhance product formation rates and mitigate cost losses stemming from the overshoot phenomenon. To address this challenge, we propose a machine-learning (ML) data-driven framework leveraging an enhanced whale optimization algorithm (WOA) to estimate product formation under diverse process conditions. Complex double-walled hollow turbine blades serve as a representative case within our proposed framework. We constructed a database using simulation data, employed feature engineering to identify crucial features and streamline inputs, and utilized a whale optimization algorithm-back-propagation neural network (WOA-BP) as the foundational ML model. To enhance WOA-BP's performance, we introduce an optimization algorithm, ICWOA-BP, incorporating cubic chaotic mapping adaptation. Experimental evaluation of ICWOA-BP demonstrated an average mean absolute error (MAE) of 0.001995 mm, reflecting a 36.21% reduction in prediction error compared to conventional models, as well as two well-known optimization algorithms (PSO, QANA). Consequently, ICWOA-BP emerges as an effective tool for early prediction of dimensional quality in complex double-walled hollow turbine blades.

Elemental diffusion coefficient prediction in conventional alloys using machine learning

This paper presents the Machine Learned Diffusion Coefficient Estimator, a comprehensive machine learning framework designed to predict diffusion coefficients in impure metallic (IM) and multi-component alloy (MCA) media. The framework incorporates five machine learning models, each tailored to specific diffusion modes: (1) impurity and (2) self-diffusion in IM media, and (3) self, (4) impurity, and (5) chemical diffusion in MCA media. These models use statistical aggregations of atomic descriptors for both the diffusing elements and the diffusion media, along with the temperature of the diffusion process, as features. Models are trained using the random forest and deep neural network algorithms, with performance evaluated through the coefficient of determination (R2), mean squared error (MSE), and uncertainty estimates. The models within this framework achieve an impressive R2 score above 0.90 with MSE less than 10−16 m2/s, demonstrating high predictive accuracy and reliability for diffusion coefficient.

An alloy-agnostic machine learning framework for process mapping in laser powder bed fusion

PurposeThis study aims to introduce an image-based method to determine the processing window for a given alloy system using laser powder bed fusion equipment based on achieving the desired melting mode across multiple materials for powder-free specimens. The method uses a convolutional neural network trained to classify different track morphologies across different alloy systems to select appropriate printing settings. This method is intended for the development of new alloy systems, where the powder feedstock may be unavailable, or prohibitively expensive to manufacture.Design/methodology/approachA convolutional neural network is designed from scratch to identify the 4 key melting modes that are observed in laser powder bed fusion additive manufacturing across different alloy systems. To increase the prediction accuracy and generalisation accuracy across different materials, the network is trained using a novel hybrid data set that combines fully unsupervised learning with semi-supervised learning.FindingsThis study demonstrates that our convolutional network with a novel hybrid training approach can be generalised across different materials, and k-fold validation shows that the model retains good accuracy with changing training conditions. The model can predict the processing maps for the different alloys with an accuracy of up to 96% in some cases. It is also shown that powder-free single-track experiments are a useful indicator for predicting the final print quality of a component.Originality/valueThe “invariant information clustering” (IIC) approach is applied to process optimisation for additive manufacturing, and a novel hybrid data set construction approach that accounts for uncertainty in the ground truth data, enables the trained convolutional model to perform across a range of different materials and most importantly, generalise to materials outside of the training data set. Compared to the traditional cross-sectioning approach, this method considers the whole length of the single track when determining the melting mode.

Federated learning-based natural language processing: a systematic literature review

Federated learning (FL) is a decentralized machine learning (ML) framework that allows models to be trained without sharing the participants’ local data. FL thus preserves privacy better than centralized machine learning. Since textual data (such as clinical records, posts in social networks, or search queries) often contain personal information, many natural language processing (NLP) tasks dealing with such data have shifted from the centralized to the FL setting. However, FL is not free from issues, including convergence and security vulnerabilities (due to unreliable or poisoned data introduced into the model), communication and computation bottlenecks, and even privacy attacks orchestrated by honest-but-curious servers. In this paper, we present a systematic literature review (SLR) of NLP applications in FL with a special focus on FL issues and the solutions proposed so far. Our review surveys 36 recent papers published in relevant venues, which are systematically analyzed and compared from multiple perspectives. As a result of the survey, we also identify the most outstanding challenges in the area.

Countering Social Media Cybercrime Using Deep Learning: Instagram Fake Accounts Detection

An online social media platform such as Instagram has become a popular communication channel that millions of people are using today. However, this media also becomes an avenue where fake accounts are used to inflate the number of followers on a targeted account. Fake accounts tend to alter the concepts of popularity and influence on the Instagram media platform and significantly impact the economy, politics, and society, which is considered cybercrime. This paper proposes a framework to classify fake and real accounts on Instagram based on a deep learning approach called the Long Short-Term Memory (LSTM) network. Experiments and comparisons with existing machine and deep learning frameworks demonstrate considerable improvement in the proposed framework. It achieved a detection accuracy of 97.42% and 94.21% on two publicly available Instagram datasets, with F-measure scores of 92.17% and 89.55%, respectively. Further experiments on the Twitter dataset reveal the effectiveness of the proposed framework by achieving an impressive accuracy rate of 99.42%.

Triple and quadruple optimization for feature selection in cancer biomarker discovery

The proliferation of omics data has advanced cancer biomarker discovery but often falls short in external validation, mainly due to a narrow focus on prediction accuracy that neglects clinical utility and validation feasibility. We introduce three- and four-objective optimization strategies based on genetic algorithms to identify clinically actionable biomarkers in omics studies, addressing classification tasks aimed at distinguishing hard-to-differentiate cancer subtypes beyond histological analysis alone. Our hypothesis is that by optimizing more than one characteristic of cancer biomarkers, we may identify biomarkers that will enhance their success in external validation. Our objectives are to: (i) assess the biomarker panel’s accuracy using a machine learning (ML) framework; (ii) ensure the biomarkers exhibit significant fold-changes across subtypes, thereby boosting the success rate of PCR or immunohistochemistry validations; (iii) select a concise set of biomarkers to simplify the validation process and reduce clinical costs; and (iv) identify biomarkers crucial for predicting overall survival, which plays a significant role in determining the prognostic value of cancer subtypes. We implemented and applied triple and quadruple optimization algorithms to renal carcinoma gene expression data from TCGA. The study targets kidney cancer subtypes that are difficult to distinguish through histopathology methods. Selected RNA-seq biomarkers were assessed against the gold standard method, which relies solely on clinical information, and in external microarray-based validation datasets. Notably, these biomarkers achieved over 0.8 of accuracy in external validations and added significant value to survival predictions, outperforming the use of clinical data alone with a superior c-index. The provided tool also helps explore the trade-off between objectives, offering multiple solutions for clinical evaluation before proceeding to costly validation or clinical trials.

Accurate Prediction of Bangla Text Article Categorization by Utilizing Novel Bangla Stemmer

Text categorization involves assigning predefined category labels to an unlabeled document. With the exponential growth in the accessibility and availability of digital documents over the past decade, this field significantly attracted the scientific community that immensely demands rapid and accurate categorization of these documents. Relying on experts for manual classification is time-consuming and resource-intensive. Consequently, labeling unlabeled digital documents faster more accurately, and more efficiently is inescapable. One promising approach to addressing this demand is the use of machine learning algorithms. Training these algorithms on a large dataset of labeled texts lets them learn patterns and predicted unlabeled documents. This strategy might greatly expedite the categorizing process while retaining a substantial level of accuracy through leveraging artificial intelligence. These algorithms have also enhanced natural language processing techniques, making them more accurate at classifying unlabeled digital documents. In this study, we propose a novel machine-learning computational framework to address this challenge. Our framework incorporates a novel Bangla stemmer, which reduces words to their stems. We then employed TF-IDF for document vectorization, a statistical measure assessing word relevance for categorization purposes. Experimental results reveal that our framework significantly enhances prediction performance, achieving an impressive 95.3% prediction accuracy.

Rapid diagnosis of the geospatial distribution of intertidal macroalgae using large-scale UAVs

Macroalgae have been used as indicators of the health of coastal ecosystems, they function as sinks of CO2 and are essential contributors to primary production. With the increase in anthropogenic activities, it is crucial to assess the impact of such activities on these ecosystems. As traditional surveying techniques, although accurate, are time-consuming and their area coverage is limited, novel techniques are required to monitor the coverage and diversity of intertidal macroalgae. We propose a methodology using the free-source Semi-Automatic Classification Plugin from QGIS to use UAV and multispectral cameras for the spatiotemporal monitoring of intertidal macroalgae. We also compared the performance of six classifiers: Minimum Distance (MD), Maximum Likelihood (ML), Spectral Angle Mapping (SAM), Multi-Layer Perceptron (MLP), Random Forest (RF) and Support Vector Machine (SVM), for three types of macroalgae classification: general, taxonomical groups and species. As proof of concept, an intertidal rocky shore in a marine protected area (NW Spain) was studied for four months. RF and SVM achieved similar results, with both being recommended for the general (OASVM = 97.4±1.7 and OARF = 98.3±1.7) and taxonomical groups (OASVM = 91.6±1.9 and OARF = 89.2±4.5). SVM and ML were found to be more suitable for species classification (OASVM = 77.4±11.4 and OAML = 74.2±9.7). SAM and MLP provided the least performant species classifiers because of the overlap in the macroalgae spectral signatures. The plugin showed limitations when tuning the input parameters of the MLP classifier and did not let to add a validation dataset. Additionally, we present an open-access GIS web application, Alganat 2000 GIS web, to facilitate the monitoring and management of coastal areas. We conclude that the proposed methodology using the SVM or ML classifiers is an effective tool for assessing intertidal macroalgal assemblages. Its easy and rapid implementation is beneficial for researchers who are not very familiar with coding and machine learning frameworks and reduces the time and cost of fieldwork. As future work, we propose the combination of the multispectral bands with topographic and spectral indices and to research the application of deep learning models to the classification of intertidal macroalgae.

Machine Learning Approaches for Predicting Power Conversion Efficiency in Organic Solar Cells: A Comprehensive Review

Organic solar cells (OSCs), renowned for their lightweight, cost efficiency, and adaptability nature, stand out as a promising option for developing renewable energy. Improving the power conversion efficiency (PCE) of OSCs is essential, and researchers are delving into novel materials to achieve this. Traditional approaches are often laborious and costly, highlighting the need for predictive modeling. Machine learning (ML), especially via quantitative structure–property relationship (QSPR) models, is streamlining material development, with a goal to exceed a 20% PCE. In this review, the application of ML in OSCs is explored, and recent studies utilizing ML approaches for PCE prediction are reviewed, encompassing empirical functions, ML algorithms, self‐devised ML frameworks, and the combination with automated experimental technologies. First, the benefits of ML in predicting PCE for OSCs are addressed. Second, the development of high‐efficiency predictive models for both fullerene and nonfullerene acceptors is delved into. The impact of various ML algorithm models on PCE prediction is then assessed, taking into account the construction of predictive models. Moreover, the quality of databases and the selection of descriptors are considered. Databases and descriptors based on experimental studies are further categorized. Finally, prospects for the future development of OSCs are proposed.

Fusion of spectral and topographic features for land use mapping using a machine learning framework for a regional scale application.

This study investigated the dynamics of land use and land cover (LULC) modelling, mapping, and assessment in the Kegalle District ofSri Lanka, where policy decision-making is crucial in agricultural development where LULC temporal datasets are not readily available.Employing remotely sensed datasets and machine learning algorithms, the work presented here aims to compare the accuracy of three classification approaches in mapping LULC categories across the time in the study area primarily using the Google Earth Engine (GEE). Three classifiers namely random forest (RF), support vector machines (SVM), and classification and regression trees (CART) were used in LULC modelling, mapping, and change analysis. Different combinations of input features were investigated to improve classification performance. Developed models were optimised using the grid search cross-validation (CV) hyperparameter optimisation approach. It was revealed that the RF classifier constantly outstrips SVM and CART in terms of accuracy measures, highlighting its reliability in classifying the LULC. Land cover changes were examined for two periods, from 2001 to 2013 and 2013 to 2022, implying major alterations such as the conversion of rubber and coconut areas to built-up areas and barren lands. For suitable classification with higher accuracy, the study suggests utilising high spatial resolution satellite data, advanced feature selection approaches, and a combination of several spatial and spatial-temporal data sources. The study demonstrated practical applications of derived temporal LULC datasets for land management practices in agricultural development activities in developing nations.

AWDP-FL: An Adaptive Differential Privacy Federated Learning Framework

Data security and user privacy concerns are receiving increasing attention. Federated learning models based on differential privacy offer a distributed machine learning framework that protects data privacy. However, the noise introduced by the differential privacy mechanism may affect the model’s usability, especially when reasonable gradient clipping is absent. Fluctuations in the gradients can lead to issues like gradient explosion, compromising training stability and potentially leaking privacy. Therefore, gradient clipping has become a crucial method for protecting both model performance and data privacy. To balance privacy protection and model performance, we propose the Adaptive Weight-Based Differential Privacy Federated Learning (AWDP-FL) framework, which processes model gradient parameters at the neural network layer level. First, by designing and recording the change trends of two-layer historical gradient sequences, we analyze and predict gradient variations in the current iteration and calculate the corresponding weight values. Then, based on these weights, we perform adaptive gradient clipping for each data point in each training batch, which is followed by gradient momentum updates based on the third moment. Before uploading the parameters, Gaussian noise is added to protect privacy while maintaining model accuracy. Theoretical analysis and experimental results validate the effectiveness of this framework under strong privacy constraints.

Species assignment from seal diet samples using shape analyses in a machine learning framework

Abstract The identification and quantification of prey ingested is a limiting factor in trophic ecology studies and is fundamental for assessing the impact of a predator on prey populations. Vendace (Coregonus albula) and whitefish (C. lavaretus) are two congeneric species, which are commonly preyed on by Baltic ringed seals (Pusa hispida). The otoliths of these two species are, however, very similar and distinguishing between them in the seal diet using visual inspection has so far been challenging. Here, otolith shape outline analyses were used in combination with machine learning techniques to discriminate between eroded vendace and whitefish otoliths from ringed seal diet samples. An experiment of in vitro digestion of the otoliths was performed to train a machine learning model. Our model is able to self-assign known digested otoliths back to their species of origin with >90% accuracy. Furthermore, 89% (N = 690) of the otoliths collected from digestive tract samples could be successfully assigned to species level, i.e. vendace or whitefish. This method is readily applicable for improved understanding of ringed seal feeding habits and predator–prey interactions, as well as large-scale applications to generate seal-predation matrix inputs for stock assessments of vendace and whitefish. Further development of the machine learning techniques to discriminate between prey species in seal and other piscivorous diets is strongly encouraged.

Prediction of adolescent depression from prenatal and childhood data from ALSPAC using machine learning

Depression is a major cause of disability and mortality for young people worldwide and is typically first diagnosed during adolescence. In this work, we present a machine learning framework to predict adolescent depression occurring between ages 12 and 18 years using environmental, biological, and lifestyle features of the child, mother, and partner from the child’s prenatal period to age 10 years using data from 8467 participants enrolled in the Avon Longitudinal Study of Parents and Children (ALSPAC). We trained and compared several cross-sectional and longitudinal machine learning techniques and found the resulting models predicted adolescent depression with recall (0.59 ± 0.20), specificity (0.61 ± 0.17), and accuracy (0.64 ± 0.13), using on average 39 out of the 885 total features (4.4%) included in the models. The leading informative features in our predictive models of adolescent depression were female sex, parental depression and anxiety, and exposure to stressful events or environments. This work demonstrates how using a broad array of evidence-driven predictors from early in life can inform the development of preventative decision support tools to assist in the early detection of risk for mental illness.

Machine-Learning Prediction of Protein Function from the Portrait of Its Intramolecular Electric Field.

We introduce a machine learning framework designed to predict enzyme functionality directly from the heterogeneous electric fields inherent to protein active sites. We apply this method to a curated data set of heme-iron oxidoreductases, spanning three enzyme classes: monooxygenases, peroxidases, and catalases. Conventional analysis, focused on simplistic, point electric fields along the Fe-O bond, is shown to be inadequate for accurate activity prediction. Our model demonstrates that the enzyme's heterogeneous 3-D electric field, alone, can accurately predict its function, without relying on additional protein-specific information. Through feature selection, we uncover key electric field components that not only validate previous studies but also underscore the crucial role of multiple components beyond the traditionally emphasized electric field along the Fe-O bond in heme enzymes. Furthermore, by integrating protein dynamics, principal component analysis, clustering, and QM/MM calculations, we reveal that while dynamic complexities in protein structures can obscure predictions, the model still retains its accuracy. This research significantly advances our understanding of how protein scaffolds possess signature electric fields tailored to their functions at the active site. Moreover, it presents a novel electrostatics-based tool to harness these signature electric fields for predicting enzyme function.

Generating tyre tread designs using a sensory evaluation regression model and a generative model

This study introduces a method for generating tyre tread images that are highly rated in terms of ‘slip resistance’ sensory evaluations. It involves constructing a regression model to predict ‘slip resistance’ sensory evaluations in tyre tread images and subsequently developing an image generation model informed by this regression model. The regression model was developed using a machine learning framework based on ensemble learning and SHapley Additive exPlanations. The model was then used to filter training data for the image generation process, leading to the creation of a tyre tread image generation model aimed at producing images with high ‘slip resistance’ sensory evaluations, using StyleGAN. The proposed method was applied in a case study using real data, and its effectiveness was validated by comparison with a multiple regression analysis approach. The results revealed that more accurate results were achieved with the random forest (RF) regression model than with the multiple regression model due to the RF regression model’s emphasis on non-linear features. Furthermore, a significant difference was observed between the ‘slip resistance’ sensory evaluation values of the images generated by StyleGAN trained with training data screened by the RF regression model and those from the training data.

Soil Properties Classification in Sustainable Agriculture Using Genetic Algorithm-Optimized and Deep Neural Networks

Optimization of land management and agricultural practices require precise classification of soil properties. This study presents a method to fine-tune deep neural network (DNN) hyperparameters for multiclass classification of soil properties using genetic algorithms (GAs) with knowledge-based generation of hyperparameters. The focus is on classifying soil attributes, including nutrient availability (0.78 ± 0.11), nutrient retention capacity (0.86 ± 0.05), rooting conditions (0.85 ± 0.07), oxygen availability to roots (0.84 ± 0.05), excess salts (0.96 ± 0.02), toxicity (0.96 ± 0.01), and soil workability (0.84 ± 0.09), with these accuracies representing the results from classification with variations from cross-validation. A dataset from the USA, which includes land-use distribution, aspect distribution, slope distribution, and climate data for each plot, is utilized. A GA is applied to explore a wide range of hyperparameters, such as the number of layers, neurons per layer, activation functions, optimizers, learning rates, and loss functions. Additionally, ensemble methods such as random forest and gradient boosting machines were employed, demonstrating comparable accuracy to the DNN approach. This research contributes to the advancement of precision agriculture by providing a robust machine learning (ML) framework for accurate soil property classification. By enabling more informed and efficient land management decisions, it promotes sustainable agricultural practices that optimize resource use and enhance soil health for long-term ecological balance.

Prediction of Diabetes, Cardiac and Parkinson’s disease using machine learning and streamlit framework

Prediction of Diabetes, Cardiac and Parkinson’s disease using machine learning and streamlit framework

Beamforming-integrated neural networks for ultrasound imaging

Sparse matrix beamforming (SMB) is a computationally efficient reformulation of delay-and-sum (DAS) beamforming as a single sparse matrix multiplication. This reformulation can potentially dovetail with machine learning platforms like TensorFlow and PyTorch that already support sparse matrix operations. In this work, using SMB principles, we present the development of beamforming-integrated neural networks (BINNs) that can rationally infer ultrasound images directly from pre-beamforming channel-domain radiofrequency (RF) datasets. To demonstrate feasibility, a toy BINN was first designed with two 2D-convolution layers that were respectively placed both before and after an SMB layer. This toy BINN correctly updated kernel weights in all convolution layers, demonstrating efficiency in both training (PyTorch – 133 ms, TensorFlow – 22 ms) and inference (PyTorch – 4 ms, TensorFlow – 5 ms). As an application demonstration, another BINN with two RF-domain convolution layers, an SMB layer, and three image-domain convolution layers was designed to infer high-quality B-mode images in vivo from single-shot plane-wave channel RF data. When trained using 31-angle compounded plane wave images (3000 frames from 22 human volunteers), this BINN showed mean-square logarithmic error improvements of 21.3 % and 431 % in the inferred B-mode image quality respectively comparing to an image-to-image convolutional neural network (CNN) and an RF-to-image CNN with the same number of layers and learnable parameters (3,777). Overall, by including an SMB layer to adopt prior knowledge of DAS beamforming, BINN shows potential as a new type of informed machine learning framework for ultrasound imaging.