Evaluation Of Machine Learning Models Research Articles

A novel dataset and local interpretable model-agnostic explanations (LIME) for monkeypox prediction

In image-based diagnosis, machine learning recently showed great potential, particularly in the detection of cancer, the identification of tumour cells, and the diagnosis of COVID-19. Similar methods could be used to detect monkeypox on human skin, however there isn’t a public dataset with data on monkeypox that can be used to train and evaluate machine learning models. In order to address this, the dataset “Monkeypox2022” has been created and made available on GitHub (https://github.com/Angmo21720/data-set). Images from internet and open-source sources were used to construct the collection; these sources did not impose any restrictions on use, including commercial use. Additionally, the enhanced VGG16 model was proposed and ran two Studies for comparison. According to the findings, the model’s AUC Score for identifying monkeypox patients in Study One was 0.972, while its AUC Score for identifying such patients in Study Two was 0.748. Deeper understanding of the characteristics that distinguish a monkeypox infection is provided by using Local Interpretable Model-Agnostic Explanations (LIME), which is used to decipher predictions and reveal the features utilized by a model in decision-making.

EvalPM: a framework for evaluating machine learning models for particulate matter prediction

Air pollution through particulate matter (PM) is one of the largest threats to human health. To understand the causes of PM pollution and enact suitable countermeasures, reliable predictions of future PM concentrations are required. In the scientific literature, many methods exist for machine learning (ML)-based PM prediction, though their quality is difficult to compare because, among other things, they use different data sets and evaluate the resulting predictions differently. For a new data set, it is not apparent which of the existing prediction methods is best suited. In order to ease the assessment of said models, we present evalPM, a framework to easily create, evaluate, and compare different ML models for immission-based PM prediction. To achieve this, the framework provides flexibility regarding data sets, input features, target variables, model types, hyperparameters, and model evaluation. It has a modular design consisting of several components, each providing at least one required flexibility. The individual capabilities of the framework are demonstrated using 16 different models from the related literature by means of temporal prediction of PM concentrations for four European data sets, showing the capabilities and advantages of the evalPM framework. In doing so, it is shown that the framework allows fast creation and evaluation of ML-based PM prediction models.

Using expert-reviewed CSAM to train CNNs and its anthropological analysis

Machine learning methods for the identification of child sexual abuse materials (CSAM) have been previously studied, however, they have serious limitations. Firstly, the training sets used to train the appropriate machine learning algorithms were not previously annotated by a forensic expert in anthropology. Secondly, previously presented solutions have rarely used models trained using real pornographic content involving children. Thirdly, previous studies have not presented a detailed justification for the classification decisions made, which is important due to the recent guidelines of the European Commission (Artificial Intelligence Act).The aim of the study was to train convolution neural networks (CNNs) using expert-labelled CSAM images and thereby identify the elements of the body and/or the environment that are critical for classifications by the neural network. To train and evaluate machine learning models, we used 60,000 images equally divided into four classes (CSAM images, images displaying sexual activity involving adults, images of people without sexual activity, and images not containing people). We used four neural network architectures: MobileNet, ResNet152, xResNet152 and its modification ResNet-s, designed for the purpose of research.The trained models provided high accuracy of classifying CSAM images: xResNet152 (F1 = 0.93, 92,8%), xResNet-s (F1 = 0.93, 93,1%), ResNet152 (F1 = 0.90, 91,39%), MobileNet (F1 ranged from 0.85 to 0.87, accuracy ranged from 86% to 87%).The results of the conducted research suggest that using expert knowledge (in sexology and anthropology) significantly improved the accuracy of the models. In regard to further anthropological analysis, the results indicate that the breasts, face and torso are crucial areas for the classification of pornographic content with children's participation. Results suggests that the ResNet-s neural network may be a reliable tool for clinical work and to support the work of experts witnesses in the field of anthropology.The study design received a positive opinion of the Ethics Committee of the Faculty of Mathematics, Informatics and Mechanics of the University of Warsaw. The clinical material was used for research purposes with the consent of the relevant prosecutor's offices. Authors provided free version of Windows application to classify CSAM for forensic experts, policemen and prosecutors at the OSF repository (DOI: 10.17605/OSF.IO/RU7JX).

Lightweight Multi-Class Support Vector Machine-Based Medical Diagnosis System with Privacy Preservation.

Machine learning, powered by cloud servers, has found application in medical diagnosis, enhancing the capabilities of smart healthcare services. Research literature demonstrates that the support vector machine (SVM) consistently demonstrates remarkable accuracy in medical diagnosis. Nonetheless, safeguarding patients' health data privacy and preserving the intellectual property of diagnosis models is of paramount importance. This concern arises from the common practice of outsourcing these models to third-party cloud servers that may not be entirely trustworthy. Few studies in the literature have delved into addressing these issues within SVM-based diagnosis systems. These studies, however, typically demand substantial communication and computational resources and may fail to conceal classification results and protect model intellectual property. This paper aims to tackle these limitations within a multi-class SVM medical diagnosis system. To achieve this, we have introduced modifications to an inner product encryption cryptosystem and incorporated it into our medical diagnosis framework. Notably, our cryptosystem proves to be more efficient than the Paillier and multi-party computation cryptography methods employed in previous research. Although we focus on a medical application in this paper, our approach can also be used for other applications that need the evaluation of machine learning models in a privacy-preserving way such as electricity theft detection in the smart grid, electric vehicle charging coordination, and vehicular social networks. To assess the performance and security of our approach, we conducted comprehensive analyses and experiments. Our findings demonstrate that our proposed method successfully fulfills our security and privacy objectives while maintaining high classification accuracy and minimizing communication and computational overhead.

Performance Evaluation of Five Machine Learning Algorithms for Estimating Reference Evapotranspiration in an Arid Climate

The Food and Agriculture Organization recommends that the Penman–Monteith Method contains Equation 56 (PMF) as a widely accepted standard for reference evapotranspiration (ETo) calculation. Despite this, the PMF cannot be employed when meteorological variables are constrained; therefore, alternative models for ETo estimation requiring fewer variables must be chosen, which means that they perform at least as well as, if not better than, the PMF in terms of accuracy and efficiency. This study evaluated five machine learning (ML) algorithms to estimate ETo and compared their results with the standardized PMF. For this purpose, ML models were trained using monthly time series climatic data. The created ML models underwent testing to determine ETo under varying meteorological input combinations. The results of ML models were compared to assess their accuracy and validate their performance using several statistical indicators, errors (root-mean-square (RMSE), mean absolute error (MAE)), model efficiency (NSE), and determination coefficient (R2). The process of evaluating ML models involved the utilization of radar charts, Smith graphs, heatmaps, and bullet charts. Based on our findings, satisfactory results have been obtained using RBFFNN based on M12 input combinations (mean temperature (Tmean), mean relative humidity (RHmean), sunshine hours (Sh)) for ETo estimation. The RBFFNN model exhibited the most precise estimation as RMSE obtained values of 0.30 and 0.22 during the training and testing phases, respectively. In addition, during training and testing, the MAE values for this model were recorded as 0.15 and 0.17, respectively. The highest R2 and NSE values were noted as 0.98 and 0.99 for the RBFNN during performance analysis, respectively. The scatter plots and spatial variations of the RBFNN and PMF in the studied region indicated that the RBFNN had the highest efficacy (R2, NSE) and lowest errors (RMSE, MAE) as compared with the other four ML models. Overall, our study highlights the potential of ML models for ETo estimation in the arid region (Jacobabad), providing vital insights for improving water resource management, helping climate change research, and optimizing irrigation scheduling for optimal agricultural water usage in the region.

Machine learning to predict ceftriaxone resistance using single nucleotide polymorphisms within a global database of Neisseria gonorrhoeae genomes.

Antimicrobial resistance (AMR) in Neisseria gonorrhoeae is an urgent global health issue. Machine learning (ML) is a powerful tool that can aid in identifying mutations and predicting their impact on AMR. The study aimed to use ML models to predict ceftriaxone susceptibility and decreased susceptibility (S/DS). A public database of N. gonorrhoeae genomes with minimum inhibitory concentration (MIC) data was used to evaluate seven ML models using 97 single nucleotide polymorphisms (SNPs) known to be associated with ceftriaxone resistance. Ceftriaxone MICs ≤ 0.064 mg/L were classified as susceptible, and ceftriaxoneMICs > 0.064 mg/L were classified as DS. The contributions of individual SNPs to predict S/DS were calculated using SHapley Additive exPlanation (SHAP) values. An ML model was retrained using different combinations of SNPs with the highest SHAP values. The performance of ML models was assessed using different metrics including area under the curve (AUC) and balanced accuracy (bAcc). The ML analyses included 9,540 N. gonorrhoeae genomes; 368 (0.04%) were classified as DS. Of the models evaluated, the model trained with a random forest classifier had the highest performance (AUC 0.965; bAcc 0.926). A model retrained the top five SNPs, according to SHAP values, demonstrated a similar performance (AUC 0.916; bAcc 0.879) as the model with 97 SNPs. An ML approach using mutations in N. gonorrhoeae can be used to predict S/DS to ceftriaxone. The results highlight a practical application of ML to identify mutations most associated with S/DS to ceftriaxone, which can aid in the development of assays to predict AMR. IMPORTANCE Antimicrobial resistance in Neisseria gonorrhoeae is an urgent global health issue. The objectives of the study were to use a global collection of 12,936 N. gonorrhoeae genomes from the PathogenWatch database to evaluate different machine learning models to predict ceftriaxone susceptibility/decreased susceptibility using 97 mutations known to be associated with ceftriaxone resistance. We found the random forest classifier model had the highest performance. The analysis also reported the relative contributions of different mutations within the ML model predictions, allowing for the identification of the mutations with the highest importance for ceftriaxone resistance. A machine learning model retrained with the top five mutations performed similarly to the model using all 97 mutations. These results could aid in the development of molecular tests to detect resistance to ceftriaxone in N. gonorrhoeae. Moreover, this approach could be applied to building and evaluating machine learning models for predicting antimicrobial resistance in other pathogens.

Comparison of Prognostic Models for Functional Outcome in Aneurysmal Subarachnoid Hemorrhage Based on Machine Learning

Controversy exists regarding the superiority of the performance of prognostic tools based on advanced machine learning (ML) algorithms for patients with aneurysmal subarachnoid hemorrhage (aSAH). However, it is unclear whether ML prognostic models will benefit patients due to the lack of a comprehensive assessment. We aimed to develop and evaluate ML models for predicting unfavorable functional outcomes for aSAH patients and identify the model with the greatest performance. In this retrospective study, a dataset of 955 patients with aSAH was used to construct and validate prognostic models for functional outcomes assessed using the modified Rankin scale during a follow-up period of 3-6 months. Clinical scores and clinical and radiological features on admission and secondary complications were used to construct models based on 5 ML algorithms (i.e., logistic regression [LR], k-nearest neighbor, extreme gradient boosting, random forest, and artificial neural network). For evaluation among the models, the area under the receiver operating characteristic curve, area under the precision-recall curve, calibration curve, and decision curve analysis were used. Composite models had significantly higher area under the receiver operating characteristic curves than did simple models in predicting unfavorable functional outcomes. Compared with other composite models (random forest and extreme gradient boosting) with good calibration, LR had the highest area under the precision-recall score and showed the greatest benefit in decision curve analysis. Of the 5 studied ML models, the conventional LR model outperformed the advanced algorithms in predicting the prognosis and could be a useful tool for health care professionals.

Machine Learning Models in Prediction of Treatment Response After Chemoembolization with MRI Clinicoradiomics Features.

To evaluate machine learning models, created with radiomics and clinicoradiomics features, ability to predict local response after TACE. 188 treatment-naïve patients (150 responders, 38 non-responders) with HCC who underwent TACE were included in this retrospective study. Laboratory, clinical and procedural information were recorded. Local response was evaluated by European Association for the Study of the Liver criteria at 3-months. Radiomics features were extracted from pretreatment pre-contrast enhanced T1 (T1WI) and late arterial-phase contrast-enhanced T1 (CE-T1) MRI images. After data augmentation, data were split into training and test sets (70/30). Intra-class correlations, Pearson's correlation coefficients were analyzed and followed by a sequential-feature-selection (SFS) algorithm for feature selection. Support-vector-machine (SVM) models were trained with radiomics and clinicoradiomics features of T1WI, CE-T1 and the combination of both datasets, respectively. Performance metrics were calculated with the test sets. Models' performances were compared with Delong's test. 1128 features were extracted. In feature selection, SFS algorithm selected 18, 12, 24 and 8 features in T1WI, CE-T1, combined datasets and clinical features, respectively. The SVM models area-under-curve was 0.86 and 0.88 in T1WI; 0.76, 0.71 in CE-T1 and 0.82, 0.91 in the combined dataset, with and without clinical features, respectively. The only significant change was observed after inclusion of clinical features in the combined dataset (p = 0.001). Higher WBC and neutrophil levels were significantly associated with lower treatment response in univariant analysis (p = 0.02, for both). Machine learning models created with clinical and MRI radiomics features, may have promise in predicting local response after TACE. Level 4, Case-control study.

Pol-InSAR-Island - A benchmark dataset for multi-frequency Pol-InSAR data land cover classification

This paper presents Pol-InSAR-Island, the first publicly available multi-frequency Polarimetric Interferometric Synthetic Aperture Radar (Pol-InSAR) dataset labeled with detailed land cover classes, which serves as a challenging benchmark dataset for land cover classification. In recent years, machine learning has become a powerful tool for remote sensing image analysis. While there are numerous large-scale benchmark datasets for training and evaluating machine learning models for the analysis of optical data, the availability of labeled SAR or, more specifically, Pol-InSAR data is very limited. The lack of labeled data for training, as well as for testing and comparing different approaches, hinders the rapid development of machine learning algorithms for Pol-InSAR image analysis. The Pol-InSAR-Island benchmark dataset presented in this paper aims to fill this gap. The dataset consists of Pol-InSAR data acquired in S- and L-band by DLR's airborne F-SAR system over the East Frisian island Baltrum. The interferometric image pairs are the result of a repeat-pass measurement with a time offset of several minutes. The image data are given as 6 × 6 coherency matrices in ground range on a 1 m × 1m grid. Pixel-accurate class labels, consisting of 12 different land cover classes, are generated in a semi-automatic process based on an existing biotope type map and visual interpretation of SAR and optical images. Fixed training and test subsets are defined to ensure the comparability of different approaches trained and tested prospectively on the Pol-InSAR-Island dataset. In addition to the dataset, results of supervised Wishart and Random Forest classifiers that achieve mean Intersection-over-Union scores between 24% and 67% are provided to serve as a baseline for future work. The dataset is provided via KITopenData: https://doi.org/10.35097/1700.

Q8VaxStance: Dataset Labeling System for Stance Detection towards Vaccines in Kuwaiti Dialect

The Kuwaiti dialect is a particular dialect of Arabic spoken in Kuwait; it differs significantly from standard Arabic and the dialects of neighboring countries in the same region. Few research papers with a focus on the Kuwaiti dialect have been published in the field of NLP. In this study, we created Kuwaiti dialect language resources using Q8VaxStance, a vaccine stance labeling system for a large dataset of tweets. This dataset fills this gap and provides a valuable resource for researchers studying vaccine hesitancy in Kuwait. Furthermore, it contributes to the Arabic natural language processing field by providing a dataset for developing and evaluating machine learning models for stance detection in the Kuwaiti dialect. The proposed vaccine stance labeling system combines the benefits of weak supervised learning and zero-shot learning; for this purpose, we implemented 52 experiments on 42,815 unlabeled tweets extracted between December 2020 and July 2022. The results of the experiments show that using keyword detection in conjunction with zero-shot model labeling functions is significantly better than using only keyword detection labeling functions or just zero-shot model labeling functions. Furthermore, for the total number of generated labels, the difference between using the Arabic language in both the labels and prompt or a mix of Arabic labels and an English prompt is statistically significant, indicating that it generates more labels than when using English in both the labels and prompt. The best accuracy achieved in our experiments in terms of the Macro-F1 values was found when using keyword and hashtag detection labeling functions in conjunction with zero-shot model labeling functions, specifically in experiments KHZSLF-EE4 and KHZSLF-EA1, with values of 0.83 and 0.83, respectively. Experiment KHZSLF-EE4 was able to label 42,270 tweets, while experiment KHZSLF-EA1 was able to label 42,764 tweets. Finally, the average value of annotation agreement between the generated labels and human labels ranges between 0.61 and 0.64, which is considered a good level of agreement.

Radiomics and Machine Learning for prediction of two-year disease-specific mortality and KRAS mutation status in metastatic colorectal cancer

PurposeColorectal cancer is usually accompanied by liver metastases. The prediction of patient evolution is essential for the choice of the appropriate therapy. The aim of this study is to develop and evaluate machine learning models to predict KRAS gene mutations and 2-year disease-specific mortality from medical images. MethodsClinical and follow-up information was collected from patients with metastatic colorectal cancer who had undergone computed tomography prior to liver resection. The dominant liver lesion was segmented in each scan and radiomic features were extracted from the volumes of interest. The 65% of the cases were employed to perform feature selection and to train machine learning algorithms through cross-validation. The best performing models were assembled and evaluated in the remaining cases of the cohort. ResultsFor the mortality model development, 101 cases were used as training set (64 alive, 37 deceased) and 35 as test set (22 alive, 13 deceased); while for KRAS mutation models, 55 cases were used for training (31 wild-type, 24 mutated) and 30 for testing (17 wild-type, 13 mutated). The ensemble of top performing models resulted in an area under the receiver operating characteristic curve of 0.878 for mortality and 0.905 for KRAS prediction. ConclusionsPredicting the prognosis of patients with metastatic colorectal cancer is useful for making timely decisions about the best treatment options. This study presents a noninvasive method based on quantitative analysis of baseline images to identify factors influencing patient outcomes, with the aim of incorporating these tools as support systems.

Anthropogenic Object Localization: Evaluation of Broad-Area High-Resolution Imagery Scans Using Deep Learning in Overhead Imagery.

Too often, the testing and evaluation of object detection, as well as the classification techniques for high-resolution remote sensing imagery, are confined to clean, discretely partitioned datasets, i.e., the closed-world model. In recent years, the performance on a number of benchmark datasets has exceeded 99% when evaluated using cross-validation techniques. However, real-world remote sensing data are truly big data, which often exceed billions of pixels. Therefore, one of the greatest challenges regarding the evaluation of machine learning models taken out of the clean laboratory setting and into the real world is the difficulty of measuring performance. It is necessary to evaluate these models on a grander scale, namely, tens of thousands of square kilometers, where it is intractable to the ground truth and the ever-changing anthropogenic surface of Earth. The ultimate goal of computer vision model development for automated analysis and broad area search and discovery is to augment and assist humans, specifically human-machine teaming for real-world tasks. In this research, various models have been trained using object classes from benchmark datasets such as UC Merced, PatternNet, RESISC-45, and MDSv2. We detail techniques to scan broad swaths of the Earth with deep convolutional neural networks. We present algorithms for localizing object detection results, as well as a methodology for the evaluation of the results of broad-area scans. Our research explores the challenges of transitioning these models out of the training-validation laboratory setting and into the real-world application domain. We show a scalable approach to leverage state-of-the-art deep convolutional neural networks for the search, detection, and annotation of objects within large swaths of imagery, with the ultimate goal of providing a methodology for evaluating object detection machine learning models in real-world scenarios.

Health Data Sharing towards Knowledge Creation

Data sharing and service reuse in the health sector pose significant privacy and security challenges. The European Commission recognizes health data as a unique and cost-effective resource for research, while the OECD emphasizes the need for privacy-protecting data governance systems. In this paper, we propose a novel approach to health data access in a hospital environment, leveraging homomorphic encryption to ensure privacy and secure sharing of medical data among healthcare entities. Our framework establishes a secure environment that enforces GDPR adoption. We present an Information Sharing Infrastructure (ISI) framework that seamlessly integrates artificial intelligence (AI) capabilities for data analysis. Through our implementation, we demonstrate the ease of applying AI algorithms to treated health data within the ISI environment. Evaluating machine learning models, we achieve high accuracies of 96.88% with logistic regression and 97.62% with random forest. To address privacy concerns, our framework incorporates Data Sharing Agreements (DSAs). Data producers and consumers (prosumers) have the flexibility to express their prefearences for sharing and analytics operations. Data-centric policy enforcement mechanisms ensure compliance and privacy preservation. In summary, our comprehensive framework combines homomorphic encryption, secure data sharing, and AI-driven analytics. By fostering collaboration and knowledge creation in a secure environment, our approach contributes to the advancement of medical research and improves healthcare outcomes. A real case application was implemented between Portuguese hospitals and universities for this data sharing.

Performance evaluation of Machine Learning models to predict heart attack

Coronary Artery Disease is the type of cardiovascular disease (CVD) that happens when the blood vessels which stream the blood toward the heart, either become tapered or blocked. Of this, the heart is incapable to push sufficient blood to encounter its requirements. This would lead to angina (chest pain). CVDs are the leading cause of mortality worldwide. According to WHO, in the year 2019 17.9 million people deceased from CVD. Machine Learning is a type of artificial intelligence that uses algorithms to help analyse large datasets more efficiently. It can be used in medical research to help process large amounts of data quickly, such as patient records or medical images. By using Machine Learning techniques and methods, scientists can automate the analysis of complex and large datasets to gain deeper insights into the data. Machine Learning is a type of technology that helps with gathering data and understanding patterns. Recently, researchers in the healthcare industry have been using Machine Learning techniques to assist with diagnosing heart-related diseases. This means that the professionals involved in the diagnosis process can use Machine Learning to help them figure out what is wrong with a patient and provide appropriate treatment. This paper evaluates different machine learning models performances. The Supervised Learning algorithms are used commonly in Machine Learning which means that the training is done using labelled data, belonging to a particular classification. Such classification methods like Random Forest, Decision Tree, K-Nearest Neighbour, XGBoost algorithm, Naive Bayes, and Support Vector Machine will be used to assess the cardiovascular disease by Machine Learning.

Machine Learning Classification of Body Part, Imaging Axis, and Intravenous Contrast Enhancement on CT Imaging.

Purpose: The development and evaluation of machine learning models that automatically identify the body part(s) imaged, axis of imaging, and the presence of intravenous contrast material of a CT series of images. Methods: This retrospective study included 6955 series from 1198 studies (501 female, 697 males, mean age 56.5years) obtained between January 2010 and September 2021. Each series was annotated by a trained board-certified radiologist with labels consisting of 16 body parts, 3 imaging axes, and whether an intravenous contrast agent was used. The studies were randomly assigned to the training, validation and testing sets with a proportion of 70%, 20% and 10%, respectively, to develop a 3D deep neural network for each classification task. External validation was conducted with a total of 35,272 series from 7 publicly available datasets. The classification accuracy for each series was independently assessed for each task to evaluate model performance. Results: The accuracies for identifying the body parts, imaging axes, and the presence of intravenous contrast were 96.0% (95% CI: 94.6%, 97.2%), 99.2% (95% CI: 98.5%, 99.7%), and 97.5% (95% CI: 96.4%, 98.5%) respectively. The generalizability of the models was demonstrated through external validation with accuracies of 89.7 - 97.8%, 98.6 - 100%, and 87.8 - 98.6% for the same tasks. Conclusions: The developed models demonstrated high performance on both internal and external testing in identifying key aspects of a CT series.

Identifying Metabolic Syndrome Easily and Cost Effectively Using Non-Invasive Methods with Machine Learning Models.

The objective of this study was to employ machine learning (ML) models utilizing non-invasive factors to achieve early and low-cost identification of MetS in a large physical examination population. The study enrolled 9171 participants who underwent physical examinations at Northern Jiangsu People's Hospital in 2009 and 2019, to determine MetS based on criteria established by the Chinese Diabetes Society. Non-invasive characteristics such as gender, age, body mass index (BMI), systolic blood pressure (SBP), and diastolic blood pressure (DBP) were collected and used as input variables to train and evaluate ML models for MetS identification. Several ML models were used for MetS identification, including logistic regression (LR), k-nearest neighbors algorithm (k-NN), naive bayesian (NB), decision tree (DT), random forest (RF), artificial neural network (ANN), and support vector machine (SVM). Our ML models all showed good performance in the 10-fold cross-validation except for the SVM model. In the external validation, the NB model exhibited the best performance with an AUC of 0.976, accuracy of 0.923, sensitivity of 98.32%, and specificity of 91.32%. This study proposed a new non-invasive method for early and low-cost identification of MetS by using ML models. This approach has the potential to serve as a highly sensitive, convenient, and cost-effective tool for large-scale MetS screening.

PlasmoFAB: a benchmark to foster machine learning for Plasmodium falciparum protein antigen candidate prediction.

Machine learning methods can be used to support scientific discovery in healthcare-related research fields. However, these methods can only be reliably used if they can be trained on high-quality and curated datasets. Currently, no such dataset for the exploration of Plasmodium falciparum protein antigen candidates exists. The parasite P.falciparum causes the infectious disease malaria. Thus, identifying potential antigens is of utmost importance for the development of antimalarial drugs and vaccines. Since exploring antigen candidates experimentally is an expensive and time-consuming process, applying machine learning methods to support this process has the potential to accelerate the development of drugs and vaccines, which are needed for fighting and controlling malaria. We developed PlasmoFAB, a curated benchmark that can be used to train machine learning methods for the exploration of P.falciparum protein antigen candidates. We combined an extensive literature search with domain expertise to create high-quality labels for P.falciparum specific proteins that distinguish between antigen candidates and intracellular proteins. Additionally, we used our benchmark to compare different well-known prediction models and available protein localization prediction services on the task of identifying protein antigen candidates. We show that available general-purpose services are unable to provide sufficient performance on identifying protein antigen candidates and are outperformed by our models that were trained on this tailored data. PlasmoFAB is publicly available on Zenodo with DOI 10.5281/zenodo.7433087. Furthermore, all scripts that were used in the creation of PlasmoFAB and the training and evaluation of machine learning models are open source and publicly available on GitHub here: https://github.com/msmdev/PlasmoFAB.

ОЦЕНКА ТЕХНИЧЕСКОГО СОСТОЯНИЯ ГАЗОТУРБИННОЙ УСТАНОВКИ ПРИ ПОМОЩИ МЕТОДОВ МАШИННОГО ОБУЧЕНИЯ ПО ИСКУССТВЕННЫМ ДАННЫМ

Continuous monitoring of the technical condition of gas turbines, defect identification, failure prevention, and optimization of operation, maintenance, and repair processes are relevant tasks for the operators of this equipment. Various machine learning methods that are already being used in the field of gas turbines can help solve these tasks. The limiting factor in this regard is the lack of real operational data. This study examines the possibility of using synthetic data for training and testing machine learning models to determine the level of technical condition of a gas turbine installation. An open dataset created by other researchers using a mathematical model of a marine gas turbine engine was selected for analysis. The research presents the accuracy values obtained by different methods of evaluating machine learning models. The random forest model demonstrated the best results. It was found that when developing machine learning-based solutions for engineering tasks, additional methods for assessing the accuracy of predictions are required. The further development of this work is associated with the development of a proprietary mathematical model of a gas turbine installation capable of considering the influence of specific defects to create datasets for analysis and further research

Evaluation of machine learning models for prediction of daily reference evapotranspiration in semi-arid India

Reference evapotranspiration (ET ) is controlled by climatic factors; hence, its estimation provides an idea 0 about the atmospheric demand of water. Machine learning techniques like elastic net (ELNET), K-nearest neighbours (KNN), multivariate adaptive regression splines (MARS), partial least squares regression (PSLR), random forest (RF), support vector regression (SVR), XGBoost and cubist were employed to predict daily reference evapotranspiration based on daily weather parameters of twenty years. Penman-Monteith method was used as the reference method for ET estimation. All models performed well during calibration showing 0 higher coefficient of determination (R2) which ranged from 0.97 (for PLSR) to 1 (for cubist models). Mean absolute error during calibration ranges from 0.027 mm d-1 for cubist to 0.607 mm d-1 for ELNET. Cubist model (R2 = 1, MAE = 0.017 mm d-1, RMSE = 0.027 mm d-1) outperformed other models during the calibration. During validation, the coefficient of determination (R2) for the machine learning models varied from 0.819 to 1, RMSE varied from 0.06 to 0.60 mm d-1 and MAE varied from 0.031 to 0.38 mm d-1. Based on statistical parameters, best performance was observed for cubist model (R2 = 1, RMSE = 0.06 mm d-1, MAE = 0.031 mm d-1) among the studied machine learning models for the prediction of reference evapotranspiration. Hence, the cubist model may be used to estimate daily reference evapotranspiration for the studied region.

Performance Evaluation of Machine Learning Models for Cyber Threat Detection and Prevention in Mobile Money Services

Performance Evaluation of Machine Learning Models for Cyber Threat Detection and Prevention in Mobile Money Services