Machine Learning Classifiers Research Articles

XGBoost-SHAP-based interpretable diagnostic framework for knee osteoarthritis: a population-based retrospective cohort study

ObjectiveTo use routine demographic and clinical data to develop an interpretable individual-level machine learning (ML) model to diagnose knee osteoarthritis (KOA) and to identify highly ranked features.MethodsIn this retrospective, population-based cohort study, anonymized questionnaire data was retrieved from the Wu Chuan KOA Study, Inner Mongolia, China. After feature selections, participants were divided in a 7:3 ratio into training and test sets. Class balancing was applied to the training set for data augmentation. Four ML classifiers were compared by cross-validation within the training set and their performance was further analyzed with an unseen test set. Classifications were evaluated using sensitivity, specificity, positive predictive value, negative predictive value, accuracy, area under the curve(AUC), G-means, and F1 scores. The best model was explained using Shapley values to extract highly ranked features.ResultsA total of 1188 participants were investigated in this study, among whom 26.3% were diagnosed with KOA. Comparatively, XGBoost with Boruta exhibited the highest classification performance among the four models, with an AUC of 0.758, G-means of 0.800, and F1 scores of 0.703. The SHAP method reveals the top 17 features of KOA according to the importance ranking, and the average of the experience of joint pain was recognized as the most important features.ConclusionsOur study highlights the usefulness of machine learning in unveiling important factors that influence the diagnosis of KOA to guide new prevention strategies. Further work is needed to validate this approach.

Prediction of talent selection in elite male youth soccer across 7 seasons: A machine-learning approach

ABSTRACT This study aimed to investigate the relative importance of parameters from several domains associated to both selecting or de-selecting players with regards to the next age group within a professional German youth soccer academy across a 7-year period. Following a mixed-longitudinal approach, physical, physiological, psychological, skill-, health-, age-, and position-related parameters were collected from 409 male players (980 datapoints) from the U12 to U19 age groups. Supervised machine learning classifiers were used to predict the selection status regarding the next age group. The XGBoost models (ROC-AUC: 0.69, F1-Score: 0.84) revealed that physical and physiological (linear sprint, change-of-direction sprint, countermovement jump, aerobic speed reserve) as well as skill-related parameters (soccer-specific skill) were most important for being selected or de-selected regarding the next age group across the entire sample and all age groups. The majority of psychological parameters (motive structure, motive attention, motive competition, cognitive flexibility) were of medium importance. No clear pattern was observed for the health-, age-, and position-related parameters. Our study provides insights into key parameters for talent selection thereby contributing to an overall talent management strategy in highly trained youth soccer players. In particular, coaches and key stakeholders might focus on physical, physiological, and skill-related parameters.

Early diagnosis of Alzheimer’s disease and mild cognitive impairment using MRI analysis and machine learning algorithms

Early diagnosis of Alzheimer’s disease (AD) and mild cognitive impairment (MCI) is crucial to prevent their progression. In this study, we proposed the analysis of magnetic resonance imaging (MRI) based on features including; hippocampus (HC) area size, HC grayscale statistics and texture features (mean, standard deviation, skewness, kurtosis, contrast, correlation, energy, homogeneity, entropy), lateral ventricle (LV) area size, gray matter area size, white matter area size, cerebrospinal fluid area size, patient age, weight, and cognitive score. Five machine learning classifiers; K-nearest neighborhood (KNN), support vector machine (SVM), random forest (RF), decision tree (DT), and multi-layer perception (MLP) were used to distinguish between groups: cognitively normal (CN) vs AD, early MCI (EMCI) vs late MCI (LMCI), CN vs EMCI, CN vs LMCI, AD vs EMCI, and AD vs LMCI. Additionally, the correlation and dependence were calculated to examine the strength and direction of association between each extracted feature and each classification of the group. The average classification accuracies in 20 trials were 95% (SVM), 71.50% (RF), 82.58% (RF), 84.91% (SVM), 85.83% (RF), and 85.08% (RF), respectively, with the best accuracies being 100% (SVM, RF, and MLP), 83.33% (RF), 91.66% (RF), 95% (SVM, and MLP), 96.66% (RF), and 93.33% (DT). Cognitive scores, HC and LV area sizes, and HC texture features demonstrated significant potential for diagnosing AD and its subtypes for all groups. RF and SVM showed better performance in distinguishing between groups. These findings highlight the importance of using 2D-MRI to identify key features containing critical information for early diagnosis of AD.

Effective DDoS attack detection in software-defined vehicular networks using statistical flow analysis and machine learning.

Vehicular Networks (VN) utilizing Software Defined Networking (SDN) have garnered significant attention recently, paralleling the advancements in wireless networks. VN are deployed to optimize traffic flow, enhance the driving experience, and ensure road safety. However, VN are vulnerable to Distributed Denial of Service (DDoS) attacks, posing severe threats in the contemporary Internet landscape. With the surge in Internet traffic, this study proposes novel methodologies for effectively detecting DDoS attacks within Software-Defined Vehicular Networks (SDVN), wherein attackers commandeer compromised nodes to monopolize network resources, disrupting communication among vehicles and between vehicles and infrastructure. The proposed methodology aims to: (i) analyze statistical flow and compute entropy, and (ii) implement Machine Learning (ML) algorithms within SDN Intrusion Detection Systems for Internet of Things (IoT) environments. Additionally, the approach distinguishes between reconnaissance, Denial of Service (DoS), and DDoS traffic by addressing the challenges of imbalanced and overfitting dataset traces. One of the significant challenges in this integration is managing the computational load and ensuring real-time performance. The ML models, especially complex ones like Random Forest, require substantial processing power, which necessitates efficient data handling and possibly leveraging edge computing resources to reduce latency. Ensuring scalability and maintaining high detection accuracy as network traffic grows and evolves is another critical challenge. By leveraging a minimal subset of features from a given dataset, a comparative study is conducted to determine the optimal sample size for maximizing model accuracy. Further, the study evaluates the impact of various dataset attributes on performance thresholds. The K-nearest Neighbor, Random Forest, and Logistic Regression supervised ML classifiers are assessed using the BoT-IoT dataset. The results indicate that the Random Forest classifier achieves superior performance metrics, with Precision, F1-score, Accuracy, and Recall rates of 92%, 92%, 91%, and 90%, respectively, over five iterations.

A benchmark for computational analysis of animal behavior, using animal-borne tags

BackgroundAnimal-borne sensors (‘bio-loggers’) can record a suite of kinematic and environmental data, which are used to elucidate animal ecophysiology and improve conservation efforts. Machine learning techniques are used for interpreting the large amounts of data recorded by bio-loggers, but there exists no common framework for comparing the different machine learning techniques in this domain. This makes it difficult to, for example, identify patterns in what works well for machine learning-based analysis of bio-logger data. It also makes it difficult to evaluate the effectiveness of novel methods developed by the machine learning community.MethodsTo address this, we present the Bio-logger Ethogram Benchmark (BEBE), a collection of datasets with behavioral annotations, as well as a modeling task and evaluation metrics. BEBE is to date the largest, most taxonomically diverse, publicly available benchmark of this type, and includes 1654 h of data collected from 149 individuals across nine taxa. Using BEBE, we compare the performance of deep and classical machine learning methods for identifying animal behaviors based on bio-logger data. As an example usage of BEBE, we test an approach based on self-supervised learning. To apply this approach to animal behavior classification, we adapt a deep neural network pre-trained with 700,000 h of data collected from human wrist-worn accelerometers.ResultsWe find that deep neural networks out-perform the classical machine learning methods we tested across all nine datasets in BEBE. We additionally find that the approach based on self-supervised learning out-performs the alternatives we tested, especially in settings when there is a low amount of training data available.ConclusionsIn light of these results, we are able to make concrete suggestions for designing studies that rely on machine learning to infer behavior from bio-logger data. Therefore, we expect that BEBE will be useful for making similar suggestions in the future, as additional hypotheses about machine learning techniques are tested. Datasets, models, and evaluation code are made publicly available at https://github.com/earthspecies/BEBE, to enable community use of BEBE.

A multimodal deep-learning model based on multichannel CT radiomics for predicting pathological grade of bladder cancer.

To construct a predictive model using deep-learning radiomics and clinical risk factors for assessing the preoperative histopathological grade of bladder cancer according to computed tomography (CT) images. A retrospective analysis was conducted involving 201 bladder cancer patients with definite pathological grading results after surgical excision at the organization between January 2019 and June 2023. The cohort was classified into a test set of 81 cases and a training set of 120 cases. Hand-crafted radiomics (HCR) and features derived from deep-learning (DL) were obtained from computed tomography (CT) images. The research builds a prediction model using 12 machine-learning classifiers, which integrate HCR, DL features, and clinical data. Model performance was estimated utilizing decision-curve analysis (DCA), the area under the curve (AUC), and calibration curves. Among the classifiers tested, the logistic regression model that combined DL and HCR characteristics demonstrated the finest performance. The AUC values were 0.912 (training set) and 0.777 (test set). The AUC values of clinical model achieved 0.850 (training set) and 0.804 (test set). The AUC values of the combined model were 0.933 (training set) and 0.824 (test set), outperforming both the clinical and HCR-only models. The CT-based combined model demonstrated considerable diagnostic capability in differentiating high-grade from low-grade bladder cancer, serving as a valuable noninvasive instrument for preoperative pathological evaluation.

Variability in hot sub-luminous stars and binaries: Machine-learning analysis of Gaia DR3 multi-epoch photometry

Hot sub-luminous stars represent a population of stripped and evolved red giants that is located on the extreme horizontal branch. Since they exhibit a wide range of variability due to pulsations or binary interactions, it is crucial to unveil their intrinsic and extrinsic variability to understand the physical processes of their formation. In the Hertzsprung-Russell diagram, they overlap with interacting binaries such as cataclysmic variables (CVs). By leveraging the most recent clustering algorithm tools, we investigate the variability of 1,576 candidate hot subdwarf variables using comprehensive data from Gaia DR3 multi-epoch photometry and Transiting Exoplanet Survey Satellite (TESS) observations. We present a novel approach that uses the t-distributed stochastic neighbour embedding and the uniform manifold approximation and projection dimensionality reduction algorithms to facilitate the identification and classification of different populations of variable hot subdwarfs and CVs in a large dataset. In addition to the publicly available Gaia time-series statistics table, we adopted additional statistical features that enhanced the performance of the algorithms. The clustering results led to the identification of 85 new hot subdwarf variables based on Gaia and TESS light curves and of 108 new variables based on Gaia light curves alone, including reflection-effect systems, HW Vir, ellipsoidal variables, and high-amplitude pulsating variables. A significant number of known CVs (140) distinctively cluster in the 2D feature space among an additional 152 objects that we consider candidates for new CVs. This study paves the way for more efficient and comprehensive analyses of stellar variability from ground- and space-based observations, and for the application of machine-learning classifications of candidate variable stars in large surveys.

ChatGPT-Assisted Machine Learning for Chronic Disease Classification and Prediction: A Developmental and Validation Study

ChatGPT-Assisted Machine Learning for Chronic Disease Classification and Prediction: A Developmental and Validation Study

Ransomware Over Modern Web Browsers: A Novel Strain and a New Defense Mechanism

Ransomware is an increasingly prevalent form of malware targeting end-users, governments, and businesses. As it has evolved, adversaries added new capabilities to their arsenal. We propose a next-generation browser-based ransomware, RøB , which performs its malicious actions via web technologies, File System Access API (FSA) and WebAssembly (Wasm). RøB uses this API through the victims’ browsers; hence, it does not require the victims to download and install malicious binaries. We performed extensive evaluations with 3 different OSs, 23 file formats, 29 distinct directories, 5 cloud providers, and 4 antivirus solutions. Our evaluations show that RøB can encrypt various types of files in the local and cloud-integrated directories, external storage devices, and network-shared folders of victims. Our experiments also reveal that popular cloud solutions, Box Individual and Apple iCloud can be severely affected by RøB . Moreover, we conducted tests with commercial antivirus software such as AVG, Avast, Kaspersky, Malware Bytes that perform sensitive directory and suspicious behavior monitoring against ransomware. We verified that RøB can evade these antivirus software and encrypt victim files. Moreover, existing ransomware detection solutions in the literature also cannot be a remedy against RøB due to its distinct features. Therefore, in this paper, we also propose RøBguard , a new detection system for RøB -like attacks. RøBguard monitors the web applications that use the FSA API via function hooking and uses a machine learning classifier to detect RøB -like attacks. We implemented a proof of concept version of RøBguard and our evaluation results show that RøBguard can detect RøB -like browser-based ransomware attacks effectively. We also provide future research directions that should be addressed in this domain.

Mapping land cover in the low Arctic using multiple sources of earth observation data

Land cover maps in high latitude regions are important for permafrost modeling and mapping and other climate change related studies. National and global land cover products cannot meet the requirements for regional permafrost mapping due to their broad land cover types. This study produced a land cover map in a low Arctic region using multi-source earth observation data, including different modes of RADARSAT-2 (RS2), Sentinel 2 (S2), and the high resolution ArcticDEM. We assessed the performances of two machine learning classification algorithms (Random Forest (RF) and Support Vector Machine (SVM)) with different combinations of input data, including evaluating the effects of incidence angle from different RS2 modes on classification results. This study concludes that the combination of two imaging modes of RS2, S2 composites and the high resolution ArcticDEM achieves a more accurate land cover map (the overall accuracy is 93.6%) than other combinations with the overall accuracies ranging from 38% to 87%; the performance of RF is better than that of SVM, the overall accuracy difference ranges from 21% to 1%; and the shallow SAR incidence angles outperform steep ones in classification results of different combinations for both classifiers.

MRI brain tumor classification based on CNN features and machine learning classifiers

MRI brain tumor classification based on CNN features and machine learning classifiers

Machine learning derived physical activity in preschool children with developmental coordination disorder.

To compare the device-measured physical activity behaviours of preschool children with typical motor development to those with probable developmental coordination disorder (pDCD) and at risk for developmental coordination disorder (DCDr). A total of 497 preschool children (4-5 years) in the Coordination and Activity Tracking in CHildren (CATCH) study completed repeated motor assessments and wore an ActiGraph GT3X on the right hip at baseline for 1 week. We calculated physical activity metrics from raw accelerometer data using a validated random forest classification machine learning model for preschool-age children. Analysis of variance (ANOVA) and linear regression models compared physical activity between typically developing children, children at risk for DCDr, and those with pDCD identified based on motor scores at baseline and averaged over time, accounting for age, sex, and accelerometer wear time. We found no differences in daily time spent sedentary, in light physical activity, or moderate-to-vigorous physical activity between typically developing children, children at risk for DCDr, and those with pDCD. However, children in the DCD groups spent less time doing ambulatory activities (walking/running) than typically developing children. Analysis of variance: baseline classification, DCDr to typically developing, run: F = 5.34, p = 0.005, classification averaged over time, DCDr to typically developing, walk: F = 5.82, p = 0.003. Regressions: DCDr compared to typically developing for walk: B = -3.47 (standard error 1.05), p < 0.001, pDCD compared to typically developing for run: B = -1.82 (standard error 0.62), p = 0.004. Designing interventions for preschool children with motor difficulties targeting specific physical activity types (walk/run) may help mitigate physical activity intensity differences observed later in childhood.

Does Using Artificial Intelligence in Citizen Science Support Volunteers&amp;rsquo; Learning? An Experimental Study in Ornithology

One of the oldest and largest biodiversity-related citizen science (CS) projects is eBird (https://ebird.org/home), developed by the Cornell Lab of Ornithology. It provides a mobile application for birdwatchers to record checklists of when, where, and how they have seen or heard birds. The Cornell Lab has also developed a mobile application, Merlin, which uses a deep convolutional neural network to help users automatically identify bird species from photos, sounds (converted to spectrograms), or descriptions. This research investigates how the use of machine learning (ML) classification models affects the learning of novice birders. Our participants (computer science students with no previous background in ornithology) were randomly divided into three groups: one using the eBird application and identifying bird species themselves; one using the Merlin application, which uses ML to automatically identify birds from photos or sounds; and a control group. Participants were tested on their knowledge of birds before and after participating in the project to see how using the ML classification model affected their learning. We also interviewed selected participants after the post-test to understand what they had done and what might explain the results. Our results show that novice participants who participate in a CS project for even a short time significantly improve their content knowledge of familiar birds in their neighbourhood, and that eBird users outperform Merlin users on the knowledge post-test. Although AI may improve volunteer productivity and retention, there is a risk that it may reduce their learning. Further research with different participant profiles and project designs is needed to understand how to optimise volunteer productivity, retention, and learning in AI-assisted CS projects.

Mediation Analysis of Diabetes and Heart Diseases Influenced by Obesity Using Machine Learning Classifiers

Purpose:This study examines the impact of body mass index on diabetes and heart disease among Indians. Multi-morbidity ailments are associated with diabetes. To understand the relationship between diabetes, body mass index, and heart disease, a study is undertaken.Methods:The present study established a relationship between diabetes, heart disease and body mass index using mediation analysis and machine learning classifiers. R software with Hayes Macro Process and Python was used as a statistical tool to conclude the study.Results:As a result of the present findings, the body mass index mediates the relationship between diabetes and heart disease and cannot be countered. This study's indirect impact is 4.6231, statistically significant at a 95% confidence interval (3.1333, 9.6556). Thesignificance of the indirect effect of diabetes on heart disease is evident as (BootLLCI), and (BootULCI) are both positive anddo not contain zero. This indicates that there is a substantial mediation effect present. In classification, the TensorFlow classifiershows 99% accuracy and 97% precession, while the Linear S.V.C., NuSVC and Logistic Regression have an accuracy of 98%, 96% and 97%, which shows that the machine learning classifiers are more significant for the study.Conclusion:Our study examines how Body Mass Index (B.M.I.) mediates diabetes and heart disease, which are statistically significant. Despite the close relationship between heart disease and diabetes, little is known about the pathways involved. Machine Learning Classifiers show that the risk of diabetes, heart disease and other diseases increases due to deterioration of body mass index.

Mutual-Energy Inner Product Optimization Method for Constructing Feature Coordinates and Image Classification in Machine Learning

As a key task in machine learning, data classification is essential to find a suitable coordinate system to represent the data features of different classes of samples. This paper proposes the mutual-energy inner product optimization method for constructing a feature coordinate system. First, by analyzing the solution space and eigenfunctions of the partial differential equations describing a non-uniform membrane, the mutual-energy inner product is defined. Second, by expressing the mutual-energy inner product as a series of eigenfunctions, it shows the significant advantage of enhancing low-frequency features and suppressing high-frequency noise, compared to the Euclidean inner product. And then, a mutual-energy inner product optimization model is built to extract the data features, and the convexity and concavity properties of its objective function are discussed. Next, by combining the finite element method, a stable and efficient sequential linearization algorithm is constructed to solve the optimization model. This algorithm only solves positive definite symmetric matrix equations and linear programming with a few constraints, and its vectorized implementation is discussed. Finally, the mutual-energy inner product optimization method is used to construct feature coordinates, and multi-class Gaussian classifiers are trained on the MINST training set. Good prediction results of the Gaussian classifiers are achieved on the MINST test set.

Fusion Learning from Non-contrast CT Scans for the Detection of Hemorrhagic Transformation in Stroke Patients.

Hemorrhagic transformation (HT) is a potentially catastrophic complication after acute ischemic stroke. Prevention of HT risk is crucial because it worsens prognosis and increases mortality. This study aimed at developing and validating a computer-aided diagnosis system using pretreatment non-contrast computed tomography (CT) scans for HT prediction in stroke patients undergoing revascularization. This retrospective study included all acute ischemic stroke patients with non-contrast CT before reperfusion therapy who also underwent follow-up MRI from January 2018 to December 2022. Among the 188 evaluated patients, any degree of HT at follow-up imaging was observed in 103 patients. HT diagnosis via MRI was defined as the reference standard for neuroradiologists. Using a database of 2076 serial non-contrast CT images of the brain, pretrained deep learning architectures such as convolutional neural networks and vision transformers (ViTs) were used for feature extraction. The performance of the predictive HT risk model was evaluated via tenfold cross-validation in machine learning classifiers. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were evaluated. Using an individual deep learning architecture, DenseNet201 features achieved the highest accuracy of 87% and an AUC of 0.8863 in the classifier of the subspace ensemble k-nearest neighbor. By combining the DenseNet201 and ViT features, the accuracy and AUC can be improved to 88% and 0.8987, respectively, which are significantly better than those of using ViT alone. Detecting HT in stroke patients is a meaningful but challenging issue. On the basis of the model approach, HT diagnosis would be more automatic, efficient, and consistent, which would be helpful in clinic use.

Improving Computer Vision Interpretability: Transparent Two-Level Classification for Complex Scenes

Abstract Treating images as data has become increasingly popular in political science. While existing classifiers for images reach high levels of accuracy, it is difficult to systematically assess the visual features on which they base their classification. This paper presents a two-level classification method that addresses this transparency problem. At the first stage, an image segmenter detects the objects present in the image and a feature vector is created from those objects. In the second stage, this feature vector is used as input for standard machine learning classifiers to discriminate between images. We apply this method to a new dataset of more than 140,000 images to detect which ones display political protest. This analysis demonstrates three advantages to this paper’s approach. First, identifying objects in images improves transparency by providing human-understandable labels for the objects shown on an image. Second, knowing these objects enables analysis of which distinguish protest images from non-protest ones. Third, comparing the importance of objects across countries reveals how protest behavior varies. These insights are not available using conventional computer vision classifiers and provide new opportunities for comparative research.

Identifying crowdfunding storytellers who deliver successful projects: a machine learning approach

Crowdfunding plays a key role in financial technology to provide individuals and enterprises with funding opportunities to establish start-ups and/or new business ventures. It is mainly used to link projects’ creators and backers, collect money and plan fundraising projects via social networks. This paper proposes a machine learning-enabled approach to analyse Kickstarter numerical and textual data and predict the successful funding and delivery of crowdfunding projects. It offers crowdfunding stakeholders benefits including creator credibility assessment, project risk reduction, and backer confidence enhancement. This research proposes a data preprocessing approach to prepare the dataset and extract the relevant features for the predictions. Besides, it trains and compares five numerical machine learning classification models and three text-mining methods to find the best-fitted numerical and textual analysis approaches. According to the results, the proposed SVM model outperforms the numerical benchmarks in terms of Accuracy, Precision, Recall, F1 score, and model Training latency. Moreover, BERT gives the best results if the dataset is complex, while Word2vec works better with simple features in textual analysis.

Prediction of Disease Stage by Machine Learning Classification Methods for Covid-19 Patients

Supervised machine learning classificitaion algorithms have been widely used in many fields in recent years. Especially, health is one of the most important areas where machine learning studies are carried out successfully. The aim of this study is to develop models that predict the disease stage of people who apply to hospital with the diagnosis of Covid-19. Inadequacies such as intensive care occupancy, insufficiency of beds, and shortage of respiratory equipment are among these problems, and this has left healthcare workers faced with the overwhelming burden of patients. Therefore, estimating the disease stages of Covid-19 patients at an early stage is of great importance. The data set used in the study includes the clinical and laboratory data of the patients during in their admission to the hospital.It has been tried to develop models that predict disease stage by using Logistic Regression, Random Forest and Support Vector Machine algorithms in the data set. The random forest model with 9 variables was the best performing model. With the models obtained, it will be ensured that the hospital management receives information in order to see the necessary treatment for low-risk or high-risk patients and to avoid medical system inadequacies.

Predicting Online Shopping Behavior: Using Machine Learning and Google Analytics to Classify User Engagement

User engagement metrics, including engaged sessions, average engagement time, bounce rate, and conversions, provide significant insights into online behavior. This study utilizes Google Analytics data insights and predictive statistics to analyze these metrics and apply classification models to enhance digital marketing strategies. Relationships among key metrics including event count, sessions, purchase revenue, transactions, and bounce rate, were examined using descriptive statistics, revealing factors affecting user engagement. Machine learning classifiers, such as decision trees (DTs), Naive Bayes (NB), and k-nearest neighbors (k-NN), were assessed for their effectiveness in classifying engagement levels. DTs achieved a classification accuracy of 97.98%, outperforming NB (65.00%) and k-NN (97.90%). Furthermore, techniques like pruning are applied for performance optimization. Primarily, this paper goas is to generate a series of recommendations to help the decision-makers and marketers optimizing the marketing strategies. This study highlights the significance of artificial intelligence (AI) integration in digital marketing as a best practice for optimizing decision-making processes.