ML Methods Research Articles

Machine learning methods, applications and economic analysis to predict heart failure hospitalisation risk: a scoping review protocol

IntroductionMachine learning (ML) has emerged as a powerful tool for uncovering patterns and generating new information. In cardiology, it has shown promising results in predictive outcomes risk assessment of heart...

An Efficient Ensemble Machine Learning Model for Cardiovascular Disease Prediction Using Digital Health Records

Abstract: Detecting cardiovascular problems early is crucial for timely treatment. In our study, we employed machine learning to analyze a diverse set of information about individuals' lives and health, to predict cardiovascular disease. Ensuring data accuracy and addressing missing information were prioritized in our approach. Experimenting with different solo ML & ensemble ML methods, comprised of Random Forest and XGBoost with tuning, we achieved a notable 92% accuracy in identifying potential heart issues. Remarkably, combining multiple machine learning methods through ensemble learning proved even more effective than individual methods. Expanding our methodology to include Light GBM, Extra Tree, Decision Tree, SVM, Naive Bayes, QDA, & Adaboost enhanced the comprehensiveness of our analysis. Additionally, delving into ensemble learning methods such as bagging, boosting, tuning, & stacking further pushed the boundaries of predictive accuracy. In essence, our research outstands the potency of diverse ensemble machine-learning techniques and algorithms in early cardiovascular prediction. Ensemble methods, which combine different algorithms, emerged as powerful tools without relying on complex terminology.

Clinical Features Predicting COVID-19 Severity Risk at the Time of Hospitalization.

The global spread of COVID-19 has led to significant mortality and morbidity worldwide. Early identification of COVID-19 patients who are at high risk of developing severe disease can help in improved patient management, care, and treatment, as well as in the effective allocation of hospital resources. The severity prediction at the time of hospitalization can be extremely helpful in deciding the treatment of COVID-19 patients. To this end, this study presents an interpretable artificial intelligence (AI) model, named COVID-19 severity predictor (CoSP) that predicts COVID-19 severity using the clinical features at the time of hospital admission. We utilized a dataset comprising 64 demographic and laboratory features of 7,416 confirmed COVID-19 patients that were collected at the time of hospital admission. The proposed hierarchical CoSP model performs four-class COVID severity risk prediction into asymptomatic, mild, moderate, and severe categories. CoSP yielded better performance with good interpretability, as observed via Shapley analysis on COVID severity prediction compared to the other popular ML methods, with an area under the received operating characteristic curve (AUC-ROC) of 0.95, an area under the precision-recall curve (AUPRC) of 0.91, and a weighted F1-score of 0.83. Out of 64 initial features, 19 features were inferred as predictive of the severity of COVID-19 disease by the CoSP model. Therefore, an AI model predicting COVID-19 severity may be helpful for early intervention, optimizing resource allocation, and guiding personalized treatments, potentially enabling healthcare professionals to save lives and allocate resources effectively in the fight against the pandemic.

Characterization of the complete mitochondrial genome of Spirobolus grahami (Diplopoda: Spirobolidae) with phylogenetic analysis

Diplopoda is one of the most diverse and important groups of soil arthropods, but little research has been done on their phylogenetic relationship and evolution. Here, we sequenced and annotated the complete mitochondrial genomes of Spirobolus grahami. The total mitogenome of S. grahami was typical circular, double-stranded molecules, with 14,875 bp in length, including 13 protein-coding genes, 22 tRNAs, two rRNAs, and one control region. Base composition analysis suggested that the mitochondrial sequences were biased toward A and T, with A + T content of 58.68%. The mitogenomes of S. grahami exhibited negative AT and positive GC skews. Most of the 13 PCGs had ATN as the start codon, except COX1 start with CGA, and most PCGs ended with the T stop codon. The dN/dS values for most PCGs were lower than 1, suggesting that purifying selection was likely the main driver of mitochondrial PCG evolution. Phylogenetic analyses based on 13 PCGs using BI and ML methods support the classification of genus Spirobolus and Tropostreptus. Glomeridesmus spelaeus is distantly related to the other Diplopoda species.

Prediction of thrust bearing’s performance in Mixed Lubrication regime

A hydrodynamic thrust bearing could be forced to operate in mixed lubrication regime under various circumstances. At this state, the tribological characteristics of the bearing could be affected significantly and the developed phenomena would have a severe impact on the performance of the mechanism. Until recently, researchers were modeling the hydrodynamic lubrication problem of the thrust bearings either with analytical or with numerical solutions. The analytical solutions are very simple and do not provide enough accuracy in describing the actual problem. To add to that, following only computational methodologies, can lead to time consuming and complex algorithms that need to be repeated every time the operating conditions change, in order to draw safe conclusions. Recent technological advances, especially on the field of computer science, have provided tools that enhance and accelerate the modeling of thrust bearings’ operation. The aim of this study is to examine the application of Artificial Neural Networks as Machine Learning models, that are trained to predict the coefficient of friction for lubricated pad thrust bearings in mixed lubrication regime. The hydrodynamic analysis of the thrust bearing is performed by solving the Average 2-D Reynolds equation numerically. In order to describe the roughness of the profiles, both the flow factors suggested by N. Patir and H.S. Cheng (1978) and the model of J.A. Greenwood and J. H. Tripp (1970) are taken into consideration. Three lubricants, the SAE 0W30, the SAE 10W40 and the SAE 10W60, are tested and compared for a variety of operating velocities and applied coatings. The numerical analysis results are used as training datasets for the machine learning algorithms. Four different ML methods are applied in this investigation: Artificial Neural Networks (ANNs), Multi- Variable Quadratic Polynomial Regression, Quadratic SVM and Regression Trees. The coefficient of determination, R2 is calculated and used to determine the most accurate ML method for the current study. The results showed that ANNs provide very good accuracy in the prediction of friction coefficient compared to the rest of the ML models discussed.

When Causal Inference Meets Graph Machine Learning

Graphs (i.e., networks) are ubiquitous in daily life, as they can effectively model a plethora of real-world systems with connected units, such as social networks and biological networks. Recent years have witnessed rapid development in graph-based machine learning (GML) in various high-impact domains. Currently, the mainstream GML methods are based on statistical learning, e.g., utilizing the statistical correlations between node features, graph structure, and labels for node classification. However, statistical learning has been widely criticized for only capturing the superficial relations between variables in the data system, and consequently, rendering the lack of trustworthiness in real-world applications. Therefore, it is crucial to understand the causality in the data system and the learning process. Causal inference is the discipline that investigates the causality inside a system, for example, to identify and estimate the causal effect of a certain treatment (e.g., wearing a face mask) on an important outcome (e.g., COVID-19 infection). Involving the concepts and philosophy of causal inference in ML methods is often considered significant for human-level intelligence and can serve as the foundation of artificial intelligence (AI). However, most traditional causal inference studies rely on strong assumptions, and focus on independent and identically distributed (i.i.d.) data, while causal inference on graphs is faced with many barriers. Therefore, we aim to bridge the gap between causal inference and GML.

HyperFast: Instant Classification for Tabular Data

Training deep learning models and performing hyperparameter tuning can be computationally demanding and time-consuming. Meanwhile, traditional machine learning methods like gradient-boosting algorithms remain the preferred choice for most tabular data applications, while neural network alternatives require extensive hyperparameter tuning or work only in toy datasets under limited settings. In this paper, we introduce HyperFast, a meta-trained hypernetwork designed for instant classification of tabular data in a single forward pass. HyperFast generates a task-specific neural network tailored to an unseen dataset that can be directly used for classification inference, removing the need for training a model. We report extensive experiments with OpenML and genomic data, comparing HyperFast to competing tabular data neural networks, traditional ML methods, AutoML systems, and boosting machines. HyperFast shows highly competitive results, while being significantly faster. Additionally, our approach demonstrates robust adaptability across a variety of classification tasks with little to no fine-tuning, positioning HyperFast as a strong solution for numerous applications and rapid model deployment. HyperFast introduces a promising paradigm for fast classification, with the potential to substantially decrease the computational burden of deep learning. Our code, which offers a scikit-learn-like interface, along with the trained HyperFast model, can be found at https://github.com/AI-sandbox/HyperFast.

Rapid Visual Screening Feature Importance for Seismic Vulnerability Ranking via Machine Learning and SHAP Values

Structures inevitably suffer damage after an earthquake, with severity ranging from minimal damage of nonstructural elements to partial or even total collapse, possibly with loss of human lives. Thus, it is essential for engineers to understand the crucial factors that drive a structure towards suffering higher degrees of damage in order for preventative measures to be taken. In the present study, we focus on three well-known damage thresholds: the Collapse Limit State, Ultimate Limit State, and Serviceability Limit State. We analyze the features obtained via Rapid Visual Screening to determine whether or not a given structure crosses these thresholds. To this end, we use machine learning to perform binary classification for each damage threshold, and use explainability to quantify the effect of each parameter via SHAP values (SHapley Additive exPlanations). The quantitative results that we obtain demonstrate the potential applicability of ML methods for recalibrating the computation of structural vulnerability indices using data from recent earthquakes.

Early diagnosis of obsessives-compulsive disorder through gene expression analysis using machine learning models

The primary aim of this research study is to develop an early diagnosis method for obsessive-compulsive disorder (OCD) by utilizing gene expression analysis and Machine Learning techniques. Gene expression data from both blood and brain samples were collected from the gene expression omnibus (GEO) database. As OCD cannot currently be detected through instruments, it relies on clinical symptoms that are often misinterpreted. To address this, a novel hybrid feature selection approach that combines statistical and ML methods is proposed to identify down-regulated genes that may play a crucial role in the development of OCD. The results of the earlier studies point to important implications and emphasize the significance of down-regulated gene expression in OCD. Currently, gene expression profiling is used as an investigative tool to identify the specific cell receptors associated with certain conditions, followed by targeted medication to alleviate symptoms. Our proposed method achieved high accuracy rates of 83% for blood data and 92% for brain data when compared to other feature selection methods such as MIFS, CFS, and mRMR, using various Machine Learning models. These results demonstrate the effectiveness of our approach in early OCD diagnosis using gene expression analysis. https://github.com/Naseerullah-Qureshi/classify_gene_expression/blob/main/gene_expression.txt.

Machine Learning for Stimulated Reservoir Volume (SRV) Prediction Using 4-D Micro-seismic Data

New methodology of stable, high accuracy estimation and optimization of stimulated reservoir volume (SRV) forecast is presented in this paper. It includes time-related data segmentation, new multilevel feature engineering, analysis of associations and importance of engineered variables. Among first-level feature engineered variables are three quantile-type variables qRangeDepth, qRangeNorth, and qRangeEast. hese three quantile-type variables are used for SRV estimation. In addition to quantile-type variables, two first level variables -trange and event minute are constructed as the first -level variables. These two variables give compact characterization of distribution of microseismic events in time and are used as predictor variables in ML SRV forecast. Second and third level engineered variables are built via transformation of variables of the first level. Although in this paper we focus on the SRV forecast, the same ideas are applicable to the characterization and forecasting of the plume volume in carbon storage and monitoring applications. A linear regression method and two ML methods - random forest, and regression tree are used for the SRV forecast. It is demonstrated that in the case of selection of appropriate set of first and second level predictor variables even simplistic linear regression may produce accurate SRV forecasts. Still, machine learning methods produce more accurate forecasts characterized by high values of accuracy parameters r.squared and correlation between SRV and its forecast values. Our results can have a significant impact on the proper design of a hydraulic fracturing operation. It can also be used for monitoring CO2 plume in carbon sequestration sites.

Analyse and Predict the Detection of the Cyber - Attack Process by Using a Machine-Learning Approach

Crimes committed online rank among the most critical global concerns. Daily, they cause country and citizen economies to suffer massive financial losses. With the proliferation of cyber-attacks, cybercrime has also been on the rise. To effectively combat cybercrime, it is essential to identify its perpetrators and understand their methods. Identifying and preventing cyber-attacks are difficult tasks. To combat these concerns, however, new research has produced safety models and forecast tools grounded on artificial intelligence. Numerous methods for predicting criminal behaviour are available in the literature. While they may not be perfect, they may help in cybercrime and cyber-attack tactic prediction. To find out whether an attack happened and, if so, who was responsible, one way to look at this problem is by using real-world data. There is data about the crime, the perpetrator's demographics, the amount of property damaged, and the entry points for the assault. Potentially, by submitting applications to forensics teams, victims of cyber-attacks may get information. This study uses ML methods to analyse cyber-crime consuming two patterns and to forecast how the specified characteristics will furnish to the detection of the cyber-attack methodology and perpetrator. Based on the comparison of eight distinct machine-learning methods, one can say that their accuracy was quite comparable. The Support Vector Machine (SVM) Linear outperformed all other cyber-attack tactics in terms of accuracy. The initial model gave us a decent notion of the assaults that the victims would face. The most successful technique for detecting malevolent actors was logistic regression, according to the success rate. To anticipate who the perpetrator and victim would be, the second model compared their traits. A person’s chances of being a victim of a cyber-attack decrease as their income and level of education rise. The proposed idea is expected to be used by departments dealing with cybercrime. Cyber-attack identification will also be made easier, and the fight against them will be more efficient.

Estimating Fuel Consumption of an Agricultural Robot by Applying Machine Learning Techniques during Seeding Operation

The integration of agricultural robots in precision farming plays a pivotal role in tackling the pressing demands of minimizing energy usage, enhancing productivity, and maximizing crop yield to meet the needs of an expanding global population and depleting non-renewable resources. Evaluating the energy expenditure is vital when assessing agricultural machinery systems. Through the reduction of fuel consumption, operational costs can be curtailed while simultaneously minimizing the overall environmental footprint left by these machines. Accurately calculating fuel usage empowers farmers to make well-informed decisions about their farming operations, resulting in more sustainable and productive methods. In this study, the ASABE model was applied to predict the fuel consumption of the studied robot. Results show that the ASABE model can predict the fuel consumption of the robot with an average error equal to 27.5%. Moreover, different machine-learning techniques were applied to develop an effective and novel model for estimating the fuel consumption of an agricultural robot. The proposed GPR model (gaussian process regression) considers four operational features of the studied robot: total operational time, total traveled distance, automatic working distance, and automatic turning distance. The GPR model with four features, considering hyperparameter optimization, showed the best performance (R-squared validation = 0.93, R-squared test = 1.00) among other models. Furthermore, three different ML methods (gradient boosting, random forest, and XGBoost) were considered in this study and compared with the developed GPR model. The results show that the GPR model outperformed the mentioned models. Moreover, the one-way ANOVA test results revealed that the predicted values from the GPR model and observation do not have significantly different means. The results of the sensitivity analysis show that the traveled distance and the total time have a significant correlation with the fuel consumption of the studied robot.

The first two whole mitochondrial genomes for the genus Dactylis species: assembly and comparative genomics analysis

BackgroundOrchardgrass (Dactylis glomerata L.), a perennial forage, has the advantages of rich leaves, high yield, and good quality and is one of the most significant forage for grassland animal husbandry and ecological management in southwest China. Mitochondrial (mt) genome is one of the major genetic systems in plants. Studying the mt genome of the genus Dactylis could provide more genetic information in addition to the nuclear genome project of the genus.ResultsIn this study, we sequenced and assembled two mitochondrial genomes of Dactylis species of D. glomerata (597, 281 bp) and D. aschersoniana (613, 769 bp), based on a combination of PacBio and Illumina. The gene content in the mitochondrial genome of D. aschersoniana is almost identical to the mitochondrial genome of D. glomerata, which contains 22–23 protein-coding genes (PCGs), 8 ribosomal RNAs (rRNAs) and 30 transfer RNAs (tRNAs), while D. glomerata lacks the gene encoding the Ribosomal protein (rps1) and D. aschersoniana contains one pseudo gene (atp8). Twenty-three introns were found among eight of the 30 protein-coding genes, and introns of three genes (nad 1, nad2, and nad5) were trans-spliced in Dactylis aschersoniana. Further, our mitochondrial genome characteristics investigation of the genus Dactylis included codon usage, sequences repeats, RNA editing and selective pressure. The results showed that a large number of short repetitive sequences existed in the mitochondrial genome of D. aschersoniana, the size variation of two mitochondrial genomes is due largely to the presence of a large number of short repetitive sequences. We also identified 52–53 large fragments that were transferred from the chloroplast genome to the mitochondrial genome, and found that the similarity was more than 70%. ML and BI methods used in phylogenetic analysis revealed that the evolutionary status of the genus Dactylis.ConclusionsThus, this study reveals the significant rearrangements in the mt genomes of Pooideae species. The sequenced Dactylis mt genome can provide more genetic information and improve our evolutionary understanding of the mt genomes of gramineous plants.

Machine Learning Inspired Nanowire Classification Method based on Nanowire Array Scanning Electron Microscope Images

Background This article introduces an innovative classification methodology to identify nanowires within scanning electron microscope images. Methods Our approach employs advanced image manipulation techniques in conjunction with machine learning-based recognition algorithms. The effectiveness of our proposed method is demonstrated through its application to the categorization of scanning electron microscopy images depicting nanowires arrays. Results The method’s capability to isolate and distinguish individual nanowires within an array is the primary factor in the observed accuracy. The foundational data set for model training comprises scanning electron microscopy images featuring 240 III-V nanowire arrays grown with metal organic chemical vapor deposition on silicon substrates. Each of these arrays consists of 66 nanowires. The results underscore the model’s proficiency in discerning distinct wire configurations and detecting parasitic crystals. Our approach yields an average F1 score of 0.91, indicating high precision and recall. Conclusions Such a high level of performance and accuracy of ML methods demonstrate the viability of our technique not only for academic but also for practical commercial implementation and usage.

Comparing Performances of Predictive Models of Toxicity after Radiotherapy for Breast Cancer Using Different Machine Learning Approaches.

Different ML models were compared to predict toxicity in RT on a large cohort (n = 1314). The endpoint was RTOG G2/G3 acute toxicity, resulting in 204/1314 patients with the event. The dataset, including 25 clinical, anatomical, and dosimetric features, was split into 984 for training and 330 for internal tests. The dataset was standardized; features with a high p-value at univariate LR and with Spearman ρ>0.8 were excluded; synthesized data of the minority were generated to compensate for class imbalance. Twelve ML methods were considered. Model optimization and sequential backward selection were run to choose the best models with a parsimonious feature number. Finally, feature importance was derived for every model. The model's performance was compared on a training-test dataset over different metrics: the best performance model was LightGBM. Logistic regression with three variables (LR3) selected via bootstrapping showed performances similar to the best-performing models. The AUC of test data is slightly above 0.65 for the best models (highest value: 0.662 with LightGBM). No model performed the best for all metrics: more complex ML models had better performances; however, models with just three features showed performances comparable to the best models using many (n = 13-19) features.

Multi-Temporal Sentinel-1 and Sentinel-2 Data for Orchards Discrimination in Khairpur District, Pakistan Using Spectral Separability Analysis and Machine Learning Classification

Generating orchards spatial distribution maps within a heterogeneous landscape is challenging and requires fine spatial and temporal resolution images. This study examines the effectiveness of Sentinel-1 (S1) and Sentinel-2 (S2) satellite data of relatively high spatial and temporal resolutions for discriminating major orchards in the Khairpur district of the Sindh province, Pakistan using machine learning methods such as random forest (RF) and a support vector machine. A Multicollinearity test (MCT) was performed among the multi-temporal S1 and S2 variables to remove those with high correlations. Six different feature combination schemes were tested, with the fusion of multi-temporal S1 and S2 (scheme-6) outperforming all other combination schemes. The spectral separability between orchards pairs was assessed using Jeffries-Matusita (JM) distance, revealing that orchard pairs were completely separable in the multi-temporal fusion of both sensors, especially the indistinguishable pair of dates-mango. The performance difference between RF and SVM was not significant, SVM showed a slightly higher accuracy, except for scheme-4 where RF performed better. This study concludes that multi-temporal fusion of S1 and S2 data, coupled with robust ML methods, offers a reliable approach for orchard classification. Prospectively, these findings will be helpful for orchard monitoring, improvement of yield estimation and precision based agricultural practices.

Predicting Individual Treatment Effects: Challenges and Opportunities for Machine Learning and Artificial Intelligence

AbstractPersonalized medicine seeks to identify the right treatment for the right patient at the right time. Predicting the treatment effect for an individual patient has the potential to transform treatment of patients and drastically improve patients outcomes. In this work, we illustrate the potential for ML and AI methods to yield useful predictions of individual treatment effects. Using the predicted individual treatment effects (PITE) framework which uses baseline covariates (features) to predict whether a treatment is expected to yield benefit for a given patient compared to an alternative intervention we provide an illustration of the potential of such approaches and provide a detailed discussion of opportunities for further research and open challenges when seeking to predict individual treatment effects.

A combination of multivariate statistics and machine learning techniques in groundwater characterization and quality forecasting

Globally, the quality of groundwater has proven to have been affected by some natural and human activities in recent years. To ensure there is good drinking water (Sustainable Development Goal 6.3, there is a need to elucidate the groundwater quality status of the area of interest. The groundwater in the northwestern parts of Ghana is not yet well characterized. Hence, this study employed a multi-method approach of hydrochemistry, water quality index (WQI), multivariate statistics, and machine models: multiple linear regression (MLR), decision tree regression (DTR), random forest regression (RFR), and artificial neural network (ANN), are combined in the characterization and prediction of the water quality in the area. They are robust in providing conclusions on groundwater assessment that can be relied upon for decision-making processes regarding groundwater usage and monitoring. Except for NO3− and TDS exceeding their standard levels in 22 and 2 locations, respectively, the other physicochemical parameters are within acceptable limits. The groundwater is generally good for domestic usage based on the WQI, with 79.2% of excellent to good waters. The groundwater evolved from Na-type, Cl-type, and Cl(SO4)-Ca(Mg) facies. Agricultural activities are the main source of human impact on the groundwater. Silicate mineral dissolution and ion exchange processes are the natural processes that affect groundwater mineralization, with mineral dissolution being the dominant process. Based on the performance metrics: MAE, MSE and RMSE of the ML methods considered in the WQI forecasting, the order of performance of the models is ANN > RFR > DTR > MLR, with the following respective R2 values 0.9974, 0.9193, 0.8966 and 0.8886.

Predicting gold accessibility from mineralogical characterization using machine learning algorithms

Information about accessibility is of great relevance for gold recovery studies. Obtaining these variables from machine learning models can greatly assist in quickly determining accessibility. Few studies have been published relating the mineralogy of the gold ore process and the application of artificial intelligence, mainly algorithms in predicting variables related to gold recovery and extraction. Accessibility is an important variable for understanding the ability to recover gold from a cyanide solution, which can occur through fractures or some other means that provides access to the solution and consequent leaching of the gold grain. This study aims to present a model capable of predicting the accessibility variable using a data set with 168 characterization results from different ML methods, such as Linear Regression (LR), Random Forest (RF), Sequential Minimum Optimization for Support Vector Machine (SMOreg) and Gaussian Processes (GP). In this context, it was possible to establish that the random forest model performed best by presenting a coefficient of determination R2 (0.77), MAE (11.76), and RMSE (14.48). It was also reported from the SHAP analysis that the Au_grade, exposed_a, and As_grade showed the highest contribution level towards the perdition process of the model.

Sentiment Analysis Using Deep Learning Approaches on Multi-Domain Dataset in Telugu Language

Recent advancements in Natural Language Processing (NLP) have made sentiment analysis an essential component of a variety of NLP jobs, including recommendation systems, question answering, and business intelligence products. While sentiment analysis research has, to put it mildly, been widely pursued in English, Telugu has barely ever attempted the task. The majority of research works concentrate on analysing the sentiments of Tweets, news, or reviews containing Hindi and English words. There is a growing interest among academics in studying how people express their thoughts and views in Indian languages like Bengali, Telugu, Malayalam, Tamil and so on. Due to a paucity of labelled datasets, microscopic investigation on Indian languages has been published to our knowledge. This work suggested a sentence-level sentiment analysis on multi-domain datasets that has been collected in Telugu. Deep learning models have been used in this work because it demonstrates the significant expertise in sentiment analysis and is widely regarded as the cutting-edge model in Telugu Sentiment Analysis. Our proposed work investigates a productive Bidirectional Long Short-Term Memory (BiLSTM) Network and Bidirectional GRU Network (BiGRU) for improving Telugu Sentiment Analysis by encapsulating contextual information from Telugu feature sequences using Forward-Backward encapsulation. Further, the model has been deployed by merging the domains so as to predict the accuracy and other performance metrics. The experimental test findings show that the deep learning models outperform when compared with the baseline traditional ML methods in four benchmark sentiment analysis datasets. There is evidence that the proposed sentiment analysis method has improved precision, recall, F1-score and accuracy in certain cases. The proposed model has achieved the F1-score of 86% for song datasets when compared with the other existing models.