Standard Machine Learning Methods Research Articles

Genome-Wide Association Study of Treatment-Resistant Depression: Shared Biology With Metabolic Traits.

Treatment-resistant depression (TRD) occurs in roughly one-third of all individuals with major depressive disorder (MDD). Although research has suggested a significant common variant genetic component of liability to TRD, with heritability estimated at 8% when compared with non-treatment-resistant MDD, no replicated genetic loci have been identified, and the genetic architecture of TRD remains unclear. A key barrier to this work has been the paucity of adequately powered cohorts for investigation, largely because of the challenge in prospectively investigating this phenotype. The objective of this study was to perform a well-powered genetic study of TRD. Using receipt of electroconvulsive therapy (ECT) as a surrogate for TRD, the authors applied standard machine learning methods to electronic health record data to derive predicted probabilities of receiving ECT. These probabilities were then applied as a quantitative trait in a genome-wide association study of 154,433 genotyped patients across four large biobanks. Heritability estimates ranged from 2% to 4.2%, and significant genetic overlap was observed with cognition, attention deficit hyperactivity disorder, schizophrenia, alcohol and smoking traits, and body mass index. Two genome-wide significant loci were identified, both previously implicated in metabolic traits, suggesting shared biology and potential pharmacological implications. This work provides support for the utility of estimation of disease probability for genomic investigation and provides insights into the genetic architecture and biology of TRD.

Auditory event-related potential differentiates girls with Rett syndrome from their typically-developing peers with high accuracy: Machine learning study

Rett Syndrome (RTT) is a rare neurodevelopmental disorder caused by mutation in the MECP2 gene. No cures are still available, but several clinical trials are ongoing. Here we examine neurophysiological correlates of auditory processing for ability to differentiate patients with RTT from typically developing (TD) peers applying standard machine learning (ML) methods and pipelines. Capitalized on the available event-related potential (ERP) data recorded in response to tone presented at different rates (stimulus onset asynchrony 900, 1800 and 3600 ms) from 24 patients with RTT and 27 their TD peer. We considered the most common ML models that are widely used for classification tasks. These include both linear models (logistic regression, support-vector machine with linear kernel) and tree-based nonlinear models (random forest, gradient boosting). Based on these methods we were able to differentiate RTT from TD children with high accuracy (with up to 0.94 ROC-AUC score), which was evidently higher at the fastest presentation rate. Importance analysis and perturbation importance pointed out that the most important feature for classification is P2-N2 peak-to-peak amplitude, consistently across the approaches and blocks with different presentation rate. The results suggest the unique pattern of ERP characteristics for RTT and points to features of importance. The results might be relevant for establishing outcome measures for clinical trials.

Should we really use graph neural networks for transcriptomic prediction?

The recent development of deep learning methods have undoubtedly led to great improvement in various machine learning tasks, especially in prediction tasks. This type of methods have also been adapted to answer various problems in bioinformatics, including automatic genome annotation, artificial genome generation or phenotype prediction. In particular, a specific type of deep learning method, called graph neural network (GNN) has repeatedly been reported as a good candidate to predict phenotypes from gene expression because its ability to embed information on gene regulation or co-expression through the use of a gene network. However, up to date, no complete and reproducible benchmark has ever been performed to analyze the trade-off between cost and benefit of this approach compared to more standard (and simpler) machine learning methods. In this article, we provide such a benchmark, based on clear and comparable policies to evaluate the different methods on several datasets. Our conclusion is that GNN rarely provides a real improvement in prediction performance, especially when compared to the computation effort required by the methods. Our findings on a limited but controlled simulated dataset shows that this could be explained by the limited quality or predictive power of the input biological gene network itself.

Gated recurrent unit network: A promising approach to corporate default prediction

AbstractThis paper presents a promising approach using gated recurrent unit (GRU) network to predict bankruptcy based on the whole sequence of financial statements of the companies listed on an unregulated market. This approach contrasts with the traditional literature where default prediction is usually tackled with methods that do not fully account for a company's history. The GRU network can be used to model long‐term dependencies thanks to its update and reset gates, which prevent the vanishing gradient problem and decide how much of the past information is relevant for predicting a default. This aspect may be of utmost importance in alternative markets where the signal detection problem is particularly strong. The performance of the GRU network is compared against the performance of other standard machine learning methods including Cox proportional hazards, gradient‐boosted Cox proportional hazards model, extreme gradient boosting, random survival forest, and standard recurrent neural networks on the basis of a broad selection of performance metrics. The GRU network not only outperforms standard machine learning methods in out‐of‐sample forecasts but also seems to be more robust in terms of in‐ versus out‐of‐sample performance.

Machine learning classification based on k-Nearest Neighbors for PolSAR data.

In this work, we focus on obtaining insights of the performances of some well-known machine learning image classification techniques (k-NN, Support Vector Machine, randomized decision tree and one based on stochastic distances) for PolSAR (Polarimetric Synthetic Aperture Radar) imagery. We test the classifiers methods on a set of actual PolSAR data and provide some conclusions. The aim of this work is to show that suitable adapted standard machine learning methods offer excellent performances vs.computational complexity trade-off for PolSAR image classification. In this work, we evaluate well-known machine learning techniques for PolSAR (Polarimetric Synthetic Aperture Radar) image classification, including K-Nearest Neighbors (KNN), Support Vector Machine (SVM), randomized decision tree, and a method based on the Kullback-Leibler stochastic distance. Our experiments with real PolSAR data show that standard machine learning methods, when adapted appropriately, offer a favourable trade-off between performance and computational complexity. The KNN and SVM perform poorly on these data, likely due to their failure to account for the inherent speckle presence and properties of the studied reliefs. Overall, our findings highlight the potential of the Kullback-Leibler stochastic distance method for PolSAR image classification.

S-TBN: A New Neural Decoding Model to Identify Stimulus Categories From Brain Activity Patterns.

Neural decoding is still a challenging and a hot topic in neurocomputing science. Recently, many studies have shown that brain network patterns containing rich spatiotemporal structural information represent the brain's activation information under external stimuli. In the traditional method, brain network features are directly obtained using the standard machine learning method and provide to a classifier, subsequently decoding external stimuli. However, this method cannot effectively extract the multidimensional structural information hidden in the brain network. Furthermore, studies on tensors have show that the tensor decomposition model can fully mine unique spatiotemporal structural characteristics of a spatiotemporal structure in data with a multidimensional structure. This research proposed a stimulus-constrained Tensor Brain Network (s-TBN) model that involves the tensor decomposition and stimulus category-constraint information. The model was verified on real neuroimaging data obtained via magnetoencephalograph and functional mangetic resonance imaging). Experimental results show that the s-TBN model achieve accuracy matrices of greater than 11.06% and 18.46% on the accuracy matrix compared with other methods on two modal datasets. These results prove the superiority of extracting discriminative characteristics using the STN model, especially for decoding object stimuli with semantic information.

A Comparison of Standard Statistical, Machine Learning and Deep Learning Methods in Forecasting the Time Series

A Comparison of Standard Statistical, Machine Learning and Deep Learning Methods in Forecasting the Time Series

A Deep Learning Approach to Real-time Electricity Load Forecasting

In light of the increasing importance of accurate and real-time electrical demand forecasting, this research presents a deep learning model with the goal of dramatically improving predictive accuracy. Conventional methods of forecasting, such as linear regression, have trouble capturing the complex patterns included in data about electricity usage. Standard machine learning methods are shown to be wanting when compared to the suggested deep Long Short-Term Memory (LSTM) model. Mean Absolute Error (MAE) of 5.454 and Mean Squared Error (MSE) of 18.243 demonstrate the deep LSTM model's proficiency in tackling this problem. The linear regression, on the other hand, achieved a MAE of 47.352 and an MSE of 65.606, which is lower than the proposed model. Because of its greater predictive precision and reliability, the deep LSTM model is a viable option for accurate, real-time prediction of electricity demand.

Panel data nowcasting: The case of price–earnings ratios

AbstractThe paper uses structured machine learning regressions for nowcasting with panel data consisting of series sampled at different frequencies. Motivated by the problem of predicting corporate earnings for a large cross‐section of firms with macroeconomic, financial, and news time series sampled at different frequencies, we focus on the sparse‐group LASSO regularization which can take advantage of the mixed‐frequency time series panel data structures. Our empirical results show the superior performance of our machine learning panel data regression models over analysts' predictions, forecast combinations, firm‐specific time series regression models, and standard machine learning methods.

Human behavior in free search online shopping scenarios can be predicted from EEG activation using Hjorth parameters.

The present work investigates whether and how decisions in real-world online shopping scenarios can be predicted based on brain activation. Potential customers were asked to search through product pages on e-commerce platforms and decide, which products to buy, while their EEG signal was recorded. Machine learning algorithms were then trained to distinguish between EEG activation when viewing products that are later bought or put into the shopping card as opposed to products that are later discarded. We find that Hjorth parameters extracted from the raw EEG can be used to predict purchase choices to a high level of accuracy. Above-chance predictions based on Hjorth parameters are achieved via different standard machine learning methods with random forest models showing the best performance of above 80% prediction accuracy in both 2-class (bought or put into card vs. not bought) and 3-class (bought vs. put into card vs. not bought) classification. While conventional EEG signal analysis commonly employs frequency domain features such as alpha or theta power and phase, Hjorth parameters use time domain signals, which can be calculated rapidly with little computational cost. Given the presented evidence that Hjorth parameters are suitable for the prediction of complex behaviors, their potential and remaining challenges for implementation in real-time applications are discussed.

In search of a precursor for crystal nucleation of hard and charged colloids.

The interplay between crystal nucleation and the structure of the metastable fluid has been a topic of significant debate over recent years. In particular, it has been suggested that even in simple model systems such as hard or charged colloids, crystal nucleation might be foreshadowed by significant fluctuations in local structure around the location where the nucleus first arises. We investigate this using computer simulations of spontaneous nucleation events in both hard and charged colloidal systems. To detect local structural variations, we use both standard and unsupervised machine learning methods capable of finding hidden structures in the metastable fluid phase. We track numerous nucleation events for the face-centered cubic and body-centered cubic crystals on a local level and demonstrate that all signs of crystallinity emerge simultaneously from the very start of the nucleation process. We thus conclude that we observe no precursor for the crystal nucleation of hard and charged colloids.

DeepAmes: A deep learning-powered Ames test predictive model with potential for regulatory application

The Ames assay is required by the regulatory agencies worldwide to assess the mutagenic potential risk of consumer products. As well as this in vitro assay, in silico approaches have been widely used to predict Ames test results as outlined in the International Council for Harmonization (ICH) guidelines. Building on this in silico approach, here we describe DeepAmes, a high performance and robust model developed with a novel deep learning (DL) approach for potential utility in regulatory science. DeepAmes was developed with a large and consistent Ames dataset (>10,000 compounds) and was compared with other five standard Machine Learning (ML) methods. Using a test set of 1,543 compounds, DeepAmes was the best performer in predicting the outcome of Ames assay. In addition, DeepAmes yielded the best and most stable performance up to when compounds were >30% outside of the applicability domain (AD). Regarding the potential for regulatory application, a revised version of DeepAmes with a much-improved sensitivity of 0.87 from 0.47. In conclusion, DeepAmes provides a DL-powered Ames test predictive model for predicting the results of Ames tests; with its defined AD and clear context of use, DeepAmes has potential for utility in regulatory application.

Machine Learning Seams of Conical Intersection: A Characteristic Polynomial Approach.

The machine learning of potential energy surfaces (PESs) has undergone rapid progress in recent years. The vast majority of this work, however, has been focused on the learning of ground state PESs. To reliably extend machine learning protocols to excited state PESs, the occurrence of seams of conical intersections between adiabatic electronic states must be correctly accounted for. This introduces a serious problem, for at such points, the adiabatic potentials are not differentiable to any order, complicating the application of standard machine learning methods. We show that this issue may be overcome by instead learning the coordinate-dependent coefficients of the characteristic polynomial of a simple decomposition of the potential matrix. We demonstrate that, through this approach, quantitatively accurate machine learning models of seams of conical intersection may be constructed.

Small patient datasets reveal genetic drivers of non-small cell lung cancer subtypes using machine learning for hypothesis generation

Aim: Many small datasets of significant value exist in the medical space that are being underutilized. Due to the heterogeneity of complex disorders found in oncology, systems capable of discovering patient subpopulations while elucidating etiologies are of great value as they can indicate leads for innovative drug discovery and development. Methods: Two small non-small cell lung cancer (NSCLC) datasets (GSE18842 and GSE10245) consisting of 58 samples of adenocarcinoma (ADC) and 45 samples of squamous cell carcinoma (SCC) were used in a machine intelligence framework to identify genetic biomarkers differentiating these two subtypes. Utilizing a set of standard machine learning (ML) methods, subpopulations of ADC and SCC were uncovered while simultaneously extracting which genes, in combination, were significantly involved in defining the subpopulations. A previously described interactive hypothesis-generating method designed to work with ML methods was employed to provide an alternative way of extracting the most important combination of variables to construct a new data set. Results: Several genes were uncovered that were previously implicated by other methods. This framework accurately discovered known subpopulations, such as genetic drivers associated with differing levels of aggressiveness within the SCC and ADC subtypes. Furthermore, phyosphatidylinositol glycan anchor biosynthesis, class X (PIGX) was a novel gene implicated in this study that warrants further investigation due to its role in breast cancer proliferation. Conclusions: The ability to learn from small datasets was highlighted and revealed well-established properties of NSCLC. This showcases the utility of ML techniques to reveal potential genes of interest, even from small datasets, shedding light on novel driving factors behind subpopulations of patients.

Detection of Monkeypox Cases Based on Symptoms Using XGBoost and Shapley Additive Explanations Methods.

The monkeypox virus poses a novel public health risk that might quickly escalate into a worldwide epidemic. Machine learning (ML) has recently shown much promise in diagnosing diseases like cancer, finding tumor cells, and finding COVID-19 patients. In this study, we have created a dataset based on the data both collected and published by Global Health and used by the World Health Organization (WHO). Being entirely textual, this dataset shows the relationship between the symptoms and the monkeypox disease. The data have been analyzed, using gradient boosting methods such as Extreme Gradient Boosting (XGBoost), CatBoost, and LightGBM along with other standard machine learning methods such as Support Vector Machine (SVM) and Random Forest. All these methods have been compared. The research aims to provide an ML model based on symptoms for the diagnosis of monkeypox. Previous studies have only examined disease diagnosis using images. The best performance has belonged to XGBoost, with an accuracy of 1.0 in reviews. To check the model's flexibility, k-fold cross-validation is used, reaching an average accuracy of 0.9 in 5 different splits of the test set. In addition, Shapley Additive Explanations (SHAP) helps in examining and explaining the output of the XGBoost model.

Explanatory Interactive Machine Learning

The most promising standard machine learning methods can deliver highly accurate classification results, often outperforming standard white-box methods. However, it is hardly possible for humans to fully understand the rationale behind the black-box results, and thus, these powerful methods hamper the creation of new knowledge on the part of humans and the broader acceptance of this technology. Explainable Artificial Intelligence attempts to overcome this problem by making the results more interpretable, while Interactive Machine Learning integrates humans into the process of insight discovery. The paper builds on recent successes in combining these two cutting-edge technologies and proposes how Explanatory Interactive Machine Learning (XIL) is embedded in a generalizable Action Design Research (ADR) process – called XIL-ADR. This approach can be used to analyze data, inspect models, and iteratively improve them. The paper shows the application of this process using the diagnosis of viral pneumonia, e.g., Covid-19, as an illustrative example. By these means, the paper also illustrates how XIL-ADR can help identify shortcomings of standard machine learning projects, gain new insights on the part of the human user, and thereby can help to unlock the full potential of AI-based systems for organizations and research.

Integrating and optimizing genomic, weather, and secondary trait data for multiclass classification.

Modern plant breeding programs collect several data types such as weather, images, and secondary or associated traits besides the main trait (e.g., grain yield). Genomic data is high-dimensional and often over-crowds smaller data types when naively combined to explain the response variable. There is a need to develop methods able to effectively combine different data types of differing sizes to improve predictions. Additionally, in the face of changing climate conditions, there is a need to develop methods able to effectively combine weather information with genotype data to predict the performance of lines better. In this work, we develop a novel three-stage classifier to predict multi-class traits by combining three data types-genomic, weather, and secondary trait. The method addressed various challenges in this problem, such as confounding, differing sizes of data types, and threshold optimization. The method was examined in different settings, including binary and multi-class responses, various penalization schemes, and class balances. Then, our method was compared to standard machine learning methods such as random forests and support vector machines using various classification accuracy metrics and using model size to evaluate the sparsity of the model. The results showed that our method performed similarly to or better than machine learning methods across various settings. More importantly, the classifiers obtained were highly sparse, allowing for a straightforward interpretation of relationships between the response and the selected predictors.

Machine Learning Applications in Malware Classification: A Meta-Analysis Literature Review

With a text mining and bibliometrics approach, this study reviews the literature on the evolution of malware classification using machine learning. This work takes literature from 2008 to 2022 on the subject of using machine learning for malware classification to understand the impact of this technology on malware classification. Throughout this study, we seek to answer three main research questions: RQ1: Is the application of machine learning for malware classification growing? RQ2: What is the most common machine-learning application for malware classification? RQ3: What are the outcomes of the most common machine learning applications? The analysis of 2186 articles resulting from a data collection process from peerreviewed databases shows the trajectory of the application of this technology on malware classification as well as trends in both the machine learning and malware classification fields of study. This study performs quantitative and qualitative analysis using statistical and N-gram analysis techniques and a formal literature review to answer the proposed research questions. The research reveals methods such as support vector machines and random forests to be standard machine learning methods for malware classification in efforts to detect maliciousness or categorize malware by family. Machine learning is a highly researched technology with many applications, from malware classification and beyond.

Athlete’s Performance Analysis Based on Improved Machine Learning Approach on Wearable Devices

Today, every nation strives for international recognition in a variety of sports. Governments invest in games and sports to raise the performance of their teams and athletes to get notoriety. Numerous people are involved in sports execution, including team management, coaches, and biomechanists who monitor athlete fitness and work to achieve remarkable results. Performance analysis is greatly aided by technological integration in sports management. The performance analysis of athletes is evaluated in this research using an upgraded machine learning approach on Improved Machine Learning approach on Wearable Devices (IMLA-WD). This design strategy utilizes wearable devices to collect health data, which is then fed into a machine-learning model to monitor athletes' progress. The athletes' performance is evaluated using standard machine learning methods, and the deep neural network monitors their health status. With a health prediction accuracy of 98.65%, the statistical findings of the proposed model demonstrate the highest performance compared to existing methodologies.

Clinical applications of artificial intelligence and machine learning in the modern cardiac intensive care unit

The depth and breadth of data produced in the modern cardiac intensive care unit (CICU) poses challenges to clinicians and researchers. Artificial intelligence (AI) and machine learning (ML) methodologies have been increasingly used to provide insights into this complex patient population. Major analytical tasks where ML methodology can be applied in the CICU and other critical care settings include mortality risk stratification, prognostication, non-fatal event prediction, diagnosis, phenotyping, identification of occult heart disease from the electrocardiogram and interpretation of echocardiographic images. In this review, we will discuss existing and future applications of different ML methods for CICU and other critical care populations, including penalized regression, standard ML methods (e.g., tree-based and other non-linear approaches) and advanced ML methods (e.g., deep learning and neural networks). While comparatively few published studies have applied ML methods in CICU populations, a more robust literature including patients with acute cardiovascular disease and non-cardiovascular critical illness can provide insights into CICU care. The CICU of the future is likely to utilize a sophisticated array of ML algorithms to streamline patient care by facilitating early recognition, diagnosis, phenotyping, and intervention for critically ill or deteriorating patients to improve providers’ cognitive load.