Statistical Learning Algorithms Research Articles

Comparative analysis of the predictive power of statistical models of macroeconomic indicators in conditions of permanent crises

Subject. The article discusses prediction of the state of the economy, the accuracy of forecasts of traditional models during crises, the need to find more effective model specifications to predict macro indicators. Objectives. The purpose is to carry out a comparative analysis of the predictive ability of ensemble methods in comparison with a set of models, including traditional statistical algorithms and machine learning algorithms. Methods. The comparative analysis of predictive ability of the models and interpretation of results obtained were performed using a dynamic factor model (DFM), a neural network with long-term short-term memory (LSTM), and integrated gradient methods (IG). Results. We performed the analysis of predictive ability of the ensemble model to forecast GDP, which combines DFM and LSTM to account for both linear and nonlinear dependencies in the data; the analysis of predictive power of various indicators, which showed that an increase in forecast error is observed for all models except DFM, the ensemble model with an error correction structure, and ARMAX. The obtained results can be used to build models of macroeconomic indicators in order to make strategic decisions by enterprises of various industries operating in a highly uncertain environment. Conclusions. The combination of DFM and LSTM in the ensemble provides higher accuracy forecasts than LSTM and competitor models, however, with less predictive power than DFM.

Spatiotemporal modelling of airborne birch and grass pollen concentration across Switzerland: A comparison of statistical, machine learning and ensemble methods

BackgroundStatistical and machine learning models are commonly used to estimate spatial and temporal variability in exposure to environmental stressors, supporting epidemiological studies. We aimed to compare the performances, strengths and limitations of six different algorithms in the retrospective spatiotemporal modeling of daily birch and grass pollen concentrations at a spatial resolution of 1 km across Switzerland. MethodsDaily birch and grass pollen concentrations were available from 14 measurement sites in Switzerland for 2000–2019. To develop the spatiotemporal models, we considered spatiotemporal, spatial and temporal predictors including meteorological factors, land-use, elevation, species distribution and Normalized Difference Vegetation Index (NDVI). We used six statistical and machine learning algorithms: LASSO, Ridge, Elastic net, Random forest, XGBoost and ANNs. We optimized model structures through feature selection and grid search techniques to obtain the best predictive performance. We used train-test split and cross-validation to avoid overfitting and overoptimistic performance indicators. We then combined these six models through multiple linear regression to develop an ensemble hybrid model. ResultsThe 5th-95th percentiles of birch and grass pollen concentrations were 0–151 and 0–105 grains/m3, respectively. The hybrid ensemble model achieved the best RMSE on the test dataset for both birch and grass pollen with 94.4 and 19.7 grains/m3, respectively.Nonlinear models (Random forest, XGBoost and ANNs) achieved lower test RMSE's than linear models (LASSO, Ridge, Elastic net) for both pollen types, with RMSE's ranging from 105.9 to 140.5 grains/m3 for birch and from 20.0 to 25.4 grains/m3 for grass pollen. The Random forest algorithm yielded the best spatial and temporal performance among the six evaluated modelling methods. The ensemble hybrid model outperformed the six linear and nonlinear algorithms. Country-wide pollen concentration, land use, weather, and NDVI were important predictors. ConclusionNonlinear algorithms outperformed linear models and accurately explained complex, nonlinear relationships between environmental factors and measured concentrations.

Advanced Cybersecurity Enhancement through Mathematical Models: A Comprehensive Approach Using Statistical Methods, Cryptographic Techniques, and Machine Learning Algorithms (SVM, Random Forest, and Neural Networks) for Behavior-Based Malware Detection

Advanced Cybersecurity Enhancement through Mathematical Models: A Comprehensive Approach Using Statistical Methods, Cryptographic Techniques, and Machine Learning Algorithms (SVM, Random Forest, and Neural Networks) for Behavior-Based Malware Detection

Markers of early post-stroke cognitive impairment

Aim. To identify significant indicators of cognitive dysfunction based on discriminant analysis and to assess the influence of the course, nature and localization of ischemic stroke on the cognitive status of the patient.Materials and methods. We examined 290 patients diagnosed with ischemic stroke in the carotid artery area. Depending on presence of cognitive dysfunction according to the Montreal Cognitive Assessment Scale (MoSA) patients were divided into 2 groups: 240 patients with cognitive decline (≤25 point by MoCA) and 50 patients without it. In order to verify the markers, anamnestic characteristics were assessed, cognitive-functional indicators (according to the scales of the National Institutes of Health, MoCA, Bartel, Rankin, IQCODE questionnaire, additional scales to assess praxis, semantic aphasia, perception and executive function), data of neuroimaging studies. For statistical analysis machine learning algorithms and Python with its libraries (Pandas and SciPy) were implied.Results. The main neuropsychological indicators for patients with early post-stroke cognitive impairment were decline in the areas of perception, executive function, memory and semantic information processing, affective disturbances and physical fatigue. Relevant indicators identified during estimation of the instrumental and clinical examination results were severity of IS, left frontal and right parietal localisations of ischemia focus, presence of cortical atrophy and leukoaraiosis.Conclusion. Based on multi-factor analysis of clinical and paraclinical parameters using machine learning algorithms, the main markers of cognitive decline of early post-stroke impairments were identified. This will allow us to optimise the choice of neurocognitive rehabilitation strategies and to personalise the approach in the further management of the stroke patient.

Revolutionising waste management with the impact of Long Short-Term Memory networks on recycling rate predictions

This study explores the efficacy of Long Short-Term Memory (LSTM) networks in predicting recycling rates and enhancing resource allocation in waste management systems. It addresses the limitations of traditional statistical models and machine learning algorithms that struggle with sequential data and temporal dependencies. The methodology comprised collecting extensive datasets from public repositories, configuring the LSTM network architecture, training the model with historical data, and testing various activation functions and hyperparameters. The model’s performance was rigorously compared to traditional models and alternative machine learning algorithms using metrics such as Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and R-squared (R2). The findings demonstrate that the LSTM model significantly outperformed traditional approaches, achieving an MAE of 3.5%, an RMSE of 2.8%, and an R2 of 0.92. These results underscore the superior capability of LSTM networks to capture complex temporal patterns in recycling data, offering substantial improvements in predictive accuracy and reliability. Consequently, the study highlights the potential of LSTM networks to revolutionize waste management strategies, contributing to more effective and sustainable practices.

The Multicollinearity Effect on the Performance of Machine Learning Algorithms: Case Examples in Healthcare Modelling

Background: The data extracted from various fields inherently consists of extremely correlated measurements in parallel with the exponential increase in the size of the data that need to be interpreted owing to the technological advances. This problem, called the multicollinearity, influences the performance of both statistical and machine learning algorithms. Statistical models proposed as a potential remedy to this problem have not been sufficiently evaluated in the literature. Therefore, a comprehensive comparison of statistical and machine learning models is required for addressing the multicollinearity problem. Methods: Statistical models (including Ridge, Liu, Lasso and Elastic Net regression) and the eight most important machine learning algorithms (including Cart, Knn, Mlp, MARS, Cubist, Svm, Bagging and XGBoost) are comprehensively compared by using two different healthcare datasets (including Body Fat and Cancer) having multicollinearity problem. The performance of the models is assessed through cross validation methods via root mean square error, mean absolute error and r-squared criteria. Results: The results of the study revealed that statistical models outperformed machine learning models in terms of root mean square error, mean absolute error and r-squared criteria in both training and testing performance. Particularly the Liu regression often achieved better relative performance (up to 7.60% to 46.08% for Body Fat data set and up to 1.55% to 21.53% for Cancer data set on training performance and up to 1.56% to 38.08% for Body Fat data set and up to 3.50% to 23.29% for Cancer data set on testing performance) among regression methods as well as compared to machine algorithms. Conclusions: Liu regression is mostly disregarded in the machine learning literature, but since it outperforms the most powerful and widely used machine learning algorithms, it appears to be a promising tool in almost all fields, especially for regression-based studies including data with multicollinearity problem.

Patent Keyword Analysis Using Bayesian Zero-Inflated Model and Text Mining

Patent keyword analysis is used to analyze the technology keywords extracted from collected patent documents for specific technological fields. Thus, various methods related to this type of analysis have been researched in the industrial engineering fields, such as technology management and new product development. To analyze the patent document data, we have to search for patents related to the target technology and preprocess them to construct the patent–keyword matrix for statistical and machine learning algorithms. In general, a patent–keyword matrix has an extreme zero-inflated problem. This is because each keyword occupies one column even if it is included in only one document among all patent documents. General zero-inflated models have a limit at which the performance of the model deteriorates when the proportion of zeros becomes extremely large. To solve this problem, we applied a Bayesian inference to a general zero-inflated model. In this paper, we propose a patent keyword analysis using a Bayesian zero-inflated model to overcome the extreme zero-inflated problem in the patent–keyword matrix. In our experiments, we collected practical patents related to digital therapeutics technology and used the patent–keyword matrix preprocessed from them. We compared the performance of our proposed method with other comparative methods. Finally, we showed the validity and improved performance of our patent keyword analysis. We expect that our research can contribute to solving the extreme zero-inflated problem that occurs not only in patent keyword analysis, but also in various text big data analyses.

A Critical Review of Inductive Logic Programming Techniques for Explainable AI.

Despite recent advances in modern machine learning algorithms, the opaqueness of their underlying mechanisms continues to be an obstacle in adoption. To instill confidence and trust in artificial intelligence (AI) systems, explainable AI (XAI) has emerged as a response to improve modern machine learning algorithms' explainability. Inductive logic programming (ILP), a subfield of symbolic AI, plays a promising role in generating interpretable explanations because of its intuitive logic-driven framework. ILP effectively leverages abductive reasoning to generate explainable first-order clausal theories from examples and background knowledge. However, several challenges in developing methods inspired by ILP need to be addressed for their successful application in practice. For example, the existing ILP systems often have a vast solution space, and the induced solutions are very sensitive to noises and disturbances. This survey paper summarizes the recent advances in ILP and a discussion of statistical relational learning (SRL) and neural-symbolic algorithms, which offer synergistic views to ILP. Following a critical review of the recent advances, we delineate observed challenges and highlight potential avenues of further ILP-motivated research toward developing self-explanatory AI systems.

An Investigation in to Sports Analytics and Related Tools

This study explores the application of sports analytics and the utilization of various tools in the sports industry. Sports analytics, also known as sports data analytics, is the application of data analysis and statistical approaches to extract important insights. Additionally, it aids in making well-informed judgments and enhancing performance in the realm of sports. Sports data analytics include a diverse array of methods and approaches, such as statistical analysis, machine learning, data visualization, predictive modeling, and optimization algorithms. The objective of sports business analytics is to get practical insights from a dataset in order to enhance decision-making and foster innovation in the sports industry

Uncertainty quantification by direct propagation of shallow ensembles

Statistical learning algorithms provide a generally-applicable framework to sidestep time-consuming experiments, or accurate physics-based modeling, but they introduce a further source of error on top of the intrinsic limitations of the experimental or theoretical setup. Uncertainty estimation is essential to quantify this error, and to make application of data-centric approaches more trustworthy. To ensure that uncertainty quantification is used widely, one should aim for algorithms that are accurate, but also easy to implement and apply. In particular, including uncertainty quantification on top of an existing architecture should be straightforward, and add minimal computational overhead. Furthermore, it should be easy to manipulate or combine multiple machine-learning predictions, propagating uncertainty over further modeling steps. We compare several well-established uncertainty quantification frameworks against these requirements, and propose a practical approach, which we dub direct propagation of shallow ensembles, that provides a good compromise between ease of use and accuracy. We present benchmarks for generic datasets, and an in-depth study of applications to the field of atomistic machine learning for chemistry and materials. These examples underscore the importance of using a formulation that allows propagating errors without making strong assumptions on the correlations between different predictions of the model.

Deep Learning-Based Dynamic Risk Prediction of Venous Thromboembolism for Patients With Ovarian Cancer in Real-World Settings From Electronic Health Records.

Patients with epithelial ovarian cancer (EOC) have an elevated risk for venous thromboembolism (VTE). To assess the risk of VTE, models were developed by statistical or machine learning algorithms. However, few models have accommodated deep learning (DL) algorithms in realistic clinical settings. We aimed to develop a predictive DL model, exploiting rich information from electronic health records (EHRs), including dynamic clinical features and the presence of competing risks. We extracted EHRs of 1,268 patients diagnosed with EOC from January 2007 through December 2017 at the National Cancer Center, Korea. DL survival networks using fully connected layers, temporal attention, and recurrent neural networks were adopted and compared with multi-perceptron-based classification models. Prediction accuracy was independently validated in the data set of 423 patients newly diagnosed with EOC from January 2018 to December 2019. Personalized risk plots displaying the individual interval risk were developed. DL-based survival networks achieved a superior area under the receiver operating characteristic curve (AUROC) between 0.95 and 0.98 while the AUROC of classification models was between 0.85 and 0.90. As clinical information benefits the prediction accuracy, the proposed dynamic survival network outperformed other survival networks for the test and validation data set with the highest time-dependent concordance index (0.974, 0.975) and lowest Brier score (0.051, 0.049) at 6 months after a cancer diagnosis. Our visualization showed that the interval risk fluctuating along with the changes in longitudinal clinical features. Adaption of dynamic patient clinical features and accounting for competing risks from EHRs into the DL algorithms demonstrated VTE risk prediction with high accuracy. Our results show that this novel dynamic survival network can provide personalized risk prediction with the potential to assist risk-based clinical intervention to prevent VTE among patients with EOC.

A new hybrid machine learning model for predicting the renewal life of patents.

In almost every country, patents need to be renewed multiple times after they are granted. A patentee assesses the value of the patent and then pays a renewal fee to keep it active for another stipulated period. The factors that characterize the value of a patent is subjective. This paper aims to address the research gap of building an accurate model for predicting the renewal life (often considered as a substitute for the patent value) of Indian patents, and identification of significant factors that influence the renewal life. This study uses an extensive data set collected from the Indian Patent Office for all granted patents filed between 1995 and 2005. The popular statistical and machine learning algorithms do not result in accurate predictive models, because the patent renewal life distribution (at least for the Indian patents) shows unusual spikes at the two extreme values, which makes the modeling task more challenging. We propose a new two-stage hybrid model by combining an efficient multi-class classifier and a binomial regression model for predicting the complex renewal data distribution. We conducted a comparative analysis of the proposed model with several state-of-the-art machine learning and statistical models. The results show that the proposed hybrid model gives 90% accuracy as compared to the best competitor which gives only 40% accuracy.

Application of Statistical Learning Algorithms in Thermal Stress Assessment in Comparison with the Expert Judgment Inherent to the Universal Thermal Climate Index (UTCI)

Application of Statistical Learning Algorithms in Thermal Stress Assessment in Comparison with the Expert Judgment Inherent to the Universal Thermal Climate Index (UTCI)

Machine Learning-Enhanced Pairs Trading

Forecasting returns in financial markets is notoriously challenging due to the resemblance of price changes to white noise. In this paper, we propose novel methods to address this challenge. Employing high-frequency Brazilian stock market data at one-minute granularity over a full year, we apply various statistical and machine learning algorithms, including ARIMA, Bidirectional Long Short-Term Memory (BiLSTM) with attention, Transformers, N-BEATS, N-HiTS, Convolutional Neural Networks (CNNs), and Temporal Convolutional Networks (TCNs) to predict changes in the price ratio of closely related stock pairs. Our findings indicate that a combination of reversion and machine learning-based forecasting methods yields the highest profit-per-trade. Additionally, by allowing the model to abstain from trading when the predicted magnitude of change is small, profits per trade can be further increased. Our proposed forecasting approach, utilizing a blend of methods, demonstrates superior accuracy compared to individual methods for high-frequency data.

Rapid detection and quantification of melamine, urea, sucrose, water, and milk powder adulteration in pasteurized milk using Fourier transform infrared (FTIR) spectroscopy coupled with modern statistical machine learning algorithms

There is an evident requirement for a rapid, efficient, and simple method to screen the authenticity of milk products in the market. Fourier transform infrared (FTIR) spectroscopy stands out as a promising solution. This work employed FTIR spectroscopy and modern statistical machine learning algorithms for the identification and quantification of pasteurized milk adulteration. Comparative results demonstrate modern statistical machine learning algorithms will improve the ability of FTIR spectroscopy to predict milk adulteration compared to partial least square (PLS). To discern the types of substances utilized in milk adulteration, a top-performing multiclassification model was established using multi-layer perceptron (MLP) algorithm, delivering an impressive prediction accuracy of 97.4 %. For quantification purposes, bayesian regularized neural networks (BRNN) provided the best results for the determination of both melamine, urea and milk powder adulteration, while extreme gradient boosting (XGB) and projection pursuit regression (PPR) gave better results in predicting sucrose and water adulteration levels, respectively. The regression models provided suitable predictive accuracy with the ratio of performance to deviation (RPD) values higher than 3. The proposed methodology proved to be a cost-effective and fast tool for screening the authenticity of pasteurized milk in the market.

Rock Mineral Volume Inversion Using Statistical and Machine Learning Algorithms for Enhanced Geothermal Systems in Upper Rhine Graben, Eastern France

AbstractAccurately determining the mineralogical composition of rocks is essential for precise assessments of key petrophysical properties like effective porosity, water saturation, clay volume, and permeability. Mineral volume inversion is particularly critical in geological contexts characterized by heterogeneity, such as in the Upper Rhine Graben (URG), where both carbonate and siliciclastic formations are prevalent. The estimation of mineral volumes poses challenges that involve both linear and nonlinear relationships associated with geophysical data. To address this complexity, our methodology strategically integrates the robust insights from standard statistical approaches with three machine learning (ML) algorithms: multi‐layer perceptron, random forest regression, and gradient boosting regression. Furthermore, we propose a new hybrid ensemble model that incorporates a weighted average of multiple ML approaches to predict mineral composition within the Muschelkalk and Buntsandstein formations of the URG. ML techniques for mineral composition prediction in these formations exhibit robust predictive performance. The predicted mineral volumes align closely with quantitative estimates derived from X‐ray diffraction analysis. Additionally, they are in good qualitative agreement with mineral descriptions obtained from cores and cuttings of the Muschelkalk and Buntsandstein formations.

Simulation-Based Assessment of Classification Methods: Statistical Models vs. Machine Learning Algorithms

Simulation-Based Assessment of Classification Methods: Statistical Models vs. Machine Learning Algorithms

Heat shock protein 72 supports extracellular matrix production in metastatic mammary tumors.

This study identified tumorigenic processes most dependent on murine heat shock protein 72 (HSP72) in the mouse mammary tumor virus-PyMT mammary tumor model, which give rise to spontaneous mammary tumors that exhibit HSP72-dependent metastasis to the lung. RNA-seq expression profiling of Hspa1a/Hspa1b (Hsp72) WT and Hsp72-/- primary mammary tumors discovered significantly lower expression of genes encoding components of the extracellular matrix (ECM) in Hsp72 knockout mammary tumors compared to WT controls. In vitro studies found that genetic or chemical inhibition of HSP72 activity in cultured collagen-expressing human or murine cells also reduces mRNA and protein levels of COL1A1 and several other ECM-encoding genes. In search of a possible mechanistic basis for this relationship, we found HSP72 to support the activation of the tumor growth factor-β-suppressor of mothers against decapentaplegic-3 signaling pathway and evidence of suppressor of mothers against decapentaplegic-3 and HSP72 coprecipitation, suggesting potential complex formation. Human COL1A1 mRNA expression was found to have prognostic value for HER2+breast tumors over other breast cancer subtypes, suggesting a possible human disease context where targeting HSP72 may have a therapeutic rationale. Analysis of human HER2+breast tumor gene expression data using a gene set comprising ECM-related gene and protein folding-related geneas an input to the statistical learning algorithm, Galgo, found a subset of these genes that can collectively stratify patients by relapse-free survival, further suggesting a potential interplay between the ECM and protein-folding genes may contribute to tumor progression.

Plant Disease Detection

Abstract: This project addresses the critical challenge of plant disease detection in India, a nation where agriculture supports nearly 70% of the population. Traditional procedures for identifying plant diseases are both labor-intensive and time-consuming. To enhance efficiency and accuracy, this project introduces an innovative approach that integrates machine learning and image processing techniques. By analyzing images of leaves, the system identifies diseases based on key factors such as leaf color, damage extent, area, and texture parameters. Unlike conventional methods that depend on visual inspections or chemical processes, this automated system provides a cost-effective and efficient solution for monitoring extensive crop fields. By leveraging statistical machine learning and advanced image processing algorithms, the proposed solution ensures precise disease detection with minimized computational complexity and reduced prediction time

A strategy integrating GC–MS, UPLC profiling, and DNA metabarcoding for characterization and discrimination of the medicinal and culinary pieces from four Curcuma species

Curcumae pieces prepared in ready-to-use forms, consumed extensively for their significant medicinal and culinary benefits, present a challenge in discrimination due to their heterogeneous origins, analogous morphologies, and varied components. This study explored the metabolites of nine Curcumae pieces (NCPs) by Gas Chromatography-Mass Spectrometry (GC–MS) and Ultra-Performance Liquid Chromatography/Ultra High-Performance Liquid Chromatography–Quadrupole Time of Flight–Mass Spectrometry (UPLC/UHPLC–QTOF–MS) technologies, augmented by multivariate statistical analysis (Principal Component Analysis, PCA; Orthogonal Partial Least Squares Discriminant Analysis, OPLS–DA) and machine learning algorithms (Random Forest, RF). Twenty-six key differential markers in volatile and non-volatile metabolites each were found subsequently. Additionally, DNA metabarcoding was employed to develop specific Single Nucleotide Polymorphism (SNP) markers for the precise identification of Curcuma species in single and mixed samples, and for CRa even contained in Chinese patent prescriptions. The integration of these three methodologies facilitates a more reliable authentication of NCPs, enhancing the overall strategy for their global discrimination. This research contributes to the advancement of quality control and the development of functional foods or products derived from Curcumae pieces, tailored to meet specific health and dietary requirements.