Statistical Machine Learning Methods Research Articles

Learning to Learn in Interactive Constraint Acquisition

Constraint Programming (CP) has been successfully used to model and solve complex combinatorial problems. However, modeling is often not trivial and requires expertise, which is a bottleneck to wider adoption. In Constraint Acquisition (CA), the goal is to assist the user by automatically learning the model. In (inter)active CA, this is done by interactively posting queries to the user, e.g. does this partial solution satisfy your (unspecified) constraints or not. While interactive CA methods learn the constraints, the learning is related to symbolic concept learning, as the goal is to learn an exact representation. However, a large number of queries is required to learn the model, which is a major limitation. In this paper, we aim to alleviate this limitation by tightening the connection of CA and Machine Learning (ML), by, for the first time in interactive CA, exploiting statistical ML methods. We propose to use probabilistic classification models to guide interactive CA queries to the most promising parts. We discuss how to train classifiers to predict whether a candidate expression from the bias is a constraint of the problem or not, using both relation-based and scope-based features. We then show how the predictions can be used in all layers of interactive CA: the query generation, the scope finding, and the lowest-level constraint finding. We experimentally evaluate our proposed methods using different classifiers and show that our methods greatly outperform the state of the art, decreasing the number of queries needed to converge by up to 72%.

Age-related changes in human brain functional connectivity using graph theory and machine learning techniques in resting-state fMRI data.

Aging is the basis of neurodegeneration and dementia that affects each endemic in the body.Normal aging in the brain is associated with progressive slowdown and disruptions in various abilities such as motor ability, cognitive impairment, decreasing information processing speed, attention, and memory. With the aggravation of global aging, more research focuses on brain changes in the elderly adult. The graph theory, in combination with functional magnetic resonance imaging (fMRI), makes it possible to evaluate the brain network functional connectivity patterns in different conditions with brain modeling. We have evaluated the brain network communication model changes in three different age groups (including 8 to 15years, 25 to 35years, and 45 to 75years) in lifespan pilot data from the human connectome project (HCP). Initially, Pearson correlation-based connectivity networks were calculated and thresholded. Then, network characteristics were compared between the three age groups by calculating the global and local graph measures. In the resting state brain network, we observed decreasing global efficiency and increasing transitivity with age. Also, brain regions, including the amygdala, putamen, hippocampus, precuneus, inferior temporal gyrus, anterior cingulate gyrus, and middle temporal gyrus, were selected as the most affected brain areas with age through statistical tests and machine learning methods. Using feature selection methods, including Fisher score and Kruskal-Wallis, we were able to classify three age groups using SVM, KNN, and decision-tree classifier. The best classification accuracy is in the combination of Fisher score and decision tree classifier obtained, which was 82.2%. Thus, by examining the measures of functional connectivity using graph theory, we will be able to explore normal age-related changes in the human brain, which can be used as a tool to monitor health with age.

Effect of rock properties on wear and cutting performance of multi blade circular saw with iron based multi-layer diamond segments

This study is an attempt for comprehensive, combining experimental data with advanced analytical techniques and machine learning for a thorough understanding of the factors influencing the wear and cutting performance of multi-blade diamond disc cutters on granite blocks. A series of sawing experiments were performed to evaluate the wear and cutting performance of multi blade diamond disc cutters with varying diameters in the processing of large-sized granite blocks. The multi-layer diamond segments comprising the Iron (Fe) based metal matrix were brazed on the sawing blades. The segment’s wear was studied through micrographs and data obtained from the Field Emission Scanning Electron Microscopy (FESEM) and Energy Dispersive X-ray (EDS). Granite rock samples of nine varieties were tested in the laboratory to determine the quantitative rock parameters. The contribution of individual rock parameters and their combined effects on wear and cutting performance of multi blade saw were correlated using statistical machine learning methods. Moreover, predictive models were developed to estimate the wear and cutting rate based on the most significant rock properties. The point load strength index, uniaxial compressive strength, and deformability, Cerchar abrasivity index, and Cerchar hardness index were found to be the significant variables affecting the sawing performance.

Developing predictive precision medicine models by exploiting real-world data using machine learning methods

Computational Medicine encompasses the application of Statistical Machine Learning and Artificial Intelligence methods on several traditional medical approaches, including biochemical testing which is extremely valuable both for early disease prognosis and long-term individual monitoring, as it can provide important information about a person's health status. However, using Statistical Machine Learning and Artificial Intelligence algorithms to analyze biochemical test data from Electronic Health Records requires several preparatory steps, such as data manipulation and standardization. This study presents a novel approach for utilizing Electronic Health Records from large, real-world databases to develop predictive precision medicine models by exploiting Artificial Intelligence. Furthermore, to demonstrate the effectiveness of this approach, we compare the performance of various traditional Statistical Machine Learning and Deep Learning algorithms in predicting individuals' future biochemical test outcomes. Specifically, using data from a large real-world database, we exploit a longitudinal format of the data in order to predict the future values of 15 biochemical tests and identify individuals at high risk. The proposed approach and the extensive model comparison contribute to the personalized approach that modern medicine aims to achieve.

Alzheimer's disease detection using data fusion with a deep supervised encoder.

Alzheimer's disease (AD) is affecting a growing number of individuals. As a result, there is a pressing need for accurate and early diagnosis methods. This study aims to achieve this goal by developing an optimal data analysis strategy to enhance computational diagnosis. Although various modalities of AD diagnostic data are collected, past research on computational methods of AD diagnosis has mainly focused on using single-modal inputs. We hypothesize that integrating, or "fusing," various data modalities as inputs to prediction models could enhance diagnostic accuracy by offering a more comprehensive view of an individual's health profile. However, a potential challenge arises as this fusion of multiple modalities may result in significantly higher dimensional data. We hypothesize that employing suitable dimensionality reduction methods across heterogeneous modalities would not only help diagnosis models extract latent information but also enhance accuracy. Therefore, it is imperative to identify optimal strategies for both data fusion and dimensionality reduction. In this paper, we have conducted a comprehensive comparison of over 80 statistical machine learning methods, considering various classifiers, dimensionality reduction techniques, and data fusion strategies to assess our hypotheses. Specifically, we have explored three primary strategies: (1) Simple data fusion, which involves straightforward concatenation (fusion) of datasets before inputting them into a classifier; (2) Early data fusion, in which datasets are concatenated first, and then a dimensionality reduction technique is applied before feeding the resulting data into a classifier; and (3) Intermediate data fusion, in which dimensionality reduction methods are applied individually to each dataset before concatenating them to construct a classifier. For dimensionality reduction, we have explored several commonly-used techniques such as principal component analysis (PCA), autoencoder (AE), and LASSO. Additionally, we have implemented a new dimensionality-reduction method called the supervised encoder (SE), which involves slight modifications to standard deep neural networks. Our results show that SE substantially improves prediction accuracy compared to PCA, AE, and LASSO, especially in combination with intermediate fusion for multiclass diagnosis prediction.

Interpretable machine learning for in-hospital mortality risk prediction in patients with ST-elevation myocardial infarction after percutaneous coronary interventions

Background and objectiveDespite the constant improvement of coronary heart disease (CHD) diagnostics and treatment methods it remains one of the main causes of death in most countries around the world. And myocardial infarction with ST segment elevation on the electrocardiogram (STEMI) still is one of the most dangerous clinical variants of CHD. This study aims to develop an explainable machine learning model for in-hospital mortality (IHM) risk prediction in STEMI patients after myocardial revascularization by percutaneous coronary intervention (PCI). MethodsA single-center observational retrospective study was conducted, enrolling 4677 electronic medical records of patients with STEMI after PCI, which were analyzed using statistical analysis and machine learning methods. A pool of potential IHM predictors was identified, and prognostic models were developed and validated based on multivariate logistic regression, random forest, and stochastic gradient boosting methods at two stages of hospital treatment: during the initial physicians examination in the emergency department and immediately after PCI surgery. To explain the IHM prognosis, threshold values of IHM risk factors were determined using 3 grid search methods for optimal cut-off points, calculating centroids and SHapley Additive exPlanations (SHAP). ResultsIHM prognostic models were developed using clinical and functional status data of STEMI patients during two stages of hospital treatment. The IHM prediction accuracy according to the first scenario was AUC = 0.85, and according to the second - AUC = 0.9. Predictors identified and validated in the models were converted into risk factors. Models whose parameters were risk factors demonstrated high forecast accuracy (AUC = 0.87), with the best model formed using the SHAP method. ConclusionsFor the forecast result interpretation risk factors obtained by categorizing continuous variables can be used by assessing the impact of the latter on the end point using the SHAP method.

ПРИМЕНЕНИЕ ИСКУССТВЕННОГО ИНТЕЛЛЕКТА ДЛЯ АНАЛИЗА РИСКОВ НА РЫНКЕ ЦЕННЫХ БУМАГ

This scientific article is devoted to the actual topic of application of artificial intelligence for risk analysis in the securities market. The article considers the role of artificial intelligence in analyzing data, identifying potential risks and making informed investment decisions. The research methodology is based on theoretical and empirical methods, as well as the results of domestic scientific research. The results of the study emphasize the importance of understanding and managing risk in financial markets, including the concept of portfolio and systemic risk. The author analyzes various methods of risk analysis, including statistical models, fundamental analysis, technical analysis, scenario analysis, and machine learning methods. Special attention is given to the application of neural networks, genetic algorithms, and other artificial intelligence techniques to improve price forecasting, optimal portfolio determination, and risk management. The conclusion emphasizes the promising direction of using artificial intelligence in financial analytics, but notes the need to consider potential challenges and limitations, such as market volatility, data availability and quality, interpretability of models, and ethical considerations.

An intercomparison of weather normalization of PM2.5 concentration using traditional statistical methods, machine learning, and chemistry transport models

Traditional statistical methods (TSM) and machine learning (ML) methods have been widely used to separate the effects of emissions and meteorology on air pollutant concentrations, while their performance compared to the chemistry transport model has been less fully investigated. Using the Community Multiscale Air Quality Model (CMAQ) as a reference, a series of experiments was conducted to comprehensively investigate the performance of TSM (e.g., multiple linear regression and Kolmogorov–Zurbenko filter) and ML (e.g., random forest and extreme gradient boosting) approaches in quantifying the effects of emissions and meteorology on the trends of fine particulate matter (PM2.5) during 2013−2017. Model performance evaluation metrics suggested that the TSM and ML methods can explain the variations of PM2.5 with the highest performance from ML. The trends of PM2.5 showed insignificant differences (p > 0.05) for both the emission-related (PM2.5EMI\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{\\rm{PM}}}_{2.5}^{{\\rm{EMI}}}$$\\end{document}) and meteorology-related components between TSM, ML, and CMAQ modeling results. PM2.5EMI\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{\\rm{PM}}}_{2.5}^{{\\rm{EMI}}}$$\\end{document} estimated from ML showed the least difference to that from CMAQ. Considering the medium computing resources and low model biases, the ML method is recommended for weather normalization of PM2.5. Sensitivity analysis further suggested that the ML model with optimized hyperparameters and the exclusion of temporal variables in weather normalization can further produce reasonable results in emission-related trends of PM2.5.

Comparative analysis of data classification methods for prediction of trade-in auto prices

In the presented article, machine learning algorithms are used to predict the price of cars. Forecasting the price of cars is one of the most important issues in modern times, because the number of car users is increasing year by year worldwide. Therefore, it may be interesting for many car owners to know the approximate price of cars in advance. The vehicle’s build date, mileage, size and other parameters are essential data for the machine learning process. Based on these parameters, the data needed to predict the price of any car was classified and a training dataset was created. Based on this data set, some interesting predictions were made. The purpose of the article is to consider the pre-processing of data, to determine and analyze what achievements the direction of artificial intelligence has achieved on the basis of predicting the price of cars. Hybrid forecasting methods using statistical analysis and machine learning methods were used in the research.

Robustness Analysis on Stock Market Prediction Method

Stock prediction is a vital approach used to forecast stock’s future prices by analyzing historical data. This technique plays a crucial role in assisting investors in making informed decisions in the stock market. There are various methods employed for stock prediction, with the most common being traditional statistical methods and machine learning methods. Traditional statistical methods involve models such as ARIMA and GARCH, while machine learning methods involve LSTM and GRU models.This research paper aims to compare the performance of SARIMA , LSTM and GRU models in stock prediction. To enhance their performance, the models will be optimized by incorporating DWT-ARIMA-XGBoost[1], LSTM-XGBoost[2] and GRU-XGBoost techniques. These methods will then be deployed to predict stock prices, with a subsequent comparison of the results obtained from the three models. After comparing those models, this paper selects four models that have high level of prediction to study their generalization. This analysis will provide valuable insights into the effectiveness and suitability of these models for stock prediction tasks. The paper evaluates these models based on their prediction accuracy, generalization ability.

Stock Price Forecasting: Traditional Statistical Methods and Deep Learning Methods

Statistical method and machine learning methods can be applied to stock forecasting to assist users in making decisions in a variety of practical applications. This paper uses 50 past stock prices to predict future stock prices in this experiment. Both ARIMA and LSTM models are used in this study to predict the price. The close price on transaction days makes up the dataset in this study. The performance of two models is evaluated by MAE, MSE, and RMSE after running them in this research. And the result shows that both the LSTM model and ARIMA model predict the stock price well. To be specific, MSE values of ARIMA model are 1.17151, 1.55678, 6.38663 and 159.58281. MSE values of LSTM model are 1.18464, 1.07799, 4.87162 and 97.59937. In these two kinds of methods, LSTM model has a better performance than ARIMA model.

Based on machine learning model for prediction of CO2 adsorption of synthetic zeolite in two-step solid waste treatment

The rising environmental issues caused by carbon dioxide emissions and accumulation of industrial solid waste accelerate the development of carbon capture utilization and storage (CCUS), especially the technology using industrial solid waste as a raw material to prepare environmentally friendly and sustainable porous materials to capture CO2. This study developed four models including support vector regression(SVR), multivariate adaptive regression spline(Mars), random forest(RF), and gradient boosting machine(GBM) based on 762 CO2 adsorption datasets of zeolites synthesized from five different industrial solid waste materials to predict the CO2 adsorption capacity and analyze impact of various factors on CO2 adsorption performance during synthesis and adsorption processes. The results suggested that gradient boosting machine(GBM) and the support vector regression(SVR) have good accuracy and generalization performance. The R2 of the model reached 0.99 and 0.96 respectively, which is in good agreement with the laboratory data. In general, the specific surface area(S) and adsorption pressure(P) during the adsorption process of zeolite have a great influence on the final adsorption performance. The correlation between the specific surface area(S) and the hydrothermal reaction temperature(T2) is the largest, and its Pearson Correlation Coefficient is 0.61. This study paved a new approach for the accumulation treatment of industrial solid waste and low-carbon industry via statistical analysis and machine learning method, which is beneficial to environmental protection and sustainable development.

Evaluation of Statistical Machine Learning Methods and Service Design Techniques Applications in Hotel Industry

Evaluation of Statistical Machine Learning Methods and Service Design Techniques Applications in Hotel Industry

Forecasting performance of machine learning, time series, and hybrid methods for low‐ and high‐frequency time series

One of the main objectives of the time series analysis is forecasting, so both Machine Learning methods and statistical methods have been proposed in the literature. In this study, we compare the forecasting performance of some of these approaches. In addition to traditional forecasting methods, which are the Naive and Seasonal Naive Methods, S/ARIMA, Exponential Smoothing, TBATS, Bayesian Exponential Smoothing Models with Trend Modifications and STL Decomposition, the forecasts are also obtained using seven different machine learning methods, which are Random Forest, Support Vector Regression, XGBoosting, BNN, RNN, LSTM, and FFNN, and the hybridization of both statistical time series and machine learning methods. The data set is selected proportionally from various time domains in M4 Competition data set. Thereby, we aim to create a forecasting guide by considering different preprocessing approaches, methods, and data sets having various time domains. After the experiment, the performance and impact of all methods are discussed. Therefore, most of the best models are mainly selected from machine learning methods for forecasting. Moreover, the forecasting performance of the model is affected by both the time frequency and forecast horizon. Lastly, the study suggests that the hybrid approach is not always the best model for forecasting. Hence, this study provides guidelines to understand which method will perform better at different time series frequencies.

Possible Alternatives: Identifying and Quantifying Adulteration in Buffalo, Goat, and Camel Milk Using Mid-Infrared Spectroscopy Combined with Modern Statistical Machine Learning Methods.

Adulteration of higher priced milks with cheaper ones to obtain extra profit can adversely affect consumer health and the market. In this study, pure buffalo milk (BM), goat milk (GM), camel milk (CM), and their mixtures with 5-50% (vol/vol) cow milk or water were used. Mid-infrared spectroscopy (MIRS) combined with modern statistical machine learning was used for the discrimination and quantification of cow milk or water adulteration in BM, GM, and CM. Compared to partial least squares (PLS), modern statistical machine learning-especially support vector machines (SVM), projection pursuit regression (PPR), and Bayesian regularized neural networks (BRNN)-exhibited superior performance for the detection of adulteration. The best prediction models for the different predictive traits are as follows: The binary classification models developed by SVM resulted in differentiation of CM-cow milk, and GM/CM-water mixtures. PLS resulted in differentiation of BM/GM-cow milk and BM-water mixtures. All of the above models have 100% classification accuracy. SVM was used to develop multi-classification models for identifying the high and low proportions of cow milk in BM, GM, and CM, as well as the high and low proportions of water adulteration in BM and GM, with correct classification rates of 94%, 100%, 100%, 99%, and 100%, respectively. In addition, a PLS-based model was developed for identifying the high and low proportions of water adulteration in CM, with correct classification rates of 100%. A regression model for quantifying cow milk in BM was developed using PCA + BRNN, with RMSEV = 5.42%, and RV2 = 0.88. A regression model for quantifying water adulteration in BM was developed using PCA + PPR, with RMSEV = 1.70%, and RV2 = 0.99. Modern statistical machine learning improved the accuracy of MIRS in predicting BM, GM, and CM adulteration more effectively than PLS.

Investigation of anomalies in federal budget revenues

Studies of abnormal values play an important role in understanding the generating processes and are one of the key ones in financial analysis and forecasting. This article is devoted to the study of anomalies in federal budget revenues. Based on statistical data for the period from 2008 to 2022, abnormal values of budget revenues in different time periods were revealed using statistical models and machine learning methods, their advantages and disadvantages are shown. Practical recommendations on the use of machine learning methods to identify anomalies in the field of public finance are given.

On‐Chip Near‐Infrared Spectral Sensing with Minimal Plasmon‐Modulated Channels

AbstractSpectroscopy is widely used in biomedicine, agriculture, remote sensing, industrial process monitoring, etc. The emerging on‐chip integration techniques further extend these applications to on‐site and portable operating modes. However, realizing high‐applicability and low‐cost spectroscopy chips is still a major challenge particularly in infrared due to the requirements on a large number of spectral channels. Here, a compact and low‐cost near‐infrared spectral sensing platform is demonstrated, releasing the burden of scaling up the spectral channel and simultaneously keeping a high accuracy in chemical classification. The monolithically integrated spectral sensor consists of a small amount of plasmon‐modulated narrowband photodetection units and operates with on‐chip photoelectric responses via a statistical machine learning method. Both plasmonic resonances with low spectral correlation and advanced regression algorithms contribute to high sensing accuracy in the case of limited hardware resources. Chemical concentration quantification, plastic sorting, and scanning spectral imaging are realized using such a sensor with up to twelve spectral channels. This concept avoids the uncertainty of various spectral reconstruction algorithms, high cost, and processing complexity and therefore provides promising strategies for miniaturized spectral sensing platforms.

Qualifying data on railroad track vibrations: a hybrid data preprocessing flow of statistical and machine learning approaches

ABSTRACT With the growing trend for increased train speed, steel rails may suffer from quality problems due to both overloading and/or the high speed of moving trains. However, before any further analysis can be performed to gain in-depth knowledge, the relevant vibration data sets must be curated, cleansed, preprocessed, and filtered very carefully after they are recorded and collected by the installed sensor equipment. This study proposes a systematic methodological flow to obtain data sets ready for subsequent analysis from messy source data. It hybridized several statistical and unsupervised machine learning methods, with the final aim to establish meaningful rules to determine suitable data sets by referring to domain knowledge. This flow was verified using a relatively large database of records of physical vibrations measured in 2019 at specific locations along a curve of an actual railroad track. As the flow can be used to qualify empirical data sets required in practice, further analysis is provided for the effectiveness of each rule, differences in determination between the rules, and the effects of combining more than one rule.

Integrative metabolome and lipidome analyses of plasma in neovascular macular degeneration

Age-related macular degeneration (AMD) causes irreversible vision-loss among the elderly in industrial countries. Neovascular AMD (nAMD), which refers to late-stage AMD, is characterized by severe vision-threatening choroidal neovascularization (CNV). Herein, we constructed a global metabolic network of nAMD, based on untargeted metabolomic and lipidomic analysis of plasma samples collected from sixty subjects (30 nAMD patients and 30 age-matched controls). Among the nAMD and control groups, 62 and 44 significantly different metabolites were detected in the positive and negative ion modes, respectively. Grouping analysis further showed that lipid and lipid-like molecule-based superclasses contained the highest number of significantly different metabolites. Lipidomic analysis revealed that 53 lipids among the nAMD and control groups differed significantly; these belonged to four major lipid categories (glycerophospholipids, sphingolipids, glycerolipids, and fatty acids). A discriminative biomarker panel comprising 16 metabolites and lipids, which was constructed using multivariate statistical machine learning methods, could effectively identify nAMD cases. Among these 16 compounds, eight were lipids that belonged to three lipid categories (glycerophospholipids, sphingolipids, and prenol lipids). The top three biomarkers with the highest importance scores were all lipids (a glycerophospholipid and two sphingolipids), highlighting the crucial role played by glycerophospholipid and sphingolipid pathways in nAMD. These differences between the metabolic and lipid profiles of nAMD patients and elderly individuals without AMD provide a readout of the overall metabolic status of nAMD. Further insights into the identified discriminative biomarkers may pave the way for future diagnostic and therapeutic interventions for nAMD.

An optimized hybrid methodology for short‐term traffic forecasting in telecommunication networks

AbstractWith the rapid development of telecommunication networks, the predictability of network traffic is of significant interest in network analysis and optimization, bandwidth allocation, and load balancing adjustment. Consequently, in recent years, significant research attention has been paid to forecasting telecommunication network traffic. Telecommunication traffic forecasting problems can be considered a time‐series problem, wherein periodic historical data is fed as the input to a model. Time‐series forecasting approaches are broadly categorized as statistical machine learning (ML) methods and their combinations. Statistical approaches forecast linear characteristics of time‐series data, unable to capture nonlinear and complex patterns. ML‐based approaches can model nonlinear characteristics of data. In recent years, hybrid approaches combining statistical and ML‐based approaches have been widely used to model linear and nonlinear data characteristics. However, the performance of these approaches highly depends on feature selection techniques and hyper‐parameter tuning of ML methods. A novel hybrid method is proposed for short‐term traffic forecasting based on feature selection and hyperparameter optimization to address this problem. It combines statistical and ML methods to model linear and nonlinear components of data. First, a novel feature selection technique, modified mutual information based on a linear combination of targets, is proposed to find the candidate input variables. Next, a combination of vector auto regressive moving average (VARMA), long short‐term memory (LSTM), and multilayer perceptron (MLP), called VARMA‐LSTM‐MLP forecaster, is suggested to forecast short‐term traffic. A hybrid metaheuristic algorithm, composed of firefly and BAT, is employed to find the optimal set of hyper‐parameter values. The proposed method is assessed by a real‐world dataset containing Tehran city's daily telecommunication data in IRAN. The evaluation results demonstrate that the proposed method outperforms the existing methods in terms of mean squared error and mean absolute error.