Statistical Learning Techniques Research Articles

Improving Short-Term, Near-Surface Temperature Forecasts by Integrating Weather Pattern Information into Model Output Statistics

Abstract Dynamical numerical weather prediction has remarkably improved over the last decades. Yet postprocessing techniques are needed to calibrate forecasts which are based on statistical and machine learning techniques. With recent advances in the derivation of year-round, large-scale atmospheric circulations, or weather regimes, the question arises of whether this information can be valuable within forecast postprocessing methods. This paper investigates this by proposing a bias correction scheme to integrate the atmospheric circulation state derived from empirical orthogonal functions, referred to as weather patterns, for deterministic short-term, near-surface temperature forecasts based on least absolute shrinkage and selection operator (LASSO) regression. We propose a computational study which first evaluates different weather pattern definitions (spatial domain) to improve temperature forecasts in Europe. As a bias could be associated with the weather pattern at the model initialization time or at the realization time of the forecast, both variants are tested in this study. We show that forecasted weather patterns with the identical spatial domain as the forecast show best skill reaching mean-square-error skill improvements of up to 3% (day ahead) or 1% (week ahead). Only considering land surface improvements in Europe, improvements of 4%–6% for day-ahead forecasts and 1%–5% for week-ahead forecasts are observable. We believe that this study not only introduces a simple yet effective tool to reduce bias in temperature forecasts but also contributes to the active discussion of how valuable weather patterns are and how to use them within forecast calibration techniques.

Predictive modeling and optimization of Rockwell hardness of additively manufactured PEEK using RSM, ANFIS and RNN integrated with PSO

The purpose of this research is to investigate the correlation between statistical and machine learning techniques and additive manufacturing, with a specific focus on predicting the Rockwell hardness of FDM-printed polyether ether ketone (PEEK) components. These components have a significant impact on various industries, such as aerospace, biomedical, and automobile. The study analyzes the hardness by conducting experimental analysis of four process parameters, including infill density, layer height, printing speed, and infill pattern. The research utilizes Response Surface Methodology (RSM), Adaptive Neuro-Fuzzy Inference System (ANFIS), and Recurrent Neural Network (RNN) to accurately predict the Rockwell hardness of the printed parts, with an average deviation of less than 5% from the experimental value. The study also investigates how hardness varies with FDM process parameters using contour and surface plots. Furthermore, the study utilizes RNN integrated with the Particle Swarm Optimization (PSO) algorithm to optimize Rockwell hardness. This approach achieved a peak Rockwell hardness value of 66.89 RHN under conditions of 80% infill density, 0.1mm layer height, 25 mm sec−1 printing speed, and an octet infill pattern. Microstructural examinations and test results corroborate the findings derived from parametric analysis and optimization efforts.

Temporal Dynamics and Predictive Modelling of Streamflow and Water Quality Using Advanced Statistical and Ensemble Machine Learning Techniques

Temporal Dynamics and Predictive Modelling of Streamflow and Water Quality Using Advanced Statistical and Ensemble Machine Learning Techniques

Classification of Garlic (Allium sativum L.) Crops by Fertilizer Differences Using Ground-Based Hyperspectral Imaging System

In contemporary agriculture, enhancing the efficient production of crops and optimizing resource utilization have become paramount objectives. Garlic growth and quality are influenced by various factors, with fertilizers playing a pivotal role in shaping both aspects. This study aimed to develop classification models for distinguishing garlic fertilizer application differences by employing statistical and machine learning techniques, such as partial least squares (PLS), based on data acquired from a ground-based hyperspectral imaging system in the agricultural sector. The garlic variety chosen for this study was Hongsan, and the fertilizer application plots were segmented into three distinct sections. Data were acquired within the VIS/NIR wavelength range using hyperspectral imaging. Following data acquisition, the standard normal variate (SNV) pre-processing technique was applied to enhance the dataset. To identify the optimal wavelengths, various techniques such as sequential forward selection (SFS), successive projections algorithm (SPA), variable importance in projection (VIP), and interval partial least squares (iPLS) were employed, resulting in the selection of 12 optimal wavelengths. For the fertilizer application difference model, six integrated vegetation indices were chosen for comparison with existing growth indicators. Using the same methodology, the model construction showed accuracies of 90.7% for PLS. Thus, the proposed model suggests that efficient regulation of garlic fertilizer application can be achieved by utilizing statistical and machine learning techniques.

Predicting Lead Changes in Tennis Matches based on Machine Learning Models

Based on men's singles data from the 2023 Wimbledon Tennis Championships, this study applies advanced statistical and machine learning techniques (including Principal Component Analysis PCA, Uniform Streaming Approximation and Projection UMAP, Random Forests, and Extreme Random Trees Extra-Trees) to explore the dynamics of lead changes during matches. By constructing mathematical models, the study predicts lead changes at key moments of the race that have a decisive impact on the race strategy and outcome. The experimental results show that the Extreme Random Trees model performs well on all evaluation metrics, especially in terms of accuracy, recall and F1 score, demonstrating its ability to handle complex interactions and variations in the data. In addition, the CatBoost model performed well in identifying actual leading changes, but with lower accuracy, possibly due to overfitting certain data patterns. The hybrid model, on the other hand, demonstrated a balanced performance, although it failed to outperform the extreme random tree model in terms of overall effectiveness.

Predictive mapping of surface roughness in turning of hardened AISI 4340 using carbide tools

This study presents a novel approach to predict surface roughness in the hard turning of AISI 4340 steel using carbide tools, aimed to develop a comprehensive predictive map. The hypothesis that surface roughness can be accurately predicted using a linear regression model was tested and confirmed. Experimental results showed surface roughness in the range of 1.946 to 5.636 microns. Statistical analysis revealed a normal distribution of surface roughness data with linear regression as the best-fit model, significantly determined by feed rate and explaining 98.41% of the variance. Machine learning validated this model, achieving high prediction accuracy (R² = 96.91%, MSE = 0.058, RMSE = 0.242). The innovative predictive map, created using a full factorial design, demonstrated a strong agreement between predicted and validated values. This work highlights the potential of integrating statistical and machine learning techniques for precise surface roughness prediction, recommending industrial validation to enhance machining productivity.

Predicting Recurrent Deficiency and Suboptimal Monitoring of Thiamin Deficiency in Patients with Metabolic and Bariatric Surgery.

Vitamin B1 (thiamine) deficiency (TD) after metabolic and bariatric surgery (MBS) is often insidious and, if unrecognized, can lead to irreversible damage or death. As TD symptoms are vague and overlap with other disorders, we aim to identify predictors of recurrent TD and failure to collect B1 labs. We analyzed a large sample of data from patients with MBS (n = 878) to identify potential predictors of TD risk. We modeled recurrent TD and failure to collect B1 labs using classical statistical and machine learning (ML) techniques. We identified clusters of labs associated with increased risk of recurrent TD: micronutrient deficiencies, abnormal blood indices, malnutrition, and fluctuating electrolyte levels (aIRR range: 1.62-4.68). Additionally, demographic variables associated with lower socioeconomic status were predictive of recurrent TD. ML models predicting characteristics associated with failure to collect B1 labs achieved 75-81% accuracy, indicating that clinicians may fail to match symptoms with the underlying condition. Our analysis suggests that both clinical and social factors can increase the risk of life-threatening TD episodes in some MBS patients. Identifying these indicators can help with diagnosis and treatment.

CUSTOMER SEGMENTATION USING RFM ANALYSIS

This research presents a comprehensive approach to customer segmentation using Recency, Frequency, and Monetary (RFM) analysis, combining statistical insights, data visualization, and machine learning techniques. The study utilizes a real-world dataset obtained from a retail environment, aiming to categorize customers based on their recent purchasing behavior, visit frequency, and monetary contributions to the store. The code begins with data preparation and exploration, ensuring data integrity by addressing issues such as negative quantities and missing customer identifiers. Following this, the Recency, Frequency, and Monetary metrics are computed, providing a holistic view of customer engagement and spending patterns. Visualizations, including violin plots, histograms, and box plots, are employed to intuitively convey the distribution of these metrics. The research then delves into the quantile-based segmentation of customers, allowing for a more granular classification. Quantiles are calculated to divide customers into four segments for each RFM metric. The resulting quantile labels are applied to the dataset, enabling the creation of a compound RFM quantile that combines recency, frequency, and monetary information. This combined quantile facilitates the definition of distinct customer segments. To further enhance the interpretability of customer segments, the study introduces a set of rules for labeling customers based on their RFM quantiles. These rules yield segments such as "Best Customer," "Loyal Customer," "Big Spender," "Dead Beats," and "Lost Customer." The resulting customer segmentation is presented visually through histograms and a pie chart, providing a clear and concise representation of the distribution of customers across different segments. Moreover, the research integrates machine learning models, including XGBClassifier, and CatBoostClassifier, to explore the potential of automating the segmentation process and predicting customer segments based on historical data. However, the machine learning aspect is introduced with commented-out sections, leaving room for further exploration and experimentation. In conclusion, this research contributes a comprehensive and detailed code implementation for RFM-based customer segmentation. The integration of visualization techniques aids in the interpretation of customer behavior, while the inclusion of machine learning models opens avenues for predictive analytics in customer segmentation. The presented approach provides valuable insights for businesses seeking to tailor marketing and customer relationship strategies based on individualized customer segments. KEYWORDS— Customer Segmentation, Frequency, Monetary Value, Recency, RFM Analysis.

Utilizing plant-based biogenic volatile organic compounds (bVOCs) to detect aflatoxin in peanut plants, pods, and kernels

Mycotoxins and particularly aflatoxin are a concern for peanut growers, processors, and consumers. Aflatoxins are responsible for approximately 25 % of liver cancer cases worldwide, while also costing millions of dollars in lost revenue on an annual basis. Current detection techniques based on high pressure liquid chromatography (HPLC) are time, labor and cost intensive. Peanut product can be held multiple days before testing is complete, costing processors additional revenue loss. Over the past few years, The Georgia Tech Research Institute (GTRI) has been investigating the use of plant-based biogenic volatile organic compounds (bVOCs) for monitoring the status of peanut plants. Initially, GTRI utilized these bVOCs to monitor for heat/drought stress in peanut plants. They quickly saw very promising indications of the ability of bVOCs to monitor other parameters in these plants. More recently, the team has started to investigate the of use of bVOCs to monitor aflatoxin development in plants, pods, and kernels pre- and post-harvest. A field trial for detection of aflatoxin using bVOCs was conducted in August–September of 2020 where three test groups were prepared: plants treated with Aspergillus fungus; plants treated with Afla-Guard (biocontrol agent); plants not treated – acting as a control group. Plant-based bVOCs were collected from the plants before treatment, and once a week post treatment using Stir Bar Sorptive Extraction (SBSE) devices or Twisters®. Each Twister® was then analyzed via gas chromatography–mass spectrometry (GC/MS). Pods from tested plants were harvested and sent to GTRI where bVOCs were collected and analyzed using GC/MS. Several statistical analysis and machine learning techniques were applied to all the collected GC/MS data. It was found using only bVOCs, that Random Forest classification performed well for the analysis of the pod and kernel samples with an F1 score of 0.80. On the other hand, Linear Discriminate Analysis (LDA) was only able to correctly classify 50 % of plant-based samples solely on bVOCs alone, which may be due to training models developed using original labels assuming no cross contamination at the field level. These results indicate the potential for bVOC screening for aflatoxin as an important way to lower impacts to growers, shellers, and consumers.

Functional clustering of NPLs recovery curves

The recovery performance of a portfolio of Non-Performing Loans can be measured in terms of recovery rate and liquidation time jointly through a “recovery curve” representative of recovery rates over time. When portfolio heterogeneity is very high, it is informative to estimate more than just one curve by dividing the portfolio into several homogeneous subsets, i.e. clusters, and calculating a recovery curve for each of them. The aim of this work is to estimate the optimal portfolio partition and the smoothed recovery curves of each cluster by means of non-parametric statistical learning techniques.

A comprehensive investigation of morphological features responsible for cerebral aneurysm rupture using machine learning

Cerebral aneurysms are a silent yet prevalent condition that affects a significant global population. Their development can be attributed to various factors, presentations, and treatment approaches. The importance of selecting the appropriate treatment becomes evident upon diagnosis, as the severity of the disease guides the course of action. Cerebral aneurysms are particularly vulnerable in the circle of Willis and pose a significant concern due to the potential for rupture, which can lead to irreversible consequences, including fatality. The primary objective of this study is to predict the rupture status of cerebral aneurysms. To achieve this, we leverage a comprehensive dataset that incorporates clinical and morphological data extracted from 3D real geometries of previous patients. The aim of this research is to provide valuable insights that can help make informed decisions during the treatment process and potentially save the lives of future patients. Diagnosing and predicting aneurysm rupture based solely on brain scans is a significant challenge with limited reliability, even for experienced physicians. However, by employing statistical methods and machine learning techniques, we can assist physicians in making more confident predictions regarding rupture likelihood and selecting appropriate treatment strategies. To achieve this, we used 5 classification machine learning algorithms and trained them on a substantial database comprising 708 cerebral aneurysms. The dataset comprised 3 clinical features and 35 morphological parameters, including 8 novel morphological features introduced for the first time in this study. Our models demonstrated exceptional performance in predicting cerebral aneurysm rupture, with accuracy ranging from 0.76 to 0.82 and precision score from 0.79 to 0.83 for the test dataset. As the data are sensitive and the condition is critical, recall is prioritized as the more crucial parameter over accuracy and precision, and our models achieved outstanding recall score ranging from 0.85 to 0.92. Overall, the best model was Support Vector Machin with an accuracy and precision of 0.82, recall of 0.92 for the testing dataset and the area under curve of 0.84. The ellipticity index, size ratio, and shape irregularity are pivotal features in predicting aneurysm rupture, respectively, contributing significantly to our understanding of this complex condition. Among the multitude of parameters under investigation, these are particularly important. In this study, the ideal roundness parameter was introduced as a novel consideration and ranked fifth among all 38 parameters. Neck circumference and outlet numbers from the new parameters were also deemed significant contributors.

Evaluation of Total Risk-Weighted Assets in Islamic Banking through Fintech Innovations

The assessment of total risk-weighted assets (LTRWAs) in the banking sector is of the utmost importance. It serves as a critical component for regulatory compliance, risk management, and capital adequacy. By accurately assessing LTRWAs, banks can effectively meet regulatory requirements, efficiently allocate capital resources, and proactively manage risks. Moreover, the accurate assessment of LTRWAs supports performance evaluation and fosters investor confidence in the financial stability of banks. This study presents statistical analyses and machine learning methods to identify factors influencing LTRWAs. Data from Bahrain, Jordan, Qatar, the United Arab Emirates, and Yemen, spanning from 2010 to 2021, was utilized. Various statistical tests and models, including ordinary least squares, fixed effect, random effect, correlation, variance inflation factor, tolerance tests, and fintech models, were conducted. The results indicated significant impacts of the unemployment rate, inflation rate, natural logarithm of the loan-to-asset ratio, and natural logarithm of total assets on LTRWAs in regression models. The dataset was divided into a training group (90% of the data) and a testing group (10% of the data) to evaluate the predictive capabilities of various fintech models, including an adaptive network-based fuzzy inference system (ANFIS), a hybrid neural fuzzy inference system (HyFIS), a fuzzy system with the heuristic gradient descent (FS.HGD), and fuzzy inference rules with the descent method (FIR.DM) models. The selection of the optimal model is contingent upon assessing its performance according to specific error criteria. The HyFIS model outperformed others with lower errors in predicting LTRWAs. Independent t-tests confirmed statistically significant differences between original and predicted LTRWA for all models, with HyFIS showing closer predictions. This study provides valuable insights into LTRWA prediction using advanced statistical and machine learning techniques, based on a dataset from multiple countries and years.

Assessment of groundwater quality using water quality index, multivariate statistical analysis and machine learning techniques in the vicinity of an open dumping yard

Assessment of groundwater quality using water quality index, multivariate statistical analysis and machine learning techniques in the vicinity of an open dumping yard

Analysis and Prediction of Stock Price using HMM and Facebook’s Prophet Computational Models

In recent times, the utilization of Statistical and Machine Learning techniques has gained prominence in the realm of financial data analysis. These methods are applied to various types of financial data, encompassing textual information, numerical data, and graphical representations. This study aims to compare the performance of two prominent forecasting methods, Hidden Markov Models and Facebook’s Prophet in the context of stock price prediction. Assessing the predictive accuracy, interpretability, and adaptability of both approaches through empirical experiments and case studies sheds light on their respective advantages and limitations. These experiments demonstrate that the predicted stock prices are in closer proximity to the actual price when compared to using a single data source. Furthermore, the achieved MAPE are 0.01, 0.025 and respectively, outperforming conventional methodologies. Our validation of effectiveness extends to real-world datasets encompassing the NIFTY50 Index. These findings offer valuable insights for researchers and practitioners seeking effective strategies for stock price prediction.

A Review Study on Outbreak Prediction of Covid19 By using Machine Learning

In December 2019, Wuhan City, China, discovered a new infectious disease, COVID-19. Over 70 million people have been infected and one million people have died as a result of COVID-19. Defeating such a deadly, infectious disease requires accurate models that predict COVID-19 outbreaks. Using prediction models, governments can plan budgets and facilities for fighting diseases, and take control measures to make better decisions and take control measures. For example, they can determine how many medicines and medical equipment to manufacture or import, as well as how many medical personnel are needed to fight the disease. The COVID-19 outbreak has subsequently been predicted in several countries and continents using regression and classification models. A recent study that incorporated statistical and machine learning techniques was reviewed to predict COVID-19 outbreaks in the future. Ground truth datasets are used, their characteristics are investigated, models are developed, predictor variables are identified, statistical and machine learning methods are applied, performance metrics are calculated, and finally comparisons are made. By applying machine learning methods, the survey results indicate that we can make predictions about whether a patient will become infected with COVID-19, how outbreak trends will develop, and which age groups will be affected the most

밀집 샘플링 기법을 이용한 네트워크 트래픽 예측 성능 향상

If the future can be predicted from network traffic data, which is a time series, it can achieve effects such as efficient resource allocation, prevention of malicious attacks, and energy saving. Many models based on statistical and deep learning techniques have been proposed, and most of these studies have focused on improving model structures and learning algorithms. Another approach to improving the prediction performance of the model is to obtain a good-quality data. With the aim of obtaining a good-quality data, this paper applies a dense sampling technique that augments time series data to the application of network traffic prediction and analyzes the performance improvement. As a dataset, UNSW-NB15, which is widely used for network traffic analysis, is used. Performance is analyzed using RMSE, MAE, and MAPE. To increase the objectivity of performance measurement, experiment is performed independently 10 times and the performance of existing sparse sampling and dense sampling is compared as a box plot. As a result of comparing the performance by changing the window size and the horizon factor, dense sampling consistently showed a better performance.

Spatial distribution models and biodiversity of phytoplankton cysts in the Black Sea

The current study employed diverse statistical and machine learning techniques to investigate the biodiversity and spatial distribution of phytoplankton cysts in the Black Sea. The MaxEnt distribution modeling technique was used to forecast the habitat suitability for the cysts of three potentially toxic microalgal taxa (Lingulodinium polyedra, Polykrikos hartmannii, and Alexandrium spp.). The key variables controlling the habitat suitability of Alexandrium spp. and L. polyedra were nitrates and temperature, while for the P. hartmannii cysts, nitrates and salinity. The region with the highest likelihood of L. polyedra cyst occurrence appears to be in the western coastal and shelf waters, which coincides with the areas where L. polyedra red tides have been documented. The projected habitat suitability of the examined species partially overlapped, perhaps as a result of their cohabitation within the phytoplankton community and shared preferences for specific environmental conditions, demonstrating similar survival strategies. The north-western region of the Black Sea was found to be the most suitable environment for the studied potentially toxic species, presumably posing a greater risk for the onset of blooming events. Two distinct aspects of cysts’ ecology and settlement were observed: the dispersal of cysts concerns their movement within the water column from one place to another prior to settling, while habitat suitability pertains to the particular environment required for their survival, growth, and germination. Therefore, it is crucial to validate the model in order to accurately determine a suitable habitat as well as understand the transportation patterns linked to the particular hydrodynamic properties of the water column and the distinct features of the local environment.

Development of a Novel Non-invasive Metabolomics Assay to Predict Implantation Potential of Human Embryos.

Can a set of metabolites present in embryo culture media correlate with embryo implantation? Case-control study in two phases: discovery phase (101 samples) and validation phase (169 samples), collected between 2018 and 2022, with a total of 218 participants. Culture media samples with known implantation outcomes were collected after blastocyst embryo transfer (including both PGT and non-PGT cycles) and were analyzed using chromatography followed by mass spectrometry. The spectra were processed and analyzed using statistical and machine learning techniques to identify biomarkers associated with embryo implantation, and to develop a predictive model. In the discovery phase, 148 embryo implantation biomarkers were identified using high resolution equipment, and 47 of them were characterized. Our results indicate a significant enrichment of tryptophan metabolism, arginine and proline metabolism, and lysine degradation biochemical pathways. After transferring the method to a lower resolution equipment, a model able to assign a Metabolite Pregnancy Index (MPI) to each embryo culture media was developed, taking the concentration of 36 biomarkers as input. Applying this model to 20% of the validation samples (N=34) used as the test set, an accuracy of 85.29% was achieved, with a PPV (Positive Predictive Value) of 88% and a NPV (Negative Predictive Value) of 77.78%. Additionally, informative results were obtained for all the analyzed samples. Metabolite concentration in the media after in vitro culture shows correlation with embryo implantation potential. Furthermore, the mathematical combination of biomarker concentrations using Artificial Intelligence techniques can be used to predict embryo implantation outcome with an accuracy of around 85%.

Prediction of the operational performance of a vehicle seat thermal management system using statistical and machine learning techniques

To assess and forecast the operational performance of a modified car seat for thermal management using an air conditioning system, statistical and machine learning (ML) models were used. By extending evaporator/condenser coils beneath the back and cushion surfaces of the car seat and using operational data on the HVAC system, such as seat temperature readings, an interval of operation was gathered. Using a data mining approach, statistically relevant factors and varying the compressor speed from 500 to 1600 rpm under various scenarios to model the system were selected. Utilizing key feature variables, our data-driven approach yielded predictions with favorable accuracy for the Coefficient of Performance (COP) of the HVAC system. By using the Akaike Information Criterion (AIC) to improve the Linear Regression (LR) model, the Root Mean Square Error (RMSE) dropped to 0.20, the Mean Absolute Error (MAE) dropped to 0.16, and the Coefficient of Determination (R2) increased to 98 %. The Random Forest (RF) model, optimized with hyperparameters, demonstrated moderate predictive capability, with RMSE (0.52), MAE (0.37), and R2 (94 %). Furthermore, polynomial feature augmentation, individual and combined predictor analysis, and iterative predictor combinations all improved predictive accuracy. Detailed information on the algorithms was given for the sake of other researchers.

Forecasting cryptocurrency returns using classical statistical and deep learning techniques

The emergence of cryptocurrencies has generated enthusiasm and concern in the modern global economy. However, their high volatility, erratic price fluctuations, and tendency to exhibit price bubbles have made investors cautious about investing in them. Consequently, it is essential to develop methods and models to forecast cryptocurrency returns to benefit investors, traders, and the scientific community. Despite the considerable volume of research on Bitcoin price forecasting, other cryptocurrencies have received little attention in academic literature. Additionally, the current body of literature on predicting cryptocurrency prices or returns emphasizes the use of in-sample methodologies. However, this method is susceptible to overfitting. To address these gaps in the literature, this study employs autoregressive moving average (ARMA), generalized autoregressive conditional heteroskedasticity (GARCH), exponential generalized autoregressive conditional heteroskedasticity (EGARCH), and long short-term memory (LSTM) deep learning neural networks to forecast returns for the ten most actively traded digital currencies: Bitcoin, Ethereum, Ripple, Chainlink, Litecoin, Cardano, Ethereum Classic, Bitcoin Cash, Tether, and Binance Coin. To assess the accuracy of the two models, this study utilizes an out-of-sample method with data gathered sequentially from November 9, 2017, to September 18, 2022. The results indicate that all models exhibit high accuracy, as evidenced by their low root mean square error (RMSE), mean absolute error (MAE), and mean squared error (MSE) values. Meanwhile, the hybrid EGARCH-LSTM or GARCH-LSTM models demonstrate slightly better accuracy compared with the other models. The findings are valuable for investors, traders, and researchers involved in cryptocurrency forecasting.