Statistical Learning Research Articles

Application of a decision tree approach to predict energy consumption in lightweight buildings under subtropical climate

PurposeLightweight building systems have emerged as alternatives to reduce the high environmental impact of conventional masonry. However, in subtropical climates, the low thermal inertia of lightweight envelopes negatively affects energy performance. The purpose of this paper is to investigate the thermophysical parameters that influence heating and cooling energy consumption in lightweight residential buildings under subtropical climates and develop a model to predict these parameters using statistical and machine learning tools.Design/methodology/approachA database was created with computer simulation data on the energy performance of 2048 building conditions generated by factorial combination of 10 parameters. Sensitivity analysis was performed to identify which parameters contribute most to energy performance indicators. Subsequently, decision trees were created using a classification and regression tree (CART) algorithm to visualize parameters and improve energy performance indicators, particularly cooling energy consumption.FindingsLow thermal transmittance and ground contact are interesting strategies for low thermal capacity buildings. Furthermore, the findings showed that relying only on the most influential properties does not ensure good energy performance; rather, it is the adequate combination of envelope properties that leads to good energy efficiency. The tree developed by CART can be used as a guide to assist designers and researchers in the initial selection of building envelopes, demonstrating the impact of each choice on electrical energy consumption for indoor climate control.Originality/valueBy adopting a global approach to assess the thermal performance of lightweight buildings, this study makes a significant contribution to synthesizing the results of a complex and time-consuming methodology into a guide for optimizing envelope design decisions and directing efforts and resources toward efficient strategies.

Refined selection of individuals for preventive cardiovascular disease treatment with a transformer-based risk model

Abstract Background Although statistical models have been commonly used to identify patients at risk of cardiovascular disease (CVD) for preventive therapy, these models tend to over-recommend therapy [1]. Moreover, in populations with pre-existing diseases, the current approach is to indiscriminately treat all individuals, as modelling in this context is currently inadequate. Aims To develop and validate a novel deep learning model, the Transformer-based Risk assessment survival model (TRisk) model, for prediction of 10-year risk of CVD in both the general population and individuals with diabetes. Methods An open cohort of 3 million adults aged 25 to 84 years free of CVD and statin use was identified using linked primary and secondary electronic health records from 291 and 98 general practices in England, which were used for TRisk model development and external validation, respectively. Comparison against two commonly recommended statistical models (QRISK3 [2] and SCORE2 [3]) and a deep learning derivation of QRISK3 was conducted. Additional analyses compared discriminatory performance in other age groups, by sex, and across categories of socioeconomic status. Similar analyses were conducted in a subgroup of 59 thousand patients with diabetes. Results TRisk demonstrated superior discrimination (C index in the general population: 0.910; 95% confidence interval [CI]: 0.906 to 0.913). TRisk’s performance was found to be less sensitive to population age range than the benchmark models and outperformed other models also in all stratified analyses. All models were overall well-calibrated. In decision curve analyses, TRisk demonstrated greater net benefit than benchmark models across the range of relevant thresholds. In impact analyses, at the widely recommended 10% risk threshold and the higher 15% threshold for the general population, TRisk reduced both the total number of patients classified at high risk (by 27% and 39% respectively) and the number of false negatives as compared with currently recommended strategies. TRisk similarly outperformed other models in patients with diabetes. Compared with the widely recommended treat-all policy approach for patients with diabetes, TRisk at a 10% risk threshold would lead to deselection of 24% of individuals with a small fraction of false negatives (0.2% of cohort). Conclusion TRisk enabled a more targeted selection of individuals at risk of CVD compared to benchmark statistical and deep learning models, in both the general population and patients with diabetes. Incorporation of TRisk into routine clinical care would allow a reduction in the number of treatment-eligible patients in general population and subgroup with diabetes by approximately one-third and one-quarter respectively while preventing at least as many events as with currently adopted approaches.Models' discrimination on both cohortsImpact analyses for both cohorts

Methods to Handle Missing depression (PHQ-8 Score) Values among Adults in the United Arab Emirates

Abstract Background The UAE Healthy Future Study (UAEHFS) is one of the first large prospective cohort studies in the Gulf region which examines causes and risk factors for chronic diseases among adult UAE nationals. Missing values are often unavoidable in empirical research and can in many cases, lead to bias. The aim of this study is to estimate the percentage of depression in the UAEHFS pilot data using the eight-item Patient Health Questionnaire (PHQ-8) variables, using different statistical methods. Methods Five common statistical machine learning methods of handling missing values were included in this analysis. These are mode imputation, k-nearest neighbor (KNN) imputation, classification, and regression trees (CART), random forest (RF) imputations, and random sample from observed values (Sample). 100 multiple imputations were used. Results 487 (94.2 %) eligible participants were included in the analysis. 231 (44.7%) were included in the complete case analysis. The median age was 30 years (Interquartile-Range: 23 - 38). More males (67.8%) than females included in the analysis. The estimated percentage of depression was 8.4%, 8.9%, 9.9%, 12.5%, 15.4% and 17.9% by the mode, complete case, sample, RF, CART, and KNN respectively. In additional analyses, the estimated proportions of depression were 11.5% by the Complete Case, 11.9% by KNN, 13.2% by K-means clustering, and 13.2% by Random Forest. Conclusions The estimated percentage of depression in the UAEHFS pilot data varies between the applied methods of handling missing values. This shows that the problem of missing values in the variables is not negligible. Further research is needed using multiple imputations in the main UAEHFS dataset after completing recruitment. Key messages • For the depression missing values, we recommend using multiple imputations not to generate data but to prevent the exclusion of observed data. • To have a better estimate of the percentage of depression, is recommended to use different machine learning methods.

Machine Learning techniques for predicting inpatient probability and LOS from EU Injury Database

Abstract Background Unlike statistical inference Machine Learning (ML) makes repeatable predictions without prior assumptions about the underlying relationships among variables. The Full Data Set (FDS) of the EU-IDB (European Injury DataBase) consent to explore risk of hospitalization due to injuries using many predictors. The Length Of Stay of patients in hospital (LOS) is a relevant proxy of complexity of treatment and resources consumption. Methods The IDB-FDS provides more than 3.800.000 ED records, in years 2008-19 for 19 Countries. LASSO (Least Absolute Shrinkage and Selection Operator) cross-validated linearized regression technique (linear for LOS) was used for variable selection and parameter regularization. Inpatients are those admitted or transferred to hospital. Days of hospitalization were used for LOS. A cross-validated Generalized Linear Model was performed on 5 folds randomly sampled assigning 80% of records to training and 20% to testing samples. Results The strongest predictors of hospital admission risk, selected by the model were in order of importance: EUROCOST-39 diagnoses categories, Age Group, Intent, Mechanism Of Injury, Activity When Injured, Transport Injury Event, Sex Of Patient, Place Of Occurrence. EUROCOST-39 represents 61,9% of explained variability and age group 19,4%. The strongest predictors of LOS were substantially the same: EUROCOST-39 86,2% of explained variability and age group 8,6%. Conclusions The main part of variability in the ML model is explained by diagnoses reclassified according to a disability standardization method. For instance, in the maximum training sample risk of hospitalization ranges from odd 0,76% for hand/fingers sprain up to 154,02% for brain concussion. In the median sample LOS ranges from 0,45 days for strain of hand/fingers up to 11,65 for multi-trauma. A combination of more disabling injury, older age and mechanism of injury (i.e. threat to breathing) increases enormously the risk of hospitalization and LOS. Key messages • Machine Learning techniques applied on EU-IDB can provide identification of relevant risk factors of hospital admission from injuries for targeting preventive measures and organizing health care. • Artificial Intelligence consents to analyse big amount of data as dimension or analytical detail of information for identifying patterns and predictors of specific indicators for injuries or diseases.

Mortgage Loan Data Exploration with Non-parametric Statistical and Machine Learning Perspectives

Mortgage Loan Data Exploration with Non-parametric Statistical and Machine Learning Perspectives

Research and Prediction of China's Air Environment

Abstract: With the rapid development of China's economy, environmental issues, particularly air pollution, have become increasingly severe, posing significant threats to human health and ecosystems. This study employed statistical learning methods to analyze air pollution composition data from thirty major Chinese cities throughout 2019-2022. The study used algorithms like regression, classification, fitting, and filtering to find changes in multi-source heterogeneous data patterns and predict how air pollutant concentrations would change over time and space. At the same time,the results indicate that the primary factors influencing the Air Quality Index (AQI) are fine particulate matter (PM2.5) and inhalable particulate matter (PM10). Additionally, this research is significant for improving public health, advancing environmental protection, fostering economic growth, strengthening laws and regulations, ensuring social stability, and promoting scientific and cultural advancements. Through this research, scientific bases can be provided for formulating policies that reduce disease, enhance government credibility, drive innovation in atmospheric research, and preserve cultural heritage.

Some notes on the consequences of pretreatment of multivariate data

With the advent of data technologies, we have various types of data, such as structured, unstructured and semi-structured. Performing certain statistical or machine learning techniques may require careful preprocessing or pretreatment of the data to make them suitable for analysis. For example, given a data matrix X, which represents n multivariate observations or cases on p variables or features, the columns/rows of X may be pretreated before applying statistical or machine learning techniques to the data. While centering and/or scaling the variables do not alter the correlation structure nor the graphical representation of the data, centering/scaling the observations do. We investigate various row pretreatment methods more closely and show with theoretical proofs and numerical examples that centering/scaling the rows of X changes both the graphical structure of the observations in the multi-dimensional space and the correlation structure among the variables. There may be good reasons for performing row centering/scaling on the data and we are not against it, but analysts who use such row operations should be aware of the geometrical and correlation structures one has performed on the data and should also demonstrate that the process results in a new, more appropriate structure for their questions.

Quantum Machine Learning—Quo Vadis?

The book Quantum Machine Learning: What Quantum Computing Means to Data Mining, by Peter Wittek, made quantum machine learning popular to a wider audience. The promise of quantum machine learning for big data is that it will lead to new applications due to the exponential speed-up and the possibility of compressed data representation. However, can we really apply quantum machine learning for real-world applications? What are the advantages of quantum machine learning algorithms in addition to some proposed artificial problems? Is the promised exponential or quadratic speed-up realistic, assuming that real quantum computers exist? Quantum machine learning is based on statistical machine learning. We cannot port the classical algorithms directly into quantum algorithms due to quantum physical constraints, like the input–output problem or the normalized representation of vectors. Theoretical speed-ups of quantum machine learning are usually analyzed in the literature by ignoring the input destruction problem, which is the main bottleneck for data encoding. The dilemma results from the following question: should we ignore or marginalize those constraints or not?

Detecting marine heatwaves below the sea surface globally using dynamics-guided statistical learning

Extreme warm water events, known as marine heatwaves, cause a variety of adverse impacts on the marine ecosystem. They are occurring more and more frequently across the global ocean. Yet monitoring marine heatwaves below the sea surface is still challenging due to the sparsity of in situ temperature observations. Here, we propose a statistical learning method guided by ocean dynamics and optimal prediction theory, to detect subsurface marine heatwaves based on the observable sea surface temperature and sea surface height. This dynamics-guided statistical learning method shows good skills in detecting subsurface marine heatwaves in the oceanic epipelagic zone over many parts of the global ocean. It outperforms both the classical ordinary least square regression and popular deep learning methods that do not effectively exploit ocean dynamics, with clear dynamical interpretation for its outperformance. Our study provides a useful statistical learning method for near real-time monitoring of subsurface marine heatwaves at a global scale and highlights the importance of exploiting ocean dynamics for enhancing the efficiency and interpretability of statistical learning.

Predicting non-responders to lifestyle intervention in prediabetes: a machine learning approach.

The clinical care process for people with prediabetes starts with lifestyle intervention, often escalating to more intense treatment due to the low success rate of the first-line intervention. Clinicians lack clear guidelines on which patients would benefit from early treatment with more intensive therapeutic options, so we aimed to develop an algorithm to early identify non-responders to lifestyle intervention for prediabetes. Several statistical and machine learning algorithms were screened with internal cross-validation on the basis of accuracy and discrimination ability to correctly classify patients that would fail to normalize fasting glycemia within one year of being prescribed a lifestyle intervention, solely based on the first examination measurements. Of the many screened algorithm, only a random forest model performed with sufficient accuracy to exceed the historical failure rate of patients within our center, with an accuracy of 0.689 (CI 0.669, 0.710) and an AUROC of 0.687 (CI 0.673, 0.701). This study showcases the ability of machine learning models to provide useful insight in clinical practice leveraging knowledge contained in routinely collected data.

Policy interventions and urban characteristics in modeling electric vehicle charging infrastructure utilization

The surge in electric vehicles adoption necessitates understanding the impact of policy interventions on public electric vehicle charging infrastructure in urban areas. This research investigates the influence of pricing frameworks on the usage of public charging facilities by analyzing both behavioral and spatial attributes of these infrastructures. Utilizing open data from Palo Alto, United States, this study employs descriptive statistical methods and interpretable machine learning approaches to scrutinize the relationship between policy initiatives and charging behaviors. The analysis underscores the significance of spatial attributes on charging behaviors. Policy interventions yield noticeable alterations in charging metrics, with locations near commercial hubs showing higher utilization, while local and frequent users resist fee adjustments. The research emphasizes the necessity for customized strategies to optimize infrastructure development and management, offering a framework for policymakers and stakeholders in sustainable urban transportation. Future research should explore similar interventions in diverse urban settings using real-time data and advanced optimization techniques to better tailor policies to the unique characteristics of specific facilities.

On learning the optimal regularization parameter in inverse problems

Abstract Selecting the best regularization parameter in inverse problems is a classical and yet challenging problem. Recently, data-driven approaches have become popular to tackle this challenge. These approaches are appealing since they do require less a priori knowledge, but their theoretical analysis is limited. In this paper, we propose and study a statistical machine learning approach, based on empirical risk minimization. Our main contribution is a theoretical analysis, showing that, provided with enough data, this approach can reach sharp rates while being essentially adaptive to the noise and smoothness of the problem. Numerical simulations corroborate and illustrate the theoretical findings. Our results are a step towards grounding theoretically data-driven approaches to inverse problems.

EXPRESS: Individual differences in distributional statistical learning: better frequency 'discriminators' are better 'estimators'.

People can easily extract and encode statistical information from their environment. However, research has primarily focused on conditional statistical learning (i.e. the ability to learn joint and conditional relationships between stimuli) and has largely neglected distributional statistical learning (i.e. the ability to learn the frequency and variability of distributions). For example, learning that 'E' is more common in the English alphabet than 'Z.' In this paper, we investigate how distributional learning can be measured by exploring the relationship between, and psychometric properties of, four different measures of distributional learning - from the ability to discriminate relative frequencies to the ability to estimate frequencies. We identified moderate relationships between four distributional learning measures and these tasks accounted for a substantial portion of the variance in performance across tasks (44.3%). A measure of divergent validity (intrinsic motivation) did not significantly correlate with any statistical learning measure and accounted for a separate portion of the variance across tasks. More complex measures also showed better reliability and internal consistency. Our results suggest that distributional statistical learning encompasses both the ability to discriminate between relative frequencies and directly estimating them.

Machine Learning Models Informed by Connected Mixture Components for Short- and Medium-Term Time Series Forecasting

This paper presents a new approach in the field of probability-informed machine learning (ML). It implies improving the results of ML algorithms and neural networks (NNs) by using probability models as a source of additional features in situations where it is impossible to increase the training datasets for various reasons. We introduce connected mixture components as a source of additional information that can be extracted from a mathematical model. These components are formed using probability mixture models and a special algorithm for merging parameters in the sliding window mode. This approach has been proven effective when applied to real-world time series data for short- and medium-term forecasting. In all cases, the models informed by the connected mixture components showed better results than those that did not use them, although different informed models may be effective for various datasets. The fundamental novelty of the research lies both in a new mathematical approach to informing ML models and in the demonstrated increase in forecasting accuracy in various applications. For geophysical spatiotemporal data, the decrease in Root Mean Square Error (RMSE) was up to 27.7%, and the reduction in Mean Absolute Percentage Error (MAPE) was up to 45.7% compared with ML models without probability informing. The best metrics values were obtained by an informed ensemble architecture that fuses the results of a Long Short-Term Memory (LSTM) network and a transformer. The Mean Squared Error (MSE) for the electricity transformer oil temperature from the ETDataset had improved by up to 10.0% compared with vanilla methods. The best MSE value was obtained by informed random forest. The introduced probability-informed approach allows us to outperform the results of both transformer NN architectures and classical statistical and machine learning methods.

Incorporating driving behavior into vehicle fuel consumption prediction: methodology development and testing

This study is a significant endeavor involving the development and testing of a comprehensive methodology to incorporate driving behavior into the analysis and prediction of vehicle fuel consumption. It underscores the crucial importance of understanding how different driving behavior affect fuel efficiency. The framework we present is a theoretical construct and a practical tool. It provides a robust, multi-step process for linking driving behavior to fuel consumption, leveraging both traditional statistical methods and advanced machine learning techniques to derive actionable insights. To test the framework, we used a naturalistic data that includes about 5408 different road users in a mixed traffic environment and urban settings in Germany. We applied a microscopic fuel consumption model to calibrate the framework and an unsupervised clustering algorithm to classify the behavior of the driver interacting with each other and with vulnerable road users. The framework includes developing Linear regression model as a baseline, which yields an R-squared of 0.511 and a Mean Squared Error (MSE) of 0.031, indicating moderate predictive accuracy. The final step includes choosing Random Forest as a better model, which achieves a higher R-squared of 0.956 and a lower MSE of 0.003. We also found that conservative and aggressive driving leads to significantly higher and more discrepancy in fuel consumption than normal driving behavior. These insights can promote more efficient driving practices, leading to significant fuel savings and environmental benefits.

Elucidating a statistical learning brain network: Coordinate-based meta-analyses and functional connectivity profiles of artificial grammar learning in healthy adults

Abstract Language rehabilitation centers on modifying its use through experience-based neuroplasticity. Statistical learning of language is essential to its acquisition and likely its rehabilitation following brain injury, but its corresponding brain networks remain elusive. Coordinate-based meta-analyses were conducted to identify common and distinct brain activity across 25 studies coded for meta-data and experimental contrasts (Grammatical and Ungrammatical). The resultant brain regions served as seeds for profiling functional connectivity in large task-independent and task-dependent data sets. Hierarchical clustering of these profiles grouped brain regions into three subnetworks associated with statistical learning processes. Functional decoding clarified the mental operations associated with those subnetworks. Results support a left-dominant language sub-network and two cognitive control networks as scaffolds for language rule identification, maintenance, and application in healthy adults. These data suggest that cognitive control is necessary to track regularities across stimuli and imperative for rule identification and application of grammar. Future empirical investigation of these brain networks in language learning in individuals with brain injury will clarify their prognostic role in language recovery.

Classification of Corn Diseases and Pests Using Fuzzy Naïve Bayes Method

Corn is an essential dietary source for humans and animals. In addition to being a food source, corn has numerous benefits as a manufacturing commodity. The quality of grain crops must be considered to minimise the likelihood of disease and pest infestations. Therefore, the diseases and pests that attack corn plants must be classified so that farmers can control them during the growth period of corn plants. The fuzzy naive Bayes method is a statistical machine learning method that can be used to classify the diseases and pests of corn crops based on colour space-transformed digital images. This study aims to classify corn plant diseases and pests using the fuzzy naive Bayes method. Digital images of corn plant diseases and pests were transformed into a red, green and blue colour space model. The following seven classes of corn plant diseases and pests were classified: leaf rust disease, downy mildew disease, leaf blight disease, Locusta pest, Heliotis armigera pest, Spodoptera frugiperdita pest and non-pathogenic pest. With this method, the classification model achieves an accuracy of 87.83%, a macro precision of 34.91%, a macro recall of 35.90% and a macro f-score of 33.82%.

The efficacy of a food supplement in the treatment of tinnitus with comorbid headache: A Statistical and Machine Learning analysis with a literature review.

Introduction Tinnitus, the perception of sound without an external auditory stimulus, affects approximately 10% to 15% of the population and is often associated with significant comorbidities such as headaches. These conditions can severely impact the quality of life. The aim of this study was to evaluate the efficacy of a food supplement in reducing the symptoms of both tinnitus and headache in patients experiencing these conditions concurrently. Methods This prospective study included 32 patients (21 males and 11 females) aged between 23 and 68 years (mean age 49.38 years) who were experiencing both tinnitus and headache. The study assessed the impact of a food supplement on tinnitus and headache over a 90-day treatment period using three main instruments: the Tinnitus Handicap Inventory (THI), the Headache Impact Test (HIT-6), and a Visual Analog Scale (VAS) for discomfort. Statistical analyses, including paired t-tests, were conducted to compare pre- and post-treatment scores. In the same dataset, Ridge Regression, a linear regression model with L2 regularization, was used to predict post-treatment scores (THI90, HIT90, VAS90). Results The results indicated a statistically significant reduction in all three measures after 90 days of treatment. The mean THI score decreased from 29.81 to 27.06 (p = 0.011), the mean HIT-6 score decreased from 50.41 to 48.75 (p = 0.019), and the mean VAS score for discomfort decreased from 7.63 to 7.13 (p = 0.033). The optimal Ridge Regression model was found with an `alpha` value of approximately 3.73. The performance metrics on the test set were as follows: Mean Squared Error (MSE) of 13.91 and an R-squared score of 0.61, indicating that the model explains approximately 61% of the variance in the post-treatment scores. These results indicate that pre-treatment scores are significant predictors of post-treatment outcomes, and gender plays a notable role in predicting HIT and VAS scores post-treatment. Conclusion This study demonstrates that a food supplement is effective in reducing the symptoms of tinnitus and headache in patients suffering from both conditions. The significant improvements in THI, HIT-6, and VAS scores indicate a positive impact on patient quality of life. Further research with larger sample sizes and more detailed subgroup analyses is recommended to fully understand the differential impacts of treatment across various demographics.

Research progress on correlative prediction factors and prediction models of endometriosis associated ovarian carcinoma.

Endometriosis is a common benign disease in women of childbearing age, with a malignant change rate of about 1%. Endometriosis associated ovarian cancer (EAOC), which usually occurs in the ovaries, is a serious threat to women's health. Early identification of high-risk groups of EMs malignant transformation is of great significance for the prevention and treatment of EAOC. However, there is still a lack of specific and sensitive prediction factors. In recent years, scholars at home and abroad have used traditional statistical methods and machine learning to explore EAOC related prediction factors and prediction models. This paper mainly reviews and evaluates the diagnosis and prediction model of EAOC. Studies were identified by searching the CNKI, PubMed and Web of Science Core Collection, (WOSCC) till 2023, Data which met the inclusion criteria of clinical studies were evaluated about the quality. This paper analyzes and summarizes the prediction factors and prediction models in the literature. After screening, 7 relevant studies were finally obtained. Prediction factors included: age, menstruation, menopausal status, course of disease, infertility associated with endometriosis, history of single estrogen use during menopause, serological indexes: human epididymis protein 4, carbohydrate antigen 125(CA125), ovarian malignancy risk algorithm, indications for ultrasound examination: cyst shape, structure and blood flow signal, etc. Prediction models: Alignment diagram, Multivariate logistic regression model, Gail model, Gradient Boosting Decision Tree and Lasso-logistics regression. Related models were in good agreement with the actual situation, and have good sensitivity and specificity. The relevant prediction factors and prediction models were summarized to provide reference and new thinking for the research of prediction models in the field of EAOC, in order to develop standardized long-term management strategies for high-risk groups of EAOC and realize the advance of the diagnosis threshold of patients with EAOC.

Fixing imbalanced binary classification: An asymmetric Bayesian learning approach.

Most statistical and machine learning models used for binary data modeling and classification assume that the data are balanced. However, this assumption can lead to poor predictive performance and bias in parameter estimation when there is an imbalance in the data due to the threshold election for the binary classification. To address this challenge, several authors suggest using asymmetric link functions in binary regression, instead of the traditional symmetric functions such as logit or probit, aiming to highlight characteristics that would help the classification task. Therefore, this study aims to introduce new classification functions based on the Lomax distribution (and its variations; including power and reverse versions). The proposed Bayesian functions have proven asymmetry and were implemented in a Stan program into the R workflow. Additionally, these functions showed promising results in real-world data applications, outperforming classical link functions in terms of metrics. For instance, in the first example, comparing the reverse power double Lomax (RPDLomax) with the logit link showed that, regardless of the data imbalance, the RPDLomax model assigns effectively lower mean posterior predictive probabilities to failure and higher probabilities to success (21.4% and 63.7%, respectively), unlike Logistic regression, which does not clearly distinguish between the mean posterior predictive probabilities for these two classes (36.0% and 39.5% for failure and success, respectively). That is, the proposed asymmetric Lomax approach is a competitive model for differentiating binary data classification in imbalanced tasks against the Logistic approach.