Statistical Learning Research Articles

Modeling mortality with Kernel Principal Component Analysis (KPCA) method

Abstract As the global population continues to age, effective management of longevity risk becomes increasingly critical for various stakeholders. Accurate mortality forecasting serves as a cornerstone for addressing this challenge. This study proposes to leverage Kernel Principal Component Analysis (KPCA) to enhance mortality rate predictions. By extending the traditional Lee-Carter model with KPCA, we capture nonlinear patterns and complex relationships in mortality data. The newly proposed KPCA Lee-Carter algorithm is empirically tested and demonstrates superior forecasting performance. Furthermore, the model’s robustness was tested during the COVID-19 pandemic, showing that the KPCA Lee-Carter algorithm effectively captures increased uncertainty during extreme events while maintaining narrower prediction intervals. This makes it a valuable tool for mortality forecasting and risk management. Our findings contribute to the growing body of literature where actuarial science intersects with statistical learning, offering practical solutions to the challenges posed by an aging world population.

Constructing phylogenetic trees for microbiome data analysis: A mini-review

As next-generation sequencing technologies advance rapidly and the cost of metagenomic sequencing continues to decrease, researchers now face an unprecedented volume of microbiome data. This surge has stimulated the development of scalable microbiome data analysis methods and necessitated the incorporation of phylogenetic information into microbiome analysis for improved accuracy. Tools for constructing phylogenetic trees from 16S rRNA sequencing data are well-established, as the highly conserved regions of the 16S gene are limited, simplifying the identification of marker genes. In contrast, metagenomic and whole genome shotgun (WGS) sequencing involve sequencing from random fragments of the entire gene, making identification of consistent marker genes challenging owing to the vast diversity of genomic regions, resulting in a scarcity of robust tools for constructing phylogenetic trees. Although bacterial sequence tree construction tools exist for upstream bioinformatics, many downstream researchers—those integrating these trees into statistical models or machine learning—are either unaware of these tools or find them difficult to use due to the steep learning curve of processing raw sequences. This is compounded by the fact that public datasets often lack phylogenetic trees, providing only abundance tables and taxonomic classifications. To address this, we present a comprehensive review of phylogenetic tree construction techniques for microbiome data (16S rRNA or whole-genome shotgun sequencing). We outline the strengths and limitations of current methods, offering expert insights and step-by-step guidance to make these tools more accessible and widely applicable in quantitative microbiome data analysis.

Highly selective extraction of gold from wasted random-access memory using a hybrid nanocomposite: Statistical, DFT, and machine learning modeling

This study successfully developed an advanced protocol for recovering gold (Au(III)) from spent random-access memory (RAM) utilizing a mesoporous FeNi2S4 integrated with g-C3N4 nanosheets. The innovative design of this hybrid material proved essential for achieving high efficiency and selectivity in adsorbing Au(III) ions from RAM. Extensive characterization of the g-C3N4 nanosheets and the FeNi2S4@g-C3N4 nanocomposite, covering physicochemical, textural, structural, morphology, and elemental composition, revealed that the strategic incorporation of FeNi2S4 significantly enhanced the composite's adsorption capabilities. The mesoporous FeNi2S4@g-C3N4 composite demonstrated superior performance, with an adsorption capacity of 203.12mg/g at an optimal pH of 2, and retained its high efficiency across more than 10 adsorption-desorption cycles. Furthermore, FeNi2S4@g-C3N4 exhibited strong resistance to interference ions, maintaining over 96% selectivity for Au(III) trapping. Gradient Boosting and Extra Trees models were instrumental in accurately predicting and optimizing the Au(III) adsorption process. The adsorption mechanism was clarified through zeta potential and DFT measurements, identifying strong chemisorption interaction between Au(III) and FeNi2S4@g-C3N4 sorbent. The study concludes that the FeNi2S4@g-C3N4 nanocomposite is an efficient and reusable material for selectively extracting Au(III) from RAM, providing a sustainable and scalable method for recovering valuable metals from complex waste streams, as supported by thorough experimental and computational evidence.

Successful sensitization of 2.5-year-olds to other-race faces through bimodal training

The present study investigated the potential for sensitizing 2.5-year-old Caucasian infants to other-race faces (Asian faces). In the domain of face perception, infants become less sensitive to facial distinctions of other-race faces through perceptual narrowing at the end of the first year of life. Nevertheless, infants around 12 months can regain their sensitivity to other-race faces. For instance, exposing them to a specific statistical distribution and employing the mechanisms of statistical learning is one way to enhance their discriminatory abilities towards other-race faces. Following this idea, we investigated if even older infants around 2.5 years can be sensitized to other-race faces. We trained the infants with a bimodal distribution of a morphed continuum of Asian female faces with faces closer to the endpoints presented most frequently. We assessed infants’ discrimination of Asian faces by measuring their looking times after the training phase. The 2.5-year-olds showed a difference in looking times after the training, indicating that the exposure to a bimodal frequency distribution led to a successful discrimination between Asian faces. These findings demonstrate that 2.5-year-olds can be sensitized to other-race faces by exposing them to a bimodal distribution of such faces, underlining the plasticity of face perception in childhood.

Unified Machine Learning Techniques for High-Dimensional Survival Data Analysis

Survival analysis is a statistical method used to analyze time-to-event data. The existence of states where event results become invisible after a given time is one of the primary difficulties with this regulation. This is considered censorship. This kind of data may be found in a variety of applications, such as mechanical system failure rates, patient mortality rates during clinical trials, and the length of unemployment in a community. Assessing the so-called survival and hazard functions is one of the primary goals of survival analysis. Many machine-learning algorithms have also been developed to tackle censored data and other difficult problems with real-world data. Machine-learning techniques enhance survival analysis by incorporating flexible modeling approaches, handling high-dimensional data, capturing nonlinear relationships, and accounting for censorship. They provide powerful tools to extract meaningful insights and make accurate predictions in time-to-event analysis across various healthcare, finance, and engineering domains. The standard statistical approaches and machine learning techniques advanced for survival analysis are structured in this unified framework, as is the implementation of some machine learning techniques applying to the GBSG2 survival analysis dataset.

Empowering Communities Through Resilient River Management

This paper focuses on empowering the society with various rescue strategies in the event of a natural disaster namely floods. This paper deals with applying the state of the art technologies such as machine learning in order to combat floods. The key aspects of a flood management system and the existing systems were then identified. The proposed system was then developed. Statistical, hydrological and Machine learning algorithms such as Random forest, support vector machines, Naïve Bayes, decision tree algorithms were further applied on the data parameters captured. to predict the overflow probability in different regions of the state. A comparison was then undertaken to determine the most suited algorithm to perform better river management. The paper then concluded highlighting the need for applying machine learning techniques for flood management.

Predicting the time to get back to work using statistical models and machine learning approaches

BackgroundWhether machine learning approaches are superior to classical statistical models for survival analyses, especially in the case of lack of proportionality, is unknown.ObjectivesTo compare model performance and predictive accuracy of classic regressions and machine learning approaches using data from the Inspiring Families programme.MethodsThe Inspiring Families programme aims to support members of families with complex issues to return to work. We explored predictors of time to return to work with proportional hazards (Semi-Parametric Cox in Stata) and (Flexible Parametric Parmar-Royston in Stata) against the Survival penalised regression with Elastic Net penalty (scikit-survival), (conditional) Survival Forest algorithm (pySurvival), and (kernel) Survival Support Vector Machine (pySurvival).ResultsAt baseline we obtained data on 61 binary variables from all 3161 participants. No model appeared superior, with a low predictive power (concordance index between 0.51 and 0.61). The median time for finding the first job was about 254 days. The top five contributing variables were ‘family issues and additional barriers’, ‘restriction of hours’, ‘available CV’, ‘self-employment considered’ and ‘education’. The Harrell’s Concordance index was range from 0.60 (Cox model) to 0.71 (Random Survival Forest) suggesting a better fit for the machine learning approaches. However, the comparison for predicting median time on a selected scenario based showed only minor differences.ConclusionImplementing a series of survival models with and without proportional hazards background provides a useful insight as well as better interpretation of the coefficients affected by non-linearities. However, that better fit does not translate to substantially higher predictive power and accuracy from using machine learning approaches. Further tuning of the machine learning algorithms may provide improved results.

Frontal Deactivation and the Efficacy of Statistical Learning: Neural Mechanisms Accompanying Exposure to Visual Statistical Sequences.

Statistical learning is the cognitive ability to rapidly identify structure and meaning in unfamiliar streams of sensory experience, even in the absence of feedback. Despite extensive studies, the neurocognitive mechanisms underlying this phenomenon still require further clarification under varying cognitive conditions. Here, we examined neural mechanisms during the first exposure to visually presented sequences in 47 healthy participants. We used two types of visual objects: abstract symbols and pictures of cartoon-like animals. This allowed us to compare informational processing mechanisms with defined distinguishing features. Participants achieved better performance for sequences with easy-to-name than difficult-to-name abstract stimuli. fMRI results revealed greater activation in widespread brain regions in response to random versus statistical sequences for all stimuli types. Behavioral accuracy was associated with increased deactivation of the ventromedial pFC for easy-to-name statistical versus random sequences. For difficult-to-name statistical versus random sequences, performance correlated with dmPFC deactivation. ROI analysis showed a generally positive involvement of the caudate head in sequence processing with significantly stronger activity during the first run of performing the task. Functional connectivity analysis of prefrontal deactivation regions revealed significant connectivity with nodes of the salience network for both object types and inverse connectivity with the caudate head only for easy-to-name objects. The results indicated that distinct subregions of pFC modulate task performance depending on the visual stimulus characteristic. They also showed that among striatal regions, only the head of the caudate was sensitive to initial exposure to visual statistical information.

Introduction to deep learning methods for multi‐species predictions

Abstract Predicting species distributions and entire communities is crucial for ecologists, to enhance our understanding of the drivers behind species distributions and community assembly and to provide quantitative data for conservation efforts. Popular species distribution models use statistical and machine learning methods but face limitations with multi‐species predictions at the community level, hindered by scalability and data imbalance sensitivity. This paper explores the potential of deep learning methods to overcome these challenges and provide more accurate multi‐species predictions. Specifically, we introduced four distinct deep learning models that use site × species community data but differ in their internal structure or on the input environmental data structure: (1) a multi‐layer perceptron (MLP) model for tabular data (e.g. in‐situ/raster climate or soil data), (2) a convolutional neural network (CNN) and (3) a vision transformer (ViT) models tailored for image data (e.g. aerial ortho‐photographs, satellite imagery), and a multimodal model that integrates both tabular and image data. We also show how adapted loss functions can address imbalance issues. We applied these deep learning models to a plant community dataset comprising 130,582 vegetation surveys encompassing 2522 species located in the French Alps. The tabular environmental data consisted of climate, terrain and soil information, while the images were derived from aerial photographs. All models achieved approximately 70% true skill statistics on hold‐out data, demonstrating high predictive capacity for community data, the multimodal model being the best performing one. Additionally, we showcased how interpretability tools can illuminate community structure as seen by deep learning models. Deep learning models offer a broad array of features for predicting entire species communities. They handle imbalance issues and accommodate various data types, from tabular datasets to images, while also being equipped with insightful interpretation tools. The versatility extends to tabular datasets and images, with no clear superiority between the two. The last hidden layers can provide valuable features for modelling other species, and the trained models can be used to support transfer learning to related tasks. The field of ecology now possesses an additional, potent tool in its arsenal that can foster basic and fundamental research.

Missing data interpolation in well logs based on generative adversarial network and improved krill herd algorithm

Accurate logging data is crucial in geology and petroleum engineering for tasks such as geological modelling, reservoir simulation, and decision-making regarding well repair, water injection, and oil recovery. However, logging instrument failure occurs due to complex conditions such as high temperature and pressure, resulting in incomplete data and posing challenges for reservoir evaluation and development. The existing interpolation methods are primarily based on statistical and machine learning methods, lacking deep mining of hidden associations between logging items. Aiming at the problem of incomplete well-logging data, an incomplete well-logging data interpolation method based on a generative adversarial network and an improved krill herd algorithm is proposed. The results show that the proposed method has stable interpolation for well-logging data missing with different missing rates and any missing positions. Compared with other GANs (GAN, WGAN, and WGAN-GP), the RMSE of the proposed method is reduced by 57.63%, and the R2 is increased by 7.94%. The proposed method is compared with statistical methods (averaging and cubic spline interpolation) and machine learning methods (k-nearest neighbor, support vector machine, and random forest). The experimental results show that the proposed model has stable reconstruction performance for logging data with different missing rates and any missing positions.

Statistical Learning Theory and Occam’s Razor: The Core Argument

Statistical learning theory is often associated with the principle of Occam’s razor, which recommends a simplicity preference in inductive inference. This paper distills the core argument for simplicity obtainable from statistical learning theory, built on the theory’s central learning guarantee for the method of empirical risk minimization. This core “means-ends” argument is that a simpler hypothesis class or inductive model is better because it has better learning guarantees; however, these guarantees are model-relative and so the theoretical push towards simplicity is checked by our prior knowledge.

Melodic contour supersedes short-term statistical learning in expressive accentuation.

Sensory systems are permanently bombarded with complex stimuli. Cognitive processing of such complex stimuli may be facilitated by accentuation of important elements. In the case of music listening, alteration of some surface features -such as volume and duration- may facilitate the cognitive processing of otherwise high-level information, such as melody and harmony. Hence, musical accents are often aligned with intrinsically salient elements in the stimuli, such as highly unexpected notes. We developed a novel listening paradigm based on an artificial Markov-chain melodic grammar to probe the hypothesis that listeners prefer structurally salient events to be consistent with salient surface properties such as musical accents. We manipulated two types of structural saliency: one driven by Gestalt principles (a note at the peak of a melodic contour) and one driven by statistical learning (a note with high surprisal, or information content [IC], as defined by the artificial melodic grammar). Results suggest that for all listeners, the aesthetic preferences in terms of surface properties are well predicted by Gestalt principles of melodic shape. In contrast, despite demonstrating good knowledge of novel statistical properties of the melodies, participants did not demonstrate a preference for accentuation of high-IC notes. This work is a first step in elucidating the interplay between intrinsic, Gestalt-like and acquired, statistical properties of melodies in the development of expressive musical properties, with a focus on the appreciation of dynamic accents (i.e. a transient increase in volume). Our results shed light on the implementation of domain-general and domain-specific principles of information processing during music listening.

Identifying homogeneous hydrological zones for flood prediction using multivariable statistical methods and machine learning

One method for estimating floods in areas lacking statistical data is the use of regional frequency analysis based on machine learning. In this study, statistical and clustering-based approaches were evaluated for flood estimation in the Karkheh watershed. The hydrological homogeneity of the obtained zones was then assessed using linear moments and heterogeneity adjustment methods proposed by Hosking and Wallis. Then, the ZDIST statistic was used to calculate the three-parameter distributions for stations within each hydrologically homogeneous cluster. These parameters were computed using linear moments, and floods with different return periods at each station were estimated using regional relationships. The results indicated the creation of two clusters in this area, with five stations in cluster one and 11 stations in cluster two. The statistical homogeneity values for clusters one and two were calculated as 0.33 and 0.17, respectively, indicating the homogeneity of each region. Generalized Pearson type III and generalized extreme value distributions were selected as the best regional distributions for clusters 1 and 2, respectively. The results also showed that floods could be estimated for return periods of 2, 5, 25 years, and more. The highest estimated flood is predicted at the Jelugir-e Majin station, where the flood with a 2-year return period reaches 1034 m3 s−1. This increases to 5360 m3 s−1 for a 100-year return period. The approach presented in this study is recommended for similar regions lacking complete information.

PitRSDNet: Predicting intra‐operative remaining surgery duration in endoscopic pituitary surgery

AbstractAccurate intra‐operative Remaining Surgery Duration (RSD) predictions allow for anaesthetists to more accurately decide when to administer anaesthetic agents and drugs, as well as to notify hospital staff to send in the next patient. Therefore, RSD plays an important role in improved patient care and minimising surgical theatre costs via efficient scheduling. In endoscopic pituitary surgery, it is uniquely challenging due to variable workflow sequences with a selection of optional steps contributing to high variability in surgery duration. This article presents PitRSDNet for predicting RSD during pituitary surgery, a spatio‐temporal neural network model that learns from historical data focusing on workflow sequences. PitRSDNet integrates workflow knowledge into RSD prediction in two forms: (1) multi‐task learning for concurrently predicting step and RSD; and (2) incorporating prior steps as context in temporal learning and inference. PitRSDNet is trained and evaluated on a new endoscopic pituitary surgery dataset with 88 videos to show competitive performance improvements over previous statistical and machine learning methods. The findings also highlight how PitRSDNet improves RSD precision on outlier cases utilising the knowledge of prior steps.

Autoencoder-Based Neural Network Model for Anomaly Detection in Wireless Body Area Networks

In medical healthcare services, Wireless Body Area Networks (WBANs) are enabler tools for tracking healthcare conditions by monitoring some critical vital signs of the human body. Healthcare providers and consultants use such collected data to assess the status of patients in intensive care units (ICU) at hospitals or elderly care facilities. However, the collected data are subject to anomalies caused by faulty sensor readings, malicious attacks, or severe health degradation situations that healthcare professionals should investigate further. As a result, anomaly detection plays a crucial role in maintaining data quality across various real-world applications, including healthcare, where it is vital for the early detection of abnormal health conditions. Numerous techniques for anomaly detection have been proposed in the literature, employing methods like statistical analysis and machine learning to identify anomalies in WBANs. However, the lack of normal datasets makes training supervised machine learning models difficult, highlighting the need for unsupervised approaches. In this paper, a novel, efficient, and effective unsupervised anomaly detection model for WBANs is developed using the autoencoder convolutional neural network (CNN) technique. Due to their ability to reconstruct data in a completely unsupervised manner using reconstruction error, autoencoders hold great potential. Real-world physiological data from the PhysioNet dataset evaluated the suggested model’s performance. The experimental findings demonstrate the model’s efficacy, which provides high detection accuracy, as reported F1-Score is 0.96 with a batch size of 256 along with a mean squared logarithmic error (MSLE) below 0.002. Compared to existing unsupervised models, the proposed model outperforms them in effectiveness and efficiency.

Predictive modeling of river blockage severity from debris flows: Integrating statistical and machine learning approaches with insights from Sichuan Province, China

River blockages caused by debris flows pose serious threats to the environment and infrastructure. This study introduces the river blockage index (RBI) as a key measure of river blockage severity. We collected data from 60 debris flow events to build a comprehensive dataset. To enhance model robustness and accuracy, we optimized variable selection using multicollinearity analysis and the Akaike information criterion (AIC). Four statistical models were developed, including multiple linear, logarithmic, power, and exponential regressions. We also constructed models based on machine learning algorithms, including random forests and gradient boosting decision trees, and tested them using 5-fold cross-validation. After confirming that the training dataset met linear statistical assumptions, we built robust regression models. We tested the significance of the regression equations and coefficients using F-tests and t-tests. Hyperparameters of the machine learning algorithms were optimized through Bayesian methods. Model performance was evaluated using metrics such as R2, adjusted R2, mean absolute error (MAE), mean relative error (MRE), and root mean square error (RMSE). Results show that the most important factors influencing RBI are catchment area (A) and the discharge ratio between the debris flow and the main river (Q). Among the statistical models, the logarithmic and power models performed best due to their simplicity and efficiency. The random forest model demonstrated the highest predictive accuracy and stability overall. By combining statistical methods with machine learning, we improved prediction accuracy and provided practical guidance for disaster prevention strategies. This approach overcomes the limitations of numerical simulations and experimental studies, offering a more flexible and efficient method for RBI prediction. Future work will extend these findings to other geological settings to further enhance model adaptability and performance.

Robust Bayesian causal estimation for causal inference in medical diagnosis

Causal effect estimation is a critical task in statistical learning that aims to find the causal effect on subjects by identifying causal links between a number of predictor (or, explanatory) variables and the outcome of a treatment. In a regressional framework, we assign a treatment and outcome model to estimate the average causal effect. Additionally, for high dimensional regression problems, variable selection methods are also used to find a subset of predictor variables that maximises the predictive performance of the underlying model for better estimation of the causal effect. In this paper, we propose a different approach. We focus on the variable selection aspects of high dimensional causal estimation problem. We suggest a cautious Bayesian group LASSO (least absolute shrinkage and selection operator) framework for variable selection using prior sensitivity analysis. We argue that in some cases, abstaining from selecting (or, rejecting) a predictor is beneficial and we should gather more information to obtain a more decisive result. We also show that for problems with very limited information, expert elicited variable selection can give us a more stable causal effect estimation as it avoids overfitting. Lastly, we carry a comparative study with synthetic dataset and show the applicability of our method in real-life situations.

DySurv: dynamic deep learning model for survival analysis with conditional variational inference.

Machine learning applications for longitudinal electronic health records often forecast the risk of events at fixed time points, whereas survival analysis achieves dynamic risk prediction by estimating time-to-event distributions. Here, we propose a novel conditional variational autoencoder-based method, DySurv, which uses a combination of static and longitudinal measurements from electronic health records to estimate the individual risk of death dynamically. DySurv directly estimates the cumulative risk incidence function without making any parametric assumptions on the underlying stochastic process of the time-to-event. We evaluate DySurv on 6 time-to-event benchmark datasets in healthcare, as well as 2 real-world intensive care unit (ICU) electronic health records (EHR) datasets extracted from the eICU Collaborative Research (eICU) and the Medical Information Mart for Intensive Care database (MIMIC-IV). DySurv outperforms other existing statistical and deep learning approaches to time-to-event analysis across concordance and other metrics. It achieves time-dependent concordance of over 60% in the eICU case. It is also over 12% more accurate and 22% more sensitive than in-use ICU scores like Acute Physiology and Chronic Health Evaluation (APACHE) and Sequential Organ Failure Assessment (SOFA) scores. The predictive capacity of DySurv is consistent and the survival estimates remain disentangled across different datasets. Our interdisciplinary framework successfully incorporates deep learning, survival analysis, and intensive care to create a novel method for time-to-event prediction from longitudinal health records. We test our method on several held-out test sets from a variety of healthcare datasets and compare it to existing in-use clinical risk scoring benchmarks. While our method leverages non-parametric extensions to deep learning-guided estimations of the survival distribution, further deep learning paradigms could be explored.

Interbrain neural correlates of self and other integration in joint statistical learning

While statistical learning is often studied individually, its collective representation through self-other integration remains unclear. This study examines dynamic self-other integration and its multi-brain mechanism using simultaneous recordings from dyads. Participants (N = 112) each repeatedly responded to half of a fixed stimulus sequence with either an active partner (joint context) or a passive observer (baseline context). Significant individual statistical learning was evident in the joint context, characterized by decreased reaction time (RT) and intra-brain neural responses, followed by a quadratic trend (i.e., first increasing and then decreasing) upon insertion of an interference sequence. More importantly, Brain-to-Brain Coupling (BtBC) in the theta band also showed learning and modulation-related trends, with its slope negatively and positively correlating with the slopes of RT and intra-brain functional connectivity, respectively. These results highlight the dynamic nature of self-other integration in joint statistical learning, with statistical regularities implicitly and spontaneously modulating this process. Notably, the BtBC serves as a key neural correlate underlying the dynamics of self-other integration.

Integrative multi-environmental genomic prediction in apple

Abstract Genomic prediction for multiple environments can aid the selection of genotypes suited to specific soil and climate conditions. Methodological advances allow effective integration of phenotypic, genomic (additive, non-additive), and large-scale environmental (enviromic) data into multi-environmental genomic prediction models. These models can also account for genotype-by-environment interaction, utilize alternative relationship matrices (kernels), or substitute statistical approaches with deep learning. However, the application of multi-environmental genomic prediction in apple remained limited, likely due to the challenge of building multi-environmental datasets and structurally complex models. Here, we applied efficient statistical and deep learning models for multi-environmental genomic prediction of eleven apple traits with contrasting genetic architectures by integrating genomic- and enviromic-based model components. Incorporating genotype-by-environment interaction effects into statistical models improved predictive ability by up to 0.08 for nine traits compared to the benchmark model. This outcome, based on Gaussian and Deep kernels, shows these alternatives can effectively substitute the standard G-BLUP. Including non-additive and enviromic-based effects resulted in a predictive ability very similar to the benchmark model. The deep learning approach achieved the highest predictive ability for three traits with oligogenic genetic architectures, outperforming the benchmark by up to 0.10. Our results demonstrate that the tested statistical models capture genotype-by-environment interactions particularly well, and the deep learning models efficiently integrate data from diverse sources. This study will foster the adoption of multi-environmental genomic prediction to select apple cultivars adapted to diverse environmental conditions, providing an opportunity to address climate change impacts.