Statistical Learning Methods Research Articles

Developing an Early Warning System for Financial Networks: An Explainable Machine Learning Approach.

Identifying the influential variables that provide early warning of financial network instability is challenging, in part due to the complexity of the system, uncertainty of a failure, and nonlinear, time-varying relationships between network participants. In this study, we introduce a novel methodology to select variables that, from a data-driven and statistical modeling perspective, represent these relationships and may indicate that the financial network is trending toward instability. We introduce a novel variable selection methodology that leverages Shapley values and modified Borda counts, in combination with statistical and machine learning methods, to create an explainable linear model to predict relationship value weights between network participants. We validate this new approach with data collected from the March 2023 Silicon Valley Bank Failure. The models produced using this novel method successfully identified the instability trend using only 14 input variables out of a possible 3160. The use of parsimonious linear models developed by this method has the potential to identify key financial stability indicators while also increasing the transparency of this complex system.

Nonlinear Ensemble Deep Learning Model for Energy Consumption Prediction with Bayesian Optimization

Accurate prediction of electric energy consumption is crucial for efficient load dispatching, energy utilization, and grid operation. Traditional statistical and classical machine learning methods struggle with the nonlinear nature of energy consumption data, often leading to higher prediction errors. Additionally, deep learning models using a single approach face challenges such as convergence to local minima and poor generalization. This paper proposes a nonlinear ensemble deep learning model for residential energy consumption prediction, incorporating Bayesian optimization for hyperparameter tuning. The model combines Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), and 1D Convolutional Neural Networks (1D-CNN), leveraging their powerful nonlinear feature learning capabilities. A k-means clustering approach is used to preprocess and reduce variability in the data, enhancing the ensemble model's performance. The ensemble model was tested on real energy consumption data from two districts in Addis Ababa, showing significant improvements in prediction accuracy with lower MAE, RMSE, and MAPE values compared to single models and un-clustered data. The integration of clustering and Bayesian optimization further enhanced model generalizability and minimized overfitting, demonstrating the effectiveness of a nonlinear approach in capturing complex energy consumption patterns.

A multi-task deep learning model based on comprehensive feature integration and self-attention mechanism for predicting response to anti-PD1/PD-L1

BackgroundImmune checkpoint inhibitor (ICI) has been widely used in the treatment of advanced cancers, but predicting their efficacy remains challenging. Traditional biomarkers are numerous but exhibit heterogeneity within populations. For comprehensively utilizing the ICI-related biomarkers, we aim to conduct multidimensional feature selection and deep learning model construction. MethodsWe used statistical and machine learning methods to map features of different levels to next-generation sequencing gene expression. We integrated genes from different sources into the feature input of a deep learning model, by means of self-attention mechanism. ResultsWe performed feature selection at the single-cell sequencing level, PD-L1 (CD274) analysis level, tumor mutational burden (TMB)/mismatch repair (MMR) level, and somatic copy number alteration (SCNA) level, obtaining 96 feature genes. Based on the pan-cancer dataset, we trained a multi-task deep learning model. We tested the model in the bladder urothelial carcinoma testing set 1 (AUC = 0.62, n = 298), bladder urothelial carcinoma testing set 2 (AUC = 0.66, n = 89), non-small cell lung cancer testing set (AUC = 0.85, n = 27), and skin cutaneous melanoma testing set (AUC = 0.71, n = 27). ConclusionOur study demonstrates the potential of the deep learning model for integrating multidimensional features in predicting the outcome of ICI. Our study also provides a potential methodological case for medical scenarios requiring the integration of multiple levels of features.

Estimating the spatiotemporal distribution of PM2.5 concentrations in Tianjin during the Chinese Spring Festival: Impact of fireworks ban

Settingoff fireworks during the Spring Festival (SF) is a traditional practice in China. However, because of its environmental impact, the Chinese government has banned this practice completely. Existing evaluations of the effectiveness of firework prohibition policies (FPPs) lack spatiotemporal perspectives, making it difficult to comprehensively assess their effects on air quality. Consequently, this study used remote sensing technology based on aerosol optical depth and multiple variables, compared nine statistical learning methods, and selected the optimal model, transformer, to estimate daily spatiotemporal continuous PM2.5 concentration datasets for Tianjin from 2016 to 2020. The overall model accuracy reached a root mean square error of 15.30 μg/m³, a mean absolute error of 9.55 μg/m³, a mean absolute percentage error of 21.07%, and an R2 of 0.88. Subsequently, we analysed the variations in PM2.5 concentrations from three time dimensions–the entire year, winter, and SF periods–to exclude the impact of interannual variations on the experimental results. Additionally, we quantitatively estimated firework-specific PM2.5 concentrations based on time-series forecasting. The results showed that during the three years following the implementation of the FPPs, firework-specific PM2.5 concentrations decreased by 52.70%, 49.76%, and 86.90%, respectively, compared to the year before the implementation of the FPPs. Spatially, the central urban area and industrial zones are more affected by FPPs than the suburbs. However, daily variations of PM2.5 concentrations during the SF showed that high concentrations of PM2.5 produced in a short period will return to normal rapidly and will not cause lasting effects. Therefore, the management of fireworks needs to consider both environmental protection and the public's emotional attachment to traditional customs, rather than simply imposing a blanket ban on fireworks. We advocate improving firework policies in four aspects–production, sales, supervision, and control–to promote sustainable development of the ecological environment and human society.

Robust discriminant analysis

AbstractDiscriminant analysis (DA) is one of the most popular methods for classification due to its conceptual simplicity, low computational cost, and often solid performance. In its standard form, DA uses the arithmetic mean and sample covariance matrix to estimate the center and scatter of each class. We discuss and illustrate how this makes standard DA very sensitive to suspicious data points, such as outliers and mislabeled cases. We then present an overview of techniques for robust DA, which are more reliable in the presence of deviating cases. In particular, we review DA based on robust estimates of location and scatter, along with graphical diagnostic tools for visualizing the results of DA.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Robust Methods Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification

Machine learning prediction and explanatory models of serious infections in patients with rheumatoid arthritis treated with tofacitinib

BackgroundPatients with rheumatoid arthritis (RA) have an increased risk of developing serious infections (SIs) vs. individuals without RA; efforts to predict SIs in this patient group are ongoing. We assessed the ability of different machine learning modeling approaches to predict SIs using baseline data from the tofacitinib RA clinical trials program.MethodsThis analysis included data from 19 clinical trials (phase 2, n = 10; phase 3, n = 6; phase 3b/4, n = 3). Patients with RA receiving tofacitinib 5 or 10 mg twice daily (BID) were included in the analysis; patients receiving tofacitinib 11 mg once daily were considered as tofacitinib 5 mg BID. All available patient-level baseline variables were extracted. Statistical and machine learning methods (logistic regression, support vector machines with linear kernel, random forest, extreme gradient boosting trees, and boosted trees) were implemented to assess the association of baseline variables with SI (logistic regression only), and to predict SI using selected baseline variables using 5-fold cross-validation. Missing values were handled individually per prediction model.ResultsA total of 8404 patients with RA treated with tofacitinib were eligible for inclusion (15,310 patient-years of total follow-up) of which 473 patients reported SIs. Amongst other baseline factors, age, previous infection, and corticosteroid use were significantly associated with SI. When applying prediction modeling for SI across data from all studies, the area under the receiver operating characteristic (AUROC) curve ranged from 0.656 to 0.739. AUROC values ranged from 0.599 to 0.730 in data from phase 3 and 3b/4 studies, and from 0.563 to 0.643 in data from ORAL Surveillance only.ConclusionsBaseline factors associated with SIs in the tofacitinib RA clinical trial program were similar to established SI risk factors associated with advanced treatments for RA. Furthermore, while model performance in predicting SI was similar to other published models, this did not meet the threshold for accurate prediction (AUROC > 0.85). Thus, predicting the occurrence of SIs at baseline remains challenging and may be complicated by the changing disease course of RA over time. Inclusion of other patient-associated and healthcare delivery-related factors and harmonization of the duration of studies included in the models may be required to improve prediction.Trial registrationClinicalTrials.gov: NCT00147498; NCT00413660; NCT00550446; NCT00603512; NCT00687193; NCT01164579; NCT00976599; NCT01059864; NCT01359150; NCT02147587; NCT00960440; NCT00847613; NCT00814307; NCT00856544; NCT00853385; NCT01039688; NCT02187055; NCT02831855; NCT02092467.

Predicting Hydrocarbon Strain Energy via a Group Equivalent Machine Learning Approach.

Strain energy is a fundamental measure of the steric and configurational properties of organic molecules. The ability to estimate strain energy through quantum chemical simulations requires at minimum the knowledge of an initial set of nuclear coordinates. In general, such knowledge is not categorically known when screening or generating large numbers of molecule candidates in the context of molecular design. We present a machine learning approach to predict hydrocarbon strain energies using Benson group equivalents. A featurization strategy is crafted by concatenating the molecule group equivalent counts with easily computable molecular fingerprints. The data are obtained from electronic structure calculations we performed on a set of 166 previously synthesized strained hydrocarbons. These data are provided and include gas phase enthalpies of formation and associated optimized atomic coordinates. The strain energy prediction accuracy of several statistical learning methods is evaluated, and their respective merits and limitations are discussed.

Campus Network Traffic Prediction and Anomaly Detection Based on Deep Learning

This paper proposes an intelligent solution for network traffic prediction and anomaly detection in campus networks, addressing the increasingly severe network security challenges. The proposed approach innovatively integrates Convolutional Neural Networks (CNN) and Long Short-Term Memory networks (LSTM) to simultaneously extract local features and capture dynamic temporal dependencies of network traffic, significantly improving prediction accuracy. Based on this, an adaptive threshold anomaly detection algorithm is designed to automatically adjust detection sensitivity according to traffic variations, achieving a better balance between accuracy and recall rates. Additionally, an anomaly visualization scheme is presented, intuitively displaying the spatiotemporal distribution of network anomalies through heatmaps, assisting administrators in decision-making. Large-scale experiments demonstrate that this approach can effectively identify various security threats such as DDoS attacks, scanning probes, and botnets, with an overall detection rate exceeding 90% while maintaining a low false positive rate. Compared to traditional statistical and machine learning methods, the proposed approach exhibits stronger adaptability and generalization capabilities, providing crucial support for building an intelligent, precise, and reliable campus network security protection system. Future work will focus on further improving the real-time performance and robustness of the solution, expanding its application in new network scenarios such as IoT and edge computing.

The impact of the European Union emissions trading system on carbon dioxide emissions: a matrix completion analysis

Despite the negative externalities on the environment and human health, today’s economies still produce excessive carbon dioxide emissions. As a result, governments are trying to shift production and consumption to more sustainable models that reduce the environmental impact of carbon dioxide emissions. The European Union, in particular, has implemented an innovative policy to reduce carbon dioxide emissions by creating a market for emission rights, the emissions trading system. The objective of this paper is to perform a counterfactual analysis to measure the impact of the emissions trading system on the reduction of carbon dioxide emissions. For this purpose, a recently-developed statistical machine learning method called matrix completion with fixed effects estimation is used and compared to traditional econometric techniques. We apply matrix completion with fixed effects estimation to the prediction of missing counterfactual entries of a carbon dioxide emissions matrix whose elements (indexed row-wise by country and column-wise by year) represent emissions without the emissions trading system for country-year pairs. The results obtained, confirmed by robust diagnostic tests, show a significant effect of the emissions trading system on the reduction of carbon dioxide emissions: the majority of European Union countries included in our analysis reduced their total carbon dioxide emissions (associated with selected industries) by about 15.4% during the emissions trading system treatment period 2005–2020, compared to the total carbon dioxide emissions (associated with the same industries) that would have been achieved in the absence of the emissions trading system policy. Finally, several managerial/practical implications of the study are discussed, together with its possible extensions.

A review on advancements in feature selection and feature extraction for high-dimensional NGS data analysis.

Recent advancements in biomedical technologies and the proliferation of high-dimensional Next Generation Sequencing (NGS) datasets have led to significant growth in the bulk and density of data. The NGS high-dimensional data, characterized by a large number of genomics, transcriptomics, proteomics, and metagenomics features relative to the number of biological samples, presents significant challenges for reducing feature dimensionality. The high dimensionality of NGS data poses significant challenges for data analysis, including increased computational burden, potential overfitting, and difficulty in interpreting results. Feature selection and feature extraction are two pivotal techniques employed to address these challenges by reducing the dimensionality of the data, thereby enhancing model performance, interpretability, and computational efficiency. Feature selection and feature extraction can be categorized into statistical and machine learning methods. The present study conducts a comprehensive and comparative review of various statistical, machine learning, and deep learning-based feature selection and extraction techniques specifically tailored for NGS and microarray data interpretation of humankind. A thorough literature search was performed to gather information on these techniques, focusing on array-based and NGS data analysis. Various techniques, including deep learning architectures, machine learning algorithms, and statistical methods, have been explored for microarray, bulk RNA-Seq, and single-cell, single-cell RNA-Seq (scRNA-Seq) technology-based datasets surveyed here. The study provides an overview of these techniques, highlighting their applications, advantages, and limitations in the context of high-dimensional NGS data. This review provides better insights for readers to apply feature selection and feature extraction techniques to enhance the performance of predictive models, uncover underlying biological patterns, and gain deeper insights into massive and complex NGS and microarray data.

Nutrient Stress Symptom Detection in Cucumber Seedlings Using Segmented Regression and a Mask Region-Based Convolutional Neural Network Model

The health monitoring of vegetable and fruit plants, especially during the critical seedling growth stage, is essential to protect them from various environmental stresses and prevent yield loss. Different environmental stresses may cause similar symptoms, making visual inspection alone unreliable and potentially leading to an incorrect diagnosis and delayed corrective actions. This study aimed to address these challenges by proposing a segmented regression model and a Mask R-CNN model for detecting the initiation time and symptoms of nutrient stress in cucumber seedlings within a controlled environment. Nutrient stress was induced by applying two different treatments: an indicative nutrient deficiency with an electrical conductivity (EC) of 0 dSm−1, and excess nutrients with a high-concentration nutrient solution and an EC of 6 dSm−1. Images of the seedlings were collected using an automatic image acquisition system two weeks after germination. The early initiation of nutrient stress was detected using a segmented regression analysis, which analyzed morphological and textural features extracted from the images. For the Mask R-CNN model, 800 seedling images were annotated based on the segmented regression analysis results. Nutrient-stressed seedlings were identified from the initiation day to 4.2 days after treatment application. The Mask R-CNN model, implemented using ResNet-101 for feature extraction, leveraged transfer learning to train the network with a smaller dataset, thereby reducing the processing time. This study identifies the top projected canopy area (TPCA), energy, entropy, and homogeneity as prospective indicators of nutritional deficits in cucumber seedlings. The results from the Mask R-CNN model are promising, with the best-fit image achieving an F1 score of 93.4%, a precision of 93%, and a recall of 94%. These findings demonstrate the effectiveness of the integrated statistical and machine learning (ML) methods for the early and accurate diagnosis of nutrient stress. The use of segmented regression for initial detection, followed by the Mask R-CNN for precise identification, emphasizes the potential of this approach to enhance agricultural practices. By facilitating the early detection and accurate diagnosis of nutrient stress, this approach allows for quicker and more precise treatments, which improve crop health and productivity. Future research could expand this methodology to other crop types and field conditions to enhance image processing techniques, and researchers may also integrate real-time monitoring systems.

Data-driven determination of plant growth stages for improved weather index insurance design

PurposeThis paper aims to improve the delimitation of plant growth stages in the context of weather index insurance design. We propose a data-driven phase division that minimizes estimation errors in the weather-yield relationship and investigate whether it can substitute an expert-based determination of plant growth phases. We combine this procedure with various statistical and machine learning estimation methods and compare their performance.Design/methodology/approachUsing the example of winter barley, we divide the complete growth cycle into four sub-phases based on phenology reports and expert instructions and evaluate all combinations of start and end points of the various growth stages by their estimation errors of the respective yield models. Some of the most commonly used statistical and machine learning methods are employed to model the weather-yield relationship with each selected method we applied.FindingsOur results confirm that the fit of crop-yield models can be improved by disaggregation of the vegetation period. Moreover, we find that the data-driven approach leads to similar division points as the expert-based approach. Regarding the statistical model, in terms of yield model prediction accuracy, Support Vector Machine ranks first and Polynomial Regression last; however, the performance across different methods exhibits only minor differences.Originality/valueThis research addresses the challenge of separating plant growth stages when phenology information is unavailable. Moreover, it evaluates the performance of statistical and machine learning methods in the context of crop yield prediction. The suggested phase-division in conjunction with advanced statistical methods offers promising avenues for improving weather index insurance design.

Financial Crisis Analysis and Prediction in Africa using Machine Learning and Statistical Learning Methods

Financial Crisis Analysis and Prediction in Africa using Machine Learning and Statistical Learning Methods

Association between ultrafine particles and asthma onset in children: comparison of statistical and machine learning methods to predict exposures

Association between ultrafine particles and asthma onset in children: comparison of statistical and machine learning methods to predict exposures

Convolutional Neural Network Processing of Radio Emission for Nuclear Composition Classification of Ultra-High-Energy Cosmic Rays

Ultra-high-energy cosmic rays (UHECRs) are extremely rare energetic particles of ordinary matter in the Universe, traveling astronomical distances before reaching the Earth’s atmosphere. When primary cosmic rays interact with atmospheric nuclei, cascading extensive air showers (EASs) of secondary elementary particles are developed. Radio detectors have proven to be a reliable method for reconstructing the properties of EASs, such as the shower’s axis, its energy, and its maximum (Xmax). This aids in understanding fundamental astrophysical phenomena, like active galactic nuclei and gamma-ray bursts. Concurrently, data science has become indispensable in UHECR research. By applying statistical, computational, and deep learning methods to both real-world and simulated radio data, researchers can extract insights and make predictions. We introduce a convolutional neural network (CNN) architecture designed to classify simulated air shower events as either being generated by protons or by iron nuclei. The classification achieved a stable test error of 10%, with Accuracy and F1 scores of 0.9 and an MCC of 0.8. These metrics indicate strong prediction capability for UHECR’s nuclear composition, based on data that can be gathered by detectors at the world’s largest cosmic rays experiment on Earth, the Pierre Auger Observatory, which includes radio antennas, water Cherenkov detectors, and fluorescence telescopes.

Exploring Latent Constructs through Multimodal Data Analysis

AbstractThis study presents a comprehensive analysis of three types of multimodal data‐response accuracy, response times, and eye‐tracking data‐derived from a computer‐based spatial rotation test. To tackle the complexity of high‐dimensional data analysis challenges, we have developed a methodological framework incorporating various statistical and machine learning methods. The results of our study reveal that hidden state transition probabilities, based on eye‐tracking features, may be contingent on skill mastery estimated from the fluency CDM model. The hidden state trajectory offers additional diagnostic insights into spatial rotation problem‐solving, surpassing the information provided by the fluency CDM alone. Furthermore, the distribution of participants across different hidden states reflects the intricate nature of visualizing objects in each item, adding a nuanced dimension to the characterization of item features. This complements the information obtained from item parameters in the fluency CDM model, which relies on response accuracy and response time. Our findings have the potential to pave the way for the development of new psychometric and statistical models capable of seamlessly integrating various types of multimodal data. This integrated approach promises more meaningful and interpretable results, with implications for advancing the understanding of cognitive processes involved in spatial rotation tests.

Research on information fusion of security analysts’ stock recommendations based on two-dimensional D-S evidence theory

Security analysts play a vital role as an information intermediary in the stock market. Their stock recommendations are important references for investors. The efficiency of investment decision-making could be improved by judging the reliability of stock recommendations based on analyst characteristics and fusing the recommendations. We propose an information fusion method for security analysts’ stock recommendations based on two-dimensional Dempster-Shafer (D-S) evidence theory, which comprehensively considers the external and internal characteristics of analysts. The characteristics of analysts are used to measure the reliability of the stock recommendations and modify the evidence, then the D-S fusion rule is used for evidence fusion. Compared with the forecast results of statistical methods and machine learning methods, the two-dimensional D-S evidence theory model we proposed has a higher forecast accuracy, which effectively improves the information efficiency of the stock market and helps investors to make decisions efficiently and scientifically.

Tensor-based statistical learning methods for diagnosing product quality defects in multistage manufacturing processes

Multistage manufacturing processes with identical stages provide three-dimensional process data in which the first dimension represents the process (control/sensing) variable, the second is the stage, and the third is the measurement/sampling/data acquisition time point. Diagnosing quality faults in such processes often requires the simultaneous identification of crucial process variables and stages associated with product quality anomalies. Most existing diagnosis methods convert 3D data into a 2D matrix, resulting in loss of information and reduced diagnostic accuracy and stability. To address this challenge, we propose a penalized tensor regression model that regresses the product quality index against its 3D process data. For the estimation of high-dimensional regression coefficients with the limited amount of historical data, we apply the CANDECOMP/PARAFAC and Tucker decompositions to the coefficient tensor, which significantly reduces the number of parameters to be estimated. Based on the decompositions, a new regularization term is designed to enable the joint identification of critical process variables and stages. To estimate the parameters, we develop the block coordinate proximal descent algorithm and provide its convergence guarantee. Numerical studies demonstrate that the proposed methods can enhance diagnostic stability and on average improve the diagnostic accuracy by around 20% over existing benchmarks.

The 2023 Mw 6.8 Morocco earthquake induced atmospheric and ionospheric anomalies

Earth observations through Global Navigation Satellite System (GNSS) and Remote Sensing (RS) technologies play a significant role in natural hazard surveillance, particularly in the context of earthquake prediction and detection. This study introduces a distinctive Deep Learning (DL) based approach to identify ionospheric and atmospheric precursors, utilizing data from multiple satellite sources and provides a comprehensive analysis of spatiotemporally varying precursors, contributing to the understanding and monitoring of seismic activity in earthquake-prone regions. In our investigation of the Morocco earthquake on September 08, 2023 (Mw 6.8), we analyzed various precursors including Total Electron Content (TEC), Air Pressure (AP), Relative Humidity (RH), Outgoing Longwave Radiation (OLR), and Air Temperature (AT). Our study aims to identify a synchronized anomalous window of potential earthquake precursors using Standard Deviation (STDEV), Continuous Wavelet Transform (CWT), and Long Short-Term Memory Inputs (LSTM) network. Both statistical and deep learning methods revealed abnormal fluctuations as precursors occurring within 8–9 days before the earthquake near the epicenter. Additionally, we detected geomagnetic anomalies in the ionosphere 6 days prior to and 4 days after the earthquake, coinciding with active geomagnetic storm days. This research underlined the importance of combining multiple earthquake precursors using statistical and deep learning approaches to support the understanding of the Lithosphere-Atmosphere-Ionosphere-Coupling (LAIC) phenomena.

Investigating alignment-free machine learning methods for HIV-1 subtype classification.

Many viruses are organized into taxonomies of subtypes based on their genetic similarities. For human immunodeficiency virus 1 (HIV-1), subtype classification plays a crucial role in infection management. Sequence alignment-based methods for subtype classification are impractical for large datasets because they are costly and time-consuming. Alignment-free methods involve creating numerical representations for genetic sequences and applying statistical or machine learning methods. Despite their high overall accuracy, existing models perform poorly on less common subtypes. Furthermore, there is limited work investigating the impact of sequence vectorization methods, in particular natural language-inspired embedding methods, on HIV-1 subtype classification. We present a comprehensive analysis of sequence vectorization methods across machine learning methods. We report a k-mer-based XGBoost model with a balanced accuracy of 0.84, indicating that it has good overall performance for both common and uncommon HIV-1 subtypes. We also report a Word2Vec-based support vector machine that achieves promising results on precision and balanced accuracy. Our study sheds light on the effect of sequence vectorization methods on HIV-1 subtype classification and suggests that natural language-inspired encoding methods show promise. Our results could help to develop improved HIV-1 subtype classification methods, leading to improved individual patient outcomes, and the development of subtype-specific treatments. Source code is available at https://www.github.com/kwade4/HIV_Subtypes.