Large Number Of Predictors Research Articles

Prediction models for children/adolescents with obesity/overweight: A systematic review and meta-analysis

The incidence of obesity and overweight in children and adolescents is increasing worldwide and becomes a global health concern. This study aims to evaluate the accuracy of available prediction models in early identification of obesity and overweight in general children or adolescents and identify predictive factors for the models, thus provide a reference for subsequent development of risk prediction tools for obesity and overweight in children or adolescents. Related publications were obtained from several databases such as PubMed, Embase, Cochrane Library, and Web of Science from their inception to September 18th, 2022. The novel Prediction Model Risk of Bias Assessment Tool (PROBAST) was employed to assess the bias risk of the included studies. R4.2.0 and Stata15.1 softwares were used to conduct meta-analysis. This study involved 45 cross-sectional and/or prospective studies with 126 models. Meta-analyses showed that the overall pooled index of concordance (c-index) of prediction models for children/adolescents with obesity and overweight in the training set was 0.769 (95% CI 0.754–0.785) and 0.835(95% CI 0.792–0.879), respectively. Additionally, a large number of predictors were found to be related to children's lifestyles, such as sleep duration, sleep quality, and eating speed. In conclusions, prediction models can be employed to predict obesity/overweight in children and adolescents. Most predictors are controllable factors and are associated with lifestyle. Therefore, the prediction model serves as an excellent tool to formulate effective strategies for combating obesity/overweight in pediatric patients.

GTWS-MLrec: global terrestrial water storage reconstruction by machine learning from 1940 to present

Abstract. Terrestrial water storage (TWS) includes all forms of water stored on and below the land surface, and is a key determinant of global water and energy budgets. However, TWS data from measurements by the Gravity Recovery and Climate Experiment (GRACE) satellite mission are only available from 2002, limiting global and regional understanding of the long-term trends and variabilities in the terrestrial water cycle under climate change. This study presents long-term (i.e., 1940–2022) and relatively high-resolution (i.e., 0.25∘) monthly time series of TWS anomalies over the global land surface. The reconstruction is achieved by using a set of machine learning models with a large number of predictors, including climatic and hydrological variables, land use/land cover data, and vegetation indicators (e.g., leaf area index). The outcome, machine-learning-reconstructed TWS estimates (i.e., GTWS-MLrec), fits well with the GRACE/GRACE-FO measurements, showing high correlation coefficients and low biases in the GRACE era. We also evaluate GTWS-MLrec with other independent products such as the land–ocean mass budget, atmospheric and terrestrial water budget in 341 large river basins, and streamflow measurements at 10 168 gauges. The results show that our proposed GTWS-MLrec performs overall as well as, or is more reliable than, previous TWS datasets. Moreover, our reconstructions successfully reproduce the consequences of climate variability such as strong El Niño events. The GTWS-MLrec dataset consists of three reconstructions based on (a) mascons of the Jet Propulsion Laboratory of the California Institute of Technology, the Center for Space Research at the University of Texas at Austin, and the Goddard Space Flight Center of NASA; (b) three detrended and de-seasonalized reconstructions; and (c) six global average TWS series over land areas, both with and without Greenland and Antarctica. Along with its extensive attributes, GTWS_MLrec can support a wide range of geoscience applications such as better understanding the global water budget, constraining and evaluating hydrological models, climate-carbon coupling, and water resources management. GTWS-MLrec is available on Zenodo through https://doi.org/10.5281/zenodo.10040927 (Yin, 2023).

Forecasting tourist flows in the COVID‐19 era using nonparametric mixed‐frequency VARs

AbstractIt is widely recognized that the tourism industry is susceptible to crisis or natural disaster. Although some literature has studied the consequences of the crisis and disaster, there remains a lack of study on the effect of COVID‐19. Against this background, this paper investigates the tourist flow forecasting by adopting an advanced nonparametric mixed‐frequency vector autoregressions model using Bayesian additive regression trees. This is particularly suitable for forecasting the presence of extreme observations, for example, the COVID‐19 pandemic. We investigate tourism demand forecasting using a large number of predictors, including industrial production index, CPI, exchange rate, economic policy uncertainty, Google trends index, and COVID‐19 infection rate. The data used for this study relate to tourist flows in Chinese Hong Kong, Japan, and South Korea. Empirical study demonstrates that this novel model significantly outperforms the traditional mixed‐frequency vector autoregressions model to quarterly tourist flow forecasting. Therefore, this model can significantly enhance tourism forecast accuracy in the face of extreme events. This study contributes to the literature on tourism forecasting and provides policymakers with policy implications.

Message similarity as a proxy to repetitive thinking: Associations with non-suicidal self-injury and suicidal ideation on social media

Detecting signs of rumination (specifically negative repetitive thinking) in social media data could be beneficial to predicting mental health, as rumination is associated with mental health problems. The objective of this study is to evaluate message similarity as a proxy measure of repetitive thinking (a core component of rumination), using data from Vent — a social media platform dedicated to emotion sharing. Our analyses suggest that automatic assessment of message similarity aligns with human judgements. Furthermore, message similarity is positively associated with signs of suicidal ideation/non-suicidal self-injury, and this association is stronger in users who post predominantly negative content, in line with observations from the literature. This effect has a relatively small magnitude and may be less visible in machine learning models with a large number of predictors. Our data suggests that message similarity may be a useful way of capturing repetitive content, which can potentially be used as a stepping stone to automatic rumination detection in social media data.

Simplifying Complexity: Linearization Method for Partial Least Squares Regression

This research investigates Romera’s local linearization approach as a variance prediction method in partial least squares (PLS) regression. By addressing limitations in the original PLS regression formula, the local linearization approach aims to improve accuracy and stability in variance predictions. Extensive simulations are conducted to assess the method's performance, demonstrating its superiority over traditional algebraic methods and showcasing its computational advantages, particularly with a large number of predictors. Additionally, the study introduces a novel computational technique utilizing bootstrap parameters, enhancing computational stability and robustness. Overall, the research provides valuable insights into the local linearization approach's effectiveness, guiding researchers and practitioners in selecting more reliable and efficient regression modeling techniques.

Improving prediction of COVID-19 mortality using machine learning in the Spanish SEMI-COVID-19 registry.

COVID-19 is responsible for high mortality, but robust machine learning-based predictors of mortality are lacking. To generate a model for predicting mortality in patients hospitalized with COVID-19 using Gradient Boosting Decision Trees (GBDT). The Spanish SEMI-COVID-19 registry includes 24,514 pseudo-anonymized cases of patients hospitalized with COVID-19 from 1 February 2020 to 5 December 2021. This registry was used as a GBDT machine learning model, employing the CatBoost and BorutaShap classifier to select the most relevant indicators and generate a mortality prediction model by risk level, ranging from 0 to 1. The model was validated by separating patients according to admission date, using the period 1 February to 31 December 2020 (first and second waves, pre-vaccination period) for training, and 1 January to 30 November 2021 (vaccination period) for the test group. An ensemble of ten models with different random seeds was constructed, separating 80% of the patients for training and 20% from the end of the training period for cross-validation. The area under the receiver operating characteristics curve (AUC) was used as a performance metric. Clinical and laboratory data from 23,983 patients were analyzed. CatBoost mortality prediction models achieved an AUC performance of 84.76 (standard deviation 0.45) for patients in the test group (potentially vaccinated patients not included in model training) using 16 features. The performance of the 16-parameter GBDT model for predicting COVID-19 hospital mortality, although requiring a relatively large number of predictors, shows a high predictive capacity.

Reconstruction of Meteorological Records by Methods Based on Dimension Reduction of the Predictor Dataset

The reconstruction or prediction of meteorological records through the Analog Ensemble (AnEn) method is very efficient when the number of predictor time series is small. Thus, in order to take advantage of the richness and diversity of information contained in a large number of predictors, it is necessary to reduce their dimensions. This study presents methods to accomplish such reduction, allowing the use of a high number of predictor variables. In particular, the techniques of Principal Component Analysis (PCA) and Partial Least Squares (PLS) are used to reduce the dimension of the predictor dataset without loss of essential information. The combination of the AnEn and PLS techniques results in a very efficient hybrid method (PLSAnEn) for reconstructing or forecasting unstable meteorological variables, such as wind speed. This hybrid method is computationally demanding but its performance can be improved via parallelization or the introduction of variants in which all possible analogs are previously clustered. The multivariate linear regression methods used on the new variables resulting from the PCA or PLS techniques also proved to be efficient, especially for the prediction of meteorological variables without local oscillations, such as the pressure.

Improvement of satellite-derived surface solar irradiance estimations using spatio-temporal extrapolation with statistical learning

Estimations of solar surface irradiance (SSI) derived from meteorological satellites are widely used by various actors in the solar industry. However, even state-of-the-art empirical and physical SSI retrieval models exhibit significant errors; the estimations provided by these models are thus traditionally corrected using ground-based measurements of SSI as references. The literature is rich with such correction methods, often called adaptation techniques. Most of the proposed models, however, are local or site-specific, i.e., they do not extrapolate the correction in space and are only applicable to the location of the ground-based measurements.In this work, we propose a novel global adaptation technique, that can extrapolate the correction in both space and time. To that end, we leverage (1) a dense network of measurement stations across France, (2) a relatively large number of predictors, and (3) a non-linear, sophisticated regression algorithm, the Extreme Gradient Boosting. The model is applied to the HelioClim3 database; its performance is benchmarked against raw HelioClim3 estimations, and alternative, simpler adaptation techniques.Our analysis shows that this global model significantly improves satellite-derived SSI estimations from the HelioClim3 database, even when the evaluation is carried out on measurement stations that were not part of the training set of the algorithm. Our proposed model also outperforms all tested alternative global adaptation techniques.These results suggest that global adaptation techniques leveraging advanced machine learning and high dimensionality have the potential to significantly improve satellite-derived SSI estimations, notably more than traditional adaptation approaches. There is certainly room for improvement, but the development of such techniques is a promising research topic.

Separating the wheat from the chaff: Bayesian regularization in dynamic social networks

In recent years there has been an increasing interest in the use of relational event models for dynamic social network analysis. The basis of these models is the concept of an “event”, defined as a triplet of time, sender, and receiver of some social interaction. The key question that relational event models aim to answer is what drives the pattern of social interactions among actors. Researchers often consider a very large number of predictors in their studies (including exogenous effects, endogenous network effects, and interaction effects). However, employing an excessive number of effects may lead to overfitting and inflated Type-I error rates. Moreover, the fitted model can easily become overly complex and the implied social interaction behavior difficult to interpret. A potential solution to this problem is to apply Bayesian regularization using shrinkage priors to recognize which effects are truly nonzero (the “wheat”) and which effects can be considered as (largely) irrelevant (the “chaff”). In this paper, we propose Bayesian regularization methods for relational event models using four different priors for both an actor and a dyad relational event model: a flat prior model with no shrinkage, a ridge estimator with a normal prior, a Bayesian lasso with a Laplace prior, and a horseshoe prior. We apply these regularization methods in three different empirical applications. The results reveal that Bayesian regularization can be used to separate the wheat from the chaff in models with a large number of effects by yielding considerably fewer significant effects, resulting in a more parsimonious description of the social interaction behavior between actors in dynamic social networks, without sacrificing predictive performance.

A large-scale machine learning study of sociodemographic factors contributing to COVID-19 severity.

Understanding sociodemographic factors behind COVID-19 severity relates to significant methodological difficulties, such as differences in testing policies and epidemics phase, as well as a large number of predictors that can potentially contribute to severity. To account for these difficulties, we assemble 115 predictors for more than 3,000 US counties and employ a well-defined COVID-19 severity measure derived from epidemiological dynamics modeling. We then use a number of advanced feature selection techniques from machine learning to determine which of these predictors significantly impact the disease severity. We obtain a surprisingly simple result, where only two variables are clearly and robustly selected-population density and proportion of African Americans. Possible causes behind this result are discussed. We argue that the approach may be useful whenever significant determinants of disease progression over diverse geographic regions should be selected from a large number of potentially important factors.

Machine Learning Models and Intra-Daily Market Information for the Prediction of Italian Electricity Prices

In this paper we assess how intra-day electricity prices can improve the prediction of zonal day-ahead wholesale electricity prices in Italy. We consider linear autoregressive models with exogenous variables (ARX) with and without interactions among predictors, and non-parametric models taken from the machine learning literature. In particular, we implement Random Forests and support vector machines, which should automatically capture the relevant interactions among predictors. Given the large number of predictors, ARX models are also estimated using LASSO regularization, which improves predictions when regressors are many and selects the important variables. In addition to zonal intra-day prices, among the predictors we include also the official demand forecasts and wind generation expectations. Our results show that the prediction performance of the simple ARX model is mostly superior to those of machine learning models. The analysis of the relevance of exogenous variables, using variable importance measures, reveals that intra-day market information successfully contributes to the forecasting performance, although the impact differs among the estimated models.

Evaluation of statistical climate reconstruction methods based on pseudoproxy experiments using linear and machine-learning methods

Abstract. Three different climate field reconstruction (CFR) methods are employed to reconstruct spatially resolved North Atlantic–European (NAE) and Northern Hemisphere (NH) summer temperatures over the past millennium from proxy records. These are tested in the framework of pseudoproxy experiments derived from two climate simulations with comprehensive Earth system models. Two of these methods are traditional multivariate linear methods (principal component regression, PCR, and canonical correlation analysis, CCA), whereas the third method (bidirectional long short-term memory neural network, Bi-LSTM) belongs to the category of machine-learning methods. In contrast to PCR and CCA, Bi-LSTM does not need to assume a linear and temporally stable relationship between the underlying proxy network and the target climate field. In addition, Bi-LSTM naturally incorporates information about the serial correlation of the time series. Our working hypothesis is that the Bi-LSTM method will achieve a better reconstruction of the amplitude of past temperature variability. In all tests, the calibration period was set to the observational period, while the validation period was set to the pre-industrial centuries. All three methods tested herein achieve reasonable reconstruction performance on both spatial and temporal scales, with the exception of an overestimation of the interannual variance by PCR, which may be due to overfitting resulting from the rather short length of the calibration period and the large number of predictors. Generally, the reconstruction skill is higher in regions with denser proxy coverage, but it is also reasonably high in proxy-free areas due to climate teleconnections. All three CFR methodologies generally tend to more strongly underestimate the variability of spatially averaged temperature indices as more noise is introduced into the pseudoproxies. The Bi-LSTM method tested in our experiments using a limited calibration dataset shows relatively worse reconstruction skills compared to PCR and CCA, and therefore our working hypothesis that a more complex machine-learning method would provide better reconstructions for temperature fields was not confirmed. In this particular application with pseudoproxies, the implied link between proxies and climate fields is probably close to linear. However, a certain degree of reconstruction performance achieved by the nonlinear LSTM method shows that skill can be achieved even when using small samples with limited datasets, which indicates that Bi-LSTM can be a tool for exploring the suitability of nonlinear CFRs, especially in small data regimes.

Robust Variable Selection Based on Relaxed Lad Lasso

Least absolute deviation is proposed as a robust estimator to solve the problem when the error has an asymmetric heavy-tailed distribution or outliers. In order to be insensitive to the above situation and select the truly important variables from a large number of predictors in the linear regression, this paper introduces a two-stage variable selection method named relaxed lad lasso, which enables the model to obtain robust sparse solutions in the presence of outliers or heavy-tailed errors by combining least absolute deviation with relaxed lasso. Compared with lasso, this method is not only immune to the rapid growth of noise variables but also maintains a better convergence rate, which is Opn−1/2. In addition, we prove that the relaxed lad lasso estimator has the property of consistency at large samples; that is, the model selects the number of important variables with a high probability of convergence to one. Through the simulation and empirical results, we further verify the outstanding performance of relaxed lad lasso in terms of prediction accuracy and the correct selection of informative variables under the heavy-tailed distribution.

Development and validation of a prediction model for early mortality after transcatheter aortic valve implantation (TAVI) based on the Netherlands Heart Registration (NHR): The TAVI-NHR risk model.

The currently available mortality prediction models (MPM) have suboptimal performance when predicting early mortality (30-days) following transcatheter aortic valve implantation (TAVI) on various external populations. We developed and validated a new TAVI-MPM based on a large number of predictors with recent data from a national heart registry. We included all TAVI-patients treated in the Netherlands between 2013 and 2018, from the Netherlands Heart Registration. We used logistic-regression analysis based on the Akaike Information Criterion for variable selection. We multiply imputed missing values, but excluded variables with >30% missing values. For internal validation, we used ten-fold cross-validation. For temporal (prospective) validation, we used the 2018-data set for testing. We assessed discrimination by the c-statistic, predicted probability accuracy by the Brier score, and calibration by calibration graphs, and calibration-intercept and calibration slope. We compared our new model to the updated ACC-TAVI and IRRMA MPMs on our population. We included 9144 TAVI-patients. The observed early mortality was 4.0%. The final MPM had 10 variables, including: critical-preoperative state, procedure-acuteness, body surface area, serum creatinine, and diabetes-mellitus status. The median c-statistic was 0.69 (interquartile range [IQR] 0.646-0.75). The median Brier score was 0.038 (IQR 0.038-0.040). No signs of miscalibration were observed. The c-statistic's temporal-validation was 0.71 (95% confidence intervals 0.64-0.78). Our model outperformed the updated currently available MPMs ACC-TAVI and IRRMA (p value < 0.05). The new TAVI-model used additional variables and showed fair discrimination and good calibration. It outperformed the updated currently available TAVI-models on our population. The model's good calibration benefits preprocedural risk-assessment and patient counseling.

The Bank of Korea watch

Traders closely monitor the Bank of Korea (BOK) base-rate decisions since the short rate is the primary factor in bond and currency valuations. The Survey of Professional Forecasters(SPF) has been widely used and is considered the most reliable BOK base-rate decision forecast. In this study, we investigate whether the SPF’s prediction ability can be further improved. To this end, we use a dynamic multinomial ordered probit prediction model of the BOK base rate with a large number of predictors and apply a Bayesian variable selection algorithm. Through an empirical exercise, we show that our approach substantially outperforms the SPF in terms of out-of-sample prediction. The key predictors found are SPF, short-term bond yields, lagged base rate, federal funds rate, and inflation expectation survey data. Furthermore, allowing the prediction ability to change over time is essential for improving predictive accuracy.

Forecasting stock market volatility with a large number of predictors: New evidence from the MS-MIDAS-LASSO model.

This paper explores the effectiveness of predictors, including nine economic policy uncertainty indicators, four market sentiment indicators and two financial stress indices, in predicting the realized volatility of the S&P 500 index. We employ the MIDAS-RV framework and construct the MIDAS-LASSO model and its regime switching extension (namely, MS-MIDAS-LASSO). First, among all considered predictors, the economic policy uncertainty indices (especially the equity market volatility index) and the CBOE volatility index are the most noteworthy predictors. Although the CBOE volatility index has the best predictive ability for stock market volatility, its predictive ability has weakened during the COVID-19 epidemic, and the equity market volatility index is best during this period. Second, the MS-MIDAS-LASSO model has the best predictive performance compared to other competing models. The superior forecasting performance of this model is robust, even when distinguishing between high- and low-volatility periods. Finally, the prediction accuracy of the MS-MIDAS-LASSO model even outperforms the traditional LASSO strategy and its regime switching extension. Furthermore, the superior predictive performance of this model has not changed with the outbreak of the COVID-19 epidemic.

A generative approach to modeling data with quantitative and qualitative responses

In many scientific areas, data with mixed quantitative and qualitative (QQ) responses are commonly encountered with a large number of predictors. By exploring the association between QQ responses, existing approaches often consider a joint model of QQ responses given the predictor variables. However, the dependency among predictive variables also provides useful information for fitting QQ responses. Hence in this work, we propose a novel generative approach to jointly model the QQ responses by incorporating the dependency information of predictors. The proposed method is computationally efficient and provides accurate parameter estimation under a penalized likelihood framework. Moreover, because of the generative approach framework, the asymptotically theoretical results of the proposed method are established under some regularity conditions. The performance of the proposed method is examined through simulations and real case studies in material science and genetics.

Forecasting with Partial Least Squares When a Large Number of Predictors Are Available

Forecasting with Partial Least Squares When a Large Number of Predictors Are Available

Fast and Flexible Bayesian Inference in Time-varying Parameter Regression Models

In this article, we write the time-varying parameter (TVP) regression model involving K explanatory variables and T observations as a constant coefficient regression model with KT explanatory variables. In contrast with much of the existing literature which assumes coefficients to evolve according to a random walk, a hierarchical mixture model on the TVPs is introduced. The resulting model closely mimics a random coefficients specification which groups the TVPs into several regimes. These flexible mixtures allow for TVPs that feature a small, moderate or large number of structural breaks. We develop computationally efficient Bayesian econometric methods based on the singular value decomposition of the KT regressors. In artificial data, we find our methods to be accurate and much faster than standard approaches in terms of computation time. In an empirical exercise involving inflation forecasting using a large number of predictors, we find our models to forecast better than alternative approaches and document different patterns of parameter change than are found with approaches which assume random walk evolution of parameters.

Semiparametric Model Averaging Prediction for Lifetime Data via Hazards Regression

AbstractForecasting survival risks for time-to-event data is an essential task in clinical research. Practitioners often rely on well-structured statistical models to make predictions for patient survival outcomes. The nonparametric proportional hazards model, as an extension of the Cox proportional hazards model, involves an additive nonlinear combination of covariate effects for hazards regression and may be more flexible. When there are a large number of predictors, nonparametric smoothing for different variables cannot be simultaneously optimal using the conventional fitting program. To address such a limitation and still maintain the nonparametric flavour, we present a novel model averaging method to produce model-based prediction for survival outcome and our method automatically offers optimal smoothing for individual nonparametric functional estimation. The proposed semiparametric model averaging prediction (SMAP) method basically approximates the underlying unstructured nonparametric regression function by a weighted sum of low-dimensional nonparametric submodels. The weights are obtained from maximizing the partial likelihood constructed for the aggregated model. Theoretical properties are discussed for the estimated model weights. Simulation studies are conducted to examine the performance of SMAP under various evaluation criteria. Two real examples from genetic research studies motivated our work and are analysed by the proposed SMAP to produce new scientific findings.