Number Of Candidate Variables Research Articles

The predictability of cross-sectional returns in high frequency

Stock return forecast is of great importance to trading, hedging, and portfolio management. In this article, we apply LASSO and random forest to make rolling one-minute-ahead return forecasts of Dow Jones stocks, using the cross-section of lagged returns of S&P 500 components as candidate predictors. Although the number of candidate variables is large, the negative out-of-sample R2 suggests that the predictions from LASSO and random forest give larger mean-squared error than the historical average. So, there is no evidence of predictability in the cross-sectional returns of large stocks in high frequency. The predictability presented by Chinco et al. (2019) might be due to the interaction between large and small stocks.

Machine learning methods for \u201cwicked\u201d problems: exploring the complex drivers of modern slavery

Forty million people are estimated to be in some form of modern slavery across the globe. Understanding the factors that make any particular individual or geographical region vulnerable to such abuse is essential for the development of effective interventions and policy. Efforts to isolate and assess the importance of individual drivers statistically are impeded by two key challenges: data scarcity and high dimensionality, typical of many “wicked problems”. The hidden nature of modern slavery restricts available data points; and the large number of candidate variables that are potentially predictive of slavery inflate the feature space exponentially. The result is a “small n, large p” setting, where overfitting and significant inter-correlation of explanatory variables can render more traditional statistical approaches problematic. Recent advances in non-parametric computational methods, however, offer scope to overcome such challenges and better capture the complex nature of modern slavery. We present an approach that combines non-linear machine-learning models and strict cross-validation methods with novel variable importance techniques, emphasising the importance of stability of model explanations via a Rashomon-set analysis. This approach is used to model the prevalence of slavery in 48 countries, with results bringing to light the importance of new predictive factors—such as a country’s capacity to protect the physical security of women, which has been previously under-emphasised in quantitative models. Further analyses uncover that women are particularly vulnerable to exploitation in areas where there is poor access to resources. Our model was then leveraged to produce new out-of-sample estimates of slavery prevalence for countries where no survey data currently exists.

Klasifikasi Penentuan Jenis Obat Menggunakan Algoritma Decision Tree

Data Mining merupakan suatu proses untuk mengidentifikasi informasi dan pengetahuan yang bermanfaat. Klasifikasi merupakan salah satu teknik data mining yang dapat digunakan untuk prediksi, dimana nilai yang diprediksi berupa label. Klasifikasi penentuan jenis obat bertujuan untuk memprediksi jenis obat yang tepat untuk pasien dengan menganalisis dataset yang telah diperoleh. Data yang digunakan pada penelitian ini adalah data hasil catatan medis pasien berdasarkan gejala penyakit yang diderita namun belum diketahui jenis obatnya. Dataset yang digunakan merupakan data sekunder, yang berasal dari kaggle.com. Data terdiri dari 200 record dengan 6 variabel (Usia, Jenis Kelamin, Tingkat Tekanan Darah, Tingkat Kolesterol, Na to K dan Jenis Obat) dimana 5 variabel sebagai predictor dan 1 variabel sebagai class target. Data kemudian dipresentasikan kedalam bentuk pohon keputusan dengan suatu model matematis menggunakan bahasa pemograman R. Untuk menyelesaikan permasalahan, maka digunakan sebuah metode klasifikasi dalam data mining yaitu decision tree C4.5. Algoritma C4.5 digunakan untuk menemukan hubungan antara calon sejumlah variabel, sehingga menjadi sebuah variabel target klasifikasi dengan pembagian data menjadi 2 yaitu 70% data training dan 30% data testing. Hasil pengujian yang diperoleh pada penelitian ini berupa aturan dan tingkat nilai accuracy sebesar 100%, sehingga dapat dsimpulkan kinerja algoritma C4.5 dinilai sangat baik dalam memprediksi jenis obat.

On model selection from a finite family of possibly misspecified time series models

Consider finite parametric time series models. “I have $n$ observations and $k$ models, which model should I choose on the basis of the data alone” is a frequently asked question in many practical situations. This poses the key problem of selecting a model from a collection of candidate models, none of which is necessarily the true data generating process (DGP). Although existing literature on model selection is vast, there is a serious lacuna in that the above problem does not seem to have received much attention. In fact, existing model selection criteria have avoided addressing the above problem directly, either by assuming that the true DGP is included among the candidate models and aiming at choosing this DGP, or by assuming that the true DGP can be asymptotically approximated by an increasing sequence of candidate models and aiming at choosing the candidate having the best predictive capability in some asymptotic sense. In this article, we propose a misspecification-resistant information criterion (MRIC) to address the key problem directly. We first prove the asymptotic efficiency of MRIC whether the true DGP is among the candidates or not, within the fixed-dimensional framework. We then extend this result to the high-dimensional case in which the number of candidate variables is much larger than the sample size. In particular, we show that MRIC can be used in conjunction with a high-dimensional model selection method to select the (asymptotically) best predictive model across several high-dimensional misspecified time series models.

Regularization parameter selection for penalized empirical likelihood estimator

Penalized estimation is a useful technique for variable selection when the number of candidate variables is large. A crucial issue in penalized estimation is the selection of the regularization parameter because the performance of the estimator largely depends on an appropriate choice. However, no theoretically sound selection method currently exists for the penalized estimation of moment restriction models. To address this important issue, we develop a novel information criterion, which we call the empirical likelihood information criterion, to select the regularization parameter of the penalized empirical likelihood estimator. The information criterion is derived as an estimator of the expected value of the Kullback–Leibler information criterion from an estimated model to the true data generating process.

Mode jumping MCMC for Bayesian variable selection in GLMM

Generalized linear mixed models (GLMM) are used for inference and prediction in a wide range of different applications providing a powerful scientific tool. An increasing number of sources of data are becoming available, introducing a variety of candidate explanatory variables for these models. Selection of an optimal combination of variables is thus becoming crucial. In a Bayesian setting, the posterior distribution of the models, based on the observed data, can be viewed as a relevant measure for the model evidence. The number of possible models increases exponentially in the number of candidate variables. Moreover, the space of models has numerous local extrema in terms of posterior model probabilities. To resolve these issues a novel MCMC algorithm for the search through the model space via efficient mode jumping for GLMMs is introduced. The algorithm is based on that marginal likelihoods can be efficiently calculated within each model. It is recommended that either exact expressions or precise approximations of marginal likelihoods are applied. The suggested algorithm is applied to simulated data, the famous U.S. crime data, protein activity data and epigenetic data and is compared to several existing approaches.

Variable selection - A review and recommendations for the practicing statistician.

Statistical models support medical research by facilitating individualized outcome prognostication conditional on independent variables or by estimating effects of risk factors adjusted for covariates. Theory of statistical models is well‐established if the set of independent variables to consider is fixed and small. Hence, we can assume that effect estimates are unbiased and the usual methods for confidence interval estimation are valid. In routine work, however, it is not known a priori which covariates should be included in a model, and often we are confronted with the number of candidate variables in the range 10–30. This number is often too large to be considered in a statistical model. We provide an overview of various available variable selection methods that are based on significance or information criteria, penalized likelihood, the change‐in‐estimate criterion, background knowledge, or combinations thereof. These methods were usually developed in the context of a linear regression model and then transferred to more generalized linear models or models for censored survival data. Variable selection, in particular if used in explanatory modeling where effect estimates are of central interest, can compromise stability of a final model, unbiasedness of regression coefficients, and validity of p‐values or confidence intervals. Therefore, we give pragmatic recommendations for the practicing statistician on application of variable selection methods in general (low‐dimensional) modeling problems and on performing stability investigations and inference. We also propose some quantities based on resampling the entire variable selection process to be routinely reported by software packages offering automated variable selection algorithms.

Regularization Parameter Selection for Penalized Empirical Likelihood Estimator

Penalized estimation is a useful technique for variable selection when the number of candidate variables is large. A crucial issue in penalized estimation is the selection of the regularization parameter because the performance of the estimator largely depends on an appropriate choice. However, no theoretically sound selection method currently exists for the penalized estimation of moment restriction models. To address this important issue, we develop a novel information criterion, which we call the empirical likelihood information criterion, to select the regularization parameter of the penalized empirical likelihood estimator. The information criterion is derived as an estimator of the expected value of the Kullback–Leibler information criterion from an estimated model to the true data generating process. We present a Monte Carlo simulation that demonstrates the efficacy of the proposed method.

Handling co-dependence issues in resampling-based variable selection procedures: a simulation study

ABSTRACTIf a number of candidate variables are available, variable selection is a key task aiming to identify those candidates which influence the outcome of interest. Methods as backward elimination, forward selection, etc. are often implemented, despite their drawbacks. One of these drawbacks is the instability of their results with respect to small perturbations in the data. To handle this issue, resampling-based procedures have been introduced; using a resampling technique, e.g. bootstrap, these procedures generate several pseudo-samples that are used to compute the inclusion frequency of each variable, i.e. the proportion of pseudo-samples in which the variable is selected. Based on the inclusion frequencies, it is possible to discriminate between relevant and irrelevant variables. These procedures may fail in case of correlated variables. To deal with this issue, two procedures based on 2×2 tables of inclusion frequencies have been developed in the literature. In this paper we analyse the behaviours of these two procedures and the role of their tuning parameters in an extensive simulation study.

Investigation of powered 2-wheeler accident involvement in urban arterials by considering real-time traffic and weather data

ABSTRACTObjective: Understanding the various factors that affect accident risk is of particular concern to decision makers and researchers. The incorporation of real-time traffic and weather data constitutes a fruitful approach when analyzing accident risk. However, the vast majority of relevant research has no specific focus on vulnerable road users such as powered 2-wheelers (PTWs). Moreover, studies using data from urban roads and arterials are scarce. This study aims to add to the current knowledge by considering real-time traffic and weather data from 2 major urban arterials in the city of Athens, Greece, in order to estimate the effect of traffic, weather, and other characteristics on PTW accident involvement.Methods: Because of the high number of candidate variables, a random forest model was applied to reveal the most important variables. Then, the potentially significant variables were used as input to a Bayesian logistic regression model in order to reveal the magnitude of their effect on PTW accident involvement.Results: The results of the analysis suggest that PTWs are more likely to be involved in multivehicle accidents than in single-vehicle accidents. It was also indicated that increased traffic flow and variations in speed have a significant influence on PTW accident involvement. On the other hand, weather characteristics were found to have no effect.Conclusions: The findings of this study can contribute to the understanding of accident mechanisms of PTWs and reduce PTW accident risk in urban arterials.

Tree-Based Conditional Portfolio Sorts: The Relation between Past and Future Stock Returns

Which variables provide independent information about the cross-section of future returns? Portfolio sorts and Fama-MacBeth regressions cannot easily answer this question when the number of candidate variables is large and when cross-terms might be important. We introduce a new method based on ideas from the machine learning literature that can be used in this context. Applying the method to past-return-based prediction of future returns, short-term returns become the most important predictors. A trading strategy based on our findings has an information ratio twice as high as a Fama-MacBeth regression accounting for two-way interactions. Transaction costs do not explain the results.

Map-based prediction of organic carbon in headwater streams improved by downstream observations from the river outlet

Abstract. In spite of the great abundance and ecological importance of headwater streams, managers are usually limited by a lack of information about water chemistry in these headwaters. In this study we test whether river outlet chemistry can be used as an additional source of information to improve the prediction of the chemistry of upstream headwaters (size < 2 km2), relative to models based on map information alone. We use the concentration of total organic carbon (TOC), an important stream ecosystem parameter, as the target for our study. Between 2000 and 2008, we carried out 17 synoptic surveys in 9 mesoscale catchments (size 32–235 km2). Over 900 water samples were collected in total, primarily from headwater streams but also including each catchment's river outlet during every survey. First we used partial least square regression (PLS) to model the distribution (median, interquartile range (IQR)) of headwater stream TOC for a given catchment, based on a large number of candidate variables including sub-catchment characteristics from GIS, and measured river chemistry at the catchment outlet. The best candidate variables from the PLS models were then used in hierarchical linear mixed models (MM) to model TOC in individual headwater streams. Three predictor variables were consistently selected for the MM calibration sets: (1) proportion of forested wetlands in the sub-catchment (positively correlated with headwater stream TOC), (2) proportion of lake surface cover in the sub-catchment (negatively correlated with headwater stream TOC), and (3) river outlet TOC (positively correlated with headwater stream TOC). Including river outlet TOC improved predictions, with 5–15 % lower prediction errors than when using map information alone. Thus, data on water chemistry measured at river outlets offer information which can complement GIS-based modelling of headwater stream chemistry.

Variable selection for modeling the absolute magnitude at maximum of Type Ia supernovae

Abstract We discuss what is an appropriate set of explanatory variables in order to predict the absolute magnitude at the maximum of Type Ia supernovae. In order to have a good prediction, the error for future data, which is called the “generalization error,” should be small. We use cross-validation in order to control the generalization error and a LASSO-type estimator in order to choose the set of variables. This approach can be used even in the case that the number of samples is smaller than the number of candidate variables. We studied the Berkeley supernova database with our approach. Candidates for the explanatory variables include normalized spectral data, variables about lines, and previously proposed flux ratios, as well as the color and light-curve widths. As a result, we confirmed the past understanding about Type Ia supernovae: (i) The absolute magnitude at maximum depends on the color and light-curve width. (ii) The light-curve width depends on the strength of Si ii. Recent studies have suggested adding more variables in order to explain the absolute magnitude. However, our analysis does not support adding any other variables in order to have a better generalization error.

Statistical strategies for constructing health risk models with multiple pollutants and their interactions: possible choices and comparisons.

BackgroundAs public awareness of consequences of environmental exposures has grown, estimating the adverse health effects due to simultaneous exposure to multiple pollutants is an important topic to explore. The challenges of evaluating the health impacts of environmental factors in a multipollutant model include, but are not limited to: identification of the most critical components of the pollutant mixture, examination of potential interaction effects, and attribution of health effects to individual pollutants in the presence of multicollinearity.MethodsIn this paper, we reviewed five methods available in the statistical literature that are potentially helpful for constructing multipollutant models. We conducted a simulation study and presented two data examples to assess the performance of these methods on feature selection, effect estimation and interaction identification using both cross-sectional and time-series designs. We also proposed and evaluated a two-step strategy employing an initial screening by a tree-based method followed by further dimension reduction/variable selection by the aforementioned five approaches at the second step.ResultsAmong the five methods, least absolute shrinkage and selection operator regression performs well in general for identifying important exposures, but will yield biased estimates and slightly larger model dimension given many correlated candidate exposures and modest sample size. Bayesian model averaging, and supervised principal component analysis are also useful in variable selection when there is a moderately strong exposure-response association. Substantial improvements on reducing model dimension and identifying important variables have been observed for all the five statistical methods using the two-step modeling strategy when the number of candidate variables is large.ConclusionsThere is no uniform dominance of one method across all simulation scenarios and all criteria. The performances differ according to the nature of the response variable, the sample size, the number of pollutants involved, and the strength of exposure-response association/interaction. However, the two-step modeling strategy proposed here is potentially applicable under a multipollutant framework with many covariates by taking advantage of both the screening feature of an initial tree-based method and dimension reduction/variable selection property of the subsequent method. The choice of the method should also depend on the goal of the study: risk prediction, effect estimation or screening for important predictors and their interactions.

Gaining insight with recursive partitioning of generalized linear models

Recursive partitioning algorithms separate a feature space into a set of disjoint rectangles. Then, usually, a constant in every partition is fitted. While this is a simple and intuitive approach, it may still lack interpretability as to how a specific relationship between dependent and independent variables may look. Or it may be that a certain model is assumed or of interest and there is a number of candidate variables that may non-linearly give rise to different model parameter values. We present an approach that combines generalized linear models (GLM) with recursive partitioning that offers enhanced interpretability of classical trees as well as providing an explorative way to assess a candidate variable's influence on a parametric model. This method conducts recursive partitioning of a GLM by (1) fitting the model to the data set, (2) testing for parameter instability over a set of partitioning variables, (3) splitting the data set with respect to the variable associated with the highest instability. The outcome is a tree where each terminal node is associated with a GLM. We will show the method's versatility and suitability to gain additional insight into the relationship of dependent and independent variables by two examples, modelling voting behaviour and a failure model for debt amortization, and compare it to alternative approaches.

The benefit of data-based model complexity selection via prediction error curves in time-to-event data

The fitting of predictive survival models usually involves determination of model complexity parameters. Up to now, there was no general applicable model selection criterion for semi- or non-parametric approaches. The integrated prediction error curve, an estimator of the integrated Brier score, has the ability to close this gap and allows a reasonable, data-based choice of complexity parameters for any kind of model where risk predictions can be obtained. Random survival forests are used as example throughout the article. Here, a critical complexity parameter might be the number of candidate variables at each node. Model selection by our integrated prediction error curve criterion is compared to a frequently used rule of thumb, investigating the potential benefit regarding prediction performance. For that, simulated microarray survival data as well as two real data sets of patients with diffuse large-B-cell lymphoma and of patients with neuroblastoma are used. It is shown, that the optimal parameter value depends on the amount of information in the data and that a data-based selection can therefore be beneficial in several settings.

Investigation of habitat preferences of Iranian jerboa (Allactaga firouzi Womochel 1978)

We studied habitat preferences of the only known population of Iranian jerboa in Iran. Binary logistic regression was used to identify habitat variables that might be important to Iranian jerboa. The number of candidate variables was reduced using logistic regression retaining variables that differed between occupied (presence) and non-used (absence) plots. Logistic regression model could successfully distinguish between used and non-used areas. Habitat use of Iranian Jerboa is mainly affected by vegetation type and percent cover of bare soil.

Gibbs posterior for variable selection in high-dimensional classification and data mining

In the popular approach of “Bayesian variable selection” (BVS), one uses prior and posterior distributions to select a subset of candidate variables to enter the model. A completely new direction will be considered here to study BVS with a Gibbs posterior originating in statistical mechanics. The Gibbs posterior is constructed from a risk function of practical interest (such as the classification error) and aims at minimizing a risk function without modeling the data probabilistically. This can improve the performance over the usual Bayesian approach, which depends on a probability model which may be misspecified. Conditions will be provided to achieve good risk performance, even in the presence of high dimensionality, when the number of candidate variables “K” can be much larger than the sample size “n.” In addition, we develop a convenient Markov chain Monte Carlo algorithm to implement BVS with the Gibbs posterior.

Multi-step methods for choosing the best set of variables in regression analysis

In a recent article (Konno and Yamamoto in ISE 07-01, Department of Industrial and Systems Engineering, Chuo University, February 2007), one of the authors formulated the problem of choosing the best set of explanatory variables from a large number of candidate variables in a linear regression model as a mixed 0---1 integer linear programming problem and showed that it can be solved by the state-of-the-art integer programming software. In this paper, we will propose multi-step methods for calculating a close to optimal solution of the problem which may not be solved by a single-step method presented in Konno and Yamamoto (ISE 07-01, Department of Industrial and Systems Engineering, Chuo University, February 2007). It will be shown that a multi-step method can generate a nearly optimal solution within a fraction of computation time of the single step method. Also, we will demonstrate that the best set of variables in terms of the squared error can be recovered under normality assumption.

Generalization error for multi-class margin classification

In this article, we study rates of convergence of the generalization error of multi-class margin classifiers. In particular, we develop an upper bound theory quantifying the generalization error of various large margin classifiers. The theory permits a treatment of general margin losses, convex or nonconvex, in presence or absence of a dominating class. Three main results are established. First, for any fixed margin loss, there may be a trade-off between the ideal and actual generalization performances with respect to the choice of the class of candidate decision functions, which is governed by the trade-off between the approximation and estimation errors. In fact, different margin losses lead to different ideal or actual performances in specific cases. Second, we demonstrate, in a problem of linear learning, that the convergence rate can be arbitrarily fast in the sample size n depending on the joint distribution of the input/output pair. This goes beyond the anticipated rate O(n−1). Third, we establish rates of convergence of several margin classifiers in feature selection with the number of candidate variables p allowed to greatly exceed the sample size n but no faster than exp(n).