Subset Of Explanatory Variables Research Articles

Sparse Poisson regression via mixed-integer optimization.

We present a mixed-integer optimization (MIO) approach to sparse Poisson regression. The MIO approach to sparse linear regression was first proposed in the 1970s, but has recently received renewed attention due to advances in optimization algorithms and computer hardware. In contrast to many sparse estimation algorithms, the MIO approach has the advantage of finding the best subset of explanatory variables with respect to various criterion functions. In this paper, we focus on a sparse Poisson regression that maximizes the weighted sum of the log-likelihood function and the L2-regularization term. For this problem, we derive a mixed-integer quadratic optimization (MIQO) formulation by applying a piecewise-linear approximation to the log-likelihood function. Optimization software can solve this MIQO problem to optimality. Moreover, we propose two methods for selecting a limited number of tangent lines effective for piecewise-linear approximations. We assess the efficacy of our method through computational experiments using synthetic and real-world datasets. Our methods provide better log-likelihood values than do conventional greedy algorithms in selecting tangent lines. In addition, our MIQO formulation delivers better out-of-sample prediction performance than do forward stepwise selection and L1-regularized estimation, especially in low-noise situations.

Modelling and forecasting national introduction times for successive generations of mobile telephony

An accurate prediction of the timing of a country's introduction of a new generation of mobile telephony benefits numerous agents including suppliers of network and consumer equipment, regulators, and network planners. We consider the estimation and prediction of the time interval between the international introduction of a generation of mobile telephony and its introduction into a specific country when a decision maker judges the introduction of a newer technology a worthwhile investment. Using literature-based socio-economic and geographical variables, we examine how well variation in international introduction times of four generations of mobile telephony in 172 countries can be explained and forecast. We model and forecast introduction times at two levels of granularity: we use Cox's proportional hazards model for the introduction time; we partition countries into introduction time-based segments and model segment membership using multinomial logistic regression. Our modelling of each generation considers three subsets of explanatory variables: All variables, socio-economic Covariates only, Regional dummies only. Over successive generations, the Covariates only models reveal the changing relevance of each socio-economic covariate. Model-based forecasting of the introduction time of the next generation is performed under three hypotheses making different uses of the information available at the time the relevant generation is launched internationally. However, changing socio-economic environments coupled with changing models impair forecasting accuracy, the lower accuracy of modelled introduction times is concentrated in 20% of countries. We speculate about the nature of the unobserved factors affecting these countries' decision processes.

Best subset selection via cross-validation criterion

This paper is concerned with the cross-validation criterion for selecting the best subset of explanatory variables in a linear regression model. In contrast with the use of statistical criteria (e.g., Mallows’ $$C_p$$, the Akaike information criterion, and the Bayesian information criterion), cross-validation requires only mild assumptions, namely, that samples are identically distributed and that training and validation samples are independent. For this reason, the cross-validation criterion is expected to work well in most situations involving predictive methods. The purpose of this paper is to establish a mixed-integer optimization approach to selecting the best subset of explanatory variables via the cross-validation criterion. This subset-selection problem can be formulated as a bilevel MIO problem. We then reduce it to a single-level mixed-integer quadratic optimization problem, which can be solved exactly by using optimization software. The efficacy of our method is evaluated through simulation experiments by comparison with statistical-criterion-based exhaustive search algorithms and $$L_1$$-regularized regression. Our simulation results demonstrate that, when the signal-to-noise ratio was low, our method delivered good accuracy for both subset selection and prediction.

Development of a model ensemble to predict Peary caribou populations in the Canadian Arctic Archipelago

AbstractIn the field of biological conservation, mathematical modeling has been an indispensable tool to advance our understanding of population dynamics. Modeling rare and endangered species with complex ecophysiological tools can be challenging due to the constraints imposed by data availability. One strategy to overcome the mismatch between what we are trying to learn from a modeling exercise and the available empirical knowledge is to develop statistical models that tend to be more parsimonious. In the present study, we introduce a spatially explicit modeling framework to examine the strength and nature of the relationships of snow density and vegetation abundance with Peary caribou (Rangifer tarandus pearyi) populations. Peary caribou are vital to the livelihood and culture of High Arctic Inuit communities, but changing climatic conditions and anthropogenic disturbances may affect the integrity of this endemic species population. Owing to an estimated decline of over 35% during the last three generations, a recent assessment by the Committee on the Status of Endangered Wildlife in Canada assigned a Threatened status to Peary caribou in 2015. Recognizing the uncertainty typically associated with the selection of the best subset of explanatory variables and their optimal functional relationship with the response variable, we examined four models across six island complexes (Banks, Axel Heiberg, Melville, Bathurst, Mackenzie King, and Boothia) of the Arctic Archipelago and formulated two ensembles to synthesize their predictions into averaged Peary caribou population distributions. Our analysis showed that an ensemble strategy with region‐specific weights displayed the highest performance and most balanced error across the six island complexes. The causal linkages between snow, vegetation abundance, and Peary caribou did manifest themselves with the models examined, but the noise‐to‐signal ratios of the corresponding regression coefficients were generally high and there were instances where they were not discernible from zero. We also present a sensitivity analysis exercise that elucidates the influence of the observation/imputation errors on the model‐training phase, thereby highlighting the importance of assigning realistic error estimates that will not hamper the identification of important cause–effect relationships. Our study identifies critical augmentations of the available scientific knowledge that necessitate to design the optimal management actions of Peary caribou populations across the Canadian Arctic Archipelago.

Could land surface phenology be used to discriminate Mediterranean pine species?

Land surface phenology (LSP) can improve the monitoring of forest areas and their change processes. The aim of this study was to characterize the temporal dynamics in Mediterranean pines and evaluate the potential of LSP for species discrimination. We used 661 mono-specific plots for five different Pinus species (Pinus halepensis, P. pinea, P. pinaster; P. sylvestris, P. nigra) and the MOD13Q1-NDVI time series (2000–2016) to perform the analyses. The time series were smoothed to extract the phenological parameters and calculate multi-temporal metrics, to synthesize the inter-annual variability. The potential of LSP for discriminating between Pinus species was evaluated by the application of the Random Forest (RF) classifier from different subsets of explanatory variables: i) the smooth time series; ii) the multi-temporal metrics; and iii) the multi-temporal metrics plus the auxiliary physical variables. This latter subset was also used as input to a Classification and Regression Tree (CART) algorithm to better explain the differences between Pinus species regarding LSP parameters and other environmental drivers. The analysis showed two different patterns: an important NDVI decrease during the summer for P. halepensis, P. pinea, and P. pinaster; and lower NDVI variation along the year for P. sylvestris. P. nigra showed a heterogeneous intra-specific behavior, having locations with different patterns. We distinguished Pinus species plots with a global accuracy of 0.82, when we used multi-temporal metrics of LSP and auxiliary physical variables. More generally, the Mediterranean Pinus species could be differentiated considering the 23rd of July as the start of season and 179 km and 1100 m as distance to the coastline and elevation, respectively.

Consistency Inference Property of QIC in Selecting the True Working Correlation Structure for Generalized Estimating Equations

The generalized estimating equations (GEE) is one of the statistical approaches for the analysis of longitudinal data with correlated response. A working correlation structure for the repeated measures of the outcome variable of a subject needs to be specified by this method and the GEE estimator for the regression parameter will be the most efficient if the working correlation matrix is correctly specified. Hence it is desirable to choose a working correlation matrix that is the closest to the underlying structure among a set of working correlation structures. The quasi-likelihood Information criteria (QIC) was proposed for the selection of the working correlation structure and the best subset of explanatory variables in GEE. However, its success rate in selecting the true correlation structure has been established to be about 29.4%. Likewise, past studies have shown that its bias increases with the number of parameters. By considering longitudinal data with binary response, we establish numerically through simulations the consistency property of QIC in selecting the true working correlation structure and the conditions for its consistency. Further, we propose a modified QIC that penalizes for the number of parameter estimates in the original QIC and numerically establish that the penalization enhances the consistency of QIC in selecting the true working correlation structure. The results indicate that QIC selects the true correlation structure with probability approaching one if only parsimonious structures are considered otherwise the selection rates are less than 50% regardless of the increase in the sample size, measurements per subject and level of correlation. Further, we established that the probability of selecting the true correlation structure <i>R₀</i> almost surely converges to one when we penalize for the number of correlation parameters estimated.

Assessment of Machine Learning Performance for Decision Support in Venture Capital Investments

The venture capital (VC) industry offers opportunities for investment in early-stage companies where uncertainty is very high. Unfortunately, the tools investors currently have available are not robust enough to reduce risk and help them managing uncertainty better. Machine learning data-driven approaches can bridge this gap, as they already do in the hedge fund industry. These approaches are now possible because data from thousands of companies over the world is available through platforms such as Crunchbase. Previous academic efforts have focused only on predicting two classes of exits, i.e., being acquired by other company or offering shares to the public, using only one or a few subsets of explanatory variables. These events are typically related to high returns, but also higher risk, making hard for a venture fund to get repeatable and sustainable returns. On the contrary, we will try to predict more possible outcomes including a subsequent funding round or the closure of the company using a large set of signals. In this way, our approach would provide VC investors with more information to set up a portfolio with lower risk that may eventually achieve higher returns than those based on finding unicorns (i.e., companies with a valuation higher than one billion dollars). We will analyze the performance of several machine learning methods in a dataset of over 120,000 early-stage companies in a realistic setting that tries to predict their progress in a 3-year time window. Results show that machine learning can support venture investors in their decision-making processes to find opportunities and better assessing the risk of potential investments.

Modeling of cigarette smoke constituents - From intense to less intense smoking regime

Since it was first required to measure and to report NFDPM and nicotine yields in a limited number of countries, there has been an increasing trend for more testing and reporting requirements. Historically, the ISO 3308 smoking regime has been used to determine NFDPM and nicotine yields. However recommendations from the World Health Organization, now include the use of two smoking regimes such as the ISO 3308 and the WHO TobLabNet Official Method SOP01, the latter being considered as an intense smoking regime. Considering the increase in data produced and similarities between some smoke constituents formed during combustion, we explored possible correlations between emissions under intense and less intense smoking conditions. A set of 22 commercial cigarettes was tested. Eighty five smoke constituents were determined under both intense and less intense regimes. In addition 36 tobacco constituents, 14 cigarette design parameters and eight cigarette burning features were determined. A computational process was designed to implement multiple linear regression analyses enabling the identification of the best subsets of explanatory variables among emissions under intense conditions, cigarette design parameters, tobacco constituents and burning parameters. We succeeded in building simple linear models, involving four to six variables, while reaching satisfactory goodness of fit and R-squared values ranging from 0.87 to 1.00. Our findings suggest, in the range of products tested, that the additional data gained by using a second smoking regime does not necessarily increase the volume of information and consequently does not necessarily improve knowledge. This study supports the premise that the application of two smoking regimes does not produce a more comprehensive product characterisation compared to using one.

Predicting crash frequency for multi-vehicle collision types using multivariate Poisson-lognormal spatial model: A comparative analysis

According to crash configuration and pre-crash conditions, traffic crashes are classified into different collision types. Based on the literature, multi-vehicle crashes, such as head-on, rear-end, and angle crashes, are more frequent than single-vehicle crashes, and most often result in serious consequences. From a methodological point of view, the majority of prior studies focused on multivehicle collisions have employed univariate count models to estimate crash counts separately by collision type. However, univariate models fail to account for correlations which may exist between different collision types. Among others, multivariate Poisson lognormal (MVPLN) model with spatial correlation is a promising multivariate specification because it not only allows for unobserved heterogeneity (extra-Poisson variation) and dependencies between collision types, but also spatial correlation between adjacent sites. However, the MVPLN spatial model has rarely been applied in previous research for simultaneously modelling crash counts by collision type. Therefore, this study aims at utilizing a MVPLN spatial model to estimate crash counts for four different multi-vehicle collision types, including head-on, rear-end, angle, and sideswipe collisions. To investigate the performance of the MVPLN spatial model, a two-stage model and a univariate Poisson lognormal model (UNPLN) spatial model were also developed in this study. Detailed information on roadway characteristics, traffic volume, and crash history were collected on 407 homogeneous segments from Malaysian federal roads. The results indicate that the MVPLN spatial model outperforms the other comparing models in terms of goodness-of-fit measures. The results also show that the inclusion of spatial heterogeneity in the multivariate model significantly improves the model fit, as indicated by the Deviance Information Criterion (DIC). The correlation between crash types is high and positive, implying that the occurrence of a specific collision type is highly associated with the occurrence of other crash types on the same road segment. These results support the utilization of the MVPLN spatial model when predicting crash counts by collision manner. In terms of contributing factors, the results show that distinct crash types are attributed to different subsets of explanatory variables.

Logical Analysis of Data as a tool for the analysis of Probabilistic Discrete Choice Behavior

Probabilistic Discrete Choice Models (PDCM) have been extensively used to interpret the behavior of heterogeneous decision makers that face discrete alternatives. The classification approach of Logical Analysis of Data (LAD) uses discrete optimization to generate patterns, which are logic formulas characterizing the different classes. Patterns can be seen as rules explaining the phenomenon under analysis. In this work we discuss how LAD can be used as the first phase of the specification of PDCM. Since in this task the number of patterns generated may be extremely large, and many of them may be nearly equivalent, additional processing is necessary to obtain practically meaningful information. Hence, we propose computationally viable techniques to obtain small sets of patterns that constitute meaningful representations of the phenomenon and allow to discover significant associations between subsets of explanatory variables and the output. We consider the complex socio-economic problem of the analysis of the utilization of the Internet in Italy, using real data gathered by the Italian National Institute of Statistics.

Approaches to select suitable subset of explanatory variables for establishing fuzzy regression models

Approaches to select suitable subset of explanatory variables for establishing fuzzy regression models

Multivariate Linear Regression and CART Regression Analysis of TBM Performance at Abu Hamour Phase-I Tunnel

Abstract The first phase of the Abu Hamour drainage and storm tunnel was completed in early 2017. The 9.5 km long, 3.7 m diameter tunnel was excavated with two Earth Pressure Balance (EPB) Tunnel Boring Machines from Herrenknecht. TBM operation processes were monitored and recorded by Data Acquisition and Evaluation System. The authors coupled collected TBM drive data with available information on rock mass properties, cleansed, completed with secondary variables and aggregated by weeks and shifts. Correlations and descriptive statistics charts were examined. Multivariate Linear Regression and CART regression tree models linking TBM penetration rate (PR), penetration per revolution (PPR) and field penetration index (FPI) with TBM operational and geotechnical characteristics were performed for the conditions of the weak/soft rock of Doha. Both regression methods are interpretable and the data were screened with different computational approaches allowing enriched insight. The primary goal of the analysis was to investigate empirical relations between multiple explanatory and responding variables, to search for best subsets of explanatory variables and to evaluate the strength of linear and non-linear relations. For each of the penetration indices, a predictive model coupling both regression methods was built and validated. The resultant models appeared to be stronger than constituent ones and indicated an opportunity for more accurate and robust TBM performance predictions.

Automatic Analysis of Retinal Vascular Parameters for Detection of Diabetes in Indian Patients with No Retinopathy Sign.

This study has investigated the association between retinal vascular parameters with type II diabetes in Indian population with no observable diabetic retinopathy. It has introduced two new retinal vascular parameters: total number of branching angles (TBA) and average acute branching angles (ABA) as potential biomarkers of diabetes in an explanatory model. A total number of 180 retinal images (two (left and right) × two (ODC and MC) × 45 subjects (13 diabetics and 32 nondiabetics)) were analysed. Stepwise linear regression analysis was performed to model the association between type II diabetes with the best subset of explanatory variables (predictors), consisting of retinal vascular parameters and patients' demographic information. P value of the estimated coefficients (P < 0.001) indicated that, at α level of 0.05, the newly introduced retinal vascular parameters, that is, TBA and ABA together with CRAE, mean tortuosity, SD of branching angle, and VB, are related to type II diabetes when there is no observable sign of retinopathy.

The Basin and Range Dixie Valley Geothermal Wellfield, Nevada, USA—A test bed for developing an Enhanced Geothermal System exploration favorability methodology

The Dixie Valley Geothermal Wellfield (DVGW), a Basin and Range type system in central Nevada USA, is used as an exploration case history for Enhanced Geothermal Systems (EGS). It encompasses an area of approximately 170km2. The wellfield produces 67MW of geothermal power and it contains a number of high temperature non-productive wells. This wellfield was chosen as the calibration site for the development of an EGS exploration methodology due to its extensive body of geoscience data and information on the geothermal system and most importantly, well results in the public domain. This existing body of data (ca. 2011) was in part re-interpreted to produce a baseline conceptual model in terms of a number of serial cross-sections for the available data sets: geology, seismic reflection, resistivity, temperature, gravity-magnetic, and p-wave velocity (Vp). The exploration methodology was calibrated against available geothermal well results. Based on a comprehensive review of all available geoscience and well data, EGS favorability maps were generated, from +1km above sea level (asl) to −4km asl at 0.5km intervals, for the three key EGS parameters of interest: rock type, temperature, and stress. Complimentary trust (“confidence-in-the-data-used”) maps were also created at the same scale to, among other things, indicate where additional data may be required.Quantitative geostatistical analysis of the geoscience data was conducted, among other factors, to address the question of whether the baseline geoscience data could be used to predict EGS favorability without the advantage of existing well temperature data. Classification and Regression Tree (CART) was one of a number of geostatistical methods applied to the baseline geoscience data and it provided the most promising results. In CART, the response variable (RV) is predicted while using explanatory variables (EVs). The geoscience parameters (EVs) considered in the CART analysis included temperature, Vp, resistivity from magnetotellurics, Coulomb Stress Change (CSC), dilatational strain (from CSC modeling), vertical stress, lithology based on geologic analysis, lithology based on gravity–magnetic modeling (G–M lithology) and the presence or absence of a fault. Temperature increases with depth in the DVGW. Vertical stress also increases with depth and it was deemed as a redundant EV. As such, a CART sensitivity analysis was applied to the baseline data set with and without vertical stress being considered as an EV to determine the effect of removing vertical stress and to evaluate with subsets of EVs that could be predictive of key EGS parameters. R2-values ranging from 0.611 to 0.841 were obtained for the RVs: temperature, lithology, productive vs. non-productive hydrothermal cells and expected EGS favorable cells (the response variables) using both cross-section and well data and not considering vertical stress. However, these CART results were not used in the generation of the favorability maps because this is the first analysis of its kind that the authors are aware of and more testing at other sites needs to be done; the raw total baseline data set described above was considered the most appropriate for this study.

Mixed integer second-order cone programming formulations for variable selection in linear regression

This study concerns a method of selecting the best subset of explanatory variables in a multiple linear regression model. Goodness-of-fit measures, for example, adjusted R2, AIC, and BIC, are generally used to evaluate a subset regression model. Although variable selection with regard to these measures is usually performed with a stepwise regression method, it does not always provide the best subset of explanatory variables. In this paper, we propose mixed integer second-order cone programming formulations for selecting the best subset of variables with respect to adjusted R2, AIC, and BIC. Computational experiments show that, in terms of these measures, the proposed formulations yield better solutions than those provided by common stepwise regression methods.

The Influence of Land Cover, Vertical Structure, and Socioeconomic Factors on Outdoor Water Use in a Western US City

To help inform urban water conservation and planning, we evaluated spatial patterns and correlative relationships among physical land cover properties, socioeconomic and demographic characteristics, and single-family outdoor residential water use in Aurora, Colorado, a rapidly-growing suburb in the semi-arid Colorado Front Range. Using high resolution land cover maps and lidar-derived vertical structural data, we quantified land cover composition and vertical structural characteristics for detached, single-family residential parcels. These data were combined with socioeconomic and demographic datasets from the 2010 US Census and local government agencies and used in Random Forest analyses of outdoor water use estimated from residential water meter records, with separate analyses conducted using parcels and census block groups as sampling units. Conditional variable importance measures from Random Forest analyses and comparisons of the predictive accuracy of models developed using subsets of explanatory variables were used to assess the relative importance of physical and socioeconomic variables in predicting outdoor water use. Models developed using the subset of land cover variables had the highest predictive accuracy, followed by vertical structural variables, and lastly, socioeconomic/demographic variables. At both the parcel and census block group scale, there was significant spatial clustering in outdoor water use as indicated by various spatial statistical analyses. Our approach demonstrates the value of high resolution land cover and structure data for understanding urban water use patterns and can be used for targeting water conservation efforts.

Variable selection in Bayesian generalized linear-mixed models: an illustration using candidate gene case-control association studies.

The problem of variable selection in the generalized linear-mixed models (GLMMs) is pervasive in statistical practice. For the purpose of variable selection, many methodologies for determining the best subset of explanatory variables currently exist according to the model complexity and differences between applications. In this paper, we develop a "higher posterior probability model with bootstrap" (HPMB) approach to select explanatory variables without fitting all possible GLMMs involving a small or moderate number of explanatory variables. Furthermore, to save computational load, we propose an efficient approximation approach with Laplace's method and Taylor's expansion to approximate intractable integrals in GLMMs. Simulation studies and an application of HapMap data provide evidence that this selection approach is computationally feasible and reliable for exploring true candidate genes and gene-gene associations, after adjusting for complex structures among clusters.

Stratified Weibull Regression Model for Interval-Censored Data

Interval censored outcomes arise when a silent event of interest is known to have occurred within a specific time period determined by the times of the last negative and first positive diagnostic tests. There is a rich literature on parametric and non-parametric approaches for the analysis of interval-censored outcomes. A commonly used strategy is to use a proportional hazards (PH) model with the baseline hazard function parameterized. The proportional hazards assumption can be relaxed in stratified models by allowing the baseline hazard function to vary across strata defined by a subset of explanatory variables. In this paper, we describe and implement a new R package straweib, for fitting a stratified Weibull model appropriate for interval censored outcomes. We illustrate the R package straweib by analyzing data from a longitudinal oral health study on the timing of the emergence of permanent teeth in 4430 children.

Diameter growth prediction for individual Pinus occidentalis Sw. trees

Predictive equations calibrated with remeasurement data from 25 permanent plots having individually identified trees were used to project stem diameter of Pinus occidentalis Sw. in Dominican Republic. The general form of the models used to fit the growth and yield functions included fixed effect covariates related to three subsets of explanatory variables: initial tree size, stand attributes, and competition indexes. The subsets were incrementally added in a stepwise fashion for each of the three response variables and the intercept of the model was allowed to vary randomly between plots. The models evaluated included a yield function that predicted future diameter at year t (dt), a periodic annual increment model using five-year diameter increment (id5) and its natural log transformation [ln(id5+0.01)]. For trees that were not remeasured exactly 5 years after the first measurement, id5 was computed by averaging the mean annual increment nearest the 5 year point and multiplying by five. Each approach was evaluated using an independent validation data set based on seven goodness-of-fit statistics, graphical display of residuals and the variance components of each model combination. LMM, including fixed and random parameters, provided a better fit among the models tested. For estimating future diameter, accuracy of predictions is within one centimeter for a five-year projection interval, and bias is negligible. All the models had moderately improved fit statistics when random effects were included in the evaluation. The degree of improvement behaved in a similar manner for most fit statistics, with differences in the range of values for MSE, RMSE and RMSE% of 0.53, 0.23 and 1.05, respectively. The absolute difference between the extreme values for Bias and relative Bias (%) in all the models was 0.20 and 0.92. The differences in values for MAD were only 0.15 and R2 values were practically equivalent.

Non-monotonic penalizing for the number of structural breaks

This paper first reduces the problem of detecting structural breaks in a random walk to that of finding the best subset of explanatory variables in a regression model and then tailors various subset selection criteria to this specific problem. Of particular interest are those new criteria, which are obtained by means of simulation using the efficient algorithm of Bai and Perron (J Appl Econom 18:1---22, 2003). Unlike conventional variable selection methods, which penalize new variables entering a model either in the same way (e.g., AIC and BIC) or milder (e.g., MRIC and $$\mathrm {FPE}_\mathrm{{sub}}$$ ) than already included variables, they do not follow any monotonic penalizing scheme. In general, their non-monotonicity is more pronounced in the case of fat tails. The characteristics of the different criteria are illustrated using bootstrap samples from the Nile data set.