Large Number Of Explanatory Variables Research Articles

Smoothed empirical likelihood estimation and automatic variable selection for an expectile high-dimensional model

We consider a linear model which can have a large number of explanatory variables, the errors with an asymmetric distribution or the values of the explained variable are missing at random. In order to take in account these several situations, we consider the non parametric empirical likelihood (EL) estimation method. Because a constraint in EL contains an indicator function then a smoothed function instead of the indicator will be considered. Two smoothed expectile maximum EL methods are proposed, one of which will automatically select the explanatory variables. For each of the methods we obtain the convergence rate of the estimators and their asymptotic normality. The smoothed expectile empirical log-likelihood ratio process follow asymptotically a chi-square distribution and moreover the adaptive LASSO smoothed expectile maximum EL estimator satisfies the sparsity property which guarantees the automatic selection of zero model coefficients. In order to implement these methods, we propose four algorithms.

Improving the Estimation of Structural Parameters of a Mixed Conifer–Broadleaf Forest Using Structural, Textural, and Spectral Metrics Derived from Unmanned Aerial Vehicle Red Green Blue (RGB) Imagery

Forest structural parameters are crucial for assessing ecological functions and forest quality. To improve the accuracy of estimating these parameters, various approaches based on remote sensing platforms have been employed. Although remote sensing yields high prediction accuracy in uniform, even-aged, simply structured forests, it struggles in complex structures, where accurately predicting forest structural parameters remains a significant challenge. Recent advancements in unmanned aerial vehicle (UAV) photogrammetry have opened new avenues for the accurate estimation of forest structural parameters. However, many studies have relied on a limited set of remote sensing metrics, despite the fact that selecting appropriate metrics as powerful explanatory variables and applying diverse models are essential for achieving high estimation accuracy. In this study, high-resolution RGB imagery from DJI Matrice 300 real-time kinematics was utilized to estimate forest structural parameters in a mixed conifer–broadleaf forest at the University of Tokyo Hokkaido Forest (Hokkaido, Japan). Structural and textual metrics were extracted from canopy height models, and spectral metrics were extracted from orthomosaics. Using random forest and multiple linear regression models, we achieved relatively high estimation accuracy for dominant tree height, mean tree diameter at breast height, basal area, mean stand volume, stem density, and broadleaf ratio. Including a large number of explanatory variables proved advantageous in this complex forest, as its structure is influenced by numerous factors. Our results will aid foresters in predicting forest structural parameters using UAV photogrammetry, thereby contributing to sustainable forest management.

Data-driven process adjustment policies for quality improvement

Common objectives in machine learning research are to predict the output quality of manufacturing processes, to perform root cause analysis in case of reduced quality, and to propose intervention strategies. The cost of reduced quality must be weighed against the cost of the interventions, which depend on required downtime, personnel costs, and material costs. Furthermore, there is a risk of false negatives, i.e., failure to identify the true root causes, or false positives, i.e., adjustments that further reduce the quality. A policy for process adjustments describes when and where to perform interventions, and we say that a policy is worthwhile if it reduces the expected operational cost. In this paper, we describe a data-driven alarm and root cause analysis framework, that given a predictive and explanatory model trained on high-dimensional process and quality data, can be used to search for a worthwhile adjustment policy. The framework was evaluated on large-scale simulated process and quality data. We find that worthwhile adjustment policies can be derived also for problems with a large number of explanatory variables. Interestingly, the performance of the adjustment policies is almost exclusively driven by the quality of the model fits. Based on these results, we discuss key areas of future research, and how worthwhile adjustment policies can be implemented in real world applications.

Forecasting Inflation in Russia Using a TVP Model with Bayesian Shrinkage

The paper emphasizes the relevance of improving methodological tools for macroeconomic forecasting. In particular, it is pointed out, that models with a large number of explanatory variables on relatively short samples can often overfit in-sample and, thus, forecast poorly. The article reviews studies on forecasting inflation in Russia and explains the applicability of the model with Bayesian shrinkage of time-varying parameters based on hierarchical normal-gamma prior. Models of this type allow for possible nonlinearities in relationships between regressors and inflation and, at the same time, can deal with the problem of overfitting.The choice of a system of statistical indicators used to forecast monthly inflation in Russia during the period 2011–2022 is substantiated. It is shown that at short forecast horizons (of one to three months) Bayesian normal-gamma shrinkage TVP model with a large set of inflation predictors outperforms in forecasting accuracy, measured by mean absolute and squared errors, its linear counterpart, linear and Bayesian autoregression models without predictors, as well as naive models (based on random walk). At the horizon of six months, the autoregression model with Bayesian shrinkage exhibits the best forecast performance. As the forecast horizon rises (up to one year), statistical differences in the quality of forecasts of competing models of Russian inflation decrease.The developed method can be used by the Bank of Russia and executive authorities for rapid assessment of inflation forecasts until the end of the year in order to evaluate risks of inflation deviation from the target level and elaborate preventive economic policy measures.

Robust nonparametric regression: Review and practical considerations

Nonparametric regression models offer a way to understand and quantify relationships between variables without having to identify an appropriate family of possible regression functions. Although many estimation methods for these models have been proposed in the literature, most of them can be highly sensitive to the presence of a small proportion of atypical observations in the training set. A review of outlier robust estimation methods for nonparametric regression models is provided, paying particular attention to practical considerations. Since outliers can also influence negatively the regression estimator by affecting the selection of bandwidths or smoothing parameters, a discussion of robust alternatives for this task is also included. Using many of the “classical” nonparametric regression estimators (and their robust counterparts) can be very challenging in settings with a moderate or large number of explanatory variables, so recently proposed robust nonparametric regression methods that scale well with a growing number of covariates are also discussed.

Local attributes and migration balance – evidence for different age and skill groups from a machine learning approach

AbstractMany European regions are currently experiencing a significant population decline and, related to this, are increasingly confronted with labour shortage. Migration is a main driver of changes in regional labour supply and the local level of human capital. A region's ability to attract residents thus becomes more and more important for its growth prospects. We use a large panel dataset for the period 2003 to 2017 to investigate the relationship between local attributes and the migration balance of regions in Germany. In particular, we examine whether the factors that determine the migration balance of regions significantly differ across age and skill groups because their contribution to regional human capital likely varies. Our econometric specification can be understood as an aggregate formulation of a two‐region random utility model. The dataset includes 30 factors that might potentially influence a region's migration balance. Given this large number of explanatory variables and significant multicollinearity issues, we apply machine learning techniques [least absolute shrinkage and selection operator (LASSO), complete subset regression] to identify important local characteristics. Our results point to a robust negative relationship between the net migration rate and population density, yet locations in close proximity to large urban centres seem to be rather attractive destination regions, and the size of the effects differs significantly across age and skill groups. Moreover, labour market conditions and some amenities are significantly correlated with the region's migration balance. However, the former and, in particular, facilities for vocational training matter primarily for young workers.

Unplanned Food Access

Problem, research strategy, and findings U.S. suburbs have increasingly diversified in the past 2 decades. Most working-class and lower income immigrant households settle in older suburbs of metro regions, reversing their decline. In this study I examined the multifaceted contribution of immigrant food entrepreneurs toward the wellbeing of a multiethnic, working-class suburb (Upper Darby, PA). This mixed-methods study relied on historical research, a survey, and field observation to explore how immigrant-run food stores shaped the food environment. The historical research examined the contribution of immigrant food businesses to vacancy and food access over time and found that persistent operation, ownership, and business transfer of ethnic food businesses stabilized the community and provided continuous access to food. A cross-sectional survey of a sample of residents demonstrated that ethnic food businesses contributed to the diversity and density of the food environment, creating a healthier environment and enabling residents to navigate it according to their needs, preferences, and budgets. The sample size was relatively small (n = 115) for this exploratory research design. Insufficient data points and large number of explanatory variables posed challenges for some statistical tests (e.g., multinomial logistic regression), which did not yield strong results. Takeaway for practice Community development stakeholders, including planners, recognize newcomers’ contributions toward labor markets and local economies but have paid less attention to the health effects of immigrant revitalization. This study’s findings call planners’ attention to equitable development practices, such as creating economic inclusion pathways for immigrant populations, as an important strategy to invest in equitable, multicultural, and healthy communities.

Sensitivity analysis of factors relevant to extreme imbalance between procurement plans and actual demand: Case study of the Japanese electricity market

In a liberalized power system, operators control the power system to compensate for imbalances, which are the differences between the scheduled power procurement in electricity markets and the actual power supply. Interpreting the influence of factors that affect extremely large imbalances (i.e., extreme imbalance events) enables power system operators to implement appropriate system operation plans. Conventionally, such imbalance analysis has focused only on a limited number of factors and described variation in imbalance by utilizing highly interpretable statistical models, assuming that the probability of imbalance events varies monotonically with changes in those factors. In particular, sensitivity analysis of such models can be a powerful tool to support the decision-making of system operators. However, when dealing with the increasingly complex behavior of markets involving many actors, a flexible statistical analysis framework is required to identify informative factors among the various observable quantities, and to describe nonmonotonic relationships when essentially necessary. This study focuses on the statistical behavior of the odds ratio of the extreme imbalance events by concentrating on an inherently large number of explanatory variables. The authors propose a model-based approach using a class of partially linear additive models and a variable selection method to reveal the statistical relationships between the relevant variables and extreme imbalance events. The framework further provides an analysis scheme to determine the response sensitivity of relevant variables to the odds ratio of extreme imbalance events. The usefulness of the framework was demonstrated by applying the approach to a real-world dataset collected in the Japanese electricity market system. The results show that the proposed approach based on partially linear additive models works well to describe the extreme imbalance events; the constructed models derive the interpretable sensitivity curves to clarify the impact of informative variables to extreme events, while identifying monotonicity/nonmonotonicity among variables.

Partial Least Squares Analysis in Electrical Brain Activity

Abstract: Partial least squares (PLS) method has been designed for handling two common problems in the data that are encountered in most of the applied sciences including the neuroimaging data: 1) Collinearity problem among explanatory variables (X) or among dependent variables (Y); 2) Small number of observations with large number of explanatory variables. The idea behind this method is to explain as much as possible covariance between two blocks of X and Y variables by a small number of uncorrelated variables. Apart from the other applied sciences in which PLS are used, in the application of imaging data PLS has been used to identify task dependent changes in activity, changes in the relations between brain and behavior, and to examine functional connectivity of one or more brain regions. The aim of this paper is to give some information about PLS and apply on electroencephalography (EEG) data to identify stimulation dependent changes in EEG activity.

Determinants of corruption in developing countries: case of Tunisia

PurposeThis paper aims to investigate the economic, political or sociocultural determinants of corruption in Tunisia.Design/methodology/approachTo better understand the main determinants of corruption in Tunisia. This study uses The Bayesian Model Averaging (BMA) model, which allows us to include a large number of explanatory variables and for a shorter period.FindingsThe results show that economic freedom is the most important variable of corruption in Tunisia. In second place comes the subsidies granted by the government, which is one of the best shelters of corruption in Tunisia through their use for purposes different from those already allocated to them. Third, this paper finds the high unemployment rate, which, in turn, is getting worse even nowadays. The other three factors considered as causal but of lesser importance are public expenditures, the human development index (HDI) and education. Education, the HDI and the unemployment rate are all socio-economic factors that promote corruption.Originality/valueThe realization of this study will lead to triple net contributions. The first is to introduce explicitly and simultaneously political, social and economic determinants of corruption in developing countries. Second, unlike previous studies based on the simple and generalized regression model, the present research uses another novel and highly developed estimation method. More precisely, this study uses the BMA model, on the set of annual data for a period of 1998–2018. The third contribution of this research resides in the choice of the sample.

Portfolio Choices with Many Big Models

This paper proposes a Bayesian-averaging heterogeneous vector autoregressive portfolio choice strategy with many big models that outperforms existing methods out-of-sample on numerous daily, weekly, and monthly datasets. The strategy assumes that excess returns are approximately determined by a time-varying regression with a large number of explanatory variables that are the sample means of past returns. Investors consider the possibility that every period there is a regime change by keeping track of many models, but doubt that any specification is able to perfectly predict the distribution of future returns, and compute portfolio choices that are robust to model misspecification. This paper was accepted by Tyler Shumway, finance.

Using Machine-Learning Methods to Improve Surface Wind Speed from the Outputs of a Numerical Weather Prediction Model

The relationship between the wind speed derived from the outputs of a numerical-weather-prediction model and from observations is explored using statistical and machine-learning models. Eight years of wind-speed measurements at a height of 10 m (from 2010 to 2017) from 171 stations spread over mainland France and Corsica are used for reference. Operational analyses from the European Center for Medium Range Weather Forecasts (ECMWF) provide the model information not only on the surface flow, but on other aspects of the atmospheric state at the location (or above) each station. In a first step, a large number of explanatory variables are used as input to several models (linear regressions, k-nearest neighbours, random forests, and gradient boosting). The modelled wind speed in the ECMWF analyses, by itself, has root-mean-square errors over all stations distributed widely around a median of 1.42 m s $$^{-1}$$ . Using statistical post-processing and making use of a historical dataset for training, the median of the root-mean-square errors at all stations can be reduced down to 1.07 m s $$^{-1}$$ when modelled with linear regressions, and down to 0.94 m s $$^{-1}$$ with the machine-learning models (random forests or gradient boosting). Yet more significant decreases are found for coastal stations where the errors are largest. The random-forest models are further explored to reduce the list of explanatory variables: a list of 25 explanatory variables, mainly consisting of flow variables (wind speed, velocity components, horizontal gradients of geopotential on different isobaric surfaces, wind shear between 10 and 100 m) and including marginally some temperature variables, appears as a good compromise between performance and simplicity. Finally, as a preliminary test for further work, the relation thus captured between the model outputs and the observed wind speed at a given time is applied to forecasts of the numerical-weather-prediction model, for lead times up to 24 h. The machine-learning model is found to be essentially as relevant on the forecasts as it was on the analyses, encouraging further use and development of these approaches for local wind-speed forecasts.

Decomposition of the gender wage gap using the LASSO estimator

ABSTRACT We use the LASSO estimator to select among a large number of explanatory variables in wage regressions for a decomposition of the gender wage gap. The LASSO selection with a one standard error rule removes about a quarter of the regressors. We use the LASSO-selected regressors for OLS-based gender wage decompositions. This approach results in a smaller error variance than in OLS without LASSO-selection. The explained gender wage gap is 1%-point greater than in the conventional OLS model.

Macroeconomic Content of Characteristics-Based Asset Pricing Models: A Machine Learning Analysis

We explore whether the empirical success of seven characteristics-based asset pricing models can be explained by their ability to identify macroeconomic risks. We find that although the stochastic discount factors (SDFs) of some models are weakly related to macroeconomic shocks, the SDFs' non-market components are totally unrelated to them. The result also holds for macroeconomic news and real-time macroeconomic shocks. Our analysis involves a comprehensive set of more than 100 macroeconomic indicators and uses machine learning techniques to mitigate the overfitting problem caused by a large number of explanatory variables. Our paper illustrates how machine learning can be used for analyzing the explainability of one variable by many others.

A fast and consistent variable selection method for high-dimensional multivariate linear regression with a large number of explanatory variables

We put forward a variable selection method for selecting explanatory variables in a normality-assumed multivariate linear regression. It is cumbersome to calculate variable selection criteria for all subsets of explanatory variables when the number of explanatory variables is large. Therefore, we propose a fast and consistent variable selection method based on a generalized $C_{p}$ criterion. The consistency of the method is provided by a high-dimensional asymptotic framework such that the sample size and the sum of the dimensions of response vectors and explanatory vectors divided by the sample size tend to infinity and some positive constant which are less than one, respectively. Through numerical simulations, it is shown that the proposed method has a high probability of selecting the true subset of explanatory variables and is fast under a moderate sample size even when the number of dimensions is large.

Machine learning in agricultural and applied economics

AbstractThis review presents machine learning (ML) approaches from an applied economist’s perspective. We first introduce the key ML methods drawing connections to econometric practice. We then identify current limitations of the econometric and simulation model toolbox in applied economics and explore potential solutions afforded by ML. We dive into cases such as inflexible functional forms, unstructured data sources and large numbers of explanatory variables in both prediction and causal analysis, and highlight the challenges of complex simulation models. Finally, we argue that economists have a vital role in addressing the shortcomings of ML when used for quantitative economic analysis.

The principal problem with principal components regression

Principal components regression (PCR) reduces a large number of explanatory variables in a regression model down to a small number of principal components. PCR is thought to be more useful, the mor...

Modelling vehicular crash mortalities in Ghana

Deaths due to road accidents are a major concern to many stakeholders in Ghana especially because road accidents only come second behind malaria for cause of deaths. Statistical models can be helpful in evaluating the effect of factors responsible for mortality and morbidity during vehicular accide nts. There is often a spoilt for choice on the type of models that may be used to explain a particular phenomenon. Picking a model can be based on the researcher’s knowledge or experience and the simplicity of the model. However, in common applications, the models applied are often not adequate to accurately and efficiently explain the underlying phenomenon particularly when it fails to address certain characteristics of the data. In this paper, an appropriate statistical model on the number of vehicular deaths in Ghana is fitted. The Poisson, Negative Binomial (NB), Zero-Inflation Poisson (ZIP) and Zero-Inflation Negative Binomial (ZINB) models, estimated by the method of maximum likelihood, are compared to determine the most appropriate model for the data at hand. In addition, due to the large number of explanatory variables, the backward model selection procedure was adopted to select the most significant factors associated with crash fatalities. After a careful model building process, the ZINB model was identified as the most appropriate for modelling road crash mortality. The model also identified factors such as shoulder type, time of crash, driver’s sex, road environment landmarks, among others as having significant effect on the fatalities during vehicular accidents in Ghana. It is recommended that authorities focus on installing reflective markings on the shoulders of roads and increase education of drivers in adhering to road regulations while also paying keen attention to road environmental landmarks.

Test by adaptive LASSO quantile method for real-time detection of a change-point

This article proposes a test statistic based on the adaptive LASSO quantile method to detect in real-time a change in a linear model. The model can have a large number of explanatory variables and the errors don’t satisfy the classical assumptions for a statistical model. For the proposed test statistic, the asymptotic distribution under $$H_0$$ is obtained and the divergence under $$H_1$$ is shown. It is shown via Monte Carlo simulations, in terms of empirical sizes, of empirical powers and of stopping time detection, that the useful test statistic for applications is better than other test statistics proposed in literature. Two applications on the air pollution and in the health field data are also considered.

Large numbers of explanatory variables: a probabilistic assessment.

Recently, Cox and Battey (2017 Proc. Natl Acad. Sci. USA 114, 8592–8595 (doi:10.1073/pnas.1703764114)) outlined a procedure for regression analysis when there are a small number of study individuals and a large number of potential explanatory variables, but relatively few of the latter have a real effect. The present paper reports more formal statistical properties. The results are intended primarily to guide the choice of key tuning parameters.