Semiparametric Approach Research Articles

Single Functional Index Quantile Regression for Functional Data with Missing Data at Random

The primary goal of this research was to estimate the quantile of a conditional distribution using a semi-parametric approach in the presence of randomly missing data, where the predictor variable belongs to a semi-metric space. The authors assumed a single index structure to link the explanatory and response variable. First, a kernel estimator was proposed for the conditional distribution function, assuming that the data were selected from a stationary process with missing data at random (MAR). By imposing certain general conditions, the study established the model’s uniform almost complete consistencies with convergence rates.

Multiresponse semiparametric regression model approach to standard growth charts design for assessing nutritional status of East Java toddlers

Multiresponse semiparametric regression model approach to standard growth charts design for assessing nutritional status of East Java toddlers

US Economic Policy Uncertainties and Economic Conditions of Fifty (50) US States: A Semi-parametric Smooth Varying-Coefficient Approach

US Economic Policy Uncertainties and Economic Conditions of Fifty (50) US States: A Semi-parametric Smooth Varying-Coefficient Approach

Methods and Software to Analyze Gene-Environment Interactions under a Case-Mother-Control-Mother Design with Partially Missing Child Genotype

Introduction: The case-mother-control-mother design allows to study fetal and maternal genetic factors together with environmental exposures on early life outcomes. Mendelian constraints and conditional independence between child genotype and environmental factors enabled semiparametric likelihood methods to estimate logistic models with greater efficiency than standard logistic regression. Difficulties in child genotype collection require methods handling missing child genotype. Methods: We review a stratified retrospective likelihood and two semiparametric likelihood approaches: a prospective one and a modified retrospective one, the latter either modeling the maternal genotype as a function of covariates or leaving their joint distribution unspecified (robust version). We also review software implementing these modeling alternatives, compare their statistical properties in a simulation study, and illustrate their application, focusing on gene-environment interactions and partially missing child genotype. Results: The robust retrospective likelihood provides generally unbiased estimates, with standard errors only slightly larger than when modeling maternal genotype based on exposure. The prospective likelihood encounters maximization problems. In the application to the association of small-for-gestational-age babies with CYP2E1 and drinking water disinfection by-products, the retrospective likelihood allowed a full array of covariates, while the prospective likelihood was limited to few covariates. Conclusion: We recommend the robust version of the modified retrospective likelihood.

SMALL AREA ESTIMATION OF MEAN YEARS SCHOOL IN KABUPATEN BOGOR USING SEMIPARAMETRIC P-SPLINE

The Fay-Herriot model, generally uses the EBLUP (Empirical Best Linear Unbiased Prediction) method, is less flexible due to the assumption of linearity. The P-Spline semiparametric model is a modification of the Fay-Herriot model which can accommodate the presence of two components, linear and nonlinear predictors. This paper also deals spatial dependence among the random area effects so that a model with spatially autocorrelated errors will be implemented, known as the SEBLUP (Spatial Empirical Best Linear Unbiased Prediction) method. Using data from SUSENAS, PODES, and some publication from BPS, the main objective of this study is to estimate the mean years school at kecamatan level in Kabupaten Bogor using EBLUP, Semiparametric P-Spline approach and SEBLUP method. The results show that based on the RRMSE value, the cubic P-Spline model with three knots predicts the mean years school better than EBLUP. Meanwhile, the addition of spatial effects into the small area estimation has not been able to improve the estimated value of the P-Spline semiparametric approach.

Assessment of non-linearity in calorie-income relationship in Pakistan.

This article considers the issue of assessing non-linearity in the relationship between calorie consumption and income using non-parametric and semi-parametric approaches. These methodologies are implemented on the cross-sectional household survey data conducted in Pakistan in 2010-2011. This framework takes account of the heterogeneity among families and potential non-linearity in the relationship. The findings show that the calorie-income elasticity is considerable and statistically significant across estimating methodologies. The results also demonstrate that the elasticity is larger for the substantially poorer households of the sample. By incorporating the explanatory variables in a manageable way in the parametric section of regression procedures, the semi-parametric analysis also reveals a slight increase in calorie response to increases in income at various income levels.

If You Give Bureaucrats an Inch, Will They Take a Yard? Lessons from Threshold Regulatory Reform in Italy

The role of discretion in public procurement has been the subject of wide debate in regulation literature and practice. Greater discretion potentially may allow for greater efficiency in procurement. However, when social capital and institutional quality are low, more discretion can easily open the door to bribery and corruption. In this paper, adopting a semiparametric approach and using a large dataset of public works managed by Italian municipalities, we empirically assess the effects of a reform in the regulation of public works in Italy, approved in 2011, that has increased the discretion of bureaucrats in the choice of the firms. The results provide evidence that the reform has exerted a positive, although mild, effect on the performance of public works execution. However, the positive role of the reform is more relevant in those areas where the effect of public service norms and prosocial behaviors are higher.

A Semiparametric Approach to Model-Based Sensitivity Analysis in Observational Studies.

When drawing causal inference from observational data, there is almost always concern about unmeasured confounding. One way to tackle this is to conduct a sensitivity analysis. One widely-used sensitivity analysis framework hypothesizes the existence of a scalar unmeasured confounder U and asks how the causal conclusion would change were U measured and included in the primary analysis. Work along this line often makes various parametric assumptions on U, for the sake of mathematical and computational convenience. In this article, we further this line of research by developing a valid sensitivity analysis that leaves the distribution of U unrestricted. Compared to many existing methods in the literature, our method allows for a larger and more flexible family of models, mitigates observable implications (Franks et al., 2019), and works seamlessly with any primary analysis that models the outcome regression parametrically. We construct both pointwise confidence intervals and confidence bands that are uniformly valid over a given sensitivity parameter space, thus formally accounting for unknown sensitivity parameters. We apply our proposed method on an influential yet controversial study of the causal relationship between war experiences and political activeness using observational data from Uganda.

Semiparametric estimation of expected shortfall and its application in finance

AbstractMeasuring risk effectively is crucial for managing risk in financial markets. The expected shortfall has become an increasingly popular metric for risk in recent years. How to estimate it is important in statistics and financial econometrics. Based on the single index quantile regression, we introduce a new semiparametric approach, namely, weighted single index quantile regression. We assess the performance of the proposed expected shortfall estimator with backtesting. Our simulation results indicate that the estimator has a good finite sample performance and often outperforms existing methods. By applying the new method to both a market index and individual stocks, we show that it not only exhibits the best performance but also reveals an insight about the effect of the COVID pandemic, that is, the pandemic increases the market risk.

Penalized splines model to estimate time-varying reproduction number for Covid-19 in India: A Bayesian semi-parametric approach

Statistical modelling is pivotal in assessing intensity of a stochastic processes. Novel Corona virus disease demanded proactive measures to understand the severity of disease spread and to plan its control accordingly. We propose estimation of reproduction number as a crucial factor to monitor the random dynamics of Covid-19 in India. In the present paper, semi-parametric regression based on penalized splines embedded under Bayesian formulation is utilised to estimate reproduction number while incorporating effects of underreporting and delay in reporting for the actual number of daily occurrences. Monte Carlo Markov Chain approximations are utilised to perform simulation study and thereby to assess the impact of the reporting probability and misspecification of delay pattern on potential for further substance of the pandemic. For a cycle of reporting on weekly basis, the proposed penalized spline Bayesian framework fits closest to the empirical data drawn for a two-day delay in reporting with approximately half of the actual cases being reported. The present paper is a contribution towards estimation of the true daily reproduction number of Covid-19 incidences in its next generation cycle.

Nonparametric Approach to Copula Estimation in Compounding The Joint Impact of Storm Surge and Rainfall Events in Coastal Flood Analysis

The joint probability modelling of storm surges and rainfall events is the main task in assessing compound flood risk in low-lying coastal areas. These extreme or non-extreme events may not be dangerous if considered individually but can intensify flooding impact if they occur simultaneously or successively. Recently, the copula approach has been widely accepted in compound flooding but is often limited to parametric or semiparametric distribution settings in a limited number of cases. However, both parametric and semiparametric approaches assume the prior distribution type for univariate marginals and copula joint density. In that case, there is a high risk of misspecification if the underlying assumption is violated. In addition, both approaches suffer from a lack of flexibility. This study uses bivariate copula density in the nonparametric distribution setting. The joint copula structure is approximated nonparametrically by employing the Bernstein copula estimator and Beta kernel copula density, and their performances are also compared. The proposed model is tested with 46 years of rainfall and storm surge observations collected on Canada's west coast. The marginal distribution of the selected flood variables is modelled using nonparametric kernel density estimation (KDE). Based on the different model compatibility tests, the Bernstein copula with normal KDE margins defined the joint dependence structure well. The selected nonparametric copula model is further employed to estimate joint and conditional return periods. It is found that flood hazard characteristics occurrence simultaneously is less frequent in AND-joint cases than in OR-joint cases. Also, the derived model is further used to estimate failure probability (FP) statistics to assess the variation of bivariate hydrologic risk during the project lifetime. It is found that FP statistics could be underestimated when neglecting the compound effect of storm surge and rainfall in the coastal flood risk.

Application of Bayesian Semi-Parametric and Hierarchical Models for Analyzing Dispersed Traffic Barriers Crash Data

Introduction: Despite the traffic barriers effectiveness in reduction of the severity of run-off road crashes, the severity of barrier crashes still accounts for a significant fraction of road fatalities. Although extensive research has already been conducted in studying traffic barrier crashes, those studies mostly either consider the severity or frequency of crashes. Here, the equivalent property damage only (EPDO) was used to account for both aspects of crashes. While modeling EPDO crashes, there are challenges associated with that type of dataset including its sparse distribution, and the presence of heterogeneity in the dataset due to aggregation of various crash types. Methods: Ignoring the sparse nature of the data might result in biased or even erroneous results. Thus, in this study we identify factors to barriers EPDO crashes while considering the discussed challenges. Those consideration are especially important as in the next step we will employ the modeling results for conducting the cost-benefit analysis. Two main methods were considered in this study to address the discussed challenges including parametric and non-parametric Bayesian hierarchical models. A semiparametric Bayesian approach was used to relax the normality assumption by using a mixture of multivariate Dirichlet prior, defining a flexible nonparametric model for the random effects’ distribution, and using grouping to account for the heterogeneity due to the structure of the dataset. On the other hand, Bayesian hierarchical models with two distributions of Poisson and negative binomial with similar levels of hierarchy were considered. These models were chosen as closest models to the Bayesian semiparametric model. The incorporated models were compared in terms of deviance information criterion (DIC). Results and Discussion: The results highlighted that although the semi-parametric method outperforms the Bayesian hierarchical model with Poisson distribution, the Bayesian hierarchical model with negative binomial (NB) distribution outperform the semi-parametric model. The findings might be related to the severe sparse nature of the EPDO, which cannot optimally be accounted by semiparametric approach, and the model needs more flexibility. Conclusion: It was found that being unrestrained, driving in interstate system, driving in clear weather, light conditions, and driving in a higher traffic all increase the likelihood of EPDO crashes. Also, while some predictors were significant in less accommodative models of semi-parametric or Poisson models, they were not for Negative binomial model.

Semiparametric estimation of INAR models using roughness penalization

Popular models for time series of count data are integer-valued autoregressive (INAR) models, for which the literature mainly deals with parametric estimation. In this regard, a semiparametric estimation approach is a remarkable exception which allows for estimation of the INAR models without any parametric assumption on the innovation distribution. However, for small sample sizes, the estimation performance of this semiparametric estimation approach may be inferior. Therefore, to improve the estimation accuracy, we propose a penalized version of the semiparametric estimation approach, which exploits the fact that the innovation distribution is often considered to be smooth, i.e. two consecutive entries of the PMF differ only slightly from each other. This is the case, for example, in the frequently used INAR models with Poisson, negative binomially or geometrically distributed innovations. For the data-driven selection of the penalization parameter, we propose two algorithms and evaluate their performance. In Monte Carlo simulations, we illustrate the superiority of the proposed penalized estimation approach and argue that a combination of penalized and unpenalized estimation approaches results in overall best INAR model fits.

Characterizing height-diameter relationships for Caribbean trees using mixed-effects random forest algorithm

Accurately characterizing the relationships between tree height and diameter at breast height (DBH) is vital to quantify productivity and predict tree growth for forest management. This study makes use of data from forest inventories in the Caribbean islands of Puerto Rico, Trinidad and the U.S. Virgin Islands to compare the efficacy of parametric mixed models, random forest and mixed-effects random forest algorithms (MERF) for modeling tree species-specific H-D relationships. MERF as a semi-parametric approach is advantageous for handling high dimensional and complex datasets using the random forest algorithm, as well as to provide species-specific predictions with random effects in the mixed model framework. A total of 37,124 observations used in analyses were collected from 645 permanent sample plots across three Caribbean islands, including 309 species in Puerto Rico, 121 species in the U.S. Virgin Islands and 34 species in Trinidad.Results showed that MERF can be used to reliably predict total tree height for diverse species, which produced more accurate predictions than the other two models. In addition to DBH, competition index was found to be the most important variable among all other stand and environmental variables in all islands. Adding climate-related variables can improve the prediction accuracy of total tree height in parametric mixed models, especially when sample size is small. Precipitation variables were selected most often, confirming the overall importance of precipitation to forest height. The results of this work will not only provide species-specific H-D relationship models for Caribbean trees, but also advance knowledge in modeling H-D relationships with broad application for mixed-species forests.

Modeling peak electricity demand: A semiparametric approach using weather-driven cross-temperature response functions

We propose a novel method to model daily peak electricity demand using temperature and additional hourly and daily weather covariates, such as humidity and wind speed. Rather than enter into the temperature response function additively, the additional covariates may flexibly impact the demand response to temperature. Such flexibility allows differential responses to the actual temperature based on the heat index and wind chill factor, for example. Most notably, we find that ignoring humidity substantially underestimates the effect of high temperatures, while ignoring the effect of cloud cover overestimates the effect of low temperatures. Time of day also matters: a demand response to the same temperature may be different at different times of day. Moreover, accounting for weather-related covariates improves the model’s ability to explain daily peak demand.

Bayesian semiparametric approach to quantile nonlinear dynamic factor analysis models with mixed ordered and nonignorable missing data

In classical nonlinear dynamic factor analysis models (NDFAMs), we assume manifest variables follow the normal distribution. However, in some applications, the normality assumption may be unreasonable or questionable. This paper proposes a quantile NDFAM in which manifest variables may be observed or unobserved but associated with the observed ordinal indicators or subject to missingness, and develops a Bayesian semiparametric method to estimate unknown parameters and latent variables by utilizing the Dirichlet process prior to approximate unknown distribution of random effects. Logistic and probit models are adopted to specify propensity scores for missing manifests and covariates, respectively. An exponential family distribution is assumed for missing covariates. Bayesian local influence is presented to assess the effect of minor perturbations to data, priors, and sampling distributions. Several simulation studies and a real example are illustrated by the proposed methodologies.

Early life environments and cognition in adulthood: New evidence using a semiparametric approach and quantile regression

Theories and empirical evidence document the importance of early life environmental factors on later life cognition. A next question is how and in what dimension associations between early life environments and later life cognition vary. Using data from the UK Biobank in conjunction with time-place-specific infant mortality rates, we assessed heterogeneous and non-linear associations between early life conditions and later life cognition. We found that the association between the infant mortality rate and later life cognition increased once the UK achieved very low infant mortality rates, suggesting that additional decreases in infant mortality rates in an industrialized society continue to improve later life cognition. We also found that infant mortality rates have stronger effects at upper quantiles of the cognition distribution. This implies that adverse early life environments may have an important role for an early manifestation of cognitive aging.

Semiparametric multinomial mixed-effects models: A university students profiling tool

Many applicative studies deal with multinomial responses and hierarchical data. Performing clustering at the highest level of grouping, in multilevel multinomial regression, is also often of interest. In this study we analyse Politecnico di Milano data with the aim of profiling students, modelling their probabilities of belonging to different categories and considering their nested structure within engineering degree programmes. In particular, we are interested in clustering degree programmes standing on their effects on different types of student career. To this end, we propose an EM algorithm for implementing semiparametric mixed-effects models dealing with a multinomial response. The novel semiparametric approach assumes the random effects to follow a multivariate discrete distribution with an a priori unknown number of support points, that is, allowed to differ across response categories. The advantage of this modelling is twofold: the discrete distribution on random effects allows, first, to express the marginal density as a weighted sum, avoiding numerical problems in the integration step, typical of the parametric approach, and, second, to identify a latent structure at the highest level of the hierarchy where groups are clustered into subpopulations.

Domestic total factor productivity with trade and heterogenous foreign direct investment in developing countries: a case of Vietnamese manufacturing

PurposeThis paper aims to examine spillover effects of heterogenous foreign direct investment (FDI) enterprises (domestic vs. export-oriented) through horizontal and vertical linkages and absorptive capacity effect on domestic firms' total factor productivity (TFP). It clarifies the spillover effect on domestic firms in accordance with industrial zones, business size, technology sector and geographical agglomeration, respectively.Design/methodology/approachThe dataset used is based on Vietnamese manufacturing firms during 2011–2014, input–output (I–O) Table 2012. This paper is conducted in two steps: (1) TFP is estimated by using a semi-parametric approach developed by Levinsohn and Petrin (2003); (2) Regression with panel data for domestic firms, applying the fixed effect method.FindingsIn terms of domestic-oriented FDI (DFDI) enterprise group: TFP spillover through horizontal linkages is found negative for domestic firms but positive for those participating in export. Additionally, backward linkages have a negative impact on TFP for most domestic enterprises, except for those operating in the high-tech sector. In terms of export-oriented FDI (EFDI) enterprise group, horizontal linkages have a negative impact on domestic firms' TFP including domestic ones participating in export whereas backward linkage is an important channel with positive effects. Absorptive capacity enables firms to improve productivity through linkages with EFDI and DFDI enterprises. Exporters located in industrial zones or regions with numerous exporters can receive better impacts through backward linkages EFDI.Originality/valueComprehensively, this is the first paper to detect FDI heterogeneity in their behavior when entering a developing country like Vietnam. The added value in this study comes from the export ability of local firms which is in line with Melitz (2003) theory that they can excel in absorping the TFP spillover from competing with DFDI competitors or from supplying to EFDI enterprises. Moreover, the role of small and medium-sized enterprises (SMEs), low technology, high technology and learning by regions affecting the impact through both horizontal and vertical linkages are included for analysis.

Dynamics of national development and regional disparity: evidence from 184 countries

PurposeThe existence of the regional Kuznets curve, i.e. an inverted U-shaped relationship between regional disparity and economic development is widely debated and discussed. The bell-shaped curve of the spatial growth process where during the initial phase inequality increases and then reduces is theoretically supported by Myrdal (1957), Hirschman (1958), and Williamson (1965). It becomes important to understand regional Kuznets curve globally. Understanding the relationship between regional disparity and economic development becomes essential for public policy for balanced regional growth.Design/methodology/approachRegional Kuznets Curve which is an inverted U-shaped relationship between regional disparity and economic development is not a new phenomenon. Theoretical framework by Myrdal (1957), Hirschman (1958), and Williamson (1965) support the an inverted U-shaped relationship. To understand the relationship between regional disparity and economic development, the authors investigate the regional Kuznets curve by using data for 184 countries and 1765 subnational regions. Using parametric, semi-parametric, and non-parametric, it is found that there exists an inverted U-shaped relationship between regional disparity and economic development. The presence of the regional Kuznets curve is observed. As the theoretical framework suggests, regional inequality increases with income initially and decreases after attaining a certain level of income. This study identifies two stages of divergence-convergence where in the first stage, divergence across regions in a country happens with increasing income and in the later stage, convergence across regions in a country occurs with increasing income.FindingsUsing the parametric approach (panel data analysis), semi-parametric and non-parametric approaches, it is found that there exists a regional Kuznets curve. It is found that there exists an inverted-U relationship between regional inequality and per capita GNI. This suggests that the divergence-convergence passes through two stages. In the first stage, divergence across regions in a country happens with increasing income while in the later stage convergence occurs.Originality/valueThis research work has done three important things which fill the research gap that exists in the literature: (1) constructing the Gini coefficient to measure the regional inequality for 184 countries using 1765 subnational regional data; (2) using a parametric approach (panel data analysis) to understand the regional Kuznets phenomenon and (3) using a semi-parametric approach and non-parametric approach to understand the regional Kuznets phenomenon.