Parametric Versus Semi and Nonparametric Regression Models

Hamdy F F Mahmoud

doi:10.5539/ijsp.v10n2p90

Abstract

There are three common types of regression models: parametric, semiparametric and nonparametric regression. The model should be used to fit the real data depends on how much information is available about the form of the relationship between the response variable and explanatory variables, and the random error distribution that is assumed. Researchers need to be familiar with each modeling approach requirements. In this paper, differences between these models, common estimation methods, robust estimation, and applications are introduced. For parametric models, there are many known methods of estimation, such as least squares and maximum likelihood methods which are extensively studied but they require strong assumptions. On the other hand, nonparametric regression models are free of assumptions regarding the form of the response-explanatory variables relationships but estimation methods, such as kernel and spline smoothing are computationally expensive and smoothing parameters need to be obtained. For kernel smoothing there two common estimators: local constant and local linear smoothing methods. In terms of bias, especially at the boundaries of the data range, local linear is better than local constant estimator.&nbsp; Robust estimation methods for linear models are well studied, however the robust estimation methods in nonparametric regression methods are limited. A robust estimation method for the semiparametric and nonparametric regression models is introduced.

Highlights

The aim of this paper is to answer many questions that researchers may have when they fit a real data, such as what are the differences between parametric, semi, and nonparamertic models? which one should be used to model a real data set? what estimation method should be used? and which modeling approach is better? These questions and others are addressed by examples in this paper and the R code for plots and analyses presented are available in the Appendix.Assume that a researcher collected data about a response variable, y, and k explanatory variables, (x1, x2, . . . , xk)
single index model (SIM) is more flexible compared to parametric models and does not lack from the curse of dimensionality problem compared to nonparametric models
It assumes that the link between the response and the explanatory variables is unknown and should be estimated nonparametrically. This gives the single index model two main advantages over parametric and nonparametric models: (1) It avoids misspecifying the link function and its misleading results (Horowitz and Hardle, 1996) and (2) the reduction of dimension which is achieved by assuming the link function to be a univariate function applied to the projection of explanatory covariate vector onto some direction

Summary

Introduction

The aim of this paper is to answer many questions that researchers may have when they fit a real data, such as what are the differences between parametric, semi, and nonparamertic models? which one should be used to model a real data set? what estimation method should be used? and which modeling approach is better? These questions and others are addressed by examples in this paper and the R code for plots and analyses presented are available in the Appendix.Assume that a researcher collected data about a response variable, y, and k explanatory variables, (x1, x2, . . . , xk). The aim of this paper is to answer many questions that researchers may have when they fit a real data, such as what are the differences between parametric, semi, and nonparamertic models? Which one should be used to model a real data set? The model choice from parametric, semiparametric or nonparametric regression model depends on the prior knowledge about the functional form of the relationship, f (·, ·), and the random error distribution. If the form is known and it is correct, the parametric choice can model the data set well. The most common functional form is the linear model, as a type of parametric regression, which is frequently used to describe the relationship between a dependent variable and explanatory variables. Parametric linear models require the estimation of a finite number of parameters, β

Objectives

Methods

Conclusion