Regression Models Augmented with Direct Stochastic Gradient Estimators

Abstract

Traditional regression assumes that the only data available are measurements of the value of the dependent variable for each combination of values for the independent variable. However, in many settings in stochastic (Monte Carlo) simulation, directly estimated derivative information is also available via techniques such as perturbation analysis or the likelihood ratio method. In this paper, we investigate potential modeling improvements that can be achieved by exploiting this additional gradient information in the regression setting. Using least squares and maximum likelihood estimation, we propose various direct gradient augmented regression (DiGAR) models that incorporate direct gradient estimators, starting with a one-dimensional independent variable and then extending to multidimensional input. For some special settings, we are able to characterize the variance of the estimated parameters in DiGAR and compare them analytically with the standard regression model. For a more typical stochastic simulation setting, we investigate the potential effectiveness of the augmented model by comparing it with standard regression in fitting a functional relationship for a simple queueing model, including both one-dimensional and four-dimensional examples. The preliminary empirical results are quite encouraging, as they indicate how DiGAR can capture trends that the standard model would miss. Even in queueing examples where there is a high correlation between the output and the gradient estimators, the basic DiGAR model that does not explicitly account for these correlations performs significantly better than the standard regression model.