True Regression Function Research Articles

A graphical selection method for parametric models in noisy inhomogeneous regression

A common problem in physics is to fit regression data by a parametric class of functions, and to decide whether a certain functional form allows for a good fit of the data. Common goodness of fit methods are based on the calculation of the distribution of certain statistical quantities under the assumption that the model under consideration {\it holds true}. This proceeding bears methodological flaws, e.g. a good ``fit'' - albeit the model is wrong - might be due to over-fitting, or to the fact that the chosen statistical criterion is not powerful enough against the present particular deviation between model and true regression function. This causes particular difficulties when models with different numbers of parameters are to be compared. Therefore the number of parameters is often penalised additionally. We provide a methodology which circumvents these problems to some extent. It is based on the consideration of the error distribution of the goodness of fit criterion under a broad range of possible models - and not only under the assumption that a given model holds true. We present a graphical method to decide for the most evident model from a range of parametric models of the data. The method allows to quantify statistical evidence {\it for} the model (up to some distance between model and true regression function) and not only {\it absence of evidence} against, as common goodness of fit methods do. Finally we apply our method to the problem of recovering the luminosity density of the Milky Way from a de-reddened {\it COBE/DIRBE} L-band map. We present statistical evidence for flaring of the stellar disc inside the solar circle.

Testing of monotonicity in parametric regression models

In data analysis concerning the investigation of the relationship between a dependent variable Y and an independent variable X, one may wish to determine whether this relationship is monotone or not. This determination may be of interest in itself, or it may form part of a (nonparametric) regression analysis which relies on monotonicity of the true regression function. In this paper we generalize the test of positive correlation by proposing a test statistic for monotonicity based on fitting a parametric model, say a higher-order polynomial, to the data with and without the monotonicity constraint. The test statistic has an asymptotic chi-bar-squared distribution under the null hypothesis that the true regression function is on the boundary of the space of monotone functions. Based on the theoretical results, an algorithm is developed for evaluating significance of the test statistic, and it is shown to perform well in several null and nonnull settings. Extensions to fitting regression splines and to misspecified models are also briefly discussed.

Goodness of fit test for isotonic regression

We consider the problem of hypothesis testing within a monotone regression model. We propose a new test of the hypothesis H 0 : “ ƒ = ƒ 0 ” against the composite alternative H a : “ ƒ ≠ ƒ 0 ” under the assumption that the true regression function f is decreasing. The test statistic is based on the -distance between the isotonic estimator of f and the function f 0 , since it is known that a properly centered and normalized version of this distance is asymptotically standard normally distributed under H 0 . We study the asymptotic power of the test under alternatives that converge to the null hypothesis.

Optimal smoothing in semiparametric index approximation of regression functions

The problem of approximating a general regression function m(x) = E (Y IX = x) is addressed. As in the case of the c1assical L2-type projection pursuit regression considered by Hall (1989), we propose to approximate m(x) through a regression of Y given an index, that is a unidimensional projection of X. The orientation vector defining the projection of X is taken to be the optimum of a Kullback-Leibler type criterion. The first step of the c1assical projection pursuit regression and the single-index models (SIM) are obtained as particular cases. We define a kernel-based estimator of the 'optimal' orientation vector and we suggest a simple empirical bandwidth selection rule. Finally, the true regression function m(•) is approximated through a kernel regression of Y given the estimated index. Our procedure extends the idea of Hardle, Hall and Ichimura (1993) which propose, in the case of SIM, to minimize an empirical L2-type criterion simultaneously with respect to the orientation vector and the bandwidth. We show that a same bandwidth of order n - 1/5 can be used for the root-n estimation of the orientation and for the kernel approximation of the true regression function. Our methodology could be extended to more accurate multi-index approximations.

Goodness of fit test for isotonic regression

We consider the problem of hypothesis testing within a monotone regression model. We propose a new test of the hypothesis H 0 : “ƒ = ƒ 0” against the alternative H a : “ƒ ≠ ƒ 0” under the assumption that the true regression function ƒ is decreasing. The statistic of the test rest on the L 1 -distance between the isotonic estimator of ƒ and the function ƒ 0 in such a way that it is asymptotically standard normally distributed under H 0. We study the asymptotic power of the test under alternatives that converge to the null hypothesis.

Regression of Genotypic on Phenotypic Value of a Ratio-Defined Character

An exact expression for the joint probability density function of the phenotypic and genotypic values is examined for the ratio-defined character in which two component characters are assumed to follow a bivariate normal law and to have positive values. An approximation to the function is derived in terms of parameters of two component characters. A nonlinear approximation to the true regression function of the genotypic value on the phenotypic value is obtained. Similar approximations to the regression functions of the genotypic values of two component characters on the phenotypic value of the ratio-defined character are also given. An example is used to discuss the validity of the given approximations and to illustrate the geometric shapes of the regression functions.

An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression

Abstract Nonparametric regression is a set of techniques for estimating a regression curve without making strong assumptions about the shape of the true regression function. These techniques are therefore useful for building and checking parametric models, as well as for data description. Kernel and nearest-neighbor regression estimators are local versions of univariate location estimators, and so they can readily be introduced to beginning students and consulting clients who are familiar with such summaries as the sample mean and median.

Bump hunting in regression analysis

Computer techniques for scatter-plot smoothing are valuable exploratory analysis tools. However, one must exercise caution when making inferences based on features of the smoothed data. Here, the size and number of observed modes are studied: the asymptotic distributions are determined under the assumption of constancy of the true regression function, and sufficient conditions for consistency of the estimated number of modes and their locations are given in the general case.

Optimal designs for regression models with possible bias

In regression problems, departures of the true regression function from the assumed model can render experimental designs which are optimal for the assumed model inefficient. If departures from the assumed model are possible, designs need to be modified so as to protect against bias introduced by fitting the wrong model. In some instances the experimenter may be able to specify what departures from the assumed model are likely and how likely these departures may be. Such information should be useful in selecting an experimental design. In this paper we investigate such situations by assuming that a prior distribution on possible departures from the assumed model is known. A general model is given and optimal designs are obtained for some polynomial models with certain types of prior distributions over the set of all polynomials.

Model robust extrapolation designs

We seek designs which are optimal in some sense for extrapolation when the true regression function is in a certain class of regression functions. More precisely, the class is defined to be the collection of regression functions such that its ( h + 1)-th derivative is bounded. The class can be viewed as representing possible departures from an ‘ideal’ model and thus describes a model robust setting. The estimates are restricted to be linear and the designs are restricted to be with minimal number of points. The design and estimate sought is minimax for mean square error. The optimal designs for cases X = [0, ∞] and X = [-1, 1], where X is the place where observations can be taken, are discussed.

Simultaneous confidence and prediction intervals for nonlinear regression models with application to a groundwater flow model

Methods are presented for computing three types of simultaneous confidence and prediction intervals (exact, likelihood ratio, and linearized) on output from nonlinear regression models with normally distributed residuals. The confidence intervals can be placed on individual regression parameters or on the true regression function at any number of points in the domain of the independent variables, and the prediction intervals can be placed on any number of future observations. The confidence intervals are analogous to simultaneous Scheffè intervals for linear models and the prediction intervals are analogous to the prediction intervals of Hahn (1972). All three types of intervals can be computed efficiently by using the same straightforward Lagrangian optimization scheme. The prediction intervals can be treated in the same computational framework as the confidence intervals by including the random errors as pseudoparameters in the Lagrangian scheme. The methods are applied to a hypothetical groundwater model for flow to a well penetrating a leaky aquifer. Three different data sets are used to demonstrate the effect of sampling strategies on the intervals. For all three data sets, the linearized confidence intervals are inferior to the exact and likelihood ratio intervals, with the latter two being very similar; however, all three types of prediction intervals yielded similar results. The third data set (time drawdown data at only a single observation well) points out many of the problems that can arise from extreme nonlinear behavior of the regression model.

Strong consistency of nearest neighbor regression function estimators

For a well-known class of nonparametric regression function estimators of nearest neighbor type the uniform measure of deviation from the estimators to the true regression function is studied. Under weak regularity conditions it is shown that the estimators are uniformly consistent with probability one and the corresponding rate of convergence is near-optimal.

Robust designs for nearly linear regression

In this paper we seek designs and estimators which are optimal in some sense for multivariate linear regression on cubes and simplexes when the true regression function is unknown. More precisely, we assume that the unknown true regression function is the sum of a linear part plus some contamination orthogonal to the set of all linear functions in the L 2 norm with respect to Lebesgue measure. The contamination is assumed bounded in absolute value and it is shown that the usual designs for multivariate linear regression on cubes and simplices and the usual least squares estimators minimize the supremum over all possible contaminations of the expected mean square error. Additional results for extrapolation and interpolation, among other things, are discussed. For suitable loss functions optimal designs are found to have support on the extreme points of our design space.

A STUDY OF THE ACCURACY OF SUBKOVIAK'S SINGLE‐ADMINISTRATION ESTIMATE OF THE COEFFICIENT OF AGREEMENT USING TWO TRUE‐SCORE ESTIMATES

Recently, Subkoviak (1976) developed a single-administration procedure for estimating the reliability of a criterion-referenced test composed of dichotomously scored items. The resulting reliability index is termed the coefficient of agreement. The procedure represents an important methodological development for criterion-referenced testing because, in line with suggestions by Hambleton and Novick (1973), the coefficient estimates the proportion of mastery classifications that are consistent on two test administrations, while avoiding the necessity of multiple test administrations. Application of the procedure requires an estimate of each examinee's relative true score (subsequently referred to as true score). The true score is defined as the expected value of the proportion-correct score. Subkoviak (1976) suggests using linear regression true-score estimates, but raises a question about the adequacy of the estimates. Although it is unlikely that the regression of true score on observed score is precisely linear, the regression function should be monotonically non-decreasing. Therefore, a linear regression function should provide a good approximation to the regression function (Dawes and Corrigan, 1974). In particular, when the true-score variance is small, a situation that is common in criterion-referenced testing (Hambleton and Novick, 1973; Popham and Husek, 1969), the approximation of the linear to the true regression function should be quite good. Thus, the use of linear regression estimates may be expected to produce reasonably accurate estimates of the coefficient of agreement.

Posterior Consistency for Coefficient Estimation and Model Selection in the General Linear Hypothesis

Berk (1970), LeCam (1953) and others have given conditions for the consistency of posterior distributions from a sequence of random variables. They have required that the sequence be i.i.d. We show that their results, Berk's in particular, may be extended to the general linear hypothesis with normal errors model (where the sequence of observations of the dependent variable need not be i.i.d.). We do not assume that the distribution governing the sequence of dependent variables has a regression function which satisfies the assumed model nor do we assume its errors are normal. Consistency is shown for both fixed and random sampling designs. We show that the convergence is to a projection of only the true regression function upon the space of regression functions given by the model. Finally, we assume that several such models are under consideration, each with a prior probability. We determine conditions for the a.s. convergence of their posterior probabilities to a degenerate distribution. Not all these conditions may be derived by any simple extension of Berk's results.

Estimates for the Points of Intersection of Two Polynomial Regressions

Abstract Let t* denote an unknown abscissa of intersection of two true regression functions μ1(t) and μ2(t). Under normality assumptions with no restraints on t* the maximum likelihood estimator of t* is shown to be the corresponding intersection of the sample regressions. When this estimate exists confidence intervals J can usually be obtained for t* by an application of the Student t-distribution. When t* is restrained to some known interval I, the ML estimate may or may not fall in I. A restrained ML estimate proposed is the limiting point of ∩I ∩J as the length of I ∩ J approaches zero. Confidence limits are obtained for the restrained estimate. Many practical difficulties are discussed.

Estimates for the Points of Intersection of Two Polynomial Regressions

Abstract Let t* denote an unknown abscissa of intersection of two true regression functions μ1(t) and μ2(t). Under normality assumptions with no restraints on t* the maximum likelihood estimator of t* is shown to be the corresponding intersection of the sample regressions. When this estimate exists confidence intervals J can usually be obtained for t* by an application of the Student t-distribution. When t* is restrained to some known interval I, the ML estimate may or may not fall in I. A restrained ML estimate proposed is the limiting point of ∩I ∩J as the length of I ∩ J approaches zero. Confidence limits are obtained for the restrained estimate. Many practical difficulties are discussed.