Rejoinder: Fisher Lecture: Dimension Reduction in Regression

Abstract

I am grateful to all of the discussants for their com ments which raise a number of important and insight ful issues, and add significantly to the breadth of ideas. Following a few introductory comments on the need for a new regression genre that centers on dimension reduction, I turn to the discussants' remarks. The development in the 1960s and early 1970s of diagnostic methods for regression produced a major shift in regression methodology. When a diagnostic produces compelling evidence of a deficiency in the current model or data it is natural to pursue remedial action, leading to a new model and a new round of di agnostics, proceeding in this way until the required di agnostic checks are passed. By the late 1970s this type of iterative model development paradigm was widely represented in the applied sciences and was formal ized in the statistical literature by Box (1979, 1980) and Cook and Weisberg (1982). With the availability of desktop computing starting in the mid-1980s, it is now possible to apply in reasonable time batteries of graph ical and numerical diagnostics to many regressions. Advances in computing and other technologies now allow scientists to routinely formulate regressions in which the number p of predictors is considerably larger than that normally considered in the past. Such large-/? regressions necessitate a new type of analysis for at least two reasons. First, the standard iterative par adigm for model development can become untenable when p is large. Recognizing the variety of graphi cal diagnostics that could be used and the possibility of iteration, a thorough analysis might require assess ment of many plots in addition to various numerical diagnostics. Experience has shown that the paradigm can often become imponderable when applied with too many predictors. Second, in some regressions, partic ularly those associated with high-throughput technolo gies, the sample size n may be smaller than p, lead ing to operational problems in addition to ponderability