Reduced Rank Regression Model Research Articles

Reduced rank regression models with latent variables in Bayesian functional data analysis

In functional data analysis (FDA) it is of interest to generalize techniques of multivariate analysis like canonical correlation analysis or regression to functions which are often observed with noise. In the proposed Bayesian approach to FDA two tools are combined: (i) a special Demmler-Reinsch like basis of interpolation splines to represent functions parsimoniously and flexibly; (ii) latent variable models initially introduced for probabilistic principal components analysis or canonical correlation analysis of the corresponding coefficients. In this way partial curves and non-Gaussian measurement error schemes can be handled. Bayesian inference is based on a variational algorithm such that computations are straight forward and fast corresponding to an idea of FDA as a toolbox for explorative data analysis. The performance of the approach is illustrated with synthetic and real data sets.

Fixed rank solutions of the matrix equation with statistical applications

By applying the canonical correlation decomposition of matrix pairs, the general fixed rank least square solutions of matrix equation Xβ=Y are derived. As statistical applications, an algorithm for computing the least square estimator of the multivariate reduced rank regression model Y=Xβ+ϵ, r(β)=t is given.

Asymptotic expansions and higher order properties of semi-parametric estimators in a system of simultaneous equations

Asymptotic expansions are made for the distributions of the Maximum Empirical Likelihood (MEL) estimator and the Estimating Equation (EE) estimator (or the Generalized Method of Moments (GMM) in econometrics) for the coefficients of a single structural equation in a system of linear simultaneous equations, which corresponds to a reduced rank regression model. The expansions in terms of the sample size, when the non-centrality parameters increase proportionally, are carried out to O ( n − 1 ) . Comparisons of the distributions of the MEL and GMM estimators are made. Also, we relate the asymptotic expansions of the distributions of the MEL and GMM estimators to the corresponding expansions for the Limited Information Maximum Likelihood (LIML) and the Two-Stage Least Squares (TSLS) estimators. We give useful information on the higher order properties of alternative estimators including the semi-parametric inefficiency factor under the homoscedasticity assumption.

Seemingly unrelated reduced-rank regression model

Reduced-rank regression models proposed by Anderson [1951. Estimating linear restrictions on regression coefficients for multivariate normal distributions. Ann. Math. Statist. 22, 327–351] have been used in various applications in social and natural sciences. In this paper we combine the features of these models with another popular, seemingly unrelated regression model proposed by Zellner [1962. An efficient method of estimating seemingly unrelated regressions and tests for aggregation bias. J. Amer. Statist. Assoc. 57, 348–368]. In addition to estimation and inference aspects of the new model, we also discuss an application in the area of marketing.

Forecasting performance of multivariate time series models with full and reduced rank: an empirical examination

Since Quenouille's influential work on multiple time series, much progress has been made towards the goal of parameter reduction and model fit. Relatively less attention has been paid to the systematic evaluation of out-of-sample forecast performance of multivariate time series models. In this paper, we update the hog data set studied by Quenouille (and other researchers who followed him). We re-estimate his model with extended observations (1867–1966), and generate recursive one- to four-steps-ahead forecasts for the period of 1967 through 2000. These forecasts are compared to forecasts from an unrestricted vector autoregression, a reduced rank regression model, an index model and a cointegration-based error correction model. The error correction model that takes into account both nonstationarity of the data and rank reduction performs best at all four forecasting horizons. However, differences among competing models are statistically insignificant in most cases. No model consistently encompasses the others at all four horizons.

The customer satisfaction in a reduced rank regression framework

In this paper we propose a methodology for the estimation of customer satisfaction conceived as a latent variable specified in the American Customer Satisfaction Index (ACSI) structural model. The current proposal puts forward: the approaches of structural equations, since it involves the dual problem of indeterminacy of the latent scores and the normality assumed; the PLS approach, because of its drawbacks (shown in depth in the paper). The ACSI model will be estimated in a reduced rank regression (RRR) framework, showing that under a non restrictive hypothesis, shared by PLS, the structural model can be viewed as a RRR model between two blocks of manifest variables. Finally, in the paper an application is shown to assess the students’ satisfaction in respect to the service of a big real estate agency, operating in the houses‐to‐let market, in Bologna (Italy) for 2002.

Tests of Rank in Reduced Rank Regression Models

There has recently been renewed research interest in the development of tests of the rank of a matrix. This article evaluates the performance of some asymptotic tests of rank determination in reduced rank regression models together with bootstrapped versions through simulation experiments. The bootstrapped procedures significantly improve on the performance of the corresponding asymptotic tests. The article also presents a Monte Carlo exercise comparing the forecasting performance of reduced rank and unrestricted vector autoregressive (VAR) models in which the former appear superior. The tests of rank considered here are then applied to construct reduced rank VAR models for leading indicators of U.K. economic activity. These more parsimonious multivariate representations display an improvement in forecasting performance over that of unrestricted VAR models.

Latent variable multivariate regression modeling

The latent variable multivariate regression (LVMR) model is made up of two sets of variables, X and Y, both of which contain a latent variable structure plus random error. The wide applicability of this model is illustrated in this paper with several real examples. The chemometrics community has developed several empirical methods to estimate the latent structure in this model, including partial least squares regression (PLS) and principal components regression (PCR). However, the majority of the statistical work in this area relies on the standard or reduced rank regression models, thus ignoring the latent variable nature of the X data. Considering methods like PLS and PCR in the context of these models has led to some misleading conclusions. This paper reaffirms the claim made frequently in the chemometrics literature that the reason PLS and PCR have been successful is that they take into account the latent variable structure in the data. It is also shown through several examples that the LVMR model provides the means to model more effectively many datasets in applied science resulting in improved techniques for process monitoring, experimental design and prediction. The focus in this paper is on the general model rather than on parameter estimation methods.

Multivariate prediction for QSAR

The biological activity of a chemical substance is often characterized by more than just one measured variable. In a QSAR (quantitative structure–activity relationships) context, this implies that one would like to predict several responses for given physico–chemical predictors. Reduced rank regression models are multivariate prediction methods which take advantage of the multivariate nature of the response and are easily interpretable. Prediction criteria for the selection of the rank, that is the dimensionality of the model, are discussed. Simulations and an example suggest that reduced rank regression can give better predictions than univariate methods or the multivariate competitor partial least squares.

A Reduced Rank Regression Approach to Tests of Asset Pricing

Both the Arbitrage Pricing Theory (APT) and the Capital Asset Pricing Model (CAPM) place restrictions of the cross‐sectional variation of conditional expectations of asset returns and of macro indicators. We show that these restrictions imposed on the reference statistical models lead to special cases of the reduced rank regression model. The maximum likelihood problem is solved by canonical correlation analysis. Likelihood ratio tests about the number of factors underlying stock returns are straightforward to calculate, thus allowing discrimination between competing financial theories. Moreover LR tests on the relevance of each macroeconomic indicator within a chosen model can be implemented. Some of the tests are illustrated by an application to Italian stock market data.

Reduced Rank Regression Using Generalized Method of Moments Estimators with Extensions to Structural Breaks in Reduced Rank Models

Generalized Method of Moments (GMM) Estimators are derived for Reduced Rank Regression Models, the Error Correction Cointegration Model (ECCM) and the Incomplete Simultaneous Equations Model (INSEM). The GMM (2SLS) estimators of the cointegrating vector in the ECCM are shown to have norma limiting distributions. Tests for the number of unit roots can be constructed straightforwardly and have Dickey-Fuller type limiting distributions. Two extensions of the ECCM, which are important in practice, are analyzed. First, cointegration estimators and tests allowing for structural shifts in the variance (heteroscedasticity) of the seriesare derived and analyzed using both a Generalized LeastSquares Estimator and a White Covariance Matrix Estimator.The resulting cointegrating vectors estimators have again normal limiting distributions while the cointegration tests have identical limiting distributions which differ from the Dickey-Fuller type. Second, cointegrating vector estimators and tests are derived which allow for structural breaks in the cointegrating vector and/or multiplicator. The limiting distributions of the estimators are again shown to be normal and the limiting distributions of the cointegration tests differ from the Dickey-Fuller type.

Bayesian reduced rank regression in econometrics

The reduced rank regression model arises repeatedly in theoretical and applied econometrics. To date the only general treatment of this model have been frequentist. This paper develops general methods for Bayesian inference with noninformative reference priors in this model, based on a Markov chain sampling algorithm, and procedures for obtaining predictive odds ratios for regression models with different ranks. These methods are used to obtain evidence on the number of factors in a capital asset pricing model.

Bayesian analysis of reduced rank regression

Multivariate regression is a well known method to relate multivariate responses to multivariate predictors. When the responses follow a factor analysis model and the latent factors can be linked with the predictors, a reduced rank regression model is obtained which is defined as a multivariate regression model with rank restrictions on the coefficient matrix. Many linear multivariate methods can be interpreted as special cases of such a reduced rank regression model. A Bayesian analysis with informative conjugate priors is presented, emphasizing the importance of an adequate parametrization to obtain interpretable results. Gibbs sampling a Markov chain Monte Carlo method, is proposed to calculate the posterior and predictive distribution. The methods are motivated and illustrated by an example where quantitative relationships between biological activity and chemical structure are searched for (QSAR).

The APT Model as Reduced-Rank Regression

Integrating the two steps of an arbitrage pricing theory (APT) model leads to a reduced-rank regression (RRR) model. So the results on RRR can be used to estimate APT models, making estimation very simple. We give a succinct derivation of estimation of RRR, derive the asymptotic variance of RRR estimators for a general case, and discuss how undersized samples (more assets than time periods) can be dealt with.

A Reduced Rank Regression Model for Local Variation in Solar Radiation

A Reduced Rank Regression Model for Local Variation in Solar Radiation

Estimating a reduced rank regression model for non‐normal variables

AbstractA method using third order moments for estimating the regression coefficients as well as the latent state scores of the reduced‐rank regression model when the latent variable(s) are non‐normally distributed is presented in this paper. It is shown that the factor analysis type indeterminacy of the regression coefficient matrices is eliminated. A real life example of the proposed method is presented. Differences of this solution with the reduced‐rank regression eigen solution are discussed.

Fitting longitudinal reduced-rank regression models by alternating least squares

An alternating least squares method for iteratively fitting the longitudinal reduced-rank regression model is proposed. The method uses ordinary least squares and majorization substeps to estimate the unknown parameters in the system and measurement equations of the model. In an example with cross-sectional data, it is shown how the results conform closely to results from eigenanalysis. Optimal scaling of nominal and ordinal variables is added in a third substep, and illustrated with two examples involving cross-sectional and longitudinal data.

Inference on covariance matrices under rank restrictions

In reduced-rank regression, a matrix of expectations is modeled as a lower rank matrix. In factor analysis, a covariance matrix is modeled as a linear function of a diagonal matrix and a lower rank matrix. A common set of test procedures can be applied to both models when the following conditions are met: the rank of the lower rank matrix is small, the diagonal matrix in the factor model is an identity, and the reduced-rank regression model is not replicated. This paper gives three results for making inferences under these conditions: (1) It is shown that locally best invariant test of sphericity in the factor analysis model is identical to the locally best invariant test of rank-O against rank- r in the reduced-rank model, provided that r is small. (2) An extended table of null percentiles of the likelihood ratio test statistic for rank-1 alternatives in both models is computed. (3) Confidence intervals for parameters in a rank-1 alternative are given. The performance of the proposed methods is evaluated in a simulation study. The methods are illustrated on two-way fixed and mixed effects classifications.

Reduced-rank models for interaction in unequally replicated two-way classifications

This paper extends reduced-rank regression models for application to interaction in unequally replicated two-way classifications having no empty cells. The a × b matrix of interaction parameters is modeled as a linear function of a rank-ϱ matrix, where ϱ ≤ min[( a − 1), ( b − 1)]. For fixed ϱ, the maximum likelihood estimator of the interaction matrix is derived. The likelihood ratio test of ϱ = m versus ϱ = m + r and a union-intersection test of ϱ = 0 versus ϱ ≥ r are constructed and illustrated. The union-intersection test is particularly useful for simultaneous inference on product interaction contrasts.

Procedures for Reduced-Rank Regression

SUMMARY We discuss in this paper procedures for the analysis of the reduced-rank regression model. A new method is proposed for parameter estimation which is justified by a least- squares analysis employing matrix singular-value decomposition and the Eckart-Young theorem. The application of the model is illustrated by the regression analysis of gasoline distillation measurements on composition data obtained by gas-liquid chromatography. IN the study of the experimental properties of mixtures, a linear model is often proposed to relate response to composition. The statistical technique of linear regression analysis is then appropriate and it is often applied severally when there are a number of responses of interest. Now the responses are often inter-related, so that, for instance, it may be possible to use an empirical linear relationship to predict the approximate value of a certain response from a knowledge of the others. The procedure for determining the regression coefficients of response on composition should in these circumstances be modified to reflect the known presence of such relationships (whose linearity is implied by the mutual linear dependence of responses on composition). This leads to consideration of the multivariate regression model with a constraint imposed on the rank of the matrix of coefficients, sometimes termed reduced-rank regression. Such models have been studied, e.g. by Izenman (1975) and also by Burket (1964), who used a factor analysis model. We have applied reduced-rank regression to the investigation of hydrocarbon fuels, whose composition is routinely characterized by the technique of gas-liquid chromatography (g.l.c.). The responses are, in the main, engine test ratings (each having a nominal repeatability or associated precision), which are often known to be correlated or otherwise interrelated. They are, for instance, frequently known to be critically dependent on the same portion of the g.l.c. chromatogram. The precision of the g.l.c. determination is quite high and the resulting data vector may be regarded as deterministic. The test results on the other hand typically exhibit a high degree of scatter. Under these circumstances a multivariate regression model was considered appropriate. The rank restriction was employed to ascertain whether a relatively small number of specific features of the chromatogram could be used to predict fuel properties, and to identify these features. It is not unusual for chromatographic data on a hydrocarbon fuel to comprise a very large number of resolved peaks, typically 200-300. This number is usually much larger than the number of fuel samples available or economically justified for experimentation. A composition vector corresponding to a chromatogram with good resolution will contain a large number of components, many of which may be effectively zero. A reduction to a smaller number of components may be achieved by, say, bulking the