Reduced Rank Regression Research Articles

Reduced rank regression via adaptive nuclear norm penalization.

We propose an adaptive nuclear norm penalization approach for low-rank matrix approximation, and use it to develop a new reduced rank estimation method for high-dimensional multivariate regression. The adaptive nuclear norm is defined as the weighted sum of the singular values of the matrix, and it is generally non-convex under the natural restriction that the weight decreases with the singular value. However, we show that the proposed non-convex penalized regression method has a global optimal solution obtained from an adaptively soft-thresholded singular value decomposition. The method is computationally efficient, and the resulting solution path is continuous. The rank consistency of and prediction/estimation performance bounds for the estimator are established for a high-dimensional asymptotic regime. Simulation studies and an application in genetics demonstrate its efficacy.

Tuning Parameter Selection for Underdetermined Reduced-Rank Regression

Multivariate regression is one of the most widely applied multivariate statistical methods with many uses across a range of disciplines. But the number of parameters increases exponentially with dimension and reduced-rank regression (RRR) is a well known approach to dimension reduction. But traditional RRR applies only to an overdetermined system. For increasingly common undetermined systems this issue can be managed by regularization, e.g., with a quadratic penalty. A significant problem is then the choice of the two tuning parameters: one discrete i.e., the rank; the other continuous i.e., the Tikhonov penalty parameter. In this paper we resolve this problem via Stein's unbiased risk estimator (SURE). We compare SURE to cross-validation and apply it on both simulated and real data sets.

Variation of serum metabolites related to habitual diet: a targeted metabolomic approach in EPIC-Potsdam

Serum metabolites have been linked to higher risk of chronic diseases but determinants of serum metabolites are not clear. We aimed to investigate the association between habitual diet as a modifiable risk factor and relevant serum metabolites. This cross-sectional study comprised 2380 EPIC-Potsdam participants. Intake of 45 food groups was assessed by food frequency questionnaire and concentrations of 127 serum metabolites were measured by targeted metabolomics. Reduced rank regression was used to find dietary patterns that explain the maximum variation of metabolites. In the multivariable-adjusted model, the proportion of explained variation by habitual diet was ranked as follows: acyl-alkyl-phosphatidylcholines (5.7%), sphingomyelins (5.1%), diacyl-phosphatidylcholines (4.4%), lyso-phosphatidylcholines (4.1%), acylcarnitines (3.5%), amino acids (2.2%) and hexose (1.6%). A pattern with high intake of butter and low intake of margarine was related to acylcarnitines, acyl-alkyl-phosphatidylcholines, lyso-phosphatidylcholines and hydroxy-sphingomyelins, particularly with saturated and monounsaturated fatty acid side chains. A pattern with high intake of red meat and fish and low intake of whole-grain bread and tea was related to hexose and phosphatidylcholines. A pattern consisting of high intake of potatoes, dairy products and cornflakes particularly explained methionine and branched chain amino acids. Dietary patterns related to type 2 diabetes-relevant metabolites included high intake of red meat and low intake of whole-grain bread, tea, coffee, cake and cookies, canned fruits and fish. Dietary patterns characterized by intakes of red meat, whole-grain bread, tea and coffee were linked to relevant metabolites and could be potential targets for chronic disease prevention.

Common Factors in Corporate Capital Structure

This paper reexamines the controversy over capital structure adjustment speed. We show that firms move leverage towards target much faster than generally understood. To do this we use reduced rank regressions to extract 4 common factors for corporate leverage using 146 variables, including value weighted aggregates of standard firm attributes, and macroeconomic variables. Firm specific factor loadings on all of these are estimated. The model provides estimates of time varying leverage targets. More than half of the leverage variation by the median firm is explained, and there is significant predictive ability for several years out-of-sample. Firms actively adjust both debt and equity to move leverage in the direction of the estimated targets. A straw man null hypothesis of purely random financing actions is easily rejected. Slow speeds of adjustment reported in previous studies may reflect the use of static leverage targets while firms actually care about time varying targets.

Dietary pattern analysis and biomarkers of low-grade inflammation: a systematic literature review

The purpose of the present literature review was to investigate and summarize the current evidence on associations between dietary patterns and biomarkers of inflammation, as derived from epidemiological studies. A systematic literature search was conducted using PubMed, Web of Science, and EMBASE, and a total of 46 studies were included in the review. These studies predominantly applied principal component analysis, factor analysis, reduced rank regression analysis, the Healthy Eating Index, or the Mediterranean Diet Score. No prospective observational study was found. Patterns identified by reduced rank regression as being statistically significantly associated with biomarkers of inflammation were almost all meat-based or "Western" patterns. Studies using principal component analysis or a priori-defined diet scores found that meat-based or "Western-like" patterns tended to be positively associated with biomarkers of inflammation, predominantly C-reactive protein, while vegetable- and fruit-based or "healthy" patterns tended to be inversely associated. While results of the studies were inconsistent, interventions with presumed healthy diets resulted in reductions of almost all investigated inflammatory biomarkers. In conclusion, prospective studies are warranted to confirm the reported findings and further analyze associations, particularly by investigating dietary patterns as risk factors for changes in inflammatory markers over time.

Dietary patterns and risk of dementia in an elderly Japanese population: the Hisayama Study

To our knowledge, there are no previous reports that assessed the association between dietary patterns and risk of dementia in Asian populations. We investigated dietary patterns and their potential association with risk of incident dementia in a general Japanese population. A total of 1006 community-dwelling Japanese subjects without dementia, aged 60-79 y, were followed up for a median of 15 y. The reduced rank regression procedure was used to efficiently determine their dietary patterns. Estimated risk conferred by a particular dietary pattern on the development of dementia was computed by using a Cox proportional hazards model. Seven dietary patterns were extracted; of these, dietary pattern 1 was correlated with high intakes of soybeans and soybean products, vegetables, algae, and milk and dairy products and a low intake of rice. During the follow-up, 271 subjects developed all-cause dementia. Of these individuals, 144 subjects had Alzheimer disease (AD), and 88 subjects had vascular dementia (VaD). After adjustment for potential confounders, risks of development of all-cause dementia, AD, and VaD were reduced by 0.66 (95% CI: 0.46, 0.95), 0.65 (95% CI: 0.40, 1.06), and 0.45 (95% CI: 0.22, 0.91), respectively, in subjects in the highest quartile of score for dietary pattern 1 compared with subjects in the lowest quartile. Our findings suggest that a higher adherence to a dietary pattern characterized by a high intake of soybeans and soybean products, vegetables, algae, and milk and dairy products and a low intake of rice is associated with reduced risk of dementia in the general Japanese population.

Trajectories of an unhealthy dietary pattern from 1991 to 2006 and diabetes in Chinese adults: A new approach to study longitudinal diet

We used an innovative technique to capture trajectories of a dietary pattern and assess their relationship to diabetes. We included 4780 adults with 3 to 6 waves of diet data (1991 to 2006) and HbA1c measure in 2009 from the China Health and Nutrition Survey. Diet was assessed by a household food inventory and three 24‐h recalls. We used Reduced Rank Regression (SAS 9.3) to identify a dietary pattern in 2006 that positively predicted HbA1c (high in wheat flour, buns & breads, cookies, pastries & deep‐fried wheat, corn, and soy‐milk; and low in rice, fresh leafy vegetables and lard & butter). A score for this pattern was estimated for each subject at each wave. We used Latent class trajectory analysis (Mplus 6) to group subjects that share similar trajectories of their dietary pattern's score over time. We estimated the prevalence ratio (PR) for diabetes (HbA1c ≥6.5) adjusting by age, gender, geographical region and education:The results provide insights into how the trajectory of an unhealthy dietary pattern affects diabetes. The highest prevalence was among those that started high and increase their score over time (# 2); whereas those that started even higher, but had an important decrease over time (# 5) had a prevalence close to those that remained low the entire follow‐up (# 1). Trajectory PR (95% CI) # intercept slope 1 −1.2 0.03 1 2 1.8 0.08 2.3 (1.5, 3.6) 3 1.4 0.00 1.9 (1.4, 4.5) 4 0.0 0.02 1.3 (0.9, 1.8) 5 2.9 −0.19 1.2 (0.7, 2.3) Grant Funding Source: CONACYT (grant 212018), NIH (R01‐ HD30880), the Fogarty International Center (D43‐TW009077)

Abstract P298: An Obesogenic Neighborhood Environment Factor Score and Changes in BMI in the Coronary Artery Risk Development in Young Adults (CARDIA) Study

We sought to identify obesogenic elements of the neighborhood environment using 15 years of individual-level and GIS-derived data in the CARDIA cohort (n=5,115; aged 18-30 at year 0, 1985/86), and to test associations with 15-year BMI change. We first used reduced rank regression (RRR) to derive a weighted combination (factor score) of environmental variables that explained variability in obesity-promoting diet and physical activity (PA) behaviors. RRR outcome variables were diet quality, fast food consumption, and PA; predictors of these behaviors were neighborhood-level indicators of walkability, PA facilities, restaurants and food stores, food prices, cost of living, population density, and neighborhood SES. In our model, a high score reflected: 1) a greater number of fee-based PA facilities, 2) higher food prices, and 3) higher neighborhood SES. We next quantified the association between the factor score and BMI change, hypothesizing an inverse association. In repeated-measures linear regression (4 waves; n=16,343), we observed significant inverse associations between the time-varying environmental factor score and BMI increases, with larger increases among those younger at baseline (figure), adjusting for time-varying covariates. Using a novel application of RRR, we identified an obesogenic score of environmental components that explained variability in diet and PA behaviors and was associated with BMI change in CARDIA.

Evaluation of methodologies for assessing the overall diet: dietary quality scores and dietary pattern analysis

This paper aims to describe different approaches for studying the overall diet with advantages and limitations. Studies of the overall diet have emerged because the relationship between dietary intake and health is very complex with all kinds of interactions. These cannot be captured well by studying single dietary components. Three main approaches to study the overall diet can be distinguished. The first method is researcher-defined scores or indices of diet quality. These are usually based on guidelines for a healthy diet or on diets known to be healthy. The second approach, using principal component or cluster analysis, is driven by the underlying dietary data. In principal component analysis, scales are derived based on the underlying relationships between food groups, whereas in cluster analysis, subgroups of the population are created with people that cluster together based on their dietary intake. A third approach includes methods that are driven by a combination of biological pathways and the underlying dietary data. Reduced rank regression defines linear combinations of food intakes that maximally explain nutrient intakes or intermediate markers of disease. Decision tree analysis identifies subgroups of a population whose members share dietary characteristics that influence (intermediate markers of) disease. It is concluded that all approaches have advantages and limitations and essentially answer different questions. The third approach is still more in an exploration phase, but seems to have great potential with complementary value. More insight into the utility of conducting studies on the overall diet can be gained if more attention is given to methodological issues.

Reduced-rank vector generalized linear models with two linear predictors

Vector generalized linear models (VGLMs) as implemented in the vgamR package permit multiple parameters to depend (via inverse link functions) on linear predictors. However it is often the case that one wishes different parameters to be related to each other in some way (i.e., to jointly satisfy certain constraints). Prominent and important examples of such cases include the normal or Gaussian family where one wishes to model the variance as a function of the mean, e.g., variance proportional to the mean raised to some power. Another example is the negative binomial family whose variance is approximately proportional to the mean raised to some power. It is shown that such constraints can be implemented in a straightforward manner via reduced rank regression (RRR) and easily used via the rrvglm() function. To this end RRR is briefly described and applied so as to impose parameter constraints in VGLMs with two parameters. The result is a rank-1 RR-VGLM. Numerous examples are given, some new, of the use of this technique. The implication here is that RRR offers hitherto undiscovered potential usefulness to many statistical distributions.

Dietary patterns and risk of elevated C-reactive protein concentrations 12 years later

Inflammation mediates several chronic diseases. Micronutrients can act on inflammation, either through modulating cytokine production or by scavenging by-products of activated white cells. Identifying dietary patterns (DP) reflecting these mechanisms and relating them to inflammation is of interest. The objective of the study was to identify DP specifically associated with intakes of nutrients potentially involved in inflammatory processes in a middle-aged population and investigate long-term associations between these DP and C-reactive protein (CRP) status assessed several years later. Subjects included in the Supplementation in Vitamins and Mineral Antioxidants 2 cohort study, having available data on dietary assessment carried out in 1994-5 and CRP measurement in 2007-9, were included in the analysis. DP were extracted with reduced rank regression (RRR), using antioxidant micronutrients and PUFA as response variables. Associations between CRP measurements >3 mg/l and extracted DP were then examined with logistic regression models providing OR and 95% CI. A total of 2031 subjects (53·2% women, mean follow-up duration: 12·5 years) were included in the analyses. Of the four extracted DP, a DP with high loading values of vegetables and vegetable oils, leading to high intakes of antioxidant micronutrients and essential fatty acids, was significantly and negatively associated with risk of elevated CRP (OR 0·88; 95% CI 0·78, 0·98). Conversely, a DP reflecting a high n-6:n-3 fatty acid intake ratio was positively and significantly associated with elevated CRP (adjusted OR 1·15; 95% CI 1·00, 1·32). DP extracted with RRR provide support for further exploration of relationships between dietary behaviour and inflammation.

Dietary Pattern Low in Fruits Explains Variations in Inflammation and in Biomarkers of Cardiovascular Disease in Latinos Diagnosed with Type-2 Diabetes

Aims: 1) To identify a dietary pattern associated with plasma interleukin-6 (IL-6), tumor necrosis factor alpha (TNF α) and monocyte chemotactic protein-1 (MCP-1) in Latinos diagnosed with T2D. 2) To examine whether the “pro-inflammation” dietary pattern obtained is associated with additional cardiovascular disease risk in this Latino population. Methodology: Reduced rank regression was used to determine the cross-sectional relationship between food patterns and plasma inflammatory biomarkers in Latinos (26 men/77 women, 32-76 y) diagnosed with type-2 diabetes (T2D). Reduced rank

Plasma Uric Acid Is Associated with Increased Risk of Type 2 Diabetes Independent of Diet and Metabolic Risk Factors

Current evidence suggests a direct association of uric acid with diabetes risk, but it is still unclear whether this is independent of risk factors such as obesity and diet. We aimed to investigate whether plasma uric acid concentrations are independently associated with incident type 2 diabetes and to investigate the role of a uric acid–related dietary pattern in this association. We used a case-cohort nested in the European Prospective Investigation into Cancer and Nutrition-Netherlands study. The study included 2318 subcohort members and 845 incident diabetes cases, with a mean follow-up of 10 y. At baseline, blood samples were taken and diet was assessed using a validated FFQ. A uric acid–related dietary pattern was derived with reduced rank regression. Diabetes was mainly self-reported and verified against general practitioner records. Plasma uric acid was (mean ± SD) 231 ± 54.6 μmol/L in the subcohort. After adjustment for established diabetes risk factors such as age, the HR (highest vs. lowest quartile of uric acid) for diabetes was 4.36 (95% CI: 3.22, 5.90). Further adjustment for adiposity attenuated the HR to 1.86 (95% CI: 1.32, 2.62). Additional adjustment for hypertension and biochemical markers, such as TG, slightly attenuated the association [HR = 1.43 (95% CI: 0.97, 2.10)]. A uric acid–related dietary pattern did not confound the association. In conclusion, this study supports that high uric acid concentrations are associated with increased diabetes risk, although a large part of the association can be explained by the degree of adiposity.

Macroeconomic Forecasting and Structural Analysis Through Regularized Reduced-Rank Regression

This paper proposes a strategy to detect and impose reduced-rank restrictions in medium vector autoregressive models. In this framework, it is known that Canonical Correlation Analysis (CCA) does not perform well because inversions of large covariance matrices are required. We propose a method that combines the richness of reduced-rank regression with the simplicity of naive univariate forecasting methods. In particular, we suggest to use a proper shrinkage estimator of the autocovariance matrices that are involved in the computation of CCA, thus obtaining a method that is asymptotically equivalent to CCA, but it is numerically more stable in finite samples. Simulations and empirical applications document the merits of the proposed approach both in forecasting and in structural analysis.

Sparse Reduced-Rank Regression for Simultaneous Dimension Reduction and Variable Selection

The reduced-rank regression is an effective method in predicting multiple response variables from the same set of predictor variables. It reduces the number of model parameters and takes advantage of interrelations between the response variables and hence improves predictive accuracy. We propose to select relevant variables for reduced-rank regression by using a sparsity-inducing penalty. We apply a group-lasso type penalty that treats each row of the matrix of the regression coefficients as a group and show that this penalty satisfies certain desirable invariance properties. We develop two numerical algorithms to solve the penalized regression problem and establish the asymptotic consistency of the proposed method. In particular, the manifold structure of the reduced-rank regression coefficient matrix is considered and studied in our theoretical analysis. In our simulation study and real data analysis, the new method is compared with several existing variable selection methods for multivariate regression and exhibits competitive performance in prediction and variable selection.

Generalized canonical correlation analysis for disparate data fusion

Manifold matching works to identify embeddings of multiple disparate data spaces into the same low-dimensional space, where joint inference can be pursued. It is an enabling methodology for fusion and inference from multiple and massive disparate data sources. In this paper we focus on a method called Canonical Correlation Analysis (CCA) and its generalization Generalized Canonical Correlation Analysis (GCCA), which belong to the more general Reduced Rank Regression (RRR) framework. We present an efficiency investigation of CCA and GCCA under different training conditions for a particular text document classification task.

Multilevel dimensionality-reduction methods

When data sets are multilevel (group nesting or repeated measures), different sources of variations must be identified. In the framework of unsupervised analyses, multilevel simultaneous component analysis (MSCA) has recently been proposed as the most satisfactory option for analyzing multilevel data. MSCA estimates submodels for the different levels in data and thereby separates the “within”-subject and “between”-subject variations in the variables. Following the principles of MSCA and the strategy of decomposing the available data matrix into orthogonal blocks, and taking into account the between- and the within data structures, we generalize, in a multilevel perspective, multivariate models in which a matrix of response variables can be used to guide the projections (formed by responses predicted by explanatory variables or by a limited number of their combinations/composites) into choices of meaningful directions. To this end, the current paper proposes the multilevel version of the multivariate regression model and dimensionality-reduction methods (used to predict responses with fewer linear composites of explanatory variables). The principle findings of the study are that the minimization of the loss functions related to multivariate regression, principal-component regression, reduced-rank regression, and canonical-correlation regression are equivalent to the separate minimization of the sum of two separate loss functions corresponding to the between and within structures, under some constraints. The paper closes with a case study of an application focusing on the relationships between mental health severity and the intensity of care in the Lombardy region mental health system.

Mineral Concentrations in Hair of Belgian Elementary School Girls: Reference Values and Relationship with Food Consumption Frequencies

Although evidence suggests that hair elements may reflect dietary habits and/or mineral intake, this topic remains controversial. This study therefore presents age-specific reference values for hair concentrations of Ca, Cu, Fe, Na, Mg, P and Zn using the LMS method of Cole, and investigates the relationship between dietary habits (i.e. food consumption frequencies) and hair mineral concentrations in 218 Belgian elementary school girls by reduced rank regression (RRR). Hair minerals were quantitatively determined via inductively coupled plasma-mass spectrometry after microwave-assisted acid digestion of 6-cm long vertex posterior hair samples. The Children's Eating Habits Questionnaire-Food Frequency Questionnaire was used to obtain information on food consumption frequency of 43 food items in the month preceding hair collection. The established reference ranges were in line with data for other childhood or adolescent populations. The retained RRR factors explained 40, 50, 45, 46, 44 and 48% of the variation of Ca, Cu, Fe, Mg, P and Zn concentrations in hair, respectively. Although this study demonstrated that a large proportion of hair mineral variation may be influenced by food consumption frequencies in elementary school girls, a number of food groups known to be rich sources of minerals did not show a relation with certain hair minerals. Future research should focus on mechanisms and processes involved in mineral incorporation and accumulation in scalp hair, in order to fully understand the importance and influence of diet on hair minerals.

Identification of gene pathways implicated in Alzheimer's disease using longitudinal imaging phenotypes with sparse regression

We present a new method for the detection of gene pathways associated with a multivariate quantitative trait, and use it to identify causal pathways associated with an imaging endophenotype characteristic of longitudinal structural change in the brains of patients with Alzheimer's disease (AD). Our method, known as pathways sparse reduced-rank regression (PsRRR), uses group lasso penalised regression to jointly model the effects of genome-wide single nucleotide polymorphisms (SNPs), grouped into functional pathways using prior knowledge of gene–gene interactions. Pathways are ranked in order of importance using a resampling strategy that exploits finite sample variability. Our application study uses whole genome scans and MR images from 99 probable AD patients and 164 healthy elderly controls in the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. 66,182 SNPs are mapped to 185 gene pathways from the KEGG pathway database. Voxel-wise imaging signatures characteristic of AD are obtained by analysing 3D patterns of structural change at 6, 12 and 24 months relative to baseline. High-ranking, AD endophenotype-associated pathways in our study include those describing insulin signalling, vascular smooth muscle contraction and focal adhesion. All of these have been previously implicated in AD biology. In a secondary analysis, we investigate SNPs and genes that may be driving pathway selection. High ranking genes include a number previously linked in gene expression studies to β-amyloid plaque formation in the AD brain (PIK3R3, PIK3CG, PRKCA and PRKCB), and to AD related changes in hippocampal gene expression (ADCY2, ACTN1, ACACA, and GNAI1). Other high ranking previously validated AD endophenotype-related genes include CR1, TOMM40 and APOE.

Identification of a dietary pattern prospectively associated with increased adiposity during childhood and adolescence

Background:Specific dietary risk factors for excess adiposity in young people are poorly understood. However, studies in adults suggest dietary energy density, fat and fibre are critical dietary factors.Objective:To examine longitudinal relationships between a dietary pattern (DP) characterised by dietary energy density, % total energy from fat and fibre density and fat mass (FM) in children from 7 to 15 years of age.Design:Subjects were 6772 children from the UK Avon Longitudinal Study of Parents and Children. Dietary intake was assessed using a 3-day food diary at 7, 10 and 13 years of age. An energy-dense, high-fat, low-fibre DP was identified using reduced rank regression and subjects scored for the DP at each age. FM was measured at 11, 13 and 15 years and FM index (FMI) calculated as FM/height(x). Longitudinal models were adjusted for dietary misreporting, physical activity and maternal factors.Results:DP z-scores at all ages were positively associated with later FMI. A 1 s.d. unit increase in DP z-score was longitudinally associated with an average increase in FMI z-score of 0.04 s.d. units (95% confidence interval (CI), 0.01–0.07). For each 1 s.d. unit increase in DP z-score, the odds of being in the highest quintile for FMI (as a marker of excess adiposity) increased by 13% (95% CI, 1–27%).Conclusions:Dietary habits during childhood are associated with increased adiposity in adolescence, with specific implications for dietary energy density, fat and fibre intake. Improving diet quality may reduce the risk of obesity in young people.