Low-dimensional Latent Factors Research Articles

Joint Inference of Diffusion and Structure in Partially Observed Social Networks Using Coupled Matrix Factorization

Access to complete data in large-scale networks is often infeasible. Therefore, the problem of missing data is a crucial and unavoidable issue in the analysis and modeling of real-world social networks. However, most of the research on different aspects of social networks does not consider this limitation. One effective way to solve this problem is to recover the missing data as a pre-processing step. In this paper, a model is learned from partially observed data to infer unobserved diffusion and structure networks. To jointly discover omitted diffusion activities and hidden network structures, we develop a probabilistic generative model called “DiffStru.” The interrelations among links of nodes and cascade processes are utilized in the proposed method via learning coupled with low-dimensional latent factors. Besides inferring unseen data, latent factors such as community detection may also aid in network classification problems. We tested different missing data scenarios on simulated independent cascades over LFR networks and real datasets, including Twitter and Memetracker. Experiments on these synthetic and real-world datasets show that the proposed method successfully detects invisible social behaviors, predicts links, and identifies latent features.

A Behrens–Fisher problem for general factor models in high dimensions

We revisit the well-known Behrens–Fisher problem in an original and challenging high-dimensional framework, and propose a testing procedure which accommodates a low-dimensional latent factor model. The developed inferential framework is general, as it applies to problems where the underlying populations may be non-normal, the dimension of the population mean vectors may highly exceed the sample size, the design may be unbalanced, and the loading factor dimensions may be different. Under a high-dimensional asymptotic regime, combined with fairly weak technical conditions, we show that null limiting distributions of the test statistics follow a weighted mixture of chi-square distributions, which depends only on the spectrum of the noise covariance matrix and the number of latent factors. As these latter are usually unknown in practice, we exploit an estimation procedure which builds on recent advances in random matrix theory. The asymptotic power of the proposed test is established. A numerical study confirms good analytical properties of the new test that compares favorably to existing procedures used in a similar context. Real data applications are demonstrated with a study of a leukemia data set.

Inference of Multiplicative Factors Underlying Neural Variability in Calcium Imaging Data.

Understanding brain function requires disentangling the high-dimensional activity of populations of neurons. Calcium imaging is an increasingly popular technique for monitoring such neural activity, but computational tools for interpreting extracted calcium signals are lacking. While there has been a substantial development of factor analysis-type methods for neural spike train analysis, similar methods targeted at calcium imaging data are only beginning to emerge. Here we develop a flexible modeling framework that identifies low-dimensional latent factors in calcium imaging data with distinct additive and multiplicative modulatory effects. Our model includes spike-and-slab sparse priors that regularize additive factor activity and gaussian process priors that constrain multiplicative effects to vary only gradually, allowing for the identification of smooth and interpretable changes in multiplicative gain. These factors are estimated from the data using a variational expectation-maximization algorithm that requires a differentiable reparameterization of both continuous and discrete latent variables. After demonstrating our method on simulated data, we apply it to experimental data from the zebrafish optic tectum, uncovering low-dimensional fluctuations in multiplicative excitability that govern trial-to-trial variation in evoked responses.

Tackling temporal-dynamic service composition in cloud manufacturing systems: A tensor factorization-based two-stage approach

Service composition (SC) is a critical matching process in cloud manufacturing (CMfg) mainly to organize geographically-scattered manufacturing services for manufacturing tasks. Currently, most of the existing CMfg SC approaches assume that cloud service data are always static without considering the dynamics in cloud environments. Meanwhile, existing approaches rely heavily on easy-to-get dense historical service data for SC. However, adapting such ideal approaches to fast-changing temporal-dynamic cloud environments possibly leads to unsatisfactory matching results. This paper proposes a hybrid two-stage approach to deal with temporal-dynamic CMfg SC problem by incorporating tensor factorization (TF). The approach has two stages: temporal-dynamic prediction (Stage I) and many-objective SC optimization (Stage II). For Stage I, a non-negative TF based CMfg SC predictor is proposed to obtain average attribute values of candidate services in desired time slots easily and conveniently. The predictor extracts temporal-dynamic QoS values of candidate services into low-dimensional latent factor space and build a low-rank approximation. Based on the predicted service data, Stage II optimizes temporal-dynamic CMfg SC solutions by an improved many-objective evolutionary optimizer, GrEA-X. GrEA-X is reinforced with multiple variation operators and a modified mating pool selection procedure to enhance its exploration ability in a given time limit. Numerical experiments are then carried out to validate the efficacy of the proposed approach. Based on these results, we confirm the superior performance of our approach over other benchmarks.

A spatial Bayesian latent factor model for image‐on‐image regression

Image-on-image regression analysis, using images to predict images, is a challenging task, due to (1) the high dimensionality and (2) the complex spatial dependence structures in image predictors and image outcomes. In this work, we propose a novel image-on-image regression model, by extending a spatial Bayesian latent factor model to image data, where low-dimensional latent factors are adopted to make connections between high-dimensional image outcomes and image predictors. We assign Gaussian process priors to the spatially varying regression coefficients in the model, which can well capture the complex spatial dependence among image outcomes as well as that among the image predictors. We perform simulation studies to evaluate the out-of-sample prediction performance of our method compared with linear regression and voxel-wise regression methods for different scenarios. The proposed method achieves better prediction accuracy by effectively accounting for the spatial dependence and efficiently reduces image dimensions with latent factors. We apply the proposed method to analysis of multimodal image data in the Human Connectome Project where we predict task-related contrast maps using subcortical volumetric seedmaps.

Computing Large-Scale Matrix and Tensor Decomposition With Structured Factors: A Unified Nonconvex Optimization Perspective

During the past 20 years, low-rank tensor and matrix decomposition models (LRDMs) have become indispensable tools for signal processing, machine learning, and data science. LRDMs represent high-dimensional, multiaspect, and multimodal data using low-dimensional latent factors in a succinct and parsimonious way. LRDMs can serve a variety of purposes, e.g., data embedding (dimensionality reduction), denoising, latent variable analysis, model parameter estimation, and big data compression; see [1]-[5] for surveys of applications.

Stimulus-choice (mis)alignment in primate area MT.

For stimuli near perceptual threshold, the trial-by-trial activity of single neurons in many sensory areas is correlated with the animal's perceptual report. This phenomenon has often been attributed to feedforward readout of the neural activity by the downstream decision-making circuits. The interpretation of choice-correlated activity is quite ambiguous, but its meaning can be better understood in the light of population-wide correlations among sensory neurons. Using a statistical nonlinear dimensionality reduction technique on single-trial ensemble recordings from the middle temporal (MT) area during perceptual-decision-making, we extracted low-dimensional latent factors that captured the population-wide fluctuations. We dissected the particular contributions of sensory-driven versus choice-correlated activity in the low-dimensional population code. We found that the latent factors strongly encoded the direction of the stimulus in single dimension with a temporal signature similar to that of single MT neurons. If the downstream circuit were optimally utilizing this information, choice-correlated signals should be aligned with this stimulus encoding dimension. Surprisingly, we found that a large component of the choice information resides in the subspace orthogonal to the stimulus representation inconsistent with the optimal readout view. This misaligned choice information allows the feedforward sensory information to coexist with the decision-making process. The time course of these signals suggest that this misaligned contribution likely is feedback from the downstream areas. We hypothesize that this non-corrupting choice-correlated feedback might be related to learning or reinforcing sensory-motor relations in the sensory population.

Cold-start link prediction in multi-relational networks based on network dependence analysis

Cold-start link prediction has been a hot issue in complex network. Different with most of existing methods, this paper utilizes multiple interactions to predict a specific type of links. In this paper, multiple interactions are abstracted as multi-relational networks, and robust principle component analysis is employed to extract low-dimensional latent factors from sub-networks. Then a distribution free independence test, projection correlation, is introduced to efficiently analyze dependence between target and auxiliary sub-networks. Furthermore, associated auxiliary networks are exploited for cold-start link prediction, which aims to forecast potential links for new/isolated nodes in target sub-networks. Experimental results on 8 bioinformatics datasets validate rationality and effectiveness of the method.

A High Dimensional Two-Sample Test Under a Low Dimensional Factor Structure

Existing high dimensional two-sample tests usually assume that different elements of a high dimensional predictor are weakly dependent. Such a condition can be violated when data follow a low dimensional latent factor structure. As a result, the recently developed two-sample testing methods are not directly applicable. To fulfill such a theoretical gap, we propose here a Factor Adjusted two-Sample Testing (FAST) procedure to accommodate the low dimensional latent factor structure. Under the null hypothesis, together with fairly weak technical conditions, we show that the proposed test statistic is asymptotically distributed as a weighted chi-square distribution with a finite number of degrees of freedom. This leads to a totally different test statistic and inference procedure, as compared with those of Bai and Saranadasa (1996) and Chen and Qin (2010). Simulation studies are carried out to examine its finite sample performance. A real example on China stock market is analyzed for illustration purpose.

A high dimensional two-sample test under a low dimensional factor structure

Existing high dimensional two-sample tests usually assume that different elements of a high dimensional predictor are weakly dependent. Such a condition can be violated when data follow a low dimensional latent factor structure. As a result, the recently developed two-sample testing methods are not directly applicable. To fulfill such a theoretical gap, we propose here a Factor Adjusted two-Sample Testing (FAST) procedure to accommodate the low dimensional latent factor structure. Under the null hypothesis, together with fairly weak technical conditions, we show that the proposed test statistic is asymptotically distributed as a weighted chi-square distribution with a finite number of degrees of freedom. This leads to a totally different test statistic and inference procedure, as compared with those of Bai and Saranadasa (1996) and Chen and Qin (2010). Simulation studies are carried out to examine its finite sample performance. A real example on China stock market is analyzed for illustration purpose.

A latent factor linear mixed model for high‐dimensional longitudinal data analysis

High-dimensional longitudinal data involving latent variables such as depression and anxiety that cannot be quantified directly are often encountered in biomedical and social sciences. Multiple responses are used to characterize these latent quantities, and repeated measures are collected to capture their trends over time. Furthermore, substantive research questions may concern issues such as interrelated trends among latent variables that can only be addressed by modeling them jointly. Although statistical analysis of univariate longitudinal data has been well developed, methods for modeling multivariate high-dimensional longitudinal data are still under development. In this paper, we propose a latent factor linear mixed model (LFLMM) for analyzing this type of data. This model is a combination of the factor analysis and multivariate linear mixed models. Under this modeling framework, we reduced the high-dimensional responses to low-dimensional latent factors by the factor analysis model, and then we used the multivariate linear mixed model to study the longitudinal trends of these latent factors. We developed an expectation-maximization algorithm to estimate the model. We used simulation studies to investigate the computational properties of the expectation-maximization algorithm and compare the LFLMM model with other approaches for high-dimensional longitudinal data analysis. We used a real data example to illustrate the practical usefulness of the model.

Factor profiled sure independence screening

We propose a method of factor profiled sure independence screening for ultrahigh-dimensional variable selection. The objective of this method is to identify nonzero components consistently from a sparse coefficient vector. The new method assumes that the correlation structure of the high-dimensional data can be well represented by a set of low-dimensional latent factors, which can be estimated consistently by eigenvalue-eigenvector decomposition. The estimated latent factors should then be profiled out from both the response and the predictors. Such an operation, referred to as factor profiling, produces uncorrelated predictors. Therefore, sure independence screening can be applied subsequently and the resulting screening result is consistent for model selection, a major advantage that standard sure independence screening does not share. We refer to the new method as factor profiled sure independence screening. Numerical studies confirm its outstanding performance. Copyright 2012, Oxford University Press.

Hierarchical multilinear models for multiway data

Reduced-rank decompositions provide descriptions of the variation among the elements of a matrix or array. In such decompositions, the elements of an array are expressed as products of low-dimensional latent factors. This article presents a model-based version of such a decomposition, extending the scope of reduced-rank methods to accommodate a variety of data types such as longitudinal social networks and continuous multivariate data that are cross-classified by categorical variables. The proposed model-based approach is hierarchical, in that the latent factors corresponding to a given dimension of the array are not a priori independent, but exchangeable. Such a hierarchical approach allows more flexibility in the types of patterns that can be represented.

Factor Profiling for Ultra High Dimensional Variable Selection

We propose here a novel method of factor profiling (FP) for ultra high dimensional variable selection. The new method assumes that the correlation structure of the high dimensional data can be well represented by a set of low-dimensional latent factors (Fan et al., 2008). The latent factors can then be estimated consistently by eigenvalue-eigenvector decomposition. They should be profiled out subsequently from both the response and predictors. Such an operation is referred to as FP. Obviously, FP produces uncorrelated predictors. Thereafter, the method of sure independent screening (Fan and Lv, 2008, SIS) can be applied immediately. This leads to profiled independent screening (PIS). PIS is shown to be selection consistent, even if the predictor dimension is substantially larger than the sample size. To further improve PIS, a novel method of profiled sequential screening (PSS) is proposed. PSS shares similar strength as forward regression (Wang, 2009a) but is computationally even simpler. Numerical studies are presented to corroborate our theoretical findings.