Latent Factor Model Research Articles

Off-Policy Evaluation in Doubly Inhomogeneous Environments

– This work aims to study off-policy evaluation (OPE) under scenarios where two key reinforcement learning (RL) assumptions—temporal stationarity and individual homogeneity are both violated. To handle the “double inhomogeneities”, we propose a class of latent factor models for the reward and transition functions, under which we develop a general OPE framework that consists of both model-based and model-free approaches. To our knowledge, this is the first article that develops statistically sound OPE methods in offline RL with double inhomogeneities. It contributes to a deeper understanding of OPE in environments, where standard RL assumptions are not met, and provides several practical approaches in these settings. We establish the theoretical properties of the proposed value estimators and empirically show that our approach outperforms state-of-the-art methods. Finally, we illustrate our method on a dataset from the Medical Information Mart for Intensive Care. An R implementation of the proposed procedure is available at https://github.com/ZeyuBian/2FEOPE . Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Latent growth mixture models as latent variable multigroup factor models: Comment on McNeish et al. (2023).

McNeish et al. argue for the general use of covariance pattern growth mixture models because these models do not involve the assumption of random effects, demonstrate high rates of convergence, and are most likely to identify the correct number of latent subgroups. We argue that the covariance pattern growth mixture model is a single random intercept model. It and other models considered in their article are special cases of a general model involving slope and intercept factors. We argue growth mixture models are multigroup invariance hypotheses based on unknown subgroups. Psychometric models in which trajectories are modeled using slope factor loadings which vary by latent subgroup are often conceptually preferable. Convergence rates for mixture models can be substantially improved by using a variance component start value taken from analyses with one fewer class and by specifying multifactor models in orthogonal form. No single latent growth model is appropriate across all research contexts and, instead, the most appropriate latent mixture model must be "right-sized" to the data under consideration. Reanalysis of a real-world longitudinal data set of posttraumatic stress disorder symptomatology reveals a three-group model involving exponential decline, further suggesting that the four-group "cat's cradle" pattern frequently reported is artefactual. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Ab initio uncertainty quantification in scattering analysis of microscopy.

Estimating parameters from data is a fundamental problem in physics, customarily done by minimizing a loss function between a model and observed statistics. In scattering-based analysis, it is common to work in the reciprocal space. Researchers often employ their domain expertise to select a specific range of wave vectors for analysis, a choice that can vary depending on the specific case. We introduce another paradigm that defines a probabilistic generative model from the beginning of data processing and propagates the uncertainty for parameter estimation, termed the ab initio uncertainty quantification (AIUQ). As an illustrative example, we demonstrate this approach with differential dynamic microscopy (DDM) that extracts dynamical information through minimizing a loss function for the squared differences of the Fourier-transformed intensities, at a selected range of wave vectors. We first show that the conventional way of estimation in DDM is equivalent to fitting a temporal variogram in the reciprocal space using a latent factor model as the generative model. Then we derive the maximum marginal likelihood estimator, which optimally weighs the information at all wave vectors, therefore eliminating the need to select the range of wave vectors. Furthermore, we substantially reduce the computational cost of computing the likelihood function without approximation, by utilizing the generalized Schur algorithm for Toeplitz covariances. Simulated studies of a wide range of dynamical systems validate that the AIUQ method improves estimation accuracy and enables model selection with automated analysis. The utility of AIUQ is also demonstrated by three distinct sets of experiments: first in an isotropic Newtonian fluid, pushing limits of optically dense systems compared to multiple particle tracking; next in a system undergoing a sol-gel transition, automating the determination of gelling points and critical exponent; and lastly, in discerning anisotropic diffusive behavior of colloids in a liquid crystal. These studies demonstrate that the new approach does not require manually selecting the wave vector range and enables automated analysis.

Alternating nonnegative least squares-incorporated regularized symmetric latent factor analysis for undirected weighted networks

An Undirected Weighted Network (UWN) can be precisely quantified as an adjacency matrix whose inherent characteristics are fully considered in a Symmetric Nonnegative Latent Factor (SNLF) model for its good representation accuracy. However, an SNLF model uses a sole latent factor matrix to precisely describe the topological characteristic of a UWN, i.e., symmetry, thereby impairing its representation learning ability. Aiming at addressing this issue, this paper proposes an Alternating nonnegative least squares-incorporated Regularized Symmetric Latent factor analysis (ARSL) model. First of all, equation constraints composed of multiple matrices are built in its learning objective for well describing the symmetry of a UWN. Note that it adopts an L2-norm-based regularization scheme to relax such constraints for making such a symmetry-aware learning objective solvable. Then, it designs an alternating nonnegative least squares-incorporated algorithm for optimizing its parameters efficiently. Empirical studies on four UWNs demonstrate that an ARSL model outperforms the state-of-the-art models in terms of representation accuracy, as well as achieves promising computational efficiency.

Direct estimation and inference of higher-level correlations from lower-level measurements with applications in gene-pathway and proteomics studies.

This paper tackles the challenge of estimating correlations between higher-level biological variables (e.g. proteins and gene pathways) when only lower-level measurements are directly observed (e.g. peptides and individual genes). Existing methods typically aggregate lower-level data into higher-level variables and then estimate correlations based on the aggregated data. However, different data aggregation methods can yield varying correlation estimates as they target different higher-level quantities. Our solution is a latent factor model that directly estimates these higher-level correlations from lower-level data without the need for data aggregation. We further introduce a shrinkage estimator to ensure the positive definiteness and improve the accuracy of the estimated correlation matrix. Furthermore, we establish the asymptotic normality of our estimator, enabling efficient computation of P-values for the identification of significant correlations. The effectiveness of our approach is demonstrated through comprehensive simulations and the analysis of proteomics and gene expression datasets. We develop the R package highcor for implementing our method.

Fast Variational Inference for Bayesian Factor Analysis in Single and Multi-Study Settings

Factors models are commonly used to analyze high-dimensional data in both single-study and multi-study settings. Bayesian inference for such models relies on Markov chain Monte Carlo (MCMC) methods, which scale poorly as the number of studies, observations, or measured variables increase. To address this issue, we propose new variational inference algorithms to approximate the posterior distribution of Bayesian latent factor models using the multiplicative gamma process shrinkage prior. The proposed algorithms provide fast approximate inference at a fraction of the time and memory of MCMC-based implementations while maintaining comparable accuracy in characterizing the data covariance matrix. We conduct extensive simulations to evaluate our proposed algorithms and show their utility in estimating the model for high-dimensional multi-study gene expression data in ovarian cancers. Overall, our proposed approaches enable more efficient and scalable inference for factor models, facilitating their use in high-dimensional settings. An R package VIMSFA implementing our methods is available on GitHub (github.com/blhansen/VI-MSFA). Supplementary materials for this article are available online.

Structured prior distributions for the covariance matrix in latent factor models

Factor models are widely used for dimension reduction in the analysis of multivariate data. This is achieved through decomposition of a p×p\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$p \ imes p$$\\end{document} covariance matrix into the sum of two components. Through a latent factor representation, they can be interpreted as a diagonal matrix of idiosyncratic variances and a shared variation matrix, that is, the product of a p×k\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$p \ imes k$$\\end{document} factor loadings matrix and its transpose. If k≪p\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$k \\ll p$$\\end{document}, this defines a parsimonious factorisation of the covariance matrix. Historically, little attention has been paid to incorporating prior information in Bayesian analyses using factor models where, at best, the prior for the factor loadings is order invariant. In this work, a class of structured priors is developed that can encode ideas of dependence structure about the shared variation matrix. The construction allows data-informed shrinkage towards sensible parametric structures while also facilitating inference over the number of factors. Using an unconstrained reparameterisation of stationary vector autoregressions, the methodology is extended to stationary dynamic factor models. For computational inference, parameter-expanded Markov chain Monte Carlo samplers are proposed, including an efficient adaptive Gibbs sampler. Two substantive applications showcase the scope of the methodology and its inferential benefits.

Block-diagonal idiosyncratic covariance estimation in high-dimensional factor models for financial time series

Estimation of high-dimensional covariance matrices in latent factor models is an important topic in many fields and especially in finance. Since the number of financial assets grows while the estimation window length remains of limited size, the often used sample estimator yields noisy estimates which are not even positive definite. Under the assumption of latent factor models, the covariance matrix is decomposed into a common low-rank component and a full-rank idiosyncratic component. In this paper we focus on the estimation of the idiosyncratic component, under the assumption of a grouped structure of the time series, which may arise due to specific factors such as industries, asset classes or countries. We propose a generalized methodology for estimation of the block-diagonal idiosyncratic component by clustering the residual series and applying shrinkage to the obtained blocks in order to ensure positive definiteness. We derive two different estimators based on different clustering methods and test their performance using simulation and historical data. The proposed methods are shown to provide reliable estimates and outperform other state-of-the-art estimators based on thresholding methods.

Dynamic factor models for binary data in circular spaces: an application to the US Supreme Court

Abstract Latent factor models are widely used in the social and behavioural sciences as scaling tools to map discrete multivariate outcomes into low-dimensional, continuous scales. In political science, dynamic versions of classical factor models have been widely used to study the evolution of justices’ preferences in multi-judge courts. In this paper, we discuss a new dynamic factor model that relies on a latent circular space that can accommodate voting behaviours in which justices commonly understood to be on opposite ends of the ideological spectrum vote together on a substantial number of otherwise closely divided opinions. We apply this model to data on nonunanimous decisions made by the US Supreme Court between 1937 and 2021, and show that for most of this period, voting patterns can be better described by a circular latent space.

A supervised Bayesian factor model for the identification of multi-omics signatures.

Predictive biological signatures provide utility as biomarkers for disease diagnosis and prognosis, as well as prediction of responses to vaccination or therapy. These signatures are identified from high-throughput profiling assays through a combination of dimensionality reduction and machine learning techniques. The genes, proteins, metabolites, and other biological analytes that compose signatures also generate hypotheses on the underlying mechanisms driving biological responses, thus improving biological understanding. Dimensionality reduction is a critical step in signature discovery to address the large number of analytes in omics datasets, especially for multi-omics profiling studies with tens of thousands of measurements. Latent factor models, which can account for the structural heterogeneity across diverse assays, effectively integrate multi-omics data and reduce dimensionality to a small number of factors that capture correlations and associations among measurements. These factors provide biologically interpretable features for predictive modeling. However, multi-omics integration and predictive modeling are generally performed independently in sequential steps, leading to suboptimal factor construction. Combining these steps can yield better multi-omics signatures that are more predictive while still being biologically meaningful. We developed a supervised variational Bayesian factor model that extracts multi-omics signatures from high-throughput profiling datasets that can span multiple data types. Signature-based multiPle-omics intEgration via lAtent factoRs (SPEAR) adaptively determines factor rank, emphasis on factor structure, data relevance and feature sparsity. The method improves the reconstruction of underlying factors in synthetic examples and prediction accuracy of coronavirus disease 2019 severity and breast cancer tumor subtypes. SPEAR is a publicly available R-package hosted at https://bitbucket.org/kleinstein/SPEAR.

A Collaborative Filtering-Based Recommender Systems approach for Multifarious applications

Recommender systems are crucial in today's IT landscape, enhancing user experiences in various industries. Collaborative filtering (CF) is a key approach, using historical interaction data to predict user preferences. This paper presents CF advancements, with a focus on latent factor models that represent users and items in a compact feature space. It also addresses sparsity issues with techniques like neighborhood-based approaches and content augmentation. Contextual CF, which incorporates temporal and contextual dynamics, is explored through methods like matrix factorization with side information and session-based recommendation. Evaluation metrics such as MAE and RMSE, along with novel ranking-based metrics, provide a comprehensive assessment of recommendation quality. In this paper we outline cutting-edge CF techniques, showcasing their mechanisms and applications and were able to achieve accurate recommendations of almost 90% using MAE and RMSE metrics. By integrating latent factor modeling, sparsity mitigation, contextual enrichment, and advanced evaluation, it paves the way for the next generation of personalized recommendation systems, tailored to meet evolving demands in modern information environments.

Individualized dynamic latent factor model for multi-resolutional data with application to mobile health

Summary Mobile health has emerged as a major success for tracking individual health status, due to the popularity and power of smartphones and wearable devices. This has also brought great challenges in handling heterogeneous, multi-resolution data that arise ubiquitously in mobile health due to irregular multivariate measurements collected from individuals. In this paper, we propose an individualized dynamic latent factor model for irregular multi-resolution time series data to interpolate unsampled measurements of time series with low resolution. One major advantage of the proposed method is the capability to integrate multiple irregular time series and multiple subjects by mapping the multi-resolution data to the latent space. In addition, the proposed individualized dynamic latent factor model is applicable to capturing heterogeneous longitudinal information through individualized dynamic latent factors. Our theory provides a bound on the integrated interpolation error and the convergence rate for B-spline approximation methods. Both the simulation studies and the application to smartwatch data demonstrate the superior performance of the proposed method compared to existing methods.

Cross-domain information fusion and personalized recommendation in artificial intelligence recommendation system based on mathematical matrix decomposition

Given the challenges of inter-domain information fusion and data sparsity in collaborative filtering algorithms, this paper proposes a cross-domain information fusion matrix decomposition algorithm to enhance the accuracy of personalized recommendations in artificial intelligence recommendation systems. The study begins by collecting Douban movie rating data and social network information. To ensure data integrity, Levenshtein distance detection is employed to remove duplicate scores, while natural language processing technology is utilized to extract keywords and topic information from social texts. Additionally, graph convolutional networks are utilized to convert user relationships into feature vectors, and a unique thermal coding method is used to convert discrete user and movie information into binary matrices. To prevent overfitting, the Ridge regularization method is introduced to gradually optimize potential feature vectors. Weighted average and feature connection techniques are then applied to integrate features from different fields. Moreover, the paper combines the item-based collaborative filtering algorithm with merged user characteristics to generate personalized recommendation lists.In the experimental stage, the paper conducts cross-domain information fusion optimization on four mainstream mathematical matrix decomposition algorithms: alternating least squares method, non-negative matrix decomposition, singular value decomposition, and latent factor model (LFM). It compares these algorithms with the non-fused approach. The results indicate a significant improvement in score accuracy, with mean absolute error and root mean squared error reduced by 12.8% and 13.2% respectively across the four algorithms. Additionally, when k = 10, the average F1 score reaches 0.97, and the ranking accuracy coverage of the LFM algorithm increases by 54.2%. Overall, the mathematical matrix decomposition algorithm combined with cross-domain information fusion demonstrates clear advantages in accuracy, prediction performance, recommendation diversity, and ranking quality, and improves the accuracy and diversity of the recommendation system. By effectively addressing collaborative filtering challenges through the integration of diverse techniques, it significantly surpasses traditional models in recommendation accuracy and variety.

A Prediction-Sampling-Based Multilayer-Structured Latent Factor Model for Accurate Representation to High-Dimensional and Sparse Data.

Performing highly accurate representation learning on a high-dimensional and sparse (HiDS) matrix is of great significance in a big data-related application such as a recommender system. A latent factor (LF) model is one of the most efficient approaches to the HiDS matrix representation. However, an LF model's representation learning ability relies heavily on an HiDS matrix's known data density, which is extremely low due to numerous missing data entities. To address this issue, this work proposes a prediction-sampling-based multilayer-structured LF (PMLF) model with twofold ideas: 1) constructing a loosely connected multilayered LF architecture to increase the known data density of an input HiDS matrix by generating synthetic data layer by layer and 2) constraining this synthetic data generating process through a random prediction-sampling strategy and nonlinear activations to avoid overfitting. In the experiments, PMLF is compared with six state-of-the-art LF-and deep neural network (DNN)-based models on four HiDS matrices from industrial applications. The results demonstrate that PMLF outperforms its peers in well-balancing prediction accuracy and computational efficiency.

Deep Latent Factor Model for Spatio-Temporal Forecasting

Latent factor models can perform spatio-temporal forecasting (i.e., predicting future responses at unmeasured as well as measured locations) by modeling temporal dependence using latent factors and considering spatial dependence using a spatial prior on factor loadings. However, they may fail to capture complex spatio-temporal dependence because the latent factors are typically assumed to follow a classical linear time series model, such as a vector autoregressive model. In this article, we propose a deep latent factor model for spatio-temporal forecasting that can model complex spatio-temporal dependence more flexibly by leveraging the high expressive power of a deep neural network. Specifically, the latent factors are modeled using a recurrent neural network and the factor loadings are modeled using a distance-based Gaussian process. The proposed model allows the number of latent factors to be inferred from the data using a beta-Bernoulli process, which enables computationally more efficient implementation compared to previous methods. We derive a stochastic variational inference algorithm for scalable inference of the proposed model and validate the model using simulated and real data examples.

Spatial Comprehensive Well-Being Composite Indicators Based on Bayesian Latent Factor Model: Evidence from Italian Provinces

AbstractThis paper proposes spatial comprehensive composite indicators to evaluate the well-being levels and ranking of Italian provinces with data from the Equitable and Sustainable Well-Being dashboard. We use a method based on Bayesian latent factor models, which allow us to include spatial dependence across Italian provinces, quantify uncertainty in the resulting estimates, and estimate data-driven weights for elementary indicators. The results reveal that our data-driven approach changes the resulting composite indicator rankings compared to those produced by traditional composite indicators’ approaches. Estimated social and economic well-being is unequally distributed among southern and northern Italian provinces. In contrast, the environmental dimension appears less spatially clustered, and its composite indicators also reach above-average levels in the southern provinces. The time series of well-being composite indicators of Italian macro-areas shows clustering and macro-areas discrimination on larger territorial units.

International trade fluctuations: Global versus regional factors

AbstractThis paper examines the relative importance of global, regional, country and idiosyncratic factors as well as the determinants that underpin fluctuations in international trade flows across different regions of the world. Our analysis starts by using a Bayesian dynamic latent factor model (BDFM) to simultaneously estimate the four dynamic factors, followed by the application of Bayesian model averaging to identify the variables that explain the shares of variance. Our key findings are: (i) international factors are the most important in explaining fluctuations in international trade, suggesting that the interconnections between economies and policies/shocks at the regional and global level tend to be more important than country‐level factors and (ii) regional integration, particularly when the agreement goes beyond trade in goods, is positively related to the share of the regional factor and inversely related to the importance of the global factor. Furthermore, the regional factor is more important in the case of economically large trade blocks. Overall, our analysis illustrates the usefulness of applying a BDFM model to study the co‐movements of international trade series.

Efficient computation of high-dimensional penalized generalized linear mixed models by latent factor modeling of the random effects.

Modern biomedical datasets are increasingly high-dimensional and exhibit complex correlation structures. Generalized linear mixed models (GLMMs) have long been employed to account for such dependencies. However, proper specification of the fixed and random effects in GLMMs is increasingly difficult in high dimensions, and computational complexity grows with increasing dimension of the random effects. We present a novel reformulation of the GLMM using a factor model decomposition of the random effects, enabling scalable computation of GLMMs in high dimensions by reducing the latent space from a large number of random effects to a smaller set of latent factors. We also extend our prior work to estimate model parameters using a modified Monte Carlo Expectation Conditional Minimization algorithm, allowing us to perform variable selection on both the fixed and random effects simultaneously. We show through simulation that through this factor model decomposition, our method can fit high-dimensional penalized GLMMs faster than comparable methods and more easily scale to larger dimensions not previously seen in existing approaches.

SURGE: uncovering context-specific genetic-regulation of gene expression from single-cell RNA sequencing using latent-factor models.

Genetic regulation of gene expression is a complex process, with genetic effects known to vary across cellular contexts such as cell types and environmental conditions. We developed SURGE, a method for unsupervised discovery of context-specific expression quantitative trait loci (eQTLs) from single-cell transcriptomic data. This allows discovery of the contexts or cell types modulating genetic regulation without prior knowledge. Applied to peripheral blood single-cell eQTL data, SURGE contexts capture continuous representations of distinct cell types and groupings of biologically related cell types. We demonstrate the disease-relevance of SURGE context-specific eQTLs using colocalization analysis and stratified LD-score regression.

Individual and environmental variables related to outdoor walking among older adults: Verifying a model to guide the design of interventions targeting outdoor walking.

To estimate the relationships between individual and environmental variables and outdoor walking (OW) in older adults with OW limitations through verifying a conceptual model. Baseline data from 205 older adults participating in a randomized trial of a park-based OW program were analyzed using structural equation modeling. We evaluated a three latent factor model: OW (accelerometry and self-report); individual factors (balance; leg strength; walking self-confidence, speed and endurance; mental health; education; income; car access); and environmental factors (neighbourhood walkability components). Mean age was 75 years; 73% were women. Individual factors was significantly associated with OW (β = 0.39, p < .01). Environmental factors was not directly associated with OW but was indirectly linked to OW through its significant covariance with the individual factors (β = 0.22, p < .01). The standardized factor loadings from the individual factors on walking self-confidence and walking capacity measures exceeded 0.65. Better walking capacity and more confidence in the ability to walk outdoors are associated with higher OW in older adults. Better neighbourhood walkability is indirectly associated with more OW. The conceptual model demonstrates an individual and environment association; if the capacity of the individual is increased (potentially through walking interventions), they may be able to better navigate environmental challenges.