Sparse Factor Analysis Research Articles

A Linkage Cultivation Model of Students’ Information Literacy and Critical Thinking Skills in Education Informatization Environment

Abstract This paper first screens the indicators by examining the current state of information literacy and critical thinking. The sparse principal component analysis (SPCA) algorithm and factor analysis model are combined to deeply mine the influencing factors of information literacy and critical thinking and provide an optimization path for the linkage cultivation model of information literacy and critical thinking. Finally, we validate and analyze the enhanced critical thinking abilities. The results show that the first four eigenroot coefficients of the correlation coefficient matrices of the proposed four main factor variables are 4.31, 3.4, 3.07, and 2.74, respectively. The eigenroots’ explanatory degrees are 33.2%, 25.7%, 21.9%, and 15.1%, respectively. The four extracted principal factors can explain 95.85% of the information in the original index. The four critical indicators of questioning, discriminating, comparing, and controlling are significantly positively correlated between the two. The synthesis of the usefulness of the four critical indicators revealed that 97.5% agreed that they could improve students’ information literacy and critical thinking skills. This study positively supports and promotes the improvement of students’ information literacy and critical thinking skills in the educational information technology environment.

When It Counts—Econometric Identification of the Basic Factor Model Based on GLT Structures

Despite the popularity of factor models with simple loading matrices, little attention has been given to formally address the identifiability of these models beyond standard rotation-based identification such as the positive lower triangular (PLT) constraint. To fill this gap, we review the advantages of variance identification in simple factor analysis and introduce the generalized lower triangular (GLT) structures. We show that the GLT assumption is an improvement over PLT without compromise: GLT is also unique but, unlike PLT, a non-restrictive assumption. Furthermore, we provide a simple counting rule for variance identification under GLT structures, and we demonstrate that within this model class, the unknown number of common factors can be recovered in an exploratory factor analysis. Our methodology is illustrated for simulated data in the context of post-processing posterior draws in sparse Bayesian factor analysis.

A new Bayesian factor analysis method improves detection of genes and biological processes affected by perturbations in single-cell CRISPR screening

Clustered regularly interspaced short palindromic repeats (CRISPR) screening coupled with single-cell RNA sequencing has emerged as a powerful tool to characterize the effects of genetic perturbations on the whole transcriptome at a single-cell level. However, due to its sparsity and complex structure, analysis of single-cell CRISPR screening data is challenging. In particular, standard differential expression analysis methods are often underpowered to detect genes affected by CRISPR perturbations. We developed a statistical method for such data, called guided sparse factor analysis (GSFA). GSFA infers latent factors that represent coregulated genes or gene modules; by borrowing information from these factors, it infers the effects of genetic perturbations on individual genes. We demonstrated through extensive simulation studies that GSFA detects perturbation effects with much higher power than state-of-the-art methods. Using single-cell CRISPR data from human CD8+ T cells and neural progenitor cells, we showed that GSFA identified biologically relevant gene modules and specific genes affected by CRISPR perturbations, many of which were missed by existing methods, providing new insights into the functions of genes involved in T cell activation and neurodevelopment.

Generalized cumulative shrinkage process priors with applications to sparse Bayesian factor analysis.

The paper discusses shrinkage priors which impose increasing shrinkage in a sequence of parameters. We review the cumulative shrinkage process (CUSP) prior of Legramanti et al. (Legramanti et al. 2020 Biometrika 107, 745-752. (doi:10.1093/biomet/asaa008)), which is a spike-and-slab shrinkage prior where the spike probability is stochastically increasing and constructed from the stick-breaking representation of a Dirichlet process prior. As a first contribution, this CUSP prior is extended by involving arbitrary stick-breaking representations arising from beta distributions. As a second contribution, we prove that exchangeable spike-and-slab priors, which are popular and widely used in sparse Bayesian factor analysis, can be represented as a finite generalized CUSP prior, which is easily obtained from the decreasing order statistics of the slab probabilities. Hence, exchangeable spike-and-slab shrinkage priors imply increasing shrinkage as the column index in the loading matrix increases, without imposing explicit order constraints on the slab probabilities. An application to sparse Bayesian factor analysis illustrates the usefulness of the findings of this paper. A new exchangeable spike-and-slab shrinkage prior based on the triple gamma prior of Cadonna et al. (Cadonna et al. 2020 Econometrics 8, 20. (doi:10.3390/econometrics8020020)) is introduced and shown to be helpful for estimating the unknown number of factors in a simulation study. This article is part of the theme issue 'Bayesian inference: challenges, perspectives, and prospects'.

An algorithm for sparse factor analysis with common factors and/or specific factors dissociated from errors

An algorithm for sparse factor analysis with common factors and/or specific factors dissociated from errors

Sparse Multiple Factor Analysis, sparse STATIS, and sparse DiSTATIS with applications to sensory evaluation

AbstractComponent‐based multitable methods, such as multiple factor analysis (MFA), STATIS, and DiSTATIS, are routinely used to analyze multiblock data, which are now common in chemometrics and sensory evaluation studies. These blocks of data form data tables that—for example, in sensory evaluation—describe how different assessors evaluate a set of products either on a set of descriptors or on the similarity between products. To analyze these data, component‐based multitable methods extract orthogonal components explaining most of the variance of the data. However, when the data tables are heterogeneous or have complex structures, a single component space does not represent the data well and can give components that are difficult to interpret. Previous literature solved this interpretation problem by eliminating irrelevant variables—a process called sparsification—while keeping the components orthogonal. Here, we extended such methods to develop sparsification algorithms for three multitable methods, namely, “sparse MFA” (sMFA), “sparse STATIS” (sSTATIS), and “sparse DiSTATIS” (sDiSTATIS). In these sparse methods, we sparsified the data tables to identify the most informative assessors or products. In sMFA, we show how group sparsity can be used to sparsify whole tables (i.e., assessors or products), hereby greatly increasing the interpretability of sMFA's outcome. In sSTATIS and sDiSTATIS, we developed two different sparsification approaches: One approach creates subgroups of products and simplifies the components to facilitate interpretation; whereas the other approach creates subgroups of assessors and alleviates the problem of heterogeneity. We showed with three examples how these sparse methods increase interpretability of the results in sensory evaluation.

Mediation analysis with latent factors using simultaneous group‐wise and parameter‐wise penalization

Mediation analysis aims to uncover the underlying mechanism of how an exposure variable affects the outcome of interest through one or more than one mediating variables. In the event that the number of candidate mediators is large, variable selection or dimension reduction techniques are often utilized to reduce the dimension of the initial set of mediators. In this paper, we propose a latent variable approach using sparse factor analysis with both group‐wise and parameter‐wise penalization to remove irrelevant candidate mediators and estimate the latent factors simultaneously. After the low‐dimensional latent mediating factors are obtained, the direct and indirect effects can be estimated and tested from a multivariate mediation model. To demonstrate the practical applications of the proposed methodology, we apply it to a weight behaviour dataset and an environmental dataset, separately.

Bayesian sparse factor analysis with kernelized observations

Multi-view problems can benefit from latent representations since they find low-dimensional projections that fairly capture the correlations among the multiple views that characterise the data. On the other hand, high-dimensionality and non-linear issues are traditionally handled by kernel methods, inducing a (non)-linear function between the latent projection and the data itself. However, they usually come with exposition to overfitting. Here, we combine Bayesian factor analysis with what we refer to as kernelized observations, in which the proposed model focuses on reconstructing not the data itself, but its relationship with other data points measured by a kernel function. In turn, we extend previous Bayesian FA formulations to be able to model non-linear data relationships by means of kernelized data representations and, at the same time, include additional facilities to obtain compact kernel representations by means of an automatic selection of Bayesian Relevance Vectors (RVs), feature relevance analysis and, even, obtain an automatic multiple kernel learning approach. Besides, this is flexibly included into a modular framework where we can easily adapt the model capabilities to the data needs and, even, combine them with previous FA functionalities such as heterogeneous data representations or semi-supervised learning. Using several public databases, we demonstrate the potential of the approach (and its extensions) w.r.t. common multi-view learning models such as kernel canonical correlation analysis, heterogeneous incomplete – variational autoenconder or manifold relevance determination, where the proposed model shows its ability to outperform the baselines while indistinctly combining model extensions.

BAYESIAN JOINT MODELING OF CHEMICAL STRUCTURE AND DOSE RESPONSE CURVES.

Today there are approximately 85,000 chemicals regulated under the Toxic Substances Control Act, with around 2,000 new chemicals introduced each year. It is impossible to screen all of these chemicals for potential toxic effects, either via full organism in vivo studies or in vitro high-throughput screening (HTS) programs. Toxicologists face the challenge of choosing which chemicals to screen, and predicting the toxicity of as yet unscreened chemicals. Our goal is to describe how variation in chemical structure relates to variation in toxicological response to enable in silico toxicity characterization designed to meet both of these challenges. With our Bayesian partially Supervised Sparse and Smooth Factor Analysis (BS3FA) model, we learn a distance between chemicals targeted to toxicity, rather than one based on molecular structure alone. Our model also enables the prediction of chemical dose-response profiles based on chemical structure (i.e., without in vivo or in vitro testing) by taking advantage of a large database of chemicals that have already been tested for toxicity in HTS programs. We show superior simulation performance in distance learning and modest to large gains in predictive ability compared to existing methods. Results from the high-throughput screening data application elucidate the relationship between chemical structure and a toxicity-relevant high-throughput assay. An R package for BS3FA is available online at https://github.com/kelrenmor/bs3fa.

Failure to Identify Robust Latent Variables of Positive or Negative Valence Processing Across Units of Analysis

BackgroundThe heterogeneous nature of mood and anxiety disorders highlights a need for dimensionally based descriptions of psychopathology that inform better classification and treatment approaches. Following the Research Domain Criteria approach, this investigation sought to derive constructs assessing positive and negative valence domains across multiple units of analysis. MethodsAdults with clinically impairing mood and anxiety symptoms (N = 225) completed comprehensive assessments across several units of analysis. Self-report assessments included nine questionnaires that assess mood and anxiety symptoms and traits reflecting the negative and positive valence systems. Behavioral assessments included emotional reactivity and distress tolerance tasks, during which skin conductance and heart rate were measured. Neuroimaging assessments included fear conditioning and a reward processing task. The latent variable structure underlying these measures was explored using sparse Bayesian group factor analysis. ResultsGroup factor analysis identified 11 latent variables explaining 31.2% of the variance across tasks, none of which loaded across units of analysis or tasks. Instead, variance was best explained by individual latent variables for each unit of analysis within each task. Post hoc analyses 1) showed associations with small effect sizes between latent variables that were derived separately from functional magnetic resonance imaging and self-report data and 2) showed that some latent variables are not directly related to individual valence system constructs. ConclusionsThe lack of latent structure across units of analysis highlights challenges of the Research Domain Criteria approach and suggests that while dimensional analyses work well to reveal within-task features, more targeted approaches are needed to reveal latent cross-modal relationships that could illuminate psychopathology.

Sparse Bayesian factor analysis for structural damage detection under unknown environmental conditions

Damage detection of civil engineering structures needs to consider the effect of normal environmental variations on structural dynamic properties. This study develops a novel structural damage detection method using factor analysis in the sparse Bayesian learning framework. The unknown changing environmental factors that affect the structural dynamic properties are treated as latent variables in the model. The automatic relevance determination prior is adopted for the factor loading matrix for model selection. All variables and parameters, including the factor loading matrix, error vector and latent variables, are solved using the iterative expectation-maximization technique. The variables are then used to reconstruct structural responses. The Euclidean norm of the error vector is calculated as the damage indicator to detect possible damage when limited vibration data are available. Two laboratory-tested examples are utilized to verify the effectiveness of the proposed method. Results demonstrate that the number of underlying environmental factors and structural damage can be accurately identified, even though the changing environmental data are unavailable. The proposed method has the advantages of online monitoring and automatic identification of underlying environmental factors.

Bayesian Joint Modeling of Chemical Structure and Dose Response Curves

Today there are approximately 85,000 chemicals regulated under the Toxic Substances Control Act, with around 2,000 new chemicals introduced each year. It is impossible to screen all of these chemicals for potential toxic effects either via full organism in vivo studies or in vitro high-throughput screening (HTS) programs. Toxicologists face the challenge of choosing which chemicals to screen, and predicting the toxicity of as-yet-unscreened chemicals. Our goal is to describe how variation in chemical structure relates to variation in toxicological response to enable in silico toxicity characterization designed to meet both of these challenges. With our Bayesian partially Supervised Sparse and Smooth Factor Analysis (BS3FA) model, we learn a distance between chemicals targeted to toxicity, rather than one based on molecular structure alone. Our model also enables the prediction of chemical dose-response profiles based on chemical structure (that is, without in vivo or in vitro testing) by taking advantage of a large database of chemicals that have already been tested for toxicity in HTS programs. We show superior simulation performance in distance learning and modest to large gains in predictive ability compared to existing methods. Results from the high-throughput screening data application elucidate the relationship between chemical structure and a toxicity-relevant high-throughput assay. An R package for BS3FA is available online at https://github.com/kelrenmor/bs3fa.

Exploring gene-gene interaction in family-based data with an unsupervised machine learning method: EPISFA.

Gene-gene interaction (G × G) is thought to fill the gap between the estimated heritability of complex diseases and the limited genetic proportion explained by identified single-nucleotide polymorphisms. The current tools for exploring G × G were often developed for case-control designs with less considerations for their applications in families. Family-based studies are robust against bias led from population stratification in genetic studies and helpful in understanding G × G.We proposed a new algorithm epistasis sparse factor analysis (EPISFA) and epistasis sparse factor analysis for linkage disequilibrium (EPISFA-LD) based on unsupervised machine learning to screen G × G. Extensive simulations were performed to compare EPISFA/EPISFA-LD with a classical family-based algorithm FAM-MDR (family-based multifactor dimensionality reduction). The results showed that EPISFA/EPISFA-LD is a tool of both high power and computational efficiency that could be applied in family designs and is applicable within high-dimensionality datasets. Finally, we applied EPISFA/EPISFA-LD to a real dataset drawn from the Fangshan/family-based Ischemic Stroke Study in China. Five pairs of G × G were discovered by EPISFA/EPISFA-LD, including three pairs verified by other algorithms (FAM-MDR and logistic), and an additional two pairs uniquely identified by EPISFA/EPISFA-LD only. The results from EPISFA might offer new insights for understanding the genetic etiology of complex diseases. EPISFA/EPISFA-LD was implemented in R. All relevant source code as well as simulated data could be freely downloaded from https://github.com/doublexism/episfa.

A Comparison of Simple Structure Rotation Criteria in Temporal Exploratory Factor Analysis for Event-Related Potential Data

Abstract. It is challenging to apply exploratory factor analysis (EFA) to event-related potential (ERP) data because such data are characterized by substantial temporal overlap (i.e., large cross-loadings) between the factors, and, because researchers are typically interested in the results of subsequent analyses (e.g., experimental condition effects on the level of the factor scores). In this context, relatively small deviations in the estimated factor solution from the unknown ground truth may result in substantially biased estimates of condition effects (rotation bias). Thus, in order to apply EFA to ERP data researchers need rotation methods that are able to both recover perfect simple structure where it exists and to tolerate substantial cross-loadings between the factors where appropriate. We had two aims in the present paper. First, to extend previous research, we wanted to better understand the behavior of the rotation bias for typical ERP data. To this end, we compared the performance of a variety of factor rotation methods under conditions of varying amounts of temporal overlap between the factors. Second, we wanted to investigate whether the recently proposed component loss rotation is better able to decrease the bias than traditional simple structure rotation. The results showed that no single rotation method was generally superior across all conditions. Component loss rotation showed the best all-round performance across the investigated conditions. We conclude that Component loss rotation is a suitable alternative to simple structure rotation. We discuss this result in the light of recently proposed sparse factor analysis approaches.

SOFAR: Large-Scale Association Network Learning.

Many modern big data applications feature large scale in both numbers of responses and predictors. Better statistical efficiency and scientific insights can be enabled by understanding the large-scale response-predictor association network structures via layers of sparse latent factors ranked by importance. Yet sparsity and orthogonality have been two largely incompatible goals. To accommodate both features, in this paper we suggest the method of sparse orthogonal factor regression (SOFAR) via the sparse singular value decomposition with orthogonality constrained optimization to learn the underlying association networks, with broad applications to both unsupervised and supervised learning tasks such as biclustering with sparse singular value decomposition, sparse principal component analysis, sparse factor analysis, and spare vector autoregression analysis. Exploiting the framework of convexity-assisted nonconvex optimization, we derive nonasymptotic error bounds for the suggested procedure characterizing the theoretical advantages. The statistical guarantees are powered by an efficient SOFAR algorithm with convergence property. Both computational and theoretical advantages of our procedure are demonstrated with several simulations and real data examples.

Advanced Dietary Patterns Analysis Using Sparse Latent Factor Models in Young Adults

Principal components analysis (PCA) has been the most widely used method for deriving dietary patterns to date. However, PCA requires arbitrary ad hoc decisions for selecting food variables in interpreting dietary patterns and does not easily accommodate covariates. Sparse latent factor models can be utilized to address these issues. The objective of this study was to compare Bayesian sparse latent factor models with PCA for identifying dietary patterns among young adults. Habitual food intake was estimated in 2730 sedentary young adults from the Training Interventions and Genetics of Exercise Response (TIGER) Study [aged 18-35 y; body mass index (BMI; in kg/m2): 26.5±6.1] who exercised <30 min/wk during the previous 30 d without restricting caloric intake before study enrollment. A food-frequency questionnaire was used to generate the frequency intakes of 102 food items. Sparse latent factor modeling was applied to the standardized food intakes to derive dietary patterns, incorporating additional covariates (sex, race/ethnicity, and BMI). The identified dietary patterns via sparse latent factor modeling were compared with the PCA derived dietary patterns. Seven dietary patterns were identified in both PCA and sparse latent factor analysis. In contrast to PCA, the sparse latent factor analysis allowed the covariate information to be jointly accounted for in the estimation of dietary patterns in the model and offered probabilistic criteria to determine the foods relevant to each dietary pattern. The derived patterns from both methods generally described common dietary behaviors. Dietary patterns 1-4 had similar food subsets using both statistical approaches, but PCA had smaller sets of foods with more cross-loading elements between the 2 factors. Overall, the sparse latent factor analysis produced more interpretable dietary patterns, with fewer of the food items excluded from all patterns. Sparse latent factor models can be useful in future studies of dietary patterns by reducing the intrinsic arbitrariness involving the choice of food variables in interpreting dietary patterns and incorporating covariates in the assessment of dietary patterns.

Molecular characterization of breast and lung tumors by integration of multiple data types with functional sparse-factor analysis.

Effective cancer treatment is crucially dependent on the identification of the biological processes that drive a tumor. However, multiple processes may be active simultaneously in a tumor. Clustering is inherently unsuitable to this task as it assigns a tumor to a single cluster. In addition, the wide availability of multiple data types per tumor provides the opportunity to profile the processes driving a tumor more comprehensively. Here we introduce Functional Sparse-Factor Analysis (funcSFA) to address these challenges. FuncSFA integrates multiple data types to define a lower dimensional space capturing the relevant variation. A tailor-made module associates biological processes with these factors. FuncSFA is inspired by iCluster, which we improve in several key aspects. First, we increase the convergence efficiency significantly, allowing the analysis of multiple molecular datasets that have not been pre-matched to contain only concordant features. Second, FuncSFA does not assign tumors to discrete clusters, but identifies the dominant driver processes active in each tumor. This is achieved by a regression of the factors on the RNA expression data followed by a functional enrichment analysis and manual curation step. We apply FuncSFA to the TCGA breast and lung datasets. We identify EMT and Immune processes common to both cancer types. In the breast cancer dataset we recover the known intrinsic subtypes and identify additional processes. These include immune infiltration and EMT, and processes driven by copy number gains on the 8q chromosome arm. In lung cancer we recover the major types (adenocarcinoma and squamous cell carcinoma) and processes active in both of these types. These include EMT, two immune processes, and the activity of the NFE2L2 transcription factor. We validate the breast cancer findings on the METABRIC set and demonstrate the translatability of the TCGA breast cancer factors to METABRIC. In summary, FuncSFA is a robust method to perform discovery of key driver processes in a collection of tumors through unsupervised integration of multiple molecular data types and functional annotation.

Generalized Bayesian Factor Analysis for Integrative Clustering with Applications to Multi-Omics Data.

Integrative clustering is a clustering approach for multiple datasets, which provide different views of a common group of subjects. It enables analyzing multi-omics data jointly to, for example, identify the subtypes of diseases, cells, and so on, capturing the complex underlying biological processes more precisely. On the other hand, there has been a great deal of interest in incorporating the prior structural knowledge on the features into statistical analyses over the past decade. The knowledge on the gene regulatory network (pathways) can potentially be incorporated into many genomic studies. In this paper, we propose a novel integrative clustering method which can incorporate the prior graph knowledge. We first develop a generalized Bayesian factor analysis (GBFA) framework, a sparse Bayesian factor analysis which can take into account the graph information. Our GBFA framework employs the spike and slab lasso (SSL) prior to impose sparsity on the factor loadings and the Markov random field (MRF) prior to encourage smoothing over the adjacent factor loadings, which establishes a unified shrinkage adaptive to the loading size and the graph structure. Then, we use the framework to extend iCluster+, a factor analysis based integrative clustering approach. A novel variational EM algorithm is proposed to efficiently estimate the MAP estimator for the factor loadings. Extensive simulation studies and the application to the NCI60 cell line dataset demonstrate that the propose method is superior and delivers more biologically meaningful outcomes.

Sparse multiple factor analysis to integrate genetic data, neuroimaging features, and attention-deficit/hyperactivity disorder domains.

We proposed the application of a multivariate cross-sectional framework based on a combination of a variable selection method and a multiple factor analysis (MFA) in order to identify complex meaningful biological signals related to attention-deficit/hyperactivity disorder (ADHD) symptoms and hyperactivity/inattention domains. The study included 135 children from the general population with genomic and neuroimaging data. ADHD symptoms were assessed using a questionnaire based on ADHD-DSM-IV criteria. In all analyses, the raw sum scores of the hyperactivity and inattention domains and total ADHD were used. The analytical framework comprised two steps. First, zero-inflated negative binomial linear model via penalized maximum likelihood (LASSO-ZINB) was performed. Second, the most predictive features obtained with LASSO-ZINB were used as input for the MFA. We observed significant relationships between ADHD symptoms and hyperactivity and inattention domains with white matter, gray matter regions, and cerebellum, as well as with loci within chromosome 1. Multivariate methods can be used to advance the neurobiological characterization of complex diseases, improving the statistical power with respect to univariate methods, allowing the identification of meaningful biological signals in Imaging Genetic studies.

Structure and stability of genetic variance-covariance matrices: A Bayesian sparse factor analysis of transcriptional variation in the three-spined stickleback.

The genetic variance-covariance matrix (G) is a quantity of central importance in evolutionary biology due to its influence on the rate and direction of multivariate evolution. However, the predictive power of empirically estimated G-matrices is limited for two reasons. First, phenotypes are high-dimensional, whereas traditional statistical methods are tuned to estimate and analyse low-dimensional matrices. Second, the stability of G to environmental effects and over time remains poorly understood. Using Bayesian sparse factor analysis (BSFG) designed to estimate high-dimensional G-matrices, we analysed levels variation and covariation in 10,527 expressed genes in a large (n=563) half-sib breeding design of three-spined sticklebacks subject to two temperature treatments. We found significant differences in the structure of G between the treatments: heritabilities and evolvabilities were higher in the warm than in the low-temperature treatment, suggesting more and faster opportunity to evolve in warm (stressful) conditions. Furthermore, comparison of G and its phenotypic equivalent P revealed the latter is a poor substitute of the former. Most strikingly, the results suggest that the expected impact of G on evolvability-as well as the similarity among G-matrices-may depend strongly on the number of traits included into analyses. In our results, the inclusion of only few traits in the analyses leads to underestimation in the differences between the G-matrices and their predicted impacts on evolution. While the results highlight the challenges involved in estimating G, they also illustrate that by enabling the estimation of large G-matrices, the BSFG method can improve predicted evolutionary responses to selection.