Factor Loading Matrix Research Articles

Selecting the number of factors in approximate factor models using group variable regularization

. We propose a novel method for the estimation of the number of factors in approximate factor models. The model is based on a penalized maximum likelihood approach incorporating an adaptive hierarchical lasso penalty function that enables setting entire columns of the factor loadings matrix to zero, which corresponds to removing uninformative factors. Additionally, the model is capable of estimating weak factors by allowing for sparsity in the non zero columns. We prove that the proposed estimator consistently estimates the true number of factors. Simulation experiments reveal superior selection accuracies for our method in finite samples over existing approaches, especially for data with cross-sectional and serial correlations. In an empirical application on a large macroeconomic dataset, we show that the mean squared forecast errors of an approximate factor model are lower if the number of included factors is selected by our method.

Structured prior distributions for the covariance matrix in latent factor models

Factor models are widely used for dimension reduction in the analysis of multivariate data. This is achieved through decomposition of a p×p\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$p \ imes p$$\\end{document} covariance matrix into the sum of two components. Through a latent factor representation, they can be interpreted as a diagonal matrix of idiosyncratic variances and a shared variation matrix, that is, the product of a p×k\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$p \ imes k$$\\end{document} factor loadings matrix and its transpose. If k≪p\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$k \\ll p$$\\end{document}, this defines a parsimonious factorisation of the covariance matrix. Historically, little attention has been paid to incorporating prior information in Bayesian analyses using factor models where, at best, the prior for the factor loadings is order invariant. In this work, a class of structured priors is developed that can encode ideas of dependence structure about the shared variation matrix. The construction allows data-informed shrinkage towards sensible parametric structures while also facilitating inference over the number of factors. Using an unconstrained reparameterisation of stationary vector autoregressions, the methodology is extended to stationary dynamic factor models. For computational inference, parameter-expanded Markov chain Monte Carlo samplers are proposed, including an efficient adaptive Gibbs sampler. Two substantive applications showcase the scope of the methodology and its inferential benefits.

Assessment of cumulative incidences of mung bean crop production in lowlands of south Ethiopia through multiple factor analysis

Ethiopian farmers face a combination of risks in agricultural production during a single crop season. However, cumulative incidence assessment of the risks in the agriculture was not researched. Thus, this study aimed at assessing the cumulative incidence components in the production of the mung bean crop in the lowlands of South Ethiopia. A cross-sectional study was used with 384 mung bean farmers selected from lowlands of Gamo, Gofa and Wolaita zones. Thirteen original risk types were used to be categorized into a few factors through multiple factor analysis. Factor analysis identified latent variables with an Eigenvalues greater than one. The economic risk factor, the climate-related risk factor, and the systematic risk factor were the first three factors that explain 92.87 % of the inertia cumulatively. The rotated factor loading matrix indicated that under factor one, the encountered risks are an increase in the price of inputs, an increase in the price of food, and financial incidence, which accounted 50 % of the total variance. Under factor two extended dry spells, crop diseases, and wild animals’ damage and livestock diseases and deaths variables that accounted for 23.24 % of the total variance. The risk types categorized under systematic risks were social risks and political risks that accounted for 19.61 % of the total variance of risks encountered by mung bean farmers in the study area. Therefore, disentangling systematic risks is important by providing much-needed information to mung bean farmers and policymakers regarding systematic risk management priorities.

Identification of Bletilla striata and related decoction pieces: a data fusion method combining electronic nose, electronic tongue, electronic eye, and high-performance liquid chromatography data.

Introduction: We here describe a new method for distinguishing authentic Bletilla striata from similar decoctions (namely, Gastrodia elata, Polygonatum odoratum, and Bletilla ochracea schltr). Methods: Preliminary identification and analysis of four types of decoction pieces were conducted following the Chinese Pharmacopoeia and local standards. Intelligent sensory data were then collected using an electronic nose, an electronic tongue, and an electronic eye, and chromatography data were obtained via high-performance liquid chromatography (HPLC). Partial least squares discriminant analysis (PLS-DA), support vector machines (SVM), and back propagation neural network (BP-NN) models were built using each set of single-source data for authenticity identification (binary classification of B. striata vs. other samples) and for species determination (multi-class sample identification). Features were extracted from all datasets using an unsupervised approach [principal component analysis (PCA)] and a supervised approach (PLS-DA). Mid-level data fusion was then used to combine features from the four datasets and the effects of feature extraction methods on model performance were compared. Results and Discussion: Gas chromatography-ion mobility spectrometry (GC-IMS) showed significant differences in the types and abundances of volatile organic compounds between the four sample types. In authenticity determination, the PLS-DA and SVM models based on fused latent variables (LVs) performed the best, with 100% accuracy in both the calibration and validation sets. In species identification, the PLS-DA model built with fused principal components (PCs) or fused LVs had the best performance, with 100% accuracy in the calibration set and just one misclassification in the validation set. In the PLS-DA and SVM authenticity identification models, fused LVs performed better than fused PCs. Model analysis was used to identify PCs that strongly contributed to accurate sample classification, and a PC factor loading matrix was used to assess the correlation between PCs and the original variables. This study serves as a reference for future efforts to accurately evaluate the quality of Chinese medicine decoction pieces, promoting medicinal formulation safety.

Factor-augmented Model for Functional Data

We propose modeling raw functional data as a mixture of a smooth function and a high-dimensional factor component. The conventional approach to retrieving the smooth function from the raw data is through various smoothing techniques. However, the smoothing model is inadequate to recover the smooth curve or capture the data variation in some situations. These include cases where there is a large amount of measurement error, the smoothing basis functions are incorrectly identified, or the step jumps in the functional mean levels are neglected. A factor-augmented smoothing model is proposed to address these challenges, and an iterative numerical estimation approach is implemented in practice. Including the factor model component in the proposed method solves the aforementioned problems since a few common factors often drive the variation that cannot be captured by the smoothing model. Asymptotic theorems are also established to demonstrate the effects of including factor structures on the smoothing results. Specifically, we show that the smoothing coefficients projected on the complement space of the factor loading matrix are asymptotically normal. As a byproduct of independent interest, an estimator for the population covariance matrix of the raw data is presented based on the proposed model. Extensive simulation studies illustrate that these factor adjustments are essential in improving estimation accuracy and avoiding the curse of dimensionality. The superiority of our model is also shown in modeling Australian temperature data.

인자의 불확정성을 반영한 인자점수 추출 및 인자점수 회귀모형에의 응용

Factor analysis is a multivariate analysis method widely used to extract latent factors from high dimensional data and interpret multidimensional data through dimension reduction. Additionally, factor scores of individual observations for the extracted factors are calculated and used in a subsequent regression analysis. This factor model has the characteristics of parameter indeterminacy which means parameters such as the factor loading matrix or unique variance are not uniquely determined, and factor indeterminacy which represents that latent factors are not uniquely determined even in situations where the parameters of the factor model are determined. In this paper, we examine the indeterminacy of factor scores and propose a new method of calculating factor scores that maximize the correlation coefficient between a response variable and a factor in a factor score regression model. Simulations and real data analysis were conducted to compare the existing factor scores using the regression method and the proposed factor scores utilizing the indeterminate part of factors in factor score regression setting. As a result of data analysis, it was confirmed that the estimation accuracy of regression model was higher when using the proposed factor scores.

A method to evaluate the rank condition for CCE estimators

We develop a binary classifier to evaluate whether the rank condition (RC) is satisfied or not for the Common Correlated Effects (CCE) estimator. The RC postulates that the number of unobserved factors, m, is not larger than the rank of the unobserved matrix of average factor loadings, ϱ. When this condition fails, the CCE estimator is inconsistent, in general. Despite its importance, to date this rank condition could not be verified. The difficulty lies in the fact that factor loadings are unobserved, such that ϱ cannot be directly determined. The key insight in this article is that ϱ can be consistently estimated with existing techniques through the matrix of cross-sectional averages of the data. Similarly, m can be estimated consistently from the data using existing methods. Thus, a binary classifier, constructed by comparing estimates of m and ϱ, correctly determines whether the RC is satisfied or not as . We illustrate the practical relevance of testing the RC by studying the effect of the Dodd-Frank Act on bank profitability. The RC classifier reveals that the rank condition fails for a subperiod of the sample, in which case the estimated effect of bank size on profitability appears to be biased upwards.

Performance of Principal Components Estimation Method on the Quality of Factor Analysis without and with Varimax Rotation

The objective of this work was to evaluate the performance of the principal components estimation method on the quality of estimates of some parameters of exploratory factor analysis (EFA), without and with varimax rotation. To this end, 18 parametric correlation matrices (r) were imposed. These matrices were obtained from combinations between six different values of parametric communalities of four normally distributed random variables with three different proportions of distances between the parametric factorial loadings of the first two orthogonal factors. For each matrix r, the following parameters were defined: the first two eigenvalues (l1 and l2), the matrix of factor loadings (G), the four communalities () and the matrix of the specific factors (Y). After the 36 factor analyses, the respective estimates of these parameters were obtained, and their respective absolute deviations between the estimates obtained a posteriori and the parameters known a priori were evaluated. With the results of the Student's t test at 5% significance applied to the response surface analysis, it was concluded that the principal components estimation method for estimating orthogonal factors was not adequate and the varimax rotation improved relatively little the quality of the SFA estimates.

Airpak-based art and design talent training innovation model framework

Abstract In this paper, the 15 original variables of the Airpak-based influence factor scale for art and design talent training were first determined, and the five-level scoring method was applied to score the 15 influence factors. Then, we used factor analysis to factorize the data, standardize the sample data, construct a coordinate system with common factors, and use the projection of each variable in the coordinate system instead of the original variables to construct the correlation coefficient matrix R. Then, we solved the correlation coefficient matrix between each variable, selected the number of common factors, and calculated the eigenvectors of the factor loading matrix A to construct the influence factor analysis model. Finally, the model is used to analyze the weight of each influencing factor of Airpak-based art and design talent cultivation and to build an innovative path for talent cultivation. In terms of students’ own factors, the influence weight of learning enthusiasm on talent cultivation is 0.48, and the influence weights of learning achievement and design talent are 0.43 and 0.545, respectively; in terms of teaching level factors, the influence weights of teachers’ teaching ability, professional level and design experience on talent cultivation are 0.44, 0.5 and 0.37 respectively. The research of this paper has an important reference and reference value for the cultivation of art and design talents.

Dynamic ICAR Spatiotemporal Factor Models

We propose a novel class of dynamic factor models for spatiotemporal areal data. This novel class of models assumes that the spatiotemporal process may be represented by some few latent factors that evolve through time according to dynamic linear models. As the dimension of the vector of latent factors is typically much smaller than the number of subregions, our proposed class of models may achieve substantial dimension reduction. At each time point, the vector of observations is linearly related to the vector of latent factors through a matrix of factor loadings. Each column of this matrix may be seen as a vectorized map of factor loadings relating one latent factor to the vector of observations. Thus, to account for spatial dependence, we assume that each column of the matrix of factor loadings follows an intrinsic conditional autoregressive (ICAR) process. Hence, we call our class of models the Dynamic ICAR Spatiotemporal Factor Models (DIFM). We develop a Gibbs sampler for exploration of the posterior distribution. In addition, we develop model selection through a Laplace-Metropolis estimator of the predictive density. We present two case studies. The first case study, which is for simulated data, demonstrates that our DIFMs are identifiable and that our proposed inferential procedure works well at recovering the underlying data generating process. Finally, the second case study demonstrates the utility and flexibility of our DIFM framework with an application to the drug overdose epidemic in the United States from 2015 to 2021.

Bayesian Exploratory Factor Analysis via Gibbs Sampling

Bayesian methods to infer model dimensionality in factor analysis generally assume a lower triangular structure for the factor loadings matrix. Consequently, the ordering of the outcomes influences the results. Therefore, we propose a method to infer model dimensionality without imposing any prior restriction on the loadings matrix. Our approach considers a relatively large number of factors and includes auxiliary multiplicative parameters, which may render null the unnecessary columns in the loadings matrix. The underlying dimensionality is then inferred based on the number of nonnull columns in the factor loadings matrix, and the model parameters are estimated with a postprocessing scheme. The advantages of the method in selecting the correct dimensionality are illustrated via simulations and using real data sets.

Model Construction of Influencing Factors of Physical Education Core Quality Cultivation Based on Multiple Linear Regression

Abstract This paper mainly discusses the formation and development of the core literacy of sports majors. Then, this paper establishes the factor model that affects the professional quality of youth sports employing multiple linear regression analysis. First, this paper uses factor analysis to observe 19-factor patterns. After the maximum variance rotation method, four dimensions were determined from the factor loading matrix, and confirmatory factor analysis was performed. Finally, this paper uses the multiple linear regression method to establish young students’ core literacy education model. This paper analyzes the model and draws the influence of four factors of school, family, society and individual on adolescents.

Integrative analysis of multi-omics and imaging data with incorporation of biological information via structural Bayesian factor analysis.

With the rapid development of modern technologies, massive data are available for the systematic study of Alzheimer's disease (AD). Though many existing AD studies mainly focus on single-modality omics data, multi-omics datasets can provide a more comprehensive understanding of AD. To bridge this gap, we proposed a novel structural Bayesian factor analysis framework (SBFA) to extract the information shared by multi-omics data through the aggregation of genotyping data, gene expression data, neuroimaging phenotypes and prior biological network knowledge. Our approach can extract common information shared by different modalities and encourage biologically related features to be selected, guiding future AD research in a biologically meaningful way. Our SBFA model decomposes the mean parameters of the data into a sparse factor loading matrix and a factor matrix, where the factor matrix represents the common information extracted from multi-omics and imaging data. Our framework is designed to incorporate prior biological network information. Our simulation study demonstrated that our proposed SBFA framework could achieve the best performance compared with the other state-of-the-art factor-analysis-based integrative analysis methods. We apply our proposed SBFA model together with several state-of-the-art factor analysis models to extract the latent common information from genotyping, gene expression and brain imaging data simultaneously from the ADNI biobank database. The latent information is then used to predict the functional activities questionnaire score, an important measurement for diagnosis of AD quantifying subjects' abilities in daily life. Our SBFA model shows the best prediction performance compared with the other factor analysis models. Code are publicly available at https://github.com/JingxuanBao/SBFA. qlong@upenn.edu.

Two-Stage Procrustes Rotation with Sparse Target Matrix and Least Squares Criterion with Regularization and Generalized Weighting

In factor analysis, a factor loading matrix is often rotated to a simple target matrix for its simplicity. For the purpose, Procrustes rotation minimizes the discrepancy between the target and rotated loadings using two types of approximation: 1) approximate the zeros in the target by the non-zeros in the loadings, and 2) approximate the non-zeros in the target by the non-zeros in the loadings. The central issue of Procrustes rotation considered in the article is that it equally treats the two types of approximation, while the former is more important for simplifying the loading matrix. Furthermore, a well-known issue of Simplimax is the computational inefficiency in estimating the sparse target matrix, which yields a considerable number of local minima. The research proposes a new rotation procedure that consists of the following two stages. The first stage estimates sparse target matrix with lesser computational cost by regularization technique. In the second stage, a loading matrix is rotated to the target, emphasizing on the approximation of non-zeros to zeros in the target by least squares criterion with generalized weighing that is newly proposed by the study. The simulation study and real data examples revealed that the proposed method surely simplifies loading matrices.

Exploring Oblique Rotation Factor to Restructure Deep Hyperspectral Image Classification

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Factor analysis (FA)</i> is commonly used in fields such as economics and now being introduced as a new tool on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dimensionality reduction (DR)</i> for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hyperspectral image classification (HSIC)</i> , but FA usually employed orthogonal rotation to directly maximize the separation among factors, which would oversimplify the relationships between variables and factors, worse still, the orthogonal rotation often distorts the true relationships between underlying traits in real life and can not always accurately represent these relationships. To this end, this letter proposes a DR algorithm about FA based on oblique rotation Oblimax to improve HSIC. Firstly, the common factors are extracted from the hyperspectral data to form a factor loading matrix which will be obliquely rotated, then its factor score is estimated to obtain the eigen dimensions for the hyperspectral data, thus realizing DR. On the basis of the successful DR, a deep classifier is constructed, specially, a double-branch structure about <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3 dimensional-convolutional neural networks (3D-CNN)</i> with different sizes is restructured to extract multi-scale spatial-spectral features, and early fusion is performed on the features, then <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2 dimensional-convolutional neural networks (2D-CNN)</i> is restructured to reduce the computational complexity and learns more spatial features. Finally, the accuracy of the proposed algorithm on the datasets Indian Pines, Kennedy Space Center and Muufl Gulfport, respectively achieves 99.78%, 99.95% and 95.57%. It shows that the proposed algorithm in this letter has advantages in improving the classification accuracy and reducing the complexity of computation.

Gaussian Orthogonal Latent Factor Processes for Large Incomplete Matrices of Correlated Data

We introduce Gaussian orthogonal latent factor processes for modeling and predicting large correlated data. To handle the computational challenge, we first decompose the likelihood function of the Gaussian random field with a multi-dimensional input domain into a product of densities at the orthogonal components with lower-dimensional inputs. The continuous-time Kalman filter is implemented to compute the likelihood function efficiently without making approximations. We also show that the posterior distribution of the factor processes is independent, as a consequence of prior independence of factor processes and orthogonal factor loading matrix. For studies with large sample sizes, we propose a flexible way to model the mean, and we derive the marginal posterior distribution to solve identifiability issues in sampling these parameters. Both simulated and real data applications confirm the outstanding performance of this method.

Detection of weathering effects and mineral additive content of LDPE composites using hyperspectral imaging

The use of hyperspectral imaging and multivariate analysis to detect weathering effects and mineral additive content in LDPE composites is demonstrated. Principal component analysis of hyperspectral images was used to obtain models of composite gradation by additive content and presence of weathering effects. The analysis of the factor loading matrix of spectral wavelength reflectance coefficients revealed the most significant bands contributing to the gradation of composite samples by additive content and presence of weathering effects.

DISTRIBUTION OF HEAVY METALS IN SOILS ON THE TERRITORY OF THE SMALL TOWN (ON THE EXAMPLE OF NESVIZH)

The results of geochemical studies of soils on the territory of Nesvizh are presented in the context of the distribution of heavy metals. Statistical data processing has been performed. For the purposes of the study, the regional and local geochemical background was calculated. The features of enrichment and pollution of urban soils by the studied components are revealed. The concentration coefficients and the total pollution coefficient are calculated. Monocomponent and polycomponent maps were constructed, reflecting the revealed regularities in the distribution of heavy metals. The results of correlation and factor analyses are presented. Associations of heavy metals, typical for the soils of the city, were revealed. Factor loadings are calculated. A graphical representation of the matrix of factor loadings is given. The factors responsible for the formation of the technogenic situation have been established, and their contribution has been determined. The map of factor loadings has been constructed. The interpretation of geochemical data both at the level of individual points and in the space of the entire city was carried out.

A regularization method for linking brain and behavior.

In a world of big data and computational resources, there has been a growing interest in further validating computational models of decision making by subjecting them to more rigorous constraints. One prominent area of study is model-based cognitive neuroscience, where measures of neural activity are explained and interpreted through the lens of a cognitive model. Although some early work has developed the statistical framework for exploiting the covariation between brain and behavior through factor analysis linking functions, current methods are still far from providing parsimonious accounts of high-dimensional (e.g., voxel-level) data. In this article, we contribute to this endeavor by investigating the fidelity of regularization methods such as the Lasso. Here, a combination of local and global penalty terms are applied to pressure elements of the factor loading matrix toward zero, reducing the false alarm rate. Such penalties facilitate the emergence of parsimonious network structure in the study of neural activation, giving way to clearer interpretations of high-dimensional data. We show through a set of three simulation studies and one application to real data that the Lasso can be an effective regularization method in the context of linking complex patterns of brain data to theoretical explanations of decisions. Although our analyses are specific to linking brain to behavior, the structure of the model is invariant to the type of high-dimensional data under investigation. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Quasi-maximum likelihood estimation of break point in high-dimensional factor models

This paper proposes a quasi-maximum likelihood (QML) estimator of the break point for large-dimensional factor models with a single structural break in the factor loading matrix. We show that the QML estimator is consistent for the true break point when the covariance matrix of the pre- or post-break factor loading (or both) is singular. Consistency here means that the deviation of the estimated break date from the true break date k0 converges to zero as the sample size grows. This is a much stronger result than the break fraction kˆ/T being T-consistent (super-consistent) for k0/T. Also, singularity occurs for most types of structural changes, except for a rotational change. Even for a rotational change, the QML estimator is still T-consistent in terms of the break fraction. Simulation results confirm the theoretical properties of our estimator, and it significantly outperforms existing estimators for change points in factor models.