Loading Matrices Research Articles

Formation of the Silicon-Containing Porous Carrier Materials Using Bacterial Cells

Four types of the loading matrices based on triethoxymethylsilane and tetraethoxysilane in the 50/50 vol % ratio were prepared. The matrices were obtained either using Paracoccus yeei bacterial cells VKM B-3302 as templates or without them. The materials were annealed at 600 and 800 °C. The effect of the cells on the resulting loading matrices was analyzed using scanning electron microscopy. It was found that the addition of P.yeei allows for the formation of the more branched surface, which is likely to result in a higher specific surface area.

Spatiotemporal factor models for functional data with application to population map forecast

The proliferation of mobile devices has led to the collection of large amounts of population data. This situation has prompted the need to utilize this rich, multidimensional data in practical applications. In response to this trend, we have integrated functional data analysis (FDA) and factor analysis to address the challenge of predicting hourly population changes across various districts in Tokyo. Specifically, by assuming a Gaussian process, we avoided the large covariance matrix parameters of the multivariate normal distribution. In addition, the data were both time and spatially dependent between districts. To capture various characteristics, a Bayesian factor model was introduced, which modeled the time series of a small number of common factors and expressed the spatial structure through factor loading matrices. Furthermore, the factor loading matrices were made identifiable and sparse to ensure the interpretability of the model. We also proposed a Bayesian shrinkage method as a systematic approach for factor selection. Through numerical experiments and data analysis, we investigated the predictive accuracy and interpretability of our proposed method. We concluded that the flexibility of the method allows for the incorporation of additional time series features, thereby improving its accuracy.

MacroPARAFAC for handling rowwise and cellwise outliers in incomplete multiway data

Multiway data extend two-way matrices into higher-dimensional tensors, often explored through dimensional reduction techniques. In this paper, we study the Parallel Factor Analysis (PARAFAC) model for handling multiway data, representing it more compactly through a concise set of loading matrices and scores. We assume that the data may be incomplete and could contain both rowwise and cellwise outliers, signifying cases that deviate from the majority and outlying cells dispersed throughout the data array. To address these challenges, we present a novel algorithm designed to robustly estimate both loadings and scores. Additionally, we introduce an enhanced outlier map to distinguish various patterns of outlying behavior. Through simulations and the analysis of fluorescence Excitation-Emission Matrix (EEM) data, we demonstrate the robustness of our approach. Our results underscore the effectiveness of diagnostic tools in identifying and interpreting unusual patterns within the data.

Tensor factor adjustment for image classification with pervasive noises

AbstractThis paper studies a tensor factor model that augments samples from multiple classes. The nuisance common patterns shared across classes are characterised by pervasive noises, and the patterns that distinguish different classes are represented by class‐specific components. Additionally, the pervasive component is modelled by the production of a low‐rank tensor latent factor and several factor loading matrices. This augmented tensor factor model can be expanded to a series of matrix variate tensor factor models and estimated using principal component analysis. The ranks of latent factors are estimated using a modified eigen‐ratio method. The proposed estimators have fast convergence rates and enjoy the blessing of dimensionality. The proposed factor model is applied to address the challenge of overlapping issues in image classification through a factor adjustment procedure. The procedure is shown to be powerful through synthetic experiments and an application to COVID‐19 pneumonia diagnosis from frontal chest X‐ray images.

Modeling and Learning on High-Dimensional Matrix-Variate Sequences

We propose a new matrix factor model, named RaDFaM, which is strictly derived from the general rank decomposition and assumes a high-dimensional vector factor model structure for each basis vector. RaDFaM contributes a novel class of low-rank latent structures that trade off between signal intensity and dimension reduction from a tensor subspace perspective. Based on the intrinsic separable covariance structure of RaDFaM, for a collection of matrix-valued observations, we derive a new class of PCA variants for estimating loading matrices, and sequentially the latent factor matrices. The peak signal-to-noise ratio of RaDFaM is proved to be superior in the category of PCA-type estimators. We also establish an asymptotic theory including the consistency, convergence rates, and asymptotic distributions for components in the signal part. Numerically, we demonstrate the performance of RaDFaM in applications such as matrix reconstruction, supervised learning, and clustering, on uncorrelated and correlated data, respectively. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Semi-parametric tensor factor analysis by iteratively projected singular value decomposition

Abstract This paper introduces a general framework of Semi-parametric TEnsor Factor Analysis (STEFA) that focuses on the methodology and theory of low-rank tensor decomposition with auxiliary covariates. Semi-parametric TEnsor Factor Analysis models extend tensor factor models by incorporating auxiliary covariates in the loading matrices. We propose an algorithm of iteratively projected singular value decomposition (IP-SVD) for the semi-parametric estimation. It iteratively projects tensor data onto the linear space spanned by the basis functions of covariates and applies singular value decomposition on matricized tensors over each mode. We establish the convergence rates of the loading matrices and the core tensor factor. The theoretical results only require a sub-exponential noise distribution, which is weaker than the assumption of sub-Gaussian tail of noise in the literature. Compared with the Tucker decomposition, IP-SVD yields more accurate estimators with a faster convergence rate. Besides estimation, we propose several prediction methods with new covariates based on the STEFA model. On both synthetic and real tensor data, we demonstrate the efficacy of the STEFA model and the IP-SVD algorithm on both the estimation and prediction tasks.

The sparse dynamic factor model: a regularised quasi-maximum likelihood approach

The concepts of sparsity, and regularised estimation, have proven useful in many high-dimensional statistical applications. Dynamic factor models (DFMs) provide a parsimonious approach to modelling high-dimensional time series, however, it is often hard to interpret the meaning of the latent factors. This paper formally introduces a class of sparse DFMs whereby the loading matrices are constrained to have few non-zero entries, thus increasing interpretability of factors. We present a regularised M-estimator for the model parameters, and construct an efficient expectation maximisation algorithm to enable estimation. Synthetic experiments demonstrate consistency in terms of estimating the loading structure, and superior predictive performance where a low-rank factor structure may be appropriate. The utility of the method is further illustrated in an application forecasting electricity consumption across a large set of smart meters.

When It Counts—Econometric Identification of the Basic Factor Model Based on GLT Structures

Despite the popularity of factor models with simple loading matrices, little attention has been given to formally address the identifiability of these models beyond standard rotation-based identification such as the positive lower triangular (PLT) constraint. To fill this gap, we review the advantages of variance identification in simple factor analysis and introduce the generalized lower triangular (GLT) structures. We show that the GLT assumption is an improvement over PLT without compromise: GLT is also unique but, unlike PLT, a non-restrictive assumption. Furthermore, we provide a simple counting rule for variance identification under GLT structures, and we demonstrate that within this model class, the unknown number of common factors can be recovered in an exploratory factor analysis. Our methodology is illustrated for simulated data in the context of post-processing posterior draws in sparse Bayesian factor analysis.

Sample canonical correlation coefficients of high-dimensional random vectors with finite rank correlations

Consider two random vectors x˜=Az+C11∕2x∈Rp and y˜=Bz+C21∕2y∈Rq, where x∈Rp, y∈Rq and z∈Rr are independent random vectors with i.i.d. entries of zero mean and unit variance, C1 and C2 are p×p and q×q deterministic population covariance matrices, and A and B are p×r and q×r deterministic factor loading matrices. With n independent observations of x˜ and y˜, we study the sample canonical correlations between them. Under the sharp fourth moment condition on the entries of x, y and z, we prove the BBP transition for the sample canonical correlation coefficients (CCCs). More precisely, if a population CCC is below a threshold, then the corresponding sample CCC converges to the right edge of the bulk eigenvalue spectrum of the sample canonical correlation matrix and satisfies the famous Tracy-Widom law; if a population CCC is above the threshold, then the corresponding sample CCC converges to an outlier that is detached from the bulk eigenvalue spectrum. We prove our results in full generality, in the sense that they also hold for near-degenerate population CCCs and population CCCs that are close to the threshold.

A fast parallel tensor decomposition with optimal stochastic gradient descent: an application in structural damage identification

AbstractStructural health monitoring (SHM) provides an economic approach which aims to enhance understanding the behavior of structures by continuously collecting data through multiple networked sensors attached to the structure. These data are then utilized to gain insight into the health of a structure and make timely and economic decisions about its maintenance. The generated SHM sensing data are non-stationary and exist in a correlated multi-way form which makes the batch/off-line learning and standard two-way matrix analysis unable to capture all of these correlations and relationships. In this sense, the online tensor data analysis has become an essential tool for capturing underlying structures in higher-order datasets stored in a tensor $${\mathcal {X}} \in {\mathbb {R}} ^{I_1 \times \cdots \times I_N} $$ X ∈ R I 1 × ⋯ × I N . The CANDECOMP/PARAFAC (CP) decomposition has been extensively studied and applied to approximate X by N loading matrices $$A(1),\ldots ,A(N)$$ A ( 1 ) , … , A ( N ) where N represents the order of the tensor. We propose a novel algorithm, FP-CPD, to parallelize the CANDECOMP/PARAFAC (CP) decomposition of a tensor $${\mathcal {X}} \in {\mathbb {R}} ^{I_1 \times \cdots \times I_N} $$ X ∈ R I 1 × ⋯ × I N . Our approach is based on stochastic gradient descent (SGD) algorithm which allows us to parallelize the learning process, and it is very useful in online setting since it updates $${\mathcal {X}}^{t+1}$$ X t + 1 in one single step. Our SGD algorithm is augmented with Nesterov’s accelerated gradient and perturbation methods to accelerate and guarantee convergence. The experimental results using laboratory-based and real-life structural datasets indicate fast convergence and good scalability.

Nonlinear excitation and mesh characteristics model for spiral bevel gears

Accurate and rapid calculation of nonlinear excitation (NE) and loaded mesh characteristics (LMC) of gear pairs is a key to achieving rapid iterative optimization of gear system designs. Therefore, an analytical model (AM) was proposed for the accurate and numerically efficient calculation of NE and LMC for application to spiral bevel gears (SBG). First, a tooth contact analysis of an SBG was completed using Coons surface technology and mesh theory to determine the contact path and unloaded transmission error (UTE). Thus, an AM for the principal and relative normal curvatures of the SBG was derived using the Euler's formula. Based on the traditional calculation method for a Hertz contact ellipse, the relation between the relative curvatures was introduced to derive the formulas for the major and minor axes using the Muller's method, which could improve the programmability and avoid numerical instability of the program. Then, analytical models for the contact ellipse and pressure distribution on the contact tooth surfaces were derived using the Hertz model and geometric theory. Subsequently, based on a single tooth LMC, infinitesimal method, elasticity, and series stiffness model, a single mesh stiffness (SMS) model for SBGs was developed, which incorporates transverse tooth stiffness, transverse gear foundation stiffness, axial tooth stiffness, axial gear foundation stiffness, and Hertz nonlinear contact stiffness. Based on the SMS and UTE, compliance and load matrices were established, and the time-varying mesh stiffness, loaded transmission error, mesh force, and multi-tooth LMC models of the SBG were determined. Finally, the proposed method was validated against nonlinear FE simulations using examples. Compared to FE simulations, the proposed method requires significantly less computational effort and can be further extended to optimization or system analysis problems.

Detecting approximate replicate components of a high-dimensional random vector with latent structure

High-dimensional feature vectors are likely to contain sets of measurements that are approximate replicates of one another. In complex applications, or automated data collection, these feature sets are not known a priori, and need to be determined. This work proposes a class of latent factor models on the observed, high-dimensional, random vector X∈Rp, for defining, identifying and estimating the index set of its approximately replicate components. The model class is parametrized by a p×K loading matrix A that contains a hidden sub-matrix whose rows can be partitioned into groups of parallel vectors. Under this model class, a set of approximate replicate components of X corresponds to a set of parallel rows in A: these entries of X are, up to scale and additive error, the same linear combination of the K latent factors; the value of K is itself unknown. The problem of finding approximate replicates in X reduces to identifying, and estimating, the location of the hidden sub-matrix within A, and of the partition H of its row index set H. Both H and H can be fully characterized in terms of a new family of criteria based on the correlation matrix of X, and their identifiability, as well as that of the unknown latent dimension K, are obtained as consequences. The constructive nature of the identifiability arguments enables computationally efficient procedures, with consistency guarantees. Furthermore, when the loading matrix A has a particular sparse structure, provided by the errors-in-variable parametrization, the difficulty of the problem is elevated. The task becomes that of separating out groups of parallel rows that are proportional to canonical basis vectors from other, possibly dense, parallel rows in A. This is met under a scale assumption, via a principled way of selecting the target row indices, guided by the successive maximization of Schur complements of appropriate covariance matrices. The resulting procedure is an enhanced version of that developed for recovering general parallel rows in A. It is also computationally efficient, consistent. It has immediate applications to latent space overlapping clustering and the estimation of loading matrices that satisfy a canonical parametrization.

Rotation to Sparse Loadings Using L^p Losses and Related Inference Problems

Researchers have widely used exploratory factor analysis (EFA) to learn the latent structure underlying multivariate data. Rotation and regularised estimation are two classes of methods in EFA that they often use to find interpretable loading matrices. In this paper, we propose a new family of oblique rotations based on component-wise L^p loss functions (0 < ple 1) that is closely related to an L^p regularised estimator. We develop model selection and post-selection inference procedures based on the proposed rotation method. When the true loading matrix is sparse, the proposed method tends to outperform traditional rotation and regularised estimation methods in terms of statistical accuracy and computational cost. Since the proposed loss functions are nonsmooth, we develop an iteratively reweighted gradient projection algorithm for solving the optimisation problem. We also develop theoretical results that establish the statistical consistency of the estimation, model selection, and post-selection inference. We evaluate the proposed method and compare it with regularised estimation and traditional rotation methods via simulation studies. We further illustrate it using an application to the Big Five personality assessment.

Matrix Factor Analysis: From Least Squares to Iterative Projection

In this article, we study large-dimensional matrix factor models and estimate the factor loading matrices and factor score matrix by minimizing square loss function. Interestingly, the resultant estimators coincide with the Projected Estimators (PE) in Yu et al. which was proposed from the perspective of simultaneous reduction of the dimensionality and the magnitudes of the idiosyncratic error matrix. In other word, we provide a least-square interpretation of the PE for the matrix factor model, which parallels to the least-square interpretation of the PCA for the vector factor model. We derive the convergence rates of the theoretical minimizers under sub-Gaussian tails. Considering the robustness to the heavy tails of the idiosyncratic errors, we extend the least squares to minimizing the Huber loss function, which leads to a weighted iterative projection approach to compute and learn the parameters. We also derive the convergence rates of the theoretical minimizers of the Huber loss function under bounded fourth or even th moment of the idiosyncratic errors. We conduct extensive numerical studies to investigate the empirical performance of the proposed Huber estimators relative to the state-of-the-art ones. The Huber estimators perform robustly and much better than existing ones when the data are heavy-tailed, and as a result can be used as a safe replacement in practice. An application to a Fama-French financial portfolio dataset demonstrates the empirical advantage of the Huber estimator.

IPoLNG-An unsupervised model for the integrative analysis of single-cell multiomics data.

Single-cell multiomics technologies, where the transcriptomic and epigenomic profiles are simultaneously measured in the same set of single cells, pose significant challenges for effective integrative analysis. Here, we propose an unsupervised generative model, iPoLNG, for the effective and scalable integration of single-cell multiomics data. iPoLNG reconstructs low-dimensional representations of the cells and features using computationally efficient stochastic variational inference by modelling the discrete counts in single-cell multiomics data with latent factors. The low-dimensional representation of cells enables the identification of distinct cell types, and the feature by factor loading matrices help characterize cell-type specific markers and provide rich biological insights on the functional pathway enrichment analysis. iPoLNG is also able to handle the setting of partial information where certain modality of the cells is missing. Taking advantage of GPU and probabilistic programming, iPoLNG is scalable to large datasets and it takes less than 15min to implement on datasets with 20,000 cells.

Two-Stage Procrustes Rotation with Sparse Target Matrix and Least Squares Criterion with Regularization and Generalized Weighting

In factor analysis, a factor loading matrix is often rotated to a simple target matrix for its simplicity. For the purpose, Procrustes rotation minimizes the discrepancy between the target and rotated loadings using two types of approximation: 1) approximate the zeros in the target by the non-zeros in the loadings, and 2) approximate the non-zeros in the target by the non-zeros in the loadings. The central issue of Procrustes rotation considered in the article is that it equally treats the two types of approximation, while the former is more important for simplifying the loading matrix. Furthermore, a well-known issue of Simplimax is the computational inefficiency in estimating the sparse target matrix, which yields a considerable number of local minima. The research proposes a new rotation procedure that consists of the following two stages. The first stage estimates sparse target matrix with lesser computational cost by regularization technique. In the second stage, a loading matrix is rotated to the target, emphasizing on the approximation of non-zeros to zeros in the target by least squares criterion with generalized weighing that is newly proposed by the study. The simulation study and real data examples revealed that the proposed method surely simplifies loading matrices.

Numerical and Experimental Study on the Dynamics of the Tendon/Top Tension Riser System of a Tension-Leg Platform

In this study, the dynamics of the tendon/top tension riser (TTR) system of a tension-leg platform (TLP) are investigated through an experiment and by using absolute nodal coordinate formulation (ANCF). First, the model test of the TLP system is conducted in the water tank of Harbin Engineering University to examine the motion response of the TLP and the dynamic response characteristics of the tendon and TTR. The test scale ratio is set to 1: 66.3. Then, on the basis of the ANCF, the stiffness, external load, and mass matrices of the element are deduced to establish the motion equation of the tendon/riser. Finally, the static and dynamic characteristics of the tendon/TTR system of TLP are analyzed systematically by using the ANCF method. The results are compared with commercial software and test results. The motion response of tendon/TTR is affected by the TLP movement and environmental load simultaneously. The analysis proves the effectiveness and accuracy of the ANCF method despite the low number of riser units, suggesting the superiority of the ANCF method for calculating the dynamics of tendon/riser in the field of ocean engineering.

An efficient rectangular optimization method for sparse orthogonal collimator based small animal irradiation

Objective. Intensity-modulated radiotherapy (IMRT) is widely used in clinical radiotherapy, treating varying malignancies with conformal doses. As the test field for clinical translation, preclinical small animal experiments need to mimic the human radiotherapy condition, including IMRT. However, small animal IMRT is a systematic challenge due to the lack of corresponding hardware and software for miniaturized targets. Approach. The sparse orthogonal collimators (SOC) based on the direct rectangular aperture optimization (RAO) substantially simplified the hardware for miniaturization. This study investigates and evaluates a significantly improved RAO algorithm for complex mouse irradiation using SOC. Because the Kronecker product representation of the rectangular aperture is the main limitation of the computational performance, we reformulated matrix multiplication in the data fidelity term using multiplication with small matrices instead of the Kronecker product of the dose loading matrices. Solving the optimization problem was further accelerated using the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA). Main results. Four mouse cases, including a liver, a brain tumor, a concave U-target, and a complex total marrow irradiation (TMI) case, were included in this study with manually delineated targets and OARs. Seven coplanar-field SOC IMRT (sIMRT) plans were compared with idealistic fluence map based IMRT (iIMRT) plans. For the first three cases with simpler and smaller targets, the differences between sIMRT plans and iIMRT plans in the planning target volumes (PTV) statistics are within 1%. For the TMI case, the sIMRT plans are superior in reducing hot spots (also termed D max) of PTV, kidneys, lungs, heart, and bowel by 20.5%, 31.5%, 24.67%, 20.13%, and 17.78%, respectively. On average, in four cases in this study, the sIMRT plan conformity is comparable to that of the iIMRT’s with lightly increased R50 and Integral Dose by 2.23% and 2.78%. Significance. The significantly improved sIMRT optimization method allows fast plan creation in under 1 min for smaller targets and makes complex TMI planning feasible while achieving comparable dosimetry to idealistic IMRT with fluence map optimization.

An enhanced artificial hummingbird algorithm and its application in truss topology engineering optimization

In view of the shortcomings such as slow search speed, low optimization precision and premature convergence of artificial hummingbird algorithm, an enhanced artificial hummingbird algorithm based on golden sine factor named DGSAHA is proposed. Firstly, chaos mapping is used to generate the initial candidate solution to increase the diversity of the population, which lays the foundation for the global search. Then, perturb the individuals by means of the differential variation between individuals on the group, thereby enhancing the diversity of the population, preserving the excellent individuals, eliminating the inferior individuals, and guiding the search process to approach the global optimal solution, avoiding the phenomenon of premature convergence. Finally, the golden sine factor were introduced in the guided foraging stage is conducive to the full exploration of the global optimal solution, reducing the search space for individuals to approach the optimal solution. And, it facilitates the balance between “exploration” and “exploitation” of algorithm. Thereby, the accuracy and speed of the DGSAHA can be improved to a certain extent. 25 classic functions, the CEC2014 and CEC2019 benchmark functions were tested, and several representative meta-heuristic algorithms and its improved algorithm are compared for evaluate the validity of DGSAHA. Meanwhile, the dimensional scalability of the variable-dimensional test function is discussed. The results of non-parametric statistical analysis and performance index show that DGSAHA in this paper has better comprehensive optimization performance, significantly improves the search speed and convergence precision, and has strong ability to get rid of the local optimal solution. Finally, the performance of DGSAHA and the practicability of truss structure are answered by three engineering examples of plane and space truss topology optimization problem. This optimization problem considers not only the static constraints such as stress, displacement and buckling, but also the dynamic constraints of frequency and motion stability. In order to avoid singularity and unnecessary analysis, the stiffness, mass and load matrices are reconstructed in finite element analysis. A lighter truss structure than the existing solution is obtained. The validity, extensibility and practicability of the algorithm are further illustrated.

Influence Surface Coefficients of Plates Resting on Pasternak Foundation

In this study, internal force influence surface coefficients of plates resting on Pasternak foundation are obtained using the finite element method applying the classical Müller-Breslau Principle. The two-parameter elastic foundation is represented by the inclusion of the elastic bedding and shear parameter matrix terms of a 4-noded soil finite element to the corresponding stiffness matrix terms of the plate finite element used in the implementation. The loading matrices used to determine the influence surfaces directly are derived from the governing equations and the element matrices which are obtained using the Betti’s law. Internal force influence surface coefficients of plates with and without elastic foundation are given comparatively through numerical examples, confirming the values with another approach and the literature.