Sparse Loadings Research Articles

Nonintrusive load disaggregation by fusion of graph signal processing and CNN

Abstract Nonintrusive load disaggregation techniques play a pivotal role in power system planning and decision making for demand response. The sparsity of low frequency total power data features, coupled with the model’s challenge in effectively extracting and utilizing spatial and temporal correlations in diverse load data, presents obstacles to achieving high precision disaggregation in nonintrusive load disaggregation. To address these challenges, this paper proposes a disaggregation framework that integrates graph signal processing (GSP) with a convolutional neural network (CNN). In this framework, the GSP module tackles the issue of sparse load features, while the CNN disaggregation module extracts and exploits various features, such as spatial and temporal correlations between different loads. The GSP module directly provides load features to the CNN disaggregation module, assigning weights to the outputs for selection and thereby enhancing the accuracy of the disaggregation process. Our method is trained and validated using real datasets, demonstrating satisfactory performance when compared to existing methods.

Decoupling Shrinkage and Selection in Gaussian Linear Factor Analysis

Factor analysis is a popular method for modeling dependence in multivariate data. However, determining the number of factors and obtaining a sparse orientation of the loadings are still major challenges. In this paper, we propose a decision-theoretic approach that brings to light the relationship between model fit, factor dimension, and sparse loadings. This relation is done through a summary of the information contained in the multivariate posterior. A two-step strategy is used in our method. First, given the posterior samples from the Bayesian factor analysis model, a series of point estimates with a decreasing number of factors and different levels of sparsity are recovered by minimizing an expected penalized loss function. Second, the degradation in model fit between the posterior of the full model and the recovered estimates is displayed in a summary. In this step, a criterion is proposed for selecting the factor model with the best trade-off between fit, sparseness, and factor dimension. The findings are illustrated through a simulation study and an application to personality data. We used different prior choices to show the flexibility of the proposed method.

Constrained Decoupled Dictionary Learning of Task fMRI data

AbstractBackgroundDictionary learning techniques have the potential to efficiently detect functional connectivity impairments in distributed brain networks associated with Alzheimer’s Disease (AD) when applied to task functional magnetic resonance imaging (tfMRI) data. However, traditional methods are constrained in what brain networks they can identify due to the statistical independence assumptions they make, and they do not leverage the structure of the tfMRI task when identifying networks. We used a new sparse dictionary learning technique to identify brain functional network correlates of cardiometabolic risk variables in a community‐based cohort.MethodThe Bogalusa Heart Study (BHS) is community‐based cohort study of atherosclerosis and risk factors for cardiovascular disease. fMRI data was collected from 100 participants of BHS during performance of a Stroop task. Spatially concatenated tfMRI data was factorized into a group‐wise temporal dictionary matrix and a set of scan‐level spatial patterns while biasing a subset of learned temporal patterns to be paradigm related. Scan‐level spatial patterns obtained as a result were aggregated and factorized to derive a group‐level spatial dictionary matrix and sparse loading coefficients. The sparse loading coefficients obtained from this Constrained Decoupled Dictionary Learning (CDDL) method were used as descriptive biomarkers, the relevance of which was established by using them to predict cardiometabolic variables including fasting glucose level, homeostatic model assessment for insulin resistance (HOMA‐IR) index and fasting insulin level using support vector regression and a leave‐one out cross‐validation. The performance of the algorithm was calculated in terms of Mean Absolute Percentage Error (MAPE) between actual and predicted cardiometabolic variable valuesResultThe SVM regressor was able to predict the cardiometabolic variables with low MAPE. The prediction of glucose values achieved the least MAPE for all kernel types (Table 1).ConclusionCombining prior knowledge, decoupling of spatial and temporal patterns, and high‐dimensional regression, we generated brain functional network patterns from real‐world tfMRI data that associated with known cardiometabolic indicators of brain health. These brain functional network patterns thus have the potential to serve as indicators of brain health.

Vapor-Induced Assembly of a Platinum(II) Complex Loaded on Layered Double Hydroxide Nanoparticles.

Controlled self-assembly of PtII complexes is key to the development of optical and stimuli-responsive materials, but designing and precisely controlling them is still difficult owing to weak intermolecular interactions. Herein, we report the successful water-vapor-induced assembly of an anionic PtII complex [Pt(CN)2 (ppy)]- (Hppy=2-phenylpyridine) electrostatically loaded onto cationically charged layered double hydroxide (LDH) nanoparticles consisting of Mg2+ and Al3+ ions. When the PtII complexes were densely loaded onto the LDH nanoparticles, the assembly was maintained, even in dilute aqueous media. In the case of sparse loading, the PtII complexes were loaded discretely in the dry state; however, when water vapor was adsorbed, the increased mobility of the PtII complexes led to their assembly on the LDH nanoparticles. The presence of water vapor led to a drastic change in luminescence from green to orange.

A critical assessment of sparse PCA (research): why (one should acknowledge that) weights are not loadings

Principal component analysis (PCA) is an important tool for analyzing large collections of variables. It functions both as a pre-processing tool to summarize many variables into components and as a method to reveal structure in data. Different coefficients play a central role in these two uses. One focuses on the weights when the goal is summarization, while one inspects the loadings if the goal is to reveal structure. It is well known that the solutions to the two approaches can be found by singular value decomposition; weights, loadings, and right singular vectors are mathematically equivalent. What is often overlooked, is that they are no longer equivalent in the setting of sparse PCA methods which induce zeros either in the weights or the loadings. The lack of awareness for this difference has led to questionable research practices in sparse PCA. First, in simulation studies data is generated mostly based only on structures with sparse singular vectors or sparse loadings, neglecting the structure with sparse weights. Second, reported results represent local optima as the iterative routines are often initiated with the right singular vectors. In this paper we critically re-assess sparse PCA methods by also including data generating schemes characterized by sparse weights and different initialization strategies. The results show that relying on commonly used data generating models can lead to over-optimistic conclusions. They also highlight the impact of choice between sparse weights versus sparse loadings methods and the initialization strategies. The practical consequences of this choice are illustrated with empirical datasets.

Small-Variance Priors in Bayesian Factor Analysis with Ordinal Data

To evaluate multidimensional factor structure, a popular method that combines features of confirmatory and exploratory factor analysis is Bayesian structural equation modeling with small-variance normal priors (BSEM-N). This simulation study evaluated BSEM-N as a variable selection and parameter estimation tool in factor analysis with sparse cross-loading structures, focusing on ordered categorical data. A sensitivity analysis was conducted by assigning eight choices of small-variance priors on all potential cross-loadings. Results indicated that variable selection was performed well in a sparse loading structure in which the number of essential cross-loadings was small and the magnitudes were relatively large. Characteristics of ordinal items did not seem to have a sizable impact on parameter estimation. If the number of cross-loading estimates were small and centered around zero, BSEM-N may serve more efficiently as a tool for parameter estimation.

Allocation of HV/MV Substations in Iraqi Power Grid Using Delaunay Triangulation

In provincial areas, allocation of the HV/MV substations aims to assign a proper set of numbers and locations of substations to supply loads spatially distributed over a large geographic area with continually emerging towns. This requires identifying the sparse load centers, the existing substations, and the new substations required to cover the potential extension. This paper proposes a method to allocate the main substations to the targeted area based on the lines’ length which are assigned by Delaunay triangulation and geographical coordinates. The proposed configuration is identified by a criterion that is tailored to the Iraqi sub-transmission system in which the standard distances among the substation follow specific patterns. The standard distances that are identified by the proposed methods are represented by the edges of Delaunay triangles. This paper aims to improve the voltage regulation of the distribution grid and avoid problems of overloaded feeders by assigning the proposed locations that reduce the lines’ lengths. The proposed method is verified using simulation tests on Diyala 9-bus system which is a part of the Iraqi 132 kV power system. The results show that the proposed configuration of the substations is reasonably valid and quite close to the real locations of the towns of the system with efficient lengths for the lines.

An extension of fast iterative shrinkage‐thresholding algorithm to Riemannian optimization for sparse principal component analysis

AbstractSparse principal component analysis (PCA), an important variant of PCA, attempts to find sparse loading vectors when conducting dimension reduction. This paper considers the nonsmooth Riemannian optimization problem associated with the ScoTLASS model for sparse PCA which can impose orthogonality and sparsity simultaneously. A Riemannian proximal method is proposed in the work of Chen et al. for the efficient solution of this optimization problem. In this paper, two acceleration schemes are introduced. First and foremost, we extend the FISTA method from the Euclidean space to the Riemannian manifold to solve sparse PCA, leading to the accelerated Riemannian proximal gradient method. Since the Riemannian optimization problem for sparse PCA is essentially nonconvex, a restarting technique is adopted to stabilize the accelerated method without sacrificing the fast convergence. Second, a diagonal preconditioner is proposed for the Riemannian proximal subproblem which can further accelerate the convergence of the Riemannian proximal methods. Numerical evaluations establish the computational advantages of the proposed methods over the existing proximal gradient methods on a manifold. Additionally, a short result concerning the convergence of the Riemannian subgradients of a sequence is established, which, together with the result in the work of Chen et al., can show the stationary point convergence of the Riemannian proximal methods.

Sparse modeling and monitoring for industrial processes using sparse, distributed principal component analysis

Driven by the strong demand for sparsity in dimensional reduction techniques, a sparse modeling and monitoring approach based on sparse, distributed principal component analysis (SDPCA) is proposed to achieve sparsity. To this end, the data set is first divided into highly correlated blocks (HCBs) and one remainder block (RB) on the basis of the mutual-information-based correlation matrix. From this, the sparse loading vectors for the HCBs are obtained using the PCA models, while for the RB, it is obtained using the sparse PCA model. It is worth noting that the sparsity in SDPCA enables the sparse loading vectors to produce interpretable principal components, which keeps the correlations between the highly correlated variables and achieves the sparsity for the weakly correlated ones. Moreover, to fully appreciate the interpretation of the sparse principal components, a fault diagnosis strategy named blockwise contribution plots is proposed by first determining the faulty block, and then, identifying the faulty variables. Compared with PCA and SPCA, the proposed SDPCA detects more faulty samples and gives more accurate diagnosis results.

Performances of some high dimensional regression methods: sparse principal component regression

Principal component analysis (PCA) is widely used technique in data processing and dimensionality reduction, but it has some drawbacks since each principal component is a linear combination of explanatory variables. As an alternative, the sparse PCA (SPCA) is a very appealing method which produces principal components with sparse loadings. On the other hand, combining PCA on explanatory variables with least squares regression yields to principal component regression (PCR). In PCR, the components are obtained using only explanatory variables, not considered the effect of the dependent variable. Considering the dependent variable, the sparse PCR (SPCR) enables to obtain sparse principal component loadings. But the main drawback of it is the computational cost. Taking into consideration the general structure of PCR, we combine (S)PCA with some sparse regression methods and compared with the classical PCR and last introduced method, SPCR. Extensive simulation studies and real data examples are implemented to show their performances. The results are supported by a reasonable computation time study.

Poisson reduced-rank models with sparse loadings

High-dimensional Poisson reduced-rank models have been considered for statistical inference on low-dimensional locations of the individuals based on the observations of high-dimensional count vectors. In this study, we assume sparsity on a so-called loading matrix to enhance its interpretability. The sparsity assumption leads to the use of $$L_1$$ penalty, for the estimation of the loading. We provide novel computational and theoretical analyses for the corresponding penalized Poisson maximum likelihood estimation. We establish theoretical convergence rates for the parameters under weak-dependence conditions; this implies consistency even in large-dimensional problems. To implement the proposed method involving several computational issues, including nonconvex log-likelihoods, $$L_1$$ penalty, and orthogonal constraints, we developed an iterative algorithm. Further, we propose a Bayesian-Information-Criteria-based penalty parameter selection, which works well in the implementation. Some numerical evidence is provided by conducting real-data-based simulation analyses and the proposed method is illustrated with the analysis of German party manifesto data.

MWRSPCA: online fault monitoring based on moving window recursive sparse principal component analysis

This paper proposes a moving window recursive sparse principal component analysis (MWRSPCA)-based online fault monitoring scheme, aim at providing an online fault monitoring solution for large-scale complex industrial processes (e.g., chemical industry processes) with time-varying and dynamically changing characteristics. It establishes a sparse principal component analysis (SPCA) model based on the sliding window block matrixes to perform process monitoring and incorporates normal process monitoring data set simultaneously to the model training set to update the monitoring model online, so that the process monitoring model has strong adaptability to time-varying processes. A recursive computing procedure of the corresponding sparse loading matrixes is derived based on a modified rank-one matrix approximation algorithm, so that the computational complexity of the process monitoring model is greatly decreased and the real-time monitoring capability can be guaranteed. The effectiveness of the proposed method is verified by the benchmark Tennessee-Eastman process. Compared with traditional fault monitoring methods, the proposed method can effectively improve the fault detection accuracies with lower false alarm rates, which is suitable for the fault monitoring of time-varying, long-term and continuous complex industrial processes.

Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes

Given a multivariate data set, sparse principal component analysis (SPCA) aims to extract several linear combinations of the variables that together explain the variance in the data as much as possible, while controlling the number of nonzero loadings in these combinations. In this paper we consider 8 different optimization formulations for computing a single sparse loading vector; these are obtained by combining the following factors: we employ two norms for measuring variance (L2, L1) and two sparsity-inducing norms (L0, L1), which are used in two different ways (constraint, penalty). Three of our formulations, notably the one with L0 constraint and L1 variance, have not been considered in the literature. We give a unifying reformulation which we propose to solve via a natural alternating maximization (AM) method. We show the the AM

Designating the geographical origin of Iranian almond and red jujube oils using fluorescence spectroscopy and l1-penalized chemometric methods

Assessment of the geographical origin of medical plants (MP) has received a lot of attention in traditional medicine for determination of the quality of the products and detection of food/drug adulteration. In the present contribution, the sparse version of N-way partial least square discriminant analysis (sNPLS-DA) has been used for classification of almond and red jujube oil samples using their excitation-emission (EEM) fluorescence spectra, for the first time. Seven types of almond seeds and six species of red jujube samples were collected from mountains and hills in Fars and Khorasan provinces in Iran, respectively. The sNPLS-DA algorithm with l1-norm constraint was utilized for calculation of sparse loading vectors. This method merges variable selection and modeling steps into one phase, and due to sparse loading vectors, models are easily interpretable. The number of latent variables and non-zero elements in the first and second loading spaces for the sNPLS-DA models in this work was optimized using a central composite design (CCD) approach. The optimized models were used for classification of almond and red jujube oil samples according to their geographical origins in Iran. The average accuracies of the sNPLS-DA models were 0.95 and 0.89 for bootstrapping cross validation and 0.97 and 1.00 for external validation for almond and red jujube samples, respectively. The performance of the sNPLS-DA models was compared with PLS-DA, parallel factor analysis (PARAFAC), counter propagation artificial neural networks (CPANN), and supervised Kohonen networks (SKN). The results of the optimized sNPLS-DA models were superior to PARAFAC-DA and comparable to those of CPANN and SKN models. The results in this work reveal that the sNPLS-DA method can be used as a reliable and interpretable tool for quality control and classification of food samples using their EEM fluorescence data.

Sparse multiple co-Inertia analysis with application to integrative analysis of multi -Omics data

BackgroundMultiple co-inertia analysis (mCIA) is a multivariate analysis method that can assess relationships and trends in multiple datasets. Recently it has been used for integrative analysis of multiple high-dimensional -omics datasets. However, its estimated loading vectors are non-sparse, which presents challenges for identifying important features and interpreting analysis results. We propose two new mCIA methods: 1) a sparse mCIA method that produces sparse loading estimates and 2) a structured sparse mCIA method that further enables incorporation of structural information among variables such as those from functional genomics.ResultsOur extensive simulation studies demonstrate the superior performance of the sparse mCIA and structured sparse mCIA methods compared to the existing mCIA in terms of feature selection and estimation accuracy. Application to the integrative analysis of transcriptomics data and proteomics data from a cancer study identified biomarkers that are suggested in the literature related with cancer disease.ConclusionProposed sparse mCIA achieves simultaneous model estimation and feature selection and yields analysis results that are more interpretable than the existing mCIA. Furthermore, proposed structured sparse mCIA can effectively incorporate prior network information among genes, resulting in improved feature selection and enhanced interpretability.

Prior Sensitivity in Bayesian Structural Equation Modeling for Sparse Factor Loading Structures.

Bayesian structural equation modeling (BSEM) is a flexible tool for the exploration and estimation of sparse factor loading structures; that is, most cross-loading entries are zero and only a few important cross-loadings are nonzero. The current investigation was focused on the BSEM with small-variance normal distribution priors (BSEM-N) for both variable selection and model estimation. The prior sensitivity in BSEM-N was explored in factor analysis models with sparse loading structures through a simulation study (Study 1) and an empirical example (Study 2). Study 1 examined the prior sensitivity in BSEM-N based on the model fit, population model recovery, true and false positive rates, and parameter estimation. Seven shrinkage priors on cross-loadings and five noninformative/vague priors on other model parameters were examined. Study 2 provided a real data example to illustrate the impact of various priors on model fit and parameter selection and estimation. Results indicated that when the 95% credible intervals of shrinkage priors barely covered the population cross-loading values, it resulted in the best balance between true and false positives. If the goal is to perform variable selection, a sparse cross-loading structure is required, preferably with a minimal number of nontrivial cross-loadings and relatively high primary loading values. To improve parameter estimates, a relatively large prior variance is preferred. When cross-loadings are relatively large, BSEM-N with zero-mean priors is not recommended for the estimation of cross-loadings and factor correlations.

Band Selection Using Combined Divergence–Correlation Index and Sparse Loadings Representation for Hyperspectral Image Classification

Band selection and feature extraction are the two paradigms of hyperspectral dimensionality reduction. While feature extraction methods have many desirable properties, band selection methods keep the actual reflectances intact, which results in better interpretation of the behavior of the materials. Keeping this in mind, several feature extraction-based band selection methods have been proposed in the past, which can bridge the gap between these two approaches and combine their salient advantages. In this article, a feature extraction-based, clustering-ranking type band selection method is proposed in which the band selection is performed in two stages. In the clustering stage, a new and improved representation of bands using their component loadings for the top principal components is introduced, which is employed to cluster similar bands into groups using sparse subspace clustering. In the ranking stage, a new metric called combined divergence–correlation index is introduced to elect the most discriminative as well as least correlated bands as cluster representatives. Experimental results indicate that the proposed method can proficiently select a set of distinct and discriminative bands, which can help in effective hyperspectral classification. Additionally, it was also shown that the selected bands show rich information content, low interband correlation and high-class separabilities.

A sparse loading-based contribution method for multivariate control performance diagnosis

An automatic and accurate control performance diagnosis algorithm is necessary for general chemical processes with a number of control loops when a performance change occurs. The popular contribution plots for control performance diagnosis have severe smearing effects of diagnosis and the problems are still not solved. A sparse loading based contribution plot is proposed in this paper to enhance the diagnosis performance. A complete iterative optimization procedure related to how to obtain the sparse loadings is developed. A numerical example and a period of practical industrial data are used to demonstrate the efficiency of the proposed method in comparison with the conventional contribution plots.

Enhancing Interpretability in Factor Analysis by Means of Mathematical Optimization

Exploratory Factor Analysis (EFA) is a widely used statistical technique to discover the structure of latent unobserved variables, called factors, from a set of observed variables. EFA exploits the property of rotation invariance of the factor model to enhance factors’ interpretability by building a sparse loading matrix. In this paper, we propose an optimization-based procedure to give meaning to the factors arising in EFA by means of an additional set of variables, called explanatory variables, which may include in particular the set of observed variables. A goodness-of-fit criterion is introduced which quantifies the quality of the interpretation given this way. Our methodology also exploits the rotational invariance of EFA to obtain the best orthogonal rotation of the factors, in terms of the goodness-of-fit, but making them match to some of the explanatory variables, thus going beyond traditional rotation methods. Therefore, our approach allows the analyst to interpret the factors not only in terms of the observed variables, but in terms of a broader set of variables. Our experimental results demonstrate how our approach enhances interpretability in EFA, first in an empirical dataset, concerning volumes of reservoirs in California, and second in a synthetic data example.

A sparse fault degradation oriented fisher discriminant analysis (FDFDA) algorithm for faulty variable isolation and its industrial application

In a fault process, the variables may be influenced differently. In order to improve the diagnosis performance, it is an important issue to isolate those significant faulty variables that cover informative fault effects. However, those variables are selected one by one and their correlations are not considered in the previous work. As sparse-relevant methods can automatically and efficiently isolate significant correlated variables, it is natural to consider applying the criteria of sparsity to separate the significantly influenced faulty variables and analyze them by specific methods. First, the sparse version of the fault degradation oriented Fisher discriminant analysis (FDFDA) algorithm is proposed to produce informative discriminant directions with sparse loadings. Subsequently, a faulty variable selection strategy is proposed based on the sparse FDFDA algorithm to select significantly influenced faulty variables. By iteratively isolating correlated variables along each sparse fault direction, all the faulty variables can be automatically selected until the left fault data and normal data share the similar characteristics. Therefore, the whole measurement variables can be divided into faulty variable set and normal variable set. Then different fault diagnosis models can be developed according to their different characteristics for each fault class. For online application, a probabilistic fault diagnosis strategy is proposed to determine the fault cause of the new sample by the largest synthetic probability that integrates the diagnosis results of two variable sets. The performance of the proposed fault diagnosis method is illustrated using the data from the cut-made process of cigarette.