Sparse Principal Component Analysis Research Articles

Feature Selection and Damage Identification for Urban Railway Track Using Bayesian Globally Sparse Principal Component Analysis

Urban railway track infrastructures often suffer from damage that affects their service performance due to a variety of factors. In this study, an unsupervised feature selection and damage identification method based on globally sparse probabilistic principal component analysis (PCA) is proposed for urban railway tracks using the monitoring data of train-induced dynamic responses. A Bayesian framework is proposed for generating principal components on a basis of vectors (original variables) with a global sparseness pattern instead of separate patterns in a traditional sparse PCA. In this framework, a variational expectation-maximization algorithm is employed to obtain the tractable calculation of the marginal likelihood function for learning all uncertain parameters of the Bayesian model. The obtained principal components are linear combinations of the very same set of important variables, making our method better interpretable than the traditional sparse PCA. We can clearly understand which original variables are most relevant for describing the data. The track damage is identified simply by discriminating the corresponding measured dynamic responses using the binary elements of the latent vector inferred from the Bayesian globally sparse PCA algorithm. The usefulness is demonstrated by successfully identifying the track bed plate crack damage through the actual train-induced dynamic responses collected from the structural health monitoring system of an urban railway track infrastructure, where the method is able to achieve F1 scores of 90% or higher for various scenarios.

Identifying and interpreting the factors in factor models via sparsity: Different approaches

SummaryThis paper considers different approaches for identifying the factor structure and interpreting the factors without imposing their interpretation via restrictions: sparse PCA and factor rotations. We establish a new consistency result for the factors estimated by sparse PCA. Monte Carlo simulations show that our methods accurately estimate the factor structure, even in small samples. We apply them to large datasets about international business cycles and the US economy. For each empirical application, they identify the same factor structure, offering a clear economic interpretation. These exploratory methods can in particular justify or complement approaches that impose the factor structure a priori.

Optimal Convex Lifted Sparse Phase Retrieval and PCA With an Atomic Matrix Norm Regularizer

We present novel analysis and algorithms for solving sparse phase retrieval and sparse principal component analysis (PCA) with convex lifted matrix formulations. The key innovation is a new mixed atomic matrix norm that, when used as regularization, promotes low-rank matrices with sparse factors. We show that convex programs with this atomic norm as a regularizer provide near-optimal sample complexity and error rate guarantees for sparse phase retrieval and sparse PCA. While we do not know how to solve the convex programs exactly with an efficient algorithm, for the phase retrieval case we carefully analyze the program and its dual and thereby derive a practical heuristic algorithm. We show empirically that this practical algorithm performs similarly to existing state-of-the-art algorithms.

Temporal relationships between latent symptoms in psychosis: a longitudinal experience sampling methodology study

IntroductionA variety of dimensions of psychopathology are observed in psychosis. However, the validation of clinical assessment scales, and their latent variable structure, is often derived from cross-sectional rather than longitudinal data, limiting our understanding of how variables interact and reinforce one another.ObjectivesUsing experience sampling methodology (ESM) and analytic approaches optimised for longitudinal data, we assess potential latent variables of commonly-reported symptoms in psychosis, and explore the temporal relationship between them.MethodsN=36 participants with a diagnosis of schizophrenia or schizoaffective disorder provided data for up to one year, as part of the Sleepsight study. Using a smartphone app, participants self-reported clinical symptoms once daily for a mean duration of 323 days (SD: 88), with a response rate of 69%. Symptoms were rated using seven-point Likert scale items. Items included symptoms traditionally implicated in psychosis (feeling “cheerful”, “anxious”, “relaxed”, “irritable”, “sad”, “in control”, “stressed”, “suspicious”, “trouble concentrating”, “preoccupied by thoughts”, “others dislike me”, “confused”, “others influence my thoughts” and “unusual sights and sounds”). We used a sparse PCA (SPCA) model to identify latent variables in the longitudinal data. SPCA has previously been applied to longitudinal ESM data, and was developed to achieve a compromise between the explained variance and the interpretability of the principal components. We then used a multistage exploratory and confirmatory differential time-varying effect model (DTVEM) to explore the temporal relationship between the latent variables. DTVEM generates a standardised β coefficient reflecting the strength of relationship between variables across multiple time lags. Only significant lags (p<0.05) are reported here.ResultsThe SPCA analysis identified five latent variables, explaining 61.4% of the total variance. Tentative interpretation of the SPCA loadings suggested these latent variables corresponded to i) cognitive symptoms, ii) feeling in-control, iii) thought interference and perceptual disturbance, iv) irritability and stress and v) paranoia. Time lag analysis revealed an effect of feeling in-control on subsequent cognitive symptoms (β=-0.19), and of cognitive symptoms on subsequent thought interference and perceptual disturbance (β=0.14). Irritability and stress was also associated with subsequent cognitive symptoms (β=0.09).ConclusionsUsing longitudinal data, we employ novel methodology to identify potential latent symptoms among commonly reported symptoms in psychosis. We identify five latent symptoms, and elucidate important temporal relationships between them. These findings may inform our understanding of the psychopathology of psychosis, potentially offering data-driven simplification of clinical assessment and novel insights for future research.Disclosure of InterestNone Declared

Forecasting the realized volatility of Energy Stock Market: A multimodel comparison

The realized volatility forecasting of energy sector stocks facilitates the establishment of corresponding risk warning mechanisms and investor decisions. In this paper, we collected two different energy sector indices and used different methods, namely principal component analysis (PCA) and sparse principal component analysis (SPCA), to extract features, and combined LSTM and GRU to construct 12 different models. The results show that the SPCA-LSTM model we constructed has the best forecasting performance in the realized volatility forecasting of energy indices, and SPCA has better forecasting results than PCA in the feature extraction stage. The results of the robustness test indicate that our results are robust.

Analyzing a series of thermal infrared images to identify defects using a hybrid approach that combines robust principal component analysis and image segmentation

Thermography is widely used to identify defects; however, analyzing just the images containing important information instead of analyzing each individual image from a series of thermal infrared images is an important capability. Principal component analysis (PCA) is the traditionally used dimensionality reduction approach for projecting a given dataset onto a low-dimensional space. In doing so, several images can be extracted from this low-dimensional space that contain significant information about the given data. The problem with this is that PCA is sensitive to noise and shadows. Robust PCA (RPCA), Sparse PCA (SPCA), and Kernel PCA (KPCA) have therefore been developed and employed for defect detection. The question then arises as to which method can preserve the most information from a series of thermal infrared images after the data are projected onto the low-dimensional space. This study analyzed a series of thermal infrared images covered with shadows using different PCAs. By comparing the results, it was found that RPCA can preserve the most information of the given data. Image segmentation was employed to segment the extracted images such that the thermal patterns could be easily identified from the segmented regions. The shadow locations could be identified by selecting the areas whose shadow effects estimated using image segmentation were less than 1.0. The results were encouraging and the proposed scheme can be feasibly employed for defect detection.

Speckle-noise filtering based on non-local mean sparse principal component analysis method

Speckle-noise filtering is an extensive and key process in coherent interferometric techniques to obtain important and accurate information from the recorded interferogram or fringe pattern. The speckle-noise inherent to these interferograms, recorded by digital image sensors in these optical techniques, is eliminated by an appropriate image processing method. Since the beginning and further development of these techniques, a wide variety of speckle noise filtering methods and techniques have been proposed. The present research work aims to exploit the performance of the non-local sparse principal component analysis (NLS-PCA) method for speckle noise reduction as applied to the interferometric fringe patterns obtained from digital holographic and digital speckle pattern interferometric techniques. The performance of the NLS-PCA is evaluated on numerical simulations using different types of speckle fringe patterns resulting in the method being highly effective in filtering the speckle noise and providing superior results evaluated in terms of peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), Edge preservation Index (EPI), and Equivalent Number of look (ENL), in comparison to widely known methods such as windowed Fourier transform method, Lee filter, Weiner filter. Furthermore, the proposed method maintains the finer details and preserves the fringe edges effectively. The performance of the NLS-PCA method is also demonstrated on experimental speckle fringe patterns and digital holograms.

Application of dense neural networks for manifold-based modeling of flame-wall interactions

Artifical neural networks (ANNs) are universal approximators capable of learning any correlation between arbitrary input data with corresponding outputs, which can also be exploited to represent a low-dimensional chemistry manifold in the field of combustion. In this work, a procedure is developed to simulate a premixed methane-air flame undergoing side-wall quenching utilizing an ANN chemistry manifold. In the investigated case, the flame characteristics are governed by two canonical problems: the adiabatic flame propagation in the core flow and the non-adiabatic flame- wall interaction governed by enthalpy losses to the wall. Similar to the tabulation of a Quenching Flamelet-Generated Manifold (QFM), the neural network is trained on a 1D head-on quenching flame database to learn the intrinsic chemistry manifold. The control parameters (i.e. the inputs) of the ANN are identified from thermo-chemical state variables by a sparse principal component analysis (PCA) without using prior knowledge about the flame physics. These input quantities are then transported in the coupled CFD solver and used for manifold access during simulation runtime. The chemical source terms are corrected at the manifold boundaries to ensure bounded- ness of the thermo-chemical state at all times. Finally, the ANN model is assessed by comparison to simulation results of the 2D side-wall quenching (SWQ) configuration with detailed chemistry and with a flamelet-based manifold (QFM).

Development of a Water Quality Index Using Sparse Principal Component Analysis for the Tigris River in Iraq

Development of a Water Quality Index Using Sparse Principal Component Analysis for the Tigris River in Iraq

Identification of Subgroups of Activity Participation in Group of Community-Dwelling Older Adults.

Occupational therapy in public health is advancing for groups of older adults, but few studies have focused on measuring population-based activity engagement. This study aimed to identify subgroups based on the daily activity pattern of community-dwelling older adults. Japanese older adults aged ≥65 years responded to a 128-activity-item questionnaire. Sparse principal component analysis was applied to summarize the activities, and k-means clustering was used to identify subgroups. Respondents were divided into three subgroups (subgroup 1: n = 39, high instrumental/leisure and social/cognitive activity; subgroup 2: n = 51, high instrumental/leisure and low social/cognitive activity; subgroup 3: n = 44, low instrumental/leisure and social/cognitive activity). The three subgroups were associated with sociodemographics (sex, area, education, and activity level) (p < .05). This study highlighted the activity-based subgroups of older adults and provided a novel viewpoint regarding occupation-based community assessments in addition to previous community occupational therapy practice.

Feature grouping and sparse principal component analysis with truncated regularization

We propose a new method for principal component analysis (PCA) called feature grouping and sparse principal component analysis (FGSPCA). This method is designed to capture both grouping and sparsity structures in factor loadings simultaneously. To achieve this, we use a non‐convex truncated regularization that can adjust for sparsity and grouping effects automatically. This regularization encourages factor loadings with similar values to be either grouped together for feature grouping or be zero for feature selection, helping to reduce model complexity and improve interpretation. While other structured PCA methods often require prior knowledge to construct the regularization term, FGSPCA can capture grouping and sparsity structures without any prior information. We solve the resulting non‐convex optimization problem using an alternating algorithm that combines difference‐of‐convex programming, the augmented Lagrange method, and coordinate descent method. Our experiments show that FGSPCA performs well and efficiently on both synthetic and real‐world datasets. An implementation of FGSPCA is available on GitHub https://github.com/HaiyanJiang/FGSPCA.

Transformer Aging Diagnosis Method Based on Raman Spectroscopy Wavelet Packet-SPCA Feature Extraction

Ensuring the safety and reliable operation of a power transformer is a top priority in grid work. There is an urgent need to study methods for measuring and evaluating the aging state of oil–paper insulation in older transformers. Raman spectroscopy is widely used in material detection; the spectra can represent the inner components of transformer oil and help diagnose the aging states of the transformer efficiently. In this article, thermal accelerated aging experiments were conducted, and oil–paper insulation samples with different aging states were obtained. The aging states of oil–paper insulation were tagged by the degree of polymerization (DP) value of the paper. Four wavelet packet decompositions were used for the Raman spectra, and sparse principal component analysis was used to extract the features in the wavelet packet coefficients. A 14-D feature was built and trained with a multiclassification support vector machine (SVM), and a diagnosis model for aging states was established. The results show that the accuracy of the diagnosis model reaches 94.9%. Seven samples of transformers in operation also verify the method’s effectiveness. This method has practical significance for quickly detecting the aging states of operating transformers.

Phenotyping Key Fruit Quality Traits in Olive Using RGB Images and Back Propagation Neural Networks

To predict oil and phenol concentrations in olive fruit, the combination of back propagation neural networks (BPNNs) and contact-less plant phenotyping techniques was employed to retrieve RGB image-based digital proxies of oil and phenol concentrations. Fruits of cultivars (×3) differing in ripening time were sampled (~10-day interval, ×2 years), pictured and analyzed for phenol and oil concentrations. Prior to this, fruit samples were pictured and images were segmented to extract the red (R), green (G), and blue (B) mean pixel values that were rearranged in 35 RGB-based colorimetric indexes. Three BPNNs were designed using as input variables (a) the original 35 RGB indexes, (b) the scores of principal components after a principal component analysis (PCA) pre-processing of those indexes, and (c) a reduced number (28) of the RGB indexes achieved after a sparse PCA. The results show that the predictions reached the highest mean R 2 values ranging from 0.87 to 0.95 (oil) and from 0.81 to 0.90 (phenols) across the BPNNs. In addition to the R 2 , other performance metrics were calculated (root mean squared error and mean absolute error) and combined into a general performance indicator (GPI). The resulting rank of the GPI suggests that a BPNN with a specific topology might be designed for cultivars grouped according to their ripening period. The present study documented that an RGB-based image phenotyping can effectively predict key quality traits in olive fruit supporting the developing olive sector within a digital agriculture domain.

Self-Learning Sparse PCA for Multimode Process Monitoring

This article proposes a novel sparse principal component analysis algorithm with self-learning ability for multimode process monitoring, where the successive modes are learned in a sequential fashion. Different from traditional multimode monitoring methods, a small set of data are collected when a novel mode arrives. The proposed method remembers the learned knowledge by selectively slowing down the changes of parameters important for the previous modes, where the importance measure is estimated by synaptic intelligence. The sufficient condition of fault detectability is proved to provide a comprehensive understanding of the proposed method. Besides, the computation and storage resources are saved in the long run, because it is not necessary to retrain the model from scratch frequently and data are discarded once they have been learned. More importantly, the model furnishes excellent interpretability and the catastrophic forgetting problem is further alleviated owing to the sparsity of parameters. In addition, the hyperparameters are discussed to understand the proposed method comprehensively and the computational complexity is analyzed. Compared with several state-of-the-art approaches, a numerical case, and a practical pulverizing system are adopted to illustrate the effectiveness of the proposed algorithm.

MS-SSPCANet: A powerful deep learning framework for tool wear prediction

Reliable tool condition monitoring (TCM) system becomes increasingly critical to machinery efficiency, workpiece quality, and production cost in modern manufacturing industry. However, existing monitoring models have to face model generalization capacity or other problems, which significantly hampers their further application. Principal component analysis network (PCANet) is a promising interpretable deep learning model, and has been applied in image recognition. Nevertheless, the architecture still needs optimizing. In this paper, a novel multi-scale stacked sparse principal component analysis network (MS-SSPCANet) is proposed to predict tool wear. At first, time–frequency images are generated by short-time Fourier transform (STFT). Then, sparsity, multi-scale filters and multi-scale pooling layer are introduced to optimize PCANet structure. A hybrid pooling layer named GA-SPP, combining global averaging and spatial pyramid pooling layers, is applied to select features. And the selected ones from different filter scales are fused. Finally, support vector regression (SVR) is trained for tool wear prediction. Two milling experiments are conducted, and the vibration signal and corresponding tool wear data are acquired simultaneously. The following performance comparison experiments illustrate the better prediction capacity of the proposed model. Additionally, sparsity and two multi-scale operation (convoluting and pooling) are proven effective, and MS-SSPCANet also outperforms LeNet even after adding −6 dB noise.

When central bank research meets Google search: A sentiment index of global financial stress

We construct a sentiment-based index of global financial stress (s-GFS index) for the period January 2004-December 2020. It builds on a novel methodological approach, which synthesizes the intensity of Google search for specific terms and word collocations related to financial instability and their prior selection based on the titles and abstracts of more than 2,000 working papers posted on the Basel Bank for International Settlements Central Bank Research Hub. The s-GFS index obtained by means of sparse principal component analysis (PCA) accurately captures major episodes of global financial instability during the observation period, playing a pivotal role for the US financial stress as well as industrial production in the USA, the Eurozone and China. It also Granger causes several well-known measures of global financial instability based on sentiment and “hard” data, e.g. the VIX index, as well as the overall dynamics of the global financial cycle, thereby emphasizing the usefulness of sentiment-based measures in monitoring worldwide financial stress.

A Hybrid Model for GRU Ultra-Short-Term Wind Speed Prediction Based on Tsfresh and Sparse PCA

Wind power is a popular renewable energy source, and the accurate prediction of wind speed plays an important role in improving the power generation efficiency of wind turbines and ensuring the normal operation of wind power equipment. Due to the instability and randomness of wind speed, it is difficult to achieve accurate prediction by traditional prediction methods. To improve the power generation efficiency of wind turbines and realize the predictability of wind speed, a hybrid wind speed prediction model based on GRUs (gated recurrent units) was constructed in this paper based on a deep neural network and feature extraction method. The hybrid model feature extraction module was implemented based on a combination of Tsfresh (a python package for time series feature extraction) and sparse PCA (sparse principal component analysis), and the network structure and other hyperparameters of the GRU module were determined through experiments. The model was validated using actual wind measurement data from a wind farm on the west coast of the United States. The results showed that the proposed model had less computational time and higher computational accuracy than the SARIMAX (seasonal auto-regressive integrated moving average with exogenous factors) and LSTM (long short-term memory) models.

A semismooth Newton based augmented Lagrangian method for nonsmooth optimization on matrix manifolds

This paper is devoted to studying an augmented Lagrangian method for solving a class of manifold optimization problems, which have nonsmooth objective functions and nonlinear constraints. Under the constant positive linear dependence condition on manifolds, we show that the proposed method converges to a stationary point of the nonsmooth manifold optimization problem. Moreover, we propose a globalized semismooth Newton method to solve the augmented Lagrangian subproblem on manifolds efficiently. The local superlinear convergence of the manifold semismooth Newton method is also established under some suitable conditions. We also prove that the semismoothness on submanifolds can be inherited from that in the ambient manifold. Finally, numerical experiments on compressed modes and (constrained) sparse principal component analysis illustrate the advantages of the proposed method.

Determination of the dominant physical processes in downward-propagating flame spread over a solid fuel using machine learning

Flame spread over solid fuel (FSS) plays a key role in solid-fuel combustion and fire-related phenomenon. The mechanisms of flame spread over solid fuel are commonly described by means of dimensionless numbers or scaling analysis that describes the balanced relationship of several processes. However, these approaches rely on prior knowledge or explicit assumption of the relevant physical processes, and it is difficult to spatially distinguish among multiple physical processes. This work demonstrated a generalized way using an unsupervised machine learning method based on the Gaussian mixture models and sparse principal component analysis (GMM-SPCA) to automatically delineates the spatial domain of the FSS from a numerical simulation into several regions that are dominated by the balance between different physical processes. The idea of equation space is employed such that each coordinate in the equation space corresponds to a specific physical process as represented by the individual term in the corresponding governing equation. The dominant heat/mass transport processes for both gas and solid phases have been analyzed, and their spatial correspondence for the fields of temperature, flow, and species has been discussed. Some critical characteristics, such as the flame stand-off distance profile, the triple flame structure, and the pyrolysis zone of the solid fuel have been properly identified and quantified. It is demonstrated that the generalized GMM-SPCA method provides an intuitive insight into the heat and mass transfer processes of the FSS for further development of the flame spread model.

Gearbox Fault Diagnosis Based on a Sparse Principal Component-Generalized Regression Neural Network

The fault diagnosis of urban rail transit gearboxes has the characteristics of complex vibration signals and large amounts of data. The daily scheduled maintenance cannot meet the needs of gearbox maintenance, so it is necessary to predict the fault types in advance. In this paper, a method of gearbox fault prediction based on sparse principal component analysis and a generalized regression neural network is presented, and the result of fault prediction can provide a reference for making maintenance plans. Based on principal component analysis (PCA), a sparse principal component is obtained by adding the LASSO penalty term, which reduces the risk of overfitting of PCA while obtaining a sparse solution. Then, the sparse reduced dimension principal component is input into the generalized regression neural network model for fault diagnosis. The results show that the fault diagnosis method based on the sparse principal component-generalized regression neural network model has high accuracy and is time-consuming.