Probabilistic Principal Component Analysis Research Articles

Deep Learning Based Atrial Fibrillation Detection Using Combination of Dimensionality Reduction Techniques and RR Interval Features

Purpose: Accurate detection of Atrial Fibrillation (AF) has great significance in the field of medical science which can reduce the rate of mortality and morbidity. The present study focuses on Electrocardiography (ECG) signal classification using dimensionality reduction techniques combined with R wave to R wave interval (RR interval) features. Materials and Methods: In the first approach, Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Independent Component Analysis (ICA), and Probabilistic Principal Component Analysis (PPCA) are performed independently on denoised ECG signal using Discrete Wavelet Transform (DWT) for the classification of ECG signal. In the second approach, the dimensionality reduction techniques combined with RR interval features are used for the classification of ECG signal. Results: Machine Learning (ML) algorithms such as Decision Tree (DT), Support Vector Machine (SVM), and Deep Learning (DL) algorithms such as Long Short Term Memory (LSTM) and Bi-Directional LSTM (BiLSTM) are used for classification purposes. Conclusion: The proposed methodology provided an overall accuracy of 93.65% with PCA and LSTM classifier and an overall accuracy of 99.45% with PCA combined with RR interval features and LSTM classifier. The developed technology has potential applications in many practical solutions.

Moving Beyond Medical Statistics: A Systematic Review on Missing Data Handling in Electronic Health Records.

Background: Missing data in electronic health records (EHRs) presents significant challenges in medical studies. Many methods have been proposed, but uncertainty exists regarding the current state of missing data addressing methods applied for EHR and which strategy performs better within specific contexts. Methods: All studies referencing EHR and missing data methods published from their inception until 2024 March 30 were searched via the MEDLINE, EMBASE, and Digital Bibliography and Library Project databases. The characteristics of the included studies were extracted. We also compared the performance of various methods under different missingness scenarios. Results: After screening, 46 studies published between 2010 and 2024 were included. Three missingness mechanisms were simulated when evaluating the missing data methods: missing completely at random (29/46), missing at random (20/46), and missing not at random (21/46). Multiple imputation by chained equations (MICE) was the most popular statistical method, whereas generative adversarial network-based methods and the k nearest neighbor (KNN) classification were the common deep-learning-based or traditional machine-learning-based methods, respectively. Among the 26 articles comparing the performance among medical statistical and machine learning approaches, traditional machine learning or deep learning methods generally outperformed statistical methods. Med.KNN and context-aware time-series imputation performed better for longitudinal datasets, whereas probabilistic principal component analysis and MICE-based methods were optimal for cross-sectional datasets. Conclusions: Machine learning methods show significant promise for addressing missing data in EHRs. However, no single approach provides a universally generalizable solution. Standardized benchmarking analyses are essential to evaluate these methods across different missingness scenarios.

MAMLCDA: A Meta-Learning Model for Predicting circRNA-Disease Association Based on MAML Combined With CNN.

Circular RNAs (circRNAs) exist in vivo and are a class of noncoding RNA molecules. They have a single-stranded, closed, annular structure. Many studies have shown that circRNAs and diseases are linked. Therefore, it is critical to build a reliable and accurate predictor to find the circRNA-disease association. In this paper, we presented a meta-learning model named MAMLCDA to identify the circRNA-disease association, which is based on model-agnostic meta-learning (MAML) combined with CNN classification. Specifically, similarities between diseases and circRNAs are extracted and integrated to characterize their relationships, and k-means is used to cluster majority samples and select a certain number of samples from each cluster to obtain the same number of negative samples as the positive samples. To further reduce the dimension of the features and save operation time, we applied probabilistic principal component analysis (PPCA) to compact the integrated circRNA and disease similarity network feature vectors. The feature vectors are converted into images. At this time, the prediction problem is transformed into the 2-way 1-shot problem of the image and input into the model with MAML as the meta-learner and CNN as the base-learner. Comparison results of five-fold cross-validation on two benchmark datasets illustrate that MAMLCDA outperforms several state-of-the-art approaches with the best accuracies of 95.33% and 98%. Therefore, MAMLCDA can help to understand the pathogenesis of complex diseases at the circRNA level.

A novel machine learning model for breast cancer detection using mammogram images.

The most fatal disease affecting women worldwide now is breast cancer. Early detection of breast cancer enhances the likelihood of a full recovery and lowers mortality. Based on medical imaging, researchers from all around the world are developing breast cancer screening technologies. Due to their rapid progress, deep learning algorithms have caught the interest of many in the field of medical imaging. This research proposes a novel method in mammogram image feature extraction with classification and optimization using machine learning in breast cancer detection. The input image has been processed for noise removal, smoothening, and normalization. The input image features were extracted using probabilistic principal component analysis for detecting the presence of tumors in mammogram images. The extracted tumor region is classified using the Naïve Bayes classifier and transfer integrated convolution neural networks. The classified output has been optimized using firefly binary grey optimization and metaheuristic moth flame lion optimization. The experimental analysis has been carried out in terms of different parameters based on datasets. The proposed framework used an ensemble model for breast cancer that made use of the proposed Bayes + FBGO and TCNN + MMFLO classifier and optimizer for diverse mammography image datasets. The INbreast dataset was evaluated using the proposed Bayes + FBGO and TCNN + MMFLO classifiers, which achieved 95% and 98% accuracy, respectively.

A framework for longitudinal latent factor modelling of treatment response in clinical trials with applications to Psoriatic Arthritis and Rheumatoid Arthritis

Objective:Clinical trials involve the collection of a wealth of data, comprising multiple diverse measurements performed at baseline and follow-up visits over the course of a trial. The most common primary analysis is restricted to a single, potentially composite endpoint at one time point. While such an analytical focus promotes simple and replicable conclusions, it does not necessarily fully capture the multi-faceted effects of a drug in a complex disease setting. Therefore, to complement existing approaches, we set out here to design a longitudinal multivariate analytical framework that accepts as input an entire clinical trial database, comprising all measurements, patients, and time points across multiple trials. Methods:Our framework composes probabilistic principal component analysis with a longitudinal linear mixed effects model, thereby enabling clinical interpretation of multivariate results, while handling data missing at random, and incorporating covariates and covariance structure in a computationally efficient and principled way. Results:We illustrate our approach by applying it to four phase III clinical trials of secukinumab in Psoriatic Arthritis (PsA) and Rheumatoid Arthritis (RA). We identify three clinically plausible latent factors that collectively explain 74.5% of empirical variation in the longitudinal patient database. We estimate longitudinal trajectories of these factors, thereby enabling joint characterisation of disease progression and drug effect. We perform benchmarking experiments demonstrating our method’s competitive performance at estimating average treatment effects compared to existing statistical and machine learning methods, and showing that our modular approach leads to relatively computationally efficient model fitting. Conclusion:Our multivariate longitudinal framework has the potential to illuminate the properties of existing composite endpoint methods, and to enable the development of novel clinical endpoints that provide enhanced and complementary perspectives on treatment response.

A Review of Statistical-Based Fault Detection and Diagnosis with Probabilistic Models

As industrial processes grow increasingly complex, fault identification becomes challenging, and even minor errors can significantly impact both productivity and system safety. Fault detection and diagnosis (FDD) has emerged as a crucial strategy for maintaining system reliability and safety through condition monitoring and abnormality recovery to manage this challenge. Statistical-based FDD methods that rely on large-scale process data and their features have been developed for detecting faults. This paper overviews recent investigations and developments in statistical-based FDD methods, focusing on probabilistic models. The theoretical background of these models is presented, including Bayesian learning and maximum likelihood. We then discuss various techniques and methodologies, e.g., probabilistic principal component analysis (PPCA), probabilistic partial least squares (PPLS), probabilistic independent component analysis (PICA), probabilistic canonical correlation analysis (PCCA), and probabilistic Fisher discriminant analysis (PFDA). Several test statistics are analyzed to evaluate the discussed methods. In industrial processes, these methods require complex matrix operation and cost computational load. Finally, we discuss the current challenges and future trends in FDD.

Publisher Correction: The Effectiveness of a Probabilistic Principal Component Analysis Model and Expectation Maximisation Algorithm in Treating Missing Daily Rainfall Data

Publisher Correction: The Effectiveness of a Probabilistic Principal Component Analysis Model and Expectation Maximisation Algorithm in Treating Missing Daily Rainfall Data

Deep Probabilistic Principal Component Analysis for Process Monitoring.

Probabilistic latent variable models (PLVMs), such as probabilistic principal component analysis (PPCA), are widely employed in process monitoring and fault detection of industrial processes. This article proposes a novel deep PPCA (DePPCA) model, which has the advantages of both probabilistic modeling and deep learning. The construction of DePPCA includes a greedy layer-wise pretraining phase and a unified end-to-end fine-tuning phase. The former establishes a hierarchical deep structure based on cascading multiple layers of the PPCA module to extract high-level features. The latter builds an end-to-end connection between the raw inputs and the final outputs to further improve the representation of the model to high-level features. After constructing the model structure of DePPCA, we first present the detailed training processes of the pretraining and fine-tuning stages, then clarify the theoretical merits of the proposed model from the perspective of variational inference. For process monitoring purposes, we develop two statistics based on the established DePPCA. The monitoring performance of these two statistics can remain superior even if the features extracted by DePPCA are significantly compressed to univariate. This makes the feature extraction process and online monitoring procedure of DePPCA quite fast. In other words, the proposed DePPCA can achieve accurate and efficient process monitoring by only extracting one feature for each sample. Finally, the effectiveness of DePPCA is evaluated on the Tennessee Eastman (TE) process and the multiphase flow (MPF) facility.

Effective application of biosensor analytical techniques in drug testing

Abstract This study explores biosensor technology, focusing on its application in drug detection through advanced quantitative analysis methods: partial least squares (PLS) and probabilistic principal component analysis (PPCA). We developed a rapid quantitative calibration model using azure A, B, and C—metabolites of pefloxacin mesylate and methylene blue— demonstrated through surface-enhanced Raman spectroscopy. The findings highlight the superior accuracy of PLS and PPCA in predicting drug concentrations, with pefloxacin mesylate detection deviations maintained between 0.24%-0.98% and 0.35%-1.02%, respectively. PLS proved to be slightly more effective. This study confirms the potential of biosensor technology in ensuring drug safety, offering substantial support for public health protection and regulatory compliance.

Factors Affecting Micro, Small, and Medium-Sized Enterprise Development in Developing Asia: Findings from a Probabilistic Principal Component Analysis

Factors Affecting Micro, Small, and Medium-Sized Enterprise Development in Developing Asia: Findings from a Probabilistic Principal Component Analysis

Streaming variational probabilistic principal component analysis for monitoring of nonstationary process

Modern industrial processes are characteristic of nonstationary and uncertainty. To address these issues, this paper proposes a probabilistic principal component analysis based model that utilizes streaming variational inference for online monitoring of nonstationary process. The model parameters are updated in a streaming/batch-wise manner to track the nonstationary behavior in the process. In the updating procedures, the posterior distributions obtained from the previous data batch are used as priors for the current model, whose parameters can be estimated using the variational Bayesian inference. The streaming variational inference enables recursive parameter estimation over time. Additionally, in order to improve computational efficiency, a distributed update strategy is also applied to the proposed model. For online process monitoring, the noncentral chi-square distributions are derived to estimate the statistical control limits for each data batch, resulting in better adaptation to the nonstationary characteristics. The effectiveness and superiority of the proposed method are validated through simulation example and real-world industrial application.

Robust Bilinear Probabilistic PCA Using a Matrix Variate t Distribution.

The bilinear probabilistic principal component analysis (BPPCA) was introduced recently as a model-based dimension reduction technique on matrix data. However, BPPCA is based on the Gaussian assumption and hence is vulnerable to potential outlying matrix-valued observations. In this article, we present a new robust extension of BPPCA, called BPPCA using a matrix variate t distribution ( t BPPCA), that is built upon a matrix variate t distribution. Like the multivariate t , this distribution offers an additional robustness tuning parameter, which can downweight outliers. By introducing a Gamma distributed latent weight variable, this distribution can be represented hierarchically. With this representation, two efficient accelerated expectation-maximization (EM)-like algorithms for parameter estimation are developed. Experiments on a number of synthetic and real datasets are conducted to understand t BPPCA and compare with several closely related competitors, including its vector-based counterpart. The results reveal that t BPPCA is generally more robust and accurate in the presence of outliers. Moreover, the expected latent weights under t BPPCA can be effectively used for outliers' detection, which is much more reliable than its vector-based counterpart due to its better robustness.

Generative models for anomaly detection and design-space dimensionality reduction in shape optimization

Our work presents a novel approach to shape optimization, with the twofold objective to improve the efficiency of global optimization algorithms while promoting the generation of high-quality designs during the optimization process free of geometrical anomalies. This is accomplished by reducing the number of the original design variables defining a new reduced subspace where the geometrical variance is maximized and modeling the underlying generative process of the data via probabilistic linear latent variable models such as factor analysis and probabilistic principal component analysis. We show that the data follows approximately a Gaussian distribution when the shape modification method is linear and the design variables are sampled uniformly at random, due to the direct application of the central limit theorem. The degree of anomalousness is measured in terms of Mahalanobis distance, and the paper demonstrates that abnormal designs tend to exhibit a high value of this metric. This enables the definition of a new optimization model where anomalous geometries are penalized and consequently avoided during the optimization loop. The procedure is demonstrated for hull shape optimization of the DTMB 5415 model, extensively used as an international benchmark for shape optimization problems. The global optimization routine is carried out using Bayesian optimization and the DIRECT algorithm. From the numerical results, the new framework improves the convergence of global optimization algorithms, while only designs with high-quality geometrical features are generated through the optimization routine thereby avoiding the wastage of precious computationally expensive simulations.

Hierarchical Classification for Large-Scale Learning

Deep neural networks (DNNs) have drawn much attention due to their success in various vision tasks. Current DNNs are used on data with a relatively small number of classes (e.g., 1000 or less) and employ a fully connected layer for classification, which allocates one neuron for each class and thus, per-example, the classification scales as O(K) with the number of classes K. This approach is computationally intensive for many real-life applications where the number of classes is very large (e.g., tens of thousands of classes). To address this problem, our paper introduces a hierarchical approach for classification with a large number of classes that scales as O(K) and could be extended to O(logK) with a deeper hierarchy. The method, called Hierarchical PPCA, uses a self-supervised pretrained feature extractor to obtain meaningful features and trains Probabilistic PCA models on the extracted features for each class separately, making it easy to add classes without retraining the whole model. The Mahalanobis distance is used to obtain the classification result. To speed-up classification, the proposed Hierarchical PPCA framework clusters the image class models, represented as Gaussians, into a smaller number of super-classes using a modified k-means clustering algorithm. The classification speed increase is obtained by Hierarchical PPCA assigning a sample to a small number of the most likely super-classes and restricting the image classification to the image classes corresponding to these super-classes. The fact that the model is trained on each class separately makes it applicable to training on very large datasets such as the whole ImageNet with more than 10,000 classes. Experiments on three standard datasets (ImageNet-100, ImageNet-1k,and ImageNet-10k) indicate that the hierarchical classifier can achieve a superior accuracy with up to a 16-fold speed increase compared to a standard fully connected classifier.

A new monitoring approach of time-varying and nonlinear processes with application to penicillin fermentation process

In the actual production process, time-varying and nonlinear problems are numerous important problems to be considered, in view of these problems, a process monitoring approach based on locally weighted probabilistic kernel principal component analysis (LWPKPCA) is proposed. First, the method selects the normal process data with a high similarity to the test samples as training data of the local model, and continuously updates the local model according to the test samples to build an accurate time-varying model. Second, by weighting the data of different importance, the role of data similar to test samples in the modeling process is strengthened. Third, the LWPKPCA model is applied to process monitoring, the monitoring indicators are established in a high-dimensional space and used to detect faults. Finally, on the basis of LWPKPCA, the penicillin fermentation process (PFP) is taken to evaluate the monitoring performance of the proposed methods. According to the comparison of the experiment results, the detection rate and accuracy rate of the LWPKPCA method is considerably better than those of probabilistic principal component analysis and probabilistic kernel principal component analysis methods. The results demonstrate that the proposed method is suitable for processing time-varying data with nonlinear characteristics, and the LWPKPCA process monitoring method is effective for improving the performance of fault detection.

Formula omitted]CA-GAM-ID: Coupling of probabilistic principal components analysis with generalised additive model to predict the [formula omitted]barriers for intrusion detection

Drastic advancement in computing technology and the dramatic increase in the usage of explainable machine learning algorithms provide a promising platform for developing robust intrusion detection algorithms. However, the development of these algorithms is constrained by their applicability over specific scenarios of Wireless Sensor Networks (WSNs). We introduced a hybrid framework by combining Probabilistic Principal Component Analysis (P2CA) and Generalised Additive Model (GAM), which is performing well for all the scenarios of WSNs. To demonstrate our framework’s broad applicability, we evaluated its performance over three publicly available intrusion detection datasets (i.e., LT-FS-ID, AutoML-ID, and FF-ANN-ID), each from different scenarios. Our findings highlight that the presented framework can accurately predict the number of k−barriers for all three datasets. Furthermore, we conducted a comprehensive performance comparison between our proposed framework and benchmark algorithms, which revealed that our approach outperforms all of them. Additionally, we evaluated the framework’s versatility by testing its performance on datasets unrelated to intrusion detection, specifically ALE datasets. Notably, our approach accurately predicted the response variable in these datasets and exceeded the performance of its primary algorithm, further demonstrating its robustness and adaptability.The implications of this research are substantial. By developing a robust intrusion detection framework that performs well across diverse WSN scenarios, we address a critical need for reliable network security in various domains, including industrial IoT, smart cities, and environmental monitoring. Our findings not only enhance the understanding of intrusion detection in WSNs but also pave the way for developing more sophisticated and adaptable systems to safeguard sensitive data and critical infrastructure.

Synthetic data generation by probabilistic PCA

Synthetic data generation by probabilistic PCA

A Hybrid Cluster Variational Autoencoder Model for Monitoring the Multimode Blast Furnace System

Efficient monitoring of the blast furnace system is crucial for maintaining high production efficiency and ensuring product quality. This article introduces a hybrid cluster variational autoencoder model for monitoring the blast furnace ironmaking process which exhibits multimode behaviors. In contrast to traditional approaches, this method utilizes neural networks to learn data features and effectively handles the diverse feature types observed in different production modes. Through the utilization of a clustering process within the hidden layer of the variational autoencoder, the proposed technique facilitates efficient fault detection in the context of multimodal blast furnace data. Based on the variational autoencoder model, this study further establishes a unified monitoring index and defines a method for computing the control limits. The application of the model to real blast furnace data reveals its proficiency in accurately identifying faults across diverse modes; compared with the probabilistic principal component analysis based on the local nearest neighbor standardization method and the recursive probabilistic principal component analysis, the model shows a reduction in false positives by up to 10.3% and a substantial reduction of 19.2% in the missed detection rate. This method achieves a remarkable false detection rate of only 0.2% and 0 instances of missed detection.

BOSS: Bones, organs and skin shape model

A virtual anatomical model of a patient can be a valuable tool for enhancing clinical tasks such as workflow automation, patient-specific X-ray dose optimization, markerless tracking, positioning, and navigation assistance in image-guided interventions. For these tasks, it is highly desirable that the patient’s surface and internal organs are of high quality for any pose and shape estimate. At present, the majority of statistical shape models (SSMs) are restricted to a small number of organs or bones or do not adequately represent the general population. To address this, we propose a deformable human shape and pose model that combines skin, internal organs, and bones, learned from CT images. By modeling the statistical variations in a pose-normalized space using probabilistic PCA while also preserving joint kinematics, our approach offers a holistic representation of the body that can be beneficial for automation in various medical applications. In an interventional setup, our model could, for example, facilitate automatic system/patient positioning, organ-specific iso-centering, automated collimation or collision prediction. We assessed our model’s performance on a registered dataset, utilizing the unified shape space, and noted an average error of 3.6 mm for bones and 8.8 mm for organs. By utilizing solely skin surface data or patient metadata like height and weight, we find that the overall combined error for bone-organ measurement is 8.68 mm and 8.11 mm, respectively. To further verify our findings, we conducted additional tests on publicly available datasets with multi-part segmentations, which confirmed the effectiveness of our model. In the diverse TotalSegmentator dataset, the errors for bones and organs are observed to be 5.10mm and 8.72mm, respectively. Our work shows that anatomically parameterized statistical shape models can be created accurately and in a computationally efficient manner. The proposed approach enables the construction of shape models that can be directly integrated into to various medical applications.

A data-driven soft-sensing approach using probabilistic latent variable model with functional data framework

Functional principal component analysis (FPCA) and functional partial least squares (FPLS) are two mainstream functional data analysis (FDA) methods, which have been commonly used to extract deep information hidden in the original data space. However, the process data always contain random noise, which affects the performance of FDA models. To overcome this issue, two functional probabilistic latent variable models (FPLVMs), including functional probabilistic principal component analysis (FPPCA) and functional probabilistic partial least squares (FPPLS) are proposed in this work. First, the process data are converted into functional data using the FDA. Subsequently, a log-likelihood function considering the noise factor and functional latent variables is designed. Finally, the regression model parameters are estimated using an expectation–maximisation algorithm. In contrast to FPPCA, FPPLS decomposes the process data and the key variable with constrained latent variables, which is similar to the partial least squares (PLS) and the principal component analysis (PCA). Moreover, the degeneration mechanism from FPLVMs into probabilistic latent variable models and latent variable models is discussed. An adaptive strategy with functional covariance is used to satisfy the online predictive capabilities of the model. Finally, the proposed approach is validated using a numerical case, the Tennessee Eastman process and an industrial o-xylene distillation column for evaluation.