Computation Of Covariance Matrix Research Articles

A Bayesian Structural Modal Updating Method Based on Sparse Grid and Ensemble Kalman Filter

This study presents a sparse grid interpolation and ensemble Kalman filter (EnKF)-based Markov Chain Monte Carlo (MCMC) method (SG-EnMCMC). Initiating with the formulation of a recursive equation for the state space vector, derived from the structural dynamic equation, this study adopts a dimensionality reduction strategy. This approach involves a separation of physical parameters and the state space vector. The acquisition of physical parameters is accomplished through sampling, utilizing sample moments to substitute population moments, thereby mitigating the need for computationally high-dimensional covariance matrix calculations. To further streamline the recursive equation of the state space vector, a sparse grid method is employed for interpolation. This step simplifies the process while ensuring superior accuracy compared to the Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF). Subsequent to this, acceptance rates and the final parameter posterior distribution within the MCMC framework are derived. The efficiency of the proposed method is assessed through validation in two shaking table experiments.

Sparse subnetwork inference for neural network epistemic uncertainty estimation with improved Hessian approximation

Despite significant advances in deep neural networks across diverse domains, challenges persist in safety-critical contexts, including domain shift sensitivity and unreliable uncertainty estimation. To address these issues, this study investigates Bayesian learning for uncertainty handling in modern neural networks. However, the high-dimensional, non-convex nature of the posterior distribution poses practical limitations for epistemic uncertainty estimation. The Laplace approximation, as a cost-efficient Bayesian method, offers a practical solution by approximating the posterior as a multivariate normal distribution but faces computational bottlenecks in precise covariance matrix computation and storage. This research employs subnetwork inference, utilizing only a subset of the parameter space for Bayesian inference. In addition, a Kronecker-factored and low-rank representation is explored to reduce space complexity and computational costs. Several corrections are introduced to converge the approximated curvature to the exact Hessian matrix. Numerical results demonstrate the effectiveness and competitiveness of this method, whereas qualitative experiments highlight the impact of Hessian approximation granularity and parameter space utilization in Bayesian inference on mitigating overconfidence in predictions and obtaining high-quality uncertainty estimates.

Analysis of statistical uncertainties of non-contact temperature and emissivity measurements in the optical and near infrared ranges

The aim of this report is the analysis of the uncertainties of non-contact measurement of temperature and emissivity determined by the least sum of squares (LSS) method. Analytical expressions are obtained for the uncertainties of the temperature T and the emissivity ε by three methods: (1) linear regression using the Wien approximation, (2) non-linear regression with covariance matrix calculation using the Planck’s equation, and (3) hybrid method that combines linear and non-linear regression methods. It was found that the better the condition for the Wien’s approximation is fulfilled, the worse the accuracy of the emissivity measurement compared to the accuracy of the temperature measurement. The new optimal strategy is proposed to determine the T and the emissivity ε for non-contact measurements. It consists of three steps: (1) the temperature of the radiating body is found by the Wien approximation, TW; (2) the obtained value TW is used as an initial approximation for one-dimensional “hybrid” minimization. Minimization is carried out by one of the numerical methods of one-dimensional minimization. As a result, the values of ε0 and T0 are found at with the sum of the squared residuals, S0, reaches a minimum; (3) Measurement uncertainties are estimated using the covariance matrix method.

Improved privacy-preserving PCA using optimized homomorphic matrix multiplication

Principal Component Analysis (PCA) is a pivotal technique widely utilized in the realms of machine learning and data analysis. It aims to reduce the dimensionality of a dataset while minimizing the loss of information. In recent years, there have been endeavors to utilize homomorphic encryption in privacy-preserving PCA algorithms for secure cloud computing. These approaches commonly employ a PCA routine known as PowerMethod, which takes the covariance matrix as input and generates an approximate eigenvector corresponding to the primary component of the dataset. However, their performance is constrained by the absence of an efficient homomorphic covariance matrix computation circuit and an accurate homomorphic vector normalization strategy in the PowerMethod algorithm. In this study, we propose a novel approach to privacy-preserving PCA that addresses these limitations, resulting in superior efficiency, accuracy, and scalability compared to previous approaches.We attain such efficiency and precision through the following contributions: (i) We implement space and speed optimization techniques for a homomorphic matrix multiplication method, specifically tailored for parallel computing scenarios. (ii) Leveraging the benefits of this optimized matrix multiplication, we devise an efficient homomorphic circuit for computing the covariance matrix homomorphically. (iii) Utilizing the covariance matrix, we develop a novel and efficient homomorphic circuit for the PowerMethod that incorporates a universal homomorphic vector normalization strategy to enhance both its accuracy and practicality.Our privacy-preserving PCA scheme, implemented using our innovative homomorphic PowerMethod circuit, surpasses the state-of-the-art approach with an average speedup of 1.9 times on datasets with size 200×256. Notably, our scheme demonstrates an even more remarkable estimated speedup of 25 times when applied to larger datasets of size 60000×256, along with an R2 accuracy improvement of up to 0.285, showcasing efficiency and accuracy that has not been reported by previous approaches.

DESI mock challenge: constructing DESI galaxy catalogues based on FastPM simulations

ABSTRACT Together with larger spectroscopic surveys such as the Dark Energy Spectroscopic Instrument (DESI), the precision of large-scale structure studies and thus the constraints on the cosmological parameters are rapidly improving. Therefore, one must build realistic simulations and robust covariance matrices. We build galaxy catalogues by applying a halo occupation distribution (HOD) model upon the FastPM simulations, such that the resulting galaxy clustering reproduces high-resolution N-body simulations. While the resolution and halo finder are different from the reference simulations, we reproduce the reference galaxy two-point clustering measurements – monopole and quadrupole – to a precision required by the DESI Year 1 emission line galaxy sample down to non-linear scales, i.e. $k\lt 0.5\, h\, \mathrm{Mpc}^{-1}$ or $s\gt 10\, \mathrm{Mpc}\, h^{-1}$. Furthermore, we compute covariance matrices based on the resulting FastPM galaxy clustering – monopole and quadrupole. We study for the first time the effect of fitting on Fourier conjugate (e.g. power spectrum) on the covariance matrix of the Fourier counterpart (e.g. correlation function). We estimate the uncertainties of the two parameters of a simple clustering model and observe a maximum variation of 20 per cent for the different covariance matrices. Nevertheless, for most studied scales the scatter is between 2 and 10 per cent. Consequently, using the current pipeline we can precisely reproduce the clustering of N-body simulations and the resulting covariance matrices provide robust uncertainty estimations against HOD fitting scenarios. We expect our methodology will be useful for the coming DESI data analyses and their extension for other studies.

Robustness of baryon acoustic oscillations measurements with photometric redshift uncertainties

ABSTRACT We investigate the robustness of baryon acoustic oscillations (BAO) measurements with a photometric galaxy sample using mock galaxy catalogues with various sizes of photometric redshift (photo-z) uncertainties. We first conduct the robustness of BAO measurements, assuming we have a perfect knowledge of photo-z uncertainties. We find that the BAO shift parameter α can be constrained in an unbiased manner even for 3 per cent photometric redshift uncertainties up to z ∼ 1. For instance, α = 1.006 ± 0.078 with 95 per cent confidence level is obtained from 3 per cent photo-z uncertainty data at z = 1.03 using the sample of M* ≥ 1010.25 M⊙ h−2. We also find that a sparse galaxy sample, e.g. <2 × 10−4 [h Mpc−1]3, causes additional noise in the covariance matrix calculation and can bias the constraint on α. Following this, we look into the scenario where incorrect photometric redshift uncertainties are assumed in the fitting model. We find that underestimating the photo-z uncertainty leads to a degradation in the constraining power on α. However, the constrained value of α is not biased. We also quantify the constraining power on Ωm0 assuming the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST)-like covariance and find that the 95 per cent confidence level is σ(Ωm0) ∼ 0.03–0.05 corresponding to the photo-z uncertainties of 1–3 per cent, respectively. Finally, we examine whether the skewness in the photometric redshift can bias the constraint on α and confirm that the constraint on α is unbiased, even assuming a Gaussian photo-z uncertainty in our model.

Structure, dynamics and free energy studies on the effect of point mutations on SARS-CoV-2 spike protein binding with ACE2 receptor.

The ongoing COVID-19 pandemic continues to infect people worldwide, and the virus continues to evolve in significant ways which can pose challenges to the efficiency of available vaccines and therapeutic drugs and cause future pandemic. Therefore, it is important to investigate the binding and interaction of ACE2 with different RBD variants. A comparative study using all-atom MD simulations was conducted on ACE2 binding with 8 different RBD variants, including N501Y, E484K, P479S, T478I, S477N, N439K, K417N and N501Y-E484K-K417N on RBD. Based on the RMSD, RMSF, and DSSP results, overall the binding of RBD variants with ACE2 is stable, and the secondary structure of RBD and ACE2 are consistent after the point mutation. Besides that, a similar buried surface area, a consistent binding interface and a similar amount of hydrogen bonds formed between RBD and ACE2 although the exact residue pairs on the binding interface were modified. The change of binding free energy from point mutation was predicted using the free energy perturbation (FEP) method. It is found that N501Y, N439K, and K417N can strengthen the binding of RBD with ACE2, while E484K and P479S weaken the binding, and S477N and T478I have negligible effect on the binding. Point mutations modified the dynamic correlation of residues in RBD based on the dihedral angle covariance matrix calculation. Doing dynamic network analysis, a common intrinsic network community extending from the tail of RBD to central, then to the binding interface region was found, which could communicate the dynamics in the binding interface region to the tail thus to the other sections of S protein. The result can supply unique methodology and molecular insight on studying the molecular structure and dynamics of possible future pandemics and design novel drugs.

High-level synthesis assisted, low-latency, area- and power-optimized FPGA implementation of MUSIC algorithm for direction-of-arrival estimation

For multiple signal-processing applications, estimating the target’s speed, distance, and elevation angle by using applications running in real time and consequently, fast data rates is important. Custom hardwares, such as field programmable gate arrays (FPGAs), are often used to prototype and deploy such signal-processing algorithms. Multiple signal classification (MUSIC) is typically used for direction-of-arrival (DOA) estimation. It involves complex mathematical operations such as covariance matrix computation, eigen value decomposition, and peak value search for signal values. With a novel high-level synthesis (HLS)-based design approach that optimizes the bit widths for the eigen value decomposition logic, this study presents an area and speed-optimal implementation of MUSIC algorithm on FPGA. The final implementation involving the covariance matrix computation, eigen value decomposition, and peak search was implemented on Xilinx Zynq (XC7Z020) FPGA. Because of its optimum bit widths, area utilization was minimized and operation frequency on the target FPGA was maximized. The proposed design works in real time to estimate the DOA in less than 1.7 µs (15 % faster than reported values) and uses up to 30 % less resources on the target FPGA compared with other reported implementations.

Detecting Anomaly in Chemical Sensors via L1-Kernel-Based Principal Component Analysis

We propose a kernel-PCA-based method to detect anomaly in chemical sensors. We use temporal signals produced by chemical sensors to form vectors to perform the principal component analysis (PCA). We estimate the kernel-covariance matrix of the sensor data and compute the eigenvector corresponding to the largest eigenvalue of the covariance matrix. The anomaly can be detected by comparing the difference between the actual sensor data and the reconstructed data from the dominant eigenvector. In this letter, we introduce a new multiplication-free kernel, which is related to the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> -norm for the anomaly detection task. The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> -kernel PCA is not only computationally efficient but also energy-efficient because it does not require any actual multiplications during the kernel covariance matrix computation. Our experimental results show that our kernel-PCA method achieves a higher area under curvature score (0.7483) than the baseline regular PCA method (0.7366).

A hybrid data-driven and mechanistic model soft sensor for estimating CO2 concentrations for a carbon capture pilot plant

Integrating post-combustion carbon capture and storage (CCS) facilities into fossil fuel power plants is considered an important step for reaching global carbon emission reduction targets. When the number of gas analyzers in such CCS units is limited, variations in the load of the power plant pose a challenge to determine the gaseous CO2 concentration profile in the absorber. A dynamic hybrid model for estimating the carbon capture absorber’s gaseous CO2 concentration profile has been proposed in this study. The model is built using actual process data collected from a carbon capture pilot plant and it combines data-driven and reaction kinetic (mechanistic) modeling approaches to act as a soft sensor along the absorber column. A subset of the process data is used for training the data-driven models and for estimating the parameters of the mechanistic model respectively. Dimensionality reduction techniques are applied to the data-driven model to reduce the input size and hence the size of the dynamic model elements. The outputs of the two models are fused by comparing the computed covariance matrices. A particular challenge for this work is that the collected process data has missing spatial labels and temporal values for the CO2 concentration measurements. The presented models are obtained using an encoding and interpolation approach for the missing information. For comparison, an alternative approach based on semi-supervised learning has been implemented. The performance of the resulting soft sensors is verified by using process data from previously unseen operating conditions. The soft sensor based on the proposed hybrid model outperforms the soft sensor trained by semi-supervised autoencoder. Overall the results indicate that the proposed approach can estimate the CO2 concentration percentage with an average root mean squared error of 0.123.

Temporal Correlation Detection Based on 3D NAND Flash In-Memory Computing

The detection of temporal correlations among event-driven data streams is widely demanded in data-intensive applications such as edge computing. In this work, temporal correlation detection is enabled by the combination of 3D NAND flash and in-memory computing. Different from mathematical scheme by covariance matrix calculation, here the detection is implemented by exploiting the physical process of charge tunneling and accumulation. Thus the intrinsic charge-trapping dynamics in 3D NAND flash can be leveraged to perform in-memory computing. The computing results are stored and imprinted in the threshold voltages of the 3D NAND device array, enabling co-existence of computation and storage in the same devices. On this basis, the validation of temporal correlation detection further paves the way for 3D NAND application in high-level non-von Neumann computing.

Neural networks and standard cosmography with newly calibrated high redshift GRB observations

Gamma-ray bursts (GRBs) detected at high redshift can be used to trace the cosmic expansion history. However, the calibration of their luminosity distances is not an easy task in comparison to Type Ia Supernovae (SNeIa). To calibrate these data, correlations between their luminosity and other observed properties of GRBs need to be identified, and we must consider the validity of our assumptions about these correlations over their entire observed redshift range. In this work, we propose a new method to calibrate GRBs as cosmological distance indicators using SNeIa observations with a machine learning architecture. As well we include a new data GRB calibrated sample using extended cosmography in a redshift range above z > 3.6. An overview of this machine learning technique was developed in [1] to study the evolution of dark energy models at high redshift. The aim of the method developed in this work is to combine two networks: a Recurrent Neural Network (RNN) and a Bayesian Neural Network (BNN). Using this computational approach, denoted RNN+BNN, we extend the network's efficacy by adding the computation of covariance matrices to the Bayesian process. Once this is done, the SNeIa distance-redshift relation can be tested on the full GRB sample and therefore used to implement a cosmographic reconstruction of the distance-redshift relation in different regimes. Thus, our newly-trained neural network is used to constrain the parameters describing the kinematical state of the Universe via a cosmographic approach at high redshifts (up to z ≈ 10), wherein we require a very minimal set of assumptions on the deep learning arquitecture itself that do not rely on dynamical equations for any specific theory of gravity.

STAP Training Samples Selection Based on GIP and Volume Cross Correlation

Space-time adaptive processing requires reliable training samples to support clutter covariance matrix calculation of the cell under test in the process of moving target detection. However, the heterogeneous clutter and the existence of outliers in the training samples violate the assumption of independent identically distributed with the cell under test, which is crucial for statistical estimation. To solve this problem, this letter suggests a new strategy to select training samples that are homogeneous with the cell under test. First, the generalized inner product method is used to filter training samples with outliers and generate the corresponding load matrix. Second, a distance metric between the training sample clutter subspace and the cell under test clutter subspace is formed by the volume cross-correlation function, the distance metric is used to judge the clutter similarity between the training sample and the cell under test. Finally, the training samples with similar clutter distribution characteristics to the cell under test are selected. Experimental results show that Space-time adaptive processing performance can be improved effectively, especially in heterogeneous environments.

Block-enhanced precision matrix estimation for large-scale datasets

The ℓ1-regularized Gaussian maximum likelihood method is a common approach for sparse precision matrix estimation, but one that poses a computational challenge for high-dimensional datasets. We present a novel ℓ1-regularized maximum likelihood method for performant large-scale sparse precision matrix estimation utilizing the block structures in the underlying computations. We identify the computational bottlenecks and contribute a block coordinate descent update as well as a block approximate matrix inversion routine, which is then parallelized using a shared-memory scheme. We demonstrate the effectiveness, accuracy, and performance of these algorithms. Our numerical examples and comparative results with various modern open-source packages reveal that these precision matrix estimation methods can accelerate the computation of covariance matrices by two to three orders of magnitude, while keeping memory requirements modest. Furthermore, we conduct large-scale case studies for applications from finance and medicine with several thousand random variables to demonstrate applicability for real-world datasets.

State-of-charge estimation of lithium-ion battery pack by using an adaptive extended Kalman filter for electric vehicles

State-of-charge (SOC) estimation is an important aspect for modern battery management systems. Extended Kalman filter (EKF) has been extensively used in battery SOC estimation. However, EKF cannot obtain accurate estimation results when the model parameters have strong uncertainty or/and the accurate initial value of noise covariance matrix is unknown. To overcome these defects, the parameters of Lithium-ion battery model on the basis of the second-order resistor–capacitor (RC) equivalent model are identified, and then an improved adaptive EKF (IAEKF) of SOC estimation method for Lithium-ion battery pack is proposed for enhancing estimation accurate and robustness. In IAEKF, the statistical characteristics of measurement noise is adaptively corrected using a forgetting factor, namely, Sage–Husa EKF (SHEKF), and the error covariance matrix is adaptively corrected in accordance with the innovation, in which the calculation of the actual innovation covariance matrix adopts the variable sliding window length. Results of numerical simulation and experiment show that the proposed SOC estimation method can accurately estimate SOC under complex driven condition and has strong robustness to the uncertainty of model parameters and the initial value of the noise covariance matrix.

Performance Analysis of Robust GRCR Based Spectrum Detector Using Compressed Sensing with Non-reconstruction Model

In cognitive radio applications, a robust detector is essentially required for the determination of spectrum sensing. Gerschgorin radii and centers ratio (GRCR) detector is a robust detector for cooperative spectrum sensing techniques. Covariance matrix generates the test statistics for signal established from one or supplementary sources. Though the method is robust against noise uncertainty, it is not suitable for wideband sensing due to the complexity associated with the computation of covariance matrix. To tackle this challenge of extensive communication cost and high processing time complexity, an efficient GRCR detector using compressive sensing with non-reconstruction is proposed here. This method introduces relevance for multiple received signals by using same measurement matrix to all received signals. Computational complexity is analysed and the proposed method is compared with the existing method through ROC simulations, and it shown that the proposed method performs better even in the low SNR range of -20 dB. Throughput analysis is validated through simulations.

Shared Linear Encoder-Based Multikernel Gaussian Process Latent Variable Model for Visual Classification.

Multiview learning has been widely studied in various fields and achieved outstanding performances in comparison to many single-view-based approaches. In this paper, a novel multiview learning method based on the Gaussian process latent variable model (GPLVM) is proposed. In contrast to existing GPLVM methods which only assume that there are transformations from the latent variable to the multiple observed inputs, our proposed method simultaneously takes a back constraint into account, encoding multiple observations to the latent variable by enjoying the Gaussian process (GP) prior. Particularly, to overcome the difficulty of the covariance matrix calculation in the encoder, a linear projection is designed to map different observations to a consistent subspace first. The obtained variable in this subspace is then projected to the latent variable in the manifold space with the GP prior. Furthermore, different from most GPLVM methods which strongly assume that the covariance matrices follow a certain kernel function, for example, radial basis function (RBF), we introduce a multikernel strategy to design the covariance matrix, being more reasonable and adaptive for the data representation. In order to apply the presented approach to the classification, a discriminative prior is also embedded to the learned latent variables to encourage samples belonging to the same category to be close and those belonging to different categories to be far. Experimental results on three real-world databases substantiate the effectiveness and superiority of the proposed method compared with state-of-the-art approaches.

Machine Learning-Based Similarity Attacks for Chaos-based Cryptosystems

When the chaotic block cryptographic algorithms are performed on hardware devices, the leakages of power consumption etc. are crucial information which can be used to analyse the security of the cryptosystems. Template Attack (TA) can recover the secret key. However, there are still some challenges for TA such as irreversible covariance matrix and exponentiation calculation overflow. Machine Learning-based Similarity Attacks (MLSAs) are proposed to effectively analyse the sensitive information of the chaotic block cryptosystem. The proposed method consists of three steps: parameter tuning, learning and attacking. For the parameter tuning, the profiling traces are categorised according to the Hamming weights of sensitive intermediate data. Then a 10-fold cross-validation is executed to determine the corresponding parameter settings for learning algorithms. In the learning step, the profiling traces and Hamming weight labels are used to train machine learning models, and in the attacking step different similarity measure methods are used to calculate similarities between actual and hypothetical Hamming weight labels to attack the secret keys. Performance analyses demonstrate that the proposed MLSAs have higher success rates than TA and lower computational time consumptions under most of scenarios. Therefore, the MLSAs can efficiently attack and analyse hardware security of chaotic block cryptosystems.

Adaptive Subcarrier Grouping for MMSE-SMI Adaptive Array Interference Suppression

Inter-user interference (IUI) suppression is a prime matter for mobile communication systems. Channel state information (CSI), meanwhile, is severely affected by the receiver additive noise effect. Sample matrix inversion (SMI) based minimum mean square error (MMSE) adaptive array is a powerful means to suppress noise enhancement with the aid of data symbols. However, its effectiveness is limited under an insufficient number of symbols available. Based on the multi-carrier transmission system, this paper proposes an adaptive subcarrier grouping to expand the number of received symbols for a covariance matrix calculation. The grouping size is adaptively adjusted by observing the dispersion of adjacent subcarriers. Simulative results reveal the effectiveness of the proposed adaptive algorithm; superior bit error rate (BER) performance can be attained under various Rician $K$ factors.

Photovoltaic Generation Prediction of CCIPCA Combined with LSTM

In order to remedy problems encompassing large-scale data being collected by photovoltaic (PV) stations, multiple dimensions of power prediction mode input, noise, slow model convergence speed, and poor precision, a power prediction model that combines the Candid Covariance-free Incremental Principal Component Analysis (CCIPCA) with Long Short-Term Memory (LSTM) network was proposed in this study. The corresponding model uses factor correlation coefficient to evaluate the factors that affect PV generation and obtains the most critical factor of PV generation. Then, it uses CCIPCA to reduce the dimension of PV super large-scale data to the factor dimension, avoiding the complex calculation of covariance matrix of algorithms such as Principal Component Analysis (PCA) and to some extent eliminating the influence of noise made by PV generation data acquisition equipment and transmission equipment such as sensors. The training speed and convergence speed of LSTM are improved by the dimension-reduced data. The PV generation data of a certain power station over a period is collected from SolarGIS as sample data. The model is compared with Markov chain power generation prediction model and GA-BP power generation prediction model. The experimental results indicate that the generation prediction error of the model is less than 3%.