Matrix Estimation Research Articles

Toeplitz covariance estimation with applications to MUSIC

We consider the maximum likelihood estimation of a covariance matrix that is known to be Toeplitz. The corresponding optimization problem is nonconvex, and to compute a local minimizer we propose an efficient implementation of Newton’s method. The cost per iteration of a straightforward implementation is of order O(n4), where n+1 is the dimension of the covariance matrix. By exploiting fast Fourier transforms to assemble the gradient and Hessian we reduce the iteration cost to order O(n3). Through extensive simulations we demonstrate that for direction-of-arrival estimation, the performance of MUSIC is significantly enhanced by using it together with the maximum likelihood Toeplitz covariance estimate. The simulations also demonstrate that our open-source implementation is orders of magnitude faster than two existing Toeplitz covariance estimators.

Music Curriculum Research Using a Large Language Model, Cloud Computing and Data Mining Technologies

This paper presents a method to enhance the scientific nature of the music curriculum model by integrating a large language model, cloud computing and data mining technology for the analysis of the music teaching curriculum model. To maintain the integrity of the mixing matrix while employing the frequency hopping frequency, the paper suggests dividing the mixing matrix into a series of sub-matrices along the vertical time axis. This approach transforms wideband music signal processing into a narrowband processing problem. Additionally, two hybrid matrix estimation algorithms are proposed in this paper using underdetermined conditions. Furthermore, utilizing the estimated mixing matrix and the detected time-frequency support domain, the paper employs the subspace projection algorithm for underdetermined blind separation of music signals in the time-frequency domain. This procedure, along with the integration of the estimated direction of arrival (DoA), enables the completion of frequency-hopping network station music signal sorting. Extensive simulation teaching demonstrates that the music curriculum model proposed in this paper, based on a large language model, cloud computing and data mining technologies, significantly enhances the quality of modern music teaching.

Spectral and optical properties of a [formula omitted] molecular complex confined in InAs/GaAs Quantum Camels: A spatial finite element method treatment

The evolution of eigenstates and related optical absorption (sum of linear and nonlinear contributions) of a singly ionized double donor (D2+) confined in InAs/GaAs (quantum-well-dot nanostructure) quantum camels were theoretically investigated. Within the effective mass and the compact density matrix approximations, the energy levels and optical absorption were characterized as a function of a hydrostatic pressure field applied, the sample temperature, and the wetting layer thickness via a spatial finite element method analysis. Results show that an increase of hydrostatic pressure (sample temperature) between 0 to 30 kbar (4 to 300 K) leads to a stabilization (destabilization) of the energy levels and a decrease (increase) of the absorption peak value with a red-shift (blue-shift). On the other hand, an increase in the wetting layer thickness leads to a reduction in the energy levels, and a monotonous evolution from quantum-dot states to wetting-layer states, especially in the excited ones, is seen. In addition, an increase in the absorption peak value is reported, which can be intended as an electron mobility enhancement. The donor-electron interactions reduce the wavefunction’s spatially-extended feature, enhancing the absorption peak in the analyzed |1〉→|6〉 transition. In the |1〉→|2〉 optical transition, the resonant peak can be tuned within the band ≈25–48 meV, slightly beyond the Terahertz frequency range.

Multivariate realized volatility: an analysis via shrinkage methods for Brazilian market data

IntroductionRealized volatility analysis of assets in the Brazilian market within a multivariate framework is the focus of this study. Despite the success of volatility models in univariate scenarios, challenges arise due to increasing dimensionality of covariance matrices and lower asset liquidity in emerging markets.MethodsIn this study, we utilize intraday stock trading data from the Brazilian Market to compute daily covariance matrices using various specifications. To mitigate dimensionality issues in covariance matrix estimation, we implement penalization restrictions on coefficients through regressions with shrinkage techniques using Ridge, LASSO, or Elastic Net estimators. These techniques are employed to capture the dynamics of covariance matrices.ResultsComparison of covariance construction models is performed using the Model Confidence Set (MCS) algorithm, which selects the best models based on their predictive performance. The findings indicate that the method used for estimating the covariance matrix significantly impacts the selection of the best models. Additionally, it is observed that more liquid sectors demonstrate greater intra-sectoral dynamics.DiscussionWhile the results benefit from shrinkage techniques, the high correlation between assets presents challenges in capturing stock or sector idiosyncrasies. This suggests the need for further exploration and refinement of methods to better capture the complexities of volatility dynamics in emerging markets like Brazil.

Communication channel parameters estimation using a ray tracing model in heterogeneous networks

Currently, modern networks LTE, Wi-Fi, 5GNR are combined into a single heterogeneous network, which leads to an increase in intra-system interference. According to the integrated optimization method, to distribute data flows in a heterogeneous network, taking into account the intra-system interference influence, the collective dynamic routing method is used. At the analysis stage of this method it is necessary to estimate the channel data rate in all valid routes. To obtain correct channel data rates estimates, it is necessary to reliably estimate the channel matrix. Today, the ray tracing model is widely used to estimate the channel matrix. The verification results of the using ray tracing model given in [1] show the need for additional research to increase the channel matrix estimates reliability. Therefore increasing the reliability of channel matrix transmission coefficients estimates by analyzing the influence of the number of rays taken into account is the actual task. The work analyzes the influence of the number of rays taken into account on the reliability of estimating the complex transmission coefficients of the channel matrix for the cases of the presence and absence of line of sight be-tween the transmitter and the receiver. Expressions are obtained to take into account the influence of diffracted, singly reflected and passed through an obstacle rays. A program has been developed to find all possible rays arriving at the receiver location point and determine their parameters based on a digital 3D map indicating the map objects materials. Using the developed program, it is shown that taking into account the influence of additional rays makes it possible to increase the channel matrix estimation reliability, and therefore, the modified ray tracing model can be used to estimate channel data rates when applying the collective dynamic routing method in heterogeneous net-works.

Flat minima generalize for low-rank matrix recovery

Abstract Empirical evidence suggests that for a variety of overparameterized nonlinear models, most notably in neural network training, the growth of the loss around a minimizer strongly impacts its performance. Flat minima—those around which the loss grows slowly—appear to generalize well. This work takes a step towards understanding this phenomenon by focusing on the simplest class of overparameterized nonlinear models: those arising in low-rank matrix recovery. We analyse overparameterized matrix and bilinear sensing, robust principal component analysis, covariance matrix estimation and single hidden layer neural networks with quadratic activation functions. In all cases, we show that flat minima, measured by the trace of the Hessian, exactly recover the ground truth under standard statistical assumptions. For matrix completion, we establish weak recovery, although empirical evidence suggests exact recovery holds here as well. We complete the paper with synthetic experiments that illustrate our findings.

Element Aggregation for Estimation of High-Dimensional Covariance Matrices

This study addresses the challenge of estimating high-dimensional covariance matrices in financial markets, where traditional sparsity assumptions often fail due to the interdependence of stock returns across sectors. We present an innovative element-aggregation method that aggregates matrix entries to estimate covariance matrices. This method is designed to be applicable to both sparse and non-sparse matrices, transcending the limitations of sparsity-based approaches. The computational simplicity of the method’s implementation ensures low complexity, making it a practical tool for real-world applications. Theoretical analysis then confirms the method’s consistency and effectiveness with its convergence rate in specific scenarios. Additionally, numerical experiments validate the method’s superior algorithmic performance compared to conventional methods, as well as the reduction in relative estimation errors. Furthermore, empirical studies in financial portfolio optimization demonstrate the method’s significant risk management benefits, particularly its ability to effectively mitigate portfolio risk even with limited sample sizes.

Exploiting Time–Frequency Sparsity for Dual-Sensor Blind Source Separation

This paper explores the important role of blind source separation (BSS) techniques in separating M mixtures including N sources using a dual-sensor array, i.e., M=2, and proposes an efficient two-stage underdetermined BSS (UBSS) algorithm to estimate the mixing matrix and achieve source recovery by exploiting time–frequency (TF) sparsity. First, we design a mixing matrix estimation method by precisely identifying high clustering property single-source TF points (HCP-SSPs) with a spatial vector dictionary based on the principle of matching pursuit (MP). Second, the problem of source recovery in the TF domain is reformulated as an equivalent sparse recovery model with a relaxed sparse condition, i.e., enabling the number of active sources at each auto-source TF point (ASP) to be larger than M. This sparse recovery model relies on the sparsity of an ASP matrix formed by stacking a set of predefined spatial TF vectors; current sparse recovery tools could be utilized to reconstruct N>2 sources. Experimental results are provided to demonstrate the effectiveness of the proposed UBSS algorithm with an easily configured two-sensor array.

Robustly Train Normalizing Flows via KL Divergence Regularization

In this paper, we find that the training of Normalizing Flows (NFs) are easily affected by the outliers and a small number (or high dimensionality) of training samples. To solve this problem, we propose a Kullback–Leibler (KL) divergence regularization on the Jacobian matrix of NFs. We prove that such regularization is equivalent to adding a set of samples whose covariance matrix is the identity matrix to the training set. Thus, it reduces the negative influence of the outliers and the small sample number on the estimation of the covariance matrix, simultaneously. Therefore, our regularization makes the training of NFs robust. Ultimately, we evaluate the performance of NFs on out-of-distribution (OoD) detection tasks. The excellent results obtained demonstrate the effectiveness of the proposed regularization term. For example, with the help of the proposed regularization, the OoD detection score increases at most 30% compared with the one without the regularization.

Underdetermined blind source separation based on third‐order cumulant and tensor compression

AbstractA method for Underdetermined Blind Source Separation is proposed using third‐order cumulants and tensor compression. To effectively suppress symmetrical distributed noise, the third‐order cumulant is considered. Additionally, the complexity of high‐dimensional tensors can be reduced through high order singular value decomposition (HOSVD) for compression purposes. The method begins by calculating the third‐order cumulant tensor for whitening signals at different time delays, and then stacks several cumulants into a fourth‐order tensor. The HOSVD decomposition is applied to the fourth‐order tensor, compressing the high‐dimensional tensor into a low‐dimensional core tensor. Next, the core tensor is further decomposed using the canonical polyadic decomposition, and the resulting factor matrices are fused to obtain an estimation of the mixed matrix. Finally, leveraging the signal independence, a matrix diagonalisation method is employed to recover the source signals. Theoretical analysis and simulation results demonstrate that the proposed method effectively suppresses the influence of Gaussian noise, reduces computational complexity, and saves computational time. Moreover, compared with five representative approaches, the proposed method achieves superior separation results. Specifically, for the 3 × 4 mixed model with a signal‐to‐noise ratio of 20 dB, the average relative error of speech signal and radio signal are −11.02 and −6.8 dB respectively.

A joint estimation algorithm for single-input multiple-output underwater acoustic communications

In single-input multiple-output (SIMO) underwater acoustic (UWA) communications, the receiver based on passive time reversal (PTR) combined with decision feedback equalizer (DFE) is widely used but has a limited performance. A multi-channel joint estimation algorithm based on sparse Bayesian learning (MJSBL) is proposed in this paper to exploit the diverse gain from multi-channels, where reasonable prior distribution functions are selected for parameters in the probabilistic model. Afterwards, the algorithm is derived by the mean-field variational inference (VI), iteratively updating the estimation of symbols, channels and noise variation. As a result, the maximum likelihood estimation of the dictionary matrix, as well as the maximum posterior estimation of the channel vectors and noise variance, can be approximated. Simulation and experimental results verify that compared to typical single-channel and multi-channel algorithms, the systems always have lower bit error rates (BERs) and symbol error rates (SERs) with the MJSBL algorithm for different communications distances and symbol block lengths.

Functional principal component analysis for incomplete space–time data

Environmental signals, acquired, e.g., by remote sensing, often present large gaps of missing observations in space and time. In this work, we present an innovative approach to identify the main variability patterns, in space–time data, when data may be affected by complex missing data structures. We formalize the problem in the framework of functional data analysis, proposing an innovative method of functional principal component analysis (fPCA) for incomplete space–time data. The functional nature of the proposed method permits to borrow information from measurements observed at nearby spatio-temporal locations. The resulting functional principal components are smooth fields over the considered spatio-temporal domain, and can lead to interesting insights in the spatio-temporal dynamic of the phenomenon under study. Moreover, they can be used to provide a reconstruction of the missing entries, also under severe missing data patterns. The proposed model combines a weighted rank-one approximation of the data matrix with a roughness penalty. We show that the estimation problem can be solved using a majorize–minimization approach, and provide a numerically efficient algorithm for its solution. Thanks to a discretization based on finite elements in space and B-splines in time, the proposed method can handle multidimensional spatial domains with complex shapes, such as water bodies with complicated shorelines, or curved spatial regions with complex orography. As shown by simulation studies, the proposed space–time fPCA is superior to alternative techniques for Principal Component Analysis with missing data. We further highlight the potentiality of the proposed method for environmental problems, by applying space–time fPCA to the study of the lake water surface temperature (LWST) of Lake Victoria, in Central Africa, starting from satellite measurements with large gaps. LWST is considered one of the fundamental indicators of how climate change is affecting the environment, and is recognized as an essential climate variable.

TDOA-Based Indoor Localization via Linear Fusion With Low-Rank Matrix Approximation

TDOA-Based Indoor Localization via Linear Fusion With Low-Rank Matrix Approximation

Covariance Matrix Estimation for High-Throughput Biomedical Data with Interconnected Communities

Estimating a covariance matrix is central to high-dimensional data analysis. Empirical analyses of high-dimensional biomedical data, including genomics, proteomics, microbiome, and neuroimaging, among others, consistently reveal strong modularity in the dependence patterns. In these analyses, intercorrelated high-dimensional biomedical features often form communities or modules that can be interconnected with others. While the interconnected community structure has been extensively studied in biomedical research (e.g., gene co-expression networks), its potential to assist in the estimation of covariance matrices remains largely unexplored. To address this gap, we propose a procedure that leverages the commonly observed interconnected community structure in high-dimensional biomedical data to estimate large covariance and precision matrices. We derive the uniformly minimum variance unbiased estimators for covariance and precision matrices in closed forms and provide theoretical results on their asymptotic properties. Our proposed method enhances the accuracy of covariance- and precision-matrix estimation and demonstrates superior performance compared to the competing methods in both simulations and real data analyses.

Dynamic Realized Minimum Variance Portfolio Models

This article introduces a dynamic minimum variance portfolio (MVP) model using nonlinear volatility dynamic models, based on high-frequency financial data. Specifically, we impose an autoregressive dynamic structure on MVP processes, which helps capture the MVP dynamics directly. To evaluate the dynamic MVP model, we estimate the inverse volatility matrix using the constrained l 1 -minimization for inverse matrix estimation (CLIME) and calculate daily realized non-normalized MVP weights. Based on the realized non-normalized MVP weight estimator, we propose the dynamic MVP model, which we call the dynamic realized minimum variance portfolio (DR-MVP) model. To estimate a large number of parameters, we employ the least absolute shrinkage and selection operator (LASSO) and predict the future MVP and establish its asymptotic properties. Using high-frequency trading data, we apply the proposed method to MVP prediction.

Ambient data-driven participation factors related to oscillation modes based on subspace dynamic mode decomposition

Analyzing the connection between the variables participating in the oscillation and modes in electromechanical oscillation is particularly important for maintaining system stability. This paper proposes an ambient data-driven method based on Subspace Dynamic Mode Decomposition (Sub-DMD) to extract the Participation Factors (PFs) related to system state and algebraic variables in electromechanical oscillation modes. This method uses ambient data to calculate the low-dimensional approximate matrix of the system, and the PFs is extracted by using eigen-decomposition and mode energy. Compare the extracted PFs with the results of power generation dispatch. The effectiveness of the proposed method is demonstrated by using simulation data from IEEE 4-generator 2-area test system and IEEE 16-generator 68-bus test system.

Stability tests for neutral‐type delay systems: A delay Lyapunov matrix and piecewise linear approximation of the functional argument approach

AbstractNew necessary and sufficient stability tests for neutral‐type linear time delay systems are presented. They are based on functionals with prescribed derivative depending on the delay Lyapunov matrix and recent stability/instability theorems. A piecewise linear approximation of the initial function argument in the functional is employed. As a result, the functional is approximated via a quadratic form whose matrix provides stability criteria. This matrix involves the integrals based on the delay Lyapunov matrix which are handled via an efficient computational procedure. The main feature of these tests is that they are verified in a finite number of operations, which is achieved through rigorous bounding of the approximation error on a specific class of functions. Some examples which validate the obtained results are presented and discussed.

Bayesian estimation of cluster covariance matrices of unknown form

We develop a flexible Bayesian model for cluster covariance matrices in large dimensions where the number of clusters and the assignment of cross-sectional units to a cluster are a-priori unknown and estimated from the data. In a cluster covariance matrix, the variances and covariances are equal within each diagonal block, while the covariances are equal in each off-diagonal block. This reduces the number of parameters by pooling those parameters together that are in the same cluster. In order to treat the number of clusters and the cluster assignments as unknowns, we build a random partition model which assigns a prior distribution over the space of partitions of the data into clusters. Sampling from the posterior over the space of partitions creates a flexible estimator because it averages across a wide set of cluster covariance matrices. We illustrate our methods on linear factor models and large vector autoregressions.

Real-Time Speed Estimation for an Induction Motor: An Automated Tuning of an Extended Kalman Filter Using Voltage-Current Sensors.

This paper aims at achieving real-time optimal speed estimation for an induction motor using the Extended Kalman filter (EKF). Speed estimation is essential for fault diagnosis in Motor Current Signature Analysis (MCSA). The estimation accuracy is obtained by exploring the noise covariance matrices estimation of the EKF algorithm. The noise covariance matrices are determined using a modified subspace model identification approach. In order to reach this goal, this method compares an estimated model of a deterministic system, derived from available input-output datasets (using voltage-current sensors), with the discrete-time state-space representation used in the Kalman filter equations. This comparison leads to the determination of model uncertainties, which are subsequently represented as noise covariance matrices. Based on the fifth-order nonlinear model of the induction motor, the rotor speed is estimated with the optimized EKF algorithm, and the algorithm is tested experimentally.

Comparing multi-class classifier performance by multi-class ROC analysis: A nonparametric approach

The area under the Receiver Operating Characteristic (ROC) curve (AUC) is a standard metric for quantifying and comparing binary classifiers. Real world applications often require classification into multiple (more than two) classes. For multi-class classifiers that produce class membership scores, a popular multi-class AUC (MAUC) variant is to average the pairwise AUC values (Hand and Till, 2001). Due to the complicated correlation patterns, the variance of MAUC is often estimated numerically using resampling techniques. This work is a generalization of DeLong’s nonparameteric approach for binary AUC analysis (DeLong et al., 1988) to MAUC. We first derive the closed-form expression of the covariance matrix of the pairwise AUCs within a single MAUC. Then by dropping higher order terms, we obtain an approximate covariance matrix with a compact, matrix factorization form, which then serves as the basis for variance estimation of a single MAUC. We further extend this approach to estimate the covariance of correlated MAUCs that arise from multiple competing classifiers. For the special case of binary correlated AUCs, our results coincide with that of DeLong. Our numerical studies confirm the accuracy of the variance and covariance estimates. We provide the source code of the proposed covariance estimation of correlated MAUCs on GitHub (https://tinyurl.com/euj6wvsz) for its easy adoption by machine learning and statistical analysis packages to quantify and compare multi-class classifiers.