Low-rank Approximation Techniques Research Articles

Nonlinear acoustic echo cancellation using low-complexity low-rank recursive least-squares algorithms

Adaptive exponential functional link networks (AEFLN) are a type of linear-in-the-parameter nonlinear filters, which have shown enhanced modeling capability for nonlinear systems. However, the convergence speed of traditional AEFLN is generally slow for long impulse responses, hence, AEFLN trained using recursive least-squares becomes an attractive choice to achieve faster convergence. Moreover, huge computational burden of RLS makes it unsuitable in practical applications like echo cancellation. While low complexity versions of RLS are widely available in literature, they still suffer from low convergence speed. To address this issue, we propose a nonlinear acoustic echo cancellation (NAEC) system using AEFLN-RLS, based on the nearest Kronecker product (NKP) decomposition and low-rank approximation technique, which not only reduces computational complexity but also achieves improved convergence speed (especially tracking). To further improve the echo cancellation performance in non-stationary conditions, a variable regularization approach based NKP-AEFLN-RLS system is also proposed. To also reduce the computational complexity further, dichotomous coordinate descent (DCD) updates are incorporated into the proposed NKP-AEFLN-RLS NAEC system and its variable regularization version. Experimental results show the effectiveness of the proposed algorithms, with the variable regularized version of the algorithm using DCD iterations showing the best compromise between convergence capability and lower computational complexity.

From low-rank retractions to dynamical low-rank approximation and back

In algorithms for solving optimization problems constrained to a smooth manifold, retractions are a well-established tool to ensure that the iterates stay on the manifold. More recently, it has been demonstrated that retractions are a useful concept for other computational tasks on manifold as well, including interpolation tasks. In this work, we consider the application of retractions to the numerical integration of differential equations on fixed-rank matrix manifolds. This is closely related to dynamical low-rank approximation (DLRA) techniques. In fact, any retraction leads to a numerical integrator and, vice versa, certain DLRA techniques bear a direct relation with retractions. As an example for the latter, we introduce a new retraction, called KLS retraction, that is derived from the so-called unconventional integrator for DLRA. We also illustrate how retractions can be used to recover known DLRA techniques and to design new ones. In particular, this work introduces two novel numerical integration schemes that apply to differential equations on general manifolds: the accelerated forward Euler (AFE) method and the Projected Ralston–Hermite (PRH) method. Both methods build on retractions by using them as a tool for approximating curves on manifolds. The two methods are proven to have local truncation error of order three. Numerical experiments on classical DLRA examples highlight the advantages and shortcomings of these new methods.

A fast multipole boundary element method for acoustics in viscothermal fluids

Standard numerical models in acoustics rely on the isentropic Helmholtz equation. Its derivation assumes adiabatic and reversible, i.e., dissipation-free, wave propagation. Sound waves in fluids are, however, subject to viscous and thermal losses. These losses originate from viscous friction and heat conduction, leading to the formation of acoustic boundary layers. Considering these effects becomes significant in setups with acoustic cavities of similar dimension as the boundary layers. Recently, boundary element methods (BEM) accounting for the viscothermal dissipation have been proposed. These methods limit the discretization to the surface of the fluid domain and require significantly fewer degrees of freedom than comparable finite element models. However, the BEM coefficient matrices are fully populated, resulting in high computational costs and storage requirements. This study develops a new BEM formulation for acoustics in viscothermal fluids that uses the fast multipole method - a low-rank approximation technique based on a hierarchical subdivision of the computational domain - to alleviate this shortcoming. It is shown that the fast viscothermal BEM improves the algorithmic complexity over the conventional formulation, reducing both the memory requirements and solution time. The results indicate good scalability of the method making it feasible for larger applications such as acoustic metamaterials.

Low-rank compression techniques in integral methods for eddy currents problems

Volume integral methods for the solution of eddy current problems are very appealing in practice since only the conducting regions need to be meshed. However, they require the assembly and storage of a dense stiffness matrix. With the objective of cutting down assembly time and memory occupation, low-rank approximation techniques like the Adaptive Cross Approximation (ACA) have been considered a major breakthrough. Recently, the VINCO framework has been introduced to reduce significantly memory occupation and computational time thanks to a novel factorization of the dense stiffness matrix. The aim of this paper is introducing a new matrix compression technique enabled by the VINCO framework. We compare the performance of VINCO framework approaches with state-of-the-art alternatives in terms of memory occupation, computational time and accuracy by solving benchmark eddy current problems at increasing mesh sizes; the comparisons are carried out using both direct and iterative solvers. The results clearly indicate that the so-called VINCO-FAIME approach which exploits the Fast Multipole Method (FMM) has the best performance.

Computation of the self-diffusion coefficient with low-rank tensor methods: application to the simulation of a cross-diffusion system

Cross-diffusion systems arise as hydrodynamic limits of lattice multi-species interacting particle models. The objective of this work is to provide a numerical scheme for the simulation of the cross-diffusion system identified in [J. Quastel, Comm. Pure Appl. Math., 45 (1992), pp. 623–679]. To simulate this system, it is necessary to provide an approximation of the so-called self-diffusion coefficient matrix of the tagged particle process. Classical algorithms for the computation of this matrix are based on the estimation of the long-time limit of the average mean square displacement of the particle. In this work, as an alternative, we propose a novel approach for computing the self-diffusion coefficient using deterministic low-rank approximation techniques, as the minimum of a high-dimensional optimization problem. The computed self-diffusion coefficient is then used for the simulation of the cross-diffusion system using an implicit finite volume scheme.

An Improved Constrained LMS Algorithm for Fast Adaptive Beamforming Based on a Low Rank Approximation

Adaptive beamformers use data from sensor arrays to capture signal from a desired direction without any distortion, in the presence of interfering signals from other directions in a noisy environment. Most beamformers achieve this goal by minimizing their variance while applying distortionless and null constraints in the direction of the desired and interfering signals, respectively. Constrained least-mean-square (CLMS) algorithms have been developed to iteratively update the weights of such beamformers. In this brief, we propose a novel and improved CLMS beamforming algorithm based on a low rank approximation technique called the nearest Kronecker product decomposition. By decomposing the weight vector into a sequence of Kronecker products of smaller vectors, the original weight update process is converted into updates of smaller vectors. The decomposition allows us to control the trade-off between steady-state performance and faster convergence based on the rank of the beamforming system. We derive the update rules of the proposed algorithm, tabulate its computational complexity and perform simulation study to show its superiority.

Aggregation in non-uniform systems with advection and localized source

We explore analytically and numerically agglomeration driven by advection and localized source. The system is inhomogeneous in one dimension, viz along the direction of advection. It is characterized by the kinetic coefficients—the advection velocity, diffusion coefficient and the reaction kernel, quantifying the aggregation rates. We analyze a simplified model with mass-independent advection velocity, diffusion coefficient, and reaction rates. We also examine a model with mass-dependent coefficients arising in the context of aggregation with sedimentation. For the quasi-stationary case and simplified model, we obtain an exact solution for the spatially dependent agglomerate densities. For the case of mass-dependent coefficients we report a new conservation law and develop a scaling theory for the densities. For the numerical efficiency we exploit the low-rank approximation technique; this dramatically increases the computational speed and allows simulations of very large systems. The numerical results are in excellent agreement with the predictions of our theory.

Low-Rank Approximation of Frequency Response Analysis of Perforated Cylinders under Uncertainty

Frequency response analysis under uncertainty is computationally expensive. Low-rank approximation techniques can significantly reduce the solution times. Thin perforated cylinders, as with all shells, have specific features affecting the approximation error. There exists a rich thickness-dependent boundary layer structure, leading to local features becoming dominant as the thickness tends to zero. Related to boundary layers, there is also a connection between eigenmodes and the perforation patterns. The Krylov subspace approach for proportionally damped systems with uncertain Young’s modulus is compared with the full system, and via numerical experiments, it is shown that the relative accuracy of the low-rank approximation of perforated shells measured in energy depends on the dimensionless thickness. In the context of frequency response analysis, it then becomes possible that, at some critical thicknesses, the most energetic response within the observed frequency range is not identified correctly. The reference structure used in the experiments is a trommel screen with a non-regular perforation pattern with two different perforation zones. The low-rank approximation scheme is shown to be feasible in computational asymptotic analysis of trommel designs when the proportional damping model is used.

Probabilistic Power Flow Analysis Based on Partial Least Square and Arbitrary Polynomial Chaos Expansion

This paper presents a new algorithm based on the partial least square (PLS) techniques and the arbitrary polynomial chaos expansion (aPCE) for the probabilistic power flow (PPF) analysis of a power system having many uncertain variables. The proposed method uses the nonlinear PLS to transform a set of random input variables to a smaller number of de-correlated random variables. Then, the aPCE technique is applied to generate the basis polynomial functions and build a surrogate model of the power system response. The algorithm has been implemented and tested with the modified IEEE 118-bus and European 1354-bus systems. The numerical results indicate that, similar to the sparse PCE method and the low-rank approximation technique, the proposed method can be applied to high-dimensional PPF problems. Nonetheless, the proposed approach uses smaller computation time, and estimates statistical characteristics of the response with higher accuracy.

Rank-adaptive structure-preserving model order reduction of Hamiltonian systems

This work proposes an adaptive structure-preserving model order reduction method for finite-dimensional parametrized Hamiltonian systems modeling non-dissipative phenomena. To overcome the slowly decaying Kolmogorov width typical of transport problems, the full model is approximated on local reduced spaces that are adapted in time using dynamical low-rank approximation techniques. The reduced dynamics is prescribed by approximating the symplectic projection of the Hamiltonian vector field in the tangent space to the local reduced space. This ensures that the canonical symplectic structure of the Hamiltonian dynamics is preserved during the reduction. In addition, accurate approximations with low-rank reduced solutions are obtained by allowing the dimension of the reduced space to change during the time evolution. Whenever the quality of the reduced solution, assessed via an error indicator, is not satisfactory, the reduced basis is augmented in the parameter direction that is worst approximated by the current basis. Extensive numerical tests involving wave interactions, nonlinear transport problems, and the Vlasov equation demonstrate the superior stability properties and considerable runtime speedups of the proposed method as compared to global and traditional reduced basis approaches.

Denoising 3D magnetic resonance images based on weighted tensor nuclear norm minimization using balanced nonlocal patch tensors

The denoising of magnetic resonance (MR) images is important to improve the accuracy of organ tissue information recognition in the process of medical diagnosis. This paper proposes a 3D MR image denoising method based on weighted tensor nuclear norm minimization using balanced nonlocal patch tensors. In order to make utilization of the nonlocal self-similarity in high-dimensional images and the correlation among different dimensions, a high-dimensional adaptive clustering technology is developed to construct highly correlated 3D MR image patches into nonlocal patch tensors. Since the generated tensor exhibits low-rank characteristics, a low-rank tensor approximation technique based on the tensor singular value decomposition framework can be used to denoise the MR data. To improve the denoising accuracy, the local noise level information of different nonlocal patch tensors is considered. Besides, a method for constructing more balanced 3D nonlocal patch tensors is proposed to exploit the effectiveness of the tensor singular value decomposition. Experimental results show that the proposed method has a greater improvement over the existing MR image denoising methods in terms of objective measurement and subjective visual quality on simulated and real 3D MR images with different categories and noise levels.

Distributed adaptive Newton methods with global superlinear convergence

This paper considers the distributed optimization problem where each node of a peer-to-peer network minimizes a finite sum of objective functions by communicating with its neighboring nodes. In sharp contrast to the existing literature where the fastest distributed algorithms converge either with a global linear or a local superlinear rate, we propose a distributed adaptive Newton (DAN) algorithm with a global quadratic convergence rate. Our key idea lies in the design of a finite-time set-consensus method with Polyak’s adaptive stepsize. Moreover, we introduce a low-rank matrix approximation (LA) technique to compress the innovation of Hessian matrix so that each node only needs to transmit message of dimension O(p) (where p is the dimension of decision vectors) per iteration, which is essentially the same as that of first-order methods. Nevertheless, the resulting DAN-LA converges to an optimal solution with a global superlinear rate. Numerical experiments on logistic regression problems are conducted to validate their advantages over existing methods.

A comparison between current-based integral equations approaches for eddy current problems

In this paper, a comparison between two current-based Integral Equations approaches for eddy current problems is presented. In particular, the very well-known and widely adopted loop-current formulation (or electric vector potential formulation) is compared to the less common J-φ formulation. Pros and cons of the two formulations with respect to the problem size are discussed, as well as the adoption of low-rank approximation techniques. Although rarely considered in the literature, it is shown that the J-φ formulation may offer some useful advantages when large problems are considered. Indeed, for large-scale problems, while the computational efforts required by the two formulations are comparable, the J-φ formulation does not require any particular attention when non-simply connected domains are considered.

Optimized cycle basis in volume integral formulations for large scale eddy-current problems

We present a Volume Integral formulation for the solution of large scale eddy-current problems coupled with low-rank approximation techniques. Two alternative approaches are introduced to map the problem unknowns into a subset of grid elements forming a base of global or mixed (global and local) cycles, respectively, and guarantee the well-posedness of the problem both in simply and multiply connected domains. The paper shows that the adoption of mixed cycles is computationally more efficient than global ones. In particular, integral formulations based on global cycles cannot be safely coupled with low-rank approximation techniques, which, however, are crucial to increase the size of the largest solvable problem, like the ones involving conducting structures in magnetic confinement fusion devices. The aim of this paper is to demonstrate how such bottleneck can be overcome by considering local and global cycles differently, on the basis of the cohomology theory. An improved, efficient, and robust algorithm for computing a base of global cycles is described in detail. In particular, the presented algorithm is able to almost minimize the cohomology basis length, i.e. the number of mesh edges forming such a basis, in order to allow an efficient solution of large scale problems. Furthermore, a novel and general method to handle global and local cycles together, in the context of low-rank approximated matrices, is shown to be efficient for the solution of large scale eddy-current problems in multiply connected domains. Along the manuscript, pseudo-codes are given, which clarify the proposed methods and help to implement them by Volume Integral Equation practitioners.

High Performance Multivariate Geospatial Statistics on Manycore Systems

Modeling and inferring spatial relationships and predicting missing values of environmental data are some of the main tasks of geospatial statisticians. These routine tasks are accomplished using multivariate geospatial models and the cokriging technique. The latter requires the evaluation of the expensive Gaussian log-likelihood function, which has impeded the adoption of multivariate geospatial models for large multivariate spatial datasets. However, this large-scale cokriging challenge provides a fertile ground for supercomputing implementations for the geospatial statistics community as it is paramount to scale computational capability to match the growth in environmental data coming from the widespread use of different data collection technologies. In this article, we develop and deploy large-scale multivariate spatial modeling and inference on parallel hardware architectures. To tackle the increasing complexity in matrix operations and the massive concurrency in parallel systems, we leverage low-rank matrix approximation techniques with task-based programming models and schedule the asynchronous computational tasks using a dynamic runtime system. The proposed framework provides both the dense and the approximated computations of the Gaussian log-likelihood function. It demonstrates accuracy robustness and performance scalability on a variety of computer systems. Using both synthetic and real datasets, the low-rank matrix approximation shows better performance compared to exact computation, while preserving the application requirements in both parameter estimation and prediction accuracy. We also propose a novel algorithm to assess the prediction accuracy after the online parameter estimation. The algorithm quantifies prediction performance and provides a benchmark for measuring the efficiency and accuracy of several approximation techniques in multivariate spatial modeling.

Numerical Study of Low Rank Approximation Methods for Mechanics Data and Its Analysis

This paper proposes a comparison of the numerical aspect and efficiency of several low rank approximation techniques for multidimensional data, namely CPD, HOSVD, TT-SVD, RPOD, QTT-SVD and HT. This approach is different from the numerous papers that compare the theoretical aspects of these methods or propose efficient implementation of a single technique. Here, after a brief presentation of the studied methods, they are tested in practical conditions in order to draw hindsight at which one should be preferred. Synthetic data provides sufficient evidence for dismissing CPD, T-HOSVD and RPOD. Then, three examples from mechanics provide data for realistic application of TT-SVD and ST-HOSVD. The obtained low rank approximation provides different levels of compression and accuracy depending on how separable the data is. In all cases, the data layout has significant influence on the analysis of modes and computing time while remaining similarly efficient at compressing information. Both methods provide satisfactory compression, from 0.1% to 20% of the original size within a few percent error in $$L^2$$ norm. ST-HOSVD provides an orthonormal basis while TT-SVD doesn’t. QTT is performing well only when one dimension is very large. A final experiment is applied to an order 7 tensor with $$(4 \times 8\times 8\times 64\times 64\times 64\times 64)$$ entries (32 GB) from complex multi-physics experiment. In that case, only HT provides actual compression (50%) due to the low separability of this data. However, it is better suited for higher order d. Finally, these numerical tests have been performed with pydecomp , an open source python library developed by the author.

Tensor based framework for Distributed Denial of Service attack detection

Distributed Denial of Service (DDoS) attacks are one of the most important security threats, since multiple compromised systems perform massive attacks over a victim, overwhelming its bandwidth and/or resources. Such attacks can be detected, for example, by using supervised machine learning based solutions previously trained on large DDoS attack datasets in order to automatically identify malicious patterns present in the incoming traffic. In addition, since large datasets show inherent multidimensional structures, tensor based detection techniques can outperform the matrix based counterparts. In this context, the development of a DDoS attack detection framework which exploits both machine learning and tensor based approaches is crucial. To face this challenge, this paper proposes a novel tensor based framework for DDoS attack detection using concepts of multiple denoising, tensor decomposition and machine learning supervised classification. Moreover, we also propose an extension of the recent Multiple Denoising algorithm such that the noise present in the dataset instances is more efficiently attenuated. Finally, we validate the effectiveness of our proposed framework through comparison with state-of-the-art low-rank approximation techniques as well as with related works. The proposed approach outperforms its competitor schemes in terms of accuracy, detection rate and false alarm rate.

Error-Robust Distributed Denial of Service Attack Detection Based on an Average Common Feature Extraction Technique.

In recent years, advanced threats against Cyber–Physical Systems (CPSs), such as Distributed Denial of Service (DDoS) attacks, are increasing. Furthermore, traditional machine learning-based intrusion detection systems (IDSs) often fail to efficiently detect such attacks when corrupted datasets are used for IDS training. To face these challenges, this paper proposes a novel error-robust multidimensional technique for DDoS attack detection. By applying the well-known Higher Order Singular Value Decomposition (HOSVD), initially, the average value of the common features among instances is filtered out from the dataset. Next, the filtered data are forwarded to machine learning classification algorithms in which traffic information is classified as a legitimate or a DDoS attack. In terms of results, the proposed scheme outperforms traditional low-rank approximation techniques, presenting an accuracy of , detection rate of and false alarm rate of for a dataset corruption level of with a random forest algorithm applied for classification. In addition, for error-free conditions, it is found that the proposed approach outperforms other related works, showing accuracy, detection rate and false alarm rate of , and , respectively, for the gradient boosting classifier.

Skin effect and proximity effect analysis of stranded conductor based on mixed order MoM with adaptive cross approximation algorithm

In this paper, an efficient adaptive cross approximation (ACA) method is employed for the skin effect and proximity effect analysis. Multi-layer stranded cables with multiple strands and different materials are used for analysis to approximate the effect. The skin and proximity matrices are computed by the ACA low-rank matrix approximation technique. The ACA method can provide better solutions through different iterations. The ACA algorithm is very stable and useful for calculating a low rank matrix approximation, because the adaptive mesh refinement for generating optimal computation. The ACA approach provides the less computational memory and less CPU time. Compare the calculated results with standard work results (such as reference) to verify the effectiveness of the adaptive cross-approximation algorithm.

A Novel Enriched Version of Truncated Nuclear Norm Regularization for Matrix Completion of Inexact Observed Data

In recent years, matrix completion has become one of the main concepts in data science. Truncated nuclear norm regularization (TNNR) approximation of the rank function is an example of the favorite approaches in matrix completion that performs better than other approximations like the nuclear norm. In all TNNR based methods, the observed data is considered to be exact, so they do not give an appropriate solution for inexact observed data. But, in real applications, in addition to missing data, observed data may be inaccurate. In this paper, we proposed a novel method based on TNNR that can deal with missing data and inexact observed data by adding some terms in the objective and constraint of the matrix completion problem. Experimental results confirm the quality of our proposed method in comparison with the other state of the art matrix completion and low-rank approximation techniques.