Rank Approximation Research Articles

GEVD Based on Multichannel Wiener Filter for Removal of EEG Artifacts

The electroencephalography (EEG) signals are contaminated by ocular artifacts usually called as ElectroOculoGraphy(EOG) artifacts. This occurs due to an eye movement and repeatedly blinking eyes, it is a major barrier to overcome when analyzing ElectroEncephaloGram (EEG) data. In this paper, Generalized Eigen Value Decomposition (GEVD) algorithm based on Multichannel Wiener filter (MWF) was proposed. In the GEVD algorithm, the covariance matrix of the artifact is identified and substituted by low rank approximation. For both real and hybrid EEG data is demonstrated using this algorithm and also compared with other existing methods for removal of artifacts. This paper determines generic, robust and fast algorithm for artifact removal of various types of EEG signals. Signal to Error Ratio (SER) and Artifact to Residue Ratio (ARR) both are expressed in dBs. The better performance of artifact removal is expressed with high SER which measures clean EEG distortion and ARR measures the artifact estimation.

Convergence analysis of LSQR for compact operator equations

In this paper we analyze the behavior of the LSQR algorithm for the solution of compact operator equations in Hilbert spaces. We present results concerning existence of Krylov solutions and the rate of convergence in terms of an ℓp sequence where p depends on the summability of the singular values of the operator. Under stronger regularity requirements we also consider the decay of the error. Finally we study the approximation of the dominant singular values of the operator attainable with the bidiagonal matrices generated by the Lanczos bidiagonalization and the arising low rank approximations. Some numerical experiments on classical test problems are presented.

Modified Low Rank Approximation for Detection of Weak Target by Noise space Exploitation in Through Wall Imaging

Low dielectric materials referred as weak targets are very difficult to detect behind the wall in through wall imaging (TWI) due to strong reflections from wall. TWI Experimental data collected for low dielectric target behind the wall and transceiver on another side of the wall. Recently several researchers are using low-rank approximation (LRA) for reduction of random noise in the various data. Explore the possibilities of using LRA for TWI data for improving the detection of low dielectric material. A novel approach using modification of LRA with exploiting the noise subspace in singular value decomposition (SVD) to detect weak target behind the wall is introduced. LRA consider data has low rank in f-x domain for noisy data, local windows are implemented in LRA approach to satisfy the principle assumptions required by the LRA algorithm itself. Decomposed TWI data in the noise space of the SVD to detect the weak target adaptively. Results for modified LRA for detection of weak target behind the wall are very encouraging over LRA.

Even-order Toeplitz tensor: framework for multidimensional structured linear systems

In this paper, we introduce tensors with Toeplitz structure. These structured tensors occur in different kinds of applications such as discretization of multidimensional PDE’s or Fredholm integral equations with an invariant kernel. We investigate the main properties of the new structured tensor and show the tensor contractive product with such tenors can be carried out with the fast Fourier transform. Also, we show that approximation of Toeplitz tensors with a specially structured tensor (that will be named $$\circledast $$-product tensors) can be reduced to the rank-1 approximation of a smaller tensor. Tensor equations with such $$\circledast $$-product coefficient tenors can be solved by a direct method. So, this approximation of a Toeplitz tensor can be used to find an approximate solution of the original tensor equation or can be used as a preconditioner. Our main goal is to show the ability of the tensor framework to handle structured multidimensional problems in their original format.

Non-orthogonal approximate joint diagonalization of non-Hermitian matrices in the least-squares sense

A non-orthogonal approximate joint diagonalization (AJD) algorithm of a set of non-Hermitian matrices is presented. Specifically, the proposed algorithm aims to find two distinct general (not necessarily orthogonal nor square) diagonalizing matrices which minimize the least-squares (LS) criterion based on the gradient and an optimal rank-1 approximation approach. It can be used to compute the canonical polyadic decomposition (CPD) of the third-order tensor. Simulation results demonstrate that the proposed algorithm has good convergence, robustness and accuracy properties. The joint blind source separation (JBSS) problem of two datasets can be effectively solved based on the proposed algorithm.

Despeckling of 3D ultrasound image using tensor low rank approximation

In this paper, a new method for despeckling of 3D ultrasound images based on tensor low rank approximation technique is presented. The main motive of this paper is to establish a new denoising method which can incorporate the redundancy present in 3D ultrasound image in all directions unlike the existing techniques. Hence, a substantial improvement in the overall performance can be achieved by preserving the edges and other finer details of the image. In order to achieve this goal, the 3D ultrasound image is modeled in terms of low rank tensor approximation of grouped tensors which is solved by minimizing the Tensor Nuclear Norm (TNN) in tensor-Singular Value Decomposition (t-SVD) framework. The resulting optimization problem is solved by the method of inexact Augmented Lagrangian Multipliers (ALM). To the best of our understanding, this is the first attempt to model the 3D ultrasound image despeckling problem as tensor low rank recovery technique. The proposed algorithm is compared with popular state-of-art works in literature including a low rank based approach termed as DLRA [1] designed for 2D ultrasound images, and it is found that our scheme offers superior performance than its counterparts.

Low-Rank Tensor Completion Based on Log-Det Rank Approximation and Matrix Factorization

Rank evaluation plays a key role in low-rank tensor completion and tensor nuclear norm is often used as a substitute of rank in the optimization due to its convex property. However, this replacement often incurs unexpected errors, and since singular value decomposition is frequently involved, the computation cost of the norm is high, especially in handling large scale matrices from the mode-n unfolding of a tensor. This paper presents a novel tensor completion method, in which a non-convex logDet function is utilized to approximate the rank and a matrix factorization is adopted to reduce the evaluation cost of the function. The study shows that the logDet function is a much tighter rank approximation than the nuclear norm and the matrix factorization can significantly reduce the size of matrix that needs to be evaluated. In the implementation of the method, alternating direction method of multipliers is employed to obtain the optimal tensor completion. Several experiments are carried out to validate the method and the results show that the proposed method is effective.

Computing lower rank approximations of matrix polynomials

Given an input matrix polynomial whose coefficients are floating point numbers, we consider the problem of finding the nearest matrix polynomial which has rank at most a specified value. This generalizes the problem of finding a nearest matrix polynomial that is algebraically singular with a prescribed lower bound on the dimension given in a previous paper by the authors. In this paper we prove that such lower rank matrices at minimal distance always exist, satisfy regularity conditions, and are all isolated and surrounded by a basin of attraction of non-minimal solutions. In addition, we present an iterative algorithm which, on given input sufficiently close to a rank-at-most matrix, produces that matrix. The algorithm is efficient and is proven to converge quadratically given a sufficiently good starting point. An implementation demonstrates the effectiveness and numerical robustness of our algorithm in practice.

Learning a Compact Vein Discrimination Model With GANerated Samples

Despite the great success achieved by convolutional neural networks (CNNs) in various image understanding tasks, it is still difficult for CNNs to be applied to vein recognition tasks due to the problems of insufficient training datasets, intra-class variations, and inter-class similarities. Besides, due to the essential requirement on the storage of millions of parameters for CNN, it is challenging to use a CNN for designing a vein-based embedded person identification system. In this paper, these two problems are addressed by learning a discriminative and compact vein recognition model. For the first problem, a hierarchical generative adversarial network (HGAN) consisting of a constrained CNN and a CycleGAN is proposed for data augmentation. Two similarity losses are defined for estimating the self-similarity and inter-class dissimilarity, and a CycleGAN model is properly trained with these two losses for better task-specific training sample generation. After obtaining a baseline vein recognition model fine-tuned on the augmented datasets, the existence of parameter redundancy in the over-parameterized network motivates the proposal of model compression by way of filter pruning and low rank approximation, thus making the compressed model more suitable for deployment on embedded systems. Through the vein recognition experiments with two different datasets and an additional palmprint recognition experiment, the proposed algorithms are shown to yield a highly compact model while keeping the accuracy acceptable for application.

Gradient nuclear norm minimization-based image filter

Designing an efficient filtering technique is an ill-posed problem especially for image affected from high density of noise. The majority of existing techniques suffer from edge degradation and texture distortion issues. Therefore, in this paper, an efficient weighted nuclear norm minimization (NNM)-based filtering technique to preserve the edges and texture information of filtered images is proposed. The proposed technique significantly improves the quantitative improvements on the low rank approximation of nonlocal self-similarity matrices to deal with the overshrink problem. Extensive experiments reveal that the proposed technique preserves edges and texture details of filtered image with lesser number of visual artifacts on visual quality. The proposed technique outperforms the existing techniques over the competitive filtering techniques in terms of structural similarity index metric (SSIM), peak signal-to-noise ratio (PSNR) and edge preservation index (EPI).

Nonconvex tensor rank minimization and its applications to tensor recovery

Low-rank tensor recovery (LRTR) has recently emerged as the potent tools for representing multidimensional data. One of the most popular LRTR is tensor rank minimization that acts as the estimating tensor rank for a given tensor. However, the existing convex tensor rank approximation methods suffer from the serious rank estimation bias due to neglecting the physical meanings of singular values along each mode. In this paper, we propose a new method to approximate the tensor rank by using the nonconvex logarithmic surrogate function of the singular values, and the redefined rank approximation can further reduce to a convex weighted nuclear norm minimization (WNNM) problem. By embedding the tensor rank function into the tensor completion (TC) and tensor robust PCA (TRPCA) frameworks, new models are formulated to enhance tensor processing. Additionally, by introducing relaxation forms of the proposed tensor rank function, the alternating direction method of multipliers (ADMM) can be adopted for the models. The proposed nonconvex tensor rank minimization method can achieve state-of-the-art performance in tensor recovery, including tensor completion and background subtraction.

A Novel Rank Approximation Method for Mixture Noise Removal of Hyperspectral Images

Mixture noise removal is a fundamental problem in hyperspectral images’ (HSIs) processing that holds significant practical importance for subsequent applications. This problem can be recast as an approximation issue of a low-rank matrix. In this paper, a novel smooth rank approximation (SRA) model is proposed to cope with these mixture noises for HSIs. The crux idea is to devise a general smooth function under some assumptions to directly approximate the rank function, which attempts to explore a closer approximation than conventional methods. This new optimization model can be easily solved by the convex analysis tool and can remove the mixture noises of HSIs quickly and effectively. Subsequently, we give a feasible iterative algorithm, and the corresponding convergence analysis is discussed mathematically. Experimental results from the simulated data set as well as real data sets illustrate that the proposed SRA method significantly outperforms the state-of-the-art methods on HSI denoising.

UWB Radar for Non-contact Heart Rate Variability Monitoring and Mental State Classification.

Heart rate variability (HRV), as measured by ultra-wideband (UWB) radar, enables contactless monitoring of physiological functioning in the human body. In the current study, we verified the reliability of HRV extraction from radar data, under limited transmitter power. In addition, we conducted a feasibility study of mental state classification from HRV data, measured using radar. Specifically, arctangent demodulation with calibration and low rank approximation have been used for radar signal pre-processing. An adaptive continuous wavelet filter and moving average filter were utilized for HRV extraction. For the mental state classification task, performance of support vector machine, k-nearest neighbors and random forest classifiers have been compared. The developed system has been validated on human participants, with 10 participants for HRV extraction, and three participants for the proof-of-concept mental state classification study. The results of HRV extraction demonstrate the reliability of time-domain parameter extraction from radar data. However, frequency-domain HRV parameters proved to be unreliable under low SNR. The best average overall mental state classification accuracy achieved was 82.34%, which has important implications for the feasibility of mental health monitoring using UWB radar.

Unbiased predictive risk estimation of the Tikhonov regularization parameter: convergence with increasing rank approximations of the singular value decomposition

The truncated singular value decomposition may be used to find the solution of linear discrete ill-posed problems in conjunction with Tikhonov regularization and requires the estimation of a regularization parameter that balances between the sizes of the fit to data function and the regularization term. The unbiased predictive risk estimator is one suggested method for finding the regularization parameter when the noise in the measurements is normally distributed with known variance. In this paper we provide an algorithm using the unbiased predictive risk estimator that automatically finds both the regularization parameter and the number of terms to use from the singular value decomposition. Underlying the algorithm is a new result that proves that the regularization parameter converges with the number of terms from the singular value decomposition. For the analysis it is sufficient to assume that the discrete Picard condition is satisfied for exact data and that noise completely contaminates the measured data coefficients for a sufficiently large number of terms, dependent on both the noise level and the degree of ill-posedness of the system. A lower bound for the regularization parameter is provided leading to a computationally efficient algorithm. Supporting results are compared with those obtained using the method of generalized cross validation. Simulations for two-dimensional examples verify the theoretical analysis and the effectiveness of the algorithm for increasing noise levels, and demonstrate that the relative reconstruction errors obtained using the truncated singular value decomposition are less than those obtained using the singular value decomposition. This is a pre-print of an article published in BIT Numerical Mathematics. The final authenticated version is available online at: https://doi.org/10.1007%2Fs10543-019-00762-7.

Sparse low rank approximation of potential energy surfaces with applications in estimation of anharmonic zero point energies and frequencies

We propose a method that exploits sparse representation of potential energy surfaces (PES) on a polynomial basis set selected by compressed sensing. The method is useful for studies involving large numbers of PES evaluations, such as the search for local minima, transition states, or integration. We apply this method for estimating zero point energies and frequencies of molecules using a three step approach. In the first step, we interpret the PES as a sparse tensor on polynomial basis and determine its entries by a compressed sensing based algorithm using only a few PES evaluations. Then, we implement a rank reduction strategy to compress this tensor in a suitable low-rank canonical tensor format using standard tensor compression tools. This allows representing a high dimensional PES as a small sum of products of one dimensional functions. Finally, a low dimensional Gauss–Hermite quadrature rule is used to integrate the product of sparse canonical low-rank representation of PES and Green’s function in the second-order diagrammatic vibrational many-body Green’s function theory (XVH2) for estimation of zero-point energies and frequencies. Numerical tests on molecules considered in this work suggest a more efficient scaling of computational cost with molecular size as compared to other methods.

A Survey of Related Research on Compression and Acceleration of Deep Neural Networks

Deep networks have achieved great success in many areas in recent years. However, with the increasing sophistication of deep neural networks (DNNs), the memory consumption and computational cost expand exponentially, greatly hindering their application in mobile devices and other limited resources. Therefore, there is impending necessity to consider model compression and acceleration without affecting the inference accuracy. In this paper, we review the recent popular techniques for compressing and accelerating deep networks. Those methods could be broadly divided into four categories: parameter pruning and sharing, low rank approximation, sparse regularization constraints and network weight low-bit quantization. The advantages and disadvantages of different compression and acceleration methods are also described in detail, other types of approach are also introduced in our paper, and future prospects for the field are given finally.

Operator Scaling: Theory and Applications

In this paper, we present a deterministic polynomial time algorithm for testing whether a symbolic matrix in non-commuting variables over $${\mathbb {Q}}$$ is invertible or not. The analogous question for commuting variables is the celebrated polynomial identity testing (PIT) for symbolic determinants. In contrast to the commutative case, which has an efficient probabilistic algorithm, the best previous algorithm for the non-commutative setting required exponential time (Ivanyos et al. in Comput Complex 26(3):717–763, 2017) (whether or not randomization is allowed). The algorithm efficiently solves the “word problem” for the free skew field, and the identity testing problem for arithmetic formulae with division over non-commuting variables, two problems which had only exponential time algorithms prior to this work. The main contribution of this paper is a complexity analysis of an existing algorithm due to Gurvits (J Comput Syst Sci 69(3):448–484, 2004), who proved it was polynomial time for certain classes of inputs. We prove it always runs in polynomial time. The main component of our analysis is a simple (given the necessary known tools) lower bound on central notion of capacity of operators (introduced by Gurvits 2004). We extend the algorithm to actually approximate capacity to any accuracy in polynomial time, and use this analysis to give quantitative bounds on the continuity of capacity (the latter is used in a subsequent paper on Brascamp–Lieb inequalities). We also extend the algorithm to compute not only singularity, but actually the (non-commutative) rank of a symbolic matrix, yielding a factor 2 approximation of the commutative rank. This naturally raises a relaxation of the commutative PIT problem to achieving better deterministic approximation of the commutative rank. Symbolic matrices in non-commuting variables, and the related structural and algorithmic questions, have a remarkable number of diverse origins and motivations. They arise independently in (commutative) invariant theory and representation theory, linear algebra, optimization, linear system theory, quantum information theory, approximation of the permanent and naturally in non-commutative algebra. We provide a detailed account of some of these sources and their interconnections. In particular, we explain how some of these sources played an important role in the development of Gurvits’ algorithm and in our analysis of it here.

Fast kernel extreme learning machine for ordinal regression

Ordinal regression is a special kind of machine learning problem, which aims to label patterns with an ordinal scale. Due to the ubiquitous existence of the ordering information in many practical cases, ordinal regression has received much attention and can be found in a great variety of applications. Meanwhile, Kernel Extreme Learning Machine (KELM), as the extension of Extreme Learning Machine in the framework of kernel learning, has shown its strength in many machine learning tasks. Nevertheless, existing KELM methods have paid little attention to ordinal regression especially in large-scale situations. In this paper, we consider the kernel technique and ordinal scales in labels at the same time, and propose a new KELM model for ordinal regression by exploiting a quadratic cost-sensitive encoding scheme. In order to make the training process more efficient in the big data scenario, a fast algorithm is designed based on the low rank approximation. The incomplete Cholesky factorization and the Sherman–Morrison–Woodbury formula are utilized together to avoid computing the inverse of the kernel matrix. The time complexity of the new algorithm is thus linear with the number of the training instances, which makes it suitable to the large-scale occasions. Numerical experiments on multiple public datasets validate the effectiveness and efficiency of the proposed methods.

Preliminary results of DSA denoising based on a weighted low-rank approach using an advanced neurovascular replication system.

2D digital subtraction angiography (DSA) has become an important technique for interventional neuroradiology tasks, such as detection and subsequent treatment of aneurysms. In order to provide high-quality DSA images, usually undiluted contrast agent and a high X-ray dose are used. The iodinated contrast agent puts a burden on the patients' kidneys while the use of high-dose X-rays expose both patients and medical staff to a considerable amount of radiation. Unfortunately, reducing either the X-ray dose or the contrast agent concentration usually results in a sacrifice of image quality. To denoise a frame, the proposed spatiotemporal denoising method utilizes the low-rank nature of a spatially aligned temporal sequence where variation is introduced by the flow of contrast agent through a vessel tree of interest. That is, a constrained weighted rank-1 approximation of the stack comprising the frame to be denoised and its temporal neighbors is computed where the weights are used to prevent the contribution of non-similar pixels toward the low-rank approximation. The method has been evaluated using a vascular flow phantom emulating cranial arteries into which contrast agent can be manually injected (Vascular Simulations Replicator, Vascular Simulations, Stony Brook NY, USA). For the evaluation, image sequences acquired at different dose levels as well as different contrast agent concentrations have been used. Qualitative and quantitative analyses have shown that with the proposed approach, the dose and the concentration of the contrast agent could both be reduced by about 75%, while maintaining the required image quality. Most importantly, it has been observed that the DSA images obtained using the proposed method have the closest resemblance to typical DSA images, i.e., they preserve the typical image characteristics best. Using the proposed denoising approach, it is possible to improve the image quality of low-dose DSA images. This improvement could enable both a reduction in contrast agent and radiation dose when acquiring DSA images, thereby benefiting patients as well as clinicians. Since the resulting images are free from artifacts and as the inherent characteristics of the images are also preserved, the proposed method seems to be well suited for clinical images as well.

Clustering Over Multiple Evolving Data Streams of the Traffic Cyber-Physical Systems

Extracting and retaining useful information from multiple data streams of the traffic Cyber-Physical Systems (CPS) generated by practical applications have attracted an increasing amount of attentions from related researchers. In this paper, the Incremental Clustering framework is proposed for multiple sensor data streams by low rank approximation Matrix Factorization (IC-MF) monitors the distribution of clusters over multiple sensor data streams based on their correlation. In the IC-MF, both the low-rank matrix approximation and matrix factorization-based clustering are applied, and IC-MF incorporates the historical results and the relationship between the nodes of the current step and previous step. To improve the accuracy of the increments, a low-rank approximation of the adjacency matrix is obtained at each time step, and makes IC-MF work directly in the low-rank subspace. The main idea of IC-MF is to make use of the similarity between two consecutive time steps to quickly update the approximating subspace. The performance and efficiency of the algorithm are demonstrated by traffic CPS experiments on the real and synthetics data sets. The results show that the proposed algorithm surpasses the existing methods for clustering multiple evolving data streams over time.