Approximate Singular Value Decomposition Research Articles

An approximate tensor singular value decomposition approach for the fast grouping of whole-body human poses

We propose a fast method to group similar human poses from single-view depth maps using an approximate singular value decomposition approach in the tensor domain. To this end, the input tensor is decomposed, and a representation tensor is learned using the tensor-QR decomposition and ℓ2,1 norm minimization. The spatial structure and, thereby, the spatial information in each depth map, vital in performing accurate grouping of human poses, is preserved by adopting the 3D tensor approach in treating the data. For the seamless integration of discriminatory information of each pose, a new dissimilarity measure based on sub-tensors is devised and integrated into the optimization problem. Experimental analysis showcases the ability of the proposed method to achieve 10%–15% improvement in the grouping results compared with the state-of-the-art counterparts. The proposed algorithm also converges in less than ten iterations on average, thereby improving CPU time.

Subspace screening rule for multi-label estimator with sparsity-inducing regularization

Multi-label learning has drawn wide attention for the last decade. To exploit the correlation between labels, a multi-label classifier based on the nuclear norm has been proposed recently, which joints Ranking support vector machine (RankSVM) and Binary Relevance (BR) with robust Low-rank learning (RBRL). Therefore, it has satisfactory classification outcomes in the application. Nonetheless, tackling the large-scale problem still remains a challenge for RBRL. Motivated by this, a Subspace Screening Rule (SSR) for RBRL is proposed to accelerate the solving process. Its primary strategy is to reduce the size of the matrix variable to be estimated, based on the fact that a low-rank matrix can be represented by a few subspaces. Specifically, at each iteration we delete a majority of subspaces with zero coefficients in the optimal solution by matrix decomposition and optimality condition. Then, we solve the small-scale reduced problem rather than the initial large-scale matrix problem. An excellent acceleration effect is obtained. To further improve the solving speed, Approximate Singular Value Decomposition (ASVD) and Accelerated Proximal Gradient (APG) are employed in the different stages. Extensive experiments on seven benchmark datasets as well as an artificial dataset demonstrate the efficiency of SSR.

Debris Flow Prediction Based on the Fast Multiple Principal Component Extraction and Optimized Broad Learning

In the current research of debris flow geological disaster prediction, determining reasonable disaster-inducing factors and ensuring the accuracy and rapidity of the prediction model are considered vital issues, and also, essential foundations for disaster early warning and disaster prevention and mitigation. Aiming at the problems of low prediction accuracy and long prediction time in the current debris flow research, firstly, six debris flow impact factors were selected relying on the fast multiple principal component extraction (FMPCE) algorithm, including rainfall, slope gradient, gully bed gradient, relative height difference, soil moisture content and pore water pressure. Next, based on the broad learning (BL) algorithm, the debris flow prediction model based on FMPCE and the optimized BL is established with the input of debris flow-inducing factors and the output of debris flow probability. Then the model is optimized using matrix stochastic approximate singular value decomposition (SVD), and the debris flow disaster prediction model, based on SVDBL, is constructed. The prediction results of the optimized model are compared with those of the gradient descent optimized the BP neural network model(GD-BP), Support Vector Machines model(SVM) based on grid search and BL model. The results show that the accuracy of SVDBL is 7.5% higher than that of GD-BP, 3% higher than that of SVM and 0.5% higher than that of BL. The RMSE sum of SVDBL was 0.05870, 0.0478 and 0.0227 less than that of GD-BPSVM and BL, respectively; the MAPE sum of SVDBL was 1.95%, 1.66% and 0.49% less than that of GD-BPSVM and BL; the AUC values of SVDBL were 12.75%, 7.64% and 2.79% higher than those of the above three models, respectively. In addition, the input dataset is expanded to compare the training time of each model. The simulation results show that the prediction accuracy of this model is the highest and the training time is the shortest after the dataset is expanded. This study shows that the BL can be used for debris flow prediction, and can also provide references for disaster early warning and prevention.

ANALYTICAL SINGULAR VALUE DECOMPOSITION FOR A CLASS OF STOICHIOMETRY MATRICES.

We present the analytical singular value decomposition of the stoichiometry matrix for a spatially discrete reaction-diffusion system. The motivation for this work is to develop a matrix decomposition that can reveal hidden spatial flux patterns of chemical reactions. We consider a 1D domain with two subregions sharing a single common boundary. Each of the subregions is further partitioned into a finite number of compartments. Chemical reactions can occur within a compartment, whereas diffusion is represented as movement between adjacent compartments. Inspired by biology, we study both (1) the case where the reactions on each side of the boundary are different and only certain species diffuse across the boundary and (2) the case where reactions and diffusion are spatially homogeneous. We write the stoichiometry matrix for these two classes of systems using a Kronecker product formulation. For the first scenario, we apply linear perturbation theory to derive an approximate singular value decomposition in the limit as diffusion becomes much faster than reactions. For the second scenario, we derive an exact analytical singular value decomposition for all relative diffusion and reaction time scales. By writing the stoichiometry matrix using Kronecker products, we show that the singular vectors and values can also be written concisely using Kronecker products. Ultimately, we find that the singular value decomposition of the reaction-diffusion stoichiometry matrix depends on the singular value decompositions of smaller matrices. These smaller matrices represent modified versions of the reaction-only stoichiometry matrices and the analytically known diffusion-only stoichiometry matrix. Lastly, we present the singular value decomposition of the model for the Calvin cycle in cyanobacteria and demonstrate the accuracy of our formulation. The MATLAB code, available at www.github.com/MathBioCU/ReacDiffStoicSVD, provides routines for efficiently calculating the SVD for a given reaction network on a 1D spatial domain.

Algorithm 1022: Efficient Algorithms for Computing a Rank-Revealing UTV Factorization on Parallel Computing Architectures

Randomized singular value decomposition (RSVD) is by now a well-established technique for efficiently computing an approximate singular value decomposition of a matrix. Building on the ideas that underpin RSVD, the recently proposed algorithm “randUTV” computes a full factorization of a given matrix that provides low-rank approximations with near-optimal error. Because the bulk of randUTV is cast in terms of communication-efficient operations such as matrix-matrix multiplication and unpivoted QR factorizations, it is faster than competing rank-revealing factorization methods such as column-pivoted QR in most high-performance computational settings. In this article, optimized randUTV implementations are presented for both shared-memory and distributed-memory computing environments. For shared memory, randUTV is redesigned in terms of an algorithm-by-blocks that, together with a runtime task scheduler, eliminates bottlenecks from data synchronization points to achieve acceleration over the standard blocked algorithm based on a purely fork-join approach. The distributed-memory implementation is based on the ScaLAPACK library. The performance of our new codes compares favorably with competing factorizations available on both shared-memory and distributed-memory architectures.

Approximate Computing Circuits for Embedded Tactile Data Processing

In this paper, we demonstrate the feasibility and efficiency of approximate computing techniques (ACTs) in the embedded Support Vector Machine (SVM) tensorial kernel circuit implementation in tactile sensing systems. Improving the performance of the embedded SVM in terms of power, area, and delay can be achieved by implementing approximate multipliers in the SVD. Singular Value Decomposition (SVD) is the main computational bottleneck of the tensorial kernel approach; since digital multipliers are extensively used in SVD implementation, we aim to optimize the implementation of the multiplier circuit. We present the implementation of the approximate SVD circuit based on the Approximate Baugh-Wooley (Approx-BW) multiplier. The approximate SVD achieves an energy consumption reduction of up to 16% at the cost of a Mean Relative Error decrease (MRE) of less than 5%. We assess the impact of the approximate SVD on the accuracy of the classification; showing that approximate SVD increases the Error rate (Err) within a range of one to eight percent. Besides, we propose a hybrid evaluation test approach that consists of implementing three different approximate SVD circuits having different numbers of approximated Least Significant Bits (LSBs). The results show that energy consumption is reduced by more than five percent with the same accuracy loss.

An Iterative Coordinate Descent Algorithm to Compute Sparse Low-Rank Approximations

In this paper, we describe a new algorithm to build a few sparse principal components from a given data matrix. Our approach does not explicitly create the covariance matrix of the data and can be viewed as an extension of the Kogbetliantz algorithm to build an approximate singular value decomposition for a few principal components. We show the performance of the proposed algorithm to recover sparse principal components on various datasets from the literature and perform dimensionality reduction for classification applications.

Compacting the density matrix in quantum dynamics: Singular value decomposition of the surprisal and the dominant constraints for anharmonic systems.

We introduce a practical method for compacting the time evolution of the quantum state of a closed physical system. The density matrix is specified as a function of a few time-independent observables where their coefficients are time-dependent. The key mathematical step is the vectorization of the surprisal, the logarithm of the density matrix, at each time pointof interest. The time span used depends on the required spectral resolution. The entire course of the system evolution is represented as a matrix where each column is the vectorized surprisal at the given time point. Using the singular value decomposition (SVD) of this matrix, we generate realistic approximations for the time-independent observables and their respective time-dependent coefficients. This allows for a simplification of the algebraic procedure for determining the dominant constraints (the time-independent observables) in the sense of the maximal entropy approach. A non-stationary coherent initial state of a Morse oscillator is used to introduce the approach. We derive the analytical exact expression for the surprisal as a function of time, and this offers a benchmark for comparison with the accurate but approximate SVD results. We discuss two examples of a Morse potential of different anharmonicities, H2 and I2 molecules. We further demonstrate the approach for a two-coupled electronic state problem, the well-studied non-radiative decay of pyrazine from its bright state. Five constraints are found to be enough to capture the ultrafast electronic population exchange and to recover the dynamics of the wave packet in both electronic states.

Efficient video hashing based on low‐rank frames

Video hashing is a useful technology for diverse video applications, such as digital watermarking, copy detection and content authentication. This paper proposes a novel efficient video hashing based on low-rank frames. A key contribution is the low-rank frame calculation using the low-rank approximation of singular value decomposition (SVD). As the large singular values of SVD are stable to digital operations, video hash extraction using low-rank frames can provide good robustness. Since most energy is contained within the large singular values, low-rank frames also contribute to discrimination. Moreover, two-dimensional discrete wavelet transform (DWT) is applied to every low-rank frame and the mean of low-frequency DWT coefficients is selected as a hash element. Since these coefficients can represent input data approximately, hash discrimination is thus ensured. Experiments with 16,850 videos are carried out to test performances of the proposed algorithm. The results show that the proposed algorithm outperforms some well-known video hashing algorithms in computational time and classification about discrimination and robustness.

Approximate Multiplication of Sparse Matrices with Limited Space

Approximate matrix multiplication with limited space has received ever-increasing attention due to the emergence of large-scale applications. Recently, based on a popular matrix sketching algorithm---frequent directions, previous work has introduced co-occuring directions (COD) to reduce the approximation error for this problem. Although it enjoys the space complexity of O((m_x+m_y)l) for two input matrices X∈ℝ^{m_x ╳ n} and Y∈ℝ^{m_y ╳ n} where l is the sketch size, its time complexity is O(n(m_x+m_y+l)l), which is still very high for large input matrices. In this paper, we propose to reduce the time complexity by exploiting the sparsity of the input matrices. The key idea is to employ an approximate singular value decomposition (SVD) method which can utilize the sparsity, to reduce the number of QR decompositions required by COD. In this way, we develop sparse co-occuring directions, which reduces the time complexity to Õ((nnz(X)+nnz(Y))l+nl^2) in expectation while keeps the same space complexity as O((m_x+m_y)l), where nnz(X) denotes the number of non-zero entries in X and the Õ notation hides constant factors as well as polylogarithmic factors. Theoretical analysis reveals that the approximation error of our algorithm is almost the same as that of COD. Furthermore, we empirically verify the efficiency and effectiveness of our algorithm.

A Reformulation of Degree of Disequilibrium Analysis for Automatic Selection of Kinetic Constraints in the Rate-Controlled Constrained-Equilibrium Method

Abstract The rate-controlled constrained equilibrium (RCCE) is a model reduction scheme for chemical kinetics. It describes the evolution of a complex chemical system with acceptable accuracy with a number of rate controlling constraints on the associated constrained-equilibrium states of the system, much lower than the number of species in the underlying detailed kinetic model (DKM). Successful approximation of the constrained-equilibrium states requires accurate identification of the constraints. One promising procedure is the fully automatable Approximate Singular Value Decomposition of the Actual Degrees of Disequilibrium (ASVDADD) method that is capable of identifying the best constraints for a given range of thermodynamic conditions and a required level of approximation. ASVDADD is based on simple algebraic analysis of the results of the underlying DKM simulation and is focused on the behavior of the degrees of disequilibrium (DoD) of the individual chemical reactions. In this paper, we introduce an alternative ASVDADD algorithm. Unlike the original ASVDADD algorithm that require the direct computation of the DKM-derived DoDs and the identification of the set of linearly independent reactions, in the alternative algorithm, the components of the overall degree of disequilibrium vector can be computed directly by casting the DKM as an RCCE simulation considering a set of linearly independent constraints equaling the number of chemical species in size. The effectiveness and robustness of the derived constraints from the alternative procedure is examined in hydrogen/oxygen and methane/oxygen ignition delay simulations and the results are compared with those obtained from DKM.

Adaptive Image Denoising with Block-Rotation-Based SVD Filtering and Edge Detection

ABSTRACT Image denoising is essential in image processing. However, existing denoising methods are not optimized for the images with distinct straight edges and lines that traverse in different directions (such as images used in autonomous driving). This paper proposes an image denoising method that uses singular value decomposition (SVD) and block-rotation-based operations and has two features: the non-fixed size of block division and the rotation operations. In the method, we first propose an image division approach, which is used to divide an image into sub-blocks of different sizes, to ensure that the line(s) or edge(s) in each sub-block have roughly one main direction. Second, we decompose an image into sub-blocks according to the image division approach. Third, we rotate each sub-block to ensure that the main direction of the edge(s) is horizontal or vertical. Fourth, we perform SVD on each sub-block, and we use the low-rank approximation of SVD to obtain each denoised sub-block. Finally, we rotate the approximation of each sub-block back to the original direction of the corresponding noisy sub-block, and then we reconstruct the denoised image using these denoised sub-blocks. Experiments show the effectiveness of this method compared with the SVD-based methods.

Compression of Cross-Section Data Size for High-Resolution Core Analysis Using Dimensionality Reduction Technique

Compression of cross-section data used for high-resolution core analysis is performed using a dimensionality reduction technique based on the singular value decomposition (SVD) and low-rank approximation. The size of cross-section data can be a significant issue in high-resolution core analysis using detailed energy and spatial resolutions. This study addresses this issue focusing on the similarity of multigroup cross sections among different spatial regions. A data compression method using the SVD and low-rank approximation is applied for the multigroup microscopic cross sections of heterogeneous material regions obtained by a lattice physics calculation with burnup and branch calculations. Weighting by nuclide number densities and neutron spectra is considered to improve the efficiency of compression for cross sections. Single-assembly transport calculations with the method of characteristics are carried out using the original cross sections and the reconstructed cross sections after data compression. The accuracy of data compression for cross sections is evaluated by comparing the multiplication factor and multigroup scalar fluxes. The results indicate that the present data compression for microscopic cross sections can reduce approximately 99.7% of the original cross-section data size under the present calculation condition.

Theory of adaptive SVD regularization for deep neural networks

Deep networks can learn complex problems, however, they suffer from overfitting. To solve this problem, regularization methods have been proposed that are not adaptable to the dynamic changes in the training process. With a different approach, this paper presents a regularization method based on the Singular Value Decomposition (SVD) that adjusts the learning model adaptively. To this end, the overfitting can be evaluated by condition numbers of the synaptic matrices. When the overfitting is high, the matrices are substituted with their SVD approximations. Some theoretical results are derived to show the performance of this regularization method. It is proved that SVD approximation cannot solve overfitting after several iterations. Thus, a new Tikhonov term is added to the loss function to converge the synaptic weights to the SVD approximation of the best-found results. Following this approach, an Adaptive SVD Regularization (ASR) is proposed to adjust the learning model with respect to the dynamic training characteristics. ASR results are visualized to show how ASR overcomes overfitting. The different configurations of Convolutional Neural Networks (CNN) are implemented with different augmentation schemes to compare ASR with state-of-the-art regularization methods. The results show that on MNIST, F-MNIST, SVHN, CIFAR-10 and CIFAR-100, the accuracies of ASR are 99.4%, 95.7%, 97.1%, 93.2% and 55.6%, respectively. Although ASR improves the overfitting and validation loss, its elapsed time is not significantly greater than the learning without regularization.

Study of the Constraint Selection Through ASVDADD Method for Rate-Controlled Constrained-Equilibrium Modeling on Ethanol Oxidation Without PLOG Reactions

Abstract Reduction of the detail chemical kinetic mechanism is important in solving complex combustion simulation. In this work, a model reduction scheme rate-controlled constrained-equilibrium (RCCE) is considered in predicting the oxidation of ethanol. A detail kinetic mechanism by Merinov from Lawrence Livermore National Laboratory (LLNL) is used in modeling this reduction technique. The RCCE method considers constrained equilibrium states which subjected to a lower number of constraints compared to the number of species. It then has to solve a smaller number of differential equations compared to the number of equations required in solving the detailed kinetic model (DKM). The accuracy of this solution depends on the selection of the constraint. A systematic procedure which will help in identifying the constraint at an optimal level of accuracy is an essential for RCCE modeling. A fully automated Approximate Singular Value Decomposition of the Actual Degrees of Disequilibrium (ASVDADD) method is used in this study to derive the constraint for RCCE simulation. ASVDADD uses an algorithm which follows the simple algebraic analysis on results of underlying DKM to find the degree of disequilibrium (DoD) of the individual chemical reactions. The number of constraints which will be used in RCCE simulation can be selected to reduce the number of equations required to solve. In the current work, this ASVDADD method is applied on ethanol oxidation to select the constraint for RCCE simulation. Both DKM and RCCE calculations on ethanol fuel are demonstrated to compare the result of temperature distribution and an ignition delay time for validating the method.

A Fast Neural Network Learning Algorithm with Approximate Singular Value Decomposition

Abstract The learning of neural networks is becoming more and more important. Researchers have constructed dozens of learning algorithms, but it is still necessary to develop faster, more flexible, or more accurate learning algorithms. With fast learning we can examine more learning scenarios for a given problem, especially in the case of meta-learning. In this article we focus on the construction of a much faster learning algorithm and its modifications, especially for nonlinear versions of neural networks. The main idea of this algorithm lies in the usage of fast approximation of the Moore–Penrose pseudo-inverse matrix. The complexity of the original singular value decomposition algorithm is O(mn2). We consider algorithms with a complexity of O(mnl),where l<n and l is often significantly smaller than n. Such learning algorithms can be applied to the learning of radial basis function networks, extreme learning machines or deep ELMs, principal component analysis or even missing data imputation.

Hypercomplex Tensor Completion via Convex Optimization

Expressing multidimensional information as a value in hypercomplex number systems (e.g., quaternion, octonion, etc.) has great potential, in data sciences, e.g., signal processing, to enjoy their nontrivial algebraic benefits which are not available in standard real or complex vector systems. Strategic utilizations of such benefits would include, e.g., hypercomplex singular value decomposition (SVD) and low-rank approximation of matrices. In this paper, as powerful mathematical tools for wider signal processing applications of hypercomplex number systems, we first propose novel definitions of SVD and best low-rank approximation of matrices based on algebraic translations of Cayley–Dickson (C–D) number systems. We then derive an algorithmic solution to hypercomplex tensor completion problem based on a convex optimization technique. Numerical experiments in a scenario of color tensor completion problem show that the proposed algorithm recovers much more faithfully the original color information, masked randomly by noise, than part-wise tensor completion algorithms.

Characterization of dominated splittings for operator cocycles acting on Banach spaces

Versions of the Oseledets multiplicative ergodic theorem for cocycles acting on infinite-dimensional Banach spaces have been investigated since the pioneering work of Ruelle in 1982 and are a topic of continuing research interest. For a cocycle to induce a continuous splitting in which the growth in one subbundle exponentially dominates the growth in a complementary subbundle requires additional assumptions; a necessary and sufficient condition for the existence of such a dominated splitting (also known as exponential separation) was recently given by J. Bochi and N. Gourmelon for invertible finite-dimensional cocycles in discrete time. We extend this result to cocycles of injective, bounded, and possibly noncompact linear maps acting on Banach spaces (in both discrete and continuous time) using an essentially geometric approach based on a notion of approximate singular value decomposition in Banach spaces. Our method is constructive, and in the finite-dimensional case yields explicit growth estimates on the dominated splitting which may be of independent interest.

A matrix completion-based multiview learning method for imputing missing values in buoy monitoring data

Buoy systems have been deployed to monitor coastal waters, and these systems generate massive amounts of data with high temporal resolution. In practice, however, many of these records are often lost, which adversely impacts further data analysis. Current imputation methods place more emphasis on spatial and temporal correlations but ignore the crucial interactions between different parameters. To fully employ these interactions and further improve the imputation performance, a matrix completion-based multiview learning (MC-MVL) method is proposed in this paper to fill in the missing values in buoy monitoring data. MC-MVL considers three hybrid views, i.e., the temporal-parameter view, the spatial-parameter view and the spatiotemporal view, and each view is formulated as a matrix. The fixed-point continuation with approximate singular value decomposition (FPCA), the non-convex (Non-Convex) and the inexact augmented Lagrange multipliers (IALM) algorithms are separately utilized to reconstruct these three matrices. Then, a ridge regression-based multiview learning algorithm is used to aggregate the estimates of the three views into the final results. We used a buoy monitoring dataset from the Zhejiang coastal area to verify the imputation ability of the proposed model. The results confirm that MC-MVL achieves better performance than do 8 baseline approaches.

A Novel Fuzzy Rough Sets Theory Based CF Recommendation System

Collaborative Filtering (CF) is one of the popular methodology in recommender systems. It suffers from the data sparsity problem, recommendation inaccuracy and big-error in predictions. In this paper, the efficient advisory tool is implemented for the younger generation to choose their right career based on their knowledge. It acquires the notions of indiscernible relation from Fuzzy Rough Sets Theory (FRST) and propose a novel algorithm named as Fuzzy Rough Set Theory Based Collaborative Filtering Algorithm (FRSTBCF). To evaluate the model, data is prepared using the cross validation method. Based on that, ratings are evaluated by calculating the MAE (mean average error), MSE (means squared error) and RMSE (root means squared error) values. Further the correctness of the model is measured by finding rates like Accuracy, Specificity, Sensitivity, Precision & False Positive Rate. The proposed FRSTBCF algorithm is compared with the traditional algorithms experiment results such as Item Based Collaborative Filtering using the cosine similarity (IBCF-COS), IBCF using the pearson correlation (IBCF-COR), IBCF using the Jaccard similarity (IBCF-JAC) and Singular Value Decomposition approximation (SVD). The proposed algorithm gives better error rate and its precision value is comparatively identical with the existing system.