Exploitation Of Sparsity Research Articles

Mathtt {Tenscalc}: a toolbox to generate fast code to solve nonlinear constrained minimizations and compute Nash equilibria

We describe the toolbox mathtt {Tenscalc} that generates specialized C-code to solve nonlinear constrained optimizations and to compute Nash equilibria. mathtt {Tenscalc} is aimed at scenarios where one needs to solve very fast a large number of optimizations that are structurally similar. This is common in applications where the optimizations depend on measured data and one wants to compute optima for large or evolving datasets, e.g., in robust estimation and classification, maximum likelihood estimation, model predictive control (MPC), moving horizon estimation (MHE), and combined MPC-MHE (which requires the computation of a saddle-point equilibria). mathtt {Tenscalc} is mostly aimed at generating solvers for optimizations with up to a few thousands of optimization variables/constraints and solve times up to a few milliseconds. The speed achieved by the solver arises from a combination of features: reuse of intermediate computations across and within iterations of the solver, detection and exploitation of matrix sparsity, avoidance of run-time memory allocation and garbage collection, and reliance on flat code that improves the efficiency of the microprocessor pipelining and caching. All these features have been automated and embedded into the code generation process. We include a few representative examples to illustrate how the speed and memory footprint of the solver scale with the size of the problem.

On the Performance of Efficient Channel Estimation Strategies for Hybrid Millimeter Wave MIMO System.

Millimeter wave (mmWave) relying upon the multiple output multiple input (MIMO) is a new potential candidate for fulfilling the huge emerging bandwidth requirements. Due to the short wavelength and the complicated hardware architecture of mmWave MIMO systems, the conventional estimation strategies based on the individual exploitation of sparsity or low rank properties are no longer efficient and hence more modern and advance estimation strategies are required to recapture the targeted channel matrix. Therefore, in this paper, we proposed a novel channel estimation strategy based on the symmetrical version of alternating direction methods of multipliers (S-ADMM), which exploits the sparsity and low rank property of channel altogether in a symmetrical manner. In S-ADMM, at each iteration, the Lagrange multipliers are updated twice which results symmetrical handling of all of the available variables in optimization problem. To validate the proposed algorithm, numerous computer simulations have been carried out which straightforwardly depicts that the S-ADMM performed well in terms of convergence as compared to other benchmark algorithms and also able to provide global optimal solutions for the strictly convex mmWave joint channel estimation optimization problem.

Estimating Doubly-Selective Channels for Hybrid mmWave Massive MIMO Systems: A Doubly-Sparse Approach

In mmWave massive multiple-input multiple-output (mMIMO) systems, hybrid (digital/analog) structure has been a prevalent option to balance system cost and performance. To facilitate transceiver design in hybrid mmWave mMIMO, acquiring an accurate channel state information is critical. To this end, a novel doubly-sparse approach is proposed to estimate doubly-selective mmWave channels under hybrid mMIMO. Via the judiciously designed training pattern, the well-utilized beamspace sparsity alongside the under-investigated delay-domain sparsity that mmWave channels exhibit can be jointly exploited to assist channel estimation. Thanks to our careful two-stage (random-probing and steering-probing) design, the proposed channel estimator possesses strong robustness against the double (frequency and time) selectivity whilst enjoying the benefits brought by the exploitation of double sparsity. Compared with existing alternatives, our proposed mmWave channel estimator not only works in doubly-selective channels, but also largely reduces the training overhead, storage demand as well as computational complexity.

Dynamically Rescaled Hamiltonian Monte Carlo for Bayesian Hierarchical Models

Dynamically rescaled Hamiltonian Monte Carlo is introduced as a computationally fast and easily implemented method for performing full Bayesian analysis in hierarchical statistical models. The method relies on introducing a modified parameterization so that the reparameterized target distribution has close to constant scaling properties, and thus is easily sampled using standard (Euclidian metric) Hamiltonian Monte Carlo. Provided that the parameterizations of the conditional distributions specifying the hierarchical model are “constant information parameterizations” (CIPs), the relation between the modified- and original parameterization is bijective, explicitly computed, and admit exploitation of sparsity in the numerical linear algebra involved. CIPs for a large catalogue of statistical models are presented, and from the catalogue, it is clear that many CIPs are currently routinely used in statistical computing. A relation between the proposed methodology and a class of explicitly integrated Riemann manifold Hamiltonian Monte Carlo methods is discussed. The methodology is illustrated on several example models, including a model for inflation rates with multiple levels of nonlinearly dependent latent variables. Supplementary materials for this article are available online.

The Race to Improve Radar Imagery: An overview of recent progress in statistical sparsity-based techniques

The exploitation of sparsity has significantly advanced the field of radar imaging over the last few decades, leading to substantial improvements in the resolution and quality of the processed images. More recent developments in compressed sensing (CS) suggest that statistical sparsity can lead to further performance benefits by imposing sparsity as a statistical prior on the considered signal. In this article, a comprehensive survey is made of recent progress on statistical sparsitybased techniques for various radar imagery applications.

STRUCTURELESS BUNDLE ADJUSTMENT WITH SELF-CALIBRATION USING ACCUMULATED CONSTRAINTS

Abstract. Bundle adjustment based on collinearity is the most widely used optimization method within image based scene reconstruction. It incorporates observed image coordinates, exterior and intrinsic camera parameters as well as object space coordinates of the observed points. The latter dominate the resulting nonlinear system, in terms of the number of unknowns which need to be estimated. In order to reduce the size of the problem regarding memory footprint and computational effort, several approaches have been developed to make the process more efficient, e.g. by exploitation of sparsity or hierarchical subdivision. Some recent developments express the bundle problem through epipolar geometry and scale consistency constraints which are free of object space coordinates. These approaches are usually referred to as structureless bundle adjustment. The number of unknowns in the resulting system is drastically reduced. However, most work in this field is focused on optimization towards speed and considers calibrated cameras, only. We present our work on structureless bundle adjustment, focusing on precision issues as camera calibration and residual weighting. We further investigate accumulation of constraint residuals as an approach to decrease the number of rows of the Jacobian matrix.

STRUCTURELESS BUNDLE ADJUSTMENT WITH SELF-CALIBRATION USING ACCUMULATED CONSTRAINTS

Bundle adjustment based on collinearity is the most widely used optimization method within image based scene reconstruction. It incorporates observed image coordinates, exterior and intrinsic camera parameters as well as object space coordinates of the observed points. The latter dominate the resulting nonlinear system, in terms of the number of unknowns which need to be estimated. In order to reduce the size of the problem regarding memory footprint and computational effort, several approaches have been developed to make the process more efficient, e.g. by exploitation of sparsity or hierarchical subdivision. Some recent developments express the bundle problem through epipolar geometry and scale consistency constraints which are free of object space coordinates. These approaches are usually referred to as structureless bundle adjustment. The number of unknowns in the resulting system is drastically reduced. However, most work in this field is focused on optimization towards speed and considers calibrated cameras, only. We present our work on structureless bundle adjustment, focusing on precision issues as camera calibration and residual weighting. We further investigate accumulation of constraint residuals as an approach to decrease the number of rows of the Jacobian matrix.

Identifying the neuroanatomical basis of cognitive impairment in Alzheimer's disease by correlation- and nonlinearity-aware sparse Bayesian learning.

Predicting cognitive performance of subjects from their magnetic resonance imaging (MRI) measures and identifying relevant imaging biomarkers are important research topics in the study of Alzheimer's disease. Traditionally, this task is performed by formulating a linear regression problem. Recently, it is found that using a linear sparse regression model can achieve better prediction accuracy. However, most existing studies only focus on the exploitation of sparsity of regression coefficients, ignoring useful structure information in regression coefficients. Also, these linear sparse models may not capture more complicated and possibly nonlinear relationships between cognitive performance and MRI measures. Motivated by these observations, in this work we build a sparse multivariate regression model for this task and propose an empirical sparse Bayesian learning algorithm. Different from existing sparse algorithms, the proposed algorithm models the response as a nonlinear function of the predictors by extending the predictor matrix with block structures. Further, it exploits not only inter-vector correlation among regression coefficient vectors, but also intra-block correlation in each regression coefficient vector. Experiments on the Alzheimer's Disease Neuroimaging Initiative database showed that the proposed algorithm not only achieved better prediction performance than state-of-the-art competitive methods, but also effectively identified biologically meaningful patterns.

Compressive sensing: From theory to applications, a survey

Compressive sensing (CS) is a novel sampling paradigm that samples signals in a much more efficient way than the established Nyquist sampling theorem. CS has recently gained a lot of attention due to its exploitation of signal sparsity. Sparsity, an inherent characteristic of many natural signals, enables the signal to be stored in few samples and subsequently be recovered accurately, courtesy of CS. This article gives a brief background on the origins of this idea, reviews the basic mathematical foundation of the theory and then goes on to highlight different areas of its application with a major emphasis on communications and network domain. Finally, the survey concludes by identifying new areas of research where CS could be beneficial.

Simplified p-norm-like constraint LMS algorithm for efficient estimation of underwater acoustic channels

Underwater acoustic channels are recognized for being one of the most difficult propagation media due to considerable difficulties such as: multipath, ambient noise, time-frequency selective fading. The exploitation of sparsity contained in underwater acoustic channels provides a potential solution to improve the performance of underwater acoustic channel estimation. Compared with the classic l0 and l1 norm constraint LMS algorithms, the p-norm-like (lp) constraint LMS algorithm proposed in our previous investigation exhibits better sparsity exploitation performance at the presence of channel variations, as it enables the adaptability to the sparseness by tuning of p parameter. However, the decimal exponential calculation associated with the p-norm-like constraint LMS algorithm poses considerable limitations in practical application. In this paper, a simplified variant of the p-norm-like constraint LMS was proposed with the employment of Newton iteration method to approximate the decimal exponential calculation. Numerical simulations and the experimental results obtained in physical shallow water channels demonstrate the effectiveness of the proposed method compared to traditional norm constraint LMS

Exploiting Sparsity in Operational-space Dynamics

This paper presents a new method for calculating operational-space inertia matrices, and other related quantities, for branched kinematic trees. It is based on the exploitation of branch-induced sparsity in the joint-space inertia matrix and the task Jacobian. Detailed cost figures are given for the new method, and its efficacy is demonstrated by means of a realistic example based on the ASIMO Next-Generation humanoid robot. In this example, the new method is shown to be 6.7 times faster than the basic matrix method, and 1.6 times faster than the efficient low-order algorithm of Rodriguez et al. Furthermore, cost savings of more than 50,000 arithmetic operations are obtained in the calculation of the inertia-weighted pseudoinverse of the task Jacobian and its null-space projection matrix. Additional examples are considered briefly, in order to further compare the new method with the algorithm of Rodriguez et al.

Large-Scale Molecular Optimization from Distance Matrices by a D.C. Optimization Approach

A so-called DCA method based on a d.c.\ (difference of convex functions) optimization approach (algorithm) for solving large-scale distance geometry problems is developed. Different formulations of equivalent d.c.\ programs in the $l_{1}$-approach are stated via the Lagrangian duality without gap relative to d.c.\ programming, and new nonstandard nonsmooth reformulations in the $l_{\infty }$-approach (resp., the $l_{1}-l_{\infty }$-approach) are introduced. Substantial subdifferential calculations permit us to compute sequences of iterations in the DCA quite simply. The computations actually require matrix-vector products and only one Cholesky factorization (resp., with an additional solution of a convex program) in the $l_{1}$-approach (resp., the $l_{1}-l_{\infty }$-approach) and allow the exploitation of sparsity in the large-scale setting. Two techniques---respectively, using shortest paths between all pairs of atoms to generate the complete dissimilarity matrix and the spanning trees procedure---are investigated in order to compute a good starting point for the DCA. Finally, many numerical simulations of the molecular optimization problems with up to 12567 variables are reported, which prove the practical usefulness of the nonstandard nonsmooth reformulations, the globality of found solutions, and the robustness and efficiency of our algorithms.

A Preconditioned Conjugate Gradient Approach to Linear Equality Constrained Minimization

We propose a new framework for the application of preconditioned conjugate gradients in the solution of large-scale linear equality constrained minimization problems. This framework allows for the exploitation of structure and sparsity in the context of solving the reduced Newton system (despite the fact that the reduced system may be dense). Numerical experiments performed on a variety of test problems from the Netlib LP collection indicate computational promise.

Introduction to the Special Section on Sparse and Structured Matrices and Their Applications

The field of sparse matrices is a broad and important area of the computational sciences that includes structured matrices and those with seemingly little or no structure. The relevance of the field is highlighted by the wide range of application areas that require the exploitation of matrix sparsity and structure in order to achieve a solution given real-world constraints on computing resources and/or time. Applications in which sparse matrices appear include structural analysis, computational fluid dynamics, economic modeling, financial analysis, numerical optimization, statistical modeling, power network analysis, electromagnetics, meteorology, medical imaging, data mining, and many more. A number of significant advancements in sparse matrix computations have been made in recent years. These advances have led to new challenges as multidisciplinary problems are now ambitiously posed, and they symbolize the growth in the area and demonstrate the dependence of the future of many fields on sparse matrix methods. Moreover, they give rise to a host of new problems that have yet to be addressed by the sparse matrix community. The Second SIAM Conference on Sparse Matrices, which was held in Coeur d'Alene, Idaho, October 9--11, 1996, was organized specifically to address some of the challenging issues in sparse matrix computations. This special issue of SIAM Journal on Matrix Analysis and Applications consists of some of the outstanding papers that were presented at the Sparse Matrix conference. Because of an error in scheduling, the paper titled ``Using Generalized Cayley Transformations within an Inexact Rational Krylov Sequence Method" by Richard Lehoucq, which should have appeared in this section, was published in SIAM Journal on Matrix Analysis and Applications, volume 20, number 1. We are grateful to Paul van Dooren, the Editor-in-Chief of SIAM Journal on Matrix Analysis and Applications, and the SIAM office for agreeing to publish a special section on sparse matrices. We would like to thank Roland Freund, Anne Greenbaum, Joseph Liu, and Zdenek Strakos for serving on the Guest Editorial Board; their effort and cooperation in handling the papers were much appreciated. Finally, we would like to express thanks to all the authors who have submitted papers to the special section; it would not have become a reality without their support.

Efficient computation of gradients and Jacobians by dynamic exploitation of sparsity in automatic differentiation

Automatic differentiation (AD) is a technique that augments computer codes with statements for the computation of derivatives. The computational workhorse of AD-generated codes for first-order derivatives is the linear combination of vectors. For many large-scale problems, the vectors involved in this operation are inherently sparse. If the underlying function is a partially separable one (e.g., if its Hessian is sparse), many of the intermediate gradient vectors computed by AD will also be sparse, even though the final gradient is likely to be dense. For large Jacobians computations, every intermediate derivative vector is usually at least as sparse as the least sparse row of the final Jacobian. In this paper, we show that dynamic exploitation of the sparsity inherent in derivative computation can result in dramatic gains in runtime and memory savings. For a set of gradient problems exhibiting implicit sparsity, we report on the runtime and memory requirements of computing the gradients with the ADIFOR (...

A DIAKOPTICAL TECHNIQUE FOR SHORT CIRCUIT STUDIES OF LARGE SIZE POWER SYSTEM NETWORKS

ABSTRACT The paper presents a diakoptical technique for deriving the impedance matrices required for short circuit studies of large size power system networks by tearing the system into smaller subsystems by cutting appropriate tie lines. The bus admittance matrix is formed for the sequence networks of each subsystem and then modified later by means of diakoptics. The simulation of mutual couplings is taken into account in the zero sequence networks. Only columns of the impedance matrices corresponding to the faulted buses are computed in the proposed technique which is based on factorized bus admittance matrices and full exploitation of sparsity. The proposed technique can either be used in a single–processor computer for sequential solution of torn subsystems or in a multicomputer configuration for a faster solution by parallel processing of torn subsystems. Both symmetrical and unsymmetrical faults are studied by using the proposed technique which produces exact results as the untorn system. No iterati...

Solution of sparse positive definite systems on a hypercube

The solution of large sparse positive definite systems of equations typically involves four steps: ordering, data structure set-up (symbolic factorization), numerical factorization, and triangular solution. This article describes how these four phases are implemented on a hypercube multiprocessor. The role of elimination trees in the exploitation of sparsity and the identification of parallelism is explained, and pseudo-code algorithms are provided for some of the important algorithms. Numerical experiments run on an Intel iPSC multiprocessor are presented in order to provide some indication of the performance of the various algorithms.

Remarks on the choice of a stiff ordinary differential equation solver

This paper discusses several aspects of the solution of fluid-flow problems. A model problem is described which captures several of the salient characteristics of such problems. The model problem is derived from the one-dimensional Euler equations. It corresponds to the subcooled liquid region of a three-region reactor steam-generator model. The defining partial differential equations are discretized spatially using the method of pseudo-characteristics. The resulting system of ordinary differential equations (ODEs) is then solved by the method of lines using each of several available ODE solvers. In addition to demonstrating that fluid-flow problems can be solved effectively using adaptive mathematical software, the results also illustrate the typical performance of each solver as well as the relative merits of special solver features (e.g. automatic method switching and exploitation of problem sparsity or bandedness). The model problem is also modified and used to illustrate several commonly observed characteristics of fluid-flow models. The results illustrate the potential effect on solver performance of characteristics such as inaccurate water property calculations. Consequently, they explain apparent anomalies often attributed incorrectly to poor solver design and algorithmic implementation. The results also support the choice of the sparse ODE solver LSODES for the solution of fluid-flow problems.

A Computational Technique for the Simulation of a Riverine Well System

A waterworks consisting of 11 wells have been developed along the Danube gravel aquifer to supply drinking water for 5 settlements. The operation had been undisturbed as long as gravel raining has started and a river boating section of 2 Km has been constructed. These two activities has greatly affected the seepage system of the wells. A two dimensional laminar simulation model has been developed in order to simulate and analyse the above two interventions on the operation of the waterworks. In fact the objective was to select the right locations of gravel mining and the boating section which don’t disturb the operation in a significant way. For that purpose a finite difference simulation was developed and the resulting linear equations were solved by an efficient direct solution technique. The method used allows a generalization of the ordinary grid concept by the so called field redefinition. We could redefine any rectangular part of the original or that of any soon redefined field and that way we could employ a substantially smaller mesh size in the critical parts of the investigated area. The coupled direct solution technique tries to minimize computer time and core usage by exploitation of matrix sparsity and disk storage.

Solution of sparse linear least squares problems using givens rotations

We describe a direct method for solving sparse linear least squares problems. The storage required for the method is no more than that needed for the conventional normal equations approach. However, the normal equations are not computed; orthogonal transformations are applied to the coefficient matrix, thus avoiding the potential numerical instability associated with computing the normal equations. Our approach allows full exploitation of sparsity, and permits the use of a fixed (static) data structure during the numerical computation. Finally, the method processes the coefficient matrix one row at a time, allowing for the convenient use of auxiliary storage and updating operations.