Sparse Formulation Research Articles

Optimal Agile Satellite Target Scheduling with Learned Dynamics

Target sequencing is an important aspect of agile Earth-observing satellite scheduling. The objective of the problem is to find a feasible imaging sequence that maximizes the cumulative value of unique heterogeneously valued requests; no expendable resource constraints (power and data storage) are considered. Two challenges are encountered: transition times between targets are dictated by dynamics of an arbitrary controller, and the solution space is combinatorial with respect to request count. To find dynamically accurate time-dependent attitude maneuver transition times, a neural network is trained on simulated slew data; this allows for the generation of more physically accurate, and thus better performing, schedules than when using the traditional approach of lower-order models for slew feasibility. Next, slews between requests are represented by a sparsified graph, and a mixed-integer linear program is formulated to solve them; the sparse formulation keeps the problem tractable over longer planning horizons and larger numbers of requests when compared to the standard approach. The solutions are optimal up to the quality of the transition time estimator and a time discretization. Finally, the solutions are verified in a high-fidelity simulation, demonstrating validity when deployed on a lifelike system. Time-dependent request values and multiple satellites are also considered.

A least-squares identification method for vehicle cornering stiffness identification from common vehicle sensor data

In this study, we present a robust framework for vehicle tire cornering stiffness identification, leveraging commonly available lateral acceleration and yaw rate measurements. In contrast to most of the state-of-the-art approaches, our methodology does not rely on (augmented) state estimation, but on batch least-squares optimisation of larger time windows. This batch-approach has as a major benefit that many of the observability issues which are encountered in estimation-based methods are completely eliminated. By immediately exploiting the measurement equations, rather than comparing dynamic simulation responses, we obtain a very efficient and sparse formulation which enables online processing. The methodology was validated on (openly available) datasets from two different vehicles – a Ferrari 250LM and Range Rover Evoque, delivering consistent and accurate results for both vehicles. This work shows that batch optimisation can be a very promising alternative for the more common state-estimation approaches for extracting reliable vehicle cornering stiffnesses in various driving scenarios with a much more straightforward tuning.

Sparse polynomial chaos expansions for uncertainty quantification in thermal tomography

This contribution presents the identification strategy of thermal parameters relying solely on data measured on boundaries — thermal tomography. The idea is to obtain crucial information about the thermal properties inside the domain under consideration while keeping the test sample intact. Such methodology perfectly fits into historic preservation where it is of particular interest to perform only non-destructive surface measurements. We propose an advanced, accelerated, and reliable inverse solver for thermal tomography problems. Here, Bayesian inference is addressed as a method, where unknown parameters are modeled as random variables regularizing the inverse problem. The obtained results are probability distributions – posterior distributions – summarizing all available information and any remaining uncertainty in the values of thermal parameters. Novelties of our approach consist in the combination of (i) formulation of parameter identification in a probabilistic setting, and (ii) use of the surrogate models based on the sparse polynomial chaos expansion. This new sparse formulation significantly reduces the total number of polynomial terms and represents the main achievement of this paper.

A Bayesian hierarchical sparse factor model for estimating simultaneous covariance matrices for gestational outcomes in consecutive pregnancies.

Covariance estimation for multiple groups is a key feature for drawing inference from a heterogeneous population. One should seek to share information about common features in the dependence structures across the various groups. In this paper, we introduce a novel approach for estimating the covariance matrices for multiple groups using a hierarchical latent factor model that shrinks the factor loadings across groups toward a global value. Using a sparse spike and slab model on these loading coefficients allows for a sparse formulation of our model. Parameter estimation is accomplished through a Markov chain Monte Carlo scheme, and a model selection approach is used to select the number of factors to use. We validate our model through extensive simulation studies. Finally, we apply our methodology to the NICHD Consecutive Pregnancies Study to estimate the correlations between birth weights and gestational ages of three consecutive birth within four different subgroups (underweight, normal, overweight, and obese) of women.

Sensing matrix design for wideband channel estimation in millimeter‐wave hybrid multiple‐input multiple‐output system

AbstractHybrid analog/digital multiple‐input multiple‐output (MIMO) system is proposed to support low complexity multi‐stream communication in millimeter‐wave (mmWave) frequencies. The main idea of channel estimation methods for mmWave hybrid MIMO systems is based on leveraging the sparsity nature of the channel in the angle and delay domain. Sparse signal recovery algorithms have been used to reconstruct the channel coefficients. Due to the lack of information about the angle of arrival and angle of departure before channel estimation, random values are common choices for precoders and combiners. Moreover, the training pilot symbols are considered to be random variables. In this article, we propose sequential time domain sensing matrix design and sequential frequency domain sensing matrix design algorithms to design these variables in order to enhance sparse signal recovery performance and consequently the channel estimation accuracy. Our methods are based on reducing the correlation between the sensing matrix columns, which is defined in the sparse formulation of the channel estimation problem. We design the training radio frequency precoders/combiners phase angle values and pilot symbol vectors with the aid of a sequential procedure that has a reasonable complexity. Simulation results show that utilizing the designed values instead of random variables increased the performance of the channel estimation, especially when we deal with a higher number of variables.

Concentration of Posterior Model Probabilities and Normalized L0 Criteria

We study frequentist properties of Bayesian and L0 model selection, with a focus on (potentially non-linear) high-dimensional regression. We propose a construction to study how posterior probabilities and normalized L0 criteria concentrate on the (Kullback-Leibler) optimal model and other subsets of the model space. When such concentration occurs, one also bounds the frequentist probabilities of selecting the correct model, type I and type II errors. These results hold generally, and help validate the use of posterior probabilities and L0 criteria to control frequentist error probabilities associated to model selection and hypothesis tests. Regarding regression, we help understand the effect of the sparsity imposed by the prior or the L0 penalty, and of problem characteristics such as the sample size, signal-to-noise, dimension and true sparsity. A particular finding is that one may use less sparse formulations than would be asymptotically optimal, but still attain consistency and often also significantly better finite-sample performance. We also prove new results related to misspecifying the mean or covariance structures, and give tighter rates for certain non-local priors than currently available.

Exploiting Sparsity for Semi-Algebraic Set Volume Computation

We provide a systematic deterministic numerical scheme to approximate the volume (i.e., the Lebesgue measure) of a basic semi-algebraic set whose description follows a correlative sparsity pattern. As in previous works (without sparsity), the underlying strategy is to consider an infinite-dimensional linear program on measures whose optimal value is the volume of the set. This is a particular instance of a generalized moment problem which in turn can be approximated as closely as desired by solving a hierarchy of semidefinite relaxations of increasing size. The novelty with respect to previous work is that by exploiting the sparsity pattern we can provide a sparse formulation for which the associated semidefinite relaxations are of much smaller size. In addition, we can decompose the sparse relaxations into completely decoupled subproblems of smaller size, and in some cases computations can be done in parallel. To the best of our knowledge, it is the first contribution that exploits correlative sparsity for volume computation of semi-algebraic sets which are possibly high-dimensional and/or non-convex and/or non-connected.

Similarity Based Block Sparse Subset Selection for Video Summarization

Video summarization (VS) is generally formulated as a subset selection problem where a set of representative keyframes or key segments is selected from an entire video frame set. Though many sparse subset selection based VS algorithms have been proposed in the past decade, most of them adopt linear sparse formulation in the explicit feature vector space of video frames, and don’t consider the local or global relationships among frames. In this paper, we first extend the conventional sparse subset selection for VS into kernel block sparse subset selection (KBS3) to utilize the advantage of kernel sparse coding and introduce a local inter-frame relationship through packing of frame blocks. Going a step further, we propose a similarity based block sparse subset selection (SB2S3) model by applying a specially designed transformation matrix on the KBS3 model in order to introduce a kind of global inter-frame relationship through the similarity. Finally, a greedy pursuit based algorithm is devised for the proposed NP-hard model optimization. The proposed SB2S3 has the following advantages: 1) through the similarity between each frame and any other frame, the global relationship among all frames can be considered; 2) through block sparse coding, the local relationship of adjacent frames is further considered; and 3) it has a wider application, since features can derive similarity, but not vice versa. It is believed that the effect of modeling such global and local relationships among frames in this paper, is similar to that of modeling the long-range and short-range dependencies among frames in deep learning based methods. Experimental results on three benchmark datasets have demonstrated that the proposed approach is superior to not only other sparse subset selection based VS methods but also most unsupervised deep-learning based VS methods.

On a computationally scalable sparse formulation of the multidimensional and nonstationary maximum entropy principle

Data-driven modelling and computational predictions based on maximum entropy principle (MaxEnt-principle) aim at finding as-simple-as-possible - but not simpler then necessary - models that allow to avoid the data overfitting problem. We derive a multivariate non-parametric and non-stationary formulation of the MaxEnt-principle and show that its solution can be approximated through a numerical maximisation of the sparse constrained optimization problem with regularization. Application of the resulting algorithm to popular financial benchmarks reveals memoryless models allowing for simple and qualitative descriptions of the major stock market indexes data. We compare the obtained MaxEnt-models to the heteroschedastic models from the computational econometrics (GARCH, GARCH-GJR, MS-GARCH, GARCH-PML4) in terms of the model fit, complexity and prediction quality. We compare the resulting model log-likelihoods, the values of the Bayesian Information Criterion, posterior model probabilities, the quality of the data autocorrelation function fits as well as the Value-at-Risk prediction quality. We show that all of the considered seven major financial benchmark time series (DJI, SPX, FTSE, STOXX, SMI, HSI and N225) are better described by conditionally memoryless MaxEnt-models with nonstationary regime-switching than by the common econometric models with finite memory. This analysis also reveals a sparse network of statistically-significant temporal relations for the positive and negative latent variance changes among different markets. The code is provided for open access.

Variations on the Convolutional Sparse Coding Model

Over the past decade, the celebrated sparse representation model has achieved impressive results in various signal and image processing tasks. A convolutional version of this model, termed convolutional sparse coding (CSC), has been recently reintroduced and extensively studied. CSC brings a natural remedy to the limitation of typical sparse enforcing approaches of handling global and high-dimensional signals by local, patch-based, processing. While the classic field of sparse representations has been able to cater for the diverse challenges of different signal processing tasks by considering a wide range of problem formulations, almost all available algorithms that deploy the CSC model consider the same $\ell _1 - \ell _2$ problem form. As we argue in this paper, this CSC pursuit formulation is also too restrictive as it fails to explicitly exploit some local characteristics of the signal. This work expands the range of formulations for the CSC model by proposing two convex alternatives that merge global norms with local penalties and constraints. The main contribution of this work is the derivation of efficient and provably converging algorithms to solve these new sparse coding formulations.

Full-waveform inversion of salt models using shape optimization and simulated annealing

A good starting model is imperative in full-waveform inversion (FWI) because it solves a least-squares inversion problem using a local gradient-based optimization method. A suboptimal starting model can result in cycle skipping leading to poor convergence and incorrect estimation of subsurface properties. This problem is especially crucial for salt models because the strong velocity contrasts create substantial time shifts in the modeled seismogram. Incorrect estimation of salt bodies leads to velocity inaccuracies in the sediments because the least-squares gradient aims to reduce traveltime differences without considering the sharp velocity jump between sediments and salt. We have developed a technique to estimate velocity models containing salt bodies using a combination of global and local optimization techniques. To stabilize the global optimization algorithm and keep it computationally tractable, we reduce the number of model parameters by using sparse parameterization formulations. The sparse formulation represents sediments using a set of interfaces and velocities across them, whereas a set of ellipses represents the salt body. We use very fast simulated annealing (VFSA) to minimize the misfit between the observed and synthetic data and estimate an optimal model in the sparsely parameterized space. The VFSA inverted model is then used as a starting model in FWI in which the sediments and salt body are updated in the least-squares sense. We partition model updates into sediment and salt updates in which the sediments are updated like conventional FWI, whereas the shape of the salt is updated by taking the zero crossing of an evolving level set surface. Our algorithm is tested on two 2D synthetic salt models, namely, the Sigsbee 2A model and a modified SEG Advanced Modeling Program (SEAM) Phase I model while fixing the top of the salt. We determine the efficiency of the VFSA inversion and imaging improvements from the level set FWI approach and evaluate a few sources of uncertainty in the estimation of salt shapes.

Estimation of Factor Analytic Mixed Models for the Analysis of Multi-treatment Multi-environment Trial Data

An efficient and widely used method of analysis for multi-environment trial (MET) data in plant improvement programs involves a linear mixed model with a factor analytic (FA) model for the variety by environment effects. The variance structure is generally constructed as the kronecker product of two matrices that relate to the variety and environment dimensions, respectively. In many applications, the FA variance structure is assumed for the environment dimension and either an identity matrix or a known matrix, such as the numerator relationship matrix, is used for the variety dimension. The factor analytic linear mixed model can be fitted to large and complex MET datasets using the sparse formulation of the average information algorithm of Thompson et al. (Aust N Z J Stat 45:445–459, 2003) or the extension provided by Kelly et al. (Genet Sel Evo, 41:1186–1297, 2009) for the case of a known non-identity matrix for the variety dimension. In this paper, we present a sparse formulation of the average information algorithm for a more general separable variance structure where all components are parametric and one component has an FA structure. The approach is illustrated using a large and highly unbalanced MET dataset where there is a factorial treatment structure. Supplementary materials accompanying this paper appear online.

Solution to a three-dimensional axisymmetric inverse electromagnetic casting problem

ABSTRACTA new method based on topology optimization is proposed for solving a three-dimensional axisymmetric inverse problem regarding the configuration of electric currents used in the electromagnetic casting technique of the metallurgical industry. These electric currents must generate an electromagnetic field such that when certain mass of liquid metal is placed in the field it levitates acquiring a predefined axisymmetric shape. The method proposed is able to obtain a suitable configuration of electric currents, and is very attractive from the computational point of view, since it is based on a sparse convex quadratic programming formulation for which there are very efficient interior-point solution algorithms.

Surrogates for the matrix l0-quasinorm in sparse feedback design: Numerical study of the efficiency

Some formulations of the optimal control problem require the resulting controller to be sparse; i.e., to contain zero elements in the gain matrix. On one hand, sparse feedback leads to the drop of performance as compared to the optimal control; on the other hand, it confers useful properties to the system. For instance, sparse controllers allow to design distributed systems with decentralized feedback. Some sparse formulations require the gain matrix of the controller to have a special sparse structure which is characterized by the presence of zero rows in the matrix. In this paper, various approximations to the number of nonzero rows of a matrix are considered and applied to sparse feedback design in optimal control problems for linear systems. Along with a popular approach based on using the matrix $\ell_1$-norm, more complex nonconvex surrogates are proposed and discussed, those surrogates being minimized via special numerical procedures. The efficiency of the approximations is compared via numerical experiments.

Exact algorithms for the order picking problem

Order picking is the problem of collecting a set of products in a warehouse in a minimum amount of time. It is currently a major bottleneck in supply-chain because of its cost in time and labor force. This article presents two exact and effective algorithms for this problem. Firstly, a sparse formulation in mixed-integer programming is strengthened by preprocessing and valid inequalities. Secondly, a dynamic programming approach generalizing known algorithms for two or three cross-aisles is proposed and evaluated experimentally. Performances of these algorithms are reported and compared with the Traveling Salesman Problem (TSP) solver Concorde.

VARIABLE SELECTION FOR BAYESIAN SURVIVAL MODELS USING BREGMAN DIVERGENCE MEASURE

The variable selection has been an important topic in regression and Bayesian survival analysis. In the era of rapid development of genomics and precision medicine, the topic is becoming more important and challenging. In addition to the challenges of handling censored data in survival analysis, we are facing increasing demand of handling big data with too many predictors where most of them may not be relevant to the prediction of the survival outcome. With the desire of improving upon the accuracy of prediction, we explore the Bregman divergence criterion in selecting predictive models. We develop sparse Bayesian formulation for parametric regression and semiparametric regression models and demonstrate how variable selection is done using the predictive approach. Model selections for a simulated data set, and two real-data sets (one for a kidney transplant study, and the other for a breast cancer microarray study at the Memorial Sloan-Kettering Cancer Center) are carried out to illustrate our methods.

Sparse Feedback Design in Discrete-Time Linear Systems

The subject of this paper is the analysis of sparse state feedback design procedures for linear discrete-time systems. By sparsity we mean the presence of zero rows in the gain matrix; this requirement is natural in the engineering practice when designing “economy” control systems which make use of a small amount of control inputs. Apart from the design of stabilizing sparse controllers, the linear-quadratic regulation problem is considered in the sparse formulation. Also, we consider a regularization scheme typical to the l1-optimization theory. The efficiency of the approach is illustrated via numerical examples.

Analysis of Sparse Cutting Planes for Sparse MILPs with Applications to Stochastic MILPs

In this paper, we present an analysis of the strength of sparse cutting planes for mixed integer linear programs (MILP) with sparse formulations. We examine three kinds of problems: packing problems, covering problems, and more general MILPs with the only assumption that the objective function is nonnegative. Given an MILP instance of one of these three types, assume that we decide on the support of cutting planes to be used and the strongest inequalities on these supports are added to the linear programming relaxation. We present bounds on the ratio of optimal value of the LP relaxation after adding cuts and the optimal value of the MILP that depends only on the sparsity structure of the constraint matrix and the support of sparse cuts selected, that is, these bounds are completely data independent. These results also shed light on the strength of single-scenario cuts for two-stage stochastic MILPs.

Channel Estimation for Hybrid Architecture-Based Wideband Millimeter Wave Systems

Hybrid analog and digital precoding allows millimeter wave (mmWave) systems to achieve both array and multiplexing gain. The design of the hybrid precoders and combiners, though, is usually based on the knowledge of the channel. Prior work on mmWave channel estimation with hybrid architectures focused on narrowband channels. Since mmWave systems will be wideband with frequency selectivity, it is vital to develop channel estimation solutions for hybrid architectures-based wideband mmWave systems. In this paper, we develop a sparse formulation and compressed sensing-based solutions for the wideband mmWave channel estimation problem for hybrid architectures. First, we leverage the sparse structure of the frequency-selective mmWave channels and formulate the channel estimation problem as a sparse recovery in both time and frequency domains. Then, we propose explicit channel estimation techniques for purely time or frequency domains and for combined time/frequency domains. Our solutions are suitable for both single carrier-frequency domain equalization and orthogonal frequency-division multiplexing systems. Simulation results show that the proposed solutions achieve good channel estimation quality, while requiring small training overhead. Leveraging the hybrid architecture at the transceivers gives further improvement in estimation error performance and achievable rates.

Novel sparse LSSVR models in primal weight space for robust system identification with outliers

Recent demands from big data applications have strongly motivated a successful sparse formulation of the least squares support vector regression (LSSVR) model in primal weight space. Such an approach, which has been called fixed-size LSSVR (FS-LSSVR), is built upon an approximation of the nonlinear feature mapping based on Nyström method to overcome the usual memory constraints and high computational costs of the standard non-sparse LSSVR model. Despite this advance, an important modeling issue remains unaddressed by the FS-LSSVR model. As in the standard LSSVR model, the performance of FS-LSSVR model is greatly degraded when estimation data is corrupted with non-Gaussian noise or outliers. Bearing this major issue in mind, we introduce two robust variants of the FS-LSSVR model based on M-estimation framework and the weighted least squares method. The proposed approaches, henceforth called Robust FS-LSSVR (RFS-LSSVR) and Reweighted Robust FS-LSSVR (R2FS-LSSVR) models, produce solutions that are simultaneously robust to outliers and sparse, making use of only a small sample of training patterns as prototype vectors. We evaluate the performances of both algorithms in benchmarking nonlinear system identification problems with synthetic and real-world datasets (including a large scale dataset) corresponding to SISO and MIMO systems, whose estimation outputs are contaminated with outliers. The obtained results indicate that our proposed approaches consistently outperforms existing robust models designed in dual space (e.g. W-LSSVR and IR-LSSVR models), specially as the amount of outliers in the data increases.