L1 Optimization Research Articles

Robust fault tolerant tracking controller design for unknown inputs T–S models with unmeasurable premise variables

This work concerns the tracking problem of uncertain Takagi–Sugeno fuzzy continuous systems with unmeasurable premise variables and affected by unknown inputs. The aim is to synthesize a fault tolerant controller (FTC) ensuring trajectory tracking of a desired reference model. To emit the original plant system, a norm bounded parametric uncertainty is employed in building the T–S model. The control scheme is based on a fuzzy observer to estimate both faults and faulty system states; a proportional integer (PI) observer to estimate constant faults is then adopted. Using descriptor redundancy property and L2 optimization to attenuate the unknown inputs effect, a solution is proposed in terms of bilinear matrix inequalities (BMIs). The performances of the proposed approach are pointed out by accentuating on a model of wastewater treatment plant (WWTP) through numerical results.

Compressive Sampling for Remote Control Systems

In remote control, efficient compression or representation of control signals is essential to send them through rate-limited channels. For this purpose, we propose an approach of sparse control signal representation using the compressive sampling technique. The problem of obtaining sparse representation is formulated by cardinality-constrained L2 optimization of the control performance, which is reducible to L1-L2 optimization. The low rate random sampling employed in the proposed method based on the compressive sampling, in addition to the fact that the L1-L2 optimization can be effectively solved by a fast iteration method, enables us to generate the sparse control signal with reduced computational complexity, which is preferable in remote control systems where computation delays seriously degrade the performance. We give a theoretical result for control performance analysis based on the notion of restricted isometry property (RIP). An example is shown to illustrate the effectiveness of the proposed approach via numerical experiments.

Inferring Rankings Using Constrained Sensing

We consider the problem of recovering a function over the space of permutations (or, the symmetric group) over n elements from given partial information; the partial information we consider is related to the group theoretic Fourier Transform of the function. This problem naturally arises in several settings such as ranked elections, multi-object tracking, ranking systems, and recommendation systems. Inspired by the work of Donoho and Stark in the context of discrete-time functions, we focus on non-negative functions with a sparse support (support size <;<; domain size). Our recovery method is based on finding the sparsest solution (through l0 optimization) that is consistent with the available information. As the main result, we derive sufficient conditions for functions that can be recovered exactly from partial information through l0 optimization. Under a natural random model for the generation of functions, we quantify the recoverability conditions by deriving bounds on the sparsity (support size) for which the function satisfies the sufficient conditions with a high probability as n → ∞. l0 optimization is computationally hard. Therefore, the popular compressive sensing literature considers solving the convex relaxation, l1 optimization, to find the sparsest solution. However, we show that l1 optimization fails to recover a function (even with constant sparsity) generated using the random model with a high probability as n → ∞. In order to overcome this problem, we propose a novel iterative algorithm for the recovery of functions that satisfy the sufficient conditions. Finally, using an Information Theoretic framework, we study necessary conditions for exact recovery to be possible.

A Template-Matching Method for Optical Power Monitoring

In this letter, we propose a template-matching (TM) method for monitoring optical power from greatly broadened optical spectrum acquired by a scanning tunable optical filter (TOF). Conventional methods, such as least squares (LS) and regularized spare recovery (RSR) methods, suffer performance degradation when the power discrepancies between adjacent channels are significant. In addition to the exploitation of prior information on channels and signal modulation formats, the proposed TM method further uses L1 optimization to alleviate the relative error in power estimation, especially for channels with weak signal power. Experimental results demonstrated that the TM method can be used to accurately monitor signal power of mixed 10/40-Gb/s wavelength-division-multiplexing (WDM) channels with 50-GHz channel spacing via commercial low-cost TOFs. This technique also has higher immunity to system errors as well as the dynamic range of the signal power discrepancies between adjacent channels.

ADAPTIVE DATA ANALYSIS VIA SPARSE TIME-FREQUENCY REPRESENTATION

We introduce a new adaptive method for analyzing nonlinear and nonstationary data. This method is inspired by the empirical mode decomposition (EMD) method and the recently developed compressed sensing theory. The main idea is to look for the sparsest representation of multiscale data within the largest possible dictionary consisting of intrinsic mode functions of the form {a(t) cos (θ(t))}, where a ≥ 0 is assumed to be smoother than cos (θ(t)) and θ is a piecewise smooth increasing function. We formulate this problem as a nonlinear L1 optimization problem. Further, we propose an iterative algorithm to solve this nonlinear optimization problem recursively. We also introduce an adaptive filter method to decompose data with noise. Numerical examples are given to demonstrate the robustness of our method and comparison is made with the EMD method. One advantage of performing such a decomposition is to preserve some intrinsic physical property of the signal, such as trend and instantaneous frequency. Our method shares many important properties of the original EMD method. Because our method is based on a solid mathematical formulation, its performance does not depend on numerical parameters such as the number of shifting or stop criterion, which seem to have a major effect on the original EMD method. Our method is also less sensitive to noise perturbation and the end effect compared with the original EMD method.

Optical coherence tomography by using frequency measurements in wavelength domain

Optical coherence tomography (OCT) reconstruction by using frequency measurements in the wavelength domain is presented in this paper. The method directly recovers the axial scan by formulating the frequency domain OCT (FD-OCT) into an algebraic reconstruction problem. In this way, the need for interpolation is removed. Then by solving the problem with ℓ1 optimization, the computational load is significantly reduced. It is demonstrated by experiment and simulation that the proposed method can achieve high resolution and longer imaging depth compared to the FD-OCT method.

An L1 design of GCF compensation filter

This paper introduces the design of the second order GCF (Generalized Comb Filter) compensator based on L1 optimization. The problem is represented by a set of nonlinear equations solved by the Newton–Raphson algorithm. The proof of the uniqueness of the solution is also provided. One can reduce the complexity of the design approximating the original set of nonlinear equations by the set of linear equations, which depends only on the passband edge frequency ωp. A design example demonstrates the effectiveness of the proposed design.

L2-norm multiple kernel learning and its application to biomedical data fusion

BackgroundThis paper introduces the notion of optimizing different norms in the dual problem of support vector machines with multiple kernels. The selection of norms yields different extensions of multiple kernel learning (MKL) such as L∞, L1, and L2 MKL. In particular, L2 MKL is a novel method that leads to non-sparse optimal kernel coefficients, which is different from the sparse kernel coefficients optimized by the existing L∞ MKL method. In real biomedical applications, L2 MKL may have more advantages over sparse integration method for thoroughly combining complementary information in heterogeneous data sources.ResultsWe provide a theoretical analysis of the relationship between the L2 optimization of kernels in the dual problem with the L2 coefficient regularization in the primal problem. Understanding the dual L2 problem grants a unified view on MKL and enables us to extend the L2 method to a wide range of machine learning problems. We implement L2 MKL for ranking and classification problems and compare its performance with the sparse L∞ and the averaging L1 MKL methods. The experiments are carried out on six real biomedical data sets and two large scale UCI data sets. L2 MKL yields better performance on most of the benchmark data sets. In particular, we propose a novel L2 MKL least squares support vector machine (LSSVM) algorithm, which is shown to be an efficient and promising classifier for large scale data sets processing.ConclusionsThis paper extends the statistical framework of genomic data fusion based on MKL. Allowing non-sparse weights on the data sources is an attractive option in settings where we believe most data sources to be relevant to the problem at hand and want to avoid a "winner-takes-all" effect seen in L∞ MKL, which can be detrimental to the performance in prospective studies. The notion of optimizing L2 kernels can be straightforwardly extended to ranking, classification, regression, and clustering algorithms. To tackle the computational burden of MKL, this paper proposes several novel LSSVM based MKL algorithms. Systematic comparison on real data sets shows that LSSVM MKL has comparable performance as the conventional SVM MKL algorithms. Moreover, large scale numerical experiments indicate that when cast as semi-infinite programming, LSSVM MKL can be solved more efficiently than SVM MKL.AvailabilityThe MATLAB code of algorithms implemented in this paper is downloadable from http://homes.esat.kuleuven.be/~sistawww/bioi/syu/l2lssvm.html.

Multivariate quantiles and multiple-output regression quantiles: From L1 optimization to halfspace depth

A new multivariate concept of quantile, based on a directional version of Koenker and Bassett's traditional regression quantiles, is introduced for multivariate location and multiple-output regression problems. In their empirical version, those quantiles can be computed efficiently via linear programming techniques. Consistency, Bahadur representation and asymptotic normality results are established. Most importantly, the contours generated by those quantiles are shown to coincide with the classical halfspace depth contours associated with the name of Tukey. This relation does not only allow for efficient depth contour computations by means of parametric linear programming, but also for transferring from the quantile to the depth universe such asymptotic results as Bahadur representations. Finally, linear programming duality opens the way to promising developments in depth-related multivariate rank-based inference.

Reconstruction of compressive sensing and semi-QR factorization

In this paper,the signal reconstruction algorithms of Compressive Sensing(CS) were discussed and a new method to enhance the efficiency was found,and the quality of recovered images was improved: proximate QR factorization of measurement matrix.The exact reconstruction of minimum l0 norm is NP-complete problem.Minimum l1 norm reconstruction can approximate compressible vectors with high probability.In the study,the quality of solutions of l1 optimization can be enhanced further by changing the singular values of the measurement matrix with QR factorization.We illustrated the effectiveness of QR factorization of the measurement matrix and gave a comparison of the Gaussian random matrix and its QR factorization.

Remote bitumen content estimation of Athabasca oil sand from hyperspectral infrared reflectance spectra using Gaussian singlets and derivative of Gaussian wavelets

Modeling of the total bitumen content, TBC, in Athabasca oil sands was undertaken on the basis of its hyperspectral reflectance spectra. Spectra (8 cm −1 resolution) were obtained that covered both the short-wave infrared and thermal infrared (TIR: 3.00–30.00 μm). Two methods, Gaussian fitting and wavelet analyses, were investigated to identify useful bitumen features as well as the best removal of the baseline. We aim to obtain the best determination of the TBC for a suitable suite of test and validation oil sands samples. The Gaussian model relied explicitly on features at 2.282 and 2.532 μm though these were only two of 10 features simultaneously fit with a quadratic baseline to the range of 2.230–2.603 μm of the spectra. The wavelet model relied on bitumen features selected at 2.274, 2.396 and 3.725 μm that could be isolated from the baseline and noise. Both models yielded similar dispersion in their estimates of TBC (±∼1–2%) while the wavelet model proved to be more robust when applied to the validation and blind data suites. We also considered the effects of using the L2 optimization (classical least-squares) and L1 optimization (minimization of largest outlier) schemes for both models. Both schemes produced similar results for the model suite of samples for TBC but the L1 was superior when applied to the validation and blind data suites. The wavelet model using the L1 optimization appeared to be quite robust producing estimates of TBC (±∼1.0–1.7%).

Dynamic Updating for $\ell_{1}$ Minimization

The theory of compressive sensing (CS) suggests that under certain conditions, a sparse signal can be recovered from a small number of linear incoherent measurements. An effective class of reconstruction algorithms involve solving a convex optimization program that balances the L1 norm of the solution against a data fidelity term. Tremendous progress has been made in recent years on algorithms for solving these L1 minimization programs. These algorithms, however, are for the most part static: they focus on finding the solution for a fixed set of measurements. In this paper, we will discuss "dynamic algorithms" for solving L1 minimization programs for streaming sets of measurements. We consider cases where the underlying signal changes slightly between measurements, and where new measurements of a fixed signal are sequentially added to the system. We develop algorithms to quickly update the solution of several different types of L1 optimization problems whenever these changes occur, thus avoiding having to solve a new optimization problem from scratch. Our proposed schemes are based on homotopy continuation, which breaks down the solution update in a systematic and efficient way into a small number of linear steps. Each step consists of a low-rank update and a small number of matrix-vector multiplications -- very much like recursive least squares. Our investigation also includes dynamic updating schemes for L1 decoding problems, where an arbitrary signal is to be recovered from redundant coded measurements which have been corrupted by sparse errors.

Underwater channel estimation and tracking using mixed norm optimization techniques.

A mixed norm optimization technique to track the impulse response of the underwater acoustic channel is presented. The channel is assumed to follow a delay‐Doppler spread function model, which is known to be sparsely distributed over the complex channel coefficients. The ill‐conditioned nature of this estimation problem, along with the need to optimize over a complex field, poses challenges to employing popular sparse estimation techniques that currently exist in the literature. The time‐varying nature of the underwater channel also makes efficient tracking difficult. The motivation behind our proposed algorithm is to robustly estimate and efficiently track the sparse delay‐Doppler coefficients directly over the complex field. We have formulated the channel estimation problem as that of minimizing over a convex combination of the L1 norm of the sparse coefficients to be estimated and the L2 norm of the estimation error. We analyze the difference between our proposed algorithm to existing L2‐constrained L1 optimization techniques and present results showing performance on simulation generated and field data. [Work supported in part by ONR Grant Nos. N00014‐05‐10085 and N00014‐07‐10184 and in part by Postdoctoral Scholar Program at the Woods Hole Oceanographic Institution, with funding provided by the Doherty Foundation.]

Reduced-Order Model-Based Feedback Control of Flow Over an Obstacle Using Center Manifold Methods

In this paper, we consider a boundary control problem governed by the two-dimensional Burgers’ equation for a configuration describing convective flow over an obstacle. Flows over obstacles are important as they arise in many practical applications. Burgers’ equations are also significant as they represent a simpler form of the more general Navier–Stokes momentum equation describing fluid flow. The aim of the work is to develop a reduced-order boundary control-oriented model for the system with subsequent nonlinear control law design. The control objective is to drive the full order system to a desired 2D profile. Reduced-order modeling involves the application of an L2 optimization based actuation mode expansion technique for input separation, demonstrating how one can obtain a reduced-order Galerkin model in which the control inputs appear as explicit terms. Controller design is based on averaging and center manifold techniques and is validated with full order numerical simulation. Closed-loop results are compared to a standard linear quadratic regulator design based on a linearization of the reduced-order model. The averaging∕center manifold based controller design provides smoother response with less control effort and smaller tracking error.

Compressed Sensing Shape Estimation of Star-Shaped Objects in Fourier Imaging

Recent theory of compressed sensing informs us that near-exact recovery of an unknown sparse signal is possible from a very limited number of Fourier samples by solving a convex L1 optimization problem. The main contribution of the present letter is a compressed sensing-based novel nonparametric shape estimation framework and a computational algorithm for binary star shape objects, whose radius functions belong to the space of bounded-variation functions. Specifically, in contrast with standard compressed sensing, the present approach involves directly reconstructing the shape boundary under sparsity constraint. This is done by converting the standard pixel-based reconstruction approach into estimation of a nonparametric shape boundary on a wavelet basis. This results in an L1 minimization under a nonlinear constraint, which makes the optimization problem nonconvex. We solve the problem by successive linearization and application of one-dimensional L1 minimization, which significantly reduces the number of sampling requirements as well as the computational burden. Fourier imaging simulation results demonstrate that high quality reconstruction can be quickly obtained from a very limited number of samples. Furthermore, the algorithm outperforms the standard compressed sensing reconstruction approach using the total variation norm.

Equivalence principle for optimization of sparse versus low‐spread representations for signal estimation in noise

AbstractEstimation of a sparse signal representation, one with the minimum number of nonzero components, is hard. In this paper, we show that for a nontrivial set of the input data the corresponding optimization problem is equivalent to and can be solved by an algorithm devised for a simpler optimization problem. The simpler optimization problem corresponds to estimation of signals under a low‐spread constraint. The goal of the two optimization problems is to minimize the Euclidian norm of the linear approximation error with an lp penalty on the coefficients, for p = 0 (sparse) and p = 1 (low‐spread), respectively. The l0 problem is hard, whereas the l1 problem can be solved efficiently by an iterative algorithm. Here we precisely define the l0 optimization problem, construct an associated l1 optimization problem, and show that for a set with open interior of the input data the optimizers of the two optimization problems have the same support. The associated l1 optimization problem is used to find the support of the l0 optimizer. Once the support of the l0 problem is known, the actual solution is easily found by solving a linear system of equations. However, we point out our approach does not solve the harder optimization problem for all input data and thus may fail to produce the optimal solution in some cases. © 2005 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 15, 10–17, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.20034

ORDER REDUCTION & STRUCTURE SIMPLIFICATION FOR NONLINEAR SYSTEMS

This paper considers the problem of passing from a nonlinear simple-structure model to a low-order approximation by preserving simplicity. The approximation problem is often best posed as an L2 optimization problem. This optimization problem is the core of our order reduction method so-called system matrices optimization. The simplification is formulated as special secondary conditions which can be added to the original optimization problem. In order to find the simplest reduced order model a search process should be performed that an appropriate and effective fitness function and some techniques for shrinking the search space and accelerating the search process is presented.

Stochastic inverse heat conduction using a spectral approach

AbstractAn adjoint‐based functional optimization technique in conjunction with the spectral stochastic finite element method is proposed for the solution of an inverse heat conduction problem in the presence of uncertainties in material data, process conditions and measurement noise. The ill‐posed stochastic inverse problem is restated as a conditionally well‐posed L2 optimization problem. The gradient of the objective function is obtained in a distributional sense by defining an appropriate stochastic adjoint field. The L2 optimization problem is solved using a conjugate‐gradient approach. Accuracy and effectiveness of the proposed approach is appraised with the solution of several stochastic inverse heat conduction problems. Copyright © 2004 John Wiley &amp; Sons, Ltd.

Robust ripple-free deadbeat control design

This paper deals with the design of ripple-free deadbeat controllers with performance or performance robustness optimized over controllers within a prescribed settling time. The performance objective of interest is the minimization of the maximum absolute tracking error. This leads to consider linfinity optimization problems. On the contrary, the optimization of the performance robustness leads to consider l1 optimization problems. An example is provided to illustrate the results.

Study of the acoustic feedback problem of active noise control by using the l1 and l2 vector space optimization approaches

This study attempts to explore the acoustic feedback problem that is frequently encountered in feedforward active noise control (ANC) structure from the standpoint of control theories. The analysis is carried out on the basis of the Youla’s parametrization and the l1-norm and l2-norm vector space optimization. The ANC problem is formulated as a model matching procedure and is solved via linear programming in the dual vector space. These methods alleviate the problems caused by the nonminimum phase (NMP) zeros of the cancellation path. The ANC algorithms are implemented by using a digital signal processor. Various types of noises were chosen for validating the developed algorithms. The proposed methods are also compared with the well-known filtered-u least-mean square (FULMS) method. The experimental results show that the acoustic feedback problem significantly degrades the performance and stability of the ANC system regardless which method is used. Insights into the difficulties due to acoustic feedback are addressed along the line of control theories.