L1-norm Minimization Problem Research Articles

Affine projection sign algorithm with ℓ1 minimization-based variable step-size

In this paper, a new affine projection sign algorithm with a variable step-size, robust in impulsive noise environments, is proposed. For that purpose, ℓ1-norm of the a posteriori error vector is minimized under a box constraint on the step-size. Since the proposed ℓ1-norm minimization problem is non-differentiable convex one, an efficient numerical procedure is developed for the affine projection structure algorithm. Finally, it is demonstrated that the proposed algorithm, not requiring any a priori knowledge, yields more robust convergence in various impulsive noise environments than conventional algorithms with robustness to outliers and recent variable step-size algorithms with a priori information.

SPARSITY SCORE: A NOVEL GRAPH-PRESERVING FEATURE SELECTION METHOD

As thousands of features are available in many pattern recognition and machine learning applications, feature selection remains an important task to find the most compact representation of the original data. In the literature, although a number of feature selection methods have been developed, most of them focus on optimizing specific objective functions. In this paper, we first propose a general graph-preserving feature selection framework where graphs to be preserved vary in specific definitions, and show that a number of existing filter-type feature selection algorithms can be unified within this framework. Then, based on the proposed framework, a new filter-type feature selection method called sparsity score (SS) is proposed. This method aims to preserve the structure of a pre-defined l1graph that is proven robust to data noise. Here, the modified sparse representation based on an l1-norm minimization problem is used to determine the graph adjacency structure and corresponding affinity weight matrix simultaneously. Furthermore, a variant of SS called supervised SS (SuSS) is also proposed, where the l1graph to be preserved is constructed by using only data points from the same class. Experimental results of clustering and classification tasks on a series of benchmark data sets show that the proposed methods can achieve better performance than conventional filter-type feature selection methods.

Fast and Accurate Algorithms for Re-Weighted $\ell _{1}$-Norm Minimization

To recover a sparse signal from an underdetermined system, we often solve a constrained l1-norm minimization problem. In many cases, the signal sparsity and recovery performance can be further improved by replacing the l1 norm with a “weighted” l1 norm. Without prior information about the signal's nonzero elements, the procedure for selecting weights is iterative in nature. Common approaches update the weights at every iteration using the solution of a weighted l1 problem from the previous iteration. This paper presents two homotopy-based algorithms that efficiently solve reweighted l1 problems. First, we present an algorithm that quickly updates the solution of a weighted l1 problem as the weights change. Since the solution changes only slightly with small changes in weights, we develop a homotopy algorithm that replaces old weights with new ones in a small number of computationally inexpensive steps. Second, we propose an algorithm that solves a weighted l1 problem by adaptively selecting weights while estimating the signal. This algorithm integrates the reweighting into every step along the homotopy path by changing the weights according to changes in the solution and its support, allowing us to achieve a high quality signal reconstruction by solving a single homotopy problem. We compare both algorithms' performance, in terms of reconstruction accuracy and computational complexity, against state-of-the-art solvers and show that our methods have smaller computational cost. We also show that the adaptive selection of the weights inside the homotopy often yields reconstructions of higher quality.

Sparse Hyperspectral Unmixing Based on Constrained lp - l2 Optimization

Linear spectral unmixing is an effective technique to estimate the abundances of materials present in each hyperspectral image pixel. Recently, sparse-regression-based unmixing approaches have been proposed to tackle this problem. Mostly, <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norm minimization is used to approximate the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> norm minimization problem in terms of computational complexity. In this letter, we model the hyperspectral unmixing as a constrained sparse <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">lp</i> - <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> (0 <; <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i> <; 1) optimization problem and propose to solve it via the iteratively reweighted least squares algorithm. Experimental results on a series of simulated data sets and a real hyperspectral image demonstrate that the proposed method can achieve performance improvement over the state-of-the-art <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> - <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> method.

High resolution range-reflectivity estimation of radar targets via compressive sampling and Memetic Algorithm

Recent results of Compressive Sampling (CS) have demonstrated its feasibility in high-resolution radar targets estimation and imaging [2,10,14,15,17,19,23,29,30,32–34]. However, the signal recovery is reduced to seeking a sparse solution to an underdetermined linear system of equations. It is potentially very difficult because even finding a solution that approximates the true minimum is NP-hard. In this paper, we introduce Memetic Algorithm (MA) to solve this non-convex l0-norm minimization problem, and design a compressive receiver for high-resolution range-reflectivity estimation of multiple radar targets. A double-population MA is proposed, where the position population is used to evaluate the ranges, and the coefficient population is used to realize a local search of target reflectivities. By combining the global search with a local searching operation to exploit the available knowledge in the recovery, the proposed MA outperforms the general purpose optimization algorithms in terms of the quality of solution. Some experiments are taken to investigate the performance of this compressive receiver at different sampling rates, and the results show the superiority to its counterparts in both noiseless environment and noisy, cluttered environment.

Compressed sensing signal recovery via forward–backward pursuit

Recovery of sparse signals from compressed measurements constitutes an l0 norm minimization problem, which is unpractical to solve. A number of sparse recovery approaches have appeared in the literature, including l1 minimization techniques, greedy pursuit algorithms, Bayesian methods and nonconvex optimization techniques among others. This manuscript introduces a novel two stage greedy approach, called the Forward-Backward Pursuit (FBP). FBP is an iterative approach where each iteration consists of consecutive forward and backward stages. The forward step first expands the support estimate by the forward step size, while the following backward step shrinks it by the backward step size. The forward step size is larger than the backward step size, hence the initially empty support estimate is expanded at the end of each iteration. Forward and backward steps are iterated until the residual power of the observation vector falls below a threshold. This structure of FBP does not necessitate the sparsity level to be known a priori in contrast to the Subspace Pursuit or Compressive Sampling Matching Pursuit algorithms. FBP recovery performance is demonstrated via simulations including recovery of random sparse signals with different nonzero coefficient distributions in noisy and noise-free scenarios in addition to the recovery of a sparse image.

A Compressed Sensing Based Basis-pursuit Formulation of the Room Algorithm

Regulatory on off minimization (ROOM) is a popular metabolic modeling strategy for obtaining the fluxes of various metabolic reactions in a mutant. It is based on minimization of the number of flux changes with respect to the wild-type. The ROOM approach involves solving an integer programming problem. In ROOM, the number of integer decision variables is equal to the number of reactions in the metabolic network under consideration. Typically, metabolic networks of interest are genome scale implying that the number of reactions in the network and hence the number of integer decision variables is large. The ROOM approach thus has inherent difficulties associated with large scale integer programming problems. In the current work, motivated by the emerging area of compressed sensing, we propose a reformulation, known as basis-pursuit, of the ROOM algorithm. The proposed formulation is an L1 norm minimization problem and is thus convex in nature. The proposed approach is used to obtain the flux profiles for various mutants of the Synechocystis species strain PCC 6803. The results are compared with the existing ROOM approach. It is observed that the proposed algorithm performs better in most cases. Use of compressed sensing based formulation creates exciting possibilities of efficiently reformulating various other metabolic network analysis problems.

Rotated Orthogonal Transform (ROT) for Motion-Compensation Residual Coding

Discrete cosine transform (DCT) is the orthogonal transform that is most commonly used in image and video compression. The motion-compensation residual (MC-residual) is also compressed with the DCT in most video codecs. However, the MC-residual has different characteristics from a nature image. In this paper, we develop a new orthogonal transform-rotated orthogonal transform (ROT) that can perform better on the MC-residual than the DCT for coding purposes. We derive the proposed ROT based on orthogonal-constrained L1-Norm minimization problem for its sparse property. Using the DCT matrix as the starting point, a better orthogonal transform matrix is derived. In addition, by exploring inter-frame dependency and local motion activity, transmission of substantial side information is avoided. The experiment results confirm that, with small computation overhead, the ROT is adaptive to change of local spatial characteristic of MC-residual frame and provides higher compression efficiency for the MC-residual than DCT, especially for high- and complex-motion videos.

Evolution-enhanced multiscale overcomplete dictionaries learning for image denoising

In this paper, a multiscale overcomplete dictionary learning approach is proposed for image denoising by exploiting the multiscale property and sparse representation of images. The images are firstly sparsely represented by a translation invariant dictionary and then the coefficients are denoised using some learned multiscale dictionaries. Dictionaries learning can be reduced to a non-convex l0-norm minimization problem with multiple variables, so an evolution-enhanced algorithm is proposed to alternately optimize the variables. Some experiments are taken on comparing the performance of our proposed method with its counterparts on some benchmark natural images, and the superiorities of our proposed method to its counterparts can be observed in both the visual result and some numerical guidelines.

Sparse Representation for Target Detection in Hyperspectral Imagery

In this paper, we propose a new sparsity-based algorithm for automatic target detection in hyperspectral imagery (HSI). This algorithm is based on the concept that a pixel in HSI lies in a low-dimensional subspace and thus can be represented as a sparse linear combination of the training samples. The sparse representation (a sparse vector corresponding to the linear combination of a few selected training samples) of a test sample can be recovered by solving an l0-norm minimization problem. With the recent development of the compressed sensing theory, such minimization problem can be recast as a standard linear programming problem or efficiently approximated by greedy pursuit algorithms. Once the sparse vector is obtained, the class of the test sample can be determined by the characteristics of the sparse vector on reconstruction. In addition to the constraints on sparsity and reconstruction accuracy, we also exploit the fact that in HSI the neighboring pixels have a similar spectral characteristic (smoothness). In our proposed algorithm, a smoothness constraint is also imposed by forcing the vector Laplacian at each reconstructed pixel to be minimum all the time within the minimization process. The proposed sparsity-based algorithm is applied to several hyperspectral imagery to detect targets of interest. Simulation results show that our algorithm outperforms the classical hyperspectral target detection algorithms, such as the popular spectral matched filters, matched subspace detectors, adaptive subspace detectors, as well as binary classifiers such as support vector machines.

Recursive Algorithm for L1 Norm Estimation in Linear Models

L1 norm estimator has been widely used as a robust parameter estimation method for outlier detection. Different algorithms have been applied for L1 norm minimization among which the linear programming problem based on the simplex method is well known. In the present contribution, in order to solve an L1 norm minimization problem in a linear model, an interior point algorithm is developed which is based on Dikin's method. The method can be considered as an appropriate alternative for the classical simplex method, which is sometimes time-consuming. The proposed method, compared with the simplex method, is thus easier for implementation and faster in performance. Furthermore, a recursive form of the Dikin's method is derived, which resembles the recursive least-squares method. Two simulated numerical examples show that the proposed algorithm gives as accurate results as the simplex method but in considerably less time. When dealing with a large number of observations, this algorithm can thus be used instead of the iteratively reweighted least-squares method and the simplex method.

Adaptive Repetitive Control Systems for Fast Rejection of Periodic Disturbances with Uncertain Multiple Periods

In this paper, a discrete-time repetitive control problem is considered for the cases where the disturbance consists of periodic signals with several different periods but the periods are uncertain. A new adaptive repetitive control system for such cases is proposed and consists of three elements; a stabilizing controller, a period estimator and an adaptive reduced order periodic signal generator. The controller not only assures the stability of the overall adaptive repetitive control system including the time-varying generator but also assigns all poles of the closed-loop system on the disk with a given radius. The design problem is reduced to a simple and feasible l1 norm minimization problem. The estimator estimates multiple periods of disturbance from the input/output data of the plant. The algorithm consists of the well-known recursive least squares method. In order to reject periodic disturbances with uncertain multiple periods, the adaptive generator is adjusted on-line by using the period estimated. The repetitive controller with the generator is implemented with much less memory elements than the previous ones. As a result, the degree of the controller is significantly reduced and much faster convergence of the controlled error to zero is provided. The effectiveness is demonstrated by simulations.

Optimization of frame representations for compressed sensing and Mercedes-Benz frame

We prove that there is no single uniform tight frame in Euclidean (unitary) space such that a solution of the l1-norm minimization problem for the frame representation is attained on the frame coefficients. Then we find an exact solution of the l1-minimization problem for the Mercedes-Benz frame in ℝN. We also give some examples of connections between optimization problems of various types.

L1 Least Square for Cancer Diagnosis using Gene Expression Data

The performance of most methods for cancer diagnosis using gene expression data greatly depends on careful model selection. Least square for classification has no need of model selection. However, a major drawback prevents it from successful application in microarray data classification: lack of robustness to outliers. In this paper we cast linear regression as a constrained l1-norm minimization problem to greatly alleviate its sensitivity to outliers, and hence the name l1 least square. The numerical experiment shows that l1 least square can match the best performance achieved by support vector machines (SVMs) with careful model selection.

L1 solution of linear inequalities

The numerical solution of a possible inconsistent system of linear inequalities in the l1 sense is considered. The non-differentiable l1 norm minimization problem is approximated by a piecewise quadratic Huber smooth function. A continuation algorithm is designed to find an l1 solution of the inequality system. In the case where the linear inequality system is consistent, a solution is obtained by solving any smoothed problem. Otherwise, the algorithm is shown to terminate in a finite number of iterations. We also consider an alternative smoothing scheme which shares similar properties with the first one, but results in an improved computational performance of the continuation algorithm on inconsistent systems. Numerical experiments are conducted to test the efficiency of the algorithm.

Optimal tracking and rejection of persistent bounded signals for mimo continuous systems

This paper introduces a design methodology for an optimal stabilizing compensator that not only minimizes the maximum tracking error but also minimizes the maximum amplitude of the undesired part of the output of a multivariable (MIMO) continuous-time system due to any persistent bounded disturbances. For this purpose a two-parameter compensator scheme is adopted in the design of the feedback system. This paper emphasizes that the optimal compensator design problem is equivalent to a constrained L1-norm minimization problem and its solution can be obtained by standard non-linear programming routines.