L1 Regularization Method Research Articles

Reconstruction of electrical capacitance tomography images based on fast linearized alternating direction method of multipliers for two-phase flow system

Electrical capacitance tomography (ECT) has been applied to two-phase flow measurement in recent years. Image reconstruction algorithms play an important role in the successful applications of ECT. To solve the ill-posed and nonlinear inverse problem of ECT image reconstruction, a new ECT image reconstruction method based on fast linearized alternating direction method of multipliers (FLADMM) is proposed in this paper. On the basis of theoretical analysis of compressed sensing (CS), the data acquisition of ECT is regarded as a linear measurement process of permittivity distribution signal of pipe section. A new measurement matrix is designed and L1 regularization method is used to convert ECT inverse problem to a convex relaxation problem which contains prior knowledge. A new fast alternating direction method of multipliers which contained linearized idea is employed to minimize the objective function. Simulation data and experimental results indicate that compared with other methods, the quality and speed of reconstructed images are markedly improved. Also, the dynamic experimental results indicate that the proposed algorithm can fulfill the real-time requirement of ECT systems in the application.

Image Super-Resolution via Adaptive Regularization and Sparse Representation.

Previous studies have shown that image patches can be well represented as a sparse linear combination of elements from an appropriately selected over-complete dictionary. Recently, single-image super-resolution (SISR) via sparse representation using blurred and downsampled low-resolution images has attracted increasing interest, where the aim is to obtain the coefficients for sparse representation by solving an l0 or l1 norm optimization problem. The l0 optimization is a nonconvex and NP-hard problem, while the l1 optimization usually requires many more measurements and presents new challenges even when the image is the usual size, so we propose a new approach for SISR recovery based on regularization nonconvex optimization. The proposed approach is potentially a powerful method for recovering SISR via sparse representations, and it can yield a sparser solution than the l1 regularization method. We also consider the best choice for lp regularization with all p in (0, 1), where we propose a scheme that adaptively selects the norm value for each image patch. In addition, we provide a method for estimating the best value of the regularization parameter λ adaptively, and we discuss an alternate iteration method for selecting p and λ . We perform experiments, which demonstrates that the proposed regularization nonconvex optimization method can outperform the convex optimization method and generate higher quality images.

Regularized Lattice Sphere Decoding for Block Data Transmission Systems

This paper presents Lattice Sphere Decoding (LSD) with regularization techniques for block data transmission systems. It has shown that a small condition number ($$\tau $$?) results in better detection performance. This paper aims to reduce this value to its smallest possible value. Regularization technique offers reduction in condition number ($$\tau $$?) that improves LSD performance. In this work, two regularization techniques are utilized on LSD. First, $$\hbox {L}_{1}$$L1-regularization method is introduced, which sums the mixed norms. Second, $$\hbox {L}_{2}$$L2- regularization method, which is the most commonly used method of regularization for ill-conditioned problems in mathematics. We derive the exact relationship between the LSD performance and condition number ($$\tau $$?) as well as the relationship between LSD initial radius (d) and condition number ($$\tau $$?). The derived equations show their convergence to the fact that the performance increases as the radius (d) increases. Simulation results show that the LSD with $$\hbox {L}_{1}$$L1-regularization technique offers smaller condition number ($$\tau $$?), and therefore, produces better system performance. From the performance results and the complexity analysis, it is apparent that the proposed techniques achieve a good balance between complexity and performance.

Classification of spectral data using fused lasso logistic regression

Spectral data contain powerful information that can be used to identify unknown compounds and their chemical structures. In this paper, we study fused lasso logistic regression (FLLR) to classify the spectral data into two groups. We show that the FLLR has a grouping property on regression coefficients, which simultaneously selects a group of highly correlated variables together. Both the sparsity and the grouping property of the FLLR provide great advantages in the analysis of the spectral data. In particular, it resolves the well-known peak misalignment problem of the spectral data by providing data dependent binning, and provides a better interpretable classifier than other ℓ1-regularization methods. We also analyze the gas chromatography/mass spectrometry data to classify the origin of herbal medicines, and illustrate the advantages of the FLLR over other existing ℓ1-regularized methods.

Statistical downscaling to predict monthly rainfall using linear regression with L_1 regularization (LASSO)

Regression with L1 regularization (lasso) method was compared to principal component regression (PCR) in Statistical Downscaling (SDS) modeling to predict monthly rainfall. SDS modeling uses ill-conditioned (high correlation/multicolliniear) covariates, which can be solved with selection or shrinkage methods. In this study, we used two GCMs with dierent characteristics as covariates (CMIP5 and GPCP version 2.2). The results shows that the lasso method gave better results (smaller RMSE and RMSEP) than PCR for GPCP version 2.2, and as good as PCR for CMIP5 covariates.

Variable selection in regression with compositional covariates

Motivated by research problems arising in the analysis of gut microbiome and metagenomic data, we consider variable selection and estimation in high-dimensional regression with compositional covariates. We propose an l1 regularization method for the linear log-contrast model that respects the unique features of compositional data. We formulate the proposed procedure as a constrained convex optimization problem and introduce a coordinate descent method of multipliers for efficient computation. In the high-dimensional setting where the dimensionality grows at most exponentially with the sample size, model selection consistency and $\ell _{\infty }$ bounds for the resulting estimator are established under conditions that are mild and interpretable for compositional data. The numerical performance of our method is evaluated via simulation studies and its usefulness is illustrated by an application to a microbiome study relating human body mass index to gut microbiome composition.

Complex-valued sparse reconstruction via arctangent regularization

Complex-valued sparse reconstruction is conventionally solved by transforming it into real-valued problems. However, this method might not work efficiently and correctly, especially when the size of the problem is large, or the mutual coherence is high. In this paper, we present a novel algorithm called the arctangent regularization (ATANR), which can handle the complex-valued problems of large size and high mutual coherence directly. The ATANR is implemented with the iterative least squares (IRLS) framework, and accelerated by the dimension reduction and active set selection steps. Further, we summarize and analyze the common properties of a penalty kernel which is suitable for sparse reconstruction. The analyses show that the key difference, between the arctangent kernel and the ℓ1 norm, is that the first order derivative of ATANR is close to zero for a nonzero variable. This will make ATANR less sensitive to the regularization parameter λ than ℓ1 regularization methods. Finally, lots of numerical experiments validate that ATANR usually has better performance than the conventional ℓ1 regularization methods, not only for the random signs ensemble, but also for the sensing matrix with high mutual coherence, such as the resolution enhancement case.

Half-Quadratic-Based Iterative Minimization for Robust Sparse Representation

Robust sparse representation has shown significant potential in solving challenging problems in computer vision such as biometrics and visual surveillance. Although several robust sparse models have been proposed and promising results have been obtained, they are either for error correction or for error detection, and learning a general framework that systematically unifies these two aspects and explores their relation is still an open problem. In this paper, we develop a half-quadratic (HQ) framework to solve the robust sparse representation problem. By defining different kinds of half-quadratic functions, the proposed HQ framework is applicable to performing both error correction and error detection. More specifically, by using the additive form of HQ, we propose an ℓ1-regularized error correction method by iteratively recovering corrupted data from errors incurred by noises and outliers; by using the multiplicative form of HQ, we propose an ℓ1-regularized error detection method by learning from uncorrupted data iteratively. We also show that the ℓ1-regularization solved by soft-thresholding function has a dual relationship to Huber M-estimator, which theoretically guarantees the performance of robust sparse representation in terms of M-estimation. Experiments on robust face recognition under severe occlusion and corruption validate our framework and findings.

Image reconstruction based on L1 regularization and projection methods for electrical impedance tomography

Electrical impedance tomography (EIT) is a technique for reconstructing the conductivity distribution by injecting currents at the boundary of a subject and measuring the resulting changes in voltage. Image reconstruction in EIT is a nonlinear and ill-posed inverse problem. The Tikhonov method with L(2) regularization is always used to solve the EIT problem. However, the L(2) method always smoothes the sharp changes or discontinue areas of the reconstruction. Image reconstruction using the L(1) regularization allows addressing this difficulty. In this paper, a sum of absolute values is substituted for the sum of squares used in the L(2) regularization to form the L(1) regularization, the solution is obtained by the barrier method. However, the L(1) method often involves repeatedly solving large-dimensional matrix equations, which are computationally expensive. In this paper, the projection method is combined with the L(1) regularization method to reduce the computational cost. The L(1) problem is mainly solved in the coarse subspace. This paper also discusses the strategies of choosing parameters. Both simulation and experimental results of the L(1) regularization method were compared with the L(2) regularization method, indicating that the L(1) regularization method can improve the quality of image reconstruction and tolerate a relatively high level of noise in the measured voltages. Furthermore, the projected L(1) method can also effectively reduce the computational time without affecting the quality of reconstructed images.

Parallelism, uniqueness, and large‐sample asymptotics for the Dantzig selector

The Dantzig selector (Candès and Tao, 2007) is a popular ℓ1-regularization method for variable selection and estimation in linear regression. We present a very weak geometric condition on the observed predictors which is related to parallelism and, when satisfied, ensures the uniqueness of Dantzig selector estimators. The condition holds with probability 1, if the predictors are drawn from a continuous distribution. We discuss the necessity of this condition for uniqueness and also provide a closely related condition which ensures uniqueness of lasso estimators (Tibshirani, 1996). Large sample asymptotics for the Dantzig selector, i.e. almost sure convergence and the asymptotic distribution, follow directly from our uniqueness results and a continuity argument. The limiting distribution of the Dantzig selector is generally non-normal. Though our asymptotic results require that the number of predictors is fixed (similar to (Knight and Fu, 2000)), our uniqueness results are valid for an arbitrary number of predictors and observations.

Metasample-Based Sparse Representation for Tumor Classification

A reliable and accurate identification of the type of tumors is crucial to the proper treatment of cancers. In recent years, it has been shown that sparse representation (SR) by l1-norm minimization is robust to noise, outliers and even incomplete measurements, and SR has been successfully used for classification. This paper presents a new SR-based method for tumor classification using gene expression data. A set of metasamples are extracted from the training samples, and then an input testing sample is represented as the linear combination of these metasamples by l1-regularized least square method. Classification is achieved by using a discriminating function defined on the representation coefficients. Since l1-norm minimization leads to a sparse solution, the proposed method is called metasample-based SR classification (MSRC). Extensive experiments on publicly available gene expression data sets show that MSRC is efficient for tumor classification, achieving higher accuracy than many existing representative schemes.

L1 regularization method in electrical impedance tomography by using the L1-curve (Pareto frontier curve)

Electrical impedance tomography (EIT), as an inverse problem, aims to calculate the internal conductivity distribution at the interior of an object from current–voltage measurements on its boundary. Many inverse problems are ill-posed, since the measurement data are limited and imperfect. To overcome ill-posedness in EIT, two main types of regularization techniques are widely used. One is categorized as the projection methods, such as truncated singular value decomposition (SVD or TSVD). The other categorized as penalty methods, such as Tikhonov regularization, and total variation methods. For both of these methods, a good regularization parameter should yield a fair balance between the perturbation error and regularized solution. In this paper a new method combining the least absolute shrinkage and selection operator (LASSO) and the basis pursuit denoising (BPDN) is introduced for EIT. For choosing the optimum regularization we use the L1-curve (Pareto frontier curve) which is similar to the L-curve used in optimising L2-norm problems. In the L1-curve we use the L1-norm of the solution instead of the L2 norm. The results are compared with the TSVD regularization method where the best regularization parameters are selected by observing the Picard condition and minimizing generalized cross validation (GCV) function. We show that this method yields a good regularization parameter corresponding to a regularized solution. Also, in situations where little is known about the noise level σ, it is also useful to visualize the L1-curve in order to understand the trade-offs between the norms of the residual and the solution. This method gives us a means to control the sparsity and filtering of the ill-posed EIT problem. Tracing this curve for the optimum solution can decrease the number of iterations by three times in comparison with using LASSO or BPDN separately.

Missing values: sparse inverse covariance estimation and an extension to sparse regression

We propose an l1-regularized likelihood method for estimating the inverse covariance matrix in the high-dimensional multivariate normal model in presence of missing data. Our method is based on the assumption that the data are missing at random (MAR) which entails also the completely missing at random case. The implementation of the method is non-trivial as the observed negative log-likelihood generally is a complicated and non-convex function. We propose an efficient EM algorithm for optimization with provable numerical convergence properties. Furthermore, we extend the methodology to handle missing values in a sparse regression context. We demonstrate both methods on simulated and real data.

Superficial Mycoses

Electrical capacitance tomography (ECT) has been applied to two-phase flow measurement in recent years. Image reconstruction algorithms play an important role in the successful applications of ECT. To solve the ill-posed and nonlinear inverse problem of ECT image reconstruction, a new ECT image reconstruction method based on fast linearized alternating direction method of multipliers (FLADMM) is proposed in this paper. On the basis of theoretical analysis of compressed sensing (CS), the data acquisition of ECT is regarded as a linear measurement process of permittivity distribution signal of pipe section. A new measurement matrix is designed and L1 regularization method is used to convert ECT inverse problem to a convex relaxation problem which contains prior knowledge. A new fast alternating direction method of multipliers which contained linearized idea is employed to minimize the objective function. Simulation data and experimental results indicate that compared with other methods, the quality and speed of reconstructed images are markedly improved. Also, the dynamic experimental results indicate that the proposed algorithm can fulfill the real-time requirement of ECT systems in the application.

Computational Data Mining in Cancer Bioinformatics and Cancer Epidemiology

Computational Data Mining in Cancer Bioinformatics and Cancer Epidemiology