A Sparse-Group Lasso

In many learning tasks with structural properties, structural sparsity methods help induce sparse models, usually leading to better interpretability and higher generalization performance. One popular approach is to use group sparsity regularization that enforces sparsity on the clustered groups of features, while another popular approach is to adopt graph sparsity regularization that considers sparsity on the link structure of graph embedded features. Both the group and graph structural properties co-exist in many applications. However, group sparsity and graph sparsity have not been considered simultaneously yet. In this paper, we propose a g2-regularization that takes group and graph sparsity into joint consideration, and present an effective approach for its optimization. Experiments on both synthetic and real data show that, enforcing group-graph sparsity lead to better performance than using group sparsity or graph sparsity only.

We present a multi-parameter inverse scattering method with group sparsity constraint, which improves the image reconstruction results in case the images of different parameters have similar shape or structure. In inverse scattering imaging, the system of equations to be solved is often under-determined where the number of measurements is less than the number of image pixels. This becomes even worse for objects described by a multi-parameter material model where the total number of unknowns grows by a factor of the number of material parameters. However, in many cases, images of different material parameters, such as permittivity and conductivity, share similar structure or shape, where group sparsity regularization can be used to effectively capture this prior information. We compare the inverse scattering of 3-parameter dispersive material objects using Tikhonov, sparsity, and group-sparsity regularization. After minimizing to the same measurement objective function error, compared with Tikhonov and sparsity regularization, the group sparsity regularized inversion produces a better shape and contrast reconstruction of all 3 parameters and achieves the lowest overall image pixel value error.

In this paper, linear regression machine learning techniques are applied to determine the quality of green tea samples. The data set is obtained by applying Differential Pulse Voltammetry (DPV) on green tea samples using Epigallocatechin-3-gallate (EGCG) specific sensor based on Molecular Imprinted Polymer (MIP) technique. Multiple linear regression models have been developed using this dataset that gives more hidden insight of the dataset and helps to find the input feature importance out of it. Regularization techniques are applied on linear regression like Ridge regression (L2 Penalty), Lasso regression (L1 Penalty) and ElasticNet regression (combination of L1 and L2 Penalty) considered to reduce overfitting of the model and to provide better prediction. The variation of cross validation score vs regularization parameter for different regularized techniques of linear regression are also taken under consideration and best value of the regularization parameter is calculated to develop the model for getting better prediction with high accuracy. From the result obtained from model metrics, a clear picture is portrayed how lasso regression performs better than ridge regression for this dataset and eliminates the less important features to develop the model as sparsity can be useful in practice if we have a high dimensional dataset with many features that are not effective for modelling. The beauty of ElasticNet Regression model is also highlighted how both L1 and L2 penalty go hand in hand to give prediction at a high accuracy.

Various sparse regularizers have been applied to machine learning problems, among which structured sparsity has been proposed for a better adaption to structured data. In this paper, motivated by effectively classifying linked data (e.g. Web pages, tweets, articles with references, and biological network data) where a group structure exists over the whole dataset and links exist between specific samples, we propose a joint sparse representation model that combines group sparsity and graph sparsity, to select a small number of connected components from the graph of linked samples, meanwhile promoting the sparsity of edges that link samples from different groups in each connected component. Consequently, linked samples are selected from a few sparsely-connected groups. Both theoretical analysis and experimental results on four benchmark datasets show that the joint sparsity model outperforms traditional group sparsity model and graph sparsity model, as well as the latest group-graph sparsity model.

Sparse Learning for Neural Networks with A Generalized Sparse Regularization

Many studies in health services research rely on regression models with a large number of covariates or predictors. In this article, we introduce novel methodology to estimate and perform model selection for high-dimensional non-parametric multivariate regression problems, with application to many healthcare studies. We particularly focus on multi-responses or multi-task regression models. Because of the complexity of the dependence between predictors and the multiple responses, we exploit model selection approaches that consider various level of groupings between and within responses. The novelty of the method lies in its ability to account simultaneously for between and within group sparsity in the presence of non-linear effects. We also propose a new set of algorithms that can identify inactive and active predictors that are common to all responses or to a subset of responses. Our modeling approach is applied to uncover factors that impact healthcare expenditure for children insured through the Medicaid benefits program. We provide important findings on the association between healthcare expenditure and a large number of well-cited factors for two neighboring states, Georgia and North Carolina, which have similar demographics but different Medicaid systems. We also validate our methods with a benchmark cancer data set and simulated data examples.

Block-wise compressed image often suffers from the blocking artifacts. In this paper, we propose a novel deblocking scheme for compressed image, by combining image's sparse property and its self-similarity together, called group sparsity optimization. Instead of processing each image patch individually, in the proposed scheme, similar patches in one group are required to be well-represented on learned dictionary collaboratively, using group sparsity regularization. The group sparsity not only imposes every patch's representation to be sparse, bus also requires patches' coefficients in the group share the similar pattern. The experiment results on standard test images demonstrate that our scheme can improve the PSNR of the compressed images by an average of 1.25 dB, and outperform state of the art deblocking approaches.

A variety of high throughput genome-wide assays enable the exploration of genetic risk factors underlying complex traits. Although these studies have remarkable impact on identifying susceptible biomarkers, they suffer from issues such as limited sample size and low reproducibility. Combining individual studies of different genetic levels/platforms has the promise to improve the power and consistency of biomarker identification. In this paper, we propose a novel integrative method, namely sparse group multitask regression, for integrating diverse omics datasets, platforms, and populations to identify risk genes/factors of complex diseases. This method combines multitask learning with sparse group regularization, which will: (1) treat the biomarker identification in each single study as a task and then combine them by multitask learning; (2) group variables from all studies for identifying significant genes; (3) enforce sparse constraint on groups of variables to overcome the “small sample, but large variables” problem. We introduce two sparse group penalties: sparse group lasso and sparse group ridge in our multitask model, and provide an effective algorithm for each model. In addition, we propose a significance test for the identification of potential risk genes. Two simulation studies are performed to evaluate the performance of our integrative method by comparing it with conventional meta-analysis method. The results show that our sparse group multitask method outperforms meta-analysis method significantly. In an application to our osteoporosis studies, 7 genes are identified as significant genes by our method and are found to have significant effects in other three independent studies for validation. The most significant gene SOD2 has been identified in our previous osteoporosis study involving the same expression dataset. Several other genes such as TREML2, HTR1E, and GLO1 are shown to be novel susceptible genes for osteoporosis, as confirmed from other studies.

Recently, sparse representation has attracted a lot of interest in various areas. However, the standard sparse representation does not consider the intrinsic structure, i.e., the nonzero elements occur in clusters, called group sparsity. Furthermore, there is no dictionary learning method for group sparse representation considering the geometrical structure of space spanned by atoms. In this paper, we propose a novel dictionary learning method, called Dictionary Learning with Group Sparsity and Graph Regularization (DL-GSGR). First, the geometrical structure of atoms is modeled as the graph regularization. Then, combining group sparsity and graph regularization, the DL-GSGR is presented, which is solved by alternating the group sparse coding and dictionary updating. In this way, the group coherence of learned dictionary can be enforced small enough such that any signal can be group sparse coded effectively. Finally, group sparse representation with DL-GSGR is applied to 3-D medical image denoising and image fusion. Specifically, in 3-D medical image denoising, a 3-D processing mechanism (using the similarity among nearby slices) and temporal regularization (to perverse the correlations across nearby slices) are exploited. The experimental results on 3-D image denoising and image fusion demonstrate the superiority of our proposed denoising and fusion approaches.

High resolution and high quality are now the requirements in synthetic aperture radar (SAR) research. The sliding spotlight mode can obtain high azimuth resolution because of its large azimuth bandwidth. Group sparse penalty can effectively suppress azimuth ambiguities to improve image quality. Generalized mini-max concave (GMC) penalty is a kind of nonconvex penalty, which is widely used in SAR imaging. In this paper, a novel sliding spotlight SAR imaging method based on group sparsity and nonconvex regularization is proposed. Compared with matched filtering method, the proposed method can suppress noise and azimuth ambiguities. Both simulations and Qilu-1(QL-1) real SAR data experiments verify the effectiveness of the proposed method.

To the Editor: The incidence of type 1 diabetes (T1D) is increasing worldwide, especially in children ages 0–4 years, decreasing the mean age at diagnosis.1 Several environmental risk factors for T1D-related islet autoimmunity have been identified, including exposure to respiratory infections in very early life.2 In young children, ambient air pollution may exacerbate inflammation and promote respiratory diseases.3,4 We hypothesized that exposure to high levels of ambient air pollution is associated with earlier onset of T1D. We analyzed data of DiMelli, a population-based register monitoring incident diabetes in children and youths in Bavaria, Germany, since 2009.5 At the registration of each patient, a structured questionnaire is completed by the attending physician, and a blood sample is drawn, which is used to determine islet autoantibodies to insulin, glutamic acid decarboxylase, IA-2, or ZnT8. Here, we used data for 671 patients registered up to May 2013 (mean age at diagnosis: 9.6 years) who were positive for at least 1 islet autoantibody and whose residential addresses were available from the questionnaire. The concentrations of particulate matter with an aerodynamic diameter of <10 μm (PM10), nitrogen dioxide (NO2), PM2.5 and PM2.5 absorbance, normalized difference vegetation index (NDVI) as a measure of greenness, and distance to the nearest major at the residential addresses of patients were obtained from various sources (see online Supplementary material, https://links.lww.com/EDE/A875). These served as possible explanatory variables in linear regression and quantile regression6 models using the 10th, 30th, 50th (median), 70th, and 90th percentiles of age at diagnosis as the dependent variables. Models were adjusted for sex, parental education, family history of T1D, and patient's body mass index at diagnosis, and additionally for level of urbanization in sensitivity analyses. Exposure to high levels of PM10 and NO2 was associated with a shift in the 10th percentile of the age at diagnosis to lower values (−1.40 [95% confidence interval {CI}: −2.24, −0.56] years per 2 SD increase in PM10; −1.28 [95% CI: −2.14, −0.42] years per 2 SD increase in NO2), but not for higher percentiles or for the mean age at diagnosis (Figure 1). Considering that the 10th percentile of the age at diagnosis was 3.29 years in the whole dataset, the first 10% of children with high exposure (+2 SDs) to PM10 were predicted to develop T1D by the age of 1.89 years (ie, 3.29−1.40), whereas the 10th percentile in children with low exposure (−2 SDs) was estimated to be 4.69 years (ie, 3.29 + 1.40). No clear associations were observed for PM2.5, PM2.5 absorbance, NDVI, or the distance to the nearest major road. If we included the level of urbanization as an additional confounder, similar results were obtained, except for the association between PM2.5 and the 10th percentile of age at diagnosis (−1.37 [95% CI: −1.97, −0.77] per 2 SD increase). Boxplots of the inflammatory markers interleukin (IL)-1β, IL-6, IL-8, and tumor necrosis factor indicated no clear associations with manifestation age and PM10 exposure level (Supplementary Figure, https://links.lww.com/EDE/A875).FIGURE: Point estimates and 95% confidence intervals for differences in the age at diagnosis of type 1 diabetes (T1D) per 2 SD increases in PM10, NO2, NDVI, PM2.5, and PM2.5 absorbance, and per 500 m increase in distance to the nearest major road, with adjustment for sex, parental education, family history of T1D, and body mass index. The dots represent specific quantile regression estimates and are connected by dashes to visualise trends by age quantiles. The horizontal grey lines represent the linear regression coefficients and their respective confidence intervals. The horizontal line depicts y = 0 as a reference.Our findings indicate that high exposure to the traffic-related air pollutants PM10, NO2 and possibly PM2.5 accelerates the manifestation of T1D, but only in very young children. Interestingly, findings from a previous study suggested that PM10 was associated with an increased risk of T1D in children ages < 5 years.7 Our results were independent of urbanization level, indicating that air pollutants, not urbanization-related lifestyle habits or for example higher temperatures in urbanized areas, might be responsible for the observed associations. However, we did not observe any clear associations with inflammatory markers to show that air pollution accelerates the onset of T1D by inducing a more severe inflammatory state in young children. ACKNOWLEDGEMENTS We thank Ramona Puff, Anja Wosch, Kathrin Hofer, Claudia Matzke, Marlon Scholz, Vanessa Dietrich, and Petra Becker (Institute of Diabetes Re search, Helmholtz Zentrum München) for their contributions to registry coordination, patient recruitment, data management, and expert technical assistance. We also thank all of the participating clinics and investigators for their support in recruiting participants in DiMelli. A list of the main participating clinics can be found in the online Supplementary material, https://links.lww.com/EDE/A875. Andreas Beyerlein Miriam Krasmann Institute of Diabetes Research Helmholtz Zentrum München and Forschergruppe Diabetes Klinikum rechts der Isar Technische Universität München Neuherberg, Germany Elisabeth Thiering Institute of Epidemiology I Helmholtz Zentrum München Neuherberg, Germany Dennis Kusian Institute of Diabetes Research Helmholtz Zentrum München and Forschergruppe Diabetes Klinikum rechts der Isar Technische Universität München Neuherberg, Germany Iana Markevych Institute of Epidemiology I Helmholtz Zentrum München Neuherberg, Germany Orietta D'Orlando Institute of Diabetes Research Helmholtz Zentrum München and Forschergruppe Diabetes Klinikum rechts der Isar Technische Universität München Neuherberg, Germany Katharina Warncke Department of Pediatrics Klinikum rechts der Isar Technische Universität München Munich, Germany Susanne Jochner Department of Ecology and Ecosystem Management Ecoclimatology Technische Universität München Freising, Germany Joachim Heinrich Institute of Epidemiology I Helmholtz Zentrum München Neuherberg, Germany Anette-Gabriele Ziegler Institute of Diabetes Research Helmholtz Zentrum München and Forschergruppe Diabetes Klinikum rechts der Isar Technische Universität München Neuherberg, Germany [email protected]

Compressed sensing MRI based on image decomposition model and group sparsity

The article is concerned with the use of Markov chain Monte Carlo methods for posterior sampling in Bayesian nonparametric mixture models.In particular, we consider the problem of slice sampling mixture models for a large class of mixing measures generalizing the celebrated Dirichlet process. Such a class of measures, known in the literature as σ-stable Poisson-Kingman models, includes as special cases most of the discrete priors currently known in Bayesian nonparametrics, for example, the two-parameter Poisson-Dirichlet process and the normalized generalized Gamma process. The proposed approach is illustrated on some simulated data examples. This article has online supplementary material.

Current high-throughput sequencing technologies allow cost-efficient genotyping of millions of single nucleotide polymorphisms (SNPs) for hundreds of samples. However, the tools that are currently available for constructing linkage maps are not well suited for large datasets. Linkage maps of large datasets would be helpful in de novo genome assembly by facilitating comprehensive genome validation and refinement by enabling chimeric scaffold detection, as well as in family-based linkage and association studies, quantitative trait locus mapping, analysis of genome synteny and other complex genomic data analyses. We describe a novel tool, called Lepidoptera-MAP (Lep-MAP), for constructing accurate linkage maps with ultradense genome-wide SNP data. Lep-MAP is fast and memory efficient and largely automated, requiring minimal user interaction. It uses simultaneously data on multiple outbred families and can increase linkage map accuracy by taking into account achiasmatic meiosis, a special feature of Lepidoptera and some other taxa with no recombination in one sex (no recombination in females in Lepidoptera). We demonstrate that Lep-MAP outperforms other methods on real and simulated data. We construct a genome-wide linkage map of the Glanville fritillary butterfly (Melitaea cinxia) with over 40 000 SNPs. The data were generated with a novel in-house SOLiD restriction site-associated DNA tag sequencing protocol, which is described in the online supplementary material. Java source code under GNU general public license with the compiled classes and the datasets are available from http://sourceforge.net/users/lep-map.

We propose a denoising method by integrating group sparsity and TV regularization based on self-similarity of the image blocks. By using the block matching technique, we introduce some local SVD operators to get a good sparsity representation for the groups of the image blocks. The sparsity regularization and TV are unified in a variational problem and each of the subproblems can be efficiently optimized by splitting schemes. The proposed algorithm mainly contains the following four steps: block matching, basis vectors updating, sparsity regularization and TV smoothing. The self-similarity information of the image is assembled by the block matching step. By concatenating all columns of the similar image block together, we get redundancy matrices whose column vectors are highly correlated and should have sparse coefficients after a proper transformation. In contrast with many transformation based denoising methods such as BM3D with fixed basis vectors, we update local basis vectors derived from the SVD to enforce the sparsity representation. This step is equivalent to a dictionary learning procedure. With the sparsity regularization step, one can remove the noise efficiently and keep the texture well. The TV regularization step can help us to reduced the artifacts caused by the image block stacking. Besides, we mathematically show the convergence of the algorithms when the proposed model is convex (with $$p=1$$ ) and the bases are fixed. This implies the iteration adopted in BM3D is converged, which was not mathematically shown in the BM3D method. Numerical experiments show that the proposed method is very competitive and outperforms state-of-the-art denoising methods such as BM3D.