Iteratively Reweighted Research Articles

Convergence of Proximal Iteratively Reweighted Nuclear Norm Algorithm for Image Processing.

The nonsmooth and nonconvex regularization has many applications in imaging science and machine learning research due to its excellent recovery performance. A proximal iteratively reweighted nuclear norm algorithm has been proposed for the nonsmooth and nonconvex matrix minimizations. In this paper, we aim to investigate the convergence of the algorithm. With the Kurdyka-Łojasiewicz property, we prove the algorithm globally converges to a critical point of the objective function. The numerical results presented in this paper coincide with our theoretical findings.

Robust Principal Component Analysis: An IRLS Approach

The modern problems of optimization, estimation, signal processing, and image recognition deal with data of huge dimensions. It is important to develop effective methods and algorithms for such problems. An important idea is the construction of low-dimension approximations to large-scale data. One of the most popular methods for this purpose is the principal component analysis (PCA), which is, however, sensitive to outliers. There exist numerous robust versions of PCA, relying on sparsity ideas and ℓ1 techniques. The present paper offers another approach to robust PCA exploiting Huber’s functions and numerical implementation based on the Iterative Reweighted Least Squares (IRLS) method.

Object-Oriented Island Land Cover Change Detection by Iteratively Reweighted Multivariate Statistical Analysis

ABSTRACTIn studies of changes in island geographic information, a key topic is island land cover change detection using multitemporal remote sensing images; such information can provide good reference for island exploitation, marine resource conservation, marine economic development, and national defense building. This study proposes a fast and easy-to-implement algorithm of object-oriented change detection based on iteratively reweighted multivariate statistical analysis with high-resolution remote sensing images and vector data, improving effects of normal multivariate alternative detection by combining object-oriented technology with probability density function of chi-square distribution. This method builds vectors in feature space with spectral, texture, and spatial structure information of objects segmented based on boundaries and categories of features in vector data. By feature optimization and iterative extension to multivariate statistical analysis for change detection, change regions can be extracted. Experiments on Zhaoshu Island area show that the proposed method is highly effective and accurate in detection of change regions.

A General Framework for Sparsity Regularized Feature Selection via Iteratively Reweighted Least Square Minimization

A variety of feature selection methods based on sparsity regularization have been developed with different loss functions and sparse regularization functions. Capitalizing on the existing sparsity regularized feature selection methods, we propose a general sparsity feature selection (GSR-FS) algorithm that optimizes a ℓ2,r (0 < r ≤ 2) based loss function with a ℓ2,p-norm (0 < p ≤ 2) sparse regularization. The ℓ2,r-norm (0 <

Efficient Iris Localization via Optimization Model

Iris localization is one of the most important processes in iris recognition. Because of different kinds of noises in iris image, the localization result may be wrong. Besides this, localization process is time-consuming. To solve these problems, this paper develops an efficient iris localization algorithm via optimization model. Firstly, the localization problem is modeled by an optimization model. Then SIFT feature is selected to represent the characteristic information of iris outer boundary and eyelid for localization. And SDM (Supervised Descent Method) algorithm is employed to solve the final points of outer boundary and eyelids. Finally, IRLS (Iterative Reweighted Least-Square) is used to obtain the parameters of outer boundary and upper and lower eyelids. Experimental result indicates that the proposed algorithm is efficient and effective.

Efficient l q norm based sparse subspace clustering via smooth IRLS and ADMM

Recently, sparse subspace clustering, as a subspace learning technique, has been successfully applied to several computer vision applications, e.g. face clustering and motion segmentation. The main idea of sparse subspace clustering is to learn an effective sparse representation that are used to construct an affinity matrix for spectral clustering. While most of existing sparse subspace clustering algorithms and its extensions seek the forms of convex relaxation, the use of non-convex and non-smooth lq(0 < q < 1) norm has demonstrated better recovery performance. In this paper we propose an lq norm based Sparse Subspace Clustering method (lqSSC), which is motivated by the recent work that lq norm can enhance the sparsity and make better approximation to l0 than l1. However, the optimization of lq norm with multiple constraints is much difficult. To solve this non-convex problem, we make use of the Alternating Direction Method of Multipliers (ADMM) for solving the lq norm optimization, updating the variables in an alternating minimization way. ADMM splits the unconstrained optimization into multiple terms, such that the lq norm term can be solved via Smooth Iterative Reweighted Least Square (SIRLS), which converges with guarantee. Different from traditional IRLS algorithms, the proposed algorithm is based on gradient descent with adaptive weight, making it well suit for general sparse subspace clustering problem. Experiments on computer vision tasks (synthetic data, face clustering and motion segmentation) demonstrate that the proposed approach achieves considerable improvement of clustering accuracy than the convex based subspace clustering methods.

Is the Bifactor Model a Better Model or Is It Just Better at Modeling Implausible Responses? Application of Iteratively Reweighted Least Squares to the Rosenberg Self-Esteem Scale

ABSTRACTAlthough the structure of the Rosenberg Self-Esteem Scale (RSES) has been exhaustively evaluated, questions regarding dimensionality and direction of wording effects continue to be debated. To shed new light on these issues, we ask (a) for what percentage of individuals is a unidimensional model adequate, (b) what additional percentage of individuals can be modeled with multidimensional specifications, and (c) what percentage of individuals respond so inconsistently that they cannot be well modeled? To estimate these percentages, we applied iteratively reweighted least squares (IRLS) to examine the structure of the RSES in a large, publicly available data set. A distance measure, ds, reflecting a distance between a response pattern and an estimated model, was used for case weighting. We found that a bifactor model provided the best overall model fit, with one general factor and two wording-related group factors. However, on the basis of dr values, a distance measure based on individual residuals, we concluded that approximately 86% of cases were adequately modeled through a unidimensional structure, and only an additional 3% required a bifactor model. Roughly 11% of cases were judged as “unmodelable” due to their significant residuals in all models considered. Finally, analysis of ds revealed that some, but not all, of the superior fit of the bifactor model is owed to that model's ability to better accommodate implausible and possibly invalid response patterns, and not necessarily because it better accounts for the effects of direction of wording.

Rapid Land Cover Map Updates Using Change Detection and Robust Random Forest Classifiers

The paper evaluated the Landsat Automated Land Cover Update Mapping (LALCUM) system designed to rapidly update a land cover map to a desired nominal year using a pre-existing reference land cover map. The system uses the Iteratively Reweighted Multivariate Alteration Detection (IRMAD) to identify areas of change and no change. The system then automatically generates large amounts of training samples (n &gt; 1 million) in the no-change areas as input to an optimized Random Forest classifier. Experiments were conducted in the KwaZulu-Natal Province of South Africa using a reference land cover map from 2008, a change mask between 2008 and 2011 and Landsat ETM+ data for 2011. The entire system took 9.5 h to process. We expected that the use of the change mask would improve classification accuracy by reducing the number of mislabeled training data caused by land cover change between 2008 and 2011. However, this was not the case due to exceptional robustness of Random Forest classifier to mislabeled training samples. The system achieved an overall accuracy of 65%–67% using 22 detailed classes and 72%–74% using 12 aggregated national classes. “Water”, “Plantations”, “Plantations—clearfelled”, “Orchards—trees”, “Sugarcane”, “Built-up/dense settlement”, “Cultivation—Irrigated” and “Forest (indigenous)” had user’s accuracies above 70%. Other detailed classes (e.g., “Low density settlements”, “Mines and Quarries”, and “Cultivation, subsistence, drylands”) which are required for operational, provincial-scale land use planning and are usually mapped using manual image interpretation, could not be mapped using Landsat spectral data alone. However, the system was able to map the 12 national classes, at a sufficiently high level of accuracy for national scale land cover monitoring. This update approach and the highly automated, scalable LALCUM system can improve the efficiency and update rate of regional land cover mapping.

Imaging brain source extent from EEG/MEG by means of an iteratively reweighted edge sparsity minimization (IRES) strategy

Estimating extended brain sources using EEG/MEG source imaging techniques is challenging. EEG and MEG have excellent temporal resolution at millisecond scale but their spatial resolution is limited due to the volume conduction effect. We have exploited sparse signal processing techniques in this study to impose sparsity on the underlying source and its transformation in other domains (mathematical domains, like spatial gradient). Using an iterative reweighting strategy to penalize locations that are less likely to contain any source, it is shown that the proposed iteratively reweighted edge sparsity minimization (IRES) strategy can provide reasonable information regarding the location and extent of the underlying sources. This approach is unique in the sense that it estimates extended sources without the need of subjectively thresholding the solution. The performance of IRES was evaluated in a series of computer simulations. Different parameters such as source location and signal-to-noise ratio were varied and the estimated results were compared to the targets using metrics such as localization error (LE), area under curve (AUC) and overlap between the estimated and simulated sources. It is shown that IRES provides extended solutions which not only localize the source but also provide estimation for the source extent. The performance of IRES was further tested in epileptic patients undergoing intracranial EEG (iEEG) recording for pre-surgical evaluation. IRES was applied to scalp EEGs during interictal spikes, and results were compared with iEEG and surgical resection outcome in the patients. The pilot clinical study results are promising and demonstrate a good concordance between noninvasive IRES source estimation with iEEG and surgical resection outcomes in the same patients. The proposed algorithm, i.e. IRES, estimates extended source solutions from scalp electromagnetic signals which provide relatively accurate information about the location and extent of the underlying source.

Generalized Degrees of Freedom

Two popular regression methods for the multiplicative-error model are the Minimum-Unbiased-Percent Error and Minimum-Percentage Error under the Zero-Percentage Bias methods. The Minimum-Unbiased-Percent Error method, an Iteratively Reweighted Least Squares regression, does not use any constraints, while the Minimum-Percentage Error under the Zero-Percentage Bias method requires a constraint as part of the curve-fitting process. However, Minimum-Percentage Error under the Zero-Percentage Bias users do not adjust the degrees of freedom to account for constraints included in the regression process. As a result, fit statistics for the Minimum-Percentage Error under the Zero-Percentage bias equations, e.g., the standard percent error and generalized R2, can be incorrect and misleading. This results in incompatible fit statistics between Minimum-Percentage Error under the Zero-Percentage Bias and Minimum-Unbiased-Percent Error equations. This article details why degrees of freedom should be adjusted and recomme...

A parallel min-cut algorithm using iteratively reweighted least squares targeting at problems with floating-point edge weights

We formulate and derive a Parallel Iteratively Reweighted least squares Min-Cut solver (PIRMCut).We propose a novel two-level rounding procedure and prove a generalized Cheeger-type inequality.We developed an MPI based parallel implementation of PIRMCut.We demonstrate the parallel scalability of our PIRMCut implementation.We demonstrate PIRMCut significant speed improvement over a serial code. We present a parallel algorithm for the undirected st min-cut problem with floating-point valued edge weights. Our overarching algorithm uses an iteratively reweighted least squares framework. Specifically, this algorithm generates a sequence of Laplacian linear systems, which are solved in parallel. The iterative nature of our algorithm enables us to trade off solution quality for execution time, which is distinguished from those purely combinatorial algorithms that only produce solutions at optimum. We also propose a novel two-level rounding procedure that helps to enhance the quality of the approximate min-cut solution output by our algorithm. Our overall implementation, including the rounding procedure, demonstrates significant speed improvement over a state-of-the-art serial solver, where it could be up to 200 times faster on commodity platforms.

Adaptive noise dictionary construction via IRRPCA for face recognition

Recently, regression analysis has become a popular method for face recognition. Various robust regression methods have been proposed to handle with different recognition tasks. In this paper, we attempt to achieve this goal by the strategy of adding an adaptive noise dictionary (AND) to the training samples. In contrast to the previous methods, the noise dictionary (ND) is adaptive to different kinds of noise and extracted automatically. To get an effective noise dictionary, the Iteratively Reweighted Robust Principal Component Analysis (IRRPCA) is proposed. A corresponding classifier based on linear regression is presented for recognition. As this adaptive noise dictionary can describe the noise distribution of testing samples, it is robust to various kinds of noise and applicable for recognition tasks with occluded or corrupted images. This method is also extended to deal with misaligned images. Experiments are conducted on AR, Yale B, CMU PIE, CMU Multi-Pie, LFW and Pubfig databases to verify the robustness of our method to variations in occlusion, corruption, illumination, misalignment, etc.

A time series analysis of urbanization induced land use and land cover change and its impact on land surface temperature with Landsat imagery

Abstract The relationship between land use and land cover (LULC) patterns and thermal characteristic has long been studied for examining the impact of urbanization on urban thermal environment. Previous studies were inclined to the use of one or a few images without fully considering the temporal domain of the reflective and thermal infrared data on board Landsat sensors. This paper took the Atlanta metropolitan area as a case study to illustrate LULC change and its impact on land surface temperature (LST) variations. The Landsat L1T (Standard terrain correction) images from TM/ETM + from 1984 to 2011 (507 in total) were downloaded through the USGS online portal and consistently calibrated to surface reflectance and brightness temperature (BT). The cloud-, cloud shadow-, and snow-contaminated pixels were excluded in the analysis according to the metadata, and a further screening procedure based on the RIRLS (Robust Iteratively Reweighted Least Squares) technique was implemented. The time series LSTs (TSLSTs) was derived using the single channel algorithm because it required only the parameters of water vapor and land surface emissivity and had a reported error close to 1 K. The LULC classification and change detection was accomplished by using the Continuous Change Detection and Classification (CCDC) algorithm. The TSLSTs were further decomposed into the seasonal and trend components by an additive model. Results showed that the overall LULC classification and change detection accuracies were 89% and 92%, respectively. Urban growth was mainly observed in Fulton County and Gwinnett County. High-intensity urban land had the largest mean LST value of 294.9 K and yearly amplitude of 17.4 K. A comparison of the trend component between urban and non-urban land covers showed a difference of 1.8 K per decade. Next, temporal thermal signatures were created to characterize and quantify the impact of urban LULC changes. Decomposition analysis showed that the conversion of evergreen forest to medium-intensity urban land generated the largest difference in annual LST variation (5.7 K) and the largest trend difference (0.0004 K/day).

Far Field Splitting by Iteratively Reweighted $\ell^1$ Minimization

The aim of far field splitting for time-harmonic acoustic or electromagnetic waves is to decompose the far field of a wave radiated by an ensemble of several compactly supported sources into the individual far field components radiated by each of these sources separately.Without further assumptions this is an ill-posed inverse problem.Observing that far fields radiated by compactly supported sources have nearly sparse representations with respect to certain suitably transformed Fourier bases that depend on the approximate source locations, we develop an $\ell^1$ characterization of these far fields and use it to reformulate the far field splitting problem as a weighted $\ell^1$ minimization problem in the spirit of basis pursuit.To this end we assume that some a priori information on the locations of the individual source components is available. We prove that the unique solution to the weighted $\ell^1$ minimization problem coincides with the solution to the far field splitting problem, and we discuss its numerical approximation. Furthermore, we propose an iterative strategy to successively improve the required a priori information by solving a sequence of these weighted $\ell^1$ minimization problems, where estimates of the approximate locations of the individual source components that are used as a priori information for the next iteration are computed from the value of the current solution.This also gradually decreases the ill-posedness of the splitting problem, and it significantly improves the quality of the reconstructions.We present a series of numerical examples to demonstrate the performance of this algorithm.

Nonconvex Nonsmooth Low Rank Minimization via Iteratively Reweighted Nuclear Norm.

The nuclear norm is widely used as a convex surrogate of the rank function in compressive sensing for low rank matrix recovery with its applications in image recovery and signal processing. However, solving the nuclear norm-based relaxed convex problem usually leads to a suboptimal solution of the original rank minimization problem. In this paper, we propose to use a family of nonconvex surrogates of L0-norm on the singular values of a matrix to approximate the rank function. This leads to a nonconvex nonsmooth minimization problem. Then, we propose to solve the problem by an iteratively re-weighted nuclear norm (IRNN) algorithm. IRNN iteratively solves a weighted singular value thresholding problem, which has a closed form solution due to the special properties of the nonconvex surrogate functions. We also extend IRNN to solve the nonconvex problem with two or more blocks of variables. In theory, we prove that the IRNN decreases the objective function value monotonically, and any limit point is a stationary point. Extensive experiments on both synthesized data and real images demonstrate that IRNN enhances the low rank matrix recovery compared with the state-of-the-art convex algorithms.

Multiple task learning with flexible structure regularization

Due to the theoretical advances and empirical successes, Multi-task Learning (MTL) has become a popular design paradigm for training a set of tasks jointly. Through exploring the hidden relationships among multiple tasks, many MTL algorithms have been developed to enhance learning performance. In general, the complicated hidden relationships can be considered as a combination of two key structural elements: task grouping and task outlier. Based on such task relationship, here we propose a generic MTL framework with flexible structure regularization, which aims in relaxing any type of specific structure assumptions. In particular, we directly impose a joint ℓ11/ℓ21-norm as the regularization term to reveal the underlying task relationship in a flexible way. Such a flexible structure regularization term takes into account any convex combination of grouping and outlier structural characteristics among the multiple tasks. In order to derive efficient solutions for the generic MTL framework, we develop two algorithms, i.e., the Iteratively Reweighted Least Square (IRLS) method and the Accelerated Proximal Gradient (APG) method, with different emphasis and strength. In addition, the theoretical convergence and performance guarantee are analyzed for both algorithms. Finally, extensive experiments over both synthetic and real data, and the comparisons with several state-of-the-art algorithms demonstrate the superior performance of the proposed generic MTL method.

On the Convergence of Alternating Minimization for Convex Programming with Applications to Iteratively Reweighted Least Squares and Decomposition Schemes

This paper is concerned with the alternating minimization (AM) method for solving convex minimization problems where the decision variables vector is split into two blocks. The objective function is a sum of a differentiable convex function and a separable (possibly) nonsmooth extended real-valued convex function, and consequently constraints can be incorporated. We analyze the convergence rate of the method and establish a nonasymptotic sublinear rate of convergence where the multiplicative constant depends on the minimal block Lipschitz constant. We then analyze the iteratively reweighted least squares (IRLS) method for solving convex problems involving sums of norms. Based on the results derived for the AM method, we establish a nonasymptotic sublinear rate of convergence of the IRLS method. In addition, we show an asymptotic rate of convergence whose efficiency estimate does not depend on the data of the problem. Finally, we study the convergence properties of a decomposition-based approach designed t...

Structural source identification using a generalized Tikhonov regularization

This paper addresses the problem of identifying mechanical exciting forces from vibration measurements. The proposed approach is based on a generalized Tikhonov regularization that allows taking into account prior information on the measurement noise as well as on the main characteristics of sources to identify like its sparsity or regularity. To solve such a regularization problem efficiently, a Generalized Iteratively Reweighted Least-Squares (GIRLS) algorithm is introduced. Proposed numerical and experimental validations reveal the crucial role of prior information in the quality of the source identification and the performance of the GIRLS algorithm.

Automated cloud, cloud shadow, and snow detection in multitemporal Landsat data: An algorithm designed specifically for monitoring land cover change

We developed a new algorithm called Tmask (multiTemporal mask) for automated masking of cloud, cloud shadow, and snow for multitemporal Landsat images. This algorithm consists of two steps. The first step is based on a single-date algorithm called Fmask (Function of mask) that initially screens most of the clouds, cloud shadows, and snow. The second step benefits from the extra temporal information from the remaining “clear” pixels and further improves the cloud, cloud shadow, and snow mask. Three Top Of Atmosphere (TOA) reflectance bands (Bands 2, 4, and 5 — Landsat-7 band numbering) are used in a Robust Iteratively Reweighted Least Squares (RIRLS) method to estimate a time series model for each pixel. By comparing model estimates with Landsat observations for the three spectral bands, the Tmask algorithm is capable of detecting any remaining clouds, cloud shadows, and snow for an entire stack of Landsat images. Generally, this algorithm will not falsely identify land cover changes as clouds, cloud shadows, or snow, as it is capable of modeling land cover change. The multitemporal images also provide extra information for better discrimination of cloud and snow, which is difficult for single-date algorithm. A snow threshold is derived for Band 5 TOA reflectance for each pixel at each specific time based on a modified Norwegian Linear Reflectance-to-Snow-Cover (NLR) algorithm. By comparing the results of Tmask with a single-date algorithm (Fmask) for multitemporal Landsat images located at Path 12 Row 31, significant improvements are observed for identification of clouds, cloud shadows, and snow. The most significant improvement is observed for cloud shadow detection. Many of the errors in cloud, cloud shadow, and snow detection observed in Fmask are corrected by the Tmask algorithm. The goal is development of a cloud, cloud shadow, and snow detection algorithm that results in a multitemporal stack of images that is free of “noise” factors and thus suitable for detection of land cover change.

QTL mapping in autotetraploids using SNP dosage information.

Key message Dense linkage maps derived by analysing SNP dosage in autotetraploids provide detailed information about the location of, and genetic model at, quantitative trait loci. Recent developments in sequencing and genotyping technologies enable researchers to generate high-density single nucleotide polymorphism (SNP) genotype data for mapping studies. For polyploid species, the SNP genotypes are informative about allele dosage, and Hackett et al. (PLoS ONE 8:e63939, 2013) presented theory about how dosage information can be used in linkage map construction and quantitative trait locus (QTL) mapping for an F1 population in an autotetraploid species. Here, QTL mapping using dosage information is explored for simulated phenotypic traits of moderate heritability and possibly non-additive effects. Different mapping strategies are compared, looking at additive and more complicated models, and model fitting as a single step or by iteratively re-weighted modelling. We recommend fitting an additive model without iterative re-weighting, and then exploring non-additive models for the genotype means estimated at the most likely position. We apply this strategy to re-analyse traits of high heritability from a potato population of 190 F1 individuals: flower colour, maturity, height and resistance to late blight (Phytophthora infestans (Mont.) de Bary) and potato cyst nematode (Globodera pallida), using a map of 3839 SNPs. The approximate confidence intervals for QTL locations have been improved by the detailed linkage map, and more information about the genetic model at each QTL has been revealed. For several of the reported QTLs, candidate SNPs can be identified, and used to propose candidate trait genes. We conclude that the high marker density is informative about the genetic model at loci of large effects, but that larger populations are needed to detect smaller QTLs.Electronic supplementary materialThe online version of this article (doi:10.1007/s00122-014-2347-2) contains supplementary material, which is available to authorized users.