Penalty Functionals Research Articles

Convergence analysis of iteratively regularized Landweber iteration with uniformly convex constraints in Banach spaces

In Banach spaces, the convergence analysis of iteratively regularized Landweber iteration (IRLI) is recently studied via conditional stability estimates. But the formulation of IRLI does not include general non-smooth convex penalty functionals, which is essential to capture special characteristics of the sought solution. In this paper, we formulate a generalized form of IRLI so that its formulation includes general non-smooth uniformly convex penalty functionals. We study the convergence analysis and derive the convergence rates of the generalized method solely via conditional stability estimates in Banach spaces for both the perturbed and unperturbed data. We also discuss few examples of inverse problems on which our method is applicable.

Generalized Inexact Newton-Landweber Iteration for Possibly Non-Smooth Inverse Problems in Banach Spaces

In this paper, we consider a generalized inexact Newton-Landweber iteration to solve nonlinear ill-posed inverse problems in Banach spaces, where the forward operator might not be Gâteaux differentiable. The method is designed with non-smooth convex penalty terms, including L1-like and total variation-like penalty functionals, to capture special features of solutions such as sparsity and piecewise constancy. Furthermore, the inaccurate inner solver is incorporated into the minimization problem in each iteration step. Under some assumptions, based on ε-subdifferential, we establish the convergence analysis of the proposed method. Finally, some numerical simulations are provided to illustrate the effectiveness of the method for solving both smooth and non-smooth nonlinear inverse problems.

Convex regularization in statistical inverse learning problems

We consider a statistical inverse learning problem, where the task is to estimate a function $ f $ based on noisy point evaluations of $ Af $, where $ A $ is a linear operator. The function $ Af $ is evaluated at i.i.d. random design points $ u_n $, $ n = 1, ..., N $ generated by an unknown general probability distribution. We consider Tikhonov regularization with general convex and $ p $-homogeneous penalty functionals and derive concentration rates of the regularized solution to the ground truth measured in the symmetric Bregman distance induced by the penalty functional. We derive concrete rates for Besov norm penalties and numerically demonstrate the correspondence with the observed rates in the context of X-ray tomography.

Timely Updates With Priorities: Lexicographic Age Optimality

In this paper, we consider a scheduling problem, in which several streams of status update packets with different priority levels are sent through a shared channel to their destinations. We introduce a notion of <i>lexicographic age optimality</i>, or simply <i>lex-age-optimality</i>, to evaluate the performance of multi-class status update policies. In particular, a lex-age-optimal scheduling policy first minimizes the Age of Information (AoI) metrics for high-priority streams, and then, within the set of optimal policies for high-priority streams, achieves the minimum AoI metrics for low-priority streams. We propose a new scheduling policy named Preemptive Priority, Maximum Age First, Last-Generated, First-Served (PP-MAF-LGFS), and prove that the PP-MAF-LGFS scheduling policy is lex-age-optimal. This result holds (i) for minimizing any time-dependent, symmetric, and non-decreasing age penalty function; (ii) for minimizing any non-decreasing functional of the stochastic process formed by the age penalty function; and (iii) for the cases where different priority classes have distinct arrival traffic patterns, age penalty functions, and age penalty functionals. For example, the PP-MAF-LGFS scheduling policy is lex-age-optimal for minimizing the probability of age violation of a high-priority stream and the time-average age of a low-priority stream. Numerical results are provided to illustrate our theoretical findings.

Level Set-Based Structural Optimization With Uniform Wall Thickness for Wire-Fed Metal Additive Manufacturing

Abstract A new three-dimensional structural optimization is presented based on the level set method to obtain favorable designs for wire-fed metal additive manufacturing with uniform wall thickness. By exploiting the signed distance nature of a level set function, a structure under design is always defined as a thin domain with uniform thickness without employing any constrains or penalty functionals. The boundary surfaces of a thin-walled domain are defined as the surfaces with level set values of ±t/2(t: wall thickness). Design velocity can be represented in terms of curvatures of the zero-level-set surface, extended to level set grids in the narrow band. Therefore, the calculation of accurate curvatures on the zero-level set is crucial for correct design sensitivities. In this investigation, mean and Gaussian curvatures at a point on the triangle mesh of the discretized zero-level set are calculated by spatial averages over the Voronoi cell of the point, by which the sensitivity of a material volume can be calculated with optimal accuracy. To address the high computational cost by a dense regular mesh for representing thin walls, degrees of freedom in void regions is mostly removed. Design examples of beams and a T-joint structure with uniform thickness are presented to verify the effectiveness of the proposed method.

A novel two-point gradient method for regularization of inverse problems in Banach spaces

In this paper, we introduce a novel two-point gradient method for solving the ill-posed problems in Banach spaces and study its convergence analysis. The method is based on the well known iteratively regularized Landweber iteration method together with an extrapolation strategy. The general formulation of iteratively regularized Landweber iteration method in Banach spaces excludes the use of certain functions such as total variation like penalty functionals, functions, etc. The novel scheme presented in this paper allows to use such non-smooth penalty terms that can be helpful in practical applications involving the reconstruction of several important features of solutions such as piecewise constancy and sparsity. We carefully discuss the choices for important parameters, such as combination parameters and step sizes involved in the design of the method. Additionally, we discuss an example to validate our assumptions.

Inexact Newton regularization in Banach spaces based on two-point gradient method with uniformly convex penalty terms

In this paper, we propose and analyze a new inexact Newton regularization method for solving nonlinear inverse problems in Banach spaces. The method consists of an outer Newton iteration and an inner scheme which provides increments by applying a regularization technique to the local linearized equation around the current iterate. In order to accelerate the convergence, we employ a two-point gradient method as inner regularization scheme, which is based on the Landweber iteration and an extrapolation strategy. The penalty term is allowed to be non-smooth, including L1-like and total variation-like penalty functionals, to detect special features of solutions such as sparsity and piecewise constancy. We present, under certain assumptions, the detailed analysis of convergence and regularization properties of the method. Finally, some numerical experiments on elliptic parameter identification and Robin coefficient reconstruction problems are provided to illustrate the effectiveness of reconstructing the properties of sought solutions and the acceleration effect of the method.

An accelerated Kaczmarz type method for nonlinear inverse problems in Banach spaces with uniformly convex penalty

In this paper, we propose and analyze a novel Kaczmarz type method for solving inverse problems which can be written as systems of nonlinear operator equations in Banach spaces. The proposed method is formulated by combining homotopy perturbation iteration and Kaczmarz approach with uniformly convex penalty terms. The penalty term is allowed to be non-smooth, including the L1 and the total variation like penalty functionals, to reconstruct special features of solutions such as sparsity and piecewise constancy. To accelerate the iteration, we introduce a sophisticated rule to determine the step sizes per iteration. Under certain conditions, we present the convergence result of the proposed method in the exact data case. When the data is given approximately, together with a suitable stopping rule, we establish the stability and regularization properties of the method. Finally, some numerical experiments on parameter identification in partial differential equations by boundary as well as interior measurements are provided to validate the effectiveness of the proposed method.

Fast subspace optimization method for nonlinear inverse problems in Banach spaces with uniformly convex penalty terms

In this paper, we consider the subspace optimization method for solving nonlinear inverse problems in Banach spaces, which is based on the sequential Bregman projections with uniformly convex penalty term. The penalty term is allowed to be non-smooth, including L1 and total variation like penalty functionals, to reconstruct the special features of solutions such as sparsity and discontinuities. Instead of just utilizing the current gradient like the Landweber iteration, the method uses multiple search directions in each iteration to accelerate the convergence. Moreover, their step lengths are calculated by the projection onto the subspace that contains the solution set of the unperturbed problem. Under certain assumptions, we present the detailed convergence analysis when the data is given exactly. For the data containing noise, we use the discrepancy principle as stopping rule and then obtain the regularization result of the method. Finally, some numerical simulations for parameter identification problems are provided to illustrate the effectiveness of capturing the property of exact solutions and the acceleration effect of the method.

Regularization of inverse problems by two-point gradient methods in Banach spaces

In this paper, we propose and analyze a two-point gradient method for solving inverse problems in Banach spaces which is based on the Landweber iteration and an extrapolation strategy. The method allows to use non-smooth penalty terms, including the $$L^1$$ -like and the total variation-like penalty functionals, which are significant in reconstructing special features of solutions such as sparsity and piecewise constancy in practical applications. The design of the method involves the choices of the step sizes and the combination parameters which are carefully discussed. Numerical simulations are presented to illustrate the effectiveness of the proposed method.

Surrogate-Based Robust Design for a Non-Smooth Radiation Source Detection Problem

In this paper, we develop and numerically illustrate a robust sensor network design to optimally detect a radiation source in an urban environment. This problem exhibits several challenges: penalty functionals are non-smooth due to the presence of buildings, radiation transport models are often computationally expensive, sensor locations are not limited to a discrete number of points, and source intensity and location responses, based on a fixed number of sensors, are not unique. We consider a radiation source located in a prototypical 250 m × 180 m urban setting. To address the non-smooth properties of the model and computationally expensive simulation codes, we employ a verified surrogate model based on radial basis functions. Using this surrogate, we formulate and solve a robust design problem that is optimal in an average sense for detecting source location and intensity with minimized uncertainty.

An iteration regularization method with general convex penalty for nonlinear inverse problems in Banach spaces

This paper investigates the construction, analysis and implementation of a novel iterative regularization scheme with general convex penalty term for nonlinear inverse problems in Banach spaces. The method is formulated by combining the homotopy perturbation technique. The penalty term is allowed to be nonsmooth to include L1-norm or total variation-like penalty functionals, especially suitable for detecting the special features of the sought solutions such as sparsity or piecewise constant. By using tools from convex analysis in Banach spaces, we establish the detailed convergence and stability results. Numerical simulations for one-dimensional and two-dimensional parameter identification problems are performed to validate that our approach is competitive in terms of reducing the overall computational time in comparison with the existing Landweber iteration with general convex penalty. Moreover, our approach is used to the non-Gaussian noisy data to illustrate the iterative regularization in Banach spaces.

On controllability methods for the Helmholtz equation

When the Helmholtz equation is discretized by standard finite difference or finite element methods, the resulting linear system is highly indefinite and thus notoriously difficult to solve, in fact increasingly so at higher frequency. The exact controllability approach (Bristeau et al., 1994) instead reformulates the problem in the time domain and seeks the time-harmonic solution of the corresponding wave equation. By iteratively reducing the mismatch between the solution at initial time and after one period, the controllability method greatly speeds up the convergence to the time-harmonic asymptotic limit. Moreover, each conjugate gradient iteration solely relies on standard numerical algorithms, which are inherently parallel and robust against higher frequencies. The original energy functional used to penalize the departure from periodicity is strictly convex only for sound-soft scattering problems. To extend the controllability approach to general boundary-value problems governed by the Helmholtz equation, new penalty functionals are proposed, which are numerically efficient. Numerical experiments for wave scattering from sound-soft and sound-hard obstacles, inclusions, but also for wave propagation in closed wave guides illustrate the usefulness of the resulting controllability methods.

A converse result for Banach space convergence rates in Tikhonov-type convex regularization of ill-posed linear equations

Abstract We consider Tikhonov-type variational regularization of ill-posed linear operator equations in Banach spaces with general convex penalty functionals. Upper bounds for certain error measures expressing the distance between exact and regularized solutions, especially for Bregman distances, can be obtained from variational source conditions. We prove that such bounds are optimal in case of twisted Bregman distances, a specific a priori parameter choice, and low regularity of the exact solution, that is, the rate function is also an asymptotic lower bound for the error measure. This result extends existing converse results from Hilbert space settings to Banach spaces without adhering to spectral theory.

Accelerated Landweber iteration with convex penalty for linear inverse problems in Banach spaces

In recent years, Landweber iteration has been extended to solve linear inverse problems in Banach spaces by incorporating non-smooth convex penalty functionals to capture features of solutions. This method is known to be slowly convergent. However, because it is simple to implement, it still receives a lot of attention. By making use of the subspace optimization technique, we propose an accelerated version of Landweber iteration with non-smooth convex penalty which significantly speeds up the method. Numerical simulations are given to test the efficiency.

Nonstationary iterated Tikhonov regularization in Banach spaces with uniformly convex penalty terms

We consider the nonstationary iterated Tikhonov regularization in Banach spaces which defines the iterates via minimization problems with uniformly convex penalty term. The penalty term is allowed to be non-smooth to include $$L^1$$ and total variation (TV) like penalty functionals, which are significant in reconstructing special features of solutions such as sparsity and discontinuities in practical applications. We present the detailed convergence analysis and obtain the regularization property when the method is terminated by the discrepancy principle. In particular we establish the strong convergence and the convergence in Bregman distance which sharply contrast with the known results that only provide weak convergence for a subsequence of the iterative solutions. Some numerical experiments on linear integral equations of first kind and parameter identification in differential equations are reported.

Landweber iteration of Kaczmarz type with general non-smooth convex penalty functionals

The determination of solutions of many inverse problems usually requires a set of measurements which leads to solving systems of ill-posed equations. In this paper, we propose the Landweber iteration of Kaczmarz type with general uniformly convex penalty functional. The method is formulated by using tools from convex analysis. The penalty term is allowed to be non-smooth to include the L1 and total variation-like penalty functionals, which are significant in reconstructing special features of solutions such as sparsity and piecewise constancy in practical applications. Under reasonable conditions, we establish the convergence of the method. Finally, we present numerical simulations on tomography problems and parameter identification in partial differential equations to indicate the performance.

On the iteratively regularized Gauss–Newton method in Banach spaces with applications to parameter identification problems

In this paper we propose an extension of the iteratively regularized Gauss---Newton method to the Banach space setting by defining the iterates via convex optimization problems. We consider some a posteriori stopping rules to terminate the iteration and present the detailed convergence analysis. The remarkable point is that in each convex optimization problem we allow non-smooth penalty terms including $$L^1$$ and total variation like penalty functionals. This enables us to reconstruct special features of solutions such as sparsity and discontinuities in practical applications. Some numerical experiments on parameter identification in partial differential equations are reported to test the performance of our method.

Shape-constrained regularization by statistical multiresolution for inverse problems: asymptotic analysis

This paper is concerned with a novel regularization technique for solving linear ill-posed operator equations in Hilbert spaces from data that are corrupted by white noise. We combine convex penalty functionals with extreme-value statistics of projections of the residuals on a given set of sub-spaces in the image space of the operator. We prove general consistency and convergence rate results in the framework of Bregman divergences which allows for a vast range of penalty functionals. Various examples that indicate the applicability of our approach will be discussed. We will illustrate in the context of signal and image processing that the presented method constitutes a locally adaptive reconstruction method.

Convergence rates in constrained Tikhonov regularization: equivalence of projected source conditions and variational inequalities

In this paper, we enlighten the role of variational inequalities for obtaining convergence rates in Tikhonov regularization of nonlinear ill-posed problems with convex penalty functionals under convexity constraints in Banach spaces. Variational inequalities are able to cover solution smoothness and the structure of nonlinearity in a uniform manner, not only for unconstrained but, as we indicate, also for constrained Tikhonov regularization. In this context, we extend the concept of projected source conditions already known in Hilbert spaces to Banach spaces, and we show in the main theorem that such projected source conditions are to some extent equivalent to certain variational inequalities. The derived variational inequalities immediately yield convergence rates measured by Bregman distances.