Lipschitz Continuous Operators Research Articles

Double inertial extragradient algorithms for solving variational inequality problems with convergence analysis

In this paper, we introduce a novel dual inertial Tseng's extragradient method for solving variational inequality problems in real Hilbert spaces, particularly those involving pseudomonotone and Lipschitz continuous operators. Our secondary method incorporates variable step‐size, updated at each iteration based on some previous iterates. A notable advantage of these algorithms is their ability to operate without prior knowledge of Lipschitz‐type constants and without the need for any line‐search procedure. We establish the convergence theorem of the proposed algorithms under mild assumptions. To illustrate the numerical behavior of the algorithms and to make comparisons with other methods, we conduct several numerical experiments. The results of these evaluations are showcased and thoroughly examined to exemplify the practical significance and effectiveness of the proposed methods.

Forward-backward-half forward splitting algorithm with deviations

ABSTRACT In this paper, we present a forward-backward-half forward splitting algorithm with deviations for solving the structured monotone inclusion problem composed of a maximally monotone operator, a maximally monotone and Lipschitz continuous operator and a cocoercive operator. The weak and linear convergence of the proposed algorithms is established. The aim of introducing deviations is to improve the performance of the forward-backward-half forward splitting algorithm through suitable choices of the deviations. A numerical example is given to show that the two-step inertial variant of the forward-backward-half forward splitting algorithms with deviations outperforms the existing methods in Briceño-Arias and Davis [Forward-backward-half forward algorithm for solving monotone inclusions, SIAM J Optimiz, 2017;28(4):2839–2871. doi: 10.1137/17M1120099], Zong et al. [An accelerated forward-backward-half forward splitting algorithm for monotone inclusion with applications to image restoration, Optimization, 2022;73(2):401–428. doi: 10.1080/02331934.2022.2107926] and Fan et al. [Convergence of an inertial shadow Douglas–Rachford splitting algorithm for monotone inclusions, Numer Func Anal Optim, 2021;42(14):1627–1644. doi: 10.1080/01630563.2021.2001749].

Three-operator reflected forward-backward splitting algorithm with double inertial effects

In this paper, we propose a reflected forward-backward splitting algorithm with two different inertial extrapolations to find a zero of the sum of three monotone operators consisting of the maximal monotone operator, Lipschitz continuous monotone operator, and a cocoercive operator in real Hilbert spaces. One of the interesting features of the proposed algorithm is that both the Lipschitz continuous monotone operator and the cocoercive operator are computed explicitly each with one evaluation per iteration. We then obtain weak and strong convergence results under some mild conditions. We finally give a numerical illustration to show that our proposed algorithm is effective and competitive with other related algorithms in the literature.

Fast Optimistic Gradient Descent Ascent (OGDA) Method in Continuous and Discrete Time

AbstractIn the framework of real Hilbert spaces, we study continuous in time dynamics as well as numerical algorithms for the problem of approaching the set of zeros of a single-valued monotone and continuous operator V. The starting point of our investigations is a second-order dynamical system that combines a vanishing damping term with the time derivative of V along the trajectory, which can be seen as an analogous of the Hessian-driven damping in case the operator is originating from a potential. Our method exhibits fast convergence rates of order $$o \left( \frac{1}{t\beta (t)} \right) $$ o 1 t β ( t ) for $$\Vert V(z(t))\Vert $$ ‖ V ( z ( t ) ) ‖ , where $$z(\cdot )$$ z ( · ) denotes the generated trajectory and $$\beta (\cdot )$$ β ( · ) is a positive nondecreasing function satisfying a growth condition, and also for the restricted gap function, which is a measure of optimality for variational inequalities. We also prove the weak convergence of the trajectory to a zero of V. Temporal discretizations of the dynamical system generate implicit and explicit numerical algorithms, which can be both seen as accelerated versions of the Optimistic Gradient Descent Ascent (OGDA) method for monotone operators, for which we prove that the generated sequence of iterates $$(z_k)_{k \ge 0}$$ ( z k ) k ≥ 0 shares the asymptotic features of the continuous dynamics. In particular we show for the implicit numerical algorithm convergence rates of order $$o \left( \frac{1}{k\beta _k} \right) $$ o 1 k β k for $$\Vert V(z^k)\Vert $$ ‖ V ( z k ) ‖ and the restricted gap function, where $$(\beta _k)_{k \ge 0}$$ ( β k ) k ≥ 0 is a positive nondecreasing sequence satisfying a growth condition. For the explicit numerical algorithm, we show by additionally assuming that the operator V is Lipschitz continuous convergence rates of order $$o \left( \frac{1}{k} \right) $$ o 1 k for $$\Vert V(z^k)\Vert $$ ‖ V ( z k ) ‖ and the restricted gap function. All convergence rate statements are last iterate convergence results; in addition to these, we prove for both algorithms the convergence of the iterates to a zero of V. To our knowledge, our study exhibits the best-known convergence rate results for monotone equations. Numerical experiments indicate the overwhelming superiority of our explicit numerical algorithm over other methods designed to solve monotone equations governed by monotone and Lipschitz continuous operators.

Determination of unknown shear force in transverse dynamic force microscopy from measured final data

Abstract In this paper, we present a new methodology, based on the inverse problem approach, for the determination of an unknown shear force acting on the inaccessible tip of the microcantilever, which is a key component of transverse dynamic force microscopy (TDFM). The mathematical modelling of this phenomenon leads to the inverse problem of determining the shear force g ⁢ ( t ) {g(t)} acting on the inaccessible boundary x = ℓ {x=\ell} in a system governed by the variable coefficient Euler–Bernoulli equation ρ A ⁢ ( x ) ⁢ u t ⁢ t + μ ⁢ ( x ) ⁢ u t + ( r ⁢ ( x ) ⁢ u x ⁢ x + κ ⁢ ( x ) ⁢ u x ⁢ x ⁢ t ) x ⁢ x = 0 , ( x , t ) ∈ ( 0 , ℓ ) × ( 0 , T ) , \rho_{A}(x)u_{tt}+\mu(x)u_{t}+(r(x)u_{xx}+\kappa(x)u_{xxt})_{xx}=0,\quad(x,t)% \in(0,\ell)\times(0,T), subject to the homogeneous initial conditions and the boundary conditions u ⁢ ( 0 , t ) = u 0 ⁢ ( t ) , u x ⁢ ( 0 , t ) = 0 , ( u x ⁢ x ⁢ ( x , t ) + κ ⁢ ( x ) ⁢ u x ⁢ x ⁢ t ) x = ℓ = 0 , ( - ( r ⁢ ( x ) ⁢ u x ⁢ x + κ ⁢ ( x ) ⁢ u x ⁢ x ⁢ t ) x ) x = ℓ = g ⁢ ( t ) , u(0,t)=u_{0}(t),\quad u_{x}(0,t)=0,\quad(u_{xx}(x,t)+\kappa(x)u_{xxt})_{x=\ell% }=0,\quad\bigl{(}-(r(x)u_{xx}+\kappa(x)u_{xxt})_{x}\bigr{)}_{x=\ell}=g(t), from the final time measured output (displacement) u T ⁢ ( x ) := u ⁢ ( x , T ) {u_{T}(x):=u(x,T)} . We introduce the input-output map ( Φ ⁢ g ) ⁢ ( x ) := u ⁢ ( x , T ; g ) {(\Phi g)(x):=u(x,T;g)} , g ∈ 𝒢 {g\in\mathcal{G}} , and prove that it is a compact and Lipschitz continuous linear operator. Then we introduce the Tikhonov functional J ⁢ ( F ) = 1 2 ⁢ ∥ Φ ⁢ g - u T ∥ L 2 ⁢ ( 0 , ℓ ) 2 J(F)=\frac{1}{2}\lVert\Phi g-u_{T}\rVert_{L^{2}(0,\ell)}^{2} and prove the existence of a quasi-solution of the inverse problem. We derive a gradient formula for the Fréchet gradient of the Tikhonov functional through the corresponding adjoint problem solution and prove that it is a Lipschitz continuous functional. The results of the numerical experiments clearly illustrate the effectiveness and feasibility of the proposed approach.

Two-step inertial forward–reflected–anchored–backward splitting algorithm for solving monotone inclusion problems

The main purpose of this paper is to propose and study a two-step inertial anchored version of the forward–reflected–backward splitting algorithm of Malitsky and Tam in a real Hilbert space. Our proposed algorithm converges strongly to a zero of the sum of a set-valued maximal monotone operator and a single-valued monotone Lipschitz continuous operator. It involves only one forward evaluation of the single-valued operator and one backward evaluation of the set-valued operator at each iteration; a feature that is absent in many other available strongly convergent splitting methods in the literature. Finally, we perform numerical experiments involving image restoration problem and compare our algorithm with known related strongly convergent splitting algorithms in the literature.

New accelerated splitting algorithm for monotone inclusion problems

ABSTRACT Forward-reflected-backward splitting algorithm with inertial extrapolation of two inertial effects to find a zero of the sum of a maximal monotone and a Lipschitz continuous monotone operator is studied in this paper. The incorporation of two inertial effects on the extrapolation step is to further improve the convergence speed of the forward-reflected-backward splitting algorithm with one inertial effect extrapolation already proposed in the literature. The parameter of the second inertial effect of our proposed algorithm is chosen to be non-positive. Weak, strong, and linear convergence results are obtained under standard conditions in Hilbert spaces. Preliminary numerical illustrations show that our proposed algorithm is competitive with other related algorithms in the literature.

On solving elliptic obstacle problems by constant abs-linearization

We consider optimal control problems governed by an elliptic variational inequality of the first kind, namely the obstacle problem. The variational inequality is treated by penalization, which leads to optimization problems governed by a nonsmooth semi-linear elliptic PDE. The CALi algorithm is then applied for the efficient solution of these nonsmooth optimization problems. The special feature of the optimization algorithm CALi is the treatment of the nonsmooth Lipschitz-continuous operators abs, max and min, which allows to explicitly exploit the nonsmooth structure. Stationary points are located by appropriate decomposition of the optimization problem into so-called smooth constant abs-linearized problems. Each of these constant abs-linearized problems can be solved by classical means. The comprehensive algorithmic concept is presented, and its performance is discussed through examples.

Nonlinear Inverse Problems for Equations with Dzhrbashyan–Nersesyan Derivatives

The unique solvability in the sense of classical solutions for nonlinear inverse problems to differential equations, solved for the oldest Dzhrbashyan–Nersesyan fractional derivative, is studied. The linear part of the equation contains a bounded operator, a continuous nonlinear operator that depends on lower-order Dzhrbashyan–Nersesyan derivatives, and an unknown element. The inverse problem is given by an equation, special initial value conditions for lower Dzhrbashyan–Nersesyan derivatives, and an overdetermination condition, which is defined by a linear continuous operator. Applying the fixed-point method for contraction mapping a theorem on the existence of a local unique solution is proved under the condition of local Lipschitz continuity of the nonlinear mapping. Analogous nonlocal results were obtained for the case of the nonlocally Lipschitz continuous nonlinear operator in the equation. The obtained results for the problem in arbitrary Banach spaces were used for the research of nonlinear inverse problems with time-dependent unknown coefficients at lower-order Dzhrbashyan–Nersesyan time-fractional derivatives for integro-differential equations and for a linearized system of dynamics of fractional Kelvin–Voigt viscoelastic media.

Analysis and Numerical Approach of a Coupled Thermo-Electro-Mechanical System for Nonlinear Hencky-Type Materials with Nonlocal Coulomb’s Friction

The effects of an included temperature field in the contact process between a piezoelectric body and a rigid foundation with thermal and electrical conductivity are discussed. The constitutive relation of the material is assumed to be thermo-electro-elastic and involves the nonlinear elastic constitutive Hencky’s law. The frictional contact is modeled with Signorini’s conditions, the regularized Coulomb law, and the regularized electrical and thermal conductivity conditions. The resulting thermo-electromechanical model includes the temperature field as an additional state variable to take into account thermal effects alongside with those of the piezoelectric. The existence of the unique weak solution to the problem is established by using Schauder’s fixed point theorem combined with arguments from the theory of variational inequalities involving nonlinear strongly monotone Lipschitz continuous operators. A successive iteration technique to solve the problem numerically is proposed, and its convergence is established. A variant of the Augmented Lagrangian, the so-called Alternating Direction Method of multipliers (ADMM), is used to decompose the original problem into two sub-problems, solve them sequentially and update the dual variables at each iteration. The influence of the thermal boundary condition on the behavior of contact forces and electrical potential is shown through graphical illustrations.

A short note on an adaptive damped Newton method for strongly monotone and Lipschitz continuous operator equations

We consider the damped Newton method for strongly monotone and Lipschitz continuous operator equations in a variational setting. We provide a very accessible justification why the undamped Newton method performs better than its damped counterparts in a vicinity of a solution. Moreover, in the given setting, an adaptive step-size strategy be presented, which guarantees the global convergence and favours an undamped update if admissible.

ОЦIНКИ ЕФЕКТИВНОСТI ДЛЯ МЕТОДIВ З ДИВЕРГЕНЦIЄЮ БРЕГМАНА

In this paper, variants of extrapolation from the past algorithm and operator extrapolation algorithm with Bregman divergence for solving variational inequalities with monotone and Lipschitz-continuous operators, which act in a finite-dimensional real linear space, are investigated. Main results: efficiency bounds for the gap function.

РЕГУЛЯРИЗОВАНИЙ АЛГОРИТМ ОПЕРАТОРНОЇ ЕКСТРАПОЛЯЦIЇ

This work is devoted to the study of new algorithm for solving variational inequalities in Hilbert spaces. The proposed algorithm is a variant of the operator extrapolation method regularized using the Halpern scheme. The algorithm has an advantage over the Korpelevich extragradient method and the method of extrapolation from the past in terms of the amount of calculations required for the iterative step. For variational inequalities with monotone, Lipschitz continuous operators acting in Hilbert space, a theorem on strong convergence of the method is proved.

ДВОРIВНЕВI ЗАДАЧI ТА ДВОЕТАПНИЙ ПРОКСИМАЛЬНИЙ АЛГОРИТМ

In this paper, we consider bilevel problem: variational inequality problem over the set of solutions the equilibrium problems. To solve this problem, an iterative algorithm is proposed that combines the ideas of a two-stage proximal method and iterative regularization. In addition, an adaptive version of the algorithm with a rule for updating parameters without using the values of the Lipschitz constants of the bifunction was studied. For monotone bifunctions of Lipschitz type and strongly monotone Lipschitz continuous operators, the theorem on strong convergence of sequences generated by the algorithms is proved.

A novel accelerated extragradient algorithm to solve pseudomonotone variational inequalities

In this paper, we propose a new inertial iterative method to solve classical variational inequalities with pseudomonotone and Lipschitz continuous operators in the setting of a real Hilbert space. The proposed iterative scheme is basically analogous to the extragradient method used to solve the problems of variational inequalities in real Hilbert spaces. The strong convergence of the proposed algorithm is set up with the prior knowledge of Lipschitz’s constant of an operator. Finally, several computational experiments are listed to show the applicability and efficiency of the proposed algorithm.

Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces

We consider monotone inclusions defined on a Hilbert space where the operator is given by the sum of a maximal monotone operator T and a single-valued monotone, Lipschitz continuous, and expectation-valued operator V. We draw motivation from the seminal work by Attouch and Cabot (Attouch in AMO 80:547–598, 2019, Attouch in MP 184: 243–287) on relaxed inertial methods for monotone inclusions and present a stochastic extension of the relaxed inertial forward–backward-forward method. Facilitated by an online variance reduction strategy via a mini-batch approach, we show that our method produces a sequence that weakly converges to the solution set. Moreover, it is possible to estimate the rate at which the discrete velocity of the stochastic process vanishes. Under strong monotonicity, we demonstrate strong convergence, and give a detailed assessment of the iteration and oracle complexity of the scheme. When the mini-batch is raised at a geometric (polynomial) rate, the rate statement can be strengthened to a linear (suitable polynomial) rate while the oracle complexity of computing an epsilon -solution improves to {mathcal {O}}(1/epsilon ). Importantly, the latter claim allows for possibly biased oracles, a key theoretical advancement allowing for far broader applicability. By defining a restricted gap function based on the Fitzpatrick function, we prove that the expected gap of an averaged sequence diminishes at a sublinear rate of {mathcal {O}}(1/k) while the oracle complexity of computing a suitably defined epsilon -solution is {mathcal {O}}(1/epsilon ^{1+a}) where a > 1. Numerical results on two-stage games and an overlapping group Lasso problem illustrate the advantages of our method compared to competitors.

A parallel type decomposition scheme for quasi-linear abstract hyperbolic equation

Abstract The Cauchy problem for an abstract hyperbolic equation with the Lipschitz continuous operator is considered in the Hilbert space. The operator corresponding to the elliptic part of the equation is the sum of operators A 1 , A 2 , … , A m {A_{1},A_{2},\ldots,A_{m}} . Each summand is a self-adjoint and positive definite operator. A parallel type decomposition scheme for an approximate solution of the stated problem is constructed. The main idea of the scheme is that on each local interval the classical difference problems are solved in parallel (independently from each other) with the operators A 1 , A 2 , … , A m {A_{1},A_{2},\ldots,A_{m}} . The weighted average of the obtained solutions is announced as an approximate solution at the right end of the local interval. The convergence of the proposed scheme is proved and the approximate solution error is estimated, as well as the error of the difference analogue for the first-order derivative for the case when the initial problem data satisfy the natural sufficient conditions for solution existence.

Determination of an unknown shear force in cantilever Kirchhoff–Love plate from measured final data with application to atomic force microscope

We present a new and feasible formulation for the inverse problem of identifying the unknown shear force acting on the inaccessible tip of the nonhomogeneous micro-cantilever plate which is a key component in most atomic force microscopes. The mathematical modelling of this phenomena leads to the inverse problem of determining the shear force g(x 2, t) acting on the inaccessible boundary x 1 = ℓ 1 in a system governed by the variable coefficient damped Kirchhoff–Love plate equation , where (x, t) ∈ Ω T ≔ Ω × (0, T), x ≔ (x 1, x 2) and , subject to clamped-free boundary conditions, from final time measured output (displacement) u T (x) ≔ u(x, T). We prove that under appropriate conditions, this inverse problem has a unique solution. We introduce the input–output operator and prove that it is a linear compact and Lipschitz continuous operator. Based on these results, the existence of a quasi-solution of the inverse problem, as a solution of the minimization problem for the Tikhonov functional, is proved. Furthermore, an explicit gradient formula for the Fréchet derivative of the Tikhonov functional is derived. Moreover, it is proved that the Fréchet derivative is Lipschitz continuous. These results provide a mathematical basis for gradient based computational algorithm.

A New Construction and Convergence Analysis of Non-Monotonic Iterative Methods for Solving ρ-Demicontractive Fixed Point Problems and Variational Inequalities Involving Pseudomonotone Mapping

Two new inertial-type extragradient methods are proposed to find a numerical common solution to the variational inequality problem involving a pseudomonotone and Lipschitz continuous operator, as well as the fixed point problem in real Hilbert spaces with a ρ-demicontractive mapping. These inertial-type iterative methods use self-adaptive step size rules that do not require previous knowledge of the Lipschitz constant. We also show that the proposed methods strongly converge to a solution of the variational inequality and fixed point problems under appropriate standard test conditions. Finally, we present several numerical examples to show the effectiveness and validation of the proposed methods.

On compositions of special cases of Lipschitz continuous operators

Many iterative optimization algorithms involve compositions of special cases of Lipschitz continuous operators, namely firmly nonexpansive, averaged, and nonexpansive operators. The structure and properties of the compositions are of particular importance in the proofs of convergence of such algorithms. In this paper, we systematically study the compositions of further special cases of Lipschitz continuous operators. Applications of our results include compositions of scaled conically nonexpansive mappings, as well as the Douglas–Rachford and forward–backward operators, when applied to solve certain structured monotone inclusion and optimization problems. Several examples illustrate and tighten our conclusions.