Lipschitz Assumption Research Articles

Modified proximal gradient methods involving double inertial extrapolations for monotone inclusion

In this work, we propose a novel class of forward‐backward‐forward algorithms for solving monotone inclusion problems. Our approach incorporates a self‐adaptive technique to eliminate the need for explicitly selecting Lipschitz assumptions and utilizes two‐step inertial extrapolations to enhance the convergence rate of the algorithm. We establish a weak convergence theorem under mild assumptions. Furthermore, we conduct numerical tests on image deblurring and data classification as practical applications. The experimental results demonstrate that our algorithm surpasses some existing methods in the literature which shows its superior performance and effectiveness.

Nonsingular adaptive finite-time consensus control for uncertain nonlinear multi-agent systems with input quantization

This paperinvestigates the distributed consensus control for nonlinear multi-agent systems with input quantization in a directed communication topology. A nonsingular adaptive finite-time control (NAFTC) scheme is proposed by combining the filtered backstepping method with neural network control. Both chattering and singularity problems of the control signals are avoided thanks to its [Formula: see text] continuity, so that the contradiction between discontinuous- and [Formula: see text] continuous finite-time control with input quantization does not exist in our scheme. At the same time, a novel inequality for the input-output relationship of hysteresis quantizer is derived, which can simply handle the input quantization without requiring additional stability analysis. What is more, some common assumptions, such as Lipschitz assumption and Holder-continuity assumption, are not required in our scheme. Finally, stability analysis and numerical simulation are provided.

The One Step Malliavin scheme: new discretization of BSDEs implemented with deep learning regressions

AbstractA novel discretization is presented for decoupled forward–backward stochastic differential equations (FBSDE) with differentiable coefficients, simultaneously solving the BSDE and its Malliavin sensitivity problem. The control process is estimated by the corresponding linear BSDE driving the trajectories of the Malliavin derivatives of the solution pair, which implies the need to provide accurate $\varGamma $ estimates. The approximation is based on a merged formulation given by the Feynman–Kac formulae and the Malliavin chain rule. The continuous time dynamics is discretized with a theta-scheme. In order to allow for an efficient numerical solution of the arising semidiscrete conditional expectations in possibly high dimensions, it is fundamental that the chosen approach admits to differentiable estimates. Two fully-implementable schemes are considered: the BCOS method as a reference in the one-dimensional framework and neural network Monte Carlo regressions in case of high-dimensional problems, similarly to the recently emerging class of Deep BSDE methods (Han et al. (2018 Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci., 115, 8505–8510); Huré et al. (2020 Deep backward schemes for high-dimensional nonlinear PDEs. Math. Comp., 89, 1547–1579)). An error analysis is carried out to show $\mathbb{L}^2$ convergence of order $1/2$, under standard Lipschitz assumptions and additive noise in the forward diffusion. Numerical experiments are provided for a range of different semilinear equations up to $50$ dimensions, demonstrating that the proposed scheme yields a significant improvement in the control estimations.

Exponential input-to-state stability for Lur’e systems via Integral Quadratic Constraints and Zames–Falb multipliers

Abstract Absolute stability criteria that are sufficient for global exponential stability are shown, under a Lipschitz assumption, to be sufficient for the a priori stronger exponential input-to-state stability property. Important corollaries of this result are as follows: (i) absolute stability results obtained using Zames–Falb multipliers for systems containing slope-restricted nonlinearities provide exponential input-to-state-stability under a mild detectability assumption; and (ii) more generally, many absolute stability results obtained via Integral Quadratic Constraint methods provide, with the additional Lipschitz assumption, this stronger property.

Existence and optimal controls of non-autonomous for impulsive evolution equation without Lipschitz assumption

In this paper, we investigate the existence of mild solutions as well as optimal controls for non-autonomous impulsive evolution equations with nonlocal conditions. Using the Schauder’s fixed-point theorem as well as the theory of evolution family, we prove the existence of mild solutions for the concerned problem. Furthermore, without the Lipschitz continuity of the nonlinear term, the optimal control result is derived by setting up minimizing sequences twice. An example is given of the application of the results.

Mean Reflected BSDE Driven by a Marked Point Process and Application in Insurance Risk Management

This paper aims to solve a super-hedging problem along with insurance re-payment under running risk management constraints. The initial endowment for the super-hedging problem is characterized by a class of mean reflected backward stochastic differential equation driven by a marked point process (MPP) and a Brownian motion. By Lipschitz assumptions on the generators and proper integrability on the terminal value, we give the well-posedness of this kind of BSDEs by combining a representation theorem with the fixed point argument.

A recent proximal gradient algorithm for convex minimization problem using double inertial extrapolations

<abstract><p>In this study, we suggest a new class of forward-backward (FB) algorithms designed to solve convex minimization problems. Our method incorporates a linesearch technique, eliminating the need to choose Lipschitz assumptions explicitly. Additionally, we apply double inertial extrapolations to enhance the algorithm's convergence rate. We establish a weak convergence theorem under some mild conditions. Furthermore, we perform numerical tests, and apply the algorithm to image restoration and data classification as a practical application. The experimental results show our approach's superior performance and effectiveness, surpassing some existing methods in the literature.</p></abstract>

Bregman-Golden Ratio Algorithms for Variational Inequalities

Variational inequalities provide a framework through which many optimisation problems can be solved, in particular, saddle-point problems. In this paper, we study modifications to the so-called Golden RAtio ALgorithm (GRAAL) for variational inequalities—a method which uses a fully explicit adaptive step-size and provides convergence results under local Lipschitz assumptions without requiring backtracking. We present and analyse two Bregman modifications to GRAAL: the first uses a fixed step size and converges under global Lipschitz assumptions, and the second uses an adaptive step-size rule. Numerical performance of the former method is demonstrated on a bimatrix game arising in network communication, and of the latter on two problems, namely, power allocation in Gaussian communication channels and N-person Cournot completion games. In all of these applications, an appropriately chosen Bregman distance simplifies the projection steps computed as part of the algorithm.

New results for stochastic fractional pseudo-parabolic equations with delays driven by fractional Brownian motion

In this work, four problems for stochastic fractional pseudo-parabolic containing bounded and unbounded delays are investigated. The fractional derivative and the stochastic noise we consider here are the Caputo operator and the fractional Brownian motion. For the two problems involving bounded delays, we aim at establishing global existence, uniqueness, and regularity results under integral Lipschitz conditions for the non-linear source terms. Such behaviors of mild solutions are also analyzed in the unbounded delay cases but under globally and locally Lipschitz assumptions. We emphasize that our results are investigated in the novel spaces C([−r,T];Lp(Ω,Wl,q(D))), Cμ((−∞,T];Lp(Ω,Wl,q(D))), and the weighted space Fμɛ((−∞,T];Lp(Ω,Wl,q(D))), instead of usual ones C([−r,T];L2(Ω,H)), Cμ((−∞,T];L2(Ω,H)). The main technique allowing us to overcome the rising difficulties lies on some useful Sobolev embeddings between the Hilbert space H=L2(D) and Wl,q(D), and some well-known fractional tools. In addition, we also study the Hölder continuity for the mild solutions, which can be considered as one of the main novelties of this paper. Finally, we consider an additional result connecting delay stochastic fractional pseudo-parabolic equations and delay stochastic fractional parabolic equations. We show that the mild solution of the first model converges to the mild solution of the second one, in some sense, as the diffusion parameter β→0+.

Iterative algorithms for variational inequalities and fixed point problems of pseudocontraction without Lipschitz assumptions

Iterative algorithms for variational inequalities and fixed point problems of pseudocontraction without Lipschitz assumptions

Error analysis for Galerkin-BDF discretizations of DAEs with elliptic operator constraints

We are interested in a convergent numerical discretization scheme for nonlinear differential algebraic equations (DAEs) coupled with elliptic constraints. The dynamics of flow networks (for example circuits) can often be described by a DAE. However, this is only the case if all network element models are lumped models. If spatial effects cannot be neglected, distributed element models have to be included. In this paper, we address the case of elliptic models for distributed network elements. Under some monotonicity and Lipschitz assumptions for the system functions, we present an error analysis for discretizations of DAEs coupled with elliptic constraints based on a Galerkin approach for the distributed model part combined with the BDF method as time discretization of the full system.

Existence and partial approximate controllability of nonlinear Riemann–Liouville fractional systems of higher order

This article studies the existence and partial approximate controllability of higher order nonlocal semilinear fractional differential equations with Riemann–Liouville derivatives avoiding Lipschitz assumptions of nonlinear operator and nonlocal functions. To derive the existence result, we make approximate systems corresponding to the original system. For this, we construct the mild solutions in terms of fractional resolvent. Then, we prove the partial approximate controllability of the nonlinear system by using the obtained existence result. Finally, we give an example to illustrate the established theory.

Convergence Properties of Monotone and Nonmonotone Proximal Gradient Methods Revisited

Composite optimization problems, where the sum of a smooth and a merely lower semicontinuous function has to be minimized, are often tackled numerically by means of proximal gradient methods as soon as the lower semicontinuous part of the objective function is of simple enough structure. The available convergence theory associated with these methods (mostly) requires the derivative of the smooth part of the objective function to be (globally) Lipschitz continuous, and this might be a restrictive assumption in some practically relevant scenarios. In this paper, we readdress this classical topic and provide convergence results for the classical (monotone) proximal gradient method and one of its nonmonotone extensions which are applicable in the absence of (strong) Lipschitz assumptions. This is possible since, for the price of forgoing convergence rates, we omit the use of descent-type lemmas in our analysis.

An Accelerated First-Order Method for Non-convex Optimization on Manifolds

We describe the first gradient methods on Riemannian manifolds to achieve accelerated rates in the non-convex case. Under Lipschitz assumptions on the Riemannian gradient and Hessian of the cost function, these methods find approximate first-order critical points faster than regular gradient descent. A randomized version also finds approximate second-order critical points. Both the algorithms and their analyses build extensively on existing work in the Euclidean case. The basic operation consists in running the Euclidean accelerated gradient descent method (appropriately safe-guarded against non-convexity) in the current tangent space, then moving back to the manifold and repeating. This requires lifting the cost function from the manifold to the tangent space, which can be done for example through the Riemannian exponential map. For this approach to succeed, the lifted cost function (called the pullback) must retain certain Lipschitz properties. As a contribution of independent interest, we prove precise claims to that effect, with explicit constants. Those claims are affected by the Riemannian curvature of the manifold, which in turn affects the worst-case complexity bounds for our optimization algorithms.

Zeroth-order algorithms for stochastic distributed nonconvex optimization

In this paper, we consider a stochastic distributed nonconvex optimization problem with the cost function being distributed over n agents having access only to zeroth-order (ZO) information of the cost. This problem has various machine learning applications. As a solution, we propose two distributed ZO algorithms, in which at each iteration each agent samples the local stochastic ZO oracle at two points with a time-varying smoothing parameter. We show that the proposed algorithms achieve the linear speedup convergence rate O(p/(nT)) for smooth cost functions under the state-dependent variance assumptions which are more general than the commonly used bounded variance and Lipschitz assumptions, and O(p/(nT)) convergence rate when the global cost function additionally satisfies the Polyak–Łojasiewicz (P–Ł) condition, where p and T are the dimension of the decision variable and the total number of iterations, respectively. To the best of our knowledge, this is the first linear speedup result for distributed ZO algorithms. It consequently enables systematic processing performance improvements by adding more agents. We also show that the proposed algorithms converge linearly under the relatively bounded second moment assumptions and the P–Ł condition. We demonstrate through numerical experiments the efficiency of our algorithms on generating adversarial examples from deep neural networks in comparison with baseline and recently proposed centralized and distributed ZO algorithms.

An Accelerated Convex Optimization Algorithm with Line Search and Applications in Machine Learning

In this paper, we introduce a new line search technique, then employ it to construct a novel accelerated forward–backward algorithm for solving convex minimization problems of the form of the summation of two convex functions in which one of these functions is smooth in a real Hilbert space. We establish a weak convergence to a solution of the proposed algorithm without the Lipschitz assumption on the gradient of the objective function. Furthermore, we analyze its performance by applying the proposed algorithm to solving classification problems on various data sets and compare with other line search algorithms. Based on the experiments, the proposed algorithm performs better than other line search algorithms.

Bounding the Trajectories of Continuous-Time LPV Systems with Parameters known in Real Time

Predicting the exact future of a Linear Parameter-Varying (LPV) system with its parameter exclusively known in real time is by definition an impossible task. In particular, it is difficult to quantify the error induced by a Zero-Order Hold (ZOH) discretization of the parameter which is not verified in practice (e.g. when the parameter depends on the states, inputs or outputs of the continuous-time system). Under a Lipschitz assumption, this paper upper bounds — in terms of uncertain matrices — the greatest possible discrepancy between the real future of the system and its estimate based on the last known values of the input and parameter. This not only upper bounds the error due to the ZOH discretization, but also provides sufficient conditions for controllability and observability of the system in the near future by bounding its Gramians.

A simple path-aware optimization method for mobile robots*

We present an approach for a mobile robot to seek the global maximum of an initially unknown function defined over its operating space. The method exploits a Lipschitz assumption to defne an upper bound on the function from previously seen samples, and optimistically moves towards the largest upper-bound point. This point is iteratively changed whenever new samples make it clear that it is suboptimal. In simulations, the method finds the global maxima with much less computation than an existing, much more involved technique, while keeping performance acceptable. Real-robot experiments confirm the effectiveness of the approach.

Event-triggered intelligent critic control with input constraints applied to a nonlinear aeroelastic system

In this paper, we establish an event-triggered intelligent control scheme with a single critic network, to cope with the optimal stabilization problem of nonlinear aeroelastic systems. The main contribution lies in the design of a novel triggering condition with input constraints, avoiding the Lipschitz assumption on the inverse hyperbolic tangent function. Based on an improved weight updating criterion that eliminates the requirement of initial admissible control, the control law is obtained approximately by online training of a single critic network. The Lyapunov stability and the Zeno phenomenon of the closed-loop system are analysed. The feasibility of the established algorithm is verified by applying it to an optimal stabilization task of a nonlinear aeroelastic system. The results reveal that the developed approach can handle input-constrained optimal control problems, with performance comparable to the time-based method that updates control inputs at each instant, while reducing the computational and communication's load.

Numerical approximations of coupled forward–backward SPDEs

We propose and study a scheme combining the finite element method and machine learning techniques for the numerical approximations of coupled nonlinear forward–backward stochastic partial differential equations (FBSPDEs) with homogeneous Dirichlet boundary conditions. Precisely, we generalize the pioneering work of Dunst and Prohl [SIAM J. Sci. Comp., 38(2017), 2725–2755] by considering general nonlinear and nonlocal FBSPDEs with more inclusive coupling; self-contained proofs are provided and different numerical techniques for the resulting finite dimensional equations are adopted. For such FBSPDEs allowing for non-Lipschitz nonlinearity on unknown variables, we prove the existence and uniqueness of the strong solution as well as of the weak solution. Then, the finite element method in the spatial domain leads to approximations of FBSPDEs by finite-dimensional forward–backward stochastic differential equations which are numerically computed by using some deep learning-based schemes. A strong convergence analysis is addressed for the spatial discretization of coupled FBSPDEs under an additional Lipschitz assumption, and the numerical examples, including both decoupled and coupled cases, indicate that our methods are quite efficient.