Local Lipschitz Research Articles

Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradient

Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradient

Primal-Dual Extrapolation Methods for Monotone Inclusions Under Local Lipschitz Continuity

In this paper, we consider a class of monotone inclusion (MI) problems of finding a zero of the sum of two monotone operators, in which one operator is maximal monotone, whereas the other is locally Lipschitz continuous. We propose primal-dual (PD) extrapolation methods to solve them using a point and operator extrapolation technique, whose parameters are chosen by a backtracking line search scheme. The proposed methods enjoy an operation complexity of [Formula: see text] and [Formula: see text], measured by the number of fundamental operations consisting only of evaluations of one operator and resolvent of the other operator, for finding an ε-residual solution of strongly and nonstrongly MI problems, respectively. The latter complexity significantly improves the previously best operation complexity [Formula: see text]. As a byproduct, complexity results of the primal-dual extrapolation methods are also obtained for finding an ε-KKT or ε-residual solution of convex conic optimization, conic constrained saddle point, and variational inequality problems under local Lipschitz continuity. We provide preliminary numerical results to demonstrate the performance of the proposed methods. Funding: This work was partially supported by the National Science Foundation [Grant IIS-2211491], the Office of Naval Research [Grant N00014-24-1-2702], and the Air Force Office of Scientific Research [Grant FA9550-24-1-0343].

Analytical Bounds on the Local Lipschitz Constants of ReLU Networks.

In this article, we determine analytical upper bounds on the local Lipschitz constants of feedforward neural networks with rectified linear unit (ReLU) activation functions. We do so by deriving Lipschitz constants and bounds for ReLU, affine-ReLU, and max pooling functions, and combining the results to determine a network-wide bound. Our method uses several insights to obtain tight bounds, such as keeping track of the zero elements of each layer, and analyzing the composition of affine and ReLU functions. Furthermore, we employ a careful computational approach which allows us to apply our method to large networks such as AlexNet and VGG-16. We present several examples using different networks, which show how our local Lipschitz bounds are tighter than the global Lipschitz bounds. We also show how our method can be applied to provide adversarial bounds for classification networks. These results show that our method produces the largest known bounds on minimum adversarial perturbations for large networks such as AlexNet and VGG-16.

Generalized Lipschitz Tracing of Implicit Surfaces

AbstractWe present a versatile and robust framework to render implicit surfaces defined by black‐box functions that only provide function value queries. We assume that the input function is locally Lipschitz continuous; however, we presume no prior knowledge of its Lipschitz constants. Our pre‐processing step generates a discrete acceleration structure, a Lipschitz field, that provides data to infer local and directional Lipschitz upper bounds. These bounds are used to compute safe step sizes along rays during rendering. The Lipschitz field is constructed by generating local polynomial approximations to the input function, then bounding the derivatives of the approximating polynomials. The accuracy of the approximation is controlled by the polynomial degree and the granularity of the spatial resolution used during fitting, which is independent from the resolution of the Lipschitz field. We demonstrate that our process can be implemented in a massively parallel way, enabling straightforward integration into interactive and real‐time modelling workflows. Since the construction only requires function value evaluations, the input surface may be represented either procedurally or as an arbitrarily filtered grid of function samples. We query the original implicit representation upon ray trace, as such, we preserve the geometric and topological details of the input as long as the Lipschitz field supplies conservative estimates. We demonstrate our method on both procedural and discrete implicit surfaces and compare its exact and approximate variants.

Numerical solutions of regime-switching functional diffusions with infinite delay

We study a class of diffusion processes which are determined by solutions X(t) to stochastic functional differential equation with infinite memory and random switching represented by Markov chain Λ(t). Under suitable conditions, we adopt Euler-Maruyama method to deal with the convergence of numerical solutions of the corresponding stochastic differential equations. More precisely, we investigate convergence rates in the L 2-norm the stochastic functional differential equation with infinite memory and random switching under the global Lipschitz conditions. Then we also discuss L 2-convergence under the local Lipschitz conditions.

Discretization and quantification for distributionally robust optimization with decision-dependent ambiguity sets

ABSTRACT In this paper, we investigate the discrete approximation and the quantitative analysis for a class of the distributionally robust optimization problems with decision-dependent ambiguity sets. We establish the local Lipschitz continuity of the decision-dependent ambiguity set, measured by the Hausdorff distance, under a broad class of metrics known as ζ-structure and the Slater condition. Furthermore, we employ Lagrange duality and first-order growth conditions to derive quantitative analysis for the optimal value and optimal solution. We also examine the application of a classical and widely-used ambiguity set within the theoretical framework of this paper. Finally, we conduct experiments to demonstrate the computational times and variations in the optimal value.

Inverse initial‐value problems for time fractional diffusion equations in fractional Sobolev spaces

AbstractWe study the time fractional diffusion equation , , in a bounded domain with an elliptic operator and a locally Lipschitz nonlinearity on fractional Sobolev spaces, subjected to the homogeneous Dirichlet boundary condition. Data have not been measured at the initial time , but at a final time , that is, is given instead of . The problem is, therefore, called an inverse initial‐value problem. We first establish the well‐posedness of this problem on fractional Sobolev spaces and the regularity of the solution by assuming only the local Lipschitz continuity of . Second, an susceptible‐infected (shortly, SI) model with heterogeneity and a Navier–Stokes equation have been exemplified. Finally, a spatial ‐estimate for the solution and its gradient has been provided. The essential tools are asymptotic behaviours of Mittag–Leffler functions, fractional power spaces, fractional Sobolev spaces and embedding, weighted functional spaces, and estimates for heat semigroup.

Lipschitz regularity of sub-elliptic harmonic maps into CAT(0) space

Abstract We prove the local Lipschitz continuity of sub-elliptic harmonic maps between certain singular spaces, more specifically from the 𝑛-dimensional Heisenberg group into CAT ⁡ ( 0 ) \operatorname{CAT}(0) spaces. Our main theorem establishes that these maps have the desired Lipschitz regularity, extending the Hölder regularity in this setting proven in [Y. Gui, J. Jost and X. Li-Jost, Subelliptic harmonic maps with values in metric spaces of nonpositive curvature, Commun. Math. Res. 38 (2022), 4, 516–534] and obtaining same regularity as in [H.-C. Zhang and X.-P. Zhu, Lipschitz continuity of harmonic maps between Alexandrov spaces, Invent. Math. 211 (2018), 3, 863–934] for certain sub-Riemannian geometries; see also [N. Gigli, On the regularity of harmonic maps from RCD ⁢ ( K , N ) \mathrm{RCD}(K,N) to CAT ⁢ ( 0 ) \mathrm{CAT}(0) spaces and related results, preprint (2022), https://arxiv.org/abs/2204.04317; and A. Mondino and D. Semola, Lipschitz continuity and Bochner–Eells–Sampson inequality for harmonic maps from RCD ⁡ ( k , n ) \operatorname{RCD}(k,n) spaces to CAT ⁡ ( 0 ) \operatorname{CAT}(0) spaces, preprint (2022), https://arxiv.org/abs/2202.01590] for the generalisation to RCD spaces. The present result paves the way for a general regularity theory of sub-elliptic harmonic maps, providing a versatile approach applicable beyond the Heisenberg group.

Uncertainty Estimation and Out-of-Distribution Detection for Deep Learning-Based Image Reconstruction Using the Local Lipschitz.

Accurate image reconstruction is at the heart of diagnostics in medical imaging. Supervised deep learning-based approaches have been investigated for solving inverse problems including image reconstruction. However, these trained models encounter unseen data distributions that are widely shifted from training data during deployment. Therefore, it is essential to assess whether a given input falls within the training data distribution. Current uncertainty estimation approaches focus on providing an uncertainty map to radiologists, rather than assessing the training distribution fit. In this work, we propose a method based on the local Lipschitz metric to distinguish out-of-distribution images from in-distribution with an area under the curve of 99.94% for True Positive Rate versus False Positive Rate. We demonstrate a very strong relationship between the local Lipschitz value and mean absolute error (MAE), supported by a Spearman's rank correlation coefficient of 0.8475, to determine an uncertainty estimation threshold for optimal performance. Through the identification of false positives, we demonstrate the local Lipschitz and MAE relationship can guide data augmentation and reduce uncertainty. Our study was validated using the AUTOMAP architecture for sensor-to-image Magnetic Resonance Imaging (MRI) reconstruction. We demonstrate our approach outperforms baseline techniques of Monte-Carlo dropout and deep ensembles as well as the state-of-the-art Mean Variance Estimation network approach. We expand our application scope to MRI denoising and Computed Tomography sparse-to-full view reconstructions using UNET architectures. We show our approach is applicable to various architectures and applications, especially in medical imaging, where preserving diagnostic accuracy of reconstructed images remains paramount.

Lipschitz regularity of a weakly coupled vectorial almost-minimizers for the p-Laplacian

For a given constant λ>0 and a bounded Lipschitz domain D⊂Rn (n≥2), we establish that almost-minimizers of the functionalJ(v;D)=∫D∑i=1m|∇vi(x)|p+λχ{|v|>0}(x)dx,1<p<∞, where v=(v1,⋯,vm), and m∈N, exhibit optimal Lipschitz continuity in compact sets of D. Furthermore, assuming p≥2 and employing a distinctly different methodology, we tackle the issue of boundary Lipschitz regularity for v. This approach simultaneously yields an alternative proof for the optimal local Lipschitz regularity for the interior case.

Euler-Maruyama approximation for stochastic fractional neutral integro-differential equations with weakly singular kernel

This manuscript examines the problem of nonlinear stochastic fractional neutral integro-differential equations with weakly singular kernels. Our focus is on obtaining precise estimates to cover all possible cases of Abel-type singular kernels. Initially, we establish the existence, uniqueness, and continuous dependence on the initial value of the true solution, assuming a local Lipschitz condition and linear growth condition. Additionally, we develop the Euler-Maruyama method for numerical solution of the equation and prove its strong convergence under the same conditions as the well-posedness. Moreover, we determine the accurate convergence rate of this method under global Lipschitz conditions and linear growth conditions.

Large deviations for regime-switching diffusions with infinite delay

Focusing on a class of regime-switching functional diffusion processes with infinite delay, a Freidlin–Wentzell type large deviations principle (LDP) is established by using an extended contraction principle and an exponential approximation argument under a local one-side Lipschitz condition. The result is new even for functional diffusion processes with infinite delay without regime-switching. Several interesting examples are given to illustrate our results.

Iterative Learning Control Based on Neural Network and Its Application to Ni-Mn-Ga Alloy Actuator With Local Lipschitz Nonlinearity

Iterative Learning Control Based on Neural Network and Its Application to Ni-Mn-Ga Alloy Actuator With Local Lipschitz Nonlinearity

Explicit numerical approximations for SDDEs in finite and infinite horizons using the adaptive EM method: Strong convergence and almost sure exponential stability

In this paper we investigate explicit numerical approximations for stochastic differential delay equations (SDDEs) under a local Lipschitz condition by employing the adaptive Euler-Maruyama (EM) method. Working in both finite and infinite horizons, we achieve strong convergence results of the adaptive EM solution. We also obtain the order of convergence in finite horizon. In addition, we show almost sure exponential stability of the adaptive approximate solution for both SDEs and SDDEs.

On the well-posedness of the Cauchy problem for the two-component peakon system in Ck∩Wk,1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$C^k\\cap W^{k,1}$$\\end{document}

This study focuses on the Cauchy problem associated with the two-component peakon system featuring a cubic nonlinearity, constrained to the class (m,n)∈Ck(R)∩Wk,1(R)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(m,n)\\in C^{k}(\\mathbb {R}) \\cap W^{k,1}(\\mathbb {R})$$\\end{document} with k∈N∪{0}\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$k\\in \\mathbb {N}\\cup \\{0\\}$$\\end{document}. This system extends the celebrated Fokas–Olver–Rosenau–Qiao equation and the following nonlocal (two-place) counterpart proposed by Lou and Qiao: ∂tm(t,x)=∂x[m(t,x)(u(t,x)-∂xu(t,x))(u(-t,-x)+∂x(u(-t,-x)))],\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\begin{aligned} \\partial _t m(t,x)= \\partial _x[m(t,x)(u(t,x)-\\partial _xu(t,x)) (u(-t,-x)+\\partial _x(u(-t,-x)))], \\end{aligned}$$\\end{document}where m(t,x)=1-∂x2u(t,x)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$m(t,x)=\\left( 1-\\partial _{x}^2\\right) u(t,x)$$\\end{document}. Employing an approach based on Lagrangian coordinates, we establish the local existence, uniqueness, and Lipschitz continuity of the data-to-solution map in the class Ck∩Wk,1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$C^k\\cap W^{k,1}$$\\end{document}. Moreover, we derive criteria for blow-up of the local solution in this class.

Cut-and-paste for impulsive gravitational waves with Λ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\Lambda $$\\end{document}: the mathematical analysis

Impulsive gravitational waves are theoretical models of short but violent bursts of gravitational radiation. They are commonly described by two distinct spacetime metrics, one of local Lipschitz regularity and the other one even distributional. These two metrics are thought to be ‘physically equivalent’ since they can be formally related by a ‘discontinuous coordinate transformation’. In this paper we provide a mathematical analysis of this issue for the entire class of nonexpanding impulsive gravitational waves propagating in a background spacetime of constant curvature. We devise a natural geometric regularisation procedure to show that the notorious change of variables arises as the distributional limit of a family of smooth coordinate transformations. In other words, we establish that both spacetimes arise as distributional limits of a smooth sandwich wave taken in different coordinate systems which are diffeomorphically related.

Local Lipschitz Continuity in the Initial Value and Strong Completeness for Nonlinear Stochastic Differential Equations

Recently, Hairer et al. (2015) showed that there exist stochastic differential equations (SDEs) with infinitely often differentiable and globally bounded coefficient functions whose solutions fail to be locally Lipschitz continuous in the strong L p L^p -sense with respect to the initial value for every p ∈ ( 0 , ∞ ] p\in (0,\infty ] . In this article we provide conditions on the coefficient functions of the SDE and on p ∈ ( 0 , ∞ ] p \in (0,\infty ] that are sufficient for local Lipschitz continuity in the strong L p L^p -sense with respect to the initial value and we establish explicit estimates for the local Lipschitz continuity constants. In particular, we prove local Lipschitz continuity in the initial value for several nonlinear stochastic ordinary and stochastic partial differential equations in the literature such as the stochastic van der Pol oscillator, Brownian dynamics, the Cox-Ingersoll-Ross processes and the Cahn-Hilliard-Cook equation. As an application of our estimates, we obtain strong completeness for several nonlinear SDEs.

Boundedness and stability of nonlinear hybrid neutral stochastic delay differential equation with Lévy jumps under different structures

This paper investigates the boundedness and stability of a class of nonlinear hybrid neutral stochastic differential delay systems with Lévy jumps and different structures. The coefficients in this system satisfy the local Lipschitz condition and a suitable Khasminskii-type condition, and the state space of the system is separated into two subsets, the existence uniqueness, asymptotic boundedness, and exponential stability of the system are obtained by designing a new Lyapunov function and applying the M-matrix technique as well as dealing with the non-differentiable delay function. Different with the existing work, we not only consider the neutral term, but also the case of the delay function being bounded and non-differentiable. At last, numerical examples are performed to manifest the obtained results.

LINEAR AND QUASILINEAR EQUATIONS WITH SEVERAL GERASIMOV - CAPUTO DERIVATIVES

A representation of a solution of the Cauchy problem for a linear inhomogeneous equation solved with respect to the oldest derivative with several fractional Gerasimov - Caputo derivatives and with a sectorial pencil of linear closed operators at them in the case of the Holder function in the right-hand side of the equation is obtained; the uniqueness of the solution is proved. This result is used to reduce the Cauchy problem for the corresponding quasilinear equation to an integro-differential equation. The existence of a unique local solution is proved by the method of contraction operators in the case of local Lipschitz nonlinear operator depending on several Gerasimov - Caputo derivatives in the equation and a single global solution under the Lipschitz condition for this operator.

LRS: Enhancing Adversarial Transferability through Lipschitz Regularized Surrogate

The transferability of adversarial examples is of central importance to transfer-based black-box adversarial attacks. Previous works for generating transferable adversarial examples focus on attacking given pretrained surrogate models while the connections between surrogate models and adversarial trasferability have been overlooked. In this paper, we propose Lipschitz Regularized Surrogate (LRS) for transfer-based black-box attacks, a novel approach that transforms surrogate models towards favorable adversarial transferability. Using such transformed surrogate models, any existing transfer-based black-box attack can run without any change, yet achieving much better performance. Specifically, we impose Lipschitz regularization on the loss landscape of surrogate models to enable a smoother and more controlled optimization process for generating more transferable adversarial examples. In addition, this paper also sheds light on the connection between the inner properties of surrogate models and adversarial transferability, where three factors are identified: smaller local Lipschitz constant, smoother loss landscape, and stronger adversarial robustness. We evaluate our proposed LRS approach by attacking state-of-the-art standard deep neural networks and defense models. The results demonstrate significant improvement on the attack success rates and transferability. Our code is available at https://github.com/TrustAIoT/LRS.