Lipschitz Continuous Research Articles

Approximation by Schurer Type λ-Bernstein–Bézier Basis Function Enhanced by Shifted Knots Properties

In this article, a novel Schurer form of λ-Bernstein operators augmented by Bézier basis functions is presented by utilizing the features of shifted knots. The shifted knots form of Bernstein operators and the Schurer form of the Bézier basis function are used in this article, then, new operators, the Schurer type λ-Bernstein shifted knots operators are constructed in terms of the Bézier basis function. First, the test functions are calculated and the central moments for these operators are obtained. Then, Korovkin’s type approximation properties are studied by the use of a modulus of continuity of orders one and two. Finally, the convergence theorems for these new operators are obtained by using Peetre’s K-functional and Lipschitz continuous functions. In the end, some direct approximation theorems are also obtained.

Distributed leader-following bipartite consensus for one-sided Lipschitz multi-agent systems via dual-terminal event-triggered mechanism

This article analyses leader-following bipartite consensus for one-sided Lipschitz multi-agent systems by dual-terminal event-triggered output feedback control approach. A distributed observer is designed to estimate unknown system states by employing relative output information at triggering time instants, and then an event-triggered output feedback controller is proposed. Dual-terminal dynamic event-triggered mechanisms are proposed in sensor–observer channel and controller–actuator channel, which can save communication resources to a great extent, and the Zeno behavior is ruled out. A new generalized one-sided Lipschitz condition is proposed to handle the nonlinear term and achieve bipartite consensus. Some stability conditions are presented to guarantee leader-following bipartite consensus. Finally, one-link robot manipulator systems are introduced to demonstrate the availability of the designed scheme. The results demonstrate that the agents of the robot manipulators can track the reference trajectories bi-directionally, and effectively reduce communication resources by 61.22% and 68.04% at the sensor–observer and controller–actuator channels, respectively.

On Observer and Controller Design for Nonlinear Hadamard Fractional-Order One-Sided Lipschitz Systems

This paper presents an extensive investigation into the state feedback stabilization, observer design, and observer-based controller design for a specific category of nonlinear Hadamard fractional-order systems. The research extends the conventional integer-order derivative to the Hadamard fractional-order derivative, offering a more universally applicable method for system analysis. Furthermore, the traditional Lipschitz condition is adapted to a one-sided Lipschitz condition, broadening the range of systems amenable to analysis using these techniques. The efficacy of the proposed theoretical findings is illustrated through several numerical examples. For instance, in Example 1, we select an order of derivative r = 0.8; in Example 2, r is set to 0.9; and in Example 3, r = 0.95. This study enhances the comprehension and regulation of nonlinear Hadamard fractional-order systems, setting the stage for future explorations in this domain.

Observer-based control for time-delayed quasi-one-sided Lipschitz nonlinear systems under input saturation

This paper addresses the observer-based controller design problem for nonlinear time-delayed systems under input saturation. The nonlinearities are supposed to satisfy the quasi-one-sided Lipschitz condition, which is less conservative than the one-sided Lipschitz condition. Based on the nonlinear matrix inequalities, control law for nonlinear systems subject to input saturation, time delays, and unavailable states, some sufficient conditions have been developed for an augmented system containing the system state vector and the error vector to ensure the convergence of all states to zero. The paper used a decoupling approach to reduce the complexity of the corresponding observer and controller gain computations. Finally, the effectiveness of the developed results is validated using suitable examples.

Noise-resilient designs and analysis for optical neural networks

Abstract All analog signal processing is fundamentally subject to noise, and this is also the case in next generation implementations of Optical Neural Networks (ONNs). Therefore, we propose the first hardware-based approach to mitigate noise in ONNs. A tree-like and an accordion-like design are constructed from a given Neural Network (NN) that one wishes to implement. Both designs have the capability that the resulting ONNs give outputs close to the desired solution. To establish the latter, we analyze the designs mathematically. Specifically, we investigate a probabilistic framework for the tree-like design that establishes the correctness of the design, i.e., for any feed-forward NN with Lipschitz continuous activation functions, an ONN can be constructed that produces output arbitrarily close to the original. ONNs constructed with the tree-like design thus also inherit the universal approximation property of NNs. For the accordion-like design, we restrict the analysis to NNs with linear activation functions and characterize the ONNs’ output distribution using exact formulas. Finally, we report on numerical experiments with LeNet ONNs that give insight into the number of components required in these designs for certain accuracy gains. The results indicate that adding just a few components and/or adding them only in the first (few) layers in the manner of either design can already be expected to increase the accuracy of ONNs considerably. To illustrate the effect we point to a specific simulation of a LeNet implementation, in which adding one copy of the layers components in each layer reduces the Mean Squared Error (MSE) by 59.1% for the tree-like design and by 51.5% for the accordion-like design. In this scenario, the gap in accuracy of prediction between the noiseless NN and the ONNs reduces even more: 93.3% for the tree-like design and 80% for the accordion-like design.

Sharp propagation of chaos for the ensemble Langevin sampler

AbstractThe aim of this paper is to revisit propagation of chaos for a Langevin‐type interacting particle system recently proposed as a method to sample probability measures. The interacting particle system we consider coincides, in the setting of a log‐quadratic target distribution, with the ensemble Kalman sampler [SIAM J. Appl. Dyn. Syst. 19 (2020), no. 1, 412–441], for which propagation of chaos was first proved by Ding and Li in [SIAM J. Math. Anal. 53 (2021), no. 2, 1546–1578]. Like these authors, we prove propagation of chaos with an approach based on a synchronous coupling, as in Sznitman's classical argument. Instead of relying on a boostrapping argument, however, we use a technique based on stopping times in order to handle the presence of the empirical covariance in the coefficients of the dynamics. The use of stopping times to handle the lack of global Lipschitz continuity in the coefficients of stochastic dynamics originates from numerical analysis [SIAM J. Numer. Anal. 40 (2002), no. 3, 1041–1063] and was recently employed to prove mean‐field limits for consensus‐based optimization and related interacting particle systems [arXiv:2312.07373, 2023; Math. Models Methods Appl. Sci. 33 (2023), no. 2, 289–339]. In the context of ensemble Langevin sampling, this technique enables proving pathwise propagation of chaos with optimal rate, whereas previous results were optimal only up to a positive .

Delayed Interval-Valued Symmetric Stochastic Integral Equations

In this paper, delayed stochastic integral equations with an initial condition and a drift coefficient given as interval-valued mappings are considered. These equations have a certain symmetric form that distinguishes them from classical single-valued stochastic integral equations and has implications for the properties of the diameter of the values of the solutions of the equations. The main result of the paper is the theorem that there is a unique solution to the equation considered. It was obtained under the assumptions of continuity of the kernels and Lipschitz continuity of the drift and diffusion coefficients. The proof of the existence of the solution is carried out by the method of iterating successive approximations. The paper ends with theorems about the continuous dependence of the solution on the initial function, kernels and nonlinearities.

On synchronization of random nonlinear complex networks

Traditionally, stochastic disturbances arising in complex networks are often assumed to be drawn from a Wiener process, potentially limiting their applicability in real engineering scenarios. To address this limitation, we incorporate randomness to quantify the stochastic disturbances within a group of participating individuals, thereby establishing random nonlinear complex networks in a directed interacting setting. Subsequently, we demonstrate that the maximal existence interval of the unique solution to the underlying systems is determined by the properties of the associated noise and the specified Lipschitz constant. Building on this, we further show that, by making use of supermartingale and Lyapunov-based techniques, the almost sure synchronization condition of the investigated random complex system is determined by the communication topology, weight gain, and the number of participating agents. Additionally, we discuss synchronization problems within strongly connected and undirected graphs. Finally, we validate the proposed method using Chen systems.

A two-step relaxed-inertial derivative-free projection based algorithm for solving standard nonlinear pseudo-monotone equations and logistic regression problems

This paper explores a two-step inertial derivative-free projection method with a relaxation factor γ∈(0,2) for solving nonlinear pseudo-monotone equations. Unlike existing inertial algorithms for the system of nonlinear pseudo-monotone equations, the inertial step of our method involves the current iteration point and the previous two iteration points. In particular, one of the inertial parameters is nonpositive. In the proposed algorithm, the search direction possesses not only the sufficient descent property but also the trust region property, independent of the line search technique. Moreover, we also establish the global convergence and the convergence rate of the algorithm without the Lipschitz continuity of the underlying mapping. Finally, our method provides competitive results on standard nonlinear monotone and pseudo-monotone equations and logistic regression problems compared with two inertial algorithms existing in the literature.

Primal-Dual Extrapolation Methods for Monotone Inclusions Under Local Lipschitz Continuity

In this paper, we consider a class of monotone inclusion (MI) problems of finding a zero of the sum of two monotone operators, in which one operator is maximal monotone, whereas the other is locally Lipschitz continuous. We propose primal-dual (PD) extrapolation methods to solve them using a point and operator extrapolation technique, whose parameters are chosen by a backtracking line search scheme. The proposed methods enjoy an operation complexity of [Formula: see text] and [Formula: see text], measured by the number of fundamental operations consisting only of evaluations of one operator and resolvent of the other operator, for finding an ε-residual solution of strongly and nonstrongly MI problems, respectively. The latter complexity significantly improves the previously best operation complexity [Formula: see text]. As a byproduct, complexity results of the primal-dual extrapolation methods are also obtained for finding an ε-KKT or ε-residual solution of convex conic optimization, conic constrained saddle point, and variational inequality problems under local Lipschitz continuity. We provide preliminary numerical results to demonstrate the performance of the proposed methods. Funding: This work was partially supported by the National Science Foundation [Grant IIS-2211491], the Office of Naval Research [Grant N00014-24-1-2702], and the Air Force Office of Scientific Research [Grant FA9550-24-1-0343].

Dictionary Learning Method Based on K-Sparse Approximation and Orthogonal Procrustes Analysis for Reconstruction in Bioluminescence Tomography.

Bioluminescence tomography (BLT) is one kind of noninvasive optical molecular imaging technology, widely used to study molecular activities and disease progression inside live animals. By combining the optical propagation model and inversion algorithm, BLT enables three-dimensional imaging and quantitative analysis of light sources within organisms. However, challenges like light scattering and absorption in tissues, and the complexity of biological structures, significantly impact the accuracy of BLT reconstructions. Here, we propose a dictionary learning method based on K-sparse approximation and Orthogonal Procrustes analysis (KSAOPA). KSAOPA uses an iterative alternating optimization strategy, enhancing solution sparsity with k-coefficients Lipschitzian mappings for sparsity(K-LIMAPS) in the sparse coding stage, and reducing errors with Orthogonal Procrustes analysis in the dictionary update stage, leading to stable and precise reconstructions. We assessed the method performance through simulations and in vivo experiments, which showed that KSAOPA excels in localization accuracy, morphological recovery, and in vivo applicability compared to other methods.

A Wasserstein perspective of Vanilla GANs

The empirical success of Generative Adversarial Networks (GANs) caused an increasing interest in theoretical research. The statistical literature is mainly focused on Wasserstein GANs and generalizations thereof, which especially allow for good dimension reduction properties. Statistical results for Vanilla GANs, the original optimization problem, are still rather limited and require assumptions such as smooth activation functions and equal dimensions of the latent space and the ambient space. To bridge this gap, we draw a connection from Vanilla GANs to the Wasserstein distance. By doing so, existing results for Wasserstein GANs can be extended to Vanilla GANs. In particular, we obtain an oracle inequality for Vanilla GANs in Wasserstein distance. The assumptions of this oracle inequality are designed to be satisfied by network architectures commonly used in practice, such as feedforward ReLU networks. By providing a quantitative result for the approximation of a Lipschitz function by a feedforward ReLU network with bounded Hölder norm, we conclude a rate of convergence for Vanilla GANs as well as Wasserstein GANs as estimators of the unknown probability distribution.

Improving extensions for the global uniform asymptotic stability of continuous-time nonlinear systems

This paper considers the classical problems of global asymptotic stability (GAS) of nonlinear autonomous (time-invariant) systems and global uniform asymptotic stability (GUAS) of nonlinear non-autonomous (time-varying) systems. By imposing new conditions on the Lyapunov function and its derivative, the state convergence to the origin from any initial condition with bounded norm in the classical setting is extended to the initial conditions with unbounded norms in the present work. The proposed new notions of stability are called extended global asymptotic stability (EGAS) and extended global uniform asymptotic stability (EGUAS) in this paper. The proposed Lyapunov-based criteria yield the global asymptotic stability of the origin and fixed-time/predefined-time attractiveness of any selected neighbourhood of the origin. The result is that the dynamical behaviour of the state trajectories from the convergence time point of view is improved considerably. The relationship between the smoothness and input-to-state stability (ISS) properties is also studied from a quantitative point of view. Under some new sufficient conditions, it is shown that dynamical systems with smaller Lipschitz constants represent smaller ultimate bounds, and as a result, are more robust. The desirable dynamical behaviour and robustness property of the proposed stability notion makes it comparable with the finite/fixed/predefined/prescribed (exact) stable systems that are naturally non-Lipschitz at the origin. Numerical results illustrate the effectiveness of the proposed theoretical results.

Analytical Solutions for a Nonlinear Quadratic Delayed Functional Integral Inclusion With Feedback Control on the Real Half-Line

Let E be a reflexive Banach space. In this research, we are interested in the solvability of the nonlinear quadratic delayed functional integral inclusion (NQDFII) with a feedback control condition on the real half-axis. Our examination is found within the space BC(R+, E) of bounded continuous functions on the real half-axis R+ and takes values in a reflexive Banach space E beneath the assumption that the set-valued function G satisfy Lipschitz condition in E. The base we depend on in this study is the procedure related to a measure of noncompactness in the space BC(R+, E) by a given norm of continuity and applying Darbo’s fixed point theorem. Moreover, the asymptotic stability of the solution and the asymptotic dependency of the solution on the set of selections SG will be examined. Also, we give an example to illustrate the adequacy and esteem of our comes about.

Avoidance of the Lavrentiev gap for one-dimensional non-autonomous functionals with constraints

Abstract Consider a positive functional F ⁢ ( y ) := ∫ t T L ⁢ ( s , y ⁢ ( s ) , y ′ ⁢ ( s ) ) ⁢ 𝑑 s {F(y):=\int_{t}^{T}L(s,y(s),y^{\prime}(s))\,ds} , defined on the space of Sobolev functions W 1 , p ⁢ ( [ t , T ] ; ℝ n ) {W^{1,p}([t,T];{\mathbb{R}}^{n})} , where p ≥ 1 {p\geq 1} . This functional is minimized among functions y that may satisfy one or both endpoint conditions. The Lagrangian L is allowed to assume the value + ∞ {+\infty} . In numerous applications minimizers may not be explicit or even may not exist. In such circumstances, it is crucial to know that the infimum of F can be approximated using a sequence of Lipschitz functions that meet the given boundary conditions. However, there are instances where this approximation is not feasible, even with polynomial Lagrangians that meet Tonelli’s existence conditions: this situation is referred to as the Lavrentiev phenomenon. Some results are present in the literature if one requires the Lipschitz approximations to preserve just one endpoint constraint or when the Lagrangian is finite valued. The present paper deals with problems with two endpoint constraints and Lagrangians that are allowed to take extended values. As a byproduct, our Lipschitz approximations may preserve given state constraints. The extended-valued case is challenging since the phenomenon may occur even when the Lagrangian is constant on its effective domain, whose topology becomes relevant. Once assumed that the Lagrangian is radial convex on the rays of the last variable, our findings offer new insights, even when the Lagrangian L is real-valued, autonomous, and there are no state constraints.

Analytical Bounds on the Local Lipschitz Constants of ReLU Networks.

In this article, we determine analytical upper bounds on the local Lipschitz constants of feedforward neural networks with rectified linear unit (ReLU) activation functions. We do so by deriving Lipschitz constants and bounds for ReLU, affine-ReLU, and max pooling functions, and combining the results to determine a network-wide bound. Our method uses several insights to obtain tight bounds, such as keeping track of the zero elements of each layer, and analyzing the composition of affine and ReLU functions. Furthermore, we employ a careful computational approach which allows us to apply our method to large networks such as AlexNet and VGG-16. We present several examples using different networks, which show how our local Lipschitz bounds are tighter than the global Lipschitz bounds. We also show how our method can be applied to provide adversarial bounds for classification networks. These results show that our method produces the largest known bounds on minimum adversarial perturbations for large networks such as AlexNet and VGG-16.

Safe-Enhanced Autonomous Driving Technology Using Conformal Prediction Results

Abstract Autonomous driving technology significantly improves road safety, mitigates traffic congestion, and paves the way for a more efficient and connected transportation future. However, the uncertain driving scenarios poses a great challenge to the safe control of autonomous vehicles (AVs). Therefore, this paper proposes an enhanced safety control framework by integrating the quantified prediction results of human driving behaviors into the path-planning process. First, a Long Short-Term Memory (LSTM) network is employed for driver behavior prediction, based on which, the conformal prediction, a mathematical statistical method, is introduced to quantify the uncertainties of the prediction results with a probabilistic presentation. Then, a model predictive model controller (MPC) is designed to enable the path-planning ability. By using the Lipschitz constant, the inequality for obstacle avoidance is reformulated as a hard constraint in the MPC optimization framework while considering the conformal prediction results. Finally, some simulation test cases are conducted to validate the proposed method. The results demonstrate the effectiveness to guarantee the safety of AVs with uncertain prediction results.

Discretization methods and their extrapolations for two-dimensional nonlinear Volterra-Urysohn integral equations

In this paper, a class of nonlinear two-dimensional (2D) integral equations of Volterra type, i.e. Volterra-Urysohn integral equations, is studied. Following the ideas of [24], and assuming that the kernels of the integral equation are Lipschitz functions with respect to the dependent variable, the existence and uniqueness of a solution to the integral equation is shown by a technique based on the Picard iterative method. Then, the Euler and trapezoidal discretization methods are used to reduce the solution of the integral equation to the solution of a system of nonlinear algebraic equations. It is proved that the solution of the Euler method has first order convergence to the exact solution of the integral equation while the solution of the trapezoidal method has quadratic convergence. To prove the convergence of the trapezoidal method, a new Gronwall inequality is developed. Some numerical examples are given which confirm our theoretical results.

Uniform Convergence of Deep Neural Networks With Lipschitz Continuous Activation Functions and Variable Widths

Uniform Convergence of Deep Neural Networks With Lipschitz Continuous Activation Functions and Variable Widths

A Modified Three-terms CG Method with the MWWP Line Search and its Application to Image Restoration

Abstract In this article, an innovative method, which integrates the optimized three-term Polak–Ribière–Polyak (PRP) conjugate gradient (CG) method with the improved WWP line search rule, is presented and applied in the field of image restoration. The proposed search direction exhibits sufficient descent and trust region characteristics, maintaining its effectiveness regardless of the specific line search rule employed. We established the global convergence of the presented algorithm and eliminated the dependence on the Lipschitz continuity assumption. The numerical comparisons with the PRP CG method revealed the superior performance of the designed algorithm in the field of image restoration. This research outcome provides strong support for further advancements in the field of image recovery.