Nonsmooth Case Research Articles

Efficient private SCO for heavy-tailed data via averaged clipping

AbstractWe consider stochastic convex optimization for heavy-tailed data with the guarantee of being differentially private (DP). Most prior works on differentially private stochastic convex optimization for heavy-tailed data are either restricted to gradient descent (GD) or performed multi-times clipping on stochastic gradient descent (SGD), which is inefficient for large-scale problems. In this paper, we consider a one-time clipping strategy and provide principled analyses of its bias and private mean estimation. We establish new convergence results and improved complexity bounds for the proposed algorithm called AClipped-dpSGD for constrained and unconstrained convex problems. We also extend our convergent analysis to the strongly convex case and non-smooth case (which works for generalized smooth objectives with H$$\ddot{\text {o}}$$ o ¨ lder-continuous gradients). All the above results are guaranteed with a high probability for heavy-tailed data. Numerical experiments are conducted to justify the theoretical improvement.

Wolfe type duality on quasidifferentiable mathematical programs with vanishing constraints

This article is devoted to the study of duality results for optimization problems with vanishing constraints in nonsmooth case. We formulate Wolfe type dual and establish weak, strong, converse, restricted converse and strict converse duality results for mathematical programs with vanishing constraints involving quasidifferentiable functions. Under the assumption of invex and strictly invex functions with respect to a convex compact set.

Descent modulus and applications

The norm of the gradient ‖∇f(x)‖ measures the maximum descent of a real-valued smooth function f at x. For (nonsmooth) convex functions, this is expressed by the distance dist(0,∂f(x)) of the subdifferential to the origin, while for general real-valued functions defined on metric spaces by the notion of metric slope |∇f|(x). In this work we propose an axiomatic definition of descent modulus T[f](x) of a real-valued function f at every point x, defined on a general (not necessarily metric) space. The definition encompasses all above instances as well as average descents for functions defined on probability spaces. We show that a large class of functions are completely determined by their descent modulus and corresponding critical values. This result is already surprising in the smooth case: a one-dimensional information (norm of the gradient) turns out to be almost as powerful as the knowledge of the full gradient mapping. In the nonsmooth case, the key element for this determination result is the break of symmetry induced by a downhill orientation, in the spirit of the definition of the metric slope. The particular case of functions defined on finite spaces is studied in the last section. In this case, we obtain an explicit classification of descent operators that are, in some sense, typical.

Zero-cycles in families of rationally connected varieties

We study zero-cycles in families of rationally connected varieties. We show that for a smooth projective scheme over a henselian discrete valuation ring the restriction of relative zero cycles to the special fiber induces an isomorphism on Chow groups if the special fiber is separably rationally connected. We further extend this result to certain higher Chow groups and develop conjectures in the non-smooth case. Our main results generalise a result of Kollár (Publ. Res. Inst. Math. Sci. 40(3):689–708, 2004).

KCC theory for a type of nonlinear damped SD oscillators

Smooth and discontinuous (SD) oscillators are the nonlinear models that will exhibit SD dynamics due to continuous variation of the smooth parameter. Kosambi–Cartan–Chern (KCC) theory is a differential geometric theory that describes the deviations from the entire trajectory of the variational equation to the nearby trajectory. This paper presents a completely new dynamical analysis of a type of SD oscillators with nonlinear damping based on the KCC theory. First, the paper gives five KCC geometrical invariants of SD oscillators in the smooth and non-smooth cases, respectively. The results show that the geometric quantities in the non-smooth case cannot be derived directly from the geometric quantities in the smooth case. Second, from these geometric quantities, KCC stability of SD oscillators trajectory at any point except [Formula: see text] is analyzed. Lastly, numerical results show that in some regions that deviate from the equilibrium point of system, the system will exhibit complex and variable dynamical behavior due to small changes in parameters. This paper shows that the KCC theory is also a useful tool in the dynamical analysis of non-smooth systems.

A fast continuous time approach for non-smooth convex optimization using Tikhonov regularization technique

In this paper we would like to address the classical optimization problem of minimizing a proper, convex and lower semicontinuous function via the second order in time dynamics, combining viscous and Hessian-driven damping with a Tikhonov regularization term. In our analysis we heavily exploit the Moreau envelope of the objective function and its properties as well as Tikhonov regularization properties, which we extend to a nonsmooth case. We introduce the setting, which at the same time guarantees the fast convergence of the function (and Moreau envelope) values and strong convergence of the trajectories of the system to a minimal norm solution—the element of the minimal norm of all the minimizers of the objective. Moreover, we deduce the precise rates of convergence of the values for the particular choice of parameters. Various numerical examples are also included as an illustration of the theoretical results.

Dynamic tipping and cyclic folds, in a one-dimensional non-smooth dynamical system linked to climate models

We study the behaviour at tipping points close to (smoothed) non-smooth fold bifurcations in one-dimensional oscillatory forced systems with slow parameter drift. The focus is the non-smooth fold in the Stommel 2-box, and related climate models, which are piecewise-smooth continuous dynamical systems, modelling thermohaline circulation. These exhibit non-smooth fold bifurcations which arise when a saddle-point and a focus meet at a border collision bifurcation. By using techniques from the theory of non-smooth dynamical systems we are able to provide precise estimates for the general tipping behaviour close to the non-smooth fold as parameters vary. These are different from the usual tipping point estimates at a saddle–node bifurcation, with advanced tipping apparent in the non-smooth case when compared to the behaviour near smoothed approximations to this fold. We also see rapid, and non-monotone, transitions in the tipping points for oscillatory forced systems close to both non-smooth folds and saddle–node bifurcations due to the effects of both phase changes and non-smoothness. These variations can have implications for the prediction of tipping in climate systems, particularly in close proximity to a non-smooth fold.

A subgradient method with constant step-size for $$\ell _1$$-composite optimization

AbstractSubgradient methods are the natural extension to the non-smooth case of the classical gradient descent for regular convex optimization problems. However, in general, they are characterized by slow convergence rates, and they require decreasing step-sizes to converge. In this paper we propose a subgradient method with constant step-size for composite convex objectives with $$\ell _1$$ ℓ 1 -regularization. If the smooth term is strongly convex, we can establish a linear convergence result for the function values. This fact relies on an accurate choice of the element of the subdifferential used for the update, and on proper actions adopted when non-differentiability regions are crossed. Then, we propose an accelerated version of the algorithm, based on conservative inertial dynamics and on an adaptive restart strategy, that is guaranteed to achieve a linear convergence rate in the strongly convex case. Finally, we test the performances of our algorithms on some strongly and non-strongly convex examples.

The adjoint of an operator on a Banach space

Self-adjoint operators in smooth Banach spaces have been already defined in recent works. Here, we extend the concept of adjoint of an operator to the scope of (non-necessarily Hilbert) Banach spaces, obtaining in particular the notion of self-adjoint operator in the non-smooth case. As a consequence, we define the probability density operator on Banach spaces and verify most of its well-known properties.

Relationships between the maximum principle and dynamic programming for infinite dimensional stochastic control systems

The Pontryagin type maximum principle and Bellman's dynamic programming principle serve as two of the most important tools in solving optimal control problems. There is a huge literature on the study of relationship between them. The main purpose of this paper is to investigate the relationship between the Pontryagin type maximum principle and the dynamic programming principle for control systems governed by stochastic evolution equations in infinite dimensional space, with the control variables entering into both the drift and the diffusion terms. To do so, we first prove the dynamic programming principle for those systems without employing the martingale solutions. Then we establish the desired relationships in both cases that value function associated is smooth or nonsmooth. For the nonsmooth case, in particular, by employing the relaxed transposition solution, we discover the connection between the superdifferentials and subdifferentials of value function and the first-order and second-order adjoint equations.

Optimality conditions for isoperimetric continuous - time optimization problems

In this paper we deal with a nonsmooth case of isoperimetric convex continuoustime programming problem with inequality integral constraint and phase constraint, defined in L?([0, T],Rn). In order to obtain necessary optimality conditions for this problem, we will use a theorem of the alternative from [15] as a main tool.

Optimal control of a nonconserved phase field model of Caginalp type with thermal memory and double obstacle potential

In this paper, we investigate optimal control problems for a nonlinear state system which constitutes a version of the Caginalp phase field system modeling nonisothermal phase transitions with a nonconserved order parameter that takes thermal memory into account. The state system, which is a first-order approximation of a thermodynamically consistent system, is inspired by the theories developed by Green and Naghdi. It consists of two nonlinearly coupled partial differential equations that govern the phase dynamics and the universal balance law for internal energy, written in terms of the phase variable and the so-called thermal displacement, i.e., a primitive with respect to time of temperature. We extend recent results obtained for optimal control problems in which the free energy governing the phase transition was differentiable (i.e., of regular or logarithmic type) to the nonsmooth case of a double obstacle potential. As is well known, in this nondifferentiable case standard methods to establish the existence of appropriate Lagrange multipliers fail. This difficulty is overcome utilizing of the so-called deep quench approach. Namely, the double obstacle potential is approximated by a family of (differentiable) logarithmic ones for which the existence of optimal controls and first-order necessary conditions of optimality in terms of the adjoint state variables and a variational inequality are known. By proving appropriate bounds for the adjoint states of the approximating systems, we can pass to the limit in the corresponding first-order necessary conditions, thereby establishing meaningful first-order necessary optimality conditions also for the case of the double obstacle potential.

On the SQH method for solving optimal control problems with non-smooth state cost functionals or constraints

In this study, we discuss how the sequential quadratic Hamiltonian (SQH) scheme can be used to solve optimal control problems with non-smooth state constraints or non-smooth state cost terms. Theoretical and numerical aspects are both considered. Further, we connect to other recent techniques to solve such optimal control problems. The capability to handle non-smooth state terms extends the possibilities of modeling costs of the state and the dynamics of a system, e.g. to weight the deviation of the state from a desired state in an L1-norm. In this work, the non-smooth state terms are transformed into a bilinear structure on which the SQH scheme proved to solve optimal control problems fast promising an efficient numerical solution also in the non-smooth case. The novelty of the presented approach is that the non-smoothness is transformed into a bilinear structure. It is proved that the SQH method, which is based on the Pontryagin maximum principle, converges to an optimal solution of the corresponding transformed optimal control problem without any globalization techniques.

Global attractor and limit points for nonsmooth ADMM

The alternating direction method of multipliers (ADMM) is frequently used to solve optimization problems derived from machine learning and statistics. Hence it is important to investigate the asymptotic behavior of ADMM. Most works in this direction deal with smooth objective functions excluding various important applications in the nonsmooth case. In this paper, we study the asymptotic behavior of the solutions of differential inclusions arising from ADMM applied to nonsmooth objective functions. More precisely, we associate a multiflow to this differential inclusion and analyze the structure of both ω-limit points and global attractors for this multiflow.

Boundary layer solutions to singularly perturbed quasilinear systems

<p style='text-indent:20px;'>We consider weak boundary layer solutions to the singularly perturbed ODE systems of the type <inline-formula><tex-math id="M1">\begin{document}$ \varepsilon^2\left(A(x, u(x), \varepsilon)u'(x)\right)' = f(x, u(x), \varepsilon) $\end{document}</tex-math></inline-formula>. The new features are that we do not consider one scalar equation, but systems, that the systems are allowed to be quasilinear, and that the systems are spatially non-smooth. Although the results about existence, asymptotic behavior, local uniqueness and stability of boundary layer solutions are similar to those known for semilinear, scalar and smooth problems, there are at least three essential differences. First, the asymptotic convergence rates valid for smooth problems are not true anymore, in general, in the non-smooth case. Second, a specific local uniqueness condition from the scalar case is not sufficient anymore in the vectorial case. And third, the monotonicity condition, which is sufficient for stability of boundary layers in the scalar case, must be adjusted to the vectorial case.</p>

Local minimizers of the Crouzeix ratio: a nonsmooth optimization case study

Given a square matrix A and a polynomial p, the Crouzeix ratio is the norm of the polynomial on the field of values of A divided by the 2-norm of the matrix p(A). Crouzeix’s conjecture states that the globally minimal value of the Crouzeix ratio is 0.5, regardless of the matrix order and polynomial degree, and it is known that 1 is a frequently occurring locally minimal value. Making use of a heavy-tailed distribution to initialize our optimization computations, we demonstrate for the first time that the Crouzeix ratio has many other locally minimal values between 0.5 and 1. Besides showing that the same function values are repeatedly obtained for many different starting points, we also verify that an approximate nonsmooth stationarity condition holds at computed candidate local minimizers. We also find that the same locally minimal values are often obtained both when optimizing over real matrices and polynomials, and over complex matrices and polynomials. We argue that minimization of the Crouzeix ratio makes a very interesting nonsmooth optimization case study, illustrating among other things how effective the BFGS method is for nonsmooth, nonconvex optimization. Our method for verifying approximate nonsmooth stationarity is based on what may be a novel approach to finding approximate subgradients of max functions on an interval. Our extensive computations strongly support Crouzeix’s conjecture: in all cases, we find that the smallest locally minimal value is 0.5.

On volume-preserving crystalline mean curvature flow

In this work we consider the global existence of volume-preserving crystalline curvature flow in a non-convex setting. We show that a natural geometric property, associated with reflection symmetries of the Wulff shape, is preserved with the flow. Using this geometric property, we address global existence and regularity of the flow for smooth anisotropies. For the non-smooth case we establish global existence results for the types of anisotropies known to be globally well-posed.

ON COMPACT 4TH ORDER FINITE-DIFFERENCE SCHEMES FOR THE WAVE EQUATION

We consider compact finite-difference schemes of the 4th approximation order for an initial-boundary value problem (IBVP) for the n-dimensional nonhomogeneous wave equation, n≥ 1. Their construction is accomplished by both the classical Numerov approach and alternative technique based on averaging of the equation, together with further necessary improvements of the arising scheme for n≥ 2. The alternative technique is applicable to other types of PDEs including parabolic and time-dependent Schro¨dinger ones. The schemes are implicit and three-point in each spatial direction and time and include a scheme with a splitting operator for n≥ 2. For n = 1 and the mesh on characteristics, the 4th order scheme becomes explicit and close to an exact four-point scheme. We present a conditional stability theorem covering the cases of stability in strong and weak energy norms with respect to both initial functions and free term in the equation. Its corollary ensures the 4th order error bound in the case of smooth solutions to the IBVP. The main schemes are generalized for non-uniform rectangular meshes. We also give results of numerical experiments showing the sensitive dependence of the error orders in three norms on the weak smoothness order of the initial functions and free term and essential advantages over the 2nd approximation order schemes in the non-smooth case as well.

Differential Games for Fractional-Order Systems: Hamilton–Jacobi–Bellman–Isaacs Equation and Optimal Feedback Strategies

The paper deals with a two-person zero-sum differential game for a dynamical system described by differential equations with the Caputo fractional derivatives of an order α∈(0,1) and a Bolza-type cost functional. A relationship between the differential game and the Cauchy problem for the corresponding Hamilton–Jacobi–Bellman–Isaacs equation with fractional coinvariant derivatives of the order α and the natural boundary condition is established. An emphasis is given to construction of optimal positional (feedback) strategies of the players. First, a smooth case is studied when the considered Cauchy problem is assumed to have a sufficiently smooth solution. After that, to cope with a general non-smooth case, a generalized minimax solution of this problem is involved.

Soft quantum waveguides with an explicit cut-locus

We consider two-dimensional Schrödinger operators with an attractive potential in the form of a channel of a fixed profile built along an unbounded curve composed of a circular arc and two straight semi-lines. Using a test-function argument with help of parallel coordinates outside the cut-locus of the curve, we establish the existence of discrete eigenvalues. This is a special variant of a recent result of Exner (2020 J. Phys. A: Math. Theor. 53 355302) in a non-smooth case and via a different technique which does not require non-positive constraining potentials.