Second-order Variational Analysis Research Articles

Parabolic Regularity of Spectral Functions

This paper is devoted to the study of the second-order variational analysis of spectral functions. It is well-known that spectral functions can be expressed as a composite function of symmetric functions and eigenvalue functions. We establish several second-order properties of spectral functions when their associated symmetric functions enjoy these properties. Our main attention is given to characterize parabolic regularity for this class of functions. It was observed recently that parabolic regularity can play a central rule in ensuring the validity of important second-order variational properties, such as twice epi-differentiability. We demonstrates that for convex spectral functions, their parabolic regularity amounts to that of their symmetric functions. As an important consequence, we calculate the second subderivative of convex spectral functions, which allows us to establish second-order optimality conditions for a class of matrix optimization problems. Funding: The research of A. Mohammadi is funded by a postdoctoral fellowship from Georgetown University. E. Sarabi is partially supported by the U.S. National Science Foundation [Grant DMS 2108546].

Generalized damped Newton algorithms in nonsmooth optimization via second-order subdifferentials

The paper proposes and develops new globally convergent algorithms of the generalized damped Newton type for solving important classes of nonsmooth optimization problems. These algorithms are based on the theory and calculations of second-order subdifferentials of nonsmooth functions with employing the machinery of second-order variational analysis and generalized differentiation. First we develop a globally superlinearly convergent damped Newton-type algorithm for the class of continuously differentiable functions with Lipschitzian gradients, which are nonsmooth of second order. Then we design such a globally convergent algorithm to solve a structured class of nonsmooth quadratic composite problems with extended-real-valued cost functions, which typically arise in machine learning and statistics. Finally, we present the results of numerical experiments and compare the performance of our main algorithm applied to an important class of Lasso problems with those achieved by other first-order and second-order optimization algorithms.

Optimal control of nonconvex integro-differential sweeping processes

This paper is devoted to the study, for the first time in the literature, of optimal control problems for sweeping processes governed by integro-differential inclusions of the Volterra type with different classes of control functions acting in nonconvex moving sets, external dynamic perturbations, and integral parts of the sweeping dynamics. We establish the existence of optimal solutions and then obtain necessary optimality conditions for a broad class of local minimizers in such problems. Our approach to deriving necessary optimality conditions is based on the method of discrete approximations married to basic constructions and calculus rules of first-order and second-order variational analysis and generalized differentiation. The obtained necessary optimality conditions are expressed entirely in terms of the problem data and are illustrated by nontrivial examples that include applications to optimal control models of non-regular electrical circuits.

Quadratic Growth Conditions and Uniqueness of Optimal Solution to Lasso

In the previous paper Bello-Cruz et al. (J Optim Theory Appl 188:378–401, 2021), we showed that the quadratic growth condition plays a key role in obtaining Q-linear convergence of the widely used forward–backward splitting method with Beck–Teboulle’s line search. In this paper, we analyze the property of quadratic growth condition via second-order variational analysis for various structured optimization problems that arise in machine learning and signal processing. This includes, for example, the Poisson linear inverse problem as well as the ell _1-regularized optimization problems. As a by-product of this approach, we also obtain several full characterizations for the uniqueness of optimal solution to Lasso problem, which complements and extends recent important results in this direction.

Augmented Lagrangian method for second-order cone programs under second-order sufficiency

This paper addresses problems of second-order cone programming important in optimization theory and applications. The main attention is paid to the augmented Lagrangian method (ALM) for such problems considered in both exact and inexact forms. Using generalized differential tools of second-order variational analysis, we formulate the corresponding version of second-order sufficiency and use it to establish, among other results, the uniform second-order growth condition for the augmented Lagrangian. The latter allows us to justify the solvability of subproblems in the ALM and to prove the linear primal–dual convergence of this method.

Variational Analysis of Composite Models with Applications to Continuous Optimization

The paper is devoted to a comprehensive study of composite models in variational analysis and optimization the importance of which for numerous theoretical, algorithmic, and applied issues of operations research is difficult to overstate. The underlying theme of our study is a systematical replacement of conventional metric regularity and related requirements by much weaker metric subregulatity ones that lead us to significantly stronger and completely new results of first-order and second-order variational analysis and optimization. In this way, we develop extended calculus rules for first-order and second-order generalized differential constructions while paying the main attention in second-order variational theory to the new and rather large class of fully subamenable compositions. Applications to optimization include deriving enhanced no-gap second-order optimality conditions in constrained composite models, complete characterizations of the uniqueness of Lagrange multipliers, strong metric subregularity of Karush-Kuhn-Tucker systems in parametric optimization, and so on.

Parabolic regularity in geometric variational analysis

The paper is mainly devoted to systematic developments and applications of geometric aspects of second-order variational analysis that are revolved around the concept of parabolic regularity of sets. This concept has been known in variational analysis for more than two decades while being largely underinvestigated. We discover here that parabolic regularity is the key to derive new calculus rules and computation formulas for major second-order generalized differential constructions of variational analysis in connection with some properties of sets that go back to classical differential geometry and geometric measure theory. The established results of second-order variational analysis and generalized differentiation, being married to the developed calculus of parabolic regularity, allow us to obtain novel applications to both qualitative and quantitative/numerical aspects of constrained optimization including second-order optimality conditions, augmented Lagrangians, etc. under weak constraint qualifications.

Optimization and discrete approximation of sweeping processes with controlled moving sets and perturbations

This paper addresses a new class of optimal control problems for perturbed sweeping processes with measurable controls in additive perturbations of the dynamics and smooth controls in polyhedral moving sets. We develop a constructive discrete approximation procedure that allows us to strongly approximate any feasible trajectory of the controlled sweeping process by feasible discrete trajectories and also establish a W1,2-strong convergence of optimal trajectories for discretized control problems to a given local minimizer of the original continuous-time sweeping control problem of the Bolza type. Employing advanced tools of first-order and second-order variational analysis and generalized differentiation, we derive necessary optimality conditions for discrete optimal solutions under fairly general assumptions formulated entirely in terms of the given data. The obtained results give us efficient suboptimality (“almost optimality”) conditions for the original sweeping control problem that are illustrated by a nontrivial numerical example.

Criticality of Lagrange multipliers in extended nonlinear optimization

ABSTRACT The paper is devoted to the study and applications of criticality of Lagrange multipliers in variational systems, which are associated with the class of problems in composite optimization known as extended nonlinear programming (ENLP). The importance of both ENLP and the concept of multiplier criticality in variational systems has been recognized in theoretical and numerical aspects of optimization and variational analysis, while the criticality notion has never been investigated in the ENLP framework. We present here a systematic study of critical and noncritical multipliers in a general variational setting that covers, in particular, KKT systems in ENLP with establishing their verifiable characterizations as well as relationships between noncriticality and other stability notions in variational analysis. Our approach is mainly based on advanced tools of second-order variational analysis and generalized differentiation.

Second-Order Variational Analysis of Parametric Constraint and Variational Systems

This paper is devoted to the generalized differential study of the normal cone mappings associated with a large class of parametric constraint systems (PCS) that appear, in particular, in nonpolyhedral conic programming. Conducting a local second-order analysis of such systems, we focus on computing the (primal-dual) graphical derivative of the normal cone mapping under the $C^2$-cone reducibility of the constraint set together with the fairly weak metric subregularity constraint qualification and its uniform parametric counterpart known as Robinson stability. The obtained precise formulas for computing the underlying second-order object are applied to the derivation of comprehensive conditions ensuring the important stability property of isolated calmness for solution maps to parametric variational systems associated with the given PCS.

Second-order variational analysis in second-order cone programming

The paper conducts a second-order variational analysis for an important class of nonpolyhedral conic programs generated by the so-called second-order/Lorentz/ice-cream cone \({\mathcal {Q}}\). From one hand, we prove that the indicator function of \({\mathcal {Q}}\) is always twice epi-differentiable and apply this result to characterizing the uniqueness of Lagrange multipliers together with an error bound estimate in the general second-order cone programming setting involving twice differentiable data. On the other hand, we precisely calculate the graphical derivative of the normal cone mapping to \({\mathcal {Q}}\) under the metric subregularity constraint qualification and then give an application of the latter result to a complete characterization of isolated calmness for perturbed variational systems associated with second-order cone programs. The obtained results seem to be the first in the literature in these directions for nonpolyhedral problems without imposing any nondegeneracy assumptions.

Extended Euler–Lagrange and Hamiltonian Conditions in Optimal Control of Sweeping Processes with Controlled Moving Sets

This paper concerns optimal control problems for a class of sweeping processes governed by discontinuous unbounded differential inclusions that are described via normal cone mappings to controlled moving sets. Largely motivated by applications to hysteresis, we consider a general setting where moving sets are given as inverse images of closed subsets of finite-dimensional spaces under nonlinear differentiable mappings dependent on both state and control variables. Developing the method of discrete approximations and employing generalized differential tools of first-order and second-order variational analysis allow us to derive nondegenerate necessary optimality conditions for such problems in extended Euler–Lagrange and Hamiltonian forms involving the Hamiltonian maximization. The latter conditions of the Pontryagin Maximum Principle type are the first in the literature for optimal control of sweeping processes with control-dependent moving sets.

Optimal control of a nonconvex perturbed sweeping process

The paper concerns optimal control of discontinuous differential inclusions of the normal cone type governed by a generalized version of the Moreau sweeping process with control functions acting in both nonconvex moving sets and additive perturbations. This is a new class of optimal control problems in comparison with previously considered counterparts where the controlled sweeping sets are described by convex polyhedra. Besides a theoretical interest, a major motivation for our study of such challenging optimal control problems with intrinsic state constraints comes from the application to the crowd motion model in a practically adequate planar setting with nonconvex but prox-regular sweeping sets. Based on a constructive discrete approximation approach and advanced tools of first-order and second-order variational analysis and generalized differentiation, we establish the strong convergence of discrete optimal solutions and derive a complete set of necessary optimality conditions for discrete-time and continuous-time sweeping control systems that are expressed entirely via the problem data.

Full Stability of General Parametric Variational Systems

The paper introduces and studies the notions of Lipschitzian and Holderian full stability of solutions to three-parametric variational systems described in the generalized equation formalism involving nonsmooth base mappings and partial subgradients of prox-regular functions acting in Hilbert spaces. Employing advanced tools and techniques of second-order variational analysis allows us to establish complete characterizations of, as well as directly verifiable sufficient conditions for, such full stability properties under mild assumptions. Furthermore, we derive exact formulas and effective quantitative estimates for the corresponding moduli. The obtained results are specified for important classes of variational inequalities and variational conditions in both finite and infinite dimensions.

Stability Analysis for Composite Optimization Problems and Parametric Variational Systems

This paper aims to provide various applications for second-order variational analysis of extended-real-valued piecewise linear functions recently obtained by the authors. We mainly focus here on establishing relationships between full stability of local minimizers in composite optimization and Robinson's strong regularity of associated (linearized and nonlinearized) KKT systems. Finally, we address Lipschitzian stability of parametric variational systems with convex piecewise linear potentials.

Second-Order Analysis of Piecewise Linear Functions with Applications to Optimization and Stability

This paper is devoted to second-order variational analysis of a rather broad class of extended-real-valued piecewise liner functions and their applications to various issues of optimization and stability. Based on our recent explicit calculations of the second-order subdifferential for such functions, we establish relationships between nondegeneracy and second-order qualification for fully amenable compositions involving piecewise linear functions. We then provide a second-order characterization of full stable local minimizers in composite optimization and constrained minimax problems.

Generalized differentiation of piecewise linear functions in second-order variational analysis

The paper is devoted to a comprehensive second-order study of a remarkable class of convex extended-real-valued functions that is highly important in many aspects of nonlinear and variational analysis, specifically those related to optimization and stability. This class consists of lower semicontinuous functions with possibly infinite values on finite-dimensional spaces, which are labeled as “piecewise linear” ones and can be equivalently described via the convexity of their epigraphs. In this paper we calculate the second-order subdifferentials (generalized Hessians) of arbitrary convex piecewise linear functions, together with the corresponding geometric objects, entirely in terms of their initial data. The obtained formulas allow us, in particular, to justify a new exact (equality-type) second-order sum rule for such functions in the general nonsmooth setting.

Optimal control of the sweeping process over polyhedral controlled sets

The paper addresses a new class of optimal control problems governed by the dissipative and discontinuous differential inclusion of the sweeping/Moreau process while using controls to determine the best shape of moving convex polyhedra in order to optimize the given Bolza-type functional, which depends on control and state variables as well as their velocities. Besides the highly non-Lipschitzian nature of the unbounded differential inclusion of the controlled sweeping process, the optimal control problems under consideration contain intrinsic state constraints of the inequality and equality types. All of this creates serious challenges for deriving necessary optimality conditions. We develop here the method of discrete approximations and combine it with advanced tools of first-order and second-order variational analysis and generalized differentiation. This approach allows us to establish constructive necessary optimality conditions for local minimizers of the controlled sweeping process expressed entirely in terms of the problem data under fairly unrestrictive assumptions. As a by-product of the developed approach, we prove the strong W1,2-convergence of optimal solutions of discrete approximations to a given local minimizer of the continuous-time system and derive necessary optimality conditions for the discrete counterparts. The established necessary optimality conditions for the sweeping process are illustrated by several examples.

Second-Order Variational Analysis in Conic Programming with Applications to Optimality and Stability

This paper is devoted to the study of a broad class of problems in conic programming modeled via parameter-dependent generalized equations. In this framework we develop a second-order generalized differential approach of variational analysis to calculate appropriate derivatives and coderivatives of the corresponding solution maps. These developments allow us to resolve some important issues related to conic programming. They include verifiable conditions for isolated calmness of the considered solution maps, sharp necessary optimality conditions for a class of mathematical programs with equilibrium constraints, and characterizations of tilt-stable local minimizers for cone-constrained problems. The main results obtained in the general conic programming setting are specified for and illustrated by the second-order cone programming.

Variational analysis and full stability of optimal solutions to constrained and minimax problems

The main goal of this paper is to develop applications of advanced tools of first-order and second-order variational analysis and generalized differentiation to the fundamental notion of full stability of local minimizers of general classes of constrained optimization and minimax problems. In particular, we derive second-order characterizations of full stability and investigate its relationships with other notions of stability for parameterized conic programs and minimax problems. Furthermore, the developed variational approach allows us to largely unify and provide new self-contained proofs of some quite recent results in this direction for problems of constrained optimization with C2-smooth data.