Nonsmooth Problems Research Articles

Inverse problems for constrained parabolic variational-hemivariational inequalities *

In this paper we study a novel class of inverse problems for parabolic variational–hemivariational inequalities with a unilateral constraint. A theorem on the well-posedness for weak solution is established. Based on a new continuous dependence result, we prove the nonemptiness and stability of the set of optimal solutions to the corresponding inverse problems. We illustrate the results by a quasistatic nonsmooth frictional viscoelastic contact problem with a unilateral constraint for which we derive results on the unique weak solvability, and existence and stability of solutions to the associated inverse problems.

A customized inertial proximal alternating minimization for SVD-free robust principal component analysis

Robust principal component analysis (RPCA) is devoted to tackling grossly corrupted datasets with noise. However, the performance of RPCA is usually circumscribed by the lack of efficiency of singular value decomposition (SVD), which rules out its potential applications to many large-scale real-world problems. In this paper, we develop a nonconvex optimization algorithm customized to SVD-free RPCA models. The proposed algorithm, which is built upon proximal alternating linearized minimization Bolte et al. [Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math Program. 2014;146(1–2):459–494], can reduce computational efforts by partially linearizing data fidelity and increase efficiency by leveraging inertial techniques. Under the Kurdyka-Łojasiewicz assumption on the objective function and some mild premises on stepsizes, the sequence produced by the proposed algorithm converges globally to a critical point of SVD-free RPCA models. Numerical simulations on synthetic and real datasets demonstrate the compelling performance of the proposed algorithm.

Secure Transmission Design for Virtual Antenna Array-Aided Device-to-Device Multicast Communications

This paper investigates the physical-layer security in a Virtual Antenna Array (VAA)-aided Device-to-Device Multicast (D2DM) communication system, where the User Equipments (UEs) who cache the common content will form a VAA system and cooperatively multicast the content to UEs who desire it. Specifically, with the target of securing the VAA-aided D2DM communication under the threat of multiple eavesdroppers, we propose a secure beamforming scheme by jointly considering the formed VAA and the Base Station (BS). For obtaining the optimal beamforing vectors, a nonsmooth and nonconvex Weight Sum Rate Maximization Problem (WRMP) is formulated and solved using Successive Convex Approximation (SCA) approach. Furthermore, we consider the worst case that eavesdroppers cooperatively form a eavesdrop VAA to enhance the overhearing capacity. In the worst case, we modify the securing beamforming scheme, formulate the corresponding WRMP and solve it using a two-level optimization. Simulation results validate the improvements of the VAA-aided D2DM scheme in terms of communication security compared with conventional D2DM schemes.

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

A proximal subgradient algorithm with extrapolation for structured nonconvex nonsmooth problems

In this paper, we consider a class of structured nonconvex nonsmooth optimization problems, in which the objective function is formed by the sum of a possibly nonsmooth nonconvex function and a differentiable function with Lipschitz continuous gradient, subtracted by a weakly convex function. This general framework allows us to tackle problems involving nonconvex loss functions and problems with specific nonconvex constraints, and it has many applications such as signal recovery, compressed sensing, and optimal power flow distribution. We develop a proximal subgradient algorithm with extrapolation for solving these problems with guaranteed subsequential convergence to a stationary point. The convergence of the whole sequence generated by our algorithm is also established under the widely used Kurdyka–Łojasiewicz property. To illustrate the promising numerical performance of the proposed algorithm, we conduct numerical experiments on two important nonconvex models. These include a compressed sensing problem with a nonconvex regularization and an optimal power flow problem with distributed energy resources.

Small-signal stability constrained unit commitment based on decomposition and SQP-GS

This paper proposes a small-signal stability constrained unit commitment(SSSC-UC) model to ensure that the solution of SCUC can provide a sufficient small-signal stability margin. To solve the model, Benders decomposition (BD) is applied to separate the problem into a SCUC master problem, which is a mixed-integer linear programming problem, and a series of subproblems to check the AC power flow constraint and small-signal stability constraint. The small-signal stability check subproblems are non-smooth and non-linear programming problems, which are solved by Sequential Quadratic Programming with Gradient Sampling (SQP-GS). When the subproblem checks do not pass, cuts formed by the sensitivity of the spectral abscissa with respect to generator output are added to the master problem for the next iteration of the SCUC. The iteration process continues until all AC power flow and small-signal constraint violations are eliminated and a SCUC solution is found. Simulations on the New England 10-machine 39-bus system and the IEEE 54-machine 118-bus system show that our proposed SSSC-UC can guarantee small-signal stability for all time periods while SCUC cannot. Furthermore, the proposed parallel technique can speed up the computation up to 5 and 11 times respectively for the two test systems.

Optimization and Learning With Randomly Compressed Gradient Updates.

Gradient descent methods are simple and efficient optimization algorithms with widespread applications. To handle high-dimensional problems, we study compressed stochastic gradient descent (SGD) with low-dimensional gradient updates. We provide a detailed analysis in terms of both optimization rates and generalization rates. To this end, we develop uniform stability bounds for CompSGD for both smooth and nonsmooth problems, based on which we develop almost optimal population risk bounds. Then we extend our analysis to two variants of SGD: batch and mini-batch gradient descent. Furthermore, we show that these variants achieve almost optimal rates compared to their high-dimensional gradient setting. Thus, our results provide a way to reduce the dimension of gradient updates without affecting the convergence rate in the generalization analysis. Moreover, we show that the same result also holds in the differentially private setting, which allows us to reduce the dimension of added noise with "almost free" cost.

Accelerated gradient methods with absolute and relative noise in the gradient

In this paper, we investigate accelerated first-order methods for smooth convex optimization problems under inexact information on the gradient of the objective. The noise in the gradient is considered to be additive with two possibilities: absolute noise bounded by a constant, and relative noise proportional to the norm of the gradient. We investigate the accumulation of the errors in the convex and strongly convex settings with the main difference with most of the previous works being that the feasible set can be unbounded. The key to the latter is to prove a bound on the trajectory of the algorithm. We also give a stopping criterion for the algorithm and consider extensions to the cases of stochastic optimization and composite nonsmooth problems.

Convergence of a Class of Nonmonotone Descent Methods for Kurdyka–Łojasiewicz Optimization Problems

This note is concerned with a class of nonmonotone descent methods for minimizing a proper lower semicontinuous Kurdyka–Łojasiewicz (KL) function , which generates a sequence satisfying a nonmonotone decrease condition and a relative error tolerance. Under suitable assumptions, we prove that the whole sequence converges to a limiting critical point of , and, when is a KL function of exponent , the convergence admits a linear rate if and a sublinear rate associated to if . These assumptions are shown to be sufficient and necessary if in addition is weakly convex on a neighborhood of the set of critical points. Our results not only extend the convergence results of monotone descent methods for KL optimization problems but also resolve the convergence problem on the iterate sequence generated by a class of nonmonotone line search algorithms for nonconvex and nonsmooth problems.

Convergence rate of inertial forward–backward algorithms based on the local error bound condition

Abstract The ‘inertial forward–backward algorithm’ (IFB) is a powerful algorithm for solving a class of convex non-smooth minimization problems, IFB relies on an inertial parameter $\gamma _{k}$ whose tuning is crucial for achieving accelerated convergence speeds as compared to the classical forward–backward algorithm. Under the local error bound condition, it is known that IFB converges R-linearly as soon as the inertial parameter satisfies ${\sup _{k}}{\gamma _{k}} \leqslant \tilde{\gamma } <1.$ On the contrary, we are not aware of any convergence result for the case ${\sup _{k}}{\gamma _{k}} = 1.$ In this paper, we consider six different choices of inertial parameters satisfying this last condition, and show convergence of the corresponding IFB algorithms under the local error bound condition. Finally, we propose a class of inertial forward–backward algorithm with an adaptive modification (IFB_AdapM) and show that it enjoys the same convergence results.

Truncated nonsmooth Newton multigrid for phase-field brittle-fracture problems, with analysis

AbstractWe propose the truncated nonsmooth Newton multigrid method (TNNMG) as a solver for the spatial problems of the small-strain brittle-fracture phase-field equations. TNNMG is a nonsmooth multigrid method that can solve biconvex, block-separably nonsmooth minimization problems with linear time complexity. It exploits the variational structure inherent in the problem, and handles the pointwise irreversibility constraint on the damage variable directly, without regularization or the introduction of a local history field. In the paper we introduce the method and show how it can be applied to several established models of phase-field brittle fracture. We then prove convergence of the solver to a solution of the nonsmooth Euler–Lagrange equations of the spatial problem for any load and initial iterate. On the way, we show several crucial convexity and regularity properties of the models considered here. Numerical comparisons to an operator-splitting algorithm show a considerable speed increase, without loss of robustness.

A Bregman stochastic method for nonconvex nonsmooth problem beyond global Lipschitz gradient continuity

ABSTRACT In this paper, we consider solving a broad class of large-scale nonconvex and nonsmooth minimization problems by a Bregman proximal stochastic gradient (BPSG) algorithm. The objective function of the minimization problem is the composition of a differentiable and a nondifferentiable function, and the differentiable part does not admit a global Lipschitz continuous gradient. Under some suitable conditions, the subsequential convergence of the proposed algorithm is established. And under expectation conditions with the Kurdyka-Łojasiewicz (KL) property, we also prove that the proposed method converges globally. We also apply the BPSG algorithm to solve sparse nonnegative matrix factorization (NMF), symmetric NMF via non-symmetric relaxation, and matrix completion problems under different kernel generating distances, and numerically compare it with other algorithms. The results demonstrate the robustness and effectiveness of the proposed algorithm.

A Family of Multi-Step Subgradient Minimization Methods

For solving non-smooth multidimensional optimization problems, we present a family of relaxation subgradient methods (RSMs) with a built-in algorithm for finding the descent direction that forms an acute angle with all subgradients in the neighborhood of the current minimum. Minimizing the function along the opposite direction (with a minus sign) enables the algorithm to go beyond the neighborhood of the current minimum. The family of algorithms for finding the descent direction is based on solving systems of inequalities. The finite convergence of the algorithms on separable bounded sets is proved. Algorithms for solving systems of inequalities are used to organize the RSM family. On quadratic functions, the methods of the RSM family are equivalent to the conjugate gradient method (CGM). The methods are intended for solving high-dimensional problems and are studied theoretically and numerically. Examples of solving convex and non-convex smooth and non-smooth problems of large dimensions are given.

Direct, fast and convergent solvers for the non-convex and non-smooth TDoA localization problem

Time-Difference-of-Arrival (TDoA) Source Localization (SL) consists of determining the position of a target given differences of time measurements of several sensors. The TDoA SL problem is formulated as minimizing a non-convex and non-smooth least squares (LS) optimization problem, which is tackled in the existing literature mostly by solving convex relaxations or by taking the square of the measurements. In this work, we tackle the LS problem directly, by first showing that a standard fixed-point (FP) method can be derived. Due to numerical instability and lack of theoretical convergence guarantees of FP, we develop the T-NAM method. This easy-to-implement and novel method uses the Nested Alternating Minimization scheme together with the fast FISTA algorithm. We show that T-NAM converges to critical points of the original LS function – a result that, to the best of our knowledge, is unknown for the TDoA SL problem. Last, we show numerically the advantages of FP and T-NAM relatively to existing works.

On an iteratively reweighted linesearch based algorithm for nonconvex composite optimization

In this paper we propose a new algorithm for solving a class of nonsmooth nonconvex problems, which is obtained by combining the iteratively reweighted scheme with a finite number of forward–backward iterations based on a linesearch procedure. The new method overcomes some limitations of linesearch forward–backward methods, since it can be applied also to minimize functions containing terms that are both nonsmooth and nonconvex. Moreover, the combined scheme can take advantage of acceleration techniques consisting in suitable selection rules for the algorithm parameters. We develop the convergence analysis of the new method within the framework of the Kurdyka–Łojasiewicz property. Finally, we present the results of a numerical experience on microscopy image super resolution, showing that the performances of our method are comparable or superior to those of other algorithms designed for this specific application.

A generalized inertial proximal alternating linearized minimization method for nonconvex nonsmooth problems

We propose a general inertial version of the proximal alternating linearized minimization (PALM) (denoted by NiPALM) for a class of nonconvex and nonsmooth minimization problems, whose objective function is the sum of a smooth function of the entire variables and two nonsmooth functions of each variable. NiPALM is general in the sense that it contains the popular PALM, the inertial PALM (iPALM) and Gauss-Seidel type inertial PALM (GiPALM) as special cases. Under mild assumptions, namely, the underlying functions satisfy the Kurdyka-Łojasiewicz (KL) property and some suitable conditions on the parameters, we prove that each bounded sequence generated by NiPALM globally converges to a critical point. We also apply NiPALM to nonnegative matrix factorization, sparse principal component analysis, and weighted low-rank matrix restoration problems. Comparing results with those by PALM, iPALM, and GiPALM demonstrate the robustness and effectiveness of the proposed algorithm.

Generalized Inexact Newton-Landweber Iteration for Possibly Non-Smooth Inverse Problems in Banach Spaces

In this paper, we consider a generalized inexact Newton-Landweber iteration to solve nonlinear ill-posed inverse problems in Banach spaces, where the forward operator might not be Gâteaux differentiable. The method is designed with non-smooth convex penalty terms, including L1-like and total variation-like penalty functionals, to capture special features of solutions such as sparsity and piecewise constancy. Furthermore, the inaccurate inner solver is incorporated into the minimization problem in each iteration step. Under some assumptions, based on ε-subdifferential, we establish the convergence analysis of the proposed method. Finally, some numerical simulations are provided to illustrate the effectiveness of the method for solving both smooth and non-smooth nonlinear inverse problems.

Nonsmooth semi-infinite roughly B-invex multi-objective programming problems

In this paper, we will study a nonsmooth semi-infinite multi-objective programming problem involving roughly B-invex functions. Sufficient optimality conditions for the primal problem are derived. We shall formulate Mond–Weir type dual and establish weak and strong duality theorems under various roughly B-invexity assumptions.

Continuous gap contact formulation based on the screened Poisson equation

We introduce a PDE-based node-to-element contact formulation as an alternative to classical, purely geometrical formulations. It is challenging to devise solutions to nonsmooth contact problem with continuous gap using finite element discretizations. We herein achieve this objective by constructing an approximate distance function (ADF) to the boundaries of solid objects, and in doing so, also obtain universal uniqueness of contact detection. Unilateral constraints are implemented using a mixed model combining the screened Poisson equation and a force element, which has the topology of a continuum element containing an additional incident node. An ADF is obtained by solving the screened Poisson equation with constant essential boundary conditions and a variable transformation. The ADF does not explicitly depend on the number of objects and a single solution of the partial differential equation for this field uniquely defines the contact conditions for all incident points in the mesh. Having an ADF field to any obstacle circumvents the multiple target surfaces and avoids the specific data structures present in traditional contact-impact algorithms. We also relax the interpretation of the Lagrange multipliers as contact forces, and the Courant–Beltrami function is used with a mixed formulation producing the required differentiable result. We demonstrate the advantages of the new approach in two- and three-dimensional problems that are solved using Newton iterations. Simultaneous constraints for each incident point are considered.

Mini-batch stochastic subgradient for functional constrained optimization

In this paper, we consider finite sum composite optimization problems with many functional constraints. The objective function is expressed as a finite sum of two terms, one of which admits easy computation of (sub)gradients while the other is amenable to proximal evaluations. We assume a generalized bounded gradient condition on the objective which allows us to simultaneously tackle both smooth and nonsmooth problems. We also consider the cases of both with and without a quadratic functional growth property. Further, we assume that each constraint set is given as the level set of a convex but not necessarily a differentiable function. We reformulate the constrained finite sum problem into a stochastic optimization problem for which the stochastic subgradient projection method from Necoara and Singh [Stochastic subgradient projection methods for composite optimization with functional constraints; 2022 Journal of Machine Learning Research, 23, 1–35] specializes in a collection of mini-batch variants, with different mini-batch sizes for the objective function and functional constraints, respectively. More specifically, at each iteration, our algorithm takes a mini-batch stochastic proximal subgradient step aimed at minimizing the objective function and then a subsequent mini-batch subgradient projection step minimizing the feasibility violation. By specializing in different mini-batching strategies, we derive exact expressions for the stepsizes as a function of the mini-batch size and in some cases we also derive insightful stepsize-switching rules which describe when one should switch from a constant to a decreasing stepsize regime. We also prove sublinear convergence rates for the mini-batch subgradient projection algorithm which depend explicitly on the mini-batch sizes and on the properties of the objective function. Numerical results also show a better performance of our mini-batch scheme over its single-batch counterpart.