Class Of Nonconvex Research Articles

Convergence of a class of algorithms with the level-set subdifferential error bound

In this paper, we study the convergence of algorithms for a class of nonconvex and nonsmooth optimization problems via an level-set subdifferential error bound. Many algorithms (cf. forward-backward splitting algorithm and alternating proximal minimization algorithm) satisfy sufficient descent and relative error conditions. We also find that these algorithms satisfy a quadratic error condition. Under these conditions and the level-set subdifferential error bound, we study the global convergence and the convergence rates of these algorithms without the Kurdyka-Łojasiewicz property which is stronger than the level-set subdifferential error bound. Two examples are given to illustrate the broader applicability of our findings across these algorithms.

A Bregman proximal subgradient algorithm for nonconvex and nonsmooth fractional optimization problems

In this paper, we study a class of nonconvex and nonsmooth fractional optimization problem, where the numerator of which is the sum of a nonsmooth and nonconvex function and a relative smooth nonconvex function, while the denominator is relative weakly convex nonsmooth function. We propose a Bregman proximal subgradient algorithm for solving this type of fractional optimization problems. Under moderate conditions, we prove that the subsequence generated by the proposed algorithm converges to a critical point, and the generated sequence globally converges to a critical point when the objective function satisfies the Kurdyka-Łojasiewicz property. We also obtain the convergence rate of the proposed algorithm. Finally, two numerical experiments illustrate the effectiveness and superiority of the algorithm. Our results give a positive answer to an open problem proposed by Bot et al. [14].

Proximal Gradient Method with Extrapolation and Line Search for a Class of Non-convex and Non-smooth Problems

Proximal Gradient Method with Extrapolation and Line Search for a Class of Non-convex and Non-smooth Problems

On the exactness and the convergence of the l_{1} exact penalty E-function method for E-differentiable optimization problems

This paper is devoted to introduce and investigate a new exact penalty function method which is called the l_{1} exact penalty E-function method. Namely, we use the aforesaid exact penalty function method to solve a completely new class of nonconvex (not necessarily) differentiable mathematical programming problems, that is, E-differentiable minimization problems. Then, we analyze the most important from a practical point of view property of all exact penalty function methods, that is, exactness of the penalization. Thus, under appropriate E-convexity hypotheses, we prove the equivalence between the original E-differentiable extremum problem and its corresponding penalized optimization problem created in the introduced l_{1} exact penalty E-function method. Further, we also present and investigate the algorithm for this exact penalty function method which minimizes the l_{1} exact penalty E-function. The convergence theorem for the aforesaid algorithm is also established.

A generalized inertial proximal alternating linearized minimization method for nonconvex nonsmooth problems

We propose a general inertial version of the proximal alternating linearized minimization (PALM) (denoted by NiPALM) for a class of nonconvex and nonsmooth minimization problems, whose objective function is the sum of a smooth function of the entire variables and two nonsmooth functions of each variable. NiPALM is general in the sense that it contains the popular PALM, the inertial PALM (iPALM) and Gauss-Seidel type inertial PALM (GiPALM) as special cases. Under mild assumptions, namely, the underlying functions satisfy the Kurdyka-Łojasiewicz (KL) property and some suitable conditions on the parameters, we prove that each bounded sequence generated by NiPALM globally converges to a critical point. We also apply NiPALM to nonnegative matrix factorization, sparse principal component analysis, and weighted low-rank matrix restoration problems. Comparing results with those by PALM, iPALM, and GiPALM demonstrate the robustness and effectiveness of the proposed algorithm.

Short Paper - A note on the Frank–Wolfe algorithm for a class of nonconvex and nonsmooth optimization problems

Frank and Wolfe’s celebrated conditional gradient method is a well-known tool for solving smooth optimization problems for which minimizing a linear function over the feasible set is computationally cheap. However, when the objective function is nonsmooth, the method may fail to compute a stationary point. In this work, we show that the Frank–Wolfe algorithm can be employed to compute Clarke-stationary points for nonconvex and nonsmooth optimization problems consisting of minimizing upper-C 1,α functions over convex and compact sets. Furthermore, under more restrictive assumptions, we propose a new algorithm variant with stronger stationarity guarantees, namely directional stationarity and even local optimality.

An inertial alternating minimization with Bregman distance for a class of nonconvex and nonsmooth problems

An inertial alternating minimization with Bregman distance for a class of nonconvex and nonsmooth problems

Proximal ADMM for nonconvex and nonsmooth optimization

By enabling the nodes or agents to solve small-sized subproblems to achieve coordination, distributed algorithms are favored by many networked systems for efficient and scalable computation. While for convex problems, substantial distributed algorithms are available, the results for the more broad nonconvex counterparts are extremely lacking. This paper develops a distributed algorithm for a class of nonconvex and nonsmooth problems featured by (i) a nonconvex objective formed by both separate and composite components regarding the decision variables of interconnected agents, (ii) local bounded convex constraints, and (iii) coupled linear constraints. This problem is directly originated from smart buildings and is also broad in other domains. To provide a distributed algorithm with convergence guarantee, we revise the powerful alternating direction method of multiplier (ADMM) method and proposed a proximal ADMM. Specifically, noting that the main difficulty to establish the convergence for the nonconvex and nonsmooth optimization with ADMM is to assume the boundness of dual updates, we propose to update the dual variables in a discounted manner. This leads to the establishment of a so-called sufficiently decreasing and lower bounded Lyapunov function, which is critical to establish the convergence. We prove that the method converges to some approximate stationary points. We besides showcase the efficacy and performance of the method by a numerical example and the concrete application to multi-zone heating, ventilation, and air-conditioning (HVAC) control in smart buildings.

Stochastic homogenization of a class of nonconvex viscous HJ equations in one space dimension

We prove homogenization for a class of nonconvex (possibly degenerate) viscous Hamilton-Jacobi equations in stationary ergodic random environments in one space dimension. The results concern Hamiltonians of the form G(p)+V(x,ω), where the nonlinearity G is the minimum of two or more convex functions with the same absolute minimum, and the potential V is a bounded stationary process satisfying an additional scaled hill and valley condition. This condition is trivially satisfied in the inviscid case, while it is equivalent to the original hill and valley condition of A. Yilmaz and O. Zeitouni [32] in the uniformly elliptic case. Our approach is based on PDE methods and does not rely on representation formulas for solutions. Using only comparison with suitably constructed super- and sub- solutions, we obtain tight upper and lower bounds for solutions with linear initial data x↦θx. Another important ingredient is a general result of P. Cardaliaguet and P. E. Souganidis [13] which guarantees the existence of sublinear correctors for all θ outside “flat parts” of effective Hamiltonians associated with the convex functions from which G is built. We derive crucial derivative estimates for these correctors which allow us to use them as correctors for G.

Scalable Optimization Methods for Incorporating Spatiotemporal Fractionation into Intensity-Modulated Radiotherapy Planning

It has been recently shown that an additional therapeutic gain may be achieved if a radiotherapy plan is altered over the treatment course using a new treatment paradigm referred to in the literature as spatiotemporal fractionation. Because of the nonconvex and large-scale nature of the corresponding treatment plan optimization problem, the extent of the potential therapeutic gain that may be achieved from spatiotemporal fractionation has been investigated using stylized cancer cases to circumvent the arising computational challenges. This research aims at developing scalable optimization methods to obtain high-quality spatiotemporally fractionated plans with optimality bounds for clinical cancer cases. In particular, the treatment-planning problem is formulated as a quadratically constrained quadratic program and is solved to local optimality using a constraint-generation approach, in which each subproblem is solved using sequential linear/quadratic programming methods. To obtain optimality bounds, cutting-plane and column-generation methods are combined to solve the Lagrangian relaxation of the formulation. The performance of the developed methods are tested on deidentified clinical liver and prostate cancer cases. Results show that the proposed method is capable of achieving local-optimal spatiotemporally fractionated plans with an optimality gap of around 10%–12% for cancer cases tested in this study. Summary of Contribution: The design of spatiotemporally fractionated radiotherapy plans for clinical cancer cases gives rise to a class of nonconvex and large-scale quadratically constrained quadratic programming (QCQP) problems, the solution of which requires the development of efficient models and solution methods. To address the computational challenges posed by the large-scale and nonconvex nature of the problem, we employ large-scale optimization techniques to develop scalable solution methods that find local-optimal solutions along with optimality bounds. We test the performance of the proposed methods on deidentified clinical cancer cases. The proposed methods in this study can, in principle, be applied to solve other QCQP formulations, which commonly arise in several application domains, including graph theory, power systems, and signal processing.

Bregman linearized reweighted alternating minimization for robust sparse recovery

In this paper, we consider a class of nonconvex and nonsmooth minimization problems with linear constraints in the area of compressed sensing (CS). The classical alternating minimization (AM) had difficulties in solving formulated nonconvex or nonsmooth subproblems. In view of this problem, we propose a Bregman linearized reweighted alternating minimization with continuation (BLRAMC). In this algorithm, all subproblems are convex and can be thus solved efficiently. Moreover, with the introduced Kurdyka-Łojasiewicz property, it is proved that the proposed algorithm converges to a stationary point of the objective function. Numerical experiments for the robust sparse recovery with different impulsive noise are presented to substantiate the effectiveness of the proposed algorithm.

An Adaptive Proximal Bundle Method with Inexact Oracles for a Class of Nonconvex and Nonsmooth Composite Optimization

In this paper, an adaptive proximal bundle method is proposed for a class of nonconvex and nonsmooth composite problems with inexact information. The composite problems are the sum of a finite convex function with inexact information and a nonconvex function. For the nonconvex function, we design the convexification technique and ensure the linearization errors of its augment function to be nonnegative. Then, the sum of the convex function and the augment function is regarded as an approximate function to the primal problem. For the approximate function, we adopt a disaggregate strategy and regard the sum of cutting plane models of the convex function and the augment function as a cutting plane model for the approximate function. Then, we give the adaptive nonconvex proximal bundle method. Meanwhile, for the convex function with inexact information, we utilize the noise management strategy and update the proximal parameter to reduce the influence of inexact information. The method can obtain an approximate solution. Two polynomial functions and six DC problems are referred to in the numerical experiment. The preliminary numerical results show that our algorithm is effective and reliable.

A Gauss–Seidel type inertial proximal alternating linearized minimization for a class of nonconvex optimization problems

In this paper we study a broad class of nonconvex and nonsmooth minimization problems, whose objective function is the sum of a smooth function of the entire variables and two nonsmooth functions of each variable. We adopt the framework of the proximal alternating linearized minimization (PALM), together with the inertial strategy to accelerate the convergence. Since the inertial step is performed once the x-subproblem/y-subproblem is updated, the algorithm is a Gauss–Seidel type inertial proximal alternating linearized minimization (GiPALM) algorithm. Under the assumption that the underlying functions satisfy the Kurdyka–Łojasiewicz (KL) property and some suitable conditions on the parameters, we prove that each bounded sequence generated by GiPALM globally converges to a critical point. We apply the algorithm to signal recovery, image denoising and nonnegative matrix factorization models, and compare it with PALM and the inertial proximal alternating linearized minimization.

求解一类非凸非光滑问题的惯性邻近交替极小化算法

求解一类非凸非光滑问题的惯性邻近交替极小化算法

Iteratively Linearized Reweighted Alternating Direction Method of Multipliers for a Class of Nonconvex Problems

In this paper, we consider solving a class of nonconvex and nonsmooth problems frequently appearing in signal processing and machine learning research. The traditional alternating direction method of multipliers encounters troubles in both mathematics and computations in solving the nonconvex and nonsmooth subproblem. In view of this, we propose a reweighted alternating direction method of multipliers. In this algorithm, all subproblems are convex and easy to solve. We also provide several guarantees for the convergence and prove that the algorithm globally converges to a critical point of an auxiliary function with the help of the Kurdyka-{\L}ojasiewicz property. Several numerical results are presented to demonstrate the efficiency of the proposed algorithm.

Global convergence of proximal iteratively reweighted algorithm

In this paper, we investigate the convergence of the proximal iteratively reweighted algorithm for a class of nonconvex and nonsmooth problems. Such problems actually include numerous models in the area of signal processing and machine learning research. Two extensions of the algorithm are also studied. We provide a unified scheme for these three algorithms. With the Kurdyka–Łojasiewicz property, we prove that the unified algorithm globally converges to a critical point of the objective function.

Saddle point criteria in semi-infinite minimax fractional programming under (Φ,ρ)-invexity

Semi-infinite minimax fractional programming problems with both inequality and equality constraints are considered. The sets of parametric saddle point conditions are established for a new class of nonconvex differentiable semi-infinite minimax fractional programming problems under(?,?)-invexity assumptions. With the reference to the said concept of generalized convexity, we extend some results of saddle point criteria for a larger class of nonconvex semi-infinite minimax fractional programming problems in comparison to those ones previously established in the literature.

Alternating Direction Method of Multipliers for a Class of Nonconvex and Nonsmooth Problems with Applications to Background/Foreground Extraction

In this paper, we study a general optimization model, which covers a large class of existing models for many applications in imaging sciences. To solve the resulting possibly nonconvex, nonsmooth, and non-Lipschitz optimization problem, we adapt the alternating direction method of multipliers (ADMM) with a general dual step-size to solve a reformulation that contains three blocks of variables and analyze its convergence. We show that for any dual step-size less than the golden ratio, there exists a computable threshold such that if the penalty parameter is chosen above such a threshold and the sequence thus generated by our ADMM is bounded, then the cluster point of the sequence gives a stationary point of the nonconvex optimization problem. We achieve this via a potential function specifically constructed for our ADMM. Moreover, we establish the global convergence of the whole sequence if, in addition, this special potential function is a Kurdyka--Łojasiewicz function. Furthermore, we present a simple st...

Epi-convergence Properties of Smoothing by Infimal Convolution

This paper concerns smoothing by infimal convolution for two large classes of functions: convex, proper and lower semicontinous as well as for (the nonconvex class of) convex-composite functions. The smooth approximations are constructed so that they epi-converge (to the underlying nonsmooth function) and fulfill a desirable property with respect to graph convergence of the gradient mappings to the subdifferential of the original function under reasonable assumptions. The close connection between epi-convergence of the smoothing functions and coercivity properties of the smoothing kernel is established.

Gradient Descent with Proximal Average for Nonconvex and Composite Regularization

Sparse modeling has been highly successful in many real-world applications. While a lot of interests have been on convex regularization, recent studies show that nonconvexregularizers can outperform their convex counterparts in many situations.However, the resulting nonconvex optimization problems are often challenging, especiallyfor composite regularizers such as the nonconvex overlapping group lasso. In thispaper, byusing a recent mathematical tool known as the proximal average,we propose a novel proximal gradient descent method for optimization with a wide class of nonconvex and composite regularizers.Instead of directlysolving the proximal stepassociated with a composite regularizer, we average thesolutions from the proximal problems of the constituent regularizers. This simple strategy has guaranteed convergenceand low per-iteration complexity.Experimental results on a number of synthetic andreal-world data sets demonstrate the effectiveness and efficiency of theproposed optimization algorithm, and also the improved classification performanceresulting from thenonconvex regularizers.