Nonsmooth Optimization Problems Research Articles

A dynamic multiple proximal bundle algorithm for locally Lipschitz multiobjective optimization problems

We propose a method for nonsmooth nonconvex multiobjective optimization problems that is based on a bundle-type approach. The proposed method employs an extension of the redistributed proximal bundle algorithm and uses an augmented improvement function to handle different objectives. In which, at each iteration, a common piecewise linear model is used to approximate the augmented improvement function. Contrary to many existing multiobjective optimization methods, this algorithm works directly with objective functions, without using any kind of a priori chosen parameters or employing any scalarization. Under appropriate assumptions, we discuss convergence to points which satisfy a necessary condition for Pareto optimality. We provide numerical results for a set of nonsmooth convex and nonconvex multiobjective optimization problems in the form of tables, figures and performance profiles. The numerical results confirm the superiority of the proposed algorithm in the considered test problems compared to other multiobjective solvers, in the computational effort necessary to compute weakly Pareto stationary points.

Dual Kuhn-Tucker necessary conditions for strict minima of order two for nonsmooth vector optimization problems with inequality and equality constraints

In this paper, we give primal and dual Kuhn-Tucker necessary conditions for the existence of a strict local minimum of order two for vector optimization problems with equality and inequality constraints under some new regularity conditions. First, we improve the existing primal necessary conditions for such minima. Then, we apply an alternative theorem to derive dual Kuhn-Tucker necessary conditions of second and higher-order. To compare our results to the ones in the literature, we provide some examples.

Uncertainty Quantification for Scale-Space Blob Detection

AbstractWe consider the problem of blob detection for uncertain images, such as images that have to be inferred from noisy measurements. Extending recent work motivated by astronomical applications, we propose an approach that represents the uncertainty in the position and size of a blob by a region in a three-dimensional scale space. Motivated by classic tube methods such as the taut-string algorithm, these regions are obtained from level sets of the minimizer of a total variation functional within a high-dimensional tube. The resulting non-smooth optimization problem is challenging to solve, and we compare various numerical approaches for its solution and relate them to the literature on constrained total variation denoising. Finally, the proposed methodology is illustrated on numerical experiments for deconvolution and models related to astrophysics, where it is demonstrated that it allows to represent the uncertainty in the detected blobs in a precise and physically interpretable way.

A Bregman proximal subgradient algorithm for nonconvex and nonsmooth fractional optimization problems

In this paper, we study a class of nonconvex and nonsmooth fractional optimization problem, where the numerator of which is the sum of a nonsmooth and nonconvex function and a relative smooth nonconvex function, while the denominator is relative weakly convex nonsmooth function. We propose a Bregman proximal subgradient algorithm for solving this type of fractional optimization problems. Under moderate conditions, we prove that the subsequence generated by the proposed algorithm converges to a critical point, and the generated sequence globally converges to a critical point when the objective function satisfies the Kurdyka-Łojasiewicz property. We also obtain the convergence rate of the proposed algorithm. Finally, two numerical experiments illustrate the effectiveness and superiority of the algorithm. Our results give a positive answer to an open problem proposed by Bot et al. [14].

A new computational framework for log-concave density estimation

In statistics, log-concave density estimation is a central problem within the field of nonparametric inference under shape constraints. Despite great progress in recent years on the statistical theory of the canonical estimator, namely the log-concave maximum likelihood estimator, adoption of this method has been hampered by the complexities of the non-smooth convex optimization problem that underpins its computation. We provide enhanced understanding of the structural properties of this optimization problem, which motivates the proposal of new algorithms, based on both randomized and Nesterov smoothing, combined with an appropriate integral discretization of increasing accuracy. We prove that these methods enjoy, both with high probability and in expectation, a convergence rate of order 1/T up to logarithmic factors on the objective function scale, where T denotes the number of iterations. The benefits of our new computational framework are demonstrated on both synthetic and real data, and our implementation is available in a github repository LogConcComp (Log-Concave Computation).

Adaptive penalty-based neurodynamic approach for nonsmooth interval-valued optimization problem

The complex and diverse practical background drives this paper to explore a new neurodynamic approach (NA) to solve nonsmooth interval-valued optimization problems (IVOPs) constrained by interval partial order and more general sets. On the one hand, to deal with the uncertainty of interval-valued information, the LU-optimality condition of IVOPs is established through a deterministic form. On the other hand, according to the penalty method and adaptive controller, the interval partial order constraint and set constraint are punished by one adaptive parameter, which is a key enabler for the feasibility of states while having a lower solution space dimension and avoiding estimating exact penalty parameters. Through nonsmooth analysis and Lyapunov theory, the proposed adaptive penalty-based neurodynamic approach (APNA) is proven to converge to an LU-solution of the considered IVOPs. Finally, the feasibility of the proposed APNA is illustrated by numerical simulations and an investment decision-making problem.

THE PENALTY FUNKCIONS METHOD AND OPTIMALITY CONDITONS

The presented work is devoted to one of the most important areas of applied mathematics, namely mathematical programming. The following general mathematical programming problem is considered where are Lipschitz, and is closed set. The purpose of the work is to obtain optimality conditions for the problem posed using the method of penalty functions. The obtained necessary conditions are expressed in the language of subdifferentials, which are a generalization of the method of indefinite Lagrange coefficients for nonsmooth optimization problems.

Higher-order KKT optimality conditions through contingent derivatives for constrained nonsmooth vector equilibrium problems

In this paper, we deal with some higher-order optimality conditions for local strict efficient solutions to a nonsmooth vector equilibrium problem with set, cone and equality constraints. For this aim, the concept of m−stable and m−steady functions (m≥2 and integer) for single-valued functions and some constraint qualifications of higher order in terms of contingent derivatives are proposed accordingly. We analyze the sum calculus rule of mth-order adjacent set, mth-order interior set, asymptotic mth-order tangent cone and asymptotic mth-order adjacent cone. Subsequently, we employ the obtained calculus rules to treat KKT necessary and sufficient optimality conditions of higher order in terms of contingent derivatives for the mth-order (local) strict efficient solutions to such problem. Simultaneously, we employ these rules to study KKT higher-order optimality conditions for such efficient solutions for a nonsmooth vector optimization problem with constraints. Some illustrative examples are provided to demonstrate the main results of the literature.

Learning fused lasso parameters in portfolio selection via neural networks

AbstractIn recent years, fused lasso models are becoming popular in several fields, such as computer vision, classification and finance. In portfolio selection, they can be used to penalize active positions and portfolio turnover. Despite efficient algorithms and software for solving non-smooth optimization problems have been developed, the amount of regularization to apply is a critical issue, especially if we have to achieve a financial aim. We propose a data-driven approach for learning the regularization parameters in a fused lasso formulation of the multi-period portfolio selection problem, able to realize a given financial target. We design a neural network architecture based on recurrent networks for learning the functional dependence between the regularization parameters and the input data. In particular, the Long Short-Term Memory networks are considered for their ability to process sequential data, such as the time series of the asset returns. Numerical experiments performed on market data show the effectiveness of our approach.

A Penalty-Free Infeasible Approach for a Class of Nonsmooth Optimization Problems Over the Stiefel Manifold

A Penalty-Free Infeasible Approach for a Class of Nonsmooth Optimization Problems Over the Stiefel Manifold

Generalized Newton Method with Positive Definite Regularization for Nonsmooth Optimization Problems with Nonisolated Solutions

Generalized Newton Method with Positive Definite Regularization for Nonsmooth Optimization Problems with Nonisolated Solutions

Smoothing algorithms for nonsmooth optimization over the Stiefel manifold with applications to the graph Fourier basis problem

In this paper, we consider a class of nonsmooth and nonconvex optimization problems over the Stiefel manifold where the objective function is the summation of a nonconvex smooth function and a nonsmooth Lipschitz continuous convex function composed with a linear mapping. Besides, we are interested in its application to the graph Fourier basis problem. We propose three numerical algorithms for solving this problem, by combining smoothing methods and some existing algorithms for smooth optimization over the Stiefel manifold. In particular, we approximate the aforementioned nonsmooth convex function by its Moreau envelope in our smoothing methods, and prove that the Moreau envelope has many favorable properties. Thanks to this and the scheme for updating the smoothing parameter, we show that any accumulation point of the solution sequence generated by the proposed algorithms is a stationary point of the original optimization problem. Numerical experiments on building graph Fourier basis are conducted to demonstrate the efficiency of the proposed algorithms.

Normalized penalty gradient flow: A continuous-time approach for solving constrained nonconvex nonsmooth optimization problems

To avoid calculating penalty parameters, this paper introduces a continuous-time approach that combines the normalized gradient flow with the penalty method to solve the nonconvex nonsmooth optimization problem with a convex constraint set. Subsequently, this approach is extended to solve nonconvex nonsmooth optimization with a box constraint set and affine equality constraints. Compared with the current methods, the proposed approach does not require calculating the penalty parameters, allows the objective function to be nonconvex or nonsmooth, and allows the initial point to be arbitrarily selected. It is proved that the solution trajectory with any initial point converges to the critical point set of the optimization problem. Finally, the effectiveness of the proposed approach is authenticated through several numerical experiments.

Inertial self-adaptive algorithms for solving non-smooth convex optimization problems

Inertial self-adaptive algorithms for solving non-smooth convex optimization problems

FPGA Implementation of Classical Dynamic Neural Networks for Smooth and Nonsmooth Optimization Problems

FPGA Implementation of Classical Dynamic Neural Networks for Smooth and Nonsmooth Optimization Problems

DIF Analysis with Unknown Groups and Anchor Items

Ensuring fairness in instruments like survey questionnaires or educational tests is crucial. One way to address this is by a Differential Item Functioning (DIF) analysis, which examines if different subgroups respond differently to a particular item, controlling for their overall latent construct level. DIF analysis is typically conducted to assess measurement invariance at the item level. Traditional DIF analysis methods require knowing the comparison groups (reference and focal groups) and anchor items (a subset of DIF-free items). Such prior knowledge may not always be available, and psychometric methods have been proposed for DIF analysis when one piece of information is unknown. More specifically, when the comparison groups are unknown while anchor items are known, latent DIF analysis methods have been proposed that estimate the unknown groups by latent classes. When anchor items are unknown while comparison groups are known, methods have also been proposed, typically under a sparsity assumption – the number of DIF items is not too large. However, DIF analysis when both pieces of information are unknown has not received much attention. This paper proposes a general statistical framework under this setting. In the proposed framework, we model the unknown groups by latent classes and introduce item-specific DIF parameters to capture the DIF effects. Assuming the number of DIF items is relatively small, an L1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L_1$$\\end{document}-regularised estimator is proposed to simultaneously identify the latent classes and the DIF items. A computationally efficient Expectation-Maximisation (EM) algorithm is developed to solve the non-smooth optimisation problem for the regularised estimator. The performance of the proposed method is evaluated by simulation studies and an application to item response data from a real-world educational test.

Alternating and Parallel Proximal Gradient Methods for Nonsmooth, Nonconvex Minimax: A Unified Convergence Analysis

There is growing interest in nonconvex minimax problems that is driven by an abundance of applications. Our focus is on nonsmooth, nonconvex-strongly concave minimax, thus departing from the more common weakly convex and smooth models assumed in the recent literature. We present proximal gradient schemes with either parallel or alternating steps. We show that both methods can be analyzed through a single scheme within a unified analysis that relies on expanding a general convergence mechanism used for analyzing nonconvex, nonsmooth optimization problems. In contrast to the current literature, which focuses on the complexity of obtaining nearly approximate stationary solutions, we prove subsequence convergence to a critical point of the primal objective and global convergence when the latter is semialgebraic. Furthermore, the complexity results we provide are with respect to approximate stationary solutions. Lastly, we expand the scope of problems that can be addressed by generalizing one of the steps with a Bregman proximal gradient update, and together with a few adjustments to the analysis, this allows us to extend the convergence and complexity results to this broader setting. Funding: The research of E. Cohen was partially supported by a doctoral fellowship from the Israel Science Foundation [Grant 2619-20] and Deutsche Forschungsgemeinschaft [Grant 800240]. The research of M. Teboulle was partially supported by the Israel Science Foundation [Grant 2619-20] and Deutsche Forschungsgemeinschaft [Grant 800240].

A note on the convergence of deterministic gradient sampling in nonsmooth optimization

AbstractApproximation of subdifferentials is one of the main tasks when computing descent directions for nonsmooth optimization problems. In this article, we propose a bisection method for weakly lower semismooth functions which is able to compute new subgradients that improve a given approximation in case a direction with insufficient descent was computed. Combined with a recently proposed deterministic gradient sampling approach, this yields a deterministic and provably convergent way to approximate subdifferentials for computing descent directions.

Sufficient optimality, duality and saddle point analysis in terms of convexificators for nonsmooth robust fractional interval-valued optimization problems

In this study, sufficient optimality criteria for a robust fractional interval-valued optimization problem (RP) is highlighted based on the idea of convexificators. The use of generalized invexity tools to support sufficient optimality conditions is demonstrated by an example. Also, saddle point condition of a Lagrange function is examined for RP. Furthermore, Mond–Weir-type robust dual model is constructed and by employing the concept of generalized invexity, usual duality results are discussed.

Fully Distributed Algorithms for Constrained Nonsmooth Optimization Problems of General Linear Multiagent Systems and Their Application

Fully Distributed Algorithms for Constrained Nonsmooth Optimization Problems of General Linear Multiagent Systems and Their Application