Nonsmooth Optimization Problems Research Articles

Learning fused lasso parameters in portfolio selection via neural networks

In recent years, fused lasso models are becoming popular in several fields, such as computer vision, classification and finance. In portfolio selection, they can be used to penalize active positions and portfolio turnover. Despite efficient algorithms and software for solving non-smooth optimization problems have been developed, the amount of regularization to apply is a critical issue, especially if we have to achieve a financial aim. We propose a data-driven approach for learning the regularization parameters in a fused lasso formulation of the multi-period portfolio selection problem, able to realize a given financial target. We design a neural network architecture based on recurrent networks for learning the functional dependence between the regularization parameters and the input data. In particular, the Long Short-Term Memory networks are considered for their ability to process sequential data, such as the time series of the asset returns. Numerical experiments performed on market data show the effectiveness of our approach.

A Penalty-Free Infeasible Approach for a Class of Nonsmooth Optimization Problems Over the Stiefel Manifold

A Penalty-Free Infeasible Approach for a Class of Nonsmooth Optimization Problems Over the Stiefel Manifold

Generalized Newton Method with Positive Definite Regularization for Nonsmooth Optimization Problems with Nonisolated Solutions

Generalized Newton Method with Positive Definite Regularization for Nonsmooth Optimization Problems with Nonisolated Solutions

Smoothing algorithms for nonsmooth optimization over the Stiefel manifold with applications to the graph Fourier basis problem

In this paper, we consider a class of nonsmooth and nonconvex optimization problems over the Stiefel manifold where the objective function is the summation of a nonconvex smooth function and a nonsmooth Lipschitz continuous convex function composed with a linear mapping. Besides, we are interested in its application to the graph Fourier basis problem. We propose three numerical algorithms for solving this problem, by combining smoothing methods and some existing algorithms for smooth optimization over the Stiefel manifold. In particular, we approximate the aforementioned nonsmooth convex function by its Moreau envelope in our smoothing methods, and prove that the Moreau envelope has many favorable properties. Thanks to this and the scheme for updating the smoothing parameter, we show that any accumulation point of the solution sequence generated by the proposed algorithms is a stationary point of the original optimization problem. Numerical experiments on building graph Fourier basis are conducted to demonstrate the efficiency of the proposed algorithms.

Normalized penalty gradient flow: A continuous-time approach for solving constrained nonconvex nonsmooth optimization problems

To avoid calculating penalty parameters, this paper introduces a continuous-time approach that combines the normalized gradient flow with the penalty method to solve the nonconvex nonsmooth optimization problem with a convex constraint set. Subsequently, this approach is extended to solve nonconvex nonsmooth optimization with a box constraint set and affine equality constraints. Compared with the current methods, the proposed approach does not require calculating the penalty parameters, allows the objective function to be nonconvex or nonsmooth, and allows the initial point to be arbitrarily selected. It is proved that the solution trajectory with any initial point converges to the critical point set of the optimization problem. Finally, the effectiveness of the proposed approach is authenticated through several numerical experiments.

Inertial self-adaptive algorithms for solving non-smooth convex optimization problems

Inertial self-adaptive algorithms for solving non-smooth convex optimization problems

FPGA Implementation of Classical Dynamic Neural Networks for Smooth and Nonsmooth Optimization Problems

FPGA Implementation of Classical Dynamic Neural Networks for Smooth and Nonsmooth Optimization Problems

DIF Analysis with Unknown Groups and Anchor Items

Ensuring fairness in instruments like survey questionnaires or educational tests is crucial. One way to address this is by a Differential Item Functioning (DIF) analysis, which examines if different subgroups respond differently to a particular item, controlling for their overall latent construct level. DIF analysis is typically conducted to assess measurement invariance at the item level. Traditional DIF analysis methods require knowing the comparison groups (reference and focal groups) and anchor items (a subset of DIF-free items). Such prior knowledge may not always be available, and psychometric methods have been proposed for DIF analysis when one piece of information is unknown. More specifically, when the comparison groups are unknown while anchor items are known, latent DIF analysis methods have been proposed that estimate the unknown groups by latent classes. When anchor items are unknown while comparison groups are known, methods have also been proposed, typically under a sparsity assumption – the number of DIF items is not too large. However, DIF analysis when both pieces of information are unknown has not received much attention. This paper proposes a general statistical framework under this setting. In the proposed framework, we model the unknown groups by latent classes and introduce item-specific DIF parameters to capture the DIF effects. Assuming the number of DIF items is relatively small, an L1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L_1$$\\end{document}-regularised estimator is proposed to simultaneously identify the latent classes and the DIF items. A computationally efficient Expectation-Maximisation (EM) algorithm is developed to solve the non-smooth optimisation problem for the regularised estimator. The performance of the proposed method is evaluated by simulation studies and an application to item response data from a real-world educational test.

Alternating and Parallel Proximal Gradient Methods for Nonsmooth, Nonconvex Minimax: A Unified Convergence Analysis

There is growing interest in nonconvex minimax problems that is driven by an abundance of applications. Our focus is on nonsmooth, nonconvex-strongly concave minimax, thus departing from the more common weakly convex and smooth models assumed in the recent literature. We present proximal gradient schemes with either parallel or alternating steps. We show that both methods can be analyzed through a single scheme within a unified analysis that relies on expanding a general convergence mechanism used for analyzing nonconvex, nonsmooth optimization problems. In contrast to the current literature, which focuses on the complexity of obtaining nearly approximate stationary solutions, we prove subsequence convergence to a critical point of the primal objective and global convergence when the latter is semialgebraic. Furthermore, the complexity results we provide are with respect to approximate stationary solutions. Lastly, we expand the scope of problems that can be addressed by generalizing one of the steps with a Bregman proximal gradient update, and together with a few adjustments to the analysis, this allows us to extend the convergence and complexity results to this broader setting. Funding: The research of E. Cohen was partially supported by a doctoral fellowship from the Israel Science Foundation [Grant 2619-20] and Deutsche Forschungsgemeinschaft [Grant 800240]. The research of M. Teboulle was partially supported by the Israel Science Foundation [Grant 2619-20] and Deutsche Forschungsgemeinschaft [Grant 800240].

A note on the convergence of deterministic gradient sampling in nonsmooth optimization

AbstractApproximation of subdifferentials is one of the main tasks when computing descent directions for nonsmooth optimization problems. In this article, we propose a bisection method for weakly lower semismooth functions which is able to compute new subgradients that improve a given approximation in case a direction with insufficient descent was computed. Combined with a recently proposed deterministic gradient sampling approach, this yields a deterministic and provably convergent way to approximate subdifferentials for computing descent directions.

Sufficient optimality, duality and saddle point analysis in terms of convexificators for nonsmooth robust fractional interval-valued optimization problems

In this study, sufficient optimality criteria for a robust fractional interval-valued optimization problem (RP) is highlighted based on the idea of convexificators. The use of generalized invexity tools to support sufficient optimality conditions is demonstrated by an example. Also, saddle point condition of a Lagrange function is examined for RP. Furthermore, Mond–Weir-type robust dual model is constructed and by employing the concept of generalized invexity, usual duality results are discussed.

Fully Distributed Algorithms for Constrained Nonsmooth Optimization Problems of General Linear Multiagent Systems and Their Application

Fully Distributed Algorithms for Constrained Nonsmooth Optimization Problems of General Linear Multiagent Systems and Their Application

Approximate proper efficiencies in nonsmooth semi-infinite multiobjective optimization problems

This article is devoted to studying a nonsmooth semiinfinite multiobjective optimization problem (SIMP) in terms of the Mordukhovich/limiting subdifferentials. We first establish necessary and sufficient conditions for an ε−quasi positively properly efficient solution of a problem (SIMP). We also investigate Mond-Weir type dual problems under assumptions of ε−quasi pseudo-generalized convexity. Next, we provide an application to a nonsmooth fractional semi-infinite multiobjective optimization problem. Finally, some examples are given to illustrate the obtained results. The obtained results improve or include some recent known ones.

Predicting pairwise interaction affinities with ℓ0-penalized least squares–a nonsmooth bi-objective optimization based approach*

In this paper, we introduce a novel nonsmooth optimization-based method LMBM-Kron ℓ 0 LS for solving large-scale pairwise interaction affinity prediction problems. The aim of LMBM-Kron ℓ 0 LS is to produce accurate predictions using as sparse a model as possible. We apply the least squares approach with Kronecker product kernels for a loss function and a continuous formulation of ℓ 0 pseudonorm for regularization. Thus, we end up solving a nonsmooth optimization problem. In addition, we apply a specific bi-objective criterion to strike a balance between the prediction accuracy of the learned model and the sparsity of the obtained solution. We compare LMBM-Kron ℓ 0 LS with some state-of-the-art methods using three benchmark and two simulated data sets under four distinct experimental settings, including zero-shot learning. Moreover, both binary and continuous interaction affinity labels are considered with LMBM-Kron ℓ 0 LS. The results show that LMBM-Kron ℓ 0 LS finds sparse solutions without sacrificing too much in the prediction performance.

Few-shot segmentation framework for lung nodules via an optimized active contour model.

Accurate segmentation of lung nodules is crucial for the early diagnosis and treatment of lung cancer in clinical practice. However, the similarity between lung nodules and surrounding tissues has made their segmentation a longstanding challenge. Existing deep learning and active contour models each have their limitations. This paper aims to integrate the strengths of both approaches while mitigating their respective shortcomings. In this paper, we propose a few-shot segmentation framework that combines a deep neural network with an active contour model. We introduce heat kernel convolutions and high-order total variation into the active contour model and solve the challenging nonsmooth optimization problem using the alternating direction method of multipliers. Additionally, we use the presegmentation results obtained from training a deep neural network on a small sample set as the initial contours for our optimized active contour model, addressing the difficulty of manually setting the initial contours. We compared our proposed method with state-of-the-art methods for segmentation effectiveness using clinical computed tomography (CT) images acquired from two different hospitals and the publicly available LIDC dataset. The results demonstrate that our proposed method achieved outstanding segmentation performance according to both visual and quantitative indicators. Our approach utilizes the output of few-shot network training as prior information, avoiding the need to select the initial contour in the active contour model. Additionally, it provides mathematical interpretability to the deep learning, reducing its dependency on the quantity of training samples.

Approximate Karush-Kuhn-Tucker conditions for nonsmooth bilevel optimization problems

Approximate Karush-Kuhn-Tucker conditions for nonsmooth bilevel optimization problems

On approximate positively properly efficient solutions in nonsmooth semi-infinite multiobjective optimization problems with data uncertainty

On approximate positively properly efficient solutions in nonsmooth semi-infinite multiobjective optimization problems with data uncertainty

On optimality conditions and duality for multiobjective fractional optimization problem with vanishing constraints

<p>The aim of this paper is to investigate the optimality conditions for a class of nonsmooth multiobjective fractional optimization problems subject to vanishing constraints. In particular, necessary and sufficient conditions for (weak) Pareto solution are presented in terms of the Clark subdifferential. Furthermore, we construct Wolfe and Mond–Weir-type dual models and derive some duality theorems by using generalized quasiconvexity assumptions. Some examples to show the validity of our conclusions are provided.</p>

Approximate Optimality Conditions for Nonsmooth Optimization Problems

Approximate Optimality Conditions for Nonsmooth Optimization Problems

The minimax exact penalty fuzzy function method for solving convex nonsmooth optimization problems with fuzzy objective functions

The minimax exact penalty fuzzy function method for solving convex nonsmooth optimization problems with fuzzy objective functions