Convex Minimization Problem Research Articles

Inexact descent methods for convex minimization problems in Banach spaces

"Given a Lipschitz and convex objective function of an unconstrained optimization problem, defined on a Banach space, we revisit the class of regular vector fields which was introduced in our previous work on descent methods. We study, in particular, the asymptotic behavior of the sequence of values of the objective function for a certain inexact process generated by a regular vector field when the sequence of computational errors converges to zero and show that this sequence of values converges to the infimum of the given objective function of the unconstrained optimization problem."

An entropic interpolation problem for incompressible viscous fluids

In view of studying incompressible inviscid fluids, Brenier introduced in the late 80's a relaxation of a geodesic problem addressed by Arnold in 1966. Instead of inviscid fluids, the present paper is devoted to incompressible viscid fluids. A natural analogue of Brenier's problem is introduced, where generalized flows are no more supported by absolutely continuous paths, but by Brownian sample paths. It turns out that this new variational problem is an entropy minimization problem with marginal constraints entering the class of convex minimization problems. This paper explores the connection between this variational problem and Brenier's original problem. Its dual problem is derived and the general shape of its solution is described. Under the restrictive assumption that the pressure is a nice function, the kinematics of its solution is made explicit and its connection with the Navier-Stokes equation is established.

Composite convex optimization with global and local inexact oracles

We introduce new global and local inexact oracle concepts for a wide class of convex functions in composite convex minimization. Such inexact oracles naturally come from primal-dual framework, barrier smoothing, inexact computations of gradients and Hessian, and many other situations. We also provide examples showing that the class of convex functions equipped with the newly inexact second-order oracles is larger than standard self-concordant as well as Lipschitz gradient function classes. Further, we investigate several properties of convex and/or self-concordant functions under the inexact second-order oracles which are useful for algorithm development. Next, we apply our theory to develop inexact proximal Newton-type schemes for minimizing general composite convex minimization problems equipped with such inexact oracles. Our theoretical results consist of new optimization algorithms, accompanied with global convergence guarantees to solve a wide class of composite convex optimization problems. When the first objective term is additionally self-concordant, we establish different local convergence results for our method. In particular, we prove that depending on the choice of accuracy levels of the inexact second-order oracles, we obtain different local convergence rates ranging from $R$-linear and $R$-superlinear to $R$-quadratic. In special cases, where convergence bounds are known, our theory recovers the best known rates. We also apply our settings to derive a new primal-dual method for composite convex minimization problems. Finally, we present some representative numerical examples to illustrate the benefit of our new algorithms.

A Parallel Splitting Augmented Lagrangian Method for Two-Block Separable Convex Programming with Application in Image Processing

The augmented Lagrangian method (ALM) is one of the most successful first-order methods for convex programming with linear equality constraints. To solve the two-block separable convex minimization problem, we always use the parallel splitting ALM method. In this paper, we will show that no matter how small the step size and the penalty parameter are, the convergence of the parallel splitting ALM is not guaranteed. We propose a new convergent parallel splitting ALM (PSALM), which is the regularizing ALM’s minimization subproblem by some simple proximal terms. In application this new PSALM is used to solve video background extraction problems and our numerical results indicate that this new PSALM is efficient.

Inertial Haugazeau's hybrid subgradient extragradient algorithm for variational inequality problems in Banach spaces

The variational inequality problem plays an important role in nonlinear analysis and optimization. It is a generalization of the nonlinear complementarity problem. For a variational inequality problem in a Hilbert space, the extragradient algorithm with inertial effects has been studied. For a variational inequality problem in a Banach space, Nakajo introduced Haugazeau's hybrid method and Liu introduced the Halpern subgradient extragradient method. In this paper, we construct a new inertial iterative method for solving variational inequality problems in Banach spaces based on the work we mentioned above. We propose a strong convergence theorem. As applications, our result can be used to solve constrained convex minimization problems.

Convergence rates of an inertial gradient descent algorithm under growth and flatness conditions

In this paper we study the convergence properties of a Nesterov’s family of inertial schemes which is a specific case of inertial Gradient Descent algorithm in the context of a smooth convex minimization problem, under some additional hypotheses on the local geometry of the objective function F, such as the growth (or Łojasiewicz) condition. In particular we study the different convergence rates for the objective function and the local variation, depending on these geometric conditions. In this setting we can give optimal convergence rates for this Nesterov scheme. Our analysis shows that there are some situations when Nesterov’s family of inertial schemes is asymptotically less efficient than the gradient descent (e.g. in the case when the objective function is quadratic).

Composite optimization for the resource allocation problem

In this paper, we consider resource allocation problem stated as a convex minimization problem with linear constraints. To solve this problem, we use gradient and accelerated gradient descent applied to the dual problem and prove the convergence rate both for the primal iterates and the dual iterates. We obtain faster convergence rates than the ones known in the literature. We also provide economic interpretation for these two methods. This means that iterations of the algorithms naturally correspond to the process of price and production adjustment in order to obtain the desired production volume in the economy. Overall, we show how these actions of the economic agents lead the whole system to the equilibrium.

COMPUTATION OF ZEROS OF MONOTONE TYPE MAPPINGS: ON CHIDUME’S OPEN PROBLEM

For $p\geq 2$, let $E$ be a 2-uniformly smooth and $p$-uniformly convex real Banach space and let $A:E\rightarrow E^{\ast }$ be a Lipschitz and strongly monotone mapping such that $A^{-1}(0)\neq \emptyset$. For given $x_{1}\in E$, let $\{x_{n}\}$ be generated by the algorithm $x_{n+1}=J^{-1}(Jx_{n}-\unicode[STIX]{x1D706}Ax_{n})$, $n\geq 1$, where $J$ is the normalized duality mapping from $E$ into $E^{\ast }$ and $\unicode[STIX]{x1D706}$ is a positive real number in $(0,1)$ satisfying suitable conditions. Then it is proved that $\{x_{n}\}$ converges strongly to the unique point $x^{\ast }\in A^{-1}(0)$. Furthermore, our theorems provide an affirmative answer to the Chidume et al. open problem [‘Krasnoselskii-type algorithm for zeros of strongly monotone Lipschitz maps in classical Banach spaces’, SpringerPlus4 (2015), 297]. Finally, applications to convex minimization problems are given.

General viscosity iterative process for solving variational inclusion and fixed point problems involving multivalued quasi-nonexpansive and demicontractive operators with application

In this paper, we introduce and study a new iterative method which is based on viscosity general algorithm and forward-backward splitting method for finding a common element of the set of common fixed points of multivalued demicontractive and quasi-nonexpansive mappings and the set of solutions of variational inclusion with set-valued maximal monotone mapping and inverse strongly monotone mappings in real Hilbert spaces. We prove that the sequence $x_n$ which is generated by the proposed iterative algorithm converges strongly to a common element of two sets above. Finally, our theorems are applied to approximate a common solution of fixed point problems with set-valued operators and the composite convex minimization problem. Our theorems are significant improvements on several important recent results.

Robust regularized extreme learning machine with asymmetric Huber loss function

Sediment transport is one of the major challenging fields in hydrology. The tropical atmosphere, complex topography and occasional extreme precipitation are the fundamental explanations behind this challenge. Thus, the rivers in this situation contain a huge quantity of sediment, which may affect the river hydraulics. Hence, it is required to collect various parameters such as discharge, velocity, rainfall and sediment concentration to analyze the impact of sediment for river engineering practices and management. Therefore, the dataset which is collected from the river may contain outliers and noises. For improving the prediction accuracy of sediment load, we present robust regularized extreme learning machine frameworks to reduce the effect of noise by using the asymmetric Huber loss function named as AHELM and $$ \varepsilon {-} $$ insensitive Huber loss function named as $$ \varepsilon {-} $$ AHELM. Further, the problems are rewritten in the form of strongly convex minimization problems whose solutions are acquired by simple function iterative schemes. To ensure the effectiveness of the proposed approach, we have considered the real-world datasets with two types of noises. Furthermore, the proposed schemes are applied on real sediment load datasets (SLDs) which are collected from the Tawang Chu river of Arunachal Pradesh, India. The results reveal that proposed AHELM and $$ \varepsilon {-} $$ AHELM with multiquadric activation function are performed better for real-world datasets, whereas AHELM and $$ \varepsilon {-} $$ AHELM with sigmoid activation function perform efficiently and effectively for the sediment load prediction. In overall, the experimental results clearly exhibit the applicability as well as the usability of the proposed extreme learning machine with asymmetric Huber loss functions.

New ℓ2 − ℓ0 algorithm for single-molecule localization microscopy

Among the many super-resolution techniques for microscopy, single-molecule localization microscopy methods are widely used. This technique raises the difficult question of precisely localizing fluorophores from a blurred, under-resolved, and noisy acquisition. In this work, we focus on the grid-based approach in the context of a high density of fluorophores formalized by a ℓ2 least-square term and sparsity term modeled with ℓ0 pseudo-norm. We consider both the constrained formulation and the penalized formulation. Based on recent results, we formulate the ℓ0 pseudo-norm as a convex minimization problem. This is done by introducing an auxiliary variable. An exact biconvex reformulation of the ℓ2 - ℓ0 constrained and penalized problems is proposed with a minimization algorithm. The algorithms, named CoBic (Constrained Biconvex) and PeBic (Penalized Biconvex) are applied to the problem of single-molecule localization microscopy and we compare the results with other recently proposed methods.

Convergence rate analysis of proximal gradient methods with applications to composite minimization problems

First-order methods such as proximal gradient, which use Forward–Backward Splitting techniques have proved to be very effective in solving nonsmooth convex minimization problem, which is useful in solving various practical problems in different fields such as machine learning and image processing. In this paper, we propose few new forward–backward splitting algorithms, which consume less number of iterations to converge to an optimum. In addition, we derive convergence rates for the proposed formulations and show that the speed of convergence of these algorithms is significantly better than the traditional forward–backward algorithm. To demonstrate the practical applicability, we apply them to two real-world problems of machine learning and image processing. The first issue deals with the regression on high-dimensional datasets, whereas the second one is the image deblurring problem. Numerical experiments have been conducted on several publicly available real datasets to verify the obtained theoretical results. Results demonstrate the superiority of our algorithms in terms of accuracy, the number of iterations required to converge and the rate of convergence against the classical first-order methods.

Unconstrained convex minimization based implicit Lagrangian twin extreme learning machine for classification (ULTELMC)

The recently proposed twin extreme learning machine (TELM) requires solving two quadratic programming problems (QPPs) in order to find two non-parallel hypersurfaces in the feature that brings in the additional requirement of external optimization toolbox such as MOSEK. In this paper, we propose implicit Lagrangian TELM for classification via unconstrained convex minimization problem (ULTELMC) and further suggest iterative convergent schemes which eliminates the requirement of external optimization toolbox generally required in solving the quadratic programming problems (QPPs) of TELM. The solutions to the dual variables of the proposed ULTELMC are obtained using iterative schemes containing ‘plus’ function which is not differentiable. To overcome this shortcoming, the generalized derivative approach and smooth approximation approaches are suggested. Further, to test the performance of the proposed approaches, classification performances are compared with support vector machine (SVM), twin support vector machine (TWSVM), extreme learning machine (ELM), twin extreme learning machine (TELM) and Lagrangian extreme learning machine (LELM). Moreover, non-requirement to solve QPPs makes the iterative schemes find the solution faster as compared to the reported methods that finds the solution in dual space. Computational times required in finding the solutions are also presented for comparison.

An Exploratory Method for Smooth/Transient Decomposition

We consider a separation problem where the observation consists of the sum of a high amplitude smooth signal and a low amplitude transient signal. We propose a method for decomposition that relies on solving instances of a `constrained filtering problem', which is posed as a convex minimization problem. We provide a fast algorithm for solving the minimization problem, and demonstrate the effectiveness of the scheme for a vital signs monitoring experiment using radar.

Proximal point algorithms for fixed point problem and convex minimization problem

Proximal point algorithms for fixed point problem and convex minimization problem

A modified proximal point algorithm for solving variational inclusion problem in real Hilbert spaces

Abstract The purpose of this paper is to use a modified proximal point algorithm for solving variational inclusion problem in real Hilbert spaces. It is proven that the sequence generated by the proposed iterative algorithm converges strongly to the common solution of the convex minimization and variational inclusion problems.

Convergence Results for Fixed Point Problems of Accretive Operators in Banach Spaces

This paper deals with the approximate solutions of accretive maps in a uniformly convex Banach space. A weak convergence of a three - step iterative scheme involving the resolvents of accretive operators is proved. The main result is applied to a convex minimization problem in Hilbert spaces. In particular, the minimizer of a convex and proper lower semi-continuous function defined in a Hilbert space was obtained. Numerical illustration with graphical display of the convergence of the sequence obtained from the iterative scheme is also presented.

A Novel Forward-Backward Algorithm for Solving Convex Minimization Problem in Hilbert Spaces

In this work, we aim to investigate the convex minimization problem of the sum of two objective functions. This optimization problem includes, in particular, image reconstruction and signal recovery. We then propose a new modified forward-backward splitting method without the assumption of the Lipschitz continuity of the gradient of functions by using the line search procedures. It is shown that the sequence generated by the proposed algorithm weakly converges to minimizers of the sum of two convex functions. We also provide some applications of the proposed method to compressed sensing in the frequency domain. The numerical reports show that our method has a better convergence behavior than other methods in terms of the number of iterations and CPU time. Moreover, the numerical results of the comparative analysis are also discussed to show the optimal choice of parameters in the line search.

Optimality conditions for higher order polyhedral discrete and differential inclusions

The problems considered in this paper are described in polyhedral multi-valued mappings for higher order(s-th) discrete (PDSIs) and differential inclusions (PDFIs). The present paper focuses on the necessary and sufficient conditions of optimality for optimization of these problems. By converting the PDSIs problem into a geometric constraint problem, we formulate the necessary and sufficient conditions of optimality for a convex minimization problem with linear inequality constraints. Then, in terms of the Euler-Lagrange type PDSIs and the specially formulated transversality conditions, we are able to obtain conditions of optimality for the PDSIs. In order to obtain the necessary and sufficient conditions of optimality for the discrete-approximation problem PDSIs, we reduce this problem to the form of a problem with higher order discrete inclusions. Finally, by formally passing to the limit, we establish the sufficient conditions of optimality for the problem with higher order PDFIs. Numerical approach is developed to solve a polyhedral problem with second order polyhedral discrete inclusions.

New iterative schemes for solving variational inequality and fixed points problems involving demicontractive and quasi-nonexpansive mappings in Banach spaces

Abstract In this paper, we suggest and analyze a new iterative method for finding a common element of the set of fixed points of a quasi-nonexpansive mapping and the set of fixed points of a demicontractive mapping which is the unique solution of some variational inequality problems involving accretive operators in a Banach space. We prove the strong convergence of the proposed iterative scheme without imposing any compactness condition on the mapping or the space. Finally, applications of our theorems to some constrained convex minimization problems are given.