Class Of Convex Optimization Problems Research Articles

The forward–backward splitting method for finding the minimum like-norm solution of the mixed variational inequality problem

We consider a general class of convex optimization problems in which one seeks to minimize a strongly convex function over a closed and convex set, which is by itself an optimal set of another mixed variational inequality problem in a Hilbert space. Regularized forward–backward splitting method is applied to find the minimum like-norm solution of the mixed variational inequality problem under investigation.

A first-order method for solving bilevel convex optimization problems in Banach space

ABSTRACT We consider a general class of convex optimization problems in which one seeks to minimize a strongly convex function over a closed and convex set which is by itself an optimal set of another convex problem in Banach space. Regularized forward–backward splitting method is applied to find the minimum like-norm solution of the minimization problem under investigation. We also introduce a gradient-based method, called the minimal like-norm gradient method, for solving this class of problems and establish the convergence of the sequence generated by the algorithm as well as a rate of convergence of the sequence of function values.

Dynamical Primal-Dual Nesterov Accelerated Method and Its Application to Network Optimization

This article develops a continuous-time primal-dual accelerated method with an increasing damping coefficient for a class of convex optimization problems with affine equality constraints. This article analyzes critical values for parameters in the proposed method and prove that the rate of convergence in terms of the duality gap function is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$O(\frac{1}{t^2})$</tex-math></inline-formula> by choosing suitable parameters. As far as we know, this is the first continuous-time primal-dual accelerated method that can obtain the optimal rate. Then, this article applies the proposed method to two network optimization problems, a distributed optimization problem with consensus constraints and a distributed extended monotropic optimization problem, and obtains two variant distributed algorithms. Finally, numerical simulations are given to demonstrate the efficacy of the proposed method.

Accelerated Stochastic Variance Reduction for a Class of Convex Optimization Problems

Accelerated Stochastic Variance Reduction for a Class of Convex Optimization Problems

Convex optimization in sums of Banach spaces

We characterize the solution of a broad class of convex optimization problems that address the reconstruction of a function from a finite number of linear measurements. The underlying hypothesis is that the solution is decomposable as a finite sum of components, where each component belongs to its own prescribed Banach space; moreover, the problem is regularized by penalizing some composite norm of the solution. We establish general conditions for existence and derive the generic parametric representation of the solution components. These representations fall into three categories depending on the underlying regularization norm: (i) a linear expansion in terms of predefined “kernels” when the component space is a reproducing kernel Hilbert space (RKHS), (ii) a non-linear (duality) mapping of a linear combination of measurement functionals when the component Banach space is strictly convex, and, (iii) an adaptive expansion in terms of a small number of atoms within a larger dictionary when the component Banach space is not strictly convex. Our approach generalizes and unifies a number of multi-kernel (RKHS) and sparse-dictionary learning techniques for compressed sensing available in the literature. It also yields the natural extension of the classical spline-fitting techniques in (semi-)RKHS to the abstract Banach-space setting.

Ideal formulations for constrained convex optimization problems with indicator variables

Motivated by modern regression applications, in this paper, we study the convexification of a class of convex optimization problems with indicator variables and combinatorial constraints on the indicators. Unlike most of the previous work on convexification of sparse regression problems, we simultaneously consider the nonlinear non-separable objective, indicator variables, and combinatorial constraints. Specifically, we give the convex hull description of the epigraph of the composition of a one-dimensional convex function and an affine function under arbitrary combinatorial constraints. As special cases of this result, we derive ideal convexifications for problems with hierarchy, multi-collinearity, and sparsity constraints. Moreover, we also give a short proof that for a separable objective function, the perspective reformulation is ideal independent from the constraints of the problem. Our computational experiments with sparse regression problems demonstrate the potential of the proposed approach in improving the relaxation quality without significant computational overhead.

Linear regression by quantum amplitude estimation and its extension to convex optimization

Linear regression is a basic and widely-used methodology in data analysis. It is known that some quantum algorithms efficiently perform least squares linear regression of an exponentially large data set. However, if we obtain values of the regression coefficients as classical data, the complexity of the existing quantum algorithms can be larger than the classical method. This is because it depends strongly on the tolerance error $\epsilon$: the best one among the existing proposals is $O(\epsilon^{-2})$. In this paper, we propose the new quantum algorithm for linear regression, which has the complexity of $O(\epsilon^{-1})$ and keeps the logarithmic dependence on the number of data points $N_D$. In this method, we overcome bottleneck parts in the calculation, which take the form of the sum over data points and therefore have the complexity proportional to $N_D$, using quantum amplitude estimation, and other parts classically. Additionally, we generalize our method to some class of convex optimization problems.

Existence and Uniqueness of Weak Solutions for a New Class of Convex Optimization Problems Related to Image Analysis

This paper proposes a new anisotropic diffusion model in image restoration that is understood from a variational optimization of an energy functional. Initially, a family of new diffusion functions based on cubic Hermite spline is provided for optimal image denoising. After that, the existence and uniqueness of weak solutions for the corresponding Euler–Lagrange equation are proven in an appropriate functional space, and a consistent and stable numerical model is also shown. We complement this work by illustrating some experiments on different actual brain Magnetic Resonance Imaging (MRI) scans, showing the proposed model’s impressive results.

On the asymptotic behavior of the Douglas\u2013Rachford and proximal-point algorithms for convex optimization

Banjac et al. (J Optim Theory Appl 183(2):490–519, 2019) recently showed that the Douglas–Rachford algorithm provides certificates of infeasibility for a class of convex optimization problems. In particular, they showed that the difference between consecutive iterates generated by the algorithm converges to certificates of primal and dual strong infeasibility. Their result was shown in a finite-dimensional Euclidean setting and for a particular structure of the constraint set. In this paper, we extend the result to real Hilbert spaces and a general nonempty closed convex set. Moreover, we show that the proximal-point algorithm applied to the set of optimality conditions of the problem generates similar infeasibility certificates.

Inexact basic tensor methods for some classes of convex optimization problems

ABSTRACT In this paper, we analyse the Basic Tensor Methods, which use approximate solutions of the auxiliary problems. The quality of this solution is described by the residual in the function value, which must be proportional to , where is the order of the method and ϵ is the desired accuracy in the main optimization problem. We analyse in details the auxiliary schemes for the third- and second-order tensor methods. The auxiliary problems for the third-order scheme can be solved very efficiently by a linearly convergent gradient-type method with a preconditioner. The most expensive operation in this process is a preliminary factorization of the Hessian of the objective function. For solving the auxiliary problem for the second order scheme, we suggest two variants of the Fast Gradient Methods with restart, which converge as , where k is the iteration counter. Finally, we present the results of the preliminary computational experiments.

Optimizing Weighted-Sum Energy Efficiency in Downlink and Uplink NOMA Systems

In this paper, weighted sum energy efficiency (WSEE) in uplink and downlink of a multi-user non-orthogonal multiple access (NOMA) system is considered. we adopt a more realistic power consumption model where signal processing power is modeled as a linear function of transmit power. Rather than the well-known network-centric global energy efficiency (GEE) metric, which is a pseudo-concave (PC) function, the weighted sum energy efficiency metric is considered, which is not PC in general. To find the optimum user power allocation Dinkelbach-like algorithm is adopted, by which each individual fractional EE function is converted to a parametric function where under some conditions on the weights falls into a class of convex optimization problems and it is solved in the dual domain. The dual variables are updated using a sub-gradient and cutting plane-based algorithm, which here ellipsoid method is used. Since the optimum solution restricts user weights, a low complexity suboptimum algorithm that does not consider any condition on user's weights is proposed. The problem is non-convex in general; hence, epigraph form followed by successive convex approximation (SCA) is used to deal with that problem. Results demonstrate that with the user-oriented metric, one can provide different priorities to users, and by choosing proper weights entail fairness among users.

Distributed event-triggered algorithms for a class of convex optimization problems over directed networks

This paper presents two distributed event-triggered algorithms under directed communication networks to solve a class of convex optimization problems such as the economic dispatch problem (EDP) with equality constraint, while the objective of optimization is the sum of all locally convex functions. One is distributed continuous-time event-triggered optimization algorithm, and the other is discrete-time algorithm based on iteration scheme. Continuous communication is not required by adopting event-triggered communication strategy. It means that the communication cost can be reduced and unnecessary waste of network resources can be avoided. Moreover, the convergence for the proposed algorithms are rigorously proved with the aid of Lyapunov stability theory under the strongly connected and weight-balanced network topology. Finally, four numerical simulations show the effectiveness and advantages of the two novel distributed event-triggered optimization algorithms.

An asynchronous distributed gradient algorithm for economic dispatch over stochastic networks

The economic dispatch problem (EDP) in power systems is usually formulated as a category of convex optimization problems in which the collective cost function is expressed as the sum of all individual objectives of generators. It is inclined to be solved through a distributed way, which is to minimize the total generation cost while meet the demands under generator capability restrictions. In this paper, we consider this class of convex optimization problems that involve coupling linear constraint and individual box constraints. In order to make information communication of network decentralized, reliable and computationally inexpensive, an asynchronous distributed primal–dual optimization algorithm is proposed over stochastic networks. This algorithm allows nodes to use asynchronous communication strategy to update their state, and the step-sizes for seeking the optimum solution are uncoordinated constants. The asynchronous operation for generators takes link failures of networks into account. Under strongly convex assumption on objective functions, it is proved that the proposed algorithm can seek the exact optimal solution with probability one if the expectation of communication network is undirected connected. The effectiveness of the proposed optimization algorithm over stochastic networks is illustrated by provided simulation results.

Separation of Convex Sets via Barrier Cones

A closed convex subset of a normed linear space is said to have the strong separation property if it can be strongly separated from every other disjoint, closed, and convex set by a closed hyperplane. In this paper, we give some results on the separation of convex sets noticing the role of barrier cones, develop some characterizations of subsets having the strong separation property, and apply them to consider a class of convex optimization problems.

Computation-Efficient Distributed Algorithm for Convex Optimization Over Time-Varying Networks With Limited Bandwidth Communication

A novel computation-efficient quantized distributed optimization algorithm is presented in this article for solving a class of convex optimization problems over time-varying undirected networks with limited communication capacity. These convex optimization problems are usually relevant to the minimization of a sum of local convex objective functions using only local communication and local computation. In most of the existing distributed optimization algorithms, each agent needs to calculate the subgradient of its local convex objective function at each time step, which leads to extremely heavy computation. The proposed algorithm incorporates random sleep scheme into procedures of agents’ updates in a probabilistic form to reduce the computation load, and further allows for uncoordinated step-sizes of all agents. The quantized strategy is also applied, which overcomes the limitation of communication capacity. Theoretical analysis indicates that the convex optimization problems can be solved and numerical analysis shows that the computation load of subgradient can be significantly reduced by the proposed algorithm. The boundedness of the quantization levels at each time step has been explicitly characterized. Simulation examples are presented to demonstrate the effectiveness of the algorithm and the correctness of the theoretical results.

Accelerated first-order methods for large-scale convex optimization: nearly optimal complexity under strong convexity

We introduce four accelerated (sub)gradient algorithms (ASGA) for solving several classes of convex optimization problems. More specifically, we propose two estimation sequences majorizing the objective function and develop two iterative schemes for each of them. In both cases, the first scheme requires the smoothness parameter and a Holder constant, while the second scheme is parameter-free (except for the strong convexity parameter which we set zero if it is not available) at the price of applying a finitely terminated backtracking line search. The proposed algorithms attain the optimal complexity for smooth problems with Lipschitz continuous gradients, nonsmooth problems with bounded variation of subgradients, and weakly smooth problems with Holder continuous gradients. Further, for strongly convex problems, they are optimal for smooth problems while nearly optimal for nonsmooth and weakly smooth problems. Finally, numerical results for some applications in sparse optimization and machine learning are reported, which confirm the theoretical foundations.

Application of the Method of Semidefinite Relaxation for Determining the Orientation of a Solid Body in Space

In this paper, the semidefinite relaxation method is applied to the problem of determining the orientation of a solid body relative to the local horizon using satellite navigation. It is shown how the initial nonconvex quadratic programming problem is immersed in a wider class of convex optimization problems that allow an effective solution. Instead of the original computationally complex problem, a convex problem was solved, giving an approximate solution of the original problem. The proposed approach is applied to the processing of experimental data.

Dynamic models of Wasserstein-1-type unbalanced transport

We consider a class of convex optimization problems modelling temporal mass transport and mass change between two given mass distributions (the so-called dynamic formulation of unbalanced transport), where we focus on those models for which transport costs are proportional to transport distance. For those models we derive an equivalent, computationally more efficient static formulation, we perform a detailed analysis of the model optimizers and the associated optimal mass change and transport, and we examine which static models are generated by a corresponding equivalent dynamic one. Alongside we discuss thoroughly how the employed model formulations relate to other formulations found in the literature.

Solution uniqueness of convex piecewise affine functions based optimization with applications to constrained ℓ1 minimization

In this paper, we study the solution uniqueness of an individual feasible vector of a class of convex optimization problems involving convex piecewise affine functions and subject to general polyhedral constraints. This class of problems incorporates many important polyhedral constrained ℓ1 recovery problems arising from sparse optimization, such as basis pursuit, LASSO, and basis pursuit denoising, as well as polyhedral gauge recovery. By leveraging the max-formulation of convex piecewise affine functions and convex analysis tools, we develop dual variables based necessary and sufficient uniqueness conditions via simple and yet unifying approaches; these conditions are applied to a wide range of ℓ1 minimization problems under possible polyhedral constraints. An effective linear program based scheme is proposed to verify solution uniqueness conditions. The results obtained in this paper not only recover the known solution uniqueness conditions in the literature by removing restrictive assumptions but also yield new uniqueness conditions for much broader constrained ℓ1-minimization problems.

Certifying Polynomial Nonnegativity via Hyperbolic Optimization

We describe a new approach to certifying the global nonnegativity of multivariate polynomials by solving hyperbolic optimization problems---a class of convex optimization problems that generalize s...