Constrained Convex Optimization Research Articles

Acceleration techniques for level bundle methods in weakly smooth convex constrained optimization

We develop a unified level-bundle method, called accelerated constrained level-bundle (ACLB) algorithm, for solving constrained convex optimization problems. where the objective and constraint functions can be nonsmooth, weakly smooth, and/or smooth. ACLB employs Nesterov’s accelerated gradient technique, and hence retains the iteration complexity as that of existing bundle-type methods if the objective or one of the constraint functions is nonsmooth. More importantly, ACLB can significantly reduce iteration complexity when the objective and all constraints are (weakly) smooth. In addition, if the objective contains a nonsmooth component which can be written as a specific form of maximum, we show that the iteration complexity of this component can be much lower than that for general nonsmooth objective function. Numerical results demonstrate the effectiveness of the proposed algorithm.

An interior-point algorithm for linearly constrained convex optimization based on kernel function and application in non-negative matrix factorization

In this paper, an interior point method (IPM) based on a new kernel function for solving linearly constrained convex optimization problems is presented. So, firstly a survey on several trigonometric kernel functions defined in literature is done and some properties of them are studied. Then some common characteristics of these functions which help us to define a new trigonometric kernel function are obtained. We generalize the growth term of the kernel function by applying a positive parameter p and rewritten the trigonometric kernel functions defined in the literature. By the help of some simple analysis tools, we show that the IPM based on the new kernel function obtains $$O\left( \sqrt{n}\log n\log \frac{n}{\epsilon }\right) $$ iteration complexity bound for large-update methods. Finally, we illustrate some numerical results of performing IPMs based on the kernel functions for solving non-negative matrix factorization problems.

Multi-Hop Wireless Optical Backhauling for LiFi Attocell Networks: Bandwidth Scheduling and Power Control

The backhaul of hundreds of light fidelity (LiFi) base stations (BSs) constitutes a major challenge. Building on an indoor wireless optical backhauling approach, this paper presents the top-down design of a multi-hop wireless backhaul configuration for multi-tier optical attocell networks by proposing super cells. Such cells incorporate multiple clusters of attocells that are connected to the core network via a single gateway. Consequently, new challenges arise for managing the bandwidth and power of the bottleneck backhaul. By putting forward user-based bandwidth scheduling (UBS) and cell-based bandwidth scheduling (CBS) policies, the system-level modeling and analysis of the end-to-end multi-user sum rate is elaborated. In addition, optimal bandwidth scheduling under both UBS and CBS policies are formulated as constrained convex optimization problems, and solved by using the projected subgradient method. Furthermore, the transmission power of the backhaul system is opportunistically reduced using a fixed power control (FPC) strategy. The notion of backhaul bottleneck occurrence (BBO) is introduced. An accurate approximate expression of the probability of BBO is derived, and verified using Monte Carlo simulations. Several insights are provided by studying different aspects of the performance of super cells including the average sum rate, the BBO probability and the backhaul power efficiency (PE).

Model Based Robust Predictive Control of Ship Roll/Yaw Motions with Input Constraints

A robust H∞-type state feedback model predictive control (H∞-SFMPC) with input constraints is proposed to optimize the control performance during the ship sailing. Specifically, the approach employed in this paper is able to optimize the closed-loop performance with respect to an H∞-type cost function which predicts the system performance based on the actual model instead of the ideal model. As a result, the effect caused by disturbances is attenuated. The state feedback control gain for the control input of the rudder-fin joint roll/yaw control system is obtained by solving a constrained convex optimization problem in terms of linear matrix inequalities. Simulations are carried out, which reveal that the proposed approach has outstanding control performance. Furthermore, it is found that the approach also has significant robustness with respect to parameter uncertainties.

An inexact proximal generalized alternating direction method of multipliers

This paper proposes and analyzes an inexact variant of the proximal generalized alternating direction method of multipliers (ADMM) for solving separable linearly constrained convex optimization problems. In this variant, the first subproblem is approximately solved using a relative error condition whereas the second one is assumed to be easy to solve. In many ADMM applications, one of the subproblems has a closed-form solution; for instance, $$\ell _1$$ regularized convex composite optimization problems. The proposed method possesses iteration-complexity bounds similar to its exact version. More specifically, it is shown that, for a given tolerance $$\rho >0$$ , an approximate solution of the Lagrangian system associated to the problem under consideration is obtained in at most $$\mathcal {O}(1/\rho ^2)$$ (resp. $$\mathcal {O}(1/\rho )$$ in the ergodic case) iterations. Numerical experiments are presented to illustrate the performance of the proposed scheme.

A Fixed-Point Subgradient Splitting Method for Solving Constrained Convex Optimization Problems

In this work, we consider a bilevel optimization problem consisting of the minimizing sum of two convex functions in which one of them is a composition of a convex function and a nonzero linear transformation subject to the set of all feasible points represented in the form of common fixed-point sets of nonlinear operators. To find an optimal solution to the problem, we present a fixed-point subgradient splitting method and analyze convergence properties of the proposed method provided that some additional assumptions are imposed. We investigate the solving of some well known problems by using the proposed method. Finally, we present some numerical experiments for showing the effectiveness of the obtained theoretical result.

Distributed Subgradient Method for Constrained Convex Optimization with Quantized and Event-Triggered Communication

Distributed Subgradient Method for Constrained Convex Optimization with Quantized and Event-Triggered Communication

Distributed continuous-time algorithm for nonsmooth optimal consensus without sharing local decision variables

A distributed continuous-time algorithm is proposed for constrained nonsmooth convex optimization. A distinct feature of our algorithm is that it does not require agents to share their local decision variables to the network, and it still achieves the optimal solution. With the help of Lagrangian functions, exact penalty techniques, differential inclusions with maximal monotone maps and saddle-point dynamics, we prove the convergence of the proposed algorithm and show that it achieves an O(1/t) convergence rate. Numerical example also illustrates the effectiveness of the proposed method.

A Computation-Efficient Decentralized Algorithm for Composite Constrained Optimization

This paper focuses on solving the problem of composite constrained convex optimization with a sum of smooth convex functions and non-smooth regularization terms (ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norm) subject to locally general constraints. Motivated by the modern large-scale information processing problems in machine learning (the samples of a training dataset are randomly decentralized across multiple computing nodes), each of the smooth objective functions is further considered as the average of several constituent functions. To address the problem in a decentralized fashion, we propose a novel computation-efficient decentralized stochastic gradient algorithm, which leverages the variance reduction technique and the decentralized stochastic gradient projection method with constant step-size. Theoretical analysis indicates that if the constant step-size is less than an explicitly estimated upper bound, the proposed algorithm can find the exact optimal solution in expectation when each constituent function (smooth) is strongly convex. Concerning the existing decentralized schemes, the proposed algorithm not only is suitable for solving the general constrained optimization problems but also possesses low computation cost in terms of the total number of local gradient evaluations. Furthermore, the proposed algorithm via differential privacy strategy can effectively mask the privacy of each constituent function, which is more practical in applications involving sensitive messages, such as military affairs or medical treatment. Finally, numerical evidence is provided to demonstrate the appealing performance of the proposed algorithm.

Nesterov Acceleration for Equality-Constrained Convex Optimization via Continuously Differentiable Penalty Functions

We propose a framework to use Nesterov's accelerated method for constrained convex optimization problems. Our approach consists of first reformulating the original problem as an unconstrained optimization problem using a continuously differentiable exact penalty function. This reformulation is based on replacing the Lagrange multipliers in the augmented Lagrangian of the original problem by Lagrange multiplier functions. The expressions of these Lagrange multiplier functions, which depend upon the gradients of the objective function and the constraints, can make the unconstrained penalty function non-convex in general even if the original problem is convex. We establish sufficient conditions on the objective function and the constraints of the original problem under which the unconstrained penalty function is convex. This enables us to use Nesterov's accelerated gradient method for unconstrained convex optimization and achieve a guaranteed rate of convergence which is better than the state-of-the-art first-order algorithms for constrained convex optimization. Simulations illustrate our results.

Online Trajectory Optimization Using Inexact Gradient Feedback for Time-Varying Environments

This article considers the problem of online trajectory design under time-varying environments. We formulate the general trajectory optimization problem within the framework of time-varying constrained convex optimization and propose a novel version of online gradient ascent algorithm (OGA) for such problems. Respecting the online nature, we carefully select the step size of OGA at each iteration so that the iterates stay feasible. Importantly, the proposed algorithm allows noisy gradients, expanding the range of practical applicability. In contrast to the most available literature, we present the offline sublinear regret of OGA up to the path length variations of the offline optimal solution, the cumulative gradient, and the error in the gradient variations. Furthermore, we establish a lower-bound on the offline dynamic regret, which defines the optimality of any trajectory. To show the efficacy of the proposed algorithm, we consider two practical problems of interest. First, a device to device (D2D) communications setting, where the goal is to design a user trajectory while maximizing its connectivity to the internet. Second, planning energy-efficient trajectories for unmanned surface vehicles (USV) under strong disturbances in ocean environments. Different from the state-of-the-art trajectory planning algorithms that entail planning and re-planning the full trajectory using the forecast data at each time instant, the proposed algorithm is entirely online and relies mostly on the ocean velocity measurements at the vehicle location. The detailed simulation results demonstrate the significance of the proposed algorithm on both synthetic and real data sets. Video result is available at https://tinyurl.com/y3ahmhsf .

Distributed constrained convex optimization over digraphs: A Fenchel dual based approach

Distributed constrained convex optimization over digraphs: A Fenchel dual based approach

Alternating direction method of multipliers with variable metric indefinite proximal terms for convex optimization

<p style='text-indent:20px;'>This paper studies a proximal alternating direction method of multipliers (ADMM) with variable metric indefinite proximal terms for linearly constrained convex optimization problems. The proximal ADMM plays an important role in many application areas, since the subproblems of the method are easy to solve. Recently, it is reported that the proximal ADMM with a certain fixed indefinite proximal term is faster than that with a positive semidefinite term, and still has the global convergence property. On the other hand, Gu and Yamashita studied a variable metric semidefinite proximal ADMM whose proximal term is generated by the BFGS update. They reported that a slightly indefinite matrix also makes the algorithm work well in their numerical experiments. Motivated by this fact, we consider a variable metric indefinite proximal ADMM, and give sufficient conditions on the proximal terms for the global convergence. Moreover, based on the BFGS update, we propose a new indefinite proximal term which can satisfy the conditions for the global convergence. Experiments on several datasets demonstrated that our proposed variable metric indefinite proximal ADMM outperforms most of the comparison proximal ADMMs.

Random Gradient-Free Optimization for Multiagent Systems With Communication Noises Under a Time-Varying Weight Balanced Digraph

In this paper, we focus on a constrained convex optimization problem of multiagent systems under a time-varying topology. In such topology, it is not only B-strongly connected, but the communication noises are also existent. Each agent has access to its local cost function, which is a nonsmooth function. A gradient-free random protocol is come up with minimizing a sum of cost functions of all agents, which are projected to local constraint sets. First, considering the stochastic disturbances in the communication channels among agents, the upper bounds of disagreement estimate of agents’ states are obtained. Second, a sufficient condition on choosing step sizes and smoothing parameters is derived to guarantee that all agents almost surely converge to the stationary optimal point. At last, a numerical example and a comparison are provided to illustrate the feasibility of the random gradient-free algorithm.

Safe Feature Elimination for Non-negativity Constrained Convex Optimization

Inspired by recent work on safe feature elimination for $1$-norm regularized least-squares, we develop strategies to eliminate features from convex optimization problems with non-negativity constraints. Our strategy is safe in the sense that it will only remove features/coordinates from the problem when they are guaranteed to be zero at a solution. To perform feature elimination we use an accurate, but not optimal, primal-dual feasible pair, making our methods robust and able to be used on ill-conditioned problems. We supplement our feature elimination problem with a method to construct an accurate dual feasible point from an accurate primal feasible point; this allows us to use a first-order method to find an accurate primal feasible point, then use that point to construct an accurate dual feasible point and perform feature elimination. Under reasonable conditions, our feature elimination strategy will eventually eliminate all zero features from the problem. As an application of our methods we show how safe feature elimination can be used to robustly certify the uniqueness of non-negative least-squares (NNLS) problems. We give numerical examples on a well-conditioned synthetic NNLS problem and a on set of 40000 extremely ill-conditioned NNLS problems arising in a microscopy application.

Selecting the Optimal Subset of Antennas in Hyperthermia Treatment Planning

Hyperthermia treatment planning is a deeply patient-specific task which includes the optimal determination of the excitations of an array applicator. To enhance flexibility, various solutions exploiting different frequencies, antenna element, number, and applicator geometries have been proposed in the literature. Amongst them, increasing the frequency and the number of radiating elements have been effective for achieving more conformal heating. However, as each radiating element requires a power amplifier to control it, increasing the number of antennas considerably impacts the overall cost and complexity of the system. Accordingly, a procedure capable of selecting an optimal patient-specific subset of antennas from an oversized phased array applicator (with more antenna elements than available amplifiers) could help improve cost-effectiveness. In this study, we present an original approach, which allows improving performance by adaptively selecting the optimal subset of antennas to be activated for a given (redundant) applicator and a given patient. The proposed approach takes inspiration from the compressive sensing theory by embedding the sparsity promotion paradigm into a treatment planning procedure, which casts power deposition as a constrained convex optimization. Performances were demonstrated for the case of head and neck hyperthermia and benchmarked against the antenna selection procedure presently used in clinical practice.

An Automatic Collision Avoidance Algorithm for Multiple Marine Surface Vehicles

Abstract In recent years, unmanned surface vehicles have been widely used in various applications from military to civil domains. Seaports are crowded and ship accidents have increased. Thus, collision accidents occur frequently mainly due to human errors even though international regulations for preventing collisions at seas (COLREGs) have been established. In this paper, we propose a real-time obstacle avoidance algorithm for multiple autonomous surface vehicles based on constrained convex optimization. The proposed method is simple and fast in its implementation, and the solution converges to the optimal decision. The algorithm is combined with the PD-feedback linearization controller to track the generated path and to reach the target safely. Forces and azimuth angles are efficiently distributed using a control allocation technique. To show the effectiveness of the proposed collision-free path-planning algorithm, numerical simulations are performed.

An adaptive primal-dual framework for nonsmooth convex minimization

We propose a new self-adaptive and double-loop smoothing algorithm to solve composite, nonsmooth, and constrained convex optimization problems. Our algorithm is based on Nesterov’s smoothing technique via general Bregman distance functions. It self-adaptively selects the number of iterations in the inner loop to achieve a desired complexity bound without requiring to set the accuracy a priori as in variants of augmented Lagrangian methods (ALM). We prove $$\mathcal {O}\left( \frac{1}{k}\right) $$ -convergence rate on the last iterate of the outer sequence for both unconstrained and constrained settings in contrast to ergodic rates which are common in ALM as well as alternating direction method-of-multipliers literature. Compared to existing inexact ALM or quadratic penalty methods, our analysis does not rely on the worst-case bounds of the subproblem solved by the inner loop. Therefore, our algorithm can be viewed as a restarting technique applied to the ASGARD method in Tran-Dinh et al. (SIAM J Optim 28(1):96–134, 2018) but with rigorous theoretical guarantees or as an inexact ALM with explicit inner loop termination rules and adaptive parameters. Our algorithm only requires to initialize the parameters once, and automatically updates them during the iteration process without tuning. We illustrate the superiority of our methods via several examples as compared to the state-of-the-art.

Two New Customized Proximal Point Algorithms Without Relaxation for Linearly Constrained Convex Optimization

In this paper, we propose a new customized proximal point algorithm for linearly constrained convex optimization problem, and further extend the proposed method to separable convex optimization problem. Unlike the existing customized proximal point algorithms, the proposed algorithms do not involve relaxation step, but still ensure the convergence. We obtain the particular iteration schemes and the unified variational inequality perspective. The global convergence and $${\mathcal {O}}(1/k)$$-convergence rate of the proposed methods are investigated under some mild assumptions. Numerical experiments show that compared to some state-of-the-art methods, the proposed methods are effective.

A consensus algorithm based on collective neurodynamic system for distributed optimization with linear and bound constraints

In this paper, an algorithm based on collective neurodynamic system is investigated for distributed constrained convex optimization, whose objective function is a sum of smooth convex functions and non-smooth L1-norm functions. Inspired by recent advances in distributed convex optimization, the continuous-time and discrete-time distributed optimization algorithms described by collective neurodynamic systems are proposed. In the systems, each of the smooth objective functions is allocated to each node as well as each of the L1-norm functions. However, the L1-norm functions are realized by projection operators. Meanwhile, each node satisfies the local linear and bound constraints. Then a connected network is constituted from all the nodes with consensus to find the optimal solutions.