An ADMM-based heuristic algorithm for optimization problems over nonconvex second-order cone

This chapter is devoted to the blackbox subgradient algorithms with the minimal requirements for the storage of auxiliary results, which are necessary to execute these algorithms. To provide historical perspective this survey starts with the original result of Shor which opened this field with the application to the classical transportation problem. The theoretical complexity bounds for smooth and nonsmooth convex and quasiconvex optimization problems are briefly exposed in what follows to introduce the relevant fundamentals of nonsmooth optimization. Special attention in this section is given to the adaptive step size policy which aims to attain lowest complexity bounds. Nondifferentiability of objective function in convex optimization significantly slows down the rate of convergence in subgradient optimization compared to the smooth case, but there are different modern techniques that allow to solve nonsmooth convex optimization problems faster than dictate theoretical lower complexity bounds. In this work the particular attention is given to Nesterov smoothing technique, Nesterov universal approach, and Legendre (saddle point) representation approach. The new results on universal mirror prox algorithms represent the original parts of the survey. To demonstrate application of nonsmooth convex optimization algorithms to solution of huge-scale extremal problems we consider convex optimization problems with nonsmooth functional constraints and propose two adaptive mirror descent methods. The first method is of primal-dual variety and proved to be optimal in terms of lower oracle bounds for the class of Lipschitz continuous convex objectives and constraints. The advantages of application of this method to the sparse truss topology design problem are discussed in essential details. The second method can be used for solution of convex and quasiconvex optimization problems and it is optimal in terms of complexity bounds. The conclusion part of the survey contains the important references that characterize recent developments of nonsmooth convex optimization.

This paper is a follow up to the previous author's paper on convex optimization. In that paper we began the process of adjusting greedy-type algorithms from nonlinear approximation for finding sparse solutions of convex optimization problems. We modified there three the most popular in nonlinear approximation in Banach spaces greedy algorithms -- Weak Chebyshev Greedy Algorithm, Weak Greedy Algorithm with Free Relaxation and Weak Relaxed Greedy Algorithm -- for solving convex optimization problems. We continue to study sparse approximate solutions to convex optimization problems. It is known that in many engineering applications researchers are interested in an approximate solution of an optimization problem as a linear combination of elements from a given system of elements. There is an increasing interest in building such sparse approximate solutions using different greedy-type algorithms. In this paper we concentrate on greedy algorithms that provide expansions, which means that the approximant at the $m$th iteration is equal to the sum of the approximant from the previous iteration ($(m-1)$th iteration) and one element from the dictionary with an appropriate coefficient. The problem of greedy expansions of elements of a Banach space is well studied in nonlinear approximation theory. At a first glance the setting of a problem of expansion of a given element and the setting of the problem of expansion in an optimization problem are very different. However, it turns out that the same technique can be used for solving both problems. We show how the technique developed in nonlinear approximation theory, in particular, the greedy expansions technique can be adjusted for finding a sparse solution of an optimization problem given by an expansion with respect to a given dictionary.

This thesis details the design and analysis of sequential procedures for the joint inference problem associated with hypothesis testing and parameter estimation in the context of sequentially observed data. The goal achieved is to minimize the average number of samples required to meet predefined detection and estimation error levels; thus fast inference with guaranteed performance. The first half of the thesis is devoted to the design of strictly optimal procedures, i.e., procedures that use, on average, as few samples as possible and fulfill constraints on the detection and estimation error levels. The design problem is formulated as a constrained optimization problem. The selected approach is to convert the problem to an unconstrained optimization problem and subsequently, to an optimal stopping problem. The solution of the optimal stopping problem is characterized by recursively defined non-linear integral equations that are parameterized by a set of cost coefficients. It is shown that the partial derivatives of the cost function, with respect to the cost coefficients, are equal to the detection/estimation errors up to a constant scaling factor. Based on this property, the choice of the coefficients that lead to an optimal solution is formulated as a convex optimization problem. Two numerical algorithms are provided to solve this optimization problem. The first one converts the optimization problem to a linear program. The second one solves it directly via a projected quasi-Newton method. Numerical examples are given that verify the proposed design procedure. In the second part of this dissertation, asymptotically optimal procedures for sequential joint detection and estimation are detailed. To avoid the computationally expensive solutions associated with optimality, an alternative procedure is proposed, which becomes optimal when the constraints on the detection and estimation errors approach zero. After a formal definition of asymptotic optimality, an asymptotically optimal stopping rule is detailed. This stopping rule is implemented by thresholding the instantaneous cost that is parameterized by a set of cost coefficients. It is shown, asymptotically, and similarly to the strictly optimal procedure, that the partial derivatives of the solution of the optimal stopping problem and the detection/estimation errors, differ only by a constant scaling factor. Using this result, it is shown how the projected quasi-Newton method, derived for the design of optimal procedures, can be adapted to choose the cost coefficients such that the constraints on the detection and estimation errors are fulfilled. The proposed asymptotically optimal procedures are applied to example problems that are motivated by real-world applications. By means of a numerical example, it is shown that the asymptotically optimal procedure uses, on average, only slightly more samples than the strictly optimal one while requiring significantly less computational resources to be implemented. This thesis provides a coherent framework for the design and analysis of strictly optimal, as well as asymptotically optimal procedures, for the problem of sequential joint detection and estimation.

Motivated by weakly convex optimization and quadratic optimization problems, we first show that there is no duality gap between a difference of convex (DC) program over DC constraints and its associated dual problem. We then provide certificates of global optimality for a class of nonconvex optimization problems. As an application, we derive characterizations of robust solutions for uncertain general nonconvex quadratic optimization problems over nonconvex quadratic constraints.

In this paper, we consider an uncrewed aerial vehicle-assisted integrated sensing and communication system, consisting of an aerial base station (ABS) and a mobile radar (MR) equipped with a radar receiver. The ABS hovers at a specific position to provide communication services to user clusters, while the MR flies along a predefined trajectory to receive communication reflection signals for target sensing. By taking into account user communication and target sensing performance, as well as flight energy consumption of the MR, the system utility function is modeled as a weighted sum of the minimum average rate of user clusters, sensing channel gain, and flight energy consumption of the MR. The joint communication precoding design, ABS deployment, MR trajectory planning, and communication sensing scheduling problem is modeled as a constrained system utility function maximization problem. Given that the modeled optimization problem is a highly coupled and non-convex mixed-integer optimization problem, it is challenging to solve directly. To tackle this problem, we decompose the original problem into four subproblems and sequentially solve each subproblem using an alternating iteration algorithm. Specifically, for the communication precoding design subproblem, the zero-forcing algorithm is used to eliminate the interference among users and the original problem is transformed into a semi-definite programming problem. For the ABS deployment subproblem and the MR trajectory planning subproblem, the Taylor expansion and the successive convex approximation are employed and slack variables are introduced to convert the original problems into convex optimization problems. For the communication sensing scheduling subproblem, the variable relaxation method is adopted to relax the binary variables into continuous variables, and the optimization tool is used to obtain the solution. Then, two heuristic algorithms are proposed to restore the communication and sensing scheduling variables. Finally, the effectiveness of the proposed algorithms is verified through simulations.

Second-order cone programming (SOCP) problems are convex optimization problems in which a linear function is minimized over the intersection of an affine linear manifold with the Cartesian product of second-order (Lorentz) cones. Linear programs, convex quadratic programs and quadratically constrained convex quadratic programs can all be formulated as SOCP problems, as can many other problems that do not fall into these three categories. These latter problems model applications from a broad range of fields from engineering, control and finance to robust optimization and combinatorial optimization. On the other hand semidefinite programming (SDP)—that is the optimization problem over the intersection of an affine set and the cone of positive semidefinite matrices—includes SOCP as a special case. Therefore, SOCP falls between linear (LP) and quadratic (QP) programming and SDP. Like LP, QP and SDP problems, SOCP problems can be solved in polynomial time by interior point methods. The computational effort per iteration required by these methods to solve SOCP problems is greater than that required to solve LP and QP problems but less than that required to solve SDP’s of similar size and structure. Because the set of feasible solutions for an SOCP problem is not polyhedral as it is for LP and QP problems, it is not readily apparent how to develop a simplex or simplex-like method for SOCP. While SOCP problems can be solved as SDP problems, doing so is not advisable both on numerical grounds and computational complexity concerns. For instance, many of the problems presented in the survey paper of Vandenberghe and Boyd [VB96] as examples of SDPs can in fact be formulated as SOCPs and should be solved as such. In §2, 3 below we give SOCP formulations for four of these examples: the convex quadratically constrained quadratic programming (QCQP) problem, problems involving fractional quadratic functions ∗RUTCOR, Rutgers University, e-mail:alizadeh@rutcor.rutgers.edu. Research supported in part by the U.S. National Science Foundation grant CCR-9901991 †IEOR, Columbia University, e-mail: gold@ieor.columbia.edu. Research supported in part by the Department of Energy grant DE-FG02-92ER25126, National Science Foundation grants DMS-94-14438, CDA-97-26385 and DMS-01-04282.

This paper solves the time and power allocation problem for the simplest feedback scheme for the Gaussian wiretap channel, which is based on the transmission of random secret keys to be used in a one time pad manner. Specifically, the optimal transmission powers at Alice and Bob, as well as the time sharing factor between the feedback and feedforward channels, are given by the solution of a non-convex optimization problem, which is found by means of the golden section algorithm and the sequential solution of several convex optimization problems. Additionally, an specific and highly efficient procedure for the solution of the inner convex optimization problems is provided, which avoids the need for general purpose optimization packages. Finally, several simulation results illustrate the potential secrecy gains achievable with a feedback scheme as simple as the one considered in this paper.

A novel neural network to nonlinear complex-variable constrained nonconvex optimization

This paper proposes a general framework for solving multiobjective nonconvex optimization problems, i.e., optimization problems in which multiple objective functions have to be optimized simultaneously. Thereby, the nonconvexity might come from the objective or constraint functions, or from integrality conditions for some of the variables. In particular, multiobjective mixed-integer convex and nonconvex optimization problems are covered and form the motivation of our studies. The presented algorithm is based on a branch-and-bound method in the pre-image space, a technique which was already successfully applied for continuous nonconvex multiobjective optimization. However, extending this method to the mixed-integer setting is not straightforward, in particular with regard to convergence results. More precisely, new branching rules and lower bounding procedures are needed to obtain an algorithm that is practically applicable and convergent for multiobjective mixed-integer optimization problems. Corresponding results are a main contribution of this paper. What is more, for improving the performance of this new branch-and-bound method we enhance it with two types of cuts in the image space which are based on ideas from multiobjective mixed-integer convex optimization. Those combine continuous convex relaxations with adaptive cuts for the convex hull of the mixed-integer image set, derived from supporting hyperplanes to the relaxed sets. Based on the above ingredients, the paper provides a new multiobjective mixed-integer solver for convex problems with a stopping criterion purely in the image space. What is more, for the first time a solver for multiobjective mixed-integer nonconvex optimization is presented. We provide the results of numerical tests for the new algorithm. Where possible, we compare it with existing procedures.

ABSTRACTIn this paper, we present adaptive event‐triggered distributionally robust optimization stochastic model predictive control (AET‐DROSMPC) applied to DC‐DC converters subject to unknown disturbances and denial of service (DoS) attacks. The DoS attacks, causing communication interruptions on the controller and actuator (C‐A) channel, is described by using Bernoulli variables. While stochastic model predictive control (SMPC) has been extensively studied, existing approaches mostly focus on periodic stochastic model predictive control (PSMPC) and self‐triggering stochastic model predictive control (SSMPC) for systems with bounded disturbances or subject to Gaussian distribution. To address chance constraints and disturbances more effectively, we introduce the distributionally robust optimization (DRO) to transform the optimization problem with chance constraints into a convex optimization problem using second‐order cone (SOC) equivalence. Moreover, adaptive event‐triggered mechanism (AETM) is devised to reduce unnecessary sampling, lower updating frequencies of control input, and ultimately decrease the computational burden of the system, addressing the lack of consideration for exact sampling times and system computing burden. The study rigorously establishes the recursive feasibility and stability of the optimization problem subject to DoS attacks. Finally, a application is conducted to demonstrate the effectiveness and advancements of the proposed algorithm.

Goal Programming (GP) is a multi-objective optimization technique that is used when there are multiple conflicting goals in the cost function of an optimization problem. Lexicographic Goal Programming (LGP) is most commonly used form of GP when there are clear priorities in the goals that leads to a lexicographic order, i.e., priroritization. We can solve LGP problems by solving a sequence of optimization problems, where the optimal cost of an optimization problem in the sequence becomes a constraint for all optimization problems that are solved afterwards. In this paper, we present the basic LGP framework where each problem in the sequence is a convex optimization problem. We provide examples from control systems where LGP is a natural choice due to the clear priorities of objectives. LGP formulation leads to a sequence of convex optimization problems, which enables us to use polynomial time Interior Point Method (IPM) algorithms to solve these problems efficiently and potentially in real-time. In some cases, we also show that this prioritization provides convexification of the problem at hand, which would have otherwise required the solution of a non-convex optimization problem. Our primary objective in writing this paper is to illustrate the usefulness LGP formulation of control problems that can be encountered in aerospace engineering.

Rocket powered landing trajectory planning is a typical non-convex and non-smooth optimization problem. To solve this problem, a convex optimization method based on hp-adaptive pseudospectral method is proposed. Combining the advantages of these two methods, the proposed method can quickly and accurately obtain the optimal trajectory. Firstly, the original non-convex optimal control problem is transformed into a convex optimal problem by lossless convexification and change of variables convexifying the non-convex thrust constraint and nonlinear dynamics respectively. Then, t Then, the hp adaptive pseudospectral method is used to discretize the time-continuous problem, in which the position of the discrete points can be adaptively adjusted to the control discontinuities according to the curvature of the state variable. Finally, the original non-convex problem can be transformed into a second-order cone programming problem, and the highly efficient interior point method can be used to quickly solve the problem. Numerical simulation results show that the proposed method can quickly solve the non-smooth trajectory optimization problem and accurately capture the control discontinuities.

This work addresses the optimistic statement of a bilevel optimization problem with a general d.c. optimization problem at the upper level and a convex optimization problem at the lower level. First, we use the reduction of the bilevel problem to a nonconvex mathematical optimization problem using the well-known Karush-Kuhn-Tucker approach. Then we employ the novel Global Search Theory and Exact Penalty Theory to solve the resulting nonconvex optimization problem. Following this theory, the special method of local search in this problem is constructed. This method takes into account the structure of the problem in question.

In a network with a high density of wireless nodes, we model flow of information by a continuous vector field known as the information flow vector field. We use a mathematical model that translates a communication network composed of a large but finite number of sensors into a continuum of nodes on which information flow is formulated by a vector field. The magnitude of this vector field is the intensity of the communication activity, and its orientation is the direction in which the traffic is forwarded. The information flow vector field satisfies a set of Neumann boundary conditions and a partial differential equation (PDE) involving the divergence of information, but the divergence constraint and Neumann boundary conditions do not specify the information flow vector field uniquely, and leave us freedom to optimize certain measures within their feasible set. Therefore, we introduce a <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p-norm</i> flow optimization problem in which we minimize the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p-norm</i> of information flow vector field over the area of the network. This problem is a convex optimization problem, and we use sequential quadratic programming (SQP) to solve it. SQP is known for numerical stability and fast convergence to the optimal solution in convex optimization problems. By using standard SQP on <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p-norm</i> flow optimization, we prove that the solution of each iteration of SQP is uniquely specified by an elliptic PDE with generalized Neumann boundary conditions. The <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p-norm</i> flow optimization shows interesting properties for different values of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i> . For example, if p is close to one, the information routes resemble the geometric shortest paths of the sources and sinks, and for <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i> = 2, the information flow shows an analogy to electrostatics. For infinitely large values of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i> , the problem minimizes the maximum magnitude of the information vector field over the network, and hence it achieves maximum load balancing.

We formulate a frequency-division multiple access (FDMA) networking problem for wireless mobile ad-hoc networks (MANETS) to jointly optimize end-to-end user rates, routes, link capacities, transmitted power, frequency and power allocation across subcarriers and fading states. We show that the resulting non-convex optimization problem has zero duality gap. For some types of FDMA networks this result is exploited to reformulate the original problem into a (computationally tractable) convex optimization problem. We further exploit the lack of duality gap to show that conventional layering can be optimal in FDMA wireless MANETS. Specifically, if we select Lagrange multipliers appropriately, we can decompose the original problem in smaller sub-problems associated with the conventional networking layers. The solution of these per-layer optimization problems coincides with the solution of the originally formulated cross-layer optimization problem.