KKT Solution Research Articles

A Necessary and Sufficient Condition for Computed OPF Solutions to be Locally Optimal

Although a significant amount of research has been directed to developing necessary and sufficient conditions for the optimal solutions of several engineering problems, there is little work on the development of a necessary and sufficient condition for optimal power flow (OPF) solutions. This paper seeks to develop the following: (1) a necessary and sufficient condition for a local OPF solution, and (2) a necessary and sufficient condition for an OPF Karush-Kuhn-Tucker (KKT) solution. The derived necessary and sufficient conditions were numerically verified on IEEE 30-, 57- and 118-bus systems. Computationally, the derived necessary and sufficient condition is also useful in assisting existing OPF methods to obtain OPF solutions instead of KKT solutions (i.e., KKT saddle points).

Energy-Efficient Joint Collaborative and Passive Beamforming for Intelligent-Reflecting-Surface-Assisted Wireless Sensor Networks

Intelligent reflecting surface (IRS) provides a promising technology that can improve the energy efficiency of wireless communications, by building the controllable wireless propagation environment with the utilization of massive passive reflecting elements. Motivated by this, we apply the IRS to assist the data collection based on collaborative beamforming in Wireless Sensor Networks (WSNs). With the objective to maximize the network lifetime, we present an energy efficient joint collaborative and passive beamforming design for an IRS-assisted WSN, which jointly optimizes the beamforming vector for all nodes and phase shifts for IRS’s reflecting elements. To efficiently resolve the joint optimization problem, we develop a penalty dual decomposition (PDD) based algorithm combined with augmented Lagrangian method, penalty-based method and block coordinate descent algorithm, which decomposes the joint optimization problem into four simplified subproblems. Especially in solving IRS’s phase shifts optimization subproblem, we formulate this subproblem as a quadratically constrained quadratic programming (QCQP) problem and further propose a low computational complexity approximate iteration algorithm to resolve this QCQP problem, which iteratively linearizes this QCQP problem to a sequence of approximate programming with linear equality constraints, whose closed-form solution is presented. Theoretical analysis indicates the proposed algorithm can converge to a KKT solution. Simulation results indicate that the IRS-assisted design and proposed algorithm improve the energy efficiency and prolong the network lifetime as compared to alternative algorithms.

Iteration-Complexity of First-Order Augmented Lagrangian Methods for Convex Conic Programming

In this paper we consider a class of convex conic programming. In particular, we first propose an inexact augmented Lagrangian (I-AL) method that resembles the classical I-AL method for solving this problem, in which the augmented Lagrangian subproblems are solved approximately by a variant of Nesterov’s optimal first-order method. We show that the total number of first-order iterations of the proposed I-AL method for finding an -KKT solution is at most . We then propose an adaptively regularized I-AL method and show that it achieves a first-order iteration complexity , which significantly improves existing complexity bounds achieved by first-order I-AL methods for finding an -KKT solution. Our complexity analysis of the I-AL methods is based on a sharp analysis of the inexact proximal point algorithm (PPA) and the connection between the I-AL methods and inexact PPA. It is vastly different from existing complexity analyses of the first-order I-AL methods in the literature, which typically regard the I-AL methods as an inexact dual gradient method.

Communication-Efficient Federated Multitask Learning Over Wireless Networks

This article investigates the scheduling framework of the federated multitask learning (FMTL) problem with a hard-cooperation structure over wireless networks, in which the scheduling becomes more challenging due to the different convergence behaviors of different tasks. Based on the special model structure, we propose a dynamic user and task scheduling scheme with a block-wise incremental gradient aggregation algorithm, in which the neural network model is decomposed into a common feature-extraction module and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$M$ </tex-math></inline-formula> task-specific modules. Different block gradients with respect to different modules can be scheduled separately. We further propose a Lyapunov-drift-based scheduling scheme that minimizes the overall communication latency by utilizing both the instantaneous data importance and the channel state information. We prove that the proposed scheme can converge almost surely to a KKT solution of the training problem such that the data-distortion issue is resolved. Simulation results illustrate that the proposed scheme significantly reduces the communication latency compared to the state-of-the-art baseline schemes.

An inexact proximal augmented Lagrangian framework with arbitrary linearly convergent inner solver for composite convex optimization

We propose an inexact proximal augmented Lagrangian framework with explicit inner problem termination rule for composite convex optimization problems. We consider arbitrary linearly convergent inner solver including in particular stochastic algorithms, making the resulting framework more scalable facing the ever-increasing problem dimension. Each subproblem is solved inexactly with an explicit and self-adaptive stopping criterion, without requiring to set an a priori target accuracy. When the primal and dual domain are bounded, our method achieves \(O(1/\sqrt{\epsilon })\) and \(O(1/{\epsilon })\) complexity bound in terms of number of inner solver iterations, respectively for the strongly convex and non-strongly convex case. Without the boundedness assumption, only logarithm terms need to be added and the above two complexity bounds increase respectively to \({\tilde{O}}(1/\sqrt{\epsilon })\) and \({\tilde{O}}(1/{\epsilon })\), which hold both for obtaining \(\epsilon \)-optimal and \(\epsilon \)-KKT solution. Within the general framework that we propose, we also obtain \({\tilde{O}}(1/{\epsilon })\) and \({\tilde{O}}(1/{\epsilon ^2})\) complexity bounds under relative smoothness assumption on the differentiable component of the objective function. We show through theoretical analysis as well as numerical experiments the computational speedup possibly achieved by the use of randomized inner solvers for large-scale problems.

Energy-Efficient Joint Association and Precoding in Ultra-Dense C-RAN

In this work, we consider the joint optimization of virtual cell association and the precoder design for a multi-user downlink cloud radio access network (C-RAN). The optimal design of cell associations and precoders are particularly relevant due to the anticipated ultra-dense networks suffering from heavy interference. As the resulting optimization is a non-convex and combinatorial problem, we propose an iterative low complexity algorithm, employing the recently developed penalty dual decomposition (PDD) framework. In contrast to predominantly used sparsity inducing methods, the proposed method incorporates the penalty/re-weighting strategy into the algorithm, which is then solved as a sequence of convex second-order cone programs. The aforementioned feature leads to a high convergence rate and subsequently a low computational cost for the proposed algorithm. Furthermore, the proposed algorithm meets the requirements to converge to a KKT solution. Numerical simulations verify the superior performance of our proposed method in ultra-dense networks, in comparison to commonly used methods, such as the reweighted $\ell _1$ -norm, in terms of system energy efficiency.

Two-Stage Stochastic Optimization Via Primal-Dual Decomposition and Deep Unrolling

We consider a two-stage stochastic optimization problem, in which a long-term optimization variable is coupled with a set of short-term optimization variables in both objective and constraint functions. Despite that two-stage stochastic optimization plays a critical role in various engineering and scientific applications, there still lack efficient algorithms, especially when the long-term and short-term variables are coupled in the constraints. To overcome the challenge caused by tightly coupled stochastic constraints, we first establish a two-stage primal-dual decomposition (PDD) method to decompose the two-stage problem into a long-term problem and a family of short-term subproblems. Then we propose a PDD-based stochastic successive convex approximation (PDD-SSCA) algorithmic framework to find KKT solutions for two-stage stochastic optimization problems. At each iteration, PDD-SSCA first runs a short-term sub-algorithm to find stationary points of the short-term subproblems associated with a mini-batch of the state samples. Then it constructs a convex surrogate for the long-term problem based on the deep unrolling of the short-term sub-algorithm and the back propagation method. Finally, the optimal solution of the convex surrogate problem is solved to generate the next iterate. We establish the almost sure convergence of PDD-SSCA and customize the algorithmic framework to solve two important application problems. Simulations show that PDD-SSCA can achieve superior performance over existing solutions.

Convergence Analysis on an Accelerated Proximal Point Algorithm for Linearly Constrained Optimization Problems

Proximal point algorithm is a type of method widely used in solving optimization problems and some practical problems such as machine learning in recent years. In this paper, a framework of accelerated proximal point algorithm is presented for convex minimization with linear constraints. The algorithm can be seen as an extension to G u ¨ ler’s methods for unconstrained optimization and linear programming problems. We prove that the sequence generated by the algorithm converges to a KKT solution of the original problem under appropriate conditions with the convergence rate of O 1 / k 2 .

Penalty Dual Decomposition Method for Nonsmooth Nonconvex Optimization—Part I: Algorithms and Convergence Analysis

Many contemporary signal processing, machine learning and wireless communication applications can be formulated as nonconvex nonsmooth optimization problems. Often there is a lack of efficient algorithms for these problems, especially when the optimization variables are nonlinearly coupled in some nonconvex constraints. In this work, we propose an algorithm named penalty dual decomposition (PDD) for these difficult problems and discuss its various applications. The PDD is a double-loop iterative algorithm. Its inner iteration is used to inexactly solve a nonconvex nonsmooth augmented Lagrangian problem via block-coordinate-descent-type methods, while its outer iteration updates the dual variables and/or a penalty parameter. In Part I of this work, we describe the PDD algorithm and establish its convergence to KKT solutions. In Part II we evaluate the performance of PDD by customizing it to three applications arising from signal processing and wireless communications.

Strong calmness of perturbed KKT system for a class of conic programming with degenerate solutions

ABSTRACTThis paper is concerned with the strong calmness of the KKT solution mapping for a class of canonically perturbed conic programming, which plays a central role in achieving fast convergence under situations when the Lagrange multiplier associated to a solution of these conic optimization problems is not unique. We show that the strong calmness of the KKT solution mapping is equivalent to a local error bound for the solutions to the perturbed KKT system, and is also equivalent to the pseudo-isolated calmness of the stationary point mapping along with the calmness of the multiplier set mapping at the corresponding reference point. Sufficient conditions are also provided for the strong calmness by establishing the pseudo-isolated calmness of the stationary point mapping in terms of the noncriticality of the associated multiplier, and the calmness of the multiplier set mapping in terms of a relative interior condition for the multiplier set. These results cover and extend the existing ones in Hager and Gowda [Stability in the presence of degeneracy and error estimation. Math Program. 1999;85:181–192]; Izmailov and Solodov [Stabilized SQP revisited. Math Program. 2012;133:93–120] for nonlinear programming and in Cui et al. [On the asymptotic superlinear convergence of the augmented Lagrangian method for semidefinite programming with multiple solutions. 2016, arXiv: 1610.00875v1]; Zhang and Zhang [Critical multipliers in semidefinite programming. 2018, arXiv: 1801.02218v1] for semidefinite programming.

Spectral Efficiency Optimization For Millimeter Wave Multiuser MIMO Systems

As a key enabling technology for 5G wireless, millimeter wave (mmWave) communication motivates the utilization of large-scale antenna arrays for achieving highly directional beamforming. However, the high cost and power consumption of RF chains stands in the way of adoption of the optimal fullydigital precoding in large-array systems. To reduce the number of RF chains while still maintaining the spatial multiplexing gain of large-array, hybrid precoding architecture has been proposed for mmWave systems and received considerable interest in both industry and academia. However, the optimal hybrid precoding design has not been fully understood, especially for the multi-user MIMO case. This paper is the first work that directly addresses the nonconvex hybrid precoding problem of mmWave multi-user MIMO systems (without any approximation) by using penalty dual decomposition (PDD) method. The proposed PDD method have a guaranteed convergence to KKT solutions of the hybrid precoding problem under a mild assumption. Simulation results show that, even when both the transmitter and the receivers are equipped with the fewest RF chains that are required to support multi-stream transmission, hybrid precoding can still approach the performance of fully-digital precoding in both the infinite resolution phase shifter case and the finite resolution phase shifter case with several bits quantization.

Semi-Decentralized Nash Equilibrium Seeking in Aggregative Games With Separable Coupling Constraints and Non-Differentiable Cost Functions

We study the Nash equilibrium seeking problem for noncooperative agents whose decision making process can be modeled as a generalized aggregative game. Specifically, we consider players with convex local cost functions, convex local constraints, and convex separable coupling constraints, and we extend the literature on generalized aggregative games by handling possibly non-differentiable cost functions. We recast the Nash equilibrium seeking problem as the problem to find a zero of a set-valued monotone operator and show that the variational Nash equilibria correspond to KKT solutions of the original game where the shared constraints have the same Lagrange multipliers for all the players. Finally, we design a semi-decentralized algorithm with global convergence guarantee for generalized Nash equilibrium seeking.

Joint Power and Admission Control: Non-Convex &lt;formula formulatype="inline"&gt;&lt;tex Notation="TeX"&gt;$L_{q}$&lt;/tex&gt;&lt;/formula&gt; Approximation and An Effective Polynomial Time Deflation Approach

In an interference limited network, joint power and admission control (JPAC) aims at supporting a maximum number of links at their specified signal to interference plus noise ratio (SINR) targets while using a minimum total transmission power. Various convex approximation deflation approaches have been developed for the JPAC problem. In this paper, we propose an effective polynomial time non-convex approximation deflation approach for solving the problem. The approach is based on the non-convex $\ell_q$-minimization approximation of an equivalent sparse $\ell_0$-minimization reformulation of the JPAC problem where $q\in(0,1).$ We show that, for any instance of the JPAC problem, there exists a $\bar q\in(0,1)$ such that it can be exactly solved by solving its $\ell_q$-minimization approximation problem with any $q\in(0, \bar q]$. We also show that finding the global solution of the $\ell_q$ approximation problem is NP-hard. Then, we propose a potential reduction interior-point algorithm, which can return an $\epsilon$-KKT solution of the NP-hard $\ell_q$-minimization approximation problem in polynomial time. The returned solution can be used to check the simultaneous supportability of all links in the network and to guide an iterative link removal procedure, resulting in the polynomial time non-convex approximation deflation approach for the JPAC problem. Numerical simulations show that the proposed approach outperforms the existing convex approximation approaches in terms of the number of supported links and the total transmission power, particularly exhibiting a quite good performance in selecting which subset of links to support.

Lipschitz continuity of the optimal value function and KKT solution set in indefinite quadratic programs

When all the involved data in indefinite quadratic programs change simultaneously, we show the locally Lipschtiz continuity of the KKT set of the quadratic programming problem firstly, then we establish the locally Lipschtiz continuity of the KKT solution set. Finally, the similar conclusion for the corresponding optimal value function is obtained.

Outage-Constrained Power Allocation in Spectrum Sharing Systems with Partial CSI

Due to limited cooperation between the primary users and the secondary users (SUs) in practical spectrum sharing systems, the secondary transmitters and receivers are assumed to have partial channel state information related to the primary receiver. Under such an assumption, this work investigates power allocation strategies for the SUs subject to an outage probability constraint on the primary transmission and a peak transmit power constraint on the secondary transmission. The challenge lies in the non-convexity of the outage probability constraint. Firstly, we prove that strong duality holds and that the Karush-Kuhn-Tucker (KKT) conditions are necessary for optimality. The optimal solution is then derived by addressing the optimality issues of the KKT solutions. Secondly, in order to further reduce the algorithmic complexity, two suboptimal strategies are proposed. The first one is designed based on several simplifications of the optimal strategy. The second one is derived from the convex relaxation of the non-convex primal problem, which corresponds to the problem with the conventional interference temperature constraint. The performance for both suboptimal strategies is derived in closed form. All proposed strategies are shown to outperform non-adaptive power transmission. The near-optimality of the two suboptimal strategies is also validated, in particular for the first one.

Second order optimality conditions and reformulations for nonconvex quadratically constrained quadratic programming problems

In this paper, we present an optimality condition which could determine whether a given KKT solution is globally optimal. This condition is equivalent to determining if the Hessian of the corresponding Largrangian is copositive over a set. To find the corresponding Lagrangian multiplier, two linear conic programming problems are constructed and then relaxed for computational purpose. Under the new condition, we proposed a local search based scheme to find a global optimal solution and showed its effectiveness by three examples.

A Continuous Relaxation Method for Multiuser Detection Problem

Based on the binary quadratic programming model of the code division multiple access maximum likelihood multiuser detection problem, a detection strategy by the continuous relaxation method is presented. The proposed method relaxes the binary quadratic programming as a nonlinear programming, which is a quadratic objective function with simple quadratic constraints. A feasible direction method is used to solve the nonlinear programming. Based on the KKT solution of the nonlinear programming, a near optimal solution is obtained for the multiuser detection problem. Simulation results show that the bit error rate performances of a detection strategy based on the continuous relaxation method is low. Furthermore, average CPU time of continuous relaxation method is much lower than that of the semidefinite programming relaxation method, especially for the large-scale detection problems. This approach provides good approximations to the optimal maximum likelihood performance.

KKT Solution and Conic Relaxation for Solving Quadratically Constrained Quadratic Programming Problems

To find a global optimal solution to the quadratically constrained quadratic programming problem, we explore the relationship between its Lagrangian multipliers and related linear conic programming problems. This study leads to a global optimality condition that is more general than the known positive semidefiniteness condition in the literature. Moreover, we propose a computational scheme that provides clues of designing effective algorithms for more solvable quadratically constrained quadratic programming problems.

A NOTE ON KKT-INVEXITY IN NONSMOOTH CONTINUOUS-TIME OPTIMIZATION

We introduce the notion of KKT-invexity for nonsmooth continuoustime nonlinear optimization problems and prove that this notion is a necessary and sufficient condition for every KKT solution to be a global optimal solution.

Multi-objective infinite programming

Using the theory of abstract optimization problems in infinite-dimensional spaces we provide necessary optimality conditions of first and second order for weakly efficient solutions of the multi-objective infinite programming problem. Sufficient conditions are given under invexity assumptions. We generalize the notion of KKT-invexity for the multi-objective infinite problem and show that this notion is a necessary and sufficient condition for every vector KKT solution to be a weakly efficient solution. Moreover, we develop a theorem of the alternative, useful for proving some of our results.