Projected Subgradient Method Research Articles

Attack-Resilient Distributed Nash Equilibrium Seeking of Uncertain Multiagent Systems Over Unreliable Communication Networks.

This article investigates the distributed Nash equilibrium (NE) seeking problem of uncertain multiagent systems in unreliable communication networks. In this problem, the action of each agent is subject to a class of nonlinear systems with uncertain dynamics, and the communication network among agents will be affected by the nonperiodic denial of service (DoS) attacks. Note that, in this insecure network environment, the existence of DoS attacks will directly destroy the connectivity of the network, which leads to performance degradation or even failure of the most existing distributed NE seeking algorithms. To address this problem, we propose a two-stage distributed NE seeking strategy, including the attack-resilient distributed NE estimator and the neuroadaptive tracking controller. The estimator based on the projection subgradient method and the consensus protocol can converge exponentially to virtual NE against DoS attacks. Then, the neuroadaptive tracking controller is designed for uncertain multiagent systems with the output of the estimator as the reference signal such that the actual action of all agents can reach NE. Based on the Lyapunov stability theory and improved average dwell time automaton, the stability of the estimator and the controller is proven, and all signals in the closed-loop system are uniformly bounded. Numerical examples are presented to verify the effectiveness of the proposed strategy.

Superiorized Adaptive Projected Subgradient Method With Application to MIMO Detection

In this paper, we show that the adaptive projected subgradient method (APSM) is bounded perturbation resilient. To illustrate a potential application of this result, we propose a set-theoretic framework for MIMO detection, and we devise algorithms based on a superiorized APSM. Various low-complexity MIMO detection algorithms achieve excellent performance on i.i.d. Gaussian channels, but they typically incur high performance loss if realistic channel models (e.g., correlated channels) are considered. Compared to existing low-complexity iterative detectors such as individually optimal large-MIMO approximate message passing (IO-LAMA), the proposed algorithms can achieve considerably lower symbol error ratios over correlated channels. At the same time, the proposed methods do not require matrix inverses, and their complexity is similar to IO-LAMA.

A Projected Subgradient Method for the Computation of Adapted Metrics for Dynamical Systems

In this paper, we extend a recently established subgradient method for the computation of Riemannian metrics that optimizes certain singular value functions associated with dynamical systems. This extension is threefold. First, we introduce a projected subgradient method which results in Riemannian metrics whose parameters are confined to a compact convex set and we can thus prove that a minimizer exists; second, we allow inexact subgradients and study the effect of the errors on the computed metrics; and third, we analyze the subgradient algorithm for three different choices of step sizes: constant, exogenous, and Polyak. The new methods are illustrated by application to dimension and entropy estimation of the Hénon map.

A Projected Subgradient Method for Nondifferentiable Quasiconvex Multiobjective Optimization Problems

A Projected Subgradient Method for Nondifferentiable Quasiconvex Multiobjective Optimization Problems

One-Bit Compressive Sensing With Projected Subgradient Method Under Sparsity Constraints

One-bit compressive sensing theory shows that the sparse signals can be almost exactly reconstructed from a small number of one-bit quantized linear measurements. This paper presents the convergence analysis of the binary iterative hard thresholding (BIHT) algorithm which is a state-of-the-art recovery algorithm in one-bit compressive sensing. The basic idea of the convergence analysis is to view BIHT as a kind of projected subgradient method under sparsity constrains. To the best of our knowledge, this is the first convergence analysis of BIHT. We first consider a general convex function subject to sparsity constraints and connect it with the non-convex model in one-bit compressive sensing literatures. A projected subgradient method is proposed to solve the general model and some convergence results are established. A stronger convergence theorem for $\alpha $ –strongly convex functions without assumption on differentiable condition is also established. Furthermore, the corresponding stochastic projected subgradient method is provided with convergence guarantee. In our settings, BIHT is a special case of the projected subgradient method. Therefore, the convergence analysis can be applied to BIHT naturally. Then, we apply the projected subgradient method to some related non-convex optimization models arising in compressive sensing with $\ell _{1}$ -constraint, sparse support vector machines, and rectifier linear units regression. Finally, some numerical examples are presented to show the validity of our convergence analysis. The numerical experiments also show that the proposed projected subgradient method is very simple to implement, robust to sparse noise, and effective for sparse recovery problems.

Solving linear generalized Nash equilibrium problems numerically

This paper considers the numerical solution of linear generalized Nash equilibrium problems (LGNEPs). Since many methods for nonlinear problems require the nonsingularity of some second-order derivative, standard convergence conditions are not satisfied in our linear case. We provide new convergence criteria for a potential reduction algorithm (PRA) that allow its application to LGNEPs. Furthermore, we discuss a projected subgradient method (PSM) and a penalty method that exploit some known Nikaido–Isoda function-based constrained and unconstrained optimization reformulations of the LGNEP. Moreover, it is shown that normalized Nash equilibria of an LGNEP can be obtained by solving a single linear program. All proposed algorithms are tested on randomly generated instances of economic market models that are introduced and analysed in this paper and that lead to LGNEPs with shared and with non-shared constraints. It is shown that these problems have some favourable properties that can be exploited to obtain their solutions. With the PRA and in particular with the PSM we are able to compute solutions with satisfying precision even for problems with up 10,000 variables.

Robust Subspace Tracking With Missing Entries: The Set-Theoretic Approach

In this paper, an Adaptive Projected Subgradient Method (APSM) based algorithm for robust subspace tracking is introduced. A properly chosen cost function is constructed at each time instance and the goal is to seek for points, which belong to the zero level set of this function; i.e., the set of points which score a zero loss. At each iteration, an outlier detection mechanism is employed, in order to conclude whether the current data vector contains outlier noise or not. In the sequel, a sparsity-promoting greedy algorithm is employed for the outlier vector estimation allowing the purification of the corrupted data from the outlier noise, prior to any further processing. Furthermore, the case where the observation vectors are partially observed is attacked via a prediction procedure, which estimates the values of the unobserved (missing) coefficients. A theoretical analysis is carried out and the simulation experiments, within the contexts of robust subspace estimation and robust matrix completion, demonstrate the enhanced performance of the proposed scheme compared to recently developed state of the art algorithms.

Optimized Distributed Inter-Cell Interference Coordination (ICIC) Scheme Using Projected Subgradient and Network Flow Optimization

In this paper, we tackle the problem of multi-cell resource scheduling, where the objective is to maximize the weighted sum-rate through inter-cell interference coordination (ICIC). The blanking method is used to mitigate the inter-cell interference, where a resource is either used with a predetermined transmit power or not used at all, i.e., blanked. This problem is known to be strongly NP-hard, which means that it is not only hard to solve in polynomial time, but it is also hard to find an approximation algorithm with guaranteed optimality gap. In this work, we identify special scenarios where a polynomial-time algorithm can be constructed to solve this problem with theoretical guarantees. In particular, we define a dominant interference environment, in which for each user the received power from each interferer is significantly greater than the aggregate received power from all other weaker interferers. We show that the originally strongly NP-hard problem can be tightly relaxed to a linear programming problem in a dominant interference environment. Consequently, we propose a polynomial time distributed algorithm that is not only guaranteed to be tight in a dominant interference environment, but which also computes an upper bound on the optimality gap without additional computational complexity. The proposed scheme is based on the primal-decomposition method, where the problem is divided into a master-problem and multiple subproblems. We solve the master-problem iteratively using the projected-subgradient method. We also show that each subproblem has a special network flow structure. By exploiting this network structure, each subproblem is solved using the network-based optimization methods, which significantly reduces the complexity in comparison to the general-purpose convex or linear optimization methods. In comparison with baseline schemes, simulation results of the International Mobile Telecommunications-Advanced (IMT-Advanced) scenarios show that the proposed scheme achieves higher gains in aggregate throughput, cell-edge throughput, and outage probability.

Stochastic Analysis of Hyperslab-Based Adaptive Projected Subgradient Method Under Bounded Noise

This letter establishes a novel analysis of the Adaptive Projected Subgradient Method (APSM) in the intersection of the stochastic and robust estimation paradigms. Utilizing classical worst-case bounds on the noise process, drawn from the robust estimation methodology, the present study demonstrates that the hyperslab-inspired version of the APSM generates a sequence of estimates which converges to a point located, with probability one, arbitrarily close to the estimand. Numerical tests and comparisons with classical time-adaptive algorithms corroborate the theoretical findings of the study.

The Adaptive Projected Subgradient Method Constrained by Families of Quasi-nonexpansive Mappings and Its Application to Online Learning

Many online, i.e., time-adaptive, inverse problems in signal processing and machine learning fall under the wide umbrella of the asymptotic minimization of a sequence of nonnegative, convex, and continuous functions. To incorporate a priori knowledge into the design, the asymptotic minimization task is usually constrained on a fixed closed convex set, which is dictated by the available a priori information. To increase versatility toward the usage of the available information, the present manuscript extends the adaptive projected subgradient method by introducing an algorithmic scheme which incorporates a priori knowledge in the design via a sequence of strongly attracting quasi-nonexpansive mappings in a real Hilbert space. In such a way, the benefits offered to online learning tasks by the proposed method unfold in two ways: (1) the rich class of quasi-nonexpansive mappings provides a plethora of ways to cast a priori knowledge, and (2) by introducing a sequence of such mappings, the proposed scheme is able to capture the time-varying nature of a priori information. The convergence properties of the algorithm are studied, several special cases of the method with wide applicability are shown, and the potential of the proposed scheme is demonstrated by considering an increasingly important online sparse system/signal recovery task.

A projection subgradient method for solving optimization with variational inequality constraints

In this paper, we propose a projection subgradient method for solving some classical variational inequality problem over the set of solutions of mixed variational inequalities. Under the conditions that \(T\) is a \(\Theta \)-pseudomonotone mapping and \(A\) is a \(\rho \)-strongly pseudomonotone mapping, we prove the convergence of the algorithm constructed by projection subgradient method. Our algorithm can be applied for instance to some mathematical programs with complementarity constraints.

The Projected Subgradient Method for Nonsmooth Convex Optimization in the Presence of Computational Errors

We study the convergence of the projected subgradient method for constrained convex optimization in a Hilbert space. Our goal is to obtain an ε-approximate solution of the problem in the presence of computational errors, where ε is a given positive number. The results that we obtain are important in practice because computations always introduce numerical errors.

Multi-Domain Adaptive Learning Based on Feasibility Splitting and Adaptive Projected Subgradient Method

We propose the multi-domain adaptive learning that enables us to find a point meeting possibly time-varying specifications simultaneously in multiple domains, e.g. space, time, frequency, etc. The novel concept is based on the idea of feasibility splitting—dealing with feasibility in each individual domain. We show that the adaptive projected subgradient method (Yamada, 2003) realizes the multi-domain adaptive learning by employing (i) a projected gradient operator with respect to a 'fixed' proximity function reflecting the time-invariant specifications and (ii) a subgradient projection with respect to 'time-varying' objective functions reflecting the time-varying specifications. The resulting algorithm is suitable for real-time implementation, because it requires no more than metric projections onto closed convex sets each of which accommodates the specification in each domain. A convergence analysis and numerical examples are presented.

Acoustic feedback cancellation based on weighted adaptive projection subgradient method in hearing aids

Acoustic feedback is an important factor that degrades the overall performance of hearing aids, and acoustic feedback cancellation has always been the research focus in the field of signal processing in hearing aids. The newly suggested adaptive projection subgradient method (APSM) for adaptive signal processing solves the problem of difficulty in finding the exact projection operator in the realization of affine projection by taking the subgradient projection hyperplane as the searching region for relaxed projection. This work applies APSM in the acoustic feedback cancellation system of hearing aids for the first time, and proposes a weighted adaptive projection subgradient method (WAPSM), which takes into consideration the exponential decay weight factor to incorporate the prior information of estimation system. The new method is compared with the traditional NLMS algorithm and APSM algorithm in simulation experiments. Incorporating the prior information of estimation system by setting the proper weighting matrix, WAPSM achieved notable improvements on the speed, stability and accuracy of the misalignment convergence. Numerical experiments demonstrate that the proposed algorithm is more robust for low SNR and real speech segment input than the traditional algorithms.

A Flexible Peak-To-Average Power Ratio Reduction Scheme for OFDM Systems by the Adaptive Projected Subgradient Method

One of the main issues of the orthogonal frequency-division multiplexing (OFDM) modulation is the high peak-to-average power ratio (PAPR) of the transmitted signal, which adversely affects the complexity of power amplifiers. In this paper, we consider transmitters that reduce the PAPR by slightly disturbing the symbols in carriers used to transmit information and by sending dummy symbols-i.e., symbols not conveying information-in unused carriers. The optimal choice of the data and dummy symbols is determined by the solution of a convex optimization problem. To reduce the PAPR with low complexity, we apply a modified version of the adaptive projected subgradient method to a sequence of convex cost functions closely related to the original optimization problem. The resulting algorithm achieves near-optimal PAPR in practical scenarios, generalizes existing algorithms based on Polyak's method, and can easily handle multiple constraints.

Strong Convergence of Projected Subgradient Methods for Nonsmooth and Nonstrictly Convex Minimization

In this paper, we establish a strong convergence theorem regarding a regularized variant of the projected subgradient method for nonsmooth, nonstrictly convex minimization in real Hilbert spaces. Only one projection step is needed per iteration and the involved stepsizes are controlled so that the algorithm is of practical interest. To this aim, we develop new techniques of analysis which can be adapted to many other non-Fejerian methods.

Multiaccess Interference Suppression in Orthogonal Space–Time Block Coded MIMO Systems by Adaptive Projected Subgradient Method

This paper introduces adaptive filters that are effective to suppress multiple access interference (MAI) in orthogonal space-time block coded/ multiple-input multiple-output (OSTBC-MIMO) systems. We define an optimal linear filter that minimizes the mean-square error between the filter output and a scaled version of the desired output under a constraint defined by the available channel state information (CSI). The adaptive filters refine a given estimate of the optimal filter by suppressing a sequence of closed convex functions with the adaptive projected subgradient method (APSM) at each iteration. To provide robustness against imperfect CSI, the adaptive filters use not only the available CSI but also estimates of previously transmitted symbols, which usually belong to a small finite set in digital communication systems. The resulting algorithms employ computationally efficient projections onto hyperplanes or hyperslabs and do not require any matrix inversion. An efficient recursive scheme based on such an algorithm is also presented. Convergence analysis and simulation results show the excellent performance of the proposed schemes.

The Adaptive Projected Subgradient Method over the Fixed Point Set of Strongly Attracting Nonexpansive Mappings

This paper presents an algorithmic solution, the adaptive projected subgradient method, to the problem of asymptotically minimizing a certain sequence of non-negative continuous convex functions over the fixed point set of a strongly attracting nonexpansive mapping in a real Hilbert space. The method generalizes Polyak's subgradient algorithm for the convexly constrained minimization of a fixed nonsmooth function. By generating a strongly convergent and asymptotically optimal point sequence, the proposed method not only offers unifying principles for many projection-based adaptive filtering algorithms but also enhances the adaptive filtering methods with the set theoretic estimation's armory by allowing a variety of a priori information on the estimandum in the form, for example, of multiple intersecting closed convex sets.

Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions

This paper presents an algorithm, named adaptive projected subgradient method that can minimize asymptotically a certain sequence of nonnegative convex functions over a closed convex set in a real Hilbert space. The proposed algorithm is a natural extension of the Polyak's subgradient algorithm, for nonsmooth convex optimization problem with a fixed target value, to the case where the convex objective itself keeps changing in the whole process. The main theorem, showing the strong convergence of the algorithm as well as the asymptotic optimality of the sequence generated by the algorithm, can serve as a unified guiding principle of a wide range of set theoretic adaptive filtering schemes for nonstationary random processes. These include not only the existing adaptive filtering techniques; e.g., NLMS, Projected NLMS, Constrained NLMS, APA, and Adaptive parallel outer projection algorithm etc., but also new techniques; e.g., Adaptive parallel min-max projection algorithm, and their embedded constraint versions. Numerical examples show that the proposed techniques are well-suited for robust adaptive signal processing problems.

An efficient algorithm for globally minimizing a quadratic function under convex quadratic constraints

In this paper we investigate two approaches to minimizing a quadratic form subject to the intersection of finitely many ellipsoids. The first approach is the d.c. (difference of convex functions) optimization algorithm (abbr. DCA) whose main tools are the proximal point algorithm and/or the projection subgradient method in convex minimization. The second is a branch-and-bound scheme using Lagrangian duality for bounding and ellipsoidal bisection in branching. The DCA was first introduced by Pham Dinh in 1986 for a general d.c. program and later developed by our various work is a local method but, from a good starting point, it provides often a global solution. This motivates us to combine the DCA and our branch and bound algorithm in order to obtain a good initial point for the DCA and to prove the globality of the DCA. In both approaches we attempt to use the ellipsoidal constrained quadratic programs as the main subproblems. The idea is based upon the fact that these programs can be efficiently solved by some available (polynomial and nonpolynomial time) algorithms, among them the DCA with restarting procedure recently proposed by Pham Dinh and Le Thi has been shown to be the most robust and fast for large-scale problems. Several numerical experiments with dimension up to 200 are given which show the effectiveness and the robustness of the DCA and the combined DCA-branch-and-bound algorithm.