Stochastic Subgradient Algorithm Research Articles

A Stochastic Primal-Dual Method for Optimization with Conditional Value at Risk Constraints

We study a first-order primal-dual subgradient method to optimize risk-constrained risk-penalized optimization problems, where risk is modeled via the popular conditional value at risk (CVaR) measure. The algorithm processes independent and identically distributed samples from the underlying uncertainty in an online fashion, and produces an $\eta/\sqrt{K}$-approximately feasible and $\eta/\sqrt{K}$-approximately optimal point within $K$ iterations with constant step-size, where $\eta$ increases with tunable risk-parameters of CVaR. We find optimized step sizes using our bounds and precisely characterize the computational cost of risk aversion as revealed by the growth in $\eta$. Our proposed algorithm makes a simple modification to a typical primal-dual stochastic subgradient algorithm. With this mild change, our analysis surprisingly obviates the need for a priori bounds or complex adaptive bounding schemes for dual variables assumed in many prior works. We also draw interesting parallels in sample complexity with that for chance-constrained programs derived in the literature with a very different solution architecture.

A stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs

We propose a stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs. Our approach is based on a bi-objective viewpoint of chance-constrained programs that seeks solutions on the efficient frontier of optimal objective value versus risk of constraints violation. To this end, we construct a reformulated problem whose objective is to minimize the probability of constraints violation subject to deterministic convex constraints (which includes a bound on the objective function value). We adapt existing smoothing-based approaches for chance-constrained problems to derive a convergent sequence of smooth approximations of our reformulated problem, and apply a projected stochastic subgradient algorithm to solve it. In contrast with exterior sampling-based approaches (such as sample average approximation) that approximate the original chance-constrained program with one having finite support, our proposal converges to stationary solutions of a smooth approximation of the original problem, thereby avoiding poor local solutions that may be an artefact of a fixed sample. Our proposal also includes a tailored implementation of the smoothing-based approach that chooses key algorithmic parameters based on problem data. Computational results on four test problems from the literature indicate that our proposed approach can efficiently determine good approximations of the efficient frontier.

String-averaging incremental stochastic subgradient algorithms

ABSTRACTWe present a method to solve constrained convex stochastic optimization problems when the objective is a finite sum of convex functions . Our method is based on Incremental Stochastic Subgradient Algorithms and String-Averaging techniques, with an assumption that the subgradient directions are affected by random errors in each iteration. Our analysis allows the method to perform approximate projections onto the feasible set in each iteration. We provide convergence results for the case where a diminishing step-size rule is used. We test our method in a large set of random instances of a stochastic convex programming problem and we compare its performance with the robust mirror descent stochastic approximation algorithm proposed in Nemirovski et al. (Robust stochastic approximation approach to stochastic programming, SIAM J Optim 19 (2009), pp. 15741609).

Optimal distributed stochastic mirror descent for strongly convex optimization

In this paper we consider convergence rate problems for stochastic strongly-convex optimization in the non-Euclidean sense with a constraint set over a time-varying multi-agent network. We propose two efficient non-Euclidean stochastic subgradient descent algorithms based on the Bregman divergence as distance-measuring function rather than the Euclidean distances that were employed by the standard distributed stochastic projected subgradient algorithms. For distributed optimization of non-smooth and strongly convex functions whose only stochastic subgradients are available, the first algorithm recovers the best previous known rate of O(ln(T)∕T) (where T is the total number of iterations). The second algorithm is an epoch variant of the first algorithm that attains the optimal convergence rate of O(1∕T), matching that of the best previously known centralized stochastic subgradient algorithm. Finally, we report some simulation results to illustrate the proposed algorithms.

Fair Resource Allocation with QoS Guarantee in Secure Multiuser TDMA Networks

We investigate a secure multiuser time division multiple access (TDMA) system with statistical delay quality of service (QoS) guarantee in terms of secure effective capacity. An optimal resource allocation policy is proposed to minimize the β‐fair cost function of the average user power under the individual QoS constraint, which also balances the energy efficiency and fairness among the users. First, convex optimization problems associated with the resource allocation policy are formulated. Then, a subgradient iteration algorithm based on the Lagrangian duality theory and the dual decomposition theory is employed to approach the global optimal solutions. Furthermore, considering the practical channel conditions, we develop a stochastic subgradient iteration algorithm which is capable of dynamically learning the intended wireless channels and acquiring the global optimal solution. It is shown that the proposed optimal resource allocation policy depends on the delay QoS requirement and the channel conditions. The optimal policy can save more power and achieve the balance of the energy efficiency and the fairness compared with the other resource allocation policies.

Max-Margin Deep Generative Models for (Semi-)Supervised Learning.

Deep generative models (DGMs) can effectively capture the underlying distributions of complex data by learning multilayered representations and performing inference. However, it is relatively insufficient to boost the discriminative ability of DGMs. This paper presents max-margin deep generative models (mmDGMs) and a class-conditional variant (mmDCGMs), which explore the strongly discriminative principle of max-margin learning to improve the predictive performance of DGMs in both supervised and semi-supervised learning, while retaining the generative capability. In semi-supervised learning, we use the predictions of a max-margin classifier as the missing labels instead of performing full posterior inference for efficiency; we also introduce additional max-margin and label-balance regularization terms of unlabeled data for effectiveness. We develop an efficient doubly stochastic subgradient algorithm for the piecewise linear objectives in different settings. Empirical results on various datasets demonstrate that: (1) max-margin learning can significantly improve the prediction performance of DGMs and meanwhile retain the generative ability; (2) in supervised learning, mmDGMs are competitive to the best fully discriminative networks when employing convolutional neural networks as the generative and recognition models; and (3) in semi-supervised learning, mmDCGMs can perform efficient inference and achieve state-of-the-art classification results on several benchmarks.

Stochastic Subgradient Algorithms for Strongly Convex Optimization Over Distributed Networks

We study diffusion and consensus based optimization of a sum of unknown convex objective functions over distributed networks. The only access to these functions is through stochastic gradient oracles, each of which is only available at a different node; and a limited number of gradient oracle calls is allowed at each node. In this framework, we introduce a convex optimization algorithm based on stochastic subgradient descent (SSD) updates. We use a carefully designed time-dependent weighted averaging of the SSD iterates, which yields a convergence rate of $O\left(\frac{N\sqrt{N}}{(1-\sigma)T}\right)$ after $T$ gradient updates for each node on a network of $N$ nodes, where $0\leq \sigma denotes the second largest singular value of the communication matrix. This rate of convergence matches the performance lower bound up to constant terms. Similar to the SSD algorithm, the computational complexity of the proposed algorithm also scales linearly with the dimensionality of the data. Furthermore, the communication load of the proposed method is the same as the communication load of the SSD algorithm. Thus, the proposed algorithm is highly efficient in terms of complexity and communication load. We illustrate the merits of the algorithm with respect to the state-of-art methods over benchmark real life data sets.

Nonsmooth analysis and subgradient methods for averaging in dynamic time warping spaces

Time series averaging in dynamic time warping (DTW) spaces has been successfully applied to improve pattern recognition systems. This article proposes and analyzes subgradient methods for the problem of finding a sample mean in DTW spaces. The class of subgradient methods generalizes existing sample mean algorithms such as DTW Barycenter Averaging (DBA). We show that DBA is a majorize-minimize algorithm that converges to necessary conditions of optimality after finitely many iterations. Empirical results show that for increasing sample sizes the proposed stochastic subgradient (SSG) algorithm is more stable and finds better solutions in shorter time than the DBA algorithm on average. Therefore, SSG is useful in online settings and for non-small sample sizes. The theoretical and empirical results open new paths for devising sample mean algorithms: nonsmooth optimization methods and modified variants of pairwise averaging methods.

Distributed Constrained Stochastic Subgradient Algorithms Based on Random Projection and Asynchronous Broadcast over Networks

We consider a distributed constrained optimization problem over a time-varying network, where each agent only knows its own cost functions and its constraint set. However, the local constraint set may not be known in advance or consists of huge number of components in some applications. To deal with such cases, we propose a distributed stochastic subgradient algorithm over time-varying networks, where the estimate of each agent projects onto its constraint set by using random projection technique and the implement of information exchange between agents by employing asynchronous broadcast communication protocol. We show that our proposed algorithm is convergent with probability 1 by choosing suitable learning rate. For constant learning rate, we obtain an error bound, which is defined as the expected distance between the estimates of agent and the optimal solution. We also establish an asymptotic upper bound between the global objective function value at the average of the estimates and the optimal value.

Stochastic sub-gradient algorithm for distributed optimization with random sleep scheme

In this paper, we consider a distributed convex optimization problem of a multi-agent system with the global objective function as the sum of agents’ individual objective functions. To solve such an optimization problem, we propose a distributed stochastic sub-gradient algorithm with random sleep scheme. In the random sleep scheme, each agent independently and randomly decides whether to inquire the sub-gradient information of its local objective function at each iteration. The algorithm not only generalizes distributed algorithms with variable working nodes and multi-step consensus-based algorithms, but also extends some existing randomized convex set intersection results. We investigate the algorithm convergence properties under two types of stepsizes: the randomized diminishing stepsize that is heterogeneous and calculated by individual agent, and the fixed stepsize that is homogeneous. Then we prove that the estimates of the agents reach consensus almost surely and in mean, and the consensus point is the optimal solution with probability 1, both under randomized stepsize. Moreover, we analyze the algorithm error bound under fixed homogeneous stepsize, and also show how the errors depend on the fixed stepsize and update rates.

Exploiting Base Station Caching in MIMO Cellular Networks: Opportunistic Cooperation for Video Streaming

We propose a novel MIMO cooperation framework called the cache-induced opportunistic cooperative MIMO (Coop-MIMO) for video streaming in backhaul limited multicell MIMO networks. By caching a portion of the video files, the base stations (BSs) opportunistically employ Coop-MIMO transmission to achieve MIMO cooperation gain without expensive payload backhaul. We derive closed form expressions of various video streaming performance metrics for cache-induced opportunistic Coop-MIMO and investigate the impact of BS level caching and key system parameters on the performance. Specifically, we first obtain a mixed fluid-diffusion limit for the playback buffer queueing system. Then we derive the approximated video streaming performance using the mixed fluid-diffusion limit. Based on the performance analysis, we formulate the joint optimization of cache control and playback buffer management as a stochastic optimization problem. Then we derive a closed form solution for the playback buffer thresholds and develop a stochastic subgradient algorithm to find the optimal cache control. The analysis shows that the video streaming performance improves linearly with the BS cache size BC, the transmit power cost decreases exponentially with BC, and the backhaul cost decreases linearly with BC.

Distributed Optimal Rate–Reliability–Lifetime Tradeoff in Time-Varying Wireless Sensor Networks

The transmission rate, delivery reliability, and network lifetime are three fundamental but conflicting design objectives in energy-constrained wireless sensor networks (WSNs). In this paper, based on stochastic network utility maximization framework, we address the optimal rate-reliability-lifetime tradeoff with time-varying channel capacity constraint, reliability constraint, and energy constraint. By introducing the weight parameters, we combine the optimization objectives of rate, reliability, and lifetime into a single objective to characterize the tradeoff among them. However, the optimization formulation of the rate-reliability-reliability tradeoff is neither separable nor convex. Through a series of transformations, a separable problem is derived, and an efficient distributed stochastic subgradient algorithm is proposed via dual decomposition and stochastic subgradient techniques. It is proved that the proposed algorithm can converge to the global optimum with probability one. Numerical examples confirm its convergence. In addition, numerical examples investigate the impact of weight parameters on the rate utility, reliability utility, and network lifetime, which provide guidance to properly set the value of weight parameters for a desired performance of WSNs according to the realistic application's requirements.

Cache-Enabled Opportunistic Cooperative MIMO for Video Streaming in Wireless Systems

We propose a cache-enabled opportunistic cooperative MIMO (CoMP) framework for wireless video streaming. By caching a portion of the video files at the relays (RS) using a novel MDS-coded random cache scheme, the base station (BS) and RSs opportunistically employ CoMP to achieve spatial multiplexing gain without expensive payload backhaul. We study a two timescale joint optimization of power and cache control to support real-time video streaming. The cache control is to create more CoMP opportunities and is adaptive to the long-term popularity of the video files. The power control is to guarantee the QoS requirements and is adaptive to the channel state information (CSI), the cache state at the RS and the queue state information (QSI) at the users. The joint problem is decomposed into an inner power control problem and an outer cache control problem. We first derive a closed-form power control policy from an approximated Bellman equation. Based on this, we transform the outer problem into a convex stochastic optimization problem and propose a stochastic subgradient algorithm to solve it. Finally, the proposed solution is shown to be asymptotically optimal for high SNR and small timeslot duration. Its superior performance over various baselines is verified by simulations.

Consensus-based distributed optimisation of multi-agent networks via a two level subgradient-proximal algorithm

This paper presents a consensus-based stochastic subgradient algorithm for multi-agent networks to minimise multiple convex but not necessarily differential objective functions, subject to an intersection set of multiple closed convex constraint sets. Compared with the existing results an alternative subgradient algorithm is first introduced based on two level subgradient iterations, where the first level is to minimise the component functions, and the second to enforce the iterates not oscillate from the constraint set wildly. In addition, a distributed consensus-based type of the proposed subgradient algorithm is constructed within the framework of multi-agent networks for the case when the iteration index of local objective functions and local constraint sets is not homologous. Detailed convergence analysis of the proposed algorithms is established using matrix theories and super-martingale convergence theorem. In addition, a pre-step convergence factor is obtained in this study to characterise the distance between the iterations and the optimal set, while some existing literatures only present a convergence work. Simulation results are given to demonstrate the effectiveness of the developed theoretical results.

Cross-layer and end-to-end optimization for the integrated wireless and wireline network

In this paper, we study a cross-layer and end-to-end optimization problem for the integrated wireless and wireline network that consists of one wireline core network and multiple wireless access networks. We consider joint end-to-end flow control/distribution at the transport and network layers and opportunistic scheduling at the data link and physical layers. We formulate a single stochastic optimization problem and solve it by using a dual approach and a stochastic sub-gradient algorithm. The developed algorithm can be implemented in a distributed way, vertically among communication layers and horizontally among all entities in the network, clearly showing what should be done at each layer and each entity and what parameters should be exchanged between layers and between entities. Numerical results show that our cross-layer and end-to-end optimization approach provides more efficient resource allocation than the conventional layered and separated optimization approach.

Optimal Wireless Networks Based on Local Channel State Information

This paper considers distributed algorithms to optimize random access multihop wireless networks in the presence of fading. Since the associated optimization problem is neither convex nor amenable to distributed implementation, a problem approximation is introduced. This approximation is still not convex but it has zero duality gap and can be solved and decomposed into local subproblems in the dual domain. The solution method is through a stochastic subgradient descent algorithm that operates without knowledge of the fading's probability distribution and leads to an architecture composed of layers and layer interfaces. With limited amount of message passing among terminals and small computational cost, the proposed algorithm converges almost surely in an ergodic sense. Numerical results on a randomly generated network corroborate theoretical results.

Distributed primal–dual stochastic subgradient algorithms for multi‐agent optimization under inequality constraints

SUMMARYWe consider the multi‐agent optimization problem where multiple agents try to cooperatively optimize the sum of their local convex objective functions, subject to global inequality constraints and a convex constraint set over a network. Through characterizing the primal and dual optimal solutions as the saddle points of the associated Lagrangian function, which can be evaluated with stochastic errors, we propose the distributed primal–dual stochastic subgradient algorithms for two cases: (i) the time model is synchronous and (ii) the time model is asynchronous. In the first case, we obtain bounds on the convergence properties of the algorithm for a diminishing step size. In the second case, for a constant step size, we establish some error bounds on the algorithm's performance. In particular, we prove that the error bounds scale as in the number of n agents. Copyright © 2012 John Wiley & Sons, Ltd.

A method of stochastic subgradients with complete feedback stepsize rule for convex stochastic approximation problems

A stochastic subgradient algorithm for solving convex stochastic approximation problems is considered. In the algorithm, the stepsize coefficients are controlled on-line on the basis of information gathered in the course of computations according to a new, complete feedback rule derived from the concept of regularized improvement function. Convergence with probability 1 of the method is established.

Optimization of the Capacities of an Electrical Transmission Network

This paper is devoted to the optimization of the transmission lines of an electrical network. We have to determine the best capacities in order to minimize the sum of the investment cost and of the expectation of the optimal generation cost. The system is perturbated by some uncertainties, in particular the unavailability of the generation units. The main difficulty is the large size of the system. Thus, the main purpose of this work is to develop a tracking method on a practical point of view i.e. to find a fast algorithm which gives an optimal or good suboptimal solution. A method used by E.D.F. (Electricité de France) is based on linear programming algorithms for a formulation of the problem in which the expectation is approximated by the average on a bundle of trajectories. This method is too much computer time consuming. The method that we propose here is a stochastic subgradient algorithm applied to a French aggregated network. In fact, the capacities of the lines can only take integer values. In this situation an adaptation of the stochastic gradient method leads to a fast computation of good upper estimates of the optimal cost.