Non-convex Stochastic Optimization Research Articles

Online Stochastic DCA With Applications to Principal Component Analysis.

Stochastic algorithms are well-known for their performance in the era of big data. In this article, we study nonsmooth stochastic Difference-of-Convex functions (DC) programs-the major class of nonconvex stochastic optimization, which have a variety of applications in divers domains, in particular, machine learning. We propose new online stochastic algorithms based on the state-of-the-art DC Algorithm (DCA)-a powerful approach in nonconvex programming framework, in the online context of streaming data continuously generated by some (unknown) source distribution. The new schemes use the stochastic approximations (SAs) principle: deterministic quantities of the standard DCA are replaced by their noisy estimators constructed using newly arriving samples. The convergence analysis of the proposed algorithms is studied intensively with the help of tools from modern convex analysis and martingale theory. Finally, we study several aspects of the proposed algorithms on an important problem in machine learning: the expected problem in principal component analysis (PCA).

Trust Region Methods for Nonconvex Stochastic Optimization beyond Lipschitz Smoothness

In many important machine learning applications, the standard assumption of having a globally Lipschitz continuous gradient may fail to hold. This paper delves into a more general (L0, L1)-smoothness setting, which gains particular significance within the realms of deep neural networks and distributionally robust optimization (DRO). We demonstrate the significant advantage of trust region methods for stochastic nonconvex optimization under such generalized smoothness assumption. We show that first-order trust region methods can recover the normalized and clipped stochastic gradient as special cases and then provide a unified analysis to show their convergence to first-order stationary conditions. Motivated by the important application of DRO, we propose a generalized high-order smoothness condition, under which second-order trust region methods can achieve a complexity of O(epsilon(-3.5)) for convergence to second-order stationary points. By incorporating variance reduction, the second-order trust region method obtains an even better complexity of O(epsilon(-3)), matching the optimal bound for standard smooth optimization. To our best knowledge, this is the first work to show convergence beyond the first-order stationary condition for generalized smooth optimization. Preliminary experiments show that our proposed algorithms perform favorably compared with existing methods.

Joint resource allocation and beamforming design for RIS-aided symbiotic radio networks: A DRL approach

In this paper, we investigate a Reconfigurable Intelligent Surface (RIS)-assisted secure Symbiosis Radio (SR) network to address the information leakage of the primary transmitter (PTx) to potential eavesdroppers. Specifically, the RIS serves as a secondary transmitter in the SR network to ensure the security of the communication between the PTx and the Primary Receiver (PRx), and simultaneously transmits its information to the PTx concurrently by configuring the phase shifts. Considering the presence of multiple eavesdroppers and uncertain channels in practical scenarios, we jointly optimize the active beamforming of PTx and the phase shifts of RIS to maximize the secrecy energy efficiency of RIS-supported SR networks while satisfying the quality of service requirement and the secure communication rate. To solve this complicated non-convex stochastic optimization problem, we propose a secure beamforming method based on Proximal Policy Optimization (PPO), which is an efficient deep reinforcement learning algorithm, to find the optimal beamforming strategy against eavesdroppers. Simulation results show that the proposed PPO-based method is able to achieve fast convergence and realize the secrecy energy efficiency gain by up to 22% when compared to the considered benchmarks.

Distributed Adaptive Gradient Algorithm with Gradient Tracking for Stochastic Non-Convex Optimization

Distributed Adaptive Gradient Algorithm with Gradient Tracking for Stochastic Non-Convex Optimization

Decentralized Asynchronous Nonconvex Stochastic Optimization on Directed Graphs

We consider a decentralized stochastic optimization problem over a network of agents, modeled as a directed graph: Agents aim to asynchronously minimize the average of their individual losses (possibly non-convex), each one having access only to a noisy estimate of the gradient of its own function. We propose an asynchronous distributed algorithm for such a class of problems. The algorithm combines stochastic gradients with tracking in an asynchronous push-sum framework and obtains a sublinear convergence rate, matching the rate of the centralized stochastic gradient descent applied to the nonconvex minimization. Our experiments on a non-convex image classification task using convolutional neural network validate the convergence of our proposed algorithm across different number of nodes and graph connectivity percentages.

ϵ-Approximation of Adaptive Leaning Rate Optimization Algorithms for Constrained Nonconvex Stochastic Optimization.

This brief considers constrained nonconvex stochastic finite-sum and online optimization in deep neural networks. Adaptive-learning-rate optimization algorithms (ALROAs), such as Adam, AMSGrad, and their variants, have widely been used for these optimizations because they are powerful and useful in theory and practice. Here, it is shown that the ALROAs are ϵ -approximations for these optimizations. We provide the learning rates, mini-batch sizes, number of iterations, and stochastic gradient complexity with which to achieve ϵ -approximations of the algorithms.

Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation function

Abstract We consider nonconvex stochastic optimization problems where the objective functions have super-linearly growing and discontinuous stochastic gradients. In such a setting, we provide a nonasymptotic analysis for the tamed unadjusted stochastic Langevin algorithm (TUSLA) introduced in Lovas et al. (2020). In particular, we establish nonasymptotic error bounds for the TUSLA algorithm in Wasserstein-1 and Wasserstein-2 distances. The latter result enables us to further derive nonasymptotic estimates for the expected excess risk. To illustrate the applicability of the main results, we consider an example from transfer learning with ReLU neural networks, which represents a key paradigm in machine learning. Numerical experiments are presented for the aforementioned example, which support our theoretical findings. Hence, in this setting, we demonstrate both theoretically and numerically that the TUSLA algorithm can solve the optimization problem involving neural networks with ReLU activation function. Besides, we provide simulation results for synthetic examples where popular algorithms, e.g., ADAM, AMSGrad, RMSProp and (vanilla) stochastic gradient descent algorithm, may fail to find the minimizer of the objective functions due to the super-linear growth and the discontinuity of the corresponding stochastic gradient, while the TUSLA algorithm converges rapidly to the optimal solution. Moreover, we provide an empirical comparison of the performance of TUSLA with popular stochastic optimizers on real-world datasets, as well as investigate the effect of the key hyperparameters of TUSLA on its performance.

A stochastic gradient method for a class of nonlinear PDE-constrained optimal control problems under uncertainty

The study of optimal control problems under uncertainty plays an important role in scientific numerical simulations. This class of optimization problems is strongly utilized in engineering, biology and finance. In this paper, a stochastic gradient method is proposed for the numerical resolution of a nonconvex stochastic optimization problem on a Hilbert space. We show that, under suitable assumptions, strong or weak accumulation points of the iterates produced by the method converge almost surely to stationary points of the original optimization problem. Measurability and convergence rates of a stationarity measure are handled, filling a gap for applications to nonconvex infinite dimensional stochastic optimization problems. The method is demonstrated on an optimal control problem constrained by a class of elliptic semilinear partial differential equations (PDEs) under uncertainty.

Nonasymptotic Bounds for Stochastic Optimization With Biased Noisy Gradient Oracles

We introduce biased gradient oracles to capture a setting where the function measurements have an estimation error that can be controlled through a batch size parameter. Our proposed oracles are appealing in several practical contexts, for instance, risk measure estimation from a batch of independent and identically distributed samples, or simulation optimization, where the function measurements are “biased” due to computational constraints. In either case, increasing the batch size reduces the estimation error. We highlight the applicability of our biased gradient oracles in a risk-sensitive reinforcement learning setting. In the stochastic nonconvex optimization context, we analyze a variant of the randomized stochastic gradient algorithm with a biased gradient oracle. We quantify the convergence rate of this algorithm by deriving nonasymptotic bounds on its performance. Next, in the stochastic convex optimization setting, we derive nonasymptotic bounds for the last iterate of a stochastic gradient descent algorithm with a biased gradient oracle.

A new approach to training neural networks using natural gradient descent with momentum based on Dirichlet distributions

In this paper, we propose a natural gradient descent algorithm with momentum based on Dirichlet distributions to speed up the training of neural networks. This approach takes into account not only the direction of the gradients, but also the convexity of the minimized function, which significantly accelerates the process of searching for the extremes. Calculations of natural gradients based on Dirichlet distributions are presented, with the proposed approach introduced into an error backpropagation scheme. The results of image recognition and time series forecasting during the experiments show that the proposed approach gives higher accuracy and does not require a large number of iterations to minimize loss functions compared to the methods of stochastic gradient descent, adaptive moment estimation and adaptive parameter-wise diagonal quasi-Newton method for nonconvex stochastic optimization.

Last-iterate convergence analysis of stochastic momentum methods for neural networks

The stochastic momentum method is a commonly used acceleration technique for solving large-scale stochastic optimization problems. Current convergence results of stochastic momentum methods under non-convex stochastic settings mostly discuss convergence in terms of the random output and minimum output, which requires temporal and spatial statistics of historical data. On the other hand, the last-iterate convergence allows us to avoid storing or selecting past output iterates after each iteration, while maintaining rigour in convergence analysis. To this end, we address the convergence of the last iterate output (called last-iterate convergence) of the stochastic momentum methods for non-convex stochastic optimization problems, in a way which is conformal with traditional optimization theory. For generality, we prove the last-iterate convergence of the stochastic momentum methods under a unified framework, covering both stochastic heavy ball momentum and stochastic Nesterov accelerated gradient momentum, whose momentum factors can be either constant or time-varying coefficients. Finally, the last-iterate convergence of the stochastic momentum methods is verified on the benchmark MNIST and CIFAR-10 datasets. The implementation of SUM is available at: https://github.com/xudp100/SUMhttps://github.com/xudp100/SUM.

Appropriate Learning Rates of Adaptive Learning Rate Optimization Algorithms for Training Deep Neural Networks.

This article deals with nonconvex stochastic optimization problems in deep learning. Appropriate learning rates, based on theory, for adaptive-learning-rate optimization algorithms (e.g., Adam and AMSGrad) to approximate the stationary points of such problems are provided. These rates are shown to allow faster convergence than previously reported for these algorithms. Specifically, the algorithms are examined in numerical experiments on text and image classification and are shown in experiments to perform better with constant learning rates than algorithms using diminishing learning rates.

Perturbed Iterate SGD for Lipschitz Continuous Loss Functions

This paper presents an extension of stochastic gradient descent for the minimization of Lipschitz continuous loss functions. Our motivation is for use in non-smooth non-convex stochastic optimization problems, which are frequently encountered in applications such as machine learning. Using the Clarke \(\epsilon \)-subdifferential, we prove the non-asymptotic convergence to an approximate stationary point in expectation for the proposed method. From this result, a method with non-asymptotic convergence with high probability, as well as a method with asymptotic convergence to a Clarke stationary point almost surely are developed. Our results hold under the assumption that the stochastic loss function is a Carathéodory function which is almost everywhere Lipschitz continuous in the decision variables. To the best of our knowledge, this is the first non-asymptotic convergence analysis under these minimal assumptions.

Stochastic Difference-of-Convex-Functions Algorithms for Nonconvex Programming

The paper deals with stochastic difference-of-convex-functions (DC) programs, that is, optimization problems whose cost function is a sum of a lower semicontinuous DC function and the expectation of a stochastic DC function with respect to a probability distribution. This class of nonsmooth and nonconvex stochastic optimization problems plays a central role in many practical applications. Although there are many contributions in the context of convex and/or smooth stochastic optimization, algorithms dealing with nonconvex and nonsmooth programs remain rare. In deterministic optimization literature, the DC algorithm (DCA) is recognized to be one of the few algorithms able to effectively solve nonconvex and nonsmooth optimization problems. The main purpose of this paper is to present some new stochastic DCAs for solving stochastic DC programs. The convergence analysis of the proposed algorithms is carefully studied, and numerical experiments are conducted to justify the algorithms' behaviors.

Stochastic model predictive controller for wind farm frequency regulation in waked conditions

As renewable energy penetrations increase, wind farms may be required to provide ancillary services such as frequency regulation. For wind farms, two issues complicate frequency regulation: turbine-wake interactions alter the available wind power, and the future power reference signal is uncertain. This paper presents a stochastic model predictive controller (MPC) for wind farms to track an uncertain frequency regulation signal while accounting for wakes. The controller uses a simplified Park model with wake propagation delay to predict turbine-wake interactions. The controller operates without any knowledge of the future power regulation signal or wind speed. The controller solves a nonconvex stochastic optimization program using scenario optimization. Our stochastic MPC tracks with 7%–10% less error than our deterministic MPC controllers.

Sparse Hybrid Precoding for Power Minimization With an Adaptive Antenna Structure in Massive MIMO Systems

In massive multiple-input multiple-output (MIMO) systems employing hybrid analog-digital precoder, there are two commonly used antenna structures that have their own pros and cons, namely, the fully-connected antenna structure (FCAS) and the partially-connected antenna structure (PCAS). The FCAS achieves better spectrum efficiency (SE) even with reduced radio frequency (RF) chains, but the hardware cost and power consumption are still grievous. Contrarily, the PCAS has less hardware cost and power consumption, but suffers from severe performance loss. In this paper, by combining the advantages of both structures, we first propose a sparse adaptive antenna structure (SAAS) for the implementation of the hybrid precoder, which can jointly control the on/off state of all phase shifters (PS) and RF chains through a switch network. Then, a sparse hybrid precoding (SHP) optimization problem based on the proposed SAAS is established aiming at minimizing the total power consumption under individual average data rate requirements. To tackle the challenging non-smooth non-convex stochastic optimization (NSO) emerged with the SHP design and reduce the power consumption of PSs and RF chains, we propose a sparse smooth approximation based an online algorithm to find a stationary point of the NSO problem and establish its convergence. Simulations verify that the proposed antenna structure and algorithm achieve a better balance between power consumption and system throughput than the existing schemes.

Lower bounds for non-convex stochastic optimization

We lower bound the complexity of finding \(\epsilon \)-stationary points (with gradient norm at most \(\epsilon \)) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance, we prove that (in the worst case) any algorithm requires at least \(\epsilon ^{-4}\) queries to find an \(\epsilon \)-stationary point. The lower bound is tight, and establishes that stochastic gradient descent is minimax optimal in this model. In a more restrictive model where the noisy gradient estimates satisfy a mean-squared smoothness property, we prove a lower bound of \(\epsilon ^{-3}\) queries, establishing the optimality of recently proposed variance reduction techniques.

Distributed Stochastic Inertial-Accelerated Methods with Delayed Derivatives for Nonconvex Problems

Stochastic gradient methods (SGMs) are predominant approaches for solving stochastic optimization. On smooth nonconvex problems, a few acceleration techniques have been applied to improve the convergence rate of SGMs. However, little exploration has been made on applying a certain acceleration technique to a stochastic subgradient method (SsGM) for nonsmooth nonconvex problems. In addition, few efforts have been made to analyze an (accelerated) SsGM with delayed derivatives. The information delay naturally happens in a distributed system, where computing workers do not coordinate with each other. In this paper, we propose an inertial proximal SsGM for solving nonsmooth nonconvex stochastic optimization problems. The proposed method can have guaranteed convergence even with delayed derivative information in a distributed environment. Convergence rate results are established for three classes of nonconvex problems: weakly convex nonsmooth problems with a convex regularizer, composite nonconvex problems with a nonsmooth convex regularizer, and smooth nonconvex problems. For each problem class, the convergence rate is $O(1/K^{\frac{1}{2}})$ in the expected value of the gradient norm square, for $K$ iterations. In a distributed environment, the convergence rate of the proposed method will be slowed down by the information delay. Nevertheless, the slow-down effect will decay with the number of iterations for the latter two problem classes. We test the proposed method on three applications. The numerical results clearly demonstrate the advantages of using the inertial-based acceleration. Furthermore, we observe higher parallelization speed-up in asynchronous updates over the synchronous counterpart, though the former uses delayed derivatives. Our source code is available at https://github.com/RPI-OPT/Inertial-SsGM.

An Active Set Limited Memory BFGS Algorithm for Machine Learning

In this paper, a stochastic quasi-Newton algorithm for nonconvex stochastic optimization is presented. It is derived from a classical modified BFGS formula. The update formula can be extended to the framework of limited memory scheme. Numerical experiments on some problems in machine learning are given. The results show that the proposed algorithm has great prospects.

Efficient Algorithms for Minimizing Compositions of Convex Functions and Random Functions and Its Applications in Network Revenue Management

In this paper, we study a class of nonconvex stochastic optimization in the form of $\min_{x\in\mathcal{X}} F(x):=\mathbb{E}_\xi [f(\phi(x,\xi))]$, where the objective function $F$ is a composition of a convex function $f$ and a random function $\phi$. Leveraging an (implicit) convex reformulation via a variable transformation $u=\mathbb{E}[\phi(x,\xi)]$, we develop stochastic gradient-based algorithms and establish their sample and gradient complexities for achieving an $\epsilon$-global optimal solution. Interestingly, our proposed Mirror Stochastic Gradient (MSG) method operates only in the original $x$-space using gradient estimators of the original nonconvex objective $F$ and achieves $\tilde{\mathcal{O}}(\epsilon^{-2})$ sample and gradient complexities, which matches the lower bounds for solving stochastic convex optimization problems. Under booking limits control, we formulate the air-cargo network revenue management (NRM) problem with random two-dimensional capacity, random consumption, and routing flexibility as a special case of the stochastic nonconvex optimization, where the random function $\phi(x,\xi)=x\wedge\xi$, i.e., the random demand $\xi$ truncates the booking limit decision $x$. Extensive numerical experiments demonstrate the superior performance of our proposed MSG algorithm for booking limit control with higher revenue and lower computation cost than state-of-the-art bid-price-based control policies, especially when the variance of random capacity is large. KEYWORDS: stochastic nonconvex optimization, hidden convexity, air-cargo network revenue management, gradient-based algorithms