Constant Step Size Research Articles

Geometric ergodicity of SGLD via reflection coupling

We consider the geometric ergodicity of the Stochastic Gradient Langevin Dynamics (SGLD) algorithm under nonconvexity settings. Via the technique of reflection coupling, we prove the Wasserstein contraction of SGLD when the target distribution is log-concave only outside some compact sets. The time discretization and the minibatch in SGLD introduce several difficulties when applying the reflection coupling, which are addressed by a series of careful estimates of conditional expectations. As a direct corollary, the SGLD with constant step size has an invariant distribution and we are able to obtain its geometric ergodicity in terms of [Formula: see text] distance. The generalization to non-gradient drifts is also included.

Distributed event-triggered algorithm for convex optimization with coupled constraints

This paper develops a distributed primal–dual algorithm via an event-triggered mechanism to solve a class of convex optimization problems subject to local set constraints, coupled equality and inequality constraints. Different from some existing distributed algorithms with the diminishing step-sizes, our algorithm uses the constant step-sizes, and is shown to achieve an exact convergence to an optimal solution with an ergodic convergence rate of O(1/k) for general convex objective functions, where k>0 is the iteration number. Based on the event-triggered communication mechanism, the proposed algorithm can effectively reduce the communication cost without sacrificing the convergence rate. Finally, a numerical example is presented to verify the effectiveness of the proposed algorithm.

Efficient simulation of complex Ginzburg–Landau equations using high-order exponential-type methods

In this paper, we consider the task of efficiently computing the numerical solution of evolutionary complex Ginzburg–Landau equations on Cartesian product domains with homogeneous Dirichlet/Neumann or periodic boundary conditions. To this aim, we employ for the time integration high-order exponential methods of splitting and Lawson type with constant time step size. These schemes enjoy favorable stability properties and, in particular, do not show restrictions on the time step size due to the underlying stiffness of the models. The needed actions of matrix exponentials are efficiently realized by using a tensor-oriented approach that suitably employs the so-called μ-mode product (when the semidiscretization in space is performed with finite differences) or with pointwise operations in Fourier space (when the model is considered with periodic boundary conditions). The overall effectiveness of the approach is demonstrated by running simulations on a variety of two- and three-dimensional (systems of) complex Ginzburg–Landau equations with cubic or cubic-quintic nonlinearities, which are widely considered in literature to model relevant physical phenomena. In fact, we show that high-order exponential-type schemes may outperform standard techniques to integrate in time the models under consideration, i.e., the well-known second-order split-step method and the explicit fourth-order Runge–Kutta integrator, for stringent accuracies.

Decentralized Adaptive temporal-difference learning over time-varying networks and its finite-time analysis

In reinforcement learning, centralized temporal-difference (TD) learning is commonly used to solve the policy evaluation problem. However, the decentralized adaptive variant of the TD learning algorithm has rarely been investigated in multi-agent reinforcement learning. To fill this gap, based on linear function approximation, we propose a decentralized adaptive TD learning algorithm over time-varying networks, referred to as D-AdaTD. We rigorously analyze the convergence performance of D-AdaTD, i.e., the explicit finite-time analysis is established under different step-sizes. Specifically, under constant step-sizes, the average estimated value function can converge to a neighborhood of the optimal value at rate O(1/(k+1)) and the estimated parameter of each agent can converge to a neighborhood of the optimal parameter at rate O(ξk), where k is the number of iterations and ξ∈(0,1). Under diminishing step-sizes, the average estimated value function can converge to the optimal value and the average estimated parameter can converge to the optimal parameter at rate O((1+log(k+1))/k) and O((1+log(k+1))/(k+1)), respectively. In addition, we evaluate the performance of D-AdaTD via simulation experiments, which are commonly insufficient in the existing decentralized temporal-difference learning. The experimental results also validate the effectiveness of D-AdaTD.

Prelimit Coupling and Steady-State Convergence of Constant-stepsize Nonsmooth Contractive SA

Motivated by Q-learning, we study nonsmooth contractive stochastic approximation (SA) with constant stepsize. We focus on two important classes of dynamics: 1) nonsmooth contractive SA with additive noise, and 2) synchronous and asynchronous Q-learning, which features both additive and multiplicative noise. For both dynamics, we establish weak convergence of the iterates to a stationary limit distribution in Wasserstein distance. Furthermore, we propose a prelimit coupling technique for establishing steady-state convergence and characterize the limit of the stationary distribution as the stepsize goes to zero. Using this result, we derive that the asymptotic bias of nonsmooth SA is proportional to the square root of the stepsize, which stands in sharp contrast to smooth SA. This bias characterization allows for the use of Richardson-Romberg extrapolation for bias reduction in nonsmooth SA.

First Order Methods for Geometric Optimization of Crystals: Experimental Analysis

AbstractThe geometric optimization of crystal structures is a procedure widely used in computational chemistry that changes the geometrical placement of the particles inside a structure. It is called structural relaxation and constitutes a local minimization problem with a non‐convex objective function whose domain complexity increases according to the number of particles involved. This work studies the performance of the two most popular gradient methods in structural relaxation, Steepest Descent and Conjugate Gradient. Although frequently employed, there is a lack of their study in this context from an algorithmic point of view. The algorithms are initially benchmarked on the basis of a constant step size. Three concepts for designing dynamic step size rules are then examined in detail and analyzed. Results show that there is a trade‐off between convergence rate and the possibility of an experiment to succeed. In order to address this, a function is proposed as a formal means for assigning utility to each method based on preference. The function is built according to a recently introduced model of preference indication concerning algorithms with deadline and their run time. It introduces the quantification of the optimization algorithms' performance according to convergence speed and success rate, thus enabling the appointment of a specific algorithmic recipe as the best choice for balanced preferences.

Optimizing A-stable hyperbolic fitting for time efficiency: exploring constant and variable stepsize approaches

Optimizing A-stable hyperbolic fitting for time efficiency: exploring constant and variable stepsize approaches

Asynchronous progressive iterative approximation method for least squares fitting

For large data fitting, the least squares progressive iterative approximation (LSPIA) methods have been proposed by Lin and Zhang (2013) and Deng and Lin (2014), in which a constant step size is used. In this paper, we further accelerate the LSPIA method in terms of a Chebyshev semi-iterative scheme and present an asynchronous LSPIA (denoted by ALSPIA) method. The control points in ALSPIA are updated by using an extrapolated variant in which an adaptive step size is chosen according to the roots of Chebyshev polynomial. Our convergence analysis shows that ALSPIA is faster than the original LSPIA method in both singular and non-singular least squares fitting cases. Numerical examples show that the proposed algorithm is feasible and effective.

Edge-based distributed primal–dual algorithms for seeking generalized Nash equilibria

This paper introduces two novel distributed algorithms aimed at finding the generalized Nash equilibria (GNE) in noncooperative games. To tackle this, we utilize the variational inequality framework, transforming the noncooperative game into the challenge of identifying the zeros of a sum of monotone operators. Within noncooperative games, the decisions made by individual players are interlinked via shared affine constraints. Our proposed approach involves two edge-based distributed algorithms, differentiated by their access to players’ actions, whether it is full-decision information or partial-decision information. We establish the parameter range and demonstrate the convergence of both algorithms, showcasing their efficacy under local constant step-sizes. Additionally, we validate the effectiveness and advantages of these algorithms through two numerical experiments.

Effectiveness of Constant Stepsize in Markovian LSA and Statistical Inference

In this paper, we study the effectiveness of using a constant stepsize in statistical inference via linear stochastic approximation (LSA) algorithms with Markovian data. After establishing a Central Limit Theorem (CLT), we outline an inference procedure that uses averaged LSA iterates to construct confidence intervals (CIs). Our procedure leverages the fast mixing property of constant-stepsize LSA for better covariance estimation and employs Richardson-Romberg (RR) extrapolation to reduce the bias induced by constant stepsize and Markovian data. We develop theoretical results for guiding stepsize selection in RR extrapolation, and identify several important settings where the bias provably vanishes even without extrapolation. We conduct extensive numerical experiments and compare against classical inference approaches. Our results show that using a constant stepsize enjoys easy hyperparameter tuning, fast convergence, and consistently better CI coverage, especially when data is limited.

Linear convergence of decentralized estimation for statistical estimation using gradient method

There has been a growing interest in solving consensus optimization problems in a multi-agent system. It is known that there is an exactness-speed dilemma for decentralized optimization for the naive gradient method, where either we have fast linear convergence to a neighborhood of size O(η) using a fixed constant step size η, or slow convergence to the exact optimizer with a decreasing step size. We reinvestigate a special case of this problem from a statistical point of view, where each agent’s local function is a sample estimate of a common population function. It is shown that when considering the convergence to the population target instead of to the exact consensual solution, the convergence is linear up to the statistical accuracy of the problem, using a step size only depending on the network characteristics instead of the sample size N. The objective function is not required to be strongly convex (but their statistical limit is), which allows for example the absolute deviation function which is used in median regression. Such results are in stark contrast to the optimization literature, where strong convexity of the objective function is necessary for linear convergence.

Stochastic Primal–Dual Hybrid Gradient Algorithm with Adaptive Step Sizes

In this work, we propose a new primal–dual algorithm with adaptive step sizes. The stochastic primal–dual hybrid gradient (SPDHG) algorithm with constant step sizes has become widely applied in large-scale convex optimization across many scientific fields due to its scalability. While the product of the primal and dual step sizes is subject to an upper-bound in order to ensure convergence, the selection of the ratio of the step sizes is critical in applications. Up-to-now there is no systematic and successful way of selecting the primal and dual step sizes for SPDHG. In this work, we propose a general class of adaptive SPDHG (A-SPDHG) algorithms and prove their convergence under weak assumptions. We also propose concrete parameters-updating strategies which satisfy the assumptions of our theory and thereby lead to convergent algorithms. Numerical examples on computed tomography demonstrate the effectiveness of the proposed schemes.

Design of Complex Vector Controller for High-Power Induction Machine Drive

In high-power drives of large commercial vehicles, the switching frequency of the inverter is always strictly limited to avoid high switching losses. In this paper, a high-performance complex vector controller in discrete domain is proposed to reduce the cross-coupling effect and improve control performance under low switching frequency. The discretization error of the induction motor model under different coordinate systems is explored, and a high-precision discrete induction motor model is proposed for the design of the complex vector controller and flux observer. In order to improve the dynamic response of the rotor flux linkage, the designed controller takes the current as the inner loop and the rotor flux linkage as the outer loop. Besides, the signal reconstruction strategy is proposed to solve the contradiction between the variable-step characteristics caused by synchronous modulation and the discretization process based on constant step size. Finally, the designed control strategy and analysis of the system are validly verified by simulation and experimental results.

Cooperative Multiagent Reinforcement Learning With Partial Observations

In this paper, we propose a distributed zeroth-order policy optimization method for Multi-Agent Reinforcement Learning (MARL). Existing MARL algorithms often assume that every agent can observe the states and actions of all the other agents in the network. This can be impractical in large-scale problems, where sharing the state and action information with multi-hop neighbors may incur significant communication overhead. The advantage of the proposed zeroth-order policy optimization method is that it allows the agents to compute the local policy gradients needed to update their local policy functions using local estimates of the global accumulated rewards that depend on partial state and action information only and can be obtained using consensus. Specifically, to calculate the local policy gradients, we develop a new distributed zeroth-order policy gradient estimator that relies on one-point residual-feedback which, compared to existing zeroth-order estimators that also rely on one-point feedback, significantly reduces the variance of the policy gradient estimates improving, in this way, the learning performance. We show that the proposed distributed zeroth-order policy optimization method with constant stepsize converges to the neighborhood of a policy that is a stationary point of the global objective function. The size of this neighborhood depends on the agents' learning rates, the exploration parameters, and the number of consensus steps used to calculate the local estimates of the global accumulated rewards. Moreover, we provide numerical experiments that demonstrate that our new zeroth-order policy gradient estimator is more sample-efficient compared to other existing one-point estimators.

Online regularized learning algorithm for functional data

In recent years, functional linear models have attracted growing attention in statistics and machine learning for recovering the slope function or its functional predictor. This paper considers online regularized learning algorithm for functional linear models in a reproducing kernel Hilbert space. It provides convergence analysis of excess prediction error and estimation error with polynomially decaying step-size and constant step-size, respectively. Fast convergence rates can be derived via a capacity dependent analysis. Introducing an explicit regularization term extends the saturation boundary of unregularized online learning algorithms with polynomially decaying step-size and achieves fast convergence rates of estimation error without capacity assumption. In contrast, the latter remains an open problem for the unregularized online learning algorithm with decaying step-size. This paper also demonstrates competitive convergence rates of both prediction error and estimation error with constant step-size compared to existing literature.

Sufficient Conditions for Instability of the Subgradient Method with Constant Step Size

Sufficient Conditions for Instability of the Subgradient Method with Constant Step Size

Digital backpropagation based on binary logarithmic step size distribution for fibre non‐linearity compensation

AbstractCapacity crunch has become critical in recent years as commercial communication systems approach their theoretical data rate limits. This work presents a low‐complexity digital backpropagation (DBP) implementation approach based on step size distribution that uses a binary logarithmic step size method to achieve high data rate optical transmission. The proposed scheme shows performance improvements (∆Q) of 2.36, 1.19, and 0.71 dB over linear compensation, constant step size (CSS) DBP, and logarithmic step size DBP techniques in a 2400 km 112 Gbit/s DP‐16 quadrature amplitude modulation (QAM) system, respectively. At 13 dBm, a high performance (Q) of 10.9 dB (BER = 2.25 × 10−4) is achieved, above the 3.80 × 10−3 hard‐decision forward error correction (HD‐FEC) limit, using the proposed scheme. Also, the allowable transmission distance is extended by 960 km at the HD‐FEC limit over the linear compensation technique. The optimization achieves a 38% saving in the number of DBP calculation steps compared to the CSS DBP, which considerably reduces the computational cost since a few steps are required for effective non‐linearity compensation.

A Differentially Private Method for Distributed Optimization in Directed Networks via State Decomposition

In this paper, we study the problem of consensus-based distributed optimization, where a network of agents, abstracted as a directed graph, aims to minimize the sum of all agents' cost functions collaboratively. In existing distributed optimization approaches (Push-Pull/AB) for directed graphs, all agents exchange their states with neighbors to achieve the optimal solution with a constant stepsize, which may lead to the disclosure of sensitive and private information. For privacy preservation, we propose a novel state-decomposition based gradient tracking approach (SD-Push-Pull) for distributed optimization over directed networks that preserves differential privacy, which is a strong notion that protects agents' privacy against an adversary with arbitrary auxiliary information. The main idea of the proposed approach is to decompose the gradient state of each agent into two sub-states. Only one substate is exchanged by the agent with its neighbours over time, and the other one is not shared. That is to say, only one substate is visible to an adversary, protecting the sensitive information from being leaked. It is proved that under certain decomposition principles, a bound for the sub-optimality of the proposed algorithm can be derived and the differential privacy is achieved simultaneously. Moreover, the trade-off between differential privacy and the optimization accuracy is also characterized. Finally, a numerical simulation is provided to illustrate the effectiveness of the proposed approach.

The continuous stochastic gradient method: part I–convergence theory

In this contribution, we present a full overview of the continuous stochastic gradient (CSG) method, including convergence results, step size rules and algorithmic insights. We consider optimization problems in which the objective function requires some form of integration, e.g., expected values. Since approximating the integration by a fixed quadrature rule can introduce artificial local solutions into the problem while simultaneously raising the computational effort, stochastic optimization schemes have become increasingly popular in such contexts. However, known stochastic gradient type methods are typically limited to expected risk functions and inherently require many iterations. The latter is particularly problematic, if the evaluation of the cost function involves solving multiple state equations, given, e.g., in form of partial differential equations. To overcome these drawbacks, a recent article introduced the CSG method, which reuses old gradient sample information via the calculation of design dependent integration weights to obtain a better approximation to the full gradient. While in the original CSG paper convergence of a subsequence was established for a diminishing step size, here, we provide a complete convergence analysis of CSG for constant step sizes and an Armijo-type line search. Moreover, new methods to obtain the integration weights are presented, extending the application range of CSG to problems involving higher dimensional integrals and distributed data.

Finite-time error bounds for distributed linear stochastic approximation

This paper considers a novel multi-agent linear stochastic approximation algorithm driven by Markovian noise and general consensus-type interaction, in which each agent evolves according to its local stochastic approximation process which depends on the information from its neighbors. The interconnection structure among the agents is described by a time-varying directed graph. While the convergence of consensus-based stochastic approximation algorithms when the interconnection among the agents is described by doubly stochastic matrices (at least in expectation) has been studied, less is known about the case when the interconnection matrix is simply stochastic. For any uniformly strongly connected graph sequences whose associated interaction matrices are stochastic, the paper derives finite-time bounds on the mean-square error, defined as the deviation of the output of the algorithm from the unique equilibrium point of the associated ordinary differential equation. For the case of interconnection matrices being stochastic, the equilibrium point can be any unspecified convex combination of the local equilibria of all the agents in the absence of communication. Both the cases with constant and time-varying step-sizes are considered. In the case when the convex combination is required to be a straight average and interaction between any pair of neighboring agents may be uni-directional, so that doubly stochastic matrices cannot be implemented in a distributed manner, the paper proposes a push-sum-type distributed stochastic approximation algorithm and provides its finite-time bound for the time-varying step-size case by leveraging the analysis for the consensus-type algorithm with stochastic matrices and developing novel properties of the push-sum algorithm. Distributed temporal difference learning is discussed as an illustrative application.