Constant Step Size Research Articles

Numerical Approach for Delay Volterra Integro-Differential Equation

The delay integro-differential equation for the Volterra type has been solved by using the two-point multistep block (2PBM) method with constant step-size. The proposed block method of order three is formulated using Taylor expansion and will simultaneously approximate the numerical solution at two points. The 2PBM method is developed by combining the predictor and corrector formulae in the PECE mode. The predictor formulae are explicit, while the corrector formulae are implicit. The algorithm for the approximate solutions were constructed and analyzed using the 2PBM method with Newton-Cotes quadrature rules. This paper focused on constant and pantograph delay types, and the previous values are used to interpolate the delay solutions. Moreover, the studies also carried out on the stability analysis of the proposed method. Some numerical results are tested to validate the competency of the multistep block method with quadrature rule approach.

Inexact gradient projection method with relative error tolerance

A gradient projection method with feasible inexact projections is proposed in the present paper. The inexact projection is performed using a general relative error tolerance. Asymptotic convergence analysis under quasiconvexity assumption and iteration-complexity bounds under convexity assumption of the method employing constant and Armijo step sizes are presented. Numerical results are reported illustrating the potential advantages of considering inexact projections instead of exact ones in some medium scale instances of a least squares problem over the spectrohedron.

Distributed Variable Sample-Size Stochastic Optimization With Fixed Step-Sizes

In this article, we consider distributed stochastic optimization over randomly switching networks, where agents collaboratively minimize the average of all agents’ local expectation-valued convex cost functions. Due to the stochasticity in gradient observations, distributedness of local functions, and randomness of communication topologies, distributed algorithms with an exact convergence guarantee under fixed step-sizes have not been achieved yet. This work incorporates variance reduction scheme into the distributed stochastic gradient tracking algorithm, where local gradients are estimated by averaging across a variable number of sampled gradients. With an identically and independently distributed random network, we show that all agents’ iterates converge almost surely to the same optimal solution under fixed step-sizes. When the global cost function is strongly convex and the sample size increases at a geometric rate, we prove that the iterates geometrically converge to the unique optimal solution, and establish the iteration, oracle, and communication complexity. The algorithm performance, including rate and complexity analysis, are further investigated with constant step-sizes and a polynomially increasing sample size. Finally, the empirical algorithm performance are illustrated with numerical examples.

Finite-sample analysis of nonlinear stochastic approximation with applications in reinforcement learning

Motivated by applications in reinforcement learning (RL), we study a nonlinear stochastic approximation (SA) algorithm under Markovian noise, and establish its finite-sample convergence bounds under various stepsizes. Specifically, we show that when using constant stepsize (i.e., αk≡α), the algorithm achieves exponential fast convergence to a neighborhood (with radius O(αlog(1/α))) around the desired limit point. When using diminishing stepsizes with appropriate decay rate, the algorithm converges with rate O(log(k)/k). Our proof is based on Lyapunov drift arguments, and to handle the Markovian noise, we exploit the fast mixing of the underlying Markov chain. To demonstrate the generality of our theoretical results on Markovian SA, we use it to derive the finite-sample bounds of the popular Q-learning algorithm with linear function approximation, under a condition on the behavior policy. Importantly, we do not need to make the assumption that the samples are i.i.d., and do not require an artificial projection step in the algorithm. Numerical simulations corroborate our theoretical results.

Investigations of moiré artifacts induced by flux fluctuations in x-ray dark-field imaging

X-ray dark-field imaging using a grating interferometer has shown potential benefits for a variety of applications in recent years. X-ray dark-field image is commonly retrieved by using discrete Fourier transform from the acquired phase-stepping data. The retrieval process assumes a constant phase step size and a constant flux for each stepped grating position. However, stepping errors and flux fluctuations inevitably occur due to external vibrations and/or thermal drift during data acquisition. Previous studies have shown that those influences introduce errors in the acquired phase-stepping data, which cause obvious moiré artifacts in the retrieved refraction image. This work investigates moiré artifacts in x-ray dark-field imaging as a result of flux fluctuations. For the retrieved mean intensity, amplitude, visibility and dark-field images, the dependence of moiré artifacts on flux fluctuation factors is theoretically derived respectively by using a first-order Taylor series expansion. Results of synchrotron radiation experiments verify the validity of the derived analytical formulas. The spatial frequency characteristics of moiré artifacts are analyzed and compared to those induced by phase-stepping errors. It illustrates that moiré artifacts can be estimated by a weighted mean of flux fluctuation factors, with the weighting factors dependent on the moiré phase and different greatly for each retrieved image. Furthermore, moiré artifacts can even be affected by object’s features not displayed in the particular contrast. These results can be used to interpret images correctly, identify sources of moiré artifacts, and develop dedicated algorithms to remove moiré artifacts in the retrieved multi-contrast images.

Scalable distributed least square algorithms for large-scale linear equations via an optimization approach

This paper proposes scalable distributed least square algorithms for solving large-scale linear algebraic equations via multi-agent network. We consider two kinds of different decomposition structures of Ax=b, in which each agent only knows a sub-block of linear equation instead of the complete row or column information in the existing results. By introducing one extra variable, the least square problem of linear equation can be transformed into a distributed constrained optimization problem. Based on the augmented Lagrange method, two continuous-time scalable distributed primal–dual algorithms are developed, in which all the states of agents have fewer dimension and can be even scalars. Subsequently, the discrete-time distributed algorithms with constant step-sizes are proposed by using the Euler discretization method, and the explicit bounds of constant step-sizes are provided. Based on the KKT condition and Lyapunov stability, we show that the proposed continuous-time and discrete-time distributed algorithms collaboratively obtain a least square solution with a linear convergence speed, and also own an additional property that verifies whether the obtained solution is an exact solution. Finally, some simulation examples are carried out to verify the effectiveness of the proposed algorithms.

A Fitted Numerical Method for Interior-Layer Parabolic Convection–Diffusion Problems

The aim of this paper is to construct and analyze a fitted mesh finite difference method (FMFDM) for a class of time-dependent singularly perturbed convection–diffusion–reaction problems with a turning point. While such problems have been widely studied in the case of boundary layers, little has been achieved for interior layer problems. In addition, most of works have considered the coefficient functions as the functions of space only. In this work, we study problems where the coefficient functions are dependent on both space and time variables, and their solutions exhibit an interior layer. We derive the solution bounds and their derivatives. Then we use the classical implicit Euler method to discretize the time variable with a constant step-size. This method consists of two upwind finite difference schemes defined on an appropriate non-uniform mesh of Shishkin type. This mesh is fine near the turning point and coarse elsewhere. We prove that the method is almost first-order uniformly convergent with respect to the perturbation parameter. To improve the accuracy of the proposed method, we apply Richardson extrapolation technique. Numerical simulations are provided to confirm the theoretical findings.

Stability of two-step spline collocation methods for initial value problems for fractional differential equations

This paper analyzes the numerical stability of a class of two-step spline collocation methods for initial value problems for fractional differential equations. The stability region is characterized in terms of the eigenvalues of a power series, which depends on the method parameters. We provide the stability regions for several choices of the method parameters. Numerical experiments prove the effectiveness of the stability results both in a constant stepsize framework and in the case of graded meshes.

Solving Lorenz System by Using Lower Order Symmetrized Runge-Kutta Methods

Runge-Kutta is a widely used numerical method for solving the non-linear Lorenz system. This study focuses on solving the Lorenz equations with the classical parameter values by using the lower order symmetrized Runge-Kutta methods, Implicit Midpoint Rule (IMR), and Implicit Trapezoidal Rule (ITR). We show the construction of the symmetrical method and present the numerical experiments based on the two methods without symmetrization, with one- and two-step active symmetrization in a constant step size setting. For our numerical experiments, we use MATLAB software to solve and plot the graphical solutions of the Lorenz system. We compare the oscillatory behaviour of the solutions and it appears that IMR and two-step active IMR turn out to be chaotic while the rest turn out to be non-chaotic. We also compare the accuracy and efficiency of the methods and the result shows that IMR performs better than the symmetrizers, while two-step active ITR performs better than ITR and one-step active ITR. Based on the results, we conclude that different implicit numerical methods with different steps of active symmetrization can significantly impact the solutions of the non-linear Lorenz system. Since most study on solving the Lorenz system is based on explicit time schemes, we hope this study can motivate other researchers to analyze the Lorenz equations further by using Runge-Kutta methods based on implicit time schemes.

Stationary Behavior of Constant Stepsize SGD Type Algorithms

Stochastic approximation (SA) and stochastic gradient descent (SGD) algorithms are work-horses for modern machine learning algorithms. Their constant stepsize variants are preferred in practice due to fast convergence behavior. However, constant stepsize SA algorithms do not converge to the optimal solution, but instead have a stationary distribution, which in general cannot be analytically characterized. In this work, we study the asymptotic behavior of the appropriately scaled stationary distribution, in the limit when the constant stepsize goes to zero. Specifically, we consider the following three settings: (1) SGD algorithm with a smooth and strongly convex objective, (2) linear SA algorithm involving a Hurwitz matrix, and (3) nonlinear SA algorithm involving a contractive operator. When the iterate is scaled by 1/√α, where α is the constant stepsize, we show that the limiting scaled stationary distribution is a solution of an implicit equation. Under a uniqueness assumption (which can be removed in certain settings) on this equation, we further characterize the limiting distribution as a Gaussian distribution whose covariance matrix is the unique solution of an appropriate Lyapunov equation. For SA algorithms beyond these cases, our numerical experiments suggest that unlike central limit theorem type results: (1) the scaling factor need not be 1/√α, and (2) the limiting distribution need not be Gaussian. Based on the numerical study, we come up with a heuristic formula to determine the right scaling factor, and make a connection to the Euler-Maruyama discretization scheme for approximating stochastic differential equations.

Asymptotic error in Euler’s method with a constant step size

In the gradient descent method, one often focus on the convergence of the sequence generated by the algorithm, but less often on the deviation of these points from the solutions of the original continuous-time differential equation (gradient flow). This also happens when discretizing other ordinary differential equations. In the case of a discretization by explicit Euler’s method with a constant step h, we provide here sufficient conditions, in terms of strong monotonicity and co-coercivity, for the deviation between discrete and continuous solutions to tend asymptotically towards zero. This analysis could shed new light on some applications of the gradient descent algorithm.

Convergence of Two Simple Methods for Solving Monotone Inclusion Problems in Reflexive Banach Spaces

We propose two very simple methods, the first one with constant step sizes and the second one with self-adaptive step sizes, for finding a zero of the sum of two monotone operators in real reflexive Banach spaces. Our methods require only one evaluation of the single-valued operator at each iteration. Weak convergence results are obtained when the set-valued operator is maximal monotone and the single-valued operator is Lipschitz continuous, and strong convergence results are obtained when either one of these two operators is required, in addition, to be strongly monotone. We also obtain the rate of convergence of our proposed methods in real reflexive Banach spaces. Finally, we apply our results to solving generalized Nash equilibrium problems for gas markets.

On linear and super-linear convergence of Natural Policy Gradient algorithm

Markov Decision Processes are classically solved using Value Iteration and Policy Iteration algorithms. Recent interest in Reinforcement Learning has motivated the study of methods inspired by optimization, such as gradient ascent. Among these, a popular algorithm is the Natural Policy Gradient, which is a mirror descent variant for MDPs. This algorithm forms the basis of several popular RL algorithms such as Natural actor–critic, TRPO, PPO, etc., and so is being studied with growing interest. It has been shown that Natural Policy Gradient with constant step size converges with a sub-linear rate of O(1/k) to the global optimal. In this paper, we present improved finite time convergence bounds, and show that this algorithm has geometric (also known as linear) asymptotic convergence rate. We further improve this convergence result by introducing a variant of Natural Policy Gradient with adaptive step sizes. Finally, we compare different variants of policy gradient methods experimentally.

ET-DASG: An Efficient Decentralized Algorithm for Convex Optimization Over Networks

This article studies the problem of decentralized optimization to minimize a finite-sum of convex cost functions over the nodes of a network where each cost function is further considered as the average of several constituent functions. Recalling the existing work, decentralized accelerated methods that consider improving both communication and computation efficiency have not yet been investigated. Based on this, we present an effective event-triggering decentralized accelerated stochastic gradient algorithm, namely, ET-DASG. ET-DASG leverages the event-triggering strategy for improving communication efficiency, the variance-reduction technique of SAGA for promoting computation efficiency, and the Nesterov's acceleration mechanism for the accelerated convergence. We provide a convergence analysis and show that ET-DASG with well-selected constant step-size can converge in the mean to the exact optimal solution. At the same time, linear convergence rate is achieved if each constituent function is strongly convex and smooth due to the adoption of gradient-tracking scheme. Under certain conditions, we prove that for each node the time interval between two successive triggering instants is larger than the iteration interval. Finally, simulation results also confirm the appealing performance of ET-DASG.

An inertial Mann forward-backward splitting algorithm of variational inclusion problems and its applications

In this paper, we introduce the inertial Mann forward-backward splitting algorithm for solving variational inclusion problem of the sum of two operators, the one is maximally monotone and the other is monotone and Lipschitz continuous. Under standard assumptions, we prove the weak convergence theorem of the proposed algorithm. We show that the algorithm is flexible to use by choosing the variable stepsizes and two different algorithms are shown by choosing constant stepsize and update stepsize. Moreover, we apply our algorithms to solve data classification using the Wisconsin original breast cancer data set as a training set. We also compare our algorithms with the other two algorithms to show the efficiency of the algorithm and show suitably learns the training dataset and generalizes well to a hold-out dataset of the algorithm by considering overfitting. Finally, we apply our algorithms to solve signal recovery and show the efficiency of the algorithm by compare with the other two algorithms. The results of data classification and signal recovery showed that choosing the right stepsizes of the algorithm would be a good efficient for the different problems. AMS subject classification46T99, 47B02, 47H05, 47J25, 49M37.

Almost sure stability of stochastic theta methods with random variable stepsize for stochastic differential equations

In this paper, we use the non-negative discrete semimartingale convergence theorems to study the stochastic theta methods with random stepsizes to reproduce the almost sure stability of the exact solution of stochastic differential equations. Moreover, the choice of the stepsize in each step is based on the stochastic theta methods of random variable stepsize. In numerical experiments, we propose an algorithm that successfully use θ-Maruyama and θ-Milstein methods to simulate the numerical solutions of stochastic differential equations, reproduce the almost sure stability of exact solutions of SDEs and simulate the random variable stepsize in each timestep, and compared with constant stepsizes, random stepsize can speed up the decay process and reduce the iterations greatly.

Stationary Behavior of Constant Stepsize SGD Type Algorithms

Stochastic approximation (SA) and stochastic gradient descent (SGD) algorithms are work-horses for modern machine learning algorithms. Their constant stepsize variants are preferred in practice due to fast convergence behavior. However, constant stepsize SA algorithms do not converge to the optimal solution, but instead have a stationary distribution, which in general cannot be analytically characterized. In this work, we study the asymptotic behavior of the appropriately scaled stationary distribution, in the limit when the constant stepsize goes to zero. Specifically, we consider the following three settings: (1) SGD algorithm with a smooth and strongly convex objective, (2) linear SA algorithm involving a Hurwitz matrix, and (3) nonlinear SA algorithm involving a contractive operator. When the iterate is scaled by 1/α, where α is the constant stepsize, we show that the limiting scaled stationary distribution is a solution of an implicit equation. Under a uniqueness assumption (which can be removed in certain settings) on this equation, we further characterize the limiting distribution as a Gaussian distribution whose covariance matrix is the unique solution of a suitable Lyapunov equation. For SA algorithms beyond these cases, our numerical experiments suggest that unlike central limit theorem type results: (1) the scaling factor need not be 1/α, and (2) the limiting distribution need not be Gaussian. Based on the numerical study, we come up with a heuristic formula to determine the right scaling factor, and make insightful connection to the Euler-Maruyama discretization scheme for approximating stochastic differential equations.

Robust echo state network with sparse online learning

Echo state network (ESN) is an effective tool for nonlinear systems modeling. To handle irregular noises or outliers in practical systems and alleviate the overfitting issue, the robust echo state network with sparse online learning (RESN-SOL) is proposed. Firstly, the ε-insensitive loss function is introduced to replace the commonly used quadratic loss function, which is theoretically optimal for Gaussian noise distribution. Secondly, the online gradient descent algorithm is used to calculate the network readout. Notably, the better learning performance can be achieved by the constant learning rate rather than the decreasing step size. Based on this observation, the sparse online learning algorithm (SOL) is proposed, in which the constant step size is used. Particularly, the SOL is able to truncate the small weights in network readout to zero for achieving sparsity. Furthermore, the convergence of RESN-SOL is theoretically analyzed, which implies the tradeoff between learning performance and readout sparsity can be controlled by a predefined sparsity parameter. Finally, the proposed method is verified in two simulated benchmarks and an actual dynamical wastewater treatment system. Experimental results demonstrate that the RESN-SOL exhibits better robustness against outliers, network compactness and modeling accuracy than other existing algorithms.

Riemannian stochastic fixed point optimization algorithm

This paper considers a stochastic optimization problem over the fixed point sets of quasinonexpansive mappings on Riemannian manifolds. The problem enables us to consider Riemannian hierarchical optimization problems over complicated sets, such as the intersection of many closed convex sets, the set of all minimizers of a nonsmooth convex function, and the intersection of sublevel sets of nonsmooth convex functions. We focus on adaptive learning rate optimization algorithms, which adapt step-sizes (referred to as learning rates in the machine learning field) to find optimal solutions quickly. We then propose a Riemannian stochastic fixed point optimization algorithm, which combines fixed point approximation methods on Riemannian manifolds with the adaptive learning rate optimization algorithms. We also give convergence analyses of the proposed algorithm for nonsmooth convex and smooth nonconvex optimization. The analysis results indicate that, with small constant step-sizes, the proposed algorithm approximates a solution to the problem. Consideration of the case in which step-size sequences are diminishing demonstrates that the proposed algorithm solves the problem with a guaranteed convergence rate. This paper also provides numerical comparisons that demonstrate the effectiveness of the proposed algorithms with formulas based on the adaptive learning rate optimization algorithms, such as Adam and AMSGrad.

Continuous-time Distributed Heavy-ball Algorithm for Distributed Convex Optimization over Undirected and Directed Graphs

This paper proposes second-order distributed algorithms over multi-agent networks to solve the convex optimization problem by utilizing the gradient tracking strategy, with convergence acceleration being achieved. Both the undirected and unbalanced directed graphs are considered, extending existing algorithms that primarily focus on undirected or balanced directed graphs. Our algorithms also have the advantage of abandoning the diminishing step-size strategy so that slow convergence can be avoided. Furthermore, the exact convergence to the optimal solution can be realized even under the constant step size adopted in this paper. Finally, two numerical examples are presented to show the convergence performance of our algorithms.