Choice Of Step Size Research Articles

N IMPROVED VARIABLE STEP SIZE MPPT METHOD FOR PHOTOVOLTAIC ARRAY UNDER PARTIAL SHADING CONDITIONS

Photovoltaic (PV) systems are prone to partial shading effects, which cause multi-peak power points on the output curve, named local maximum power point (LMPP). The traditional perturbation and observation (P&O) algorithm will search into it when tracking maximum power. This paper proposes an improved variable step size (IVSS) perturbation tracking method to reduce the impact of perturbation step size choice on dynamic PV performance. MATLAB/Simulink software is used to develop and simulate the improved maximum power point tracking (MPPT) system. The IVSS tracking algorithm uses a program to achieve step perturbation. Finally, simulation results of the traditional P&O algorithm are compared with those achieved using the MPPT, verifying the superiority of proposed method in terms of tracking time, steady-state accuracy and maximum output power.

Improved Wolf Pack Algorithm for Solving a Class of Multi-Objective Route Optimization Problem

This paper considers a class of Multi-objective path planning and solving algorithms. The choice of step size is a very important factor that affects the iterative algorithm. This paper improves the classical Wolf Pack Algorithm (WPA) by designing an adaptive step size. This new improved wolf pack algorithm (IWPA) improves the classical wolf colony algorithm by designing an adaptive step size. It effectively solves the problem that the WPA may fall into a endless loop. The simulation results by MATLAB show that the IWPA has a good effectiveness than the WPA to solve multi-objective route optimization.

Nonlinear Kaczmarz algorithms and their convergence

This paper proposes a class of randomized Kaczmarz algorithms for obtaining isolated solutions of large-scale well-posed or overdetermined nonlinear systems of equations. This type of algorithm improves the classic Newton method. Each iteration only needs to calculate one row of the Jacobian instead of the entire matrix, which greatly reduces the amount of calculation and storage. Therefore, these algorithms are called matrix-free algorithms. According to the different probability selection patterns of choosing a row of the Jacobian matrix, the nonlinear Kaczmarz (NK) algorithm, the nonlinear randomized Kaczmarz (NRK) algorithm and the nonlinear uniformly randomized Kaczmarz (NURK) algorithm are proposed. In addition, the NURK algorithm is similar to the stochastic gradient descent (SGD) algorithm in nonlinear optimization problems. The only difference is the choice of step size. In the case of noise-free data, theoretical analysis and the results of numerical based on the classical tangential cone conditions show that the algorithms proposed in this paper are superior to the SGD algorithm in terms of iterations and calculation time.

Ergodic numerical approximation to periodic measures of stochastic differential equations

In this paper, we consider numerical approximation to periodic measure of a time periodic stochastic differential equations (SDEs) under weakly dissipative condition. For this we first study the existence of the periodic measure ρt and the large time behaviour of U(t+s,s,x)≔Eϕ(Xts,x)−∫ϕdρt, where Xts,x is the solution of the SDEs and ϕ is a test function being smooth and of polynomial growth at infinity. We prove U and all its spatial derivatives decay to 0 with exponential rate on time t in the sense of average on initial time s. We also prove the existence and the geometric ergodicity of the periodic measure of the discretised semi-flow from the Euler–Maruyama scheme and moment estimate of any order when the time step is sufficiently small (uniform for all orders). We thereafter obtain that the weak error for the numerical scheme of infinite horizon is of the order 1 in terms of the time step. We prove that the choice of step size can be uniform for all test functions ϕ. Subsequently we are able to estimate the average periodic measure with ergodic numerical schemes.

Subgradient averaging for multi-agent optimisation with different constraint sets

We consider a multi-agent setting with agents exchanging information over a possibly time-varying network, aiming at minimising a separable objective function subject to constraints. To achieve this objective we propose a novel subgradient averaging algorithm that allows for non-differentiable objective functions and different constraint sets per agent. Allowing different constraints per agent simultaneously with a time-varying communication network constitutes a distinctive feature of our approach, extending existing results on distributed subgradient methods. To highlight the necessity of dealing with a different constraint set within a distributed optimisation context, we analyse a problem instance where an existing algorithm does not exhibit a convergent behaviour if adapted to account for different constraint sets. For our proposed iterative scheme we show asymptotic convergence of the iterates to a minimum of the underlying optimisation problem for step sizes of the form ηk+1, η>0. We also analyse this scheme under a step size choice of ηk+1, η>0, and establish a convergence rate of O(lnkk) in objective value. To demonstrate the efficacy of the proposed method, we investigate a robust regression problem and an ℓ2 regression problem with regularisation.

Local Search with Groups of Step Sizes

Local search methods for continuous optimization problems tend to be sensitive to the choice of step sizes in their search directions. This paper presents the Local Search with Groups of Step Sizes (LSGSS) method, a derivative-free method that reactively updates groups of promising step sizes for each problem coordinate. The experiments demonstrate LSGSS could find the best solutions for each large-scale benchmark problem when compared to classical methods.

Construction of Explicit Symplectic Integrators in General Relativity. II. Reissner–Nordström Black Holes

Abstract In a previous paper, second- and fourth-order explicit symplectic integrators were designed for a Hamiltonian of the Schwarzschild black hole. Following this work, we continue to trace the possibility of construction of explicit symplectic integrators for a Hamiltonian of charged particles moving around a Reissner–Nordström black hole with an external magnetic field. Such explicit symplectic methods are still available when the Hamiltonian is separated into five independently integrable parts with analytical solutions as explicit functions of proper time. Numerical tests show that the proposed algorithms share desirable properties in their long-term stability, precision, and efficiency for appropriate choices of step size. For the applicability of one of the new algorithms, the effects of black hole’s charge, the Coulomb part of the electromagnetic potential and the magnetic parameter on the dynamical behavior are surveyed. Under some circumstances, the extent of chaos becomes strong with an increase of the magnetic parameter from a global phase-space structure. No variation of the black hole’s charge other than the Coulomb part affects the regular and chaotic dynamics of the particles’ orbits. A positive Coulomb part more easily induces chaos than a negative one.

FlexPD: A Flexible Framework of First-Order Primal-Dual Algorithms for Distributed Optimization

In this paper, we study the problem of minimizing a sum of convex objective functions, which are locally available to agents in a network. Distributed optimization algorithms make it possible for the agents to cooperatively solve the problem through local computations and communications with neighbors. Lagrangian-based distributed optimization algorithms have received significant attention in recent years, due to their exact convergence property. However, many of these algorithms have slow convergence or are expensive to execute. In this paper, we propose a flexible framework of first-order primal-dual algorithms (FlexPD), which allows for an arbitrary number of primal steps per iteration. This framework includes three algorithms, FlexPD-F, FlexPD-G, and FlexPD-C that can be customized for various applications with different computation and communication requirements. For strongly convex and Lipschitz gradient objective functions, by carefully controlling the stepsize choices, we establish linear convergence of our proposed framework to the optimal solution. Simulation results confirm the superior performance of our framework compared to the existing methods.

On Gradient Descent and Co-ordinate Descent methods and its variants.

This research is focused on Unconstrained Optimization problems. Among a number of methods that can be used to solve Unconstrained Optimization problems we have worked on Gradient and Coordinate Descent methods. Step size plays an important role for optimization. Here we have performed numerical experiment with Gradient and Coordinate Descent method for several step size choices. Comparison between different variants of Gradient and Coordinate Descent methods and their efficiency are demonstrated by implementing in loss functions minimization problem.

A Variable Step Size for Maximum Correntropy Criterion Algorithm with Improved Variable Kernel Width

The maximum correntropy criterion (MCC) algorithm has popularly been used in suppressing impulsive noise. It mainly relies on the choice of step size and kernel width. With the study of step size, this paper proposes a novel algorithm that updates the step size based on constructing a variable step size function that uses the moving weighted average algorithm to keep performance stable and sigmoid function to speed the convergence rate. The proposed algorithm is robust against impulsive noises and generates a large step size to accelerate the convergence in iteration beginning, and system mutation makes the system identification performance more stable when the error is small. Based on the improved variable kernel width maximum entropy criterion (IVKW‐MCC) algorithm, which overcomes the shortcoming of how to choose a reliable kernel width in the MCC algorithm, simulation results are compared with other MCC algorithms under impulsive noise interference. Simulations in the system identification scenarios show that the proposed algorithm has better performance in improving the convergence speed over other algorithms. © 2020 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Euler–Maclaurin expansions without analytic derivatives

The Euler–Maclaurin (EM) formulae relate sums and integrals. Discovered nearly 300 years ago, they have lost none of their importance over the years, and are nowadays routinely taught in scientific computing and numerical analysis courses. The two common versions can be viewed as providing error expansions for the trapezoidal rule and for the midpoint rule, respectively. More importantly, they provide a means for evaluating many infinite sums to high levels of accuracy. However, in all but the simplest cases, calculating very high-order derivatives analytically becomes prohibitively complicated. When approximating such derivatives with finite differences (FD), the choice of step size typically requires a severe trade-off between errors due to truncation and to rounding. We show here that, in the special case of EM expansions, FD approximations can provide excellent accuracy without the step size having to go to zero. While FD approximations of low-order derivatives to high orders of accuracy have many applications for solving ODEs and PDEs, the present context is unusual in that it relies on FD approximations to derivatives of very high orders. The application to infinite sums ensures that one can use centred FD formulae (which are not subject to the Runge phenomenon).

Network flows that solve least squares for linear equations

This paper presents a first-order distributed continuous-time algorithm for computing the least-squares solution to a linear equation over networks. Given the uniqueness of the solution, with nonintegrable and diminishing step size, convergence results are provided for fixed graphs. The exact rate of convergence is also established for various types of step size choices falling into that category. For the case where non-unique solutions exist, convergence to one such solution is proved for constantly connected switching graphs with square integrable step size. Validation of the results and illustration of the impact of step size on the convergence speed are made using a few numerical examples.

Distributed constrained optimization for multi-agent systems over a directed graph with piecewise stepsize

In this paper, the distributed constrained optimization problem over a directed graph is considered. We assume that the digraph has a row-stochastic adjacency matrix, which corresponds to the case when agents assign weights to the received information individually. We present an algorithm that converges to the optimal solution even with a time-varying stepsize which is not attenuated to zero. The choice of stepsizes is relatively easy. The usable type of stepsizes is added. The diminishing or constant stepsizes can be used in our algorithm. Equality constraints and set constraints are also considered in this paper. Convergence analysis of the proposed algorithm relies on a conversion theorem between column-stochastic and row-stochastic matrices. Finally, the results of the numerical simulation are provided to verify the effectiveness of the proposed algorithm.

Distributed stochastic gradient tracking methods

In this paper, we study the problem of distributed multi-agent optimization over a network, where each agent possesses a local cost function that is smooth and strongly convex. The global objective is to find a common solution that minimizes the average of all cost functions. Assuming agents only have access to unbiased estimates of the gradients of their local cost functions, we consider a distributed stochastic gradient tracking method (DSGT) and a gossip-like stochastic gradient tracking method (GSGT). We show that, in expectation, the iterates generated by each agent are attracted to a neighborhood of the optimal solution, where they accumulate exponentially fast (under a constant stepsize choice). Under DSGT, the limiting (expected) error bounds on the distance of the iterates from the optimal solution decrease with the network size $n$, which is a comparable performance to a centralized stochastic gradient algorithm. Moreover, we show that when the network is well-connected, GSGT incurs lower communication cost than DSGT while maintaining a similar computational cost. Numerical example further demonstrates the effectiveness of the proposed methods.

Adaptive complex-valued stepsize based fast learning of complex-valued neural networks

Complex-valued gradient descent algorithm is a popular tool to optimize functions of complex variables, especially for the training of complex-valued neural networks. However, the choice of suitable learning stepsize is a challenging task during the training process. In this paper, an adaptive complex-valued stepsize design method is proposed for complex-valued neural networks by generalizing the adaptable learning rate tree technique to the complex domain. The scaling and rotation factors are introduced to simultaneously adjust the amplitude and phase of complex-valued stepsize. The search range is thus expanded from half line to half plane such that better search direction is obtained at each iteration. We analyze the dynamics of the algorithm near a saddle point and find that it is very easy to escape from the saddle point to guarantee fast convergence and high accuracy. Some experimental results on function approximation and pattern classification tasks are presented to illustrate the advantages of the proposed algorithm over some previous ones.

A heuristic technique for traffic assignment with variable step size and number of iterations

The prediction of traffic flows on network links is an essential issue in urban transportation modeling and planning. Although manual assignment methods are possible for tiny networks, the networks involved in practical-size problems usually require complex calculations with large amounts of data. In the traditional incremental traffic assignment technique (TITAT), the O-D matrix is divided into N equal portions and assigned to the network at each iteration. Travel times are then updated, based on which the next portion of the O-D matrix is loaded onto the network. In this paper, the assumption of equal step size in all iterations is relaxed, and variable step size for the different iterations is proposed to increase the efficiency of this technique. It is shown that an intelligent choice of step size can cause more efficiency. To implement this concept quantitatively for a real-size problem, Mashhad, as the second-largest city of Iran with about 4.5 million trips per year, is used as the case study to analyze and evaluate the proposed technique and compare the results with those of TITAT. Results suggest that regarding the reliability of the outcomes and computational efficacy, the proposed algorithm is as good as other methods. Unlike other methods, there is no additional parameter to be calibrated, and the convergence behavior of the algorithm is promising. It is observed in this particular case, that there is an inverse linear relationship between the number of iterations and the initial step size; and that for a specified number of iterations, the best results are obtained for equal step sizes for all but the first iteration. The main advantage of this technique is that it can provide a simple and economical method for traffic assignment as compared with user equilibrium for which goodness-of-fit measure of predicted-observed flows on selected links equals 0.57.In contrast, this measure for the proposed incremental traffic assignment technique (PITAT) equals 0.52. The main shortcoming of the method is the fact that it is a heuristic method. Despite showing promising results, the convergence of the algorithm has yet to be practically proved. Considering the order of calculations, PITAT seems very attractive and can be applied as a substitute in situations similar to the case of this paper.

An improved Dai–Kou conjugate gradient algorithm for unconstrained optimization

It is gradually accepted that the loss of orthogonality of the gradients in a conjugate gradient algorithm may decelerate the convergence rate to some extent. The Dai–Kou conjugate gradient algorithm (SIAM J Optim 23(1):296–320, 2013), called CGOPT, has attracted many researchers’ attentions due to its numerical efficiency. In this paper, we present an improved Dai–Kou conjugate gradient algorithm for unconstrained optimization, which only consists of two kinds of iterations. In the improved Dai–Kou conjugate gradient algorithm, we develop a new quasi-Newton method to improve the orthogonality by solving the subproblem in the subspace and design a modified strategy for the choice of the initial stepsize for improving the numerical performance. The global convergence of the improved Dai–Kou conjugate gradient algorithm is established without the strict assumptions in the convergence analysis of other limited memory conjugate gradient methods. Some numerical results suggest that the improved Dai–Kou conjugate gradient algorithm (CGOPT (2.0)) yields a tremendous improvement over the original Dai–Kou CG algorithm (CGOPT (1.0)) and is slightly superior to the latest limited memory conjugate gradient software package CG$$\_ $$DESCENT (6.8) developed by Hager and Zhang (SIAM J Optim 23(4):2150–2168, 2013) for the CUTEr library.

Fast, Asymptotically Efficient, Recursive Estimation in a Riemannian Manifold

Stochastic optimisation in Riemannian manifolds, especially the Riemannian stochastic gradient method, has attracted much recent attention. The present work applies stochastic optimisation to the task of recursive estimation of a statistical parameter which belongs to a Riemannian manifold. Roughly, this task amounts to stochastic minimisation of a statistical divergence function. The following problem is considered: how to obtain fast, asymptotically efficient, recursive estimates, using a Riemannian stochastic optimisation algorithm with decreasing step sizes. In solving this problem, several original results are introduced. First, without any convexity assumptions on the divergence function, we proved that, with an adequate choice of step sizes, the algorithm computes recursive estimates which achieve a fast non-asymptotic rate of convergence. Second, the asymptotic normality of these recursive estimates is proved by employing a novel linearisation technique. Third, it is proved that, when the Fisher information metric is used to guide the algorithm, these recursive estimates achieve an optimal asymptotic rate of convergence, in the sense that they become asymptotically efficient. These results, while relatively familiar in the Euclidean context, are here formulated and proved for the first time in the Riemannian context. In addition, they are illustrated with a numerical application to the recursive estimation of elliptically contoured distributions.

New versions of Newton method: step-size choice, convergence domain and under-determined equations

ABSTRACT Newton method is one of the most powerful methods for finding solutions of nonlinear equations and for proving their existence. In its ‘pure’ form it has fast convergence near the solution, but small convergence domain. On the other hand damped Newton method has slower convergence rate, but weaker conditions on the initial point. We provide new versions of Newton-like algorithms, resulting in combinations of Newton and damped Newton method with special step-size choice, and estimate its convergence domain. Under some assumptions the convergence is global. Explicit complexity results are also addressed. The adaptive version of the algorithm (with no a priori constants knowledge) is presented. The method is applicable for under-determined equations (with m< n , m being the number of equations and n being the number of variables). The results are specified for systems of quadratic equations, for composite mappings and for one-dimensional equations and inequalities.

Multiclass traffic assignment model for mixed traffic flow of human-driven vehicles and connected and autonomous vehicles

Compared to existing human-driven vehicles (HDVs), connected and autonomous vehicles (CAVs) offer users the potential for reduced value of time, enhanced quality of travel experience, and seamless situational awareness and connectivity. Hence, CAV users can differ in their route choice behavior compared to HDV users, leading to mixed traffic flows that can significantly deviate from the single-class HDV traffic pattern. However, due to the lack of quantitative models, there is limited knowledge on the evolution of mixed traffic flows in a traffic network. To partly bridge this gap, this study proposes a multiclass traffic assignment model, where HDV users and CAV users follow different route choice principles, characterized by the cross-nested logit (CNL) model and user equilibrium (UE) model, respectively. The CNL model captures HDV users’ uncertainty associated with limited knowledge of traffic conditions while overcoming the route overlap issue of logit-based stochastic user equilibrium. The UE model characterizes the CAV's capability for acquiring accurate information on traffic conditions. In addition, the multiclass model can capture the characteristics of mixed traffic flow such as the difference in value of time between HDVs and CAVs and the asymmetry in their driving interactions, thereby enhancing behavioral realism in the modeling. The study develops a new solution algorithm labeled RSRS-MSRA, in which a route-swapping based strategy is embedded with a self-regulated step size choice technique, to solve the proposed model efficiently. Sensitivity analysis of the proposed model is performed to gain insights into the effects of perturbations on the mixed traffic equilibrium, which facilitates the estimation of equilibrium traffic flow and identification of critical elements under expected or unexpected events. The study results can assist transportation decision-makers to design effective planning and operational strategies to leverage the advantages of CAVs and manage traffic congestion under mixed traffic flows.