Step Size Control Research Articles

Stability of step size control based on a posteriori error estimates

A posteriori error estimates based on residuals can be used for reliable error control of numerical methods. Here, we consider them in the context of ordinary differential equations and Runge-Kutta methods. In particular, we take the approach of Dedner & Giesselmann (2016) and investigate it when used to select the time step size. We focus on step size control stability when combined with explicit Runge-Kutta methods and demonstrate that a standard I controller is unstable while more advanced PI and PID controllers can be designed to be stable. We compare the stability properties of residual-based estimators and classical error estimators based on an embedded Runge-Kutta method both analytically and in numerical experiments.

On the adaptive solution of phase‐field problems with A‐stable explicit last‐stage diagonally implicit Runge–Kutta (ELDIRK) methods

AbstractRunge–Kutta algorithms with adaptive step size control provide reliable tools for the solution of initial value problems with diagonally implicit Runge‐Kutta (DIRK) methods as the most common approach. In this contribution, the new low‐order explicit last‐stage diagonally implicit Runge–Kutta (ELDIRK) methods are investigated, combining implicit schemes with an additional explicit evaluation as an explicit last stage. This results in Butcher tableaus with two solutions of different convergence orders suitable for embedded methods, where the higher‐order solution is achieved by additional explicit evaluations. Thus, the iterative solution of non‐linear systems is omitted for the additional stage, presenting a major reduction in computational cost for the determination of a local error estimate. The key contribution is the application of the novel Butcher tableaux to phase‐field problems, solved with the finite‐element method, leading to substantial numerical investigations with an efficient approach for DIRK schemes. The most important aspects are the investigations of stability properties which lead to the novel class of A‐stable ELDIRK methods. In accordance, the study of the convergence orders is presented. A local error estimator is presented, such that adaptive step size control for the new low‐order embedded schemes based on an empirical approach for error estimation is achieved. A suitable parallel algorithm is presented with conclusive two‐dimensional phase‐field simulations based on a Kobayashi‐Warren‐Carter model. The higher‐order convergence suggested by the novel schemes is confirmed, and their effective results are demonstrated, resulting in a valuable semi‐explicit addition to the family of Runge–Kutta time integration schemes.

An improved Takagi–Sugeon–Kang-based FxLMS algorithm and its active noise control in electric bus driving position

This paper proposed an improved filtered-x least mean square (FxLMS) algorithm based on the fuzzy control rule of Takagi–Sugeon–Kang (TSK) to solve the drawback of slow convergence for standard FxLMS algorithm. The TSK-FxLMS is a control framework with two inputs and three outputs constructed by the error signal and its integral as input variables. To validate its effectiveness and applicability, in-vehicle active noise control (ANC) modelling and adaptive noise reduction at the driver’s position of an electric bus are thoroughly investigated. Firstly, the four noise signals for the different working conditions at 50 km/h, acceleration, coasting and braking are collected, and their spectral analyses are performed. Secondly, the effects of step size and fuzzy control parameters of ANC model on weight convergence and noise reduction effect are analysed and determined. Finally, the results of calculating and comparing the residual signals’ waveforms, frequency spectra, sound pressure levels and tracking performance indicate that the proposed TSK-FxLMS algorithm outperforms the standard FxLMS algorithm with faster convergence and better noise reduction effect.

FLARE: field line analysis and reconstruction for 3D boundary plasma modeling

Abstract The FLARE code is a magnetic mesh generator that is integrated within a suite of tools for the analysis of the magnetic geometry in toroidal fusion devices. A magnetic mesh is constructed from field line segments and permits fast reconstruction of field lines in 3D boundary plasma codes such as EMC3-EIRENE. Both intrinsically non-axisymmetric configurations (stellarators) and those with symmetry breaking perturbations of an axisymmetric equilibrium (tokamaks) are supported. The code itself is written in Modern Fortran with MPI support for parallel computing, and it incorporates object-oriented programming for the definition of the magnetic field and the material surface geometry. Extended derived types for a number of different magnetohydrodynamic equilibrium and plasma response models are implemented. The core element of FLARE is a field line tracer with adaptive step-size control, and this is integrated into tools for the construction of Poincaré maps and invariant manifolds of X-points. A collection of high-level procedures that generate output files for visualization is build on top of that. The analysis modules are build with Python frontends that facilitate customization of tasks and/or scripting of parameter scans.

Parameter ESTimation With the Gauss-Levenberg-Marquardt Algorithm: AnIntuitive Guide.

In this paper, we review the derivation of the Gauss-Levenberg-Marquardt (GLM) algorithm and its extension to ensemble parameter estimation. We explore the use of graphical methods to provide insights into how the algorithm works in practice and discuss the implications of both algorithm tuning parameters and objective function construction in performance. Some insights include understanding the control of both parameter trajectory and step size for GLM as a function of tuning parameters. Furthermore, for the iterative Ensemble Smoother (iES), we discuss the importance of noise on observations and show how iES can cope with non-unique outcomes based on objective function construction. These insights are valuable for modelers using PEST, PEST++, or similar parameter estimation tools.

The Basics of Evolution Strategies: The Implementation of the Biomimetic Optimization Method in Educational Modules.

With a focus on education and teaching, we provide general background information on bioinspired optimization methods by comparing the concept of optimization and the search for an optimum in engineering and biology. We introduce both the principles of Darwinian evolution and the basic evolutionary optimization procedure of evolution strategies. We provide three educational modules in work sheets that can be used by teachers and students to improve their understanding of evolution strategies. The educational module "Optimization of a Milk Carton" shows that the material consumption in producing a milk carton can be minimized using an evolution strategy with a mutative step size control. The use of a standard dice and a pocket calculator enables new milk cartons to be generated, with the offspring having the lowest material consumption becoming the parent of the next generation. The other educational modules deal with the so-called brachistochrone problem. The module "Fastest and Shortest Marble Track" provides a construction plan for a marble track whereby students can experimentally compare the "path of shortest length" with the "path of shortest time". The EvoBrach software, is used in the module "Various Marble Track Shapes" to compare the running times of a marble on a straight line, a parabola, and a brachistochrone. In conclusion, the introduction to the biomimetic method of evolution strategies and the educational modules should deepen the understanding of both optimization problems and biological evolution.

Assessing the benefits of approximately exact step sizes for Picard and Newton solver in simulating ice flow (FEniCS-full-Stokes v.1.3.2)

Abstract. Solving the momentum balance is the computationally expensive part of simulating the evolution of ice sheets. The momentum balance is described by the nonlinear full-Stokes equations, which are solved iteratively. We use the Picard iteration and Newton's method combined with Armijo step sizes and approximately exact step sizes, respectively, to solve these equations. The Picard iteration uses either no step size control or the approximately exact step sizes. We compare the variants of Newton's method and the Picard iteration in benchmark experiments, called ISMIP-HOM experiments A, B, E1, and E2. The ISMIP-HOM experiments consist of a more realistic domain and are designed to test the quality of ice models. For an even more realistic test case, we simulate the experiments E1 and E2 with a time-dependent surface. We obtain that approximately exact step sizes greatly reduce the necessary number of iterations for the Picard iteration and Newton's method with nearly no increase in the computation time for each iteration.

A novel chaos control method based on conjugate direction and adaptive step size

The Hasofer-Lind-Rackwitz-Fiessler method (HL-RF method) with the negative gradient direction as the iteration direction often oscillates or even does not converge during the iteration process due to the unreasonable control of the negative gradient direction. This greatly reduces the convergence and robustness of the HL-RF method. Chaos control method (CC method) is also a method with negative gradient direction as the iterative direction. However, in order to alleviate the oscillation problem caused by unreasonable control of negative gradient direction, it introduces a smaller iterative step size in the iterative process. Because its iterative step size is a fixed small value, the iterative efficiency of the CC method is not high. In view of the above problems, in order to reduce the iterative oscillation phenomenon and increase the iterative efficiency at the same time, it proposes a novel chaos control method based on conjugate direction and adaptive step size (CDCC method). Firstly, it uses the conjugate direction instead of the negative gradient direction, so as to fundamentally avoid the problem that the negative gradient direction is easy to cause oscillation. It effectively increases the convergence and robustness of the method. Secondly, it uses iteration information to adaptively control the iteration step size, increasing the computational efficiency of the method effectively. In addition, it also discusses the optimal value of the modification coefficient in the adaptive step size control. Finally, eight examples are given to show that the proposed method has obvious advantages in computational efficiency, convergence and robustness.

Numerical Analysis of Free Play-Induced Aeroelastic Phenomena: A Numerical Approach With Adaptive Step Size Control

This study presents a detailed numerical analysis of nonlinear aeroelastic behavior in a two degree of freedom (DOF) model, focusing on plunge and pitch motions and employing the continuation method (CM) with an adaptive step size control algorithm. The research incorporates free-play nonlinearity at the plunge hinge, a common structural nonlinearity in aeronautics that can induce detrimental limit cycle oscillations (LCOs) during flight. By examining three scenarios—linear response, unhindered plunge motion, and nonlinear stiffness behavior—the study assesses the effects of free play on flutter and LCO phenomena, including discontinuity-induced bifurcations like grazing bifurcation. Additionally, the study explores parameter variation for nonlinear flutter analysis, revealing the dynamics of grazing bifurcation and its impact on LCO behavior. The research also demonstrates the method’s superior accuracy in flutter speed estimation and mode-switching identification, despite higher computational demands. The findings underscore the diminishing influence of nonlinear free-play behavior on LCO amplitude, providing insights with significant implications for aeroelastic design and aircraft safety.

An Adaptive Learning Rate Deep Learning Optimizer Using Long and Short-Term Gradients Based on G–L Fractional-Order Derivative

AbstractDeep learning model is a multi-layered network structure, and the network parameters that evaluate the final performance of the model must be trained by a deep learning optimizer. In comparison to the mainstream optimizers that utilize integer-order derivatives reflecting only local information, fractional-order derivatives optimizers, which can capture global information, are gradually gaining attention. However, relying solely on the long-term estimated gradients computed from fractional-order derivatives while disregarding the influence of recent gradients on the optimization process can sometimes lead to issues such as local optima and slower optimization speeds. In this paper, we design an adaptive learning rate optimizer called AdaGL based on the Grünwald–Letnikov (G–L) fractional-order derivative. It changes the direction and step size of parameter updating dynamically according to the long-term and short-term gradients information, addressing the problem of falling into local minima or saddle points. To be specific, by utilizing the global memory of fractional-order calculus, we replace the gradient of parameter update with G–L fractional-order approximated gradient, making better use of the long-term curvature information in the past. Furthermore, considering that the recent gradient information often impacts the optimization phase significantly, we propose a step size control coefficient to adjust the learning rate in real-time. To compare the performance of the proposed AdaGL with the current advanced optimizers, we conduct several different deep learning tasks, including image classification on CNNs, node classification and graph classification on GNNs, image generation on GANs, and language modeling on LSTM. Extensive experimental results demonstrate that AdaGL achieves stable and fast convergence, excellent accuracy, and good generalization performance.

Direct method for solving systems of second order ordinary differential equations

In the paper a direct solver for systems of structurally partitioned second order differential equations is proposed. a general scheme of the method algorithmically oriented towards the particular structure of the system is presented. With the last stage reuse (also known as FSAL approach) an embedded pair of sixth and fourth order methods with just six stages is constructed, which provides an easy step-size control. Numerical comparison is made with the well-known Dormand — Prince method 5(4)7F having the same computation cost, showing the advantage of the proposed method.

Numerical design optimization of real size complex steel space frame structures using a novel adaptive cuckoo search method

AbstractIn this study, real size complex steel space frame structures are numerically designed to achieve optimal design weight. For this aim, standard cuckoo search algorithm is rectified through a newly proposed adaptive method over two fundamental algorithmic parameters of alien egg probability detection (Pa) and step size control (α) to overcome incompetency of trapping into local optima. Besides, to strengthen exploitation phase of newly proposed algorithm, it is boosted with greedy selection (GS). So, novel algorithm has more promising exploration and exploitation capabilities. An 8‐story, 1024‐member and a 20‐story, 1860‐member real size complex steel space frame structures are selected as design examples. Also, initially to verify the supremacy of novel algorithm, a well‐known welded beam is optimized as a benchmark structural design problem. Afterwards, the steel space frame structures are optimally designed via novel algorithm. Since ready steel section lists are utilized as selection pool for design variables, discrete programming problem is come into existence. The dead, live, snow, and wind design loads acting on frame structures are calculated in direction of ASCE 7‐05 provisions. Furthermore, the structural design constraints are determined from LRFD‐AISC specifications. Eventually, the newly proposed adaptive cuckoo search algorithm boosted with GS presents outstanding algorithmic performance.

Adaptive step-size control for global approximation of SDEs driven by countably dimensional Wiener process

In this paper, we deal with the global approximation of solutions of stochastic differential equations (SDEs) driven by countably dimensional Wiener process. Under certain regularity conditions imposed on the coefficients, we show lower bounds for exact asymptotic error behaviour. For that reason, we analyse separately two classes of admissible algorithms: based on equidistant, and possibly not equidistant meshes. Our results indicate that in both cases, decrease of any method error requires significant increase of the cost term, which is illustrated by the product of cost and error diverging to infinity. This is, however, not visible in the finite-dimensional case. In addition, we propose an implementable, path-independent Euler algorithm with adaptive step-size control, which is asymptotically optimal among algorithms using specified truncation levels of the underlying Wiener process. Our theoretical findings are supported by numerical simulation in Python language.

Automatic Time Step Control to Resolve Hydromechanically Driven Fault Reactivation, Spontaneous Nucleation, and Seismic Arrest

AbstractA physical understanding of the progression from flow‐driven (quasi‐static) poromechanical deformation to dynamic fault rupture is critical to the resilient operations of several engineering systems. These processes are bridged by a progression from fault reactivation to the spontaneous nucleation of unstable sliding. Toward addressing this challenge, novel automatic time step size control methods are developed to enable accurate and efficient simulation of these dynamics and transitions from the first principles. The controllers combine local models for discretization error and Coulomb failure conditions to automatically adjust the time step size across several orders of magnitude. The methods do not require additional empirical or theoretical input and can resolve the pre‐rupture, interseismic, and seismic periods to the allowed accuracy. The computational results reveal that the proposed methods automatically capture the onset of reactivation and nucleation for homogeneous and heterogeneous fields. Hydrodynamic and structural heterogeneity lead to disparate critical nucleation sizes compared to those predicted by theory. The results highlight its potential in predicting induced seismicity in realistic subsurface engineering systems and at practical scales.

On the accuracy, stability and computational efficiency of explicit last-stage diagonally implicit Runge–Kutta methods (ELDIRK) for the adaptive solution of phase-field problems

Runge–Kutta algorithms with adaptive step size control provide reliable tools for the solution of initial value problems. Most widespread within the vast family of Runge–Kutta methods are the diagonally implicit Runge–Kutta (DIRK) methods. In particular, the new low-order explicit last-stage diagonally implicit Runge–Kutta (ELDIRK) methods are investigated, combining implicit schemes with an additional explicit evaluation as an explicit last stage. This results in Butcher tableaus with two solutions of different convergence orders suitable for embedded methods, where the higher-order solution is achieved by additional explicit evaluations. Thus, the iterative solution of non-linear systems is omitted for the additional stage, presenting a major reduction in computational cost for the determination of a local error estimate. The key contribution is the application of the novel Butcher tableaus to phase-field problems, solved with the finite-element method, leading to substantial numerical investigations with an efficient approach for diagonally implicit Runge–Kutta schemes. The most important aspects are the extension towards fourth-order methods, the study of the convergence orders for the ELDIRK schemes, their respective stability regions and computational efficiency. A local error estimator is presented capturing the evolution of the phase-field problems, such that adaptive step size control for the new low-order embedded schemes based on an empirical approach for error estimation is achieved. A suitable parallel algorithm is presented with conclusive two-dimensional phase-field simulations based on a Kobayashi–Warren–Carter model including benchmarks for computational efficiency. The higher-order convergence suggested by the novel schemes is confirmed, and their effective results are demonstrated, resulting in a valuable semi-explicit addition to the family of Runge–Kutta time integration schemes.

Self-Adaptive PDLC Control Strategy With Smart Light Intensity Adjustment Using Photovoltaic-Thermoelectric Hybrid Energy Supply Technology

As an electronically controlled glass whose transmittance varies with voltage, polymer dispersed liquid crystal (PDLC) has the advantages of good privacy, high transmittance, fast response speed, and good heat shielding performance. It can be applied to electronically controlled building windows. However, PDLC needs to consume extra power, and electronically controlled PDLC has not yet been integrated with intelligent home environment control. In this paper, we proposed an adaptive variable voltage step-size fast control algorithm in Field Programmable Gate Array (FPGA)-based home environment control system, which is powered by photovoltaic-thermoelectric (PV-TEG) hybrid sources. The power supply voltage of the PDLC can be dynamically changed through the adaptive Pulse width modulation (PWM) duty cycle adjustment technology to realize intelligent transmittance control, thereby improving the indoor lighting environment. Experimental results show that by applying the proposed algorithm, the indoor temperature is reduced by 2°C, and the adaptive and stable regulation of the optimal indoor illumination of 500 Lux can be completed in less than 2.5 seconds. This paper provides a feasible solution for the low-energy intelligent integrated control and monitoring of the modern home environment.

Adaptive time stepping for generalized convolution quadrature

AbstractThrough the current work, we aim to improve the boundary element method (BEM) in acoustics. The proposed adaptivity is based on a time‐domain boundary element formulation. In this setting, an elegant method for solving the acoustic problem is through the transformation of the underlying partial differential equation on to the boundary of the domain as a space–time boundary integral equation. However, accurate modeling of temporal behavior is crucial, requiring careful consideration of the time step size to ensure numerical stability and capture the dynamics of the system accurately. The boundary integral equation in the time domain has a convolution in time and the generalized convolution quadrature method (gCQ) is introduced, providing a framework for numerically evaluating the convolution integral within the boundary integral equation. The gCQ method under consideration uses a higher order time stepping method from the Runge–Kutta family. The gCQ method allows for adaptive time stepping, enabling the refinement of the time step size to focus computational resources where they are most needed. The current work shows that an approach similar to that of the ordinary differential equations can be included in the gCQ to introduce an adaptive control of the time step size in the 3D boundary element formulation. Numerical experiments are conducted to validate the effectiveness of the proposed approach. The results demonstrate improved solution resolution, particularly in capturing steep changes and localized variations through the adaptive time stepping scheme. The proposed method shows promising performance, dynamically adjusting the time step size based on the evolving solution, thereby enabling efficient and accurate computations in BEM simulations of radiation phenomena.

Numerical investigations of new low‐order explicit last stage diagonal implicit Runge–Kutta schemes with the finite‐element method

AbstractInitial value problems can be solved efficiently by means of Runge–Kutta algorithms with adaptive step size control. Diagonally implicit Runge–Kutta (DIRK) methods are the most popular class among the diverse family of Runge–Kutta algorithms. In this paper, the novel class of low‐order explicit last‐stage diagonally implicit Runge–Kutta (ELDIRK) methods are explored, which combine implicit schemes with an additional explicit evaluation as an explicit last stage. ELDIRK Butcher tableaus are used to control embedded RK methods to obtain solutions of different orders. The lower‐order solution is obtained by classical implicit RK stages and the higher‐order solution is obtained by additional explicit evaluation. As a result, a significant reduction in computational cost is achieved by skipping the iterative solution of nonlinear systems for the additional step. The examination of the heat problem and the use of the innovative Butcher tableau in the finite‐element method are the main contributions of this work. Thus, it is possible to establish adaptive step size control for the new low‐order embedded methods based on an empirical method for error estimation. Two‐dimensional simulations are used to show an appropriate algorithm for the ELDIRK schemes. The new Runge–Kutta schemes' predictions of higher‐order convergence are confirmed, and their successful outcomes are illustrated.

On Kahan's automatic step-size control and an anadromic gradient descent iteration

Accelerating the basic gradient iteration is a critical issue in numerical optimization and deep-learning. Recently, William Kahan [Kahan, Automatic Step-Size Control for Minimization Iterations, Technical report, University of California, Berkeley CA, USA, 2019] proposed automatic step-size strategies for the gradient descent iteration without explicitly using any prior information of the Hessian; moreover, a new momentum-based gradient acceleration method, namely, an Anadromic Gradient Descent (AGD) iteration, was proposed. Besides the capability of accelerating the basic gradient descent iteration, AGD enables the iteration to return to the past iterates by just reversing steps in sign and order in the same updating formulation. Numerical performances of the automatic step-size strategies and AGD are claimed to be favourable. In this paper, for the quadratic model, through a revisit of some classical momentum-based gradient methods, we perform new analysis on the convergence and their optimal hyper-parameters. We also investigate the convergence behaviour of AGD with the optimal hyper-parameters and connect one Kahan's automatic step-size scheme with the long Barzilai–Borwein step-size. Numerical results are presented to reflect the theoretical convergence behaviours and demonstrate the practical performances of various momentum-based gradient methods.

Predicting wing-pylon-nacelle configuration flutter characteristics using adaptive continuation method

In this study, the aeroelastic response of a wing-pylon-nacelle system in subsonic and low supersonic flow regimes is analyzed using the continuation method in conjunction with an adaptive step size control algorithm. Idealizing the pylon and nacelle as a point mass, the computed effects of a standard structural analysis of the wing together with the pylon and nacelle are compared with those of a clean wing to build a reduced-order model for analysis. The aerodynamic forces relating to different reduced frequencies are assessed using the Doublet Lattice Method (DLM) in the subsonic flow regime and supersonic lifting surface theory relying on the unsteady linearized small-disturbance potential flow model in the low supersonic flow regime. The Rational Function Approximation (RFA) method is then utilized for the state-space formulation of the system equations, appended with the continuation method for flutter prediction. Thereafter, the linearized aeroelastic equations are resolved using the continuation method with adaptive step size, the results of which are matched with those obtained from the traditional p-k method to emphasize that the continuation method exhibits a distinct advantage in achieving better accuracy in estimating the flutter speed and identifying the “mode switching” phenomenon.