Small Step Size Research Articles

New approaches to numerical differentiation for second and third order

In this article, new numerical methods for calculation of second and third order derivatives are designed by using basic finite difference methods; forward, central and backward finite difference approaches. Those approaches are originally derived from the well-known Taylor series. Main advantage of new numerical formulas (named as Improved Backward Finite Difference Method, Improved Forward Finite Difference Method) is that they produce more accurate numerical results with smaller step size than the well-known backward and forward finite difference methods. For this purpose, some numerical examples are given to compare these new formulas with the traditional finite difference methods; backward and forward. The performance of the new methods in terms of error analysis and elapsed time for both second and third order derivative computations is also presented.

A Robust Salp Swarm Algorithm for Photovoltaic Maximum Power Point Tracking Under Partial Shading Conditions

Currently, numerous intelligent maximum power point tracking (MPPT) algorithms are capable of tackling the global optimization challenge of multi-peak photovoltaic output power under partial shading conditions, yet they often face issues such as slow convergence, low tracking precision, and substantial power fluctuations. To address these challenges, this paper introduces a hybrid algorithm that integrates an improved salp swarm algorithm (SSA) with the perturb and observe (P&O) method. Initially, the SSA is augmented with a dynamic spiral evolution mechanism and a Lévy flight strategy, expanding the search space and bolstering global search capabilities, which in turn enhances the tracking precision. Subsequently, the application of a Gaussian operator for distribution calculations allows for the adaptive adjustment of step sizes in each iteration, quickening convergence and diminishing power oscillations. Finally, the integration with P&O facilitates a meticulous search with a small step size, ensuring swift convergence and further mitigating post-convergence power oscillations. Both the simulations and the experimental results indicate that the proposed algorithm outperforms particle swarm optimization (PSO) and grey wolf optimization (GWO) in terms of convergence velocity, tracking precision, and the reduction in iteration power oscillation magnitude.

Stability Analysis of Split‐Step θ$$ \theta $$‐Milstein Scheme for Stochastic Delay Integro‐Differential Equations

ABSTRACTThis paper introduces and analyzes a split‐step ‐Milstein (SSTM) scheme specifically designed for stochastic delay integro‐differential equations (SDIDEs). Given reasonable assumptions regarding the drift term, diffusion term, and integral term, it is established that for small stepsize constraint, the SSTM method with is mean square exponential stable. Additionally, the result shows that also demonstrates the same stability under more stringent conditions on stepsize. Ultimately, two numerical experiments are presented to verify the validity of the obtained results.

A Semi-Analytical Solution to a Generalized Nonlinear Van Der Pol Equation in Plasma By MsDTM

This article examines the effectiveness of the multistage differential transform method (MsDTM) in solving equations with a very strong nonlinear term. It introduces MsDTM as a method for solving the generalized nonlinear Van der Pol equation, which features strong nonlinearity. The generalized nonlinear Van der Pol equation arises in plasma and describes the propagation of various nonlinear phenomena, such as wave propagation in astrophysical plasma. MsDTM demonstrates greater accuracy compared to other analytical and numerical methods, such as the 4th-order Runge-Kutta Method (4thRKM), due to its ability to enhance accuracy through two factors: the number of iterations and the time step size. Most numerical methods rely solely on reducing the time step size to improve accuracy, but for some types of equations, this requires an impractically small time step size, causing the method to fail. In contrast, MsDTM offers an additional means of improving accuracy by increasing the number of iterations. The paper successfully applies MsDTM to solve the Van der Pol equation and presents the results, demonstrating that the method is highly effective for equations with very strong nonlinearity.

Delayed rejection Hamiltonian Monte Carlo for sampling multiscale distributions

The efficiency of Hamiltonian Monte Carlo (HMC) can suffer when sampling a distribution with a wide range of length scales, because the small step sizes needed for stability in high-curvature regions are inefficient elsewhere. To address this we present a delayed rejection (DR) variant: if an initial HMC trajectory is rejected, we make one or more subsequent proposals each using a step size geometrically smaller than the last. To reduce the cost of DR approaches, we extend the standard delayed rejection to a probabilistic framework wherein we do not make multiple proposals at every rejection, but allow the probability of a retry to depend on the probability of accepting the previous proposal. We test the scheme in several sampling tasks, including statistical applications and multiscale model distributions such as Neal’s funnel. Delayed rejection enables sampling multiscale distributions for which standard approaches such as HMC fail to explore the tails, and improves performance five-fold over optimally-tuned HMC as measured by effective sample size per gradient evaluation. Even for simpler distributions, delayed rejection provides increased robustness to step size misspecification.

Snapback Repellers, Computational Chaos and Extreme Multistability in Discrete-Time Memristor Murali–Lakshmanan–Chua Circuit

Discretization schemes such as Euler method and Runge–Kutta techniques are extensively used to find approximate solutions of Continuous-Time (CT) dynamical system. While the approximation is good for small discretization step sizes, as pointed out by Lorenz, when the step size increases, computational chaos and computational instability are frequently observed, the former phenomenon being a precursor to the latter. By computational instability, it is meant that there is a blow up of trajectories for the Discrete-Time (DT) system. Computational chaos instead means that for certain step sizes, the DT system displays chaos while the CT counterpart is not chaotic. This paper studies the dynamics of a class of second-order maps obtained via the discretization of a Memristor Murali–Lakshmanan–Chua Circuit (MMLCC). The discretization, which is based on the DT Flux–Charge Analysis Method (FCAM), guarantees that the first integrals of a CT-MMLCC are preserved exactly for the DT system. Hence the dynamics of DT-MMLCC evolves on invariant manifolds and it is characterized by the coexistence of infinitely many different attractors (extreme multistability). The paper uses analytic techniques introduced by Marotto, based on the concept of snapback repellers and transverse homoclinic orbits, to study the chaotic behaviors of the maps. Regions of the parameter space are singled out where there exist snapback repellers for DT-MMLCC, thus implying that the maps display chaos in the Marotto and in the Li–Yorke sense. Since the corresponding CT-MMLCC does not display chaos, the observed chaos of DT-MMLCC is genuinely a consequence of the discretization scheme used in the paper, i.e. it can be actually considered as computational chaos. It is also verified that computational chaos is a precursor to computational instability of the DT-MMLCC maps. Finally, the paper analyzes the effect on chaos obtained by changing the invariant manifold where the dynamics evolves.

Minimally Distorted Adversarial Images with a Step-Adaptive Iterative Fast Gradient Sign Method

The safety and robustness of convolutional neural networks (CNNs) have raised increasing concerns, especially in safety-critical areas, such as medical applications. Although CNNs are efficient in image classification, their predictions are often sensitive to minor, for human observers, invisible modifications of the image. Thus, a modified, corrupted image can be visually equal to the legitimate image for humans but fool the CNN and make a wrong prediction. Such modified images are called adversarial images throughout this paper. A popular method to generate adversarial images is backpropagating the loss gradient to modify the input image. Usually, only the direction of the gradient and a given step size were used to determine the perturbations (FGSM, fast gradient sign method), or the FGSM is applied multiple times to craft stronger perturbations that change the model classification (i-FGSM). On the contrary, if the step size is too large, the minimum perturbation of the image may be missed during the gradient search. To seek exact and minimal input images for a classification change, in this paper, we suggest starting the FGSM with a small step size and adapting the step size with iterations. A few decay algorithms were taken from the literature for comparison with a novel approach based on an index tracking the loss status. In total, three tracking functions were applied for comparison. The experiments show our loss adaptive decay algorithms could find adversaries with more than a 90% success rate while generating fewer perturbations to fool the CNNs.

Handling effectively the rejected stages in Runge–Kutta pairs implementation

Runge–Kutta (RK) pairs are widely used for numerically solving initial value problems (IVPs), but dealing with step rejections during integration is a common occurrence. Conventionally, when a step is rejected, all calculations made during that step are discarded, and a completely new set of computations is initiated. In our research, we propose a method to address this inefficiency by repurposing the previously computed RK stages from rejected steps. Our primary focus is on the renowned RKF45 pair, consisting of fifth‐ and fourth‐order methods. When a step rejection occurs, we leverage the stages computed in prior steps and introduce just three additional stages. These stages are then used to evaluate the results with a smaller step size. This approach effectively reduces computational costs in various challenging IVPs where RK algorithms with different step sizes encounter difficulties

A 16 Gb/s serial link transceiver with active inductor based CTLE and 3-tap DFE in 12-nm FinFET CMOS

This paper presents a 16 Gb/s serial link transceiver implemented in 12-nm FinFET CMOS technology. The equalization scheme incorporates a 3-tap feed-forward equalizer (FFE) in the transmitter, a 5-stage continuous time linear equalizer (CTLE), and a 3-tap decision-feedback equalizer (DFE) in the receiver. The proposed FFE structure offers small step size and flexible configuration. The CTLE's high-frequency boost is augmented by integrating an active inductor. A high-accuracy clock and data recovery (CDR) circuit, featuring a cascaded two-step analog phase interpolator (PI) structure, is developed for data retiming. Occupying a footprint of 0.26 mm × 0.63 mm per lane, the transceiver test chip is packaged in a flip chip ball grid array (FC-BGA) module. Measurement results demonstrate a horizontal eye opening of 38 % at a bit error rate (BER) of 10−12 over a channel with 28 dB loss at 8 GHz. Moreover, the power consumption is 68.85 mW per lane at 16 Gb/s.

Real-Time Hardware Emulation of Microgrid Forming Wireless Power Transfer Systems

The prevalence of wireless charging approaches has manifested a grid-supporting potential which can be evaluated by hardware-based methodologies. In this work, real-time emulation of a bidirectional wireless power transfer (WPT) system capable of DC microgrid formation is investigated on the field-programmable gate array (FPGA). Following a detailed analysis of the transfer characteristics of the dual-active bridge with a series-compensated resonant tank in between, a unified control scheme utilizing a phase-shift strategy is proposed for a flexible voltage or current regulation, thereby enabling the DC microgrid-forming capability. Electromagnetic transient modeling is then carried out so that an accurate digital emulation platform is feasible for prototyping. The fact that the WPT has a high frequency compels a small computation step size, which poses a dramatic challenge to real-time execution. A partition-iteration approach is therefore proposed for matrix dimension reduction which ultimately results in an alleviated processing burden. In the meantime, the parallelism of configurable logic blocks and the pipelined architecture of the FPGA are explored to achieve a low hardware latency. The analytical models, as well as the proposed control method, are validated experimentally, and then real-time hardware emulation of a DC microgrid consisting of WPT systems is performed for an integration study.

A new 2D ESPH bedload sediment transport model for rapidly varied flows over mobile beds

A 2D Eulerian meshless bedload sediment transport model is developed using smoothed particle hydrodynamics (SPH) to simulate rapidly varied flows over mobile beds. In the developed model, we adopt a weakly coupled numerical approach to explicitly solve the governing equations, including 2D shallow water equations for fluid flow motion and the Exner equation for bed sediment movement at the same time step. A defined virtual bedload velocity in a weakly coupled approach is involved in the calculation of time step sizes. However, nonphysical virtual bedload velocities, which often occur in cases with nearly flat beds, require extremely small time step sizes. A formulation of the lower and upper bounds of information propagation speeds for an HLLC approximate Riemann solver is thus proposed to remedy the problem. Four case studies involving dam break flows in prismatic and erodible channels, knickpoint migration, dam erosion due to overtopping flow and dam-break flow in an erodible channel with an abrupt expansion are adopted to validate the developed model. In addition, to compare the performances of different particle interaction configurations under Cartesian uniform particle arrangements, four and eight interacting particles are obtained by changing the smoothing length. Against the measured results, the case of eight interacting particles shows more accurate predictions of erosion peaks along the measured cross-sections because the lateral flows are significant. The good agreement between the simulated and measured results shows that the developed 2D Eulerian SPH bedload sediment transport model with eight interacting particles is well suited for simulating complicated flow-induced bed erosion.

A Multi-Rate Simulation Strategy Based on the Modified Time-Domain Simulation Method and Multi-Area Data Exchange Method of Power Systems

Accurate modeling for power-electronic devices requires power systems to be simulated with considerably small step sizes (typically several microseconds), which causes unnecessary computational burden and reduces efficiency, especially for large-scale power systems. To achieve a balance between simulation precision and efficiency, this paper introduces an innovative multi-rate interface strategy based on the modified time-domain simulation (TDS) method and multi-area data exchange method. The modified TDS method transforms the initialization process into exchange of electric data among different subsystems, while the multi-area data exchange method is able to ensure numerical stability and simulation universality during the multi-rate simulation. The proposed strategy provides a robust interface that allows different subsystems to be engaged in simulations with different step sizes while exchanging data. To validate this strategy, simulations on an integrated system of IEEE 14-bus and 33-bus systems is conducted. In addition, the strategy is further applied to a real-world scenario of the subsystem in the Guangxi Power Grid in China. Analysis of the results indicates that the proposed multi-rate fast simulation strategy can significantly boost simulation efficiency while maintaining accuracy, which marks a notable improvement compared with the traditional single step size simulation.

Single image super-resolution with denoising diffusion GANS

Single image super-resolution (SISR) refers to the reconstruction from the corresponding low-resolution (LR) image input to a high-resolution (HR) image. However, since a single low-resolution image corresponds to multiple high-resolution images, this is an ill-posed problem. In recent years, generative model-based SISR methods have outperformed conventional SISR methods in performance. However, the SISR methods based on GAN, VAE, and Flow have the problems of unstable training, low sampling quality, and expensive computational cost. These models also struggle to achieve the trifecta of diverse, high-quality, and fast sampling. In particular, denoising diffusion probabilistic models have shown impressive variety and high quality of samples, but their expensive sampling cost prevents them from being well applied in the real world. In this paper, we investigate the fundamental reason for the slow sampling speed of the SISR method based on the diffusion model lies in the Gaussian assumption used in the previous diffusion model, which is only applicable for small step sizes. We propose a new Single Image Super-Resolution with Denoising Diffusion GANS (SRDDGAN) to achieve large-step denoising, sample diversity, and training stability. Our approach combines denoising diffusion models with GANs to generate images conditionally, using a multimodal conditional GAN to model each denoising step. SRDDGAN outperforms existing diffusion model-based methods regarding PSNR and perceptual quality metrics, while the added latent variable Z solution explores the diversity of likely HR spatial domain. Notably, the SRDDGAN model infers nearly 11 times faster than diffusion-based SR3, making it a more practical solution for real-world applications.

Semiglobal exponential stability of the discrete-time Arrow-Hurwicz-Uzawa primal-dual algorithm for constrained optimization

We consider the discrete-time Arrow-Hurwicz-Uzawa primal-dual algorithm, also known as the first-order Lagrangian method, for constrained optimization problems involving a smooth strongly convex cost and smooth convex constraints. We prove that, for every given compact set of initial conditions, there always exists a sufficiently small stepsize guaranteeing exponential stability of the optimal primal-dual solution of the problem with a domain of attraction including the initialization set. Inspired by the analysis of nonlinear oscillators, the stability proof is based on a non-quadratic Lyapunov function including a nonlinear cross term.

The solution for singularly perturbed differential-difference equation with boundary layers at both ends by a numerical integration method

This paper presents a numerical integration method for the solution of ‘Singularly Perturbed Differential-Difference Equations' having dual layers. It is well known that when we use already existing numerical methods to solve such problems, we get oscillatory or unsatisfactory results unless we take a very small step size, which is time-consuming and costly. To get a numerical solution for such a problem, first, the delay and advanced parameters present in the SPDDE are approximated by Taylor’s series to get an equivalent ‘Singularly Perturbed Differential Equation' of second order. Second, an asymptotically equivalent first-order differential equation is obtained from SPDE using Taylor’s transformation. Composite Simpson’s 1/3 rule is implemented to get a three-term recurrence relation. The Thomas algorithm is applied to get the solution of the tri-diagonal system of equations. Several model examples are tested and it was found that the numerical solution approximates the available/exact solution very well.

Energy-Preserving/Group-Preserving Schemes for Depicting Nonlinear Vibrations of Multi-Coupled Duffing Oscillators

In the paper, we first develop a novel automatically energy-preserving scheme (AEPS) for the undamped and unforced single and multi-coupled Duffing equations by recasting them to the Lie-type systems of ordinary differential equations. The AEPS can automatically preserve the energy to be a constant value in a long-term free vibration behavior. The analytical solution of a special Duffing–van der Pol equation is compared with that computed by the novel group-preserving scheme (GPS) which has fourth-order accuracy. The main novelty is that we constructed the quadratic forms of the energy equations, the Lie-algebras and Lie-groups for the multi-coupled Duffing oscillator system. Then, we extend the GPS to the damped and forced Duffing equations. The corresponding algorithms are developed, which are effective to depict the long term nonlinear vibration behaviors of the multi-coupled Duffing oscillators with an accuracy of O(h4) for a small time stepsize h.

On the correlation between the noise and a priori error vectors for standard and augmented affine projection algorithms

This paper analyzes the correlation matrix between the a priori error and measurement noise vectors for standard and augmented affine projection algorithms using a unified approach. This correlation stems from the dependence between the filter tap estimates and the noise samples, and has a strong influence on the mean square behavior of the algorithm. We show that the correlation matrix is upper triangular, and compute the diagonal elements in closed form, showing that they are independent of the input process statistics. Also, for white inputs we show that the matrix is fully diagonal. These results are valid in the transient and steady states, considering a possibly variable step-size. Our only assumption is that the filter order is large compared to its projection order and that the input signal is stationary. Using these results, we perform a steady-state analysis for small step size and provide a new simple closed-form expression for the mean-square error, which has comparable or better accuracy to many preexisting expressions, and is much simpler to compute. Finally, we also obtain expressions for the steady-state energy of the other components of the error vector.

Dynamic analysis and Chaos control in a discrete predator-prey model with Holling type IV and nonlinear harvesting

In this paper, we investigate the dynamical behavior of a two dimensional discretized prey predator system. The model is formulated in terms of difference equations and derived by using the higher-order implicit Runge Kutta method with a very small step size to attain a discrete time version of its continuous counterpart. The existence of fixed points as well as their local asymptotic stability are proved. Further, it is shown that the model experiences Neimark-Sacker bifurcation (NSB for short) and the periodic doubling bifurcation (PDB) in a small neighborhood of the coexistence fixed point under certain parametric conditions. This analysis utilizes bifurcation theory and the center manifold theorem. The chaos influenced by NSB is stabilized. Finally, we use numerical simulations and computer analysis to check our theories and show more complex behaviors.

Mathematical Modelling of Tuberculosis and Diabetes Co-Infection Using the Non- Standard Finite Difference Scheme

One of the major health challenge facing Africa and in particular, Kenya is the risk of Tuberculosis and Diabetes. To understand the dynamics of this, a nine compartmental model for tuberculosis-diabetes co-infection is formulated. The Non-standard finite difference Scheme (NSFD) of the model is formulated from the first-order ordinary differential equations (ode) to avoid full implicit schemes that are computationally expensive. The overly small step sizes in NSFD give the user autonomy in controlling the accuracy of the results, making it suitable for disease control applications. Numerical simulations with different step sizes of the NSFD for the TB-Diabetes model are carried out to find the optimal step size, h. A comparison of the best resultant numerical simulation based on optimal h in NSDF indicates NSFD gives better results when compared with the corresponding first-order ode. The phase-plane analysis revealed that the NSFD formulated for tuberculosis and diabetes co-infection is generally asymptotically stable. Future studies should consider formulating the proposed model with varied control parameters such as medication to compare the results with those from first-order ode.

A time-consistent stabilized finite element method for fluids with applications to hemodynamics

Several finite element methods for simulating incompressible flows rely on the streamline upwind Petrov–Galerkin stabilization (SUPG) term, which is weighted by tau _{text{SUPG}}. The conventional formulation of tau _{text{SUPG}} includes a constant that depends on the time step size, producing an overall method that becomes exceedingly less accurate as the time step size approaches zero. In practice, such method inconsistency introduces significant error in the solution, especially in cardiovascular simulations, where small time step sizes may be required to resolve multiple scales of the blood flow. To overcome this issue, we propose a consistent method that is based on a new definition of tau _{text{SUPG}}. This method, which can be easily implemented on top of an existing streamline upwind Petrov–Galerkin and pressure stabilizing Petrov–Galerkin method, involves the replacement of the time step size in tau _{text{SUPG}} with a physical time scale. This time scale is calculated in a simple operation once every time step for the entire computational domain from the ratio of the L2-norm of the acceleration and the velocity. The proposed method is compared against the conventional method using four cases: a steady pipe flow, a blood flow through vascular anatomy, an external flow over a square obstacle, and a fluid–structure interaction case involving an oscillatory flexible beam. These numerical experiments, which are performed using linear interpolation functions, show that the proposed formulation eliminates the inconsistency issue associated with the conventional formulation in all cases. While the proposed method is slightly more costly than the conventional method, it significantly reduces the error, particularly at small time step sizes. For the pipe flow where an exact solution is available, we show the conventional method can over-predict the pressure drop by a factor of three. This large error is almost completely eliminated by the proposed formulation, dropping to approximately 1% for all time step sizes and Reynolds numbers considered.