Small Step Size Research Articles

Delayed rejection Hamiltonian Monte Carlo for sampling multiscale distributions

The efficiency of Hamiltonian Monte Carlo (HMC) can suffer when sampling a distribution with a wide range of length scales, because the small step sizes needed for stability in high-curvature regions are inefficient elsewhere. To address this we present a delayed rejection (DR) variant: if an initial HMC trajectory is rejected, we make one or more subsequent proposals each using a step size geometrically smaller than the last. To reduce the cost of DR approaches, we extend the standard delayed rejection to a probabilistic framework wherein we do not make multiple proposals at every rejection, but allow the probability of a retry to depend on the probability of accepting the previous proposal. We test the scheme in several sampling tasks, including statistical applications and multiscale model distributions such as Neal’s funnel. Delayed rejection enables sampling multiscale distributions for which standard approaches such as HMC fail to explore the tails, and improves performance five-fold over optimally-tuned HMC as measured by effective sample size per gradient evaluation. Even for simpler distributions, delayed rejection provides increased robustness to step size misspecification.

Snapback Repellers, Computational Chaos and Extreme Multistability in Discrete-Time Memristor Murali–Lakshmanan–Chua Circuit

Discretization schemes such as Euler method and Runge–Kutta techniques are extensively used to find approximate solutions of Continuous-Time (CT) dynamical system. While the approximation is good for small discretization step sizes, as pointed out by Lorenz, when the step size increases, computational chaos and computational instability are frequently observed, the former phenomenon being a precursor to the latter. By computational instability, it is meant that there is a blow up of trajectories for the Discrete-Time (DT) system. Computational chaos instead means that for certain step sizes, the DT system displays chaos while the CT counterpart is not chaotic. This paper studies the dynamics of a class of second-order maps obtained via the discretization of a Memristor Murali–Lakshmanan–Chua Circuit (MMLCC). The discretization, which is based on the DT Flux–Charge Analysis Method (FCAM), guarantees that the first integrals of a CT-MMLCC are preserved exactly for the DT system. Hence the dynamics of DT-MMLCC evolves on invariant manifolds and it is characterized by the coexistence of infinitely many different attractors (extreme multistability). The paper uses analytic techniques introduced by Marotto, based on the concept of snapback repellers and transverse homoclinic orbits, to study the chaotic behaviors of the maps. Regions of the parameter space are singled out where there exist snapback repellers for DT-MMLCC, thus implying that the maps display chaos in the Marotto and in the Li–Yorke sense. Since the corresponding CT-MMLCC does not display chaos, the observed chaos of DT-MMLCC is genuinely a consequence of the discretization scheme used in the paper, i.e. it can be actually considered as computational chaos. It is also verified that computational chaos is a precursor to computational instability of the DT-MMLCC maps. Finally, the paper analyzes the effect on chaos obtained by changing the invariant manifold where the dynamics evolves.

Minimally Distorted Adversarial Images with a Step-Adaptive Iterative Fast Gradient Sign Method

The safety and robustness of convolutional neural networks (CNNs) have raised increasing concerns, especially in safety-critical areas, such as medical applications. Although CNNs are efficient in image classification, their predictions are often sensitive to minor, for human observers, invisible modifications of the image. Thus, a modified, corrupted image can be visually equal to the legitimate image for humans but fool the CNN and make a wrong prediction. Such modified images are called adversarial images throughout this paper. A popular method to generate adversarial images is backpropagating the loss gradient to modify the input image. Usually, only the direction of the gradient and a given step size were used to determine the perturbations (FGSM, fast gradient sign method), or the FGSM is applied multiple times to craft stronger perturbations that change the model classification (i-FGSM). On the contrary, if the step size is too large, the minimum perturbation of the image may be missed during the gradient search. To seek exact and minimal input images for a classification change, in this paper, we suggest starting the FGSM with a small step size and adapting the step size with iterations. A few decay algorithms were taken from the literature for comparison with a novel approach based on an index tracking the loss status. In total, three tracking functions were applied for comparison. The experiments show our loss adaptive decay algorithms could find adversaries with more than a 90% success rate while generating fewer perturbations to fool the CNNs.

Handling effectively the rejected stages in Runge–Kutta pairs implementation

Runge–Kutta (RK) pairs are widely used for numerically solving initial value problems (IVPs), but dealing with step rejections during integration is a common occurrence. Conventionally, when a step is rejected, all calculations made during that step are discarded, and a completely new set of computations is initiated. In our research, we propose a method to address this inefficiency by repurposing the previously computed RK stages from rejected steps. Our primary focus is on the renowned RKF45 pair, consisting of fifth‐ and fourth‐order methods. When a step rejection occurs, we leverage the stages computed in prior steps and introduce just three additional stages. These stages are then used to evaluate the results with a smaller step size. This approach effectively reduces computational costs in various challenging IVPs where RK algorithms with different step sizes encounter difficulties

A 16 Gb/s serial link transceiver with active inductor based CTLE and 3-tap DFE in 12-nm FinFET CMOS

This paper presents a 16 Gb/s serial link transceiver implemented in 12-nm FinFET CMOS technology. The equalization scheme incorporates a 3-tap feed-forward equalizer (FFE) in the transmitter, a 5-stage continuous time linear equalizer (CTLE), and a 3-tap decision-feedback equalizer (DFE) in the receiver. The proposed FFE structure offers small step size and flexible configuration. The CTLE's high-frequency boost is augmented by integrating an active inductor. A high-accuracy clock and data recovery (CDR) circuit, featuring a cascaded two-step analog phase interpolator (PI) structure, is developed for data retiming. Occupying a footprint of 0.26 mm × 0.63 mm per lane, the transceiver test chip is packaged in a flip chip ball grid array (FC-BGA) module. Measurement results demonstrate a horizontal eye opening of 38 % at a bit error rate (BER) of 10−12 over a channel with 28 dB loss at 8 GHz. Moreover, the power consumption is 68.85 mW per lane at 16 Gb/s.

Real-Time Hardware Emulation of Microgrid Forming Wireless Power Transfer Systems

The prevalence of wireless charging approaches has manifested a grid-supporting potential which can be evaluated by hardware-based methodologies. In this work, real-time emulation of a bidirectional wireless power transfer (WPT) system capable of DC microgrid formation is investigated on the field-programmable gate array (FPGA). Following a detailed analysis of the transfer characteristics of the dual-active bridge with a series-compensated resonant tank in between, a unified control scheme utilizing a phase-shift strategy is proposed for a flexible voltage or current regulation, thereby enabling the DC microgrid-forming capability. Electromagnetic transient modeling is then carried out so that an accurate digital emulation platform is feasible for prototyping. The fact that the WPT has a high frequency compels a small computation step size, which poses a dramatic challenge to real-time execution. A partition-iteration approach is therefore proposed for matrix dimension reduction which ultimately results in an alleviated processing burden. In the meantime, the parallelism of configurable logic blocks and the pipelined architecture of the FPGA are explored to achieve a low hardware latency. The analytical models, as well as the proposed control method, are validated experimentally, and then real-time hardware emulation of a DC microgrid consisting of WPT systems is performed for an integration study.

A new 2D ESPH bedload sediment transport model for rapidly varied flows over mobile beds

A 2D Eulerian meshless bedload sediment transport model is developed using smoothed particle hydrodynamics (SPH) to simulate rapidly varied flows over mobile beds. In the developed model, we adopt a weakly coupled numerical approach to explicitly solve the governing equations, including 2D shallow water equations for fluid flow motion and the Exner equation for bed sediment movement at the same time step. A defined virtual bedload velocity in a weakly coupled approach is involved in the calculation of time step sizes. However, nonphysical virtual bedload velocities, which often occur in cases with nearly flat beds, require extremely small time step sizes. A formulation of the lower and upper bounds of information propagation speeds for an HLLC approximate Riemann solver is thus proposed to remedy the problem. Four case studies involving dam break flows in prismatic and erodible channels, knickpoint migration, dam erosion due to overtopping flow and dam-break flow in an erodible channel with an abrupt expansion are adopted to validate the developed model. In addition, to compare the performances of different particle interaction configurations under Cartesian uniform particle arrangements, four and eight interacting particles are obtained by changing the smoothing length. Against the measured results, the case of eight interacting particles shows more accurate predictions of erosion peaks along the measured cross-sections because the lateral flows are significant. The good agreement between the simulated and measured results shows that the developed 2D Eulerian SPH bedload sediment transport model with eight interacting particles is well suited for simulating complicated flow-induced bed erosion.

A Multi-Rate Simulation Strategy Based on the Modified Time-Domain Simulation Method and Multi-Area Data Exchange Method of Power Systems

Accurate modeling for power-electronic devices requires power systems to be simulated with considerably small step sizes (typically several microseconds), which causes unnecessary computational burden and reduces efficiency, especially for large-scale power systems. To achieve a balance between simulation precision and efficiency, this paper introduces an innovative multi-rate interface strategy based on the modified time-domain simulation (TDS) method and multi-area data exchange method. The modified TDS method transforms the initialization process into exchange of electric data among different subsystems, while the multi-area data exchange method is able to ensure numerical stability and simulation universality during the multi-rate simulation. The proposed strategy provides a robust interface that allows different subsystems to be engaged in simulations with different step sizes while exchanging data. To validate this strategy, simulations on an integrated system of IEEE 14-bus and 33-bus systems is conducted. In addition, the strategy is further applied to a real-world scenario of the subsystem in the Guangxi Power Grid in China. Analysis of the results indicates that the proposed multi-rate fast simulation strategy can significantly boost simulation efficiency while maintaining accuracy, which marks a notable improvement compared with the traditional single step size simulation.

Single image super-resolution with denoising diffusion GANS

Single image super-resolution (SISR) refers to the reconstruction from the corresponding low-resolution (LR) image input to a high-resolution (HR) image. However, since a single low-resolution image corresponds to multiple high-resolution images, this is an ill-posed problem. In recent years, generative model-based SISR methods have outperformed conventional SISR methods in performance. However, the SISR methods based on GAN, VAE, and Flow have the problems of unstable training, low sampling quality, and expensive computational cost. These models also struggle to achieve the trifecta of diverse, high-quality, and fast sampling. In particular, denoising diffusion probabilistic models have shown impressive variety and high quality of samples, but their expensive sampling cost prevents them from being well applied in the real world. In this paper, we investigate the fundamental reason for the slow sampling speed of the SISR method based on the diffusion model lies in the Gaussian assumption used in the previous diffusion model, which is only applicable for small step sizes. We propose a new Single Image Super-Resolution with Denoising Diffusion GANS (SRDDGAN) to achieve large-step denoising, sample diversity, and training stability. Our approach combines denoising diffusion models with GANs to generate images conditionally, using a multimodal conditional GAN to model each denoising step. SRDDGAN outperforms existing diffusion model-based methods regarding PSNR and perceptual quality metrics, while the added latent variable Z solution explores the diversity of likely HR spatial domain. Notably, the SRDDGAN model infers nearly 11 times faster than diffusion-based SR3, making it a more practical solution for real-world applications.

Semiglobal exponential stability of the discrete-time Arrow-Hurwicz-Uzawa primal-dual algorithm for constrained optimization

We consider the discrete-time Arrow-Hurwicz-Uzawa primal-dual algorithm, also known as the first-order Lagrangian method, for constrained optimization problems involving a smooth strongly convex cost and smooth convex constraints. We prove that, for every given compact set of initial conditions, there always exists a sufficiently small stepsize guaranteeing exponential stability of the optimal primal-dual solution of the problem with a domain of attraction including the initialization set. Inspired by the analysis of nonlinear oscillators, the stability proof is based on a non-quadratic Lyapunov function including a nonlinear cross term.

The solution for singularly perturbed differential-difference equation with boundary layers at both ends by a numerical integration method

This paper presents a numerical integration method for the solution of ‘Singularly Perturbed Differential-Difference Equations' having dual layers. It is well known that when we use already existing numerical methods to solve such problems, we get oscillatory or unsatisfactory results unless we take a very small step size, which is time-consuming and costly. To get a numerical solution for such a problem, first, the delay and advanced parameters present in the SPDDE are approximated by Taylor’s series to get an equivalent ‘Singularly Perturbed Differential Equation' of second order. Second, an asymptotically equivalent first-order differential equation is obtained from SPDE using Taylor’s transformation. Composite Simpson’s 1/3 rule is implemented to get a three-term recurrence relation. The Thomas algorithm is applied to get the solution of the tri-diagonal system of equations. Several model examples are tested and it was found that the numerical solution approximates the available/exact solution very well.

Energy-Preserving/Group-Preserving Schemes for Depicting Nonlinear Vibrations of Multi-Coupled Duffing Oscillators

In the paper, we first develop a novel automatically energy-preserving scheme (AEPS) for the undamped and unforced single and multi-coupled Duffing equations by recasting them to the Lie-type systems of ordinary differential equations. The AEPS can automatically preserve the energy to be a constant value in a long-term free vibration behavior. The analytical solution of a special Duffing–van der Pol equation is compared with that computed by the novel group-preserving scheme (GPS) which has fourth-order accuracy. The main novelty is that we constructed the quadratic forms of the energy equations, the Lie-algebras and Lie-groups for the multi-coupled Duffing oscillator system. Then, we extend the GPS to the damped and forced Duffing equations. The corresponding algorithms are developed, which are effective to depict the long term nonlinear vibration behaviors of the multi-coupled Duffing oscillators with an accuracy of O(h4) for a small time stepsize h.

On the correlation between the noise and a priori error vectors for standard and augmented affine projection algorithms

This paper analyzes the correlation matrix between the a priori error and measurement noise vectors for standard and augmented affine projection algorithms using a unified approach. This correlation stems from the dependence between the filter tap estimates and the noise samples, and has a strong influence on the mean square behavior of the algorithm. We show that the correlation matrix is upper triangular, and compute the diagonal elements in closed form, showing that they are independent of the input process statistics. Also, for white inputs we show that the matrix is fully diagonal. These results are valid in the transient and steady states, considering a possibly variable step-size. Our only assumption is that the filter order is large compared to its projection order and that the input signal is stationary. Using these results, we perform a steady-state analysis for small step size and provide a new simple closed-form expression for the mean-square error, which has comparable or better accuracy to many preexisting expressions, and is much simpler to compute. Finally, we also obtain expressions for the steady-state energy of the other components of the error vector.

Mathematical Modelling of Tuberculosis and Diabetes Co-Infection Using the Non- Standard Finite Difference Scheme

One of the major health challenge facing Africa and in particular, Kenya is the risk of Tuberculosis and Diabetes. To understand the dynamics of this, a nine compartmental model for tuberculosis-diabetes co-infection is formulated. The Non-standard finite difference Scheme (NSFD) of the model is formulated from the first-order ordinary differential equations (ode) to avoid full implicit schemes that are computationally expensive. The overly small step sizes in NSFD give the user autonomy in controlling the accuracy of the results, making it suitable for disease control applications. Numerical simulations with different step sizes of the NSFD for the TB-Diabetes model are carried out to find the optimal step size, h. A comparison of the best resultant numerical simulation based on optimal h in NSDF indicates NSFD gives better results when compared with the corresponding first-order ode. The phase-plane analysis revealed that the NSFD formulated for tuberculosis and diabetes co-infection is generally asymptotically stable. Future studies should consider formulating the proposed model with varied control parameters such as medication to compare the results with those from first-order ode.

A time-consistent stabilized finite element method for fluids with applications to hemodynamics

Several finite element methods for simulating incompressible flows rely on the streamline upwind Petrov–Galerkin stabilization (SUPG) term, which is weighted by tau _{text{SUPG}}. The conventional formulation of tau _{text{SUPG}} includes a constant that depends on the time step size, producing an overall method that becomes exceedingly less accurate as the time step size approaches zero. In practice, such method inconsistency introduces significant error in the solution, especially in cardiovascular simulations, where small time step sizes may be required to resolve multiple scales of the blood flow. To overcome this issue, we propose a consistent method that is based on a new definition of tau _{text{SUPG}}. This method, which can be easily implemented on top of an existing streamline upwind Petrov–Galerkin and pressure stabilizing Petrov–Galerkin method, involves the replacement of the time step size in tau _{text{SUPG}} with a physical time scale. This time scale is calculated in a simple operation once every time step for the entire computational domain from the ratio of the L2-norm of the acceleration and the velocity. The proposed method is compared against the conventional method using four cases: a steady pipe flow, a blood flow through vascular anatomy, an external flow over a square obstacle, and a fluid–structure interaction case involving an oscillatory flexible beam. These numerical experiments, which are performed using linear interpolation functions, show that the proposed formulation eliminates the inconsistency issue associated with the conventional formulation in all cases. While the proposed method is slightly more costly than the conventional method, it significantly reduces the error, particularly at small time step sizes. For the pipe flow where an exact solution is available, we show the conventional method can over-predict the pressure drop by a factor of three. This large error is almost completely eliminated by the proposed formulation, dropping to approximately 1% for all time step sizes and Reynolds numbers considered.

Implicit coupling methods for nonlinear interactions between a large‐deformable hyperelastic solid and a viscous acoustic fluid of infinite extent

AbstractThis paper addresses the challenges in studying the interaction between high‐intensity sound waves and large‐deformable hyperelastic solids, which are characterized by nonlinearities of the hyperelastic material, the finite‐amplitude acoustic wave, and the large‐deformable fluid–solid interface. An implicit coupling method is proposed for predicting nonlinear structural‐acoustic responses of the large‐deformable hyperelastic solid submerged in a compressible viscous fluid of infinite extent. An arbitrary Lagrangian–Eulerian (ALE) formulation based on an unsplit complex‐frequency‐shifted perfectly matched layer method is developed for long‐time simulation of the nonlinear acoustic wave propagation without exhibiting long‐time instabilities. The solid and acoustic fluid domains are discretized using the finite element method, and two different options of staggered implicit coupling procedures for nonlinear structural‐acoustic interactions are developed. Theoretical formulations for stability analysis of the implicit methods are provided. The accuracy, robustness, and convergence properties of the proposed methods are evaluated by a benchmark problem, that is, a hyperelastic rod interacting with finite‐amplitude acoustic waves. The numerical results substantiate that the present methods are able to provide long‐time steady‐state solutions for a nonlinear coupled hyperelastic solid and viscous acoustic fluid system without numerical constraints of small time step sizes and long‐time instabilities. The methods are applied to investigate nonlinear dynamic behaviors of coupled hyperelastic elliptical ring and acoustic fluid systems. Physical insights into 2:1 and 4:2:1 internal resonances of the hyperelastic elliptical ring and period‐doubling bifurcations of the structural and acoustic responses of the system are provided.

Development of a Composite Implicit Time Integration Scheme for Three-Dimensional Discontinuous Deformation Analysis

Discontinuous deformation analysis (DDA) is a discontinuum-based and implicit method for investigating the deformational behavior of block systems. The constant acceleration integration (CAI) scheme characterized by unconditional stability is employed in the traditional DDA. In this study, the problems of the CAI scheme regarding the time step in DDA are pointed out. A too large or too small time step size adopted in the CAI scheme will have adverse effects on the DDA computation. To overcome the weaknesses, an alternative composite implicit time integration (CITI) scheme, which is a combination of the trapezoidal rule and the three-point backward Euler method, is implemented in the three-dimensional (3D) DDA method. Verification examples and slope numerical simulations are presented to illustrate the accuracy and effectiveness of the proposed methodology. The results showed that the CITI scheme can overcome the numerical error caused by the large time step size, and the algorithm damping is closely related to the time step size and the selected splitting parameter.

Class struggle in a Schumpeterian economy

This study explores class struggle between workers and capitalists in a Schumpeterian economy, in which economic growth is driven by innovation in a market economy. We consider the limit on the market power of firms as a policy instrument and derive its optimal levels for workers and capitalists, respectively. Capitalists prefer powerful monopolistic firms, but even workers prefer firms to have some market power because profit provides incentives for innovation. Workers’ utility-maximizing degree of monopoly power is decreasing in their discount rate but increasing in innovation productivity and the quality step size. Capitalists’ utility-maximizing degree of monopoly power is increasing in the quality step size. We use the difference in these two degrees of monopoly power to measure the severity of their conflict, which becomes less severe when workers’ discount rate falls or innovation productivity rises. Finally, at a small (large) quality step size, enlarging it mitigates (worsens) their conflict.

Distributed Estimation With Cross-Verification Under False Data-Injection Attacks.

Under false data-injection (FDI) attacks, the data of some agents are tampered with by the FDI attackers, which causes that the distributed algorithm cannot estimate the ideal unknown parameter. Due to the concealment of the malicious data tampered with by the FDI attacks, many detection algorithms against FDI attacks often have poor detection results or low detection efficiencies. To solve these problems, a conveniently distributed diffusion least-mean-square (DLMS) algorithm with cross-verification (CV) is proposed against FDI attacks. The proposed DLMS with CV (DLMS-CV) algorithm is comprised of two subsystems: one subsystem provides a detection test of agents based on the CV mechanism, while the other provides a secure distribution estimation. In the CV mechanism, a smoothness strategy is introduced, which can improve the detection performance. The convergence performance of the proposed algorithm is analyzed, and then the design method of the adaptive threshold is also formulated. In particular, the probabilities of missing alarm and false alarm are examined, and they decay exponentially to zero under sufficiently small step size. Finally, simulation experiments are provided to illustrate the effectiveness and simplicity of the proposed DLMS-CV algorithm in comparison to other algorithms against FDI attacks.

A new graph-theoretic approach for the study of the surface of a thin sheet of a viscous liquid model

In this study, we considered the model of the surface of a thin sheet of viscous liquid which is known as the Buckmaster equation (BME), and presented a new graph-theoretic polynomial collocation method named the Hosoya polynomial collocation method (HPCM) for the solution of nonlinear Buckmaster equation. In the literature, the majority of the developed numerical methods considered small time step sizes like 0.01s and 0.05s to obtain relatively accurate approximations for the nonlinear BME. This study focused on optimizing the time step sizes by adopting bigger time steps sizes like 1.0s,3.0s, and 5.0s, etc without adversely affecting accuracy. First, using the Gram- Schmidt process, we generated the orthonormal functions from the Hosoya polynomial of the path graph. Then developed the functional integration matrix using orthonormal Hosoya polynomials of path graphs. With this active matrix-involved method, the nonlinear BMEs are transformed into a system of nonlinear equations and solved the equations by Newton’s method through the Mathematica software for unknown coefficients. The exactness of the proposed strategy is tested with two numerical examples. The acquired results contrasted with the current analytical solutions to these problems. Also provided the convergence analysis, comparison of error norms, graphical plots of correlation of HPCM results, and the results of other numerical methods in the literature to validate the productivity and accuracy of the newly developed HPCM.