Large Step Size Research Articles

Nonparametric stochastic approximation with large step-sizes

We consider the random-design least-squares regression problem within the reproducing kernel Hilbert space (RKHS) framework. Given a stream of independent and identically distributed input/output data, we aim to learn a regression function within an RKHS $\mathcal{H}$, even if the optimal predictor (i.e., the conditional expectation) is not in $\mathcal{H}$. In a stochastic approximation framework where the estimator is updated after each observation, we show that the averaged unregularized least-mean-square algorithm (a form of stochastic gradient), given a sufficient large step-size, attains optimal rates of convergence for a variety of regimes for the smoothnesses of the optimal prediction function and the functions in $\mathcal{H}$.

Improvement of FDTD method regarding cloaking metamaterials by interpolation

The finite-difference time-domain (FDTD) algorithm is a powerful, versatile tool for electromagnetic analysis. In this work, the stability of FDTD is analyzed regarding metamaterials and improved by stably interpolating adjacent fields. The interpolation operation was applied to metamaterials such as the invisibility cloak, the field rotator, and the field concentrator. As a result, field distortion disappeared and computation efficiency was increased as large spatial step sizes became available. The proposed method also resulted in smaller dispersion compared with the conventional FDTD method. The suggested method is applicable to other metamaterials and can be used to efficiently analyze large-scale metamaterial structures with FDTD.

Charged particle transport in magnetic fields in EGSnrc.

To accurately and efficiently implement charged particle transport in a magnetic field in EGSnrc and validate the code for the use in phantom and ion chamber simulations. The effect of the magnetic field on the particle motion and position is determined using one- and three-point numerical integrations of the Lorentz force on the charged particle and is added to the condensed history calculation performed by the EGSnrc PRESTA-II algorithm. The code is tested with a Fano test adapted for the presence of magnetic fields. The code is compatible with all EGSnrc based applications, including egs++. Ion chamber calculations are compared to experimental measurements and the effect of the code on the efficiency and timing is determined. Agreement with the Fano test's theoretical value is obtained at the 0.1% level for large step-sizes and in magnetic fields as strong as 5 T. The NE2571 dose calculations achieve agreement with the experiment within 0.5% up to 1 T beyond which deviations up to 1.2% are observed. Uniform air gaps of 0.5 and 1 mm and a misalignment of the incoming photon beam with the magnetic field are found to produce variations in the normalized dose on the order of 1%. These findings necessitate a clear definition of all experimental conditions to allow for accurate Monte Carlo simulations. It is found that ion chamber simulation times are increased by only 38%, and a 10 × 10 × 6 cm(3) water phantom with (3 mm)(3) voxels experiences a 48% increase in simulation time as compared to the default EGSnrc with no magnetic field. The incorporation of the effect of the magnetic fields in EGSnrc provides the capability to calculate high accuracy ion chamber and phantom doses for the use in MRI-radiation systems. Further, the effect of apparently insignificant experimental details is found to be accentuated by the presence of the magnetic field.

Towards essential improvement for the Parareal-TR and Parareal-Gauss4 algorithms

Parareal is an iterative algorithm and is characterized by two propagators G and F, which are respectively associated with large step size ΔT and small step size Δt, where ΔT=JΔt and J≥2 is an integer. For symmetric positive definite (SPD) system u′(t)+Au(t)=g(t) arising from semi-discretizing time-dependent PDEs, if we fix the G-propagator to the Backward-Euler method and choose for F some L-stable time-integrator it can be proven that the convergence factors of the corresponding parareal algorithms satisfy ρ≈13, ∀J≥2 and ∀σ(A)⊂[0,+∞), where σ(A) is the spectrum of the matrix A. However, this result does not hold when time-integrators that lack L-stability, such as the Trapezoidal rule and the 4th-order Gauss RK method, are chosen as the F-propagator. The parareal algorithms using these two methods for the F-propagator are denoted by Parareal-TR and Parareal-Gauss4. In this paper, we propose a strategy to let these two parareal algorithms possess such a uniform convergence property. The idea is to choose an L-stable propagator F˜ and on each coarse time-interval [Tn,Tn+1] we perform first two steps of F˜, then followed by J−2 steps of F. Precisely, for the Trapezoidal rule we select the 2nd-order SDIRK method as the F˜-propagator, and for the 4th-order Gauss RK method we select the 4th-order Lobatto III-C method as the F˜-propagator. Numerical results are given to support our theoretical conclusions.

Advanced time integration algorithms for dislocation dynamics simulations of work hardening

Efficient time integration is a necessity for dislocation dynamics simulations of work hardening to achieve experimentally relevant strains. In this work, an efficient time integration scheme using a high order explicit method with time step subcycling and a newly-developed collision detection algorithm are evaluated. First, time integrator performance is examined for an annihilating Frank–Read source, showing the effects of dislocation line collision. The integrator with subcycling is found to significantly out-perform other integration schemes. The performance of the time integration and collision detection algorithms is then tested in a work hardening simulation. The new algorithms show a 100-fold speed-up relative to traditional schemes. Subcycling is shown to improve efficiency significantly while maintaining an accurate solution, and the new collision algorithm allows an arbitrarily large time step size without missing collisions.

Numerically efficient modified Runge–Kutta solver for fatigue crack growth analysis

We present a modified Runge–Kutta algorithm which yields a conservative estimate of the crack size for fatigue crack growth even for large integration step sizes. Conservative in this context means to overestimate the crack size. Commonly used algorithms (e.g. Euler, Runge–Kutta) usually underestimate the crack growth and only converge for small step sizes to an accurate value. As the presented algorithm overestimates the crack growth even for large and converges for small integration step sizes, it can be used for instance in Monte-Carlo based probabilistic fracture mechanics simulations, which might be otherwise computational impractical.

Fractional-order adaptive signal processing strategies for active noise control systems

Robust and computationally efficient adaptive algorithms are required in active noise control systems (ANCS) to cancel out the effects of noise in the presence of secondary path (SP) as the latter makes the identification problem more challenging. To ensure stability in such applications, the step size parameter is kept small, but it results in slow convergence which limits the usefulness of such algorithms in ANCS. In this paper, we propose a novel fractional-order adaptive filter structures such that the output from the conventional filtered-x least mean square algorithm is passed through a new update equation derived from a cost function based on a posteriori error and optimized using fractional derivatives. The proposed algorithms are designed for the feed-forward configuration of ANCS; the schemes are validated using the performance metrics of mean squared error, mean squared deviation and mean relative modeling error. We consider a number of scenarios where different step sizes and fractional orders have been used for evaluation with input signals modeled as binary or Gaussian. Simulation results show that the proposed algorithms outperform the conventional counterparts with convergence improvements in the range of 75–80 %, while they offer the same steady- state behavior even for large step sizes, thereby providing better modeling in the presence of SP.

Error analysis for the operator marching method applied to range dependent waveguides

Abstract The relations between stability and accuracy of the operator marching method (OMM) are usually conflicting in waveguides with strong range dependence. To explain this phenomenon, this study intends to present an error estimate for the OMM in range dependent waveguides. We utilize “approximation level” to measure truncation error for a marching method in various range step sizes. Then, the error estimate is developed to analyze the performances of the OMM. Through an error analysis, we verify the following features of the OMM: (i) it is valid to apply the OMM in slowly varying waveguides with very large range step sizes; (ii) the OMM may blow up suddenly when the range dependence is strong and the step size is extremely small in the same time. We also develop a three-number set to describe the stability and accuracy level of a general marching method for computing wave propagation in a waveguide. In the end, extensive numerical experiments are implemented to verify the correctness of the error analysis.

Review and Comparison of Variable Step-Size LMS Algorithms

The inherent feature of the Least Mean Squares (LMS) algorithm is the step size, and it requires careful adjustment. Small step size, required for small excess mean square error, results in slow convergence. Large step size, needed for fast adaptation, may result in loss of stability. Therefore, many modifications of the LMS algorithm, where the step size changes during the adaptation process depending on some particular characteristics, were and are still being developed. The paper reviews seventeen of the best known variable step-size LMS (VS-LMS) algorithms to the degree of detail that allows to implement them. The performance of the algorithms is compared in three typical applications: parametric identification, line enhancement, and adaptive noise cancellation. The paper suggests also one general modification that can simplify the choice of the upper bound for the step size, which is a crucial parameter for many VS-LMS algorithms.

Low-rank one-step wave extrapolation for reverse time migration

Reverse time migration (RTM) relies on accurate wave extrapolation engines to image complex subsurface structures. To construct such operators with high efficiency and numerical stability, we have developed a one-step wave extrapolation approach using complex-valued low-rank decomposition to approximate the mixed-domain space-wavenumber wave extrapolation symbol. The low-rank one-step method involves a complex-valued phase function, which is more flexible than a real-valued phase function of two-step schemes, and thus it is capable of modeling a wider variety of dispersion relations. Two novel designs of the phase function leads to the desired properties in wave extrapolation. First, for wave propagation in inhomogeneous media, including a velocity gradient term assures a more accurate phase behavior, particularly when the velocity variations are large. Second, an absorbing boundary condition, which is propagation-direction-dependent, can be incorporated into the phase function as an anisotropic attenuation term. This term allows waves to travel parallel to the boundary without absorption, thus reducing artificial reflections at wide incident angles. Using numerical experiments, we revealed the stability improvement of a one-step scheme in comparison with two-step schemes. We observed the low-rank one-step operator to be remarkably stable and capable of propagating waves using large time step sizes, even beyond the Nyquist limit. The stability property can help to minimize the computational cost of seismic modeling or RTM. The low-rank one-step wave extrapolation also handles anisotropic wave propagation accurately and efficiently. When applied to RTM in anisotropic media, the proposed method generated high-quality images.

Analogue‐to‐digital converter phase insensitive dispersion search method with large search step size for low computational cost

Accumulated chromatic dispersion (CD) estimation is crucial for frequency-domain equalisation in an optical coherent digital receiver. To solve the problems of small search step size and analogue-to-digital converter (ADC) sampling phase sensitivity from conventional CD searching algorithm, a CD searching method with modified shift parameters and waveforms is proposed in this study. The modified CD searching method allows a much larger search step size and can achieve faster CD estimation with low computational complexity. The performance of the modified CD searching method is also insensitive to ADC sampling phase. On the basis of simulation results, a very low search error probability is confirmed with a search step size of 50 km. About 1 K symbols are found to be enough for dispersion estimation. After the frequency-domain equaliser (FDE) with the estimated dispersion value, the residual dispersion length <30 km can be well compensated with a 9-tap adaptive time-domain equaliser (TDE) with fast convergence. The overall error rate performance of the tunable electrical dispersion compensation is also studied and compared with lumped and distributed optical dispersion compensation for dynamically switched optical networks.

Optimization in High Dimensions via Accelerated, Parallel, and Proximal Coordinate Descent

We propose a new randomized coordinate descent method for minimizing the sum of convex functions, each of which depends on a small number of coordinates only. Our method (APPROX) is simultaneously Accelerated, Parallel, and PROXimal; this is the first time such a method has been proposed. In the special case when the number of processors is equal to the number of coordinates, the method converges at the rate $2\bar{\omega}\bar{L} R^2/(k+1)^2 $, where $k$ is the iteration counter, $\bar{\omega}$ is a data-weighted average degree of separability of the loss function, $\bar{L}$ is the average of Lipschitz constants associated with the coordinates and individual functions in the sum, and $R$ is the distance of the initial point from the minimizer. We show that the method can be implemented without the need to perform full-dimensional vector operations, which is the major bottleneck of accelerated coordinate descent, rendering it impractical. The fact that the method depends on the average degree of separability, and not on the maximum degree, can be attributed to the use of new safe large stepsizes, leading to improved expected separable overapproximation (ESO). These are of independent interest and can be utilized in all existing parallel randomized coordinate descent algorithms based on the concept of ESO. In special cases, our method recovers several classical and recent algorithms such as simple and accelerated proximal gradient descent, as well as serial, parallel, and distributed versions of randomized block coordinate descent. Due to this flexibility, APPROX had been used successfully by the authors in a graduate class setting as a modern introduction to deterministic and randomized proximal gradient methods. Our bounds match or improve upon the best known bounds for each of the methods APPROX specializes to. Our method has applications in a number of areas, including machine learning, submodular optimization, and linear and semidefinite programming.

A multistep Legendre pseudo-spectral method for Volterra integral equations

In this paper, we extend the single-step pseudo-spectral method for Volterra integral equations of the second kind to the multistep pseudo-spectral method. We also analyze the convergence of the hp-version of the multistep pseudo-spectral method under the L2-norm, and show that the scheme enjoys high order accuracy and can be implemented in a stable and efficient manner. In addition, it is very suitable for long time calculations and large step size situations. Numerical experiments confirm theoretical expectations.

A DC-Stable, Well-Balanced, Calderón Preconditioned Time Domain Electric Field Integral Equation

The marching-on-in-time (MOT) solution of the time domain electric field integral equation (TD-EFIE) has traditionally suffered from a number of problems, including: 1) instability; 2) spurious static contributions plaguing the solution; 3) low-frequency breakdown; and 4) dense discretization breakdown. The first issue can be resolved by employing proper space-time Galerkin discretization schemes and accurate quadrature methods. The second and the third issue have been resolved by the quasi-Helmholtz Projected TD-EFIE (qHP-TDEFIE). This contribution introduces a multiplicative preconditioner which can be applied to the qHP-TDEFIE, without further modifying the original scheme. This preconditioner is based on Calderon techniques and guarantees that the MOT system can be solved efficiently using iterative methods, not only for large time step sizes but also for dense spatial discretizations, and for both simply and multiply connected geometries.

A predator-prey model with Beddington-DeAngelis functional response: a non-standard finite-difference method

In this paper, we transform a continuous-time predator-prey model with Beddington–DeAngelis functional response into a discrete-time model by nonstandard finite difference scheme (NSFD). The NSFD model shows complete dynamic consistency with its continuous counterpart for any step size. However, the discrete model of same continuous system obtained by Euler forward method shows dynamic inconsistency for larger step size. Extensive numerical simulations have been done to compare the dynamics of NSFD system and Euler system. Our analysis reveals that dynamics of NSFD model is independent of the step-size, whereas the dynamics of the standard discrete model completely depends on the step-size and produces spurious dynamics like chaos.

On the Convergence of the Iterative Shrinkage/Thresholding Algorithm With a Weakly Convex Penalty

We consider the iterative shrinkage/thresholding algorithm (ISTA) applied to a cost function composed of a data fidelity term and a penalty term. The penalty is non-convex but the concavity of the penalty is accounted for by the data fidelity term so that the overall cost function is convex. We provide a generalization of the convergence result for ISTA viewed as a forward-backward splitting algorithm. We also demonstrate experimentally that for the current setup, using large stepsizes in ISTA can accelerate convergence more than existing schemes proposed for the convex case, like TwIST or FISTA.

Multiparticle moves in acceptance rate optimized Monte Carlo.

Molecular Dynamics (MD) and Monte Carlo (MC) based simulation methods are widely used to investigate molecular and nanoscale structures and processes. While the investigation of systems in MD simulations is limited by very small time steps, MC methods are often stifled by low acceptance rates for moves that significantly perturb the system. In many Metropolis MC methods with hard potentials, the acceptance rate drops exponentially with the number of uncorrelated, simultaneously proposed moves. In this work, we discuss a multiparticle Acceptance Rate Optimized Monte Carlo approach (AROMoCa) to construct collective moves with near unit acceptance probability, while preserving detailed balance even for large step sizes. After an illustration of the protocol, we demonstrate that AROMoCa significantly accelerates MC simulations in four model systems in comparison to standard MC methods. AROMoCa can be applied to all MC simulations where a gradient of the potential is available and can help to significantly speed up molecular simulations.

Convergence Analysis of the Parareal-Euler Algorithm for Systems of ODEs with Complex Eigenvalues

Parareal is an iterative algorithm and is characterized by two propagators $$\mathscr {G}$$G and $$\mathscr {F}$$F, which are respectively associated with large step size $$\varDelta T$$ΔT and smal...

Statistical errors in interferometry with least-squares algorithms and large step sizes

This paper investigates statistical errors in determining peak positions of interference fringes with the least-squares algorithms when the phase-shifting step sizes are 2π/3, π/2, and 2π/5. Two types of random noises are considered, which are called intensity noise and position noise. With these large step sizes, the statistical errors obtained from simulated measurements are compared with the theoretical results published in Ref. [11]. The effects of the wavelength and the amplitude on the statistical errors are investigated. A discrepancy between the simulation results and the theoretical results has been observed, that is, when the step size was π/2, the statistical errors caused by position noise have shown strong peak-position dependence, which cannot be explained by Eq. (16) given in Ref. [11]. A new equation is given to account for the simulation results.

Compressive advection and multi‐component methods for interface‐capturing

SummaryThis paper develops methods for interface‐capturing in multiphase flows. The main novelties of these methods are as follows: (a) multi‐component modelling that embeds interface structures into the continuity equation; (b) a new family of triangle/tetrahedron finite elements, in particular, the P1DG‐P2(linear discontinuous between elements velocity and quadratic continuous pressure); (c) an interface‐capturing scheme based on compressive control volume advection methods and high‐order finite element interpolation methods; (d) a time stepping method that allows use of relatively large time step sizes; and (e) application of anisotropic mesh adaptivity to focus the numerical resolution around the interfaces and other areas of important dynamics. This modelling approach is applied to a series of pure advection problems with interfaces as well as to the simulation of the standard computational fluid dynamics benchmark test cases of a collapsing water column under gravitational forces (in two and three dimensions) and sloshing water in a tank. Two more test cases are undertaken in order to demonstrate the many‐material and compressibility modelling capabilities of the approach. Numerical simulations are performed on coarse unstructured meshes to demonstrate the potential of the methods described here to capture complex dynamics in multiphase flows. Copyright © 2015 John Wiley &amp; Sons, Ltd.