Convergence Rate Of Iteration Research Articles

A Time-Domain Preconditioned Truncated Newton Approach to Visco-acoustic Multiparameter Full Waveform Inversion

A truncated Newton (TRN) method for time-domain full waveform inversion (FWI) in a visco-acoustic medium has been developed based on the 2nd-order adjoint state formulation. Time-domain gradient estimation and Hessian-vector product are managed by recomputing, without numerical instabilities, the incident wavefield at the same time as the adjoint wavefield for mitigating memory issues. Generic algorithm workflow has been proposed to switch between parameterizations, thanks to the chain rule. An efficient preconditioner adapted to the multiparameter configuration is developed to enhance the convergence rate of the inner conjugate gradient iterations. An additional user-defined scaling is introduced in the preconditioner to mitigate the weak sensitivity of specific parameters to waveform variations. The importance of the inverse Hessian for mitigating interparameter trade-off is validated on a toy example. Through a realistic 2D synthetic case based on a North Sea real data application, encouraging numerical results under $(V_p,\rho, Q^{-1})$ parametrization demonstrate that considering the Hessian influence significantly improves the multiparameter reconstruction, mitigating the coupling between parameters, for a reasonable increase of the computational cost compared to standard quasi-Newton optimization strategies.

Stability analysis of a deterministic dose calculation for MRI-guided radiotherapy

Modern effort in radiotherapy to address the challenges of tumor localization and motion has led to the development of MRI guided radiotherapy technologies. Accurate dose calculations must properly account for the effects of the MRI magnetic fields. Previous work has investigated the accuracy of a deterministic linear Boltzmann transport equation (LBTE) solver that includes magnetic field, but not the stability of the iterative solution method. In this work, we perform a stability analysis of this deterministic algorithm including an investigation of the convergence rate dependencies on the magnetic field, material density, energy, and anisotropy expansion. The iterative convergence rate of the continuous and discretized LBTE including magnetic fields is determined by analyzing the spectral radius using Fourier analysis for the stationary source iteration (SI) scheme. The spectral radius is calculated when the magnetic field is included (1) as a part of the iteration source, and (2) inside the streaming-collision operator. The non-stationary Krylov subspace solver GMRES is also investigated as a potential method to accelerate the iterative convergence, and an angular parallel computing methodology is investigated as a method to enhance the efficiency of the calculation. SI is found to be unstable when the magnetic field is part of the iteration source, but unconditionally stable when the magnetic field is included in the streaming-collision operator. The discretized LBTE with magnetic fields using a space-angle upwind stabilized discontinuous finite element method (DFEM) was also found to be unconditionally stable, but the spectral radius rapidly reaches unity for very low-density media and increasing magnetic field strengths indicating arbitrarily slow convergence rates. However, GMRES is shown to significantly accelerate the DFEM convergence rate showing only a weak dependence on the magnetic field. In addition, the use of an angular parallel computing strategy is shown to potentially increase the efficiency of the dose calculation.

A Fast Preconditioned Penalty Method for American Options Pricing Under Regime-Switching Tempered Fractional Diffusion Models

A fast preconditioned penalty method is developed for a system of parabolic linear complementarity problems (LCPs) involving tempered fractional order partial derivatives governing the price of American options whose underlying asset follows a geometry Levy process with multi-state regime switching. By means of the penalty method, the system of LCPs is approximated with a penalty term by a system of nonlinear tempered fractional partial differential equations (TFPDEs) coupled by a finite-state Markov chain. The system of nonlinear TFPDEs is discretized with the shifted Grunwald approximation by an upwind finite difference scheme which is shown to be unconditionally stable. Semi-smooth Newton’s method is utilized to solve the finite difference scheme as an outer iterative method in which the Jacobi matrix is found to possess Toeplitz-plus-diagonal structure. Consequently, the resulting linear system can be fast solved by the Krylov subspace method as an inner iterative method via fast Fourier transform (FFT). Furthermore, a novel preconditioner is proposed to speed up the convergence rate of the inner Krylov subspace iteration with theoretical analysis. With the above-mentioned preconditioning technique via FFT, under some mild conditions, the operation cost in each Newton’s step can be expected to be $$\mathcal{O}(N\mathrm{log}N)$$ , where N is the size of the coefficient matrix. Numerical examples are given to demonstrate the accuracy and efficiency of our proposed fast preconditioned penalty method.

A fast preconditioned policy iteration method for solving the tempered fractional HJB equation governing American options valuation

A fast preconditioned policy iteration method is proposed for the Hamilton–Jacobi–Bellman (HJB) equation involving tempered fractional order partial derivatives, governing the valuation of American options whose underlying asset follows exponential Lévy processes. An unconditionally stable upwind finite difference scheme with shifted Grünwald approximation is first developed to discretize the established HJB equation under the tempered fractional diffusion models. Next, the policy iteration method as an outer iterative method is utilized to solve the discretized HJB equation and proven to be convergent in finite steps to its numerical solution. Given the Toeplitz-like structure of the coefficient matrix in each policy iteration, the resulting linear system can be fast solved by the Krylov subspace method as an inner iterative method via fast Fourier transform (FFT). Furthermore, a novel preconditioner is proposed to speed up the convergence rate of the inner Krylov subspace iteration with theoretical analysis to ensure the linear system can be solved in O(NlogN) operations under some mild conditions, where N is the number of spatial node points. Numerical examples are given to demonstrate the accuracy and efficiency of the proposed fast preconditioned policy method.

Improved Algebraic Preconditioning for Multiscale Electromagnetic Problems

An improved scheme for the algebraic preconditioners is proposed in this letter to accelerate the iterative convergence rate of the multiscale electromagnetic problems, in which the overcrowded leaf boxes of the multilevel fast multipole algorithm are recursively bisected with the application of the multilevel accelerated Cartesian expansion algorithm to maintain the computational efficiency. Due to a lack of some strong couplings measured by the relative distance between the testing/basis functions, the algebraic preconditioners constructed by filtering the near-field interaction matrix of the fast algorithm are consequently ineffective. To alleviate the adverse effect of the multiscale-size boxes, the proposed scheme constructs the preconditioners on higher levels rather than the leaf level of the related fast algorithm, thus speeding up the iterative convergence rate significantly. Moreover, the rank-revealing adaptive cross approximation algorithm is first introduced to compute and store the enlarged computational domain of the preconditioner. As a result, there is only a modest increase in the computational cost. Numerical experiments have been performed to demonstrate the effectiveness and efficiency of the proposed method.

Uniqueness of iterative positive solutions for the singular fractional differential equations with integral boundary conditions

The uniqueness of positive solution for a class of singular fractional differential system with integral boundary conditions is considered in this paper and many types of equation system are contained in this equation system because there are many parameters which can be changeable in this equation system. The fractional orders are involved in the nonlinearity of the boundary value problem and the nonlinearity is allowed to be singular in regard to not only time variable but also space variable. The existence of uniqueness of positive solution is mainly obtained by fixed point theorem of mixed monotone operator and the positive solution of equation system is dependent on λ. An iterative sequence and convergence rate are given which are important for practical application and an example is given to demonstrate the validity of our main results.

Numerical solution of optimization problems for semi-linear elliptic equations with discontinuous coefficients and solutions

In this work we consider optimization problems for processes described by semi-linear partial differential equations of elliptic type with discontinuous coefficients and solutions (with imperfect contact matching conditions), with controls involved in the coefficients. Finite difference approximations of optimization problems are constructed. For the numerical implementation of finite optimization problems differentiability and Lipshitz-continuity of the grid functional of the approximating grid problems are proved. An iterative method for solving boundary value problems of contact for PDEs of elliptic type with discontinuous coefficients and solutions is developed and validated. The convergence of the iterative process is investigated. And the convergence rate of iterations (with calculated constants) is estimated.

Jungck-Khan iterative scheme and higher convergence rate

In this paper, we continue the theme of analytical and numerical treatment of Jungck-type iterative schemes. In particular, we focus on a special case of Jungck-Khan iterative scheme introduced by Khan et al. [Analytical and numerical treatment of Jungck-type iterative schemes, Appl. Math. Comput. 231 (2014) 521–535] to get an insight in the strong convergence and data dependence results obtained therein. Our investigations show that this special case under different control conditions on parametric sequences provides higher convergence rate and better data dependence estimates as compared to the Jungck-Khan iterative scheme itself.

Approximate Value Iteration with Temporally Extended Actions

Temporally extended actions have proven useful for reinforcement learning, but their duration also makes them valuable for efficient planning. The options framework provides a concrete way to implement and reason about temporally extended actions. Existing literature has demonstrated the value of planning with options empirically, but there is a lack of theoretical analysis formalizing when planning with options is more efficient than planning with primitive actions. We provide a general analysis of the convergence rate of a popular Approximate Value Iteration (AVI) algorithm called Fitted Value Iteration (FVI) with options. Our analysis reveals that longer duration options and a pessimistic estimate of the value function both lead to faster convergence. Furthermore, options can improve convergence even when they are suboptimal and sparsely distributed throughout the state-space. Next we consider the problem of generating useful options for planning based on a subset of landmark states. This suggests a new algorithm, Landmark-based AVI (LAVI), that represents the value function only at the landmark states. We analyze both FVI and LAVI using the proposed landmark-based options and compare the two algorithms. Our experimental results in three different domains demonstrate the key properties from the analysis. Our theoretical and experimental results demonstrate that options can play an important role in AVI by decreasing approximation error and inducing fast convergence.

Fast All-Pairs SimRank Assessment on Large Graphs and Bipartite Domains

SimRank is a powerful model for assessing vertex-pair similarities in a graph. It follows the concept that two vertices are similar if they are referenced by similar vertices. The prior work [18] exploits partial sums memoization to compute SimRank in $O(Kmn)$ time on a graph of $n$ vertices and $m$ edges, for $K$ iterations. However, computations among different partial sums may have redundancy. Besides, to guarantee a given accuracy $\epsilon$ , the existing SimRank needs $K=\lceil \log _C \,\epsilon \rceil$ iterations, where $C$ is a damping factor, but the geometric rate of convergence is slow if a high accuracy is expected. In this paper, (1) a novel clustering strategy is proposed to eliminate duplicate computations occurring in partial sums, and an efficient algorithm is then devised to accelerate SimRank computation to $O(K d^{\prime } n^2)$ time, where $d^{\prime }$ is typically much smaller than $\frac{m}{n}$ . (2) A new differential SimRank equation is proposed, which can represent the SimRank matrix as an exponential sum of transition matrices, as opposed to the geometric sum of the conventional counterpart. This leads to a further speedup in the convergence rate of SimRank iterations. (3) In bipartite domains, a novel finer-grained partial max clustering method is developed to speed up the computation of the Minimax SimRank variation from $O(Kmn)$ to $O(Km^{\prime }n)$ time, where $m^{\prime } ({\le} m)$ is the number of edges in a reduced graph after edge clustering, which can be typically much smaller than $m$ . Using real and synthetic data, we empirically verify that (1) our approach of partial sums sharing outperforms the best known algorithm by up to one order of magnitude; (2) the revised notion of SimRank further achieves a 5X speedup on large graphs while also fairly preserving the relative order of original SimRank scores; (3) our finer-grained partial max memoization for the Minimax SimRank variation in bipartite domains is 5X-12X faster than the baselines.

Distributed Consensus-Based Weight Design for Cooperative Spectrum Sensing

In this paper, we study the distributed spectrum sensing in cognitive radio networks. Existing distributed consensus-based fusion algorithms only ensure equal gain combining of local measurements, whose performance may be incomparable to various centralized soft combining schemes. Motivated by this fact, we consider practical channel conditions and link failures, and develop new weighted soft measurement combining without a centralized fusion center. Following the measurement by its energy detector, each secondary user exchanges its own measurement statistics with its local one-hop neighbors, and chooses the information exchanging rate according to the measurement channel condition, e.g., the signal-to-noise ratio (SNR). We rigorously prove the convergence of the new consensus algorithm, and show all secondary users hold the same global decision statistics from the weighted soft measurement combining throughout the network. We also provide distributed optimal weight design under uncorrelated measurement channels. The convergence rate of the consensus iteration is given under the assumption that each communication link has an independent probability to fail, and the upper bound of the iteration number of the $ \epsilon$ -convergence is explicitly given as a function of system parameters. Simulation results show significant improvement of the sensing performance compared to existing consensus-based approaches, and the performance of the distributed weighted design is comparable to the centralized weighted combining scheme.

Technique for Intrusion Detection Based on NSST Domain Artificial Neural Networks

The issue of intrusion detection has been an active topic in both military and civilian areas, and a great many relevant algorithms and techniques have been developed accordingly. This paper addresses a novel technique based on non-subsampled shearlet transform (NSST) domain artificial neural networks (ANN) to solve the above problem, employing multi-scale geometry analysis (MGA) of NSST and the train characteristics of ANN together. Experimental results indicate that, compared with other existing conventional intrusion detection tools, the proposed one is superior to other current popular ones in both aspects of iteration numbers and convergence rates.

Exploring trust region method for the solution of logit-based stochastic user equilibrium problem

In this research paper, we explored using the trust region method to solve the logit-based SUE problem. We proposed a modified trust region Newton (MTRN) algorithm for this problem. When solving the trust region SUE subproblem, we showed that applying the well-known Steihaug-Toint method is inappropriate, since it may make the convergence rate of the major iteration very slow in the early stage of the computation. To overcome this drawback, a modified Steihaug-Toint method was proposed. We proved the convergence of our MTRN algorithm and showed its convergence rate is superlinear.For the implication of our algorithm, we proposed an important principle on how to select the basic route for each OD pair. We indicated that it is a crucial principle to accelerate the convergence rate of the minor iteration (i.e. trust region subproblem-solving iteration). In this study, other implication issues for the SUE problem are also considered, including the computation of the trial step and the strategy to ensure strict feasibility iteration point. We compared the MTRN algorithm with the Gradient Projection (GP) algorithm on the Sioux Falls network. Some results of numerical analysis are also reported.

A novel Frank–Wolfe algorithm. Analysis and applications to large-scale SVM training

Recently, there has been a renewed interest in the machine learning community for variants of a sparse greedy approximation procedure for concave optimization known as the Frank–Wolfe (FW) method. In particular, this procedure has been successfully applied to train large-scale instances of non-linear Support Vector Machines (SVMs). Specializing FW to SVM training has allowed to obtain efficient algorithms, but also important theoretical results, including convergence analysis of training algorithms and new characterizations of model sparsity.In this paper, we present and analyze a novel variant of the FW method based on a new way to perform away steps, a classic strategy used to accelerate the convergence of the basic FW procedure. Our formulation and analysis is focused on a general concave maximization problem on the simplex. However, the specialization of our algorithm to quadratic forms is strongly related to some classic methods in computational geometry, namely the Gilbert and MDM algorithms.On the theoretical side, we demonstrate that the method matches the guarantees in terms of convergence rate and number of iterations obtained by using classic away steps. In particular, the method enjoys a linear rate of convergence, a result that has been recently proved for MDM on quadratic forms.On the practical side, we provide experiments on several classification datasets, and evaluate the results using statistical tests. Experiments show that our method is faster than the FW method with classic away steps, and works well even in the cases in which classic away steps slow down the algorithm. Furthermore, these improvements are obtained without sacrificing the predictive accuracy of the obtained SVM model.

An Optimal p‐Refinement Strategy for Dynamic Iteration of Ordinary and Differential Algebraic Equations

AbstractMultiphysical simulation tasks are often numerically solved by dynamic iteration schemes. Usually, this demands the efficient and stable coupling of existing simulation software for the contributing physical subdomains or subsystem. Since the coupling is weakened by such a simulation strategy, iteration is needed to enhance the quality of the numerical approximation. By the means of error recursions, one obtains estimates for the approximation order and the reduction of error per iteration (convergence rate). It is know that the first iterations can be coarsely sampled (in time), but the last iterations need to be refined (h‐refinement) to obtain the accuracy gain of latter iterations (‘sweeps’).In this work we discuss an optimal choice of the approximation order p used in the time integration with respect to the iteration ‘sweep’ count. It is deduced from the analytical error recursion and yields a p‐refinement strategy. Numerical experiments show that our estimates are sharp and give a precise prediction of the correct convergence. (© 2013 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

On convergence acceleration in the method for the identification of the diagonal element in problems of heat radiation transfer with scattering

We consider the improvement in the method of identifying the diagonal element for the iterative solution of implicit finite-difference equations consisting of the spectral integral-differential kinetic equations of photon transport and the energy equation. The study is based on the case of the WDD-scheme in slab geometry for two versions of the algorithm. The theoretical estimates of the iteration convergence rate and examples of the test problems are given.

On the convergence rate of dynamic iteration for coupled problems with multiple subsystems

In multiphysical modeling coupled problems naturally occur. Each subproblem is commonly represented by a system of partial differential-algebraic equations. Applying the method of lines, this results in coupled differential-algebraic equations (DAEs). Dynamic iteration with windowing is a standard technique for the transient simulation of such systems. In contrast to the dynamic iteration of systems of ordinary differential equations, convergence for DAEs cannot be generally guaranteed unless some contraction condition is fulfilled. In the case of convergence, it is a linear one.In this paper, we quantify the convergence rate, i.e., the slope of the contraction, in terms of the window size. We investigate the convergence rate with respect to the coupling structure for DAE and ODE systems and also for two and more subsystems. We find higher rates (for certain coupling structures) than known before (that is, linear in the window size) and give sharp estimates for the rate. Furthermore it is revealed how the rate depends on the number of subsystems.

Critical state plasticity. Part VII: Triggering a shear band in variably saturated porous media

In a previous paper (Part VI), the impact of spatially varying density on the localization of deformation of granular materials has been quantified using mesoscopic representations of stresses and deformation. In the present paper, we extend the formulation to unsaturated porous media and investigate the effect of spatially varying degree of saturation on triggering a shear band in granular materials. Variational formulations are presented for porous solids whose voids are filled with liquid and gas. Two critical state formulations are used to characterize the solid constitutive response: one developed for clay and another for sand. Stabilized low-order mixed finite elements are used to solve the fully coupled solid-deformation/fluid-flow problem. For the first time, we present the consistent derivative of the effective stress tensor with respect to capillary pressure considering full coupling of solid deformation with fluid flow, which is essential for achieving an optimal convergence rate of Newton iteration.

An algebraic multigrid approach to solve extended finite element method based fracture problems

SUMMARYThis article proposes an algebraic multigrid (AMG) approach to solve linear systems arising from applications where strong discontinuities are modeled by the extended finite element method. The application of AMG methods promises optimal scalability for solving large linear systems. However, the straightforward (or ‘black‐box’) use of existing AMG techniques for extended finite element method problems is often problematic. In this paper, we highlight the reasons for this behavior and propose a relatively simple adaptation that allows one to leverage existing AMG software mostly unchanged. Numerical tests demonstrate that optimal iterative convergence rates can be attained that are comparable with AMG convergence rates associated with linear systems for standard finite element approximations without discontinuities. Published 2012. This article is a US Government work and is in the public domain in the USA.

Digital Predistortion Using Stochastic Conjugate Gradient Method

This paper is concerned with digital predistortion for linearization of RF high power amplifiers (HPAs). An iterative stochastic conjugate gradient (SCG) method is used in computing the predistorter in the digital predistortion. We also propose an adaptive scheme for selecting basis functions in the SCG computations. The adaptive scheme with SCG has the advantage of reducing the complexity and, at the same time, increasing the convergence rate and stability of iteration. We present some results of using the SCG in a hardware platform with a solid state HPA.