Anderson Acceleration Research Articles

Restarted Nonnegativity Preserving Tensor Splitting Methods via Relaxed Anderson Acceleration for Solving Multilinear Systems

Restarted Nonnegativity Preserving Tensor Splitting Methods via Relaxed Anderson Acceleration for Solving Multilinear Systems

Anderson Acceleration with Truncated Gram–Schmidt

Anderson Acceleration with Truncated Gram–Schmidt

Anderson accelerated preconditioning iterative method for RBF interpolation

Traditional RBF interpolation involves solving a linear system, making it computationally expensive for large datasets. Iterative-based quasi-interpolation combines RBF interpolation with iterative methods to enhance accuracy and convergence. To enhance efficiency and accuracy, we in this paper propose a novel method for RBF quasi-interpolation that combines Anderson acceleration with the asynchronous DCPI, termed Anderson-DCPI. The method alternates between the preconditioning iterative method and Anderson extrapolation, aiming to improve convergence rates. We demonstrate the convergence of Anderson-DCPI for positive definite RBF kernel functions and validate its effectiveness through a series of numerical examples.

A bronchoscopic navigation method based on neural radiation fields.

We introduce a novel approach for bronchoscopic navigation that leverages neural radiance fields (NeRF) to passively locate the endoscope solely from bronchoscopic images. This approach aims to overcome the limitations and challenges of current bronchoscopic navigation tools that rely on external infrastructures or require active adjustment of the bronchoscope. To address the challenges, we leverage NeRF for bronchoscopic navigation, enabling passive endoscope localization from bronchoscopic images. We develop a two-stage pipeline: offline training using preoperative data and online passive pose estimation during surgery. To enhance performance, we employ Anderson acceleration and incorporate semantic appearance transfer to deal with the sim-to-real gap between training and inference stages. We assessed the viability of our approach by conducting tests on virtual bronchscopic images and a physical phantom against the SLAM-based methods. The average rotation error in our virtual dataset is about 3.18 and the translation error is around 4.95 mm. On the physical phantom test, the average rotation and translation error are approximately 5.14 and 13.12 mm. Our NeRF-based bronchoscopic navigation method eliminates reliance on external infrastructures and active adjustments, offering promising advancements in bronchoscopic navigation. Experimental validation on simulation and real-world phantom models demonstrates its efficacy in addressing challenges like low texture and challenging lighting conditions.

The cell-centered positivity-preserving finite volume scheme for 3D convection–diffusion equation on distorted meshes

PurposeThis paper aims to construct positivity-preserving finite volume schemes for the three-dimensional convection–diffusion equation that are applicable to arbitrary polyhedral grids.Design/methodology/approachThe cell vertices are used to define the auxiliary unknowns, and the primary unknowns are defined at cell centers. The diffusion flux is discretized by the classical nonlinear two-point flux approximation. To ensure the fully discrete scheme has positivity-preserving property, an improved discretization method for the convection flux was presented. Besides, a new positivity-preserving vertex interpolation method is derived from the linear reconstruction in the discretization of convection flux. Moreover, the Picard iteration method may have slow convergence in solving the nonlinear system. Thus, the Anderson acceleration of Picard iteration method is used to solve the nonlinear system. A condition number monitor of matrix is employed in the Anderson acceleration method to achieve better robustness.FindingsThe new scheme is applicable to arbitrary polyhedral grids and has a second-order accuracy. The results of numerical experiments also confirm the positivity-preserving of the discretization scheme.Originality/value1. This article presents a new positivity-preserving finite volume scheme for the 3D convection–diffusion equation. 2. The new discretization scheme of convection flux is constructed. 3. A new second-order interpolation algorithm is given to eliminate the auxiliary unknowns in flux expressions. 4. An improved Anderson acceleration method is applied to accelerate the convergence of Picard iterations. 5. This scheme can solve the convection–diffusion equation on the distorted meshes with second-order accuracy.

On the efficient non-linear solver for hydraulic fracturing and well cementing simulations based on Anderson acceleration

The aim of this study is to create a fast and stable iterative technique for numerical solution of a quasi-linear elliptic pressure equation. We developed a modified version of the Anderson acceleration (AA) algorithm to fixed-point (FP) iteration method. It computes the approximation to the solutions at each iteration based on the history of vectors in extended space, which includes the vector of unknowns, the discrete form of the operator, and the equation's right-hand side. Several constraints are applied to AA algorithm, including a limitation of the time step variation during the iteration process, which allows switching to the base FP iterations to maintain convergence. Compared to the base FP algorithm, the improved version of the AA algorithm enables a reliable and rapid convergence of the iterative solution for the quasi-linear elliptic pressure equation describing the flow of particle-laden yield-stress fluids in a narrow channel during hydraulic fracturing, a key technology for stimulating hydrocarbon-bearing reservoirs. In particular, the proposed AA algorithm allows for faster computations and resolution of unyielding zones in hydraulic fractures that cannot be calculated using the FP algorithm. The quasi-linear elliptic pressure equation under consideration describes various physical processes, such as the displacement of fluids with viscoplastic rheology in a narrow cylindrical annulus during well cementing, the displacement of cross-linked gel in a proppant pack filling hydraulic fractures during the early stage of well production (fracture flowback), and multiphase filtration in a rock formation. We estimate computational complexity of the developed algorithm as compared to Jacobian-based algorithms and show that the performance of the former one is higher in modelling of flows of viscoplastic fluids. We believe that the developed algorithm is a useful numerical tool that can be implemented in commercial simulators to obtain fast and converged solutions to the non-linear problems described above.

Non-stationary Anderson acceleration with optimized damping

Anderson acceleration (AA) has a long history of use and a strong recent interest due to its potential ability to dramatically improve the linear convergence of the fixed-point iteration. Most authors are simply using and analyzing the stationary version of Anderson acceleration (sAA) with a constant damping factor or without damping. Little attention has been paid to nonstationary algorithms. However, damping can be useful and is sometimes crucial for simulations in which the underlying fixed-point operator is not globally contractive. The role of this damping factor has not been fully understood. In the present work, we consider the non-stationary Anderson acceleration algorithm with optimized damping (AAoptD) in each iteration to further speed up linear and nonlinear iterations by applying one extra inexpensive optimization. We analyze the convergence rate this procedure and develop an efficient and inexpensive implementation scheme. We show by extensive numerical experiments that the proposed non-stationary Anderson acceleration with optimized damping procedure often converges much faster than stationary AA with constant damping, adaptive damping or without damping, especially in the cases larger window sizes are needed. We also observe that simple strategies like using constant damping factors and adaptive damping factors, sometimes, work very well for some problems while sometimes they are even worse than AA without damping. Our proposed method is usually more robust than AA with constant damping and adaptive damping. Moreover, we also observed from our numerical results that damping can be good, but choosing the wrong damping factors may slow down the convergence rate. Theoretical analysis of the effects of damping factors are needed and important.

Anderson acceleration for PWR whole-core pin-by-pin Nu-TH coupling calculation in NECP-Bamboo2.0

For PWR whole-core pin-by-pin neutronics/thermal-hydraulics (Nu-TH) coupling analysis, the Picard iteration strategy is widely employed benefitting from the convenience of implementation. Nevertheless, it is limited to the global multi-physics convergence rate regarding its oscillatory convergence behaviors in the local solution of each single-physics field. In this paper, the Anderson Acceleration (AA) scheme is employed and investigated to enhance the convergence performance of the pin-by-pin Nu and subchannel TH coupling analysis code NECP-Bamboo2.0. Different from the classical relaxation scheme where the relaxation factor is predetermined and fixed, the AA scheme utilizes the linear combination of several previous solutions to correct the current solution. The coefficients of this linear combination can be obtained by solving the residual equation, making the approach more robust. In addition, the selection of the accelerated solution vector is investigated based on the tightly coupling strategy of Nu-TH in NECP-Bamboo2.0. It is illustrated that the neutron-transport solution vector could have superior theoretical acceleration performance compared to the commonly used TH solution vector in the practical PWR analysis scenario. Finally, the AA scheme in NECP-Bamboo2.0 is validated using the VERA#6 3D single-assembly transport/TH coupling benchmark problem and the VERA#7 3D whole-core multi-physics coupling benchmark. It is demonstrated by the numerical results that the AA scheme possesses good robustness and computational efficiency. For the 3D whole-core VERA#7 benchmark, compared with the classic Picard iteration with a relaxation factor of 0.5, the AA scheme based on the neutron-transport solution vector can reduce the number of global iterations and calculation time by around 80%.

Anderson acceleration with approximate calculations: Applications to scientific computing

SummaryWe provide rigorous theoretical bounds for Anderson acceleration (AA) that allow for approximate calculations when applied to solve linear problems. We show that, when the approximate calculations satisfy the provided error bounds, the convergence of AA is maintained while the computational time could be reduced. We also provide computable heuristic quantities, guided by the theoretical error bounds, which can be used to automate the tuning of accuracy while performing approximate calculations. For linear problems, the use of heuristics to monitor the error introduced by approximate calculations, combined with the check on monotonicity of the residual, ensures the convergence of the numerical scheme within a prescribed residual tolerance. Motivated by the theoretical studies, we propose a reduced variant of AA, which consists in projecting the least‐squares used to compute the Anderson mixing onto a subspace of reduced dimension. The dimensionality of this subspace adapts dynamically at each iteration as prescribed by the computable heuristic quantities. We numerically show and assess the performance of AA with approximate calculations on: (i) linear deterministic fixed‐point iterations arising from the Richardson's scheme to solve linear systems with open‐source benchmark matrices with various preconditioners and (ii) non‐linear deterministic fixed‐point iterations arising from non‐linear time‐dependent Boltzmann equations.

Finite element iterative algorithm based on Anderson acceleration technique for incompressible MHD equations

In this paper, an Anderson’s acceleration algorithm for the finite element Oseen iterative scheme of the steady incompressible magnetohydrodynamics (MHD) problem is designed and analyzed. The Anderson acceleration algorithm based on solving an optimization problem in each iteration receives great attention to accelerate linear and nonlinear iterations. We prove that the proposed algorithm has higher convergence rate than that of the Oseen iterative method for the steady incompressible MHD, and give the accelerating convergence theory under different acceleration depths m. Numerical experiments including the singular solution and benchmark problems are carried out to verify the correctness and effectiveness of the algorithm. The tests show the marked improvement in the high physical parameters (including the hydrodynamic and magnetic Reynolds numbers, and the coupling number) when the Oseen iteration fails to converge.

Phase-field simulation of dual-crack system hydrogen embrittlement in metallic materials

Hydrogen embrittlement is a severe form of failure in metallic materials, where hydrogen ions significantly reduce the load-bearing capacity of the material. The presence of hydrogen and multiple cracks complicates the fracture process, rendering fracture prediction considerably more challenging. In this work, we employ the standard phase-field damage model to investigate the propagation of dual cracks based on the hydrogen-enhanced decohesion mechanism (HEDE). Furthermore, we have enhanced the iteration approach of the phase-field method by introducing Anderson acceleration and over-relaxation as complementary strategies on top of the staggered solution scheme. This modification significantly reduces the number of iterations, thereby achieving efficient solutions. Additionally, this paper comprehensively considers the influence of various crack factors, under different crack arrangement configurations, on both the material's load-bearing capacity and the interaction among cracks. These arrangements include horizontal, inclined, collinear, parallel, internal, and boundary positions. The influencing factors encompass crack length, crack tilt angle, the lateral spacing between cracks, the vertical spacing between cracks, and the distance between cracks and boundaries. Through numerical simulations, universal patterns of the influence of various crack factors on the fracture behavior of a dual-crack system in a hydrogen environment were identified. It was demonstrated that when the influence regions of two cracks overlap, it significantly affects their propagation paths and reduces the model's load-carrying capacity.

Anderson acceleration for partially observable Markov decision processes: A maximum entropy approach

Partially observable Markov decision processes (POMDPs) is a rich mathematical framework that embraces a large class of complex sequential decision-making problems under uncertainty with limited observations. However, the complexity of POMDPs poses various computational challenges, motivating the need for an efficient algorithm that rapidly finds a good enough suboptimal solution. In this paper, we propose a novel accelerated offline POMDP algorithm exploiting Anderson acceleration (AA) that is capable of efficiently solving fixed-point problems using previous solution estimates. Our algorithm is based on the Q-function approximation (QMDP) method to alleviate the scalability issue inherent in POMDPs. Inspired by the quasi-Newton interpretation of AA, we propose a maximum entropy variant of QMDP, which we call soft QMDP, to fully benefit from AA. We prove that the overall algorithm converges to the suboptimal solution obtained by soft QMDP. Our algorithm can also be implemented in a model-free manner using simulation data. Provable error bounds on the residual and the solution are provided to examine how the simulation errors are propagated through the proposed algorithm. Finally, the performance of our algorithm is tested on several benchmark problems. According to the results of our experiments, the proposed algorithm converges significantly faster without degrading the solution quality compared to its standard counterparts.

Anderson acceleration. Convergence analysis and applications to equilibrium chemistry

In this paper, we study theoretically and numerically the Anderson acceleration method. First, we extend the convergence results of Anderson's method for a small depth to general nonlinear cases. More precisely, we prove that the Type-I and Type-II Anderson(1) are locally q-linearly convergent if the fixed point map is a contraction with a Lipschitz constant small enough. We then illustrate the effectiveness of the method by applying it to the resolution of chemical equilibria. This test case has been identified as a challenging one because of the high nonlinearity of the chemical system and stiffness of the transport phenomena. The Newton method (usually Newton-Raphson) has been adopted by quite all the equilibrium and reactive transport codes. But the often ill-conditioned Jacobian matrix and the choice of a bad initial data can lead to convergence problems, especially if solute transport produces sharp concentrations profiles. Here we propose to combine the Anderson acceleration method with a particular formulation of the equilibrium system called the method of positive continued fractions (usually used as preconditioning). As shown by the numerical simulations, this approach makes it possible to considerably improve the robustness of the resolution of chemical equilibria algorithms, especially since it is coupled with a strategy to monitor the depth of the Anderson acceleration method in order to control the condition number.

The Descartes Code (Spin Orbital Rotation of Photons)–II. The Gravitational Redshift

A new alternative model to the Schwarzschild’s solution of the Einstein's gravitational theory is presented. Descartes’s code is the old color theory (published in 1637) based on the rotation of “light globules”, however, it was rejected by Newton in 1672. In this model, the photon spin-orbital speed is modified by the gravitational field. The formulae for the wavelength, frequency, local time, momentum, energy, and temperature of photons in the gravitational field are identical in the first order both for the Schwarzschild solution and Descartes’ code. To experimentally determine the predictions of Descartes’ photon model we have to search for the experiments in the second order or for the situations where the standard model has to postulate some hidden elements (e.g., the dark matter). Some examples are presented here. The new interpretations of the bending of light by the Sun and the Shapiro time delay are given. The Anderson acceleration constant known as the Pioneer anomaly was derived as the effect of the Earth’s gravitational field on photons. Similar acceleration could be found in invaluable data from the Laser Lunar Ranging experiment. Stellar photons are modified by the Solar gravitational field at the distance of 1 AU and the acceleration constant identical to the Milgrom acceleration (MOND gravitational model) was derived. Therefore, we should study in more detail the influence of the Solar, Earth’s, and Moon’s gravitational fields on photons to avoid traps with false gravitational models.

Enhancing Electrical Impedance Tomography Reconstruction Using Learned Half-Quadratic Splitting Networks with Anderson Acceleration

Enhancing Electrical Impedance Tomography Reconstruction Using Learned Half-Quadratic Splitting Networks with Anderson Acceleration

Generalized modulus-based matrix splitting algorithm with Anderson acceleration strategy for vertical linear complementarity problems

The vertical linear complementarity problem (VLCP) with an arbitrary number l of matrices is related to many practical problems, of which the state-of-the-art modulus-based matrix splitting (MMS) method is proved to be an efficient solver. Enlightened by the Anderson acceleration which is a well-established and simple technique for speeding up fixed point iteration solvers with countless applications, we propose an Anderson accelerated generalized modulus-based matrix splitting (AA+GMMS) method for solving the VLCP. We particularly analyze the AA+GMMS method for the problem with l=2 and then generalize the method to any l. More importantly, the convergence theorems and theoretical optimal parameters of the MMS, AA+GMMS methods with any l are obtained in the positive definite case. Eventually, numerical experiments are given to demonstrate the effectiveness of the AA+GMMS method which significantly accelerates the original MMS method. In particular, we explore the parameters involved in the AA+GMMS method, and they have a small extent of impact on the suggested method, reinforcing that the AA+GMMS method is highly efficient.

Practical considerations for the adoption of Anderson acceleration in nonlinear diffusion accelerated transport

The use of the Anderson Acceleration (AA) method within the context of the coarse mesh finite difference (CMFD)-accelerated neutron transport k-eigenvalue calculation is explored for practical problems. Current application of CMFD only has a linear convergence rate. The AA method is similar to a quasi-Newton method and can improve the convergence rate of fixed-point iterations. We therefore implement AA for the 2D calculation in MPACT and study its performance in pressurized water reactor (PWR) problems. The initial results show that the traditional AA method can improve the convergence rate in a trivial problem and improve the stability of the CMFD method for some problems. However, the success of AA in the practical problems, where simulations with symmetry or domain decomposition, relies on one essential factor–the treatment of iterative angular flux boundary conditions in the AA. Additionally, the efficiency of AA is also affected by the computational cost for solving the optimization problem, and the computational cost of the transport solver and the CMFD solver. To address these issues, we propose a new implementation of AA that considers only the mutligroup scalar flux in the optimization problem, but the angular flux and the scalar flux in the mixing. The new approach achieves better stability and efficiency in practical simulations, and can be explained by the theory of Randomized Numerical Linear Algebra (RandNLA).

Anderson Acceleration of Gradient Methods with Energy for Optimization Problems

Anderson Acceleration of Gradient Methods with Energy for Optimization Problems

Asymptotic convergence analysis and influence of initial guesses on composite Anderson acceleration

Asymptotic convergence analysis and influence of initial guesses on composite Anderson acceleration

Efficient and effective algebraic splitting‐based solvers for nonlinear saddle point problems

The incremental Picard Yosida (IPY) method has recently been developed as an iteration for nonlinear saddle point problems that is as effective as Picard but more efficient. By combining ideas from algebraic splitting of linear saddle point solvers with incremental Picard‐type iterations and grad‐div stabilization, IPY improves on the standard Picard method by allowing for easier linear solves at each iteration—but without creating more total nonlinear iterations compared to Picard. This paper extends the IPY methodology by studying it together with Anderson acceleration (AA). We prove that IPY for Navier–Stokes and regularized Bingham fits the recently developed analysis framework for AA, which implies that AA improves the linear convergence rate of IPY by scaling the rate with the gain of the AA optimization problem. Numerical tests illustrate a significant improvement in convergence behavior of IPY methods from AA, for both Navier–Stokes and regularized Bingham.