Iterative Methods In Linear Algebra Research Articles

Quantum simulation of discrete linear dynamical systems and simple iterative methods in linear algebra

Quantum simulation is capable of simulating certain dynamical systems in continuous time—Schrödinger’s equations being the most direct and well known—more efficiently than classical simulation. Any linear dynamical system can in fact be transformed into a system of Schrödinger’s equations via a method called Schrödingerisation (Jin et al. 2022. Quantum simulation of partial differential equations via Schrödingerisation. ( https://arxiv.org/abs/2212.13969 ) and Jin et al. 2023. Phys. Rev. A 108 , 032603. ( doi:10.1103/PhysRevA.108.032603 )). We show how Schrödingerisation allows quantum simulation to be directly used for the simulation of continuous-time versions of general (explicit) iterative schemes or discrete linear dynamical systems. In particular, we use this new method to solve linear systems of equations and for estimating the maximum eigenvector and eigenvalue of a matrix, respectively. This method is applicable using either discrete-variable quantum systems or on hybrid continuous-variable and discrete-variable quantum systems. This framework provides an interesting alternative to solve linear algebra problems using quantum simulation.

The combination technique applied to functionals

Functionals related to a solution of a problem, usually modelled by partial differential equations, can be important quantities used to capture features of the problem. For high dimensional problems the computational cost of the functionals can be large since the numerical solution of a high dimensional partial differential equation is usually expensive to compute. We develop a new sparse grid combination technique to reduce the computational cost of such functionals. Our method is based on error splitting models of the functionals. However, it is hard to obtain a concrete error splitting model for complicated approximations. We show the connection between the decay of the surpluses and the error splitting models. By using the connection, we can also apply our combination technique to functionals when we only know their computed surpluses. Numerical experiments are provided to illustrate our idea and test the performance of our method.

Helmholtz Scattering by Random Domains: First-Order Sparse Boundary Element Approximation

We consider the numerical solution of time-harmonic acoustic scattering by obstacles with uncertain geometries for Dirichlet, Neumann, impedance, and transmission boundary conditions. In particular, we aim to quantify diffracted fields originated by small stochastic perturbations of a given relatively smooth nominal shape. Using first-order shape Taylor expansions, we derive tensor deterministic first-kind boundary integral equations for the statistical moments of the scattering problems considered. These are then approximated by sparse tensor Galerkin discretizations via the combination technique [M. Griebel, M. Schneider, and C. Zenger, A combination technique for the solution of sparse grid problems, in Iterative Methods in Linear Algebra, P. de Groen and P. Beauwens, eds., Elsevier, Amsterdam, 1992, pp. 263--281; H. Harbrecht, M. Peters, and M. Siebenmorgen, J. Comput. Phys., 252 (2013), pp. 128--141]. We supply extensive numerical experiments confirming the predicted error convergence rates with polylogarithmic growth in the number of degrees of freedom and accuracy in approximation of the moments. Moreover, we discuss implementation details such as preconditioning to finally point out further research avenues.

Non-interior-point smoothing Newton method for CP revisited and its application to support vector machines

Non-interior-point smoothing Newton method (SNM) for optimization have been widely studied for over three decades. SNM is a popular approach for solving small- and medium-scale complementarity problem (CP) and many optimization problems. The main purpose of this paper is to revisit the SNM and show that the Hessian matrix in SNM becomes increasingly ill-conditioned while smoothing parameter approaches to zero, which leads to their practical use remains limited due to computational difficulties in solving large-scale CP. To tackle this, we redesign a new smoothing method, called accelerated preconditioned smoothing method (APSM) for the efficient solution of regularized support vector machines in machine learning. With the help of suitable preconditioner, we can correct the ill-conditioning of associated smoothing Hessian matrix and thereby the associated smoothing Hessian equation can be solved in a few of iterations by using iterative methods in linear algebra. Two accelerated techniques are designed in the paper to reduce our computation time. Finally we present numerical experiments to support our theoretical guarantees and test accelerated convergence obtained by APSM. The result showed that APSM computes faster than state of art algorithm without reducing classification accuracy.

Analytical Jacobian-vector products for the matrix-free time integration of partial differential equations

Many scientific and engineering applications require the solution of large systems of initial value problems arising from method of lines discretization of partial differential equations. For systems with widely varying time scales, or with complex physical dynamics, implicit time integration schemes are preferred due to their superior stability properties. These schemes solve at each step linear systems with matrices formed using the Jacobian of the right hand side function. For large applications iterative linear algebra methods, which make use of Jacobian-vector products, are employed. This paper studies the impact that the method of computing Jacobian-vector products has on the overall performance and accuracy of the time integration process. The analysis shows that the most beneficial approach is the direct computation of exact Jacobian-vector products in the context of matrix-free time integrators. This approach does not suffer from approximation errors, reuses the parallelism and data distribution already present in the right-hand side vector computations, and avoids storing or operating on the entire Jacobian matrix.

A robust combination technique

One of the challenges for efficiently and effectively using petascale and exascale computers is the handling of run-time errors. Without such robustness, applications developed for these machines will have little chance of completing successfully. The sparse grid combination technique approximates the solution to a given problem by taking the linear combination of its solution on multiple grids. It is successful in many high performance computing applications due to its ability to tackle the curse of dimensionality. We present several approaches to fault tolerance using the combination technique. The first of these is implemented within the MapReduce model in order to utilise the existing fault tolerance of this framework. In addition, we present a method which utilises the redundancy shared by solutions on different grids. Finally, we describe a novel approach in which the solution is computed on additional grids which are used for alternative combinations if other grids experience failure. We include some results based on the solution of the 2D scalar advection PDE. References S. Balay, J. Brown, K. Buschelman, W. D. Gropp, D. Kaushik, M. G. Knepley, L. C. McInnes, B. F. Smith, and H. Zhang. PETSc Web page (2012). http://www.mcs.anl.gov/petsc. G. Bosilca, R. Delmas, J. Dongarra, and J. Langou. Algorithm-based fault tolerance applied to high performance computing. Journal of Parallel and Distributed Computing, 69(4):410--416 (2009). doi:10.1016/j.jpdc.2008.12.002. H. J. Bungartz and M. Griebel. Sparse grids. Acta Numerica, 13:147--269 (2004). doi:10.1017/S0962492904000182. F. Cappello. Fault Tolerance in Petascale/Exascale Systems: Current Knowledge, Challenges and Research Opportunities. International Journal of High Performance Computing Applications, 23(3):212--226, (2009). doi:10.1177/1094342009106189. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1):107--113 (2008). doi:10.1145/1327452.1327492. J. Garcke. Sparse grids in a nutshell. In J. Garcke and M. Griebel, editors, Sparse grids and applications, volume 88 of Lecture Notes in Computational Science and Engineering, pages 57--80. Springer (2013). doi:10.1007/978-3-642-31703-3_3. M. Griebel, M. Schneider, and C. Zenger. A combination technique for the solution of sparse grid problems. In P. de Groen and R. Beauwens, editors, Iterative Methods in Linear Algebra, pages 263--281. IMACS, Elsevier, North Holland (1992). Zbl 0785.65101. M. Hegland. Adaptive sparse grids. ANZIAM Journal, 44:C335--C353 (2003). http://journal.austms.org.au/ojs/index.php/ANZIAMJ/article/view/685. K.-H. Huang and J. A. Abraham. Algorithm-based fault tolerance for matrix operations. Computers, IEEE Transactions on, C-33(6):518--528 (1984). doi:10.1109/TC.1984.1676475. C. Zenger. Sparse Grids. In W.Hackbusch, editor, Parrallel Algorithms for Partial Differential Equations, Proceedings of the Sixth GAMM-Seminar, Kiel, 1990, volume 31 of Notes on Num. Fluid Mech., Vieweg--Verlag, 31:241--251 (1991). Zbl 0763.65091.

One dimensional combination technique and its implementation

This article introduces the 1D combination technique and its implementation with parallel programming. I discuss two primary features of the 1D combination technique: (1)~its reduction of computational cost, especially when combined with parallel programming and where high accuracy is required; and (2)~a resultant sacrifice of accuracy. However, the loss of the accuracy can be bounded thus reducing its significance. References C. Zenger. Sparse grids. in Parallel Algorithms for Partial Differential Equations, Proceedings of the Sixth GSAMMSeminar, Kiel, January 19--21, 1990, W. Hackbusch, ed., Braunschweig, 1991, Vieweg--Verlag. M. Griebel, M. Schneider, C. Zenger. A combination technique for the solution of sparse grid problems. PdeGroen, R.Buwens(Ed.). Iterative Methods in Linear Algebra, IMACS, Elsevier, North Holland, 1992, pp.263--281. http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.3530 C. Pflaum and A. Zhou. Error analysis of the combination technique. Numerische Mathematik, 84:327--350, 1999. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.54.1356 C. Pflaum. Convergence of the combination technique for second-order elliptic differential equations. SIAM J. Numer. Anal., 34(6):2431--2455, 1997. http://www.jstor.org/pss/2951959 U. Rude and A. Zhou. Multi-parameter extrapolation methods for boundary integral equaitons. Advances in Computational Mathematics, 9:173--190, 1998. http://www.springerlink.com/content/m31l311t32345607/

Automatic spectral collocation for integral, integro-differential, and integrally reformulated differential equations

Automatic Chebyshev spectral collocation methods for Fredholm and Volterra integral and integro-differential equations have been implemented as part of the chebfun software system. This system enables a symbolic syntax to be applied to numerical objects in order to pose and solve problems without explicit references to discretization. The same objects can be used in matrix-free iterative methods in linear algebra, in order to avoid very large dense matrices or allow application to problems with nonsmooth coefficients. As a further application of the ability to implement operator equations, a method of Greengard [1] for the recasting of differential equations as integral equations is generalized to mth order boundary value and generalized eigenvalue problems. In the integral form, large condition numbers associated with differentiation matrices in high-order problems are avoided. The ability to implement the recasting process generally follows from implementation of the operator expressions in chebfun. The integral method also can be extended to first-order systems, although chebfun syntax does not currently allow easy implementation in this case.

Ordering strategies and related techniques to overcome the trade-off between parallelism and convergence in incomplete factorizations

This paper is concerned with the parallel implementation of the incomplete factorization preconditioned iterative method. Although the use of such parallel ordering as multicolor ordering may increase parallelism in factorization, it often slows convergence when used in the preconditioned method, and thus may offset the gain in speed obtained with parallelization. Further, the higher the parallelism of an ordering, the slower the convergence; the lower the parallelism, the faster the convergence. This well-known trade-off between parallelism and convergence is well explained by the property of compatibility, the level of which can be clearly seen when ordering is presented in graph form (S. Doi, A. Lichnewsky, A graph-theory approach for analyzing the effects of ordering on ILU preconditioning, INRIA report 1452, 1991). In any given method, the fewer the incompatible local graphs in an ordering (i.e., the lower the parallelism), the faster the convergence (S. Doi, Appl. Numer. Math. 7 (1991) 417–436; S. Doi, in: T. Nodera (Ed.), Advances in Numerical Methods for Large Sparse Sets of Linear Systems, 7 Keio University, 1991). An ordering with no incompatible local graphs, for example, such as that implemented on vector multiprocessors by using the nested dissection technique, will have excellent convergence, but its parallelism will be limited (S. Doi, A. Lichnewsky, Int. J. High Speed Comput. 2 (1990) 143–179). To attain a better balance, a certain degree of incompatibility is necessary. In this regard, increasing the number of colors in multicolor ordering can be a useful approach (S. Fujino, S. Doi, in: R. Beauwens (Ed.), Proceeding of the IMACS Internation Symposium on Iterative Methods in Linear Algebra, March 1991; S. Doi, A. Hoshi, Int. J. Comput. Meth. 44 (1992) 143–152). Two related techniques also presented here are the overlapped multicolor ordering (T. Washio, K. Hayami, SIAM J. Sci. Comput. 16 (1995) 631–650), and a fill-in strategy selectively applied to incompatible local graphs. Experiments conducted with an SX-5/16A vector parallel supercomputer show the relative effectiveness of increasing the number of colors and also of using this approach in combination with overlapping and with fill-ins.