Numerically Stable Polynomially Coded Computing

Abstract

We study the numerical stability of polynomial based encoding methods, which has emerged to be a powerful class of techniques for providing straggler and fault tolerance in the area of coded computing. Our contributions are as follows: 1)We construct new codes for matrix multiplication that achieve the same fault/straggler tolerance as the previously constructed MatDot Codes and Polynomial Codes.2)We show that the condition number of every m ×m sub-matrix of an m ×n, n ≥ m Chebyshev-Vandermonde matrix, evaluated on the n-point Chebyshev grid, grows as O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2(n-m)</sup> ) for n > m.3)By specializing our orthogonal polynomial based constructions to Chebyshev polynomials, and using our condition number bound for Chebyshev-Vandermonde matrices, we construct new numerically stable techniques for coded matrix multiplication. We empirically demonstrate that our constructions have significantly lower numerical errors compared to previous approaches which involve inversion of Vandermonde matrices. We generalize our constructions to explore the trade-off between computation/communication and fault-tolerance.4)We propose a numerically stable specialization of Lagrange coded computing. Our approach involves the choice of evaluation points and a suitable decoding procedure. Our approach is demonstrated empirically to have lower numerical errors as compared to standard methods.