In order to compute a vibrational spectrum, one often wishes to start with a set of ab initio Born–Oppenheimer potential values at points, called fitting points, and interpolate or fit to find values of the potential at quadrature or collocation points. It is common to do this once to build a potential energy surface (PES). Once the PES is known, it can be evaluated at any point in configuration space. Gaussian Process (GP) is frequently being used to make a PES. As is the case in other interpolation methods, to use GP one must store and invert a matrix whose size is the number of fitting points. The matrix is sometimes large enough that approximations are introduced to reduce the cost of the calculation. We show that is possible to use many local Gaussian Process fits rather than one global fit. Retaining only local Gaussians and the associated points works well despite the fact that other Gaussians have tails with significant amplitude in the local region. We demonstrate that from the potential values obtained from the local fits it is possible to compute accurate energy levels of formaldehyde. In one calculation, potential values were obtained with N=120,000 fitting points by inverting matrices of size less than m=400. The local idea reduces the cost from N3 to T(m3+N), where T is the number of desired potential points.