We present optical spectroscopy and near-infrared photometry of 57 faint ($g= 19-22$) high proper motion white dwarfs identified through repeat imaging of $\approx3100$ square degrees of the Sloan Digital Sky Survey footprint by Munn et al. (2014). We use $ugriz$ and $JH$ photometry to perform a model atmosphere analysis, and identify ten ultracool white dwarfs with $T_{\rm eff}<4000$ K, including the coolest pure H atmosphere white dwarf currently known, J1657+2638, with $T_{\rm eff}= 3550 \pm 100$K. The majority of the objects with cooling ages larger than 9 Gyr display thick disc kinematics and constrain the age of the thick disc to $\geq11$ Gyr. There are four white dwarfs in our sample with large tangential velocities ($v_{tan} > 120$ km $s^{-1}$) and UVW velocities that are more consistent with the halo than the Galactic disc. For typical $0.6 M_{\odot}$ white dwarfs, the cooling ages for these halo candidates range from 2.3 to 8.5 Gyr. However, the total main-sequence + white dwarf cooling ages of these stars would be consistent with the Galactic halo if they are slightly undermassive. Given the magnitude limits of the current large scale surveys, many of the coolest and oldest white dwarfs remain undiscovered in the solar neighborhood, but upcoming surveys such as GAIA and the Large Synoptic Survey Telescope (LSST) should find many of these elusive thick disc and halo white dwarfs.