Rapid sub-nanometer neuronal deformations have been shown to occur as a consequence of action potentials in vitro, allowing for optical registration of discrete axonal and synaptic depolarizations. Such optically-measured deformations are a novel signature for recording neural activity. We demonstrate this signature can be extended to in vivo measurements through recording of rapid neuronal deformations on the population level with holographic, optical phase-based recordings. Our system demonstrates, for the first time, non-invasive recordings of in vivo tissue deformation associated with population level neuronal activity, including through-skull. We confirmed this technique across a range of neural activation models, including direct epidural focal electrical stimulation, anesthetic-induced cortical deactivation, activation of primary somatosensory cortex via whisker barrel stimulation, and pharmacologically-induced seizures. Collectively, we show holographic imaging provides a pathway for high-resolution, label-free, non-invasive recording of transcranial in vivo neural activity at depth, making it highly advantageous for studying neural function and signaling.