Head direction (HD) cells, abundant in the rat postsubiculum and anterior thalamic nuclei, fire maximally when the rat's head is facing a particular direction. The activity of a population of these cells forms a distributed representation of the animal's current heading. We describe a neural network model that creates a stable, distributed representation of head direction and updates that representation in response to angular velocity information. In contrast to earlier models, our model of the head direction system accurately tracks a series of actual rat head rotations, and, using biologically plausible neurons, it fits the single-cell tuning curves of real HD cells recorded from rats executing those same rotations. The model makes neurophysiological predictions that can be tested using current technologies.