The initial conditions for Newtonian $N$-body simulations are usually generated by applying the Zel'dovich approximation to the initial displacements of the particles using an initial power spectrum of density fluctuations generated by an Einstein-Boltzmann solver. We show that in most gauges the initial displacements generated in this way receive a first-order relativistic correction. We define a new gauge, the $N$-body gauge, in which this relativistic correction vanishes and show that a conventional Newtonian $N$-body simulation includes all first-order relativistic contributions (in the absence of radiation) if we identify the coordinates in Newtonian simulations with those in the relativistic $N$-body gauge.