We study the impact of stellar feedback in shaping the density and velocity structure of neutral hydrogen (HI) in disc galaxies. For our analysis, we carry out $\sim 4.6$pc resolution $N$-body+adaptive mesh refinement (AMR) hydrodynamic simulations of isolated galaxies, set up to mimic a Milky Way (MW), and a Large and Small Magellanic Cloud (LMC, SMC). We quantify the density and velocity structure of the interstellar medium using power spectra and compare the simulated galaxies to observed HI in local spiral galaxies from THINGS (The HI Nearby Galaxy Survey). Our models with stellar feedback give an excellent match to the observed THINGS HI density power spectra. We find that kinetic energy power spectra in feedback regulated galaxies, regardless of galaxy mass and size, show scalings in excellent agreement with super-sonic turbulence ($E(k)\propto k^{-2}$) on scales below the thickness of the HI layer. We show that feedback influences the gas density field, and drives gas turbulence, up to large (kpc) scales. This is in stark contrast to density fields generated by large scale gravity-only driven turbulence. We conclude that the neutral gas content of galaxies carries signatures of stellar feedback on all scales.