We study scalar field theories invariant under transverse diffeomorphismsin cosmological contexts. We show that in the geometric optics approximation, the corresponding particles move along geodesics and contribute with the same active mass (energy) to the gravitational field as in Diff invariant theories. However, for low-frequency (super-Hubble) modes, the contributions to the energy-momentum tensor differ from that of Diff invariant theories. This opens up a wide range of possibilities for cosmological model building. As an example, we show that the simplest TDiff invariant scalar field theory with only kinetic term could drive inflation and generate a nearly scale invariant (red-tilted) spectrum of density fluctuations. We also present a detailed analysis of cosmological perturbations and show that the breaking of full Diff invariance generically induces new non-adiabatic pressure perturbations. A simple scalar field dark matter model based on a purely kinetic term that exhibits the same clustering properties as standard cold dark matter is also presented.