We study properties of modules that appear as syzygies of acyclic complexes of projective, injective, flat or flat-cotorsion modules and obtain criteria for these complexes to be contractible or totally acyclic. Our results illustrate the importance of strongly fp-injective modules in the study of these properties. We examine implications of the existence of complete resolutions (in a certain weak sense) and the finiteness of the Gorenstein projective dimension of pure-projective modules. We also use the orthogonality in the homotopy category between complexes of flat modules and pure acyclic complexes of cotorsion modules, in order to study the syzygies of acyclic complexes of flat modules. Finally, we present some applications of our results to group rings, regarding complete resolutions and cohomological periodicity.