Understanding the viscoelastic properties of cement paste is beneficial to pumping, formwork casting and 3D printing of cement-based materials. This paper presents new insights into the viscoelastic properties of cement paste with various compositions. Several parameters, including critical strain, storage modulus and viscoelastic yield stress obtained from small amplitude oscillatory shear (SAOS) test, are selected to characterize the viscoelasticity of fresh cementitious paste. Results reveal that increasing water-to-cement (w/c) ratio reduces the storage modulus at linear viscoelastic region (LVER) and viscoelastic yield stress of cement paste, whereas higher w/c increases the critical strain. The replacement of cement by fly ash has no significant influence on the critical strain of cement paste. Due to the improvement of cohesive bonding between cement particles by nanoparticles, the incorporation of nano-Fe3O4 particles results in an increase in the storage modulus at LVER, critical strain and viscoelastic yield stress. The critical strain of cement paste gradually increases with the concentration of polycarboxylate ether (PCE) superplasticizer, which possibly can be attributed to the interactions and entanglement of PCE molecules adsorbed onto the solid particles. By contrast, cement pastes with low PCE additions exhibit an increase in the viscoelastic yield stress, while higher PCE additions significantly decrease the storage modulus at LVER and viscoelastic yield stress of cement paste.