The incoherent inelastic neutron scattering (IINS) spectra of solid p-xylenes with different deuterated molecules have been investigated. The lattice dynamics model based on the atom-atom potential set of parameters, fitted to the dynamics of solid benzene, cannot be applied directly to p-xylene. By decreasing the potential depth for the CH/D and H/DH/D interactions of the methyl groups with the neighbouring atoms, we were able to get the correct energies for their torsional vibrations and to show how they are coupled to the lattice modes. One set of atom-atom potential parameters can satisfactorily describe the lattice dynamics of four p-xylenes with differently deuterated molecules.