In this work, the conductivity of intrinsic GaSe, S doped 2.5 mass% GaSe (GaSe: S(2.5%)), and S doped 7 mass% GaSe (GaSe: S(7%)) crystals, in a frequency range of 0.3–2.5 THz, is measured by transmission terahertz time-domain spectroscopy, and fitted with Drude-Smith-Lorentz model which is introduced by lattice vibration effect. It is found that the real part of conductivity decreases with the augment of S doping, which is caused by the gradual shift of the Fermi energy level of GaSe crystals to the charge neutrality level due to the generation of substitution impurities and gap impurities by S doping, resulting in the reduction of carrier concentration. The intrinsic GaSe and GaSe: S(2.5%) have a clear lattice vibration peak at about 0.56 THz, while GaSe: S(7%) has no lattice vibration peak near 0.56 THz, which is mainly due to the S doping increasing the structural hardness of the crystal and reducing the interlayer rigidity vibration of the crystal. All three samples have the obvious narrow lattice vibration peaks at about 1.81 THz, and the intensities that first decrease and then increase with the augment of S doping, which is mainly due to the fact that a small amount of S doping can reduce the local structural defects of GaSe and weaken the intensity of the narrow lattice vibration peak, while excessive S doping can generate the β-type GaS crystal, increase the local structural defects of the crystals and the intensity of the narrow lattice vibration peak. With the increase of S doping, the intensity of the broad lattice vibration peak of GaSe crystal weakens or even disappears at about 1.07 THz and 2.28 THz, mainly due to the S doping resulting in the substitution of S for impurities and GaS gap impurities, which reduces the fundamental frequency phonon vibration intensity, thereby weakening the lattice vibration caused by the second-order phonon difference mode of the crystal. The results show that the appropriate concentration of S doping can effectively suppress the lattice vibration of GaSe crystal and reduce the conductivity and power loss in the THz band. This study provides important data support and theoretical basis for the design and fabrication of low loss THz devices.