We studied the effects of defects on coherent phonons in ion-implanted bismuth, graphite and GaAs. Ultrafast dynamics of coherent phonons in the time-domain has been investigated by means of femtosecond pump–probe reflectivity techniques. For bismuth the dephasing rate of the A 1g phonon increases linearly with increasing ion dose, which is explained by the additional dephasing of coherent phonons by the defect scattering. For graphite, the coherent acoustic phonon relaxation is also accelerated due to the additional scattering by defects. The linear dose-dependence of the decay rate is understood as scattering of propagating acoustic phonons by single vacancies. The coherent LO phonon–plasmon coupled modes are annihilated by the defect-induced carrier trapping.