The giant monopole, dipole and quadrupole responses in $^{40}$Ca, $^{90}$Zr, $^{120}$Sn and $^{208}$Pb are investigated using linear response treatment based on a stochastic one-body transport theory. Effects of the coupling to low-lying surface modes (coherent mechanism) and the incoherent mechanism due to nucleon-nucleon collisions are included beyond the usual mean-field description. We emphasize the importance of both mechanism in the fragmentation and damping of giant resonance. Calculated spectra are compared with experiment in terms of percentage of Energy-Weighted Sum-Rules in various energy regions. We obtained reasonable agreement in all cases. A special attention as been given to the fragmentation of the Giant Quadrupole Resonance in calcium and lead. In particular, the equal splitting of the $2^{+}$ in $^{40}$Ca is correctly reproduced. In addition, the appearance of fine structure in the response $^{208}$Pb is partly described by the calculations in which the coherent mechanism play an important role.