The ultrafast relaxation dynamics of a nonequilibrium phonon gas towards thermal equilibrium involves many-body collisions that cannot be properly described by perturbative approaches. Here, we develop a nonperturbative method to elucidate the microscopic mechanisms underlying the decay of laser-excited coherent phonons in the presence of electron-hole pairs, which so far are not fully understood. Our theory relies on ab initio molecular dynamics simulations on laser-excited potential-energy surfaces. Those simulations are compared with runs in which the laser-excited coherent phonon is artificially deoccupied. We apply this method to antimony and show that the decay of the ${A}_{1g}$ phonon mode at low laser fluences can be accounted mainly to three-body down-conversion processes of an ${A}_{1g}$ phonon into acoustic phonons. For higher excitation strengths, however, we see a crossover to a four-phonon process, in which two ${A}_{1g}$ phonons decay into two optical phonons.