We study the phonon-assisted relaxation processes (longitudinal relaxation time T1) within the radiative doublet of the heavy-hole-exciton in asymmetrical quantum dots. Two different relaxation mechanisms are considered: the exciton spin–acoustic phonon coupling via the strain-dependent short-range exchange interaction and the second-order quasielastic interaction between charge carriers and LO phonons. For zero magnetic fields and low temperatures, the calculated relaxation times for typical QDs are very long compared to the exciton lifetime yet they are strongly reduced in high magnetic fields (of the order of a few Tesla) and high temperatures T ≥ 100 K.