Abstract Coherent acoustic phonon (CAP) oscillation of 2D layered semiconductor/3D dielectric heterostructure generated by femtosecond laser pulse excitation can realize ultrafast photoacoustic conversion, while the photoacoustic conversion efficiency suffers from interfacial phonon scatterings of simultaneously laser-induced lattice heat. Here, taking advantage of graphene’s high thermal conductivity and large acoustic impedance, we demonstrate that phonon scatterings can be markedly mediated in a MoS$_2$/graphene/glass heterostructure via femtosecond laser pump-probe measurements. Equilibrium temperatures of MoS$_2$ lattice have been cooled down by about 45 %. As a benefit, both lifetime of CAP oscillations and pump pulse-picosecond acoustic pulse energy conversion efficiency have been enhanced by a factor of about 2. Our results constitute insights into CAP manipulations via interfacial engineering, that are fundamentally important for ultrafast photoacoustics based on 2D layered semiconductors.