We have measured the thermal conductivity parallel to the a-axis of the Zn-free and 1% Zn-substituted SrCu$_{2-x}$Zn$_x$(BO$_3$)$_2$ in magnetic fields up to 14 T, in order to examine the thermal conductivity due to the multi-triplet magnons. It has been found that the thermal conductivity peak observed in the spin gap state is suppressed by the substitution of Zn for Cu in high magnetic fields above 6 T, while it is not changed in low magnetic fields below 6 T. The results suggest that the thermal conductivity peak in the spin-gap state of SrCu$_2$(BO$_3$)$_2$ is composed of not only thermal conductivity due to phonons but also that due to the multi-triplet magnons in high fields above 6 T.