Under 808 nm excitation into the Tm3+:3H4 level, a considerable enhancement in intensity of Tm3+: 3F4→3H6 emission with respect to Tm3+: 3H4→3F4 emission is observed in Tm3+/Yb3+ codoped Lu1.6Sc0.4O3. The Tm3+-Yb3+-Tm3+ forward-backward energy transfer is proved to generate an additional route for the 3H4→3F4 nonradiative relaxation, that is, energy transfer from Tm3+:3H4 to Yb3+:2F5/2 and the subsequent back transfer from Yb3+:2F5/2 to Tm3+:3F4. The analysis of emission spectra reveals that back transfer from Yb3+ that excited by the forward energy transfer is more efficient than by absorption of 980 nm infrared light. The efficiency can reach as high as 96% with an extremely low Tm3+ concentration (0.05%). We propose that those Yb3+ ions with nearby Tm3+ ions in the forward energy transfer are preferentially excited instead of equally excited as by absorption of light. The efficiencies of the energy back transfer at different Yb3+ concentrations are evaluated, indicating that the forward-backward energy transfer can act as a dominant route for the Tm3+: 3H4→3F4 nonradiative relaxation when Yb3+ concentration is higher than 5%. A method to determine the radiative rate of Tm3+:3H4 state based on the model of cross relaxation is also demonstrated.