Through integrated molecular dynamics (MD) simulations and experimental studies, we demonstrated the feasibility of an ion-irradiation-and-annealing based phonon engineering technique to enhance thermal conductivity of carbon nanotube (CNT) films. Upon ion irradiation of CNT films, both inter-tube defects and intra-tube defects are introduced. Our MD simulations show that inter-tube defects created between neighboring tubes are much more stable than intra-tube defects created on tube graphitic planes. Upon thermal annealing, intra-tube defects are preferentially removed but inter-tube defects stay. Consequently, axial phonon transport increases due to reduced phonon scattering and off-axial phonon transport is sustained due to the high stability of inter-tube defects, leading to a conductivity enhancement upon annealing. The modeling predictions agree with experimental observations that thermal conductivities of CNT films were enhanced after 2 MeV hydrogen ion irradiations and conductivities were further enhanced upon post irradiation annealing.