Thermoelectric thin films are of great interest for self-powering Internet of Things (IoT) nodes via energy harvesting. Heusler-type Fe2VAl attracts some attention for room-temperature thermoelectric applications. However, the large overall thermal conductivity leads to a poor performance that restricts the current investigations. Herein, we report a comprehensive study of structural, thermoelectric, and magnetic properties of Fe2V0.9Ti0.1Al thin films on quartz substrate, grown by magnetron sputtering. It is found that high-temperature annealing is more effective than the long-time annealing at the low temperatures to stabilize the L21 type structure of Fe2VAl, leading to a much larger Seebeck coefficient and slightly higher thermal conductivity. More importantly, the significant off-stoichiometry phenomenon, namely Fe-rich composition, contributes to n-type conduction of thin films. Both the picosecond and nanosecond time-domain thermoreflectance techniques were employed to successfully measure the thin-film thermal conductivities, merely one-third of bulk material, demonstrating the effectiveness of utilizing nanostructuring to reduce the thermal conductivity of Fe2VAl. Moreover, magnetic susceptibility measurements reveal that thin films are ferromagnets, while the bulk sample is characterized by paramagnetism. Our finding sheds new light on the composition-structure-property relationship of Fe2VAl thin films for future related applications.