Two-dimensional materials with tunable in-plane anisotropic infrared response promise versatile applications in polarized photodetectors and field-effect transistors. Black phosphorus is a prominent example. However, it suffers from poor ambient stability. Here, we report the strain-tunable anisotropic infrared response of a layered material Nb2SiTe4, whose lattice structure is similar to the 2H-phase transition metal dichalcogenides (TMDCs) with three different kinds of building units. Strikingly, some of the strain-tunable optical transitions are crystallographic axis-dependent, even showing an opposite shift when uniaxial strain is applied along two in-plane principal axes. Moreover, G0W0-BSE calculations show good agreement with the anisotropic extinction spectra. The optical selection rules are obtained via group theory analysis, and the strain induced unusual shift trends are well explained by the orbital coupling analysis. Our comprehensive study suggests that Nb2SiTe4 is a good candidate for tunable polarization-sensitive optoelectronic devices.