Abstract This work does an extensive analysis of the optoelectronic and mechanical properties of the tri rutile structure type 3d transition metal Antimonate MSb2O6 (M = Fe, Co, Ni, or Zn), for the first time, utilizing the pseudo-potential plane wave approach within the density functional theory framework. When calculating the structural, optical, and mechanical properties, the exchange–correlation interactions were studied using the GGA-PBE functional, whereas when computing electronic, it is analyzed using the HSE06 hybrid functional. The equilibrium lattice parameters exhibit good agreement with the available experimental results. The electronic properties were estimated using the GGA-PBE and HSE06 functionals. Based on the calculated electronic properties with the GGA-PBE functional, the FeSb2O6, CoSb2O6, and NiSb2O6 materials exhibit metallic behavior with energy gap values of 0 eV, while ZnSb2O6 is a semiconductor with a narrow direct band gap (Γ–Γ) of 0.5 eV. Furthermore, the computed band gaps using the HSE06 functional are 0 eV, 0 eV, 1 eV (Γ–Γ), and 4 eV (Γ–Γ) for FeSb2O6, CoSb2O6, NiSb2O6, and ZnSb2O6, respectively. Density of states diagrams were used to gain deeper insights into the characteristics of the energy bands. The optical properties of these compounds, such as the dielectric function, energy loss function, conductivity, reflectivity, refractive index, and absorption coefficient were investigated over the energy range of 0 to 40 eV. The materials exhibited a high absorption coefficient and a significantly low reflectivity within the UV–vis energy spectrum. The negative cohesive energy Ecoh implies the chemical (thermodynamic) stability of the trirutile MSb2O6 (M = Fe, Co, Ni, or Zn). Mechanical stability is confirmed by applying the Born stability criteria using elastic constants (Cij). The absence of imaginary frequencies in the phonon spectrum calculations confirms the dynamic stability of the studied compounds. These results are consistent with previous experimental research on these materials in photocatalysis and gas sensor applications. On the other hand, these compounds possess exceptional high and broad optical absorption UV range, making them suitable for use in next-generation ultraviolet photodetectors.