In order to enhance the efficiency of direct absorption solar collectors, this study carried out an experimental analysis about the optical and photothermal conversion performance of Fe3O4, ATO (Antimony-doped tin oxide), and Fe3O4/ATO nanofluids with a total concentration of 0.1 wt%. According to the results of the experiments, Fe3O4 nanofluid outperforms ATO nanofluid in terms of optical absorption; nevertheless, at wavelengths shorter than 600 nm, it also shows significant scattering reflection. The solar-weighted absorption coefficient of Fe3O4/ATO nanofluid rose from 0.863 (mFe3O4/mTotal = 0.2) to 0.932 (mFe3O4/mTotal = 0.8) when the optical path length increased from 0.01 m to 0.06 m. Moreover, the Fe3O4/ATO hybrid nanofluid achieved a photothermal conversion efficiency of 0.932 when the mass ratio of Fe3O4 to total mass was 0.2, surpassing the efficiencies of 0.892 and 0.898 recorded for 0.1 wt% ATO and Fe3O4 nanofluids, respectively. When present together, the opposing optical characteristics of Fe3O4 and ATO boost photothermal conversion performance, which is anticipated to raise the efficiency of direct absorption solar collectors.