PbS colloidal quantum dot (CQD) photodiodes possess high photoresponsivity and broad spectral sensitivity, making them tremendously attractive for low-cost short-wave infrared (SWIR) sensing. A critical challenge in achieving high detectivity in SWIR photodiodes is to minimize their reverse bias dark current density (Jdark) and elucidate the origin of Jdark. We demonstrate in this work how the self-formed AlOx at the ZnO/Al interface dramatically reduced Jdark of the PbS CQD SWIR photodiodes. In such a simple and controlled way, the interfacial states were thoroughly passivated by post-exposure to dry air, which could block the dominant pathways for minority hole injection from Al electrode. The treated photodiode showed low Jdark down to 1.58 × 10−7 A/cm2 at –0.5 V, high photoresponsivity of 0.62 A/W, and high specific detectivity of 2.05 × 1012 Jones at 1413 nm, establishing a readily approach to optimizing detectivity in PbS CQD SWIR photodetectors.