Colloidal quantum dots (QDs) without toxic heavy metals are promising candidates for exploiting the next‐generation solar cells by virtue of their numerous merits, including solution processability, facile bandgap tunability and wide absorption spectrum range. However, a large number of photo‐generated carriers are captured by their inherent defects instead of effective dissociation, which immensely hinders the device performance. Herein, the silver‐gold‐selenide (AgAuSe) QDs with well‐treated defect states were firstly employed as the photo‐absorber of active layer to construct environment‐friendly QD solar cells, in which an inverted AgAuSe QD solar cell composed of ITO/SnO2/AgAuSe/Au/PCE10:BTP‐4Cl/MoO3/Al was fabricated. Thereinto, after the n‐i‐p infrastructure of SnO2/QD/PCE10 being determined, a layer of Au nanoparticles (NPs) was introduced into the QD/PCE10 interface to modulate the energy‐level alignment of the device. This interfacial modification significantly improved the device performance by increasing the open‐circuit voltage and fill factor, which was different from the absorption enhancement mechanism originated from the plasmonic effects of Au NPs. Lastly, the PCE10 layer was blended with BTP‐4Cl to increase the short‐circuit current density of device through its complementary absorption to AgAuSe QD, which further increased the power conversion efficiency up to 4.14% with the synergistic effect of Au NPs layer.