Bismuth (Bi)-doped glasses have the potential to be applied in tunable photonic devices owing to their broadband near-infrared (NIR) luminescence. However, it is still challenging to develop Bi-doped glasses with luminescence performance at desirable emission wavelength since it is hard to stabilize the specific luminescent Bi centers. In this work, a structural model describing the negatively charged local environments is proposed to stabilize the activated Bi centers by adding Ta2O5 and/or Al2O3 in Bi-doped silicate glasses. The results show that the structural model leads to great enhancement of the NIR emission, the broadening of the emission bandwidth and the blueshift of the emission peak, compared to the samples without Ta2O5/Al2O3 doping. The change in NIR emission performances could originate from the generation of active Bi+ centers. The latter arises from oxidation of Bi0 induced by the negatively charged local environments around Bi. Moreover, the intensity of the Bi NIR emission is enhanced, and this is attributed both to the suppression of phase separation by adding Ta2O5 and to the dispersing effect of alumina.