We have examined the influence of electron irradiation and rapid thermal annealing on photoluminescence emission from GaAsNBi alloys. Electron irradiation of a 1-eV compressively strained GaNAsBi-on-GaAs epilayer, grown by molecular beam epitaxy and subsequently rapidly thermally annealed, is found to induce much stronger photoluminescence than what is observed for an identical as-grown sample upon annealing. At the same time, annealing of the irradiated sample caused a negligible spectral blueshift and reduced alloy potential energy fluctuations. These irradiation-related phenomena occurred without a change in the alloy macroscopic composition as revealed by x-ray diffraction and are mainly related to the nitrogen incorporated into non-substitutional sites in the quaternary alloy.