This research work investigates the response of nanocrystalline lanthanum zirconium oxide (LZO) under neutron irradiation, emphasizing grain size-dependent structural stability. LZO samples were synthesized with the chemical precipitation method and the LZO pellets were annealed at two different temperatures (1200 °C and 1400 °C) to obtain two different grain size samples, namely LZO-1200 and LZO-1400. The samples were exposed to neutrons at different doses (ranging from 1.63 × 1012 n/cm2 to 5.68 × 1014 n/cm2) and the effects were studied with X-ray diffraction, scanning electron microscopy, Raman scattering and optical characterization. The structural characterization results reveal that, the grain size and strain are significantly modified in LZO-1400 samples, and such changes are absent inLZO-1200 samples. The band gap and photoluminescence intensity are reduced significantly in LZO-1400 samples, however, this effect is not observed in LZO-1200 samples. The experimental results reveal that in LZO-1200 samples, the grain boundaries act as a sink for irradiation-induced defects and there is no significant change in the grain size, strain and band gap. However, in the case of the LZO-1400 sample, the grain size is larger than the cascades induced by primary knock-on atoms (PKAs) and it results in disorder-driven fast grain growth and changes in band gap and PL intensity. The LZO-1200 samples with small grain sizes are radiation resistant.