Optical coatings formed from amorphous oxide thin films have many applications in precision measurements. The Advanced Laser Interferometer Gravitational-Wave Observatory (LIGO) and Advanced Virgo use coatings of SiO2 (silica) and TiO2:Ta2O5 (titania-doped tantala) and post-deposition annealing to 500°C to achieve low thermal noise and low optical absorption. Optical scattering by these coatings is a key limit to the sensitivity of the detectors. This paper describes optical scattering measurements for single-layer, ion-beam-sputtered thin films on fused silica substrates: two samples of Ta2O5 and two of TiO2:Ta2O5. Using an imaging scatterometer at a fixed scattering angle of 12.8°, in-situ changes in the optical scatter of each sample were assessed during post-deposition annealing to 500°C in vacuum. The scatter of three of the four coated optics was observed to decrease during the annealing process, by 25-30% for tantala and up to 74% for titania-doped tantala, while the scatter from the fourth sample held constant. Angle-resolved scatter measurements performed before and after vacuum annealing suggest some improvement in three of the four samples. These results demonstrate that post-deposition, high-temperature annealing of single-layer tantala and titania-doped tantala thin films in vacuum does not lead to an increase in scatter, and may actually improve their scatter.