High-resolution transmission electron microscopy, electron diffraction, and electron energy-loss spectroscopy provide information on the structural evolution, dielectric function, and bandgap values of nanocrystalline 10 nm thick lanthanum doped hafnia (La:HfO2) layers in TiN/La:HfO2/TiN/SiO2/Si irradiated with 24, 46, 72, and 160 MeV (0.2–1.2 MeV/u) Xe ions. Swift heavy Xe ions were expected to create significant atomic rearrangements when passed through a solid losing energy mainly through electronic excitation and ionization of the target atoms. Local heating and subsequent rapid cooling in the region around the ion track can lead to re-solidification with the formation of a new stable crystalline phase, and direct crystal-to-crystal transformations are possible. The structure evolution of hafnia nanocrystals from the orthorhombic Pbcm in the pristine layer to the tetragonal P42/nmc phase in the 160 MeV Xe ion irradiated layer was observed. The mixture of tetragonal and orthorhombic phases was found in samples irradiated with ions of intermediate energies. Textured hafnia layers were formed as a result of ion irradiation. The changes in plasmon line shape and the blueshift of the plasmon energy-loss peak from 14.9 eV in the pristine layer to 15.4 eV in 46 MeV and 15.9 eV in 160 MeV Xe ion irradiated hafnia reflect structural transformations, the increase in the Hf coordination number, and crystal orientations. Valence-electron energy-loss spectroscopy measurements showed a slight increase in the bandgap value from 6.1 eV in the pristine sample to 6.2 eV and 6.3 eV in irradiated samples with 46 and 160 MeV Xe ions, respectively, and dielectric functions changed insignificantly in irradiated hafnia layers.