Fluorite is one of the most common minerals in the crust and is of widespread economic importance. It shows strong UV-excited luminescence, variously attributed to defects within the fluorite structure and lanthanide substitutions. We present here a detailed chemical characterisation of a suite of natural fluorite samples, chosen to represent the range of compositions observed in nature. We perform X-ray excited luminescence spectroscopy on the samples as a function of temperature (20–673 K) in the wavelength range 250–800 nm to provide insights into physical defects in the lattice and their interactions with lanthanide substituents in natural fluorite. Most broad bands in the UV are attributed to electronic defects in the fluorite lattice, whereas sharp emissions are attributed to intra-ion energy cascades in trivalent lanthanides. Lanthanides are accommodated in fluorite by substitution for Ca2+ coupled with interstitial F−, O2− (substituting for F−) and a variety of electronic defect structures which provide local charge balance. The chondrite-normalised lanthanide profiles show that fluorite accommodates a greater proportion of heavy lanthanides (and Y) as the total Rare Earth Element (REE) concentration increases; whereas cell parameters decrease and then increase as substitution continues. Luminescence intensity also goes through a maximum and then decreases as a function of REE concentration. All three datasets are consistent with a model whereby lanthanides initially act as isolated centres, but, beyond a critical threshold (~ 1000 ppm), cluster into lanthanide-rich domains. Clustering results in shorter REE-O bond distances (favouring smaller heavier ions), a larger unit cell but more efficient energy transfer between lanthanides, thereby promoting non-radiative energy loss and a drop in the intensity of lanthanide emission.