The understanding of complex relationships between luminescent properties, local symmetry of an emitting center, and the host crystal structure provides a better insight into optical properties of materials. In this work, the alkaline earths CaF2 and BaF2 fluoride crystals doped with 0.1 mol% ErF3 were investigated. The crystals structure has been studied using a synchrotron and laboratory X-ray diffraction. The C3v and C4v sites symmetry were determined using erbium probed high resolution emission spectroscopy (HRPL) at low temperature (LT) of 10 K. The considerable difference in room temperature (RT) optical properties for CaF2 compared to BaF2 crystals was observed. Such difference in absorption intensity of 4.7 times of erbium 4G11/2 manifold in UV, and 7.5 times in green emission from 4S3/2 manifold, could be due to the distinction in the host crystals cationic radius (ΔrCa,Ba) and the dopant-host ionic radius (ΔrCa-Er, ΔrBa-Er). Those Δr differences influence the structure and lead to the following symmetry formation: In CaF2, the C4v and C3v isolated centers were identified, with the determined Er3+- F−i bond lengths of 2.734 Å and 4.735 Å respectively; In BaF2, only C3v isolated centers were identified with the determined Er3+- F−i bond lengths of 5.380 Å. The present work is the first study that takes into account correlations of optical properties, the local symmetry and the structure in mentioned fluorides crystals, and it could be a step forward in the lanthanide doped optical materials systematics.