We use complementary optical spectroscopy methods to directly measure the lowest crystal-field energies of the rare-earth quantum magnet LiY$_{1-x}$Ho$_{x}$F$_{4}$, including their hyperfine splittings, with more than 10 times higher resolution than previous work. We are able to observe energy level splittings due to the $^6\mathrm{Li}$ and $^7\mathrm{Li}$ isotopes, as well as non-equidistantly spaced hyperfine transitions originating from dipolar and quadrupolar hyperfine interactions. We provide refined crystal field parameters and extract the dipolar and quadrupolar hyperfine constants ${A_J=0.02703\pm0.00003}$ $\textrm{cm}^{-1}$ and ${B= 0.04 \pm0.01}$ $\textrm{cm}^{-1}$, respectively. Thereupon we determine all crystal-field energy levels and magnetic moments of the $^5I_8$ ground state manifold, including the (non-linear) hyperfine corrections. The latter match the measurement-based estimates. The scale of the non-linear hyperfine corrections sets an upper bound for the inhomogeneous line widths that would still allow for unique addressing of a selected hyperfine transition. e.g. for quantum information applications. Additionally, we establish the far-infrared, low-temperature refractive index of LiY$_{1-x}$Ho$_{x}$F$_{4}$.