The usual practice today is that milk component phenotypes are predicted using Fourier-transform infrared (FTIR) spectra and they are then, together with pedigree information, used in BLUP for calculation of individual estimated breeding values. Here, this is referred to as the indirect prediction (IP) approach. An alternative approach—a direct prediction (DP) method—is proposed, where genetic analyses are directly conducted on the milk FTIR spectral variables. Breeding values of all derived milk traits (protein, fat, fatty acid composition, and coagulation properties, among others) can then be predicted as traits correlated only to the genetic information of the spectra. For the DP, no need exists to predict the phenotypes before calculating breeding values for each of the traits—the genetic analysis is done once for the spectra, and is applicable to all traits derived from the spectra. The aim of the study was to compare the effects of DP and IP of milk composition and quality traits on prediction error variance (PEV) and genetic gain. A data set containing 27,927 milk FTIR spectral observations and milk composition phenotypes (fat, lactose, and protein) belonging to 14,869 goats of 271 herds was used for training and evaluating models. Partial least squares regression was used for calibrating prediction models for fat, protein, and lactose percentages. Restricted maximum likelihood was used to estimate variance components of the spectral variables after principal components analysis was applied to reduce the spectral dimension. Estimated breeding values were predicted for fat, lactose, and protein percentages using DP and IP methods. The DP approach reduced the mean PEV by 3.73, 4.07, and 7.04% for fat, lactose, and protein percentages, respectively, compared with the IP method. Given the reduction in PEV, relative genetic gains were 2.99, 2.78, and 4.85% for fat, lactose, and protein percentages, respectively. We concluded that more accurate estimated breeding values could be found using genetic components of milk FTIR spectra compared with single-trait animal model analyses on phenotypes predicted from the spectra separately. The potential and application is not only limited to milk FTIR spectra, but could also be extended to any spectroscopy techniques implemented in other species and for other traits.