Relaxation time and frequency spectra are not directly available by measurement. To determine them, an ill-posed inverse problem must be solved based on relaxation stress or oscillatory shear relaxation data. Therefore, the quality of spectra models has only been assessed indirectly by examining the fit of the experiment data to the relaxation modulus or dynamic moduli models. As the measures of data fitting, the mean sum of the moduli square errors were usually used, the minimization of which was an essential step of the identification algorithms. The aim of this paper was to determine a relaxation spectrum model that best approximates the real unknown spectrum in a direct manner. It was assumed that discrete-time noise-corrupted measurements of a relaxation modulus obtained in the stress relaxation experiment are available for identification. A modified relaxation frequency spectrum was defined as a quotient of the real relaxation spectrum and relaxation frequency and expanded into a series of linearly independent exponential functions that are known to constitute a basis of the space of square-integrable functions. The spectrum model, given by a finite series of these basis functions, was assumed. An integral-square error between the real unknown modified spectrum and the spectrum model was taken as a measure of the model quality. This index was proved to be expressed in terms of the measurable relaxation modulus at uniquely defined sampling instants. Next, an empirical identification index was introduced in which the values of the real relaxation modulus are replaced by their noisy measurements. The identification consists of determining the spectrum model that minimizes this empirical index. Tikhonov regularization was applied to guarantee model smoothness and noise robustness. A simple analytical formula was derived to calculate the optimal model parameters and expressed in terms of the singular value decomposition. A complete identification algorithm was developed. The analysis of the model smoothness and model accuracy for noisy measurements was carried out. The equivalence of the direct identification of the relaxation frequency and time spectra has been demonstrated when the time spectrum is modeled by a series of functions given by the product of the relaxation frequency and its exponential function. The direct identification concept can be applied to both viscoelastic fluids and solids; however, some limitations to its applicability have been pointed out. Numerical studies have shown that the proposed identification algorithm can be successfully used to identify Gaussian-like and Kohlrausch–Williams–Watt relaxation spectra. The applicability of this approach to determining other commonly used classes of relaxation spectra was also examined.