Anisotropic dielectric tensors of uniaxial van der Waals (vdW) materials are difficult to investigate at infrared frequencies. The small dimensions of high-quality exfoliated crystals prevent the use of diffraction-limited spectroscopies. Near-field microscopes coupled to broadband lasers can function as Fourier transform infrared spectrometers with nanometric spatial resolution (nano-FTIR). Although dielectric functions of isotropic materials can be readily extracted from nano-FTIR spectra, the in- and out-of-plane permittivities of anisotropic vdW crystals cannot be easily distinguished. For thin vdW crystals residing on a substrate, nano-FTIR spectroscopy probes a combination of sample and substrate responses. We exploit the information in the screening of substrate resonances by vdW crystals to demonstrate that both the in- and out-of-plane dielectric permittivities are identifiable for realistic spectra. This novel method for the quantitative nanoresolved characterization of optical anisotropy was used to determine the dielectric tensor of a bulk 2H-WSe2 microcrystal in the mid-infrared.