Diffusion tensor imaging (DTI) is amongst the simplest mathematical models available for diffusion magnetic resonance imaging, yet still by far the most used one. Despite the success of DTI as an imaging tool for white matter fibers, its anatomical underpinnings on a microstructural basis remain unclear. In this study, we used 65 myelin-stained sections of human premotor cortex to validate modeled fiber orientations and oft used microstructure-sensitive scalar measures of DTI on the level of individual voxels. We performed this validation on high spatial resolution diffusion MRI acquisitions investigating both white and gray matter. We found a very good agreement between DTI and myelin orientations with the majority of voxels showing angular differences less than 10°. The agreement was strongest in white matter, particularly in unidirectional fiber pathways. In gray matter, the agreement was good in the deeper layers highlighting radial fiber directions even at lower fractional anisotropy (FA) compared to white matter. This result has potentially important implications for tractography algorithms applied to high resolution diffusion MRI data if the aim is to move across the gray/white matter boundary. We found strong relationships between myelin microstructure and DTI-based microstructure-sensitive measures. High FA values were linked to high myelin density and a sharply tuned histological orientation profile. Conversely, high values of mean diffusivity (MD) were linked to bimodal or diffuse orientation distributions and low myelin density. At high spatial resolution, DTI-based measures can be highly sensitive to white and gray matter microstructure despite being relatively unspecific to concrete microarchitectural aspects.