Abstract Objective This article aims to determine the microstructural alterations in the auditory pathway in prelingual deaf children using diffusion tensor imaging (DTI)-derived parameters—fractional anisotropy (FA) and apparent diffusion coefficient (ADC), and secondarily to evaluate these changes in rubella and cytomegalovirus (CMV) positive cases. Method A consecutive series of consenting deaf and normal children between 1 and 7 years of age, forming the case and control groups, respectively, underwent DTI, audiological tests, and testing for rubella, CMV, and toxoplasma infections. FA and ADC were measured at four locations bilaterally: lateral lemniscus (LL), inferior colliculus, medial geniculate body, and auditory cortex (AC). Result The mean ADC values were higher and the mean FA values were lower in cases (19 males, 21 females, mean age 2.65 years) than the controls (21 males, 19 females, mean age 4.63 years) at all eight sites. Sixteen (40%), 17 (42.5%), and 7 (17.5%) cases had severe, severe to profound, and profound hearing loss, respectively, the FA and ADC values being significantly different for LL. For rubella and CMV immunoglobulin G, 20/40and 17/40 cases were positive, respectively, 11 for both, and none for toxoplasma. Significant decrease in FA was seen at LL and AC in rubella/CMV positive cases. Conclusion Microstructural changes are seen throughout the auditory pathway in prelingual deaf children, especially with rubella and/or CMV positive status. Further studies may pave the path to segregate out patient groups potentially more responsive to cochlear implant.