Sensorimotor adaptation alters mappings between motor commands and their predicted outcomes. Such remapping has been extensively studied in the visual domain, but the degree to which it occurs in modalities other than vision remains less well understood. Here, we manipulated the modality of reach target presentation to compare sensorimotor adaptation in response to perturbations of visual and auditory feedback location. We compared the extent of adaptation to perturbed sensory feedback for visual and auditory sensory modalities, and the magnitude of reach-direction aftereffects when the perturbation was removed. To isolate the contribution of implicit sensorimotor recalibration to adaptation in reach direction, we held sensory prediction errors and task-performance errors constant via a task-irrelevant clamp of sensory feedback. Seventy-two participants performed one of three experiments in which target location information and endpoint reach direction feedback were presented by loudspeakers (n = 24), headphones (n = 24), or a visual display (n = 24). Presentation durations for target stimuli (500 ms) and (non-veridical) endpoint feedback of reach direction (100 ms) were matched for visual and auditory modalities. For all three groups, when endpoint feedback was perturbed, adaptation was evident: reach-directions increased significantly in the direction opposite the clamped feedback, and a significant aftereffect persisted after participants were instructed that the perturbation had been removed. This study provides new evidence that implicit sensorimotor adaptation occurs in response to perturbed auditory feedback of reach direction, suggesting that an implicit neural process to recalibrate sensory to motor maps in response to sensory prediction errors may be ubiquitous across sensory modalities.