Apraxia is a cognitive disorder of skilled movements that characteristically affects the ability to imitate meaningless gestures, or to pantomime the use of tools. Despite substantial research, the neural underpinnings of imitation and pantomime have remained debated. An influential model states that higher motor functions are supported by different processing streams. A dorso-dorsal stream may mediate movements based on physical object properties, like reaching or grasping, whereas skilled tool use or pantomime rely on action representations stored within a ventro-dorsal stream. However, given variable results of past studies, the role of the two streams for imitation of meaningless gestures has remained uncertain, and the importance of the ventro-dorsal stream for pantomime of tool use has been questioned. To clarify the involvement of ventral and dorsal streams in imitation and pantomime, we performed voxel-based lesion-symptom mapping in a sample of 96 consecutive left-hemisphere stroke patients (mean age ± SD, 63.4 ± 14.8 years, 56 male). Patients were examined in the acute phase after ischaemic stroke (after a mean of 5.3, maximum 10 days) to avoid interference of brain reorganization with a reliable lesion-symptom mapping as best as possible. Patients were asked to imitate 20 meaningless hand and finger postures, and to pantomime the use of 14 common tools depicted as line drawings. Following the distinction between movement engrams and action semantics, pantomime errors were characterized as either movement or content errors, respectively. Whereas movement errors referred to incorrect spatio-temporal features of overall recognizable movements, content errors reflected an inability to associate tools with their prototypical actions. Both imitation and pantomime deficits were associated with lesions within the lateral occipitotemporal cortex, posterior inferior parietal lobule, posterior intraparietal sulcus and superior parietal lobule. However, the areas specifically related to the dorso-dorsal stream, i.e. posterior intraparietal sulcus and superior parietal lobule, were more strongly associated with imitation. Conversely, in contrast to imitation, pantomime deficits were associated with ventro-dorsal regions such as the supramarginal gyrus, as well as brain structures counted to the ventral stream, such as the extreme capsule. Ventral stream involvement was especially clear for content errors which were related to anterior temporal damage. However, movement errors were not consistently associated with a specific lesion location. In summary, our results indicate that imitation mainly relies on the dorso-dorsal stream for visuo-motor conversion and on-line movement control. Conversely, pantomime additionally requires ventro-dorsal and ventral streams for access to stored action engrams and retrieval of tool-action relationships.