Cognitive task control can be examined in task-switching studies. Performance costs in task switches are usually smaller with compatible stimulus-response modality mappings (visual-manual and auditory-vocal) than with incompatible mappings (visual-vocal and auditory-manual). Modality compatibility describes the modality match of sensory input and of the anticipated response effect (e.g., vocal responses produce auditory effects, so that auditory stimuli are modality-compatible with vocal responses). Fintor et al. (Psychological Research, 84(2), 380–388, 2020) found that modality compatibility also biased task choice rates in voluntary task switching (VTS). In that study, in each trial participants were presented with a visual or auditory spatial stimulus and were free to choose the response modality (manual vs. vocal). In this free-choice task, participants showed a bias to create more modality-compatible than -incompatible mappings. In the present study, we assessed the generality of Fintor et al.’s (2020) findings, using verbal rather than spatial stimuli, and more complex tasks, featuring an increased number of stimulus-response alternatives. Experiment 1 replicated the task-choice bias to preferentially create modality-compatible mappings. We also found a bias to repeat the response modality just performed, and a bias to repeat entire stimulus-response modality mappings. In Experiment 2, we manipulated the response-stimulus interval (RSI) to examine whether more time for proactive cognitive control would help resolve modality-specific crosstalk in this free-choice paradigm. Long RSIs led to a decreased response-modality repetition bias and mapping repetition bias, but the modality-compatibility bias was unaffected. Together, the findings suggest that modality-specific priming of response modality influences task choice.