Elevated levels of trait anxiety are argued to interfere with the ability to shift attention between different task sets, yet empirical support for this hypothesis is scarce. Using a task-switching paradigm in two separate studies, we compared high and low trait anxious participants' ability to switch from non-affective, positive, and negative tasks to different non-affective tasks. In Study 1 (N = 59 high and low trait anxious undergraduate students), we found that non-affective-to-non-affective switch costs were smaller than both positive-to-non-affective and negative-to-non-affective switch costs, and positive-to-non-affective switch costs were smaller than negative-to-non-affective switch costs. In Study 2 (N = 97 high and low trait anxious community members), we found that non-affective-to-non-affective switch costs and positive-to-non-affective switch costs were both smaller than negative-to-non-affective switch costs, but positive-to-non-affective and non-affective-to-non-affective switch costs did not differ. Crucially, none of the switch costs in either of the studies or in an analysis of the combined data differed between high and low trait anxious groups. While we cannot exclude the possibility that anxiety linked differences in task-switching do exist when switching from more demanding to less demanding tasks, our studies found no evidence for the general idea that elevated trait anxiety interferes with attentional shifting.