Keeping task-relevant sensory events in the focus of attention while ignoring irrelevant ones is crucial for optimizing task behavior. This attention-distraction balance might change with the perceptual demands of the ongoing task: while easy tasks might be performed with low attentional effort, difficult ones require enhanced attention. The goal of the present study was to investigate how task difficulty affected allocation of attention and distractibility in an auditory distraction paradigm. Participants performed a tone duration discrimination task in which tones were rarely, occasionally presented at a rare pitch (distracters), and task difficulty was manipulated by the duration difference between short and long tones. Short tones were consistently 200 ms long, while long tone duration was 400 ms in the easy, and 260 ms in the difficult condition. Behavioral results and deviant-minus-standard event-related potential (ERP) waveforms suggested similar magnitudes of distraction in both conditions. ERPs without such a subtraction showed that tone onsets were preceded by a negative-going trend, suggesting that participants prepared for tone onsets. In the difficult condition, N1 amplitudes to tone onsets were enhanced, indicating that participants invested more attentional resources. Increased difficulty also slowed down tone offset processing as reflected by significantly delayed offset-related P1 and N1/N2 waveforms. These results suggest that although task difficulty compels participants to attend the tones more strongly, this does not have significant impact on distraction-related processing.