Our auditory system pre-attentively scans the environment for potentially relevant irregularities in an otherwise regular acoustic input. Such irregularities may cause an involuntary attention switch, which, in turn, may disturb ongoing mental operations resulting in impaired task-performance. The protection from distraction and the re-orienting of attention to task-relevant aspects of stimulation belongs to the liability of working memory. We investigated the dependency of ERP and behavioral indicators of processes underlying distraction and the recovery from distraction from the saliency of the perturbating information. More specifically, in a duration discrimination task occasional deviations in the sound’s frequency served as distractors. The physical difference between standard an deviant frequency was 1%, 3%, 5% or 10%. Even with the smallest distractor a substantial prolongation in the duration discrimination task was obtained. Separating trials with and without reaction time prolongation revealed MMN to both sub-averages, whereas P3a and RON (indicating an involuntary attention switch and the subsequent re-orienting of attention) were confined to sub-average computed from trials with behavioral distraction. This shows that the mismatch signal elicited by small deviations does not necessarily invoke subsequent distraction. It also supports the model according to which P3a and RON reflect distraction processes of the working memory system, whereas MMN indicates activity of the pre-attentive change-detection system. Moreover, MMN, P3a, RON and reaction time prolongation did also increase with increasing deviancy. Thus the processes involved in distraction including the early pre-attentive deviancy detection, the involuntary switching of attention, and the late re-orienting are modulated in a gradual fashion by the saliency of the distracting information. This strengthens the notion of a close relationship between pre-attentive deviance detection and working memory.