The present study aims to fill a gap at the intersection of the phenomena of language mode—the state of activation of the bilingual’s languages and language processing mechanisms—and the subset problem—issues learners face when the second language has fewer of some kind of contrast than the first language. When the subset problem is present in second language acquisition, learners may struggle to acquire specific contrasts of a language and may map them incorrectly to their first language. By studying advanced learners of Spanish and considering language mode, we are able to investigate whether learners create separate categories for Spanish vowels—as opposed to simply adapting their English categories—and whether the use of such categories depends on the language being perceived. Spanish and English serve as convenient languages for study of these phenomena because Spanish has fewer vowels than English. With this, we ask: “Does language mode influence language-specific categorization?” To investigate this question, we had native English-speaking, proficient Spanish learners perform an AX task in both English and Spanish, where they identified whether two aurally presented vowel stimuli were the same or different. There was no strong effect of language mode across conditions, but we found that reaction times were significantly slower and that error rates were higher in tasks that included stimuli from more than one language. Thus, we conclude that when multiple languages are activated it is more difficult to process a given language.