Marginal phonemes exploit systemically latent possibilities of contrast but have unusual lexical distributions characterized by clustering according to expressive function or morphological structure. This paper discusses examples of marginal contrast from several languages and shows that, despite initial appearances, it is not possible to confine marginally contrasting items to well-defined strata, lexical or morphological. Marginal phonemes are structure preserving, and turn up, however infrequently, in core and non-derived environments. Explanations for clustering must accordingly be sought outside grammatical theory. Marginal contrasts are interesting to linguistic theory because they may appear to challenge the Cartesian assumptions of clear-cut categories and systems. And yet, precisely from the vantage point of the system, marginal phonemes may be seen to be exploiting latent possibilities of contrast — they are in this sense 'structure preserving'. Kiparsky (1995) argues for a 'priming effect' as a diachronic analogue of structure preservation, according to which "redundant features are likely to be phonologized if the language's phonological representations have a class node to host them." To take an example from the history of English, the phonemicization of /ʒ/ which, in initial position, is restricted to low frequency French loanwords such as gitane and gite, was primed through the prior existence of a postalveolar sibilant /ʃ/ and a voiced fricative series /v ð z/. The novel phoneme /ʒ/ represents the intersection between the voiced fricative and postalveolar series. Put differently, we might say that /ʒ/ was a grammatical combination of features before any words containing it actually populated the English lexicon. This structure-preserving quality would also appear to hold for the marginal phonemes in Abkhaz and Hup. Abkhaz has plain stops, fricatives, and ejective stops. The possibility of ejective stops entails, on a minimal interpretation, that the combination (-sonorant, contricted glottis) is licit, although it implicitly also allows ejective fricatives. The case of marginal /p'/ in Hup is more subtle. The language has contrasting series of voiceless and voiced plain stops /p t c k/ • /b d ɟ ɡ/, but in the glottalized series the voicing contrast is largely neutralized, with the exception of /p'/. In initial position, the glottalized dorsal stops are voiceless /c' k'/ (which Epps transcribes as /j' ɡ'/) but the labial and alveolar /b' d'/ are voiced. 2 There are well-known aerodynamic constraints against voicing in dorsal consonants, but the prior existence of a voiceless and voiced plain stop series and the distribution of the glottalized series between voiceless and voiced would seem to create a space for marginal /p'/.