This article examines the historical phenomenon of nonsense syllables in the chant of the Byzantine church. This practice of non-lexical singing upon the extended use of vocables, appears already in ancient Greek music and has gone by many terms, including kratēmata or teretismata. Three different hypotheses as to the historical roots and development of this singing practice are predominant, namely those of Gregorios Stathis (1979, 2014), Diane Touliatos (1989), and Gregorios Anastasiou (2005). The foundations, results and consequences of these theories are reassessed in the light of critical examination of the fourteenth century treatise Ἁρμονικά (Harmonika) of Manuel Bryennios (Magdalen College MS Gr 13), the tenth century Book of Ceremonies (in Latin cited as De cerimoniis aulae Byzantinae and in Greek cited as Ἔκθεσις τῆς βασιλείου τάξεως) of the Byzantine emperor Constantine Porphyrogennetos, and different versions of the liturgy of the so-called “Service of the Furnace” (Athens, National Library, EBE 2047; Athens, National Library, EBE 2406; Iviron Monastery, MS 1120; S-Sinai, Mt. Sinai, MS 1527; and Lavra Monastery, MS 165). The critical examination and synthesis of previous theories of the origin and development of Byzantine nonsense-syllable singing offers a number of conclusions, where I argue for new, qualified hypotheses that may be laid as foundations for further research. It is concluded that the nonsense syllables seem to have served a number of semiotic functions and purposes: as incantations (sometimes with pagan and ritual associations), as mimesis of nature (birds and cicada), for the intonation of texted chant, as well as for singing practice and solmization purposes. It is also argued that the non-semantic vocalization that this singing constitutes, amounts to a type of proto-programmatic music, with layers of mimetic singing and a rare sense of heightened expression in a context which otherwise focuses strongly on a religious text and its context. Finally, the term “mental representations” is proposed for describing the semantic function of nonsense syllables.