Vocal Sequences Research Articles

Beyond bigrams: call sequencing in the common marmoset ( Callithrix jacchus ) vocal system

Over the last two decades, an emerging body of research has demonstrated that non-human animals exhibit the ability to combine context-specific calls into larger sequences. These structures have frequently been compared with language’s syntax, whereby linguistic units are combined to form larger structures, and leveraged to argue that syntax might not be unique to language. Currently, however, the overwhelming majority of examples of call combinations are limited to simple sequences comprising just two calls which differ dramatically from the open-ended hierarchical structuring of the syntax found in language. We revisit this issue by taking a whole-repertoire approach to investigate combinatoriality in common marmosets ( Callithrix jacchus ). We use Markov chain models to quantify the vocal sequences produced by marmosets providing evidence for structures beyond the bigram, including three-call and even combinations of up to eight or nine calls. Our analyses of these longer vocal sequences are suggestive of potential further internal organization, including some amount of recombination, nestedness and non-adjacent dependencies. We argue that data-driven, whole-repertoire analyses are fundamental to uncovering the combinatorial complexity of non-human animals and will further facilitate meaningful comparisons with language’s combinatoriality.

Prevalent harmonic interaction in the bat inferior colliculus.

Animal vocalizations and human speech are typically characterized by a complex spectrotemporal structure, composed of multiple harmonics, and patterned as temporally organized sequences. However, auditory research often employed simple artificial acoustic stimuli or their combinations. Here we addressed the question of whether the neuronal responses to natural echolocation call sequences can be predicted by manipulated sequences of incomplete constituents at the midbrain inferior colliculus (IC). We characterized the extracellular single-unit activity of IC neurons in the great roundleaf bat, Hipposideros armiger (both sexes), using natural call sequences, various manipulated sequences of incomplete vocalizations, and pure tones. We report that approximately two-thirds of IC neurons exhibited a harmonic interaction. Neurons with high harmonic interactions exhibited greater selectivity to natural call sequences, and the degree of harmonic interaction was robust to the natural amplitude variations between call harmonics. For 81% of the IC neurons, the responses to the natural echolocation call sequence could not be predicted by altered sequences of missing call components. Surprisingly, nearly 70% of the neurons that showed a harmonic interaction were characterized by a single excitatory response peak as revealed by pure tones. Our results suggest that prevalent harmonic processing has already emerged in the auditory midbrain inferior colliculus in the echolocating bat.Significance Statement Auditory cortex has long been suggested to be the main region for processing complex sound such as natural vocalizations. However, this cortex-centered view of complex stimuli processing was largely based on simple stimuli or their combinations, ignoring many crucial aspects of natural communication behaviors. Here, we took a neuroethological perspective to study the harmonic processing in the inferior colliculus, in the great roundleaf bat. We show that prevalent harmonic processing has already emerged in the auditory midbrain in the echolocating bat. Critically, harmonic interactions revealed by natural call sequences cannot be predicted by responses to either pure tones or altered sequences of missing call components. Our data suggest that auditory midbrain is involved in harmonic processing in the echolocating bat.

First person – Nakul Wewhare

ABSTRACT First Person is a series of interviews with the first authors of a selection of papers published in Biology Open, helping researchers promote themselves alongside their papers. Nakul Wewhare is first author on ‘ Individual-specific associations between warble song notes and body movements in budgerigar courtship displays’, published in BiO. Nakul is a BS-MS student in the lab of Dr Anand Krishnan at the Indian Institute of Science Education and Research (IISER), Pune, India, investigating the structure and function of complex vocal sequences in animals.

Emotional vocalizations alter behaviors and neurochemical release into the amygdala

The basolateral amygdala (BLA), a brain center of emotional expression, contributes to acoustic communication by first interpreting the meaning of social sounds in the context of the listener’s internal state, then organizing the appropriate behavioral responses. We propose that modulatory neurochemicals such as acetylcholine (ACh) and dopamine (DA) provide internal-state signals to the BLA while an animal listens to social vocalizations. We tested this in a vocal playback experiment utilizing highly affective vocal sequences associated with either mating or restraint, then sampled and analyzed fluids within the BLA for a broad range of neurochemicals and observed behavioral responses of adult male and female mice. In male mice, playback of restraint vocalizations increased ACh release and usually decreased DA release, while playback of mating sequences evoked the opposite neurochemical release patterns. In non-estrus female mice, patterns of ACh and DA release with mating playback were similar to males. Estrus females, however, showed increased ACh, associated with vigilance, as well as increased DA, associated with reward-seeking. Experimental groups that showed increased ACh release also showed the largest increases in an aversive behavior. These neurochemical release patterns and several behavioral responses depended on a single prior experience with the mating and restraint behaviors. Our results support a model in which ACh and DA provide contextual information to sound analyzing BLA neurons that modulate their output to downstream brain regions controlling behavioral responses to social vocalizations.

Emotional vocalizations alter behaviors and neurochemical release into the amygdala.

The basolateral amygdala (BLA), a brain center of emotional expression, contributes to acoustic communication by first interpreting the meaning of social sounds in the context of the listener's internal state, then organizing the appropriate behavioral responses. We propose that modulatory neurochemicals such as acetylcholine (ACh) and dopamine (DA) provide internal-state signals to the BLA while an animal listens to social vocalizations. We tested this in a vocal playback experiment utilizing highly affective vocal sequences associated with either mating or restraint, then sampled and analyzed fluids within the BLA for a broad range of neurochemicals and observed behavioral responses of adult male and female mice. In male mice, playback of restraint vocalizations increased ACh release and usually decreased DA release, while playback of mating sequences evoked the opposite neurochemical release patterns. In non-estrus female mice, patterns of ACh and DA release with mating playback were similar to males. Estrus females, however, showed increased ACh, associated with vigilance, as well as increased DA, associated with reward-seeking. Experimental groups that showed increased ACh release also showed the largest increases in an aversive behavior. These neurochemical release patterns and several behavioral responses depended on a single prior experience with the mating and restraint behaviors. Our results support a model in which ACh and DA provide contextual information to sound analyzing BLA neurons that modulate their output to downstream brain regions controlling behavioral responses to social vocalizations.

Kurpian glides

Abstract This article addresses the problem of vowel-generated glides in Kurpian. In general, glides come from two sources: Gliding and Glide Insertion, for instance, word-initial /#iVC/ → [jVC] and /#iC/ → [jiC], where V stands for a vowel and C for a consonant. Even though Gliding and Glide Insertion are different phonological operations, they are united by a single goal: to eliminate onsetless syllables because a glide, regardless of whether it comes from a vowel or from insertion, provides an onset to a syllable. Onset-driven effects are particularly rich in Kurpian because they involve all kinds of vowels, including /a/ but not /ɛ/, a situation that has not been noted in the past research on crosslinguistic typologies (Levi, Susannah V. 2004. The representation of underlying glides: A cross-linguistic study. Seattle, WA: University of Washington dissertation, Levi, Susannah V. 2008. Phonemic vs. derived glides. Lingua 118. 1956–1978; Rosenthall, Sam. 1994. Vowel/glide alternation in a theory of constraint interaction. Amherst, MA: University of Massachusetts dissertation; Sands, Kathy. 2004. Patternings of vocalic sequences in the world’s languages. Santa Barbara, CA: University of California, Santa Barbra dissertation). Further, glides are found both word-medially and word-initially. In this regard, Kurpian is different from Standard Polish, which tolerates both hiatus and word-initial onsetless syllables. It is argued that the constraint interaction required cannot be accounted for in classic Optimality Theory (OT). The solution is to adopt Derivational OT.

SSTE: Syllable-Specific Temporal Encoding to FORCE-learn audio sequences with an associative memory approach

The circuitry and pathways in the brains of humans and other species have long inspired researchers and system designers to develop accurate and efficient systems capable of solving real-world problems and responding in real-time. We propose the Syllable-Specific Temporal Encoding (SSTE) to learn vocal sequences in a reservoir of Izhikevich neurons, by forming associations between exclusive input activities and their corresponding syllables in the sequence. Our model converts the audio signals to cochleograms using the CAR-FAC model to simulate a brain-like auditory learning and memorization process. The reservoir is trained using a hardware-friendly approach to FORCE learning. Reservoir computing could yield associative memory dynamics with far less computational complexity compared to RNNs. The SSTE-based learning enables competent accuracy and stable recall of spatiotemporal sequences with fewer reservoir inputs compared with existing encodings in the literature for similar purpose, offering resource savings. The encoding points to syllable onsets and allows recalling from a desired point in the sequence, making it particularly suitable for recalling subsets of long vocal sequences. The SSTE demonstrates the capability of learning new signals without forgetting previously memorized sequences and displays robustness against occasional noise, a characteristic of real-world scenarios. The components of this model are configured to improve resource consumption and computational intensity, addressing some of the cost-efficiency issues that might arise in future implementations aiming for compactness and real-time, low-power operation. Overall, this model proposes a brain-inspired pattern generation network for vocal sequences that can be extended with other bio-inspired computations to explore their potentials for brain-like auditory perception. Future designs could inspire from this model to implement embedded devices that learn vocal sequences and recall them as needed in real-time. Such systems could acquire language and speech, operate as artificial assistants, and transcribe text to speech, in the presence of natural noise and corruption on audio data.

Recursive self-embedded vocal motifs in wild orangutans

Recursive procedures that allow placing a vocal signal inside another of a similar kind provide a neuro-computational blueprint for syntax and phonology in spoken language and human song. There are, however, no known vocal sequences among nonhuman primates arranged in self-embedded patterns that evince vocal recursion or potential incipient or evolutionary transitional forms thereof, suggesting a neuro-cognitive transformation exclusive to humans. Here, we uncover that wild flanged male orangutan long calls feature rhythmically isochronous call sequences nested within isochronous call sequences, consistent with two hierarchical strata. Remarkably, three temporally and acoustically distinct call rhythms in the lower stratum were not related to the overarching rhythm at the higher stratum by any low multiples, which suggests that these recursive structures were neither the result of parallel non-hierarchical procedures nor anatomical artifacts of bodily constraints or resonances. Findings represent a case of temporally recursive hominid vocal combinatorics in the absence of syntax, semantics, phonology, or music. Second-order combinatorics, ‘sequences within sequences’, involving hierarchically organized and cyclically structured vocal sounds in ancient hominids may have preluded the evolution of recursion in modern language-able humans.

Recursive self-embedded vocal motifs in wild orangutans.

Recursive procedures that allow placing a vocal signal inside another of a similar kind provide a neuro-computational blueprint for syntax and phonology in spoken language and human song. There are, however, no known vocal sequences among nonhuman primates arranged in self-embedded patterns that evince vocal recursion or potential incipient or evolutionary transitional forms thereof, suggesting a neuro-cognitive transformation exclusive to humans. Here, we uncover that wild flanged male orangutan long calls feature rhythmically isochronous call sequences nested within isochronous call sequences, consistent with two hierarchical strata. Remarkably, three temporally and acoustically distinct call rhythms in the lower stratum were not related to the overarching rhythm at the higher stratum by any low multiples, which suggests that these recursive structures were neither the result of parallel non-hierarchical procedures nor anatomical artifacts of bodily constraints or resonances. Findings represent a case of temporally recursive hominid vocal combinatorics in the absence of syntax, semantics, phonology, or music. Second-order combinatorics, 'sequences within sequences', involving hierarchically organized and cyclically structured vocal sounds in ancient hominids may have preluded the evolution of recursion in modern language-able humans.

Courtship behaviour reveals temporal regularity is a critical social cue in mouse communication.

While animals navigating the real world face a barrage of sensory input, their brains evolved to perceptually compress multidimensional information by selectively extracting the features relevant for survival. Notably, communication signals supporting social interactions in several mammalian species consist of acoustically complex sequences of vocalisations. However, little is known about what information listeners extract from such time-varying sensory streams. Here, we utilise female mice's natural behavioural response to male courtship songs to identify the relevant acoustic dimensions used in their social decisions. We found that females were highly sensitive to disruptions of song temporal regularity and preferentially approached playbacks of intact over rhythmically irregular versions of male songs. In contrast, female behaviour was invariant to manipulations affecting the songs' sequential organisation or the spectro-temporal structure of individual syllables. The results reveal temporal regularity as a key acoustic cue extracted by mammalian listeners from complex vocal sequences during goal-directed social behaviour.

Courtship behaviour reveals temporal regularity is a critical social cue in mouse communication

While animals navigating the real world face a barrage of sensory input, their brains evolved to perceptually compress multidimensional information by selectively extracting the features relevant for survival. Notably, communication signals supporting social interactions in several mammalian species consist of acoustically complex sequences of vocalisations. However, little is known about what information listeners extract from such time-varying sensory streams. Here, we utilise female mice’s natural behavioural response to male courtship songs to identify the relevant acoustic dimensions used in their social decisions. We found that females were highly sensitive to disruptions of song temporal regularity and preferentially approached playbacks of intact over rhythmically irregular versions of male songs. In contrast, female behaviour was invariant to manipulations affecting the songs’ sequential organisation or the spectro-temporal structure of individual syllables. The results reveal temporal regularity as a key acoustic cue extracted by mammalian listeners from complex vocal sequences during goal-directed social behaviour.

Large-scale single-neuron speech sound encoding across the depth of human cortex

Understanding the neural basis of speech perception requires that we study the human brain both at the scale of the fundamental computational unit of neurons and in their organization across the depth of cortex. Here we used high-density Neuropixels arrays1–3 to record from 685 neurons across cortical layers at nine sites in a high-level auditory region that is critical for speech, the superior temporal gyrus4,5, while participants listened to spoken sentences. Single neurons encoded a wide range of speech sound cues, including features of consonants and vowels, relative vocal pitch, onsets, amplitude envelope and sequence statistics. Neurons at each cross-laminar recording exhibited dominant tuning to a primary speech feature while also containing a substantial proportion of neurons that encoded other features contributing to heterogeneous selectivity. Spatially, neurons at similar cortical depths tended to encode similar speech features. Activity across all cortical layers was predictive of high-frequency field potentials (electrocorticography), providing a neuronal origin for macroelectrode recordings from the cortical surface. Together, these results establish single-neuron tuning across the cortical laminae as an important dimension of speech encoding in human superior temporal gyrus.

Higher-order dialectic variation and syntactic convergence in the complex warble song of budgerigars.

Dialectic signatures in animal acoustic signals are key in the identification of and association with group members. Complex vocal sequences may also convey information about behavioral state, and may thus vary according to social environment. Some bird species, such as psittaciforms, learn and modify their complex acoustic signals throughout their lives. However, the structure and function of vocal sequences in open-ended vocal learners remains understudied. Here, we examined vocal sequence variation in the warble song of budgerigars, and how these change upon contact between social groups. Budgerigars are open-ended vocal learners which exhibit fission-fusion flock dynamics in the wild. We found that two captive colonies of budgerigars exhibited colony-specific differences in the syntactic structure of their vocal sequences. Individuals from the two colonies differed in the propensity to repeat certain note types, forming repetitive motifs which served as higher-order signatures of colony identity. When the two groups were brought into contact, their vocal sequences converged, and these colony-specific repetitive patterns disappeared, with males from both erstwhile colonies now producing similar sequences with similar syntactic structure. We present data suggesting that the higher-order temporal arrangement of notes/vocal units is modified throughout life by social learning as groups of birds continually associate and dissociate. Our study sheds light on the importance of examining signal structure at multiple levels of organization, and the potential for psittaciform birds as model systems to examine the influence of learning and social environment on acoustic signals.

Chimpanzees show the capacity to communicate about concomitant daily life events

One universal feature of human language is its versatility in communicating about juxtapositions of everyday events. Versatile combinatorial systems of communication can be selected for if (a) several vocal units are flexibly combined into numerous and long vocal sequences and (b) vocal sequences relate to numerous daily life events. We propose (b)is more likely during simultaneous or serial (concomitant) events than single events. We analyzed 9,391 vocal utterances across the repertoire of wild chimpanzees and their events of production. Chimpanzees used vocal sequences across a range of daily life events and twice as often during concomitant than single events. Also, utterance diversity correlated positively with event diversity. Our results show the potential of chimpanzee vocal sequences to convey combined information about numerous daily life events, a step from which generalized combinatoriality could have evolved.

Allopatric montane wren-babblers exhibit similar song notes but divergent vocal sequences

Allopatric montane wren-babblers exhibit similar song notes but divergent vocal sequences

Reversible inhibition of the basal ganglia prolongs repetitive vocalization but only weakly affects sequencing at branch points in songbirds.

Although vocal signals, including languages and songbird syllables, are composed of a finite number of acoustic elements, diverse vocal sequences are composed of a combination of these elements, which are linked together by syntactic rules. However, the neural basis of syntactic vocalization generation remains poorly understood. Here, we report that inhibition using tetrodotoxin (TTX) and manipulations of gamma-aminobutyric acid (GABA) receptors within the basal ganglia Area X or lateral magnocellular nucleus of the anterior neostriatum (LMAN) alter and prolong repetitive vocalization in Bengalese finches (Lonchura striata var. domestica). These results suggest that repetitive vocalizations are modulated by the basal ganglia and not solely by higher motor cortical neurons. These data highlight the importance of neural circuits, including the basal ganglia, in the production of stereotyped repetitive vocalizations and demonstrate that dynamic disturbances within the basal ganglia circuitry can differentially affect the repetitive temporal features of songs.

Predictive Mouse Ultrasonic Vocalization Sequences: Uncovering Behavioral Significance, Auditory Cortex Neuronal Preferences, and Social-Experience-Driven Plasticity.

Mouse ultrasonic vocalizations (USVs) contain predictable sequential structures like bird songs and speech. Neural representation of USVs in the mouse primary auditory cortex (Au1) and its plasticity with experience has been largely studied with single-syllables or dyads, without using the predictability in USV sequences. Studies using playback of USV sequences have used randomly selected sequences from numerous possibilities. The current study uses mutual information to obtain context-specific natural sequences (NSeqs) of USV syllables capturing the observed predictability in male USVs in different contexts of social interaction with females. Behavioral and physiological significance of NSeqs over random sequences (RSeqs) lacking predictability were examined. Female mice, never having the social experience of being exposed to males, showed higher selectivity for NSeqs behaviorally and at cellular levels probed by expression of immediate early gene c-fos in Au1. The Au1 supragranular single units also showed higher selectivity to NSeqs over RSeqs. Social-experience-driven plasticity in encoding NSeqs and RSeqs in adult females was probed by examining neural selectivities to the same sequences before and after the above social experience. Single units showed enhanced selectivity for NSeqs over RSeqs after the social experience. Further, using two-photon Ca2+ imaging, we observed social experience-dependent changes in the selectivity of sequences of excitatory and somatostatin-positive inhibitory neurons but not parvalbumin-positive inhibitory neurons of Au1. Using optogenetics, somatostatin-positive neurons were identified as a possible mediator of the observed social-experience-driven plasticity. Our study uncovers the importance of predictive sequences and introduces mouse USVs as a promising model to study context-dependent speech like communications.SIGNIFICANCE STATEMENT Humans need to detect patterns in the sensory world. For instance, speech is meaningful sequences of acoustic tokens easily differentiated from random ordered tokens. The structure derives from the predictability of the tokens. Similarly, mouse vocalization sequences have predictability and undergo context-dependent modulation. Our work investigated whether mice differentiate such informative predictable sequences (NSeqs) of communicative significance from RSeqs at the behavioral, molecular, and neuronal levels. Following a social experience in which NSeqs occur as a crucial component, mouse auditory cortical neurons become more sensitive to differences between NSeqs and RSeqs, although preference for individual tokens is unchanged. Thus, speech-like communication and its dysfunction may be studied in circuit, cellular, and molecular levels in mice.

VOWEL PHONEMES

The existence of word vowel phonemes in all languages, including in Indonesian is very dominant. All words in their internal structure must be found in phonemes. As a result of the existence of a very dominant vowel phoneme, its existence is able to form what is called a phoneme vocal sequence. The application of research theory in connection with efforts to describe vowel series phonemes is based on the linguistic theory of the phonological field of the phonemic subfield. Because the study focuses on the smallest unit of language sounds, namely phonemes. The research implementation is based on three strategic stages, namely:1. provision of data, 2. classification and analysis of data, 3. preparation / writing of reports. The findings of the data are more secondary, so the findings of the data focus on the availability of references, so that the existence of a large Indonesian language dictionary acts as a "smart" book. The findings of primary data are relatively difficult to do. There are seven kinds of vocal phonemes, namely / a, i, u, ê, é, è, o /. The seven vowel phonemes are relatively capable of forming vowel phonemes, so that in the end a table can be made from each of the seven vowel phonemes in question.

Deep audio embeddings for vocalisation clustering.

The study of non-human animals' communication systems generally relies on the transcription of vocal sequences using a finite set of discrete units. This set is referred to as a vocal repertoire, which is specific to a species or a sub-group of a species. When conducted by human experts, the formal description of vocal repertoires can be laborious and/or biased. This motivates computerised assistance for this procedure, for which machine learning algorithms represent a good opportunity. Unsupervised clustering algorithms are suited for grouping close points together, provided a relevant representation. This paper therefore studies a new method for encoding vocalisations, allowing for automatic clustering to alleviate vocal repertoire characterisation. Borrowing from deep representation learning, we use a convolutional auto-encoder network to learn an abstract representation of vocalisations. We report on the quality of the learnt representation, as well as of state of the art methods, by quantifying their agreement with expert labelled vocalisation types from 8 datasets of other studies across 6 species (birds and marine mammals). With this benchmark, we demonstrate that using auto-encoders improves the relevance of vocalisation representation which serves repertoire characterisation using a very limited number of settings. We also publish a Python package for the bioacoustic community to train their own vocalisation auto-encoders or use a pretrained encoder to browse vocal repertoires and ease unit wise annotation.

Front Cover: Cover Image, Volume 26, Issue 4

The cover image is based on the Research Article Slow development of vocal sequences through ontogeny in wild chimpanzees (Pan troglodytes verus) by Tatiana Bortolato et al., https://doi.org/10.1111/desc.13350 Image Credit: Liran Samuni, Tai Chimpanzee Project.