Syllable Classification Research Articles

SaLSa: A Combinatory Approach of Semi-Automatic Labeling and Long Short-Term Memory to Classify Behavioral Syllables.

Accurately and quantitatively describing mouse behavior is an important area. Although advances in machine learning have made it possible to track their behaviors accurately, reliable classification of behavioral sequences or syllables remains a challenge. In this study, we present a novel machine learning approach, called SaLSa (a combination of semi-automatic labeling and long short-term memory-based classification), to classify behavioral syllables of mice exploring an open field. This approach consists of two major steps. First, after tracking multiple body parts, spatial and temporal features of their egocentric coordinates are extracted. A fully automated unsupervised process identifies candidates for behavioral syllables, followed by manual labeling of behavioral syllables using a graphical user interface (GUI). Second, a long short-term memory (LSTM) classifier is trained with the labeled data. We found that the classification performance was marked over 97%. It provides a performance equivalent to a state-of-the-art model while classifying some of the syllables. We applied this approach to examine how hyperactivity in a mouse model of Alzheimer's disease develops with age. When the proportion of each behavioral syllable was compared between genotypes and sexes, we found that the characteristic hyperlocomotion of female Alzheimer's disease mice emerges between four and eight months. In contrast, age-related reduction in rearing is common regardless of genotype and sex. Overall, SaLSa enables detailed characterization of mouse behavior.

Lithuanian Syllable: The Hierarchy of Syllable Constituents and Syllable Weight

The article aims to establish the hierarchy of Lithuanian syllable constituents and determine how syllables should be interpreted in terms of syllable weight. The empirical data based on 311 000 syllables show that the variety of consonants in the onset is more significant than in the coda. A vowel and a succeeding approximant form the nucleus of a prosodic syllable, which shows strong coherency between a nucleus and a coda. Therefore, a CVC syllable is interpreted as [syllable Onset [rhyme Nucleus Coda]]. The division of Lithuanian syllables into light and heavy can be based on the realization of pitch accent, and it fully coincides with the traditional classification of long and short syllables in Lithuanian linguistics. Short syllables are light, their nucleus is a short vowel, and the number of consonants in the onset and the coda is unimportant. Such syllables account for almost half of all syllables in the database of this research (45%). Long syllables are heavy, and their nucleus may consist of long vowels (30%), diphthongs (12%), or mixed diphthongs (13%). When stressed, heavy syllables are pronounced in two ways: traditionally known as “acute” and “circumflex”. The stress is not dependent on the syllable weight, as both light (short) and heavy (long) syllables can be stressed. However, it has been observed that heavy (long) syllables are stressed more often than light (short) ones. Heavily stressed syllables constitute 25%, while light stressed syllables comprise 11% of all the syllables in this research database.

Convolutional Neural Network Method for Classification of Syllables in Javanese Script

Javanese script is one of the languages which are a typical Javanese culture. Javanese script is seen in its use in writing the name of a particular agency or location that has historical and tourism value. The use of Javanese script in public places makes the existence of this script seen by many people, not only by the Javanese people. Some of them have difficulty recognizing the Javanese characters they encounter. One method of pattern recognition and image processing is Convolutional Neural Network (CNN). CNN is a method that uses convolution operations in performing feature extraction on images as a basis for classification. The process consists of initial data processing, classification, and syllable formation. The classification consists of 48 classes covering Javanese script types, namely basic letters (Carakan) and voice-modifying scripts (Sandhangan). It is tested with multi-class confusion matrix scenarios to determine the accuracy, precision, and recall of the built CNN model. The CNN architecture consists of three convolution layers with max-pooling operations. The training configuration includes a learning rate of 0.0001, and the number of filters for each convolution layer is 32, 64, and 128 filters. The dropout value used is 0.5, and the number of neurons in the fully-connected layer is 1,024 neurons. The average performance value of accuracy reached 87.65%, the average precision value was 88.01%, and the average recall value was 87.70%.

Isolation-Induced Ultrasonic Vocalization in Environmental and Genetic Mice Models of Autism.

Studies in rodent models suggest that calls emitted by isolated pups serve as an early behavioral manifestation of communication deficits and autistic like behavior. Previous studies in our labs showed that gestational exposure to the pesticide chlorpyrifos (CPF) and the Mthfr-knock-out mice are associated with impaired social preference and restricted or repetitive behavior. To extend these studies, we examine how pup communication via ultrasonic vocalizations is altered in these ASD models. We implemented an unsupervised hierarchical clustering method based on the spectral properties of the syllables in order to exploit syllable classification to homogeneous categories while avoiding over-categorization. Comparative exploration of the spectral and temporal aspects of syllables emitted by pups in two ASD models point to the following: (1) Most clusters showed a significant effect of the ASD factor on the start and end frequencies and bandwidth and (2) The highest percent change due to the ASD factor was on the bandwidth and duration. In addition, we found sex differences in the spectral and temporal properties of the calls in both control groups as well as an interaction between sex and the gene/environment factor. Considering the basal differences in the characteristics of syllables emitted by pups of the C57Bl/6 and Balb/c strains used as a background in the two models, we suggest that the above spectral-temporal parameters start frequency, bandwidth, and duration are the most sensitive USV features that may represent developmental changes in ASD models.

Evidence for vocal diversity during physical interference at the perch in sympatricCarolliaspecies (Chiroptera: Phyllostomidae): a key to social organization and species coexistence?

AbstractVocal diversity might reflect the social organization and sustain the coexistence of cryptic species in sympatry. To test the extent to which vocal behaviour separates sympatric, cryptic species of a bat radiation, we compared vocalizations of genetically confirmed Carollia castanea, Carollia sowelli and Carollia perspicillata emitted by intraspecific dyads of identified individuals during corresponding physical interference interactions at the perch. Video analysis revealed a similar behaviour and interaction time across species. A sonagram-based visual classification of vocalization syllables of uninterrupted frequency–time contour discriminated 21 syllable classes. Class usage and distribution of the four shared classes differed across species. Carollia sowelli emitted the lowest number of classes in total and per interaction across species and displayed a limited number of syllable compositions in bouts. Discriminant analyses of syllables of a common, shared class provided evidence for species distinctiveness and individual-specific signatures. In general, sex did not account for data variability. The present vocalizations combine syllables reported from aggressive and submissive contexts in C. perspicillata and might express experienced ambivalence during interference at the perch. The diversity of vocal behaviour across congeners is discussed as arising from different ecological pressures during allopatric speciation and as an indicator of differences in species social organization.

Syllable and diphthong classification in the medieval Welsh bardic grammars

ABSTRACTThe medieval Welsh bardic grammars, known as ‘Gramadegau’r Penceirddiaid,’ include an extensive system of classification to describe syllable and diphthong types. While much of the rest of the linguistic description in the bardic grammars is heavily Latinate, this section is apparently innovative and oriented towards the demands of bardic composition. The syllables and diphthongs section is extensively revised over the course of its transmission, and either expanded or contracted depending on the aims and purposes of its editors. This article examines the two earliest revisions, found in Peniarth MS 20 (c.1330) and Bangor MS 1 (mid-fifteenth century) as evidence of the changing function of the grammars over the course of the fourteenth and fifteenth centuries. A case is made for the increasing use of the grammars as practical pedagogical documents from the mid-fifteenth century.

Mispronunciation Detection for Spoken Isolated Words using Segmentation and Classification under Low Resource Conditions for Kannada Language

Relocation makes it inevitable for a person to learn the local pronunciations correctly. With the advent of mobile phones, language learning can be made easy and flexible. Our research involves Kannada Kali, a mobile and cloud based application that is being developed to assist people in learning the correct pronunciations of Kannada (a language spoken in India). Automatic Speech Recognition systems which are used to aid pronunciation training require to be trained on sufficient amount of spoken target language data. Since collecting such data in not easy, the objective of our research is to detect mispronounced segments of words with minimal data. When there is scarcity of data, a comparative approach where spoken word segments are compared with the canonical pronunciations is more effective for detecting anomaly in pronunciation. Since syllables are basic independent units of pronunciation, the spoken words are segmented into syllables for effective comparison and feedback. We propose an unsupervised segmentation method called Spectrogram Formant Contour Analysis that detects syllable boundaries by analysing the change in contours of the formants in the spoken word spectrograms. The task of mispronunciation detection is more effective when the application can identify the actual syllable pronounced and communicate the correct pronunciation to the user. For the purpose of syllable classification, our method employs a novel approach where a model is trained on phonemes and given syllables as input for identification. Our study includes comparing the performance of three machine learning algorithms, namely, Convolution Neural Network, Support Vector Machines and K-Nearest Neighbours on the task of identifying phonemes when they are trained on minimal data. The accuracy of KNN on phoneme classification was the best with 80% for clean and 60% for noisy data. In case of our initial results on syllable classification for Kannada Kali, Support Vector Machines gave the highest accuracy of almost 30%.

Aero-tactile integration during speech perception: Effect of response and stimulus characteristics on syllable identification.

Integration of auditory and aero-tactile information during speech perception has been documented during two-way closed-choice syllable classification tasks [Gick and Derrick (2009). Nature 462, 502-504], but not during an open-choice task using continuous speech perception [Derrick, O'Beirne, Gorden, De Rybel, Fiasson, and Hay (2016). J. Acoust. Soc. Am. 140(4), 3225]. This study was designed to compare audio-tactile integration during open-choice perception of individual syllables. In addition, this study aimed to compare the effects of place and manner of articulation. Thirty-four untrained participants identified syllables in both auditory-only and audio-tactile conditions in an open-choice paradigm. In addition, forty participants performed a closed-choice perception experiment to allow direct comparison between these two response-type paradigms. Adaptive staircases, as noted by Watson [(1983). Percept. Psychophys. 33(2), 113-120] were used to identify the signal-to-noise ratio for identification accuracy thresholds. The results showed no significant effect of air flow on syllable identification accuracy during the open-choice task, but found a bias towards voiceless identification of labials, and towards voiced identification of velars. Comparison of the open-choice results to those of the closed-choice task show a significant difference between both response types, with audio-tactile integration shown in the closed-choice task, but not in the open-choice task. These results suggest that aero-tactile enhancement of speech perception is dependent on response type demands.

DeepSqueak: a deep learning-based system for detection and analysis of ultrasonic vocalizations.

Rodents engage in social communication through a rich repertoire of ultrasonic vocalizations (USVs). Recording and analysis of USVs has broad utility during diverse behavioral tests and can be performed noninvasively in almost any rodent behavioral model to provide rich insights into the emotional state and motor function of the test animal. Despite strong evidence that USVs serve an array of communicative functions, technical and financial limitations have been barriers for most laboratories to adopt vocalization analysis. Recently, deep learning has revolutionized the field of machine hearing and vision, by allowing computers to perform human-like activities including seeing, listening, and speaking. Such systems are constructed from biomimetic, "deep", artificial neural networks. Here, we present DeepSqueak, a USV detection and analysis software suite that can perform human quality USV detection and classification automatically, rapidly, and reliably using cutting-edge regional convolutional neural network architecture (Faster-RCNN). DeepSqueak was engineered to allow non-experts easy entry into USV detection and analysis yet is flexible and adaptable with a graphical user interface and offers access to numerous input and analysis features. Compared to other modern programs and manual analysis, DeepSqueak was able to reduce false positives, increase detection recall, dramatically reduce analysis time, optimize automatic syllable classification, and perform automatic syntax analysis on arbitrarily large numbers of syllables, all while maintaining manual selection review and supervised classification. DeepSqueak allows USV recording and analysis to be added easily to existing rodent behavioral procedures, hopefully revealing a wide range of innate responses to provide another dimension of insights into behavior when combined with conventional outcome measures.

Automatic lexical stress and pitch accent detection for L2 English speech using multi-distribution deep neural networks

This paper investigates the use of multi-distribution deep neural networks (MD-DNNs) for automatic lexical stress detection and pitch accent detection, which are useful for suprasegmental mispronunciation detection and diagnosis in second-language (L2) English speech. The features used in this paper cover syllable-based prosodic features (including maximum syllable loudness, syllable nucleus duration and a pair of dynamic pitch values) as well as lexical and syntactic features (encoded as binary variables). As stressed/accented syllables are more prominent than their neighbors, the two preceding and two following syllables are also taken into consideration. Experimental results show that the MD-DNN for lexical stress detection achieves an accuracy of 87.9% in syllable classification (primary/secondary/no stress) for words with three or more syllables. This performance is much better than those of our previous work using Gaussian mixture models (GMMs) and the prominence model (PM), whose accuracies are 72.1% and 76.3% respectively. Approached similarly as the lexical stress detector, the pitch accent detector obtains an accuracy of 90.2%, which is better than the results of using the GMMs and PM by about 9.6% and 6.9% respectively.

Automated analysis of song structure in complex birdsongs

Understanding communication and signalling has long been strived for in studies of animal behaviour. Many songbirds have a variable and complex song, closely connected to territory defence and reproductive success. However, the quantification of such variable song is challenging. In this paper, we present a novel, automated method for detection and classification of syllables in birdsong. The method provides a tool for pairwise comparison of syllables with the aim of grouping them in terms of their similarity. This allows analyses such as (1) determining repertoire size within an individual, (2) comparing song similarity between individuals within as well as between populations of a species and (3) comparing songs of different species (e.g. for species recognition). Our method is based on a particular feature representation of song units (syllables) which ensures invariance to shifts in time, frequency and amplitude. Using a single song from a great reed warbler, Acrocephalus arundinaceus, recorded in the wild, the proposed algorithm is evaluated by means of comparison to manual auditory and visual (spectrogram) song investigation by a human expert and to standard song analysis methods. Our birdsong analysis approach conforms well to manual classification and, moreover, outperforms the hitherto widely used methods based on mel-frequency cepstral coefficients and spectrogram cross-correlation. Thus, our algorithm is a methodological step forward for analyses of song (syllable) repertoires of birds singing with high complexity.

Automated classification of mouse pup isolation syllables: from cluster analysis to an Excel-based “mouse pup syllable classification calculator”

Mouse pups vocalize at high rates when they are cold or isolated from the nest. The proportions of each syllable type produced carry information about disease state and are being used as behavioral markers for the internal state of animals. Manual classifications of these vocalizations identified 10 syllable types based on their spectro-temporal features. However, manual classification of mouse syllables is time consuming and vulnerable to experimenter bias. This study uses an automated cluster analysis to identify acoustically distinct syllable types produced by CBA/CaJ mouse pups, and then compares the results to prior manual classification methods. The cluster analysis identified two syllable types, based on their frequency bands, that have continuous frequency-time structure, and two syllable types featuring abrupt frequency transitions. Although cluster analysis computed fewer syllable types than manual classification, the clusters represented well the probability distributions of the acoustic features within syllables. These probability distributions indicate that some of the manually classified syllable types are not statistically distinct. The characteristics of the four classified clusters were used to generate a Microsoft Excel-based mouse syllable classifier that rapidly categorizes syllables, with over a 90% match, into the syllable types determined by cluster analysis.

MFCC and Prosodic Feature Extraction Techniques: A Comparative Study

In this paper our main aim to provide the difference between cepstral and non-cepstral feature extraction techniques. Here we try to cover-up most of the comparative features of Mel Frequency Cepstral Coefficient and prosodic features. In speaker recognition, there are two type of techniques are available for feature extraction: Short-term features i.e. Mel Frequency Cepstral Coefficient (MFCC) and long-term features (Prosodic) extraction techniques. In this paper, we explore the usefulness of prosodic features for syllable classification and MFCC for feature extraction of a speech signal followed by comparison between them. The Me1 Frequency Cepstral Coefficients (MFCC) is one of the most important features extraction techniques, which is required among various kinds of speech applications. The MFCC features are extracted from the speaker phonemes in the presegmented speech sentences. Now days Prosodic features are currently used in most emotion recognition algorithms Prosodic features are relatively simple in their structures and known for their effectiveness in some speech recognition tasks. There are various ways of generating prosodic syllable contour features that have recently been applied to enhance systems for speaker recognition.

Unsupervised bird song syllable classification using evolving neural networks

Evolution of bird vocalizations is subjected to selection pressure related to their functions. Passerine bird songs are also under a neutral model of evolution because of the learning process supporting their transmission; thus they contain signals of individual, population, and species relationships. In order to retrieve this information, large amounts of data need to be processed. From vocalization recordings, songs are extracted and encoded as sequences of syllables before being compared. Encoding songs in such a way can be done either by ear and spectrogram visual analysis or by specific algorithms permitting reproducible studies. Here, a specific automatic method is presented to compute a syllable distance measure allowing an unsupervised classification of song syllables. Results obtained from the encoding of White-crowned Sparrow (Zonotrichia leucophrys pugetensis) songs are compared to human-based analysis.

Parametric Representations of Bird Sounds for Automatic Species Recognition

This paper is related to the development of signal processing techniques for automatic recognition of bird species. Three different parametric representations are compared. The first representation is based on sinusoidal modeling which has been earlier found useful for highly tonal bird sounds. Mel-cepstrum parameters are used since they have been found very useful in the parallel problem of speech recognition. Finally, a vector of various descriptive features is tested because such models are popular in audio classification applications, and bird song is almost like music. We briefly introduce the methods and evaluate their performance in the classification and recognition of both individual syllables and song fragments of 14 common North-European Passerine bird species

Persistence and change of vocal signals in natural populations of chickadees: annual sampling of the gargle call over eight seasons

A study of the gargle call, an agonistic call of the black-capped chickadee Poecile atricapillus, for eight successive fall/winter seasons in three resident populations in Colorado (USA) revealed the following:(1) Individuals had repertoires of calls averaging 8.1 distinct types per bird (range 2-18).(2) Among individuals in a local population many calls shared some of the component syllables constituting the whole gargle calls; often the shared syllables occurred in different locations in the sequences of syllables (syntax) making up the calls.(3) Individuals also shared many of the same calls (identical or near identical syllable sequences); thus in a local sample the ensemble of call types in the population (the population repertoire) could be found by recording less than 10 birds.(4) In each population, the ensemble of gargle call types found in a given season consisted of a core group of about 10 kinds of gargle calls that persisted over all or nearly all the eight seasons of the study, but also the ensemble included a number of call types that were present for only one or a few seasons, or that occurred intermittently across time.(5) The great majority of call types that were lost from a population were explained by the loss of individual birds (carriers) by emigration or death.(6) The great majority of call types new in a population in any given season were explained by the occurrence of a new bird in the sample (local recruitment or immigration).(7) Both qualitative classification of gargle syllables and whole gargle calls by the method of visual inspection of sound spectrograms and automated quantitative analysis of acoustic features of syllables and calls were carried out in some examples, and these comparisons gave highly concordant results.Long-term studies of cultural evolution of non-song calls are infrequent but important because the sources and strength of social selection on non-song vocalizations are often likely to differ from those social factors influencing cultural evolution in territorial songs, the more often studied kind of bird acoustic signals. As a primary finding of this study we showed, using fine-grained measurements of acoustic variables, that a population retains a set of core gargle groups, each group representing a cultural tradition (behavioral lineage) exhibiting virtually no drift in detailed structure across eight consecutive seasons. In a small set of samples of core groups going back to 1988, there also was no evidence of change within a vocal tradition. These results suggest the hypothesis of strong canalizing social selection on the acoustic structure of gargles.

A simple statistical speech recognition of mandarin monosyllables

Each mandarin syllable is represented by a sequence of vectors of linear predict coding cepstra (LPCC). Since all syllables have a simple phonetic structure, in our speech recognition, we partition the sequence of LPCC vectors of all syllables into equal segments and average the LPCC vectors in each segment. The mean vector of LPCC is used as the feature of a syllable. Our simple feature does not need any time consuming and complicated nonlinear contraction and expansion as adopted by the dynamic time-warping. We propose several probability distributions for the feature values. A simplified Bayes decision rule is used for classification of mandarin syllables. For the speaker-independent mandarin digits, the recognition rate is 98.6% if a normal distribution is used for feature values and the rate is 98.1% if an exponential distribution is used for the absolute values of the features. The feature proposed in this paper to represent a syllable is the simplest one, much easier to be extracted than any other known features. The computation for feature extraction and classification is much faster and more accurate than using the HMM method or any other known techniques.

Encoding and memory for visible speech and gestures: a comparison between young and older adults.

Two experiments explored whether older adults have developed a strategy of compensating for slower speeds of language processing and hearing loss by relying more on the visual modality. Experiment 1 examined the influence of visual articulatory movements of the face (visible speech) in auditory-visual syllable classification in young adults and older adults. Older adults showed a significantly greater influence of visible speech. Experiment 2 examined immediate recall in three spoken-language sentence conditions: speech alone, with visible speech, or with both visible speech and iconic gestures. Sentences also varied in meaningfulness and speech rate. In the old adult group, recall was better for sentences containing visible speech compared with the speech-alone sentences in the meaningful sentence condition. Old adults' recall showed no overall benefit of the presence of gestures. Young adults' recall on meaningful sentences was not higher for the visible speech compared with the speech-alone condition, whereas recall was significantly higher with the addition of iconic gestures. In the anomalous sentence condition, both young and old adults showed an advantage in recall by the presence of visible speech. The experiments provide converging evidence for old adults' greater reliance on visible speech while processing visual-spoken language.

Classification of Japanese syllables including the speech sound found in loan words

Methods of classifying Japanese syllables are discussed with regard to their use in evaluating performance of speech processing techniques for both synthesis/recognition systems and user's speaking/listening characteristics. First, by taking into account the initial/intervocalic contrast of utterance, 16 intervocalic syllables are introduced to the traditional 50 Japanese syllables chart or the list of 100 Japanese syllables that has been used in articulation test of speech transmission channels. Then, by rearranging those syllables based on place/manner of articulation and by supplementing missing kind of combinations of the preceding consonant and following vowel, more than 60 new syllables which can be pronounced in the loan words are derived. Also discussed in this paper is rationale for various levels of deriving new syllables that correspond to the special Kana letters used in writing loan words in Japanese.

Feature-based automatic syllable and stress detection

The importance of syllable structure and stress level as determinants of segmental temporal and spectral variability makes automatic syllable detection and stress estimation a very desirable goal for continuous speech recognition research. In this paper, a rule-based system is described for locating syllables in continuous speech and for making a two-level stress assignment. Location of syllable nuclei and rough estimation of syllable boundaries are performed using a smoothed midfrequency “sonorant” energy contour, a frication detector, and Sonorant consonant detectors. First pass classification of detected syllables as stressed, unstressed, or uncertain is based on the relative energy levels and utterance-position-normalized durations of syllables in a three-syllable window. Fundamental frequency information is then used to reclassify uncertain cases as either stressed or unstressed. Preliminary evaluation of the system's current performance on a multispeaker data base yielded 77% correct location and classification of syllables. Although improvement is necessary, this result is encouraging.