Perceived Speaker Identity Research Articles

Top-down effect of dialogue coherence on perceived speaker identity

A key mechanism in the comprehension of conversation is the ability for listeners to recognize who is speaking and when a speaker switch occurs. Some authors suggest that speaker change detection is accomplished through bottom-up mechanisms in which listeners draw on changes in the acoustic features of the auditory signal. Other accounts propose that speaker change detection involves drawing on top-down linguistic representations to identify who is speaking. The present study investigates these hypotheses experimentally by manipulating the pragmatic coherence of conversational utterances. In experiment 1, participants listened to pairs of utterances and had to indicate whether they heard the same or different speakers. Even though all utterances were spoken by the same speaker, our results show that when two segments of conversation are spoken by the same speaker but make sense for different speakers to say, listeners report hearing different speakers. In experiment 2 we removed pragmatic information from the same stimuli by scrambling word order while leaving acoustic information intact. In contrast to experiment 1, results from the second experiment indicate no difference between our experimental conditions. We interpret these results as a top-down effect of pragmatic expectations: knowledge of conversational structure at least partially determines a listener’s perception of speaker changes in conversation.

Individual Differences in Political Ideology and Disgust Sensitivity Affect Real-Time Spoken Language Comprehension.

Individuals' moral views have been shown to affect their event-related potentials (ERP) response to spoken statements, and people's political ideology has been shown to guide their sentence completion behavior. Using pupillometry, we asked whether political ideology and disgust sensitivity affect online spoken language comprehension. 60 native speakers of English listened to spoken utterances while their pupil size was tracked. Some of those utterances contained grammatical errors, semantic anomalies, or socio-cultural violations, statements incongruent with existing gender stereotypes and perceived speaker identity, such as “I sometimes buy my bras at Hudson's Bay,” spoken by a male speaker. An individual's disgust sensitivity is associated with the Behavioral Immune System, and may be correlated with socio-political attitudes, for example regarding out-group stigmatization. We found that more disgust-sensitive individuals showed greater pupil dilation with semantic anomalies and socio-cultural violations. However, political views differently affected the processing of the two types of violations: whereas more conservative listeners showed a greater pupil response to socio-cultural violations, more progressive listeners engaged more with semantic anomalies, but this effect appeared much later in the pupil record.

The role of between- versus within-speaker acoustic variability in vocal identity perception

Our recent studies [JASA 145(Pt. 2), 1930, (2019); this conference] show that acoustic spaces characterizing within- and between-speaker variability in voice quality have similar structures, with a few features (acoustic variability and formant dispersion) important for all speakers combined with idiosyncratic features characterizing individual talkers. These findings suggest that voice discrimination should be based on shared features, while “telling voices together” should depend on knowledge of each individual’s vocal idiosyncrasies. To test this hypothesis, we selected a set of voices that varied systematically in acoustic closeness to each other and to the center of the group acoustic space, based on values of shared features. Listeners were presented with multiple samples of each voice and were asked to sort the voices into piles according to perceived speaker identity. Based on prototype models of voice perception, we hypothesized that errors in telling voices apart would be strongly predictable from distances in the group acoustic space, but that errors in telling voices together would not be significantly associated with these distances. Separate analyses will attempt to shed light on the features and strategies involved in this second kind of judgment. [Work supported by NIH/NSF.]

Towards understanding speaker discrimination abilities in humans and machines for text-independent short utterances of different speech styles.

Little is known about human and machine speaker discrimination ability when utterances are very short and the speaking style is variable. This study compares text-independent speaker discrimination ability of humans and machines based on utterances shorter than 2 s in two different speaking styles (read sentences and speech directed towards pets, characterized by exaggerated prosody). Recordings of 50 female speakers drawn from the UCLA Speaker Variability Database were used as stimuli. Performance of 65 human listeners was compared to i-vector-based automatic speaker verification systems using mel-frequency cepstral coefficients, voice quality features, which were inspired by a psychoacoustic model of voice perception, or their combination by score-level fusion. Humans always outperformed machines, except in the case of style-mismatched pairs from perceptually-marked speakers. Speaker representations by humans and machines were compared using multi-dimensional scaling (MDS). Canonical correlation analysis showed a weak correlation between machine and human MDS spaces. Multiple regression showed that means of voice quality features could represent the most important human MDS dimension well, but not the dimensions from machines. These results suggest that speaker representations by humans and machines are different, and machine performance might be improved by better understanding how different acoustic features relate to perceived speaker identity.

The case for aural perceptual speaker identification

Once forensic speaker identification (SI) was recognized as an entity, it was predicted that valid computer based identification systems would quickly become a reality. This has not happened and the review to follow will provide some of the reasons why. Notable among them are (1) the sharp underestimation of its complexity and (2) its confounding with speaker verification (SV). Consideration of these (and related) issues will be followed by a brief history about how the need for SI developed and some of the responses to the problem. Since much of the SI development preceded the structuring of appropriate standards, the recommended stop-gap response described here is based on somewhat uncoordinated, but extensive, research. The product of that effort will be reviewed and organized into a platform which supports SI procedures consistent with the forensic model. Also discussed are the standards which have been established, their impact on SI development and its present limitations. How the cited approach interacts both with progress in verification and the developing SI machine-based identification systems also will be considered. Finally, a few suggestions will be made that should assist in upgrading the effectiveness of aural perceptual speaker identification (AP SI).

Speaker-dependent characteristics of the nasals

Investigation on human speaker identification enables us to know the indexical cues to speakers, and it may consequently lead to the effective acoustical parameters that can be used for forensic speaker recognition. It is known that speaker individuality interacts with the phonological or linguistic information contained in speech signals. As proof, the accuracy of perceptual speaker identification (PSI) performances depends on what types of sounds are presented to the listeners. In a series of our previous experiments, we have been investigating the effective sounds for PSI, and the stimuli containing a nasal were found to be the ones. In this present study, we conducted another PSI experiment in order to examine the reproducibility of the nasal effectiveness, and to see the effects of the following vowels. Coronal nasals were shown to be effective despite the different speaker set or the following vowels, and the stimuli containing a nasal were significantly better than those without it. In the second part of this paper, we introduce the results of the acoustical analysis of the stimuli. The contours of the energy transitions showed variations in shape among speakers for all three types of the analysis targets; nasals, stops, and fricatives, although the inter-speaker difference in the energy slopes for the consonant articulation was significant especially in nasal sounds. We also examined the effects of the sampling frequencies and the speech codecs, and found that the speaker-dependent shapes of these energy contours were maintained as long as the speech materials were uncompressed. The contours of the nasals appeared to be stable within a speaker, compared to other types of sounds.

Effects of linguistic contents on perceptual speaker identification: Comparison of familiar and unknown speaker identifications

There are several factors that affect human speaker recognition. In this study, two experiments were conducted in order to see the effects that the stimulus contents and the familiarity to the speakers give to the perception of the speakers. The results showed that: a) stimuli including a nasal were effective for accurate speaker identification; b) coronal nasals were more effective than the labial nasal, and c) the familiarity to the speakers gives a great influence on the performance. The tendencies a) and b) were observed both in familiar and unknown speaker identifications. The results of the acoustical analyses also showed that there were correspondences between the perception of the speaker identity and the cepstral distances among the speakers. The inter-speaker cepstral distances were greater in vowel intervals than in the consonant intervals; especially, notably they were greater in nasals than in orals in the consonant intervals.

Differential effects of the phonemes on identification of previously unknown speakers

In perceptual speaker identification, it is known that the identification accuracy depends on the contents of the stimuli presented to the listeners. In our previous experiments, we found that the stimuli containing a nasal sound are effective for human speaker identification, and that coronal sounds are more effective than labial ones. This tendency was observed in the identifications of both familiar and previously unknown speakers. In this present study, we investigated the effects of the speech contents again, by focusing on the vowels in CV monosyllabic stimuli. Through the experiment we obtained several findings: 1) stimuli containing a nasal gained significantly higher accuracy compared to stimuli with only oral sounds; 2) coronal sounds were more effective than labial sounds; 3) palatalisation caused an improvement in performance; and 4) back vowels were more effective than front vowels significantly. These results can be explained by speaker-specific morphologies of the nasal cavity and the paranasal sinuses, and also of the pharyngeal cavity. We will also show analyses and discussions on the acoustical properties of the stimuli and the performance differences among the subjects. [Work supported by Sophia University Open Research Centre from MEXT.]

Effects of stimulus contents and speaker familiarity on perceptual speaker identification

Effects of stimulus contents and speaker familiarity on perceptual speaker identification

Phoneme dependency of accuracy rates in familiar and unknown speaker identification

For perceptual speaker identification, the identification accuracy depends on the speech contents presented to subjects. Our previous studies have shown that stimuli containing nasals are effective for identifying familiar speakers [Amino et al., Acoust. Sci. Tech. 27(4) (2006)]. We have also presented the possibility that the interspeaker spectral distances reflect perceptual speaker similarities. In the present study, we conducted an experiment in which four unknown speakers were identified by 15 subjects. The stimuli were identical to those used in the previous study, in which ten speakers were identified by familiar listeners, although the speakers were fewer this time. Nine consonants in the CV structure were used as stimuli. The consonants were /d/, /t/, /z/, /s/, /r/, /j/, /m/, /n/, and /nj/; the vowel was restricted to /a/ for all CV syllables to simplify the experiment. The results showed that the nasals /n/ and /nj/ obtained higher scores. Tendencies in the differences among consonants were on the same order as those of the previous experiment, but the average scores were lower than those for familiar listeners. [Work supported by Grant-in-Aid for JSPS Fellows 17-6901.]

Speaker identification using noncontemporary speech samples

There are two noncontemporary elements important to speaker identification. In one, listeners are asked to make identifications at various latencies after hearing the speaker (time of identification is noncontemporary) or, in the second, samples of the speaker’s voice are obtained at markedly different times (noncontemporary samples). It has been found that memory for voice decays over time in the first case; hence it has been assumed that the noncontemporary samples pose just as difficult a challenge. AN aural–perceptual speaker identification project was carried out to test this second relationship. Noncontemporary samples, with latencies of 4, 8, and 32 weeks, and then 6 and 20 years, were studied. It was found that mean correct identification was initially reduced by 15% and this level was sustained for up to 6 years. It was only after 20 years that levels dropped significantly (to 33%). It is concluded that listener’s efficiency in identifying noncontemporary speech will show only modest decay over time and, hence, its use ordinarily should have only a minimal effect on the speaker identification process.

Contributions of voice-source and vocal-tract characteristics to speaker identity

A study was done on whether a voice-source model, in particular the Liljencrants–Fant model, codes information that is used by listeners to identify speakers by their voices. Automatic analysis/resynthesis techniques were used to generate so-called hybrid vowels for which the voice-source characteristics of one speaker are combined with the vocal-tract characteristics of another speaker. Listeners who were trained to recognize the speakers by their natural utterances, had to indicate which speaker they thought had produced the hybrid stimulus. It was found that there is no general rule that vocal-tract information contributes more to perceived speaker identity than voice-source information. Sometimes vocal-tract information is more important, sometimes voice-source information. The results fit the subjective cues that listeners reported they had been using to perform the speaker identification task quite well. Besides the expected importance of the vocal-tract filter, the spectral balance between high- and low-frequency components of the voice-source spectrum and the flutter of F0 proved to be important perceptual cues for speaker identity.

The role of perceived speaker identity in F0 normalization of vowels

In the experiments reported here, perceived speaker identity was controlled by manipulating the fundamental frequency (F0) range of carrier phrases in which speech tokens were embedded. In the first experiment, words from two "hood"-"hud" continua were synthesized with different F0. The words were then embedded in synthetic carrier phrases with intonation contours which reduced perceived speaker identity differences for test items with different F0. The results indicated that when perceived speaker identity differences were reduced, the effect of F0 on vowel identification was also reduced. Experiment 2 indicated that when items presented in carrier phrases are matched for speaker identity and F0 with items in isolation, there is no effect for presentation in a carrier phrase. Experiment 3 involved the presentation of vowels from the "hood"-"hud" continuum in two different intonational contexts which were judged to have been produced by different speakers, even though the F0 of the test word was identical in the two contexts. There was a shift in identification as a result of the intonational context which was interpreted as evidence for the role of perceived identity in vowel normalization. Overall, the experiments suggest that perceived speaker identity is a better predictor of vowel normalization effects than is intrinsic F0. This indicates that the role of F0 in vowel normalization is mediated through perceived speaker identity.