Pairs Of Utterances Research Articles

Recurrent neural network language generation for spoken dialogue systems

Natural Language Generation (NLG) is a critical component of spoken dialogue systems and it has a significant impact both on usability and perceived quality. Most existing NLG approaches in common use employ rules and heuristics and tend to generate rigid and stylised responses without the natural variation of human language. Moreover, these limitations also add significantly to development costs and make the delivery of cross-domain, cross-lingual dialogue systems especially complex and expensive. The first contribution of this paper is to present RNNLG, a Recurrent Neural Network (RNN)-based statistical natural language generator that can learn to generate utterances directly from dialogue act – utterance pairs without any predefined syntaxes or semantic alignments. The presentation includes a systematic comparison of the principal RNN-based NLG models available. The second contribution, is to test the scalability of the proposed system by adapting models from one domain to another. We show that by pairing RNN-based NLG models with a proposed data counterfeiting method and a discriminative objective function, a pre-trained model can be quickly adapted to different domains with only a few examples. All of the findings presented are supported by both corpus-based and human evaluations.

A Novel Machine Translation Method based on Stochastic Finite Automata Model for Spoken English

Stochastic finite automata have been applied to a variety of fields, machine trans-lation is one of them. It can learn from data and build model automatically from training sets. Stochastic finite automata are well adapted for the constrained inte-gration of pairs of sentences for language processing. In this paper, a novel ma-chine translation method based on stochastic finite automata is proposed. The method formalized rational grammars by using stochastic finite automata. Through given pairs of source and target utterances, our proposed method will produce a series of conventional rules from which a stochastic rational grammar would be inferred, and the grammar is finally converted into a finite state automa-ton. The efficacy and accuracy of our proposed method is evaluated by a large number of English-Chinese and Chinese-English machine translation experi-ments.

Development of Quantitative and Temporal Scalar Implicatures in a Felicity Judgment Task.

Experimental investigations into children’s interpretation of scalar terms show that children have difficulties with scalar implicatures in tasks. In contrast with adults, they are for instance not able to derive the pragmatic interpretation that “some” means “not all” (Noveck, 2001; Papafragou and Musolino, 2003). However, there is also substantial experimental evidence that children are not incapable of drawing scalar inferences and that they are aware of the pragmatic potential of scalar expressions. In these kinds of studies, the prime interest is to discover what conditions facilitate implicature production for children. One of the factors that seem to be difficult for children is the generation of the scalar alternative. In a Felicity Judgment Task (FJT) the alternative is given. Participants are presented with a pair of utterances and asked to choose the most felicitous description. In such a task, even 5-year-old children are reported to show a very good performance. Our study wants to build on this tradition, by using a FJT where not only “some-all” choices are given, but also “some-many” and “many-all.” In combination with a manipulation of the number of successes/failures in the stories, this enabled us to construct control, critical and ambiguous items. We compared the performance of 59 5-year-old children with that of 34 11-year-old children. The results indicated that performance of both age groups was clearly above chance, replicating previous findings. However, for the 5-year-old children, the critical and ambiguous items were more difficult than the control items and they also performed worse on these two types of items than the 11-year-old children. Interestingly with respect to the issue of scalar diversity, the 11-year-old children were also presented temporal items, which turned out to be more difficult than the quantitative ones.

Question-Answer in ‘Ayat-Ayat Cinta’ (A Study of Speech Act in Novel)

This study aims to obtain information related to the forms and the uses of linkages adjacency pair of 'question-answer' used in Ayat-Ayat Cinta, as well as understand the function of the adjacent pair in establishing the integrity of the novel discourse. This study is a qualitative study which is operationally using the principles of speech act study. The speech act referred in this study is a dialogue in the novel which becomes the data source of this research. Data was collected by the following procedure: (1) to identify the overall dialogue in the novel; Ayat-Ayat Cinta; (2) to determine the adjacent pair in the form of "questions-answers."; (3) to find the forms and the links between each pair of utterances focused on "question-answer” type. The research findings indicate that the adjacent pairs of "question-answer" are found in the following forms: (1) short sentences; (2) ellipsis (3) deep structure; and (4) informal language (slang). The function of the adjacency pairs of "question-answer" in this novel are as controlling the integrity of discourse, namely (1) to express pleasure/choice of the characters; (2) to explain the identity, background and character of the characters; and (3) to reveal the history of the past of the characters.

A robust unsupervised pattern discovery and clustering of speech signals

In this paper, a novel approach to unsupervised pattern discovery for speech signals is proposed. The proposed work deviates from the standard speech recognition task, and aims to cluster the speech utterances based on the vocabulary of a broad topic. It attempts to discover the matched sequence of phonetic units by making use of the repeated patterns between the speech signals. Identification of matched sequence of phonetic patterns helps in clustering the speech signals, automatically. The proposed approach uses the posterior features derived from Gaussian mixture model (GMM) to find the repeated structure between the speech signals. Image processing techniques are used to identify these matched acoustic patterns. An angle histogram-based method is used to extract the desired matched keyword/phrase patterns present in a pair of speech utterances. The performance of the proposed method is evaluated on Hindi and Bengali news speech corpora using standard objective measures, and also compared with state-of-the-art techniques. The matched pairs of speech utterances obtained by the proposed method are grouped into broader classes using an appropriate clustering technique. The final clusters represent the broader classes of information such as politics, sports, and weather.

Empirical feature analysis for dialogue breakdown detection

Chat-oriented dialogue systems sometimes generate inappropriate response utterances to user utterances that cause dialogue breakdown. Detecting such inappropriate utterances and suppressing them will support the continuation of the dialogue. Although a previous state-of-the-art dialogue breakdown detector leveraged dialogue-act transitions and word-based similarities between utterance pairs, these features are insufficient to evaluate the appropriateness of question-answering or relatedness between utterances that share few topic words. In this paper, we propose novel features to assess these problems, and examine their effectiveness for improving the performance of dialogue breakdown detection.

Smart Enough to Talk With Us? Foundations and Challenges for Dialogue Capable AI Systems

Smart Enough to Talk With Us? Foundations and Challenges for Dialogue Capable AI Systems

Uncovering mental representations of smiled speech using reverse correlation.

Which spectral cues underlie the perceptual processing of smiles in speech? Here, the question was addressed using reverse-correlation in the case of the isolated vowel [a]. Listeners were presented with hundreds of pairs of utterances with randomly manipulated spectral characteristics and were asked to indicate, in each pair, which was the most smiling. The analyses revealed that they relied on robust spectral representations that specifically encoded vowel's formants. These findings demonstrate the causal role played by formants in the perception of smile. Overall, this paper suggests a general method to estimate the spectral bases of high-level (e.g., emotional/social/paralinguistic) speech representations.

Acoustical and perceptual study of voice disguise by age modification in speaker verification

The task of speaker recognition is feasible when the speakers are co-operative or wish to be recognized. While modern automatic speaker verification (ASV) systems and some listeners are good at recognizing speakers from modal, unmodified speech, the task becomes notoriously difficult in situations of deliberate voice disguise when the speaker aims at masking his or her identity. We approach voice disguise from the perspective of acoustical and perceptual analysis using a self-collected corpus of 60 native Finnish speakers (31 female, 29 male) producing utterances in normal, intended young and intended old voice modes. The normal voices form a starting point and we are interested in studying how the two disguise modes impact the acoustical parameters and perceptual speaker similarity judgments.First, we study the effect of disguise as a relative change in fundamental frequency (F0) and formant frequencies (F1 to F4) from modal to disguised utterances. Next, we investigate whether or not speaker comparisons that are deemed easy or difficult by a modern ASV system have a similar difficulty level for the human listeners. Further, we study affecting factors from listener-related self-reported information that may explain a particular listener’s success or failure in speaker similarity assessment.Our acoustic analysis reveals a systematic increase in relative change in mean F0 for the intended young voices while for the intended old voices, the relative change is less prominent in most cases. Concerning the formants F1 through F4, 29% (for male) and 30% (for female) of the utterances did not exhibit a significant change in any formant value, while the remaining ∼ 70% of utterances had significant changes in at least one formant.Our listening panel consists of 70 listeners, 32 native and 38 non-native, who listened to 24 utterance pairs selected using rankings produced by an ASV system. The results indicate that speaker pairs categorized as easy by our ASV system were also easy for the average listener. Similarly, the listeners made more errors in the difficult trials. The listening results indicate that target (same speaker) trials were more difficult for the non-native group, while the performance for the non-target pairs was similar for both native and non-native groups.

TINDAK TUTUR ILOKUSI DALAM INTERAKSI PEMBELAJARAN BAHASA INDONESIA (Kajian Etnografi Komunikasi di SMA Ehipassiko School BSD)

ABSTRAKPenelitian ini bertujuan untuk memperoleh pemahaman mendalam mengenai tindak tutur ilokusi dalam interaksi pembelajaran bahasa Indonesia. Penelitian ini dilakukan dari bulan November 2016 hingga Maret 2017 di kelas XII IPS, SMA Ehipassiko School BSD. Penelitian ini difokuskan pada tindak tutur ilokusi, dengan subfokus kategori tindak tutur ilokusi, jenis tindak tutur ilokusi, dan fungsi tindak tutur ilokusi. Penelitian ini merupakan penelitian kualitatif dengan mengambil kajian etnografi komunikasi untuk meneliti tindak tutur ilokusi dalam situasi yang sebenarnya. Data penelitian dikumpulkan dengan metode observasi, simak, rekam, dan wawancara. Data tersebut kemudian dikelompokkan ke dalam pasangan tuturan dengan unit analisis situasi komunikatif, peristiwa komunikatif, dan tindak komunikatif. Hasil penelitian ini menunjukkan bahwa terdapat empat kategori tindak tutur ilokusi yang ditemukan dalam interaksi pembelajaran bahasa Indonesia di kelas XII IPS yaitu kategori asertif, direktif, komisif, dan ekspresif. Kategori-kategori tersebut tersebar ke dalam beberapa jenis yang memiliki fungsinya masing-masing.Kata Kunci : Tindak Tutur Ilokusi, Etnografi Komunikasi
 ABSTRACTThis research aims to obtain the deepest understanding about illocutionary speech act of Indonesian Language learning interaction. This research was conducted from November 2016 until March 2017 at class XII IPS, SMA Ehipassiko School BSD. This research focuses on category of illocutionary speech act, and subfocuses on type of illocutionary speech act and function of illocutionary speech act. This research ia a qualitative research using ethnography communication for research illocutionary speech act in a real situation. The data is collected through observation, listening, recording, and interview. The data is then grouped into pair of utterances with unit analysis communicative situation, communicative event, and communicative act. The result showed four categories of illocutionary speech act during Indonesia language learning interaction at grade XII IPS namely: assertive, directive, commisive, and ekspressive among several types which have their own function.Key Word : Illocutionary Speech Act, Ethnography of Communication

Partial matching and search space reduction for QbE-STD

Query-by-Example approach of spoken content retrieval has gained much attention because of its feasibility in the absence of speech recognition and its applicability in a multilingual matching scenario. This approach to retrieve spoken content is referred to as Query-by-Example Spoken Term Detection (QbE-STD). The state-of-the-art QbE-STD system performs matching between the frame sequence of query and test utterance via Dynamic Time Warping (DTW) algorithm. In realistic scenarios, there is a need to retrieve the query which does not appear exactly in the spoken document. However, the appeared instance of query might have the different suffix, prefix or word order. The DTW algorithm monotonically aligns the two sequences and hence, it is not suitable to perform partial matching between the frame sequence of query and test utterance. In this paper, we propose novel partial matching approach between spoken query and utterance using modified DTW algorithm where multiple warping paths are constructed for each query and test utterance pair. Next, we address the research issue associated with search complexity of DTW and suggest two approaches, namely, feature reduction approach and Bag-of-Acoustic-Words (BoAW) model. In feature reduction approach, the number of feature vectors is reduced by averaging across the consecutive frames within phonetic boundaries. Thus, a lesser number of feature vectors require fewer number of comparisons and hence, DTW speeds up the search computation. The search computation time gets reduced by 46–49% with a slight degradation in performance as compared to no feature reduction case. In BoAW model, we construct term frequency-inverse document frequency (tf−idf) vectors at segment-level to retrieve audio documents. The proposed segment-level BoAW model is used to match test utterance with a query using (tf−idf) vectors and the scores obtained are used to rank the test utterance. The BoAW model gave more than 80% recall value on 70% top retrieval. To re-score the detection, we further employ DTW search or modified DTW search to retrieve the spoken query from the selected utterances using BoAW model. QbE-STD experiments are conducted on different international benchmarks, namely, MediaEval spoken web search SWS 2013 and MediaEval query-by-example search on speech QUESST 2014.

Matching Speaking to Singing Voices and the Influence of Content

We tested whether speaking voices of unfamiliar people could be matched to their singing voices, and, if so, whether the content of the utterances would influence this matching performance. Our hypothesis was that enough acoustic features would remain the same between speaking and singing voices such that their identification as belonging to the same or different individuals would be possible even upon a single hearing. We also hypothesized that the contents of the utterances would influence this identification process such that voices uttering words would be easier to match than those uttering vowels. We used a within-participant design with blocked stimuli that were counterbalanced using a Latin square design. In one block, mode (speaking vs singing) was manipulated while content was held constant; in another block, content (word vs syllable) was manipulated while mode was held constant, and in the control block, both mode and content were held constant. Participants indicated whether the voices in any given pair of utterances belonged to the same person or to different people. Cross-mode matching was above chance level, although mode-congruent performance was better. Further, only speaking voices were easier to match when uttering words. We can identify speaking and singing voices as the same or different even on just a single hearing. However, content interacts with mode such that words benefit matching of speaking voices but not of singing voices. Results are discussed within an attentional framework.

Cognitive Load Reduces Perceived Linguistic Convergence Between Dyads.

Speech convergence is the tendency of talkers to become more similar to someone they are listening or talking to, whether that person is a conversational partner or merely a voice heard repeating words. To elucidate the nature of the mechanisms underlying convergence, this study uses different levels of task difficulty on speech convergence within dyads collaborating on a task. Dyad members had to build identical LEGO® constructions without being able to see each other's construction, and with each member having half of the instructions required to complete the construction. Three levels of task difficulty were created, with five dyads at each level (30 participants total). Task difficulty was also measured using completion time and error rate. Listeners who heard pairs of utterances from each dyad judged convergence to be occurring in the Easy condition and to a lesser extent in the Medium condition, but not in the Hard condition. Amplitude envelope acoustic similarity analyses of the same utterance pairs showed that convergence occurred in dyads with shorter completion times and lower error rates. Together, these results suggest that while speech convergence is a highly variable behavior, it may occur more in contexts of low cognitive load. The relevance of these results for the current automatic and socially-driven models of convergence is discussed.

Slow speech rate effects on stuttering preschoolers with disordered phonology

To study the effects of clinicians’ slow rate on the speech of children who stutter with and without a concomitant phonological disorder, an A–B–A–B single case design was used with six clinician–child dyads, where B = Clinician’s slow speech rate model. Two boys and one girl, aged 49–54 months, stuttering with disordered phonology (S + DP), were compared to three boys aged 42–50 months, stuttering with normal phonology (S + NP). Articulation rates were measured in phones per second (pps) in clinician–child adjacent utterance pairs. The S + NP dyads showed improved fluency in the B condition through a larger effect size, higher mean baseline stutter reductions and lower percentages of non-overlapping data than did the S + DP dyads. The S + DP girl showed relatively improved fluency in the B condition. S + DP children showed no articulation rate alignment (Range: 16% decrease to a 1.2% increase), whereas S + NP children averaged a 20% pps rate reduction (Range: 19.6–25.4% decrease), aligning with their clinicians who averaged a 38% pps rate reduction from baseline. The S + DP group spoke significantly (z = −4.63; p < 0.00) slower at baseline (Mdn = 6.9 pps; SE = 0.07 pps) than S + NP children in previously published samples (Mdn = 9.8 pps; SE = 0.22 pps). Results suggest that a slow rate model alone is not effective for facilitating fluency in S + DP boys with time since onset of about 2 years.

Influence of Second Language (L2) Proficiency on the Measure of Spatiotemporal Index of Bilabial Utterances in Typical Kannada (L1) English (L2) Bilingual Speakers

The Spatiotemporal Index (STI) of Lower lip (LL) across L1 (Kannada) and L2 (English) in High Proficient (HP)/Early and ˜Low Proficient (LP)/Late typical Kannada (L1)-English (L2) bilinguals was analyzed. A bilabial utterance pair matched for syllable structure and length, phonetic complexity, and overall duration across L1 and L2 were uttered 10 times in each language randomly using language switching paradigm by HP/Early (n = 7) and LP/Late (n = 7) bilinguals. The mean STI was significantly different across languages and L2 language proficiency, supporting the use of differential speech motor control strategies by participants to maintain phonetic distinctiveness across languages. The STI was more variable in HP/Early bilinguals across L1 and L2 than LP/Late bilinguals and HP/Early bilinguals showed higher variability in Kannada (L1), reflecting higher organizational flexibility and movement control along with successful suppression of the non target language (L2). The mean STI in LP/Late bilinguals did not differ significantly across languages suggesting reduced movement flexibility and inability to suppress the interfering effect of L1 on L2. The results are discussed in the perspective of the measure of STI reflecting an intra articulatory global movement variability that is sensitive in understanding the movement control differences in the speech production of bilinguals. The findings are also discussed in the backdrop of bilingual speech production models and possible influence of âcross Linguistic Interference on articulatory movement stability across L1 and L2.

Single-channel dereverberation by feature mapping using cascade neural networks for robust distant speaker identification and speech recognition

We present a feature enhancement method that uses neural networks (NNs) to map the reverberant feature in a log-melspectral domain to its corresponding anechoic feature. The mapping is done by cascade NNs trained using Cascade2 algorithm with an implementation of segment-based normalization. Experiments using speaker identification (SID) and automatic speech recognition (ASR) systems were conducted to evaluate the method. The experiments of SID system was conducted by using our own simulated and real reverberant datasets, while the CENSREC-4 evaluation framework was used as the evaluation for the ASR system. The proposed method could remarkably improve the performance of both systems by using limited stereo data and low speaker-variant data as the training data. From the evaluation using SID, we reached 26.0% and 34.8% of error rate reduction (ERR) relative to the baseline by using simulated and real data, respectively, by using only one pair of utterances for matched condition cases. Then, by using combined dataset containing 15 pairs of utterances by one speaker from three positions in a room, we could reach 93.7% of average identification rate (three known and two unknown positions), which was 42.2% of ERR relative to the use of cepstral mean normalization (CMN). From the evaluation using ASR, by using 40 pairs of utterances as the NN training data, we could reach 78.4% of ERR relative to the baseline by using simulated utterances by five speakers. Moreover, we could reach 75.4% and 71.6% of ERR relative to the baseline by using real utterances by five speakers and one speaker, respectively.

Detecting cues to deception from children's facial expressions: On the effectiveness of two visual manipulation techniques

This paper looks into the extent to which facial expressions may reveal whether a person is telling the truth or not. More specifically, it studies selected clips from video recordings of children who had participated in an interactive story paradigm that naturally elicits minimal pairs of truthful and deceptive utterances in participants. In two perception experiments, these pairs of clips (with audio removed) were shown to adult observers who were given the task to guess which member of the pair contained the deceptive utterance. Experiment 1 tested whether the likelihood of correct detection could be enhanced when clips were slowed down compared to clips at normal speed. Results revealed that this manipulation indeed had a positive effect on lie detection, albeit that the effect interacted with the order of presentation (lies are easier to see when they are shown after rather than before the truthful utterance) and kind of lie (second attempts of children to lie reveal more cues than their first attempts). Experiment 2 explored whether lie detection is different for recordings in which the full face of a child is shown, or for recordings in which either the eye or mouth region is hidden after digital manipulation. This experiment revealed that the partial presentations of the face lead to more correct deception detection than the full face presentation. Implications of the outcomes of Experiment 1 and 2 for lie detection and for a general model of nonverbal communication are discussed.

Feeling Voices

Two experiments investigated deaf individuals' ability to discriminate between same-sex talkers based on vibrotactile stimulation alone. Nineteen participants made same/different judgments on pairs of utterances presented to the lower back through voice coils embedded in a conforming chair. Discrimination of stimuli matched for F0, duration, and perceived magnitude was successful for pairs of spoken sentences in Experiment 1 (median percent correct = 83%) and pairs of vowel utterances in Experiment 2 (median percent correct = 75%). Greater difference in spectral tilt between “different” pairs strongly predicted their discriminability in both experiments. The current findings support the hypothesis that discrimination of complex vibrotactile stimuli involves the cortical integration of spectral information filtered through frequency-tuned skin receptors.

Word Co-occurrence Analysis with Utterance Pairs for Spoken Dialogue System

Word Co-occurrence Analysis with Utterance Pairs for Spoken Dialogue System

Analysis of Large-Scale SVM Training Algorithms for Language and Speaker Recognition

This paper compares a set of large scale support vector machine (SVM) training algorithms for language and speaker recognition tasks. We analyze five approaches for training phonetic and acoustic SVM models for language recognition. We compare the performance of these approaches as a function of the training time required by each of them to reach convergence, and we discuss their scalability towards large corpora. Two of these algorithms can be used in speaker recognition to train a SVM that classifies pairs of utterances as either belonging to the same speaker or to two different speakers. Our results show that the accuracy of these algorithms is asymptotically equivalent, but they have different behavior with respect to the time required to converge. Some of these algorithms not only scale linearly with the training set size, but are also able to give their best results after just a few iterations. State-of-the-art performance has been obtained in the female subset of the NIST 2010 Speaker Recognition Evaluation extended core test using a single SVM system.