Keyword Spotting Research Articles

Text-independent pronunciation quality automatic assessment system for English retelling test

An automatic grading system for spoken English retelling test is presented in this paper. Speech recognition technology is used in the system to evaluate the quality of retelling according to the pre-defined scoring rubric which includes speech fluency, pronunciation accuracy and content integrity. Scoring features for these quality aspects are firstly extracted by applying LVCSR, keyword spotting, forced alignment and confidence measurements. And then, these features are mapped to a score by using SVM model which is pre-trained on human rated test items. According to the experimental results the correlation coefficient between machine scores and expert scores is 0.729, which means that the system can be used in real examination to replace human scores. This work is partially supported by the National Natural Science Foundation of China (No. 10925419, 90920302, 10874203, 60875014, 61072124, 11074275, 11161140319).

Beam Pruning Based on Quantile for Keyword Spotting

In this paper, we propose a novel pruning method for the Viterbi decoding in the task of keyword spotting. This method aims to reduce the computation time without any sacrifice of the detection performance. Firstly, α-quantile is employed to set the beam, and it reduces the hypothesis in proportion. Secondly, a flexible beam is achieved by a linear adjustment. The experiments conducted on the King-ASR-023 database show that the proposed method leads to a significant reduction in the computation time over the baseline system with fixed width of the beam.

A Hybrid Model for Automatic Emotion Recognition in Suicide Notes

We describe the Open University team’s submission to the 2011 i2b2/VA/Cincinnati Medical Natural Language Processing Challenge, Track 2 Shared Task for sentiment analysis in suicide notes. This Shared Task focused on the development of automatic systems that identify, at the sentence level, affective text of 15 specific emotions from suicide notes. We propose a hybrid model that incorporates a number of natural language processing techniques, including lexicon-based keyword spotting, CRF-based emotion cue identification, and machine learning-based emotion classification. The results generated by different techniques are integrated using different vote-based merging strategies. The automated system performed well against the manually-annotated gold standard, and achieved encouraging results with a micro-averaged F-measure score of 61.39% in textual emotion recognition, which was ranked 1st place out of 24 participant teams in this challenge. The results demonstrate that effective emotion recognition by an automated system is possible when a large annotated corpus is available.

Multi-stream LSTM-HMM decoding and histogram equalization for noise robust keyword spotting

Highly spontaneous, conversational, and potentially emotional and noisy speech is known to be a challenge for today's automatic speech recognition (ASR) systems, which highlights the need for advanced algorithms that improve speech features and models. Histogram Equalization is an efficient method to reduce the mismatch between clean and noisy conditions by normalizing all moments of the probability distribution of the feature vector components. In this article, we propose to combine histogram equalization and multi-condition training for robust keyword detection in noisy speech. To better cope with conversational speaking styles, we show how contextual information can be effectively exploited in a multi-stream ASR framework that dynamically models context-sensitive phoneme estimates generated by a long short-term memory neural network. The proposed techniques are evaluated on the SEMAINE database-a corpus containing emotionally colored conversations with a cognitive system for "Sensitive Artificial Listening".

Tandem decoding of children's speech for keyword detection in a child-robot interaction scenario

In this article, we focus on keyword detection in children's speech as it is needed in voice command systems. We use the FAU Aibo Emotion Corpus which contains emotionally colored spontaneous children's speech recorded in a child-robot interaction scenario and investigate various recent keyword spotting techniques. As the principle of bidirectional Long Short-Term Memory (BLSTM) is known to be well-suited for context-sensitive phoneme prediction, we incorporate a BLSTM network into a Tandem model for flexible coarticulation modeling in children's speech. Our experiments reveal that the Tandem model prevails over a triphone-based Hidden Markov Model approach.

A Novel Word Spotting Method Based on Recurrent Neural Networks

Keyword spotting refers to the process of retrieving all instances of a given keyword from a document. In the present paper, a novel keyword spotting method for handwritten documents is described. It is derived from a neural network-based system for unconstrained handwriting recognition. As such it performs template-free spotting, i.e., it is not necessary for a keyword to appear in the training set. The keyword spotting is done using a modification of the CTC Token Passing algorithm in conjunction with a recurrent neural network. We demonstrate that the proposed systems outperform not only a classical dynamic time warping-based approach but also a modern keyword spotting system, based on hidden Markov models. Furthermore, we analyze the performance of the underlying neural networks when using them in a recognition task followed by keyword spotting on the produced transcription. We point out the advantages of keyword spotting when compared to classic text line recognition.

Performance evaluation for an HMM-based keyword spotter and a large-margin based one in noisy environments

Abstract Keyword spotting refers to the detection of a limited number of given keywords in speech utterances. In this paper, first we review one of the large margin based keyword spotting approach that uses a discriminative method for training the keyword spotter. Then, we evaluate the robustness of this approach in different noisy conditions. In addition; we compare the performance of this method with an HMM-based keyword spotter -which uses a generative training method- in the same noisy conditions. The experimental results show that the large-margin based keyword spotter is more robust than HMM-based system in noisy environments.

A survey of keyword spotting techniques for printed document images

This paper attempts to provide a survey of the past researches on character based as keyword based approaches used for retrieving information from document images. This survey also provides insights into the strengths and weaknesses of current techniques, relevancy lies between each technique and also the guidance in choosing the area that future work on document image retrieval could address.

SACA: Software Assisted Call Analysis – An interactive tool supporting content exploration, online guidance and quality improvement of counseling dialogues

Objective Nearly 30,000 individual inquiries are answered annually by the telephone cancer information service (CIS, KID) of the German Cancer Research Center (DKFZ). The aim was to develop a tool for evaluating these calls, and to support the complete counseling process interactively. Methods A novel software tool is introduced, based on a structure similar to a music score. Treating the interaction as a “duet”, guided by the CIS counselor, the essential contents of the dialogue are extracted automatically. For this, “trained speech recognition” is applied to the (known) counselor's part, and “keyword spotting” is used on the (unknown) client's part to pick out specific items from the “word streams”. The outcomes fill an abstract score representing the dialogue. Results Pilot tests performed on a prototype of SACA (Software Assisted Call Analysis) resulted in a basic proof of concept: Demographic data as well as information regarding the situation of the caller could be identified. Conclusion The study encourages following up on the vision of an integrated SACA tool for supporting calls online and performing statistics on its knowledge database offline. Practice implications Further research perspectives are to check SACA's potential in comparison with established interaction analysis systems like RIAS.

Bidirectional LSTM Networks for Context-Sensitive Keyword Detection in a Cognitive Virtual Agent Framework

Robustly detecting keywords in human speech is an important precondition for cognitive systems, which aim at intelligently interacting with users. Conventional techniques for keyword spotting usually show good performance when evaluated on well articulated read speech. However, modeling natural, spontaneous, and emotionally colored speech is challenging for today’s speech recognition systems and thus requires novel approaches with enhanced robustness. In this article, we propose a new architecture for vocabulary independent keyword detection as needed for cognitive virtual agents such as the SEMAINE system. Our word spotting model is composed of a Dynamic Bayesian Network (DBN) and a bidirectional Long Short-Term Memory (BLSTM) recurrent neural net. The BLSTM network uses a self-learned amount of contextual information to provide a discrete phoneme prediction feature for the DBN, which is able to distinguish between keywords and arbitrary speech. We evaluate our Tandem BLSTM-DBN technique on both read speech and spontaneous emotional speech and show that our method significantly outperforms conventional Hidden Markov Model-based approaches for both application scenarios.

Phoneme lattice construction and its application to speech recognition and keyword spotting

An arrangement is provided for using a phoneme lattice for speech recognition and/or keyword spotting. The phoneme lattice may be constructed for an input speech signal and searched to produce a textual representation for the input speech signal and/or to determine if the input speech signal contains targeted keywords. An expectation maximization (EM) trained phoneme confusion matrix may be used when searching the phoneme lattice. The phoneme lattice may be constructed in a client and sent to a server, which may search the phoneme lattice to produce a result.

Keyword Spotting in Multilingual Environments

Keyword Spotting in Multilingual Environments

Estimation of Phone Mismatch Penalty Matrices for Two-Stage Keyword Spotting

Estimation of Phone Mismatch Penalty Matrices for Two-Stage Keyword Spotting

On-line emotion recognition in a 3-D activation-valence-time continuum using acoustic and linguistic cues

For many applications of emotion recognition, such as virtual agents, the system must select responses while the user is speaking. This requires reliable on-line recognition of the user’s affect. However most emotion recognition systems are based on turnwise processing. We present a novel approach to on-line emotion recognition from speech using Long Short-Term Memory Recurrent Neural Networks. Emotion is recognised frame-wise in a two-dimensional valence-activation continuum. In contrast to current state-of-the-art approaches, recognition is performed on low-level signal frames, similar to those used for speech recognition. No statistical functionals are applied to low-level feature contours. Framing at a higher level is therefore unnecessary and regression outputs can be produced in real-time for every low-level input frame. We also investigate the benefits of including linguistic features on the signal frame level obtained by a keyword spotter.

Point Process Models for Spotting Keywords in Continuous Speech

We investigate the hypothesis that the linguistic content underlying human speech may be coded in the pattern of timings of various acoustic ldquoeventsrdquo (landmarks) in the speech signal. This hypothesis is supported by several strands of research in the fields of linguistics, speech perception, and neuroscience. In this paper, we put these scientific motivations to the test by formulating a point process-based computational framework for the task of spotting keywords in continuous speech. We find that even with a noisy and extremely sparse phonetic landmark-based point process representation, keywords can be spotted with accuracy levels comparable to recently studied hidden Markov model-based keyword spotting systems. We show that the performance of our keyword spotting system in the high-precision regime is better predicted by the median duration of the keyword rather than simply the number of its constituent syllables or phonemes. When we are confronted with very few (in the extreme case, zero) examples of the keyword in question, we find that constructing a keyword detector from its component syllable detectors provides a viable approach.

Discriminative keyword spotting

This paper proposes a new approach for keyword spotting, which is based on large margin and kernel methods rather than on HMMs. Unlike previous approaches, the proposed method employs a discriminative learning procedure, in which the learning phase aims at achieving a high area under the ROC curve, as this quantity is the most common measure to evaluate keyword spotters. The keyword spotter we devise is based on mapping the input acoustic representation of the speech utterance along with the target keyword into a vector-space. Building on techniques used for large margin and kernel methods for predicting whole sequences, our keyword spotter distills to a classifier in this vector-space, which separates speech utterances in which the keyword is uttered from speech utterances in which the keyword is not uttered. We describe a simple iterative algorithm for training the keyword spotter and discuss its formal properties, showing theoretically that it attains high area under the ROC curve. Experiments on read speech with the TIMIT corpus show that the resulted discriminative system outperforms the conventional context-independent HMM-based system. Further experiments using the TIMIT trained model, but tested on both read (HTIMIT, WSJ) and spontaneous speech (OGI Stories), show that without further training or adaptation to the new corpus our discriminative system outperforms the conventional context-independent HMM-based system.

Point process models of distinctive feature landmarks for speech recognition.

Several interrelated strands of research in linguistics, acoustic phonetics, and cognitive neuroscience suggest a host of new directions for the development of end-to-end computational models of speech perception and recognition. Natural candidates for exploration include (i) phonological representations in terms of distinctive features; (ii) nonlinear detectors for distinctive feature landmarks (or any other set of perceptually salient acoustic events), which define a sparse point process representation of the speech signal; (iii) syllable-metered temporal processing and/or syllable-sized integration windows; and (iv) point process models and hierarchical strategies for recognizing words, syllables, phonemes, and features. A computational framework around these ideas has been developed and has led to phonetic recognition and keyword spotting performance that is competitive with equivalent hidden Markov model-based systems. This framework thus connects a computational platform for benchmarking competing scientific theories with simultaneous advancement toward a viable technological solution to the speech recognition problem.

Modelling of the cochlea response as a versatile tool for acoustic signal processing

The inner ear or cochlea processes the acoustic signals that enter the oval window into a specific time‐frequency pattern. Many acoustic signal processing methods are based on this behaviour. A fundamental method is to calculate this time‐frequency response by solving the differential equation of the movement of the basilar membrane, followed by a visualisation of the excitation patterns in a time‐frequency plot. For that purpose Continuity Preserving Signal Processing (CPSP) is a promising method. In the presentation an overview will be given of a project that is carried out by TUD (University of Technology Delft) together with RUG (University of Groningen) being sponsored by STW (Dutch Technology Foundation). The project divides into four sub‐projects which are closely related: Automatic Keyword Spotting, Machine Analysis and Diagnostics, Speech Intelligibility Enhancement for Hearing Aids and Quality Assessment of Room Acoustics. Results that have been obtained in the project will be summarised. Detailed results of the sub‐projects will be presented in separate presentations.

A comparison of grapheme and phoneme-based units for Spanish spoken term detection

The ever-increasing volume of audio data available online through the world wide web means that automatic methods for indexing and search are becoming essential. Hidden Markov model (HMM) keyword spotting and lattice search techniques are the two most common approaches used by such systems. In keyword spotting, models or templates are defined for each search term prior to accessing the speech and used to find matches. Lattice search (referred to as spoken term detection), uses a pre-indexing of speech data in terms of word or sub-word units, which can then quickly be searched for arbitrary terms without referring to the original audio. In both cases, the search term can be modelled in terms of sub-word units, typically phonemes. For in-vocabulary words (i.e. words that appear in the pronunciation dictionary), the letter-to-sound conversion systems are accepted to work well. However, for out-of-vocabulary (OOV) search terms, letter-to-sound conversion must be used to generate a pronunciation for the search term. This is usually a hard decision (i.e. not probabilistic and with no possibility of backtracking), and errors introduced at this step are difficult to recover from. We therefore propose the direct use of graphemes (i.e., letter-based sub-word units) for acoustic modelling. This is expected to work particularly well in languages such as Spanish, where despite the letter-to-sound mapping being very regular, the correspondence is not one-to-one, and there will be benefits from avoiding hard decisions at early stages of processing. In this article, we compare three approaches for Spanish keyword spotting or spoken term detection, and within each of these we compare acoustic modelling based on phone and grapheme units. Experiments were performed using the Spanish geographical-domain A lbayzin corpus. Results achieved in the two approaches proposed for spoken term detection show us that trigrapheme units for acoustic modelling match or exceed the performance of phone-based acoustic models. In the method proposed for keyword spotting, the results achieved with each acoustic model are very similar.

Telephone number retrieval system and method

Methods and systems directed to telephone number retrieval are provided. A terminal requests a telephone number from a telephone number service. A voice response comprising the telephone number is received from the telephone number service. A feature vector sequence is stored in memory of the terminal as a reference pattern for the received voice response. The voice response has designated words between and among numbers of the telephone number. A keyword spotting routine searches for the designated words. Connecting words, located between and among designated words, comprising the telephone number are extracted and stored in a buffer. A number recognizer analyzes contents of the buffer. The number recognizer outputs a recognized telephone number for display on a terminal and/or storage in memory for future use.