Speech-Based Human-Machine Interaction Research Articles

Speech Enhancement Based on NMF Under Electric Vehicle Noise Condition

Speech-based human–machine interaction (HMI) is essential to electronic navigation, autonomous cars, and intelligent vehicles. The noises generated by the mechanical motion or electric power equipment degrade speech quality and result in HMI failing to work effectively. However, there is relatively little literature available on speech enhancement under electric vehicle noise condition. This paper presents a speech enhancement method based on improved nonnegative matrix factorization (ImNMF). Unlike the traditional nonnegative matrix factorization (NMF) trains its speech dictionary using speech recorded in advance which inevitably contains a little noise component, ImNMF generates the speech dictionary using the spectra of pitch and their harmonics via mathematical model. This purpose is to guarantee the purity of speech dictionary. In addition, in order to alleviate the loss of the information of the noise sample, ImNMF constructs noise dictionary by a combination of the gain adjusted spectrum frames of the noise samples separated online. Compared with traditional NMF, the ImNMF noise atoms are relatively larger. Thus, the representation of speech signal mixed with noise atoms is greatly reduced. Therefore, ImNMF can reduce distortion of reconstructed speech while enhancing the recovered speech quality. Speech enhancement and speaker verification experiments on NUST603 and TIMIT data showed that the proposed ImNMF can effectively enhance speech signal in the noise environment of electric vehicles and further can reduce the equal error rate of the speaker verification system.

User-awareness and adaptation in conversational agents

This paper considers the research question of developing user-aware and adaptive conversational agents. The conversational agent is a system which is user-aware to the extent that it recognizes the user identity and his/her emotional states that are relevant in a given interaction domain. The conversational agent is user-adaptive to the extent that it dynamically adapts its dialogue behavior according to the user and his/her emotional state. The paper summarizes some aspects of our previous work and presents work-in-progress in the field of speech-based human-machine interaction. It focuses particularly on the development of speech recognition modules in cooperation with both modules for emotion recognition and speaker recognition, as well as the dialogue management module. Finally, it proposes an architecture of a conversational agent that integrates those modules and improves each of them based on some kind of synergies among themselves.

Kinect Development Kit: A Toolkit for Gesture- and Speech-Based Human-Machine Interaction [Best of the Web

Kinect is a device for human?machine interaction, which adds two more input modalities to the palette of the user interface designer: gestures and speech. Kinect is transforming how people interact with computers, kiosks, and other motion-controlled devices from fun applications like playing a virtual violin, to applications in health care and physical therapy, retail, education, and training.

Improving Language Models in Speech-Based Human-Machine Interaction

This work focuses on speech-based human-machine interaction. Specifically, a Spoken Dialogue System (SDS) that could be integrated into a robot is considered. Since Automatic Speech Recognition is one of the most sensitive tasks that must be confronted in such systems, the goal of this work is to improve the results obtained by this specific module. In order to do so, a hierarchical Language Model (LM) is considered. Different series of experiments were carried out using the proposed models over different corpora and tasks. The results obtained show that these models provide greater accuracy in the recognition task. Additionally, the influence of the Acoustic Modelling (AM) in the improvement percentage of the Language Models has also been explored. Finally the use of hierarchical Language Models in a language understanding task has been successfully employed, as shown in an additional series of experiments.

Focus tree: modeling attentional information in task-oriented human-machine interaction

This paper introduces a new model of attentional state in task-oriented human-machine interaction. It integrates three lines of research: (i) neurocognitive understanding of the focus of attention in working memory, (ii) the notion of attention related to the theory of discourse structure in the field of computational linguistics, and (iii) investigation of a corpus that comprises recordings of spontaneous speech-based human-machine interaction. The underlying idea was to make a computationally appropriate representation of attentional information that imitates the function of a focus of attention in human perception. The introduced model addresses both the research questions of storage and processing of attentional information. Finally, the paper illustrates the model for concrete interaction domains, and discusses its implementation within a prototype spoken dialogue system.

Inducing Genuine Emotions in Simulated Speech-Based Human-Machine Interaction: The NIMITEK Corpus

Emotional corpora provide an important empirical foundation for investigation when researchers aim at implementing emotion-aware spoken dialog systems. One of the fundamental research questions is how to acquire an appropriate, realistic emotion corpus. The primary aim of this paper is to address the methodological desiderata in producing emotion corpora in human-machine interaction (HMI). It proposes a substantial refinement of the Wizard-of-Oz (WOZ) technique in order that a scenario designed to elicit affected speech in HMI could result in realistic and useful data. In addition, the paper reports about the NIMITEK corpus of affected behavior in HMI produced during a refined WOZ simulation. The evaluation of the corpus with respect to the perception of its emotional content demonstrated that the corpus contains recordings of emotions that were overtly signaled. The range of emotional reactions is indicative of the kind of emotional reactions than can be expected to occur in the interaction with the sort of spoken dialog systems considered in this study. Since the subjects were not restricted by given predetermined linguistic constraints on the language to use, their utterances are indicative of the way in which nontrained, nontechnical users probably like to converse with conversational agents as well.

PRESENCE: A Human-Inspired Architecture for Speech-Based Human-Machine Interaction

Recent years have seen steady improvements in the quality and performance of speech-based human-machine interaction driven by a significant convergence in the methods and techniques employed. However, the quantity of training data required to improve state-of-the-art systems seems to be growing exponentially and performance appears to be asymptotic to a level that may be inadequate for many real-world applications. This suggests that there may be a fundamental flaw in the underlying architecture of contemporary systems, as well as a failure to capitalize on the combinatorial properties of human spoken language. This paper addresses these issues and presents a novel architecture for speech-based human-machine interaction inspired by recent findings in the neurobiology of living systems. Called PRESENCE-PREdictive SENsorimotor Control and Emulation-this new architecture blurs the distinction between the core components of a traditional spoken language dialogue system and instead focuses on a recursive hierarchical feedback control structure. Cooperative and communicative behavior emerges as a by-product of an architecture that is founded on a model of interaction in which the system has in mind the needs and intentions of a user and a user has in mind the needs and intentions of the system.

Guest Editors' Introduction: Special Section on Emergent Systems, Algorithms and Architectures for Speech-Based Human-Machine Interaction

The nine papers selected for this special section focus on emergent systems. algorithms, and architectures for speech-based human-machine interaction. The papers address a considerable number of new ideas and applications in the field of speech-based human-machine interaction.

Call for papers - Special issue on Emergent Systems, Algorithms, and Architectures for Speech-Based Human-Machine Interaction

Call for papers - Special issue on Emergent Systems, Algorithms, and Architectures for Speech-Based Human-Machine Interaction

Call for papers - Special issue on Emergent Systems, Algorithms, and Architectures for Speech-Based Human-Machine Interaction

Call for papers - Special issue on Emergent Systems, Algorithms, and Architectures for Speech-Based Human-Machine Interaction

Call for Papers for Special Issue on Emergent Systems, Algorithms, and Architectures for Speech-Based Human-Machine Interaction

Call for Papers for Special Issue on Emergent Systems, Algorithms, and Architectures for Speech-Based Human-Machine Interaction