Voice Input Research Articles

Information retrieval interfaces in virtual reality-A scoping review focused on current generation technology.

The Information Retrieval user experience has remained largely unchanged since its inception for computers and mobile devices alike. However, recent developments in Virtual Reality hardware (pioneered by Oculus Rift in 2013) could introduce a new environment for Information Retrieval. This paper reports the results of a Scoping Literature Review (PRISMA-ScR) by rigorously examining the entire body of relevant literature with reproducible methods. The following research questions are answered: "What prototypes and concepts of Virtual Reality Information Retrieval systems with current generation hardware exist?", "How are user interaction and especially user input realised in these systems?", "What Retrieval features are used in these systems?", "How are search results displayed in these systems?" and "Can these VR IR systems compare to traditional (non-VR) IR systems?". After querying Google Scholar, Scopus and Web of Science, 1042 documents were reviewed in depth. Key features and attributes of the systems are summarised and discussed. Sketches of the user interfaces are included as well. The 30 documents that were relevant to the research questions include 16 distinct systems or theories. They discuss and utilise several user input technologies, ranging from controllers, voice input or hand tracking. Although conventional retrieval features are less common, systems enable retrieval of literature, 3D objects, images, books and texts and arrange them in a virtual space (e.g. as grids, arcs or maps). Finally, many of these systems were compared to conventional counterparts through user evaluation (n = 10). Most found user task times to be shorter or equal (n = 5, n = 3). In the seven papers that measured user performance (rate of correct solutions), three reported better performance (one equal). Notably, users always were more satisfied with the Virtual Reality systems compared to conventional ones. Possible limitations of these evaluations are demographic selection and the quality of baseline systems (control).

WITHDRAWN: Robokart for visually impaired people

Abstract This paper proposes a system that can be installed on a shopping cart that be used to assist visually impaired people. This method uses the concept of ROI, text localization and recognition by optical character recognition software and a webcam controlled by Raspberry Pi. Two micro controllers are used to program both the shopping cart movement and OCR. The user is required to give a voice input of the required product and the ROBOKART guides the user to that product.

Secure voice cryptography based on Diffie-Hellman algorithm

This article introduces a new technique for voice signals encryption & decryption to ameliorate the information security during transferring over unsecure network. The presented mechanism is based on a particular type of asymmetric key cryptography called Diffie-Hellman Algorithm. The main reason of using this algorithm is enabling users to encrypt and decrypt their messages via a shared session key that is known only for them. Firstly, the input voice signal in this scheme is enciphered by utilizing the shared secret key at the dispatcher. Secondly, the scrambled voice signal is delivered to the destination through open telecommunication network. Eventually, the cipher text signal is deciphered at the receptor by exploiting the same key to retrieve the plaintext signal. The performance of the presented voice cryptosystem is assessed via different known ciphering/deciphering voice measures for different voice signals. The computed and visual results in addition to comparison outcomes confirm that the presented cryptosystem can fulfill good enciphering and deciphering results and it is capable of tolerating various cryptographic analyses.

Advanced Safe Home Systems using Face-Recognition with Unique Passcode Systems

In this paper, we present a model for advanced security measure for safe home security system using face-recognition system and a unique pass-code for each user. Both face as well as unique passcode must be verified for the user to gain access to the building. For face-recognition system, we use HAAR cascade object detection algorithm by using HAAR classifiers. For unique passcode, we use google api for voice input submission and verification.

Physical Activity Evaluation Using a Voice Recognition App: Development and Validation Study.

Historically, the evaluation of physical activity has involved a variety of methods such as the use of questionnaires, accelerometers, behavior records, and global positioning systems, each according to the purpose of the evaluation. The use of web-based physical activity evaluation systems has been proposed as an easy method for collecting physical activity data. Voice recognition technology not only eliminates the need for questionnaires during physical activity evaluation but also enables users to record their behavior without physically touching electronic devices. The use of a web-based voice recognition system might be an effective way to record physical activity and behavior. The purpose of this study was to develop a physical activity evaluation app to record behavior using voice recognition technology and to examine the app's validity by comparing data obtained using both the app and an accelerometer simultaneously. A total of 20 participants (14 men, 6 women; mean age 19.1 years, SD 0.9) wore a 3-axis accelerometer and inputted behavioral data into their smartphones for a period of 7 days. We developed a behavior-recording system with a voice recognition function using a voice recognition application programming interface. The exercise intensity was determined from the text data obtained by the voice recognition program. The measure of intensity was metabolic equivalents (METs). From the voice input data of the participants, 601 text-converted data could be confirmed, of which 471 (78.4%) could be automatically converted into behavioral words. In the time-matched analysis, the mean daily METs values measured by the app and the accelerometer were 1.64 (SD 0.20) and 1.63 (SD 0.20), respectively, between which there was no significant difference (P=.57). There was a significant correlation between the average METs obtained from the voice recognition app and the accelerometer in the time-matched analysis (r=0.830, P<.001). In the Bland-Altman plot for METs measured by the voice recognition app as compared with METs measured by accelerometer, the mean difference between the two methods was very small (0.02 METs), with 95% limits of agreement from -0.26 to 0.22 METs between the two methods. The average METs value measured by the voice recognition app was consistent with that measured by the 3-axis accelerometer and, thus, the data gathered by the two measurement methods showed a high correlation. The voice recognition method also demonstrated the ability of the system to measure the physical activity of a large number of people at the same time with less burden on the participants. Although there were still issues regarding the improvement of automatic text data classification technology and user input compliance, this research proposes a new method for evaluating physical activity using voice recognition technology.

Development of a framework for a collaborative and personalised voice assistant

Virtual assistant is an artificial intelligence (AI) system that understands natural voice commands and completes the task for the user. There is a need for seamless integration of high level general purpose voice assistants such as Google Assistant and Amazon Alexa under a single framework. In this paper, a novel collaborative personalised voice assistant with the ability to interface hardware components for an interactive environment is proposed. The proposed assistant takes the users' voice input as the query and processes it using a natural language processing (NLP) unit which generates intents from the converted text. Based on the type of intent, the NLP unit passes to one of the two services Google Assistant or Amazon Alexa. If the query is related to requesting information, the NLP passes to Google Assistant which gives an appropriate answer as the voice input and/or a query related to controlling hardware objects, will be pass on to Amazon Alexa. To demonstrate our collaborative voice assistant, a servo motor is used as a hardware object which controls the movement of the 5-inch liquid crystal display (LCD). Results prove that the proposed collaborative voice assistant brings together the strengths of Google Assistant and Amazon Alexa in a single framework.

Research on Augmented Reality Experience Design of Tujia Brocade Products Based on Emotion Cognition

Augmented reality technology is an extension of virtual reality. Voice input and output devices, image input and image display devices, touch-tap gesture action input and visual tracking devices are used in various computer systems to improve computer efficiency and human-computer interaction accuracy, which can improve the interactivity and user experience of augmented reality systems. Augmented reality technology superimposes the virtual world and the real world, and its final interactive operation has an enhanced effect on the senses of the participants. Augmented reality has been able to be used more maturely in many fields, and its use in the art field has become a new direction for designers to explore art and practice. This study explores the user’s emotional perception process of Tujia brocade products from the theoretical level, constructs an augmented reality experience model applied to Tujia brocade, and proposes a design process for the implementation of augmented reality.

Video Creation Using Facial Animation and Speech Synthesis

We present an end-to-end system for creating talking head video from text and a single image automatically. A two-dimensional clone of an individual made entirely of a still image of a person, that conveys the given text message to the target audience through replicated facial expressions. It can send any message that is delivered in a text to it with a voice that is cloned from a few seconds of their actual voice input. Our technique yields videos of natural facial movements including blinks and brow movements, as well as mouth movements that are in synchronization with the audio. The proposed method only uses a single base model to create a personalized video, avoiding the hassle of training it every time.

Detecting audio adversarial examples for protecting speech-to-text transcription neural networks

With the increasing use of deep learning techniques in real-world applications, their vulnerabilities have received significant attention from deep-learning researchers and practitioners. In particular, adversarial examples for deep neural networks and protection methods against them have been well-studied in recent years because they have serious vulnerabilities that threaten safety in the real-world. Audio adversarial examples, which are targeted attacks, are designed such that the deep neural network-based speech-to-text systems misunderstand input voice sound. In this study, we propose a new protection method against audio adversarial examples. The proposed protection method is based on a sandbox approach, where an input voice sound is checked in the system to determine if it is an audio adversarial example. To evaluate the proposed protection method, we used actual audio adversarial examples created on deep speech, which is a typical speech-to-text transcription neural network. The experimental results show that our protection method can detect audio adversarial examples with high accuracy.

Trade-offs in the design of multimodal interaction for older adults

ABSTRACT This paper presents key aspects that designers and Human–Computer Interaction practitioners might encounter when designing multimodal interaction for older adults, focusing on the trade-offs that might occur as part of the design process. The paper gathers literature on multimodal interaction and assistive technology, and describes a set of design challenges specific for older users. Building on these main design challenges, four trade-offs in the design of multimodal technology for this target group are presented and discussed. To highlight the relevance of the trade-offs in the design process of multimodal technology for older adults, two of the four reported trade-offs are illustrated with two user studies that investigate mid-air and speech-based interaction with a tablet device. The first study explores the design trade-offs related to redundant multimodal commands in older, middle-aged and younger adults, whereas the second one investigates the design choices related to the definition of a set of mid-air one-hand gestures and voice input commands for older adults. Further reflections highlight the design trade-offs that such considerations bring in the process, providing an overview of the design choices involved and of their potential consequences.

“We Didn’t Catch That!” Using Voice Text Input on a Mixed Reality Headset in Noisy Environments

The Microsoft HoloLens, a mixed reality head-mounted display (HMD), has been demonstrated in domains such as medicine, engineering, and manufacturing. In order to interact with the device, voice input may be required. Given this range of environments, it is necessary to understand the impact of noise on voice dictation speed and accuracy. In this study, we evaluated the dictation feature of the HoloLens through speed (WPM), accuracy (WER), perceived workload, and perceived usability at three different noise levels: 40 dB, 55 dB, and 70 dB. No differences were found across noise levels in speed (67-75 WPM), or perceived workload. Accuracy and perceived usability worsened in the 70 dB noise condition. Only 37.5% of participants were able to successfully dictate in the 70 dB condition. This study shows that if the HoloLens is to be accepted in environments with high noise levels, improvements to dictation need to be made.

Voice to Indian Sign Language Conversion for Hearing Impaired People

Sign language is a universal way of communication for challenged people with speaking and hearing limitations. Multiple mediums are accessible to translate or to acknowledge sign language and convert them to text. However, the text to signing conversion systems has been rarely developed; this is often thanks to the scarcity of any sign language dictionary. Our project aims at creating a system that consists of a module that initially transforms voice input to English text and which parses the sentence, then to which Indian sign language grammar rules are applied. This is done by eliminating stop words from the reordered sentence. Indian Sign Language (ISL) does not sustain the inflections of the word. Hence, stemming is applied to vary over the words to their root/ stem class. All words of the sentence are then checked against the labels in the dictionary containing videos representing each of the words. The present systems are limited to only straight conversion of words into ISL, whereas the proposed system is innovative, as our system aims to rework these sentences into ISL as per grammar in the real domain.

A multistudy investigation of drivers and passengers’ gesture and voice input preferences for in-vehicle interactions

We conduct an examination of the preferences of drivers and passengers alike for in-vehicle interactions with a multistudy approach consisting of (1) a targeted literature survey of applications and user interfaces designed to support interactions with in-vehicle controls and systems based on gesture and voice input; (2) a large-scale survey (N = 160 participants) to understand drivers and passengers’ preferences for driving and traveling by car; and (3) an end-user elicitation study (N = 40 drivers and passengers) to collect preferences for gesture and voice input commands for in-vehicle interaction. We analyze and discuss the gesture and voice commands proposed by our participants and describe their characteristics, such as production times for gesture input and the vocabulary size of voice commands.

Public opinion mining using natural language processing technique for improvisation towards smart city.

In this digital world integrating smart city concepts, there is a tremendous scope and need for e-governance applications. Now people analyze the opinion of others before purchasing any product, hotel booking, stepping onto restaurants etc. and the respective user share their experience as a feedback towards the service. But there is no e-governance platform to obtain public opinion grievances towards covid19, government new laws, policies etc. With the growing availability and emergence of opinion rich information’s, new opportunities and challenges might arise in developing a technology for mining the huge set of public messages, opinions and alert the respective departments to take necessary actions and also nearby ambulances if its related to covid-19. To overcome this pandemic situation a natural language processing based efficient e-governance platform is demandful to detect the corona positive patients and provide transparency on the covid count and also alert the respective health ministry and nearby ambulance based on the user voice inputs. To convert the public voice messages into text, we used Hidden Markov Models (HMMs). To identify respective government department responsible for the respective user voice input, we perform pre-processing, part of speech, unigram, bigram, trigram analysis and fuzzy logic (machine learning technique). After identifying the responsible department, we perform 2 methods, (1) Automatic alert e-mail and message to the government departmental officials and nearby ambulance or covid camp if the user input is related to covis19. (2) Ticketing system for public and government officials monitoring. For experimental results, we used Java based web and mobile application to execute the proposed methodology. Integration of HMM, Fuzzy logic provides promising results.

SAIF: A Correction-Detection Deep-Learning Architecture for Personal Assistants.

Intelligent agents that can interact with users using natural language are becoming increasingly common. Sometimes an intelligent agent may not correctly understand a user command or may not perform it properly. In such cases, the user might try a second time by giving the agent another, slightly different command. Giving an agent the ability to detect such user corrections might help it fix its own mistakes and avoid making them in the future. In this work, we consider the problem of automatically detecting user corrections using deep learning. We develop a multimodal architecture called SAIF, which detects such user corrections, taking as inputs the user’s voice commands as well as their transcripts. Voice inputs allow SAIF to take advantage of sound cues, such as tone, speed, and word emphasis. In addition to sound cues, our model uses transcripts to determine whether a command is a correction to the previous command. Our model also obtains internal input from the agent, indicating whether the previous command was executed successfully or not. Finally, we release a unique dataset in which users interacted with an intelligent agent assistant, by giving it commands. This dataset includes labels on pairs of consecutive commands, which indicate whether the latter command is in fact a correction of the former command. We show that SAIF outperforms current state-of-the-art methods on this dataset.

RHIZOME IN ENGLISH-LANGUAGE POSTMODERN TEXT FROM COGNITIVE AND SYNERGETIC PERSPECTIVES

The article deals with theoretical and methodological bases of the rhizome in the postmodern text with the focus on the achievements of cognitive linguistics and synergetics. Chaotic and non-linear as well as temporal and spatial shifts in the fictional reality of the postmodern text enabled to elicit a fractal sequence consisting of 22 embedded narratives in Michael Cunningham’s The Hours. From a cognitive linguistics perspective the rhizome model and fractal textual similarities in the conceptual network of mental spaces verified the synthesis of cognitive and synergetic principles of postmodern literary text analysis. This research claims that the rhizome as an ontological category of postmodern philosophy, on the one hand, participates in the conceptualization of fictional reality in the postmodern text, and on the other hand, it constitutes a fractal ensemble of sequential embranchments in the conceptual network of mental spaces. The technique of conceptual blending (Fauconnier, Turner, 2006) was used to give prominence to conceptual and semantic embranchments of FEAR concept in the conceptual network of mental spaces of the rhizome. The author comes to the conclusion that in the generic space the character’s narrative identity is defined as its organizing centre. In addition, the iteration process of embedded narratives resulted from minimally four input mental spaces as well as conceptual and semantic embranchments of FEAR, namely FEAR-1→HEADACHE (input 1), FEAR-2→VOICE (input 2), FEAR-3→MIRROR (input 3), FEAR-n (input n). Consequently, an ensemble of similarities in Cunningham’s The Hours represented by conceptual and semantic embranchments of FEAR in the conceptual network of mental spaces reveals the specific features of the rhizome from a cognitive linguistics perspective.

Secure Voice Input on Augmented Reality Headsets

Voice-based input is usually used as the primary input method for augmented reality (AR) headsets due to immersive AR experience and good recognition performance. However, recent researches show that attackers can inject inaudible voice commands to the devices that lack voice verification. Even if we secure voice input with voice verification techniques, attackers can record the victim's voice and replay it. To defend against voice-spoofing attacks, AR headsets should be able to determine whether the voice is from the person who is using the AR headsets. Existing voice-spoofing defense systems are designed for smartphone platforms and usually fail to work due to the special locations of microphones and loudspeakers on AR headsets. To address this challenge, in this paper, we propose a voice-spoofing defense system for AR headsets by leveraging both the internal body propagation and the air propagation of human voices. Experimental results show that our system can successfully accept normal users with average accuracy of 97% and defend against two basic types of attacks with average accuracy of at least 98%. More importantly, even if the attackers can fool our distance-based model by manipulating special voice signals, our combine model can still reject them with accuracy of 100%.

Demonstrating the voice-based exploration of large data sets with CiceroDB-zero

This demonstration enables participants to explore large data sets via voice interfaces. The focus of the demonstration is on methods generating concise speech descriptions of query results, specified by users via voice input. The technical novelty of the demonstrated system lies in the fact that processing overheads are mostly moved into a pre-processing phase, generating speeches for batches of queries defined via templates. Visitors can access the demo via smart speakers on-site or via their own smart phones. They will be able to customize the generated voice descriptions and to tune voice output methods.

Comparison of real-ear insertion gains in Japanese-speaking individuals wearing hearing aids with DSLv5 and NAL-NL2

ObjectiveWe used real-ear insertion gain (REIG), with the international speech test signal (ISTS), to evaluate the amplifying characteristics of hearing aids, set for patients who have been wearing such aids for a long time in a stable manner. We further compared this to the target values of the DSLv5 and NAL-NL2 methods. MethodsThe subjects were adults with moderate sensorineural hearing loss. We examined 40 ears in 25 individuals (15 people wearing hearing aids in both ears and ten people wearing aid in only one ear). Fit assessments were performed based on the speech performance-intensity functions and tolerance of environmental noise, and the ears studied were categorized as either ears with sufficient benefit or ears with insufficient benefit. Additionally, we evaluated the REIG for international speech test signals at 65-dB and 80-dB sound pressure level (SPL). We compared the REIG and target values for voice input at 65-dB and 80-dB SPL, calculated from the DSLv5 and NAL-NL2 methods. ResultsAmong the 40 ears, 34 received sufficient benefit and six received an insufficient benefit from hearing aids. The REIG for ISTS at 65-dB in the sufficient benefit ears, at frequencies of 1,000 Hz and 2,000 Hz, were similar to the target values of NAL-NL2 and DSLv5 but were significantly lower at 250 Hz, 500 Hz, and 4,000 Hz frequencies. The compression ratio of REIG for sufficient benefit ears was similar to that of DSLv5. The REIG for ISTS at 65-dB in the insufficient benefit ears was smaller than that in the sufficient benefit ears at frequencies of 2,000 Hz and 4,000 Hz. ConclusionThis study suggested that the target values of NAL-NL2 and DSLv5 are appropriate, even for Japanese-speaking individuals, at mid-pitch sounds. Although it is necessary to investigate the necessity for low-pitch and high-pitch gains further in the future, this study provides meaningful data regarding the amplifying characteristics in Japanese-speaking individuals who have been wearing hearing aids in a stable manner.

The Development of an Android-Based Dictionary for Blind

This study aims to develop an Android-based Indonesian dictionary for the blind. This study was developed using the Borg &amp;amp; Gall development model with some adjustments to the needs of this study. The stages of product development are (1) preliminary studies, (2) planning, (3) product development, (4) product validation, (5) first phase revision, (6) trials, (7) second phase revision, and (8) dissemination. The product developed is an android-based dictionary application with voice input and output. This developed product has been validated and tested on a small scale. Validation is carried out for material expert validators, media experts, and practitioners; while small-scale trials were conducted on 5 people with visual impairments. The results of product validation and trial show that the product developed is feasible to be implemented with an average percentage rating of 91.93%. The product developed provides benefits to the blind, which makes it easier to understand the meaning of words in the dictionary and increases the independence of the blind.