Voice Input Research Articles

Voice Input and OCR (Optical Character Reader) Use in Emergency Medicine

Voice Input and OCR (Optical Character Reader) Use in Emergency Medicine

Robust Vocal Quality Feature Embeddings for Dysphonic Voice Detection.

Approximately 1.2% of the world's population has impaired voice production. As a result, automatic dysphonic voice detection has attracted considerable academic and clinical interest. However, existing methods for automated voice assessment often fail to generalize outside the training conditions or to other related applications. In this paper, we propose a deep learning framework for generating acoustic feature embeddings sensitive to vocal quality and robust across different corpora. A contrastive loss is combined with a classification loss to train our deep learning model jointly. Data warping methods are used on input voice samples to improve the robustness of our method. Empirical results demonstrate that our method not only achieves high in-corpus and cross-corpus classification accuracy but also generates good embeddings sensitive to voice quality and robust across different corpora. We also compare our results against three baseline methods on clean and three variations of deteriorated in-corpus and cross-corpus datasets and demonstrate that the proposed model consistently outperforms the baseline methods.

Privacy‐preserving hands‐free voice authentication leveraging edge technology

AbstractAlthough speech recognition technology has improved significantly over the past few years, the absence of reliable voice authentication methods has negatively affected the Internet of Things (IoT). Voice‐activated devices, by design, rely on personal voice biometrics for access‐level security, and insecure voice authentication techniques limit the development of voice interfaces that can deal with multiple users with different privileges. Furthermore, public concerns surrounding biometric systems have been reported. Outsourcing biometric data are liable to various privacy breaches, increasing vulnerability to cyberattacks from both criminal organizations and government agencies. In this article, we address the security and privacy challenges of voice authentication for IoT by presenting an edge‐based, hands‐free, single‐factor authentication scheme that consists of three security‐based features: random phrase verification, anti‐spoof verification, and text‐independent speaker recognition. The use of edge technology allows the integration of this authentication scheme with resource contained IoT devices such as voice assistants. Additionally, privacy is ensured by a novel security protocol that uses chaffing and encryption techniques to protect the users' biometric data stored locally on the edge device. Finally, the scheme does not compromise the user experience as it requires only two short voice inputs.

MULTI LANGUAGE VIRTUAL VOICE ASSISTANT

ABSTRACT— In this modern era, day to day life became smarter and interlinked with technology. We already know some voice assistants like Google Siri ,Alexa. etc. Here, we have proposed our voice assistance system. It can act as a basic medical prescriber, daily schedule reminder, note writer, calculator and a search tool and it can be easier to send emails without typing any word, searching on Google without opening the browser, and performing many other daily tasks like playing music, opening your favorite IDE with the help of a single voice command. In the current scenario, advancement in technologies is such that they can perform any task with same effectiveness or can say more effectively than us. By making this project, we realized that the concept of AI in every field is decreasing human effort and saving time. This project works on voice input and gives output through voice and displays the text on the screen. The main agenda of our voice assistant is to make people smart and give instant and computed results. The voice assistant takes the voice input through our microphone (Bluetooth and wired microphone) and it converts our voice into computer understandable language and gives the required solutions and answers which are asked by the user. This assistant connects with the world wide web to provide results that the user has questioned. Natural Language Processing algorithm helps computer machines to engage in communication using natural human language in many forms. Keywords—Speech Recognition, API, Artificial Intelligence, Desktop Assistant, Python, Text to Speech, Virtual Assistant, Voice Recognition

Audio to Sign Language Converter

Abstract: The hearing and speech disabled people have a communication problem with other people. It is hard for such individuals to express themselves since everyone is not familiar with the sign language. The aim of this paper is to design a system which is helpful for the people with hearing / speech disabilities and convert a voice in Indian sign language (ISL). The task of learning a sign language can be cumbersome for people so this paper proposes a solution to this problem using speech recognition and image processing. The Sign languages have developed a means of easy communication primarily for the deaf and hard of hearing people. In this work we propose a real time system that recognizes the voice input through Pyaudio, SPHINX and Google speech recognition API and converts it into text, followed by sign language output of text which is displayed on the screen of the machine in the form of series of images or motioned video by the help of various python libraries.

Implementation of Echo Cancellation Ana Noise Reduction System

Abstract: The advanced dispatches world is upset talking further naturally by using hands free this help the mortal being to talk more confidently without holding any of the bias similar as microphones or telephones. This paper describes an indispensable system of estimating signals corrupted by cumulative noise or hindrance. Aural echo cancellation problem was bandied out of different noise cancellation ways by concerning different parameters with their relative results. In this project we used voice signal for the input process then we can add some echo and remove noise to the voice signal. After that we can go with different applications for the voice signal. Here we used filter steps to remove the noise from the input voice signal. Here we used filter steps to remove the noise from the input voice signal.

Mark-Virtual Friend using Python

In the present era life has become smarter and more advanced. We are already familiar with some voice services like Google assistant, Siri and Alexa etc. Now in our voice support system, it can work like automatic chrome, opens youtube from a web browser, and sign into the gmail account and helps in locking the window. Mark works by entering voice and rendering voice output and displaying text on the screen. Our project's main agenda is to make work easier by delivering faster results with a computer. It compares voice input with our microphones and processes the given commands and provides necessary solutions and answers that users ask the microphone and processes the given commands and provides necessary solutions and answers that users ask. Speech Recognition translates inputted voice into text enables communication with the computer.

Sound Activity Monitor Circuit for Low Power Consumption of Always-On Microphone Applications

A novel sound activity monitor (SAM) circuit for low power consumption of always-on microphone applications is presented. To reduce average power consumption, the ultra-low-power SAM is essential and operates a readout integrated circuit (ROIC) in low power mode with silent input or in normal power mode with voice input. A novel SAM with an architecture that does not include an envelope detector is proposed to achieve low power consumption. A new architecture is also proposed to improve MEMS sensitivity by connecting the SAM input to the source follower (SF) output instead of connecting the SAM input to the MEMS port already connected to the SF. In addition, in order to prevent inefficient frequent operation mode conversion, a feature of delaying the transition to the low-power mode after the sound is silenced is implemented. The proposed architecture is designed and verified based on the standard 0.18 µm CMOS process. The SAM, which consists of two-stage amplifiers (OA, AMP2), comparators, and a logic circuit, consumes a 1 µA current. The analog path consisting of SF, OA, and AMP2 in low power mode has a maximum amplification gain of 63 dB and a noise of 72 nVrms/√Hz at 1 kHz.

Artificial Intelligence based Multi-sensor COVID-19 Screening Framework

Many countries are struggling for COVID-19 screening resources which arises the need for automatic and low-cost diagnosis systems which can help to diagnose and a large number of tests can be conducted rapidly. Instead of relying on one single method, artificial intelligence and multiple sensors based approaches can be used to decide the prediction of the health condition of the patient. Temperature, oxygen saturation level, chest X-ray and cough sound can be analyzed for the rapid screening. The multi-sensor approach is more reliable and a person can be analyzed in multiple feature dimensions. Deep learning models can be trained with multiple chest x-ray images belonging to different categories to different health conditions i.e. healthy, COVID-19 positive, pneumonia, tuberculosis, etc. The deep learning model will extract the features from the input images and based on that test images will be classified into different categories. Similarly, cough sound and short talk can be trained on a convolutional neural network and after proper training, input voice samples can be differentiated into different categories. Artificial based approaches can help to develop a system to work efficiently at a low cost.

DeepDetection: Privacy-Enhanced Deep Voice Detection and User Authentication for Preventing Voice Phishing

The deep voice detection technology currently being researched causes personal information leakage because the input voice data are stored in the detection server. To overcome this problem, in this paper, we propose a novel system (i.e., DeepDetection) that can detect deep voices and authenticate users without exposing voice data to the server. Voice phishing prevention is achieved in two-way approaches by performing primary verification through deep voice detection and secondary verification of whether the sender is the correct sender through user authentication. Since voice preprocessing is performed on the user local device, voice data are not stored on the detection server. Thus, we can overcome the security vulnerabilities of the existing detection research. We used ASVspoof 2019 and achieved an F1-score of 100% in deep voice detection and an F1 score of 99.05% in user authentication. Additionally, the average EER for user authentication achieved was 0.15. Therefore, this work can be effectively used to prevent deep voice-based phishing.

Augmented reality assisted assembly: An action regulation theory perspective on performance and user experience

The growing demand for personalized products is changing the manufacturing industry and requires flexible assembly instructions. In this context, augmented reality (AR)-based digital instructions represent a potential solution. However, empirical evidence of their effectiveness in terms of performance, user experience, and work experience remains fragmented and mixed. Moreover, theoretical arguments related to their effects are completely lacking. In this paper, we present the results of two studies conducted in a pilot factory that investigated the differences between a screen-based, a static AR-based, and an animated AR-based application. We found that task completion time was best in the animated AR condition and hedonic user experience was higher in the AR conditions than in the screen-based one. In addition, we discovered a loss of agency in the animated AR condition, represented through hesitant behavior regarding voice input. Based on the interview data, it became clear that users saw themselves in a passive role blindly executing. They transferred the authority originally assigned to them to the technology. As a result, users remained passive even though their action was required.Relevance to industry (25–50 words): When developing AR assistance for industrial work, the inherent design challenge is to develop the technology in a way that supports mental effort sufficiently to enhance performance while keeping the user in an active state. We formulate design implications as well as a theoretical frame to meet this challenge.

The Health ChatBots in Telemedicine: Intelligent Dialog System for Remote Support

The objective of the Health Bot is to analyze and elaborate on the implementation of an intelligent system that can support telemedicine services using natural language processing (NLP) technology. Health Bot is a robust, modularized, and user-friendly platform that aims to improve the patient’s interactions with the health care system. Using NLP and speech recognition algorithms, the platform can analyze and classify free text and voice input data to symptoms. The classified data are used in the machine learning (ML) training process of the artificial intelligence (AI) models in order to predict the probability of the patient suffering from a specific disease and get alerted in case of disorders. The Health Bot is acting as a medical virtual agent in order to collect any information needed in the sense of a medical interview and provide medical assessments, close appointments with the doctors, and monitor/record the patient’s health condition. In this paper, two models have been trained for the Covid-19 and heart disease cases with 98.3% and 82% accuracy, respectively.

An Implementation of Voice Assistant for Hospitality

Voice user interface has gained popularity in the recent years. A chatbot is a machine with the ability to answer automatically through a conversational interface. Instead of using mouse and keyboards as input and screen as output, a chatbot with extra voice user interface feature improve the system and enhance the user experience. A chatbot is considered as one of the most exceptional and promising expressions of human computer interaction. Voice-based chatbots or artificial intelligence (AI) devices transform humancomputer bidirectional interactions that allow users to navigate an interactive voice response (IVR) system with their voice generally using natural language. In this paper, we focus on voice based chatbots for mediating interactions between hotels and guests from both the hospitality technology providers’ and guests’ perspectives. A hotel web application with voice user interface was implemented which provides voice input/output interface to enhance the user experience. Speech recognition component was used to dictate the user voice input to text. Speech synthesis API was used for text to voice conversion. A closed domain question answering (cdQA) Natural Language Processing (NLP) solution was used for processingof query and return the best answer possible.

Design of an integrated learning approach to assist real-time deaf application using voice recognition system

Modern voice recognition is the process of predicting the human voice using emergent artificial intelligence techniques. It is extensively adopted in real-time applications for identity verification, helping deaf/dumb people, electronic voice eavesdropping, and hearing-impairment purposes. Predicting salient and discriminative process flow in the voice recognition process is the most challenging task in identifying the voice. Some existing research works find it complex to provide better prediction results. However, this work concentrates on modelling an efficient approach to predict the voice using deep learning approaches and it includes four essential phases: data acquisition phase, input voice pre-processing phase, word segmentation phase and classification. For experimentation purpose, an available online dataset was initially acquired and fed as an input to the proposed model. Subsequently, the provided input is pre-processed using an improved Mel-DCT filter for voice activity detection and to improve the classifier performance. The pre-processed image is then supplied into the word segmentation phase where the words from the input data are segmented using the Grab Cut segmentation. Finally, the classification was done with the integrated Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN). The integration is performed to handle the drawbacks of the single classifier model and to boost the prediction accuracy.

An Integrated Single Framework for Text, Image and Voice for Sentiment Mining of Social Media Posts

The wide spread pandemic COVID-19 has propelled the entire world to rely on social media interaction digitally. Social media is thus a platform to express numerous kinds of direct and indirect sentiments by human beings. Psychologically, a person tends to share his/her feelings in terms of sentiments more openly over the social media. These sentiments, when intense may polarize oneself to commit severe mis-deeds. Here arises the role of the researchers to perform a real time identification of sentiments and classify them so that a prospective mishap can be averted. In this work, an integrated framework is proposed that does an early recognition of sentiments over social media in the digital domain. Along with sentiment categorization, another module has been integrated to the framework to perform a post-predictive analysis of the same. The proposed integrated framework involves combination of two distinct mechanisms. First, the proposed work channelizes the input data in line with its characteristics text, image, and voice. The text input is directly fed to our proposed ‘Lexicon based LSTM with sentiment word mapping’ mechanism. From the input image, both text and semantics are extracted through two different blocks. One block converts image-to-text and redirects the output to the above proposed model. We proposed a new generative model (GM) to extract the semantics of the image and the second block utilizes our generative model and redirects the outcome straight to the final output buffer of the framework. The voice-to-text module has been used for transforming voice input data to text data which is redirected to our proposed Lexicon based LSTM for further processing. A comparison of the proposed work has been made with state-of-the-art techniques. Our results indicate that the overall rate of accuracy of this framework is superior to the existing methods.

Voice Recognition Security Reliability Analysis Using Deep Learning Convolutional Neural Network Algorithm

This study discusses the reliability analysis of voice recognition security using the deep learning convolutional neural network (CNN) algorithm. The CNN algorithm has learning advantages in that it is safer, faster, and more accurate. CNN also can solve user identification problems in large amounts of data. The measured voice input is ten types of user's voice with the number of iterations of 6000, 12000, and 15000 sound files. Furthermore, voice extraction features are performed to recognize conversations and retain information that is very much needed. After that, the voice file iteration data is trained to register the user's voice so that a trained model is obtained. These results measure performance (confusion matrix) to analyze the actual value compared to the predicted value in the CNN algorithm. The results obtained are that the best accuracy is obtained at 15000 sound file iterations, 96.87%, 12000 sound file iterations get 96.30%, and 6000 sound file iterations get 95.77%. CNN's performance data shows that 15000 iterations of voice files produce high accuracy. Voice recognition security helps provide high security and maintain the privacy of one's identity.

DriNet: Dynamic Backdoor Attack against Automatic Speech Recognization Models

Automatic speech recognition (ASR) is popular in our daily lives (e.g., via voice assistants or voice input). Once its security attributes are destroyed, it poses as a severe threat to a user’s life and ‘property safety’. Prior research has demonstrated that ASR systems are vulnerable to backdoor attacks. A model embedded with a backdoor behaves normally on clean samples yet misclassifies malicious samples that contain triggers. Existing backdoor attacks have mostly been conducted in the image domain. However, they can not be applied in the audio domain because of poor transferability. This paper proposes a dynamic backdoor attack method against ASR models, named DriNet. Significantly, we designed a dynamic trigger generation network to craft a variety of audio triggers. It is trained jointly with the discriminative model incorporated with an attack success rate on poisoned samples and accuracy on clean samples. We demonstrate that DriNet achieves an attack success rate of 86.4% when infecting only 0.5% of the training set without reducing its accuracy. DriNet can still achieve comparable attack performance to backdoor attacks using static triggers, further enjoying richer attack patterns. We further evaluated DriNet’s resistance to a current state-of-the-art defense mechanism. The anomaly index of DriNet is more than 37.4% smaller than that of BadNets method. The triggers generated by DriNet are hard reverse, keeping DriNet from the detectors.

Personal Voice Assistant

Abstract: In today’s modern world, our daily life has been more linked with technology. Over time the advancement in technology is unmatched. The project that we have built is a personal voice assistant. It works on voice input and displays the output through voice or text. We have used python language for the voice assistant as it provides various libraries and packages to make our project. The assistant can perform some basic functions like giving weather updates, providing time and news, opening Google, YouTube, Gmail etc., setting reminder and performing arithmetic calculations. The main aim behind the voice assistant is to reduce the usage of input devices like keyboard, mouse, etc. This will not only lessen the hardware cost of the devices but also the space taken by them. Voice assistants are efficient and consume less time and therefore they are now inbuilt in many devices that people use nowadays. Keywords: Speech Recognition, Personal Assistant, Voice Input, A.I., Python

Who Is Willing to Use Audio and Voice Inputs in Smartphone Surveys, and Why?

The ever-growing number of respondents completing web surveys via smartphones is paving the way for leveraging technological advances to improve respondents’ survey experience and, in turn, the quality of their answers. Smartphone surveys enable researchers to incorporate audio and voice features into web surveys, that is, having questions read aloud to respondents using pre-recorded audio files and collecting voice answers via the smartphone’s microphone. Moving from written to audio and voice communication channels might be associated with several benefits, such as humanizing the communication process between researchers and respondents. However, little is known about respondents’ willingness to undergo this change in communication channels. Replicating and extending earlier research, we examine the extent to which respondents are willing to use audio and voice channels in web surveys, the reasons for their (non)willingness, and respondent characteristics associated with (non)willingness. The results of a web survey conducted in a nonprobability online panel in Germany ( N = 2146) reveal that more than 50% of respondents would be willing to have the questions read aloud (audio channel) and about 40% would also be willing to give answers via voice input (voice channel). While respondents mostly name a general openness to new technologies for their willingness, they mostly name preference for written communication for their nonwillingness. Finally, audio and voice channels in smartphone surveys appeal primarily to frequent and competent smartphone users as well as younger and tech-savvy respondents.

GRaNN: feature selection with golden ratio-aided neural network for emotion, gender and speaker identification from voice signals

Compared to other features of the human body, voice is quite complex and dynamic, in a sense that a speech can be spoken in various languages with different accents and in different emotional states. Recognizing the gender, i.e. male or female from the voice of an individual, is by all accounts a minor errand for human beings. Similar goes for speaker identification if we are well accustomed with the speaker for a long time. Our ears function as the front end, accepting the sound signs which our cerebrum processes and settles on our disposition. Although being trivial for us, it becomes a challenging task to mimic for any computing device. Automatic gender, emotion and speaker identification systems have many applications in surveillance, multimedia technology, robotics and social media. In this paper, we propose a Golden Ratio-aided Neural Network (GRaNN) architecture for the said purposes. As deciding the number of units for each layer in deep NN is a challenging issue, we have done this using the concept of Golden Ratio. Prior to that, an optimal subset of features are selected from the feature vector extracted, common for all three tasks, from spectral images obtained from the input voice signals. We have used a wrapper-filter framework where minimum redundancy maximum relevance selected features are fed to Mayfly algorithm combined with adaptive beta hill climbing (AbetaHC) algorithm. Our model achieves accuracies of 99.306% and 95.68% for gender identification in RAVDESS and Voice Gender datasets, 95.27% for emotion identification in RAVDESS dataset and 67.172% for speaker identification in RAVDESS dataset. Performance comparison of this model with existing models on the publicly available datasets confirms its superiority over those models. Results also ensure that we have chosen the common feature set meticulously, which works equally well on three different pattern classification tasks. The proposed wrapper-filter framework reduces the feature dimension significantly, thereby lessening the storage requirement and training time. Finally, strategically selecting the number units in each layer in NN help increases the overall performance of all three pattern classification tasks.