Use Of Speech Recognition Research Articles

AUTOMATED LECTURE NOTE-TAKING SYSTEM

Background. In the modern world, information technologies evolve rapidly, constantly altering our approaches to learning, work, and daily life. One significant aspect of this evolution is the automation of various processes, including education. Students and teachers are faced with large volumes of information that need to be processed, stored, and used in the educational process. Taking lecture notes is still an important task that requires a lot of time and effort, thus automating this process is both relevant and necessary. Automated lecture note-taking based on video and audio materials greatly facilitates the lives of students and teachers by providing quick access to structured information. The use of speech recognition and artificial intelligence technologies to create notes from lecture materials opens up new opportunities for effective learning. These systems can significantly save time, improve the quality and accuracy of notes, and ensure their accessibility to all participants in the educational process. These systems can not only create notes but also structure them by highlighting key points and providing easy access to information. This allows students to focus on understanding and comprehending the material rather than writing it down, thereby improving the quality of learning and knowledge acquisition. Objective. The purpose of the paper is to simplify the note-taking process and improve its quality by developing a system for automated lecture note-taking based on video and audio materials, ensuring the efficient and rapid creation of structured notes from lecture materials. Methods. Analysis of Literature and Contemporary Studies: Studying scientific articles, monographs, and dissertations related to the topic of automated lecture note-taking, speech recognition, and artificial intelligence. System Analysis: Defining system requirements, analysing possible approaches and tools for implementation. Experimental Method: Developing, implementing, and testing the system. Comparative Analysis: Evaluating the effectiveness of different speech recognition tools and AI models for creating notes. Modelling and Prototyping: Creating a system prototype, testing it, and improving it based on the obtained results. Results. During the implementation of the automated lecture note-taking system, an effective Telegram bot was created, which uses "whisper-1" and "gpt-4" models to provide high-quality speech recognition and the generation of structured notes from video and audio materials. Conclusions. The developed system of automated note-taking of lectures based on video and audio materials significantly simplifies the preparation of materials for students and teachers. Integration with Telegram and implementation of the system through a Telegram bot ensure cross-platform, accessibility and ease of use and at the same time provide an opportunity to avoid creating additional web or mobile applications for a wide range of users. The use of OpenAI's "whisper-1" model demonstrates high accuracy of speech recognition, which allowed improving the quality of transcriptions compared to other tools such as Vosk or FasterWhisper.

Assessing pronunciation using dictation tools

Abstract Language institutions need efficient and reliable placement tests to ensure students are placed in appropriate classes. This can be achieved by automating the scoring of pronunciation tests via the use of speech recognition, as its reliability has been shown to be comparable to that of human raters. However, this technology can be costly as it requires development and maintenance, placing it beyond the means of many institutions. This study investigates the feasibility of assessing English second language pronunciation in placement tests through the use of a free automatic speech recognition tool, Google Voice Typing (GVT). We compared human-rated and GVT-rated scores of 56 pronunciation placement tests. Our results indicate a strong correlation between scores for the final rating and for each criterion on the rubric used by human raters. We conclude that leveraging this free speech technology could increase the test usefulness of language placement tests.

Virtual Notes Assistant Using ML

The voice assistance takes the voice input through our microphone (Bluetooth and wired microphone).It converts our voice into computer understandable language gives the required solutions and answers which are asked by the user. Voice assistance makes people smart and gives instant and computed results. The main focus is on user based information through user communication. They are able to interpret human speech. An intelligent software program called a virtual notes assistant (VNA) employing machine learning (ML) offers individualized note-taking and task management solutions. To increase productivity and lessen cognitive burden, it makes use of speech recognition, natural language processing, and user behavior analysis.

Patient Satisfaction With Speech Recognition in the Exam Room: Exploratory Survey.

Medical speech recognition technology uses a microphone and computer software to transcribe the spoken word into text and is not typically used in outpatient clinical exam rooms. Patient perceptions regarding speech recognition in the exam room (SRIER) are therefore unknown. This study aims to characterize patient perceptions of SRIER by administering a survey to consecutive patients scheduled for acute, chronic, and wellness care in three outpatient clinic sites. We used a microphone and medical speech recognition software to complete the "assessment and plan" portion of the after-visit summary in the patient's presence, immediately printed the after-visit summary, and then administered a 4-question exploratory survey to 65 consecutive patients in internal medicine and pulmonary medicine clinics at an academic medical center and a community family practice clinic in 2021 to characterize patient perceptions of SRIER. All questions were completed by all participants. When compared to patients' recollection of usual care (visits with no microphone and an after-visit summary without an "assessment and plan"), 86% (n=56) of respondents agreed or strongly agreed that their provider addressed their concerns better, and 73% (n=48) agreed or strongly agreed that they understood their provider's advice better. A total of 99% (n=64) of respondents agreed or strongly agreed that a printed after-visit summary including the "assessment and plan" was helpful. By comparing the "agree" and "strongly agree" responses to the neutral responses, we found that patients felt that clinicians using SRIER addressed their concerns better (P<.001), they understood their clinician's advice better (P<.001), and receiving a paper summary was helpful (P<.001). Patients were likely to recommend a provider using a microphone based on the Net Promoter Score of 58. This survey suggests patients have a very positive perception of speech recognition use in the exam room.

Novel Speech Recognition Systems Applied to Forensics within Child Exploitation: Wav2vec2.0 vs. Whisper.

The growth in online child exploitation material is a significant challenge for European Law Enforcement Agencies (LEAs). One of the most important sources of such online information corresponds to audio material that needs to be analyzed to find evidence in a timely and practical manner. That is why LEAs require a next-generation AI-powered platform to process audio data from online sources. We propose the use of speech recognition and keyword spotting to transcribe audiovisual data and to detect the presence of keywords related to child abuse. The considered models are based on two of the most accurate neural-based architectures to date: Wav2vec2.0 and Whisper. The systems were tested under an extensive set of scenarios in different languages. Additionally, keeping in mind that obtaining data from LEAs are very sensitive, we explore the use of federated learning to provide more robust systems for the addressed application, while maintaining the privacy of the data from LEAs. The considered models achieved a word error rate between 11% and 25%, depending on the language. In addition, the systems are able to recognize a set of spotted words with true-positive rates between 82% and 98%, depending on the language. Finally, federated learning strategies show that they can maintain and even improve the performance of the systems when compared to centralized trained models. The proposed systems set the basis for an AI-powered platform for automatic analysis of audio in the context of forensic applications of child abuse. The use of federated learning is also promising for the addressed scenario, where data privacy is an important issue to be managed.

Fine Motor Therapy Using Avatar-based AI for Autistic Children (Virtual Reality)

People who have autism spectrum disorder (ASD) frequently encounter situations in which they must give answers but are unsure of howto do so, such as questions from strangers about routine daily activities. ASD is a spectrum disorder that is characterised by impairments in social interaction and repetitive patterns of behaviour. The potential of the experience as a learning tool was increased with the addition of virtual reality (VR) support, which increased the experience's sense of presence and realism. Research that aims to use technology to improve social and communication deficits in autistic children is currently in progress.Immersive virtual reality (VR) is a relatively new technology that may be an effective therapy tool for helping autistic youngsters learn a variety of skills. The interactive scenario-based VR system described in this research was created to help autistic kids communicate more effectively. The HTC Vive Pro serves as the virtual reality headset for the game, which is currently being created using the Unity game engine. In order to test this "REAL-TIME project" and confirm the impact of the immersive environmenton the social performance of autistic children, the system makes use of speech recognition to allow natural interaction and turn-taking. In this project, we create and model a virtual kind of fine motor therapy for kids.The kids will get to experience animmersive VR environment full with fun games, interactions, and educational opportunities.

Towards Speech Recognition and Training Utilization in the Nuclear Power Main Control Room

Speech, as the material shell of language, is the external form of language, and is the symbol system that most directly records people’s thinking activities. As a means of interaction, it has the characteristics of simplicity and speed. For a long time, due to insufficient recognition accuracy, it cannot be applied in industry. However, with the rapid development of big data and machine learning technology, the training effect of speech model has greatly increased, and the reliability of speech recognition has been greatly improved. The application of speech interaction in nuclear power plants with extremely high safety requirements brings feasibility. The purpose of this system is to establish an intelligent speech control system and its related speech model training system, and use Kaldi and Python scripts as model training scripts to train the corpus. And designed experiments to verify that after using speech recognition, it can effectively reduce the operation time by about 45%, and the overall task execution efficiency has been significantly improved. Therefore, the use of speech recognition can significantly reduce the operator’s task load and reduce the probability of human error.

Automatic Communication Error Detection Using Speech Recognition and Linguistic Analysis for Proactive Control of Loss of Separation

Losses of separation (LoS) are breaches of regulations that specify the minimum distance between aircraft in controlled airspace. Erroneous communications between air traffic controllers (ATCs) and pilots are leading contributors to LoS that result in elevated risk of fatal accidents. An air traffic control system that could identify communication errors promptly would, therefore, be advantageous. Establishing such a system requires a systematic characterization of communication errors to reveal how various communication arrangements and errors influence the development of LoS. Such know-how could guide the ATCs and pilots in identifying the parts of their communication processes and content that most influence the occurrence of LoS. Existing studies of LoS focus on simulation of aircraft operation processes with little quantitative analysis about how communication issues arise and result in elevated risks of LoS. This paper presents a method for supporting automatic communication error detection through integrated use of speech recognition, text analysis, and formal modeling of airport operational processes. The proposed method focuses on: identifying communication features to guide the detection of vulnerable communications; characterizing communication errors; and Bayesian Network modeling for predicting communication errors and LoS using the features derived from ATC–pilot communications. Major findings show that incorrect read-backs by pilots are highly correlated with a majority of LoS. Results indicate the proposed method could form a basis for automating communication error detection and preventing LoS. The integrated Automatic Speech Recognition and Natural Language Processing functions may be incorporated into existing aviation applications for real-time ATC–pilot communication monitoring and preventive LoS control.

Speech Recognition Implementation Using MFCC and DTW Algorithm for Home Automation

The use of speech recognition as part of home automation, especially for smart homes, is an exciting thing that is still being developed. That is because of human needs for comfort, convenience, quality of life, and better safety. Speech recognition built in this study is used as a device to control smart home devices by identifying the commands spoken by users, especially in a state of clean speech. The command used is a predetermined consecutive word. For the extraction of voice commands, the MFCC algorithm is used to match spoken words with templates using the Dynamic Time Warping (DTW) algorithm. DTW algorithm can find the difference between 2-time series that have different lengths of time. The results of the accuracy of this system by using these algorithms were successfully carried out by 86.67%, with an average time required to identify the commands of 5.28 seconds.

Effect of Speech Recognition on Problem Solving and Recall in Consumer Digital Health Tasks: Controlled Laboratory Experiment.

BackgroundRecent advances in natural language processing and artificial intelligence have led to widespread adoption of speech recognition technologies. In consumer health applications, speech recognition is usually applied to support interactions with conversational agents for data collection, decision support, and patient monitoring. However, little is known about the use of speech recognition in consumer health applications and few studies have evaluated the efficacy of conversational agents in the hands of consumers. In other consumer-facing tools, cognitive load has been observed to be an important factor affecting the use of speech recognition technologies in tasks involving problem solving and recall. Users find it more difficult to think and speak at the same time when compared to typing, pointing, and clicking. However, the effects of speech recognition on cognitive load when performing health tasks has not yet been explored.ObjectiveThe aim of this study was to evaluate the use of speech recognition for documentation in consumer digital health tasks involving problem solving and recall.MethodsFifty university staff and students were recruited to undertake four documentation tasks with a simulated conversational agent in a computer laboratory. The tasks varied in complexity determined by the amount of problem solving and recall required (simple and complex) and the input modality (speech recognition vs keyboard and mouse). Cognitive load, task completion time, error rate, and usability were measured.ResultsCompared to using a keyboard and mouse, speech recognition significantly increased the cognitive load for complex tasks (Z=–4.08, P<.001) and simple tasks (Z=–2.24, P=.03). Complex tasks took significantly longer to complete (Z=–2.52, P=.01) and speech recognition was found to be overall less usable than a keyboard and mouse (Z=–3.30, P=.001). However, there was no effect on errors.ConclusionsUse of a keyboard and mouse was preferable to speech recognition for complex tasks involving problem solving and recall. Further studies using a broader variety of consumer digital health tasks of varying complexity are needed to investigate the contexts in which use of speech recognition is most appropriate. The effects of cognitive load on task performance and its significance also need to be investigated.

Physician use of speech recognition versus typing in clinical documentation: A controlled observational study

ImportanceSpeech recognition (SR) is increasingly used directly by clinicians for electronic health record (EHR) documentation. Its usability and effect on quality and efficiency versus other documentation methods remain unclear. ObjectiveTo study usability and quality of documentation with SR versus typing. DesignIn this controlled observational study, each subject participated in two of five simulated outpatient scenarios. Sessions were recorded with Morae® usability software. Two notes were documented into the EHR per encounter (one dictated, one typed) in randomized order. Participants were interviewed about each method’s perceived advantages and disadvantages. Demographics and documentation habits were collected via survey. Data collection occurred between January 8 and February 8, 2019, and data analysis was conducted from February through September of 2019. SettingBrigham and Women’s Hospital, Boston, Massachusetts, USA. ParticipantsTen physicians who had used SR for at least six months. Main outcomes and measuresDocumentation time, word count, vocabulary size, number of errors, number of corrections and quality (clarity, completeness, concision, information sufficiency and prioritization). ResultsDictated notes were longer than typed notes (320.6 vs. 180.8 words; p = 0.004) with more unique words (170.9 vs. 120.4; p = 0.01). Documentation time was similar between methods, with dictated notes taking slightly less time to complete than typed notes. Typed notes had more uncorrected errors per note than dictated notes (2.9 vs. 1.5), although most were minor misspellings. Dictated notes had a higher mean quality score (7.7 vs. 6.6; p = 0.04), were more complete and included more sufficient information. Conclusions and relevanceParticipants felt that SR saves them time, increases their efficiency and allows them to quickly document more relevant details. Quality analysis supports the perception that SR allows for more detailed notes, but whether dictation is objectively faster than typing remains unclear, and participants described some scenarios where typing is still preferred. Dictation can be effective for creating comprehensive documentation, especially when physicians like and feel comfortable using SR. Research is needed to further improve integration of SR with EHR systems and assess its impact on clinical practice, workflows, provider and patient experience, and costs.

Improving the Translation Environment for Professional Translators

When using computer-aided translation systems in a typical, professional translation workflow, there are several stages at which there is room for improvement. The SCATE (Smart Computer-Aided Translation Environment) project investigated several of these aspects, both from a human-computer interaction point of view, as well as from a purely technological side. This paper describes the SCATE research with respect to improved fuzzy matching, parallel treebanks, the integration of translation memories with machine translation, quality estimation, terminology extraction from comparable texts, the use of speech recognition in the translation process, and human computer interaction and interface design for the professional translation environment. For each of these topics, we describe the experiments we performed and the conclusions drawn, providing an overview of the highlights of the entire SCATE project.

Speech recognition for clinical documentation from 1990 to 2018: a systematic review.

The study sought to review recent literature regarding use of speech recognition (SR) technology for clinical documentation and to understand the impact of SR on document accuracy, provider efficiency, institutional cost, and more. We searched 10 scientific and medical literature databases to find articles about clinician use of SR for documentation published between January 1, 1990, and October 15, 2018. We annotated included articles with their research topic(s), medical domain(s), and SR system(s) evaluated and analyzed the results. One hundred twenty-two articles were included. Forty-eight (39.3%) involved the radiology department exclusively and 10 (8.2%) involved emergency medicine; 10 (8.2%) mentioned multiple departments. Forty-eight (39.3%) articles studied productivity; 20 (16.4%) studied the effect of SR on documentation time, with mixed findings. Decreased turnaround time was reported in all 19 (15.6%) studies in which it was evaluated. Twenty-nine (23.8%) studies conducted error analyses, though various evaluation metrics were used. Reported percentage of documents with errors ranged from 4.8% to 71%; reported word error rates ranged from 7.4% to 38.7%. Seven (5.7%) studies assessed documentation-associated costs; 5 reported decreases and 2 reported increases. Many studies (44.3%) used products by Nuance Communications. Other vendors included IBM (9.0%) and Philips (6.6%); 7 (5.7%) used self-developed systems. Despite widespread use of SR for clinical documentation, research on this topic remains largely heterogeneous, often using different evaluation metrics with mixed findings. Further, that SR-assisted documentation has become increasingly common in clinical settings beyond radiology warrants further investigation of its use and effectiveness in these settings.

Acoustic domain classification and recognition through ensemble based multilevel classification

To make the best use of speech recognition, it is imperative that it can recognize not just speech or speaker, but also the domain of communication. This paper proposes an approach for recognition of the acoustic domain using ensemble-based 3-level architecture instead of a single classifier for training and testing. It is estimated the predictions of various classifiers and then selects a set of three classifiers such that, any of the three classifiers must contain the target predictions and finally, these predictions are used to train another random forest classifier. It yields the final classification results of test data set. Experimental results indicate that the proposed method has consistent performance even if data size is increased with acceptable accuracy i.e. 76.36%.

Speech Recognition for Medical Dictation: Overview in Quebec and Systematic Review.

Speech recognition is increasingly used in medical reporting. The aim of this article is to identify in the literature the strengths and weaknesses of this technology, as well as barriers to and facilitators of its implementation. A systematic review of systematic reviews was performed using PubMed, Scopus, the Cochrane Library and the Center for Reviews and Dissemination through August 2017. The gray literature has also been consulted. The quality of systematic reviews has been assessed with the AMSTAR checklist. The main inclusion criterion was use of speech recognition for medical reporting (front-end or back-end). A survey has also been conducted in Quebec, Canada, to identify the dissemination of this technology in this province, as well as the factors leading to the success or failure of its implementation. Five systematic reviews were identified. These reviews indicated a high level of heterogeneity across studies. The quality of the studies reported was generally poor. Speech recognition is not as accurate as human transcription, but it can dramatically reduce turnaround times for reporting. In front-end use, medical doctors need to spend more time on dictation and correction than required with human transcription. With speech recognition, major errors occur up to three times more frequently. In back-end use, a potential increase in productivity of transcriptionists was noted. In conclusion, speech recognition offers several advantages for medical reporting. However, these advantages are countered by an increased burden on medical doctors and by risks of additional errors in medical reports. It is also hard to identify for which medical specialties and which clinical activities the use of speech recognition will be the most beneficial.

PP109 Use Of Speech Recognition In Medical Reports: A Systematic Review

Introduction:Speech recognition is increasingly used in medical reporting. The aim of this article is to identify in the literature the advantages and weaknesses of this technology, as well as barriers and facilitators to its implementation.Methods:A systematic review of systematic reviews has been conducted in PubMed, Scopus, Cochrane Library and Center for Reviews and Dissemination up to August 2017. The grey literature has also been consulted. The quality of systematic reviews has been assessed with the AMSTAR checklist. Inclusion criteria were to use speech recognition for medical reporting (front or back-end). A Survey has also been conducted in Quebec, Canada, to identify the dissemination of this technology in this province, as well as the factors of success or failure in its implementation.Results:Five systematic reviews were identified. These reviews indicated a high level of heterogeneity across studies. The quality of the studies reported was generally poor. Speech recognition is not as accurate as human transcription but can dramatically reduce the turnaround times for reporting. In front-end use, medical doctors need to spend more time for dictation and correction than with human transcription. With speech recognition, major errors can be up to three times more frequent. In back-end use, a potential increase in the productivity of transcriptionist is noted.Conclusions:Speech recognition offers some advantages for medical reporting, the main one being a reduction in turnaround times. However, these advantages are challenged by an increased burden for medical doctor and risks of additional errors in medical reports. It is also hard to identify for which medical specialties and which clinical activities the use of speech recognition will be the most beneficial.

An unsupervised deep domain adaptation approach for robust speech recognition

This paper addresses the robust speech recognition problem as a domain adaptation task. Specifically, we introduce an unsupervised deep domain adaptation (DDA) approach to acoustic modeling in order to eliminate the training–testing mismatch that is common in real-world use of speech recognition. Under a multi-task learning framework, the approach jointly learns two discriminative classifiers using one deep neural network (DNN). As the main task, a label predictor predicts phoneme labels and is used during training and at test time. As the second task, a domain classifier discriminates between the source and the target domains during training. The network is optimized by minimizing the loss of the label classifier and to maximize the loss of the domain classifier at the same time. The proposed approach is easy to implement by modifying a common feed-forward network. Moreover, this unsupervised approach only needs labeled training data from the source domain and some unlabeled raw data of the new domain. Speech recognition experiments on noise/channel distortion and domain shift confirm the effectiveness of the proposed approach. For instance, on the Aurora-4 corpus, compared with the acoustic model trained only using clean data, the DDA approach achieves relative 37.8% word error rate (WER) reduction.

Naming practice app on tablets and communication robots for people with aphasia

Aphasia is a language disorder that negatively affects a person's ability to understand and generate language. Although it has been said that patients in the chronic phase are not likely to recover their ability, a number of recent studies report that highly frequent and continuous training enables recovery. Therefore, we have started a project that aims to improve the environment in which people with aphasia can continue language learning at home. In this presentation, we demonstrate naming practice apps, using speech recognition technology for people with aphasia. They can check whether their utterances are accurate, through the use of speech recognition and they can listen to the correct pronunciation of words through the use of speech synthesis. Field trials using the app for tablets have been conducted since Dec. 2014, and for the communication robot, Pepper, since Dec. 2015. Although the average of speech recognition accuracy was 61 % (false acceptance: 2%, false rejection: 37%), the participants accepted the poor robustness of the speech recognition, and tried to speak clearly. Some participants felt as if the robot was somewhat similar to a child, and others understood its inflexibility, and so in consequence they both accepted its low speech recognition ability.

From Translation Studies and audiovisual translation to media accessibility

Abstract Recent developments in Translation Studies and translation practice have not only led to a profusion of approaches, but also to the development of new text forms and translation modes. Media Accessibility, particularly audio description (AD) and subtitling for the deaf and hard-of-hearing (SDH), is an example of such a ‘new’ mode. SDH has been evolving quickly in recent decades and new developments such as interlingual SDH and live subtitling with speech recognition bring it closer to established forms of translation and interpreting. On the one hand, interlingual SDH reintroduces Jakobson’s (1959) ‘translation proper’ while the use of speech recognition has led to the creation of a hybrid form that has affinities with both subtitling and interpreting. Audio description, for its part, cannot even be fitted into Jakobson’s ‘intersemiotic translation’ model since it involves translation from images into words. Research into AD is especially interesting since it rallies methods from adjacent disciplines, much in the same way that Holmes ([1972] 1988) described TS when it was a fledgling discipline. In 2008, Braun set out a research agenda for AD and the wealth of topics and research approaches dealt with in her article illustrate the immense complexity of this field and the work still to be done. Although AD and SDH research have developed at different paces and are concerned with different topics, converging trends do appear. Particularly the role of technology and the concept of multimodality seem to be key issues. This article aims to give an overview of current research trends in both these areas. It illustrates the possibilities of technology-driven research – particularly popular in SDH and live-subtitling research – while at the same time underlining the value of individual, human-driven approaches, which are still the main ‘modus operandi’ in the younger discipline of AD where much basic research is still required.

HyperCode: Voice aided programming

Existing input methods for inserting text into a digital device are slow and create a steep contrast of information exchange difference between a human and a computer. While programmers are fast at getting information from the computer via the screen, it is a slow process of inserting information via mouse and keyboard. Speech Recognition Software enables inserting continuous text into the computer for increased speed. In this paper, we will present a framework, which makes use of Speech Recognition as another input method for enabling programmers to write code faster and if desired (for people with disabilities) remove reliance on keyboard usage.