Speech Transcription Research Articles

Modeling Topics in User Dialog for Interactive Tablet Media

In this paper, we present a set of crowdsourcing and data processing techniques for annotating, segmenting and analyzing spoken dialog data to track topics of discussion between multiple users. Specifically, our system records the dialog between the parent and child as they interact with a reading game on a tablet, crowdsources the audio data to obtain transcribed text, and models topics of discussion from speech transcription using ConceptNet, a freely available commonsense knowledge base. We present preliminary results evaluating our technique using dialog collected using an interactive reading game for children 3-5 years of age. We successfully demonstrate the ability to form discussion topics by grouping words with similar meaning. The presented approach is entirely domain independent and in future work can be applied to a broad range of interactive entertainment applications, such as mobile devices, tablets and games.

UNINTELLIGIBLE SPEECH: LISTENERS' AWARENESS TO INDONESIAN-ACCENTED SPEECH WITH PRONUNCIATION ERRORS

There have been widely exposed researches that analyze accented and intelligible speech of foreign language; yet, the study concerning listeners' awareness of the pronunciation errors is hardly found. Therefore, the current study aims to find out segmental features and the source of errors that have caused unintelligible speech of Indonesian-accented speech and describe the listeners' awareness of the errors. This descriptive qualitative research investigates listeners' transcripts of Indonesian-accented speeches through purposive sampling. The standard orthography transcripts are then transformed into phonemic transcripts. They are analyzed by error analysis based on phonological operation by Davenport and Hannahs. The result is further checked by the listeners to generate their awareness of the errors through interviews. There are consonants causing unintelligibility more than vowels. Furthermore, it is found that there are six pronunciation features affecting listeners' awareness of speakers' pronunciation errors: 1) aspiration, 2) spelling system, 3) blended phonemes, 4) absent phonemes, 5) different articulation, and 6) homophone in the research.

Feasibility of using an automated analysis of formulation effort in patients’ spoken seizure descriptions in the differential diagnosis of epileptic and nonepileptic seizures

ObjectiveThere are three common causes of Transient Loss of Consciousness (TLOC), syncope, epileptic and psychogenic nonepileptic seizures (PNES). Many individuals who have experienced TLOC initially receive an incorrect diagnosis and inappropriate treatment. Whereas syncope can be distinguished relatively easily with a small number of “yes”/”no” questions, the differentiation of the other two causes of TLOC is more challenging. Previous qualitative research based on the methodology of Conversation Analysis has demonstrated that the descriptions of epileptic seizures contain more formulation effort than accounts of PNES. This research investigates whether features likely to reflect the level of formulation effort can be automatically elicited from audio recordings and transcripts of speech and used to differentiate between epileptic and nonepileptic seizures. MethodVerbatim transcripts of conversations between patients and neurologists were manually produced from video and audio recordings of 45 interactions (21 epilepsy and 24 PNES). The subsection of each transcript containing the person's account of their first seizure was manually extracted for the analysis. Seven automatically detectable features were designed as markers of formulation effort. These features were used to train a Random Forest machine learning classifier. ResultThere were significantly more hesitations and repetitions in descriptions of epileptic than nonepileptic seizures. Using a nested leave-one-out cross validation approach, 71% of seizures were correctly classified by the Random Forest classifier. DiscussionThis pilot study provides proof of principle that linguistic features that have been automatically extracted from audio recordings and transcripts could be used to distinguish between epileptic seizures and PNES and thereby contribute to the differential diagnosis of TLOC. Future research should explore whether additional observations can be incorporated into a diagnostic stratification tool and compare the performance of these features when they are combined with additional information provided by patients and witnesses about seizure manifestations and medical history.

Leadership Sentiment and Price Fluctuations

The relationship between prices and leadership mood was examined using the example of crude oil prices and the sentiment of the CEOs of 5 leading oil production companies between 2014 and 2019. The crude oil market was chosen due to recent price fluctuations and upcoming challenges, and for its significant for the global economy. This study uses an empirical approach based on a mood analysis of the CEO's natural language. 160 speech transcripts were analyzed using a leading aspect-based sentiment analysis machine learning algorithm to obtain sentiment data. The relation between sentiment and oil price was tested using linear regression. The results of this study show in detail that the sentiment correlates positively in times of low prices and negatively in times of high prices. The average threshold price calculated using this method was 63 USD per barrel of West Texas Intermediate (WTI) crude oil in the observed period. This corresponds to analysts who estimated the ideal oil price for 2018 at USD 60 to 70. Finally, the restrictions and prospects are discussed. Findings of this study could aid investors decision making and advance the use of sentiment analysis in economic sciences.

Context-Based Quotation Recommendation

While composing a new document, anything from a news article to an email or essay, authors often utilize direct quotes from a variety of sources. Although an author may know what point they would like to make, selecting an appropriate quote for the specific context may be time-consuming and difficult. We therefore propose a novel context-aware quote recommendation system which utilizes the content an author has already written to generate a ranked list of quotable paragraphs and spans of tokens from a given source document. We approach quote recommendation as a variant of open-domain question answering and adapt the state-of-the-art BERT-based methods from open-QA to our task. We conduct experiments on a collection of speech transcripts and associated news articles, evaluating models' paragraph ranking and span prediction performances. Our experiments confirm the strong performance of BERT-based methods on this task, which outperform bag-of-words and neural ranking baselines by more than 30% relative across all ranking metrics. Qualitative analyses show the difficulty of the paragraph and span recommendation tasks and confirm the quotability of the best BERT model's predictions, even if they are not the true selected quotes from the original news articles.

Unconventional Labour: Environmental Justice and Working-class Ecology in the New South Wales Green Bans

The New South Wales union movement embraced the principles of heritage and conservationism in the 1970s through the imposing of “green bans” – a strategy wherein union members refused to work on construction projects that were a threat to the state’s natural or built environment. Led by radicals like Builders Labourers’ Federation leader Jack Mundey, the green bans were seen in several sectors as a departure from the traditional “Old Left” priorities of securing workers’ wages and conditions. Rather than a hard shift towards radicalism, this article proposes that the green bans were instead reflective of an already existing conservationist tradition in the New South Wales union movement. This reinterpretation is predicated on a content analysis of extant historical material such as contemporaneous news articles, personal memoirs, transcripts of political speeches and archival documents related to the policing of left-wing activism in the 1960s and 1970s. The results show that an existing tradition of engagement with a broad spectrum of social issues in the New South Wales union movement predates the emergence of the New Left, including the commitment to environmental justice principles that underpinned the green bans.

Automatic speech recognition in neurodegenerative disease

Automatic speech recognition (ASR) could potentially improve communication by providing transcriptions of speech in real time. ASR is particularly useful for people with progressive disorders that lead to reduced speech intelligibility or difficulties performing motor tasks. ASR services are usually trained on healthy speech and may not be optimized for impaired speech, creating a barrier for accessing augmented assistance devices. We tested the performance of three state-of-the-art ASR platforms on two groups of people with neurodegenerative disease and healthy controls. We further examined individual differences that may explain errors in ASR services within groups, such as age and sex. Speakers were recorded while reading a standard text. Speech was elicited from individuals with multiple sclerosis, Friedreich’s ataxia, and healthy controls. Recordings were manually transcribed and compared to ASR transcriptions using Amazon Web Services, Google Cloud, and IBM Watson. Accuracy was measured as the proportion of words that were correctly classified. ASR accuracy was higher for controls than clinical groups, and higher for multiple sclerosis compared to Friedreich’s ataxia for all ASR services. Amazon Web Services and Google Cloud yielded higher accuracy than IBM Watson. ASR accuracy decreased with increased disease duration. Age and sex did not significantly affect ASR accuracy. ASR faces challenges for people with neuromuscular disorders. Until improvements are made in recognizing less intelligible speech, the true value of ASR for people requiring augmented assistance devices and alternative communication remains unrealized. We suggest potential methods to improve ASR for those with impaired speech.

Comparing Pre-trained and Feature-Based Models for Prediction of Alzheimer's Disease Based on Speech.

Introduction: Research related to the automatic detection of Alzheimer's disease (AD) is important, given the high prevalence of AD and the high cost of traditional diagnostic methods. Since AD significantly affects the content and acoustics of spontaneous speech, natural language processing, and machine learning provide promising techniques for reliably detecting AD. There has been a recent proliferation of classification models for AD, but these vary in the datasets used, model types and training and testing paradigms. In this study, we compare and contrast the performance of two common approaches for automatic AD detection from speech on the same, well-matched dataset, to determine the advantages of using domain knowledge vs. pre-trained transfer models.Methods: Audio recordings and corresponding manually-transcribed speech transcripts of a picture description task administered to 156 demographically matched older adults, 78 with Alzheimer's Disease (AD) and 78 cognitively intact (healthy) were classified using machine learning and natural language processing as “AD” or “non-AD.” The audio was acoustically-enhanced, and post-processed to improve quality of the speech recording as well control for variation caused by recording conditions. Two approaches were used for classification of these speech samples: (1) using domain knowledge: extracting an extensive set of clinically relevant linguistic and acoustic features derived from speech and transcripts based on prior literature, and (2) using transfer-learning and leveraging large pre-trained machine learning models: using transcript-representations that are automatically derived from state-of-the-art pre-trained language models, by fine-tuning Bidirectional Encoder Representations from Transformer (BERT)-based sequence classification models.Results: We compared the utility of speech transcript representations obtained from recent natural language processing models (i.e., BERT) to more clinically-interpretable language feature-based methods. Both the feature-based approaches and fine-tuned BERT models significantly outperformed the baseline linguistic model using a small set of linguistic features, demonstrating the importance of extensive linguistic information for detecting cognitive impairments relating to AD. We observed that fine-tuned BERT models numerically outperformed feature-based approaches on the AD detection task, but the difference was not statistically significant. Our main contribution is the observation that when tested on the same, demographically balanced dataset and tested on independent, unseen data, both domain knowledge and pretrained linguistic models have good predictive performance for detecting AD based on speech. It is notable that linguistic information alone is capable of achieving comparable, and even numerically better, performance than models including both acoustic and linguistic features here. We also try to shed light on the inner workings of the more black-box natural language processing model by performing an interpretability analysis, and find that attention weights reveal interesting patterns such as higher attribution to more important information content units in the picture description task, as well as pauses and filler words.Conclusion: This approach supports the value of well-performing machine learning and linguistically-focussed processing techniques to detect AD from speech and highlights the need to compare model performance on carefully balanced datasets, using consistent same training parameters and independent test datasets in order to determine the best performing predictive model.

THE CONVERSATION IMPLICATURE IN PRESIDENT JOKO WIDODO RHETORICAL AND DIPLOMATIC SPEECH

The paper aims to describe the implicature used by President Joko Widodo in his speech on international forums. This study employs a pragmatic analysis approach to identify the implicative meanings of his speech. The data were the videos and texts of the transcription of President Joko Widodo's speech at the APEC CEO Summit Summit in Beijing on November 10, 2014 and the Asian-African Conference in Jakarta on April 22 2015. The data were analyzed by using note-taking techniques comprising two stages, (1) classifying linguistic data that contains implicatures and (2) interpreting the meaning behind the implicature used, based on the context of the situation and the background of the relevant discourse. Based on the results, President Joko Widodo‘s speech was dominated by several implicatures including influencing, insinuating, convincing, threatening, complaining, clarifying, and governing.

Boys-Specific Text-Comprehension Enhancement With Dual Visual-Auditory Text Presentation Among 12-14 Years-Old Students.

Quality of language comprehension determines performance in all kinds of activities including academics. Processing of words initially develops as auditory, and gradually extends to visual as children learn to read. School failure is highly related to listening and reading comprehension problems. In this study we analyzed sex-differences in comprehension of texts in Spanish (standardized reading test PROLEC-R) in three modalities (visual, auditory, and both simultaneously: dual-modality) presented to 12–14-years old students, native in Spanish. We controlled relevant cognitive variables such as attention (d2), phonological and semantic fluency (FAS) and speed of processing (WISC subtest Coding). Girls’ comprehension was similar in the three modalities of presentation, however boys were importantly benefited by dual-modality as compared to boys exposed only to visual or auditory text presentation. With respect to the relation of text comprehension and school performance, students with low grades in Spanish showed low auditory comprehension. Interestingly, visual and dual modalities preserved comprehension levels in these low skilled students. Our results suggest that the use of visual-text support during auditory language presentation could be beneficial for low school performance students, especially boys, and encourage future research to evaluate the implementation in classes of the rapidly developing technology of simultaneous speech transcription, that could be, in addition, beneficial to non-native students, especially those recently incorporated into school or newly arrived in a country from abroad.

Animal listening

The ‘Elegy on the Nightingale’ is a curious Latin poem of uncertain (but probably post-classical) date and authorship that is transmitted by several medieval manuscripts. It offers a catalogue of animal sounds rich in what linguists call iconicity, and literary scholars, onomatopoeia: to read these verses aloud is to imitate the sounds being described. The poem begins in address to the nightingale of its title, praised for her ability to make music by mimicking all she hears. By the end has the poem itself done the same? For all their playfulness, the verses strike at the heart of our own theoretical commonplaces, starting with the supposed arbitrariness of the sign, always unsettled by such examples, exceptional though they may be. So too did the writing down of non-human sounds preoccupy ancient linguists, who sought to segregate them from language proper. Nevertheless, it is difficult to deny that these sound-words conjure what they name, especially since, in many cases, it is only our ability to match their sounds to animals we can still hear that enables us to know what the poem is saying. What happens to our understanding of the poetic text as a transcription of human speech or song when we take it seriously as a recording of non-human sound? And even more dramatically, what happens to our understanding of human language when we strive (as this poem strives, albeit surreptitiously) to listen with non-human ears? With some help from the animal imaginings of Jakob von Uexküll, this article attempts some preliminary answers.

“If the gas pipeline would be built, we lose”: transcript of Reagan's speech at the US National Security Council Meeting on the Sanctions against Soviet Union

The object of this research is the declassified transcript of the US National Security Council Meeting of July 9, 1981, dedicated to introduction of the economic and political restrictions on building of the Soviet gas main pipeline. The subject of this research is the analysis information-bearing capabilities of the office documentation of the US National Security Council Meetings for conducting the scientific assessment of sanctions policy of the US government against the Soviet Union in the 1980s as part of directives on restricting the access of the Soviet Union to foreign markets. The article examines the published protocol the US National Security Council Meeting and related documents that contain information on creating the regime to impede the construction of the gas main pipeline to Europe. The novelty of lies in the fact that this article is first within the framework of historiographical analysis to study the plans of the US President R. Reagan on interruption of the active efforts of the Soviet Union to supply Western Europe with energy. Publication of the document clearly demonstrates that the ideas of restarting the trade-economic development of the Soviet Union were later implemented in other countries in the XXI century, when the Russian Nord Stream pipeline became one of the crucial vanguards within the system of control of the US national security interests in Europe. Based on the newly introduced documents from the foreign archives of the CIA, Ronald Reagan Presidential Library, materials of the US periodical press, and memoirs, the author explores the options prepared by the US agencies aimed at complete shutdown, and restriction of access of the participants of the Soviet-German gas pipeline deal to foreign markets and resources, as well as the response of business community to trade embargo with the USSR.

Attention reinforces human corticofugal system to aid speech perception in noise

Perceiving speech-in-noise (SIN) demands precise neural coding between brainstem and cortical levels of the hearing system. Attentional processes can then select and prioritize task-relevant cues over competing background noise for successful speech perception. In animal models, brainstem-cortical interplay is achieved via descending corticofugal projections from cortex that shape midbrain responses to behaviorally-relevant sounds. Attentional engagement of corticofugal feedback may assist SIN understanding but has never been confirmed and remains highly controversial in humans. To resolve these issues, we recorded source-level, anatomically constrained brainstem frequency-following responses (FFRs) and cortical event-related potentials (ERPs) to speech via high-density EEG while listeners performed rapid SIN identification tasks. We varied attention with active vs. passive listening scenarios whereas task difficulty was manipulated with additive noise interference. Active listening (but not arousal-control tasks) exaggerated both ERPs and FFRs, confirming attentional gain extends to lower subcortical levels of speech processing. We used functional connectivity to measure the directed strength of coupling between levels and characterize “bottom-up” vs. “top-down” (corticofugal) signaling within the auditory brainstem-cortical pathway. While attention strengthened connectivity bidirectionally, corticofugal transmission disengaged under passive (but not active) SIN listening. Our findings (i) show attention enhances the brain's transcription of speech even prior to cortex and (ii) establish a direct role of the human corticofugal feedback system as an aid to cocktail party speech perception.

Recognition of Alzheimer’s Dementia From the Transcriptions of Spontaneous Speech Using fastText and CNN Models

Alzheimer’s dementia (AD) is a type of neurodegenerative disease that is associated with a decline in memory. However, speech and language impairments are also common in Alzheimer’s dementia patients. This work is an extension of our previous work, where we had used spontaneous speech for Alzheimer’s dementia recognition employing log-Mel spectrogram and Mel-frequency cepstral coefficients (MFCC) as inputs to deep neural networks (DNN). In this work, we explore the transcriptions of spontaneous speech for dementia recognition and compare the results with several baseline results. We explore two models for dementia recognition: 1) fastText and 2) convolutional neural network (CNN) with a single convolutional layer, to capture the n-gram-based linguistic information from the input sentence. The fastText model uses a bag of bigrams and trigrams along with the input text to capture the local word orderings. In the CNN-based model, we try to capture different n-grams (we usen= 2, 3, 4, 5) present in the text by adapting the kernel sizes to n. In both fastText and CNN architectures, the word embeddings are initialized using pretrained GloVe vectors. We use bagging of 21 models in each of these architectures to arrive at the final model using which the performance on the test data is assessed. The best accuracies achieved with CNN and fastText models on the text data are 79.16 and 83.33%, respectively. The best root mean square errors (RMSE) on the prediction of mini-mental state examination (MMSE) score are 4.38 and 4.28 for CNN and fastText, respectively. The results suggest that the n-gram-based features are worth pursuing, for the task of AD detection. fastText models have competitive results when compared to several baseline methods. Also, fastText models are shallow in nature and have the advantage of being faster in training and evaluation, by several orders of magnitude, compared to deep models.

Paralinguistic and linguistic fluency features for Alzheimer's disease detection

Alzheimer’s disease (AD) is one of the most common forms of dementia in the world. The Mini-Mental State Examination, a tool developed to detect AD, is composed of various tests that evaluate functional performance in several fields, one of which is language. Several symptoms are manifested in voices as a result of language and speech problems caused by AD, including frequent involuntary pauses during conversations and diction and vocabulary difficulties. Speech fluency is considered a key feature for AD detection in this research, for which two algorithms are proposed. The first algorithm is a paralinguistic system that is independent of the language and task and whose low-dimension feature vectors facilitate the training stage. This algorithm is tested on two databases (AcceXible and ADReSS), on two languages (Spanish and English) and on several tests. The second algorithm is based on analysing temporal patterns of silence between words and errors in spoken words. This approach, based on verbal fluency tests, is tested on the AcceXible database. To benchmark these algorithms, two baseline algorithms are used: the i-vector framework, a speaker modelling algorithm that has been effectively used for speech-related tasks such as speaker recognition, language identification, speaker diarization and speech-related health tasks; and a classic counting-terms algorithm, which processes transcriptions of speech. The paralinguistic system yields promising results for different tests and languages, while the silence-based system achieves high accuracy in verbal fluency tests.

Extending automatic transcripts in a unified data representation towards a prosodic-based metadata annotation and evaluation

This paper describes a framework that extends automatic speech transcripts in order to accommodate relevant information coming from manual transcripts, the speech signal itself, and other resources, like lexica. The proposed framework automatically collects, relates, computes, and stores all relevant information together in a self-contained data source, making it possible to easily provide a wide range of interconnected information suitable for speech analysis, training, and evaluating a number of automatic speech processing tasks. The main goal of this framework is to integrate different linguistic and paralinguistic layers of knowledge for a more complete view of their representation and interactions in several domains and languages. The processing chain is composed of two main stages, where the first consists of integrating the relevant manual annotations in the speech recognition data, and the second consists of further enriching the previous output in order to accommodate prosodic information. The described framework has been used for the identification and analysis of structural metadata in automatic speech transcripts. Initially put to use for automatic detection of punctuation marks and for capitalization recovery from speech data, it has also been recently used for studying the characterization of disfluencies in speech. It was already applied to several domains of Portuguese corpora, and also to English and Spanish Broadcast News corpora.

Speech Processing for Language Learning: A Practical Approach to Computer-Assisted Pronunciation Teaching

This article contributes to the discourse on how contemporary computer and information technology may help in improving foreign language learning not only by supporting better and more flexible workflow and digitizing study materials but also through creating completely new use cases made possible by technological improvements in signal processing algorithms. We discuss an approach and propose a holistic solution to teaching the phonological phenomena which are crucial for correct pronunciation, such as the phonemes; the energy and duration of syllables and pauses, which construct the phrasal rhythm; and the tone movement within an utterance, i.e., the phrasal intonation. The working prototype of StudyIntonation Computer-Assisted Pronunciation Training (CAPT) system is a tool for mobile devices, which offers a set of tasks based on a “listen and repeat” approach and gives the audio-visual feedback in real time. The present work summarizes the efforts taken to enrich the current version of this CAPT tool with two new functions: the phonetic transcription and rhythmic patterns of model and learner speech. Both are designed on a base of a third-party automatic speech recognition (ASR) library Kaldi, which was incorporated inside StudyIntonation signal processing software core. We also examine the scope of automatic speech recognition applicability within the CAPT system workflow and evaluate the Levenstein distance between the transcription made by human experts and that obtained automatically in our code. We developed an algorithm of rhythm reconstruction using acoustic and language ASR models. It is also shown that even having sufficiently correct production of phonemes, the learners do not produce a correct phrasal rhythm and intonation, and therefore, the joint training of sounds, rhythm and intonation within a single learning environment is beneficial. To mitigate the recording imperfections voice activity detection (VAD) is applied to all the speech records processed. The try-outs showed that StudyIntonation can create transcriptions and process rhythmic patterns, but some specific problems with connected speech transcription were detected. The learners feedback in the sense of pronunciation assessment was also updated and a conventional mechanism based on dynamic time warping (DTW) was combined with cross-recurrence quantification analysis (CRQA) approach, which resulted in a better discriminating ability. The CRQA metrics combined with those of DTW were shown to add to the accuracy of learner performance estimation. The major implications for computer-assisted English pronunciation teaching are discussed.

Speech to multi-language text conversion

Right from the beginning of previous century, researchers have shown interest in areas like Automatic Speech Recognition, Image Processing and Natural Language Processing. The area of Automatic Speech Recognition (ASR) has received attention over the past five decades due to its application in both commercial and military. In the recent times this can be attributed to the advancements in Artificial Intelligence and Advanced Algorithms. ASR takes speech as input and converts it in to text. ASR is employed in electronic dictionaries, Customer Call Centers, Voice Dictation and Query based Information Systems, Speech Transcription, Avionics, Smart Houses and Access Systems and many more areas. ASR can also be used to interact with handicapped people. ASR enables human beings interact with computers using speech rather than using keyboards & mouse (Vimalaand Radha V., 2012). ASR aims to provide natural machine interface where in speech acts input to the machine. Generally, ASR is based on two tasks viz. Identification of Phoneme and Whole-Word Decoding. A relationship between speech signal and speech segment that has dissimilar physical or perceptual features usually termed as phones is established in two steps. The first step deals with dimensionality reduction and second step deals with the estimation of likelihood of each phoneme. In the dimensionality reduction phase, the volume of the speech signal is decreased by extracting the relevant information using task-specific knowledge. In the next phase, the system recognizes the word sequence using a discriminative program. Traditionally ASR systems preferred the Mel frequency Cepstral coefficients (MFCC) for the first phase and Discriminative techniques for the second phase. Over the years ASR systems have evolved from being an integration of multiple trained components to “end-to-end” Deep neural architectures that link speech to text directly. The proposed work implements an MLP with AdaBoost Classifier. The MLP will be used to extract discriminative features from the speech data. Later AdaBoost classifier will map these features to the relevant set of words.

Жить и умереть Вступительное слово академика И.Т. Фролова на открытии конференции, посвященной проблемам жизни и смерти (Москва, 1993 г.) Предисловие к публикации

The transcript of the opening speech of Academician I.T. Frolov at the opening of the Conference dedicated to the problems of life and death is published. The conference was organized in 1993 by the Institute of Man of the Russian Acad­emy of Sciences. In his opening speech at the conference, I.T. Frolov summed up the results of ten years of work on the problem of the meaning of life, death and immortality of man, which began with the publication of his articles on this topic in the first issues of the journal “Voprosy Filosofii” in 1983. In his 1993 speech, I.T. Frolov spoke about the program of tanatological research of the Institute of Man of the Russian Academy of Sciences. He shared his thoughts on the prob­lem of life and death in Russian philosophy and culture, and analyzed the prospects for research of this problem by modern science. The text of the tran­script was discovered in 2020 by the Group for the Study of the Creative Heritage of I.T. Frolov of the Institute of Philosophy of the Russian Academy of Sciences.

Convolutional recurrent neural network with attention for Vietnamese speech to text problem in the operating room

We introduce automatic Vietnamese speech recognition (ASR) system for converting Vietnamese speech to text on a real operating room ambient noise recorded during liver surgery. First, we propose applying a combination between convolutional neural network (CNN) and bidirectional long short-term memory (BLSTM) for investigating local speech feature learning, sequence modelling, and transcription for speech recognition. We also extend the CNN-LSTM framework with an attention mechanism to decode the frames into a sequence of words. The CNN, LSTM and attention models are combining into a unified architecture. In addition, we combine connectionist temporal classification (CTC) and attention's loss functions in training phase. The length of the output label sequence from CTC is applied to the attention-based decoder predictions to make the final label sequence. This process helps to decrease irregular alignments and make speedup of the label sequence estimation during training and inference, instead of only relying on the data-driven attention-based encoder-decoder for estimating the label sequence in long sentences. The proposed system is evaluated using a real operating room database. The results show that our method significantly enhances the performance of the ASR system. We find that our approach provides a 13.05% in WER and outperforms standard methods.