Character Error Rate Research Articles

Deep learning-based recognition system for pashto handwritten text: benchmark on PHTI.

This article introduces a recognition system for handwritten text in the Pashto language, representing the first attempt to establish a baseline system using the Pashto Handwritten Text Imagebase (PHTI) dataset. Initially, the PHTI dataset underwent pre-processed to eliminate unwanted characters, subsequently, the dataset was divided into training 70%, validation 15%, and test sets 15%. The proposed recognition system is based on multi-dimensional long short-term memory (MD-LSTM) networks. A comprehensive empirical analysis was conducted to determine the optimal parameters for the proposed MD-LSTM architecture; Counter experiments were used to evaluate the performance of the proposed system comparing with the state-of-the-art models on the PHTI dataset. The novelty of our proposed model, compared to other state of the art models, lies in its hidden layer size (i.e., 10, 20, 80) and its Tanh layer size (i.e., 20, 40). The system achieves a Character Error Rate (CER) of 20.77% as a baseline on the test set. The top 20 confusions are reported to check the performance and limitations of the proposed model. The results highlight complications and future perspective of the Pashto language towards the digital transition.

Sampleformer: An efficient conformer-based Neural Network for Automatic Speech Recognition

The Convolution-augmented Transformer (Conformer) model, which was recently introduced, has attained state-of-the-art(SOTA) results in Automatic Speech Recognition (ASR). In this paper, a series of methodical investigations uncover that the Conformer’s design decisions may not represent the most efficient choices when operating within the constraints of a limited computational budget. After a thorough re-evaluation of the Conformer architecture’s design choices, we propose Sampleformer which reduces the Conformer architecture complexity and has more robust performance. We introduce downsampling to the Conformer Encoder, and to exploit the information in the speech features, we incorporate an additional downsampling module to enhance the efficiency and accuracy of our model. Additionally, we propose a novel and adaptable attention mechanism called multi-group attention, effectively reducing the attention complexity from O⁢(n2⁢d) to O⁢(n2⁢d⋅f/g). We performed experiments on the AISHELL-1 corpora, our 13.3 million-parameter CTC model demonstrates a 3.0%/2.6% relative reduction in character error rate (CER) on the dev/test sets, all without the utilization of a language model (LM). Additionally, the model exhibits a 30% improvement in inference compared to our CTC Conformer baseline and trains 27% faster.

Text Recognition for Library Collection in Different Light Conditions

Book arrangement problems in Indonesia mainly utilize traditional systems. However, it contributes to user frustration and dissatisfaction, leading to difficulties in the retrieval process and hindering book monitoring. Therefore, an efficient arrangement is needed to make it easier for librarians to organize collections at the library. Hence, this study proposes and evaluates Optical Character Recognition (OCR) approaches such as Tesseract-OCR and Efficient and Accurate Scene Text (EAST) by considering various light factors and device applications. An evaluation was performed to determine the effectiveness of the methods for detecting classification numbers using the Character Error Rate (CER) and Word Error Rate (WER). The results obtained can be seen from the lighting and the use of image capture tools that can affect the text detection results. The higher the lighting, the better the text detection results obtained. However, in the extraction tool using a security camera yields a better output than the others. In conclusion, the modernization of book arrangement systems by implementing OCR through security cameras could enhance user experience in the retrieval process.

KMSAV: Korean multi‐speaker spontaneous audiovisual dataset

AbstractRecent advances in deep learning for speech and visual recognition have accelerated the development of multimodal speech recognition, yielding many innovative results. We introduce a Korean audiovisual speech recognition corpus. This dataset comprises approximately 150 h of manually transcribed and annotated audiovisual data supplemented with additional 2000 h of untranscribed videos collected from YouTube under the Creative Commons License. The dataset is intended to be freely accessible for unrestricted research purposes. Along with the corpus, we propose an open‐source framework for automatic speech recognition (ASR) and audiovisual speech recognition (AVSR). We validate the effectiveness of the corpus with evaluations using state‐of‐the‐art ASR and AVSR techniques, capitalizing on both pretrained models and fine‐tuning processes. After fine‐tuning, ASR and AVSR achieve character error rates of 11.1% and 18.9%, respectively. This error difference highlights the need for improvement in AVSR techniques. We expect that our corpus will be an instrumental resource to support improvements in AVSR.

Developing children's ASR system under low-resource conditions using end-to-end architecture

The work presented in this paper aims at enhancing the performance of end-to-end (E2E) speech recognition task for children's speech under low resource conditions. For majority of the languages, there is hardly any speech data from child speakers. Furthermore, even the available children's speech corpora are limited in terms of the number of hours of data. On the other hand, large amounts of adults' speech data are freely available for research as well as commercial purposes. As a consequence, developing an effective E2E automatic speech recognition (ASR) system for children becomes a very challenging task. One may develop an ASR system using adults' speech and then use it to transcribe children's data, but this leads to very poor recognition rates due to the stark differences in the acoustic attributes of adults' and children's speech. In order to overcome these hurdles and to develop a robust children's ASR system employing E2E architecture, we have resorted to several out-of-domain and in-domain data augmentation techniques. For out-of-domain data augmentation, we have explicitly modified adults' speech to render it acoustically similar to that of children's speech before pooling into training. On the other hand, in the case of in-domain data augmentation, we have slightly modified the pitch and duration of children's speech in order to create more data capturing greater diversity. Data augmentation approaches helps in mitigating the ill-effects resulting from the scarcity of data from child domain to a certain extent. This, in turn, reduces the error rates by a large margin. In addition to data augmentation, we have also studied the efficacy of Gamma-tone frequency cepstral coefficients (GFCC) and frequency domain linear prediction (FDLP) technique along with the most commonly used Mel-frequency cepstral coefficients (MFCC) for front-end speech parameterization. Both MFCC as well as GFCC capture and model the spectral envelope of speech. On the other hand, application of linear prediction on the frequency domain representation of speech signal helps to effectively capture the temporal envelope during front-end feature extraction. Employing FDLP features that model the temporal envelope provides important cues for the perception and understanding of stop bursts and, at times, complete phonemes. This motivated us to perform a comparative experimental study of the effectiveness of the three aforementioned front-end acoustic features. In our experimental explorations, the use of proposed data augmentation in combination of FDLP features has shown a relative improvement in character error rate by 67.6% over the baseline system. The combination of data augmentation with MFCC or GFCC features is observed to result in lower recognition performances.

Oral Voice Recognition System Based on Deep Neural Network Posteriori Probability Algorithm

The unstoppable trend of globalization has made the integration of voice recognition with science and technology increasingly close, and has called for the emergence of more powerful oral voice recognition systems. At present, most speech recognition research uses Hidden Markov Model-Gaussian Mixture Modell (HMM-GMM) and onnectionist Temporal Classification (CTC) end-to-end method, which has obvious limitations. Therefore, based on neural network, this paper introduces deep neural network posterior probability algorithm into speech recognition system, and constructs the spoken speech recognition system based on deep neural network posterior probability algorithm. The design principle of the constructed speech system is discussed and the performance analysis and effectiveness evaluation are carried out. The experimental results show that the performance of the neural network-based speech recognition system constructed in this paper has made a breakthrough compared with the traditional technology, and the Character Error Rate (CER) is only 4.6%, and the End-to-End Latency and Equal Error Rate (EER) have reached a low level. This study points out the future development direction of speech recognition and provides the impetus to promote its further development

Deep Learning-Based Analysis of Ancient Greek Literary Texts in English Version: A Statistical Model Based on Word Frequency and Noise Probability for the Classification of Texts

In our paper we intend to present a methodology that we elaborated for clustering texts based on the word fre quency in the English translations of selected old Greek texts. We used the classification system of the ancient Library of Alex andria, devised by the prominent Greek scholar-poet, Callima chus in the 3rd century BC., as a basis for categorizing literary masterpieces. In our content analysis, we could determine a tri plet of a, b, c values for describing a power function that appro priately fits a curve determined by the word frequencies in the texts. In addition, we have discovered 16 special features of the different texts that correspond to various token categories inves tigated in each text, such as part of speech of the word in the con text, numerals, subordinate conjunction, symbols, etc. We have developed a cognitive model in which several hundred different subtexts were utilized for supervised learning with the aim of subtext class recognition. Concerning 200 subtexts, the triplet of a, b, c values, the classes of the subtexts, and their 16-dimen sional feature vectors were learnt for the Recurrent Neural Net work (RNN). It turned out that the Long-Short Term Memory RNN could efficiently predict which class a chosen subtext could be categorized into without considering the interpretation of the content. The influence of the non-zero error rate of new com munication services on the meaning of the transferred texts was also investigated. The impact of the noise on the classification accuracy was found to be linear, dependent on the character error rate.

Deep Learning for Accurate Recognition of Arabic Handwritten Words in Historical Documents.

Invaluable for historical research, Arabic manuscripts are often challenging to access due to their physical preservation needs. Digitization coupled with handwriting recognition offers a powerful solution for making these resources readily available. However, accurate text recognition hinges on effective segmentation into lines and words. While line segmentation has received significant attention, word detection in historical Arabic script remains an under-explored area due to the scarcity of annotated datasets. This paper addresses this gap by introducing a novel, word-level annotated database specifically designed for historical Arabic manuscripts. We further present two neural network architectures based on Transformers relying on an Arabic language model and a robust CNN-BLSTM with skip connections, guaranteeing the preservation of crucial spatial information for recognition. Validation on a dataset of 20 pages of historical manuscripts demonstrates the effectiveness of the proposed models, achieving a Character Error Rate (CER) of 4.8% with data augmentation, surpassing the state-of-the-art.

ARABIC SOFT SPELLING CORRECTION WITH T5

Spelling correction is considered a challenging task for resource-scarce languages. The Arabic language is one of these resource-scarce languages, which suffers from the absence of a large spelling correction dataset, thus datasets injected with artificial errors are used to overcome this problem. In this paper, we trained the Text-to-Text Transfer Transformer (T5) model using artificial errors to correct Arabic soft spelling mistakes. Our T5 model can correct 97.8% of the artificial errors that were injected into the test set. Additionally, our T5 model achieves a character error rate (CER) of 0.77% on a set that contains real soft spelling mistakes. We achieved these results using a 4-layer T5 model trained with a 90% error injection rate, with a maximum sequence length of 300 characters.

A hybrid model for Arabic Script Recognition based on CNN-CBAM and BLSTM

Handwriting recognition, particularly for Arabic, is a very challenging field of research due to various complex factors such as the presence of ligatures, cursive writing style, slant variations, diacritics, overlapping, and other difficult problems. This paper specifically addresses the task of recognizing offline Arabic handwritten text lines. The main contributions include the pre-processing stage and the utilization of a deep learning-based approach with data augmentation techniques. The pre-processing step involves correcting the skew of text-lines and removing any unnecessary white space in images. The deep learning architecture consists of a Convolutional Neural Network and Convolutional Block Attention Module for feature extraction, along with Bidirectional Long Short-Term Memory for sequence modelling and Connectionist Temporal Classification as a decoder. Data augmentation techniques are utilized on the images in the database to enhance the system’s ability to recognize a wide range of Arabic characters and to extend the level of abstraction in patterns due to synthetic variations. Our suggested approach has the capability of precisely recognizing Arabic handwritten texts without the necessity of character segmentation, thereby resolving various issues associated with this aspect. The results obtained from the KHATT database highlight the effectiveness of our approach, demonstrating a Word Error Rate of 14.55% and a Character Error Rate of 3.25%.

NEURAL NETWORK ARCHITECTURE FOR TEXT DECODING BASED ON SPEAKER'S LIP MOVEMENTS

In this paper, we tested a command recognition system using the SSI approach and conducted a series of experiments on modern solutions based on ALR interfaces. The main goal was to improve the accuracy of speech recognition in cases where it is not possible to use the speaker's non-noisy audio sequence, for example, at a great distance from the speaker or in a noisy environment. The obtained results showed that training the neural network on a GPU accelerator allowed to reduce the training time by 26.2 times using a high-resolution training sample with a size of the selected mouth area of 150 × 100 pixels. The results of the analysis of the selected speech recognition quality assessment metrics (word recognition rate (WRR), word error rate (WER), and character error rate (CER)) showed that the maximum word recognition rate of the speaker's speech is 96.71% and is achieved after 18 epochs of training. If we evaluate the character regonition rate of viseme recognition, the highest rate can be obtained after 13 epochs of training. Future research will focus on the use of depth cameras and stereo vision methods with increased frame rates to further improve the accuracy of voice command decoding in conditions of high background noise.

Offline Handwritten Text Extraction and Recognition Using CNN-BLSTM-CTC Network

Offline handwriting recognition is a significant research area that aims at tackling problems encountered with handwritten forms in college application and registration processes. The objective of this study is to address the problems of English language offline handwriting recognition via CNN-BLSTM-CTC neural network applied for an NCE Admission form. The system uses OpenCV for image processing, TensorFlow for neural network training and handwritten text recognition, and trains and tests it on the IAM database using image segmentation-based handwriting recognition. With the help of proper image verification, the system allows the users to upload images of the NCE Admission form provided that they strictly comply with the specified format; it denies access to images not conforming to the set standards. Following the successful delivery of a valid image, the form goes through extensive processing that includes text extraction from specific regions of interest (ROIs). The extracted texts are then passed to text recognition block. The recognized texts are then recorded in a CSV file under respective fields. The text recognition model has a CER of approximately 9.33%. The study performed with 15 NCE Admission forms found that the average Character Error Rate (CER) was approximately 12.2% for scanned images and 19.3% for camera-captured images. The results show that accuracy depends on aspects such as the quality and orientation of the image; thus, scanned images are preferred for better performance.

Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision

Abstract We introduce SPEAR-TTS, a multi-speaker text-to-speech (TTS) system that can be trained with minimal supervision. By combining two types of discrete speech representations, we cast TTS as a composition of two sequence-to-sequence tasks: from text to high-level semantic tokens (akin to “reading”) and from semantic tokens to low-level acoustic tokens (“speaking”). Decoupling these two tasks enables training of the “speaking” module using abundant audio-only data, and unlocks the highly efficient combination of pretraining and backtranslation to reduce the need for parallel data when training the “reading” component. To control the speaker identity, we adopt example prompting, which allows SPEAR-TTS to generalize to unseen speakers using only a short sample of 3 seconds, without any explicit speaker representation or speaker labels. Our experiments demonstrate that SPEAR-TTS achieves a character error rate that is competitive with state-of-the-art methods using only 15 minutes of parallel data, while matching ground-truth speech in naturalness and acoustic quality.

Gated Convolution and Stacked Self-Attention Encoder–Decoder-Based Model for Offline Handwritten Ethiopic Text Recognition

Offline handwritten text recognition (HTR) is a long-standing research project for a wide range of applications, including assisting visually impaired users, humans and robot interactions, and the automatic entry of business documents. However, due to variations in writing styles, visual similarities between different characters, overlap between characters, and source document noise, designing an accurate and flexible HTR system is challenging. The problem becomes serious when the algorithm has a low learning capacity and when the text used is complex and has a lot of characters in the writing system, such as Ethiopic script. In this paper, we propose a new model that recognizes offline handwritten Ethiopic text using a gated convolution and stacked self-attention encoder–decoder network. The proposed model has a feature extraction layer, an encoder layer, and a decoder layer. The feature extraction layer extracts high-dimensional invariant feature maps from the input handwritten image. Using the extracted feature maps, the encoder and decoder layers transcribe the corresponding text. For the training and testing of the proposed model, we prepare an offline handwritten Ethiopic text-line dataset (HETD) with 2800 samples and a handwritten Ethiopic word dataset (HEWD) with 10,540 samples obtained from 250 volunteers. The experiment results of the proposed model on HETD show a 9.17 and 13.11 Character Error Rate (CER) and Word Error Rate (WER), respectively. However, the model on HEWD shows an 8.22 and 9.17 CER and WER, respectively. These results and the prepared datasets will be used as a baseline for future research.

The Challenges of HTR Model Training: Feedback from the Project Donner le gout de l'archive a l'ere numerique

The arrival of handwriting recognition technologies offers new possibilities for research in heritage studies. However, it is now necessary to reflect on the experiences and the practices developed by research teams. Our use of the Transkribus platform since 2018 has led us to search for the most significant ways to improve the performance of our handwritten text recognition (HTR) models which are made to transcribe French handwriting dating from the 17th century. This article therefore reports on the impacts of creating transcribing protocols, using the language model at full scale and determining the best way to use base models in order to help increase the performance of HTR models. Combining all of these elements can indeed increase the performance of a single model by more than 20% (reaching a Character Error Rate below 5%). This article also discusses some challenges regarding the collaborative nature of HTR platforms such as Transkribus and the way researchers can share their data generated in the process of creating or training handwritten text recognition models.

Task-based Meta Focal Loss for Multilingual Low-resource Speech Recognition

Low-resource automatic speech recognition is a challenging task due to a lack of labeled training data. To resolve this issue, multilingual meta-learning learns a better model initialization from many source language tasks for fast adaptation to unseen target languages. However, for diverse source languages, the quantity and difficulty vary greatly because of their different data scales and phonological systems. These differences lead to task-quantity and task-difficulty imbalance issues and thus a failure of multilingual meta-learning ASR. In this work, we propose a task-based meta focal loss (TMFL) approach to address this tough challenge. Specifically, we introduce a hard-task moderator and update the meta-parameters using gradients from both the support set and query set. Our proposed approach focuses more on hard tasks and makes full use of the data from hard tasks. Moreover, we analyze the significance of the hard task moderator and interpret its significance at the sample level. Experiment results show that the proposed method, TMFL, significantly outperforms the state-of-the-art multilingual meta-learning on all target languages for the IARPA BABEL and OpenSLR datasets, especially under very-low-resource conditions. In particular, it can reduce character error rate from 72% to 60% by fine-tuning the pre-trained model with about 22 hours of Vietnamese data.

Efficient CRNN: Towards end-to-end low resource Urdu text recognition using depthwise separable convolutions and gated recurrent units

In this study, a novel technique is proposed to recognize printed text in images for Urdu, a low-resource language with a scarcity of benchmark datasets. The proposed technique is called Efficient CRNN which uses depthwise separable convolutional and bidirectional gated recurrent unit layers, followed by connectionist temporal classification loss. The proposed technique is computationally more efficient than the existing text recognition techniques, requiring fewer parameters and computations. A multi-font printed Urdu text lines corpus is also presented, consisting of 245,000 text line images rendered using 7 different fonts. The corpus is called the MMU-Extension-22 and is used to train and evaluate existing state-of-the-art end-to-end text recognition techniques. Efficient CRNN is also evaluated using the proposed corpus. The proposed technique is first trained using a total of 196,000 text line images and then tested using 49,000 images. The Efficient CRNN technique achieved the minimum character and word error rates of 0.91% and 1.49% respectively for Urdu text line recognition under different settings, outperforming the existing computationally more complex techniques. The simple nature of the proposed technique not only makes it more efficient but also more robust for Urdu text line recognition, achieving a 2.23% reduced character error rate and a 71%11Percentage Decrease = 100 * (Baseline Value - Changed Value)/Baseline Value. decrease in character error rate as compared to the best performing existing Recurrent Neural Networks based technique. Also, the proposed technique outperforms Vision Transformer-based network achieving a 0.79% reduced character error rate accounting for a 41% decrease in error. Also, the Efficient CRNN has 49.16% reduced parameters compared to the baseline Vision Transformer technique.

Online Mongolian Handwriting Recognition Based on Encoder–Decoder Structure with Language Model

Mongolian online handwriting recognition is a complex task due to the script’s intricate characters and extensive vocabulary. This study proposes a novel approach by integrating a pre-trained language model into the sequence-to-sequence(Seq2Seq) + attention mechanisms(AM) model to enhance recognition accuracy. Three fusion models, including former, latter, and complete fusion, are introduced, showing substantial improvements over the baseline model. The complete fusion model, combined with synchronized language model parameters, achieved the best results, significantly reducing character and word error rates. This research presents a promising solution for accurate Mongolian online handwriting recognition, offering practical applications in preserving and utilizing the Mongolian script.

Automatic optical inspection for detecting keycaps misplacement using Tesseract optical character recognition

<span lang="EN-US">This research study aims to develop automatic optical inspection (AOI) for detecting keycaps misplacement on the keyboard. The AOI hardware has been designed using an industrial camera with an additional mechanical jig and lighting system. Optical character recognition (OCR) using the Tesseract OCR engine is the proposed method to detect keycaps misplacement. In addition, captured images were cropped using a predefined region of interest (ROI) during the setup. Subsequently, the cropped ROIs were processed to acquire binary images. Furthermore, Tesseract processed these binary images to recognize the text on the keycaps. Keycaps misplacement could be identified by comparing the predicted text with the actual text on the golden sample. Experiments on 25 defects and 25 non-defected samples provided a classification accuracy of 97.34%, a precision of 100%, and a recall of 90.70%. Meanwhile, the character error rate (CER) obtained from the test on a total of 57 characters provided a performance of 10.53%. This outcome has implications for developing AOI for various keyboard products. In addition, the precision level of 100% signifies that the proposed method always offers correct results in detecting product defects. Such outcomes are critical in industrial applications to prevent defective products from circulating in the market.</span>

Radio2Text

Millimeter wave (mmWave) based speech recognition provides more possibility for audio-related applications, such as conference speech transcription and eavesdropping. However, considering the practicality in real scenarios, latency and recognizable vocabulary size are two critical factors that cannot be overlooked. In this paper, we propose Radio2Text, the first mmWave-based system for streaming automatic speech recognition (ASR) with a vocabulary size exceeding 13,000 words. Radio2Text is based on a tailored streaming Transformer that is capable of effectively learning representations of speech-related features, paving the way for streaming ASR with a large vocabulary. To alleviate the deficiency of streaming networks unable to access entire future inputs, we propose the Guidance Initialization that facilitates the transfer of feature knowledge related to the global context from the non-streaming Transformer to the tailored streaming Transformer through weight inheritance. Further, we propose a cross-modal structure based on knowledge distillation (KD), named cross-modal KD, to mitigate the negative effect of low quality mmWave signals on recognition performance. In the cross-modal KD, the audio streaming Transformer provides feature and response guidance that inherit fruitful and accurate speech information to supervise the training of the tailored radio streaming Transformer. The experimental results show that our Radio2Text can achieve a character error rate of 5.7% and a word error rate of 9.4% for the recognition of a vocabulary consisting of over 13,000 words.