Text Recognition Model Research Articles

HubNet: An E2E Model for Wheel Hub Text Detection and Recognition Using Global and Local Features.

Automatic detection and recognition of wheel hub text, which can boost the efficiency and accuracy of product information recording, are undermined by the obscurity and orientation variability of text on wheel hubs. To address these issues, this paper constructs a wheel hub text dataset and proposes a wheel hub text detection and recognition model called HubNet. The dataset captured images on real industrial production line scenes, including 446 images, 934 word instances, and 2947 character instances. HubNet is an end-to-end text detection and recognition model, not only comprising conventional detection and recognition heads but also incorporating a feature cross-fusion module, which improves the accuracy of recognizing wheel hub texts by utilizing both global and local features. Experimental results show that on the wheel hub text dataset, the HubNet achieves an accuracy of 86.5%, a recall of 79.4%, and an F1-score of 0.828, and the feature cross-fusion module increases the accuracy by 2% to 4%. The wheel hub dataset and the HubNet offer a significant reference for automatic detection and recognition of wheel hub text.

Retraction Note: Effective offline handwritten text recognition model based on a sequence-to-sequence approach with CNN–RNN networks

Retraction Note: Effective offline handwritten text recognition model based on a sequence-to-sequence approach with CNN–RNN networks

Exploring recursive neural networks for compact handwritten text recognition models

This paper addresses the challenge of deploying recognition models in specific scenarios in which memory size is relevant, such as in low-cost devices or browser-based applications. We specifically focus on developing memory-efficient approaches for Handwritten Text Recognition (HTR) by leveraging recursive networks. These networks reuse learned weights across successive layers, thus enabling the maintenance of depth, a critical factor associated with model accuracy, without an increase in memory footprint. We apply neural recursion techniques to models typically used in HTR that contain convolutional and recurrent layers. We additionally study the impact of kernel scaling, which allows the activations of these recursive layers to be modified for greater expressiveness with little cost to memory. Our experiments on various HTR benchmarks demonstrate that recursive networks are, indeed, a good alternative. It is noteworthy that these recursive networks not only preserve but in some instances also enhance accuracy, making them a promising solution for memory-efficient HTR applications. This research establishes the utility of recursive networks in addressing memory constraints in HTR models. Their ability to sustain or improve accuracy while being memory-efficient positions them as a promising solution for practical deployment, especially in contexts where memory size is a critical consideration, such as low-cost devices and browser-based applications.

Handwritten stenography recognition and the LION dataset

AbstractIn this paper, we establish the first baseline for handwritten stenography recognition, using the novel LION dataset, and investigate the impact of including selected aspects of stenographic theory into the recognition process. We make the LION dataset publicly available with the aim of encouraging future research in handwritten stenography recognition. A state-of-the-art text recognition model is trained to establish a baseline. Stenographic domain knowledge is integrated by transforming the target sequences into representations which approximate diplomatic transcriptions, wherein each symbol in the script is represented by its own character in the transliteration, as opposed to corresponding combinations of characters from the Swedish alphabet. Four such encoding schemes are evaluated and results are further improved by integrating a pre-training scheme, based on synthetic data. The baseline model achieves an average test character error rate (CER) of 29.81% and a word error rate (WER) of 55.14%. Test error rates are reduced significantly (p< 0.01) by combining stenography-specific target sequence encodings with pre-training and fine-tuning, yielding CERs in the range of 24.5–26% and WERs of 44.8–48.2%. An analysis of selected recognition errors illustrates the challenges that the stenographic writing system poses to text recognition. This work establishes the first baseline for handwritten stenography recognition. Our proposed combination of integrating stenography-specific knowledge, in conjunction with pre-training and fine-tuning on synthetic data, yields considerable improvements. Together with our precursor study on the subject, this is the first work to apply modern handwritten text recognition to stenography. The dataset and our code are publicly available via Zenodo.

A Novel Multi-Stage Recognition System in Handwritten Chinese Characters

Due to the various individual handwriting styles and diverse structures of Chinese characters, it is rather intricate to recognize handwritten Chinese characters precisely. In this paper, a muti-stage system that combines a Differentiable Binarization Network (DBNet) for text detection and Scene Text Recognition with a Single Visual Model (SVTR) for effective character recognition is developed, aiming to offer a potential solution for correct identification of handwritten Chinese characters in real-world scenarios. More specifically, the proposed system utilized the CASIA-HWDB2.x dataset from the Institute of Automation of the Chinese Academy of Sciences to train text detection and recognition models. For the detection, the Differentiable Binarization method in the Head Network applies a dynamic threshold map for text-background differentiation. On top of the SVTR employed as a recognition module, this research proposed a performance-optimized PP-LCNet architecture and the Guided Training of Continuous Time-Series Classifiers (GTC) with TextConAug, which is a data augmentation strategy to accommodate real-world scenarios. Briefly, DBNet and SVTR are integrated to form an end-to-end system. The output of DBNet after undergoing cropping and direction classifier, guides SVTR for a comprehensive system, ensuring seamless detection and recognition of handwritten Chinese characters. Experimental results indicate that this system achieves a promising performance and can provide accurate information on handwritten Chinese characters.

Self-Distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach

Text recognition methods are gaining rapid development. Some advanced techniques, e.g., powerful modules, language models, and un- and semi-supervised learning schemes, consecutively push the performance on public benchmarks forward. However, the problem of how to better optimize a text recognition model from the perspective of loss functions is largely overlooked. CTC-based methods, widely used in practice due to their good balance between performance and inference speed, still grapple with accuracy degradation. This is because CTC loss emphasizes the optimization of the entire sequence target while neglecting to learn individual characters. We propose a self-distillation scheme for CTC-based model to address this issue. It incorporates a framewise regularization term in CTC loss to emphasize individual supervision, and leverages the maximizing-a-posteriori of latent alignment to solve the inconsistency problem that arises in distillation between CTC-based models. We refer to the regularized CTC loss as Distillation Connectionist Temporal Classification (DCTC) loss. DCTC loss is module-free, requiring no extra parameters, longer inference lag, or additional training data or phases. Extensive experiments on public benchmarks demonstrate that DCTC can boost text recognition model accuracy by up to 2.6%, without any of these drawbacks.

The Industrial Application of Artificial Intelligence-Based Optical Character Recognition in Modern Manufacturing Innovations

This paper presents the development of a comprehensive, on-site industrial Optical Character Recognition (OCR) system tailored for reading text on iron plates. Initially, the system utilizes a text region detection network to identify the text area, enabling camera adjustments along the x and y axes and zoom enhancements for clearer text imagery. Subsequently, the detected text region undergoes line-by-line division through a text segmentation network. Each line is then transformed into rectangular patches for character recognition by the text recognition network, comprising a vision-based text recognition model and a language network. The vision network performs preliminary recognition, followed by refinement through the language model. The OCR results are then converted into digital characters and recorded in the iron plate registration system. This paper’s contributions are threefold: (1) the design of a comprehensive, on-site industrial OCR system for autonomous registration of iron plates; (2) the development of a realistic synthetic image generation strategy and a robust data augmentation strategy to address data scarcity; and (3) demonstrated impressive experimental results, indicating potential for on-site industrial applications. The designed autonomous system enhances iron plate registration efficiency and significantly reduces factory time and labor costs.

Text Recognition Model for Yiddish in Vaybertaytsh Typeface, Based on Community Regulations

Text Recognition Model for Yiddish in Vaybertaytsh Typeface, Based on Community Regulations

Offline Handwritten Text Extraction and Recognition Using CNN-BLSTM-CTC Network

Offline handwriting recognition is a significant research area that aims at tackling problems encountered with handwritten forms in college application and registration processes. The objective of this study is to address the problems of English language offline handwriting recognition via CNN-BLSTM-CTC neural network applied for an NCE Admission form. The system uses OpenCV for image processing, TensorFlow for neural network training and handwritten text recognition, and trains and tests it on the IAM database using image segmentation-based handwriting recognition. With the help of proper image verification, the system allows the users to upload images of the NCE Admission form provided that they strictly comply with the specified format; it denies access to images not conforming to the set standards. Following the successful delivery of a valid image, the form goes through extensive processing that includes text extraction from specific regions of interest (ROIs). The extracted texts are then passed to text recognition block. The recognized texts are then recorded in a CSV file under respective fields. The text recognition model has a CER of approximately 9.33%. The study performed with 15 NCE Admission forms found that the average Character Error Rate (CER) was approximately 12.2% for scanned images and 19.3% for camera-captured images. The results show that accuracy depends on aspects such as the quality and orientation of the image; thus, scanned images are preferred for better performance.

An end-to-end model for multi-view scene text recognition

Due to the increasing applications of surveillance and monitoring such as person re-identification, vehicle re-identification and sports events tracking, the necessity of text detection and end-to-end recognition is also growing. Although the past deep learning-based models have addressed several challenges such as arbitrary-shaped text, multiple scripts, and variations in the geometric structure of characters, the scope of the models is limited to a single view. This paper presents an end-to-end model for text recognition through refining the multi-views of the same scene, which is called E2EMVSTR (End-to-End Model for Multi-View Scene Text Recognition). Considering the common characteristics shared in multi-view texts, we propose a cycle consistency pairwise similarity-based deep learning model to find texts more efficiently in three input views. Further, the extracted texts are supplied to a Siamese network and semi-supervised attention embedding combinational network for obtaining recognition results. The proposed model combines natural language processing and genetic algorithm models to restore missing character information and correct wrong recognition results. In experiments on our multi-view dataset and several benchmark datasets, the proposed method is proven effective compared to the state-of-the-art methods. The dataset and codes will be made available to the public upon acceptance.

The Challenges of HTR Model Training: Feedback from the Project Donner le gout de l'archive a l'ere numerique

The arrival of handwriting recognition technologies offers new possibilities for research in heritage studies. However, it is now necessary to reflect on the experiences and the practices developed by research teams. Our use of the Transkribus platform since 2018 has led us to search for the most significant ways to improve the performance of our handwritten text recognition (HTR) models which are made to transcribe French handwriting dating from the 17th century. This article therefore reports on the impacts of creating transcribing protocols, using the language model at full scale and determining the best way to use base models in order to help increase the performance of HTR models. Combining all of these elements can indeed increase the performance of a single model by more than 20% (reaching a Character Error Rate below 5%). This article also discusses some challenges regarding the collaborative nature of HTR platforms such as Transkribus and the way researchers can share their data generated in the process of creating or training handwritten text recognition models.

Exploring the Role of Chinese Language and Literature in the Transmission of Traditional Culture by Combining the Method of Internet Text Analysis

Abstract This paper constructs a Bayesian network text recognition model based on the Bayesian network and explores the role of Chinese language literature in the dissemination of traditional culture by analyzing the embodiment of traditional culture in Chinese language literature network texts. The collection process of Chinese language and literature data in network text is analyzed from the perspective of textual data interaction. The information of node variables in a Bayesian network is used to determine the mutual relationship between Chinese language literature and traditional culture. The degree of interdependence between Chinese literature and traditional culture can be measured by combining mutual information. The results show that the correct rate of text recognition of the Bayesian text recognition model decreases slightly when the training samples are (100-300), but the correct rate always stays around 0.85, thus reflecting the effectiveness of the network recognition model in this paper. Chinese language literature has a certain role in the dissemination of traditional culture, which proves that Chinese language literature, as a carrier of traditional culture, can improve the dissemination speed of traditional culture. This study focuses on the integration of Chinese literature and traditional communication to improve a new vision.

Experimenting with Training a Neural Network in Transkribus to Recognise Text in a Multilingual and Multi-Authored Manuscript Collection

This work aims at developing an optimal strategy to automatically transcribe a large quantity of uncategorised, digitised archival documents when resources include handwritten text by multiple authors and in several languages. We present a comparative study to establish the efficiency of a single multilingual handwritten text recognition (HTR) model trained on multiple handwriting styles instead of using a separate model for every language. When successful, this approach allows us to automate the transcription of the archive, reducing manual annotation efforts and facilitating information retrieval. To train the model, we used the material from the personal archive of the Dutch glass artist Sybren Valkema (1916–1996), processing it with Transkribus.

Text-Enhanced Scene Image Super-Resolution via Stroke Mask and Orthogonal Attention

Low-resolution text images are very commonplace in real life and their information is hard to be extracted by using existing text recognition methods only. Although this problem can be solved by introducing super-resolution (SR) techniques, most existing SR methods fail to process stroke regions and background regions of input text images distinctively. In this paper, we propose a text-specific super-resolution network named Text Enhanced Attention Network (TEAN) to solve this problem. First of all, we compensate for disadvantages of traditional thresholding mask operation proposed in Text Super-Resolution Network (TSRN) by utilizing deep-learning based semantic segmentation method to get correct masks as prior semantic information and propose a Text-Segmented-Contextual-Attention (TSCA) branch on the basis of them. Besides, we design an Orthogonal Contextual Attention Module (OCAM) working with TSCA to implicitly enhance stroke regions of LR images. Secondly, to effectively fuse shallow features and deep features of SR model, we propose a convolutional structure named Weight Balanced Fusion Module (WBFM) to improve traditional feature fusion methods of SR network. Finally, extensive experiments on TextZoom dataset demonstrate that the proposed network can improve the recognition accuracy of text images on existing text recognition models. Using TEAN to process low-resolution text images improves the recognition accuracy by 25.4% on CRNN, by 17.4% on ASTER, by 17.3% on MORAN, by 20.7% on NRTR, by 17.3% on SAR and by 15.9% on MASTER compared with directly recognizing them, which attains competitive performances against state-of-the-art methods. Furthermore, cross-dataset experiments on IC15_2077 demonstrate that TEAN is helpful for scene text recognition task, especially for low-resolution images even with the cross-domain issue.

Practical single-line diagram recognition based on digital image processing and deep vision models

The Single-Line Diagram (SLD) is a standardized graphical representation of power systems. Power engineering software typically requires users to model power systems by drawing SLDs. Although existing computer vision-based SLD recognition approaches facilitate system modeling to some extent, their performance is usually limited by the low resolution of images, the imbalance in size and number of electrical symbols, and the interference between adjacent graphical elements on practical SLDs. This paper proposes a novel SLD recognition framework to address these challenges. First, a text and symbol separation module based on digital image processing is developed to enable independent analysis of text labels and electrical symbols to avoid mutual interference. To accurately recognize electrical symbols of different sizes, the proposed framework integrates the Swin Transformer into the Faster Region-based Convolutional Neural Network (Faster R-CNN), which ultimately achieves a minimum precision of 98.9% and a minimum recall of 98.12%. In addition, the framework modifies the Text Super-Resolution Network (TSRN) to improve text recognition performance on low-resolution images. Combined with the Attentional Scene Text Recognizer with Flexible Rectification (ASTER) model, the proposed text recognition model results in an average accuracy increase of 1% on the TextZoom dataset and 97.1% accuracy in recognizing practical SLD text labels. A synthetic SLD generation method is also proposed to augment the actual SLD datasets to overcome the class imbalance of electrical symbols. The proposed framework significantly improves the performance of existing engineering drawing recognition frameworks. Relevant source code is available online at https://github.com/Lattle-y/work.git.

A Metaverse text recognition model based on character-level contrastive learning

Accurate and efficient text recognition can enhance the accuracy and time efficiency of human–computer interaction and information exchange in noise Metaverse scenarios. For example, it can improve the safety of digital twin-based intelligent transportation and the intelligence of Non-Player Characters. Robust features play a vital role in the performance of scene text recognition models in noise Metaverse situations. To extract robust features, and improve the accuracy and time efficiency of scene text recognition, we propose a Character level text recognition model in Metaverse applications, called MetaChara. It contains two main components: a lightweight text feature extraction module (LightFeature), and a robust character recognition module (RobChara). LightFeature leverages the advantage of global feature aggregation in the primitive representation learning network to handle irregular text images.RobChara incorporates the capability of contrastive learning from the momentum contrast method, improving the robustness of feature extraction in MetaChara. It structures a feature queue for organized storage. By optimizing the similarity of intra-character features and maximizing inter-character differences, it makes the model better adapted to scene text recognition tasks in Metaverse. Experiment results demonstrate that MetaChara is light with 29.14 million parameters and time efficient with an average recognition speed of 1.73 s. It also achieves excellent performance in terms of FLoating-point Operations (FLOPs), registering only 59.60 billion times for each operation. MetaChara achieves an average accuracy of 0.969 for character recognition. We present a case study where MetaChara quickly and accurately recognizes scene texts within the context of autonomous driving in the Metaverse. This demonstrates how MetaChara enhances safety and improves time efficiency for intelligent transportation systems.

Comparing the Performance Characteristics and Commonalities of Chinese Modern and Contemporary Literature in Different Cultural Contexts by Combining Text Mining Techniques

Abstract This paper first investigates the expressive features and developmental commonalities of Chinese modern and contemporary literary works, explains the paradoxical nature, and there are dilemmas in the expressive features by reviewing the data, and proposes the nationalized literary model and the cosmopolitan literary model, aiming to promote the development of Chinese modern and contemporary literature. Then, based on text mining techniques, atomic cuts are performed in the lowest fifth layer using Hidden Markov Models, and rough Chinese word splitting is performed using the n-shortest path method to find out the most likely n ambiguous word splitting results and text clustering analysis based on HMM-LDA topic models is performed on top of the splitting results. The results show that the HMM-LDA text recognition model is more accurate than the HMM text recognition model in terms of recall and accuracy of entity recognition in five categories of Chinese modern and contemporary literature: poetry, prose, drama, novel, and picture. Through this study, people gain a deeper understanding of modern and contemporary literature, gain some insight, and establish a foundation for promoting modern and contemporary literature.

Towards a general open dataset and model for late medieval Castilian text recognition (HTR/OCR)

Submitted to the Journal of Data Mining and Digital Humanities, and accepted. Pending last revisions. Please cite: @article{gille_levenson_2023_towards, author = {Gille Levenson, Matthias}, date = {2023}, journaltitle = {Journal of Data Mining and Digital Humanities}, doi = {10.5281/zenodo.7387376}, editor = {Pinche, Ariane and Stokes, Peter}, issuetitle = {Special Issue: Historical documents and automatic text recognition}, title = {Towards a general open dataset and models for late medieval Castilian text recognition (HTR/OCR)}, note = {Accepted, to be published.} } GILLE LEVENSON , Matthias, « Towards a general open dataset and models for late medieval Castilian text recognition (HTR/OCR) », Journal of Data Mining and Digital Humanities (2023) : Special Issue : Historical documents and automatic text recognition, eds. Ariane PINCHE and Peter STOKES, DOI : 10.5281/zenodo.7387376. Link to the data: https://doi.org/10.5281/zenodo.7386489 Final published version

Unsupervised Domain Adaptation via Class Aggregation for Text Recognition

Cross-domain text recognition is a very challenging task due to the domain drift problem. One solution is aligning feature distributions between domains through Unsupervised Domain Adaptation (UDA). All existing methods perform feature alignment based on the whole image or semantic character features. However, visual character features without contextual semantics also contain much valuable information, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., stroke features of individual characters, which also benefits domain transfer. To this end, we propose a dual intra-Class Aggregation based unsupervised Domain Adaptation method (CADA) for text recognition, which aligns both visual and semantic character feature distributions. To our knowledge, CADA is the first to consider visual character features without contextual semantics in cross-domain text recognition tasks. Accordingly, a Single-head Self-Attention (SSA) mechanism is introduced for extracting visual character features. Thereafter, a dual intra-class aggregation strategy is designed, which performs class aggregations in both visual and semantic spaces. We test the proposed method on widely-used datasets by combining it with representative text recognition models with various decoding methods. Extensive experimental results demonstrate the superiority and generality. Moreover, there is no additional inference time introduced compared to the baselines.

A Bidirectional LSTM approach for written script auto evaluation using keywords-based pattern matching

The evaluation process necessitates significant work in order to effectively and impartially assess the growing number of new subjects and interests in courses. This paper aims at auto-evaluating and setting scores for individuals similar to those provided by humans using deep learning models. This system is built purely to decipher the English characters and numbers from images, convert them into text format, and match the existing written scripts or custom keywords provided by the invigilators to check the answers. The Handwritten Text Recognition (HTR) model fervors and implements an algorithm that is capable of evaluating written scripts based on handwriting and comparing it with the custom keywords provided, whereas the existing models using Convolutional Neural networks (CNN) or Recurrent Neural networks (RNN) suffer from the Vanishing Gradient problem. The core objective of this model is to reduce manual paper checking using Bidirectional Long Short Term Memory (BiLSTM) and CRNN (Convolutional Recurrent Neural Networks). It has been implemented more than the models built on conventional approaches in aspects of performance, efficiency, and better text recognition. The inputs given to the model are in the form of custom keywords; the system processes them through HTR and image processing techniques of segmentation; and the output formats the percentage obtained by the student, word error rate, number of words misspelt, synonyms produced, and the effective outcome. The system has the capability to identify and highlight errors made by students. This feature is advantageous for both students and teachers, as it saves a significant amount of time. Even if the keywords used by students do not align perfectly, the advanced processing models employed by the system possess the intelligence to provide a reasonable number of marks.