Text Recognition Tasks Research Articles

MAGIC: Multi-granularity domain adaptation for text recognition

Domain gaps between synthetic text and real-world text restrict current text recognition methods. One solution is to align features through Unsupervised Domain Adaptation (UDA). Most existing UDA-based text recognition methods extract global and local features to alleviate domain differences, only focusing on character-level distribution gaps. However, notable distribution gaps in character combinations exert a pivotal influence on diverse text recognition tasks. To this end, we propose a Multi-level And multi-Granularity domain adaptation with entropy loss guIded text reCognition model, named MAGIC. It integrates Global-level Domain Adaptation (GDA) to mitigate image-level domain drift and Local-level Multi-granularity Domain Adaptation (LMDA) for local feature shifts. Particularly, we design a subword-level domain discriminator to align the subword features relating to each character combination. Moreover, multi-granularity entropy minimization is used to optimize the target domain data for better domain adaptation. Experimental results on several types of text datasets demonstrate the effectiveness of MAGIC.

DPGS: Cross-cooperation guided dynamic points generation for scene text spotting

End-to-end text spotting aims to combine scene text detection and recognition into a unified framework. Dealing with the relationship between the two sub-tasks plays a pivotal role in designing effective spotters. While polygon or segmentation-based methods eliminate heuristic post-processing, they still face challenges such as background noise and high computational burden. In this study, we introduce DPGS, a coarse-to-fine learning framework that lets Dynamic Points Generation for text Spotting. DPGS simultaneously learns character representations for both detection and recognition tasks. Specifically, for each text instance, we represent the character sequence as ordered points and model them with learnable point queries. This approach progressively selects appropriate key points covering character and leverages group attention to associate similar information from different positions, improving detection accuracy. After passing through a single decoder, the point queries encode text semantics and locations, facilitating decoding to central line, boundary, script, and confidence of text through simple prediction heads. Additionally, we introduce an adaptive cooperative criterion to combine more useful feature knowledge, enhancing training efficiency. Extensive experiments show the superiority of our DPGS when handling scene text detection and recognition tasks. Compared to their respective top-1 methods, DPGS has significantly improved the average recognition accuracy by 3.7%, 1.9%, and 0.7% on the Total-Text, ICDAR15, and CTW1500 datasets, respectively.

Trademark Text Recognition Combining SwinTransformer and Feature-Query Mechanisms

The task of trademark text recognition is a fundamental component of scene text recognition (STR), which currently faces a number of challenges, including the presence of unordered, irregular or curved text, as well as text that is distorted or rotated. In applications such as trademark infringement detection and analysis of brand effects, the diversification of artistic fonts in trademarks and the complexity of the product surfaces where the trademarks are located pose major challenges for relevant research. To tackle these issues, this paper proposes a novel recognition framework named SwinCornerTR, which aims to enhance the accuracy and robustness of trademark text recognition. Firstly, a novel feature-extraction network based on SwinTransformer with EFPN (enhanced feature pyramid network) is proposed. By incorporating SwinTransformer as the backbone, efficient capture of global information in trademark images is achieved through the self-attention mechanism and enhanced feature pyramid module, providing more accurate and expressive feature representations for subsequent text extraction. Then, during the encoding stage, a novel feature point-retrieval algorithm based on corner detection is designed. The OTSU-based fast corner detector is presented to generate a corner map, achieving efficient and accurate corner detection. Furthermore, in the encoding phase, a feature point-retrieval mechanism based on corner detection is introduced to achieve priority selection of key-point regions, eliminating character-to-character lines and suppressing background interference. Finally, we conducted extensive experiments on two open-access benchmark datasets, SVT and CUTE80, as well as a self-constructed trademark dataset, to assess the effectiveness of the proposed method. Our results showed that the proposed method achieved accuracies of 92.9%, 92.3% and 84.8%, respectively, on these datasets. These results demonstrate the effectiveness and robustness of the proposed method in the analysis of trademark data.

A Method for Analyzing the Operating Data of Electric Energy Meters Based on Data Mining Analysis

Aiming at the problem of error estimation of smart meters in distribution network, a method of error estimation of smart meters based on particle swarm optimization convolutional neural network is proposed. This method establishes an intelligent energy meter error estimation model through data collection, data prediction, and preprocessing. To address the convergence issue in training, the interlayer distribution of weights is adjusted to improve training quality. This method fully utilizes template calibration information to transform indicator detection under complex conditions into simple and effective isometric segmentation, transforming label recognition from complex text detection and recognition tasks to simple and efficient binary detection tasks, with better robustness. The effectiveness and high robustness of the proposed method have been demonstrated through experimental verification.

SARN: Script-Aware Recognition Network for scene multilingual text recognition

Recent research in script-specific text recognition has achieved remarkable performance on scene text images. However, the challenges of multilingual text in real-world images have not been effectively addressed, thereby limiting the practical application. For multilingual text recognition, the prevailing single-head scheme that uses the same recognition head for all scripts suffers from the issues of enormous alphabet size and linguistic knowledge discrepancies. Meanwhile, the multi-head scheme, which assigns distinct heads for individual scripts, encounters error accumulation within the two-step inference process. In this paper, we explore the influence of script attribute information within existing methods on achieving accurate recognition. Correspondingly, we propose a script-aware recognition network encompassing script identification and text recognition, which leverages script information to make character features more discriminative in text recognition. In particular, we design the script-aware module to utilize script information at the local level, global level, and their combination, respectively. Furthermore, our method is extended to the multi-head scheme, utilizing mutual assistance between script and character features to improve the performance in both script identification and text recognition tasks. Extensive experiments demonstrate that our method obtains superior results in multilingual text recognition, and reveal the importance of script information in alleviating the issues of single-head and multi-head recognition scheme.

CNN-optimized text recognition with binary embeddings for Arabic expiry date recognition

Recognizing Arabic dot-matrix digits is a challenging problem due to the unique characteristics of dot-matrix fonts, such as irregular dot spacing and varying dot sizes. This paper presents an approach for recognizing Arabic digits printed in dot matrix format. The proposed model is based on convolutional neural networks (CNN) that take the dot matrix as input and generate embeddings that are rounded to generate binary representations of the digits. The binary embeddings are then used to perform Optical Character Recognition (OCR) on the date images. To overcome the challenge of the limited availability of dotted Arabic expiration date images, we developed a True Type Font (TTF) for generating synthetic images of Arabic dot-matrix characters. The model was trained on a synthetic dataset of 3287 images and 658 synthetic images for testing, representing realistic expiration dates from 2019 to 2027 in the format of yyyy/mm/dd and yy/mm/dd. Our model achieved an accuracy of 98.94% on the expiry date recognition with Arabic dot matrix format using fewer parameters and less computational resources than traditional CNN-based models. By investigating and presenting our findings comprehensively, we aim to contribute substantially to the field of OCR and pave the way for advancements in Arabic dot-matrix character recognition. Our proposed approach is not limited to Arabic dot matrix digit recognition but can be also extended to text recognition tasks, such as text classification and sentiment analysis.

CFOR: Character-First Open-Set Text Recognition via Context-Free Learning.

The open-set text recognition task is a generalized form of the (close-set) text recognition task, where the model is further challenged to spot and incrementally recognize novel characters in new languages, which also indicate a shift in the language model. In this work, we propose to alleviate the confounding effect such biases under an open-set setup by learning individual character representations out of their context.We propose a Character-First Open-Set Text Recognition framework that treats individual characters as first-class citizens by cotraining the feature extractor with two context-free learning tasks. First, a Context Isolation Learning task is proposed to wipe the context for each character, utilizing a character mask learned in a weak supervision manner. Second, the framework adopts an Individual Character Learning task, which is a single-character classification task with synthetic samples. Our framework can reliably spot unseen characters in Japanese with F1-score over 64%, and can adapt to recognize unseen characters in Japanese, Korean, Greek, and other languages without retraining. The framework also shows decent many-shot performance on close-set text benchmarks with 91.5% line accuracy on IIIT5k and a speed of over 69 FPS single-batched, making it a feasible universal lightweight OCR solution.

An End-to-End Tag Recognition Architecture for Industrial Meter

Optical character recognition (OCR) technology is promoting the process of automation in daily information recording and inspection of industrial meters. However, bad installation scenes and long-time running cause industrial meters to age to varying degrees, which increases the difficulty of OCR. Besides, in practice, operators typically use handheld cameras to extract tag information from industrial meters. During the shooting process, extreme lighting, free shooting angles, and shooting distance also cause many difficulties for OCR. Considering such difficulties, an end-to-end recognition architecture is developed to obtain a better OCR performance. The proposed architecture can quickly extract structured information from normal or skewed text images. A novel yolov5_adaloss with a specific penalty factor is designed to alleviate the influence of illumination, installation scene, age, skew, and distance on the classification accuracy of Tags. For images with large skew angles, a mathematical method is proposed to calculate the text inclination angle for better OCR performance. The contribution of this work is twofold. Firstly, the architecture proposed is lightweight, which only needs a low computing cost and a short inference time. Secondly, as a practical application-oriented architecture, this work does not require much training, labeling, and fine-tuning, which is easy to generalize to other structured text recognition tasks. Experiments show that the method proposed in this paper can achieve excellent performance in meter tag recognition tasks on actual industrial images of meters.

UIT-MLReceipts: A Multilingual Benchmark for Detecting and Recognizing Key Information in Receipts

The 4.0 industrial evolution has paved the way for development potential and revolution in Vietnam. In this movement, digitization appears to be necessary to transform numerous traditional economic sectors. It will provide valuable digital data for many automation applications and decision-making processes. Particularly in the retail industry, data has long played a vital factor. Hence, digitizing documents such as receipts can help businesses in management and enterprise development. Nevertheless, the digital transformation process is still slow because of the shortage of cleaned datasets for this type of document. This paper introduces a new dataset named UIT-MLReceipts for extracting key information in receipts. The task includes two sub-tasks: Receipt Text Detection (RTD) and Receipt Text Recognition (RTR). We thoroughly evaluate current state-of-the-art Receipt Text Detection using Faster R-CNN, YOLOv3, YOLOF, and Faster R-CNN with Precise RoI-Pooling on our dataset. To evaluate the performance of Receipt Text Recognition, we experiment with two text recognition baselines: RobustScanner and SATRN. Experimental results indicate that the Faster R-CNN with Precise RoI-Pooling outperforms the competitors and achieves the best mean Average Precision (mAP) score at 51.6% in the Receipt Text Detection task. With the Receipt Text Recognition task, results show that SATRN performs better.

FedBoosting: Federated learning with gradient protected boosting for text recognition

Conventional machine learning methodologies require the centralization of data for model training, which may be infeasible in situations where data sharing limitations are imposed due to concerns such as privacy and gradient protection. The Federated Learning (FL) framework enables the collaborative learning of a shared model without necessitating the centralization or sharing of data among the data proprietors. Nonetheless, in this paper, we demonstrate that the generalization capability of the joint model is suboptimal for Non-Independent and Non-Identically Distributed (Non-IID) data, particularly when employing the Federated Averaging (FedAvg) strategy as a result of the weight divergence phenomenon. Consequently, we present a novel boosting algorithm for FL to address both the generalization and gradient leakage challenges, as well as to facilitate accelerated convergence in gradient-based optimization. Furthermore, we introduce a secure gradient sharing protocol that incorporates Homomorphic Encryption (HE) and Differential Privacy (DP) to safeguard against gradient leakage attacks. Our empirical evaluation demonstrates that the proposed Federated Boosting (FedBoosting) technique yields significant enhancements in both prediction accuracy and computational efficiency in the visual text recognition task on publicly available benchmarks.

CommuSpotter: Scene Text Spotting with Multi-Task Communication

Scene text spotting is a challenging multi-task modulation for locating and recognizing texts in complex scenes. Existing end-to-end text spotters generally adopt sequentially decoupled multi-tasks, consisting of text detection and text recognition modules. Although customized modules are designed to connect the tasks closely, there is no interaction among multiple tasks, resulting in compatible information loss for the overall text spotting. Moreover, the independent and sequential modulation is unidirectional, accumulating errors from early to later tasks. In this paper, we propose CommuSpotter, which enhances multi-task communication by explicitly and concurrently sharing compatible information in overall scene text spotting. To address task-specific inconsistencies, we propose a Conversation Mechanism (CM) to extract and exchange expertise in each specific task with others. Specifically, the detection task is rectified by the text recognition task to filter out duplicated results and false positives, while the text recognition task is corrected by the rectified text detection task to replenish missing characters and decrease non-text interruptions. Consequently, the communication compensates for interaction information and breaks the sequential pipeline of error propagation. In addition, we adopt text semantic segmentation in the text recognition task, which reduces the complex design of customized modules and corresponding extra annotations. Compared with state-of-the-art methods, experimental results show that our method achieves competitive results with computation efficiency.

Optimization of Image Processing Algorithms for Character Recognition in Cultural Typewritten Documents

Linked data is used in various fields as a new way of structuring and connecting data. Cultural heritage institutions have been using linked data to improve archival descriptions and facilitate the discovery of information. Most archival records have digital representations of physical artifacts in the form of scanned images that are non-machine-readable. Optical Character Recognition (OCR) recognizes text in images and translates it into machine-encoded text. This article evaluates the impact of image processing methods and parameter tuning in OCR applied to typewritten cultural heritage documents. The approach uses a multi-objective problem formulation to minimize Levenshtein edit distance and maximize the number of words correctly identified with a non-dominated sorting genetic algorithm (NSGA-II) to tune the methods’ parameters. Evaluation results show that parameterization by digital representation typology benefits the performance of image pre-processing algorithms in OCR. Furthermore, our findings suggest that employing image pre-processing algorithms in OCR might be more suitable for typologies where the text recognition task without pre-processing does not produce good results. In particular, Adaptive Thresholding, Bilateral Filter, and Opening are the best-performing algorithms for the theater plays’ covers, letters, and overall dataset, respectively, and should be applied before OCR to improve its performance.

Text-Enhanced Scene Image Super-Resolution via Stroke Mask and Orthogonal Attention

Low-resolution text images are very commonplace in real life and their information is hard to be extracted by using existing text recognition methods only. Although this problem can be solved by introducing super-resolution (SR) techniques, most existing SR methods fail to process stroke regions and background regions of input text images distinctively. In this paper, we propose a text-specific super-resolution network named Text Enhanced Attention Network (TEAN) to solve this problem. First of all, we compensate for disadvantages of traditional thresholding mask operation proposed in Text Super-Resolution Network (TSRN) by utilizing deep-learning based semantic segmentation method to get correct masks as prior semantic information and propose a Text-Segmented-Contextual-Attention (TSCA) branch on the basis of them. Besides, we design an Orthogonal Contextual Attention Module (OCAM) working with TSCA to implicitly enhance stroke regions of LR images. Secondly, to effectively fuse shallow features and deep features of SR model, we propose a convolutional structure named Weight Balanced Fusion Module (WBFM) to improve traditional feature fusion methods of SR network. Finally, extensive experiments on TextZoom dataset demonstrate that the proposed network can improve the recognition accuracy of text images on existing text recognition models. Using TEAN to process low-resolution text images improves the recognition accuracy by 25.4% on CRNN, by 17.4% on ASTER, by 17.3% on MORAN, by 20.7% on NRTR, by 17.3% on SAR and by 15.9% on MASTER compared with directly recognizing them, which attains competitive performances against state-of-the-art methods. Furthermore, cross-dataset experiments on IC15_2077 demonstrate that TEAN is helpful for scene text recognition task, especially for low-resolution images even with the cross-domain issue.

A Metaverse text recognition model based on character-level contrastive learning

Accurate and efficient text recognition can enhance the accuracy and time efficiency of human–computer interaction and information exchange in noise Metaverse scenarios. For example, it can improve the safety of digital twin-based intelligent transportation and the intelligence of Non-Player Characters. Robust features play a vital role in the performance of scene text recognition models in noise Metaverse situations. To extract robust features, and improve the accuracy and time efficiency of scene text recognition, we propose a Character level text recognition model in Metaverse applications, called MetaChara. It contains two main components: a lightweight text feature extraction module (LightFeature), and a robust character recognition module (RobChara). LightFeature leverages the advantage of global feature aggregation in the primitive representation learning network to handle irregular text images.RobChara incorporates the capability of contrastive learning from the momentum contrast method, improving the robustness of feature extraction in MetaChara. It structures a feature queue for organized storage. By optimizing the similarity of intra-character features and maximizing inter-character differences, it makes the model better adapted to scene text recognition tasks in Metaverse. Experiment results demonstrate that MetaChara is light with 29.14 million parameters and time efficient with an average recognition speed of 1.73 s. It also achieves excellent performance in terms of FLoating-point Operations (FLOPs), registering only 59.60 billion times for each operation. MetaChara achieves an average accuracy of 0.969 for character recognition. We present a case study where MetaChara quickly and accurately recognizes scene texts within the context of autonomous driving in the Metaverse. This demonstrates how MetaChara enhances safety and improves time efficiency for intelligent transportation systems.

Unsupervised Domain Adaptation via Class Aggregation for Text Recognition

Cross-domain text recognition is a very challenging task due to the domain drift problem. One solution is aligning feature distributions between domains through Unsupervised Domain Adaptation (UDA). All existing methods perform feature alignment based on the whole image or semantic character features. However, visual character features without contextual semantics also contain much valuable information, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., stroke features of individual characters, which also benefits domain transfer. To this end, we propose a dual intra-Class Aggregation based unsupervised Domain Adaptation method (CADA) for text recognition, which aligns both visual and semantic character feature distributions. To our knowledge, CADA is the first to consider visual character features without contextual semantics in cross-domain text recognition tasks. Accordingly, a Single-head Self-Attention (SSA) mechanism is introduced for extracting visual character features. Thereafter, a dual intra-class aggregation strategy is designed, which performs class aggregations in both visual and semantic spaces. We test the proposed method on widely-used datasets by combining it with representative text recognition models with various decoding methods. Extensive experimental results demonstrate the superiority and generality. Moreover, there is no additional inference time introduced compared to the baselines.

Text detection and recognition through deep learning-based fusion neural network

&lt;p&gt;Text recognition task involves recognizing the text from the natural image; it possesses various application, which aids information extraction through data mining from street view like images. Scene text recognition involves two stages i.e., text detection and text recognition, in the past several mechanisms has been proposed for accurate identification, these mechanisms are either traditional approach or deep learning-based. All the existing deep-learning methodology fails as this comprises character data and image data, further this research develops an optimal architecture fusion neural network (FNN) for text identification and recognition. FNN comprises several layers of convolutional neural network as well as recurrent neural network. Within FNN architecture convolutional layer is utilized for the feature extraction and recurrent layer is utilized for attaining the feature classification prediction. Further, an optimal training architecture is established for the enhancement of classification accuracy. Here Devanagari MLT-19 dataset is utilized for the evaluation of FNN. Three different parameters are considered during evaluation i.e., script word identification, character recognition rate (CRR) and word recognition rate (WRR). Further comparison with existing models is performed to establish the proposed model efficiency and it shows FNN methodology observes 98.67% of script identification accuracy, 84.65% of WRR and 92.93% of CRR.&lt;/p&gt;

Advancements in OCR: A Deep Learning Algorithm for Enhanced Text Recognition

Optical Character Recognition (OCR) has significantly evolved with the rise of deep learning techniques. In this research paper, we present a novel and advanced OCR algorithm that leverages the power of deep learning for improved text recognition accuracy. Traditional OCR methods have faced limitations in handling complex layouts, noisy images, and diverse fonts, affecting overall performance. Our proposed algorithm addresses these challenges through the integration of deep neural networks, specifically convolutional and recurrent layers. The algorithm undergoes comprehensive training on large-scale datasets, enabling it to learn intricate patterns and features, resulting in robust recognition capabilities. Furthermore, we introduce an attention mechanism that enhances the model's ability to focus on critical text regions, enhancing accuracy and efficiency. Through extensive experiments and evaluations on benchmark datasets, we demonstrate the superiority of our deep learning-based OCR algorithm over conventional approaches. Our algorithm achieves state-of-the-art performance on various OCR tasks, including multilingual text recognition and document digitization. Additionally, we conduct an in-depth analysis of the algorithm's behaviour under various scenarios, such as low-resolution inputs and challenging environmental conditions. The findings from this research not only contribute to the field of OCR but also open avenues for applications in document analysis, text extraction, and content digitization in real-world scenarios. The integration of deep learning in OCR showcases its potential in revolutionising text recognition tasks, pushing the boundaries of accuracy and efficiency in this domain.

Dynamic Fall Detection Using Graph-Based Spatial Temporal Convolution and Attention Network

The prevention of falls has become crucial in the modern healthcare domain and in society for improving ageing and supporting the daily activities of older people. Falling is mainly related to age and health problems such as muscle, cardiovascular, and locomotive syndrome weakness, etc. Among elderly people, the number of falls is increasing every year, and they can become life-threatening if detected too late. Most of the time, ageing people consume prescription medication after a fall and, in the Japanese community, the prevention of suicide attempts due to taking an overdose is urgent. Many researchers have been working to develop fall detection systems to observe and notify about falls in real-time using handcrafted features and machine learning approaches. Existing methods may face difficulties in achieving a satisfactory performance, such as limited robustness and generality, high computational complexity, light illuminations, data orientation, and camera view issues. We proposed a graph-based spatial-temporal convolutional and attention neural network (GSTCAN) with an attention model to overcome the current challenges and develop an advanced medical technology system. The spatial-temporal convolutional system has recently proven the power of its efficiency and effectiveness in various fields such as human activity recognition and text recognition tasks. In the procedure, we first calculated the motion along the consecutive frame, then constructed a graph and applied a graph-based spatial and temporal convolutional neural network to extract spatial and temporal contextual relationships among the joints. Then, an attention module selected channel-wise effective features. In the same procedure, we repeat it six times as a GSTCAN and then fed the spatial-temporal features to the network. Finally, we applied a softmax function as a classifier and achieved high accuracies of 99.93%, 99.74%, and 99.12% for ImViA, UR-Fall, and FDD datasets, respectively. The high-performance accuracy with three datasets proved the proposed system’s superiority, efficiency, and generality.

A Non-Intrusive Automated Testing System for Internet of Vehicles App Based on Deep Learning

In the non-intrusive automated testing system for Internet of Vehicles (IoV) applications, automatic recognition of text and icons on vehicle central control screens is of paramount importance. However, the detection and recognition of content on vehicle central control screens are inherently complex. Additionally, during non-intrusive vehicle central control screen image testing, there is a deficiency of suitable datasets and detection methods. This deficiency renders information within vehicle application images difficult to be accurately extracted by the detection network. To address this problem, this study first constructs a dataset tailored for text detection and recognition on vehicle screens. This dataset encompasses a variety of vehicle central control images, enabling the generic text detection and recognition network to more effectively identify and interpret text within vehicle screens. Subsequently, this research proposes an enhanced Fully Convolutional Networks for Text Detection (FOTS) method for vehicle central control screen text detection and recognition. This method elevates the semantic expression capabilities of features by sharing vehicle central control screen text detection and recognition features. Furthermore, it improves multi-scale feature processing capabilities through the utilization of a feature transformation module. Validation through visual and quantitative experiments demonstrates that the proposed method can effectively accomplish text detection and recognition tasks on vehicle screens. This achievement bears significant implications for the field of automated testing in IoV applications.

Scene Text Recognition Based on Improved CRNN

Text recognition is an important research topic in computer vision. Scene text, which refers to the text in real scenes, sometimes needs to meet the requirement of attracting attention, and there is the situation such as deformation. At the same time, the image acquisition process is affected by factors such as occlusion, noise, and obstruction, making scene text recognition tasks more challenging. In this paper, we improve the CRNN model for text recognition, which has relatively low accuracy, poor performance in recognizing irregular text, and only considers obtaining text sequence information from a single aspect, resulting in incomplete information acquisition. Firstly, to address the problems of low text recognition accuracy and poor recognition of irregular text, we add label smoothing to ensure the model’s generalization ability. Then, we introduce the smoothing loss function from speech recognition into the field of text recognition, and add a language model to increase information acquisition channels, ultimately achieving the goal of improving text recognition accuracy. This method was experimentally verified on six public datasets and compared with other advanced methods. The experimental results show that this method performs well in most benchmark tests, and the improved model outperforms the original model in recognition performance.