Handwritten Chinese Character Recognition Research Articles

Deep Convolutional Neural Networks Based on Knowledge Distillation for Offline Handwritten Chinese Character Recognition

Deep convolutional neural networks (DNNs) have achieved outstanding performance in this field. Meanwhile, handwritten Chinese character recognition (HCCR) is a challenging area of research in the field of computer vision. DNNs require a large number of parameters and high memory consumption. To address these issues, this paper proposes an approach based on an attention mechanism and knowledge distillation. The attention mechanism improves the feature extraction and the knowledge distillation reduces the number of parameters. The experimental results show that ResNet18 achieves a recognition accuracy of 97.63% on the HCCR dataset with 11.25 million parameters. Compared with other methods, this study improves the performance for HCCR.

An approach for handwritten Chinese text recognition unifying character segmentation and recognition

Text line recognition methods can be categorized into explicit segmentation based and implicit segmentation based ones. Explicit segmentation based methods require character-level annotation during training, while implicit segmentation based methods, trained on line-level annotated data, face alignment drift challenges. Though some methods have been proposed to address these challenges using weakly supervised object detection, they often rely on cumbersome pseudo-box generation processes and complex decoding. In this paper, we propose a unified framework to overcome these challenges, achieving high accuracy in text recognition and character segmentation. To eliminate the need of character-level annotated real text line data in training, we introduce a novel training paradigm that utilizes character-level annotated synthetic data and line-level annotated real data jointly. For synthetic data, candidate characters are explicitly aligned with labeled characters to generate hard labels for supervising model training. For real data, implicit alignment is produced by Connectionist Temporal Classification (CTC) mapping to provide soft labels for weakly-supervised model training. And for inference, we propose two decoding strategies leveraging the advantages of Non-Maximum Suppression (NMS) and CTC decoding. Extensive experiments on benchmark datasets demonstrate the superior performance of our method in text recognition and character localization, even with minimal amounts of character-level annotated line data.

A Review of the Current Status of AI Research in Handwritten Chinese Character Recognition

With the continuous development of artificial intelligence technology, there have been attempts to utilize AI in the field of linguistics. In this context, research and development in AI-based Chinese character recognition technology has spanned 40 years, with its outcomes attracting significant attention. However, in the field of Chinese character recognition, the research into offline handwritten recognition technology is particularly challenging due to the unique characteristics of handwritten Chinese characters. The main issue in current research is the significant decrease in accuracy for characters composed of similar components or characters with similar overall shapes. The focus of research solutions has been on modern mechanical programming and other engineering aspects, while research that integrates the inherent characteristics of Chinese characters themselves has been relatively overlooked. The author believes that combining engineering technology research with a deeper understanding of the nature of Chinese characters can solve many of the current problems. This paper reviews the current state of related domestic and international research to identify areas with significant potential for improvement and progress in research, and finally proposes directions for future research.

Online Handwritten Chinese Character Recognition Based on 1-D Convolution and Two-Streams Transformers

Online Handwritten Chinese Character Recognition Based on 1-D Convolution and Two-Streams Transformers

Reconstructed SqueezeNext with C-CBAM for offline handwritten Chinese character recognition.

Handwritten Chinese character recognition (HCCR) is a difficult problem in character recognition. Chinese characters are diverse and many of them are very similar. The HCCR model consumes a large number of computational resources during runtime, making it difficult to deploy to resource-limited development platforms. In order to reduce the computational consumption and improve the operational efficiency of such models, an improved lightweight HCCR model is proposed in this article. We reconstructed the basic modules of the SqueezeNext network so that the model would be compatible with the introduced attention module and model compression techniques. The proposed Cross-stage Convolutional Block Attention Module (C-CBAM) redeploys the Spatial Attention Module (SAM) and the Channel Attention Module (CAM) according to the feature map characteristics of the deep and shallow layers of the model, targeting enhanced information interaction between the deep and shallow layers. The reformulated intra-stage convolutional kernel importance assessment criterion integrates the normalization nature of the weights and allows for structured pruning in equal proportions for each stage of the model. The quantization aware training is able to map the 32-bit floating-point weights in the pruned model to 8-bit fixed-point weights with minor loss. Pruning with the new convolutional kernel importance evaluation criterion proposed in this article can achieve a pruning rate of 50.79% with little impact on the accuracy rate. The various optimization methods can compress the model to 1.06 MB and achieve an accuracy of 97.36% on the CASIA-HWDB dataset. Compared with the initial model, the volume is reduced by 87.15%, and the accuracy is improved by 1.71%. The model proposed in this article greatly reduces the running time and storage requirements of the model while maintaining accuracy.

Offline Handwritten Chinese Character Using Convolutional Neural Network: State-of-the-Art Methods

Given the presence of handwritten documents in human transactions, including email sorting, bank checks, and automating procedures, handwritten characters recognition (HCR) of documents has been invaluable to society. Handwritten Chinese characters (HCC) can be divided into offline and online categories. Online HCC recognition (HCCR) involves the trajectory movement of the pen tip for expressing linguistic content. In contrast, offline HCCR involves analyzing and categorizing the sample binary or grayscale images of characters. As recognition technology develops, academics’ interest in Chinese character recognition has continuously increased, as it significantly affects social and economic development. Recent development in this area is promising. However, the recognition accuracy of offline HCCR is still a sophisticated challenge owing to their complexity and variety of writing styles. With the advancement of deep learning, convolutional neural network (CNN)-based algorithms have demonstrated distinct benefits in offline HCCR and have achieved outstanding results. In this review, we aim to show the different HCCR methods for tackling the complexity and variability of offline HCC writing styles. This paper also reviews different activation functions used in offline HCCR and provides valuable assistance to new researchers in offline Chinese handwriting recognition by providing a succinct study of various methods for recognizing offline HCC.

Deep Learning Inferencing with High-performance Hardware Accelerators

As computer architectures continue to integrate application-specific hardware, it is critical to understand the relative performance of devices for maximum app acceleration. The goal of benchmarking suites, such as MLPerf for analyzing machine learning (ML) hardware performance, is to standardize a fair comparison of different hardware architectures. However, there are many apps that are not well represented by these standards that require different workloads, such as ML models and datasets, to achieve similar goals. Additionally, many apps, like real-time video processing, are focused on latency of computations rather than strictly on throughput. This research analyzes multiple compute architectures that feature ML-specific hardware on a case study of handwritten Chinese character recognition. Specifically, AlexNet and a custom version of GoogLeNet are benchmarked in terms of their streaming latency and maximum throughput for optical character recognition. Considering that these models are composed of fundamental neural network operations yet architecturally different from each other, these models can stress devices in different yet insightful ways that generalizations of the performance of other models can be drawn from. Many devices featuring ML-specific hardware and optimizations are analyzed including Intel and AMD CPUs, Xilinx and Intel FPGAs, NVIDIA GPUs, and Google TPUs. Overall, ML-oriented hardware added to the Intel Xeon CPUs helps to boost throughput by 3.7× and to reduce latency by up to 34.7×, which makes the latency of Intel Xeon CPUs competitive on more parallel models. The TPU devices were limited in terms of throughput due to large data transfer times and not competitive in terms of latency. The FPGA frameworks showcase the lowest latency on the Xilinx Alveo U200 FPGA achieving 0.48 ms on AlexNet using Mipsology Zebra and 0.39 ms on GoogLeNet using Vitis-AI. Through their custom acceleration datapaths coupled with high-performance SRAM, the FPGAs are able to keep critical model data closer to processing elements for lower latency. The massively parallel and high-memory GPU devices with Tensor Core accelerators achieve the best throughput. The NVIDIA Tesla A100 GPU showcases the highest throughput at 42,513 and 52,484 images/second for AlexNet and GoogLeNet, respectively. 1

LW-ViT: The Lightweight Vision Transformer Model Applied in Offline Handwritten Chinese Character Recognition

In recent years, the transformer model has been widely used in computer-vision tasks and has achieved impressive results. Unfortunately, these transformer-based models have the common drawback of having many parameters and a large memory footprint, causing them to be difficult to deploy on mobiles as lightweight convolutional neural networks. To address these issues, a Vision Transformer (ViT) model, named the lightweight Vision Transformer (LW-ViT) model, is proposed to reduce the complexity of the transformer-based model. The model is applied to offline handwritten Chinese character recognition. The design of the LW-ViT model is inspired by MobileViT. The lightweight ViT model reduces the number of parameters and FLOPs by reducing the number of transformer blocks and the MV2 layer based on the overall framework of the MobileViT model. The number of parameters and FLOPs for the LW-ViT model was 0.48 million and 0.22 G, respectively, and it ultimately achieved a high recognition accuracy of 95.8% on the dataset. Furthermore, compared to the MobileViT model, the number of parameters was reduced by 53.8%, and the FLOPs were reduced by 18.5%. The experimental results show that the LW-ViT model has a low number of parameters, proving the correctness and feasibility of the proposed model.

Particle Swarm Optimization-Based Convolutional Neural Network for Handwritten Chinese Character Recognition

Recently, handwritten Chinese character recognition has become an important research field in computer vision. With the development of deep learning, convolutional neural networks (CNNs) have demonstrated excellent performance in computer vision. However, CNNs are typically designed manually, which requires extensive experience and may lead to redundant computations. To solve these problems, in this study, the particle swarm optimization approach is incorporated into the design of a CNN for handwritten Chinese character recognition, reducing redundant computations in the network. In this approach, each network architecture is represented by a particle, and the optimal network architecture is determined by continuously updating the particles until a global particle is identified. The experimental validation resulted in a network accuracy of 97.24% with only 1.43 million network parameters. Therefore, it is demonstrated that the proposed particle swarm optimization method can quickly and accurately find the optimal network architecture.

The Challenges of Recognizing Offline Handwritten Chinese: A Technical Review

Offline handwritten Chinese recognition is an important research area of pattern recognition, including offline handwritten Chinese character recognition (offline HCCR) and offline handwritten Chinese text recognition (offline HCTR), which are closely related to daily life. With new deep learning techniques and the combination with other domain knowledge, offline handwritten Chinese recognition has gained breakthroughs in methods and performance in recent years. However, there have yet to be articles that provide a technical review of this field since 2016. In light of this, this paper reviews the research progress and challenges of offline handwritten Chinese recognition based on traditional techniques, deep learning methods, methods combining deep learning with traditional techniques, and knowledge from other areas from 2016 to 2022. Firstly, it introduces the research background and status of handwritten Chinese recognition, standard datasets, and evaluation metrics. Secondly, a comprehensive summary and analysis of offline HCCR and offline HCTR approaches during the last seven years is provided, along with an explanation of their concepts, specifics, and performances. Finally, the main research problems in this field over the past few years are presented. The challenges still exist in offline handwritten Chinese recognition are discussed, aiming to inspire future research work.

Stroke-Based Autoencoders: Self-Supervised Learners for Efficient Zero-Shot Chinese Character Recognition

Chinese characters carry a wealth of morphological and semantic information; therefore, zero-shot Chinese character recognition with the morphology of Chinese characters has drawn significant attention. The previous methods are mainly based on radical-level decomposition or stroke-level decomposition, which usually cannot capture adequately the structural and spatial information of Chinese characters. In this paper, we develop a stroke-based autoencoder (SAE), to model the sophisticated morphology of Chinese characters with a self-supervised method. Following its canonical writing order, we first represent a Chinese character as a series of stroke images with a fixed writing order, and then our SAE model is trained to reconstruct this stroke image sequence. This pre-trained SAE model can predict the stroke image series for unseen characters, as long as their strokes or radicals are in the training set. We have designed two contrasting SAE architectures on different forms of stroke images. One is fine-tuned on existing stroke-based method for zero-shot recognition of handwritten Chinese characters, and the other is applied to enrich the Chinese word embeddings from their morphological features. The experimental results validate that after pre-training, our SAE architecture outperforms other existing methods in zero-shot recognition and enhances the representation of Chinese characters with their abundant morphological and semantic information.

Characters as graphs: Interpretable handwritten Chinese character recognition via Pyramid Graph Transformer

It is meaningful but challenging to teach machines to recognize handwritten Chinese characters. However, conventional approaches typically view handwritten Chinese characters as either static images or temporal trajectories, which may ignore the inherent geometric semantics of characters. Instead, here we first propose to represent handwritten characters as skeleton graphs, explicitly considering the natural characteristics of characters (i.e., characters as graphs). Furthermore, we propose a novel Pyramid Graph Transformer (PyGT) to specifically process the graph-structured characters, which fully integrates the advantages of Transformers and graph convolutional networks. Specifically, our PyGT can learn better graph features through (i) capturing the global information from all nodes with graph attention mechanism and (ii) modelling the explicit local adjacency structures of nodes with graph convolutions. Furthermore, the PyGT learns the multi-resolution features by constructing a progressive shrinking pyramid. Compared with existing approaches, it is more interpretable to recognize characters as geometric graphs. Moreover, the proposed method is generic for both online and offline handwritten Chinese character recognition (HCCR), and it also can be feasibly extended to handwritten text recognition. Extensive experiments empirically demonstrate the superiority of PyGT over the prevalent approaches including 2D-CNN, RNN/1D-CNN, and Vision Transformer (ViT) for HCCR. The code is available at https://github.com/ganji15/PyGT-HCCR.

A novel multilevel stacked SqueezeNet model for handwritten Chinese character recognition

To solve the problems of large number of similar Chinese characters, difficult feature extraction and inaccurate recognition, we propose a novel multilevel stacked SqueezeNet model for handwritten Chinese character recognition. First, we design a deep convolutional neural network model for feature grouping extraction and fusion. The multilevel stacked feature group extraction module is used to extract the deep abstract feature information of the image and carry out the fusion between the different feature information modules. Secondly, we use the designed down-sampling and channel amplification modules to reduce the feature dimension while preserving the important information of the image. The feature information is refined and condensed to solve the overlapping and redundant problem of feature information. Thirdly, inter-layer feature fusion algorithm and Softmax classification function constrained by L2 norm are used. We further compress the parameter clipping to avoid the loss of too much accuracy due to the clipping of important parameters. The dynamic network surgery algorithm is used to ensure that the important parameters of the error deletion are reassembled. Experimental results on public data show that the designed recognition model in this paper can effectively improve the recognition rate of handwritten Chinese characters.

Fast and Robust Online Handwritten Chinese Character Recognition With Deep Spatial and Contextual Information Fusion Network

Deep convolutional neural networks have achieved fairly high accuracy for single online handwritten Chinese character recognition (SOLHCCR). However, in real application scenarios, users always write multiple characters to form a complete sentence, and previous contextual information holds significant potential for improving the accuracy, robustness and efficiency of recognition. In this work, we first propose a simple and straightforward model named the vanilla compositional network (VCN) by coupling convolutional neural network with a sequence modeling architecture (i.e., a recurrent neural network or Transformer), which exploits the handwritten character's previous contextual information. Although VCN performs much better than the previous state-of-the-art SOLHCCR models, it is a two-stage architecture in nature. It suffers from high fragility when confronting with poorly written characters such as sloppy writing, and missing or broken strokes, due to relying heavily on contextual information. To improve the robustness of the OLHCCR model, we further propose a novel deep spatial & contextual information fusion network (DSCIFN). It utilizes an autoregresssive framework pre-trained on a large-scale sentence corpora as the backbone component, and highly integrates the spatial features of handwritten characters and their previous contextual information in a multi-layer fusion module. To verify the effectiveness of models, we reorganize a new form of online Chinese handwritten character with its previous context dataset, named OHCCC. Extensive experimental results demonstrate that DSCIFN achieves state-of-the-art performance and has increased strong robustness compared to VCN and previous SOLHCCR models. The in-depth empirical analysis and case study indicate that DSCIFN can significantly improve the efficiency of handwriting input because it does not need complete strokes to recognize a handwritten Chinese character precisely.

PF-ViT: Parallel and Fast Vision Transformer for Offline Handwritten Chinese Character Recognition.

Recently, Vision Transformer (ViT) has been widely used in the field of image recognition. Unfortunately, the ViT model repeatedly stacks 12-layer encoders, resulting in a large number of model computations, many parameters, and slow training speed, making it difficult to deploy on mobile devices. In order to reduce the computational complexity of the model and improve the training speed, a parallel and fast Vision Transformer method for offline handwritten Chinese character recognition is proposed. The method adds parallel branches of the encoder module to the structure of the Vision Transformer model. Parallel modes include two-way parallel, four-way parallel, and seven-way parallel. The original picture is fed to the encoder module after flattening and linear embedding processing operations. The core step in the encoder is the multihead attention mechanism. Multihead self-attention can learn the interdependence between image sequence blocks. In addition, the use of data expansion strategies increases the diversity of data. In the two-way parallel experiment, when the model is 98.1% accurate on the dataset, the number of parameters and the number of FLOPs are 43.11 million and 4.32 G, respectively. Compared with the ViT model, whose parameters and FLOPs are 86 million and 16.8 G, respectively, the two-way parallel model has a 50.1% decrease in parameters and a 34.6% decrease in FLOPs. This method has been demonstrated to effectively reduce the computational complexity of the model while indirectly improving image recognition speed.

S-Swin Transformer: simplified Swin Transformer model for offline handwritten Chinese character recognition.

The Transformer shows good prospects in computer vision. However, the Swin Transformer model has the disadvantage of a large number of parameters and high computational effort. To effectively solve these problems of the model, a simplified Swin Transformer (S-Swin Transformer) model was proposed in this article for handwritten Chinese character recognition. The model simplifies the initial four hierarchical stages into three hierarchical stages. In addition, the new model increases the size of the window in the window attention; the number of patches in the window is larger; and the perceptual field of the window is increased. As the network model deepens, the size of patches becomes larger, and the perceived range of each patch increases. Meanwhile, the purpose of shifting the window’s attention is to enhance the information interaction between the window and the window. Experimental results show that the verification accuracy improves slightly as the window becomes larger. The best validation accuracy of the simplified Swin Transformer model on the dataset reached 95.70%. The number of parameters is only 8.69 million, and FLOPs are 2.90G, which greatly reduces the number of parameters and computation of the model and proves the correctness and validity of the proposed model.

Evaluation and Recognition of Handwritten Chinese Characters Based on Similarities

To accurately recognize ordinary handwritten Chinese characters, it is necessary to recognize the normative level of these characters. This study proposes methods to quantitatively evaluate and recognize these characters based on their similarities. Three different types of similarities, including correlation coefficient, pixel coincidence degree, and cosine similarity, are calculated between handwritten and printed Song typeface Chinese characters. Eight features are derived from the similarities and used to verify the evaluation performance and an artificial neural network is used to recognize the character content. The results demonstrate that our proposed methods deliver satisfactory evaluation effectiveness and recognition accuracy (up to 98%~100%). This indicates that it is possible to improve the accuracy in recognition of ordinary handwritten Chinese characters by evaluating the normative level of these characters and standardizing writing actions in advance. Our study can offer some enlightenment for developing methods for the identification of handwritten Chinese characters used in transaction processing activities.

Research on Handwritten Chinese Character Recognition Based on BP Neural Network

The application of pattern recognition technology enables us to solve various human-computer interaction problems that were difficult to solve before. Handwritten Chinese character recognition, as a hot research object in image pattern recognition, has many applications in people’s daily life, and more and more scholars are beginning to study off-line handwritten Chinese character recognition. This paper mainly studies the recognition of handwritten Chinese characters by BP (Back Propagation) neural network. Establish a handwritten Chinese character recognition model based on BP neural network, and then verify the accuracy and feasibility of the neural network through GUI (Graphical User Interface) model established by Matlab. This paper mainly includes the following aspects: Firstly, the preprocessing process of handwritten Chinese character recognition in this paper is analyzed. Among them, image preprocessing mainly includes six processes: graying, binarization, smoothing and denoising, character segmentation, histogram equalization and normalization. Secondly, through the comparative selection of feature extraction methods for handwritten Chinese characters, and through the comparative analysis of the results of three different feature extraction methods, the most suitable feature extraction method for this paper is found. Finally, it is the application of BP neural network in handwritten Chinese character recognition. The establishment, training process and parameter selection of BP neural network are described in detail. The simulation software platform chosen in this paper is Matlab, and the sample images are used to train BP neural network to verify the feasibility of Chinese character recognition. Design the GUI interface of human-computer interaction based on Matlab, show the process and results of handwritten Chinese character recognition, and analyze the experimental results.

Cross-modal prototype learning for zero-shot handwritten character recognition

Traditional methods of handwritten character recognition rely on extensive labeled data. However, humans can generalize to unseen handwritten characters by watching a few printed examples in textbooks. To simulate this ability, we propose a cross-modal prototype learning method (CMPL) to realize zero-shot recognition. For each character class, a prototype is generated by mapping the printed character into a deep neural network feature space. For unseen character class, its prototype can be directly produced from a printed character sample, therefore, not requiring any handwritten samples to realize class-incremental learning. Specifically, CMPL considers different modalities simultaneously - online handwritten trajectories, offline handwritten images, and auxiliary printed character images. The joint learning of the above modalities is achieved through sharing printed prototypes between online and offline data. In zero-shot inference, we feed CMPL the printed samples to obtain corresponding class prototypes, and then the unseen handwritten character can be recognized by the nearest prototype. Our experimental results demonstrate that CMPL outperforms the state-of-the-art methods in both online and offline zero-shot handwritten Chinese character recognition. Moreover, we also show the cross-domain generalization of CMPL from two perspectives: cross-language and modern-to-ancient handwritten character recognition, focusing on the transferability between different languages and different styles (i.e., modern and historical handwritings).

Adversarial Attack and Defence on Handwritten Chinese Character Recognition

Deep Neural Networks (DNNs) have shown their powerful performance in classification; however, the robustness issue of DNNs has arisen as one primary concern, e.g., adversarial attack. So far as we know, there is not any reported work about the adversarial attack on handwritten Chinese character recognition (HCCR). To this end, the classical adversarial attack method (i.e., Projection Gradient Descent: PGD) is adopted to generate adversarial examples to evaluate the robustness of the HCCR model. Furthermore, in the training process, we use adversarial examples to improve the robustness of the HCCR model. In the experiments, we utilize a frequently-used DNN model on HCCR and evaluate its robustness on the benchmark dataset CASIA-HWDB. The experimental results show that its recognition accuracy is decreased severely on the adversarial examples, demonstrating the vulnerability of the current HCCR model. In addition, we can improve the recognition accuracy significantly after the adversarial training, demonstrating its effectiveness.