A Metaverse text recognition model based on character-level contrastive learning

Abstract

Accurate and efficient text recognition can enhance the accuracy and time efficiency of human–computer interaction and information exchange in noise Metaverse scenarios. For example, it can improve the safety of digital twin-based intelligent transportation and the intelligence of Non-Player Characters. Robust features play a vital role in the performance of scene text recognition models in noise Metaverse situations. To extract robust features, and improve the accuracy and time efficiency of scene text recognition, we propose a Character level text recognition model in Metaverse applications, called MetaChara. It contains two main components: a lightweight text feature extraction module (LightFeature), and a robust character recognition module (RobChara). LightFeature leverages the advantage of global feature aggregation in the primitive representation learning network to handle irregular text images.RobChara incorporates the capability of contrastive learning from the momentum contrast method, improving the robustness of feature extraction in MetaChara. It structures a feature queue for organized storage. By optimizing the similarity of intra-character features and maximizing inter-character differences, it makes the model better adapted to scene text recognition tasks in Metaverse. Experiment results demonstrate that MetaChara is light with 29.14 million parameters and time efficient with an average recognition speed of 1.73 s. It also achieves excellent performance in terms of FLoating-point Operations (FLOPs), registering only 59.60 billion times for each operation. MetaChara achieves an average accuracy of 0.969 for character recognition. We present a case study where MetaChara quickly and accurately recognizes scene texts within the context of autonomous driving in the Metaverse. This demonstrates how MetaChara enhances safety and improves time efficiency for intelligent transportation systems.