Text Images Research Articles

HAFE: A Hierarchical Awareness and Feature Enhancement Network for Scene Text Recognition

Scene text recognition (STR) is to obtain text information from natural text images. Compared to the optical character recognition, STR has attracted much attentions due to the variability of scene images. However, most STR methods tend to loss some spatial information due to only simply adding Resnet-45 to the front end of the encoder. To solve this problem, we propose a Hierarchical Awareness and Feature Enhancement (HAFE) network. Our proposed Hierarchical Awareness (HA) module can effectively extract the depth information of scene text images from multiple dimensions and then improve the anti-interference ability of the network. Besides, the proposed Feature Enhancement (FE) module can enhance the stability of the model through strengthening the global features, which is helpful to improve the accuracy of text recognition. On the whole, our HAFE network tends to strengthen the information in encoder, better learn the feature maps from different levels, and then improve the recognition accuracy of non-horizontal texts. Extensive comparative experiments on seven benchmark datasets show that the proposed method has the most advanced performance and outperforms most state-of-the-art ones. Our code will be released at website: https://github.com/HAFE in the future.

LAE-GAN: a novel cloud-based Low-light Attention Enhancement Generative Adversarial Network for unpaired text images

With the widespread adoption of mobile multimedia devices, the deployment of compute-intensive inference tasks on edge and resource-constrained devices, particularly in the context of low-light text detection, remains a formidable challenge. Existing deep learning approaches have shown limited effectiveness in restoring images for extremely dark scenes. To address these limitations, this paper presents a novel cloud-based Low-light Attention Enhancement Generative Adversarial Network for unpaired text images (LAE-GAN) for the non-paired text image enhancement task in extremely low-light conditions. In the first stage, compressed low-light images are transmitted from edge devices to a cloud server for image enhancement. The LAE-GAN, an end-to-end network comprising a Zero-DCE and AGM-net generator, is designed with a global and local discriminator structure. The initial illumination restoration of extremely low-light images is accomplished using the Zero-DCE network. To enhance text details, we propose an Enhanced Text Attention Mechanism (ETAM) that transforms text information into a comprehensive text attention mechanism across the entire network. The Sobel operator is employed to extract text edge information, while attention is focused on text region details through constraints imposed on the attention map and edge map. Additionally, an AGM-Net module is integrated to reduce noise and fine-tune illumination. In the second stage, the cloud server makes decisions based on user requirements and processes requests in parallel, scaling with the quantity of requests. In the third stage, the enhanced results are transmitted back to edge devices for text detection. Experimental results on widely used LOL and SID low-light datasets demonstrate significant improvements in both quantitative and qualitative analysis, surpassing state-of-the-art enhancement methods in terms of image restoration and text detection.

Pathway to authenticity? The influence of politicians’ gender and multimodal self-presentation in social media on perceived authenticity

The authenticity of political candidates has become a decisive factor in their evaluation by the public. In response, political candidates employ self-presentation techniques to appear genuine to constituents, and social media provides politicians with new tools to perform authenticity. Yet, we still know little about how politicians construct authentic images and how their gender and modes of self-presentation influence perceived authenticity. In this study, we examine how politicians’ gender (male vs female) and modes of self-presentation (text-only, image-only, and text–image combined) affect perceived political authenticity. Results from a 2 × 3 online survey experiment in Germany ( N = 1,485) show that gender alone does not explain differences in perceived political authenticity when self-presentation is held consistently. Furthermore, we do not find significant effects for different self-presentation modes in social media. Additional analyses reveal that physical attractiveness is the primary factor for higher authenticity ratings in our sample.

Pixel-Level Degradation for Text Image Super-Resolution and Recognition

In the realm of image reconstruction, deep learning-based super-resolution (SR) has established itself as a prevalent technique, particularly in the domain of text image restoration. This study aims to address notable deficiencies in existing research, including constraints imposed by restricted datasets and challenges related to model generalization. Specifically, the goal is to enhance the super-resolution network’s reconstruction of scene text image super-resolution and utilize the generated degenerate dataset to alleviate issues associated with poor generalization due to the sparse scene text image super-resolution dataset. The methodology employed begins with the degradation of images from the MJSynth dataset, using a stochastic degradation process to create eight distinct degraded versions. Subsequently, a blank image is constructed, preserving identical dimensions to the low-resolution image, with each pixel sourced randomly from the corresponding points across the eight degraded images. Following several iterations of training via Finetune, the LR-HR method is applied to the TextZoom dataset. The pivotal metric for assessment is optical character recognition (OCR) accuracy, recognized for its fundamental role in gauging the pragmatic effectiveness of this approach. The experimental findings reveal a notable enhancement in OCR accuracy when compared to the TBSRN model, yielding improvements of 2.4%, 2.3%, and 4.8% on the TextZoom dataset. This innovative approach, founded on pixel-level degradation, not only exhibits commendable generalization capabilities but also demonstrates resilience in confronting the intricate challenges inherent to text image super-resolution.

Text-Enhanced Scene Image Super-Resolution via Stroke Mask and Orthogonal Attention

Low-resolution text images are very commonplace in real life and their information is hard to be extracted by using existing text recognition methods only. Although this problem can be solved by introducing super-resolution (SR) techniques, most existing SR methods fail to process stroke regions and background regions of input text images distinctively. In this paper, we propose a text-specific super-resolution network named Text Enhanced Attention Network (TEAN) to solve this problem. First of all, we compensate for disadvantages of traditional thresholding mask operation proposed in Text Super-Resolution Network (TSRN) by utilizing deep-learning based semantic segmentation method to get correct masks as prior semantic information and propose a Text-Segmented-Contextual-Attention (TSCA) branch on the basis of them. Besides, we design an Orthogonal Contextual Attention Module (OCAM) working with TSCA to implicitly enhance stroke regions of LR images. Secondly, to effectively fuse shallow features and deep features of SR model, we propose a convolutional structure named Weight Balanced Fusion Module (WBFM) to improve traditional feature fusion methods of SR network. Finally, extensive experiments on TextZoom dataset demonstrate that the proposed network can improve the recognition accuracy of text images on existing text recognition models. Using TEAN to process low-resolution text images improves the recognition accuracy by 25.4% on CRNN, by 17.4% on ASTER, by 17.3% on MORAN, by 20.7% on NRTR, by 17.3% on SAR and by 15.9% on MASTER compared with directly recognizing them, which attains competitive performances against state-of-the-art methods. Furthermore, cross-dataset experiments on IC15_2077 demonstrate that TEAN is helpful for scene text recognition task, especially for low-resolution images even with the cross-domain issue.

Kernel Proposal Network for Arbitrary Shape Text Detection.

Segmentation-based methods have achieved great success for arbitrary shape text detection. However, separating neighboring text instances is still one of the most challenging problems due to the complexity of texts in scene images. In this article, we propose an innovative kernel proposal network (dubbed KPN) for arbitrary shape text detection. The proposed KPN can separate neighboring text instances by classifying different texts into instance-independent feature maps, meanwhile avoiding the complex aggregation process existing in segmentation-based arbitrary shape text detection methods. To be concrete, our KPN will predict a Gaussian center map for each text image, which will be used to extract a series of candidate kernel proposals (i.e., dynamic convolution kernel) from the embedding feature maps according to their corresponding keypoint positions. To enforce the independence between kernel proposals, we propose a novel orthogonal learning loss (OLL) via orthogonal constraints. Specifically, our kernel proposals contain important self-information learned by network and location information by position embedding. Finally, kernel proposals will individually convolve all embedding feature maps for generating individual embedded maps of text instances. In this way, our KPN can effectively separate neighboring text instances and improve the robustness against unclear boundaries. To the best of our knowledge, our work is the first to introduce the dynamic convolution kernel strategy to efficiently and effectively tackle the adhesion problem of neighboring text instances in text detection. Experimental results on challenging datasets verify the impressive performance and efficiency of our method. The code and model are available at https://github.com/GXYM/KPN.

SLOGAN: Handwriting Style Synthesis for Arbitrary-Length and Out-of-Vocabulary Text.

Large amounts of labeled data are urgently required for the training of robust text recognizers. However, collecting handwriting data of diverse styles, along with an immense lexicon, is considerably expensive. Although data synthesis is a promising way to relieve data hunger, two key issues of handwriting synthesis, namely, style representation and content embedding, remain unsolved. To this end, we propose a novel method that can synthesize parameterized and controllable handwriting S tyles for arbitrary-Length and O ut-of-vocabulary text based on a G enerative A dversarial N etwork (GAN), termed SLOGAN. Specifically, we propose a style bank to parameterize specific handwriting styles as latent vectors, which are input to a generator as style priors to achieve the corresponding handwritten styles. The training of the style bank requires only writer identification of the source images, rather than attribute annotations. Moreover, we embed the text content by providing an easily obtainable printed style image, so that the diversity of the content can be flexibly achieved by changing the input printed image. Finally, the generator is guided by dual discriminators to handle both the handwriting characteristics that appear as separated characters and in a series of cursive joins. Our method can synthesize words that are not included in the training vocabulary and with various new styles. Extensive experiments have shown that high-quality text images with great style diversity and rich vocabulary can be synthesized using our method, thereby enhancing the robustness of the recognizer.

Rater cognitive processes in integrated writing tasks: from the perspective of problem-solving

It is widely believed that raters’ cognition is an important aspect of writing assessment, as it has both logical and temporal priority over scores. Based on a critical review of previous research in this area, it is found that raters’ cognition can be boiled to two fundamental issues: building text images and strategies for articulating scores. Compared to the scoring contexts of previous research, the TEM 8 integrated writing task scoring scale has unique features. It is urgent to know how raters build text images and how they articulate scores for text images in the specific context of rating TEM8 compositions. In order to answer these questions, the present study conducted qualitative research by considering raters as problem solvers in the light of problem-solving theory. Hence, 6 highly experienced raters were asked to verbalize their thoughts simultaneously while rating TEM 8 essays, supplemented by a retrospective interview. Analyzing the collected protocols, we found that with regard to research question 1, the raters went through two stages by setting building text images as isolated nodes and building holistic text images for each dimension as two sub-goals, respectively. In order to achieve the first sub-goal, raters used strategies such as single foci evaluating, diagnosing, and comparing; for the second sub-goal, they mainly used synthesizing and comparing. Regarding the second question, the results showed that they resorted to two groups of strategies: demarcating boundaries between scores within a dimension and discriminating between dimensions, each group consisting of more specific processes. Each of the extracted processes was defined clearly and their relationships were delineated, on the basis of which a new working model of the rating process was finalized. Overall, the present study deepens our understanding of rating processes and provides evidence for the scoring validity of the TEM 8 integrated writing test. It also provides implications for rating practice, such as the need for the distinction between two types of analytical rating scales.

Back projection deep unrolling network for handwritten text image super resolution

Current super-resolution (SR) methods have demonstrated exceptional advancements in the domain of natural image processing. Nevertheless, these approaches do not fully address the open issues of edge blurring and distortion. In order to address the aforementioned challenges, we propose a novel deep unrolling model, coined back projection deep unrolling network (BPDUN), for super-resolution of handwritten text images leveraging the Algorithm Unrolling paradigm. We design the network model of BPDUN by unfolding and truncating the traditional iterative back projection algorithm (IBPA). We unroll IBPA into a cascade operation of three building blocks (deep denoising module, low-frequency reconstruction module, and residual projection module). BPDUN inherits the interpretability of the iterative optimization algorithm and is also designed to make the reconstructed text image more realistic and natural. Moreover, we propose a new benchmark dataset to address challenging SR problems of handwritten text image (HDT300). Extensive experiments show that BPDUN obtains an enhanced balance between the performance (quantified by PSNR and SSIM) and the cost (as measured by network parameters). Notably, BPDUN sets new benchmarks on the HDT300 dataset, surpassing previous state-of-the-art approaches by achieving up to 0.2 dB gains in PSNR.

Soft-edge-guided significant coordinate attention network for scene text image super-resolution

Soft-edge-guided significant coordinate attention network for scene text image super-resolution

DILF: Differentiable rendering-based multi-view Image–Language Fusion for zero-shot 3D shape understanding

Zero-shot 3D shape understanding aims to recognize “unseen” 3D categories that are not present in training data. Recently, Contrastive Language–Image Pre-training (CLIP) has shown promising open-world performance in zero-shot 3D shape understanding tasks by information fusion among language and 3D modality. It first renders 3D objects into multiple 2D image views and then learns to understand the semantic relationships between the textual descriptions and images, enabling the model to generalize to new and unseen categories. However, existing studies in zero-shot 3D shape understanding rely on predefined rendering parameters, resulting in repetitive, redundant, and low-quality views. This limitation hinders the model’s ability to fully comprehend 3D shapes and adversely impacts the text–image fusion in a shared latent space. To this end, we propose a novel approach called Differentiable rendering-based multi-view Image–Language Fusion (DILF) for zero-shot 3D shape understanding. Specifically, DILF leverages large-scale language models (LLMs) to generate textual prompts enriched with 3D semantics and designs a differentiable renderer with learnable rendering parameters to produce representative multi-view images. These rendering parameters can be iteratively updated using a text–image fusion loss, which aids in parameters’ regression, allowing the model to determine the optimal viewpoint positions for each 3D object. Then a group-view mechanism is introduced to model interdependencies across views, enabling efficient information fusion to achieve a more comprehensive 3D shape understanding. Experimental results can demonstrate that DILF outperforms state-of-the-art methods for zero-shot 3D classification while maintaining competitive performance for standard 3D classification. The code is available at https://github.com/yuzaiyang123/DILP.

Tamper Detection Technique for Text Images based on Vowels and Unicode Zero Length Characters

Tamper Detection Technique for Text Images based on Vowels and Unicode Zero Length Characters

Image Theory and Emotional Expression in w. j. t. Michener

W.I.T. Michel's theory of iconography centers on the dilemmas faced by images in the 20th century, such as the relationship between images and texts, how images act on life, etc. This indispensably explores people's emotional expression in art. Based on Michel's analysis of text and image, this paper explores the relationship between textual images and emotional expression, as well as the emotional transmission of images in artworks.

A text image generation model based on deep learning

The traditional text-image confrontation model utilizes a convolutional network in the discriminator to extract image features, yet this fails to involve the spatial relationship between underlying objects, resulting in a poor-quality generated image. To remedy this, a capsule network is proposed to improve the model. The convolutional network in the discriminator is replaced with a capsule network, thereby improving the robustness of the images. Through experiments on the Oxford-102 and CUB datasets, it has been found that the new model can effectively improve the quality of generated text-image. The FID value of the generated flower image decreased by 14.49%, and the FID value of the generated bird image decreased by 9.64%. Additionally, the Inception Score of images generated on the Oxford-102 and CUB datasets increased by 22.60% and 26.28%, respectively, indicating that the improved model generated richer and more meaningful image features.

‘Domani a quest’ora potresti essere qui’: multimodal practices for representing temporality in destination advertising

Materiality, mediality, locality and temporality are constitutive characteristics of texts. Various texts in mediatized everyday reality show temporal aspects that influence their perception and their functions in different ways. The present article considers diverse expressions of temporality in destination ads, a genre that is hitherto underexplored. A special focus is put on multimodal practices that express temporal aspects, for instance the timeline of pre-, on- and post-trip time, vacation time as desired time, the situationality of ads, etc. The author uses multimodal discourse analysis to examine the interplay of semiotic resources in destination ads and to describe their compositional, ideational and interactional metafunctions. The analysis shows that text–image relations are crucial for the depiction of temporality and time-boundedness, and also for the formation of a specific tourist gaze that is centred on the desire to fully enjoy vacation time.

HDevChaRNet: A deep learning-based model for recognizing offline handwritten devanagari characters

<p>Optical character recognition (OCR) converts text images into machine-readable text. Due to the non-availability of several standard datasets of Devanagari characters, researchers have used many techniques for developing an OCR system with varying recognition rates using their own created datasets. The main objective of our proposed study is to improve the recognition rate by analyzing the effect of using batch normalization (BN) instead of dropout in convolutional neural network (CNN) architecture. So, a CNN-based model HDevChaRNet (Handwritten Devanagari Character Recognition Network) is proposed in this study for same to recognize offline handwritten Devanagari characters using a dataset named Devanagari handwritten character dataset (DHCD). DHCD comprises a total of 46 classes of characters, out of which 36 are consonants, and 10 are numerals. The proposed models based on convolutional neural network (CNN) with BN for recognizing the Devanagari characters showed an improved accuracy of 98.75%, 99.70%, and 99.17% for 36, 10, and 46 classes, respectively.</p>

Text Recognition Model Based on Multi-Scale Fusion CRNN.

Scene text recognition is a crucial area of research in computer vision. However, current mainstream scene text recognition models suffer from incomplete feature extraction due to the small downsampling scale used to extract features and obtain more features. This limitation hampers their ability to extract complete features of each character in the image, resulting in lower accuracy in the text recognition process. To address this issue, a novel text recognition model based on multi-scale fusion and the convolutional recurrent neural network (CRNN) has been proposed in this paper. The proposed model has a convolutional layer, a feature fusion layer, a recurrent layer, and a transcription layer. The convolutional layer uses two scales of feature extraction, which enables it to derive two distinct outputs for the input text image. The feature fusion layer fuses the different scales of features and forms a new feature. The recurrent layer learns contextual features from the input sequence of features. The transcription layer outputs the final result. The proposed model not only expands the recognition field but also learns more image features at different scales; thus, it extracts a more complete set of features and achieving better recognition of text. The results of experiments are then presented to demonstrate that the proposed model outperforms the CRNN model on text datasets, such as Street View Text, IIIT-5K, ICDAR2003, and ICDAR2013 scenes, in terms of text recognition accuracy.

A Long-Tailed Image Classification Method Based on Enhanced Contrastive Visual Language.

To solve the problem that the common long-tailed classification method does not use the semantic features of the original label text of the image, and the difference between the classification accuracy of most classes and minority classes are large, the long-tailed image classification method based on enhanced contrast visual language trains the head class and tail class samples separately, uses text image to pre-train the information, and uses the enhanced momentum contrastive loss function and RandAugment enhancement to improve the learning of tail class samples. On the ImageNet-LT long-tailed dataset, the enhanced contrasting visual language-based long-tailed image classification method has improved all class accuracy, tail class accuracy, middle class accuracy, and the F1 value by 3.4%, 7.6%, 3.5%, and 11.2%, respectively, compared to the BALLAD method. The difference in accuracy between the head class and tail class is reduced by 1.6% compared to the BALLAD method. The results of three comparative experiments indicate that the long-tailed image classification method based on enhanced contrastive visual language has improved the performance of tail classes and reduced the accuracy difference between the majority and minority classes.

Joint super-resolution and deblurring for low-resolution text image using two-branch neural network

Joint super-resolution and deblurring for low-resolution text image using two-branch neural network

BCSR: toward arbitrarily oriented text image super-resolution via adaptive Bezier curve network

Although existing super-resolution networks based on deep learning have obtained good results, it is still challenging to achieve an ideal visual effect for irregular texts, especially spatially deformed ones. In this paper, we propose a robust Bezier Curve-based image super-resolution network (BCSR), which can efficiently handle the degradation caused by deformations. Firstly, the arbitrarily shaped text is adaptively fitted by a parameterized Bezier curve, aiming to convert a curved text box into an annotated text box. Then, we design a BezierAlign layer to calibrate between the extracted features and the input image. By importing the extracted text prior information, the accuracy of the super-resolution network can be significantly improved. It is worth highlighting that we propose a kind of text prior loss that enables the text prior image and the super-resolution text image to achieve cooperation enhancement. Extensive experiments on several standard scene text datasets demonstrate that our proposed model achieves desirable objective evaluation results and further immensely helps downstream tasks related to text recognition, especially in text instances with multi-orientation and curved shapes.