Video Text Extraction Research Articles

Multi-modal graph reasoning for structured video text extraction

Structured video text information extraction is a crucial part of video understanding for exploring the structured text fields from different category-specific videos such as scores in basketball games or identities in news. Recent natural language models and text detectors have demonstrated state-of-the-art performance in video text detection and recognition. However, understanding text from unstructured video frames is challenging in practice due to a variety of video text and dynamic text layout changes. Limited work has focused on the solutions that efficiently extract structured information from the video text. In this paper, we address this task by modeling a multi-modal attention graph on the video text. Specifically, we encode both the visual and textual features of detected text regions as nodes of the graph; the spatial layout relationship of the text regions is modeled as edges of the graph. The structured information extraction is solved by iteratively propagating text region messages along graph edges and reasoning the structured categories of graph nodes. To promote the representation capacity of the graph, we further introduce a contrastive loss on the visual embeddings of the text regions in a self-supervised manner. In order to roundly evaluate our proposed method as well as boost future research, we release a new dataset collected and annotated from several standard NBA regular seasons and playoff match videos. Experimental results demonstrate the superior performance of the proposed method over several state-of-the-art methods.

Intelligent Detection Method of English Text in Natural Scenes in Video

With the rapid development of Internet technology, breakthroughs have been made in all branches of computer vision. Especially in image detection and target tracking, deep learning techniques such as convolutional neural networks have achieved excellent results. In order to explore the applicability of machine learning technology in the field of video text recognition and extraction, a YOLOv3 network based on multiscale feature transformation and migration fusion is proposed to improve the accuracy of english text detection in natural scenes in video. Firstly, aiming at the problem of multiscale target detection in video key frames, based on the YOLOv3 network, the scale conversion module of STDN algorithm is used to reduce the low-level feature map, and a backbone network with feature reuse is constructed to extract features. Then, the scale conversion module is used to enlarge the high-level feature map, and a feature pyramid network (FPN) is built to predict the target. Finally, the improved YOLOv3 network is verified to extract key text from images. The experimental results show that the improved YOLOv3 network can effectively improve the false detection and missed detection caused by occlusion and small target, and the accuracy of English text extraction is obviously improved.

A new video text extraction using local laplacian filters and mean shift

Video text constitutes the semantic context of the video. For that reason, robust extraction of text is essential for successful video understanding, search and retrieval. Extracting text from background is an important phrase before the text can be recognized correctly. It is a challenging task because of the difficulties in text segmentation from the varied and complicated backgrounds. Therefore, this paper proposes a novel text extraction method to tackle this issue. First, we perform background complexity determination to distinguish the text lines with clear and simple background from those with complex background, which will increase the extraction speed. Then, for the text lines with complicated background and low contrast, we utilize the Local Laplacian Filters Commun ACM 58(3):81–91 [18] to enhance the details of text regions and get the Integrated Enhanced Map (IEM). Finally, we perform the Mean Shift IEEE Trans Pattern Anal Mach Intell 24(5):603–619 [4] for the segmentation on IEM and retrieve the text extraction results. Experimental evaluations based on a variety of videos dataset we collected demonstrate that our method significantly outperforms the other three video text extraction algorithms in terms of recall, precision and F-score, especially when there are challenges such as video text with different font sizes, font styles, languages, and background complexities.

Chinese text-line detection from web videos with fully convolutional networks

BackgroundIn recent years, video becomes the dominant resource of information on the Web, where the text within video usually carries significant semanticinformation. Video text extraction and recognition plays an essential role in web multimedia understanding and retrieval for big visual data analytics and applications. To deal with challenging backgrounds and embedding noises, most conventional approaches usually tend to design sophisticated pre-processing and post-progressing steps before and after text detection. In this paper, we present a simple yet powerful pipeline that directly and uniformly detects Chinese text lines for embedded captions from web videos.ResultsIn this Chinese text-line detection system, a fully convolutional network with local context is adopted to localize via an end-to-end learning way. The produced caption predictions are with the word level that could be directly fed into the character classifier. Text-line construction is then performed by heuristic strategies. A variety of experiments are conducted on several real-world web video datasets and demonstrated the effectiveness and efficiency of our proposed method.ConclusionThe proposed system can directly detect the English word and Chinese characters in the caption text-lines without word or character segmentation with the high performance on real-world web video datasets.

A Unified Framework for Tracking Based Text Detection and Recognition from Web Videos.

Video text extraction plays an important role for multimedia understanding and retrieval. Most previous research efforts are conducted within individual frames. A few of recent methods, which pay attention to text tracking using multiple frames, however, do not effectively mine the relations among text detection, tracking and recognition. In this paper, we propose a generic Bayesian-based framework of Tracking based Text Detection And Recognition (T DAR) from web videos for embedded captions, which is composed of three major components, i.e., text tracking, tracking based text detection, and tracking based text recognition. In this unified framework, text tracking is first conducted by tracking-by-detection. Tracking trajectories are then revised and refined with detection or recognition results. Text detection or recognition is finally improved with multi-frame integration. Moreover, a challenging video text (embedded caption text) database (USTB-VidTEXT) is constructed and publicly available. A variety of experiments on this dataset verify that our proposed approach largely improves the performance of text detection and recognition from web videos.

Text Detection, Tracking and Recognition in Video: A Comprehensive Survey

The intelligent analysis of video data is currently in wide demand because a video is a major source of sensory data in our lives. Text is a prominent and direct source of information in video, while the recent surveys of text detection and recognition in imagery focus mainly on text extraction from scene images. Here, this paper presents a comprehensive survey of text detection, tracking, and recognition in video with three major contributions. First, a generic framework is proposed for video text extraction that uniformly describes detection, tracking, recognition, and their relations and interactions. Second, within this framework, a variety of methods, systems, and evaluation protocols of video text extraction are summarized, compared, and analyzed. Existing text tracking techniques, tracking-based detection and recognition techniques are specifically highlighted. Third, related applications, prominent challenges, and future directions for video text extraction (especially from scene videos and web videos) are also thoroughly discussed.

An Effective Edge and Texture Based Approach towards Curved Videotext Detection and Extraction

In present day video text greatly helps video indexing and retrieval system as they often carry significant semantic information. Video text analysis is challenging due to varying background, multiple orientations and low contrast between text and non-text regions. Proposed approach explores a new framework for curved video text detection and recognition where from the observation that curve text regions can be well defined by edges size and uniform texture, Probable curved text edge detection is accomplished by processing wavelet sub bands followed by text localization by utilizing fast texture descriptor LU-transform. Binarization is achieved by maximal H-transform. A Connected Component filtering method followed by B-Spline curve fitting on centroid of each character vertically aligns each oriented character. The aligned text string is recognized by optical character recognition (OCR). Experiments on various curved video frames shows that proposed method is efficacious and robust in detecting and recognizing curved videotext.

A New Log Gabor Approach for Text Detection from Video

Text information embedded in video frames play an important role in content-based multimedia indexing and retrieval, as a result the automatic detection of texts from videos has gained wide attention in recent years. However,large variations in size, color, orientation and background complexity makes video text detection and extraction a challenging problem. In this paper an effective method for text detection and localization approach based on Log Gabor filter and Block Eigen map analysis is proposed. Log Gabor filter response is used to identify candidate text regions and Block wise Eigen map analysis is used to classify pixels as true text pixels. Finally,a heuristic based on gray scale pixel co-occurrence to eliminate false positives from the frame is used. Experimental results show the promising overall performance of the proposed method against the best features reported in the literature for the standard databases.

Effective and efficient video text extraction using key text points

Text information contains important clues for video analysis, indexing and retrieval. Effective and efficient text extraction has been a challenging and significant topic. Focusing on this issue, this study proposes a video text extraction scheme using key text points (KTPs). A KTP is defined as the point that has strong textural structure in multi-directions simultaneously. First, a KTP can be acquired by the three high-frequency subbands obtained from the wavelet transform. An anti-texture-direction-projection method is then proposed to improve the accuracy of text localisation and verification. Second, the difference at the KTPs between neighbouring frames is calculated as the similarity measure in text tracking, which can reduce the influence of dramatic background variation. Finally, the total number of the KTPs in each connected component is calculated to remove the background interference and to extract texts for text segmentation. Experimental results show that the proposed text detection and localisation algorithm is robust to the font size, style, colour and alignment of texts. Text tracking significantly improves the efficiency of text detection, and the similarity measure based on KTPs decreases the influence of scene changes and motions. The proposed text segmentation has a promising performance when processing video scenes with complex backgrounds and low contrast between texts and backgrounds.

A videotext in picture display system with embedded videotext extraction on a dual-core platform

This research utilizes embedded videotexts to achieve an intelligent multimedia display. We design a videotext in picture (TiP) display system which can extract the videotexts in the subchannel and then combine these videotexts with the main channel. This system was constructed on a dual-core platform to reach real-time videotext extraction and display. A schedulable design framework was proposed to partition the TiP display with videotext extraction in pipeline running. A data-aware transfer scheme was designed in which some data can be reused. Single instruction multiple data (SIMD) based mechanisms were created to enhance the computational efficiency on numerous convolutions and accumulations in videotext extraction. Quadruple buffering was manipulated to process the input/output in videotext extraction simultaneously. To optimize the labeling and filling tasks, the multi-banking and multi-tasking were developed. The evaluation results indicated that the proposed techniques can speed up the processing time of TiP display with videotext extraction. The equivalent comparison presented that the proposed techniques are more proficient at realizing videotext extraction.

Video Text Localization and Extraction Based on Variance Projection and Morphology

The text of the video can provide important information for video description and retrieval. First,the paper adopts Maximum Gradient algorithm which is based on the Sobel Gradient operator to detect the edge of the text. Second, with regards to the rich information of the edge of text, variance projection is employed to detect text zones and to filtrate false text zones by calculating the Regional Density. Finally, Image Morphology is employed to treat the edge of the text. It is to get position and color information of pixels of original image and extract text from complex background. A large number of experiments to prove that the algorithm is very effective.

A robust video text extraction method for character recognition

AbstractThis paper proposes a method for extracting text portions occurring in video images with high accuracy for reading by OCR. Past studies have produced methods of extracting text in a video image from its background by binarization based on a threshold, utilizing the fact that the intensity of the text is higher than that of the background. One method for determining the threshold is Shio's application of Otsu's method, assuming the distribution of two intensities of the background and characters in local blocks. However, methods based on various intensities of the background such as those of video images have the problem of not yielding a good threshold due to assumptions that are not necessarily valid. In addition, in reality, they cannot extract characters with accuracy sufficient for OCR readability because the intensity around the characters is not necessarily high due to the effects of shadowing, edge elimination, and signal conversion processing. Thus, this paper proposes a method of extracting only the text portions by robustly estimating the intensity distribution of the text portions, initially extracting high‐reliability areas as text portions, and extending the areas based on the estimated distribution. Experimental results show that the proposed method extracts text portions with higher accuracy and better OCR readability than the conventional methods. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(9): 87–96, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10148

A comprehensive method for multilingual video text detection, localization, and extraction

Text in video is a very compact and accurate clue for video indexing and summarization. Most video text detection and extraction methods hold assumptions on text color, background contrast, and font style. Moreover, few methods can handle multilingual text well since different languages may have quite different appearances. This paper performs a detailed analysis of multilingual text characteristics, including English and Chinese. Based on the analysis, we propose a comprehensive, efficient video text detection, localization, and extraction method, which emphasizes the multilingual capability over the whole processing. The proposed method is also robust to various background complexities and text appearances. The text detection is carried out by edge detection, local thresholding, and hysteresis edge recovery. The coarse-to-fine localization scheme is then performed to identify text regions accurately. The text extraction consists of adaptive thresholding, dam point labeling, and inward filling. Experimental results on a large number of video images and comparisons with other methods are reported in detail.

A new robust algorithm for video text extraction

We report on the development and implementation of a robust algorithm for extracting text in digitized color video. The algorithm first computes maximum gradient difference to detect potential text line segments from horizontal scan lines of the video. Potential text line segments are then expanded or combined with potential text line segments from adjacent scan lines to form text blocks, which are then subject to filtering and refinement. Color information is then used to more precisely locate text pixels within the detected text blocks. The robustness of the algorithm is demonstrated by using a variety of color images digitized from broadcast television for testing. The algorithm also performs well on images after JPEG compression and decompression, and on images corrupted with different types of noise.