Text Tracking Research Articles

A Unified Framework for Tracking Based Text Detection and Recognition from Web Videos.

Video text extraction plays an important role for multimedia understanding and retrieval. Most previous research efforts are conducted within individual frames. A few of recent methods, which pay attention to text tracking using multiple frames, however, do not effectively mine the relations among text detection, tracking and recognition. In this paper, we propose a generic Bayesian-based framework of Tracking based Text Detection And Recognition (T DAR) from web videos for embedded captions, which is composed of three major components, i.e., text tracking, tracking based text detection, and tracking based text recognition. In this unified framework, text tracking is first conducted by tracking-by-detection. Tracking trajectories are then revised and refined with detection or recognition results. Text detection or recognition is finally improved with multi-frame integration. Moreover, a challenging video text (embedded caption text) database (USTB-VidTEXT) is constructed and publicly available. A variety of experiments on this dataset verify that our proposed approach largely improves the performance of text detection and recognition from web videos.

Preschoolers explore interactive storybook apps: The effect on word recognition and story comprehension

Two experiments explored the effects of reading digital storybooks on tablet computers with 25 preschoolers, aged 4---5. In the first experiment, the students' word recognition scores were found to increase significantly more when students explored a digital storybook and employed the read-aloud function than when they were read to from a comparable print book. Their comprehension scores did not change significantly in the two conditions. In Experiment 2, the same students explored digital storybooks with more animation embedded in them. The students listened to the read aloud function on the tablet computer and explored digital storybooks in both conditions, but in one condition a teacher guided the talk about the story. Contrary to expectations, the students' word recognition and story comprehension scores were higher in the independent condition than in the guided condition. One explanation for the higher word recognition scores when students were reading with the tablet computer is the effect of multimedia, like hotspots and/or text tracking. Although digital storybooks are not a substitute for adult interaction, these preschoolers learned surprisingly well on their own. The importance of digital storybook design, as well as what elements to look for in an e-book to encourage literacy learning are discussed.

Graph‐based approach to scene text localisation and tracking in videos

A text localisation and tracking method is presented that finds text-regions in videos and assigns unique IDs to their trajectories. For the goal, a graph-based framework that can work with existing text detection methods is developed. To be precise, graphs are built where vertices are image-level text detection results and edges represent the correspondence scores of the vertices. From these graphs, text-region trajectories by using the graph-cut algorithm are extracted. This approach allows considering false positives and misses, as well as their patch-based tracking results at the same time, and text trajectories are reliably extracted. Finally, the results are refined by interpolating misses and filtering out false positives. The proposed method is submitted to the International Conference on Document Analysis and Recognition 2015 robust reading competition (video text localisation) and the method showed the best performance in terms of CLEAR MOT metrics and was ranked third place according to VACE metrics among the seven participating methods.

Text Detection, Tracking and Recognition in Video: A Comprehensive Survey

The intelligent analysis of video data is currently in wide demand because a video is a major source of sensory data in our lives. Text is a prominent and direct source of information in video, while the recent surveys of text detection and recognition in imagery focus mainly on text extraction from scene images. Here, this paper presents a comprehensive survey of text detection, tracking, and recognition in video with three major contributions. First, a generic framework is proposed for video text extraction that uniformly describes detection, tracking, recognition, and their relations and interactions. Second, within this framework, a variety of methods, systems, and evaluation protocols of video text extraction are summarized, compared, and analyzed. Existing text tracking techniques, tracking-based detection and recognition techniques are specifically highlighted. Third, related applications, prominent challenges, and future directions for video text extraction (especially from scene videos and web videos) are also thoroughly discussed.

Real‐time text tracking in natural scenes

The authors present a system that automatically detects, recognises and tracks text in natural scenes in real‐time. The focus of the author's method is on large text found in outdoor environments, such as shop signs, street names, billboards and so on. Built on top of their previously developed techniques for scene text detection and orientation estimation, the main contribution of this work is to present a complete end‐to‐end scene text reading system based on text tracking. They propose to use a set of unscented Kalman filters to maintain each text region's identity and to continuously track the homography transformation of the text into a fronto‐parallel view, thereby being resilient to erratic camera motion and wide baseline changes in orientation. The system is designed for continuous, unsupervised operation in a handheld or wearable system over long periods of time. It is completely automatic and features quick failure recovery and interactive text reading. It is also highly parallelised to maximise usage of available processing power and achieve real‐time operation. They demonstrate the performance of the system on sequences recorded in outdoor scenarios.

Survey of Region-Based Text Extraction Techniques for Efficient Indexing of Image/Video Retrieval

With the dramatic increase in multimedia data, escalating trend of internet, and amplifying use of image/video capturing devices; content based indexing and text extraction is gaining more and more importance in research community. In the last decade, many techniques for text extraction are reported in the literature. Methodologies of text extraction from images/videos is generally comprises of text detection and localization, text tracking, text segmentation and optical character recognition (OCR). This paper intends to highlight the contributions and limitations of text detection, localization and tracking phases. The problem is exigent due to variations in the font styles, size and color, text orientations, animations and backgrounds. The paper can serve as the beacon-house for the novice researchers of the text extraction community. Index Terms—Text extraction, Document analysis, Survey, Text localization, Text tracking.

Extraction and Recognition Chinese Character of Video Using Wavelet-fractal and Multilevel Classification

Extraction and recognition Chinese characters in a video are widely applied to fields like image annotation and retrieval. In this paper, we extract features of Chinese character based on wavelet-fractal and employ multi-level classification including K-means and Back Propagation Neural Network (BPNN) to recognize. we take the projection of image in 0 ◦ ,4 5 ◦ ,9 0 ◦ , 135 ◦ and ring as features. To verify the algorithm, we developed an extraction and recognition Chinese character system consists of text regions detection, tracking, character segmentation, and character recognition module. In the character tracking, by introducing the concepts of density, sliding window and similarity, we design an algorithm of back-searching and forward-searching the frame interval which includes specific text regions. Statistical hypothesis test is employed to segment the characters. Finally, we give full and detailed experiments according to the algorithms proposed in this paper and the results turn out these algorithms are efficient.

PARTIR PARA O TRABALHO DE CAMPO EM SOCHAUX COM “BOURDIEU NA CABEÇA”

The text tracks a long itinerary of research in which the authors worked mainly on a single informant, a militant factory worker in a French auto plant. The outcomes of the research are presented via both an account and a reflection - metholodological and theoretical - about the sociological making, the relationship between the researcher and the researched, the dilemmas and choices presented in order to capture an experience that is both individual and collective, all taken and inspired from the topics found in the works of Pierre Bourdieu. Among such topics are: the fieldwork as a tool for ethnography the representation of the popular classes, the question of the dominated culture and the social biography (and its limits). At the same the authors provide clues to what is specific and particular about their own work when facing to the work of Bourdieu.

Ruins, Rubble and Human Remains: Negotiating Culture and Violence in Post-Catastrophic Warsaw

The wartime destruction of cities and their populations is frequently quantified in statistical terms. In historical summaries of Warsaw's wartime losses it is common to read that the city was 85% destroyed with up to 60% of its population killed. While statistical comparisons may be useful to summative historical accounts, expressing destruction in percentage points ignores the nuanced material relationships between destroyed architecture and human remains. Working outside conventional narratives of the destruction and reconstruction of Warsaw, this essay explores the semantic, ideological and physical conversions that accompanied the extensive survey and ruin clearance work conducted by the Capital Reconstruction Office (Biuro Odbudowy Stolicy—BOS) in the spring of 1945. The text tracks shifts in language and graphic standards as survey information was interpreted, mapped and finally made publicly available in professional journals. Ideological shifts were accompanied by demolitions and material transmutations as the mapped ruins were cleared, piled into rubble heaps, and eventually reused as construction material. This essay views the entirety of post-war urban topography as a war memorial materially implicated in genocide and asks what this means for practices of commemoration. It calls for a re-evaluation of the relationship between the body of architecture and the human body as it challenges the reader to consider the possibility that Warsaw's post-war rubble-concrete buildings may contain human remains.

Effective and efficient video text extraction using key text points

Text information contains important clues for video analysis, indexing and retrieval. Effective and efficient text extraction has been a challenging and significant topic. Focusing on this issue, this study proposes a video text extraction scheme using key text points (KTPs). A KTP is defined as the point that has strong textural structure in multi-directions simultaneously. First, a KTP can be acquired by the three high-frequency subbands obtained from the wavelet transform. An anti-texture-direction-projection method is then proposed to improve the accuracy of text localisation and verification. Second, the difference at the KTPs between neighbouring frames is calculated as the similarity measure in text tracking, which can reduce the influence of dramatic background variation. Finally, the total number of the KTPs in each connected component is calculated to remove the background interference and to extract texts for text segmentation. Experimental results show that the proposed text detection and localisation algorithm is robust to the font size, style, colour and alignment of texts. Text tracking significantly improves the efficiency of text detection, and the similarity measure based on KTPs decreases the influence of scene changes and motions. The proposed text segmentation has a promising performance when processing video scenes with complex backgrounds and low contrast between texts and backgrounds.

Text detection, localization, and tracking in compressed video

Video text information plays an important role in semantic-based video analysis, indexing and retrieval. Video texts are closely related to the content of a video. Usually, the fundamental steps of text-based video analysis, browsing and retrieval consist of video text detection, localization, tracking, segmentation and recognition. Video sequences are commonly stored in compressed formats where MPEG coding techniques are often adopted. In this paper, a unified framework for text detection, localization, and tracking in compressed videos using the discrete cosines transform (DCT) coefficients is proposed. A coarse to fine text detection method is used to find text blocks in terms of the block DCT texture intensity information. The DCT texture intensity of an 8×8 block of an intra-frame is approximately represented by seven AC coefficients. The candidate text block regions are further verified and refined. The text block region localization and tracking are carried out by virtue of the horizontal and vertical block texture intensity projection profiles. The appearing and disappearing frames of each text line are determined by the text tracking. The final experimental results show the effectiveness of the proposed methods.

On minimizing pattern splitting in multi-track string matching

Given a pattern string P = p 1 p 2 ⋯ p m and K parallel text strings T = { T k = t 1 k ⋯ t n k | 1 ⩽ k ⩽ K } over an integer alphabet Σ, our task is to find the smallest integer κ > 0 such that P can be split into κ pieces P = P 1 … P κ , where each P i has an occurrence in some text track T k i and these partial occurrences retain the order. We study some variations of this minimum splitting problem, such as splittings with limited gaps and transposition invariance, and show how to use sparse dynamic programming to solve the variations efficiently. In particular, we show that the minimum splitting problem can be interpreted as a shortest path problem on line segments.

VIDEO CLASSIFICATION USING OBJECT TRACKING

Content description becomes important in the ubiquity of video content on the Web and consumer devices. Video classification is needed so that more appropriate description and search methods can be applied. This paper describes two methods for video content classification: a Nearest Neighbor (NN) method relying on domain knowledge and Hidden Markov Model (HMM) based method. Our approach stems from the observation that in different TV categories, there are different objects (e.g., face and text) trajectory patterns. Face and text tracking is applied to video segments to extract face and text trajectories. We used NN and HMM to classify a given video segment into predefined classes, e.g., commercial, news, situation comedy and soap. Our preliminary experimental results show classification accuracy of 75% for NN and over 80% for HMM based method on short video segments.

Cognitive correlates of visual speech understanding in hearing-impaired individuals.

This study examined the extent to which different measures of speechreading performance correlated with particular cognitive abilities in a population of hearing-impaired people. Although the three speechreading tasks (isolated word identification, sentence comprehension, and text tracking) were highly intercorrelated, they tapped different cognitive skills. In this population, younger participants were better speechreaders, and, when age was taken into account, speech tracking correlated primarily with (written) lexical decision speed. In contrast, speechreading for sentence comprehension correlated most strongly with performance on a phonological processing task (written pseudohomophone detection) but also on a span measure that may have utilized visual, nonverbal memory for letters. We discuss the implications of this pattern.

Automatic text detection and tracking in digital video

Text that appears in a scene or is graphically added to video can provide an important supplemental source of index information as well as clues for decoding the video's structure and for classification. In this work, we present algorithms for detecting and tracking text in digital video. Our system implements a scale-space feature extractor that feeds an artificial neural processor to detect text blocks. Our text tracking scheme consists of two modules: a sum of squared difference (SSD)-based module to find the initial position and a contour-based module to refine the position. Experiments conducted with a variety of video sources show that our scheme can detect and track text robustly.

Tracking with text-based messages

As an initial research prototype, TextTrack demonstrates the feasibility of applying multiple hypothesis tracking concepts from radar signal processing to natural-language text tracking and correlation problems.

Making multimedia accessible on the World Wide Web

The Web Access Project, a part of the CPB/WGBH National Center for Accessible Media (NCAM; http://www.wgbh.org/ncam), was initiated in early 1996. Its mission is to develop and test technology to make World Wide Web sites accessible to deaf, hard-of-hearing, blind or visually impaired users. The Pr oject has developed methods for adding captions and audio descriptions to movie clips, making Web-based multimedia more accessible to users with sensory impairments. NCAM has also worked with Microsoft and the World Wide Web Consortium (W3C) on tools which allow authors to use a standard markup language and timecode to synchronize multiple sound and text tracks to video. This client-side rendering technique can provide users with easy and flexible access to closed captions and audio descriptions, in addition to foreign language soundtracks and subtitles. This paper examines the work done by the Web Access Project in the field of multimedia, and also explores accessibility issues in future multimedia technology, such as Shockwave, WebTV and Java.

Attributes of delivery media in computer-assisted instruction

This study investigated the effectiveness of multimedia for training purposes, with a comparison made between three Computer Assisted Instruction (CAI) techniques. The role of delivery media was explored with 57 university students who were randomly assigned to three experimental groups receiving the same sequence of four short computer lessons, but with different presentations: (1) text and still image; (2) text, still image and audio track of the text; and (3) text, video image and audio track of the text. Two levels of knowledge type, and the presence or absence of cue pictures on recall tests gave a 3 ×2 ×2 repeated measures design with subject's memory tested by a 40 item test. Multivariate analysis showed no significant effect for group. The pattern of results was however, in accordance with the dual coding theory of short term memory processing. A partitioned χ2 analysis showed significant group effects on individual test items. Results showed greater effectiveness of video and audio CAI over stills based CAI.