Video Text Research Articles

Inter-View Dependency-Based Rate Control for 3D-HEVC

We propose an inter-view dependency-based rate control (RC) algorithm for 3D extension of high efficiency video coding (HEVC). First, considering the rate-distortion (R-D) dependency between the synthesized views and the input views (including texture videos and depth maps), a synthesized view distortion model is derived. Second, a novel distortion model for dependent views is proposed by investigating the inter-view dependency between the base and dependent views. Based on these two distortion models, a joint optimal bit allocation strategy (including texture/depth level, view level, and frame level) is developed to allocate target bits for both texture videos and depth maps of different views. Furthermore, an effective initial quantization parameter decision scheme considering the characteristics of the input video content is presented. Extensive experimental results exhibit that the proposed scheme achieves a higher rate control accuracy and provides a better R-D performance than the state-of-the-art RC algorithms.

Glottotecnologie didattiche per i migranti L’italiano fra le altre lingue nel progetto L-Pack (Citizenship Language Pack for Migrants in Europe)

Teaching languages to migrants through ICT L-Pack Project for Italian and other languages (Citizenship Language Pack for Migrants in Europe) The most recent trends in technology and the internet, commonly called Web 2.0., have determined new concepts in teaching and learning that involve autonomy, multimodality and flexibility. New approaches to ‘knowledge by technology’ are changing both teachers’ and learners’ roles, responding to their different aims and needs. At the same time, nearly 4 million people are presently involved in massive migration processes all over Europe, coming either from outside Europe or from another EU member state. The EU policies on migrants state the importance of a basic knowledge of the host country’s language, history and institutions for an effective integration process, and EU governments are expected to contribute to this aim. This paper describes the European Project ‘L-PACK: Citizenship Language Pack For Migrants in Europe’ (2011-2016), whose main aim consisted in developing a series of internet video texts, accompanied with materials and resources to lead adult migrants to A2 level (according to QCER levels scale) in different EU languages. The project has been developed in two main stages. In the first part of the programme, called L-Pack 1, from 2011 to 2013, the languages were Italian, Spanish, German, Lithuanian, Greek and Czech. In the second part, called L-Pack 2 extended, from 2014 to 2016, the project added English and French and was integrated with new resources and tools. The L-PACK course, which consists of 60 short video dialogues from everyday life, supported by comprehension activities and linguistic explanation and rules, is totally free and available through Youtube, Wikibooks and Soundcloud. The dedicated website http://www.l-pack.eu was visited by 120.000 users from 146 countries in the period 2011-2016. L-Pack teaching materials have also been used by teachers in classroom activities. The authors analyze and evaluate the results of the L-Pack Project, from pilot to dissemination, in different learning contexts both in situ and through e-learning. This is carried out on the basis of surveys concerning the project as a whole and in particular its use in Italian language teaching classes for migrants in Italy.

Study on Restaurant Images through Korea-Japan Video Text - and

Study on Restaurant Images through Korea-Japan Video Text - and

Important macroblock distinction model for multi-view plus depth video transmission over error-prone network

Multi-view plus depth (MVD) video is an efficient three dimensional (3D) video representation format that allows the sender only transmit two pairs of texture and depth videos, arbitrary virtual view can be synthesized at the receiver. However, constrained by the limited-bandwidth, inevitable packet loss will induce transmission distortion, which will propagate to virtual views thereby affects user’s stereoscopic perception. In order to relieve the virtual view’s quality degradation induced by packet loss, an important macroblock (MB) distinction model for both texture and depth videos is proposed. MBs with low important level will be actively discarded once congestion occurs. The model includes two main parts: Firstly, by considering temporal and spatial correlation of the coding structure and the distortion diffusion due to lost packets, a transmission distortion model is proposed. Secondly, a gradient based synthesis distortion model is adopted to analyze the distortion induced by depth-error. Finally, a low-complexity important MB distinction model is proposed for MVD video transmission. Experiment results show that, compared with random packet loss condition, Peak Signal-to-Noise Ratio (PSNR) of the virtual view increase by up to 15.65 dB at 20% packet loss rate, both objective and subjective quality of the virtual view are close to error-free transmission.

Adaptive Progressive Motion Vector Resolution Selection Based on Rate-Distortion Optimization.

In the state-of-the-art H.265/HEVC video coding standard, the motion vector (MV) resolution is fixed to be 1/4-pel for the entire video sequence, while the inherent video characteristics, e.g., texture complexities and motion activities, have been largely ignored. Obviously such strategy may not suffice the demand of high-accuracy motion compensation. In this paper, we propose a specially designed rate-distortion model in terms of the MV resolution by taking the video characteristics into consideration. In particular, the MV resolution selection is formulated as a rate-distortion optimization problem by analyzing the rate-distortion cost of each MV resolution candidate. To further improve the coding performance, the progressive MV resolution strategy is employed, where the optimal progressive MV resolution is determined by decision trees constructed with the rate-distortion model. In this manner, a novel adaptive progressive motion vector resolution selection scheme can be realized and the MV resolution can be adaptively adjusted based on the properties of local content. Extensive experiments and comparisons show that the proposed algorithm significantly improves the coding performance, and 1.8% BD-rate gain on average has been achieved without introducing any noticeable computational complexity.

A Robust Approach for Overlay Text Localization and Extraction in Complex Video Scene

Overlay text in video carries important semantic clues for video information retrieval and summarization. In this paper, we propose a robust method that is able to accurately locate text lines and extract text even in complex video scene. In the text localization stage, this paper adopts the method based on corner point. First, corner detection is used to extract corners as text features from video frames. Then multi-layer filtering mechanism (MLFM) is used to locate the text lines, which consists of corners clustering, corners horizontal projection, background filtering and heuristic rules. This MLFM can effectively remove the isolated corners, locate the text lines accurately and remove the background or pseudo text lines automatically. In the text extraction stage, this paper proposed a twice binarization method that combines with polarity judgment on image. The polarity judgment was used as a guide to adjust the first binarization threshold when we perform the first binarization. After the first binarization, a main proportion of the image has been processed, and the rest will be processed by the second binarization. Experimental results show that this approach can fast and robustly locate text lines and extract text in video even under complex background.

An efficient depth modeling mode decision algorithm for 3D-HEVC depth map coding

With the purpose of augmenting immersive perception, 3D video coding extension of High Efficiency Video Coding (3D-HEVC) standard is developed. This extension employs Multiview Video plus Depth (MVD) format in which immediate virtual view texture videos can be rendered by adjacent depth maps, so that bitrates and coding runtime of multiview video are saved. A good preservation of depth map is vital to synthesize high-quality virtual view texture videos. Hence, several new depth map coding tools have been introduced into 3D-HEVC, such as Depth Modeling Mode (DMM) decision. However, the numerous DMM candidates result in a huge computational complexity, which prevents 3D-HEVC for practical use. To this end, we propose an efficient DMM decision algorithm for 3D-HEVC depth map coding, which incorporates unnecessary depth blocks skipping approach and Contour pattern early termination method. Simulated experiment results state that the proposed efficient DMM decision method can save about 31.33% coding time on average with only 0.01dB of PSNR drop and just 0.60% bitrate increase on average compared to the original 3D-HEVC encoder.

Arbitrarily-oriented multi-lingual text detection in video

Text detection in arbitrarily-oriented multi-lingual video is an emerging area of research because it plays a vital role for developing real-time indexing and retrieval systems. In this paper, we propose to explore moments for identifying text candidates. We introduce a novel idea for determining automatic windows to extract moments for tackling multi-font and multi-sized text in video based on stroke width information. The temporal information is explored to find deviations between moving and non-moving pixels in successive frames iteratively, which results in static clusters containing caption text and dynamic clusters containing scene text, as well as background pixels. The gradient directions of pixels in static and dynamic clusters are analyzed to identify the potential text candidates. Furthermore, boundary growing is proposed that expands the boundary of potential text candidates until it finds neighbor components based on the nearest neighbor criterion. This process outputs text lines appearing in the video. Experimental results on standard video data, namely, ICDAR 2013, ICDAR 2015, YVT videos and on our own English and Multi-lingual videos demonstrate that the proposed method outperforms the state-of-the-art methods.

Adaptive Bit Allocation for 3D Video Coding

In the multi-view video plus depth 3D video coding, texture image and depth map are coded jointly. The texture image is utilized for displaying and synthesizing the virtual view as reference image. The depth map provides the scene geometry information and is utilized to synthesize the virtual view at the terminal through Depth-Image Based Rendering technique. The distortion of the compressed texture image and depth map will be propagated to the synthesized virtual view. Besides the coding efficiency of texture image and depth map, bit allocation between texture image and depth map also has a great effect on the synthesized virtual view quality. Several methods are proposed for bit allocation between texture image and depth map, but most of them attempt to allocate a fixed target bitrate based on virtual view distortion model to achieve optimal synthesized virtual view quality, and the modeling process brings extra complexity. In practical application, the video sequence has different contents and fixed bit ratio cannot achieve optimal performance. In this paper, we propose an adaptive bit allocation algorithm for 3D video coding. First, we present a model to estimate the synthesized virtual view distortion, and then adjust the bit ratio between adjacent views and between texture image and depth map at Group of Picture level based on the virtual view quality fluctuation. We adjust the bit ratio to achieve the optimal virtual view quality for different video contents. Experimental results demonstrate that the proposed algorithm can optimally allocate bits to achieve optimal virtual view quality under different target bitrates and for different video contents, and the computational complexity of the proposed algorithm is extremely low.

Efficient Lagrange multiplier selection algorithm for depth maps coding

The rate–distortion optimisation criterion of depth maps coding is evaluated by virtual view distortion, which motivates research modelling the distortion and corresponding Lagrange multiplier for depth maps coding. Current studies follow the assumption that the virtual view distortion introduced by texture videos distortion and depth maps distortion can be explored separately. Then the texture-videos-distortion-induced virtual view distortion is considered as a constant value during depth maps coding. A relationship between the optimal Lagrange multiplier of depth maps and texture videos distortion is investigated. Based on this relationship, an efficient Lagrange multiplier selection algorithm is proposed for depth maps coding based on quantisation parameters of texture videos and depth maps. With the proposed Lagrange multiplier, 1.36% bitrate saving with the same virtual view quality can be achieved compared with 3D high efficiency video coding reference software.

Fast Algorithm for Intra Prediction of HEVC Using Adaptive Decision Trees

High Efficiency Video Coding (HEVC) Standard, as the latest coding standard, introduces satisfying compression structures with respect to its predecessor Advanced Video Coding (H.264/AVC). The new coding standard can offer improved encoding performance compared with H.264/AVC. However, it also leads to enormous computational complexity that makes it considerably difficult to be implemented in real time application. In this paper, based on machine learning, a fast partitioning method is proposed, which can search for the best splitting structures for Intra-Prediction. In view of the video texture characteristics, we choose the entropy of Gray-Scale Difference Statistics (GDS) and the minimum of Sum of Absolute Transformed Difference (SATD) as two important features, which can make a balance between the computation complexity and classification performance. According to the selected features, adaptive decision trees can be built for the Coding Units (CU) with different size by offline training. Furthermore, by this way, the partition of CUs can be resolved as a binary classification problem. Experimental results have shown that the proposed algorithm can save over 34% encoding time on average, with a negligible Bjontegaard Delta (BD)-rate increase.

Depth Map Boundary Filter for Enhanced View Synthesis in 3D Video

In 3D video systems, view synthesis is performed at the receiver end using decoded texture and depth videos. Due to errors and noise, boundary pixels at coded depth data exhibit inaccuracy, which affects the rendering quality. In this paper, we propose a boundary refinement filter for coded depth data. Initially, we estimate the boundary region based on gradient magnitudes, using its standard deviation as a threshold. Consecutively, we replace the depth value at the boundary region with a weighted average by means of the proposed filter. Three weights are calculated in this process: depth similarity, distance, and boundary direction. Experiment results demonstrate that the proposed filter increases the PSNR of synthesized images. The improvements are confirmed subjectively as well. Hence, the quality of synthesized images is enhanced, aided by the proposed depth map filter.

Fast 3D-HEVC encoder algorithm for multiview video plus depth coding

The three-dimensional high efficiency video coding (3D-HEVC) is an extension of the HEVC standard for the compression of the multiview video plus depth (MVD) format. In 3D-HEVC, additional coding tools have been added to HEVC for improving the coding efficiency of the dependent video views and depth maps. Those tools achieve the highest possible coding efficiency, but also bring extremely high computational complexity which limits 3D-HEVC from real-time applications. In this paper, we propose a fast 3D-HEVC encoder algorithm based on the texture video and depth map correlation to reduce MVD coding computational complexity. Since the video texture and depth map represent the same scene at the same time instant (they have the same motion characteristics), it is not efficient to use all the prediction modes in 3D-HEVC mode decision procedure. The basic idea of the algorithm is to utilize the texture video and depth map property of coding information to predict the current CU prediction mode and early skip unnecessary variable-size mode decision. Experimental results demonstrate that the proposed fast 3D-HEVC encoder algorithm can save 75% computational complexity on average, while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.

Convergências e desafios para a pesquisa com imagens em movimento sob a perspectiva da análise do discurso multimodal

Neste artigo, fazemos uma revisão da literatura (Iedema, 2001; O’Halloran, 2004; Baldry e Thibault, 2007) em Análise do Discurso Multimodal em busca de categorias para segmentação e análise de imagens em movimento em gêneros diversos. O objetivo é estabelecer uma análise comparativa entre as propostas e subsidiar tanto a pesquisa quanto o ensino dos multiletramentos por meio de textos em vídeo. A partir da análise, ficou evidente que há um avanço significativo na área, pois os autores investigados apresentam uma sintonia conceitual na proposta de quatro unidades – Quadro, Tomada, Cena e Sequência. Identificamos, entretanto, alguns desafios para futuras pesquisas.

Proficient Character Recognition from Images

Reading text from photographs is a challenging problem that has received a significant amount of attention. Two key components of most systems are (i) text detection from images and (ii) text recognition, and many methods have been introduced to design better feature representations and models for both. Scene text recognition has gained significant attention from the computer vision community in recent years. Recognizing text is a difficult problem, even more so than the detection of scanned documents. To evaluate the performance of recent algorithms in detecting and recognizing text from complex images, In this proposed paper implement two method text detection and text recognition . The features extractors are Harris-Corner, Maximal Stable Extremal Regions (MSER), and dense sampling and Histogram of Oriented Gradients (HOG) descriptors. Then implement text recognition. The first one is training a character recognizer to predict the category of a character in an image patch. The second one is training a binary character classifier for each character class to predict the existence of this category in an image patch. The two schemes are compatible with two promising applications related to scene text, which are text understanding and text retrieval. Further we extend this concept with word level recognition with lexicon techniques with accurate results. And also recognition text in real time images, videos and mobile application images.

A depth video processing algorithm based on cluster dependent and corner-ware filtering

In free viewpoint video system, color video and the associated depth video are utilized to synthesize arbitrary virtual viewpoint. Hence, high quality of depth video is a necessity for virtual view rendering. However, depth video estimated by depth estimate software is inconsistent and inaccurate which decreases the quality of virtual views. To solve the problem, a depth video processing algorithm is proposed in this paper. Firstly, depth video is divided into five clusters adaptively by Fuzzy C-means clustering method. Meanwhile, the edges of depth video are detected and expanded into 8×8 block-wise. Secondly, for pixels in non-edge regions of depth video, a cluster dependent filtering method is adopted according to the feature of each cluster. Finally, corners in the corresponding texture video are detected. For pixels in edge regions of depth video, a corner-aware filtering method is used. Experimental results show that the proposed algorithm enhances the depth video and significantly improves the quality of the virtual views. The peak signal to noise ratio of virtual views rendered by using the proposed algorithm is 0.43dB higher than that of the virtual views rendered by using the original depth video. The proposed algorithm also outperforms Martin's algorithm in terms of virtual view rendering performance.

Texture-Aware Depth Prediction in 3D Video Coding

3D video has raised great interest in the last decade and currently a new 3D video coding standard, known as 3D video coding extension of High Efficiency Video Coding (3D-HEVC), has been developed. The standard investigates the coding of multiview video plus depth, which consists of texture videos and depth videos of multiple views. Depth video, as a description of geometry information of a scene, is generally composed of large flat regions separated by sharp edges. The conventional video coding may fail to generate an accurate prediction for units with sharp edges due to its block-based prediction which cannot compensate (minor) boundary changes well. In order to attack the problem, a new texture-aware depth inter-prediction method is proposed, which incorporates pixel-oriented weighting in the bi-prediction process by exploiting motion and structure similarities between texture and depth videos. Furthermore, such pixel-oriented weighting scheme can be extended to the uni-prediction process by considering more prediction blocks with small motion vector displacements. Experimental results demonstrate that the adapted 3D-HEVC codec with the proposed method can achieve better rate-distortion performance compared to the original 3D-HEVC standard codec.

Graph‐based approach to scene text localisation and tracking in videos

A text localisation and tracking method is presented that finds text-regions in videos and assigns unique IDs to their trajectories. For the goal, a graph-based framework that can work with existing text detection methods is developed. To be precise, graphs are built where vertices are image-level text detection results and edges represent the correspondence scores of the vertices. From these graphs, text-region trajectories by using the graph-cut algorithm are extracted. This approach allows considering false positives and misses, as well as their patch-based tracking results at the same time, and text trajectories are reliably extracted. Finally, the results are refined by interpolating misses and filtering out false positives. The proposed method is submitted to the International Conference on Document Analysis and Recognition 2015 robust reading competition (video text localisation) and the method showed the best performance in terms of CLEAR MOT metrics and was ranked third place according to VACE metrics among the seven participating methods.

Access to Nature Improves Well-Being

Access to Nature Improves Well-Being

Text Detection, Tracking and Recognition in Video: A Comprehensive Survey

The intelligent analysis of video data is currently in wide demand because a video is a major source of sensory data in our lives. Text is a prominent and direct source of information in video, while the recent surveys of text detection and recognition in imagery focus mainly on text extraction from scene images. Here, this paper presents a comprehensive survey of text detection, tracking, and recognition in video with three major contributions. First, a generic framework is proposed for video text extraction that uniformly describes detection, tracking, recognition, and their relations and interactions. Second, within this framework, a variety of methods, systems, and evaluation protocols of video text extraction are summarized, compared, and analyzed. Existing text tracking techniques, tracking-based detection and recognition techniques are specifically highlighted. Third, related applications, prominent challenges, and future directions for video text extraction (especially from scene videos and web videos) are also thoroughly discussed.