Original Video Stream Research Articles

New method for video stream brightness stabilization: algorithms and performance evaluation

Subject of study. In this paper, for the first time, an original method for estimating the change in the brightness of video data under the influence of changes in the lighting conditions of the scene and external noise is proposed. Algorithms for stabilizing the brightness of video data are also proposed. An objective assessment of the quality of video data pre-processed is given. The purpose of the research is to create a methodology for analyzing the variability of video data parameters under the influence of negative factors and to develop effective algorithms for stabilizing the parameters of the received video stream. The reliability of the method is tested using real video recordings pictured through various conditions. Objectives: To determine the most universal, resistant to external influences, and informative indicator necessary for an objective assessment of the quality of video data under various shooting conditions and scene lighting features; develop and programmatically implement algorithms for stabilizing video parameters based on modern programming tools. Research methods. Statistical analysis and pre-processing of video stream parameters as a random spatio-temporal process, algorithms for processing video data by digital filtering, and adaptive stabilization of video stream parameters. Research results. It has been proposed and experimentally proven that the optimal indicator of video stream quality is the average frame brightness (AFB). An algorithm for spatiotemporal processing of video data is proposed that generates a sequence of AFB values from the original video stream. The paper also proposes digital algorithms for filtering and stabilizing the brightness of a video stream and investigates the effectiveness of their application. Conclusions. The scientific novelty of the results obtained lies in a new method for analyzing and evaluating the parameters of video surveillance data and algorithms for filtering and stabilizing the brightness of the video stream. The performance of the proposed algorithms has been tested on real data. The algorithms are implemented in the Python software environment using the functions of the OpenCV library.

High-Embedded Low-Distortion Multihistogram Shift Video Reversible Data Hiding Based on DCT Coefficient

Video reversible data hiding technology can be applied to copyright protection, medical images, the military, and other fields, but it cannot guarantee high visual quality with an effective embedded capacity. In this paper, a high-embedding and low-distortion reversible data hiding scheme based on a discrete cosine transform (DCT) coefficients method is proposed. The scheme first decodes the original video stream with entropy, obtains all the DCT blocks, and selects the embeddable DCT blocks according to the capacity of the zero factor. Then, it divides the coefficients in the DCT blocks into the shift and embedding coefficients. The shift coefficients directly generate a one-dimensional histogram; the embedding coefficients generate a two-dimensional histogram according to paired strategies. Finally, the secret data can be successfully embedded according to the proposed two-dimensional histogram shift reversible data hiding scheme. This scheme performed more effectively than existing schemes in terms of the embedded capacity, peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM).

Rapid Defect Detection for Spacecraft in Infrared Reconstructed Images Using Image Mosaic Technique

In order to satisfy the real-time and large scale requirement of spacecraft nondestructive testing using optical pulsed thermography, in this research, we propose a novel approach based on image mosaic technique which can create large scale or panoramic image mosaics from a set of ordered infrared reconstruction images with overlapping areas for defect detection. First, infrared reconstruction images are extracted from the original thermal video stream by the ICA method based on the idea of blind source signal processing. Then for this research’s special mosaic object, a fast and accurate registration scheme is proposed. Abundant scale rotation invariant feature points are quickly obtained by using the hybrid feature detector and descriptor. Moreover, the feature matching process is realized by applying two-way FLANN and MSAC, parameters of the geometric transformation matrix are estimated by MSAC, and image mosaic is realized according to the geometric transformation model. The experimental results convinced the validity and efficiency of the proposed method.

HTTP/2-based Frame Discarding for Low-Latency Adaptive Video Streaming

In this article, we propose video delivery schemes insuring around 1s delivery latency with Dynamic Adaptive Streaming over HTTP (DASH), which is a standard version of HTTP Live Streaming (HLS), so as to benefit from the video representation switching between successive video segments. We also propose HTTP/2-based algorithms to apply video frame discarding policies inside a video segment when a selected DASH representation does not match with the available network resources. The current solutions with small buffer suffer from rebuffering events. Rebuffering not only impacts the Quality of Experience (QoE) but also increases the delivery delay between the displayed and the original video streams. In this work, we completely eliminate rebuffering events by developing optimal and practical video frame discarding algorithms to meet the 1s latency constraint. In all our algorithms, we request the video frames individually through HTTP/2 multiple streams, and we selectively drop the least meaningful video frames thanks to HTTP/2 stream resetting feature. Our simulations show that the proposed algorithms eliminate rebuffering while insuring an acceptable video quality with at least a Peak Signal to Noise Ratio (PSNR) of 35dB compared to 25dB of the basic First In First Out (FIFO) algorithm. We also quantify and qualify the resulting temporal distortion of the video segments per algorithm. An important number of missing video frames results in a temporal fluidity break known as video jitter . The displayed video looks like a series of snapshots. We show that both the optimal Integer Linear Program (ILP) and practical algorithms decrease the frequency and duration of the jitters. For example, practical algorithms reduce the number of crashed displayed videos (presenting one jitter longer than 1,350ms) with 22% compared to the basic FIFO algorithm. We also show that requesting video frames separately with HTTP/2 slightly increases the overhead from 4.34% to 5.76%.

A reliability enhanced video storage architecture in hybrid SLC/MLC NAND flash memory

Abstract Video-based applications have been widely used in embedded mobile devices. In such devices, NAND flash memory chip is normally used as the storage media to store videos. To enlarge the capacity of NAND flash memory, MLC (multi-level cell) technology is widely adopted with low storage reliability compared with the classical SLC (single-level cell) technology. Thus, the inherent features of NAND flash memory will significantly affect the storage reliability of video files. In this paper, we propose a reliability enhanced MPEG video storage architecture (REVSA) in hybrid SLC/MLC storage systems. In this architecture, the video stream is reconstructed as the key cluster and the non-key cluster according to the importance of video restoration. The key cluster is stored in SLC blocks to ensure the reliability, while the non-key cluster is stored in MLC blocks to utilize its capacity. To achieve this, we introduce a new layer in the storage hierarchy called video bit-stream analyzer, which includes a segmentation pipeline and a merging pipeline for MPEG video stream to build clusters and merge them to original video stream. We conducted experiments in a hybrid SLC/MLC storage system to store the car driving videos. The experimental results show that the proposed scheme can significantly improve the storage reliability of the MPEG video files and prolong the lifetime of the storage system.

Video Partitioning and Secured Keyframe Extraction of MPEG Video

Video partitioning and keyframe extraction (KFE) are the key foundations of video analysis and Content based video retrieval. The use of keyframes reduces the amount of data that is necessary in video indexing and provides the outline for dealing with the video content. In the last few years, many algorithms of keyframe extraction concentrated on the original compressed video stream. This can increase computational complexity when decompression is required before video processing. Keyframe is the frame, which can be a prototype of the salient content and information of the shot. The keyframes extracted must summarize the significant features of the video in time sequence. Therefore, there is a commensurate need of an efficient and secured keyframe selection technique in an efficient CBVR system. We propose an algorithm for keyframe extraction of compressed video shots using adaptive threshold method. Extensive computation on 200 plus video clips is performed and results are accurate and satisfactory.

Interface MB-Based Video Content Editing Transcoding

In practical multimedia systems, the content of coded video streams often needs to be re-edited at the nodes of transmitting networks. For example, logo insertion is always required for copyright protection at different local transmitting nodes. This kind of video stream editing is denoted as video content editing transcoding (VCET) in this paper. Though some techniques have been suggested for VCET, these methods cannot meet the requirement of dynamic transmitting bandwidth in practical applications. In this paper, we proposed an interface macroblock-based transcoding scheme for VCET, which can reuse the variable length codes of the original video streams as much as possible to achieve the best VCET quality. In order to ensure that the edited video streams can be transmitted by the original bandwidth, we also proposed a rate control algorithm for VCET, which can accurately control the bitrate of edited video streams according to the frame level coding bits of the original video streams. Experimental results showed that the proposed scheme achieved substantially better results in bitrate accuracy, computational complexities, and video quality than many other existing schemes.

Bandwidth efficient adaptive forward error correction mechanism with feedback channel

Multimedia content is very sensitive to packet loss and therefore multimedia streams are typically protected against packet loss, either by supporting retransmission requests or by adding redundant forward error correction (FEC) data. However, the redundant FEC information introduces significant additional bandwidth requirements, as compared to the bitrate of the original video stream. Especially on wireless and mobile networks, bandwidth availability is limited and variable. In this article, an adaptive FEC (A-FEC) system is presented whereby the redundancy rate is dynamically adjusted to the packet loss, based on feedback messages from the client. We present a statistical model of our A-FEC system and validate the proposed system under different packet loss conditions and loss probabilities. The experimental results show that 57-95% bandwidth gain can be achieved compared with a static FEC approach.

The power of connectivity: Identity preserving transformations on visual streams in the spike domain

We investigate neural architectures for identity preserving transformations (IPTs) on visual stimuli in the spike domain. The stimuli are encoded with a population of spiking neurons; the resulting spikes are processed and finally decoded. A number of IPTs are demonstrated including faithful stimulus recovery, as well as simple transformations on the original visual stimulus such as translations, rotations and zoomings. We show that if the set of receptive fields satisfies certain symmetry properties, then IPTs can easily be realized and additionally, the same basic stimulus decoding algorithm can be employed to recover the transformed input stimulus. Using group theoretic methods we advance two different neural encoding architectures and discuss the realization of exact and approximate IPTs. These are realized in the spike domain processing block by a “switching matrix” that regulates the input/output connectivity between the stimulus encoding and decoding blocks. For example, for a particular connectivity setting of the switching matrix, the original stimulus is faithfully recovered. For other settings, translations, rotations and dilations (or combinations of these operations) of the original video stream are obtained. We evaluate our theoretical derivations through extensive simulations on natural video scenes, and discuss implications of our results on the problem of invariant object recognition in the spike domain.

A Full Reference Algorithm for Dropped Frames Identification in Uncompressed Video Using Genetic Algorithm

Dropped frames identification in a given video stream is always a challenging task for research community due to required heavy computation. To preserve the quality of service during any processing over visual information, drop of frame is least desired. Many contemporary work identifies the frame drop in terms of full reference algorithm (where reference and distorted video streams are available for comparison), or reduced reference algorithm (where some information about reference video are available) or no reference algorithm (where information about reference video are not available). This paper presents a novel full reference heuristic approach using genetic algorithm which identifies the dropped frame indices in a given distorted video stream with respect to original video stream. The proposed algorithm efficiently identifies dropped frame indices even if reference video stream contains repeated frames and spatially distorted too with low or high spatial distortions. The proposed algorithm is simulated and tested with 12 video streams. Simulation results suggested that it is more efficient with a video stream having lesser repeated frames.

Automated real-time video surveillance summarization framework

Reviewing video surveillance contents for security monitoring is a time-consuming and time-limiting task. This paper presents a real-time video surveillance summarization framework intended for minimizing the time requirement for time critical tasks, based on compact moving objects in time---space. A tunnel is proposed as an individual time-dimension object. In order to summarize an endless video into a shorter duration without loss of selected targets so as to extend the understanding of any given individual object, this research utilizes three real-time algorithms. Direct shift collision detection (DSCD) is implemented for the extremely fast shifting of tunnels together in time---space. The DSCD summarized video can then be customized by technique from many different approaches. Here, early trajectory searching is applied with the same DSCD technique, and then direct distance transform is used to instantly give the trajectory similarity between tunnels and the user's query. The most important step for identifying each individual object is background subtraction. To this end, dynamic region adaptation (DRA) was used as the background subtraction algorithm to select the best foreground for each object before making a tunnel. DRA also helps DSCD to summarize the video more accurately. The proposed framework is able to provide the results by real-time performance approach without losing the major events of the original video stream.

Effect of node churn on frame interval for peer-to-peer video streaming with data-block synchronization mechanism

Coolstreaming is a mesh based peer-to-peer (P2P) video streaming system in which single video stream is decomposed into multiple sub-streams. A client-peer node retrieves the sub-streams from multiple parent-peer nodes, combining them into the original video stream. Each client-peer node has two buffers, a synchronization buffer and a cache buffer, and arriving data blocks are synchronized at the synchronization buffer and then forwarded to the cache buffer. In this buffering system, data-block synchronization is important to guarantee high video quality. In this paper, we consider the effect of churn on the performance of data-block synchronization scheme with which data blocks are simultaneously forwarded just after all the data blocks composing a macro data block arrive at the synchronization buffer. It is assumed that data blocks belonging to a sub-stream arrive at the client-peer node according to an interrupted Poisson process. The synchronization buffer is modeled as a multiple-buffer queueing system with homogeneous interrupted Poisson processes, and the mean forwarding interval is derived. Numerical examples show that the average forwarding interval increases as parent-peer nodes leave more frequently.

Queueing analysis of data block synchronization mechanism in peer-to-peer based video streaming system

In a Peer-to-Peer (P2P) based video streaming system such as Coolstreaming, a single video stream is decomposed into multiple sub-streams. A client-peer node receives the sub-streams from multiple parent-peer nodes, combining them into the original video stream. Each client-peer node has a synchronization buffer and a cache buffer. Data blocks are stored in the synchronization buffer in a sub-stream basis, and then forwarded into the cache buffer according to their sequence numbers. In this buffering system, data-block synchronization plays a crucial role to guarantee video quality. In this paper, we consider the performance of data-block synchronization scheme with which data blocks are simultaneously forwarded just after all the data blocks composing a macro data block arrive at the synchronization buffer. We model the synchronization buffer as a multiple-buffer queueing system with homogeneous Poisson arrival processes, deriving the mean forwarding interval. We also consider the frame loss probability for multiple-path video streaming, investigating how the number of sub-streams decreases the frame loss probability. Numerical examples show that increasing the number of sub-streams makes the average forwarding interval large, while the frame loss probability at the bottleneck router is improved. It is also shown that increasing the synchronization buffer decreases the average forwarding interval.

Queueing analysis of data block synchronization mechanism in peer-to-peer based video streaming system

In a Peer-to-Peer (P2P) based video streaming system such as Coolstreaming, a single video stream is decomposed into multiple sub-streams. A client-peer node receives the sub-streams from multiple parent-peer nodes, combining them into the original video stream. Each client-peer node has a synchronization buffer and a cache buffer. Data blocks are stored in the synchronization buffer in a sub-stream basis, and then forwarded into the cache buffer according to their sequence numbers. In this buffering system, data-block synchronization plays a crucial role to guarantee video quality. In this paper, we consider the performance of data-block synchronization scheme with which data blocks are simultaneously forwarded just after all the data blocks composing a macro data block arrive at the synchronization buffer. We model the synchronization buffer as a multiple-buffer queueing system with homogeneous Poisson arrival processes, deriving the mean forwarding interval. We also consider the frame loss probability for multiple-path video streaming, investigating how the number of sub-streams decreases the frame loss probability. Numerical examples show that increasing the number of sub-streams makes the average forwarding interval large, while the frame loss probability at the bottleneck router is improved. It is also shown that increasing the synchronization buffer decreases the average forwarding interval.

Design of an IPTV Multicast System for Internet Backbone Networks

The design of an IPTV multicast system for the Internet backbone network is presented and explored through extensive simulations. In the proposed system, a resource reservation algorithm such as RSVP, IntServ, or DiffServ is used to reserve resources (i.e., bandwidth and buffer space) in each router in an IP multicast tree. Each router uses an Input-Queued, Output-Queued, or Crosspoint-Queued switch architecture with unity speedup. A recently proposedRecursive Fair Stochastic Matrix Decompositionalgorithm used to compute near-perfect transmission schedules for each IP router. The IPTV traffic is shaped at the sources usingApplication-Specific Token Bucker Traffic Shapers, to limit the burstiness of incoming network traffic. The IPTV traffic is shaped at the destinations usingApplication-Specific Playback Queues, to remove residual network jitter and reconstruct the original bursty IPTV video streams at each destination. All IPTV traffic flows are regenerated at the destinations with essentially zero delay jitter and essentially-perfect QoS. The destination nodes deliver the IPTV streams to the ultimate end users using the same IPTV multicast system over a regional Metropolitan Area Network. It is shown that all IPTV traffic is delivered with essentially-perfect end-to-end QoS, with deterministic bounds on the maximum delay and jitter on each video frame. Detailed simulations of an IPTV distribution system, multicasting several hundred high-definition IPTV video streams over several essentially saturated IP backbone networks are presented.

An automated system for monitoring the visual quality and authenticity of satellite video streams using a fragile watermarking approach

This study presents an automatic, fragile-watermarking-based scheme for monitoring the visual quality and authenticity of video streams broadcast by a communications satellite. In the proposed approach, watermark information is embedded in designated motion vectors within the frames of the original video stream prior to it encryption, modulation and transmission to the satellite. Following the reception of the video stream at a terrestrial monitoring system, the watermark information is extracted and compared to that embedded in the original frames. It is shown that by monitoring the mean square error (MSE) of the extracted watermark relative to the original version, reliable estimates of the visual quality of the transmitted video stream can be obtained. Furthermore, the experimental results indicate that the watermarking scheme successfully identifies various common forms of interference, namely recompression attacks, frame removals, frame substitutions, and so forth. Therefore, by monitoring the watermark extraction results on a continuous basis, the degree of signal interference can be automatically determined and an appropriate alarm signal generated to prompt the necessary response.

User and Device Adaptation in Summarizing Sports Videos

Video summarization is defined as creating a video summary which includes only important scenes in the original video streams. In order to realize automatic video summarization, the significance of each scene needs to be determined. When targeted especially on broadcast sports videos, a play scene, which corresponds to a play, can be considered as a scene unit. The significance of every play scene can generally be determined based on the importance of the play in the game. Furthermore, the following two issues should be considered: 1) what is important depends on each user's preferences, and 2) the summaries should be tailored for media devices that each user has. Considering the above issues, this paper proposes a unified framework for user and device adaptation in summarizing broadcast sports videos. The proposed framework summarizes sports videos by selecting play scenes based on not only the importance of each play itself but also the users' preferences by using the metadata, which describes the semantic content of videos with keywords, and user profiles, which describe users' preference degrees for the keywords. The selected scenes are then presented in a proper way using various types of media such as video, image, or text according to device profiles which describe the device type. We experimentally verified the effectiveness of user adaptation by examining how the generated summaries are changed by different preference degrees and by comparing our results with/without using user profiles. The validity of device adaptation is also evaluated by conducting questionnaires using PCs and mobile phones as the media devices.

Automatic personalized video abstraction for sports videos using metadata

Video abstraction is defined as creating a video abstract which includes only important information in the original video streams. There are two general types of video abstracts, namely the dynamic and static ones. The dynamic video abstract is a 3-dimensional representation created by temporally arranging important scenes while the static video abstract is a 2-dimensional representation created by spatially arranging only keyframes of important scenes. In this paper, we propose a unified method of automatically creating these two types of video abstracts considering the semantic content targeting especially on broadcasted sports videos. For both types of video abstracts, the proposed method firstly determines the significance of scenes. A play scene, which corresponds to a play, is considered as a scene unit of sports videos, and the significance of every play scene is determined based on the play ranks, the time the play occurred, and the number of replays. This information is extracted from the metadata, which describes the semantic content of videos and enables us to consider not only the types of plays but also their influence on the game. In addition, user's preferences are considered to personalize the video abstracts. For dynamic video abstracts, we propose three approaches for selecting the play scenes of the highest significance: the basic criterion, the greedy criterion, and the play-cut criterion. For static video abstracts, we also propose an effective display style where a user can easily access target scenes from a list of keyframes by tracing the tree structures of sports games. We experimentally verified the effectiveness of our method by comparing our results with man-made video abstracts as well as by conducting questionnaires.

Essential Information Aggregation of intra coded video frame for wireless transmisson

A novel error resilient method based on Essential Information Aggregation (EIA) technique is proposed for robust wireless transmission of intra coded video frames. This method modifies the original video stream structure so as to mitigate the effect of decoding loss of synchronization within a video frame, for which Macro Block Aligned (MBA) error detection method is accompanied to be developed. Experiments on an Additive White Gaussian Noise (AWGN) wireless simulation platform and H.263+ baseline video codec show that it can obtain better subjective and objective image quality than existing error resilience approaches.

An Error Detection Method Based on Coded Block Pattern Information Verification for Wireless Video Communication

A novel error detection method based on coded block pattern (CBP) information verification is proposed for error concealment of inter-coded video frames transmitted in wireless channel. This method firstly modifies the original video stream structure by the aggregation of certain important information, and then inserts some error verification bits into the video stream for each encoded macro block (MB), these bits can be used as reference information to determine whether each encoded MB is corrupted. Experimental results on additive Gauss white noise simulation wireless channel and H.263+ baseline codec show that the proposed method can outperform other reference approaches on error detection performance. In addition, it can preserve the original video quality with a small coding overhead increase.