Video Bitstream Research Articles

An Advanced Moving Object Detection Algorithm for Automatic Traffic Monitoring in Real-World Limited Bandwidth Networks

Automated motion detection technology is an integral component of intelligent transportation systems, and is particularly essential for management of traffic and maintenance of traffic surveillance systems. Traffic surveillance systems using video communication over real-world networks with limited bandwidth often encounter difficulties due to network congestion and/or unstable bandwidth. This is especially problematic in wireless video communication. This has necessitated the development of a rate control scheme which alters the bit-rate to match the obtainable network bandwidth, thereby producing variable bit-rate video streams. However, complete and accurate detection of moving objects under variable bit-rate video streams is a very difficult task. In this paper, we propose an approach for motion detection which utilizes an analysis-based radial basis function network as its principal component. This approach is applicable not only in high bit-rate video streams, but in low bit-rate video streams, as well. The proposed approach consists of a various background generation stage and a moving object detection stage. During the various background generation stage, the lower-dimensional Eigen-patterns and the adaptive background model are established in variable bit-rate video streams by using the proposed approach in order to accommodate the properties of variable bit-rate video streams. During the moving object detection stage, moving objects are extracted via the proposed approach in both low bit-rate and high bit-rate video streams; detection results are then generated through the output value of the proposed approach. The detection results produced through our approach indicate it to be highly effective in variable bit-rate video streams over real-world limited bandwidth networks. In addition, the proposed method can be easily achieved for real-time application. Quantitative and qualitative evaluations demonstrate that it offers advantages over other state-of-the-art methods. For instance, Similarity and F1 accuracy rates produced via the proposed approach were up to 86.38% and 89.88% higher than those produced via other compared methods, respectively.

Non-Stationary Resource Allocation Policies for Delay-Constrained Video Streaming: Application to Video over Internet-of-Things-Enabled Networks

Due to the high bandwidth requirements and stringent delay constraints of multi-user wireless video transmission applications, ensuring that all video senders have sufficient transmission opportunities to use before their delay deadlines expire is a longstanding research problem. We propose a novel solution that addresses this problem without assuming detailed packet-level knowledge, which is unavailable at resource allocation time (i.e. prior to the actual compression and transmission). Instead, we translate the transmission delay deadlines of each sender's video packets into a monotonically-decreasing weight distribution within the considered time horizon. Higher weights are assigned to the slots that have higher probability for deadline-abiding delivery. Given the sets of weights of the senders' video streams, we propose the low-complexity Delay-Aware Resource Allocation (DARA) approach to compute the optimal slot allocation policy that maximizes the deadline-abiding delivery of all senders. A unique characteristic of the DARA approach is that it yields a non-stationary slot allocation policy that depends on the allocation of previous slots. This is in contrast with all existing slot allocation policies such as round-robin or rate-adaptive round-robin policies, which are stationary because the allocation of the current slot does not depend on the allocation of previous slots. We prove that the DARA approach is optimal for weight distributions that are exponentially decreasing in time. We further implement our framework for real-time video streaming in wireless personal area networks that are gaining significant traction within the new Internet-of-Things (IoT) paradigm. For multiple surveillance videos encoded with H.264/AVC and streamed via the 6tisch framework that simulates the IoT-oriented IEEE 802.15.4e TSCH medium access control, our solution is shown to be the only one that ensures all video bitstreams are delivered with acceptable quality in a deadline-abiding manner.

Double-layer video transmission over decode-and-forward wireless relay networks using hierarchical modulation.

We consider a wireless relay network with a single source, a single destination, and a multiple relay. The relays are half-duplex and use the decode-and-forward protocol. The transmit source is a layered video bitstream, which can be partitioned into two layers, a base layer (BL) and an enhancement layer (EL), where the BL is more important than the EL in terms of the source distortion. The source broadcasts both layers to the relays and the destination using hierarchical 16-QAM. Each relay detects and transmits successfully decoded layers to the destination using either hierarchical 16-QAM or QPSK. The destination can thus receive multiple signals, each of which can include either only the BL or both the BL and the EL. We derive the optimal linear combining method at the destination, where the uncoded bit error rate is minimized. We also present a suboptimal combining method with a closed-form solution, which performs very close to the optimal. We use the proposed double-layer transmission scheme with our combining methods for transmitting layered video bitstreams. Numerical results show that the double-layer scheme can gain 2-2.5 dB in channel signal-to-noise ratio or 5-7 dB in video peak signal-to-noise ratio, compared with the classical single-layer scheme using conventional modulation.

Architecture for video streaming application on heterogeneous platform

In this paper, we propose a new architecture, which is an efficient streaming media player application on heterogeneous platform. The streaming library can be used in design for reducing the memory bandwidth on processing RTSP/RTP/RTCP network protocols. And the proposed improvement has to do with the I-frame encoding alone. The architecture can receive higher rate for data transfer and packet loss in embedded system. The framework of the key components is able to adopt Direct Memory Access to reduce the time-consumption resulted from the communication between the dual cores. On the other hand, the approach of the dynamic quality adaption improves the video bitstream by periodically adjusting the values of encoding quality parameters. Through the experiment results, it is evident that the new video streaming architecture greatly enhances the coding efficacy. Our experimental results present that the decoding/encoding speed of the dual-core CPU embedded with Direct Memory Access is enhanced up to 50 %, and its usage of CPU resources and memory bandwidth are 50 % lower than that of the popular JRTPLIB.

Full-reference video quality metric assisted the development of no-reference bitstream video quality metrics for real-time network monitoring

High-quality video is being increasingly delivered over Internet Protocol networks, which means that network operators and service providers need methods to measure the quality of experience (QoE) of the video services. In this paper, we propose a method to speed up the development of no-reference bitstream objective metrics for estimating QoE. This method uses full-reference objective metrics, which makes the process significantly faster and more convenient than using subjective tests. In this process, we have evaluated six publicly available full-reference objective metrics in three different databases, the EPFL-PoliMI database, the HDTV database, and the Live Video Wireless database, all containing transmission distortions in H.264 coded video. The objective metrics could be used to speed up the development process of no-reference real-time video QoE monitoring methods that are receiving great interest from the research community. We show statistically that the full-reference metric Video Quality Metric (VQM) performs best considering all the databases. In the EPFL-PoliMI database, SPATIAL MOVIE performed best and TEMPORAL MOVIE performed worst. When transmission distortions are evaluated, using the compressed video as the reference provides greater accuracy than using the uncompressed original video as the reference, at least for the studied metrics. Further, we use VQM to train a lightweight no-reference bitstream model, which uses the packet loss rate and the interval between instantaneous decoder refresh frames, both easily accessible in a video quality monitoring system.

A Video Saliency Detection Model in Compressed Domain

Saliency detection is widely used to extract regions of interest in images for various image processing applications. Recently, many saliency detection models have been proposed for video in uncompressed (pixel) domain. However, video over Internet is always stored in compressed domains, such as MPEG2, H.264, and MPEG4 Visual. In this paper, we propose a novel video saliency detection model based on feature contrast in compressed domain. Four types of features including luminance, color, texture, and motion are extracted from the discrete cosine transform coefficients and motion vectors in video bitstream. The static saliency map of unpredicted frames (I frames) is calculated on the basis of luminance, color, and texture features, while the motion saliency map of predicted frames (P and B frames) is computed by motion feature. A new fusion method is designed to combine the static saliency and motion saliency maps to get the final saliency map for each video frame. Due to the directly derived features in compressed domain, the proposed model can predict the salient regions efficiently for video frames. Experimental results on a public database show superior performance of the proposed video saliency detection model in compressed domain.

Rate Allocation Algorithm with Successive Refinement in Peer-to-Peer Networks

We introduce a new across-peer rate allocation algorithm with successive refinement to improve the video transmission performance in P2P networks, based on the combination of multiple description coding and network coding. Successive refinement is implemented through layered multiple description codes. The algorithm is developed to maximize the expected video quality at the receivers by partitioning video bitstream into different descriptions depending on different bandwidth conditions of each peer. Adaptive rate partition adjustment is applied to ensure the real reflection of the packet drop rate in the network. Also the granularity is changed to the scale of atomic blocks instead of stream rates in prior works. Through simulation results we show that the algorithm outperforms prior algorithms in terms of video playback quality at the peer ends, and helps the system more adjustable to the peer dynamics.

FPGA Implementation of MIMO System using Xilinx System for Video Transmission

In this paper, we have introduced an architecture for real-time video transmission over multiple- input multiple-output (MIMO) wireless communication systems. We employ the 2x1 and 2x2 Alamouti MIMO technique to develop a transmission system and implement the design on FPGA using Xilinx System using the modeling and simulation capabilities of MATLAB. Furthermore, according to the contribution to the reconstructed video quality, we apply unequal error protection strategy using STBC space-time codes for each video bitstream. We present a solution using FPGA as target platform (instead of software), for implementing algorithms for transmission of educational videos using MIMO systems. In our proposed system, we have provided the system performance of a joint MPEG-2 coding scheme with convolutional channel coding and space time block coding (STBC) techniques, associated with suitable modulation method (QAM and BPSK), for video data transmission over a wireless MIMO system with Rayleigh fading noises. Simulation results show the quality of the reconstructed image can be significantly improved over only using space time coding. The comparison has been done on the basis of quality of reproduced image by measuring image PSNR based on SNR and BER value for different system model is realized over Xilinx Virtex-4 XC4VFX100FFG1517-11C FPGA based WRAP board. Keywords: FPGA (Field Programmable Gate Arrays); STBC (Space Time Block Coding), Bit error rate(BER), FPGA, Peak to signal noise ratio (PSNR), Multiple input multiple output (MIMO), WRAP board, Joint source channel coding, Space time block coding(STBC).

Confidentiality of a selectively encrypted H.264 coded video bit-stream

It is an assumption that selective encryption does not strongly protect confidentiality owing to the partial visibility of some video data. This is because, though encryption keys may be difficult to derive, an enhanced version of selectively encrypted video sequence might be found from knowledge of the unencrypted parts of the sequence. An efficient selective encryption method for syntax elements of H.264 encoded video was recently proposed at the entropy coding stage of an H.264 encoder. Using this recent scheme as an example, the purpose of this paper is a comprehensive cryptanalysis of selectively encrypted H.264 bit-streams to contradict the previous assumption that selective encryption is vulnerable. The novel cryptanalysis methods presented in this paper analyze the ability of an attacker to improve the quality of the encrypted video stream to make it watchable. The conclusion is drawn that if the syntax elements for selective encryption are chosen using statistical and structural characteristics of the video, then the selective encryption method is secure. The cryptanalysis is performed by taking into account the probability distribution of syntax elements within the video sequence, the relationship of syntax elements with linear regression analysis and the probability of successfully attacking them in order to enhance the visual quality. The results demonstrate the preservation of distorted video quality even after considering many possible attacks on: the whole video sequence; each video frame; and on small video segments known as slices.

Highly accurate moving object detection in variable bit rate video-based traffic monitoring systems.

Automated motion detection, which segments moving objects from video streams, is the key technology of intelligent transportation systems for traffic management. Traffic surveillance systems use video communication over real-world networks with limited bandwidth, which frequently suffers because of either network congestion or unstable bandwidth. Evidence supporting these problems abounds in publications about wireless video communication. Thus, to effectively perform the arduous task of motion detection over a network with unstable bandwidth, a process by which bit-rate is allocated to match the available network bandwidth is necessitated. This process is accomplished by the rate control scheme. This paper presents a new motion detection approach that is based on the cerebellar-model-articulation-controller (CMAC) through artificial neural networks to completely and accurately detect moving objects in both high and low bit-rate video streams. The proposed approach is consisted of a probabilistic background generation (PBG) module and a moving object detection (MOD) module. To ensure that the properties of variable bit-rate video streams are accommodated, the proposed PBG module effectively produces a probabilistic background model through an unsupervised learning process over variable bit-rate video streams. Next, the MOD module, which is based on the CMAC network, completely and accurately detects moving objects in both low and high bit-rate video streams by implementing two procedures: 1) a block selection procedure and 2) an object detection procedure. The detection results show that our proposed approach is capable of performing with higher efficacy when compared with the results produced by other state-of-the-art approaches in variable bit-rate video streams over real-world limited bandwidth networks. Both qualitative and quantitative evaluations support this claim; for instance, the proposed approach achieves Similarity and F1 accuracy rates that are 76.40% and 84.37% higher than those of existing approaches, respectively.

Recovery of Lost Motion Vectors Using Encoded Residual Signals

Motion-vector-based error concealment algorithms recover lost blocks by motion compensation with the estimated motion vectors. Traditionally, the estimated motion vectors for the lost block are derived by the direct usage of the neighboring available motion vectors. In this work, the residual signal in a block encoded in the video bitstream is used to measure the reliability of the corresponding motion vector. The residual information is applied to weight the neighboring available motion vectors differently, to further improve a newly proposed mathematical formulation between encoded motion vectors and extrapolated motion vectors. The proposed error concealment method using the residual information outperforms a state-of-the-art error concealment algorithm by up to 2.12 dB in sequence PSNR, and outperforms a classic DMVE method by up to 8.27 dB.

Flexible Multi-Core Platform for a Multiple-Format Video Decoder

This paper presents a new multi-core platform that can decode various video compression formats including MPEG-2, MPEG-4, AVS, and H.264/AVC. The developed multi-core platform consists of multiple media cores that are designed based on an application-specific instruction-set processor (ASIP). To improve the decoding speed of each media core, we developed several designated instructions that are useful for decoding various video codecs. In addition, an inter-connected hardware structure and a new synchronization algorithm are proposed to reduce inter-core communication overheads and a shared memory contention on the multi-core platform. We achieved a speed-up of 2.2× in decoding video bitstreams using several designated instructions on the media core. Furthermore, we achieved a speed-up of 5.56× in decoding video bitstreams on the multi-core platform by employing macroblock-row level parallelism, compared with the developed media core without designated instructions. The developed multi-core platform was implemented on a Xilinx Virtex5 LX330 field-programmable gate array (FPGA) that operates at 60 MHz.

Constructing a No-Reference H.264/AVC Bitstream-Based Video Quality Metric Using Genetic Programming-Based Symbolic Regression

In order to ensure optimal quality of experience toward end users during video streaming, automatic video quality assessment becomes an important field-of-interest to video service providers. Objective video quality metrics try to estimate perceived quality with high accuracy and in an automated manner. In traditional approaches, these metrics model the complex properties of the human visual system. More recently, however, it has been shown that machine learning approaches can also yield competitive results. In this paper, we present a novel no-reference bitstream-based objective video quality metric that is constructed by genetic programming-based symbolic regression. A key benefit of this approach is that it calculates reliable white-box models that allow us to determine the importance of the parameters. Additionally, these models can provide human insight into the underlying principles of subjective video quality assessment. Numerical results show that perceived quality can be modeled with high accuracy using only parameters extracted from the received video bitstream.

움직임 벡터의 정보 숨김을 이용한 H.264/AVC의 향상된 오류 검출 방법

제한된 통신 채널을 통한 실시간 전송을 위해 비디오 데이터의 압축이 필요하다. 압축된 영상 비트열은 전송 중에 발생되는 오류에 아주 민감하다. 전송 과정에서 오류가 발생된 패킷을 받게 되면, 복호기에서는 현재 입력되는 프레임을 잘못 재구성하게 되고 영상의 시공간적 특성에 의해 연속되는 프레임으로 전파된다. 이런 복호 영상의 화질 저하를 줄일 수 있는 방법으로 오류 검출과 오류 은닉이 있다. 이 논문에서는 오류 검출률을 높이기 위해 정보 숨김을 이용한 새로운 오류 검출 방법을 제안한다. H.264/AVC의 화면간 예측에서 발생하는 각 매크로블록의 움직임 벡터의 차분값에 특정한 정보를 숨겨 부호화하였다. 복호기에서는 전송받은 특정 정보의 체크를 통해 오류의 위치를 쉽게 찾아낼 수 있도록 하였다. H.324M 모바일 실험 도구를 이용한 실험 환경을 통해 제안한 알고리즘이 PSNR과 주관적 화질 측면에서 좋은 성능을 보임을 확인하였다. The compression of video data is intended for real-time transmission of band-limited channels. Compressed video bit-streams are very sensitive to transmission error. If we lose packets or receive them with errors during transmission, not only the current frame will be corrupted, but also the error will propagate to succeeding frames due to the spatio-temporal predictive coding structure of sequences. Error detection and concealment is a good approach to reduce the bad influence on the reconstructed visual quality. To increase concealment efficiency, we need to get some more accurate error detection algorithm. In this paper, We hide specific data into the motion vector difference of each macro-block, which is obtained from the procedure of inter prediction mode in H.264/AVC. Then, the location of errors can be detected easily by checking transmitted specific data in decoder. We verified that the proposed algorithm generates good performances in PSNR and subjective visual quality through the computer simulation by H.324M mobile simulation tool.

Toward Deployable Methods for Assessment of Quality for Scalable IPTV Services

Scalable video coding (H.264 SVC) is an attractive option for video service providers due to its ability to adapt a video's bitstream at the server to suit different network conditions and device characteristics. Lowering a video's bitrate can be achieved through reductions in frame rate, spatial resolution, and/or by increasing the quantization levels applied to the video sequence. In this paper, we first evaluate the effects of such scalability using some full-reference and no-reference video quality metrics, namely PSNR, SSIM, blocking, and blurring. No-reference metrics have the ability to capture the degradation in video quality caused by employing scalability in one or more dimensions. We study if conclusions drawn in previous works, which are based on well-known test video content, hold true for real-world broadcast content. We then discuss how, using these results for a particular content type, the use of no-reference metrics can be enabled in place of, or to supplement, existing widely used full-reference quality assessment metrics. We conduct an experimental analysis by transmitting video encoded at different scalability points over a lossy network to ascertain the effect of loss when scalability is employed in one or more dimensions. We analyze these results using a reduced reference metric called delta-blocking, which can detect visual damage of frames that causes a decrease in a user's quality of experience when perceived by the user. If the levels of packet loss are excessively high, this can lead the decoder to drop some video frames. To combat this type of frame loss, we propose a simple windowing algorithm that can automatically re-align the corresponding values for reduced-reference comparison, allowing for video quality monitoring to continue.

Region-of-interest determination and bit-rate conversion for H.264 video transcoding

Abstract This paper presents a video bit-rate transcoder for baseline profile in H.264/AVC standard to fit the available channel bandwidth for the client when transmitting video bit-streams via communication channels. To maintain visual quality for low bit-rate video efficiently, this study analyzes the decoded information in the transcoder and proposes a Bayesian theorem-based region-of-interest (ROI) determination algorithm. In addition, a curve fitting scheme is employed to find the models of video bit-rate conversion. The transcoded video will conform to the target bit-rate by re-quantization according to our proposed models. After integrating the ROI detection method and the bit-rate transcoding models, the ROI-based transcoder allocates more coding bits to ROI regions and reduces the complexity of the re-encoding procedure for non-ROI regions. Hence, it not only keeps the coding quality but improves the efficiency of the video transcoding for low target bit-rates and makes the real-time transcoding more practical. Experimental results show that the proposed framework gets significantly better visual quality.

H.264/AVC video watermarking for active fingerprinting based on Tardos code

In this paper, we present a novel approach for active fingerprinting of state-of-the-art video codec H.264/AVC. Tardos probabilistic fingerprinting code is embedded in H.264/AVC video signals using spread spectrum watermarking technique in both luma and chroma. Tardos code is embedded in intra as well as inter frames. The embedding has been performed in the nonzero quantized transformed coefficients, which are above a certain threshold while taking into account the reconstruction loop, to avoid the uncontrollable increase in bitrate of video bitstream. A comprehensive analysis of payload and PSNR trade-off is presented for the benchmark video sequences. Different linear and nonlinear collusion attacks have been performed in the pixel domain to show the robustness of the proposed approach.

Spatiotemporal saliency detection and salient region determination for H.264 videos

In this study, a spatiotemporal saliency detection and salient region determination approach for H.264 videos is proposed. After Gaussian filtering in Lab color space, the phase spectrum of Fourier transform is used to generate the spatial saliency map of each video frame. On the other hand, the motion vector fields from each H.264 compressed video bitstream are backward accumulated. After normalization and global motion compensation, the phase spectrum of Fourier transform for the moving parts is used to generate the temporal saliency map of each video frame. Then, the spatial and temporal saliency maps of each video frame are combined to obtain its spatiotemporal saliency map using adaptive fusion. Finally, a modified salient region determination scheme is used to determine salient regions (SRs) of each video frame. Based on the experimental results obtained in this study, the performance of the proposed approach is better than those of two comparison approaches.

Constrained in-loop filtering for error resiliency in high efficiency video coding

In error-prone environments, packets could be lost during the transmission of compressed video bitstream, and corrupted errors by the lost packets could be propagated until it meets a refresh point. To defect the error propagation, constrained intra-prediction in high efficiency video coding (HEVC) performs intra-prediction using only neighboring intra-coded samples. However, the errors could be propagated to intra-coded blocks by an in-loop filtering process, since the in-loop filtering is performed regardless of the prediction modes of blocks, even though constrained intra-prediction is used. In this paper, a constrained in-loop filtering is proposed to protect intra-coded samples from error propagation by adaptively applying filters depending on the prediction modes of reconstructed blocks. Simulation results show that the proposed method alleviates the distortion of the decoded pictures and thereby improves both subjective and objective quality compared with the HEVC standard.

Restoring Video Bitstream Integrity by Resolving Field Reversal and Mixed Pulldown Errors

The quality of the decoded video is affected by errors occurring at various stages of the production chain. In this paper, the errors occurring in the editing layer, due to the coexistence of different video standards in the broadcast market are addressed. The problems investigated are field reversal and mixed pulldown. Field reversal is caused when the interlaced video fields are not shown in the same order as they were captured. This results in a shaky video display, as the fields are not displayed in a chronological order. Whereas, mixed pulldown artefacts occur when the video frame-rate is up-sampled and down-sampled. These errors destroy the integrity of the bitstream and continue to corrupt the videos with which it is edited. The current solution is to visually inspect the video stream. This paper investigates these errors and describes a new technique that eliminates the visual inspection process and automatically restores the bitstream integrity. The methods are based on the principles of motion flow estimation and image correlation. The new algorithms were designed for real-time implementation and outperform existing techniques with superior performance in quality and speed.