Decoded Video Sequence Research Articles

A video hard cut detection using multifractal features

Efficient management of video sequences is based on adequate video content description. This description can be used for various purposes in different applications, telecommunication services, video and multimedia systems. Video hard cut detection represents the foundation of temporal video segmentation. In this paper, a new video hard cut detection methodology is proposed using multifractal features. Transition between two shots can be described as color and texture differences within a decoded video sequence. In the proposed methodology we formed specific structures by measuring color differences between frames. The formed structures are used for hard cut candidate detection. This is followed by multifractal representation of texture changes by Holder exponents. The proposed methodology achieves high performance using more than 750,000 frames, extracted from forty different video sequences, classified by four well known genre groups. Moreover, the proposed hard cut detection achieves high performance regardless of high level video production or complex non-linear editing for different genre groups. This is confirmed by comparison between the proposed methodology and other recent work on hard cut detection.

Study of the Subjective Visibility of Packet Loss Artifacts in Decoded Video Sequences

Packet loss is a significant cause of visual impairments in video broadcasting over packet-switched networks. There are several subjective and objective video quality assessment methods focused on the overall perception of video quality. However, less attention has been paid on the visibility of packet loss artifacts appearing in spatially and temporally limited regions of a video sequence. In this paper, we present the results of a subjective study, using a methodology where a video sequence is displayed on a touchscreen and the users tap it in the positions where they observe artifacts. We also analyze the objective features derived from those artifacts, and propose different models for combining those features into an objective metric for assessing the noticeability of the artifacts. The practical results show that the proposed metric predicts visibility of packet loss impairments with a reasonable accuracy. The proposed method can be applied for developing packetization and error recovery schemes to minimize the subjectively experienced distortion in error-prone networked video systems.

High efficient video coding using weighted entropy with optimized quantization matrix

As the concept of quantization matrix becomes an important feature in recent video CODECs, an optimized quantization matrix is being considered in the High-Efficiency Video Coding (HEVC) standard. This paper describes the entropy encoding by familiarizing optimized quantization matrix, and so higher rate of compression can be accomplished over the improved entropy encoding. Experiments show that for the eight benchmark video sequences and PSNR for varying rate of data transmission is explored. Comparative analysis is made with the improved (WE-Encoding) and standard entropy encoding based on the performance measurements. The simulation results show that the proposed method (WE-OQM) can save the originality of the decoded video sequence far better even though the compression rate is increased. In addition, the overall analysis states that the proposed method is 35.29% better than the Standard Encoding and 62.5% better than the WE-Encoding.

An optimized parallel order scheme of the deblocking filtering process for enhancing the performance of the HEVC standard using GPUs

In HEVC, deblocking filtering (DF) is responsible for about 20% of the time consumed to perform video compression. In a typical parallel DF scheme, a set of horizontal and vertical edges are processed using deblocking filters. In conventional parallel DF schemes, deblocking filters could be applied to the same edges more than once. Moreover, some edges are assigned to cores to be filtered even though those edges are not designated to be filtered. Accordingly, the used parallel hardware architecture requires more on-chip memory modules. Those challenges negatively affect HEVC performance resulting in an increase in computational complexity. In this paper, an optimized parallel DF scheme is proposed for HEVC using graphical processing units (GPUs). The proposed scheme outperforms competing ones in terms of reducing the decoding time of all frames of video sequences by average speed-up factors of 2.83 and 2.45 using the all-intra and low-delay video coding configuration modes, respectively. The proposal does not change the rate-distortion between the decoded video sequences and their original sequences.

Error-Resilient Video Encoding Using Parallel Independent Signature Processing

Soft errors resulting from encoding video sequences on unreliable hardware can create significant artifacts in decoded video sequences, contributing to extreme video quality degradation. Modern systems are required to operate under increasingly challenging constraints, including smaller feature sizes and lower operating voltage, increasing the likelihood of soft errors in the video encoding hardware. These conditions are of particular concern for energy-limited battery-operated systems since they may be required to operate in nonideal environments and/or continue operating with a practically depleted energy source. The proposed parallel independent signature processing design performs error detection and mitigation in video encoding hardware, enabling a graceful degradation of quality when encoding by using unreliable hardware. The effects of soft errors are minimized by preventing the error propagation normally associated with errors in encoded video sequences. This allows for the recovery of quality when errors are present in the video encoding system. Conventional video encoding techniques are designed to handle worst case error rates by increasing gate sizes and/or increasing the operating voltage of the system. Such designs have error-rate limits, and when these limits are reached, the systems tend to fail catastrophically, resulting in an unrecoverable signal. The proposed design allows for single upset events to translate into single transient artifacts in a decoded video sequence.

Improved entropy encoding for high efficient video coding standard

The High Efficiency Video Coding (HEVC) has better coding efficiency, but the encoding performance has to be improved to meet the growing multimedia applications. This paper improves the standard entropy encoding by introducing the optimized weighing parameters, so that higher rate of compression can be accomplished over the standard entropy encoding. The optimization is performed using the recently introduced firefly algorithm. The experimentation is carried out using eight benchmark video sequences and the PSNR for varying rate of data transmission is investigated. Comparative analysis based on the performance statistics is made with the standard entropy encoding. From the obtained results, it is clear that the originality of the decoded video sequence is preserved far better than the proposed method, though the compression rate is increased.

Post-Processing for Blocking Artifact Reduction Based on Inter-Block Correlation

Block-based coding introduces an undesirable discontinuity between neighboring blocks in reconstructed images. This image degradation, referred to as blocking artifacts, arises mainly due to the loss of inter-block correlation in the quantization process of discrete cosine transform coefficients. In many multimedia broadcasting applications, such as a television, decoded video sequences suffer from blocking artifacts. In this paper, we present a novel post-processing algorithm based on increment of inter-block correlation aimed at reducing blocking artifacts. We first smooth the three lowest frequency discrete cosine transform (DCT) coefficients between neighboring blocks, in order to reduce blocking artifacts in the flat region, which are most sensitive to the human visual system. We then group each edge block and its matched blocks together and apply group-based filtering to increase the correlation between grouped blocks. This suppresses blocking artifacts in the edge region while preserving details. In addition, the algorithm is extended to reduce flickering artifacts as well as blocking artifacts in video sequences. Experimental results show that the proposed method successfully alleviates blocking artifacts in both images and videos coded with low bit-rates.

Screen Content Coding Based on HEVC Framework

Screen content like cartoons, captures of typical computer screens or video with text overlay or news ticker is an important category of video, which needs new techniques beyond the existing video coding techniques. In this paper, we analyze the characteristics of screen content and coding efficiency of HEVC on screen content. We propose a new coding scheme, which adopts a non-transform representation, separating screen content into color component and structure component. Based on the proposed representation, two coding modes are designed for screen content to exploit the directional correlation and non-translational changes in screen video sequences. The proposed scheme is then seamlessly incorporated into the HEVC structure and implemented into HEVC range extension reference software HM9.0. Experimental results show that the proposed scheme achieves up to 52.6% bitrate saving compared with HM9.0. On average, 35.1%, 29.2% and 23.6% bitrate saving are achieved with intra, random-access and low-delay configurations, respectively. The visual quality of the decoded video sequences is also significantly improved by reducing ringing artifacts around sharp edges and reserving the shape of text without blur.

A Coding and Postprocessing Framework of Multiview Distributed Video for Wireless Video Sensor Networks

In many surveillance application scenarios of wireless video sensor networks (WVSNs), a number of video sensors are deployed, and multidimension monitored the visual information in a region of interest, forming multiview videos. Since the power, computing capability, and bandwidth are very limited in WVSNs, the conventional multiview video coding method is no longer applicable. So multiview distributed video coding (MDVC) emerged and developed rapidly. In this paper, we propose a new multiview video coding and postprocessing framework for multiview videos. First, in coding scheme, motion intense regions (MIRs) and nonmotion intense regions (NMIRs) based on sum of absolute difference (SAD) criteria are distinguished. For the MIR, the side information (SI) is generated by fusion temporal SI and interview spatial SI at the pixel level. But for the NMIR, the temporal SI is directly use as the ultimate SI. Then, to further improve the quality of the decoded image, an image postprocessing scheme is designed by using deblocking and deringing artifact filters on decoded image. Finally, a set of experimental results show that the proposed fusion SI approach can bring improvements up to 0.2–0.5 dB when compares with only temporal SI used. The subsequent decoded videos postprocessing simulation proves that the proposed postprocessing scheme can provide an additional improvement of about 0.1 dB to the decoded video sequences.

Perceptual-based distributed video coding

In this paper, we propose a perceptual-based distributed video coding (DVC) technique. Unlike traditional video codecs, DVC applies video prediction process at the decoder side using previously received frames. The predicted video frames (i.e., side information) contain prediction errors. The encoder then transmits error-correcting parity bits to the decoder to reconstruct the video frames from side information. However, channel codes based on i.i.d. noise models are not always efficient in correcting video prediction errors. In addition, some of the prediction errors do not cause perceptible visual distortions. From perceptual coding point of view, there is no need to correct such errors. This paper proposes a scheme for the decoder to perform perceptual quality analysis on the predicted side information. The decoder only requests parity bits to correct visually sensitive errors. More importantly, with the proposed technique, key frames can be encoded at higher rates while still maintaining consistent visual quality across the video sequence. As a result, even the objective PSNR measure of the decoded video sequence will increase too. Experimental results show that the proposed technique improves the R-D performance of a transform domain DVC codec both subjectively and objectively. Comparisons with a well-known DVC codec show that the proposed perceptual-based DVC coding scheme is very promising for distributed video coding framework.

Reconfigurable architecture for entropy decoding and inverse transform in H.264

Reconfigurable hardware is an effective design option for dealing with the increasing demands of flexibility and computation power in system design. This paper explores techniques to combine the two entropy decoding methods defined in the H.264 standard, context-based adaptive binary arithmetic coding (CABAC) and context-based adaptive variable length coding (CAVLC), using a coarse-grain reconfigurable architecture. An analyzing of the similarities and differences between these two decoding processes shows that CAVLC can be effectively merged into a CABAC decoder. Experimental results show that about 1.5K gates can be saved using the proposed reconfigurable cell (RC) architecture, which corresponds to a 25.4% area savings in the implementation of the CAVLC decoder. Using the idle time in RC arrays, the base cell can be extended to carry out the inverse transform with very limited overhead. The proposed entropy decoder design, which operates at 66 MHz, can decode video sequences at Baseline and Main profiles at Level 3.0 under the real-time constraint.

Two-Terminal Video Coding

Following recent works on the rate region of the quadratic Gaussian two-terminal source coding problem and limit-approaching code designs, this paper examines multiterminal source coding of two correlated, i.e., stereo, video sequences to save the sum rate over independent coding of both sequences. Two multiterminal video coding schemes are proposed. In the first scheme, the left sequence of the stereo pair is coded by H.264/AVC and used at the joint decoder to facilitate Wyner-Ziv coding of the right video sequence. The first I-frame of the right sequence is successively coded by H.264/AVC Intracoding and Wyner-Ziv coding. An efficient stereo matching algorithm based on loopy belief propagation is then adopted at the decoder to produce pixel-level disparity maps between the corresponding frames of the two decoded video sequences on the fly. Based on the disparity maps, side information for both motion vectors and motion-compensated residual frames of the right sequence are generated at the decoder before Wyner-Ziv encoding. In the second scheme, source splitting is employed on top of classic and Wyner-Ziv coding for compression of both I-frames to allow flexible rate allocation between the two sequences. Experiments with both schemes on stereo video sequences using H.264/AVC, LDPC codes for Slepian-Wolf coding of the motion vectors, and scalar quantization in conjunction with LDPC codes for Wyner-Ziv coding of the residual coefficients give a slightly lower sum rate than separate H.264/AVC coding of both sequences at the same video quality.

A Queuing-Theoretic Approach to Task Scheduling and Processor Selection for Video-Decoding Applications

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> We propose a cross-layer design for resource-constrained systems that simultaneously decode multiple video streams on multiple parallel processors, cores, or processing elements. Our proposed design explicitly considers the coder specific application characteristics such as the decoding dependencies, decoding deadlines, and distortion impacts of different video packets (e.g., frames, slices, groups of slices etc.). The key to the cross-layer design is the resource management control plane (RMCP) that coordinates the scheduling and processor selection across the active applications. The RMCP deploys a priority-queuing model that can evaluate the system congestion and predict the total expected video quality for the set of active decoding tasks. Using this model, we develop a robust distortion- and delay-aware scheduling algorithm for video packets. This algorithm aims to maximize the sum of achieved video qualities over all of the decoded video sequences. Additionally, we propose a processor selection scheme intended to minimize the delays experienced by the queued video packets. In this way, the number of missed decoding deadlines is reduced and the overall decoded video quality is increased. We compare queuing-theoretic based scheduling strategies to media agnostic scheduling strategies (i.e., earliest-deadline-first scheduling) that do not jointly consider the decoding deadlines and distortion impacts. Our results illustrate that by directly considering the video application's properties in the design of a video decoding system, significant system performance gains on the order of 4 dB peak-signal-to-noise ratio can be achieved. </para>

A Framework for Robust Watermarking of H.264-Encoded Video With Controllable Detection Performance

As H.264 digital video becomes more prevalent, the need for copyright protection and authentication methods that are appropriate for this standard will emerge. This paper proposes a robust watermarking algorithm for H.264. We employ a human visual model adapted for a 4 times 4 discrete cosine transform block to increase the payload and robustness while limiting visual distortion. A key-dependent algorithm is used to select a subset of the coefficients that have visual watermarking capacity. Furthermore, the watermark is spread over frequencies and within blocks to avoid error pooling. This increases the payload and robustness without noticeably changing the perceptual quality. We embed the watermark in the coded residuals to avoid decompressing the video; however, we detect the watermark from the decoded video sequence in order to make the algorithm robust to intraprediction mode changes. We build a theoretical framework for watermark detection based on a likelihood ratio test. This framework is used to obtain optimal video watermark detection with controllable detection performance. Our simulation results show that we achieve the desired detection performance in Monte Carlo trials. We demonstrate the robustness of our proposed algorithm to several different attacks

An Improved Multiple Description Coding Strategy Based on Path Diversity

In this paper, a source coding technique exploiting packet path diversity is presented, which combines the individual benefits of MDC (multiple description coding) and LC (layer coding) for video transmission over wireless networks. Performance comparisons between LC and MDC are discussed at the beginning of this paper. Their advantages and disadvantages are concluded. An improved coding technique named auto-resilient multiple description coding (ARMDC) is then shown for its ability to conjoin the merits and avoid the shortcomings of the above two coding schemes. Under the critical transmission conditions of error-prone wireless channels, ARMDC can make all the received packets useful, not just those consecutive from the first packet. At the same time, it can also process the feature of LC, creating a base-layer bit-stream to ensure the basic quality of video. Transmission policy of ARMDC varies with transmission environment and application limits to gain fine video quality. 3G mobile channels are simulated and then streams create by ARMDC, MDR (multiple description with restart) scheme and coding scheme without any LC or MDC are sent over the channels. Experiment results show that the decoded video sequences of ARMDC streams have better performance in R-D curve comparision than the other MDC scheme and also gain best personality effect because of the smooth sequence quality. This out-coming is well adapted to streaming and conversational applications in wireless environment.

Cost-adaptive motion estimation strategy for high-performance video encoders

An adaptive cost block matching (ACBM) motion estimation technique is proposed to achieve a good compromise between the visual quality of decoded video sequences and the computational cost. Simulation results demonstrate that the ACBM algorithm is able to obtain a better rate-distortion performance that the one given by the full-search algorithm, with reductions of up to 90% in the computational cost.

A survey of hybrid MC/DPCM/DCT video coding distortions

The motion-compensated hybrid DCT/DPCM algorithm has been successfully adopted in various video coding standards, such as H.261, H.263, MPEG-1 and MPEG-2. However, its robustness is challenged in the face of an inadequate bit allocation, either globally for the whole video sequence, or locally as a result of an inappropriate distribution of the available bits. In either of these situations, the trade-off between quality and the availability of bits results in a deterioration in the quality of the decoded video sequence, both in terms of the loss of information and the introduction of coding artifacts. These distortions are an important factor in the fields of filtering, codec design, and the search for objective psychovisual-based quality metrics; therefore, this paper presents a comprehensive analysis and classification of the numerous coding artifacts which are introduced into the reconstructed video sequence through the use of the hybrid MC/DPCM/DCT video coding algorithm. Artifacts which have already been briefly described in the literature, such as the blocking effect, ringing, the mosquito effect, MC mismatch, blurring, and color bleeding, will be comprehensively analyzed. Additionally, we will present artifacts with unique properties which have not been previously identified in the literature.

An adaptive temporal-spatial filter for MPEG decoded video signals

A new adaptive post-processing algorithm for the MPEG decoded video sequences is proposed. We use a motion compensated averaging filter to reduce the noises in the temporal domain and an adaptive spatial filter to remove noise in the spatial domain and preserve the edge of different orientations in the image. A MPEG decoded video sequence called table tennis is processed by our proposed filter. The post-processed video sequence shows that its image quality is improved, especially of the moving objects.

A real-time MPEG encoder using a programmable processor

This paper describes the design of a fully programmable multimedia system. The system, which utilizes a single multimedia video processor (MVP), is capable of performing real-time MPEG video and audio encoding at SIF resolution, 30 frames per-second. Support is provided for a windows environment, including a scalable video window for viewing decoded video sequences or monitoring video compression in real time. The system is also well suited to perform image processing, 2-D and 3-D graphics, and many other video and audio processing and compression algorithms.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>