Frame Skipping Scheme Research Articles

Improved HEVC video compression algorithm using low-complexity frame rate up conversion

The high-efficiency video coding (HEVC) standard can reduce the bit rate of compression video. However, the compression efficiency of HEVC is still challenging current internet bandwidth. This problem can be solved by reducing the number of video frames encoded by HEVC and reconstructing the reduced frames that rely on frame rate up conversion (FRUC). Unfortunately, most existing FRUC methods need to re-estimate the motion information, a step that will lead to a huge increase of video decoder complexity. An improved HEVC video compression algorithm using low-complexity FRUC is proposed based on reusing the motion vector (MV) predicted by HEVC encoding procedure to directly reconstruct the skipped frames. First, an adaptive frame-skipping scheme is used to maximize reducing the number of frames that will encode by HEVC. Second, decoder extracts the MV and residual from the received bit stream. Then an MV reliability checking and rectifying method is proposed according to adaptive residual energy analyzing. Finally, a motion-compensation method and efficient occluded area processing algorithm are proposed to reconstruct skipped frames. These methods all directly utilize the rectified MVs. Experimental results show that due to the reduced frames and decoder that does not need to re-estimate MV, the proposed method can not only significantly decrease the time of all codec procedure when comparing HEVC and other FRUC-based video compression methods but also achieve 9.7022% to 66.0030% reduction in Bjøntegaard delta bit rate at low bit rates when compared to HEVC standard. In addition, the compression of the proposed method is also better than other state-of-the-art methods.

Joint Spatial-Temporal Quality Improvement Scheme for H.264 Low Bit Rate Video Coding via Adaptive Frameskip

Conventional rate control (RC) schemes for H.264 video coding usually regulate output bit rate to match channel bandwidth by adjusting quantization parameter (QP) at fixed full frame rate, and the passive frame skipping to avoid buffer overflow usually occurs when scene changes or high motions exist in video sequences especially at low bit rate, which degrades spatial-temporal quality and causes jerky effect. In this paper, an active content adaptive frame skipping scheme is proposed instead of passive methods, which skips subjectively trivial frames by structural similarity (SSIM) measurement between the original frame and the interpolated frame via motion vector (MV) copy scheme. The saved bits from skipped frames are allocated to coded key ones to enhance their spatial quality, and the skipped frames are well recovered based on MV copy scheme from adjacent key ones at the decoder side to maintain constant frame rate. Experimental results show that the proposed active SSIM-based frameskip scheme acquires better and more consistent spatial-temporal quality both in objective (PSNR) and subjective (SSIM) sense with low complexity compared to classic fixed frame rate control method JVT-G012 and prior objective metric based frameskip method.

A Frame-Activity-Based Dynamic Frame-Skipping Transcoder

Nowadays band-limited terminals are very popular in wireless networking environments, with the result that frame-skipping technology is applied to reduce frame rate for video transmission. In this paper, we propose a dynamic frame-skipping transcoding scheme, where a frame being skipped or not is based on its activity. Whether a frame has relatively drastic motion activities or soft motion activities is evaluated among a certain number of its neighboring frames, which keeps the continuity of the transcoded sequence logically. To reduce the drift errors caused by locating overlapping blocks during the motion vector re-construction process in multi-frame skipping, we propose a new efficient overlapping block location method. A transcoder structure with the proposed dynamic frame-skipping scheme is also designed for hardware application. Simulation shows that the proposed dynamic frame-skipping scheme obtains lower complexity and higher performance, which is especially useful for real-time transcoding.

Dynamic frame-skipping transcoding with motion information considered

Frame-skipping transcoding is an efficient tool for reducing the bit rate when transmitting precoded video over bandwidth-constrained channels. In this process, the motion information is an important factor to be considered. Traditional methods consider only the motion activity information in dynamic frame-skipping transcoding. The concept of motion change is introduced to evaluate motion continuity. Combined with the motion activity information, these two parts of motion information are used to dynamically skip frames without seriously affecting the visual quality. A new frame-skipping scheme with a variable length prediction window is also proposed. Experimental results show that the proposed method can present a smooth frame rate variation, and well preserve the original motion information with tolerable delay. At the same time, spatial quality is also improved.

Variable Frame Skipping Scheme Based on Estimated Quality of Non-coded Frames at Decoder for Real-Time Video Coding

This paper proposes a block-based video encoder employing variable frame skipping (VFS) to improve the video quality in low bit rate channel. The basic idea of VFS mechanism is to decide and skip a suitable, non-fixed number of frames in temporal domain to reduce bit usage. The saved bits can be allocated to enhance the spatial quality of video. In literature, several methods of frame skipping decision have been proposed, but most of them only consider the similarities between neighboring coded frames as the decision criteria. Our proposed method takes into account the reconstruction of the skipped frames using motion-compensated frame interpolation at decoder. The proposed VFS models the reconstructed objective quality of the skipped frame and, therefore, can provide a fast estimate to the frame skipping at encoder. The proposed VFS can determine the suitable frame skipping in real time and provide the encoded video with better spatial-temporal bit allocation.

Adaptive frame skipping based on spatio-temporal complexity for low bit-rate video coding

Conventional rate control schemes focus on the adjustment of quantization values to retain a certain buffer level, and buffer adaptive frame skippings are used to avoid buffer overflow at low bit-rates. In this paper, an adaptive frame-skipping scheme is proposed to achieve the balance between spatial and temporal quality, and to maintain the buffer level. The occurrence of frame skipping is jointly dependent on the temporal and spatial complexity of the video, and on the fullness of the buffer. This helps to achieve a balanced spatial and temporal quality and to enhance the overall perceptual quality. Experimental results show that the new scheme is simple but very effective, with large average PSNR gains and consistently improved visual quality, and the improvement in perceptual quality is much significant than that in average PSNRs.

Modified intelligent scissors and adaptive frame skipping for video object segmentation

MPEG-4 introduces the concept of video object to support content-based functionalities. Video object segmentation is a crucial step for object-based coding and manipulation. In this paper, a robust semi- automatic video object segmentation scheme is proposed. To efficiently and accurately define the initial object contour, modified intelligent scissors is proposed on the basis of original intelligent scissors. It can improve about 6–8 times the processing speed with only slight sacrifice of accuracy, which just meets the requirements of initial object extraction for semi-automatic approach. To avoid errors accumulating and propagating during object tracking, an adaptive frame skipping scheme is proposed to decompose video sequence into video clips. For rigid and non-rigid video objects, two different image segmentation algorithms are utilized, and then region-based backward projection technique is adopted to interpolate the video object plane (VOPs) of other frames within every video clip. The proposed approach can cope with occlusion/disocclusion problem to most extent. Experimental results demonstrate the effectiveness and robustness of the method.

Joint bit allocation analysis for video streaming over networks

In this paper, a joint bit allocation analysis for rate control in video streaming over networks is presented. In our previous works, we proposed two simple, yet efficient, methods to reduce video quality variations, through adaptive target bits estimation control and active frame skipping, so that a near constant video quality can be achieved. Adaptive target bits estimation control scheme is finer and more efficient than active frame skipping scheme, which proactively skips some simple frames to avoid forced frame skipping for high motion frames. Both methods were implemented individually and jointly on H.264 coder and their performances for a variety of video sequences and bit rates were compared. An analytic solution for joint bit allocation in favour of adaptive target bit estimation scheme is derived.