Motion Compensated Frame Interpolation Research Articles

Variable Frame Skipping Scheme Based on Estimated Quality of Non-coded Frames at Decoder for Real-Time Video Coding

This paper proposes a block-based video encoder employing variable frame skipping (VFS) to improve the video quality in low bit rate channel. The basic idea of VFS mechanism is to decide and skip a suitable, non-fixed number of frames in temporal domain to reduce bit usage. The saved bits can be allocated to enhance the spatial quality of video. In literature, several methods of frame skipping decision have been proposed, but most of them only consider the similarities between neighboring coded frames as the decision criteria. Our proposed method takes into account the reconstruction of the skipped frames using motion-compensated frame interpolation at decoder. The proposed VFS models the reconstructed objective quality of the skipped frame and, therefore, can provide a fast estimate to the frame skipping at encoder. The proposed VFS can determine the suitable frame skipping in real time and provide the encoded video with better spatial-temporal bit allocation.

Motion compensated frame interpolation by new block-based motion estimation algorithm

In this paper, a motion compensated frame interpolation (MCI) algorithm adapting new block-based motion estimation (BME), which is overlapped block-based motion estimation (OBME), is proposed for frame rate up-conversion (FRC). Unlike conventional BME algorithms, where a video frame is divided into many non-overlapping square blocks of pixels, the proposed BME is executed using an overlapped matching block in order to get more accurate motion trajectory. To reduce computational complexity caused by the OBME, a sub-sampled pixel block is used for the OBME. The proposed OBME is executed with the various overlapped blocks that have different block sizes and sub-sampling ratios. In this paper, instead of mean absolute difference (MAD) used for general BME, a modified MAD is applied for the OBME. The MAD proposed is weighted using the magnitude of MV that corresponds to the estimating position. From the simulation results, the proposed methods show better performance in estimating "true motion" and reducing blocking artifacts with less complexity in calculation compared to a conventional method.

Hybrid global–local motion compensated frame interpolation for low bit rate video coding

A hybrid global–local motion compensated frame interpolation (HMCI) scheme for low bit rate video coding is proposed in this paper. Different from most other global–local motion compensation algorithms, which were performed at the encoder end, the proposed scheme works solely on the decoded video stream. Thus, it can be applied to any MCP (motion compensated prediction) based standard coders without bit stream syntax modification. Along with the proposed HMCI scheme, a motion-based foreground–background segmentation method is presented. Traditional intensity-based segmentation methods can only segment moving objects from still background. In contrast, the proposed approach can segment foreground objects from both moving and still background by exploiting information carried by block-based motion vectors. Both the six-parameter affine or the eight-parameter perspective models can be applied to the foreground–background segmentation and the moving background reconstruction; while the six-parameter affine model with triangular patch warping is adopted to reconstruct the foreground blocks due to its low complexity. Experiments show that the proposed HMCI scheme achieves higher subjective (visual) quality compared to the conventional block-based frame interpolation without global motion compensation, where artifacts in moving backgrounds are significantly reduced.

Frame interpolation scheme using inertia motion prediction

In this paper, a new inertia-based motion-compensated frame interpolation scheme is proposed for video transmission applications to scale up the frame rate of a decoded video at the receiving end. In our formulation, a new background segmentation and frame interpolation using inertia motion prediction are developed to further improve the performance of frame interpolation. Experiments show that the proposed method achieves better subjective and objective quality for the interpolated frames than other existing methods.

TCP-friendly Internet video streaming employing variable frame-rate encoding and interpolation

A feedback-based Internet video transmission scheme based on ITU-T H.263+ is presented. The proposed system is capable of continually accommodating its stream size and managing the packet loss recovery in response to network condition changes. It consists of multiple components: TCP-friendly end-to-end congestion control and available bandwidth estimation, encoding frame-rate control and delay-based smoothing at the sender, media-aware packetization and packet loss recovery tied with congestion control, and quality recovery tools such as motion-compensated frame interpolation at the receiver. These components are designed to meet the low computational complexity requirement so that the whole system can operate in real-time. Among these, the video-aware congestion control known as a receiver-based congestion control mechanism, the variable frame-rate H.263+ encoding, and fast motion-compensated frame interpolation components are key features. Through a seamless integration, it is demonstrated that network adaptivity is enhanced enough to mitigate the packet loss and bandwidth fluctuation, resulting in a more smooth video experience at the receiver.

Frame interpolation and bidirectional prediction of video using compactly encoded optical-flow fields and label fields

We consider the problems of motion-compensated frame interpolation (MCFI) and bidirectional prediction in a video coding environment. These applications generally require good motion estimates at the decoder. We use a multiscale optical-flow-based motion estimator that provides smooth, natural motion fields under bit-rate constraints. These motion estimates scale well with change in temporal resolution and provide considerable flexibility in the design and operation of coders and decoders. In the MCFI application, this estimator provides excellent interpolated frames that are superior to those of conventional motion estimators, both visually and in terms of peak signal-to-noise ratio (PSNR). We also consider the effect of occlusions in the bidirectional prediction application and introduce a dense label field that complements our motion estimator. This label field enables us to adaptively weight the forward and backward predictions and gives us substantial visual and PSNR improvements in the covered/uncovered regions of the sequence.

A Combination of DPCM and Motion-Compensated Frame Interpolation for the Encoding of 34 Mbit/s Colour TV Signals (Part I)

Article A Combination of DPCM and Motion-Compensated Frame Interpolation for the Encoding of 34 Mbit/s Colour TV Signals (Part I) was published on August 1, 1988 in the journal Frequenz (volume 42, issue 8).

A Combination of DPCM and Motion-Compensated Frame Interpolation for the Encoding of 34 Mbit/s Colour TV Signals (Part II)

A Combination of DPCM and Motion-Compensated Frame Interpolation for the Encoding of 34 Mbit/s Colour TV Signals (Part II)