Abstract
Frame interpolation, which generates an intermediate frame given adjacent ones, finds various applications such as frame rate up-conversion, video compression, and video streaming. Instead of using complex network models and additional data involved in the state-of-the-art frame interpolation methods, this paper proposes an approach based on an end-to-end generative adversarial network. A combined loss function is employed, which jointly considers the adversarial loss (difference between data models), reconstruction loss, and motion blur degradation. The objective image quality metric values reach a PSNR of 29.22 dB and SSIM of 0.835 on the UCF101 dataset, similar to those of the state-of-the-art approach. The good visual quality is notably achieved by approximately one-fifth computational time, which entails possible real-time frame rate up-conversion. The interpolated output can be further improved by a GAN based refinement network that better maintains motion and color by image-to-image translation.
Highlights
Frame interpolation is a technique that generates intermediate frames between the adjacent ones in a video sequence
The networks considered in video frame generation include the convolutional neural networks (CNNs) [32,33], recurrent neural networks (RNNs), long short term memory networks (LSTM), and generative adversarial network (GAN)
We have presented a lightweight GAN for frame interpolation
Summary
Frame interpolation is a technique that generates intermediate frames between the adjacent ones in a video sequence. The popular network models include the convolution neural network (CNN) and generative adversarial network (GAN), typically under the supervised learning framework. Most of the existing GAN based frame prediction methods use complicated models in which each of them takes a different responsibility such as generation or rectification [7,8]. Those methods require auxiliary data and long training time. Generative Adversarial Network (DCGAN) [4], one of the most popular GAN frameworks for random image generation, we propose using a semi-supervised frame interpolation framework (Figure 1). Section demonstrates the experiments which include qualitative and quantitative proposed approach.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
