SANERV: Scene-Adaptive Neural Representation for Videos

Abstract

Translate Article

Neural Representations for Videos (NeRV) is a neural network model that learns video representation through image-wise implicit neural representation (INR). Although early NeRV-based research has improved representation learning, it overlooked video properties. In this paper, we propose a new method called Scene-Adaptive Neural Representation for Videos (SANeRV) that considers videos’ general properties and adapts to individual video characteristics. To achieve this, we incorporate temporal redundancy, which is a typical feature of videos, by splitting the learning process into two parts: global representation and residual representation. Moreover, we adjust the network architecture based on the presence of scene changes and the dynamism of the video to cater to individual video characteristics. Experiments show that SANeRV achieves state-of-the-art performance in video regression and compression tasks for benchmark datasets.