Adaptive convolutional neural network for large change in video object segmentation

Abstract

This study tackles the semi‐supervised segmentation task for the objects that have large motion or appearance change in a video sequence, which is very challenging to the existing methods of video object segmentation (VOS). In this study, a novel adaptive approach is presented, named adaptive convolutional neural network for large change VOS, which determines when and how to fine‐tune the convolutional neural network through the motion metric and the appearance metric among consecutive video frames. Additionally, a lightweight optimisation algorithm for the predictive binary mask is introduced which is effective for pixel prediction by eliminating the discrete points cluster. To illustrate the advantages of this approach, experiments have been performed on four VOS datasets, which demonstrate that the proposed method is highly effective and could achieve the state‐of‐the‐art on these datasets.