A Four-Point Camera Calibration Method for Sport Videos

Abstract

The main objective of camera calibration is to estimate the homography between a sports field template and the corresponding field in a video frame. State-of-the-art approaches exploit deep-learning technologies to achieve this aim. However, the lack of large training datasets leads to unreliable results. Search-based approaches deliver better accuracy but at the expense of a very high computational cost. In this paper, a four-point camera calibration method is proposed to address these challenges. It consists of four stages: first, a conditional generative adversarial network (cGAN) is used to generate meaningfully segmented video frames. This cGAN is a general image-to-image translation technique which has robust performance even for small training datasets. The aim is to eliminate foreground objects that greatly impede accurate estimation of four suitable points for the subsequent stage. Then, a regression network which can estimate four points from a single segmented video frame is introduced. In this stage we aim at keeping the low computational cost. In the next processing stage, the estimated four points and the original selected four points are used to calculate the homography. In this third stage a Direct Linear Transformation (DLT) algorithm is applied. In the last processing stage, the accuracy of the estimated homography is refined by running an Enhanced Correlation Coefficient (ECC) module. The proposed method is extensively evaluated using a standard 2014 World Cup dataset. The obtained results are compared to the state-of-the-art approaches and its variations based on U-net, VGG-16, ResNet-50. The obtained results show that the proposed method is superior in terms of accuracy and computational efficiency. To demonstrate the effectiveness and robustness of the proposed method, it is also validated on National Basketball, 2018 World Cup and cross dataset. In these diverse environments, the proposed method still maintains the competitive performance.