Video Summarization for Multiple Sports Using Deep Learning

Abstract

This paper proposes a computationally inexpensive method for automatic key event extraction and subsequent summarization of sports videos using scoreboard detection. A database consisting of 1300 images was used to train a supervised learning-based object detection algorithm, You Only Look Once (YOLO). Then, for each frame of the video, once the scoreboard was detected using YOLO, the scoreboard was cropped out of the image. After this, image processing techniques were applied on the cropped scoreboard to reduce noise and false positives. Following that, the processed image was passed through an optical character recognizer (OCR) to get the score. A rule-based algorithm was run on the output of the OCR to generate the timestamps of key events based on the game. Finally, clips of length 30 s around every detected event (15 s before and after the event) were extracted from the video file and stitched together to form the highlights video. The proposed method is best suited for people who want to analyze the games and want precise timestamps of the occurrence of important events. The performance of the proposed design was tested on videos of Bundesliga, English Premier League, ICC WC 2019, IPL 2019, and Pro Kabaddi League. An average F1 score of 0.979 was achieved during the simulations. The model was also tested on different video formats, and these were .mp4, .avi, .mov, and .mkv. The algorithm was trained on five different classes of three separate games (soccer, cricket, and kabaddi). The design was implemented using Python 3.7.KeywordsYou Only Look Once (YOLO)Optical Character Recognizer (OCR)Intersection Over Union (IOU)Region of Image (ROI)True Positives (TP)True Negatives (TN)False Positives (FP)False Negatives (FN)Frames Per Second (FPS)Mean Average Precision (MAP)