Soccer Video Event Research Articles

Prediction of Shooting Events in Soccer Videos Using Complete Bipartite Graphs and Players' Spatial-Temporal Relations.

In soccer, quantitatively evaluating the performance of players and teams is essential to improve tactical coaching and players' decision-making abilities. To achieve this, some methods use predicted probabilities of shoot event occurrences to quantify player performances, but conventional shoot prediction models have not performed well and have failed to consider the reliability of the event probability. This paper proposes a novel method that effectively utilizes players' spatio-temporal relations and prediction uncertainty to predict shoot event occurrences with greater accuracy and robustness. Specifically, we represent players' relations as a complete bipartite graph, which effectively incorporates soccer domain knowledge, and capture latent features by applying a graph convolutional recurrent neural network (GCRNN) to the constructed graph. Our model utilizes a Bayesian neural network to predict the probability of shoot event occurrence, considering spatio-temporal relations between players and prediction uncertainty. In our experiments, we confirmed that the proposed method outperformed several other methods in terms of prediction performance, and we found that considering players' distances significantly affects the prediction accuracy.

Automated Event Detection and Classification in Soccer: The Potential of Using Multiple Modalities

Detecting events in videos is a complex task, and many different approaches, aimed at a large variety of use-cases, have been proposed in the literature. Most approaches, however, are unimodal and only consider the visual information in the videos. This paper presents and evaluates different approaches based on neural networks where we combine visual features with audio features to detect (spot) and classify events in soccer videos. We employ model fusion to combine different modalities such as video and audio, and test these combinations against different state-of-the-art models on the SoccerNet dataset. The results show that a multimodal approach is beneficial. We also analyze how the tolerance for delays in classification and spotting time, and the tolerance for prediction accuracy, influence the results. Our experiments show that using multiple modalities improves event detection performance for certain types of events.

Using 3D Convolutional Neural Networks for Real-time Detection of Soccer Events

Developing systems for the automatic detection of events in video is a task which has gained attention in many areas including sports. More specifically, event detection for soccer videos has been studied widely in the literature. However, there are still a number of shortcomings in the state-of-the-art such as high latency, making it challenging to operate at the live edge. In this paper, we present an algorithm to detect events in soccer videos in real time, using 3D convolutional neural networks. We test our algorithm on three different datasets from SoccerNet, the Swedish Allsvenskan, and the Norwegian Eliteserien. Overall, the results show that we can detect events with high recall, low latency, and accurate time estimation. The trade-off is a slightly lower precision compared to the current state-of-the-art, which has higher latency and performs better when a less accurate time estimation can be accepted. In addition to the presented algorithm, we perform an extensive ablation study on how the different parts of the training pipeline affect the final results.

Learning an event-oriented and discriminative dictionary based on an adaptive label-consistent K-SVD method for event detection in soccer videos

In this paper, we formulate the soccer video event detection task as a sparse representation problem by learning a supervised, discriminative and event-oriented dictionary based on learned weighted local features. To this end, we present a novel framework based on two ideas: First, we propose an approach for computing the representativeness of each video frame for each soccer event. Second, we propose an Adaptive Label-Consistent K-SVD (ALC-KSVD) algorithm to learn an event-oriented and discriminative dictionary based on the computed representativeness of frames to transfer video frames to a sparse space. To improve discrimination among frames of different events, we proposed a weighting method to identify local features that are more representative in each event category. Next, the representativeness score of each frame is calculated by aggregating the weighted local features within each frame. The calculated representativeness score of each frame indicates its belonging degree to each event. The representativeness score matrix, being a discriminative term, is combined with the reconstruction error to form an objective function to improve the discrimination ability in the sparse representation during the dictionary learning process. The obtained objective function is efficiently and optimally solved by the K-SVD algorithm. The representativeness score matrix, which is automatically calculated based on the training samples, defines an adaptive correspondence between the dictionary atoms and the labels of the frames. We demonstrate the effectiveness of the proposed framework on the detection and classification of several soccer events based on an extensive experimental investigation that was conducted using a large collection of video data. The experimental results indicate that our approach maintains good classification performance and outperforms the state-of-the-art methods.

IF-MCA: Importance Factor-Based Multiple Correspondence Analysis for Multimedia Data Analytics

Multimedia concept detection is a challenging topic due to the well-known class imbalance issue, where the data instances are distributed unevenly across different classes. This problem becomes even more prominent when the minority class that contains an extremely small proportion of the data represents the concept of interest as has occurred in many real-world applications such as frauds in banking transactions and goal events in soccer videos. Traditional data mining approaches often have difficulty handling largely skewed data distributions. To address this issue, in this paper, an importance-factor (IF)-based multiple correspondence analysis (MCA) framework is proposed to deal with the imbalanced datasets. Specifically, a hierarchical information gain analysis method, which is inspired by the decision tree algorithm, is presented for critical feature selection and IF assignment. Then, the derived IF is incorporated with the MCA algorithm for effective concept detection and retrieval. The comparison results in video concept detection using the disaster dataset and the soccer dataset demonstrate the effectiveness of the proposed framework.

Soccer Video Event Annotation by Synchronization of Attack–Defense Clips and Match Reports With Coarse-Grained Time Information

The annotation of significant events within soccer videos is a fundamental step in content-based soccer video retrieval. In this paper, we propose a soccer video annotation approach based on semantic matching with coarse time constraints, where video events and external text information (match reports) are synchronized using their semantic correspondence in the temporal sequence. Unlike the state-of-the-art soccer video analysis methods that assume that the time of an event's occurrence is given precisely by the external text information, this paper considers the problem of annotating soccer videos using match reports with coarse-grained time information. The contributions of the proposed approach are: 1) we propose a more generalized approach that synchronizes video events with text descriptions using high-level semantics with coarse time constraints, rather than assuming that the timestamp is given exactly in the text description; 2) the detection of event boundaries is improved by attack-defense transition analysis (ADTA); 3) a robust and fast center circle detection algorithm is proposed for the classification of soccer field zones and ADTA; and 4) unlike conventional audio-based whistle detection, we propose a novel Hough-transform-based algorithm from the perspective of image processing. This allows the game start time to be detected, and further helps the synchronization of video and text events. The experimental results conducted on a large number of soccer videos validate the effectiveness of the proposed approach.

High-level representation sketch for video event retrieval

Representing video events is an essential step for a wide range of visual applications. In this paper, we propose the event sketch, a high-level event representation, to depict the dynamic properties of video events composed of actions of semantic objects. We show that this representation can facilitate a novel sketch based video retrieval (SBVR) system, which has not been considered before to the best of our knowledge. In this system, users are allowed to draw the evolutions (e.g. spatiotemporal layouts and behaviors of semantic objects) on a board, and retrieve the events whose semantic objects have the similar evolutions from a database. To do this, event sketches are constructed on both the user queries and database videos, and compared under a novel graph-matching scheme based on data-driven Monta Carlo Markov chain (DDMCMC). To test our approach, we collect a novel dataset of goal events in real soccer videos, which consists actions of multiple players and shows large variability in the evolution process of the events. Experiments on this dataset and the publicly available dataset CAVIAR demonstrated the effectiveness of the proposed approach.

Event Detection and Summarization in Soccer Videos Using Bayesian Network and Copula

Semantic video analysis and automatic concept extraction play an important role in several applications; including content-based search engines, video indexing, and video summarization. As the Bayesian network is a powerful tool for learning complex patterns, a novel Bayesian network-based method is proposed for automatic event detection and summarization in soccer videos. The proposed method includes efficient algorithms for shot boundary detection, shot view classification, mid-level visual feature extraction, and construction of the related Bayesian network. The method contains of three main stages. In the first stage, the shot boundaries are detected. Using the hidden Markov model, the video is segmented into large and meaningful semantic units, called play-break sequences. In the next stage, several features are extracted from each of these units. Finally, in the last stage, in order to achieve high level semantic features (events and concepts), the Bayesian network is used. The basic part of the method is constructing the Bayesian network, for which the structure is estimated using the Chow-Liu tree. The joint distributions of random variables of the network are modeled by applying the Farlie-Gumbel-Morgenstern family of Copulas. The performance of the proposed method is evaluated on a dataset with about 9 h of soccer videos. The method is capable of detecting seven different events in soccer videos; namely, goal, card, goal attempt, corner, foul, offside, and nonhighlights. Experimental results show the effectiveness and robustness of the proposed method on detecting these events.

Hidden conditional random field-based soccer video events detection

Detect highlight event is an important step for semantic-based video retrieval. Hidden conditional random field (HCRF) is a discriminative model, which is effective in fusing observations for event inference. Mid-level semantics and their refinements are more robust than low-level visual features in event detection for learning models. To make full use of the contextual information, two aspects are taken into account during soccer video event detection. The first is parsing video sequences into event clips. The second is fusing the temporal transitions of the mid-level semantics of an event clip to determine the event type. In this study, HCRF is utilised to model the observations of mid-level semantics of an event clip for event detection. Comparisons are made with the dynamic Bayesian networks, hidden Markov model (HMM), enhanced HMM and conditional random field-based event detection approaches. Experimental results show the effectiveness of the proposed method.

HMM based soccer video event detection using enhanced mid-level semantic

Highlight detection is a fundamental step in semantics based video retrieval and personalized sports video browsing. In this paper, an effective hidden Markov models (HMMs) based soccer video event detection method based on a hierarchical video analysis framework is proposed. Soccer video shots are classified into four coarse mid-level semantics: global, median, close-up and audience. Global and local motion information is utilized for the refinement of coarse mid-level semantics. Sequential soccer video is segmented into event clips. Both the temporal transitions of the mid-level semantics and the overall features of an event clip are fused using HMMs to determine the type of event. Highlight detection performance of dynamic Bayesian networks (DBN), conditional random fields (CRF) and the proposed HMM based approach are compared. The average F-score of our highlights (including goal, shoot, foul and placed kick) detection approach is 82.92%, which outperforms that of DBN and CRF by 9.85% and 11.12% respectively. The effects of number of hidden states, overall features, and the refinement of mid-level semantics on the event detection performance are also discussed.

Semantic Concept Mining Based on Hierarchical Event Detection for Soccer Video Indexing

In this paper, we present a novel automated indexing and semantic labeling for broadcast soccer video sequences. The proposed method automatically extracts silent events from the video and classifies each event sequence into a concept by sequential association mining. The paper makes three new contributions in multimodal sports video indexing and summarization. First, we propose a novel hierarchical framework for soccer (football) video event sequence detection and classification. Unlike most existing video classification approaches, which focus on shot detection followed by shot-clustering for classification, the proposed scheme perform a top-down video scene classification which avoids shot clustering. This improves the classification accuracy and also maintains the temporal order of shots. Second, we compute the association for the events of each excitement clip using a priori mining algorithm. We pro- pose a novel sequential association distance to classify the association of the excitement clip into semantic concepts. For soccer video, we have considered goal scored by team-A, goal scored by team-B, goal saved by team-A, goal saved by team-B as semantic concepts. Third, the extracted excitement clips with semantic concept label helps us to summarize many hours of video to collection of soccer highlights such as goals, saves, corner kicks, etc. We show promising results, with correctly indexed soccer scenes, enabling structural and temporal analysis, such as video retrieval, highlight extraction, and video skimming.

A multimodal data mining framework for soccer goal detection based on decision tree logic

In this paper, we propose a new multimedia data mining framework for the extraction of soccer goal events in soccer videos by utilising both multimodal analysis and decision tree logic. The extracted events can be used to index the soccer videos. We first adopt an advanced video shot detection method to produce shot boundaries and some important visual features. Then, the visual/audio features are extracted for each shot at different granularities. This rich multimodal feature set is then filtered by a pre-filtering step in order to clean the noise as well as to reduce the irrelevant data. A decision tree model is built upon the cleaned data set and is used to classify the goal shots. We also present the experimental results for the proposed framework, which indicate the performance of the framework for soccer goal extraction.