Sports Video Annotation Research Articles

Research on Aerobics Training and Evaluation Method Based on Artificial Intelligence-Aided Modeling

Traditional aerobics training methods have the problems of lack of auxiliary teaching conditions and low-training efficiency. With the in-depth application of artificial intelligence and computer-aided training methods in the field of aerobics teaching and practice, this paper proposes a local space-time preserving Fisher vector (FV) coding method and monocular motion video automatic scoring technology. Firstly, the gradient direction histogram and optical flow histogram are extracted to describe the motion posture and motion characteristics of the human body in motion video. After normalization and data dimensionality reduction based on the principal component analysis, the human motion feature vector with discrimination ability is obtained. Then, the spatiotemporal pyramid method is used to embed spatiotemporal features in FV coding to improve the ability to identify the correctness and coordination of human behavior. Finally, the linear model of different action classifications is established to determine the action score. In the key frame extraction experiment of the aerobics action video, the ST-FMP model improves the recognition accuracy of uncertain human parts in the flexible hybrid joint human model by about 15 percentage points, and the key frame extraction accuracy reaches 81%, which is better than the traditional algorithm. This algorithm is not only sensitive to human motion characteristics and human posture but also suitable for sports video annotation evaluation, which has a certain reference significance for improving the level of aerobics training.

Sports Video Annotation and Multi-Target Tracking using Extended Gaussian Mixture model

Video offers solutions to many of the traditional problems with coach, trainer, commenter, umpires and other security issues of modern team games. This paper presents a novel framework to perform player identification and tracking technique for the sports (Kabaddi) with extending the implementation towards the event handling process which expands the game analysis of the third umpire assessment. In the proposed methodology, video preprocessing has done with Kalman Filtering (KF) technique. Extended Gaussian Mixture Model (EGMM) implemented to detect the object occlusions and player labeling. Morphological operations have given the more genuine results on player detection on the spatial domain by applying the silhouette spot model. Team localization and player tracking has done with Robust Color Table (RCT) model generation to classify each team members. Hough Grid Transformation (HGT) and Region of Interest (RoI) method has applied for background annotation process. Through which each court line tracing and labeling in the half of the court with respect to their state-of-art for foremost event handling process is performed. Extensive experiments have been conducted on real time video samples to meet out the all the challenging aspects. Proposed algorithm tested on both Self Developed Video (SDV) data and Real Time Video (RTV) with dynamic background for the greater tracking accuracy and performance measures in the different state of video samples.

A Novel Markov Logic Rule Induction Strategy for Characterizing Sports Video Footage

The grounding of high-level semantic concepts is a key requirement of video annotation systems. Rule induction can thus constitute an invaluable intermediate step in characterizing protocol-governed domains, such as broadcast sports footage. The authors propose a clause grammar template approach to the problem of rule induction in video footage of court games that employs a second-order meta-grammar for Markov Logic Network construction. The aim is to build an adaptive system for sports video annotation capable, in principle, of both learning ab initio and adaptively transferring learning between distinct rule domains. The authors tested the method using a simulated game predicate generator as well as real data derived from tennis footage via computer-vision-based approaches including HOG3D-based player-action classification, Hough-transform-based court detection, and graph-theoretic ball tracking. Experiments demonstrate that the method exhibits both error resilience and learning transfer in the court domain context. Moreover, the clause template approach naturally generalizes to any suitably constrained, protocol-governed video domain characterized by feature noise or detector error.

Multilevel Chinese takeaway process and label-based processes for rule induction in the context of automated sports video annotation.

We propose four variants of a novel hierarchical hidden Markov models strategy for rule induction in the context of automated sports video annotation including a multilevel Chinese takeaway process (MLCTP) based on the Chinese restaurant process and a novel Cartesian product label-based hierarchical bottom-up clustering (CLHBC) method that employs prior information contained within label structures. Our results show significant improvement by comparison against the flat Markov model: optimal performance is obtained using a hybrid method, which combines the MLCTP generated hierarchical topological structures with CLHBC generated event labels. We also show that the methods proposed are generalizable to other rule-based environments including human driving behavior and human actions.

Active Foreground Region Extraction and Tracking for Sports Video Annotation

Automatic video segmentation plays a vital role in sports videos annotation. This paper presents a fully automatic and computationally efficient algorithm for analysis of sports videos. Various methods of automatic shot boundary detection have been proposed to perform automatic video segmentation. These investigations mainly concentrate on detecting fades and dissolves for fast processing of the entire video scene without providing any additional feedback on object relativity within the shots. The goal of the proposed method is to identify regions that perform certain activities in a scene. The model uses some low-level feature video processing algorithms to extract the shot boundaries from a video scene and to identify dominant colours within these boundaries. An object classification method is used for clustering the seed distributions of the dominant colours to homogeneous regions. Using a simple tracking method a classification of these regions to active or static is performed. The efficiency of the proposed framework is demonstrated over a standard video benchmark with numerous types of sport events and the experimental results show that our algorithm can be used with high accuracy for automatic annotation of active regions for sport videos.

Tagging Webcast Text in Baseball Videos by Video Segmentation and Text Alignment

Sports video annotation, an active research area in the field of multimedia content understanding, is an essential process in applications, such as summarization, highlight extraction, event detection, and retrieval. This paper considers the issue in relation to the annotation of baseball videos. Conventional baseball video annotation frameworks are based primarily on video content analysis, such as scoreboard recognition and machine learning techniques, which require a substantial amount of human input to collect and organize training data. The performance of such frameworks might become unstable if they encounter audiovisual patterns not included in the training data. To address the issue, we propose a novel framework for baseball video annotation that aligns high-level webcast text with low-level video content. Several cues, which are derived from the video content and webcast text, are utilized for alignment by leveraging hierarchical agglomerative clustering and genetic algorithm optimization. In addition, we develop an unsupervised method to learn the pitch segment properties of baseball videos by Markov random walk, and thereby reduce the need for human intervention substantially. Our experiments demonstrate that the proposed framework yields a robust result against a variety of video content and enhances the automaticity in baseball video annotation.

Shots Classification for Basketball Video Based on NERFC-Means Clustering

shots classification is the fundamental for sports video annotation. This paper uses non-supervised method to cluster the shots into defined classes (in-play, close-up and free-throw) based on the low-level features of the image (the main color and the histogram). Specifically, we apply the None Euclidean Relational Fuzzy C-means (NERFCM) to cluster the shots after comparing the four Fuzzy C-means and extracting the low-level features of the frames. Experiments prove its efficiency and sensitivity.

Personalized Sports Video Customization Using Content and Context Analysis

We present an integrated framework on personalized sports video customization, which addresses three research issues: semantic video annotation, personalized video retrieval and summarization, and system adaptation. Sports video annotation serves as the foundation of the video customization system. To acquire detailed description of video content, external web text is adopted to align with the related sports video according to their semantic correspondence. Based on the derived semantic annotation, a user-participant multiconstraint 0/1 Knapsack model is designed to model the personalized video customization, which can unify both video retrieval and summarization with different fusion parameters. As a measure to make the system adaptive to the particular user, a social network based system adaptation algorithm is proposed to learn latent user preference implicitly. Both quantitative and qualitative experiments conducted on twelve broadcast basketball and football videos validate the effectiveness of the proposed method.

A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video

Sports video annotation is important for sports video semantic analysis such as event detection and personalization. In this paper, we propose a novel approach for sports video semantic annotation and personalized retrieval. Different from the state of the art sports video analysis methods which heavily rely on audio/visual features, the proposed approach incorporates web-casting text into sports video analysis. Compared with previous approaches, the contributions of our approach include the following. 1) The event detection accuracy is significantly improved due to the incorporation of web-casting text analysis. 2) The proposed approach is able to detect exact event boundary and extract event semantics that are very difficult or impossible to be handled by previous approaches. 3) The proposed method is able to create personalized summary from both general and specific point of view related to particular game, event, player or team according to user's preference. We present the framework of our approach and details of text analysis, video analysis, text/video alignment, and personalized retrieval. The experimental results on event boundary detection in sports video are encouraging and comparable to the manually selected events. The evaluation on personalized retrieval is effective in helping meet users' expectations.

Gesture spotting for low-resolution sports video annotation

Human gesture recognition plays an important role in automating the analysis of video material at a high level. Especially in sports videos, the determination of the player's gestures is a key task. In many sports views, the camera covers a large part of the sports arena, resulting in low resolution of the player's region. Moreover, the camera is not static, but moves dynamically around its optical center, i.e. pan/tilt/zoom camera. These factors make the determination of the player's gestures a challenging task. To overcome these problems, we propose a posture descriptor that is robust to shape corruption of the player's silhouette, and a gesture spotting method that is robust to noisy sequences of data and needs only a small amount of training data. The proposed posture descriptor extracts the feature points of a shape, based on the curvature scale space (CSS) method. The use of CSS makes this method robust to local noise, and our method is also robust to significant shape corruption of the player's silhouette. The proposed spotting method provides probabilistic similarity and is robust to noisy sequences of data. It needs only a small number of training data sets, which is a very useful characteristic when it is difficult to obtain enough data for model training. In this paper, we conducted experiments spotting serve gestures using broadcast tennis play video. From our experiments, for 63 shots of playing tennis, some of which include a serve gesture and while some do not, it achieved 97.5% precision rate and 86.7% recall rate.

A unified approach to the generation of semantic cues for sports video annotation

The use of video and audio features for automated annotation of audio-visual data is becoming widespread. A major limitation of many of the current methods is that the stored indexing features are too low-level—they relate directly to properties of the data. In this work we apply a further stage of processing that associates the feature measurements with real-world objects or events. The outputs, which we call “cues”, are combined to enable us to compute directly the probability of the object being present in the scene. An additional advantage of this approach is that the cues from different types of features are presented in a homogeneous way.