Shot Classification Research Articles

Strategy analysis of badminton players using deep learning from IMU and UWB wearables

Badminton is a fast-paced sport that requires a high level of skill and coordination. To improve their skills, players can use activity trackers to monitor different shots and activities. These trackers utilize inertial measurement units (IMUs), which measures acceleration and angular velocity on the rackets, and the ultra-wideband (UWB) sensors, which measure the location of the player. This study first analyzes the use of UWB localization for tracking badminton players on the court and analyzes the location where shots are played. Furthermore, this study focuses on using both IMU at the racket and wrist and UWB sensors to recognize strategies utilized in badminton matches, employing convolutional neural network (CNN) and Long-Short-Term Memory (LSTM) models. The goal is to classify thirteen badminton shots, as well as an extra class that contains non-shot activities. The output of this shot classifier is provided to the strategy recognition model, which can identify four main strategies, with eleven variations in total, alongside an additional class designated for non-strategy instances such as movement or rest intervals. We trained and tested the models on data from six skilled badminton players. The best results were achieved by using both IMU and UWB data. The proposed 2D-CNN achieved a shot classification accuracy of 90.9%, while the proposed LSTM achieved a strategy recognition accuracy of 80%. The results of this study suggest that neural networks can be used to effectively classify badminton shots and strategies to improve the training of badminton players, as well as to analyze match data.

Volleyball serves impact sound intensity and frequency spectrum affect the predictions of the future ball’s landing point according to the level of auditory-motor experience

ABSTRACT In ball sports, such as volleyball or soccer, the impact sound of the ball is used to define the shot length and predict the final ball position on the field. Several studies thoroughly investigated the role of sound intensity in predicting the ball’s landing point, neglecting the equally important contribution of the sound frequency spectrum component. Here, we investigated the contribution of the sound frequency spectrum in defining a volleyball serve’s length (i.e., distance). We recorded the sound of short and long volleyball serves (ecological sound) and changed their frequency spectrum in pink noise, preserving their intensity (noise sound). We then presented both sounds (ecological and noise) to expert volleyball players and measured their performance in short and long-shot classifications. The role of auditory-motor experience in sound classification was also explored by testing soccer players (experience on ball shots but not sport-specific auditory-motor experience) and non-athletes (no auditory-motor experience in sports). We found that hearing ecological sound leads to better sound discrimination performance than noise sound. Concurrently, we found an effect of the specific auditory-motor experience, with volleyball players outperforming non-athletes and soccer players. Interestingly, experts in different sports domains (soccer players) mainly used sound intensity for shot classification. This underlines the importance of specific auditory-motor experience in encoding specific sound frequency spectrum for the sound of action classification.

Compositional diversity in visual concept learning

Humans leverage compositionality to efficiently learn new concepts, understanding how familiar parts can combine together to form novel objects. In contrast, popular computer vision models struggle to make the same types of inferences, requiring more data and generalizing less flexibly than people do. Here, we study these distinctively human abilities across a range of different types of visual composition, examining how people classify and generate “alien figures” with rich relational structure. We also develop a Bayesian program induction model which searches for the best programs for generating the candidate visual figures, utilizing a large program space containing different compositional mechanisms and abstractions. In few shot classification tasks, we find that people and the program induction model can make a range of meaningful compositional generalizations, with the model providing a strong account of the experimental data as well as interpretable parameters that reveal human assumptions about the factors invariant to category membership (here, to rotation and changing part attachment). In few shot generation tasks, both people and the models are able to construct compelling novel examples, with people behaving in additional structured ways beyond the model capabilities, e.g. making choices that complete a set or reconfigure existing parts in new ways. To capture these additional behavioral patterns, we develop an alternative model based on neuro-symbolic program induction: this model also composes new concepts from existing parts yet, distinctively, it utilizes neural network modules to capture residual statistical structure. Together, our behavioral and computational findings show how people and models can produce a variety of compositional behavior when classifying and generating visual objects.

Enhancing Tennis Serve Scoring Efficiency: An AI Deep Learning Approach

The playing field of a tennis competition is a dynamic and complex formative environment given the following preliminary knowledge: (a) the basic technical, tactical, situational, and special types of shots used by the opponent; (b) the hitting area of the tennis player; (c) the place of service; (d) the ball drop position; and (d) batting efficiency and other related information that may improve the chances of victory. In this study, we propose an AI classification model for tennis serve scores. Using a deep learning algorithm, the model automatically tracks and classifies the serve scores of professional tennis players from video data. We first defined the players’ techniques, volleys, and placements of strokes and serves. Subsequently, we defined the referee's tennis terms and the voice in deciding on a serve score. Finally, we developed a deep learning model to automatically classify the serving position, landing position, and use of tennis techniques. The methodology was applied in the context of 10 matches played by Roger Federer and Rafael Nadal. The proposed deep learning algorithm achieved a 98.27% accuracy in the automatic classification of serve scores, revealing that Nadal outscored Federer by 2.1% in terms of serve-scoring efficiency. These results are expected to facilitate the automatic comparison and classification of shots in future studies, enabling coaches to adjust tactics in a timely manner and thereby improve the chances of winning.

Shot-ViT: Cricket Batting Shots Classification with Vision Transformer Network

Shot-ViT: Cricket Batting Shots Classification with Vision Transformer Network

Shot classification for human behavioural analysis in video surveillance applications

Human behavior analysis plays a vital role in ensuring security and safety of people in crowded public places against diverse contexts like theft detection, violence prevention, explosion anticipation etc. Analysing human behaviour by classifying of videos in to different shot types helps in extracting appropriate behavioural cues. Shots indicates the subject size within the frame and the basic camera shots include: the close-up, medium shot, and the long shot. If the video is categorised as Close-up shot type, investigating emotional displays helps in identifying criminal suspects by analysing the signs of aggressiveness and nervousness to prevent illegal acts. Mid shot can be used for analysing nonverbal communication like clothing, facial expressions, gestures and personal space. For long shot type, behavioural analysis is by extracting the cues from gait and atomic action displayed by the person. Here, the framework for shot scale analysis for video surveillance applications is by using Face pixel percentage and deep learning based method. Face Pixel ratio corresponds to the percentage of region occupied by the face region in a frame. The Face pixel Ratio is thresholded with predefined threshold values and grouped into Close-up shot, mid shot and long shot categories. Shot scale analysis based on transfer learning utilizes effective pre-trained models that includes AlexNet, VGG Net, GoogLeNet and ResNet. From experimentation, it is observed that, among the pre-trained models used for experimentation GoogLeNet tops with the accuracy of 94.61%.

A lightweight weak semantic framework for cinematographic shot classification

Shot is one of the fundamental unit in the content structure of a film, which can provide insights into the film-director’s ideas. By analyzing the properties and types of shots, we can gain a better understanding of a film’s visual language. In this paper, we delve deeply into the task of shot type classification, proposing that utilizing multimodal video inputs can effectively improve the accuracy of the task, and that shot type classification is closely related to low-level spatiotemporal semantic features. To this end, we propose a Lightweight Weak Semantic Relevance Framework (LWSRNet) for classifying cinematographic shot types. Our framework comprises two modules: a Linear Modalities Fusion module (LMF Module) capable of fusing an arbitrary number of video modalities, and a Weak Semantic 3D-CNN based Feature Extraction Backbone (WSFE Module) for classifying shot movement and scale, respectively. Moreover, to support practical cinematographic analysis, we collect FullShots, a large film shot dataset containing 27K shots from 19 movies with professionally annotations for movement and scale information. Following experimental results validate the correctness of our proposed hypotheses, while our framework also outperforms previous methods in terms of accuracy with fewer parameters and computations, on both FullShots and MovieShots datasets. Our code is available at (https://github.com/litchiar/ShotClassification).

Learning Deep Co-Occurrence Features

We exploit the computational capability of deep convolutional neural network (CNN) architecture and the natural interpretability of the co-occurrence matrix (CM) to learn deep co-occurrence features (DCOFs). The DCOFs represent the statistics of the co-occurrences of pixels thus overcoming the black box nature of traditional deep representation learning while at the same time solving the inherent computational difficulty of CM. We propose a parametric co-occurrence matrix (PCM) model to approximate the CM with multivariate Gaussian functions, and have developed three approaches to decomposing the PCM model into linear and nonlinear operations such that the model can be easily implemented using standard CNN operations and to learn the DCOFs of arbitrary shapes. The CNN implementation of the PCM model, termed PCMCNN, can be used as a standard plugin module of a deep learning system and adaptively learns the DCOFs for downstream applications. We demonstrate the broad applicability of the DCOFs and their effectiveness in fine-grained image classification tasks such as texture classification and GAN (generative adversarial network) image detection. The introduction of the PCMCNN module makes it much more compact and efficient than conventional implementations of deep learning models, achieving comparable classification performances to state of the art methods on a variety of benchmarking datasets with models that are more than 30 folds smaller and 11 times less complex. The small model size for learning the DCOFs makes the new method particularly effective for few shot classification of large number of texture categories where the small number of training samples can easily cause traditional deep learning models to overfit. This work shows the potential benefits of combining the principles of traditional handcrafted features and deep representation learning to take advantage of both for advancing state of the art.

A comparative study of machine learning and deep learning algorithms for padel tennis shot classification

Data processing in sports is a phenomenon increasingly present at all levels, from professionals in search of tools to improve their performance to beginners motivated by the quantification of their physical activity. In this work, a comparison between some of the main machine learning and deep learning algorithms is carried out in order to classify padel tennis strokes. Up to 13 representative padel tennis strokes are classified. Before a classification of padel tennis strokes is performed, a sufficiently representative data set is needed that collects numerous examples of the performance of these strokes. Since there was no similar data set in the literature, we proceeded to the creation of such a data set, for which we developed a data collection system based on an electronic device with an inertial measurement unit. Using the developed data set, the machine learning and deep learning algorithms were hyperparameterized to compare their performance under the best possible configurations. The algorithms were fed with both the temporal series of the acceleration and speed of the six degrees of freedom and also with feature engineering input, consisting in calculating the mean, maximum, and minimum values for each axis. The algorithms evaluated are: fully connected or dense neural networks, 1D convolutional neural networks, decision tree, K nearest neighbors, support vector machines, and eigenvalue classification. According to the results achieved, the best algorithm is the 1D convolutional neural network with temporal series input that achieves an accuracy higher than 93%. However, other simpler algorithms such as dense networks and support vector machines achieve similar results.

One shot learning approach for cross spectrum periocular verification.

The use of face mask during the COVID-19 pandemic has increased the popularity of the periocular biometrics in surveillance applications. Despite of the rapid advancements in this area, matching images over cross spectrum is still a challenging problem. Reason may be two-fold 1) variations in image illumination 2) small size of available data sets and/or class imbalance problem. This paper proposes Siamese architecture based convolutional neural networks which works on the concept of one-shot classification. In one shot classification, network requires a single training example from each class to train the complete model which may lead to reduce the need of large dataset as well as doesn't matter whether the dataset is imbalance. The proposed architectures comprise of identical subnetworks with shared weights whose performance is assessed on three publicly available databases namely IMP, UTIRIS and PolyU with four different loss functions namely Binary cross entropy loss, Hinge loss, contrastive loss and Triplet loss. In order to mitigate the inherent illumination variations of cross spectrum images CLAHE was used to preprocess images. Extensive experiments show that the proposed Siamese CNN model with triplet loss function outperforms the states of the art periocular verification methods for cross, mono and multi spectral periocular image matching.

FaNet: Feature-aware network for few shot classification of strip steel surface defects

Accurate classification of surface defects is one of the most important factors in achieving quality inspection for strip steel. Most existing methods are based on fully-supervised learning, which requires a large number of labeled training data samples. In the manufacturing process, collecting defective samples is time-consuming and laborious. So it is very difficult to train a fully supervised model based on few labeled samples. In this paper, we propose a feature-aware network (FaNet) for a few shot defect classification, which can effectively distinguish new classes with a small number of labeled samples. In our proposed FaNet, we use ResNet12 as our baseline. The feature-attention convolution module (FAC) is applied to extract the comprehensive feature information from the base classes, as well as to fuse semantic information by capturing the long-range feature relationships between the upper and lower layers. Meanwhile, during the test phase, an online feature-enhance integration module (FEI) is adopted to average the noise from the support set and query set defect images, further enhancing image features among the different tasks. In addition, we construct a large-scale strip steel surface defects few shot classification dataset (FSC-20) with 20 different types. Experimental results show that the proposed method achieves the best performance compared to state-of-the-art methods for the 5-way 1-shot and 5-way 5-shot tasks. The dataset and code are available at: https://github.com/VDT-2048/FSC-20.

ProZe: Explainable and Prompt-Guided Zero-Shot Text Classification

As technology accelerates the generation and communication of textual data, the need to automatically understand this content becomes a necessity. In order to classify text, being it for tagging, indexing, or curating documents, one often relies on large, opaque models that are trained on preannotated datasets, making the process unexplainable, difficult to scale, and ill-adapted for niche domains with scarce data. To tackle these challenges, we propose ProZe, a text classification approach that leverages knowledge from two sources: prompting pretrained language models, as well as querying ConceptNet, a common-sense knowledge base which can be used to add a layer of explainability to the results. We evaluate our approach empirically and we show how this combination not only performs on par with state-of-the-art zero shot classification on several domains, but also offers explainable predictions that can be visualized.

One shot ancient character recognition with siamese similarity network

Ancient character recognition is not only important for the study and understanding of ancient history but also has a profound impact on the inheritance and development of national culture. In order to reduce the study of difficult professional knowledge of ancient characters, and meanwhile overcome the lack of data, class imbalance, diversification of glyphs, and open set recognition problems in ancient characters, we propose a Siamese similarity network based on a similarity learning method to directly learn input similarity and then apply the trained model to establish one shot classification task for recognition. Multi-scale fusion backbone structure and embedded structure are proposed in the network to improve the model's ability to extract features. We also propose the soft similarity contrast loss function for the first time, which ensures the optimization of similar images with higher similarity and different classes of images with greater differences while reducing the over-optimization of back-propagation leading to model overfitting. Specially, we propose a cumulative class prototype based on our network to solve the deviation problem of the mean class prototype and obtain a good class representation. Since new ancient characters can still be found in reality, our model has the ability to reject unknown categories while identifying new ones. A large number of experiments show that our proposed method has achieved high-efficiency discriminative performance and obtained the best performance over the methods of traditional deep learning and other classic one-shot learning.

Cinematographic Shot Classification with Deep Ensemble Learning

Cinematographic shot classification assigns a category to each shot either on the basis of the field size or on the movement performed by the camera. In this work, we focus on the camera field of view, which is determined by the portion of the subject and of the environment shown in the field of view of the camera. The automation of this task can help freelancers and studios belonging to the visual creative field in their daily activities. In our study, we took into account eight classes of film shots: long shot, medium shot, full figure, american shot, half figure, half torso, close up and extreme close up. The cinematographic shot classification is a complex task, so we combined state-of-the-art techniques to deal with it. Specifically, we finetuned three separated VGG-16 models and combined their predictions in order to obtain better performances by exploiting the stacking learning technique. Experimental results demonstrate the effectiveness of the proposed approach in performing the classification task with good accuracy. Our method was able to achieve 77% accuracy without relying on data augmentation techniques. We also evaluated our approach in terms of f1 score, precision, and recall and we showed confusion matrices to show that most of our misclassified samples belonged to a neighboring class.

Shot classification and replay detection for sports video summarization

Shot classification and replay detection for sports video summarization

Domain Knowledge Powered Two-Stream Deep Network for Few-Shot SAR Vehicle Recognition

Synthetic aperture radar (SAR) target recognition faces the challenge that there are very little labeled data. Although few-shot learning methods are developed to extract more information from a small amount of labeled data to avoid overfitting problems, recent few-shot or limited-data SAR target recognition algorithms overlook the unique SAR imaging mechanism. Domain knowledge-powered two-stream deep network (DKTS-N) is proposed in this study, which incorporates SAR domain knowledge related to the azimuth angle, the amplitude, and the phase data of vehicles, making it a pioneering work in few-shot SAR vehicle recognition. The two-stream deep network, extracting the features of the entire image and image patches, is proposed for more effective use of the SAR domain knowledge. To measure the structural information distance between the global and local features of vehicles, the deep Earth mover’s distance is improved to cope with the features from a two-stream deep network. Considering the sensitivity of the azimuth angle in SAR vehicle recognition, the nearest neighbor classifier replaces the structured fully connected layer for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> -shot classification. All experiments are conducted under the configuration that the SARSIM and the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset work as a source and target task, respectively. Our proposed DKTS-N achieved 49.26% and 96.15% under ten-way one-shot and ten-way 25-shot, whose labeled samples are randomly selected from the training set. In standard operating condition (SOC) as well as three extended operating conditions (EOCs), DKTS-N demonstrated overwhelming advantages in accuracy and time consumption compared with other few-shot learning methods in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> -shot recognition tasks.

Multi-graph convolutional network for a remote sensing image few shot classification

Multi-graph convolutional network for a remote sensing image few shot classification

Classification of Tennis Shots with a Neural Network Approach.

Data analysis plays an increasingly valuable role in sports. The better the data that is analysed, the more concise training methods that can be chosen. Several solutions already exist for this purpose in the tennis industry; however, none of them combine data generation with a wristband and classification with a deep convolutional neural network (CNN). In this article, we demonstrate the development of a reliable shot detection trigger and a deep neural network that classifies tennis shots into three and five shot types. We generate a dataset for the training of neural networks with the help of a sensor wristband, which recorded 11 signals, including an inertial measurement unit (IMU). The final dataset included 5682 labelled shots of 16 players of age 13–70 years, predominantly at an amateur level. Two state-of-the-art architectures for time series classification (TSC) are compared, namely a fully convolutional network (FCN) and a residual network (ResNet). Recent advances in the field of machine learning, like the Mish activation function and the Ranger optimizer, are utilized. Training with the rather inhomogeneous dataset led to an F1 score of 96% in classification of the main shots and 94% for the expansion. Consequently, the study yielded a solid base for more complex tennis analysis tools, such as the indication of success rates per shot type.

Vision based human fall detection with Siamese convolutional neural networks

Fall detection is drawing serious attention all across the globe, as unattended fall of senior citizens creates long lasting injuries. This necessitates the deployment of automatic fall detection systems to facilitate smart care health environments for the elderly people living in various settings, viz., living independently in their homes, hospitalized or living in care homes. The proposed work employs Siamese network with one shot classification for human fall detection. Unlike the neural network that classifies the video sequences, this network learns to differentiate the video sequences by computing the similarity score. The network contains two identical CNNs, receiving pair of video sequences as the input. The features of these networks are merged at the final layer through the similarity function. Two different architectures viz., one with 2D convolutional filters and the other with depth wise convolutional filters, each operated on two set of features, RGB and optical flow features are developed. Experimental results demonstrate the effectiveness and feasibility of the proposed work compared to state-of-the methods.

Extended morphological profiles analysis of airborne hyperspectral image classification using machine learning algorithms

When morphological capabilities are used for the class of high decision hyperspectral photographs from metropolitan areas, one must not forget two crucial problems. Among which the primary one is that traditional morphological openings and closings degrade the object obstacles and distorts the items shape. Morphological profiles (MP) opening and closing via reconstruction can keep us away from this problem, however this system ends in a few unwanted consequences. In this paper, first check out morphological summaries with subjective restoration and steering MPs for the classification of excessive decision hyperspectral snap shots from city areas. Secondly, broaden a supervised face extraction to lessen the dimensionality of the engendered morphological profiles for the prediction.