Weizmann Dataset Research Articles

Research on Artificial Intelligence Technology in Accurate Recognition of Sports Training Actions

In sports training, accurate identification of athletes' movements is helpful to judge whether athletes' actions are standard or not, thus providing precise movement data for training and improving athletes' levels. The convolutional neural network VGG model was constructed based on artificial intelligence technology to extract image features and accurately recognize actions automatically. Weizmann data set was used to train and verify the model. The results of this study showed that the motion recognition algorithm based on the convolutional neural network could achieve high recognition accuracy.

Human Recognization Activity and Maximum Motion Representation in Surveillance Video

Recognizing human actions in the real-world environment finds vital applications, Hence, machine vision studies in this field become crucial. This research aims to extract human activity from video sequences. The number of activities are required for human action recognition, including the gathering of visual data, the identification and presentation of robust features, and the training of classifiers with strong discriminative abilities. The action recognition method employed in this research is based on maximal motion identification. The Region of Interest (ROI) difference picture is utilized to extract motion information, and the Block Based Motion Intensity Code (BBMIC) is extracted as a feature. The Weizmann action dataset, which includes 10 actions (including "walk," "run," "jump," "side," "bend," "wave1," "wave2," "pjump," "jack," and "skip"), was used in the trials. A variety of the tree-based classifiers, including Random-Tree, the Random-Forest, and the Decision-Tree (J48), were also used. According to the experimental findings, this method uses the Weizmann dataset to recognize the activities with an overall quality rate of 94.20%. This is more effective than other tree-based algorithms. Performance of the suggested strategy is on par with that of well-established, established procedures.

ICA and IVA bounded multivariate generalized Gaussian mixture based hidden Markov models

Machine learning (ML), a branch of artificial intelligence (AI), is an area of computational science that is concerned with the analysis and interpretation of patterns and structures in data to enable learning and decision-making without the participation of a human. Hidden Markov models (HMMs), which have been acknowledged for decades but have recently made a significant revival in machine learning, are one of the most impressively powerful probabilistic models. HMMs are frequently employed in machine learning to model heterogeneous time series data. In this paper, we integrate independent component analysis (ICA) and ICA with a bounded multivariate generalized Gaussian mixture model (ICA-BMGGMM) into the HMM approach. One limitation of ICA is that it assumes the sources to be independent from each other. This assumption can be relaxed by combining independent vectors analysis (IVA) and IVA with the BMGGMM (IVA-BMGGMM) into the HMM approach to improve their modeling capability. We validate our proposed models using a variety of applications, such as human action recognition, speech recognition, and energy disaggregation. The results presented in the paper demonstrate the effectiveness of the proposed approaches for modeling different types of data. These data include KTH and Weizmann datasets for human action recognition, TIMIT and SDR for speech recognition, REDD dataset for energy disaggregation and EEG dataset for elliptic seizure classification. For all conducted experiments, our proposed models outperform other comparing models for all performance metrics such as accuracy, sensitivity, and precision. The best detection results were found using the IVABMGGMM-HMM for the reported experiments.

In‐Sensor Reservoir Computing Based on Optoelectronic Synapse

Conventional machine vision systems suffer from great data latency and energy consumption in cognitive tasks due to the separated vision sensors, memory units, and processors. In‐sensor computing based on optoelectronic synapses allows efficient computation by directly sensing and processing optical signals. Herein, an optoelectronic synapse based on Au/ZnO:N/IGZO/TiN structure is proposed. It shows uniform optical SET and electrical RESET behaviors, with various light‐tunable plasticity. Furthermore, a 4‐bit reservoir is experimentally implemented on the device, which is ideal to construct in‐sensor reservoir computing (RC) system. By converting spatiotemporal optical signals to higher dimensional feature space, in‐sensor RC has a great advantage in processing sequential visual information. Simulation results demonstrate that the in‐sensor RC system based on the proposed synapse achieves a considerable recognition accuracy (90.45%) for the MNIST dataset with very limited 36‐30‐10 perceptron network, and a 97.14% accuracy for human action classification from sequential vision data based on the Weizmann dataset. This work proves the low training cost and great efficiency for processing spatiotemporal and sequential optical signals, which may pave a new way for future machine vision applications.

Gerçek Zamanlı İnsan Davranışı Anlamaya Doğru: Optimal-Altı Bir Şekil Tanımlayıcı

In this study, two novel shape descriptors are proposed to be used in human behavior understanding problem. First is optimal shape descriptor, which has high performance but works very slow due to high algorithmic complexity. Second is suboptimal shape descriptor, performance of which is very close to optimal one, but works much more faster. Optimal means using minimum data to represent maximum knowledge. Algorithms are run on Weizmann dataset and results are shown both as figure and video link. Classification was performed using 12 statistical features extracted from the data sets' human silhouettes. An accuracy rating of 92 percent was obtained by using Euclidean distance in classification.

Human Action Recognition (HAR) Classification Using MediaPipe and Long Short-Term Memory (LSTM)

Human Action Recognition is an important research topic in Machine Learning and Computer Vision domains. One of the proposed methods is a combination of MediaPipe library and Long Short-Term Memory concerning the testing accuracy and training duration as indicators to evaluate the model performance. This research tried to adapt proposed LSTM models to implement HAR with image features extracted by MediaPipe library. There would be a comparison between LSTM models based on their testing accuracy and training duration. This research was conducted under OSEMN methods (Obtain, Scrub, Explore, Model, and iNterpret). The dataset was preprocessed Weizmann dataset with data preprocessing and data augmentation implementations. Video features extracted by MediaPipe: Pose was used in training and validation processes on neural network models focusing on Long Short-Term Memory layers. The processes were finished by model performance evaluation based on confusion matrices interpretation and calculations of accuracy, error rate, precision, recall, and F1score. This research yielded seven LSTM model variants with the highest testing accuracy at 82% taking 10 minutes and 50 seconds of training duration.

A novel human activity recognition architecture: using residual inception ConvLSTM layer

Human activity recognition (HAR) is a very challenging problem that requires identifying an activity performed by a single individual or a group of people observed from spatiotemporal data. Many computer vision applications require a solution to HAR. To name a few, surveillance systems, medical and health care monitoring applications, and smart home assistant devices. The rapid development of machine learning leads to a great advance in HAR solutions. One of these solutions is using ConvLSTM architecture. ConvLSTM architectures have recently been used in many spatiotemporal computer vision applications.In this paper, we introduce a new layer, residual inception convolutional recurrent layer, ResIncConvLSTM, a variation of ConvLSTM layer. Also, a novel architecture to solve HAR using the introduced layer is proposed. Our proposed architecture resulted in an accuracy improvement by 7% from ConvLSTM baseline architecture. The comparisons are held in terms of classification accuracy. The architectures are trained using KTH dataset and tested against both KTH and Weizmann datasets. The architectures are also trained and tested against a subset of UCF Sports Action dataset. Also, experimental results show the effectiveness of our proposed architecture compared to other state-of-the-art architectures.

K-Means image segmentation using Mumford–Shah model

k-Means (KM) is well known for its ease of implementation as a clustering technique. It has been applied for color quantization in RGB, YUV, hyperspectral image, Lab, and other spaces, but this leads to fragmented segments as the pixels are clustered only in the color space without considering connectivity. The problem has been attacked by adding connectivity constraints, or using joint color and spatial features (r, g, b, x, y), which prevent fragmented and nonconvex segments. However, it does not take into account the complexity of the shape itself. The Mumford–Shah model has been earlier used to overcome this problem but with slow and complex mathematical optimization algorithms. We integrate Mumford–Shah model directly into KM context and construct a fast and simple implementation of the algorithm. The proposed approach uses standard KM algorithm with distance function derived from Mumford–Shah model so that it optimizes both the content and the shape of the segments jointly. We demonstrate by experiments that the proposed algorithm provides better results than comparative methods when compared using various error evaluation criteria. The algorithm is applied on 100 images in the Weizmann dataset and two remote sensing images.

Football Players’ Shooting Posture Norm Based on Deep Learning in Sports Event Video

Football is one of the favorite sports of people nowadays. Shooting is the ultimate goal of all offensive tactics in football matches. This is the most basic way to score a goal and the only way to score a goal. The choice and use of shooting technical indicators can have a great impact on the final result of the game. Therefore, how to improve the shooting technique of football players and how to adjust the shooting posture of football players are important issues faced by coaches and athletes. In recent years, deep learning has been widely used in various fields such as image classification and recognition and language processing. How to apply deep learning optimization to shooting gesture recognition is a very promising research direction. This article aims to study the football player’s shooting posture specification based on deep learning in sports event videos. Based on the analysis of target motion detection algorithm, target motion tracking algorithm, target motion recognition algorithm, and football shooting posture classification, KTH and Weizmann data sets are used. As the experimental verification data set of this article, the shooting posture of football players in the sports event video is recognized, and the accuracy of the action recognition is finally calculated to standardize the football shooting posture. The experimental results show that the Weizmann data set has a higher accuracy rate than the KTH data set and is more suitable for shooting attitude specifications.

Extraction and Recognition Method of Basketball Players’ Dynamic Human Actions Based on Deep Learning

The extraction and recognition of human actions has always been a research hotspot in the field of state recognition. It has a wide range of application prospects in many fields. In sports, it can reduce the occurrence of accidental injuries and improve the training level of basketball players. How to extract effective features from the dynamic body movements of basketball players is of great significance. In order to improve the fairness of the basketball game, realize the accurate recognition of the athletes’ movements, and simultaneously improve the level of the athletes and regulate the movements of the athletes during training, this article uses deep learning to extract and recognize the movements of the basketball players. This paper implements human action recognition algorithm based on deep learning. This method automatically extracts image features through convolution kernels, which greatly improves the efficiency compared with traditional manual feature extraction methods. This method uses the deep convolutional neural network VGG model on the TensorFlow platform to extract and recognize human actions. On the Matlab platform, the KTH and Weizmann datasets are preprocessed to obtain the input image set. Then, the preprocessed dataset is used to train the model to obtain the optimal network model and corresponding data by testing the two datasets. Finally, the two datasets are analyzed in detail, and the specific cause of each action confusion is given. Simultaneously, the recognition accuracy and average recognition accuracy rates of each action category are calculated. The experimental results show that the human action recognition algorithm based on deep learning obtains a higher recognition accuracy rate.

Robust video content analysis schemes for human action recognition.

Action recognition is a challenging time series classification task that has received much attention in the recent past due to its importance in critical applications, such as surveillance, visual behavior study, topic discovery, security, and content retrieval. The main objective of the research is to develop a robust and high-performance human action recognition techniques. A combination of local and holistic feature extraction methods used through analyzing the most effective features to extract to reach the objective, followed by using simple and high-performance machine learning algorithms. This paper presents three robust action recognition techniques based on a series of image analysis methods to detect activities in different scenes. The general scheme architecture consists of shot boundary detection, shot frame rate re-sampling, and compact feature vector extraction. This process is achieved by emphasizing variations and extracting strong patterns in feature vectors before classification. The proposed schemes are tested on datasets with cluttered backgrounds, low- or high-resolution videos, different viewpoints, and different camera motion conditions, namely, the Hollywood-2, KTH, UCF11 (YouTube actions), and Weizmann datasets. The proposed schemes resulted in highly accurate video analysis results compared to those of other works based on four widely used datasets. The First, Second, and Third Schemes provides recognition accuracies of 57.8%, 73.6%, and 52.0% on Hollywood2, 94.5%, 97.0%, and 59.3% on KTH, 94.5%, 95.6%, and 94.2% on UCF11, and 98.9%, 97.8% and 100% on Weizmann. Each of the proposed schemes provides high recognition accuracy compared to other state-of-art methods. Especially, the Second Scheme as it gives excellent comparable results to other benchmarked approaches.

Detection of individual activities in video sequences based on fast interference discovery and semi-supervised method

Auto understanding of human activities in video is an increasing necessity in some application realms. The existing methods for human’s activity identification are divided into two methods: activity recognition and activity detection. The most important challenge in activity detection realm is activity boundary false detection which decreases system accuracy. In this research, an activity detection system was suggested denoting rapid interference and sewing it. Although it has improved accuracy it has also accuracy time, activities in suggested system were replayed more usefully and influenced by creating a descriptor denoting movable and apparent form. The suggested system was tested on Weizmann dataset and reached an accuracy of 93.34%. Furthermore, the proposed system in activity recognition was tested on KTH dataset and reached an accuracy of 93.63%. When activity recognition is stated as a learning case, sufficient labeled educational examples must be used. But labeling the video data is expensive, so the useful method uses unlabeled and labeled examples, during the learning process, this idea is the basic foundation of the semi-supervised method. In this research, a semi-supervised method with co-training algorithm appearance and active learning was suggested which improved the efficiency of semi-supervised learning that was tested.

Fusion-based Gaussian mixture model for background subtraction from videos

Human Activity Recognition (HAR) aims to realise and interpret human activities from videos, and it comprises background subtraction, feature extraction and classification stages. Among those stages, the background subtraction stage is mandatory to achieve a better recognition rate while analysing the videos. The proposed Fusion-based Gaussian Mixture Model (FGMM) background subtraction algorithm extracts the foreground from videos invariant to illumination, shadows and the dynamic background. The proposed FGMM algorithm consists of three stages: background detection, colour similarity and colour distortion calculation. Here, the Jefries-Matusita distance measure is utilised to check whether the current pixel matches the Gaussian distribution, and by using this value, the background model is updated. Weighted Euclidean based colour similarity measure is used to eliminate shadows, and colour distortion measure is adopted to handle illumination variations. The extracted foreground is binarised to easily extract the foreground's interest points, which has white pixels stored into the frame. This algorithm has experimented over test sets gathered from publicly available benchmark data sets such as the K-th data set, Weizmann data set, PETS data set and change detection data set. Experimental results have proved that the proposed FGMM exhibits better accuracy in foreground detection, with better accuracy than the prevailing approaches.

Construction and evaluation of the human behavior recognition model in kinematics under deep learning

To explore the construction and evaluation of the human behavior recognition model in kinematics by deep learning, the convolution neural network (CNN) in the field of deep learning was applied to build the CNN human behavior recognition algorithm model. The image data were collected from the KTH and Weizmann datasets and trained; then, the proposed algorithm was simulated by the TensorFlow platform. The results suggested that in the analysis of recognition effect of kinematic description in two datasets, the accuracy of the histogram of optical flow orientation (HOF) method was the worst in both KTH and Weizmann datasets, while the accuracy of the constructed Visual Geometry Group-16 (VGG-16) algorithm model was the highest. In the analysis of the accuracy of the KTH dataset, the boxing action had the highest accuracy of recognition, and the running action had the lowest accuracy of recognition. The average recognition value of all kinds of actions was 91.93%. In the precision analysis of the Weizmann dataset, the bending and hand waving actions had the most accurate recognition rate, while the running action had the lowest recognition rate.

R-STDP Based Spiking Neural Network for Human Action Recognition

ABSTRACT Video surveillance systems are omnipresent and automatic monitoring of human activities is gaining importance in highly secured environments. The proposed work explores the use of the bio-inspired third generation neural network called spiking neural network (SNN) in order to recognize the action sequences present in a video. The SNN used in this work carries the neural information in terms of timing of spikes rather than the shape of the spikes. The learning technique used herein is reward-modulated spike time-dependent plasticity (R-STDP). It is based on reinforcement learning that modulates or demodulates the synaptic weights depending on the reward or the punishment signal that it receives from the decision layer. The absence of gradient descent techniques and external classifiers makes the system computationally efficient and simple. Finally, the performance of the network is evaluated on the two benchmark datasets, viz., Weizmann and KTH datasets.

Recurrent Self-Structuring Machine Learning for Video Processing using Multi-Stream Hierarchical Growing Self-Organizing Maps

The emergence of IoT and advanced multimedia information systems have undoubtedly created a proliferation of video sensor data. Although diverse machine learning approaches are utilized to extract useful insights from these data, limitations occur when processing and accommodating the large volumes of video data, which are unlabeled and have previously unseen data structures. This brings out the importance of using self-structuring intelligence that can adapt to the nature of the data and with the ability to learn from multi-modal, spatiotemporal and unstructured data. Encompassing these advances, we propose a recurrent self-structuring machine learning approach for video processing using multi-stream hierarchical recurrent growing self-organizing maps (RGSOM) architecture. We have designed, implemented and evaluated the said approach using a human activity recognition video dataset (Weizmann dataset), achieving state-of-the-art accuracy of 93.5% in the unsupervised domain. We used both spatial and temporal data from the video as separate input feature streams, where RGSOMs were used to self-structure the video data in multi-streams for visual exploratory analysis and video classification. As potential implications, this study can contribute to the existing literature in advancing self-adaptation techniques for video sensor data processing.

Particle swarm optimization with deep learning for human action recognition

A novel method for human action recognition using a deep learning network with features optimized using particle swarm optimization is proposed. The binary histogram, Harris corner points and wavelet coefficients are the features extracted from the spatiotemporal volume of the video sequence. In order to reduce the computational complexity of the system, the feature space is reduced by particle swarm optimization technique with the multi-objective fitness function. Finally, the performance of the system is evaluated using deep learning neural network (DLNN). Two autoencoders are trained independently and the knowledge embedded in the autoencoders are transferred to the proposed DLNN for human action recognition. The proposed framework achieves an average recognition rate of 91% on UT interaction set 1, 88% on UT interaction set 2, 91% on SBU interaction dataset and 94% on Weizmann dataset.

Human Action Recognition Based on Multiple Features and Modified Deep Learning Model

In order to improve the accuracy of human action recognition in video and the computational efficiency of large data sets, an action recognition algorithm based on multiple features and modified deep learning model is proposed. First, the deep network pre-training process is used to learn and optimize the RBM parameters, and the deep belief nets (DBN) model is constructed through deep learning. Then, human 13 joint points and critical points of optical flow are automatically extracted by DBN model. Second, these more abstract and more effective human motion features are combined to represent human actions. Ultimately, the entire DBN network structure is fine-tuned by support vector machine (SVM) algorithm to classify human actions. We demonstrate that human 13 joint points and critical points of optical flow are two very effective human action characterizations, our proposed approach greatly reduces the required samples, and shortens the training time of the samples, can efficiently process large data sets and can effectively recognize novel actions. We performed experiments on the KTH data set, Weizmann data set, the ballet data set and UCF101 data set to evaluate the proposed method, the experiment results show that the average recognition accuracy is over 98%, which validates its effectiveness, and show that our results are stable, reliable, and significantly better than the results of two state-of-the-art approaches on four different data sets. So, it lays a good theoretical foundation for practical applications.

Foreground Extraction and Motion Recognition Technology for Intelligent Video Surveillance

With the rapid development of computer technology and network technology, it has become possible to build a large-scale networked video surveillance system. The video surveillance system has become a new type of infrastructure necessary for modern cities. In this paper, the problem of foreground extraction and motion recognition in intelligent video surveillance is studied. The three key sub-problems, namely the extraction of motion foreground in video, the deblurring of motion foreground and the recognition of human motion, are studied and corresponding solutions are proposed. A background modeling technique based on video block is proposed. The background is modeled at the block level, which greatly reduces the spatial complexity of the algorithm. It solves the problem that the traditional Gaussian model (GMM) moving target enters the static state and is integrated into the background process. The target starts to move for a long time and there are ghosts and other problems, which reduce the processing efficiency of the lifting algorithm. The test results on the Weizmann dataset show that the proposed algorithm can achieve high human motion recognition accuracy and recognition with low computational complexity. The rate can reach 100%; the local constrained group sparse representation classification (LGSRC) model is used to classify it. The experimental results on Weizmann, KTH, UCF sports and other test datasets confirm the validity of the algorithm in this chapter. KNN, SRC voting classification accuracy.

HUMAN ACTIVITY DETECTION AND ACTION RECOGNITION IN VIDEOS USING CONVOLUTIONAL NEURAL NETWORKS

Human activity recognition from video scenes has become a significant area of research in the field of computer vision applications. Action recognition is one of the most challenging problems in the area of video analysis and it finds applications in human-computer interaction, anomalous activity detection, crowd monitoring and patient monitoring. Several approaches have been presented for human activity recognition using machine learning techniques. The main aim of this work is to detect and track human activity, and classify actions for two publicly available video databases. In this work, a novel approach of feature extraction from video sequence by combining Scale Invariant Feature Transform and optical flow computation are used where shape, gradient and orientation features are also incorporated for robust feature formulation. Tracking of human activity in the video is implemented using the Gaussian Mixture Model. Convolutional Neural Network based classification approach is used for database training and testing purposes. The activity recognition performance is evaluated for two public datasets namely Weizmann dataset and Kungliga Tekniska Hogskolan dataset with action recognition accuracy of 98.43% and 94.96%, respectively. Experimental and comparative studies have shown that the proposed approach outperformed state-of the art techniques.