Automatic Activity Recognition Research Articles

Towards Automatic Object Detection and Activity Recognition in Indoor Climbing

Rock climbing has propelled from niche sport to mainstream free-time activity and Olympic sport. Moreover, climbing can be studied as an example of a high-stakes perception-action task. However, understanding what constitutes an expert climber is not simple or straightforward. As a dynamic and high-risk activity, climbing requires a precise interplay between cognition, perception, and precise action execution. While prior research has predominantly focused on the movement aspect of climbing (i.e., skeletal posture and individual limb movements), recent studies have also examined the climber’s visual attention and its links to their performance. To associate the climber’s attention with their actions, however, has traditionally required frame-by-frame manual coding of the recorded eye-tracking videos. To overcome this challenge and automatically contextualize the analysis of eye movements in indoor climbing, we present deep learning-driven (YOLOv5) hold detection that facilitates automatic grasp recognition. To demonstrate the framework, we examined the expert climber’s eye movements and egocentric perspective acquired from eye-tracking glasses (SMI and Tobii Glasses 2). Using the framework, we observed that the expert climber’s grasping duration was positively correlated with total fixation duration (r = 0.807) and fixation count (r = 0.864); however, it was negatively correlated with the fixation rate (r = −0.402) and saccade rate (r = −0.344). The findings indicate the moments of cognitive processing and visual search that occurred during decision making and route prospecting. Our work contributes to research on eye–body performance and coordination in high-stakes contexts, and informs the sport science and expands the applications, e.g., in training optimization, injury prevention, and coaching.

Random forest based robust human activity detection system using hybrid feature

Human Activity Recognition is analyzing surveillance videos of a person’s activity. Tracking and identifying activities is essential in various applications like fight detection, mob lynching, etc. Human activity recognition may be the primary or secondary goal of a significant problem target. Surveillance for fight detection, crowd violence, public attacks, mob lynching, public robbery, etc, are few to list. The researchers are getting done in this direction, and algorithms are being proposed for automatic activity recognition. These are typically restricted to the recordings made by stationary cameras, though. For automated Human Activity Recognition, a novel skeleton-based feature called ‘Orientation Invariant Skeleton Feature (OISF)’ was introduced in the earlier work. This work uses a hybrid feature, which is a combination of ‘OISF’ features (proposed by Neelam Dwivedi et al) and ‘FV1’ features (proposed by S Kumar et al) for human activity detection systems. The hybrid features used in this paper have a low dependence on changes in camera orientation, according to experimental results. The accuracy obtained is higher than that of earlier studies using existing features and is roughly 99.50% with the ViHASi dataset and 97.25% with the KTH dataset. This demonstrates that the proposed study is suitable for commercial use.

Enhancing volleyball training: empowering athletes and coaches through advanced sensing and analysis.

Modern sensing technologies and data analysis methods usher in a new era for sports training and practice. Hidden insights can be uncovered and interactive training environments can be created by means of data analysis. We present a system to support volleyball training which makes use of Inertial Measurement Units, a pressure sensitive display floor, and machine learning techniques to automatically detect relevant behaviours and provides the user with the appropriate information. While working with trainers and amateur athletes, we also explore potential applications that are driven by automatic action recognition, that contribute various requirements to the platform. The first application is an automatic video-tagging protocol that marks key events (captured on video) based on the automatic recognition of volleyball-specific actions with an unweighted average recall of 78.71% in the 10-fold cross-validation setting with convolution neural network and 73.84% in leave-one-subject-out cross-validation setting with active data representation method using wearable sensors, as an exemplification of how dashboard and retrieval systems would work with the platform. In the context of action recognition, we have evaluated statistical functions and their transformation using active data representation besides raw signal of IMUs sensor. The second application is the "bump-set-spike" trainer, which uses automatic action recognition to provide real-time feedback about performance to steer player behaviour in volleyball, as an example of rich learning environments enabled by live action detection. In addition to describing these applications, we detail the system components and architecture and discuss the implications that our system might have for sports in general and for volleyball in particular.

Outdoor activity classification using smartphone based inertial sensor measurements

Human Activity Recognition (HAR) deals with the automatic recognition of physical activities and plays a crucial role in healthcare and sports where wearable sensors and intelligent computational techniques are used. We propose a HAR algorithm that uses the smartphones accelerometer data for human activity recognition. In particular, we present a recurrent convolutional neural network-based HAR algorithm that combines a Convolutional Neural Network (CNN) to extract temporal features from the sensor data, a Fuzzy C-Means (FCM) clustering algorithm to cluster the features extracted by the CNN, and a Long Short-Term Memory (LSTM) network to learn the temporal dependencies between the features. We evaluate the proposed methodology on two distinct datasets: the MotionSense dataset and the WISDM dataset. We evaluate the proposed CNN-FCM-LSTM model on the publicly available MotionSense dataset to classify ten activity types: 1) walking upstairs, 2) walking downstairs, 3) jogging, 4) sitting, 5) standing, 6) level ground walking, 7) jumping jacks, 8) brushing teeth, 9) writing, and 10) eating. Next, we evaluate the model’s performance on the WISDM dataset to assess its ability to generalize to unseen data. On the MotionSense test dataset, CNN-FCM-LSTM achieves a classification accuracy of 99.69%, a sensitivity of 99.62%, a specificity of 99.63%, and a false positive rate per hour (FPR/h) of 0.37%. Meanwhile, it achieves a classification accuracy of 97.27% on the WISDM dataset. The CNN-FCM-LSTM model’s capability to classify a diverse range of activities within a single architecture is noteworthy. The results suggest that the proposed CNN-FCM-LSTM model using smartphone inputs is more accurate, reliable, and robust in detecting and classifying activities than the state-of-the-art models. It should be noted that activity recognition technology has the potential to aid in studying the underpinnings of physical activity, designing more effective training regimens, and simulating the rigors of competition in sports.

Human worker activity recognition in a production floor environment through deep learning

One of the aims of the 4th industrial revolution is to seamlessly connect equipment and personnel to enable a greater level of collaboration, which in turn will result in higher operational efficiency and improved decision making strategies. In this context, this work presents a deep learning based approach for automated activity recognition of two human workers who are carrying out typical manual assembly and part handling tasks in a production floor environment. A typical smartphone is employed as a wearable sensor recording acceleration in 3 directions, total acceleration, brightness, geomagnetic field strength, orientation in space and sound wave intensity and is attached to the wrist of the two workers, who subsequently perform a large number of experiments examining 5 activities of interest. For the classification of these activities, two types of neural networks are employed. Initially, the experimental signals are fed to the variational autoencoder neural network so as to extract the appropriate features and subsequently these features are added to the last layer of a long short-term memory (LSTM) time series network, which is trained to classify the 5 activities of interest. An additional goal of the present paper is the study of long-term signals (LTS), that is time series of much longer duration than those of the 5 aforementioned activities, including enough noise due to the existence of time periods corresponding to unrelated activities. Ultimately, a new methodology is proposed combining both trained neural networks mentioned above, in order to identify the desired activities within the LTS in a real time application.

A multi-resolution fusion approach for human activity recognition from video data in tiny edge devices

Human Activity Recognition (HAR) is the process of automatic recognition of Activities of Daily Living (ADL) from human motion data captured in various data modalities by wearable and ambient sensors. Advances in Deep Learning, especially Convolutional Neural Networks (CNNs) and sequential models have revolutionalized HAR from video data sources. Although these models are incredibly accurate and utilize both spatial and temporal information, they require huge computation and memory resources — making them unsuitable for edge or wearable applications. Tiny Machine Learning (TinyML) aims to optimize these models in terms of compute and memory requirements – aiming to make them suitable for always-on resource constrained devices – leading to a reduction in communication latency and network traffic for HAR frameworks. In this paper, we propose a two-stream multi-resolution fusion architecture for HAR from video data modality. The context stream takes a resized image as input and the fovea stream takes the cropped center portion of the resized image as input, reducing the overall dimensionality. We tested two quantization methods: Post-Training Quantization (PTQ) and Quantization Aware Training (QAT) to optimize these models for deployment in edge devices and tested the performance in two challenging video datasets: KTH and UCF11. We performed ablation studies to validate the two-stream model performance. We deployed the proposed architecture in commercial resource constrained devices and monitored their performance in terms of inference latency and power consumption. The results indicate that the proposed architecture clearly outperforms other relevant single-stream models tested in this work in terms of accuracy, precision, recall, and F1-Score while also reducing the overall model size.

Optimized deep learning-based cricket activity focused network and medium scale benchmark

The recognition of different activities in sports has gained attention in recent years for its applications in various athletic events, including soccer and cricket. Cricket, in particular, presents a challenging task for automatic activity recognition methods due to its closely overlapped activities such as cover drive, and pull short, to name a few. Existing methods often rely on hand-crafted features as the limited availability of public data has restricted the scope of research to only the significant categories of cricket activities. To this end, we proposed a cricket activities dataset and an intuitive end-to-end deep learning model for cricket activity recognition. The data is collected from online sources and pre-processed through cleaning, resizing, and organizing. Similarly, an intuitive deep model is designed with a combination of time-distributed 2D CNN layers and LSTM cells for extracting and learning the spatiotemporal information from the input sequences. For benchmarking, we evaluated the model on our cricket datasets and four standard datasets namely UCF101, HMDB51, YouTube action, and Kinetics. The quantitative results show that the proposed model outperforms different variants of recurrent neural networks and achieved an accuracy of 92%, recall of 91%, and F1 score of 91%. Our code and dataset is publicly available for further research on https://drive.google.com/file/d/1c9qcAz4q00qvx4yFA3pSudWFczm1cWUL/view?usp=sharing.

Automatic Recognition of Construction Worker Activities Using Deep Learning Approaches and Wearable Inertial Sensors

The automated evaluation and analysis of employee behavior in an Industry 4.0-compliant manufacturing firm are vital for the rapid and accurate diagnosis of work performance, particularly during the training of a new worker. Various techniques for identifying and detecting worker performance in industrial applications are based on computer vision techniques. Despite widespread computer vision-based approaches, it is challenging to develop technologies that assist the automated monitoring of worker actions at external working sites where camera deployment is problematic. Through the use of wearable inertial sensors, we propose a deep learning method for automatically recognizing the activities of construction workers. The suggested method incorporates a convolutional neural network, residual connection blocks, and multi-branch aggregate transformation modules for high-performance recognition of complicated activities such as construction worker tasks. The proposed approach has been evaluated using standard performance measures, such as precision, F1-score, and AUC, using a publicly available benchmark dataset known as VTT-ConIoT, which contains genuine construction work activities. In addition, standard deep learning models (CNNs, RNNs, and hybrid models) were developed in different empirical circumstances to compare them to the proposed model. With an average accuracy of 99.71% and an average F1-score of 99.71%, the experimental findings revealed that the suggested model could accurately recognize the actions of construction workers. Furthermore, we examined the impact of window size and sensor position on the identification efficiency of the proposed method.

Deep learning-based activity recognition and fine motor identification using 2D skeletons of cynomolgus monkeys.

Video-based action recognition is becoming a vital tool in clinical research and neuroscientific study for disorder detection and prediction. However, action recognition currently used in non-human primate (NHP) research relies heavily on intense manual labor and lacks standardized assessment. In this work, we established two standard benchmark datasets of NHPs in the laboratory: MonkeyinLab (MiL), which includes 13 categories of actions and postures, and MiL2D, which includes sequences of two-dimensional (2D) skeleton features. Furthermore, based on recent methodological advances in deep learning and skeleton visualization, we introduced the MonkeyMonitorKit (MonKit) toolbox for automatic action recognition, posture estimation, and identification of fine motor activity in monkeys. Using the datasets and MonKit, we evaluated the daily behaviors of wild-type cynomolgus monkeys within their home cages and experimental environments and compared these observations with the behaviors exhibited by cynomolgus monkeys possessing mutations in the MECP2 gene as a disease model of Rett syndrome (RTT). MonKit was used to assess motor function, stereotyped behaviors, and depressive phenotypes, with the outcomes compared with human manual detection. MonKit established consistent criteria for identifying behavior in NHPs with high accuracy and efficiency, thus providing a novel and comprehensive tool for assessing phenotypic behavior in monkeys.

Recognition of crowd abnormal activities using fusion of handcrafted and deep features

Constant vigilance is extremely important at crowded public places where some unusual activities such as sudden dispersion or continuous gathering of people may lead to chaotic and disastrous situations. Automatic recognition of such collective activities of people is indeed an important task to ensure people safety. In this view, we propose a novel approach for automatic recognition of crowd merging and sudden dispersion events. The proposed method detects dispersion and merging using fusion of features extracted by deep networks with a novel set of optical flow based and density based handcrafted features. These proposed features are not affected by occlusion and illumination. These features complement the features extracted by deep network and their fusion improves the performance of event recognition by significant amount. The method is tested profoundly on benchmark public datasets as well as private datasets. Abnormal activity recognition data often suffers from high class imbalance. However, the proposed method could successfully recognize sudden dispersion and merging activities on such very small datasets having class imbalance. This proves the effectiveness and robustness of the proposed features. The proposed method also shows better performance than other state of the art methods based on deep networks.

Automated activity recognition of construction workers using single in-pocket smartphone and machine learning methods

Automatic recognition of construction workers’ activities contributes to improving productivity and reducing the potential risk of injury. Kinematics sensors have been proved feasible and efficient to recognize construction activities. However, most of the sensors need to be tightly tied to workers’ bodies, which might result in uncomfortableness and workers’ reluctance to wear the sensors. To solve the problem, this paper proposes a less physically intrusive construction activities recognition method with a single in-pocket smartphone. The smartphone was placed in the pocket in a natural and non-fixed manner, with its built-in accelerometer and gyroscope collecting motion data. Machine learning-based classifiers were trained to recognize construction activities. An experiment simulating rebar activities was designed to verify the effectiveness of the proposed method. The experiment results showed that the proposed method could identify rebar activities (with an accuracy over 94%) in a non-intrusive manner.

Context-Aware Complex Human Activity Recognition Using Hybrid Deep Learning Models

Smart devices, such as smartphones, smartwatches, etc., are examples of promising platforms for automatic recognition of human activities. However, it is difficult to accurately monitor complex human activities on these platforms due to interclass pattern similarities, which occur when different human activities exhibit similar signal patterns or characteristics. Current smartphone-based recognition systems depend on traditional sensors, such as accelerometers and gyroscopes, which are built-in in these devices. Therefore, apart from using information from the traditional sensors, these systems lack the contextual information to support automatic activity recognition. In this article, we explore environmental contexts, such as illumination (light conditions) and noise level, to support sensory data obtained from the traditional sensors using a hybrid of Convolutional Neural Network and Long Short-Term Memory (CNN–LSTM) learning models. The models performed sensor fusion by augmenting low-level sensor signals with rich contextual data to improve the models’ recognition accuracy and generalization. Two sets of experiments were performed to validate the proposed solution. The first set of experiments used triaxial inertial sensing signals to train baseline models, while the second set of experiments combined the inertial signals with contextual information from environmental sensors. The obtained results demonstrate that contextual information, such as environmental noise level and light conditions using hybrid deep learning models, achieved better recognition accuracy than the traditional baseline activity recognition models without contextual information.

Automatic Swimming Activity Recognition and Lap Time Assessment Based on a Single IMU: A Deep Learning Approach.

This study presents a deep learning model devoted to the analysis of swimming using a single Inertial Measurement Unit (IMU) attached to the sacrum. Gyroscope and accelerometer data were collected from 35 swimmers with various expertise levels during a protocol including the four swimming techniques. The proposed methodology took high inter- and intra-swimmer variability into account and was set up for the purpose of predicting eight swimming classes (the four swimming techniques, rest, wallpush, underwater, and turns) at four swimming velocities ranging from low to maximal. The overall F1-score of classification reached 0.96 with a temporal precision of 0.02 s. Lap times were directly computed from the classifier thanks to a high temporal precision and validated against a video gold standard. The mean absolute percentage error (MAPE) for this model against the video was 1.15%, 1%, and 4.07%, respectively, for starting lap times, middle lap times, and ending lap times. This model is a first step toward a powerful training assistant able to analyze swimmers with various levels of expertise in the context of in situ training monitoring.

A new approach for physical human activity recognition from sensor signals based on motif patterns and long-short term memory

Numerous studies have been carried out in recent years on the recognition, tracking, and discrimination of human activities. Automatic recognition of physical activities is often referred to as human activity recognition (HAR). There are generally vision-based and sensor-based approaches for activity recognition. The computer vision-based approach generally works well in laboratory conditions, but it can fail in real-world problems due to clutter, variable light intensity, and contrast. Sensor-based HAR systems are realized by continuously monitoring and analyzing physiological signals measured from heterogeneous sensors connected to the person's body. In this study, the Motif Patterns (MP) approach, which extracts features from sensor signals, is proposed for HAR. The success of the HAR systems depends on the effectiveness of the features extracted from the signals. The LSTM network is a special kind of recurrent neural network that has been used to make very successful predictions on time series data where long-term dependencies are. The LSTM network type offers a successful solution approach to solving long-term dependencies problems such as human activity recognition. The classification process was carried out with Long-Short Term Memory (LSTM) using MP features extracted from accelerometer, gyroscope, and magnetometer sensor signals. A large dataset of 9120 signals was used to test the proposed approach. A high success rate of 98.42 % was achieved with the proposed MP + LSTM method. As a result, it has been seen that the proposed approach has been obtained with a high success rate for HAR using sensor signals.

Gesture Recognition in Robotic Surgery With Multimodal Attention.

Automatically recognising surgical gestures from surgical data is an important building block of automated activity recognition and analytics, technical skill assessment, intra-operative assistance and eventually robotic automation. The complexity of articulated instrument trajectories and the inherent variability due to surgical style and patient anatomy make analysis and fine-grained segmentation of surgical motion patterns from robot kinematics alone very difficult. Surgical video provides crucial information from the surgical site with context for the kinematic data and the interaction between the instruments and tissue. Yet sensor fusion between the robot data and surgical video stream is non-trivial because the data have different frequency, dimensions and discriminative capability. In this paper, we integrate multimodal attention mechanisms in a two-stream temporal convolutional network to compute relevance scores and weight kinematic and visual feature representations dynamically in time, aiming to aid multimodal network training and achieve effective sensor fusion. We report the results of our system on the JIGSAWS benchmark dataset and on a new in vivo dataset of suturing segments from robotic prostatectomy procedures. Our results are promising and obtain multimodal prediction sequences with higher accuracy and better temporal structure than corresponding unimodal solutions. Visualization of attention scores also gives physically interpretable insights on network understanding of strengths and weaknesses of each sensor.

Features and Components Preferred by Adolescents in Smartphone Apps for the Promotion of Physical Activity: Focus Group Study.

BackgroundThere is solid evidence that lack of physical activity (PA) is a risk factor for chronic diseases. Sufficient levels of PA in childhood and adolescence are particularly important, as they can set the standards for PA levels in adulthood. The latest reports show that only a small percentage of adolescents reach the recommended levels of PA in European Union countries at the age of 15 years. In view of the scale of the problem, it is crucial to develop interventions that promote and support PA in adolescents. Considering their low implementation costs and ubiquitous presence, smartphone apps could be advantageous as a part of PA interventions.ObjectiveThis study aimed at investigating the attitudes and preferences of adolescents aged 16-18 years toward various PA app features and components that could (1) make the app more attractive for them and consequently (2) increase their interest and engagement with the app.MethodsTwo separate focus group discussions were conducted in 2 groups of adolescents (n=4 each) aged 16-18 years. Focus groups were carried out online via video conference. The discussions were conducted using a semistructured interview. Participants (n=8; 4 males and 4 females) had a mean age of 17.25 years (SD 0.82 years). Transcripts were analyzed following the approach by Krueger and Casey, that is, categorizing participants’ answers and comments according to the questions and themes from the focus group schedule.ResultsFeatures, such as “goal setting and planning,” “coaching and training programs,” “activity tracking,” “feedback,” and “location tracking” were appraised as attractive, motivating, and interesting. An “automatic activity recognition” feature was perceived as useful only under the condition that its precision was high. The “reminders” component was also deemed as useful only if a range of conditions was fulfilled (timeliness, opportunity for customization, etc). The features “mood and sleep tracking,” “sharing workout results via social networks,” “digital avatar and coach,” and “rewards” were generally perceived negatively and considered as useless and not motivating. In general, participants preferred features with an easy-to-navigate interface and a clear, simplistic, and straightforward layout with a modern design. Customization and personalization qualities were highly appreciated throughout an app, together with data precision.ConclusionsThis study contributes to the understanding of the features and components preferred by adolescents in apps promoting PA. Such apps should provide users with precise data, and have a simplistic modern design and a straightforward easy-to-use interface. Apps should be personalized and customizable. Desired features to be included in an app are goal setting and planning, feedback, coaching and training programs, and activity tracking. The features should involve high levels of data precision and timely delivery while taking into consideration the real-life context.

Automatic cattle activity recognition on grazing systems

The use of collars, pedometers or activity tags is expensive to record cattle's behavior in short periods (e.g. 24h). Under this particular situation, the development of low-cost and easy-to-use technologies is relevant. Similar to smartphone apps for human activity recognition, which analyzes data from embedded triaxial accelerometer sensors, we develop an Android app to record activity in cattle. Four main steps were followed: a) data acquisition for model training, b) model training, c) app deploy, and d) app utilization. For data acquisition, we developed a system in which three components were used: two smartphones and a Google Firebase account for data storage. For model training, the generated database was used to train a recurrent neural network. The performance of training was assessed by the confusion matrix. For all actual activities, the trained model provided a high prediction (> 96 %). The trained model was used to deploy an Android app by using the TensorFlow API. Finally, three cell phones (LG gm730) were used to test the app and record the activity of six Holstein cows (3 lactating and 3 non-lactating). Direct and non-systematic observations of the animals were made to contrast the activities recorded by the device. Our results show consistency between the direct observations and the activity recorded by our Android app.

Wearable Sensor-Based Human Activity Recognition in the Smart Healthcare System.

Human activity recognition (HAR) has been of interest in recent years due to the growing demands in many areas. Applications of HAR include healthcare systems to monitor activities of daily living (ADL) (primarily due to the rapidly growing population of the elderly), security environments for automatic recognition of abnormal activities to notify the relevant authorities, and improve human interaction with the computer. HAR research can be classified according to the data acquisition tools (sensors or cameras), methods (handcrafted methods or deep learning methods), and the complexity of the activity. In the healthcare system, HAR based on wearable sensors is a new technology that consists of three essential parts worth examining: the location of the wearable sensor, data preprocessing (feature calculation, extraction, and selection), and the recognition methods. This survey aims to examine all aspects of HAR based on wearable sensors, thus analyzing the applications, challenges, datasets, approaches, and components. It also provides coherent categorizations, purposeful comparisons, and systematic architecture. Then, this paper performs qualitative evaluations by criteria considered in this system on the approaches and makes available comprehensive reviews of the HAR system. Therefore, this survey is more extensive and coherent than recent surveys in this field.

Surgical Process Modeling for Open Spinal Surgeries.

Modern operating rooms are becoming increasingly advanced thanks to the emerging medical technologies and cutting-edge surgical techniques. Current surgeries are transitioning into complex processes that involve information and actions from multiple resources. When designing context-aware medical technologies for a given intervention, it is of utmost importance to have a deep understanding of the underlying surgical process. This is essential to develop technologies that can correctly address the clinical needs and can adapt to the existing workflow. Surgical Process Modeling (SPM) is a relatively recent discipline that focuses on achieving a profound understanding of the surgical workflow and providing a model that explains the elements of a given surgery as well as their sequence and hierarchy, both in quantitative and qualitative manner. To date, a significant body of work has been dedicated to the development of comprehensive SPMs for minimally invasive baroscopic and endoscopic surgeries, while such models are missing for open spinal surgeries. In this paper, we provide SPMs common open spinal interventions in orthopedics. Direct video observations of surgeries conducted in our institution were used to derive temporal and transitional information about the surgical activities. This information was later used to develop detailed SPMs that modeled different primary surgical steps and highlighted the frequency of transitions between the surgical activities made within each step. Given the recent emersion of advanced techniques that are tailored to open spinal surgeries (e.g., artificial intelligence methods for intraoperative guidance and navigation), we believe that the SPMs provided in this study can serve as the basis for further advancement of next-generation algorithms dedicated to open spinal interventions that require a profound understanding of the surgical workflow (e.g., automatic surgical activity recognition and surgical skill evaluation). Furthermore, the models provided in this study can potentially benefit the clinical community through standardization of the surgery, which is essential for surgical training.

Architecture for Automatic Recognition of Group Activities Using Local Motions and Context

Currently, the ability to automatically detect human behavior in image sequences is one of the most important challenges in the area of computer vision. Within this broad field of knowledge, the recognition of activities of people groups in public areas is receiving special attention due to its importance in many aspects including safety and security. This paper proposes a generic computer vision architecture with the ability to learn and recognize different group activities using mainly the local group’s movements. Specifically, a multi-stream deep learning architecture is proposed whose two main streams correspond to a representation based on a descriptor capable of representing the trajectory information of a sequence of images as a collection of local movements that occur in specific regions of the scene. Additional information (e.g. location, time, etc.) to strengthen the classification of activities by including it as additional streams. The proposed architecture is capable of classifying in a robust way different activities of a group as well to deal with the one-class problems. Moreover, the use of a simple descriptor that transforms a sequence of color images into a sequence of two-image streams can reduce the curse of dimensionality using a deep learning approach. The generic deep learning architecture has been evaluated with different datasets outperforming the state-of-the-art approaches providing an efficient architecture for single and multi-class classification problems.