Behavior Dataset Research Articles

Student behavior recognition for interaction detection in the classroom environment

With the development of multimedia technologies, surveillance videos and other multimedia data have received widespread attention in several fields. Surveillance videos can monitor students' learning statuses in real time. However, the current action recognition methods for teaching have limitations. First, the ethical privacy of AI and education makes public datasets on student behavior scarce. Therefore, based on the summarization of seven typical student behaviors in the classroom, course videos were obtained from the smart classroom to generate a dataset of student behavior. Compared with existing student behavior recognition datasets, the proposed dataset is distinguished by cluttered backgrounds, crowded scenes, and occlusions. Second, relational reasoning using existing methods is not ideal for distinguishing between students' body parts and small objects in a cluttered background; the interactive utilization rate of different relational features is low, and it cannot take advantage of the complementarity of different relational features, resulting in poor performance of interaction action recognition. Therefore, the attention-based relational reasoning module strengthens the interactive representation between small objects and human body parts. At the same time, considering that there is a certain complementary relationship between relational features, this study constructs a relational feature fusion module which models a human-to-human interaction relationship built upon supporting human's body part and surrounding context. Finally, the reconstructed features and human-appearance features were fused to achieve accurate interactive action recognition. Through an experimental comparison between the proposed and current mainstream algorithms on the generated student behavior dataset, it was verified that the proposed model achieves state-of-the-art performance in action recognition.

A deep learning model to forecast cattle heat stress

The accurate forecasting of feedlot cattle heat stress is pivotal to improving animal welfare and reducing the economic losses associated with heat events. This study investigated how autonomously recorded cattle behaviour datasets can be leveraged to propose new approaches to cattle heat stress forecasting. Behaviour data acquired by accelerometer sensors in two experiments using mixed breed feedlot cattle were utilised, with duration of heavy breathing used to quantify heat stress. Two time-forecasting methods were proposed for predicting the average panting duration of a herd of cattle 24-hours into the future using meteorological data, animal characteristics and treatment factors as input. The first method utilised a simple deep learning framework based on Long-Short Term Memory networks, while the second applied traditional statistical methods to HLI, AHL and animal characteristic data. An optimisation algorithm was used to select which meteorological factors and how much historic data should be considered by the deep learning model. The experiments conducted emphasise that the deep learning approach was able to effectively model the lagged effect of weather on cattle heat stress. Consequently, it was found to be more accurate than traditional statistical methods and climate indices at forecasting the cattle heat response recorded in this study. A final experiment investigated the potential for using these time forecasting models to provide decision support to producers. This preliminary study thus emphasises that autonomously derived behaviour datasets and deep learning have the potential to improve animal welfare and productivity.

MSTA-SlowFast: A Student Behavior Detector for Classroom Environments.

Detecting students' classroom behaviors from instructional videos is important for instructional assessment, analyzing students' learning status, and improving teaching quality. To achieve effective detection of student classroom behavior based on videos, this paper proposes a classroom behavior detection model based on the improved SlowFast. First, a Multi-scale Spatial-Temporal Attention (MSTA) module is added to SlowFast to improve the ability of the model to extract multi-scale spatial and temporal information in the feature maps. Second, Efficient Temporal Attention (ETA) is introduced to make the model more focused on the salient features of the behavior in the temporal domain. Finally, a spatio-temporal-oriented student classroom behavior dataset is constructed. The experimental results show that, compared with SlowFast, our proposed MSTA-SlowFast has a better detection performance with mean average precision (mAP) improvement of 5.63% on the self-made classroom behavior detection dataset.

RETRACTED ARTICLE: Simulation of video image detection in leisure sports tourism industry based on convolutional neural network

Due to the continuous development of information technology, emerging technologies such as 5G networks, Turing devices and deep learning are widely used in people's lives. At present, the sports and tourism industry has become a pillar industry supporting the national economy. Sports events have attracted many people's attention, and people are willing to devote themselves to sports activities. Participants and visitors are also increasing day by day. The article also conducts research on the rapid detection of human video behavior, obtains more representative features for this demand. This research is of great help to the development of the industry. This allows the network to manipulate the input video behavior at different scales and extract deeper video information. Therefore, this paper proposes another 3D convolutional neural network model that combines a multi-level pyramid network with an attention mechanism. The visual attributes in the video behavior data set are explicitly learned to refine the classification of similar features. This paper also proposes an integrated neural network based on visual attribute enhancement. It also shows good adaptability, and can complete video data processing based on time series. The extraction of time and space functions is realized by the two-stream neural network method, the two networks learn the corresponding pixel ratios, and it is not easy to lose the clues of the action recognition. This paper also proposes a spatiotemporal convolutional neural network video for human body recognition algorithm, which is applied to the leisure tourism industry according to the specific analysis of the above method, and promotes the development of the industry.

Driver Predicting Behavior Based on Accelerometer and Gyroscope Sensors with K-Means Algorithm Method

The transportation with an increasing population level in Indonesia will also affect the smoothness of traffic. However, with the existence of transportation that is owned by someone, it is often misused in driving. Aggressive driving behavior is a major factor in traffic accidents. As reported AAA Foundation for Traffic Safety, 106,727 dangerous accidents – 55.7 percent of the over past four year period involved drivers engaging in one or more acts of aggressive driving. Therefore, to predict driver behavior, the K-Means algorithm used the Orange Application. Using the driver behavior dataset that has been recorded with Accelerometer and Gyroscope Sensor-based applications, the results obtained where the k value is the number of cluster 1 there are 814 items, cluster 2 has 997 items, cluster 3 has 1273 items. From this test, it was found that proportion of predicted driver behavior with Aggressive qualifications was 96.4%, Normal was 92.2% and from driver behavior with Low qualifications was 74.3%. The accuracy rate of research using the K-Means algorithm to determine predictions is 0.861 or 86.1%. The results of predictions can help prevent accidents or other risks that will occur in the future.

Grazing Sheep Behaviour Recognition Based on Improved YOLOV5

Fundamental sheep behaviours, for instance, walking, standing, and lying, can be closely associated with their physiological health. However, monitoring sheep in grazing land is complex as limited range, varied weather, and diverse outdoor lighting conditions, with the need to accurately recognise sheep behaviour in free range situations, are critical problems that must be addressed. This study proposes an enhanced sheep behaviour recognition algorithm based on the You Only Look Once Version 5 (YOLOV5) model. The algorithm investigates the effect of different shooting methodologies on sheep behaviour recognition and the model’s generalisation ability under different environmental conditions and, at the same time, provides an overview of the design for the real-time recognition system. The initial stage of the research involves the construction of sheep behaviour datasets using two shooting methods. Subsequently, the YOLOV5 model was executed, resulting in better performance on the corresponding datasets, with an average accuracy of over 90% for the three classifications. Next, cross-validation was employed to verify the model’s generalisation ability, and the results indicated the handheld camera-trained model had better generalisation ability. Furthermore, the enhanced YOLOV5 model with the addition of an attention mechanism module before feature extraction results displayed a mAP@0.5 of 91.8% which represented an increase of 1.7%. Lastly, a cloud-based structure was proposed with the Real-Time Messaging Protocol (RTMP) to push the video stream for real-time behaviour recognition to apply the model in a practical situation. Conclusively, this study proposes an improved YOLOV5 algorithm for sheep behaviour recognition in pasture scenarios. The model can effectively detect sheep’s daily behaviour for precision livestock management, promoting modern husbandry development.

DeepWild: Application of the pose estimation tool DeepLabCut for behaviour tracking in wild chimpanzees and bonobos.

Studying animal behaviour allows us to understand how different species and individuals navigate their physical and social worlds. Video coding of behaviour is considered a gold standard: allowing researchers to extract rich nuanced behavioural datasets, validate their reliability, and for research to be replicated. However, in practice, videos are only useful if data can be efficiently extracted. Manually locating relevant footage in 10,000 s of hours is extremely time-consuming, as is the manual coding of animal behaviour, which requires extensive training to achieve reliability. Machine learning approaches are used to automate the recognition of patterns within data, considerably reducing the time taken to extract data and improving reliability. However, tracking visual information to recognise nuanced behaviour is a challenging problem and, to date, the tracking and pose-estimation tools used to detect behaviour are typically applied where the visual environment is highly controlled. Animal behaviour researchers are interested in applying these tools to the study of wild animals, but it is not clear to what extent doing so is currently possible, or which tools are most suited to particular problems. To address this gap in knowledge, we describe the new tools available in this rapidly evolving landscape, suggest guidance for tool selection, provide a worked demonstration of the use of machine learning to track movement in video data of wild apes, and make our base models available for use. We use a pose-estimation tool, DeepLabCut, to demonstrate successful training of two pilot models of an extremely challenging pose estimate and tracking problem: multi-animal wild forest-living chimpanzees and bonobos across behavioural contexts from hand-held video footage. With DeepWild we show that, without requiring specific expertise in machine learning, pose estimation and movement tracking of free-living wild primates in visually complex environments is an attainable goal for behavioural researchers.

Learning signatures of decision making from many individuals playing the same game.

Human behavior is incredibly complex and the factors that drive decision making-from instinct, to strategy, to biases between individuals-often vary over multiple timescales. In this paper, we design a predictive framework that learns representations to encode an individual's 'behavioral style', i.e. long-term behavioral trends, while simultaneously predicting future actions and choices. The model explicitly separates representations into three latent spaces: the recent past space, the short-term space, and the long-term space where we hope to capture individual differences. To simultaneously extract both global and local variables from complex human behavior, our method combines a multi-scale temporal convolutional network with latent prediction tasks, where we encourage embeddings across the entire sequence, as well as subsets of the sequence, to be mapped to similar points in the latent space. We develop and apply our method to a large-scale behavioral dataset from 1,000 humans playing a 3-armed bandit task, and analyze what our model's resulting embeddings reveal about the human decision making process. In addition to predicting future choices, we show that our model can learn rich representations of human behavior over multiple timescales and provide signatures of differences in individuals.

Characterization of External Female Genitalia in Five Triatoma Laporte Species of South America (Hemiptera: Reduviidae: Triatominae).

Currently, there are 158 valid species of triatomines, all of which are potential vectors of Trypanosoma cruzi, the etiological agent of Chagas disease. The correct taxonomic identification of triatomines is essential since each species hos a different epidemiological importance. The aim of the study is to compare five species of South American Triatoma. Here we present a comparative study of terminal abdominal segments in females by scanning electron microscopy (SEM) of the species Triatoma delpontei, T. jurbergi, T. infestans var. melanosoma, T. platensis, and T. vandae. The results showed diagnostic characters for the studied species. The dorsal view featured more valuable characters, with seven informative characters. Similarities were observed among T. delpontei, T. infestans var. melanosoma, and T. platensis, and between T. jurbergi and T. vandae, correlating with previous studies. Thus, female genital characters proved to be reliable and useful in the diagnosis of the Triatoma species studied here; additional studies, along with other sets of behavioral, morphological, and molecular data, helped to reinforce the hypotheses found here.

An Improved Genetic-XGBoost Classifier for Customer Consumption Behavior Prediction

AbstractIn an increasingly competitive market, predicting the customer’s consumption behavior has a vital role in customer relationship management. In this study, a new classifier for customer consumption behavior prediction is proposed. The proposed methods are as follows: (i) A feature selection method based on least absolute shrinkage and selection operator (Lasso) and Principal Component Analysis (PCA), to achieve efficient feature selection and eliminate correlations between variables. (ii) An improved genetic-eXtreme Gradient Boosting (XGBoost) for customer consumption behavior prediction, to improve the accuracy of prediction. Furthermore, the global search ability and flexibility of the genetic mechanism are used to optimize the XGBoost parameters, which avoids inaccurate parameter settings by manual experience. The adaptive crossover and mutation probabilities are designed to prevent the population from falling into the local extremum. Moreover, the grape-customer consumption behavior dataset is employed to compare the six Lasso-based models from the original, normalized and standardized data sources with the Isometric Mapping, Locally Linear Embedding, Multidimensional Scaling, PCA and Kernel Principal Component Analysis methods. The improved genetic-XGBoost is compared with several well-known parameter optimization algorithms and state-of-the-art classification approaches. Furthermore, experiments are conducted on the University of California Irvine datasets to verify the improved genetic-XGBoost algorithm. All results show that the proposed methods outperform the existing ones. The prediction results provide the decision-making basis for enterprises to formulate better marketing strategies.

Student Cheating Detection in Higher Education by Implementing Machine Learning and LSTM Techniques.

Both paper-based and computerized exams have a high level of cheating. It is, therefore, desirable to be able to detect cheating accurately. Keeping the academic integrity of student evaluations intact is one of the biggest issues in online education. There is a substantial possibility of academic dishonesty during final exams since teachers are not directly monitoring students. We suggest a novel method in this study for identifying possible exam-cheating incidents using Machine Learning (ML) approaches. The 7WiseUp behavior dataset compiles data from surveys, sensor data, and institutional records to improve student well-being and academic performance. It offers information on academic achievement, student attendance, and behavior in general. In order to build models for predicting academic accomplishment, identifying at-risk students, and detecting problematic behavior, the dataset is designed for use in research on student behavior and performance. Our model approach surpassed all prior three-reference efforts with an accuracy of 90% and used a long short-term memory (LSTM) technique with a dropout layer, dense layers, and an optimizer called Adam. Implementing a more intricate and optimized architecture and hyperparameters is credited with increased accuracy. In addition, the increased accuracy could have been caused by how we cleaned and prepared our data. More investigation and analysis are required to determine the precise elements that led to our model's superior performance.

Neural Network-Based Method for Early Diagnosis of Autism Spectral Disorder Head-Banging Behavior from Recorded Videos

Autism spectrum disorder (ASD) is a mental developmental disorder associated with social and communicational defects and Stereotypical Motor Movements (SMM). SMM is a set of repetitive motor activities associated with several mental developmental disorders like Autism. SMM has several forms like arm flapping, head banging, ear covering, and spinning with various degrees of severity that might lead to self-injury in severe cases. Developing a computer-vision-based technology to detect noticeable SMM behaviors can help in the early diagnosis of autism. In this paper, a computer vision-based neural network model was proposed to detect and recognize repetitive motor behaviors. The proposed model went through three main stages: First, data preparation. Second, human body features extraction using deep learning pose estimation and the skeleton representation model, and finally, multiclass classification to distinguish between several classes of headbanging. The proposed solution was evaluated using the Self Stimulatory Behavior Dataset (SSBD) which is a public dataset of three classes of repetitive motor behaviors associated with autism. We also collected a set of 40 videos of autistic children exhibiting headbanging from public domains like YouTube. In addition to that, we captured 25 videos of typically developing subjects mimicking headbanging. The collected and the videoed videos were used to evaluate the proposed model. This work proves the applicability of diagnosing mental developmental syndrome symptoms using vision-based techniques in cooperation with neural networks. The produced results prove that the used techniques can operate well in real-world challenging applications. The proposed model achieved 85.5% accuracy on SSBD and 93% on the collected and recorded videos.

Adaptive tuning of human learning and choice variability to unexpected uncertainty.

Human value-based decisions are notably variable under uncertainty. This variability is known to arise from two distinct sources: variable choices aimed at exploring available options and imprecise learning of option values due to limited cognitive resources. However, whether these two sources of decision variability are tuned to their specific costs and benefits remains unclear. To address this question, we compared the effects of expected and unexpected uncertainty on decision-making in the same reinforcement learning task. Across two large behavioral datasets, we found that humans choose more variably between options but simultaneously learn less imprecisely their values in response to unexpected uncertainty. Using simulations of learning agents, we demonstrate that these opposite adjustments reflect adaptive tuning of exploration and learning precision to the structure of uncertainty. Together, these findings indicate that humans regulate not only how much they explore uncertain options but also how precisely they learn the values of these options.

Video‐based real‐time assessment and diagnosis of autism spectrum disorder using deep neural networks

AbstractHuman action recognition (HAR) in untrimmed videos can make insightful predictions of human behaviour. Previous work on HAR‐included models trained on spatial and temporal annotations and could classify limited actions from trimmed videos. These methods reported limitations such as (1) performance degradation due to the lack of precision temporal regions proposal and (2) poor adaptability of the models in the clinical domain because of unrelated actions of interest. We propose an innovative method that could analyse untrimmed behavioural videos to recommend actions of interest leading to diagnostic and functional assessments for children with Autism Spectrum Disorder (ASD). Our method entails end‐to‐end behaviour action recognition (BAR) pipeline, including child detection, temporal action localization, and actions of interest identification and classification. The model trained on the data of 400 ASD children and 125 with other developmental delays (ODD) accurately identified ASD, ODD, and Neurotypical children with 79.7%, 77.2%, and 80.8% accuracy, respectively. The model's performance on an independent benchmark Self‐Stimulatory Behaviour Dataset (SSBD) reported top‐1 accuracy of 78.57% for combined localization with action recognition, significantly higher than the earlier reported outcomes.

Mars’ ACE: Using AI and Behavioral Data in Ad Testing with High Correlating Sales

Laurent Larguinat describes Mars’ ACE, a proprietary AI tool which analyzes large sets of consumer behavioral data to predict the sales effectiveness of the company’s adverts far more effectively than traditional methods.

Detection of non‐suicidal self‐injury based on spatiotemporal features of indoor activities

AbstractNon‐suicide self‐injury (NSSI) can be dangerous and difficult for guardians or caregivers to detect in time. NSSI refers to when people hurt themselves even though they have no wish to cause critical or long‐lasting hurt. To timely identify and effectively prevent NSSI in order to reduce the suicide rates of patients with a potential suicide risk, the detection of NSSI based on the spatiotemporal features of indoor activities is proposed. Firstly, an NSSI behaviour dataset is provided, and it includes four categories that can be used for scientific research on NSSI evaluation. Secondly, an NSSI detection algorithm based on the spatiotemporal features of indoor activities (NssiDetection) is proposed. NssiDetection calculates the human bounding box by using an object detection model and employs a behaviour detection model to extract the temporal and spatial features of NSSI behaviour. Thirdly, the optimal combination schemes of NssiDetection is investigated by checking its performance with different behaviour detection methods and training strategies. Lastly, a case study is performed by implementing an NSSI behaviour detection prototype system. The prototype system has a recognition accuracy of 84.18% for NSSI actions with new backgrounds, persons, or camera angles.

THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior.

Understanding object representations requires a broad, comprehensive sampling of the objects in our visual world with dense measurements of brain activity and behavior. Here, we present THINGS-data, a multimodal collection of large-scale neuroimaging and behavioral datasets in humans, comprising densely sampled functional MRI and magnetoencephalographic recordings, as well as 4.70 million similarity judgments in response to thousands of photographic images for up to 1,854 object concepts. THINGS-data is unique in its breadth of richly annotated objects, allowing for testing countless hypotheses at scale while assessing the reproducibility of previous findings. Beyond the unique insights promised by each individual dataset, the multimodality of THINGS-data allows combining datasets for a much broader view into object processing than previously possible. Our analyses demonstrate the high quality of the datasets and provide five examples of hypothesis-driven and data-driven applications. THINGS-data constitutes the core public release of the THINGS initiative (https://things-initiative.org) for bridging the gap between disciplines and the advancement of cognitive neuroscience.

Social Isolation Affects the Mimicry Response in the Use of Smartphones

Humans are social animals that rely on different ways to interact with each other. The COVID-19 pandemic strongly changed our communication strategies. Because of the importance of direct contact for our species, we predict that immediately after the forced social isolation, people were more prone to engage in direct rather than in virtual interactions, thus showing a lower mimicry response in the use of smartphones. In a non-longitudinal study, we collected behavioral data under naturalistic contexts and directly compared the data of the mimicry response gathered immediately following the Italian lockdown (May–September 2020) with those gathered one year later (May–October 2021). Contrary to our expectations, the mimicry response in the use of smartphones was higher immediately after the lockdown than a year later. Probably the large use of these devices during the lockdown translated into a greater sensitivity to be affected by others’ smartphone manipulation. Indeed, social isolation modified, at least in the short term, the ways we interact with others by making us more prone to engage in “virtual” social interactions. The bright side of the coin unveiled by our findings is that the effect seems to diminish over time. The large behavioral dataset analyzed here (1,608 events; 248 people) also revealed that the mimicry response in the use of smartphones was higher between familiar subjects than between strangers. In this view, mimicry in manipulating smartphones can be considered an example of joint action that fosters behavioral synchrony between individuals that, in the long-term, can translate into the formation of social bonding.

Intention affects fairness processing: Evidence from behavior and representational similarity analysis of event-related potential signals.

In an ultimatum game, the responder must decide between pursuing self-interest and insisting on fairness, and these choices are affected by the intentions of the proposer. However, the time course of this social decision-making process is unclear. Representational similarity analysis (RSA) is a useful technique for linking brain activity with rich behavioral data sets. In this study, electroencephalography (EEG) was used to measure the time course of neural responses to proposed allocation schemes with different intentions. Twenty-eight participants played an ultimatum game as responders. They had to choose between accepting and rejecting the fair or unfair money allocation schemes of proposers. The schemes were offered based on the proposer's selfish intention (monetary gain), altruistic intention (donation to charity), or ambiguous intention (unknown to the responder). We used a spatiotemporal RSA and inter-subject RSA (IS-RSA) to explore the connections between event-related potentials (ERPs) after offer presentation and intention presentation with four types of behavioral data (acceptance, response time, fairness ratings, and pleasantness ratings). The spatiotemporal RSA results revealed that only response time variation was linked with the difference in ERPs at 432-592 ms after offer presentation on the posterior parietal and prefrontal regions. Meanwhile, the IS-RSA results found a significant association between inter-individual differences in response time and differences in ERP activity at 596-812 ms after the presentation of ambiguous intention, particularly in the prefrontal region. This study expands the intention-based reciprocal model to the third-party context and demonstrates that brain activity can represent response time differences in social decision-making.

FMRI functional connectivity is a better predictor of general intelligence than cortical morphometric features and ICA parcellation order affects predictive performance

Intelligence, as a general cognitive ability, shows a substantial inter-subject variation. Because of its impact on our lives, there is great interest in explaining the neural substrates of these differences. We used a large set of neuroimaging and behavioral data from 805 subjects, provided by the Human Connectome Project, and applied predictive models based on elastic-net regression using functional connectivity and brain morphometric data to predict general intelligence values. Additionally, we explored the impact of brain spatial distribution of the input connectivity data in the regression model using two strategies: brain parcellation and individual components. Features derived from functional connectivity were considerably more correlated with general intelligence than cortical thickness and surface area. Considering the regularization terms in this particular prediction problem, the best performances were obtained when the impact of all the independent variables was considered in the regresion, i.e. null LASSO sparsity term. Using different parcellation schemes affected predictive performances, which indicates spatial heterogeneity in the regression. We were able to explain 17,5% of the general intelligence variance, in the best performance reached, with a brain parcellation of 25 independent components; by other hand, using only cortical morphometric features the performance reduced to 1,6% for both cortical thickness and surface area. While no component, in particular, was responsible for predicting a large portion of the variance, the spatial components with the best results comprehend parietal, frontal and occipital regions, in agreement with the Parieto-Frontal Integration Theory (P-FIT).