Action Descriptors Research Articles

Changes in the equine facial repertoire during different orthopedic pain intensities

A number of facial expressions are associated with pain in horses, however, the entire display of facial activities during orthopedic pain have yet to be described. The aim of the present study was to exhaustively map changes in facial activities in eight resting horses during a progression from sound to mild and moderate degree of orthopedic pain, induced by lipopolysaccharides (LPS) administered in the tarsocrural joint. Lameness progression and regression was measured by objective gait analysis during movement, and facial activities were described by EquiFACS in video sequences (n = 348, total length 892.5 min) of the horses obtained when resting in their box stalls. Predictive modeling identified 16 action units and action descriptors, related to ears, eyes, and lower face. Lower lip depressor (AU16), lips part (AU25), half blink (AU47), single ear forward (SEAD101) and single ear rotator (SEAD104) were selected as co-occurring significantly more in horses with pain than in horses without pain. The major change in co-occurring facial activities occurred in the transition from no pain to mild pain. In conclusion, resting horses with induced orthopedic pain showed a dynamic upper and lower facial repertoire and the relationship between level of pain intensity and facial activity appears complex.

Beyond Pattern Variance: Unsupervised 3-D Action Representation Learning With Point Cloud Sequence.

This work pays the first research effort to address unsupervised 3-D action representation learning with point cloud sequence, which is different from existing unsupervised methods that rely on 3-D skeleton information. Our proposition is built on the state-of-the-art 3-D action descriptor 3-D dynamic voxel (3DV) with contrastive learning (CL). The 3DV can compress the point cloud sequence into a compact point cloud of 3-D motion information. Spatiotemporal data augmentations are conducted on it to drive CL. However, we find that existing CL methods (e.g., SimCLR or MoCo v2) often suffer from high pattern variance toward the augmented 3DV samples from the same action instance, that is, the augmented 3DV samples are still of high feature complementarity after CL, while the complementary discriminative clues within them have not been well exploited yet. To address this, a feature augmentation adapted CL (FACL) approach is proposed, which facilitates 3-D action representation via concerning the features from all augmented 3DV samples jointly, in spirit of feature augmentation. FACL runs in a global-local way: one branch learns global feature that involves the discriminative clues from the raw and augmented 3DV samples, and the other focuses on enhancing the discriminative power of local feature learned from each augmented 3DV sample. The global and local features are fused to characterize 3-D action jointly via concatenation. To fit FACL, a series of spatiotemporal data augmentation approaches is also studied on 3DV. Wide-range experiments verify the superiority of our unsupervised learning method for 3-D action feature learning. It outperforms the state-of-the-art skeleton-based counterparts by 6.4% and 3.6% with the cross-setup and cross-subject test settings on NTU RGB + D 120, respectively. The source code is available at https://github.com/tangent-T/FACL.

Residual Neural Networks for Human Action Recognition from RGB-D Videos

Recently, the RGB-D based Human Action Recognition (HAR) has gained significant research attention due to the provision of complimentary information by different data modalities. However, the current models have experienced still unsatisfactory results due to several problems including noises and view point variations between different actions. To sort out these problems, this paper proposes two new action descriptors namely Modified Depth Motion Map (MDMM) and Spherical Redundant Joint Descriptor (SRJD). MDMM eliminates the noises from depth maps and preserves only the action related information. Further SRJD ensures resilience against view point variations and reduces the misclassifications between different actions with similar view properties. Further, to maximize the recognition accuracy, standard deep learning algorithm called as Residual Neural Network (ResNet) is used to train the system through the features extracted from MDMM and SRJD. Simulation experiments prove that the multiple data modalities are better than single data modality. The proposed approach was tested on two public datasets namely NTURGB+D dataset and UTD-MHAD dataset. The testing results declare that the proposed approach is superior to the earlier HAR methods. On an average, the proposed system gained an accuracy of 90.0442% and 92.3850% at Cross-subject and Cross-view validations respectively.

The Use of Packaging Descriptors in a Rapidly Growing Market for Capsule Cigarettes: Evidence From Mexico.

The global market for capsule cigarettes has grown rapidly, especially in Latin America. This research examined the sales trends and patterns as well as packaging characteristics of capsule cigarette in Mexico in recent years. National data on cigarette sales in Mexico between October 2018 and September 2021 licensed from NielsenIQ were supplemented with cigarette pack pictures and Internet searches to determine whether characterizing flavor, concept flavor, and action descriptors were used. Market share in sales value and volume was calculated by capsule and flavor status, descriptors, price tier, manufacturer, pack size, and cigarette length for the whole time period and for each month. Number of unique brand variants was also examined. Capsule cigarette sales totaled US$4.29 billion or 30.99 billion sticks during the assessed period, comprising 37.7% of the cigarette market share in sales value or 35.5% in sales volume. Over half of capsule cigarettes sold during this time period had concept flavor descriptors, 17.8% contained characterizing flavor descriptors, and 15.1% contained action descriptors that were suggestive of the interactive aspects of capsules. The monthly market share of capsule cigarette sales volume among all cigarette sales in Mexico increased steadily from 33.3% in October 2018 to 37.3% in September 2021. The expanding capsule cigarette market in Mexico is concerning, given the product's associated misconceptions of reduced harm and greater appeal among youth. Findings underline the need for enhanced regulations to address the public health threat posed by capsule cigarettes. Capsule cigarettes have a substantial market share in Mexico, with increasing popularity. Descriptors and other product characteristics such as cigarette length might be used to increase the appeal and target particular populations. Policy makers should consider banning capsules and flavors in cigarettes including descriptors or other indication in product presentation that could be perceived as denoting a taste, aroma, or sensation, and making the appearance and design of tobacco products more commensurate with the harm they cause by adopting plain and standardized packaging.

Spatial and temporal information fusion for human action recognition via Center Boundary Balancing Multimodal Classifier

This paper proposes a novel multimodal data classifier named Center Boundary Balancing Multimodal Classifier (CBBMC) to fuse and classify the spatial and temporal descriptors for recognizing human actions from depth video sequences. CBBMC is a composite algorithm integrating feature fusion and feature classification, in which Center Boundary Balancing Projection (CBBP) is used to balance the center and boundary information of feature class spaces. In order to solve the problem of multimodal information redundancy and isolation, two feature selection and fusion schemes of CBBMC based on embedded feature selection are presented. Moreover, two new action descriptors called Gaussian Pyramid Depth Motion Images (GP-DMI) and Depth Temporal Maps (DTM) are introduced to capture the multi-scale spatial and fine-grained temporal information of human activities. Finally, we present an effective spatial and temporal information fusion framework based on CBBMC for human action recognition. In order to evaluate the performance of the proposed approach, extensive experiments are conducted. The proposed method achieved impressive results on four benchmark datasets, namely MSR Action3D (96.33%), UTD-MHAD (94.41%), DHA (95.65%), and NTU RGB+D (83.31% cross-subject and 87.66% cross-view), even though only the depth modality was used. The experimental results demonstrate the effectiveness of our method.

Machine Vision-Based Human Action Recognition Using Spatio-Temporal Motion Features (STMF) with Difference Intensity Distance Group Pattern (DIDGP)

In recent years, human action recognition is modeled as a spatial-temporal video volume. Such aspects have recently expanded greatly due to their explosively evolving real-world uses, such as visual surveillance, autonomous driving, and entertainment. Specifically, the spatio-temporal interest points (STIPs) approach has been widely and efficiently used in action representation for recognition. In this work, a novel approach based on the STIPs is proposed for action descriptors i.e., Two Dimensional-Difference Intensity Distance Group Pattern (2D-DIDGP) and Three Dimensional-Difference Intensity Distance Group Pattern (3D-DIDGP) for representing and recognizing the human actions in video sequences. Initially, this approach captures the local motion in a video that is invariant to size and shape changes. This approach extends further to build unique and discriminative feature description methods to enhance the action recognition rate. The transformation methods, such as DCT (Discrete cosine transform), DWT (Discrete wavelet transforms), and hybrid DWT+DCT, are utilized. The proposed approach is validated on the UT-Interaction dataset that has been extensively studied by past researchers. Then, the classification methods, such as Support Vector Machines (SVM) and Random Forest (RF) classifiers, are exploited. From the observed results, it is perceived that the proposed descriptors especially the DIDGP based descriptor yield promising results on action recognition. Notably, the 3D-DIDGP outperforms the state-of-the-art algorithm predominantly.

Group Activity Recognition Based on Interaction Contextual Information in Videos Using Machine Learning

This paper is about recognizing multiple person actions occurring in videos, including individual actions, interactions,and group activities. In an environment, multiple people perform group actions such as walking in groupsand talking by facing each other. The model develops by retrieving individual person action from video sequencesby representing interactive contextual features among multiple people. The novelty of the proposed frameworkis the development of interactive action context descriptors (IAC) and classifying group activities using MachineLearning. Each individual person and other nearby people’s relative action score are encoded by IAC in thevideo frame. Individual person action descriptors are important clues for recognition of multiple person activityby developing interaction context. An action retrieval technique was formulated based on KNN for individualaction classification scores. This model also introduces Fully Connected Conditional Random Field (FCCRF) tolearn interaction context information among multiple people. FCCRF regularizes activity categorization by thespatial-temporal model. This paper also presents threshold processing to improve the performance of contextdescriptors. The experimental results compared to state-of-the-art approaches and demonstrated improvement inperformance for group activity recognition.

CalliFACS: The common marmoset Facial Action Coding System.

Facial expressions are subtle cues, central for communication and conveying emotions in mammals. Traditionally, facial expressions have been classified as a whole (e.g. happy, angry, bared-teeth), due to automatic face processing in the human brain, i.e., humans categorise emotions globally, but are not aware of subtle or isolated cues such as an eyebrow raise. Moreover, the same facial configuration (e.g. lip corners pulled backwards exposing teeth) can convey widely different information depending on the species (e.g. humans: happiness; chimpanzees: fear). The Facial Action Coding System (FACS) is considered the gold standard for investigating human facial behaviour and avoids subjective interpretations of meaning by objectively measuring independent movements linked to facial muscles, called Action Units (AUs). Following a similar methodology, we developed the CalliFACS for the common marmoset. First, we determined the facial muscular plan of the common marmoset by examining dissections from the literature. Second, we recorded common marmosets in a variety of contexts (e.g. grooming, feeding, play, human interaction, veterinary procedures), and selected clips from online databases (e.g. YouTube) to identify their facial movements. Individual facial movements were classified according to appearance changes produced by the corresponding underlying musculature. A diverse repertoire of 33 facial movements was identified in the common marmoset (15 Action Units, 15 Action Descriptors and 3 Ear Action Descriptors). Although we observed a reduced range of facial movement when compared to the HumanFACS, the common marmoset’s range of facial movements was larger than predicted according to their socio-ecology and facial morphology, which indicates their importance for social interactions. CalliFACS is a scientific tool to measure facial movements, and thus, allows us to better understand the common marmoset’s expressions and communication. As common marmosets have become increasingly popular laboratory animal models, from neuroscience to cognition, CalliFACS can be used as an important tool to evaluate their welfare, particularly in captivity.

«Encoding loci». Problems and prospects using of FACS in the interpretation of emotions and states in Maya terracotta (1st millennium AD)

The article is devoted to the study of emotions and psychoemotional states represented on Mayan terracotta (1st millennium AD.) The key direction of the analysis was the adaptation of the Facial Action Coding System (FACS) that had not been previously used when working with archaeological material. The study of a representative sample of figurines in the amount of more than 300 units made it possible to determine the Action Units (AU) represented on the faces of the characters in all three floors of the face. The combination of AU on terracotta gave an opportunity to determine the features of almost all basic groups of emotions. Much attention is paid to the characteristics of Action Descriptors (AD) and the system for detecting various wrinkles, which allowed to reflect to some extent the emotional states of the depicted characters. The article discusses in detail the difficulties of FACS adaptation related to the degree of preservation of the material, iconographic features of images, anthropological specifics, static, which do not allow fixing some Action Units, due to their dynamism (blinking, swallowing, etc.). Despite these difficulties, the article describes successful application of FACS and the prospects for work in this area, expressed in the creation of software by definition AU and AD for Maya terracotta. In the future, this will simplify the mechanism for determining emotions and allow extrapolating images to their "historical prototypes".

Action Classification for Partially Occluded Silhouettes by Means of Shape and Action Descriptors

This paper presents an action recognition approach based on shape and action descriptors that is aimed at the classification of physical exercises under partial occlusion. Regular physical activity in adults can be seen as a form of non-communicable diseases prevention, and may be aided by digital solutions that encourages individuals to increase their activity level. The application scenario includes workouts in front of the camera, where either the lower or upper part of the camera’s field of view is occluded. The proposed approach uses various features extracted from sequences of binary silhouettes, namely centroid trajectory, shape descriptors based on the Minimum Bounding Rectangle, action representation based on the Fourier transform and leave-one-out cross-validation for classification. Several experiments combining various parameters and shape features are performed. Despite the presence of occlusion, it was possible to obtain about 90% accuracy for several action classes, with the use of elongation values observed over time and centroid trajectory.

Machine Learning Model for Group Activity Recognition Based on Discriminative Interaction Contextual Relationship

This paper represents the recognition of group activity in public areas, considering personal actions and interactions between people from the field of computer vision. Modeling the interaction relationships between multiple people is essential for recognizing group activity in the video scene. In artificial intelligence applications, identifying group activities based on human interaction is often a challenging task. This paper proposed a model that formulates a group action context (GAC) descriptor. The descriptor was developed by integrating the focal person action descriptor and interaction joint context descriptor of nearby people in the video frame. The model used an efficient optimization principle based on machine learning to learn the discriminative interaction context relations between multiple persons. The proposed novel group action context descriptor is classified by support vector machine (SVM) to recognize group activity. The proposed technique effectiveness is evaluated for group activity recognition by performing experiments on a publicly available collective activity dataset. The proposed approach infers a group action class when multiple persons are together in the video sequence, especially when the interaction between people is confusing. The overall group action recognition model is interrelated with a baseline model to estimate the performance of interaction context information. The experimental result of the proposed group activity recognition model is comparable and outperforms the previous methods.

Effect of transportation and social isolation on facial expressions of healthy horses.

Horses have the ability to generate a remarkable repertoire of facial expressions, some of which have been linked to the affective component of pain. This study describes the facial expressions in healthy horses free of pain before and during transportation and social isolation, which are putatively stressful but ordinary management procedures. Transportation was performed in 28 horses by subjecting them to short-term road transport in a horse trailer. A subgroup (n = 10) of these horses was also subjected to short-term social isolation. During all procedures, a body-mounted, remote-controlled heart rate monitor provided continuous heart rate measurements. The horses’ heads were video-recorded during the interventions. An exhaustive dataset was generated from the selected video clips of all possible facial action units and action descriptors, time of emergency, duration, and frequency according to the Equine Facial Action Coding System (EquiFACS). Heart rate increased during both interventions (p<0.01), confirming that they caused disruption in sympato-vagal balance. Using the current method for ascribing certain action units (AUs) to specific emotional states in humans and a novel data-driven co-occurrence method, the following facial traits were observed during both interventions: eye white increase (p<0.001), nostril dilator (p<0.001), upper eyelid raiser (p<0.001), inner brow raiser (p = 0.042), tongue show (p<0.001). Increases in ‘ear flicker’ (p<0.001) and blink frequency (p<0.001) were also seen. These facial actions were used to train a machine-learning classifier to discriminate between the high-arousal interventions and calm horses, which achieved at most 79% accuracy. Most facial features identified correspond well with previous findings on behaviors of stressed horses, for example flared nostrils, repetitive mouth behaviors, increased eye white, tongue show, and ear movements. Several features identified in this study of pain-free horses, such as dilated nostrils, eye white increase, and inner brow raiser, are used as indicators of pain in some face-based pain assessment tools. In order to increase performance parameters in pain assessment tools, the relations between facial expressions of stress and pain should be studied further.

Theoretical studies of Thiazolyl-Pyrazoline derivatives as promising drugs against malaria by QSAR modelling combined with molecular docking and molecular dynamics simulation

ABSTRACT We investigate thiazolyl-pyrazoline derivatives as promising drugs for the anti-malarial. Protein kinase G is a primary target for treating malaria due to its essential role in Plasmodium falciparum life cycle. In this present study, several computational approaches such as QSAR modelling, molecular docking, and all-atom MD simulation are performed to screen 36 drug candidates against malaria. From QSAR analysis, three potent drugs are selected based on the strong correlation between the inhibitory action, i.e. pEC50 and various descriptors. Further, those selected drugs are used as ligand molecules for molecular docking. We predict three complexes of models 1, 2, and 3 bind to the catalytic site of protein kinase G, suggesting those ligands may become potent inhibitors for Plasmodium falciparum. To validate the structural stability of those complexes, the parameters of RMSD, RMSF, and Rg are calculated from MD simulation. All models are stable along simulation since no significant fluctuations are observed in those validity parameters. Moreover, the binding energy is estimated at each model using MM-GBSA method and model 2 becomes the most stable structure. Finally, three ligands are assumed to have potential as inhibitors and the ligand of model 2 may become the most promising drug against malaria.

Action recognition using interrelationships of 3D joints and frames based on angle sine relation and distance features using interrelationships

Human action recognition is still an uncertain computer vision problem, which could be solved by a robust action descriptor. As a solution, we proposed an action recognition descriptor using only the 3D skeleton joint’s points to perform this unsettle task. Joint’s point interrelationships and frame-frame interrelationships are presented, which is a solution backbone to achieve human action recognition. Here, many joints are related to each other, and frames depend on different frames while performing any action sequence. Joints point spatial information calculates using angle, joint’s sine relation, and distance features, whereas joints point temporal information estimates from frame-frame relations. Experiments are performed over four publicly available databases, i.e., MSR Daily Activity 3D Dataset, UTD Multimodal Human Action Dataset, KARD- Kinect Activity Recognition Dataset, and SBU Kinect Interaction Dataset, and proved that proposed descriptor outperforms as a comparison to state-of-the-art approaches on entire four datasets. Angle, Sine relation, and Distance features are extracted using interrelationships of joints and frames (ASD-R). It is all achieved due to accurately detecting spatial and temporal information of the Joint’s points. Moreover, the Support Vector Machine classifier supports the proposed descriptor to identify the right classification precisely.

Extending the MaqFACS to measure facial movement in Japanese macaques (Macaca fuscata) reveals a wide repertoire potential.

Facial expressions are complex and subtle signals, central for communication and emotion in social mammals. Traditionally, facial expressions have been classified as a whole, disregarding small but relevant differences in displays. Even with the same morphological configuration different information can be conveyed depending on the species. Due to a hardwired processing of faces in the human brain, humans are quick to attribute emotion, but have difficulty in registering facial movement units. The well-known human FACS (Facial Action Coding System) is the gold standard for objectively measuring facial expressions, and can be adapted through anatomical investigation and functional homologies for cross-species systematic comparisons. Here we aimed at developing a FACS for Japanese macaques, following established FACS methodology: first, we considered the species’ muscular facial plan; second, we ascertained functional homologies with other primate species; and finally, we categorised each independent facial movement into Action Units (AUs). Due to similarities in the rhesus and Japanese macaques’ facial musculature, the MaqFACS (previously developed for rhesus macaques) was used as a basis to extend the FACS tool to Japanese macaques, while highlighting the morphological and appearance changes differences between the two species. We documented 19 AUs, 15 Action Descriptors (ADs) and 3 Ear Action Units (EAUs) in Japanese macaques, with all movements of MaqFACS found in Japanese macaques. New movements were also observed, indicating a slightly larger repertoire than in rhesus or Barbary macaques. Our work reported here of the MaqFACS extension for Japanese macaques, when used together with the MaqFACS, comprises a valuable objective tool for the systematic and standardised analysis of facial expressions in Japanese macaques. The MaqFACS extension for Japanese macaques will now allow the investigation of the evolution of communication and emotion in primates, as well as contribute to improving the welfare of individuals, particularly in captivity and laboratory settings.

Human Action Recognition Using Time Delay Input Radial Basis Function Networks

This paper presents a fast, vision-based method for the problem of human action representation and recognition. The first problem is addressed by constructing an action descriptor from spatiotemporal data of action silhouettes based on appearance and motion features. For action classification, a new Radial Basis Function Network (RBF), called Time Delay Input Radial Basis Function Network (TDIRBF) is proposed by introducing time delay units to the RBF in a novel approach. A TDIRBF offers a few desirable features such as an easier learning process and more flexibility. The representational power and speed of the proposed method were explored using a publicly available dataset. Based on experimental results, implemented in MATLAB and on standard PCs, the average time for constructing a feature vector for a high-resolution video was just about 20 ms/frame (or 50 fps) and the classifier speed was above 15 fps. Furthermore, the proposed approach demonstrated good performance in terms of both execution time and overall performance (a new performance measure that combines accuracy and speed into one metric).

Human Action Recognition Based on Bag-of-Words

Human action recognition has gained popularity because of its wide applicability, such as in patient monitoring systems, surveillance systems, and a wide diversity of systems that contain interactions between people and electrical devices, including human computer interfaces. The proposed method includes sequential stages of object segmentation, feature extraction, action detection and then action recognition. Effective results of human actions using different features of unconstrained videos was a challenging task due to camera motion, cluttered background, occlusions, complexity of human movements, and variety of same actions performed by distinct subjects. Thus, the proposed method overcomes such problems by using the fusion of features concept for the development of a powerful human action descriptor. This descriptor is modified to create a visual word vocabulary (or codebook) which yields a Bag-of-Words representation. The True Positive Rate (TPR) and False Positive Rate (FPR) measures gave a true indication about the proposed HAR system. The computed Accuracy (Ar) and the Error (misclassification) Rate (Er) reveal the effectiveness of the system with the used dataset.

CAD: concatenated action descriptor for one and two person(s), using silhouette and silhouette's skeleton

This study introduces an action descriptor that has the ability to perform human action recognition efficiently for one and two person(s). The authors’ proposed descriptor computes information like motion, spatial–temporal, diversion with respect to the centroid, critical point and keypoint detection, whereas the existing approaches lack to address this information efficiently. Action descriptors are developed from signature-based optical flow, signature-based corner points and binary robust invariant scalable keypoints. These action descriptors are applied to silhouette and silhouette's skeleton frames. These aforementioned action descriptors lead to developing the concatenated action descriptor (CAD). In order to develop action descriptors, the reference video frame plays an important role. Weizmann (one person) and both clean and noise versions of SBU Kinect Interaction (two persons) datasets are used for the evaluation of their proposed descriptors. On the other hand, classifications are performed by using support vector machine. Experimental results demonstrate that CAD not only outperforms among the entire proposed descriptors, but also provides better performance as compared to state-of-the-art approaches.

Motion Recognition Algorithm in VR Video Based on Dual Feature Fusion and Adaptive Promotion

VR video recognition in complex environment, a motion recognition algorithm based on two-feature fusion and adaptive enhancement is proposed to solve the problems of inaccurate target position, target drift and recognition error caused by the vulnerability to light change, target rotation and occlusion. First, based on the spatio-temporal context (STC) mechanism, image sequence features are extracted through spatio-temporal context relationship and visual system characteristics to reduce the influence of light changes and occlusion on behaviors. Secondly, reliable feature point tracks are obtained through image feature point tracking and background track cutting, and a rich set of action descriptors (AD) are calculated from which local motion information, shape and static appearance information of the track are retained. After that, the principal component analysis operator is introduced to define the double feature fusion rules, and the STC feature and AD feature are combined to form a more accurate and complete feature representation. Finally, adaptive boosting algorithm (ABA) is used to train the classification through the new features obtained and complete the decision judgment of behavior and action. The experimental results show that the proposed algorithm has higher recognition accuracy and robustness compared with the current commonly used motion recognition methods, and can better adapt to complex background and motion changes.

Independence Action Recognition using Self Similarities

Exploring Self-Similarities Of Action Sequences Over Time And Observing The Striking Stability Of Human Action Recognition. Developing An Action Descriptor That Captures The Structure Of Temporal Similarities And Dissimilarities Within An Action Sequence. Self-Likeness Descriptors Are Demonstrated To Be Steady Under Execution Varieties Inside A Group Of Activities When Person Haste Changes Are Overlooked. Changes Between Two Unique Occurrences Of A Similar Class Can Be Unequivocally Recouped With Dynamic Time Traveling. Adequate Activity Separations Are As Yet Held Along These Lines To Construct A View-Autonomous Activity Acknowledgment Framework. Strangely Self-Likenesses Are Registered From Various Picture Highlights Have Comparable Properties And Can Be Utilized In A Corresponding Manner. It Depends Powerless Geometric Properties And Joins Them With AI For Proficient Cross-See Activity Acknowledgment. It Has Comparative Or Better Execution Looked At Than Related Techniques And It Performs Well Even In Outrageous Conditions, For Example, Well Perceiving Activities From Top Perspectives.