Optical Flow Computation Research Articles

Anisotropy of object nonrigidity: High-level perceptual consequences of cortical anisotropy.

We present a surprising anisotropy in perceived object nonrigidity, a complex, higher-level perceptual phenomenon, and explain it unexpectedly by the distribution of low-level neural properties in primary visual cortex. We examined the visual interpretation of two rigidly connected rotating circular rings. At speeds where observers predominantly perceived rigid rotation of the rings rotating horizontally, observers perceived only nonrigid wobbling when the image was rotated by 90°. Additionally, vertically rotating rings appeared narrower and longer compared to their physically identical horizontally rotating counterparts. We show that these perceived shape changes can be decoded from V1 outputs by incorporating documented anisotropies in orientation selectivity. We then show that even when the shapes are matched, the increased nonrigidity persists in vertical rotations, suggesting that motion mechanisms also play a role. By incorporating cortical anisotropies into optic flow computations, we show that the kinematic gradients (Divergence, Curl, Deformation) for vertical rotations align more with physical nonrigidity, while those for horizontal rotations align closer to rigidity, indicating that cortical anisotropies contribute to the orientation dependence of the perception of nonrigidity. Our results reveal how high-level percepts are shaped by low-level anisotropies, which raises questions about their evolutionary significance, particularly regarding shape constancy and motion perception.

DVDS: A deep visual dynamic slam system

Simultaneous localization and mapping (SLAM) utilizing visual sensors represent an extensively investigated research area, holding significant potential for advancements in robotics and autonomous vehicular systems. Recently, dense SLAM systems underpinned by learning-based methodologies have showcased superior accuracy and robustness compared to conventional techniques. Nevertheless, contemporary learning-based SLAM systems exhibit notable discrepancies in pose estimation, particularly within dynamic environments. In addition, the constrained receptive field of convolutional features in these methods impedes their efficacy when confronted with homogeneous, texture-less images, rendering them vulnerable to noise perturbations. We develop a novel deep visual dynamic slam (DVDS) system that exploits solely static pixels within images to retrieve the camera poses. Specifically, we formulate a dynamic object exclusion mechanism that excises dynamic constituents within the scene before the optical flow computation, thus optimizing the precision of the estimation. In addition, we unveil an efficient dispersive transformer (DisFormer) that facilitates per-pixel features in assimilating long-range information from surrounding features, culminating in constructing more precise 4D correlation volumes. Building on the DisFormer, we suggest a Disformer-based gated recurrent unit (GRU) to generate a refined flow field coupled with a confidence map, which is subsequently employed by the dense bundle adjustment layer to iteratively rectify the residuals of inverse depths and associated camera poses. The global receptive field provided by the DisFormer promotes information integration from a wider contextual window, thus improving the robustness of our SLAM system. Comprehensive experiments underscore that our proposed DVDS system manifests superior efficacy compared with state-of-the-art works across both static and dynamic scenes.

Reviewing Approaches and Techniques for Detecting Suspicious Human Behavior: A Comprehensive Survey

The paper aims to review related works that focus on detecting suspicious human behavior using machine-learning techniques. Suspicious human behavior refers to behaviors that may indicate involvement in or preparation for a crime. Detecting such behaviors before a crime is committed allows law enforcement to take early action and prevent criminal activities. One of the challenges in developing an effective detection system for suspicious human behavior is the absence of a well-defined definition for such behaviors. Different definitions can lead to various methods for designing the detection system. The paper explores different definitions and their implications on the design of detection systems and mentions two types of methods that can be used for detecting suspicious human behavior: image-based methods and saliency mapping. Image-based methods utilize image or video recognition techniques to analyze objects held by individuals or recognize specific activities. Saliency mapping, on the other hand, focuses on emphasizing the movement of individuals using techniques like optical flow calculation to generate saliency maps. Additionally, the paper highlights the increasing popularity of embedded machine learning, particularly on portable platforms. The use of embedded machine learning allows for the deployment of machine learning models on mobile or lightweight devices. This can be relevant for developing efficient and portable systems for detecting suspicious human behavior. Overall, the paper aims to provide an overview of existing works in the field of suspicious human behavior detection using machine learning, exploring different definitions and methods employed in the literature.

Real-time FPGA Implementation of an Optic Flow Algorithm Suitable for Background Oriented Schlieren Technique

Abstract To meet the demands of real-time displacement detection in embedded systems, a real-time optical flow calculation method suitable for background oriented schlieren (BOS) technology was transplanted and implemented on an Field Programmable Gate Array(FPGA) system. Firstly, through the analysis of different optical flow operation characteristics and the comparison of processing results in MATLAB with actual experimental data, an optical flow algorithm suitable for real-time processing of background shadow images was selected. Subsequently, by using hardware pipelining and multimodule parallel processing, the single-layer L&K optical flow algorithm was transplanted onto the FPGA and algorithm optimization was performed to achieve real-time optical flow calculation based on adjacent frame images. The key optimization techniques in this paper mainly include cleverly designing Double-Data-Rate Three Synchronous Dynamic Random Access Memory (DDR3) memory partitioning read and write logic to avoid image read and write conflicts; ensuring the timing alignment and data synchronization of consecutive frames by using burst read arbitration modules and two independent First In First Out (FIFO) buffers; and in optical flow calculation, constructing a 3×3 window using a shifting FIFO method to effectively save cache resources and improve computational speed. Experimental results show that the system can achieve real-time processing speeds of not less than 60fps for 1024×768 resolution images, meeting the requirements of practical applications.

Deepfake Face Detection Using Machine Learning with LSTM

Deep fake videos, which employ artificial intelligence to manipulate and generate highly convincing fake content, have emerged as a significant threat to society, potentially undermining trust in visual media. Detecting these deceptive videos is outmost importance to combat the spread of misinformation and protect the integrity of digital media. In this study, we propose a novel approach for deep fake face video detection utilizing Long Short-Term Memory (LSTM) networks, a type of Recurrent Neural Network (RNN). Our approach capitalizes on the temporal patterns and context within video sequences, harnessing the unique strengths of LSTM in capturing sequential information. We demonstrate the effectiveness of our methodology by training the LSTM network on a diverse dataset comprising both real and deep fake videos. The network’s ability to learn temporal dependencies and identify inconsistencies in facial expressions, eye movements, and other subtle cues allows it to distinguish between genuine and manipulated content. To further enhance the accuracy and robustness of our deep fake face detection system, we integrate pre-processing techniques for frame- level analysis, such as optical flow computation and facial landmarks extraction. Additionally, we employ a comprehensive ensemble of LSTM models and other machine learning algorithms to improve the overall detection performance. In our experiments, we evaluate the LSTM-based deep fake detection system on a large-scale dataset of both known and unseen deep fake videos, achieving high detection accuracy and low false positive rates. We also compare our approach with existing methods, demonstrating its superiority in terms of robustness and generalization. The results of this study signify the potential of LSTM-based models for mitigating the adverse effects of deep fake content on society. As deep fake technology continues to evolve, our approach showcases a promising step towards combating the dissemination of deceptive multimedia, promoting media integrity, and upholding trust in visual information. Keywords: LSTM Networks, Recurrent Neural Network, Optical flow computation, Facial landmarks extraction, False positive rates.

Gas–Liquid Two-Phase Flow Measurement Based on Optical Flow Method with Machine Learning Optimization Model

Gas–Liquid two-phase flows are a common flow in industrial production processes. Since these flows inherently consist of discrete phases, it is challenging to accurately measure the flow parameters. In this context, a novel approach is proposed that combines the pyramidal Lucas-Kanade (L–K) optical flow method with the Split Comparison (SC) model measurement method. In the proposed approach, videos of gas–liquid two-phase flows are captured using a camera, and optical flow data are acquired from the flow videos using the pyramid L–K optical flow detection method. To address the issue of data clutter in optical flow extraction, a dynamic median value screening method is introduced to optimize the corner point for optical flow calculations. Machine learning algorithms are employed for the prediction model, yielding high flow prediction accuracy in experimental tests. Results demonstrate that the gradient boosted regression (GBR) model is the most effective among the five preset models, and the optimized SC model significantly improves measurement accuracy compared to the GBR model, achieving an R2 value of 0.97, RMSE of 0.74 m3/h, MAE of 0.52 m3/h, and MAPE of 8.0%. This method offers a new approach for monitoring flows in industrial production processes such as oil and gas.

Multi-YOLOv8: An infrared moving small object detection model based on YOLOv8 for air vehicle

The detection of infrared moving small objects faces significant challenges in the field of object detection for air vehicles. These types of objects usually occupy a small number of pixels in an infrared image, resulting in limited feature information, considerable feature loss, low recognition accuracy, and various challenges in single-frame detection. To address these challenges, this paper proposes an efficient multi-input method named Multi-YOLOv8, which is based on the YOLOv8s model. The proposed method uses current frames as a primary input and incorporates optical flow processing images and background suppression images as auxiliary inputs to improve detection performance. In addition, an improved method is developed for optical flow computations, named the pyramidal weight-momentum Horn–Schunck (PWMHS) method, which can process optical flows efficiently and precisely. An improved version of the Wise-IoU (WIoU) v3, referred to as α*-WIoU v3, is proposed as a bounding box regression (BBR) loss function to optimize the YOLOv8 network. Further, the BiFormer module and lightweight convolution GSConv are introduced to improve the attention to key information for the objects and balance the computational cost and detection performance, respectively. Moreover, a small object detection layer is added the YOLOv8 network to improve the capability for small object detection. Finally, a warming-up training method that can reduce the dependency on auxiliary inputs and ensure model stability in case of auxiliary input failures is developed. The results of the comprehensive experiments on an open-access dataset reveal that the proposed model outperforms the mainstream models in overall performance. The proposed method can significantly enhance the detection ability of infrared moving small objects.

Vision-based algorithm for structural response measurement using movable camera and damage localization

Vibration-based Structural Health Monitoring (SHM) is widely recognized as a prominent approach for evaluating the safety and integrity of civil infrastructure. This research presents a novel algorithm for vibration measurement using a movable camera, effectively overcoming the issue of signal drift caused by camera motion. The proposed algorithm comprises three modules: optical flow computation, camera pose estimation, and motion compensation. Validation experiments by virtual images demonstrate accurate camera pose estimation and effective mitigation of drifting signals. The dynamic characteristics of the frame structure are extracted using a movable camera and the proposed algorithm. Several challenges are addressed, including accurately estimating the scale factor to compensate for the drifting signals. Experimental validation is conducted to verify the proposed approach. The results demonstrate the efficacy and promising potential of the proposed approach. Finally, this study demonstrates the practical application through damage localization experiments.

PDTE: Pyramidal deep Taylor expansion for optical flow estimation

Optical flow estimation is an important hot research in computer vision. Although existing methods had got a considerable progress in improving their performance, they still have drawbacks, such as heavily computational burden, inaccurate pixel-level offset estimation, and poor interpretability. To address these issues, this letter proposes a pyramidal deep Taylor expansion (PDTE) framework, including: First, we seriously interpreted the relationship between optical flow computation and Taylor expansion. Then, the proposed PDTE is constructed by employing global motion aggregation (GMA) to calculate each derivative part which contributes to the final estimated optical flow. Quantitative and qualitative results on the Sintel and KITTI datasets validate that the proposed PDTE scheme is effective and outperforms the state-of-the-art optical flow estimation methods. The results of extensive experiments in the ablation study demonstrate that PDTE performs well on shape preservation and the accuracy improvement of optical flow estimation, even pixel-level offset calculation.

Enhancing Infrared Optical Flow Network Computation through RGB-IR Cross-Modal Image Generation.

Due to the complexity of real optical flow capture, the existing research still has not performed real optical flow capture of infrared (IR) images with the production of an optical flow based on IR images, which makes the research and application of deep learning-based optical flow computation limited to the field of RGB images only. Therefore, in this paper, we propose a method to produce an optical flow dataset of IR images. We utilize the RGB-IR cross-modal image transformation network to rationally transform existing RGB image optical flow datasets. The RGB-IR cross-modal image transformation is based on the improved Pix2Pix implementation, and in the experiments, the network is validated and evaluated using the RGB-IR aligned bimodal dataset M3FD. Then, RGB-IR cross-modal transformation is performed on the existing RGB optical flow dataset KITTI, and the optical flow computation network is trained using the IR images generated by the transformation. Finally, the computational results of the optical flow computation network before and after training are analyzed based on the RGB-IR aligned bimodal data.

Duality-Gated Mutual Condition Network for RGBT Tracking.

Low-quality modalities contain not only a lot of noisy information but also some discriminative features in RGB-Thermal (RGBT) tracking. However, the potentials of low-quality modalities are not well explored in existing RGBT tracking algorithms. In this work, we propose a novel duality-gated mutual condition network to fully exploit the discriminative information of all modalities while suppressing the effects of data noise. In specific, we design a mutual condition module, which takes the discriminative information of a modality as the condition to guide feature learning of target appearance in another modality. Such a module can effectively enhance target representations of all modalities even in the presence of low-quality modalities. To improve the quality of conditions and further reduce data noise, we propose a duality-gated mechanism and integrate it into the mutual condition module. To deal with the tracking failure caused by sudden camera motion, which often occurs in RGBT tracking, we design a resampling strategy based on optical flow. It does not increase much computational cost since we perform optical flow calculation only when the model prediction is unreliable and then execute resampling when the sudden camera motion is detected. Extensive experiments on four RGBT tracking benchmark datasets show that our method performs favorably against the state-of-the-art tracking algorithms.

Development of artificial intelligence-based slow-motion echocardiography and clinical usefulness for evaluating regional wall motion abnormalities.

The diagnostic accuracy of exercise stress echocardiography (ESE) for myocardial ischemia requires improvement, given that it currently depends on the physicians' experience and image quality. To address this issue, we aimed to develop artificial intelligence (AI)-based slow-motion echocardiography using inter-image interpolation. The clinical usefulness of this method was evaluated for detecting regional wall-motion abnormalities (RWMAs). In this study, an AI-based echocardiographic image-interpolation pipeline was developed using optical flow calculation and prediction for in-between images. The accuracy for detecting RWMAs and image readability among 25 patients with RWMA and 25 healthy volunteers was compared between four cardiologists using slow-motion and conventional ESE. Slow-motion echocardiography was successfully developed for arbitrary time-steps (e.g., 0.125×, and 0.5×) using 1,334 videos. The RWMA detection accuracy showed a numerical improvement, but it was not statistically significant (87.5% in slow-motion echocardiography vs. 81.0% in conventional ESE; odds ratio: 1.43 [95% CI: 0.78-2.62], p = 0.25). Interreader agreement analysis (Fleiss's Kappa) for detecting RWMAs among the four cardiologists were 0.66 (95%CI: 0.55-0.77) for slow-motion ESE and 0.53 (95%CI: 0.42-0.65) for conventional ESE. Additionally, subjective evaluations of image readability using a four-point scale showed a significant improvement for slow-motion echocardiography (2.11 ± 0.73 vs. 1.70 ± 0.78, p < 0.001).In conclusion, we successfully developed slow-motion echocardiography using in-between echocardiographic image interpolation. Although the accuracy for detecting RWMAs did not show a significant improvement with this method, we observed enhanced image readability and interreader agreement. This AI-based approach holds promise in supporting physicians' evaluations.

Abstract 11702: Development of Artificial Intelligence-Based Slow-Motion Echocardiography and Clinical Usefulness for Evaluating Regional Wall Motion Abnormalities

Introduction: Improvement in accuracy and image readability of stress echocardiography needs to be addressed, given that the accuracy depends on the physicians’ experience and image quality. The study aimed to develop an artificial intelligence (AI) based slow-motion echocardiography by inter-image interpolation and evaluate the clinical usefulness for detecting regional wall motion abnormalities (RWMAs). Methods: An AI-based image interpolation pipeline was developed with the optical flow calculation between echocardiographic images and predicting in-between images. In a cohort of patients with RWMA (n=25) and healthy volunteers (n=25), the accuracy for detecting RWMAs and image-reliabilities among four cardiologists was evaluated using slow-motion and conventional exercise stress echocardiography. Results: The development of slow-motion echocardiography were successfully performed at any arbitrary-time steps (e.g., 0.125x, 0.25x, and 0.5x). The accuracy for detecting RWMAs was numerically improved, while not statistically significant (87.5% in slow-motion echocardiography vs 81.0% in conventional stress echocardiography, odds ratio: 1.43 (95% confidence interval 0.78-2.62, p=0.25). Inter-reader agreement analysis demonstrated the level of agreements (Fleiss’s Kappa) for detecting RWMAs was 0.66 (95%CI: 0.55-0.77) in slow-motion echocardiography, and 0.53 (95%CI: 0.42-0.65) in conventional stress echocardiography. As for the four-point scale subjectively-evaluated image readability, the mean score was significantly improved using slow-motion echocardiography (2.11±0.73vs 1.70±0.78, p&lt;0.001). Conclusions: Slow-motion echocardiography was successfully developed. There was no significant improvement in the accuracy for detecting RWMAs using the developed slow-motion echocardiography, while the image readability and inter-reader agreement were improved.

Research on the Analysis of Traditional Dance Performance Forms and Dance Movement Characteristics Based on Artificial Intelligence Technology

Abstract This paper analyzes the performance forms and movement characteristics of traditional dances by using the method of movement feature extraction. We construct a dance key movement extraction system by merging optical flow computing and extracting the key movements of music and dance, which increases the efficiency of traditional dance movement analysis and reduces computational complexity. We can improve the similarity matching of human posture when using picture entropy computation. It uses an optical capture technique to record lively dancing motions. The results show that the accuracy of optical flow computation is 0.8 for distinguishing the form of the cultural lion performance, 0.85 for distinguishing the martial lion performance, 0.75 for distinguishing the aerial lion performance, and 0.6 for capturing the tumbling movement of the cultural lion and 0.7 for capturing the frolicking movement under the optical motion capture system.

Brain-like position measurement method based on improved optical flow algorithm

In this paper, a brain-like navigation scheme based on fuzzy kernel C-means (FKCM) clustering assisted pyramid Lucas Kanade (LK) optical flow algorithm is developed to measure the position of vehicle. The Speed Cell and Place Cell in animals’ brain are introduced to construct the brain-like navigation mechanism which involves the optical flow method and image template matching to imitate the cells above-mentioned separately. To eliminate the singular values during optical flow calculation, the output of pyramid LK algorithm is clustered by FKCM algorithm firstly. Then, the velocity is calculated and integrated to get the position of the vehicle, and the brain-like navigation scheme is introduced to correct the position measurement errors by eliminating the accumulated errors resulting from velocity integration. The prominent advantages of the presented method are: (i) a pure visual brain-like position measurement method based on the concept of speed cells and place cells is proposed, making visual navigation more accurate and intelligent; (ii) the FKCM algorithm is used to eliminate the singular value of the pyramid LK algorithm, which improves the calculated velocity accuracy. Also, experimental comparison with classical pyramid LK algorithm is given to illustrate the superiority of the proposed method in position measurement.

Dynamic 3D phase-shifting profilometry based ona corner optical flow algorithm.

Real-time 3D reconstruction has been applied in many fields, calling for many ongoing efforts to improve the speed and accuracy of the used algorithms. Phase shifting profilometry based on the Lucas-Kanade optical flow method is a fast and highly precise method to construct and display the three-dimensional shape of objects. However, in this method, a dense optical flow calculation is required for the modulation image corresponding to the acquired deformed fringe pattern, which consumes a lot of time and affects the real-time performance of 3D reconstruction and display. Therefore, this paper proposes a dynamic 3D phase shifting profilometry based on a corner optical flow algorithm to mitigate this issue. Therein, the Harris corner algorithm is utilized to locate the feature points of the measured object, so that the optical flow needs to calculate for only the feature points which, greatly reduces the amount of calculation time. Both our experiments and simulations show that our method improves the efficiency of pixel matching by four times and 3D reconstruction by two times.

Physics-based optical flow estimation under varying illumination conditions

The physics-based optical flow (PBOF) model was derived from typical flow visualizations and it provided a more solid physical foundation for optical flow calculation. However, like many recent variational optical flow methods, it is not formulated to deal with the effect of illumination variation. In this paper, we propose a new efficient illumination-invariant optical flow estimation method, which improves the physics-based optical flow model. The proposed method introduces a multiplicative product of the velocity gradient and intensity from PBOF and an additive diffusion component to relax the brightness constancy assumption. The proposed method employs a linear transformation to reduce the influence of varying illumination on the data term. Then, a smoothness-sparsity regularization is used to constrain the minimization problem to have good edge-preserving for optical flow. Furthermore, the numerical implementation is given to solve the optimization problem. The accuracy of the developed method and the robustness against varying illumination are evaluated. The proposed method can provide motion estimation with acceptable accuracy for three optical flow datasets and cloud images of Jupiter’s great red spot, which have challenging illumination variations.

Modeling the development of cortical responses in primate dorsal ("where") pathway to optic flow using hierarchical neural field models.

Although there is a plethora of modeling literature dedicated to the object recognition processes of the ventral ("what") pathway of primate visual systems, modeling studies on the motion-sensitive regions like the Medial superior temporal area (MST) of the dorsal ("where") pathway are relatively scarce. Neurons in the MST area of the macaque monkey respond selectively to different types of optic flow sequences such as radial and rotational flows. We present three models that are designed to simulate the computation of optic flow performed by the MST neurons. Model-1 and model-2 each composed of three stages: Direction Selective Mosaic Network (DSMN), Cell Plane Network (CPNW) or the Hebbian Network (HBNW), and the Optic flow network (OF). The three stages roughly correspond to V1-MT-MST areas, respectively, in the primate motion pathway. Both these models are trained stage by stage using a biologically plausible variation of Hebbian rule. The simulation results show that, neurons in model-1 and model-2 (that are trained on translational, radial, and rotational sequences) develop responses that could account for MSTd cell properties found neurobiologically. On the other hand, model-3 consists of the Velocity Selective Mosaic Network (VSMN) followed by a convolutional neural network (CNN) which is trained on radial and rotational sequences using a supervised backpropagation algorithm. The quantitative comparison of response similarity matrices (RSMs), made out of convolution layer and last hidden layer responses, show that model-3 neuron responses are consistent with the idea of functional hierarchy in the macaque motion pathway. These results also suggest that the deep learning models could offer a computationally elegant and biologically plausible solution to simulate the development of cortical responses of the primate motion pathway.

Search for identical points in the inter-pixel space of video images

One of the ways to describe objects in images is to identify some of their characteristic points or points of attention. Areas surrounding attention points are described by descriptors (a set of features) in such a way that they can be identified and compared. On these features the search for identical points on other images is carried out by scanning them with a sliding window. The most famous descriptors and methods for finding identical points are: SIFT, SURF, GLOH, BRIEF and others. This group of methods is characterized by the fact that the displacement of identical points in video images can be arbitrary, but the accuracy of calculating their coordinates depends on the bit grid of video images and, in the best case, is equal to the interpixel distance. Another group of methods that can be used to track identical points of video images are methods built on the basis of optical flow calculation. One of the popular methods of tracking points based on optical flow calculation is the Lucas-Kanade method. It allows you to calculate the displacement of points in the interpixel space due to the solution of differential equations. To date, the Lucas-Kanade method has several modifications. A limitation of these methods is that the neighborhoods of the shifted points must overlap to a large extent. The article investigates and proposes the complex application of methods of scanning video images with a sliding window and differential calculation of optical flow, which allows to increase the accuracy and speed of calculating the coordinates of identical points in the images in relation to the search for these points only by scanning. A more accurate calculation of the coordinates of the characteristic points of the object in the interpixel space of video images will lead to a more accurate determination of the position and orientation of these objects in 3D space. The simulation was carried out using the method of rough search for identical points of video images described by invariant moments and specifying their coordinates using the Lucas-Kanade point tracking method. The simulation results indicate an increase in speed by almost an order of magnitude and, according to indirect estimates, the accuracy of calculations.

3D network with channel excitation and knowledge distillation for action recognition.

Modern action recognition techniques frequently employ two networks: the spatial stream, which accepts input from RGB frames, and the temporal stream, which accepts input from optical flow. Recent researches use 3D convolutional neural networks that employ spatiotemporal filters on both streams. Although mixing flow with RGB enhances performance, correct optical flow computation is expensive and adds delay to action recognition. In this study, we present a method for training a 3D CNN using RGB frames that replicates the motion stream and, as a result, does not require flow calculation during testing. To begin, in contrast to the SE block, we suggest a channel excitation module (CE module). Experiments have shown that the CE module can improve the feature extraction capabilities of a 3D network and that the effect is superior to the SE block. Second, for action recognition training, we adopt a linear mix of loss based on knowledge distillation and standard cross-entropy loss to effectively leverage appearance and motion information. The Intensified Motion RGB Stream is the stream trained with this combined loss (IMRS). IMRS surpasses RGB or Flow as a single stream; for example, HMDB51 achieves 73.5% accuracy, while RGB and Flow streams score 65.6% and 69.1% accuracy, respectively. Extensive experiments confirm the effectiveness of our proposed method. The comparison with other models proves that our model has good competitiveness in behavior recognition.