Benchmark Datasets Research Articles

A molecular video-derived foundation model for scientific drug discovery

Accurate molecular representation of compounds is a fundamental challenge for prediction of drug targets and molecular properties. In this study, we present a molecular video-based foundation model, named VideoMol, pretrained on 120 million frames of 2 million unlabeled drug-like and bioactive molecules. VideoMol renders each molecule as a video with 60-frame and designs three self-supervised learning strategies on molecular videos to capture molecular representation. We show high performance of VideoMol in predicting molecular targets and properties across 43 drug discovery benchmark datasets. VideoMol achieves high accuracy in identifying antiviral molecules against common diverse disease-specific drug targets (i.e., BACE1 and EP4). Drugs screened by VideoMol show better binding affinity than molecular docking, revealing the effectiveness in understanding the three-dimensional structure of molecules. We further illustrate interpretability of VideoMol using key chemical substructures.

MACHINE LEARNING, DEEP LEARNING AND ENSEMBLE LEARNING BASED APPROACHES FOR INTRUSION DETECTION ENHANCEMENT

With the extensive application of Internet of Things and wireless sensor networks (WSNs) in various real-time fields, they have become increasingly vulnerable to several types of attacks that could have a serious impact in their functionalities. Recently, intrusion detection systems have become one of the crucial security components. We propose in this work an approach based on machine and deep learning for DoS and DDoS attack detection. We have evaluated, analyzed, and compared the efficiency of three learning models separately, including deep neural network (DNN), random forest, and decision tree, using the standard metrics of evaluation such as accuracy, precision, F1-score, and recall. In our contribution, an approach ensemble learning was introduced in order to enhance the accuracy rate of classification. Our study was carried out using two well-known benchmark and real-time datasets, CICIoT-2023 and WSN-DS, intended for wireless sensor networks and IoT. These datasets containing various types of attacks. CICIoT2023 dataset contains 33 attacks divided into 7 classes, while WSN-DS dataset contains four types of DoS attacks, including Blackhole, Grayhole, Flooding, and TDMA. The experiment result demonstrate the effectiveness of our approach in attacks detection with high accuracy achieved close to perfect. Our approach result demonstrates that using the ensemble learning technique works better than each ML architecture independently.

Deep emotion recognition in textual conversations: a survey

Emotion Recognition in Conversations (ERC) is a key step towards successful human–machine interaction. While the field has seen tremendous advancement in the last few years, new applications and implementation scenarios present novel challenges and opportunities. These range from leveraging the conversational context, speaker, and emotion dynamics modelling, to interpreting common sense expressions, informal language, and sarcasm, addressing challenges of real-time ERC, recognizing emotion causes, different taxonomies across datasets, multilingual ERC, and interpretability. This survey starts by introducing ERC, elaborating on the challenges and opportunities of this task. It proceeds with a description of the emotion taxonomies and a variety of ERC benchmark datasets employing such taxonomies. This is followed by descriptions comparing the most prominent works in ERC with explanations of the neural architectures employed. Then, it provides advisable ERC practices towards better frameworks, elaborating on methods to deal with subjectivity in annotations and modelling and methods to deal with the typically unbalanced ERC datasets. Finally, it presents systematic review tables comparing several works regarding the methods used and their performance. Benchmarking these works highlights resorting to pre-trained Transformer Language Models to extract utterance representations, using Gated and Graph Neural Networks to model the interactions between these utterances, and leveraging Generative Large Language Models to tackle ERC within a generative framework. This survey emphasizes the advantage of leveraging techniques to address unbalanced data, the exploration of mixed emotions, and the benefits of incorporating annotation subjectivity in the learning phase.

LEVERAGING CORPUS LINGUISTICS AND DATA-DRIVEN DEEP LEARNING FOR TEXTUAL EMOTION ANALYSIS

Emotions have played a major part in the conversation, as they express context to the conversation. Text or words in conversation contain contextual and lexical meanings. In recent times, obtaining emotion from the text has been an attractive area of research. With the emergence of machine learning (ML) algorithms and hardware to aid the ML method, identifying emotion from the text with ML provides significant and promising solutions. The main objective of Textual Emotion Analysis (TEA) is to analyze and extract the user’s emotional states in the text. Many different Complex Systems and Deep Learning (DL) algorithms have been fast-paced developed and proved their effectiveness in several fields including audio, image, and natural language processing (NLP). This has moved researchers away from the classical ML to DL for their academic research work. This study develops a new Corpus Linguistics and Data-Driven Deep Learning for Textual Emotion Analysis (CLD3L-TEA) technique. The CLD3L-TEA technique mainly investigates the distinct types of emotions that endure in the social media text. In the CLD3L-TEA model, the raw data can be pre-processed in distinct ways. Next, a multi-weighted TF–IDF model is used to generate feature vectors. For the identification of emotions, the CLD3L-TEA technique applied a gated recurrent unit (GRU). At last, the hyperparameter range of the GRU model is executed by the Fractal Harris Hawks Optimization (HHO) model. The experimental validation of the CLD3L-TEA technique on a benchmark dataset illustrates the supremacy of this technique over recent approaches.

Fusing semantic and syntactic information for aspect sentiment triplet extraction

Aspect Sentiment Triplet Extraction (ASTE) aims to extract aspect terms, sentiment polarity and opinion terms explaining the reason for the sentiment from a sentence in the form of triplets. Many existing studies model the context by graph neural networks to learn relevant information from the generated graphs. However, some sentences may have syntactic errors or lack significant grammar, which may lead to poor results on the dataset of the model. In this paper, we propose the Fusing Semantic and Syntactic Information for Aspect Sentiment Triplet Extraction (FSSI) model, which incorporates both syntactic structure and semantic relevance in the context. Specifically, we construct a syntactic graph convolutional network to obtain comprehensive syntactic structure information and a semantic graph convolutional network to obtain global semantic relevance of sentences using the self-attention mechanism. Furthermore, we concatenate the graph representations generated by the two graph convolution networks to obtain the final enhanced representation. Finally, we apply an effective inference strategy to extract triplets. Extensive experimental results on benchmark datasets show that our model outperforms state-of-the-art approaches.

Take good care of your fish: fish re-identification with synchronized multi-view camera system

IntroductionFish re-identification (re-ID) is of great significance for fish monitoring and can contribute to aquaculture and fish breeding. Synchronizing information from different cameras is beneficial for optimizing re-ID performance.MethodsWe constructed the first underwater fish re-identification benchmark dataset (FS48) under three camera conditions. FS48 encompasses 48 different fish identities, 10,300 frames, and 39,088 bounding boxes, covering various lighting conditions and background environments. Additionally, we developed the first robust and accurate fish re-identification baseline, FSNet, which fuses information from three camera positions by extracting features from synchronized video frames of each position and combining the synchronized information.ResultsThe experimental results show that FS48 is universal and of high quality. FSNet has an effective network design and demonstrates good performance, achieving better re-identification performance by combining information from three positions, helping improve overall re-test accuracy, and evaluating the effectiveness of re-identification among detectors.DiscussionOur dataset will be released upon acceptance of this paper, which is expected to further promote the development of underwater fish re-identification.

A vessel bifurcation liver CT landmark pair dataset for evaluating deformable image registration algorithms.

Evaluating deformable image registration (DIR) algorithms is vital for enhancing algorithm performance and gaining clinical acceptance. However, there is a notable lack of dependable DIR benchmark datasets for assessing DIR performance except for lung images. To address this gap, we aim to introduce our comprehensive liver computed tomography (CT) DIR landmark dataset library. This dataset is designed for efficient and quantitative evaluation of various DIR methods for liver CTs, paving the way for more accurate and reliable image registration techniques. Forty CT liver image pairs were acquired from several publicly available image archives and authors' institutions under institutional review board (IRB) approval. The images were processed with a semi-automatic procedure to generate landmark pairs: (1) for each case, liver vessels were automatically segmented on one image; (2) landmarks were automatically detected at vessel bifurcations; (3) corresponding landmarks in the second image were placed using two deformable image registration methods to avoid algorithm-specific biases; (4) a comprehensive validation process based on quantitative evaluation and manual assessment was applied to reject outliers and ensure the landmarks' positional accuracy. This workflow resulted in an average of ∼56 landmark pairs per image pair, comprising a total of 2220 landmarks for 40 cases. The general landmarking accuracy of this procedure was evaluated using digital phantoms and manual landmark placement. The landmark pair target registration errors (TRE) on digital phantoms were 0.37±0.26 and 0.55±0.34mm respectively for the two selected DIR algorithms used in our workflow, with 97% of landmark pairs having TREs below 1.5mm. The distances from the calculated landmarks to the averaged manual placement were 1.27±0.79mm. All data, including image files and landmark information, are publicly available at Zenodo (https://zenodo.org/records/13738577). Instructions for using our data can be found on our GitHub page at https://github.com/deshanyang/Liver-DIR-QA. The landmark dataset generated in this work is the first collection of large-scale liver CT DIR landmarks prepared on real patient images. This dataset can provide researchers with a dense set of ground truth benchmarks for the quantitative evaluation of DIR algorithms within the liver.

ChemXTree: A Feature-Enhanced Graph Neural Network-Neural Decision Tree Framework for ADMET Prediction.

The rapid progression of machine learning, especially deep learning (DL), has catalyzed a new era in drug discovery, introducing innovative approaches for predicting molecular properties. Despite the many methods available for feature representation, efficiently utilizing rich, high-dimensional information remains a significant challenge. Our work introduces ChemXTree, a novel graph-based model that integrates a Gate Modulation Feature Unit (GMFU) and neural decision tree (NDT) in the output layer to address this challenge. Extensive evaluations on benchmark data sets, including MoleculeNet and eight additional drug databases, have demonstrated ChemXTree's superior performance, surpassing or matching the current state-of-the-art models. Visualization techniques clearly demonstrate that ChemXTree significantly improves the separation between substrates and nonsubstrates in the latent space. In summary, ChemXTree demonstrates a promising approach for integrating advanced feature extraction with neural decision trees, offering significant improvements in predictive accuracy for drug discovery tasks and opening new avenues for optimizing molecular properties.

StreaMD: the toolkit for high-throughput molecular dynamics simulations.

Molecular dynamics simulations serve as a prevalent approach for investigating the dynamic behaviour of proteins and protein-ligand complexes. Due to its versatility and speed, GROMACS stands out as a commonly utilized software platform for executing molecular dynamics simulations. However, its effective utilization requires substantial expertise in configuring, executing, and interpreting molecular dynamics trajectories. Existing automation tools are constrained in their capability to conduct simulations for large sets of compounds with minimal user intervention, or in their ability to distribute simulations across multiple servers. To address these challenges, we developed a Python-based tool that streamlines all phases of molecular dynamics simulations, encompassing preparation, execution, and analysis. This tool minimizes the required knowledge for users engaging in molecular dynamics simulations and can efficiently operate across multiple servers within a network or a cluster. Notably, the tool not only automates trajectory simulation but also facilitates the computation of free binding energies for protein-ligand complexes and generates interaction fingerprints across the trajectory. Our study demonstrated the applicability of this tool on several benchmark datasets. Additionally, we provided recommendations for end-users to effectively utilize the tool.Scientific contributionThe developed tool, StreaMD, is applicable to different systems (proteins, ligands and their complexes including co-factors) and requires a little user knowledge to setup and run molecular dynamics simulations. Other features of StreaMD are seamless integration with calculation of MM-GBSA/PBSA binding free energies and protein-ligand interaction fingerprints, and running of simulations within distributed environments. All these will facilitate routine and massive molecular dynamics simulations.

A tied-weight autoencoder for the linear dimensionality reduction of sample data

Dimensionality reduction is a method used in machine learning and data science to reduce the dimensions in a dataset. While linear methods are generally less effective at dimensionality reduction than nonlinear methods, they can provide a linear relationship between the original data and the dimensionality-reduced representation, leading to better interpretability. In this research, we present a tied-weight autoencoder as a dimensionality reduction model with the merit of both linear and nonlinear methods. Although the tied-weight autoencoder is a nonlinear dimensionality reduction model, we approximate it to function as a linear model. This is achieved by removing the hidden layer units that are largely inactivated by the input data, while preserving the model’s effectiveness. We evaluate the proposed model by comparing its performance with other linear and nonlinear models using benchmark datasets. Our results show that the proposed model performs comparably to the nonlinear model of a similar autoencoder structure to the proposed model. More importantly, we show that the proposed model outperforms the linear models in various metrics, including the mean square error, data reconstruction, and the classification of low-dimensional projections of the input data. Thus, our study provides general recommendations for best practices in dimensionality reduction.

Large-Kernel Central Block Masked Convolution and Channel Attention-Based Reconstruction Network for Anomaly Detection of High-Resolution Hyperspectral Imagery

In recent years, the rapid advancement of drone technology has led to an increasing use of drones equipped with hyperspectral sensors for ground imaging. Hyperspectral data captured via drones offer significantly higher spatial resolution, but this also introduces more complex background details and larger target scales in high-resolution hyperspectral imagery (HRHSI), posing substantial challenges for hyperspectral anomaly detection (HAD). Mainstream reconstruction-based deep learning methods predominantly emphasize spatial local information in hyperspectral images (HSIs), relying on small spatial neighborhoods for reconstruction. As a result, large anomalous targets and background details are often well reconstructed, leading to poor anomaly detection performance, as these targets are not sufficiently distinguished from the background. To address these limitations, we propose a novel HAD network for HRHSI based on large-kernel central block masked convolution and channel attention, termed LKCMCA. Specifically, we first employ the pixel-shuffle technique to reduce the size of anomalous targets without losing image information. Next, we design a large-kernel central block masked convolution to make the network pay more attention to the surrounding background information, enabling better fusion of the information between adjacent bands. This, coupled with an efficient channel attention mechanism, allows the network to capture deeper spectral features, enhancing the reconstruction of the background while suppressing anomalous targets. Furthermore, we introduce an adaptive loss function by down-weighting anomalous pixels based on the mean absolute error. This loss function is specifically designed to suppress the reconstruction of potentially anomalous pixels during network training, allowing our model to be considered an excellent background reconstruction network. By leveraging reconstruction error, the model effectively highlights anomalous targets. Meanwhile, we produced four benchmark datasets specifically for HAD tasks using existing HRHSI data, addressing the current shortage of HRHSI datasets in the HAD field. Extensive experiments demonstrate that our LKCMCA method achieves superior detection performance, outperforming ten state-of-the-art HAD methods on all datasets.

Positive Unlabeled Learning Selected Not At Random (PULSNAR): class proportion estimation without the selected completely at random assumption

Positive and unlabeled (PU) learning is a type of semi-supervised binary classification where the machine learning algorithm differentiates between a set of positive instances (labeled) and a set of both positive and negative instances (unlabeled). PU learning has broad applications in settings where confirmed negatives are unavailable or difficult to obtain, and there is value in discovering positives among the unlabeled (e.g., viable drugs among untested compounds). Most PU learning algorithms make the selected completely at random (SCAR) assumption, namely that positives are selected independently of their features. However, in many real-world applications, such as healthcare, positives are not SCAR (e.g., severe cases are more likely to be diagnosed), leading to a poor estimate of the proportion, α, of positives among unlabeled examples and poor model calibration, resulting in an uncertain decision threshold for selecting positives. PU learning algorithms vary; some estimate only the proportion, α, of positives in the unlabeled set, while others calculate the probability that each specific unlabeled instance is positive, and some can do both. We propose two PU learning algorithms to estimate α, calculate calibrated probabilities for PU instances, and improve classification metrics: i) PULSCAR (positive unlabeled learning selected completely at random), and ii) PULSNAR (positive unlabeled learning selected not at random). PULSNAR employs a divide-and-conquer approach to cluster SNAR positives into subtypes and estimates α for each subtype by applying PULSCAR to positives from each cluster and all unlabeled. In our experiments, PULSNAR outperformed state-of-the-art approaches on both synthetic and real-world benchmark datasets.

SimCDL: A Simple Framework for Contrastive Dictionary Learning

In this paper, we propose a novel approach to the dictionary learning (DL) initialization problem, leveraging the SimCLR framework from deep learning in a self-supervised manner. Dictionary learning seeks to represent signals as sparse combinations of dictionary atoms, but effective initialization remains challenging. By applying contrastive learning, we encourage similar representations for augmented versions of the same sample while distinguishing between different samples. This results in a more diverse and incoherent set of atoms, which enhances the performance of DL applications in classification and anomaly detection tasks. Our experiments across several benchmark datasets demonstrate the effectiveness of our method for improving dictionary learning initialization and its subsequent impact on performance in various applications.

Structure-aware annotation of leucine-rich repeat domains.

Protein domain annotation is typically done by predictive models such as HMMs trained on sequence motifs. However, sequence-based annotation methods are prone to error, particularly in calling domain boundaries and motifs within them. These methods are limited by a lack of structural information accessible to the model. With the advent of deep learning-based protein structure prediction, existing sequenced-based domain annotation methods can be improved by taking into account the geometry of protein structures. We develop dimensionality reduction methods to annotate repeat units of the Leucine Rich Repeat solenoid domain. The methods are able to correct mistakes made by existing machine learning-based annotation tools and enable the automated detection of hairpin loops and structural anomalies in the solenoid. The methods are applied to 127 predicted structures of LRR-containing intracellular innate immune proteins in the model plant Arabidopsis thaliana and validated against a benchmark dataset of 172 manually-annotated LRR domains.

Natural Language Processing and Machine Learning-Based Solution of Cold Start Problem Using Collaborative Filtering Approach

In today’s digital era, the abundance of online services presents users with a daunting array of choices, spanning from streaming platforms to e-commerce websites, leading to decision fatigue. Recommendation algorithms play a pivotal role in aiding users in navigating this plethora of options, among which collaborative filtering (CF) stands out as a prevalent technique. However, CF encounters several challenges, including scalability issues, privacy implications, and the well-known cold start problem. This study endeavors to mitigate the cold start problem by harnessing the capabilities of natural language processing (NLP) applied to user-generated reviews. A unique methodology is introduced, integrating both supervised and unsupervised NLP approaches facilitated by sci-kit learn, utilizing benchmark datasets across diverse domains. This study offers scientific contributions through its novel approach, ensuring rigor, precision, scalability, and real-world relevance. It tackles the cold start problem in recommendation systems by combining natural language processing (NLP) with machine learning and collaborative filtering techniques, addressing data sparsity effectively. This study emphasizes reproducibility and accuracy while proposing an advanced solution that improves personalization in recommendation models. The proposed NLP-based strategy enhances the quality of user-generated content, consequently refining the accuracy of Collaborative Filtering-Based Recommender Systems (CFBRSs). The authors conducted experiments to test the performance of the proposed approach on benchmark datasets like MovieLens, Jester, Book-Crossing, Last.fm, Amazon Product Reviews, Yelp, Netflix Prize, Goodreads, IMDb (Internet movie Database) Data, CiteULike, Epinions, and Etsy to measure global accuracy, global loss, F-1 Score, and AUC (area under curve) values. Assessment through various techniques such as random forest, Naïve Bayes, and Logistic Regression on heterogeneous benchmark datasets indicates that random forest is the most effective method, achieving an accuracy rate exceeding 90%. Further, the proposed approach received a global accuracy above 95%, a global loss of 1.50%, an F-1 Score of 0.78, and an AUC value of 92%. Furthermore, the experiments conducted on distributed and global differential privacy (GDP) further optimize the system’s efficacy.

AI4ACEIP: A Computing Tool to Identify Food Peptides with High Inhibitory Activity for ACE by Merged Molecular Representation and Rich Intrinsic Sequence Information Based on an Ensemble Learning Strategy.

Hypertension is a common chronic disorder and a major risk factor for cardiovascular diseases. Angiotensin-converting enzyme (ACE) converts angiotensin I to angiotensin II, causing vasoconstriction and raising blood pressure. Pharmacotherapy is the mainstay of traditional hypertension treatment, leading to various negative side effects. Some food-derived peptides can suppress ACE, named ACEIP with fewer undesirable effects. Therefore, it is crucial to seek strong dietary ACEIP to aid in hypertension treatment. In this article, we propose a new model called AI4ACEIP to identify ACEIP. AI4ACEIP uses a novel two-layer stacked ensemble architecture to predict ACEIP relying on integrated view features derived from sequence, large language models, and molecular-based information. The analysis of feature combinations reveals that four selected integrated feature pairs exhibit enhancing performance for identifying ACEIP. For finding meta models with strong abilities to learn information from integrated feature pairs, PowerShap, a feature selection method, is used to select 40 optimal feature and meta model combinations. Compared with seven state-of-the-art methods on the source and clear benchmark data sets, AI4ACEIP significantly outperformed by 8.47 to 20.65% and 5.49 to 14.42% for Matthew's correlation coefficient. In brief, AI4ACEIP is a reliable model for ACEIP prediction and is freely available at https://github.com/abcair/AI4ACEIP.

AdaGIN: Adaptive Graph Interaction Network for Click-Through Rate Prediction

The goal of click-through rate (CTR) prediction in recommender systems is to effectively work with input features. However, existing CTR prediction models face three main issues. First, many models use a basic approach for feature combinations, leading to noise and reduced accuracy. Second, there is no consideration for the varying importance of features in different interaction orders, affecting model performance. Third, current model architectures struggle to capture different interaction signals from various semantic spaces, leading to sub-optimal performance. To address these issues, we propose the Adaptive Graph Interaction Network (AdaGIN) with the Graph Neural Networks-based Feature Interaction Module (GFIM), the Multi-semantic Feature Interaction Module (MFIM), and the Negative Feedback-based Search (NFS) algorithm. GFIM explicitly aggregates information between features and assesses their importance, while MFIM captures information from different semantic spaces. NFS uses negative feedback to optimize model complexity. Experimental results show AdaGIN outperforms existing models on large-scale public benchmark datasets.

Revisiting representation learning of color information: Color medical image segmentation incorporating quaternion

Currently, color medical image segmentation methods commonly extract color and texture features mixed together by default, however, the distribution of color information and texture information is different: color information is represented differently in different color channels of a color image, while the distribution of texture information remains the same. Such a simple and brute-force feature extraction pattern will inevitably result in a partial bias in the model's semantics understanding. In this paper, we decouple the representation learning for color-texture information, and propose a novel network for color medical image segmentation, named CTNet. Specifically, CTNet introduces the Quaternion CNN (QCNN) module to capture the correlation among different color channels of color medical images to generate color features, and uses a designed local-global texture feature integrator (LoG) to mine the textural features from local to global. Moreover, a multi-stage features interaction strategy is proposed to minimize the semantic understanding gap of color and texture features in CTNet, so that they can be subsequently fused to generate a unified and robust feature representation. Comparative experiments on four different color medical image segmentation benchmark datasets show that CTNet strikes an optimal trade-off between segmentation accuracy and computational overhead when compared to current state-of-the-art methods. We also conduct extensive ablation experiments to verify the effectiveness of the proposed components. Our code will be available at https://github.com/Notmezhan/CTNet.

NFSA-DTI: A Novel Drug–Target Interaction Prediction Model Using Neural Fingerprint and Self-Attention Mechanism

Existing deep learning methods have shown outstanding performance in predicting drug–target interactions. However, they still have limitations: (1) the over-reliance on locally extracted features by some single encoders, with insufficient consideration of global features, and (2) the inadequate modeling and learning of local crucial interaction sites in drug–target interaction pairs. In this study, we propose a novel drug–target interaction prediction model called the Neural Fingerprint and Self-Attention Mechanism (NFSA-DTI), which effectively integrates the local information of drug molecules and target sequences with their respective global features. The neural fingerprint method is used in this model to extract global features of drug molecules, while the self-attention mechanism is utilized to enhance CNN’s capability in capturing the long-distance dependencies between the subsequences in the target amino acid sequence. In the feature fusion module, we improve the bilinear attention network by incorporating attention pooling, which enhances the model’s ability to learn local crucial interaction sites in the drug–target pair. The experimental results on three benchmark datasets demonstrated that NFSA-DTI outperformed all baseline models in predictive performance. Furthermore, case studies illustrated that our model could provide valuable insights for drug discovery. Moreover, our model offers molecular-level interpretations.

Two-Stream Modality-Based Deep Learning Approach for Enhanced Two-Person Human Interaction Recognition in Videos

Human interaction recognition (HIR) between two people in videos is a critical field in computer vision and pattern recognition, aimed at identifying and understanding human interaction and actions for applications such as healthcare, surveillance, and human–computer interaction. Despite its significance, video-based HIR faces challenges in achieving satisfactory performance due to the complexity of human actions, variations in motion, different viewpoints, and environmental factors. In the study, we proposed a two-stream deep learning-based HIR system to address these challenges and improve the accuracy and reliability of HIR systems. In the process, two streams extract hierarchical features based on the skeleton and RGB information, respectively. In the first stream, we utilised YOLOv8-Pose for human pose extraction, then extracted features with three stacked LSM modules and enhanced them with a dense layer that is considered the final feature of the first stream. In the second stream, we utilised SAM on the input videos, and after filtering the Segment Anything Model (SAM) feature, we employed integrated LSTM and GRU to extract the long-range dependency feature and then enhanced them with a dense layer that was considered the final feature for the second stream module. Here, SAM was utilised for segmented mesh generation, and ImageNet was used for feature extraction from images or meshes, focusing on extracting relevant features from sequential image data. Moreover, we newly created a custom filter function to enhance computational efficiency and eliminate irrelevant keypoints and mesh components from the dataset. We concatenated the two stream features and produced the final feature that fed into the classification module. The extensive experiment with the two benchmark datasets of the proposed model achieved 96.56% and 96.16% accuracy, respectively. The high-performance accuracy of the proposed model proved its superiority.