Attentional Modulation Research Articles

Advancing skeleton-based human behavior recognition: multi-stream fusion spatiotemporal graph convolutional networks

In the realm of daily human interactions, a rich tapestry of behaviors and actions is observed, encompassing a wealth of informative cues. In the era of burgeoning big data, extensive repositories of images and videos have risen to prominence as the primary conduits for disseminating information. Grasping the intricacies of human behaviors depicted within these multimedia contexts has evolved into a pivotal quandary within the domain of computer vision. The technology of behavior recognition finds its practical application across domains such as human-computer interaction, intelligent surveillance, and anomaly detection, exhibiting a robust blend of pragmatic utility and scholarly significance. The present study introduces an innovative human body behavior recognition framework anchored in skeleton sequences and multi-stream fused spatiotemporal graph convolutional networks. Developed upon the foundation of graph convolutional networks, this method encompasses three pivotal refinements tailored to ameliorate extant challenges. First and foremost, in response to the complex task of capturing distant interdependencies among nodes within graph convolutional networks, we incorporate a spatial attention module. This module adeptly encapsulates long-term node interdependencies via precision-laden positional information, thus engendering interconnections that span diverse temporal and spatial contexts. Subsequently, to elevate the discernment of channel information within the network and to optimize the allocation of attention across distinct channels, we introduce a channel attention mechanism. This augmentation fortifies the discernment of motion-related features. Lastly, confronting the lacuna of information gaps prevalent within single-stream data, we deploy a multi-stream fusion methodology to fortify model outputs, ultimately fostering more precise prognostications concerning action classifications. Empirical results bear testament to the efficacy of the proposed multi-stream fused spatiotemporal graph convolutional network paradigm for skeleton-centric behavior recognition, evincing a pinnacle recognition accuracy of 96.0% on the expansive NTU-RGB+D skeleton dataset, alongside a zenithal accuracy of 37.3% on the Kinetics-Skeleton dataset—emanating from RGB data and furthered through pose estimation.

Vessel re-identification by a hierarchical perceptual aggregation network with inclination-aware attention

Abstract Vessel re-identification (re-ID) is a crucial task in maritime supervision, enhancing maritime safety and improving the maritime situational awareness system. However, distinct from land-based scenarios involving vehicles or pedestrians, vessels, as enormous rigid bodies situated in the dynamic marine environment, face unique challenges such as significant variations in the scale of discriminative features and unpredictable sway. Furthermore, there is a limited number of publicly available datasets for vessel re-ID in complex backgrounds. In this paper, to overcome these challenges, a novel Hierarchical Perceptual Aggregation Network with Inclination-Aware Attention (HPAN-IAA) is proposed. HPAN-IAA comprises two main modules: the Hierarchical Perceptual Aggregation Block (HPAB) and the Inclination-Aware Attention Block (IAAB). Specifically, in HPAB, a hierarchical perceptual function is introduced to decompose visual information of vessels into discriminative features at multiple levels. These feature maps with different levels of detail from diverse network layers are then fused together by concatenation, resulting in a comprehensive feature representation that effectively integrates information across various scales. Conversely, to address the irregular variations and random omissions in discriminative feature distribution caused by unpredictable vessel sway, in IAAB, the Channel Collaborative Attention Module and the Pyramidal Spatial Attention Module are designed to adaptively extract potential discriminative features within each channel and spatial dimension, enhancing model’s ability in effectively extracting and utilizing irregularly changing discriminative features. Moreover, we propose a novel vessel re-ID dataset—VesselReID-2258. Extensive experiments conducted on VesselReID-2258 and the publicly available dataset VesselReID demonstrate that HPAN-IAA outperforms the current state-of-the-art methods,achieving superior performance with mean Average Precision scores of 0.861 and 0.823.

An Efficient Group Convolution and Feature Fusion Method for Weed Detection

Weed detection is a crucial step in achieving intelligent weeding for vegetables. Currently, research on vegetable weed detection technology is relatively limited, and existing detection methods still face challenges due to complex natural conditions, resulting in low detection accuracy and efficiency. This paper proposes the YOLOv8-EGC-Fusion (YEF) model, an enhancement based on the YOLOv8 model, to address these challenges. This model introduces plug-and-play modules: (1) The Efficient Group Convolution (EGC) module leverages convolution kernels of various sizes combined with group convolution techniques to significantly reduce computational cost. Integrating this EGC module with the C2f module creates the C2f-EGC module, strengthening the model’s capacity to grasp local contextual information. (2) The Group Context Anchor Attention (GCAA) module strengthens the model’s capacity to capture long-range contextual information, contributing to improved feature comprehension. (3) The GCAA-Fusion module effectively merges multi-scale features, addressing shallow feature loss and preserving critical information. Leveraging GCAA-Fusion and PAFPN, we developed an Adaptive Feature Fusion (AFF) feature pyramid structure that amplifies the model’s feature extraction capabilities. To ensure effective evaluation, we collected a diverse dataset of weed images from various vegetable fields. A series of comparative experiments was conducted to verify the detection effectiveness of the YEF model. The results show that the YEF model outperforms the original YOLOv8 model, Faster R-CNN, RetinaNet, TOOD, RTMDet, and YOLOv5 in detection performance. The detection metrics achieved by the YEF model are as follows: precision of 0.904, recall of 0.88, F1 score of 0.891, and mAP0.5 of 0.929. In conclusion, the YEF model demonstrates high detection accuracy for vegetable and weed identification, meeting the requirements for precise detection.

Grape Target Detection Method in Orchard Environment Based on Improved YOLOv7

In response to the poor detection performance of grapes in orchards caused by issues such as leaf occlusion and fruit overlap, this study proposes an improved grape detection method named YOLOv7-MCSF based on the You Only Look Once v7 (YOLOv7) framework. Firstly, the original backbone network is replaced with MobileOne to achieve a lightweight improvement of the model, thereby reducing the number of parameters. In addition, a Channel Attention (CA) module was added to the neck network to reduce interference from the orchard background and to accelerate the inference speed. Secondly, the SPPFCSPC pyramid pooling is embedded to enhance the speed of image feature fusion while maintaining a consistent receptive field. Finally, the Focal-EIoU loss function is employed to optimize the regression prediction boxes, accelerating their convergence and improving regression accuracy. The experimental results indicate that, compared to the original YOLOv7 model, the YOLOv7-MCSF model achieves a 26.9% reduction in weight, an increase in frame rate of 21.57 f/s, and improvements in precision, recall, and mAP of 2.4%, 1.8%, and 3.5%, respectively. The improved model can efficiently and in real-time identify grape clusters, providing technical support for the deployment of mobile devices and embedded grape detection systems in orchard environments.

CINet: A Constraint- and Interaction-Based Network for Remote Sensing Change Detection.

Remote sensing change detection (RSCD), which utilizes dual-temporal images to predict change locations, plays an essential role in long-term Earth observation missions. Although many deep learning based RSCD models perform well, challenges remain in effectively extracting change information between dual-temporal images and fully leveraging interactions between their feature maps. To address these challenges, a constraint- and interaction-based network (CINet) for RSCD is proposed. Firstly, a constraint mechanism is introduced that uses labels to control the backbone of the network during training to enhance the consistency of the unchanged regions and the differences between the changed regions in the extracted dual-temporal images. Secondly, a Cross-Spatial-Channel Attention (CSCA) module is proposed, which realizes the interaction of valid information between dual-temporal feature maps through channels and spatial attention and uses multi-level information for more accurate detection. The verification results show that compared with advanced parallel methods, CINet achieved the highest F1 scores on all six widely used remote sensing benchmark datasets, reaching a maximum of 92.00 (on LEVIR-CD dataset). These results highlight the excellent ability of CINet to detect changes in various practical scenarios, demonstrating the effectiveness and feasibility of the proposed constraint enhancement and CSCA module.

HandFI: Multilevel Interacting Hand Reconstruction Based on Multilevel Feature Fusion in RGB Images

Interacting hand reconstruction presents significant opportunities in various applications. However, it currently faces challenges such as the difficulty in distinguishing the features of both hands, misalignment of hand meshes with input images, and modeling the complex spatial relationships between interacting hands. In this paper, we propose a multilevel feature fusion interactive network for hand reconstruction (HandFI). Within this network, the hand feature separation module utilizes attentional mechanisms and positional coding to distinguish between left-hand and right-hand features while maintaining the spatial relationship of the features. The hand fusion and attention module promotes the alignment of hand vertices with the image by integrating multi-scale hand features while introducing cross-attention to help determine the complex spatial relationships between interacting hands, thereby enhancing the accuracy of two-hand reconstruction. We evaluated our method with existing approaches using the InterHand 2.6M, RGB2Hands, and EgoHands datasets. Extensive experimental results demonstrated that our method outperformed other representative methods, with performance metrics of 9.38 mm for the MPJPE and 9.61 mm for the MPVPE. Additionally, the results obtained in real-world scenes further validated the generalization capability of our method.

GFN: A Garbage Classification Fusion Network Incorporating Multiple Attention Mechanisms

With the increasing global attention to environmental protection and the sustainable use of resources, waste classification has become a critical issue that needs urgent resolution in social development. Compared with the traditional manual waste classification methods, deep learning-based waste classification systems offer significant advantages. This paper proposes an innovative deep learning framework, Garbage FusionNet (GFN), aimed at tackling the waste classification challenge. GFN enhances classification performance by integrating the local feature extraction strengths of ResNet with the global information processing capabilities of the Vision Transformer (ViT). Furthermore, GFN incorporates the Pyramid Pooling Module (PPM) and the Convolutional Block Attention Module (CBAM), which collectively improve multi-scale feature extraction and emphasize critical features, thereby increasing the model’s robustness and accuracy. The experimental results on the Garbage Dataset and Trashnet demonstrate that GFN achieves superior performance compared with other comparison models.

Bidirectional interaction directional variance attention model based on increased-transformer for thyroid nodule classification

Malignant thyroid nodules are closely linked to cancer, making the precise classification of thyroid nodules into benign and malignant categories highly significant. However, the subtle differences in contour between benign and malignant thyroid nodules, combined with the texture features obscured by the inherent noise in ultrasound images, often result in low classification accuracy in most models. To address this, we propose a Bidirectional Interaction Directional Variance Attention Model based on Increased-Transformer, named IFormer-DVNet. This paper proposes the Increased-Transformer, which enables global feature modeling of feature maps extracted by the Convolutional Feature Extraction Module (CFEM). This design maximally alleviates noise interference in ultrasound images. The Bidirectional Interaction Directional Variance Attention module (BIDVA) dynamically calculates attention weights using the variance of input tensors along both vertical and horizontal directions. This allows the model to focus more effectively on regions with rich information in the image. The vertical and horizontal features are interactively combined to enhance the model's representational capability. During the model training process, we designed a Multi-Dimensional Loss function (MD Loss) to stretch the boundary distance between different classes and reduce the distance between samples of the same class. Additionally, the MD Loss function helps mitigate issues related to class imbalance in the dataset. We evaluated our network model using the public TNCD dataset and a private dataset. The results show that our network achieved an accuracy of 76.55% on the TNCD dataset and 93.02% on the private dataset. Compared to other state-of-the-art classification networks, our model outperformed them across all evaluation metrics.

High-Frequency Workpiece Image Recognition Model Based on Hybrid Attention Mechanism

High-frequency workpieces are specialized items characterized by complex internal textures and minimal variance in properties. Under intricate lighting conditions, existing mainstream image recognition models struggle with low precision when applied to the identification of high-frequency workpiece images. This paper introduces a high-frequency workpiece image recognition model based on a hybrid attention mechanism, HAEN. Initially, the high-frequency workpiece dataset is enhanced through geometric transformations, random noise, and random lighting adjustments to augment the model’s generalization capabilities. Subsequently, lightweight convolution, including one-dimensional and dilated convolutions, is employed to enhance convolutional attention and reduce the model’s parameter count, extracting original image features with robustness to strong lighting and mitigating the impact of lighting conditions on recognition outcomes. Finally, lightweight re-estimation attention modules are integrated at various model levels to reassess spatial information in feature maps and enhance the model’s representation of depth channel features. Experimental results demonstrate that the proposed model effectively extracts features from high-frequency workpiece images under complex lighting, outperforming existing models in image classification tasks with a precision of 97.23%.

Research on lightweight detection methods for the golden snub-nosed monkey based on YOLOv5n

To enable rapid detection of golden snub-nosed monkeys in complex environments, reduce the human costs associated with tracking and observing these monkeys, and accelerate the development of intelligent forest monitoring, we propose the PCB-YOLOv5n-prune model. This model is designed for lightweight devices and is based on channel pruning and module reconstruction. First, we constructed a dataset that combines annotations of the golden snub-nosed monkey's face and body, with some data converted to grayscale. We mixed and expanded five data styles to decrease reliance on color and enhance the informational content. Next, we applied the Sparse Group Lasso selection operator method to slim down the YOLOv5n primitive model for golden snub-nosed monkey detection, improving the detection speed of the underlying network. We then introduced a lightweight convolutional module, PConv, to create the improved residual branching module, CPB, which reduces model computation and memory access. Additionally, we incorporated a lightweight attention module, ECA, to adaptively weight channel features, facilitating local cross-channel information interaction. Finally, we integrated the ByteTrack multi-target tracking algorithm to enable continuous tracking of golden snub-nosed monkeys and visualize detection results. Experimental results demonstrate that the PCB-YOLOv5n-prune model reduces the number of parameters, floating point operations, and model weight by 61 %, 56 %, and 55 %, respectively, compared to the original YOLOv5n model, while significantly improving detection speed.

Enhancing Retinal Vessel Segmentation with a Simplified nnU-Net Model and CBAM

To better diagnose and monitor ocular diseases, segmentation of blood vessels on the retina is crucial, but manual annotation is labor-intensive and error-prone. To deal with these problems, many automatic segmentation algorithms have already been developed, with deep learning techniques, leading the advancements. This study aims to enhance retinal vessel segmentation by refining the no-new-U-Net (nnU-Net) model through two key modifications: the integration of the Convolutional Block Attention Module (CBAM) attention blocks and the replacement of the original Stage 1 nnU-Net with a simplified Residual Neural Network (ResNet)-like network. The incorporation of CBAM improves feature extraction by focusing on the most relevant features, while the ResNet-like architecture simplifies the network structure, making it more efficient. Experimental results on the Retina Blood Vessel Dataset reveal that the modified nnU-Net model significantly outperforms the original in terms of both Dice score and prediction time. Specifically, it achieves superior accuracy in identifying fine blood vessels and demonstrates reduced memory usage, showcasing its potential for delivering more precise and efficient retinal vessel segmentation.

A mutual inclusion mechanism for precise boundary segmentation in medical images.

Accurate image segmentation is crucial in medical imaging for quantifying diseases, assessing prognosis, and evaluating treatment outcomes. However, existing methods often fall short in integrating global and local features in a meaningful way, failing to give sufficient attention to abnormal regions and boundary details in medical images. These limitations hinder the effectiveness of segmentation techniques in clinical settings. To address these issues, we propose a novel deep learning-based approach, MIPC-Net, designed for precise boundary segmentation in medical images. Our approach, inspired by radiologists' working patterns, introduces two distinct modules: 1. Mutual Inclusion of Position and Channel Attention (MIPC) Module: To improve boundary segmentation precision, we present the MIPC module. This module enhances the focus on channel information while extracting position features and vice versa, effectively enhancing the segmentation of boundaries in medical images. 2. Skip-Residue Module: To optimize the restoration of medical images, we introduce Skip-Residue, a global residual connection. This module improves the integration of the encoder and decoder by filtering out irrelevant information and recovering the most crucial information lost during the feature extraction process. We evaluate the performance of MIPC-Net on three publicly accessible datasets: Synapse, ISIC2018-Task, and Segpc. The evaluation uses metrics such as the Dice coefficient (DSC) and Hausdorff Distance (HD). Our ablation study confirms that each module contributes to the overall improvement of segmentation quality. Notably, with the integration of both modules, our model outperforms state-of-the-art methods across all metrics. Specifically, MIPC-Net achieves a 2.23 mm reduction in Hausdorff Distance on the Synapse dataset, highlighting the model's enhanced capability for precise image boundary segmentation. The introduction of the novel MIPC and Skip-Residue modules significantly improves feature extraction accuracy, leading to better boundary recognition in medical image segmentation tasks. Our approach demonstrates substantial improvements over existing methods, as evidenced by the results on benchmark datasets.

A Lightweight Small Target Detection Algorithm for UAV Platforms

The targets in the aerial view of UAVs are small, scenes are complex, and background noise is strong. Additionally, the low computational capability of UAVs is challenged when trying to meet the requirements of large neural networks. Therefore, a lightweight object detection algorithm tailored for UAV platforms, called RSG-YOLO, is proposed. The algorithm introduces an attention module constructed with receptive field attention and coordinate attention, which helps reduce background noise interference while improving long-range information dependency. It also introduces and refines a fine-grained downsampling structure to minimize the loss of target information during the downsampling process. A general efficient layer aggregation network enhances the base feature extraction module, improving gradient flow information. Additionally, a detection layer rich in small target information is added, while redundant large object detection layers are removed, achieving a lightweight design while enhancing detection accuracy. Experimental results show that, compared to the baseline algorithm, the improved algorithm increases the P, R, mAP@0.5, and mAP@0.5:0.95 by 6.9%, 7.2%, 8.4%, 5.8%, respectively, on the VisDrone 2019 dataset, and by 5.7%, 9%, 9.3%, 3.6%, respectively, on the TinyPerson dataset, while reducing the number of parameters by 23.3%. This significantly enhances the model’s detection performance and robustness, making it highly suitable for object detection tasks on low-computing-power UAV platforms.

Classification of arteriovenous fistula sounds using a convolutional block attention module and long short-term memory neural network.

The assessment of vascular accessibility in patients undergoing hemodialysis is predominantly reliant on manual inspection, a method that is associated with several limitations. In this study, we propose an alternative approach by recording the acoustic signals produced by the arteriovenous fistula (AVF) and employing deep learning techniques to analyze these sounds as an objective complement to traditional AVF evaluation methods. Auscultation sounds were collected from 800 patients, with each recording lasting between 24 and 30s. Features were extracted by combining Mel-Frequency Cepstral Coefficients with Mel-Spectrogram data, generating a novel set of feature parameters. These parameters were subsequently used as input to a model that integrates the Convolutional Block Attention Module and a Long Short-Term Memory neural network, designed to classify the severity of AVF stenosis based on two sound categories (normal and abnormal). The experimental results demonstrate that the CBAM-LSTM model achieves an Area Under the Receiver Operating Characteristic curve of 99%, Precision of 99%, Recall of 97%, and F1 Score of 98%. Comparative analysis with other models, including VGG, Bi-LSTM, DenseNet121, and ResNet50, indicates that the proposed CBAM-LSTM model outperforms these alternatives in classifying AVF stenosis severity. These findings suggest the potential of the CBAM-LSTM model as a reliable tool for monitoring AVF maturation.

An automatic coke optical texture recognition method based on semantic segmentation model

Article An automatic coke optical texture recognition method based on semantic segmentation model Xialin Wang 1, Xiu Kan 1,*, Zhen Zhang 1, and Weizhou Sun 2 1 Shanghai University of Engineering Science, Shanghai 201620, China 2 Anhui University of Technology, Anhui 243002, China * Correspondence: xiu.kan@sues.edu.cn Received: 1 October 2023 Accepted: 30 October 2024 Published: 24 December 2024 Abstract: To solve the segmentation problem of coke optical texture in coke photomicrograph, a semantic segmentation method is proposed based on the multi-scale feature fusion and attention strategy in this paper. The multi-scale module is com­bined with convolutional block attention module (CBAM) to design a feature extraction strategy, and the Coke-Net network model is established to extract the coke optical texture from coke photomicrographs. The relationship between pixels is fully considered to refine the segmentation edge, and the extraction results with spatial consistency are output to complete the precise segmentation of the coke optical structure. The ablation experiment and contrast experiment are used to demonstrate the effectiveness of the proposed method in coke optical texture extraction.

Research on belt deviation diagnosis of belt conveyors based on deep learning

Abstract Due to the slow detection speed, low accuracy, and small detection range of existing methods for detecting belt deviation in belt conveyors, this paper introduces an enhanced ultra-fast lane detection (UFLD) algorithm that leverages deep learning for the detection of belt deviation. Based on the UFLD algorithm, a variable step-size row anchor division method is proposed, and the simple parameter-free attention module is added to the network to enhance the network model’s focus on edge information of conveyor belts. Furthermore, improvements are made to the convolution operations in the ResNet-18 Stem and the downsampling operations in the residual modules, thereby enhancing the network’s ability to recognize the edges of conveyor belts. Based on the established experimental platform, a high-definition camera equipped with a track-type inspection robot was used to inspect the entire belt conveyor, covering the whole of the transmission line. The conveyor belt operation datasets collected under various working conditions were used to train and comparatively study the Hough Transform, DHT, YOLOv5, YOLOv8, LaneNet, SAD, and UFLD algorithms. The experimental outcomes demonstrate that the algorithm introduced in this article outperforms the other algorithms, achieving an F1-measure of 90.31%, an accuracy rate of 94.19 %, and a detection speed of 71 frames per second, meeting the real-time diagnostic needs for belt misalignment in the coal mining industry.

A Weakly Supervised Crowd Counting Method via Combining CNN and Transformer

During the past five years, there has been an increasing trend of weakly supervised crowd counting methods being developed since such methods just rely on count-level annotations and avoid a laborious labeling process. But, the existing weakly supervised methods usually fail to achieve comparable counting performance to the fully supervised methods. To improve the accuracy of crowd counting tasks, we propose to combine the convolutional neural network (CNN) and Transformer frameworks. Since CNN focuses on capturing local detail information and Transformer can effectively extract global context information, we believe that the combination of CNN and Transformer could learn more efficient feature representations for crowd images. Our proposed framework is named CrowdCCT (Crowd Counting via CNN and Transformer), and it is composed of a CNN feature extraction part, a Transformer feature extraction part, and a counting regression part. In the CNN part, we utilize DenseNet121 to learn rich semantic features with its inherent dense connection structure. In the Transformer part, we introduce two attention modules, Multi-Scale Dilated Attention (MSDA) and Location-Enhanced Attention (LEA), working together to extract more expressive features. The output features are then fed into the regression part to generate the predicted counting results. Experiments on four crowd counting benchmark datasets demonstrate that our proposed CrowdCCT can achieve superior performance. Also, the experimental results validate the feasibility and effectiveness of combining CNN and Transformer for weakly supervised counting tasks. Our work could be expected to promote further combination research on CNN and Transformer.

ETiSeg-Net: edge-aware self attention to enhance tissue segmentation in histopathological images

AbstractDigital pathology employing Whole Slide Images (WSIs) plays a pivotal role in cancer detection. Nevertheless, the manual examination of WSIs for the identification of various tissue regions presents formidable challenges due to its labor-intensive nature and subjective interpretation. Convolutional Neural Network (CNN) based semantic segmentation algorithms have emerged as valuable tools for assisting in this task by automating ROI delineation. The incorporation of attention modules and carefully designed loss functions has shown promise in further augmenting the performance of these algorithms. However, there exists a notable gap in research regarding the utilization of attention modules specifically for tissue segmentation, thereby constraining our comprehension and application of these modules in this context. This study introduces ETiSeg-Net (Edge-aware self attention to enhance Tissue Segmentation), a CNN-based semantic segmentation model that uses a novel edge-based attention module to achieve effective delineation of class boundaries. In addition, an innovative iterative training strategy is devised to efficiently optimize the model parameters. The study also conducts a comprehensive investigation into the impact of attention modules and loss functions on the efficacy of semantic segmentation models. Qualitative and quantitative evaluations of these semantic segmentation models are conducted using publicly available datasets. The findings underscore the potential of attention modules in enhancing the accuracy and effectiveness of tissue semantic segmentation.

Smoke Detection Transformer: An Improved Real-Time Detection Transformer Smoke Detection Model for Early Fire Warning

As one of the important features in the early stage of fires, the detection of smoke can provide a faster early warning of a fire, thus suppressing the spread of the fire in time. However, the features of smoke are not apparent; the shape of smoke is not fixed, and it is easy to be confused with the background outdoors, which leads to difficulties in detecting smoke. Therefore, this study proposes a model called Smoke Detection Transformer (Smoke-DETR) for smoke detection, which is based on a Real-Time Detection Transformer (RT-DETR). Considering the limited computational resources of smoke detection devices, Enhanced Channel-wise Partial Convolution (ECPConv) is introduced to reduce the number of parameters and the amount of computation. This approach improves Partial Convolution (PConv) by using a selection strategy that selects channels containing more information for each convolution, thereby increasing the network’s ability to learn smoke features. To cope with smoke images with inconspicuous features and irregular shapes, the Efficient Multi-Scale Attention (EMA) module is used to strengthen the feature extraction capability of the backbone network. Additionally, in order to overcome the problem of smoke being easily confused with the background, the Multi-Scale Foreground-Focus Fusion Pyramid Network (MFFPN) is designed to strengthen the model’s attention to the foreground of images, which improves the accuracy of detection in situations where smoke is not well differentiated from the background. Experimental results demonstrate that Smoke-DETR has achieved significant improvements in smoke detection. In the self-building dataset, compared to RT-DETR, Smoke-DETR achieves a Precision that has reached 86.2%, marking an increase of 3.6 percentage points. Similarly, Recall has achieved 80%, showing an improvement of 3.6 percentage points. In terms of mAP50, it has reached 86.2%, with a 3.8 percentage point increase. Furthermore, mAP50 has reached 53.9%, representing a 3.6 percentage point increase.

Parallel Spatio-Temporal SlowFast Model for Behavior Detection in AI-Enhanced Classrooms: A Vision-Based Approach

Education is pivotal in shaping future generations, with artificial intelligence (AI) technologies revolutionizing conventional classroom approaches. Understanding student behavior is essential for improving teaching quality and learning outcomes. However, in large classrooms, monitoring each student becomes a challenge. Therefore, we propose an intelligent framework utilizing deep learning to create a vision-based classroom capable of autonomously analyzing student behavior and attention. Our approach, Parallel Spatio-Temporal SlowFast (PST-SlowFast), combines spatial and temporal attention mechanisms to enhance behavior detection accuracy. The key contributions of the PST-SlowFast model lie in its ability to handle spatial and temporal features in videos effectively, facilitated by integrating both spatial and temporal attention modules. By leveraging these attention mechanisms, the model captures relevant spatial and temporal features, enhancing performance in behavior detection tasks. The experimental results demonstrate the effectiveness of the PST-SlowFast model in recognizing behaviors such as reading, writing, and hand-raising. The PST-SlowFast model achieves an average mAP@50 of 88.8%, which is an improvement of 14% compared to state-of-the-art models such as YOLOv5x and YOLOv8x. These findings indicate the promise of the PST-SlowFast model for real-world applications in educational settings.