Representational Capacity Research Articles

Cross-modal learning representation using new margin combination for speech recognition task

Cross-modal retrieval aims to elucidate the fusion of information, mimic human learning, and advance the field. The main challenge in cross-modal matching is to build a shared subspace reflecting semantic proximity. Previous works fail to capture asymmetric relevance by adopting symmetric similarity computations. To overcome these shortcomings, an efficient approach called quaternion representation learning (QRL) is introduced for better cross-modal matching. Thus, a better representation of the shared semantics is offered by virtue of its richer representation capacity of the quaternionic space and its strong expressive power. Transfer learning is a crucial aspect in this context. By leveraging pre-trained models, the knowledge gained from one task or domain can be effectively transferred to another, allowing for improved performance and generalization. In this study, transfer learning is employed to enhance the cross-modal retrieval system. Specifically, a pre-trained ResNet-512 model is utilized in conjunction with the proposed total margin (TM) loss function, which combines the QRL approach with the novel adaptive mean margin (AMM) methodology. The TM loss function, coupled with the pre-trained ResNet-512 model, is evaluated on the Audio-Visual Arabic Speech Database (AVAS) and the Arabic Visual Speech Database (AVSD), along with other audio-visual datasets. Experimental results demonstrate the effectiveness of the TM loss function in consistently improving performance on both databases. The recall scores (R@k) and mean average precision (mAP) values achieved on the AVAS Database are as follows: R@1: 42.1±0.7, R@2: 70.2±0.1, R@5: 78.5±1.0, and mAP: 53.0±1.1. Similarly, on the AVSD Database, the results are R@1: 41.7±0.3, R@2: 69.2±1.1, R@5: 78.0±0.3, and mAP: 52.7±0.5. By incorporating transfer learning and the TM loss function into the cross-modal retrieval framework, this study demonstrates the potential for improving clustering efficiency and enhancing visual and speech understanding. The combination of pre-trained models and the TM loss function offers a promising avenue for advancing crossmodal matching techniques and achieving state-of-the-art performance.

Developmental origin of a language-cognition interface in infants: Gateway to advancing core knowledge?

Spelke's sweeping proposal requires greater precision in specifying the place of language in early cognition. We now know by 3 months of age, infants have already begun to forge a link between language and core cognition. This precocious link, which unfolds dynamically over development, may indeed offer an entry point for acquiring higher-order, abstract conceptual and representational capacities.

Interval type-2 fuzzy stochastic configuration networks for soft sensor modeling of industrial processes

Soft sensors have been widely applied to predict key variables that are difficult to measure for industrial process modeling. In this paper, a novel randomized interval type-2 fuzzy neural network with parallel learning, called IT2F-PSCN, is presented for soft sensor modeling of industrial processes. It trains the upper and lower bounds of the interval type-2 fuzzy logic system separately to facilitate type reduction, thereby integrating the fuzzy logic system with stochastic configuration networks. To achieve the appropriate structure and parameters of the model, we develop a two-phase training scheme. In the first phase, a sparse rule interpolation method with stochastic configuration is applied to generate new fuzzy rules. In the second phase, the hidden layer is constructed through parallel stochastic configuration to enhance the nonlinear representational capacity. The validity of IT2F-PSCN is confirmed by a series of experiments, including four benchmark data modelings, simulation on the Tennessee Eastman process, and soft sensor modeling for the slurry grade of the first rougher in a zinc flotation process. The experimental results indicate that the proposed IT2F-PSCN performs favorably compared with other methods.

Self-supervised Gaussian Restricted Boltzmann Machine via joint contrastive representation and contrastive divergence

In this paper, we propose a novel self-supervised Gaussian Restricted Boltzmann Machine with contrastive learning (CL-GRBM), which fuses contrastive representation learning and contrastive divergence to optimize and enhance the representation of GRBM. Built upon the concept of contrastive representation learning, CL-GRBM aims to enhance the representation capacity of the GRBM. Firstly, a pair of positive samples are constructed by one times Gibbs sampling with the original data. Then, the contrastive loss is used to pull the positive samples closer and push other samples further apart, making similar representations closer and different representations farther apart. In summary, during the training process of CL-GRBM using the CD algorithm, the objective is to align the sampling probability distribution of the visible layer in CL-GRBM as closely as possible with the empirical distribution of the original data. In the feature space of the sampling probabilities in the hidden layer, the distance between positive samples is minimized to capture the intrinsic structure of the data. According to the experimental verification, the proposed CL-GRBM shows better performance than contrastive models. As a shallow self-supervised model, it has even better performance than some excellent deep self-supervised models.

Capacity bounds for hyperbolic neural network representations of latent tree structures

We study the representation capacity of deep hyperbolic neural networks (HNNs) with a ReLU activation function. We establish the first proof that HNNs can ɛ-isometrically embed any finite weighted tree into a hyperbolic space of dimension d at least equal to 2 with prescribed sectional curvature κ<0, for any ɛ>1 (where ɛ=1 being optimal). We establish rigorous upper bounds for the network complexity on an HNN implementing the embedding. We find that the network complexity of HNN implementing the graph representation is independent of the representation fidelity/distortion. We contrast this result against our lower bounds on distortion which any ReLU multi-layer perceptron (MLP) must exert when embedding a tree with L>2d leaves into a d-dimensional Euclidean space, which we show at least Ω(L1/d); independently of the depth, width, and (possibly discontinuous) activation function defining the MLP.

FEDM: a convolutional neural network based fertilised egg detection model

ABSTRACT 1. The production of goose eggs holds significant economic value on a global scale and the quality of fertilised eggs is crucial for the successful hatching and sustained development of the poultry industry. Developing a low-cost fertilised egg identification system that is suitable for large-scale testing is of great significance. However, existing methods are expensive and have high environmental detection requirements, which limit their promotion. 2. To address this issue, an improved object detection model called FEDM based on YOLOv5 is proposed, which has been shown to be outstanding among nine models. The main network of YOLOv5 is enhanced with the SENet attention mechanism to improve the feature selection capability. The C3_DCNv3 is introduced to enhance the detection ability of blood vessels in the fertilised eggs. The application of Dyhead significantly improved the representation capacity of the object detection head without any computational overhead. The loss function is replaced with MPDIoU to simplify the calculation process. 3. Experimental results from the augmented dataset showed that the average precision of the FEDM reached 96.7%, which is a 5.5% improvement compared to the YOLOv5s model. FEDM exhibited better detection performance on eggs from different shooting angles than the YOLOv5 algorithm and achieves high detection speed. 4. The FEDM secured significant advancement on the detection rate of the fourth day fertilised egg compared to the YOLOv5 algorithm. Based on this result, savings and space utilisation can be made, which has practical application value.

Efficient learning in spiking neural networks

Spiking neural networks (SNNs) are a large class of neural model distinct from ‘classical’ continuous-valued networks such as multilayer perceptrons (MLPs). With event-driven dynamics and a continuous-time model in contrast to the discrete-time model of their classical counterparts, they offer interesting advantages in representational capacity and energy consumption. Spiking networks may also be more biologically plausible, offering more insights into neuroscience. However, developing models of learning for SNNs has historically proven challenging: as discrete-time systems, their dynamics are much more complex and they cannot benefit from the strong theoretical developments in MLPs such as convergence proofs and optimal gradient descent. Nor do they gain automatically from algorithmic improvements that have produced efficient matrix inversion and batch training methods. Most of the existing research has focused on the most well-studied learning mechanism in SNNs, spike-timing-dependent plasticity (STDP), and although there has been progress, there are also notable pathologies that have often been solved with a variety of ad-hoc techniques. While efforts have been made to map SNNs to classical convolutional neural networks (CNNs), these have not yet shown any decisive efficiency advantage over conventional CNNs. More promising research directions lie in the realm of pure spiking learning models that exploit the inherent temporal dynamics (and often leverage recurrency). Metrics are needed; one possibility would be a measure of total energy cost per unit reduction in error. This tutorial overview looks at existing techniques for learning in SNNs and offers some thoughts for future directions.

The clinical and imaging data fusion model for single-period cerebral CTA collateral circulation assessment.

The Chinese population ranks among the highest globally in terms of stroke prevalence. In the clinical diagnostic process, radiologists utilize computed tomography angiography (CTA) images for diagnosis, enabling a precise assessment of collateral circulation in the brains of stroke patients. Recent studies frequently combine imaging and machine learning methods to develop computer-aided diagnostic algorithms. However, in studies concerning collateral circulation assessment, the extracted imaging features are primarily composed of manually designed statistical features, which exhibit significant limitations in their representational capacity. Accurately assessing collateral circulation using image features in brain CTA images still presents challenges. To tackle this issue, considering the scarcity of publicly accessible medical datasets, we combined clinical data with imaging data to establish a dataset named RadiomicsClinicCTA. Moreover, we devised two collateral circulation assessment models to exploit the synergistic potential of patients' clinical information and imaging data for a more accurate assessment of collateral circulation: data-level fusion and feature-level fusion. To remove redundant features from the dataset, we employed Levene's test and T-test methods for feature pre-screening. Subsequently, we performed feature dimensionality reduction using the LASSO and random forest algorithms and trained classification models with various machine learning algorithms on the data-level fusion dataset after feature engineering. Experimental results on the RadiomicsClinicCTA dataset demonstrate that the optimized data-level fusion model achieves an accuracy and AUC value exceeding 86% . Subsequently, we trained and assessed the performance of the feature-level fusion classification model. The results indicate the feature-level fusion classification model outperforms the optimized data-level fusion model. Comparative experiments show that the fused dataset better differentiates between good and bad side branch features relative to the pure radiomics dataset. Our study underscores the efficacy of integrating clinical and imaging data through fusion models, significantly enhancing the accuracy of collateral circulation assessment in stroke patients.

Microalgae detection based on improved YOLOv5

AbstractAccurate detection of algae in microscopic image plays a crucial role in water quality monitoring. However, the existing object detection methods still face challenges in accurately detecting different categories of algae in microscopic image. In order to improve the accuracy, an improved YOLOv5s model is proposed for microalgae detection, by combining a Receptive Field Enhancement (RFE) module, the Wise‐IoU v3 dynamic non‐monotonic focal loss function, and a Dynamic head (Dyhead), which is termed receptive field enhancement wise‐IoU dyhead (RWD)‐you only look once (YOLO). Firstly, to detect microalgae of various scales, the Bottleneck in the C3 module of YOLOv5s is replaced with a more reasonable RFE module. Secondly, Wise‐IoU v3 is applied to enhance detection accuracy by assigning varying weights between high‐quality and low‐quality images. Finally, Dyhead is introduced to enhance the representation capacity of the detection head by integrating three attention mechanisms: scale awareness, spatial awareness, and task awareness. The proposed RWD‐YOLO model significantly enhances the accuracy of algae detection in microscopic image. Specifically, the experimental results on the microalgae dataset show that the RWD‐YOLO achieves an mAP@0.5 of 93.2% and an mAP@0.5:0.95 of 65.1%. Compared to the original YOLOv5s, mAP@0.5 and mAP@0.5:0.95 are improved by 3.7% and 5.7%, respectively.

Misalignment-Resistant Deep Unfolding Network for multi-modal MRI super-resolution and reconstruction

Multi-modal Magnetic Resonance Imaging (MRI) super-resolution (SR) and reconstruction aims to obtain a high-quality target image from corresponding sparsely sampled signals under the guidance of a reference image. However, existing techniques typically assume that the input multi-modal MR images are well aligned, which is challenging to achieve in clinical practice. This naive assumption has made their algorithms vulnerable to misalignment scenarios. Moreover, they often neglect many non-local common characteristics within and between modalities. In this work, we proposed a MisAlignment-Resistant Deep Unfolding Network (MAR-DUN) embedded in the tailored gradient descent module (GDM) and proximal mapping module (PMM) for multi-modal MRI SR and reconstruction. In the GDM, we employ an adaptive step-size sub-network (ASS-Net) to enhance the texture representation capacity of the proposed MAR-DUN. Furthermore, in the PMM, we propose a cross-modality non-local module (CNLM) featuring the inverse deformation layer (IDL). The IDL aligns features between the target and reference images by adaptively learning their spatial transformations, thus enhancing the robustness of the proposed network and allowing the CNLM to further explore the cross-modality non-local characteristics. On the other hand, the proposed CNLM aims to establish both the intra-modality and inter-modality non-local dependencies for fully exploiting the correlations between the target and reference images. Extensive experimental results show that our proposed method consistently achieves state-of-the-art reconstruction performance in alignment and misalignment scenarios, demonstrating its significant promise for real-world applications.

An information entropy-based fuzzy stochastic configuration network for robust data modeling

In data-driven modeling, the generalization performance of a neural network based on the minimum mean square error criterion may be affected when the samples contain noise. To handle uncertain data modeling problems, this study contributes a novel robust data modeling method called information entropy-based fuzzy stochastic configuration networks (IEFSCNs), which incorporates the maximum correntropy criterion (MCC) to reflect the squared error of the training sample set and uses the quantized minimum error entropy (QMEE) criterion to mine the distribution structure of the modeling data and suppress outlier data. Fuzzy systems are designed to generate nodes in the fuzzy transformation layer. A nonlinear enhancement layer is established as a hidden layer to enhance the representational capacity of the target system. The fuzzy transformation and nonlinear enhancement layers are fully connected to determine the outputs of the IEFSCNs. In addition, an incremental parameter learning algorithm is developed based on a supervision mechanism for optimizing the cost function, and its convergence property is demonstrated. Owing to the superiorities of the cost function by synthesizing MCC and QMEE, IEFSCNs can improve the modeling accuracy and reduce noise suppression. The performances are evaluated using various regression and classification tasks. Compared with existing methods, including SCNs, RVFL, F-SCNs, RSC-KDE, RSC-MCC, and SM-RSC, the proposed method demonstrates an effective robust modeling performance by reducing network structure redundancy while enhancing its ability to suppress noisy data.

Spatial domain decomposition-based physics-informed neural networks for practical acoustic propagation estimation under ocean dynamics.

Practical acoustic propagation modeling is significantly affected by ocean dynamics, and then can be exploited in numerous oceanic applications, where "practical" refers to modeling acoustic propagation in simulations that mimic real-world ocean environments. Physics-based numerical models provide approximate solutions of wave equation and rely on accurate prior environmental knowledge while the environment of practical scenarios is largely unknown. In contrast, data-driven machine learning offers a promising avenue to estimate practical acoustic propagation by learning from dataset. However, collecting such a high-quality, noise-free, and dense dataset remains challenging. Under the practical hurdle posed by the above approaches, the emergence of physics-informed neural network (PINN) presents an alternative to integrate physics and data but with limited representation capacity. In this work, a framework, termed spatial domain decomposition-based physics-informed neural networks (SPINNs), is proposed to enhance the representation capacity in spatially dependent oceanic scenarios and effectively learn from incomplete and biased prior physics and noisy dataset. Experiments demonstrate SPINNs' advantages over PINN in practical acoustic propagation estimation. The learning capacity of SPINNs toward physics and dataset during training is further analyzed. This work holds promise for practical applications and future expansion.

Mask-guided discriminative feature network for occluded person re-identification

In recent years, although research on person re-identification (ReID) has made significant progress, occluded person ReID remains a major challenge. In real-world scenes, persons are often occluded by various obstacles such as vehicles, umbrellas, and other persons. This leads to the noisy interference and the loss of visual information, which result in poor ReID performance of occluded persons. To address this issue, we propose an end-to-end Mask-guided Discriminative Feature Network (MDFNet). First, MDFNet adopts a dual-branch architecture with a shared encoder as the Feature Extraction module for paired images. Each pair of images consist of one image from the training set and its corresponding occluded image generated through an occlusion augmentation strategy. Second, MDFNet utilizes a Mask-guided Discriminative Feature Enhancement and Fusion (MDFEF) module to achieve the fusion and enhancement of global and local features for high-quality person representations. MDFEF module effectively suppresses the interference of occlusion, enriches the representation capacity of person features, and enables the model to focus more on the important features in non-occluded regions. Furthermore, MDFNet introduces a sparse pairwise loss for enabling the model to dynamically adapt to intra-class variations and reduce the negative impact of complex occlusions. The experimental results on four challenging person ReID datasets demonstrate the effectiveness of the proposed method.

Supervised Contrastive Discrete Hashing for cross-modal retrieval

Cross-modal hashing aims at mapping data points from varied modalities to a shared Hamming space for providing swift and adaptive search services. The recent progress in contrastive learning has significantly enhanced the retrieval performance of cross-modal hashing, owing to its strong ability in powerful representations learning. However, most of them face two primary challenges: (1) they select negative samples for anchors randomly in each mini-batch, which may bring false negative samples that destroy the intrinsic semantic similarity in the learned hash codes; (2) the continuous relaxation strategy is commonly used to generate discrete hash codes, which results in quantization errors and consequently impairing the quality of hash codes. To alleviate these issues, we propose a deep supervised cross-modal hashing scheme, termed Supervised Contrastive Discrete Hashing (SCDH). Specifically, we extend the unsupervised contrastive method to the supervised setting to construct the positive and negative data points with class labels, avoiding the performance decreasing caused by false negative data. Besides, multiple positive and negative samples for each anchor can be chosen from the generated global dictionary bank by class labels, which can better capture the global data structure. Moreover, discrete hash codes can be generated directly, enhancing the feature representational capacity of hash codes. Comprehensive experiments conducted on two public datasets affirm the superior performance of the proposed SCDH over several other cutting-edge methods.

A Continuous Non-Invasive Blood Pressure Prediction Method Based on Deep Sparse Residual U-Net Combined with Improved Squeeze and Excitation Skip Connections.

Arterial blood pressure (ABP) serves as a pivotal clinical metric in cardiovascular health assessments, with the precise forecasting of continuous blood pressure assuming a critical role in both preventing and treating cardiovascular diseases. This study proposes a novel continuous non-invasive blood pressure prediction model, DSRUnet, based on deep sparse residual U-net combined with improved SE skip connections, which aim to enhance the accuracy of using photoplethysmography (PPG) signals for continuous blood pressure prediction. The model first introduces a sparse residual connection approach for path contraction and expansion, facilitating richer information fusion and feature expansion to better capture subtle variations in the original PPG signals, thereby enhancing the network's representational capacity and predictive performance and mitigating potential degradation in the network performance. Furthermore, an enhanced SE-GRU module was embedded in the skip connections to model and weight global information using an attention mechanism, capturing the temporal features of the PPG pulse signals through GRU layers to improve the quality of the transferred feature information and reduce redundant feature learning. Finally, a deep supervision mechanism was incorporated into the decoder module to guide the lower-level network to learn effective feature representations, alleviating the problem of gradient vanishing and facilitating effective training of the network. The proposed DSRUnet model was trained and tested on the publicly available UCI-BP dataset, with the average absolute errors for predicting systolic blood pressure (SBP), diastolic blood pressure (DBP), and mean blood pressure (MBP) being 3.36 ± 6.61 mmHg, 2.35 ± 4.54 mmHg, and 2.21 ± 4.36 mmHg, respectively, meeting the standards set by the Association for the Advancement of Medical Instrumentation (AAMI), and achieving Grade A according to the British Hypertension Society (BHS) Standard for SBP and DBP predictions. Through ablation experiments and comparisons with other state-of-the-art methods, the effectiveness of DSRUnet in blood pressure prediction tasks, particularly for SBP, which generally yields poor prediction results, was significantly higher. The experimental results demonstrate that the DSRUnet model can accurately utilize PPG signals for real-time continuous blood pressure prediction and obtain high-quality and high-precision blood pressure prediction waveforms. Due to its non-invasiveness, continuity, and clinical relevance, the model may have significant implications for clinical applications in hospitals and research on wearable devices in daily life.

Discovery of Novel Aldose Reductase Inhibitors via the Integration of Ligand-Based and Structure-Based Virtual Screening with Experimental Validation.

Aldose reductase plays a central role in diabetes mellitus (DM) associated complications by converting glucose to sorbitol, resulting in a harmful increase of reactive oxygen species (ROS) in various tissues, such as the heart, vasculature, neurons, eyes, and kidneys. We employed a comprehensive approach, integrating both ligand- and structure-based virtual screening followed by experimental validation. Initially, candidate compounds were extracted from extensive drug and chemical libraries using the DeepChem's GraphConvMol algorithm, leveraging its capacity for robust molecular feature representation. Subsequent refinement employed molecular docking and molecular dynamics (MD) simulations, which are crucial for understanding compound-receptor interactions and dynamic behavior in a simulated physiological environment. Finally, the candidate compounds were subjected to experimental validation of their biological activity using an aldose reductase inhibitor screening kit. The comprehensive approach led to the identification of a promising compound, demonstrating significant potential as an aldose reductase inhibitor. This comprehensive approach not only yields a potential therapeutic intervention for DM-related complications but also establishes an integrated protocol for drug development, setting a new benchmark in the field.

Vision Transformers with Hierarchical Attention

This paper tackles the high computational/space complexity associated with multi-head self-attention (MHSA) in vanilla vision transformers. To this end, we propose hierarchical MHSA (H-MHSA), a novel approach that computes sell-attention in a hierarchical fashion. Specifically, we first divide the input image into patches as commonly done, and each patch is viewed as a token. Then, the proposed H-MHSA learns token relationships within local patches, serving as local relationship modeling. Then, the small patches are merged into larger ones, and H-MHSA models the global dependencies for the small number of the merged tokens. At last, the local and global attentive features are aggregated to obtain features with powerful representation capacity. Since we only calculate attention for a limited number of tokens at each step, the computational load is reduced dramatically. Hence, H-MHSA can efficiently model global relationships among tokens without sacrificing fine-grained information. With the H-MHSA module incorporated, we build a family of hierarchical-attention-based transformer networks, namely HAT-Net. To demonstrate the superiority of HAT-Net in scene understanding, we conduct extensive experiments on fundamental vision tasks, including image classification, semantic segmentation, object detection and instance segmentation. Therefore, HAT-Net provides a new perspective for vision transformers. Code and pretrained models are available at https://github.com/yun-liu/HAT-Net.

“An Australian beauty-lover based in Singapore”: negotiating Asian Australian identity in the beauty vlogosphere

Abstract This article explores how the conventions and culture of beauty vlogging elucidate Asian Australian identity and politics. Drawing on a qualitative analysis of Asian Australian beauty vlogger Tina Yong’s vlogs, I explore how national, ethnic, and racial identities are negotiated through platform logics, commercial imperatives, and community participation. Yong’s vlogs offer tangible ways to narrate the mundane, everyday experiences of Asian Australian identity that are underrepresented in mainstream Australian media. At the same time, Yong’s self-representation is made legible through her engagements with the commercialized beauty vlogosphere and her transnational audience, which tend to reduce the potential for illuminating hybrid and culturally specific Asian Australian experiences, communities, and politics. While Yong’s vlogs generate productive dialogue on race/ethnicity and race-based affinity communities, platform and commercial logics hinder her representational capacity as an Asian Australian subject who is doubly marginalized within the global beauty vlogosphere.

A Method for Underwater Biological Detection Based on Improved YOLOXs

This article proposes a lightweight underwater biological target detection network based on the improvement of YOLOXs, addressing the challenges of complex and dynamic underwater environments, limited memory in underwater devices, and constrained computational capabilities. Firstly, in the backbone network, GhostConv and GhostBottleneck are introduced to replace standard convolutions and the Bottleneck1 structure in CSPBottleneck_1, significantly reducing the model’s parameter count and computational load, facilitating the construction of a lightweight network. Next, in the feature fusion network, a Contextual Transformer block replaces the 3 × 3 convolution in CSPBottleneck_2. This enhances self-attention learning by leveraging the rich context between input keys, improving the model’s representational capacity. Finally, the positioning loss function Focal_EIoU Loss is employed to replace IoU Loss, enhancing the model’s robustness and generalization ability, leading to faster and more accurate convergence during training. Our experimental results demonstrate that compared to the YOLOXs model, the proposed YOLOXs-GCE achieves a 1.1% improvement in mAP value, while reducing parameters by 24.47%, the computational load by 26.39%, and the model size by 23.87%. This effectively enhances the detection performance of the model, making it suitable for complex and dynamic underwater environments, as well as underwater devices with limited memory. The model meets the requirements of underwater target detection tasks.

Learning Reliable Dense Pseudo-Labels for Point-Level Weakly-Supervised Action Localization

Point-level weakly-supervised temporal action localization aims to accurately recognize and localize action segments in untrimmed videos, using only point-level annotations during training. Current methods primarily focus on mining sparse pseudo-labels and generating dense pseudo-labels. However, due to the sparsity of point-level labels and the impact of scene information on action representations, the reliability of dense pseudo-label methods still remains an issue. In this paper, we propose a point-level weakly-supervised temporal action localization method based on local representation enhancement and global temporal optimization. This method comprises two modules that enhance the representation capacity of action features and improve the reliability of class activation sequence classification, thereby enhancing the reliability of dense pseudo-labels and strengthening the model’s capability for completeness learning. Specifically, we first generate representative features of actions using pseudo-label feature and calculate weights based on the feature similarity between representative features of actions and segments features to adjust class activation sequence. Additionally, we maintain the fixed-length queues for annotated segments and design a action contrastive learning framework between videos. The experimental results demonstrate that our modules indeed enhance the model’s capability for comprehensive learning, particularly achieving state-of-the-art results at high IoU thresholds.