Loss Of Contrast Research Articles

CLIB: Contrastive learning of ignoring background for underwater fish image classification.

Aiming at the problem that the existing methods are insufficient in dealing with the background noise anti-interference of underwater fish images, a contrastive learning method of ignoring background called CLIB for underwater fish image classification is proposed to improve the accuracy and robustness of underwater fish image classification. First, CLIB effectively separates the subject from the background in the image through the extraction module and applies it to contrastive learning by composing three complementary views with the original image. To further improve the adaptive ability of CLIB in complex underwater images, we propose a multi-view-based contrastive loss function, whose core idea is to enhance the similarity between the original image and the subject and maximize the difference between the subject and the background, making CLIB focus more on learning the core features of the subject during the training process, and effectively ignoring the interference of background noise. Experiments on the Fish4Knowledge, Fish-gres, WildFish-30, and QUTFish-89 public datasets show that our method performs well, with improvements of 1.43-6.75%, 8.16-8.95%, 13.1-14.82%, and 3.92-6.19%, respectively, compared with the baseline model, further validating the effectiveness of CLIB.

Unsupervised domain adaptive building semantic segmentation network by edge-enhanced contrastive learning

Unsupervised domain adaptation (UDA) is a weakly supervised learning technique that classifies images in the target domain when the source domain has labeled samples, and the target domain has unlabeled samples. Due to the complexity of imaging conditions and the content of remote sensing images, the use of UDA to accurately extract artificial features such as buildings from high-spatial-resolution (HSR) imagery is still challenging. In this study, we propose a new UDA method for building extraction, the contrastive domain adaptation network (CDANet), by utilizing adversarial learning and contrastive learning techniques. CDANet consists of a single multitask generator and dual discriminators. The generator employs a region and edge dual-branch structure that strengthens its edge extraction ability and is beneficial for the extraction of small and densely distributed buildings. The dual discriminators receive the region and edge prediction outputs and achieve multilevel adversarial learning. During adversarial training processing, CDANet aligns the cross-domain of similar pixel features in the embedding space by constructing the regional pixelwise contrastive loss. A self-training (ST) strategy based on pseudolabel generation is further utilized to address the target intradomain discrepancy. Comprehensive experiments are conducted to validate CDANet on three publicly accessible datasets, namely the WHU, Austin, and Massachusetts. Ablation experiments show that the generator network structure, contrastive loss and ST strategy all improve the building extraction accuracy. Method comparisons validate that CDANet achieves superior performance to several state-of-the-art methods, including AdaptSegNet, AdvEnt, IntraDA, FDANet and ADRS, in terms of F1 score and mIoU.

Vehicle Classification Algorithm Based on Improved Vision Transformer

Vehicle classification technology is one of the foundations in the field of automatic driving. With the development of deep learning technology, visual transformer structures based on attention mechanisms can represent global information quickly and effectively. However, due to direct image segmentation, local feature details and information will be lost. To solve this problem, we propose an improved vision transformer vehicle classification network (IND-ViT). Specifically, we first design a CNN-In D branch module to extract local features before image segmentation to make up for the loss of detail information in the vision transformer. Then, in order to solve the problem of misdetection caused by the large similarity of some vehicles, we propose a sparse attention module, which can screen out the discernible regions in the image and further improve the detailed feature representation ability of the model. Finally, this paper uses the contrast loss function to further increase the intra-class consistency and inter-class difference of classification features and improve the accuracy of vehicle classification recognition. Experimental results show that the accuracy of the proposed model on the datasets of vehicle classification BIT-Vehicles, CIFAR-10, Oxford Flower-102, and Caltech-101 is higher than that of the original vision transformer model. Respectively, it increased by 1.3%, 1.21%, 7.54%, and 3.60%; at the same time, it also met a certain real-time requirement to achieve a balance of accuracy and real time.

Multimodal Pre-training for Sequential Recommendation via Contrastive Learning

Sequential recommendation systems often suffer from data sparsity, leading to suboptimal performance. While multimodal content, such as images and text, has been utilized to mitigate this issue, its integration within sequential recommendation frameworks remains challenging. Current multimodal sequential recommendation models are often unable to effectively explore and capture correlations among behavior sequences of users and items across different modalities, either neglecting correlations among sequence representations or inadequately capturing associations between multimodal data and sequence data in their representations. To address this problem, we explore multimodal pre-training in the context of sequential recommendation, with the aim of enhancing fusion and utilization of multimodal information. We propose a novel M ultimodal P re-training for S equential R ecommendation (MP4SR) framework, which utilizes contrastive losses to capture the correlation among different modality sequences of users, as well as the correlation among different modality sequences of users and items. MP4SR consists of three key components: 1) multimodal feature extraction, 2) a backbone network, Multimodal Mixup Sequence Encoder (M 2 SE), and 3) pre-training tasks. After utilizing pre-trained encoders to generate initial multimodal features of items, M 2 SE adopts a complementary sequence mixup strategy to fuse different modality sequences, and leverages contrastive learning to capture modality interactions at the sequence-to-sequence and sequence-to-item levels. Extensive experiments on four real-world datasets demonstrate that MP4SR outperforms state-of-the-art approaches in both normal and cold-start settings. We further highlight the efficacy of incorporating multimodal pre-training in sequential recommendation representation learning, serving as an effective regularizer and optimizing the parameter space for the recommendation task.

An interpretable multiscale lifting wavelet contrast network for planetary gearbox fault diagnosis with small samples

Previous deep learning-based fault diagnosis methods for planetary gearbox require numerous training samples and lack the necessary interpretability. Aiming at the problems of insufficient interpretability of deep models and the absence of feature mining capability with small samples, this study presents an interpretable multiscale lifting wavelet contrast network for planetary gearbox fault diagnosis. First, an interpretable multiscale lifting wavelet network is designed to achieve comprehensive and credible features mining from fault signals. Secondly, an interactive channel attention mechanism is constructed to choose feature maps with different frequency components. It can further confirm the interpretability of the lifting wavelet layer while improving the accuracy of the model. Finally, a time-frequency contrast loss is designed to simultaneously optimizing the distribution of time-frequency domain features. The effectiveness and interpretability of the model are analyzed through various visualization approaches. Experimental results on two planetary gearbox datasets indicate that our method is an interpretable and effective fault recognition method with small samples, and it holds a promising future for engineering applications.

Open-set domain adaptive fault diagnosis based on supervised contrastive learning and a complementary weighted dual adversarial network

In an actual industrial environment, the complex working environment of mechanical equipment may lead to new faults in the target domain, called the open-set domain adaptation problem. Recently, open-set adaptive fault diagnosis has been extensively employed. However, most studies not only require pre-set fixed thresholds to identify unknown class features but also ignore the learning of discriminable features under specific tasks, which affects the diagnostic performance. Hence, this paper proposes a complementary weighted dual adversarial network combined with supervised contrastive learning (CWDAN-SCL) to address the open-set cross-different working fault diagnosis of bearings. Specifically, a novel complementary weighted adversarial learning strategy is designed using supervised classification and uncertainty measurement to effectively control the participation of target domain features in the domain adaptation process and achieve the alignment of shared class fault features between the source and target domains. Moreover, an adaptive unknown fault separation module is designed using an adversarial learning method to construct a hyperplane between shared and unknown class fault features in the target domain to identify unknown class faults accurately. Additionally, a supervised contrastive loss term is designed based on contrastive learning and label knowledge to improve the aggregation of fault features of the same class and enhance the model’s generalization ability in target domain diagnosis tasks. Subsequently, the efficacy and advancement of the proposed method are substantiated through experimentation on two datasets. The experimental results illustrate that the average diagnostic performance of the proposed method is 91.73 %. This study contributes a dependable diagnostic approach for ascertaining the health status of rotating machinery equipment.

Heterogeneous biomedical entity representation learning for gene-disease association prediction.

Understanding the genetic basis of disease is a fundamental aspect of medical research, as genes are the classic units of heredity and play a crucial role in biological function. Identifying associations between genes and diseases is critical for diagnosis, prevention, prognosis, and drug development. Genes that encode proteins with similar sequences are often implicated in related diseases, as proteins causing identical or similar diseases tend to show limited variation in their sequences. Predicting gene-disease association (GDA) requires time-consuming and expensive experiments on a large number of potential candidate genes. Although methods have been proposed to predict associations between genes and diseases using traditional machine learning algorithms and graph neural networks, these approaches struggle to capture the deep semantic information within the genes and diseases and are dependent on training data. To alleviate this issue, we propose a novel GDA prediction model named FusionGDA, which utilizes a pre-training phase with a fusion module to enrich the gene and disease semantic representations encoded by pre-trained language models. Multi-modal representations are generated by the fusion module, which includes rich semantic information about two heterogeneous biomedical entities: protein sequences and disease descriptions. Subsequently, the pooling aggregation strategy is adopted to compress the dimensions of the multi-modal representation. In addition, FusionGDA employs a pre-training phase leveraging a contrastive learning loss to extract potential gene and disease features by training on a large public GDA dataset. To rigorously evaluate the effectiveness of the FusionGDA model, we conduct comprehensive experiments on five datasets and compare our proposed model with five competitive baseline models on the DisGeNet-Eval dataset. Notably, our case study further demonstrates the ability of FusionGDA to discover hidden associations effectively. The complete code and datasets of our experiments are available at https://github.com/ZhaohanM/FusionGDA.

Adaptive self-supervised learning for sequential recommendation

Sequential recommendation typically utilizes deep neural networks to mine rich information in interaction sequences. However, existing methods often face the issue of insufficient interaction data. To alleviate the sparsity issue, self-supervised learning is introduced into sequential recommendation. Despite its effectiveness, we argue that current self-supervised learning-based (i.e., SSL-based) sequential recommendation models have the following limitations: (1) using only a single self-supervised learning method, either contrastive self-supervised learning or generative self-supervised learning. (2) employing a simple data augmentation strategy in either the graph structure domain or the node feature domain. We believe that they have not fully utilized the capabilities of both self-supervised methods and have not sufficiently explored the advantages of combining graph augmentation schemes. As a result, they often fail to learn better item representations. In light of this, we propose a novel multi-task sequential recommendation framework named Adaptive Self-supervised Learning for sequential Recommendation (ASLRec). Specifically, our framework combines contrastive and generative self-supervised learning methods adaptively, simultaneously applying different perturbations at both the graph topology and node feature levels. This approach constructs diverse augmented graph views and employs multiple loss functions (including contrastive loss, generative loss, mask loss, and prediction loss) for joint training. By encompassing the capabilities of various methods, our model learns item representations across different augmented graph views to achieve better performance and effectively mitigate interaction noise and sparsity. In addition, we add a small proportion of random uniform noise to item representations, making the item representations more uniform and mitigating the inherent popularity bias in interaction records. We conduct extensive experiments on three publicly available benchmark datasets to evaluate our model. The results demonstrate that our approach achieves state-of-the-art performance compared to 14 other competitive methods: the hit rate (HR) improved by over 14.39%, and the normalized discounted cumulative gain (NDCG) increased by over 18.67%.

Prototypical contrastive learning based oriented detector for kitchen waste

Automatic detection of kitchen waste enables the identification and quantification of non-degradable materials, such as plastics, metals, and other substances that cannot be easily decomposed. This approach increases the efficiency of the waste disposal process with significant time and cost savings compared to manual sorting. However, detecting non-degradable waste automatically presents a great challenge due to the deformability and directional uncertainty of the objects in kitchen waste images. Object deformation leads to a lack of structural features, making it more difficult to distinguish objects. Direction uncertainty leads to redundant backgrounds when detecting with the horizontal box detector. To address these issues, we propose a kitchen waste detector (KWDet) for the automatic detection of non-degradable waste in kitchen waste. First, the KWDet introduces the oriented bounding box representation and rotated region of interest (RRoI) learner, which enables the oriented detection of non-degradable waste in kitchen waste. Second, the KWDet constructs a prototypical contrastive learning (PCL) based detection head to improve the classification performance by learning discriminative features of different non-degradable waste in kitchen waste. In the PCL based detection head, the prototype-sample contrastive loss is proposed to pull samples and prototypes from the same classes together while pushing samples and prototypes from the different classes apart. After that, we design the inter-prototype contrastive loss to learn the discriminative prototypes of different non-degradable waste. Finally, to evaluate the proposed KWDet, we construct a large kitchen waste detection dataset (KWDD), containing 13,873 kitchen waste images and 213,565 instances collected from Zhengzhou China with 5 classes. To validate the effectiveness of our proposed method, we performed ablation experiments and comparison experiments on the KWDD dataset. The experiments demonstrate that KWDet achieves a state-of-the-art (SOTA) mAP50 of 71.5%, surpassing other object detection methods by more than 1.6% mAP50. Furthermore, our approach’s Macro-F1 score outperforms the SOTA method by 4.0%. Extensive experiments on the KWDD demonstrate that the prototypical contrastive learning-based oriented detector is able to extract more robust features and better characterize the target location, suggesting that the KWDet is a powerful support for intelligent kitchen waste sorting.

Towards diverse image-to-image translation via adaptive normalization layer and contrast learning

A nice image-to-image translation framework is able to acquire an explicit and credible mapping relationship between the source domain and target domains while satisfying two requirements. One is simplicity, the other is extensibility over multiple translation tasks. To this end, we design a concise but versatile generative model for image-to-image translation. Our method includes three major ingredients. First, inspired by popular unconditional normalization layers, named Spatially Adaptive Normalization(SPADE). We introduce a novel Semantics-Appearance Spatially Adaptive Normalization (SA-SPADE), taking into account both semantic structure and style appearance. This enables semantic composition and style appearance information to be sufficiently captured and integrated by our normalization layers. Thanks to SA-SPADE, our model extends to multiple image-to-image translation tasks in an unsupervised or supervised way. Second, we carefully designed two symmetrical network branches to provide semantic and appearance information for our normalization layer, namely Semantic Branch (SB) and Appearance Branch(AB) respectively. Third, we propose novel Semantic-aware Contrastive Loss (SCL) and Appearance-aware Contrastive Loss (ACL)based on newly un-/self- supervised contrastive learning. That is, SCL guarantees domain-invariant (e.g., pose, structure) representations between the generated image and the input image, while ACL ensures domain-specific representations (e.g., color, texture) between the generated image and the reference image. As a result, we verify the effectiveness of our method by comparing it with various task-dependent image translation models in both qualitative and quantitative evaluations.

Learning from Feature and Global Topologies: Adaptive Multi-View Parallel Graph Contrastive Learning

To address the limitations of existing graph contrastive learning methods, which fail to adaptively integrate feature and topological information and struggle to efficiently capture multi-hop information, we propose an adaptive multi-view parallel graph contrastive learning framework (AMPGCL). It is an unsupervised graph representation learning method designed to generate task-agnostic node embeddings. AMPGCL constructs and encodes feature and topological views to mine feature and global topological information. To encode global topological information, we introduce an H-Transformer to decouple multi-hop neighbor aggregations, capturing global topology from node subgraphs. AMPGCL learns embedding consistency among feature, topology, and original graph encodings through a multi-view contrastive loss, generating semantically rich embeddings while avoiding information redundancy. Experiments on nine real datasets demonstrate that AMPGCL consistently outperforms thirteen state-of-the-art graph representation learning models in classification accuracy, whether in homophilous or non-homophilous graphs.

TDF-Net: Trusted Dynamic Feature Fusion Network for breast cancer diagnosis using incomplete multimodal ultrasound

Ultrasound is a critical imaging technique for diagnosing breast cancer. However, the multimodal breast ultrasound diagnostic process is time-consuming and labor-intensive, heavily dependent on the physician’s extensive expertise. Therefore, developing a computer-aided diagnosis system for breast cancer is essential. Existing diagnostic systems fail to consider the varying impacts of different ultrasound modalities on diagnostic results and struggle to address the issue of missing modalities in clinical practice. Consequently, this paper proposes the Trusted Dynamic Feature Fusion Network (TDF-Net) for diagnosing breast cancer using incomplete multimodal ultrasound data. Initially, this method introduces a dual-branch feature extraction module to capture modality-specific information. Meanwhile, a contrastive clustering loss is designed to enforce the consistency constraint, ensuring the coherence of different modal features within the semantic space for each sample. Additionally, an invertible neural network-based recovery method is suggested to establish mappings between different modalities, enabling the recovery of missing modalities. Finally, a trusted dynamic feature fusion module based on the Dirichlet distribution is proposed to quantify each modality’s contribution to the diagnostic result by considering uncertainty, thereby achieving the dynamic fusion of each modality’s features across different samples. The proposed method is validated on an established multimodal breast ultrasound dataset, demonstrating superior diagnostic performance compared to existing methods, with an average AUC of 98.31%, a 95% confidence interval of [96.98%, 99.64%], and a p-value < 0.05. A pilot study is planned to assess the effectiveness and usability of TDF-Net in clinical settings.

Spatial-aware contrastive learning for cross-domain medical image registration.

With the rapid advancement of medical imaging technologies, precise image analysis and diagnosis play a crucial role in enhancing treatment outcomes and patient care. Computed tomography (CT) and magnetic resonance imaging (MRI), as pivotal technologies in medical imaging, exhibit unique advantages in bone imaging and soft tissue contrast, respectively. However, cross-domain medical image registration confronts significant challenges due to the substantial differences in contrast, texture, and noise levels between different imaging modalities. The purpose of this study is to address the major challenges encountered in the field of cross-domain medical image registration by proposing a spatial-aware contrastive learning approach that effectively integrates shared information from CT and MRI images. Our objective is to optimize the feature space representation by employing advanced reconstruction and contrastive loss functions, overcoming the limitations of traditional registration methods when dealing with different imaging modalities. Through this approach, we aim to enhance the model's ability to learn structural similarities across domain images, improve registration accuracy, and provide more precise imaging analysis tools for clinical diagnosis and treatmentplanning. With prior knowledge that different domains of images (CT and MRI) share same content-style information, we extract equivalent feature spaces from both images, enabling accurate cross-domain point matching. We employ a structure resembling that of an autoencoder, augmented with designed reconstruction and contrastive losses to fulfill our objectives. We also propose region mask to solve the conflict between spatial correlation and distinctiveness, to obtain a better representation space. Our research results demonstrate the significant superiority of the proposed spatial-aware contrastive learning approach in the domain of cross-domain medical image registration. Quantitatively, our method achieved an average Dice similarity coefficient (DSC) of 85.68%, target registration error (TRE) of 1.92 mm, and mean Hausdorff distance (MHD) of 1.26 mm, surpassing current state-of-the-art methods. Additionally, the registration processing time was significantly reduced to 2.67 s on a GPU, highlighting the efficiency of our approach. The experimental outcomes not only validate the effectiveness of our method in improving the accuracy of cross-domain image registration but also prove its adaptability across different medical image analysis scenarios, offering robust support for enhancing diagnostic precision and patient treatmentoutcomes. The spatial-aware contrastive learning approach proposed in this paper introduces a new perspective and solution to the domain of cross-domain medical image registration. By effectively optimizing the feature space representation through carefully designed reconstruction and contrastive loss functions, our method significantly improves the accuracy and stability of registration between CT and MRI images. The experimental results demonstrate the clear advantages of our approach in enhancing the accuracy of cross-domain image registration, offering significant application value in promoting precise diagnosis and personalized treatment planning. In the future, we look forward to further exploring the application of this method in a broader range of medical imaging datasets and its potential integration with other advanced technologies, contributing more innovations to the field of medical image analysis andprocessing.

A self-supervised embedding of cell migration features for behavior discovery over cell populations

Background and objective:Recent studies point out that the dynamics and interaction of cell populations within their environment are related to several biological processes in immunology. Hence, single-cell analysis in immunology now relies on spatial omics. Moreover, recent literature suggests that immunology scenarios are hierarchically organized, including unknown cell behaviors appearing in different proportions across some observable control and therapy groups. These dynamic behaviors play a crucial role in identifying the causes of processes such as inflammation, aging, and fighting off pathogens or cancerous cells. In this work, we use a self-supervised learning approach to discover these behaviors associated with cell dynamics in an immunology scenario. Materials and methods:Specifically, we study the different responses of control group and therapy groups in a scenario involving inflammation due to infarct, with a focus on neutrophil migration within blood vessels. Starting from a set of hand-crafted spatio-temporal features, we use a recurrent neural network to generate embeddings that properly describe the dynamics of the migration processes. The network is trained using a novel multi-task contrastive loss that, on the one hand, models the hierarchical structure of our scenario (groups-behaviors-samples) and, on the other, ensures temporal consistency within the embedding, enforcing that subsequent temporal samples obtained from a given cell stay close in the latent space. Results:Our experimental results demonstrate that the resulting embeddings improve the separability of cell behaviors and log-likelihood of the therapies, when compared to the hand-crafted feature extraction and recent methods from the state of the art, even with dimensionality reduction (16 vs. 21 hand-crafted features). Conclusions:Our approach enables single-cell analyses at a population level, being able to automatically discover shared behaviors among different groups. This, in turn, enables the prediction of the therapy effectiveness based on their proportions within a study group.

Leveraging segmentation-guided spatial feature embedding for overall survival prediction in glioblastoma with multimodal magnetic resonance imaging

Background and objectivePatients with glioblastoma have a five-year relative survival rate of less than 5 %. Thus, accurately predicting the overall survival (OS) of patients with glioblastoma is crucial for effective treatment planning. MethodsTo fully leverage the imaging characteristics of glioblastomas, we propose a segmentation-guided regression method for predicting OS of patients with brain tumors using multimodal magnetic resonance imaging. Specifically, a brain tumor segmentation network was first pre-trained without leveraging survival information. Subsequently, the survival regression network was jointly trained with the guidance of brain tumor segmentation, focusing on tumor voxels and suppressing irrelevant backgrounds. ResultsOur proposed framework, based on the well-known backbone of UNETR++, achieved a Dice score of 0.7910, Spearman correlation of 0.4112, and Harrell's concordance index of 0.6488. The model consistently showed promising results compared with baseline methods on two different datasets (BraTS and UCSF-PDGM). Furthermore, ablation studies on our training configurations demonstrated that both the pre-training segmentation network and contrastive loss significantly improved all metrics for OS prediction. ConclusionsIn this study, we propose a joint learning framework based on a pre-trained segmentation backbone for OS prediction by leveraging a brain tumor segmentation map. By utilizing a spatial feature map, our model can operate using a sliding-window approach, which can be adopted by varying the matrix sizes and resolutions of the input images.

Realization of Reliable and Effective Authentication in Intelligent Systems by Using Visual Biometrics Methods

The main purpose of this article is to consider the aspects of ensuring security and increasing the efficiency of the authentication process in intelligent systems using visual biometrics. The work is aimed at the development and improvement of authentication systems using advanced biometric identification methods. An intelligent system has been created that ensures secure authentication of users of the current system, using a Siamese neural network. In addition to the implementation of basic security measures in the form of hashing and saving user logins and passwords, the implementation of two-factor authentication is important nowadays, which significantly strengthens the protection of user data and prevents most modern methods of hacking and stealing user data. Two-factor authentication is implemented as a technology for searching, recognizing and comparing the faces of system users, as visual biometrics is more secure than other types of two-factor authentication. Different variations of the possible implementation of Siamese neural network using Contrastive loss function and more modern Triplet loss function were reviewed and accordingly, a neural network using Triplet loss function was accomplished and trained. After training and verifying the correct operation of the neural network, it was integrated into the created intelligent system, thanks to which an effective way of recognizing the face of the system user was created, saving the received information in the database and further comparing the current user with the stored face during authentication. As a result, a secure and reliable intelligent system was created that cutting down the risk of unapproved access to the user account and uses an effective and modern method of user authentication.

Efficient Representation Learning for Generalized Category Discovery

Generalized Category Discovery (GCD) aims to discover and cluster data from previously unseen classes. Numerous studies have delved into GCD; however, they either overlooked the significance of representation learning or naively adapt the contrastive loss from self-supervised learning without further refinements. Therefore, in this paper, we propose a simple yet effective method to address this task. Our first contribution is to design a robust Divide-and-Combine strategy with an improved contrastive loss to generate informative features. Subsequently, in light of the scarce supervised signals, we propose a Neighborhood-based Sampling module based on the analysis of cluster-level alignment and uniformity to better discover novel categories. Ultimately, to boost the model’s ability to learn from unlabeled data, we adopt a fusion training approach utilizing combinational patch features and the deterministic threshold to selectively identify high-confidence samples. We unify these learning processes within a single framework and jointly optimize. Extensive experiments across three general datasets and four fine-grained benchmark datasets show the compelling evidence for the superiority of our proposed method—Efficient Representation Learning for Generalized Category Discovery (ERL4GCD).

Event-driven weakly supervised video anomaly detection

Inspired by the observations of human working manners, this work proposes an event-driven method for weakly supervised video anomaly detection. Complementary to the conventional snippet-level anomaly detection, this work designs an event analysis module to predict the event-level anomaly scores as well. It first generates event proposals simply via a temporal sliding window and then constructs a cascaded causal transformer to capture temporal dependencies for potential events of varying durations. Moreover, a dual-memory augmented self-attention scheme is also designed to capture global semantic dependencies for event feature enhancement. The network is learned with a standard multiple instance learning (MIL) loss, together with normal-abnormal contrastive learning losses. During inference, the snippet- and event-level anomaly scores are fused for anomaly detection. Experiments show that the event-level analysis helps to detect anomalous events more continuously and precisely. The performance of the proposed method on three public datasets demonstrates that the proposed approach is competitive with state-of-the-art methods.

UET4Rec: U-net encapsulated transformer for sequential recommender

Recommending a tempting sequence of items according to a user’s previous history of purchases and clicks, for instance, in the online shopping portals is challenging. And yet it is a crucial task for all service providers. One of the core components in the recommender systems is a sequential model with which an input sequence is transformed into the predicted items. Among many, deep neural networks, such as RNN, LSTM, and Transformer, have been favored for this purpose. However, improving the performance of these models remains an important task. To address this, we propose a novel sequential model by combining a U-net and Transformer, called U-net Encapsulated Transformer. This hybrid architecture places a transformer model between a convolutional encoder and its decoder, wherein each convolutional layer processes 1D signal, i.e. text. The primary benefit of this structure is that the computational burden is reduced since the embedding size of the input to the Transformer is drastically decreased as the signal has to go through the multi-layer convolutional encoder. This solution leverages recommendation lists for action predictions and uses user feedback as direct rewards for updating the model. In addition, the loss function mechanism is improved by including contrastive and reinforcement learning losses. Evaluation of the proposed model, including extensive ablation study, is carried out on four standard benchmark datasets, such as RC15, RetailRocket, MovieLens-1M, and Amazon-Beauty, demonstrating superior performance compared to state-of-the-art methods.

Semi-supervised human action recognition via dual-stream cross-fusion and class-aware memory bank

Human action recognition, a crucial task for understanding complex human behavior from sensor data, faces significant challenges. This study specifically addresses two of these challenges: the complex interplay between joint positions and their velocities in action data, and improving model generalizability in the context of limited labeled data. To this end, we present the dual-stream cross-fusion and class-aware memory bank (DSCF-CAMB) framework, a novel semi-supervised approach. It employs a dual-stream cross-fusion (DSCF) mechanism to independently encode joint and velocity data streams, and through cross-fusion, enriches the action representation by exploring correlations between joint positions and their velocities. Meanwhile, to address the challenges posed by unlabeled data, we have developed a contrastive loss based class-aware memory bank (CAMB) that carefully selects high-confidence positive and negative sample embeddings and filters out potential noise-contributing samples. This strategy significantly improves the robustness and accuracy of the framework in action recognition. The experimental results show that our method outperforms the state-of-the-art methods by an average of 2.7% and 1.5% in top-1 accuracy on the renowned NTU RGB+D and NW-UCLA datasets, respectively, demonstrating the superiority of our method in human action recognition.