Improve Feature Representation Research Articles

Prediction of Myocardial Infarction Using a Combined Generative Adversarial Network Model and Feature-Enhanced Loss Function.

Accurate risk prediction for myocardial infarction (MI) is crucial for preventive strategies, given its significant impact on global mortality and morbidity. Here, we propose a novel deep-learning approach to enhance the prediction of incident MI cases by incorporating metabolomics alongside clinical risk factors. We utilized data from the KORA cohort, including the baseline S4 and follow-up F4 studies, consisting of 1454 participants without prior history of MI. The dataset comprised 19 clinical variables and 363 metabolites. Due to the imbalanced nature of the dataset (78 observed MI cases and 1376 non-MI individuals), we employed a generative adversarial network (GAN) model to generate new incident cases, augmenting the dataset and improving feature representation. To predict MI, we further utilized multi-layer perceptron (MLP) models in conjunction with the synthetic minority oversampling technique (SMOTE) and edited nearest neighbor (ENN) methods to address overfitting and underfitting issues, particularly when dealing with imbalanced datasets. To enhance prediction accuracy, we propose a novel GAN for feature-enhanced (GFE) loss function. The GFE loss function resulted in an approximate 2% improvement in prediction accuracy, yielding a final accuracy of 70%. Furthermore, we evaluated the contribution of each clinical variable and metabolite to the predictive model and identified the 10 most significant variables, including glucose tolerance, sex, and physical activity. This is the first study to construct a deep-learning approach for producing 7-year MI predictions using the newly proposed loss function. Our findings demonstrate the promising potential of our technique in identifying novel biomarkers for MI prediction.

Quantum driven deep learning for enhanced diabetic retinopathy detection

Traditional convolutional neural networks (CNNs) have shown potential for recognizing retinopathy caused by diabetes (DR). However, developing quantum computing has the possibility for improved feature representation. We propose a hybrid approach that combines classical CNNs with quantum circuits to capitalize on both classical and quantum information for DR classification. Using the Keras and Qiskit frameworks, our model encodes picture features into quantum states, allowing for richer representations. Through experiments on a collection of retinal pictures, our model displays competitive performance, with excellent reliability and precision in categorizing DR severity levels. This combination of classical and quantum paradigms offers a fresh approach to enhancing DR diagnosis and therapy.

Learning with noisy labels using collaborative sample selection and contrastive semi-supervised learning

Learning with noisy labels (LNL) has been extensively studied, with existing approaches typically following a framework that alternates between clean sample selection and semi-supervised learning (SSL). However, this approach has a limitation: the clean set selected by the Deep Neural Network (DNN) classifier, trained through self-training, inevitably contains noisy samples. This mixture of clean and noisy samples leads to misguidance in DNN training during SSL, resulting in impaired generalization performance due to confirmation bias caused by error accumulation in sample selection. To address this issue, we propose a method called Collaborative Sample Selection (CSS), which leverages the large-scale pre-trained model CLIP. CSS aims to remove the mixed noisy samples from the identified clean set. We achieve this by training a 2-Dimensional Gaussian Mixture Model (2D-GMM) that combines the probabilities from CLIP with the predictions from the DNN classifier. To further enhance the adaptation of CLIP to LNL, we introduce a co-training mechanism with a contrastive loss in semi-supervised learning. This allows us to jointly train the prompt of CLIP and the DNN classifier, resulting in improved feature representation, boosted classification performance of DNNs, and reciprocal benefits to our Collaborative Sample Selection. By incorporating auxiliary information from CLIP and utilizing prompt fine-tuning, we effectively eliminate noisy samples from the clean set and mitigate confirmation bias during training. Experimental results on multiple benchmark datasets demonstrate the effectiveness of our proposed method in comparison with the state-of-the-art approaches.

Involution Transformer based U-Net for Landmark Detection in Ultrasound Images for Diagnosis of Infantile DDH.

The B-mode ultrasound based computer-aided diagnosis (CAD) has demonstrated its effectiveness for diagnosis of Developmental Dysplasia of the Hip (DDH) in infants, which can conduct the Graf's method by detecting landmarks in hip ultrasound images. However, it is still necessary to explore more valuable information around these landmarks to enhance feature representation for improving detection performance in the detection model. To this end, a novel Involution Transformer based U-Net (IT-UNet) network is proposed for hip landmark detection. The IT-UNet integrates the efficient involution operation into Transformer to develop an Involution Transformer module (ITM), which consists of an involution attention block and a squeeze-and-excitation involution block. The ITM can capture both the spatial-related information and long-range dependencies from hip ultrasound images to effectively improve feature representation. Moreover, an Involution Downsampling block (IDB) is developed to alleviate the issue of feature loss in the encoder modules, which combines involution and convolution for the purpose of downsampling. The experimental results on two DDH ultrasound datasets indicate that the proposed IT-UNet achieves the best landmark detection performance, indicating its potential applications.

Enhancing Representation of Spiking Neural Networks via Similarity-Sensitive Contrastive Learning

Spiking neural networks (SNNs) have attracted intensive attention as a promising energy-efficient alternative to conventional artificial neural networks (ANNs) recently, which could transmit information in form of binary spikes rather than continuous activations thus the multiplication of activation and weight could be replaced by addition to save energy. However, the binary spike representation form will sacrifice the expression performance of SNNs and lead to accuracy degradation compared with ANNs. Considering improving feature representation is beneficial to training an accurate SNN model, this paper focuses on enhancing the feature representation of the SNN. To this end, we establish a similarity-sensitive contrastive learning framework, where SNN could capture significantly more information from its ANN counterpart to improve representation by Mutual Information (MI) maximization with layer-wise sensitivity to similarity. In specific, it enriches the SNN’s feature representation by pulling the positive pairs of SNN's and ANN's feature representation of each layer from the same input samples closer together while pushing the negative pairs from different samples further apart. Experimental results show that our method consistently outperforms the current state-of-the-art algorithms on both popular non-spiking static and neuromorphic datasets.

Correlation Matching Transformation Transformers for UHD Image Restoration

This paper proposes UHDformer, a general Transformer for Ultra-High-Definition (UHD) image restoration. UHDformer contains two learning spaces: (a) learning in high-resolution space and (b) learning in low-resolution space. The former learns multi-level high-resolution features and fuses low-high features and reconstructs the residual images, while the latter explores more representative features learning from the high-resolution ones to facilitate better restoration. To better improve feature representation in low-resolution space, we propose to build feature transformation from the high-resolution space to the low-resolution one. To that end, we propose two new modules: Dual-path Correlation Matching Transformation module (DualCMT) and Adaptive Channel Modulator (ACM). The DualCMT selects top C/r (r is greater or equal to 1 which controls the squeezing level) correlation channels from the max-pooling/mean-pooling high-resolution features to replace low-resolution ones in Transformers, which can effectively squeeze useless content to improve the feature representation in low-resolution space to facilitate better recovery. The ACM is exploited to adaptively modulate multi-level high-resolution features, enabling to provide more useful features to low-resolution space for better learning. Experimental results show that our UHDformer reduces about ninety-seven percent model sizes compared with most state-of-the-art methods while significantly improving performance under different training sets on 3 UHD image restoration tasks, including low-light image enhancement, image dehazing, and image deblurring. The source codes will be made available at https://github.com/supersupercong/UHDformer.

NaMa: Neighbor-Aware Multi-Modal Adaptive Learning for Prostate Tumor Segmentation on Anisotropic MR Images

Accurate segmentation of prostate tumors from multi-modal magnetic resonance (MR) images is crucial for diagnosis and treatment of prostate cancer. However, the robustness of existing segmentation methods is limited, mainly because these methods 1) fail to adaptively assess subject-specific information of each MR modality for accurate tumor delineation, and 2) lack effective utilization of inter-slice information across thick slices in MR images to segment tumor as a whole 3D volume. In this work, we propose a two-stage neighbor-aware multi-modal adaptive learning network (NaMa) for accurate prostate tumor segmentation from multi-modal anisotropic MR images. In particular, in the first stage, we apply subject-specific multi-modal fusion in each slice by developing a novel modality-informativeness adaptive learning (MIAL) module for selecting and adaptively fusing informative representation of each modality based on inter-modality correlations. In the second stage, we exploit inter-slice feature correlations to derive volumetric tumor segmentation. Specifically, we first use a Unet variant with sequence layers to coarsely capture slice relationship at a global scale, and further generate an activation map for each slice. Then, we introduce an activation mapping guidance (AMG) module to refine slice-wise representation (via information from adjacent slices) for consistent tumor segmentation across neighboring slices. Besides, during the network training, we further apply a random mask strategy to each MR modality to improve feature representation efficiency. Experiments on both in-house and public (PICAI) multi-modal prostate tumor datasets show that our proposed NaMa performs better than state-of-the-art methods.

MEFF – A model ensemble feature fusion approach for tackling adversarial attacks in medical imaging

Adversarial attacks pose a significant threat to deep learning models, specifically medical images, as they can mislead models into making inaccurate predictions by introducing subtle distortions to the input data that are often imperceptible to humans. Although adversarial training is a common technique used to mitigate these attacks on medical images, it lacks the flexibility to address new attack methods and effectively improve feature representation. This paper introduces a novel Model Ensemble Feature Fusion (MEFF) designed to combat adversarial attacks in medical image applications. The proposed model employs feature fusion by combining features extracted from different DL models and then trains Machine Learning classifiers using the fused features. It uses a concatenation method to merge the extracted features, forming a more comprehensive representation and enhancing the model's ability to classify classes accurately. Our experimental study has performed a comprehensive evaluation of MEFF, considering several challenging scenarios, including 2D and 3D images, greyscale and colour images, binary classification, and multi-label classification. The reported results demonstrate the robustness of using MEFF against different types of adversarial attacks across six distinct medical image applications. A key advantage of MEFF is its capability to incorporate a wide range of adversarial attacks without the need to train from scratch. Therefore, it contributes to developing a more diverse and robust defence strategy. More importantly, by leveraging feature fusion and ensemble modelling, MEFF enhances the resilience of DL models in the face of adversarial attacks, paving the way for improved robustness and reliability in medical image analysis.

A Parallel Convolutional Network Based on Spiking Neural Systems.

Deep convolutional neural networks have shown advanced performance in accurately segmenting images. In this paper, an SNP-like convolutional neuron structure is introduced, abstracted from the nonlinear mechanism in nonlinear spiking neural P (NSNP) systems. Then, a U-shaped convolutional neural network named SNP-like parallel-convolutional network, or SPC-Net, is constructed for segmentation tasks. The dual-convolution concatenate (DCC) and dual-convolution addition (DCA) network blocks are designed, respectively, in the encoder and decoder stages. The two blocks employ parallel convolution with different kernel sizes to improve feature representation ability and make full use of spatial detail information. Meanwhile, different feature fusion strategies are used to fuse their features to achieve feature complementarity and augmentation. Furthermore, a dual-scale pooling (DSP) module in the bottleneck is designed to improve the feature extraction capability, which can extract multi-scale contextual information and reduce information loss while extracting salient features. The SPC-Net is applied in medical image segmentation tasks and is compared with several recent segmentation methods on the GlaS and CRAG datasets. The proposed SPC-Net achieves 90.77% DICE coefficient, 83.76% IoU score and 83.93% F1 score, 86.33% ObjDice coefficient, 135.60 Obj-Hausdorff distance, respectively. The experimental results show that the proposed model can achieve good segmentation performance.

Drug–target affinity prediction with extended graph learning-convolutional networks

BackgroundHigh-performance computing plays a pivotal role in computer-aided drug design, a field that holds significant promise in pharmaceutical research. The prediction of drug–target affinity (DTA) is a crucial stage in this process, potentially accelerating drug development through rapid and extensive preliminary compound screening, while also minimizing resource utilization and costs. Recently, the incorporation of deep learning into DTA prediction and the enhancement of its accuracy have emerged as key areas of interest in the research community. Drugs and targets can be characterized through various methods, including structure-based, sequence-based, and graph-based representations. Despite the progress in structure and sequence-based techniques, they tend to provide limited feature information. Conversely, graph-based approaches have risen to prominence, attracting considerable attention for their comprehensive data representation capabilities. Recent studies have focused on constructing protein and drug molecular graphs using sequences and SMILES, subsequently deriving representations through graph neural networks. However, these graph-based approaches are limited by the use of a fixed adjacent matrix of protein and drug molecular graphs for graph convolution. This limitation restricts the learning of comprehensive feature representations from intricate compound and protein structures, consequently impeding the full potential of graph-based feature representation in DTA prediction. This, in turn, significantly impacts the models’ generalization capabilities in the complex realm of drug discovery.ResultsTo tackle these challenges, we introduce GLCN-DTA, a model specifically designed for proficiency in DTA tasks. GLCN-DTA innovatively integrates a graph learning module into the existing graph architecture. This module is designed to learn a soft adjacent matrix, which effectively and efficiently refines the contextual structure of protein and drug molecular graphs. This advancement allows for learning richer structural information from protein and drug molecular graphs via graph convolution, specifically tailored for DTA tasks, compared to the conventional fixed adjacent matrix approach. A series of experiments have been conducted to validate the efficacy of the proposed GLCN-DTA method across diverse scenarios. The results demonstrate that GLCN-DTA possesses advantages in terms of robustness and high accuracy.ConclusionsThe proposed GLCN-DTA model enhances DTA prediction performance by introducing a novel framework that synergizes graph learning operations with graph convolution operations, thereby achieving richer representations. GLCN-DTA does not distinguish between different protein classifications, including structurally ordered and intrinsically disordered proteins, focusing instead on improving feature representation. Therefore, its applicability scope may be more effective in scenarios involving structurally ordered proteins, while potentially being limited in contexts with intrinsically disordered proteins.

CONet: Crowd and occlusion-aware network for occluded human pose estimation

Human pose estimation has numerous applications in motion recognition, virtual reality, human–computer interaction, and other related fields. However, multi-person pose estimation in crowded and occluded scenes is challenging. One major issue about the current top-down human pose estimation approaches is that they are limited to predicting the pose of a single person, even when the bounding box contains multiple individuals. To address this problem, we propose a novel Crowd and Occlusion-aware Network (CONet) using a divide-and-conquer strategy. Our approach includes a Crowd and Occlusion-aware Head (COHead) which estimates the pose of both the occluder and the occluded person using two separate branches. We also use the attention mechanism to guide the branches for differentiated learning, aiming to improve feature representation. Additionally, we propose a novel interference point loss to enhance the model’s anti-interference ability. Our CONet is simple yet effective, and it outperforms the state-of-the-art model by +1.6 AP, achieving 71.6 AP on CrowdPose. Our proposed model has achieved state-of-the-art results on the CrowdPose dataset, demonstrating its effectiveness in improving the accuracy of human pose estimation in crowded and occluded scenes. This achievement highlights the potential of our model in many real-world applications where accurate human pose estimation is crucial, such as surveillance, sports analysis, and human–computer interaction.

A Hierarchical Graph V-Net With Semi-Supervised Pre-Training for Histological Image Based Breast Cancer Classification.

Numerous patch-based methods have recently been proposed for histological image based breast cancer classification. However, their performance could be highly affected by ignoring spatial contextual information in the whole slide image (WSI). To address this issue, we propose a novel hierarchical Graph V-Net by integrating 1) patch-level pre-training and 2) context-based fine-tuning, with a hierarchical graph network. Specifically, a semi-supervised framework based on knowledge distillation is first developed to pre-train a patch encoder for extracting disease-relevant features. Then, a hierarchical Graph V-Net is designed to construct a hierarchical graph representation from neighboring/similar individual patches for coarse-to-fine classification, where each graph node (corresponding to one patch) is attached with extracted disease-relevant features and its target label during training is the average label of all pixels in the corresponding patch. To evaluate the performance of our proposed hierarchical Graph V-Net, we collect a large WSI dataset of 560 WSIs, with 30 labeled WSIs from the BACH dataset (through our further refinement), 30 labeled WSIs and 500 unlabeled WSIs from Yunnan Cancer Hospital. Those 500 unlabeled WSIs are employed for patch-level pre-training to improve feature representation, while 60 labeled WSIs are used to train and test our proposed hierarchical Graph V-Net. Both comparative assessment and ablation studies demonstrate the superiority of our proposed hierarchical Graph V-Net over state-of-the-art methods in classifying breast cancer from WSIs. The source code and our annotations for the BACH dataset have been released at https://github.com/lyhkevin/Graph-V-Net.

A novel vision transformer network for rolling bearing remaining useful life prediction

The accurate predictions of remaining useful life (RUL) have become a key and extremely challenging problem. Due to the limitations of the classical convolutional neural network and recurrent neural network structure, the attention mechanism has been introduced to improve feature representation of the long-term bearing degradation data. Transformer network based on attention mechanism is successfully applied in many fields and recognized as an excellent creation for deep learning models. In this paper, a novel lightweight mobile vision transformer (MobileViT) architecture based on deep networks is proposed for the RUL predictions. This new network is named prognostics separable vision Transformer (ProgSViT), which combines the separable convolution and MobileViT. In ProgSViT network, the separable convolutions are first constructed for extracting local feature from the input vibration signal, and the new vision transformer architecture is proposed to learn the global feature representations. In improved MobileViT model, the loss function is optimized, and a new training strategy is provided. Finally, the obtained features are input to the global average pool layers and the full connection layers to perform RUL estimation. Experiment results present the proposed ProgSViT network surpasses the other models in RUL predicting, which possesses higher precision and computational efficiency.

RC-Net: A region-level context network for hyperreflective dots segmentation in retinal OCT images

Due to chronic exposure to hyperglycemia, the ocular biomechanical homeostasis of diabetics is disturbed, which may cause multiple ocular diseases, such as diabetic retinopathy (DR). In retinal optical coherence tomography (OCT) images of patients with DR, hyperreflective dots (HRDs) are spot-shaped areas with high reflectivity and high local contrast. Quantitative analyses of HRDs are the key to make appropriate decisions on diagnosis and treatment. The small scale, similarity to background and interference of other foci bring great challenges to accurate and automatic segmentation of HRDs. To address above challenges, we propose a novel region-level context network (RC-Net), which introduces region-level position attention module (RPAM) and region-level channel attention module (RCAM) for HRDs segmentation. Four RPAMs are embedded behind ordinary convolution blocks to extract rich contextual dependencies, which help to cluster pixels in same class and reveal differences between HRDs and background. To further improve feature representation, a RCAM is embedded into the top of the encoder structure to adaptively utilize the semantic dependencies between different channels. We evaluate the proposed RC-Net on a dataset, which consists of 450 2D OCT B-scans. The mean and standard deviation of dice similarity coefficient (DSC), intersection over union (IoU), recall (R) and precision (P) are 75.29±0.42%, 62.27±0.54%, 78.36±1.79% and 75.34±0.52%, respectively. The experiments show that the proposed RC-Net achieves superior segmentation performance of HRDs segmentation to state-of-the-art methods.

Deep Neural Decision Forest (DNDF): A Novel Approach for Enhancing Intrusion Detection Systems in Network Traffic Analysis.

Intrusion detection systems, also known as IDSs, are widely regarded as one of the most essential components of an organization's network security. This is because IDSs serve as the organization's first line of defense against several cyberattacks and are accountable for accurately detecting any possible network intrusions. Several implementations of IDSs accomplish the detection of potential threats throughout flow-based network traffic analysis. Traditional IDSs frequently struggle to provide accurate real-time intrusion detection while keeping up with the changing landscape of threat. Innovative methods used to improve IDSs' performance in network traffic analysis are urgently needed to overcome these drawbacks. In this study, we introduced a model called a deep neural decision forest (DNDF), which allows the enhancement of classification trees with the power of deep networks to learn data representations. We essentially utilized the CICIDS 2017 dataset for network traffic analysis and extended our experiments to evaluate the DNDF model's performance on two additional datasets: CICIDS 2018 and a custom network traffic dataset. Our findings showed that DNDF, a combination of deep neural networks and decision forests, outperformed reference approaches with a remarkable precision of 99.96% by using the CICIDS 2017 dataset while creating latent representations in deep layers. This success can be attributed to improved feature representation, model optimization, and resilience to noisy and unbalanced input data, emphasizing DNDF's capabilities in intrusion detection and network security solutions.

An Improvements of Deep Learner Based Human Activity Recognition with the Aid of Graph Convolution Features

Many researchers are now focusing on Human Action Recognition (HAR), which is based on various deep-learning features related to body joints and their trajectories from videos. Among many schemes, Joints and Trajectory-pooled 3D-Deep Geometric Positional Attention-based Hierarchical Bidirectional Recurrent convolutional Descriptors (JTDGPAHBRD) can provide a video descriptor by learning geometric features and trajectories of the body joints. But the spatial-temporal dynamics of the different geometric features of the skeleton structure were not explored deeper. To solve this problem, this article develops the Graph Convolutional Network (GCN) in addition to the JTDGPAHBRD to create a video descriptor for HAR. The GCN can obtain complementary information, such as higher-level spatial-temporal features, between consecutive frames for enhancing end-to-end learning. In addition, to improve feature representation ability, a search space with several adaptive graph components is created. Then, a sampling and computation-effective evolution scheme are applied to explore this space. Moreover, the resultant GCN provides the temporal dynamics of the skeleton pattern, which are fused with the geometric features of the skeleton body joints and trajectory coordinates from the JTDGPAHBRD to create a more effective video descriptor for HAR. Finally, extensive experiments show that the JTDGPAHBRD-GCN model outperforms the existing HAR models on the Penn Action Dataset (PAD).

Learning High-Order Multi-View Representation by New Tensor Canonical Correlation Analysis

Canonical correlation analysis (CCA) has attracted great interest in multi-view representation. However, most of the CCA methods heavily rely on the matrix structure, which may neglect the prior geometric information in high-order data. To deal with the above issue, we first propose a novel tensor CCA formulation with orthogonality, called TCCA-O, based on the Tucker decomposition to preserve the orthogonality. Then, we incorporate a structured sparse regularization term into the TCCA-O, called TCCA-OS, to improve feature representation. In addition, we develop an efficient alternating direction method of multipliers (ADMM)-based algorithm to solve TCCA-OS and conduct numerical comparisons on four public datasets. The results validate the advantages of the proposed methods in terms of classification accuracy, parameter sensitivity, noise robustness, and model stability. In particular, TCCA-O and TCCA-OS improve the classification accuracy by at least 10.03% and 10.36%, respectively, over the state-of-the-art CCA methods on the Caltech101-7 dataset.

Medimatrix: innovative pre-training of grayscale images for rheumatoid arthritis diagnosis revolutionises medical image classification.

Efficient and accurate medical image classification (MIC) methods face two major challenges: (1) high similarity between images of different disease classes; and (2) generating large medical image datasets for training deep neural networks is challenging due to privacy restrictions and the need for expert ground truth annotations. In this paper, we introduce a novel deep learning method called pre-training grayscale images with supervised learning for MIC (MediMatrix). Instead of pre-training on color ImageNet, our approach uses MediMatrix on grayscale ImageNet. To improve the performance of the network, we introduce ShuffleAttention (SA), a self-attention mechanism. By combining SA with the multiple residual structure (ResSA block) and replacing short-cut connections with dense residual connections between corresponding layers (densepath), our network can dynamically adjust channel attention weights and receive image inputs of different sizes, resulting in improved feature representation and better discrimination of similarities between different categories. MediMatrix effectively classifies X-ray images of rheumatoid arthritis (RA), enabling efficient screening without the need for expert analysis or invasive testing. Through extensive experiments, we demonstrate the superiority of MediMatrix over state-of-the-art methods and that color is not critical for rich natural image classification. Our results highlight the potential of computer-aided diagnosis combined with MediMatrix as a valuable screening tool for early detection and intervention in RA.

LCReg: Long-tailed image classification with Latent Categories based Recognition

In this work, we tackle the challenging problem of long-tailed image recognition. Previous long-tailed recognition approaches mainly focus on data augmentation or re-balancing strategies for the tail classes to give them more attention during model training. However, these methods are limited by the small number of training images for the tail classes, which results in poor feature representations. To address this issue, we propose the Latent Categories based long-tail Recognition (LCReg) method. Our hypothesis is that common latent features shared by head and tail classes can be used to improve feature representation. Specifically, we learn a set of class-agnostic latent features shared by both head and tail classes, and then use semantic data augmentation on the latent features to implicitly increase the diversity of the training sample. We conduct extensive experiments on five long-tailed image recognition datasets, and the results show that our proposed method significantly improves the baselines.

MemFormer: A memory based unified model for anomaly detection on metro railway tracks

Anomaly detection on metro railway tracks is crucial to maintain the safety of train transportation. Unsupervised methods require separate models for various scenarios, rendering them unsuitable for unified detection in the variable trackway scenario. The problem of “abnormal reconstruction” persists in the unified detection model, which preserves abnormal features in the output and hinders the model’s ability to recognize anomalies. In addition, real-time capability is still a huge challenge currently faced. In this study, we present a unified model, MemFormer, that employs a memory module, layer-wise normal queries, and a cascade convolution-based multi-head self-attention mechanism (CC-MHSA) to overcome the aforementioned issues. Firstly, a memory module is constructed to store diverse normal features that aid in the uniform modeling of a decision boundary for various scenarios. Secondly, we utilized the normal features existed in the memory module and layer-wise normal queries to optimize the attention mechanism, which suppresses the reconstruction of abnormal features. Thirdly, the proposed CC-MHSA benefits from cascaded convolution to improve feature representation and reduce parameter size, thereby reshaping self-attention calculation and reducing model inference time. Under the unified case, our method achieved detection accuracy of {99.2%, 97.5%} and localization accuracy of {99.7%, 97.1%}, respectively, on the metro trackway foreign object anomaly detection dataset and MVTec-AD dataset. The model infer speed is 47.6 FPS, exceeding that of the state-of-the-art alternatives.