This research introduces an improved method for identifying colorectal tumors through a combination of deep convolutional neural networks (CNNs), transfer learning, and sophisticated image processing techniques used on histopathological images. The suggested ensemble-based on ResNet50 and enhanced with a dual attention mechanism-surpasses individual model architectures by enhancing both accuracy and interpretability, allowing the model to emphasize crucial tissue areas pertinent to diagnosis. Segmentation techniques, such as watershed and distance transform, are utilized to define tumor margins and possible lesion regions. The dataset, obtained from Kather et al. (2019), includes 5,000 histopathological images spanning eight unique categories (tumor, stroma, complex, lymph, debris, mucosa, adipose, empty). The experimental findings demonstrate impressive results, achieving a training accuracy of 98.74%, a validation accuracy of 94.35%, an F1-score of 0.94, a recall of 0.94, a precision of 0.95, a specificity of 0.96, and a Cohen's kappa score of 0.9354, signifying outstanding inter-class consensus. These results showcase the model's strength across different class distributions and emphasize its possible clinical value in aiding the early identification and management of colorectal cancer.

Abstract Fault diagnosis is essential for condition-based maintenance, directly impacting the reliability and safety of satellite power systems. The abundance of telemetry data enhances the performance of data-driven diagnostic methods. Transformer models address the limitation of traditional methods by capturing long-term dependencies and learning high-dimensional feature representations. However, Transformers cannot identify the importance of internal feature elements, and existing attention mechanisms often introduce redundant complexity, potentially degrading performance. To tackle this, we propose a Global Sparse Attention (GSA) mechanism that applies sparse encoding to two feature dimensions and fuses them via an outer product to generate a global attention mask. This design improves model efficiency, precisely identifies key feature regions, and enhances diagnostic accuracy. Experiments on a satellite power platform demonstrate that our method outperforms existing approaches.

Cell-penetrating peptides (CPPs) are short peptides consisting of 5 to 50 amino acids and useful for drug delivery and intracellular localization. Laboratory-based techniques are often lengthy and resource-intensive, whereas computational approaches offer a rapid and cost-effective solution. To address these limitations, this research introduces a predictive model called pCPPs-sADNN leveraging feature fusion, integrating embeddings from the protein pre-trained language models Protein Text-to-Text Transfer Transformer and Evolutionary Scale Modeling, along with Conjoint Triad Features. By combining the distinct derived feature sets, generates an enhanced and robust features vector. Furthermore, we employed Random Forest-based Recursive Feature Elimination for feature selection and used the Adaptive Synthetic Sampling Approach to address class imbalance by generating synthetic minority samples. The hybrid feature set was subsequently utilized to train a deep neural network enhanced with an attention mechanism. The proposed pCPPs-sADNN model achieved a high training accuracy of 98.58% and an AUC of 0.99. In evaluation on test dataset, pCPPs-sADNN demonstrated strong performance with an accuracy of 96.84% and an AUC of 0.99.

Deep learning-based approaches for integrating multi-omics data offer a novel perspective on cancer recurrence prediction. However, existing methods struggle to manage the complex relationships within multi-omics data and the intrinsic correlations between samples, leading to suboptimal prediction accuracy. To tackle these challenges, we propose a multi-omics cancer recurrence prediction model (MultCPM), which employs a multi-head attention mechanism to extract key information from biological pathways. Integrated with a hierarchical fusion module, the model performs layered integration of omics data to effectively capture their interdependence. Ultimately, the fused information is consolidated into a unified feature matrix, refining critical features and their relationships across omics. Results from 5-fold cross-validation, repeated five times on Breast Cancer (BRCA), Bladder Cancer (BLCA), and Liver Cancer (LIHC) datasets, demonstrate that the MultCPM model achieves superior prediction performance and robustness. Additionally, Deep SHapley Additive exPlanations (DeepSHAP) was utilized to analyze the model’s interpretability, revealing key genes closely associated with cancer recurrence, thus providing valuable insights for biological research and the development of cancer recurrence prediction algorithms. The code is publicly available at https://github.com/dowell2016/MultCPM .

Time Series Classification (TSC) plays a crucial role in machine learning applications across domains such as healthcare, finance, and industrial systems. In these domains, TSC requires accurate predictions and reliable explanations, as misclassifications may lead to severe consequences. In addition, scalability issues, including training time and memory consumption, are critical for practice usage. To address these challenges, we propose ProtoPGTN, a prototype-based interpretable framework that unifies gated transformers with prototype reasoning for scalable time series classification. Unlike existing prototype-based interpretable TSC models which rely on recurrent structure for sequence processing and Euclidean distance for similarity computation, ProtoPGTN adapts Gated Transformer Networks (GTN), which uses an attention mechanism to capture both temporal and spatial long-range dependencies in time series data and integrates the prototype learning framework from ProtoPNet with cosine similarity to enhance metric consistency and interpretability. Extensive experiments are conducted on 165 publicly available datasets from the UCR and UEA repositories, covering both univariate and multivariate tasks. Results show that ProtoPGTN obtains at least the same performance as existing prototype-based interpretable models on both multivariate and univariate datasets. The average accuracy on multivariate and univariate datasets stands at 67.69% and 76.99%, respectively. ProtoPGTN achieves up to 20× faster training and up to 200× lower memory consumption than existing prototype-based interpretable models.

Electrophysiological correlates of meaning-based attentional guidance mechanism as a function of cognitive loads in visual search for words.

Contrast and Gain-Aware Attention: A Plug-and-Play Feature Fusion Attention Module for Torso Region Fetal Plane Identification.

A lightweight deep learning model with attention mechanisms for hypertensive retinopathy classification.

Research on e-commerce special commodity recommendation system based on attention mechanism and Dense Net model

Static-dynamic class-level perception consistency in video semantic segmentation.

Humans demonstrate the ability to focus auditory attention in noisy environments, enabling them to concentrate on a specific speaker at a cocktail party. Neuroscientific research has shown that auditory attention itself is a dynamic brain activity that evolves over time, which has inspired studies on electroencephalography (EEG)-based auditory attention detection (AAD). This paper proposes a neural attention mechanism model named GSANet, which employs a self-attention mechanism to model the temporal dynamics of EEG signals while dynamically assigning weights to EEG channels through a graph attention mechanism. In brief, GSANet simulates the neural attention mechanisms of the human brain to extract discriminative representations from EEG signals for training high-performance classifiers. We conducted experiments on two public datasets, KUL and DTU, achieving overall decoding accuracies of 94.5% and 79.2%, respectively, under a 1-second decision window, significantly outperforming baseline models across all comparative conditions. The code of our proposed method will be available at: https://github.com/dalin6666/GSANet.

Memristor-Based Brain Emotional Learning Neural Network With Attention Mechanism and Its Application

Fault diagnosis of wind turbine based on dual-channel feature aggregation network with attentional mechanism

Fault diagnosis method for rolling bearing based on attention mechanism and BiTCN model

A multi-task deep attention network for simultaneous rapid quantification of sucrose, glucose, and fructose contents in pumpkin using FT-NIR spectroscopy.

Dominant preference decoupling and guided perturbed preference injection for cross-domain sequence recommendation.

A bearing fault diagnosis model with enhanced feature extraction based on the Kolmogorov–Arnold representation Theorem and an attention mechanism

Intelligent compressive strength prediction of sustainable rubberised concrete using an optimised interpretable deep CNN-LSTM model with attention mechanism

AI-Driven molecule generation and bioactivity prediction: A multi-model approach combining VAE, graph and language-based neural networks.

Camouflaged Object Segmentation (COS) faces significant challenges due to the scarcity of annotated data, where meticulous pixel-level annotation is both labor-intensive and costly, primarily due to the intricate object-background boundaries. Addressing the core question, "Can COS be effectively achieved in a zero-shot manner without manual annotations for any camouflaged object?", we propose an affirmative solution. We examine the learned attention patterns for camouflaged objects and introduce a robust zero-shot COS framework. Our findings reveal that while transformer models for salient object segmentation (SOS) prioritize global features in their attention mechanisms, camouflaged object segmentation exhibits both global and local attention biases. Based on these findings, we design a framework that adapts with the inherent local pattern bias of COS while incorporating global attention patterns and a broad semantic feature space derived from SOS. This enables efficient zero-shot transfer for COS. Specifically, We incorporate a Masked Image Modeling (MIM) based image encoder optimized for Parameter-Efficient Fine-Tuning (PEFT), a Multimodal Large Language Model (M-LLM), and a Multi-scale Fine-grained Alignment (MFA) mechanism. The MIM encoder captures essential local features, while the PEFT module learns global and semantic representations from SOS datasets. To further enhance semantic granularity, we leverage the M-LLM to generate caption embeddings conditioned on visual cues, which are meticulously aligned with multi-scale visual features via MFA. This alignment enables precise interpretation of complex semantic contexts. Moreover, we introduce a learnable codebook to represent the M-LLM during inference, significantly reducing computational demands while maintaining performance. Our framework demonstrates its versatility and efficacy through rigorous experimentation, achieving state-of-the-art performance in zero-shot COS with $F_{\beta }^{w}$Fβw scores of 72.9% on CAMO and 71.7% on COD10K. By removing the M-LLM during inference, we achieve an inference speed comparable to that of traditional end-to-end models, reaching 18.1 FPS. Additionally, our method excels in polyp segmentation, and underwater scene segmentation, outperforming challenging baselines in both zero-shot and supervised settings, thereby implying its potentiality in various segmentation tasks.