Encoder Module Research Articles

Attention-based cross-frequency graph convolutional network for driver fatigue estimation.

Fatigue driving significantly contributes to global vehicle accidents and fatalities, making driver fatigue level estimation crucial. Electroencephalography (EEG) is a proven reliable predictor of brain states. With Deep Learning (DL) advancements, brain state estimation algorithms have improved significantly. Nonetheless, EEG's multi-domain nature and the intricate spatial-temporal-frequency correlations among EEG channels present challenges in developing precise DL models. In this work, we introduce an innovative Attention-based Cross-Frequency Graph Convolutional Network (ACF-GCN) for estimating drivers' reaction times using EEG signals from theta, alpha, and beta bands. This method utilizes a multi-head attention mechanism to detect long-range dependencies between EEG channels across frequencies. Concurrently, the transformer's encoder module learns node-level feature maps from the attention-score matrix. Subsequently, the Graph Convolutional Network (GCN) integrates this matrix with feature maps to estimate driver reaction time. Our validation on a publicly available dataset shows that ACF-GCN outperforms several state-of-the-art methods. We also explore the brain dynamics within the cross-frequency attention-score matrix, identifying theta and alpha bands as key influencers in fatigue estimating performance. The ACF-GCN method advances brain state estimation and provides insights into the brain dynamics underlying multi-channel EEG signals.

Dance2Music-Diffusion: leveraging latent diffusion models for music generation from dance videos

With the rapid development of social networks, short videos have become a popular form of content, especially dance videos. In this context, research on automatically generating music for dance videos shows significant practical value. However, existing studies face challenges such as limited richness in music timbre and lack of synchronization with dance movements. In this paper, we propose Dance2Music-Diffusion, a novel framework for music generation from dance videos using latent diffusion models. Our approach includes a motion encoder module for extracting motion features and a music diffusion generation module for generating latent music representations. By integrating dance type monitoring and latent diffusion techniques, our framework outperforms existing methods in generating complex and rich dance music. We conducted objective and subjective evaluations of the results produced by various existing models on the AIST++ dataset. Our framework shows outstanding performance in terms of beat recall rate, consistency with GT beats, and coordination with dance movements. This work represents the state of the art in automatic music generation from dance videos, is easy to train, and has implications for enhancing entertainment experiences and inspiring innovative dance productions. Sample videos of our generated music and dance can be viewed at https://youtu.be/eCvLdLdkX-Y. The code is available at https://github.com/hellto/dance2music-diffusion.

STFM: Accurate Spatio-Temporal Fusion Model for Weather Forecasting

Meteorological prediction is crucial for various sectors, including agriculture, navigation, daily life, disaster prevention, and scientific research. However, traditional numerical weather prediction (NWP) models are constrained by their high computational resource requirements, while the accuracy of deep learning models remains suboptimal. In response to these challenges, we propose a novel deep learning-based model, the Spatiotemporal Fusion Model (STFM), designed to enhance the accuracy of meteorological predictions. Our model leverages Fifth-Generation ECMWF Reanalysis (ERA5) data and introduces two key components: a spatiotemporal encoder module and a spatiotemporal fusion module. The spatiotemporal encoder integrates the strengths of convolutional neural networks (CNNs) and recurrent neural networks (RNNs), effectively capturing both spatial and temporal dependencies. Meanwhile, the spatiotemporal fusion module employs a dual attention mechanism, decomposing spatial attention into global static attention and channel dynamic attention. This approach ensures comprehensive extraction of spatial features from meteorological data. The combination of these modules significantly improves prediction performance. Experimental results demonstrate that STFM excels in extracting spatiotemporal features from reanalysis data, yielding predictions that closely align with observed values. In comparative studies, STFM outperformed other models, achieving a 7% improvement in ground and high-altitude temperature predictions, a 5% enhancement in the prediction of the u/v components of 10 m wind speed, and an increase in the accuracy of potential height and relative humidity predictions by 3% and 1%, respectively. This enhanced performance highlights STFM’s potential to advance the accuracy and reliability of meteorological forecasting.

Locus coeruleus modulation of prefrontal dynamics and encoding of flexible rule switching.

Behavioral flexibility, the ability to adjust behavioral strategies in response to changing environmental contingencies and internal demands, is fundamental to cognitive functions. Despite a large body of pharmacology and lesion studies, the underlying neurophysiological correlates and mechanisms that support flexible rule switching are under active investigation. To address this question, we trained mice to distinguish complex sensory cues comprising different perceptual dimensions (set shifting). Endoscopic calcium imaging revealed that medial prefrontal cortex (mPFC) neurons exhibited pronounced dynamic changes during rule switching. Notably, prominent encoding capacity in the mPFC was associated with switching across, but not within perceptual dimensions. We then showed the functional importance of the ascending input from the locus coeruleus (LC), as LC inhibition impaired rule switching behavior and impeded mPFC dynamic processes and encoding. Our results highlight the pivotal role of the mPFC in set shifting processes and demonstrate the profound impact of ascending neuromodulation on shaping prefrontal neural dynamics and behavioral flexibility.

SDCINet: A novel cross-task integration network for segmentation and detection of damaged/changed building targets with optical remote sensing imagery

Buildings are primary locations for human activities and key focuses in the military domain. Rapidly detecting damaged/changed buildings (DCB) and conducting detailed assessments can effectively aid urbanization monitoring, disaster response, and humanitarian assistance. Currently, the tasks of object detection (OD) and change detection (CD) for DCB are almost independent of each other, making it difficult to simultaneously determine the location and details of changes. Based on this, we have designed a cross-task network called SDCINet, which integrates OD and CD, and have created four dual-task datasets focused on disasters and urbanization. SDCINet is a novel deep learning dual-task framework composed of a consistency encoder, differentiation decoder, and cross-task global attention collaboration module (CGAC). It is capable of modeling differential feature relationships based on bi-temporal images, performing end-to-end pixel-level prediction, and object bounding box regression. The bi-direction traction function of CGAC is used to deeply couple OD and CD tasks. Additionally, we collected bi-temporal images from 10 locations worldwide that experienced earthquakes, explosions, wars, and conflicts to construct two datasets specifically for damaged building OD and CD. We also constructed two datasets for changed building OD and CD based on two publicly available CD datasets. These four datasets can serve as data benchmarks for dual-task research on DCB. Using these datasets, we conducted extensive performance evaluations of 18 state-of-the-art models from the perspectives of OD, CD, and instance segmentation. Benchmark experimental results demonstrated the superior performance of SDCINet. Ablation experiments and evaluative analyses confirmed the effectiveness and unique value of CGAC.

OAR-UNet: Enhancing Long-Distance Dependencies for Head and Neck OAR Segmentation

Accurate segmentation of organs at risk (OARs) is a crucial step in the precise planning of radiotherapy for head and neck tumors. However, manual segmentation methods using CT images, which are still predominantly applied in clinical settings, are inefficient and expensive. Additionally, existing segmentation methods struggle with small organs and have difficulty managing the complex interdependencies between organs. To address these issues, this study proposed an OAR-UNet segmentation method based on a U-shaped architecture with two key designs. To tackle the challenge of segmenting small organs, a Local Feature Perception Module (LFPM) is developed to enhance the sensitivity of the method to subtle structures. Furthermore, a Cross-shaped Transformer Block (CSTB) with a cross-shaped attention mechanism is introduced to improve the ability of the model to capture and process long-distance dependency information. To accelerate the convergence of the Transformer, we designed a Local Encoding Module (LEM) based on depthwise separable convolutions. In our experimental evaluation, we utilized two publicly available datasets, SegRap2023 and PDDCA, achieving Dice coefficients of 78.22% and 89.42%, respectively. These results demonstrate that our method outperforms both previous classic methods and state-of-the-art (SOTA) methods.

CvFormer: Cross-view transFormers with pre-training for fMRI analysis of human brain

In recent years, functional magnetic resonance imaging (fMRI) has been widely utilized to diagnose neurological disease, by exploiting the region of interest (RoI) nodes as well as their connectivities in human brain. However, most of existing works only rely on either RoIs or connectivities, neglecting the potential for complementary information between them. To address this issue, we study how to discover the rich cross-view information in fMRI data of human brain. This paper presents a novel method for cross-view analysis of fMRI data of the human brain, called Cross-view transFormers (CvFormer). CvFormer employs RoI and connectivity encoder modules to generate two separate views of the human brain, represented as RoI and sub-connectivity tokens. Then, basic transformer modules can be used to process the RoI and sub-connectivity tokens, and cross-view modules integrate the complement information across two views. Furthermore, CvFormer uses a global token for each branch as a query to exchange information with other branches in cross-view modules, which only requires linear time for both computational and memory complexity instead of quadratic time. To enhance the robustness of the proposed CvFormer, we propose a two-stage strategy to train its parameters. To be specific, RoI and connectivity views can be firstly utilized as self-supervised information to pre-train the CvFormer by combining it with contrastive learning and then fused to finetune the CvFormer using label information. Experiment results on two public ABIDE and ADNI datasets can show clear improvements by the proposed CvFormer, which can validate its effectiveness and superiority.

Tissue segmentation for traumatic brain injury based on multimodal MRI image fusion-semantic segmentation

Accurate segmentation of traumatic brain injury (TBI) has great significance for physicians to diagnose and assess a patient’s condition. The utilization of multimodal information plays a critical role in TBI segmentation. However, most of the existing methods mainly focus on direct extraction and selection of deep semantic features, whereas in this paper, we use image fusion as an auxiliary task for feature learning based on multimodal feature extraction to achieve more sufficient fusion of multimodal features. Therefore, we design a multimodal image fusion-semantic segmentation based framework. The proposed approach mainly consists of a semantic encoder module, a semantic segmentation module and an image fusion module. The semantic encoder compresses the input image into a smaller feature space to extract semantic features. The semantic segmentation module utilizes both the detailed information extracted by the encoder and the semantic information of high-level features extracted from the semantic segmentation module to generate the segmentation results. The image fusion module fuses semantic feature information from different modalities as an auxiliary task to semantic segmentation. Furthermore, to enhance the model’s performance even further, an uncertainty-based approach is employed, which dynamically adjusts the loss weights for the image fusion task and the semantic segmentation task during the model training process. The proposed method is evaluated on a private dataset, and compared with other widely recognized methods. It demonstrates outstanding performance in both Dice score and Recall metrics.

STSNet: A cross-spatial resolution multi-modal remote sensing deep fusion network for high resolution land-cover segmentation

Recently, deep learning models have found extensive application in high-resolution land-cover segmentation research. However, the most current research still suffers from issues such as insufficient utilization of multi-modal information, which limits further improvement in high-resolution land-cover segmentation accuracy. Moreover, differences in the size and spatial resolution of multi-modal datasets collectively pose challenges to multi-modal land-cover segmentation. Therefore, we propose a high-resolution land-cover segmentation network (STSNet) with cross-spatial resolution spatio-temporal-spectral deep fusion. This network effectively utilizes spatio-temporal-spectral features to achieve information complementary among multi-modal data. Specifically, STSNet consists of four components: (1) A high resolution and multi-scale spatial-spectral encoder to jointly extract subtle spatial-spectral features in hyperspectral and high spatial resolution images. (2) A long-term spatio-temporal encoder formulated by spectral convolution and spatio-temporal transformer block to simultaneously delineates the spatial, temporal and spectral information in dense time series Sentinel-2 imagery. (3) A cross-resolution fusion module to alleviate the spatial resolution differences between multi-modal data and effectively leverages complementary spatio-temporal-spectral information. (4) A multi-scale decoder integrates multi-scale information from multi-modal data. We utilized airborne hyperspectral remote sensing imagery from the Shenyang region of China in 2020, with a spatial resolution of 1authors declare that they have no known competm, a spectral number of 249, and a spectral resolution ≤ 5 nm, and its Sentinel dense time-series images acquired in the same period with a spatial resolution of 10 m, a spectral number of 10, and a time-series number of 31. These datasets were combined to generate a multi-modal dataset called WHU-H2SR-MT, which is the first open accessed large-scale high spatio-temporal-spectral satellite remote sensing dataset (i.e., with >2500 image pairs sized 300 m × 300 m for each). Additionally, we employed two open-source datasets to validate the effectiveness of the proposed modules. Extensive experiments show that our multi-scale spatial-spectral encoder, spatio-temporal encoder, and cross-resolution fusion module outperform existing state-of-the-art (SOTA) algorithms in terms of overall performance on high-resolution land-cover segmentation. The new multi-modal dataset will be made available at http://irsip.whu.edu.cn/resources/resources_en_v2.php, along with the corresponding code for accessing and utilizing the dataset at https://github.com/RS-Mage/STSNet.

TDT-MIL: a framework with a dual-channel spatial positional encoder for weakly-supervised whole slide image classification.

The classic multiple instance learning (MIL) paradigm is harnessed for weakly-supervised whole slide image (WSI) classification. The spatial position relationship located between positive tissues is crucial for this task due to the small percentage of these tissues in billions of pixels, which has been overlooked by most studies. Therefore, we propose a framework called TDT-MIL. We first serially connect a convolutional neural network and transformer for basic feature extraction. Then, a novel dual-channel spatial positional encoder (DCSPE) module is designed to simultaneously capture the complementary local and global positional information between instances. To further supplement the spatial position relationship, we construct a convolutional triple-attention (CTA) module to attend to the inter-channel information. Thus, the spatial positional and inter-channel information is fully mined by our model to characterize the key pathological semantics in WSI. We evaluated TDT-MIL on two publicly available datasets, including CAMELYON16 and TCGA-NSCLC, with the corresponding classification accuracy and AUC up to 91.54%, 94.96%, and 90.21%, 94.36%, respectively, outperforming state-of-the-art baselines. More importantly, our model possesses a satisfactory capability in solving the imbalanced WSI classification task using an ingenious but interpretable structure.

LLMs: Their Past, Promise, and Problems

Transformer-based large language models are currently at the forefront of modern artificial intelligence. Their prominence followed from the seminal paper Attention is All You Need [1]. Vaswani and his colleagues suggested placing attention mechanisms within the encoder and decoder modules of autoencoders rather than using them to focus between these two modules. In this paper we present first the seminal insights of early AI that lead to deep learning. We then describe the mathematical tools necessary for understanding the current generation of LLMs and follow this with a brief description of the transformer architecture. We then provide examples of LLMs in action and conclude with some observations of their promise and problems.

Intelligent fault diagnosis of rotating machine via Expansive dual-attention fusion Transformer enhanced by semi-supervised learning

The precise identification of faults is an essential component in the effective operation and maintenance of rotating machinery (RM). The increasing adoption of deep learning in this field is likely due to the vast amounts of automatically generated monitoring data and its robust ability to learn and detect errors effectively. Deep learning techniques generally demonstrate high efficacy, particularly when limited fault data is available or when employing a two-stage diagnostic approach, involving manual feature extraction followed by fault classification. To address these challenges, we introduce the Expansive Dual-Attention Fusion Transformer (EDAF-Transformer), which is trained through a semi-supervised approach using minimal labeled data. Our method begins with a detailed explanation of the foundational theories behind transformers and semi-supervised learning strategies. This establishes the basis of our approach to intelligent fault diagnosis, particularly under the constraints of raw vibration signals and limited labeled data availability. We then present our novel model, the EDAF-Transformer , which comprises two main components: the Expansive Dual-Attention Enhanced Modules and the 1-D Twin Global Self-Attention Transformer Encoder Module. The first component employs an Expansive Dual-branch Attention Enhanced Module (EDAEM) within a dual-branch attention architecture. This design effectively enlarges the receptive field, capturing both global and local features. The second component, the Twin Sparse Self-Attention Fusion Module (TSSAFM), integrates Global Sparse Self-Attention with Squeeze-and-Excitation (SE) attention. This module is designed to enhance feature encoding capabilities while simultaneously reducing computational demands. Furthermore, we implement an uncertainty self-training semi-supervised strategy focused on balanced unlabeled sample selection. This approach significantly enhances the generalizability and performance of the EDAF-Transformer . We extensively tested our semi-supervised fault diagnosis method on publicly available motor bearing data and a specially curated transmission shaft dataset. The results from these tests demonstrate that our method surpasses other intelligent fault diagnosis methods in effectiveness.

Semantic-Aware Dynamic Generation Networks for Few-Shot Human-Object Interaction Recognition.

Recognizing human-object interaction (HOI) aims at inferring various relationships between actions and objects. Although great progress in HOI has been made, the long-tail problem and combinatorial explosion problem are still practical challenges. To this end, we formulate HOI as a few-shot task to tackle both challenges and design a novel dynamic generation method to address this task. The proposed approach is called semantic-aware dynamic generation networks (SADG-Nets). Specifically, SADG-Net first assigns semantic-aware task representations for different batches of data, which further generates dynamic parameters. It obtains the features that highlight intercategory discriminability and intracategory commonality adaptively. In addition, we also design a dual semantic-aware encoder module (DSAE-Module), that is, verb-aware and noun-aware branches, to yield both action and object prototypes of HOI for each task space, which generalizes to novel combinations by transferring similarities among interactions. Extensive experimental results on two benchmark datasets, that is, humans interacting with common objects (HICO)-FS and trento universal HOI (TUHOI)-FS, illustrate that our SADG-Net achieves superior performance over state-of-the-art approaches, which proves its impressive effectiveness on few-shot HOI recognition.

CTNet: a convolutional transformer network for EEG-based motor imagery classification

Brain-computer interface (BCI) technology bridges the direct communication between the brain and machines, unlocking new possibilities for human interaction and rehabilitation. EEG-based motor imagery (MI) plays a pivotal role in BCI, enabling the translation of thought into actionable commands for interactive and assistive technologies. However, the constrained decoding performance of brain signals poses a limitation to the broader application and development of BCI systems. In this study, we introduce a convolutional Transformer network (CTNet) designed for EEG-based MI classification. Firstly, CTNet employs a convolutional module analogous to EEGNet, dedicated to extracting local and spatial features from EEG time series. Subsequently, it incorporates a Transformer encoder module, leveraging a multi-head attention mechanism to discern the global dependencies of EEG's high-level features. Finally, a straightforward classifier module comprising fully connected layers is followed to categorize EEG signals. In subject-specific evaluations, CTNet achieved remarkable decoding accuracies of 82.52% and 88.49% on the BCI IV-2a and IV-2b datasets, respectively. Furthermore, in the challenging cross-subject assessments, CTNet achieved recognition accuracies of 58.64% on the BCI IV-2a dataset and 76.27% on the BCI IV-2b dataset. In both subject-specific and cross-subject evaluations, CTNet holds a leading position when compared to some of the state-of-the-art methods. This underscores the exceptional efficacy of our approach and its potential to set a new benchmark in EEG decoding.

Evaluating Named Entity Recognition: A comparative analysis of mono- and multilingual transformer models on a novel Brazilian corporate earnings call transcripts dataset

Since 2018, when the Transformer architecture was introduced, Natural Language Processing has gained significant momentum with pre-trained Transformer-based models that can be fine-tuned for various tasks. Most models are pre-trained on large English corpora, making them less applicable to other languages, such as Brazilian Portuguese. In our research, we identified two models pre-trained in Brazilian Portuguese (BERTimbau and PTT5) and two multilingual models (mBERT and mT5). BERTimbau and mBERT use only the Encoder module, while PTT5 and mT5 use both the Encoder and Decoder. Our study aimed to evaluate their performance on a financial Named Entity Recognition (NER) task and determine the computational requirements for fine-tuning and inference. To this end, we developed the Brazilian Financial NER (BraFiNER) dataset, comprising sentences from Brazilian banks’ earnings calls transcripts annotated using a weakly supervised approach. Additionally, we introduced a novel approach that reframes the token classification task as a text generation problem. After fine-tuning the models, we evaluated them using performance and error metrics. Our findings reveal that BERT-based models consistently outperform T5-based models. While the multilingual models exhibit comparable macro F1-scores, BERTimbau demonstrates superior performance over PTT5. In terms of error metrics, BERTimbau outperforms the other models. We also observed that PTT5 and mT5 generated sentences with changes in monetary and percentage values, highlighting the importance of accuracy and consistency in the financial domain. Our findings provide insights into the differing performance of BERT- and T5-based models for the NER task.

A lightweight CNN-Transformer network for pixel-based crop mapping using time-series Sentinel-2 imagery

Deep learning approaches have provided state-of-the-art performance in crop mapping. Recently, several studies have combined the strengths of two dominant deep learning architectures, Convolutional Neural Networks (CNNs) and Transformers, to classify crops using remote sensing images. Despite their success, many of these models utilize patch-based methods that require extensive data labeling, as each sample contains multiple pixels with corresponding labels. This leads to higher costs in data preparation and processing. Moreover, previous methods rarely considered the impact of missing values caused by clouds and no-observations in remote sensing data. Therefore, this study proposes a lightweight multi-stage CNN-Transformer network (MCTNet) for pixel-based crop mapping using time-series Sentinel-2 imagery. MCTNet consists of several successive modules, each containing a CNN sub-module and a Transformer sub-module to extract important features from the images, respectively. An attention-based learnable positional encoding (ALPE) module is designed in the Transformer sub-module to capture the complex temporal relations in the time-series data with different missing rates. Arkansas and California in the U.S. are selected to evaluate the model. Experimental results show that the MCTNet has a lightweight advantage with the fewest parameters and memory usage while achieving the superior performance compared to eight advanced models. Specifically, MCTNet obtained an overall accuracy (OA) of 0.968, a kappa coefficient (Kappa) of 0.951, and a macro-averaged F1 score (F1) of 0.933 in Arkansas, and an OA of 0.852, a Kappa of 0.806, and an F1 score of 0.829 in California. The results highlight the importance of each component of the model, particularly the ALPE module, which enhanced the Kappa of MCTNet by 4.2% in Arkansas and improved the model’s robustness to missing values in remote sensing data. Additionally, visualization results demonstrated that the features extracted from CNN and Transformer sub-modules are complementary, explaining the effectiveness of the MCTNet.

MBHFuse: A multi- branch heterogeneous global and local infrared and visible image fusion with differential convolutional amplification features

Fusing infrared and visible imagery aims to harness their complementary spectral data, enhancing output image quality, sharpness, and content. However, convolutional neural networks (CNNs) are not capable of capturing long-range dependencies, while the Transformer architecture faces the challenge of consuming huge computational resources. To address these critical challenges, this paper proposes a novel framework named MBHFuse, which employs a multi-branch heterogeneous global and local image fusion approach. To effectively extract global and local features, we design a multi-branch heterogeneous encoder module and introduce a differential convolution amplification module (DCAM) to further extract complementary information. Additionally, we devise a new loss function, incorporating multi-branch feature decomposition loss, intensity loss, gradient loss, mean squared error loss, and structural similarity loss, for training the proposed MBHFuse model. Through extensive experiments on public datasets, we demonstrate that the proposed framework outperforms other state-of-the-art (SOTA) methods in both qualitative and quantitative evaluations. Our code will be available at https://github.com/sunyichen1994/MBHFuse.

TADGCN: A Time-Aware Dynamic Graph Convolution Network for long-term traffic flow prediction

Traffic flow prediction is a crucial component of the Intelligent Transportation System (ITS) with substantial implications for traffic management and daily travel. While short-term traffic forecasting has shown promise, long-term prediction presents unique challenges. The existing short-term approaches fall short in long-term scenarios due to two primary issues: (1) ignoring modeling the interaction relationships among dynamic spatial dependencies, and (2) extracting long-term temporal dependencies without capturing the time information in-depth. To mitigate the above issues, we propose a novel model called TADGCN, a Time-Aware Dynamic Graph Convolution Network, to effectively perform long-term traffic flow prediction. TADGCN captures latent interactions among dynamic spatial dependencies using Attention-Driven Dynamic Adaptive Graph Convolution Network (ADAGCN) modules. The time-aware capability is effectively enhanced using Time-Aware Joint Multi-View Temporal Encoder (TJMTE) modules due to the construction of a time-aware matrix. We conduct diversified experiments on three real-world datasets (England, PEMS04, and PEMS08) to evaluate the performance of our proposed method TADGCN. The results reveal that TADGCN achieves the best accuracy in all cases against 16 competitive state-of-the-art baselines.

INA-Net: An integrated noise-adaptive attention neural network for enhanced medical image segmentation

Skin cancer, a growing health concern, poses a significant threat. Automated segmentation of skin lesions is crucial for precise clinical diagnosis and treatment. However, challenges such as hair interference, vascular occlusion, and ambiguous lesion boundaries in dermoscopic images complicate accurate segmentation. Recent research has focused on capturing multiscale-enhanced global information across spatial and channel dimensions to improve the segmentation of ambiguous boundaries. Despite these advancements, performance remains suboptimal in the presence of complex disturbances. To address this, we propose INA-Net, an Integrated Noise-Adaptive Attention Neural Network for enhanced medical image segmentation. INA-Net proposes a lightweight noise-range attention module that integrates local and simulated noise features with global information to improve robustness against complex mixed noises. Additionally, our proposed Edge-aware Spatial Attention (ESA) and Multi-scale Channel Attention (MCA) enhance feature information and boundary perception. The proposed Dynamic Noise Encoding module and Fourier Wavelet Analysis (FW-Parser) further improve robustness by handling high-frequency noise components. INA-Net effectively manages hair interference, vascular occlusion, and ambiguous lesion boundaries, thereby providing critical support for accurate lesion identification. Evaluations on the ISIC2016, ISIC2018, and PH2 datasets demonstrate outstanding performance, with IOU scores of 87.53%, 84.74%, and 85.30%, respectively. These results surpass those of other models with comparable complexity and size.

Multi-scale feature extraction and TrasMLP encoder module for ocean HABs segmentation

Due to tiny edge and texture details of harmful algae blooms(HABs), existing segmentation networks are not effective for HABs segmentation. In order to solve the above problems, this paper proposes a Multi-scale Feature extraction and TrasMLP Encoder Fusion-based network (MFTS). To tackle the complex morphological characteristics and the complex backgrounds of HABs, a TrasMlp module which can effectively identify long-range patterns and adapt network parameters is introduced, enabling accurately parsing of complex algae images. Secondly, the deep convolution module is constructed by combining deep separable convolution with a two-channel attention mechanism to separate the target region from the background. In addition, this paper proposes a Weighted Feature Fusion of Deep Convolution and INRS Encoder Module classification network(FDIR) is proposed to quantify the performance of the image segmentation network. The segmentation results on HABs dataset from AICO Lab show that our proposed MFTS model achieves a miou of 90.02%, outperforming the performance of classical segmentation networks such as U-Net and Mask R-CNN. Compared to original HABs dataset, the segmented result shows a 5.1% improvement in the classification accuracy of the FDIR model.