Encoder-decoder Model Research Articles

Fusing deep metric learning with KNN for 12-lead multi-labelled ECG classification

Electrocardiograms (ECG) provides essential information of the electrical activity of the heart which is important for the auto-diagnosis of cardiovascular arrhythmia. Although several state-of-art models based on deep learning have been proposed for auto-classification of cardiac arrhythmia, their performance efficiency still require further improvement. This paper presented an auto-detection algorithm based on the encoder-decoder model, where the encoder and decoder are responsible for feature extraction and classification respectively. The advantage of the developed model is at combining metric learning with residual neural networks, extracting discriminative features whilst reducing the complexity of network, therefore, increasing the computational efficiency of the model by reducing its computing demand. Moreover, it incorporates temporal features of RR intervals (i.e., the time interval between two successive R-wave peaks of ECG signals) into feature embeddings for feature fusion. Through evaluating the distance between the feature embeddings produced by the deep metric model, the K-Nearest Neighbours algorithm was implemented as a decoder to process the classification based on the feature embeddings. Additionally, the frame blocking pre-treats the raw ECG records, unifying the size of all records. By testing the developed model over 12-lead ECG recordings from the CPSC dataset, the proposed algorithm achieved an average F1 score of 0.874, outperforming other contemporary classification models with an increased F1 score by 0.09 in average. It also showed a promising efficiency with a small model size and GFLOPs. In conclusion, an efficient deep metric-based model with feature infusion for classifying multiple rhythmic states of the heart has been developed, showing promising potentials for future practical applications.

A Clinical Information Extractor Integrated with Global Semantic Features via Dynamic Attention Mechanism

Electronic medical records have been rolled out in the past decades to facilitate the medical exports daily routine. However, the number of electronic medical records increases dramatically, which also causes huge workloads for the front-line clinical workers when they face the writing-up work. In this sense, researchers in the artificial intelligence domain wish to automate this process by constructing a natural language processing system, and medical information extraction is one of the key steps amongst the entire work. In this paper, we focus on medical information extraction from doctor-patient dialogues, and propose a novel encoder-decoder model which incorporates global information into the dialogue windows. The experiment on the MIE dataset suggests our model outperforms the compared baseline models, and achieves the state-of-the-art results, which proves our models effectiveness.

Multilingual semantic fusion network for text recognition in the wild

Most current approaches in the literature of scene text recognition train the language model via a text dataset far sparser than in natural language processing, resulting in inadequate training. Therefore, we propose a simple transformer encoder–decoder model called the multilingual semantic fusion network (MSFN) that can leverage prior linguistic knowledge to learn robust language features. First, we label the text dataset with forward, backward sequences, and subwords, which are extracted by tokenization with linguistic information. Then we introduce a multilingual model to the decoder corresponding to three different channels of the labeled dataset. The final output is fused by different channels to get more accurate results. In experiments, MSFN achieves cutting-edge performance across six benchmark datasets, and extensive ablative studies have proven the effectiveness of the proposed method. Code is available at https://github.com/lclee0577/MLViT.

TAN: A Transferable Adversarial Network for DNN-Based UAV SAR Automatic Target Recognition Models

Recently, the unmanned aerial vehicle (UAV) synthetic aperture radar (SAR) has become a highly sought-after topic for its wide applications in target recognition, detection, and tracking. However, SAR automatic target recognition (ATR) models based on deep neural networks (DNN) are suffering from adversarial examples. Generally, non-cooperators rarely disclose any SAR-ATR model information, making adversarial attacks challenging. To tackle this issue, we propose a novel attack method called Transferable Adversarial Network (TAN). It can craft highly transferable adversarial examples in real time and attack SAR-ATR models without any prior knowledge, which is of great significance for real-world black-box attacks. The proposed method improves the transferability via a two-player game, in which we simultaneously train two encoder–decoder models: a generator that crafts malicious samples through a one-step forward mapping from original data, and an attenuator that weakens the effectiveness of malicious samples by capturing the most harmful deformations. Particularly, compared to traditional iterative methods, the encoder–decoder model can one-step map original samples to adversarial examples, thus enabling real-time attacks. Experimental results indicate that our approach achieves state-of-the-art transferability with acceptable adversarial perturbations and minimum time costs compared to existing attack methods, making real-time black-box attacks without any prior knowledge a reality.

Grasping Complex-Shaped and Thin Objects Using a Generative Grasping Convolutional Neural Network

Vision-based pose detection and grasping complex-shaped and thin objects are challenging tasks. We propose an architecture that integrates the Generative Grasping Convolutional Neural Network (GG-CNN) with depth recognition to identify a suitable grasp pose. First, we construct a training dataset with data augmentation to train a GG-CNN with only RGB images. Then, we extract a segment of the tool using a color segmentation method and use it to calculate an average depth. Additionally, we apply and evaluate different encoder–decoder models with a GG-CNN structure using the Intersection Over Union (IOU). Finally, we validate the proposed architecture by performing real-world grasping and pick-and-place experiments. Our framework achieves a success rate of over 85.6% for picking and placing seen surgical tools and 90% for unseen surgical tools. We collected a dataset of surgical tools and validated their pick and place with different GG-CNN architectures. In the future, we aim to expand the dataset of surgical tools and improve the accuracy of the GG-CNN.

Mutation prediction and phylogenetic analysis of SARS-CoV2 protein sequences using LSTM based encoder-decoder model

The ongoing evolution and mutation of SARS-CoV2 pose a significant challenge to the development of effective medication, as genetic changes can render previously developed drugs ineffective. To address this issue, researchers are exploring various strategies to predict and assess the emergence of novel SARS-CoV2 strains through phylogenetic analysis and mutation prediction. In recent years, deep learning approaches have been applied to studying viruses, including SARS-CoV2, to improve our understanding of virus evolution, structure, categorization, and prediction. In this study, a novel deep learning approach is proposed to predict and assess SARS-CoV2 protein sequences. Specifically, Long Short-Term Memory (LSTM) is utilized to predict protein sequences from aligned input sequences, with a bioinformatics tool used to detect mutations. The deep learning model proposed in this study exhibits high accuracy in predicting several key SARS-CoV2 protein sequences, including spike, replicase, putative, ORF1a, and nucleocapsid. The study uses genome sequencing data from the National Center for Biotechnology Information (NCBI) and demonstrates a 98% accuracy in predicting genomic sequences, with minimal changes observed in protein sequences. This study represents a significant improvement over previous research, which has focused only on predicting mutations in viral RNA sequences using datasets from other viruses.

Morphologically Motivated Input Variations and Data Augmentation in Turkish-English Neural Machine Translation

Success of neural networks in natural language processing has paved the way for neural machine translation (NMT), which rapidly became the mainstream approach in machine translation. Significant improvement in translation performance has been achieved with breakthroughs such as encoder-decoder networks, attention mechanism, and Transformer architecture. However, the necessity of large amounts of parallel data for training an NMT system and rare words in translation corpora are issues yet to be overcome. In this article, we approach NMT of the low-resource Turkish-English language pair. We employ state-of-the-art NMT architectures and data augmentation methods that exploit monolingual corpora. We point out the importance of input representation for the morphologically rich Turkish language and make a comprehensive analysis of linguistically and non-linguistically motivated input segmentation approaches. We prove the effectiveness of morphologically motivated input segmentation for the Turkish language. Moreover, we show the superiority of the Transformer architecture over attentional encoder-decoder models for the Turkish-English language pair. Among the employed data augmentation approaches, we observe back-translation to be the most effective and confirm the benefit of increasing the amount of parallel data on translation quality. This research demonstrates a comprehensive analysis on NMT architectures with different hyperparameters, data augmentation methods, and input representation techniques, and proposes ways of tackling the low-resource setting of Turkish-English NMT.

Solving Turkish math word problems by sequence-to-sequence encoder-decoder models

Solving math word problems (MWP) is a challenging task due to the semantic gap between natural language texts and mathematical equations. The main purpose of the task is to take a written math problem as input and produce a proper equation as output for solving that problem. This paper describes a sequence-to-sequence (seq2seq) neural model for automatically solving Turkish MWPs based on their semantic meanings in the text. It comprises a bidirectional encoder to comprehend the semantics of the problem by encoding the input sequence and a decoder with attention to extract the equation by tracking the semantic meanings of the output symbols. We investigate the success of several embedding types, pretrained language models, and neural models. Our research is novel in the sense that there exist no studies in Turkish on this natural language processing task that utilizes pretrained language models and neural models. There is also no Turkish dataset that can be used to train the neural models for the MWP task. As the first large-scale Turkish MWP dataset, we translated the well-known English MWP datasets into Turkish using a machine translation system. Although Turkish is an agglutinative and grammatically challenging language, the proposed models achieve around 72% accuracy on the dataset compiled from three English datasets.

Research on Short-term Time Series Forecasting Method Based on Feature Fusion of Electric Power

In the power industry, time series forecasting has many application scenarios, such as accurate electricity consumption forecasting and load forecasting, which can provide a reference for power grid staff and reduce decision-making costs. To improve the accuracy of short-term load forecasting, this paper proposes a hybrid forecasting model based on an improved encoder-decoder architecture model and a Temporal Convolutional Network. Based on the gate recurrent unit, the model in this paper introduces the encoder-decoder architecture, adds the attention mechanism for improving the prediction accuracy, and performs feature fusion with the Temporal Convolutional Network, building a dual-feature fusion model finally. Experiments show that the method proposed in this paper reduces the evaluation metrics RMSE, MAPE and MAE by 7.7%, 15.2%, and 13.9% respectively compared to the Long Short-Term Memory Networks. Compared with the Temporal Convolutional Network, it is reduced by 2.6%, 4.8%, and 4%.

Deep capsule encoder–decoder network for surrogate modeling and uncertainty quantification

AbstractWe propose a novel capsule‐based deep encoder–decoder model for surrogate modeling and uncertainty quantification of systems in mechanics from sparse data. The proposed framework is developed by adapting Capsule Network (CapsNet) architecture into an image‐to‐image regression encoder–decoder network. Specifically, the aim is to exploit the benefits of CapsNet over convolution neural network (CNN) – retaining pose and position information related to an entity to name a few. The performance of the proposed approach is illustrated by solving two different variants of elliptic partial differential equations (PDE): a stochastic version without a source term having an input dimensionality of 1024 and a deterministic pressure Poisson equation for flow past a cylinder. The first PDE, that is, the stochastic PDE (SPDE), also governs systems in mechanics such as steady heat conduction, groundwater flow, or other diffusion processes. In this article, the problem definition for this SPDE is such that it does not restrict the random diffusion field to a particular covariance structure, and the more strenuous task of response prediction for an arbitrary diffusion field is solved. Finally, we also evaluate the performance of our model on the uncertainty propagation problem based on this equation. The obtained results from the performance evaluation of the developed model on the mentioned problems indicate that the proposed approach is accurate, data efficient, and robust.

Exploiting multimodal CNN architecture for automated teeth segmentation on dental panoramic X-ray images.

Panoramic X-ray images are the major source used in field of dental image segmentation. However, such images suffers from the disturbances like low contrast, presence of jaw bones, nose bones, spinal bone, and artifacts. Thus, to observe these images manually is a tedious task, requires expertise of dentist and is time consuming. Hence, there is need to develop an automated tool for teeth segmentation. Recently, few deep models have been developed for dental image segmentation. But, such models possess large number of training parameters, thus making the segmentation a very complex task. Also, these models are based only on conventional CNN and lacks in exploiting multimodal CNN features for dental image segmentation. Thus, to address these issues, a novel encoder-decoder model based on multimodal-feature extraction for automatic segmentation of teeth area is proposed. The encoder has three different CNN based architectures: conventional CNN, atrous-CNN, and separable CNN to encode rich contextual information. Whereas decoder contains a single stream of deconvolutional layers for segmentation. The proposed model is tested on 1500 panoramic X-ray images and uses very less parameters when compared to state-of-the-art methods. Besides this, the precision and recall are 95.01% and 94.06%, which out performs the state-of-the art methods.

Series-wise attention network for wind power forecasting considering temporal lag of numerical weather prediction

Numerical Weather Prediction (NWP), which provides approximate weather information in the next few days, is an essential feature in wind power forecasting (WPF). However, the forecasted wind speed in NWP (NWPWS) shows temporal lag compared to the actual wind speed. And the time-varying nature of the temporal lag challenges the WPF methods on selecting the NWP temporal features within valid time windows. To this end, this paper first summarizes the characteristics of the temporal lag and finds the weak inertia property within a few hours. Consequently, a series-wise mechanism, temporal lag attention (TLA), is proposed to extract valid NWP information for the ultra-short-term WPF by considering the temporal lag’s interference. Three fundamental parts included in TLA, Block-Sparse Attention Range, Lag Recognition, and Feature Fusion, are conducted sequentially. Block-Sparse Attention Range firstly screens out the critical range from the wide distribution of the temporal lag to simplify the subsequent calculation; Lag Recognition real-timely compares the series-wise similarity between actual wind speed and NWPWS time series to determine the top-k most likely temporal lags with their probability; Feature Fusion finally generates the lag-fixed weighted NWPWS as the valid NWP information. TLA has good scalability and can be integrated into the matured WPF model as the feature processing module. Real-world cases verify the accuracy improvement by integrating TLA into a modified encoder–decoder model (MED). Compared to the common point-wise attention mechanism, TLA can mitigate the negative influence of NWPWS temporal lag on WPF more effectively. Meanwhile, the sparse attention range is also proven to benefit the higher WPF accuracy and lower training cost.

A quantile-based encoder-decoder framework for multi-step ahead runoff forecasting

Deep neural network (DNN) models have become increasingly popular in the hydrology community. However, most studies are related to (rainfall-) runoff simulation and comparatively fewer studies have focused on runoff forecasting. In this study, quantile-based (q = 0.05, 0.5, 0.95) encoder-decoder (ED) models that use long short-term memory network (LSTM) and dense network (DN) blocks were developed for three and five days ahead runoff forecasting. Through linear (LW) and non-linear (NLW) wavelet selection, hybrid models LSTM-DN, LSTM-DN-LW, LSTM-DN-NLW, ED, ED-LW, and ED-NLW were developed. For each lead time (LT = 3, 5) and value of q, different model configurations were created using different input lag lengths (IL = 15, 45, 180). The developed models were tested for runoff forecasting using three basins (with different characteristics) from the Catchment Attributes and MEteorology for Large-sample Studies (CAMELS) dataset. The models were compared using deterministic (e.g., the Kling-Gupta efficiency [KGE] metric) and probabilistic (e.g., reliability) statistical metrics. While the models showed high variability in performance across the three basins (KGE = 0.308–0.979 for the q = 0.5 models), very high accuracy (up to KGE = 0.979) was achieved for one of the basins with high snowmelt. The ED-NLW model was found to generally outperform the other models. Although the LSTM-DN model had the highest median KGE (0.434 across all configurations), the ED and ED-NLW models had higher reliability than LSTM-DN (90% and 91%, respectively, considering a 90% confidence level). Models coupled with NLW performed superior to those that used LW. All ED models had high reliability despite two of the basins achieving median KGE values of ∼ 0.390, highlighting that quantile-based models can generate reliable forecast intervals even when the KGE of the median forecast (q = 0.5) is low. An additional experiment generated synthetic precipitation forecasts with varying degrees of accuracy. The models were trained using accurate precipitation forecasts and tested using both accurate and inaccurate precipitation forecasts. While up to a 120% improvement in KGE was found when accurate precipitation forecasts were used as input to the models, using inaccurate precipitation forecasts resulted in a substantial decrease in reliability. Overall, the results of this study can serve as a benchmark for future studies developing probabilistic DNN models for runoff forecasting.

Graph neural networks for representing multivariate resource usage: A multiplayer mobile gaming case-study

The emergence of Multiplayer Mobile Gaming (MMG) applications is intertwined with a plethora of Quality of Service and Quality of Experience requirements. Resource usage prediction can provide valuable insights into the corresponding orchestration and management process in the form of several proactive functionalities in resource scaling, service migration, task offloading and scheduling. These processes are crucial in the Cloud and Edge environments exploited by MMG applications. Thus, producing accurate resource usage predictions concerning these types of applications is of paramount importance. To that end, we propose a resource usage representation paradigm based on Graph Neural Networks (GNNs). The novelty of this approach is based on the process of leveraging the dependencies that exist among the various types of computational resources. Furthermore, we expand upon this representation approach to develop a GNN-based Encoder-Decoder model that caters to the complexities of resource usage and can provide multi-step resource usage predictions. This model is compared against numerous well-established Encoder-Decoder and Deep Learning prediction models to assess its efficiency. Finally, the proposed model is incorporated in a proactive Horizontal Autoscaling solution that manages to outperform a standard reactive Horizontal Autoscaling approach in the context of a large-scale simulation, in terms of various performance metrics, while keeping the volume of the required computational resources to a minimum. The findings of this work showcase the importance of developing novel approaches in order to represent resource usage and the numerous benefits in the context of application performance and resource consumption that may derive from such scientific endeavors.

DapNet-HLA: Adaptive dual-attention mechanism network based on deep learning to predict non-classical HLA binding sites

Human leukocyte antigen (HLA) plays a vital role in immunomodulatory function. Studies have shown that immunotherapy based on non-classical HLA has essential applications in cancer, COVID-19, and allergic diseases. However, there are few deep learning methods to predict non-classical HLA alleles. In this work, an adaptive dual-attention network named DapNet-HLA is established based on existing datasets. Firstly, amino acid sequences are transformed into digital vectors by looking up the table. To overcome the feature sparsity problem caused by unique one-hot encoding, the fused word embedding method is used to map each amino acid to a low-dimensional word vector optimized with the training of the classifier. Then, we use the GCB (group convolution block), SENet attention (squeeze-and-excitation networks), BiLSTM (bidirectional long short-term memory network), and Bahdanau attention mechanism to construct the classifier. The use of SENet can make the weight of the effective feature map high, so that the model can be trained to achieve better results. Attention mechanism is an Encoder-Decoder model used to improve the effectiveness of RNN, LSTM or GRU (gated recurrent neural network). The ablation experiment shows that DapNet-HLA has the best adaptability for five datasets. On the five test datasets, the ACC index and MCC index of DapNet-HLA are 4.89% and 0.0933 higher than the comparison method, respectively. According to the ROC curve and PR curve verified by the 5-fold cross-validation, the AUC value of each fold has a slight fluctuation, which proves the robustness of the DapNet-HLA. The codes and datasets are accessible at https://github.com/JYY625/DapNet-HLA.

Modified Double U-Net Architecture for Medical Image Segmentation

Semantic segmentation is the process of labeling each pixel of an input image with its appropriate class. In biomedical image processing, semantic segmentation is a critical preprocessing step. After the evolution of deep learning, specifically, the U-Net, which is based on the encoder–decoder model, researchers have extensively used several modified versions of U-Net for semantic segmentation. This manuscript presents a novel architecture called modified Double U-Net for various biomedical image segmentation tasks. The modified Double U-Net takes full advantage of Double U-Net and ensemble learning. It is the combination of two U-Net architectures stacked on top of each other. The first U-Net uses an ensemble of pretrained Xception, DenseNet, and VGG-19. We have deliberately used a pretrained model so that the features learned from the ImageNet classification task can be reused for our semantic segmentation task. Another U-Net is stacked at the bottom to capture more information that can be used in our semantic segmentation task. The performance of our proposed model has been evaluated on five different medical datasets, namely, the Data Science Bowl Challenge-2018, the CVC-ClinicDB dataset, which has complex images, such as smaller and flat polyps, the ISIC-2018 challenge dataset, the Kvasir-Instrument dataset, and the INBreast dataset achieving state-of-the-art performance. Conducting fivefold cross-validation on the five datasets mentioned above intersection over union of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$89.13~\pm ~2.88$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$85.22~\pm ~1.26$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$83.13~\pm ~3.56$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$85.22~\pm ~1.26$ </tex-math></inline-formula> , and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$95.22~\pm ~2.56$ </tex-math></inline-formula> were obtained.

Unrolling a rain-guided detail recovery network for singleimage deraining

BackgroundOwing to the rapid development of deep networks, single-image deraining tasks have progressed significantly. Various architectures have been designed to recursively or directly remove rain, and most rain streaks can be removed using existing deraining methods. However, many of them cause detail loss, resulting in visual artifacts. MethodTo resolve this issue, we propose a novel unrolling rain-guided detail recovery network (URDRN) for single-image deraining based on the observation that the most degraded areas of a background image tend to be the most rain-corrupted regions. Furthermore, to address the problem that most existing deep-learningbased methods trivialize the observation model and simply learn end-to-end mapping, the proposed URDRN unrolls a single-image deraining task into two subproblems: rain extraction and detail recovery. ResultSpecifically, first, a context aggregation attention network is introduced to effectively extract rain streaks; thereafter, a rain attention map is generated as an indicator to guide the detail recovery process. For the detail recovery sub-network, with the guidance of the rain attention map, a simple encoder–decoder model is sufficient to recover the lost details. Experiments on several well-known benchmark datasets show that the proposed approach can achieve performance similar to those of other state-of-the-art methods.

A Weakly Supervised Learning Framework for Salient Object Detection via Hybrid Labels

Fully-supervised salient object detection (SOD) methods have made great progress, but such methods often rely on a large number of pixel-level annotations, which are time-consuming and labour-intensive. In this paper, we focus on a new weakly-supervised SOD task under hybrid labels, where the supervision labels include a large number of coarse labels generated by the traditional unsupervised method and a small number of real labels. To address the issues of label noise and quantity imbalance in this task, we design a new pipeline framework with three sophisticated training strategies. In terms of model framework, we decouple the task into label refinement sub-task and salient object detection sub-task, which cooperate with each other and train alternately. Specifically, the R-Net is designed as a two-stream encoder-decoder model equipped with Blender with Guidance and Aggregation Mechanisms (BGA), aiming to rectify the coarse labels for more reliable pseudo-labels, while the S-Net is a replaceable SOD network supervised by the pseudo labels generated by the current R-Net. Note that, we only need to use the trained S-Net for testing. Moreover, in order to guarantee the effectiveness and efficiency of network training, we design three training strategies, including alternate iteration mechanism, group-wise incremental mechanism, and credibility verification mechanism. Experiments on five SOD benchmarks show that our method achieves competitive performance against weakly-supervised/unsupervised methods both qualitatively and quantitatively. The code and results can be found from the link of <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://rmcong.github.io/proj_Hybrid-Label-SOD.html</uri> .

A Convolutional Self-Attention Network for CSI Reconstruction in MIMO System

A convolutional self-attention network-based channel state information reconstruction method is presented to address the issue of low reconstruction accuracy of channel state information in Multiple-Input Multiple-Output (MIMO) at a high compression rate. First, an encoder-decoder structure-based channel state information reconstruction model is built. The feature is extracted by the encoder’s convolutional network, and the information is compressed by adding an attention block. At the same time, the compressed information is nonuniform quantized to prevent the transmission process from using up too much bandwidth. A dequantization module and an attention block are added to the decoder to reduce the impact of noise on the matrix, converting the continuous value into a discrete value to increase reconstruction accuracy and using the long-time cosine annealing training approach. According to the simulation results, when compared to CsiNet, Lightweight CNN, CRNet, and CLNet, convergence speed is improved by 17.64%, indoor reconstruction precision is improved by an average of 37.4%, and outside reconstruction accuracy is improved by an average of 32.5% under all compressions.

TrEnD: A transformer-based encoder-decoder model with adaptive patch embedding for mass segmentation in mammograms.

Breast cancer is one of the most prevalent malignancies diagnosed in women. Mammogram inspection in the search and delineation of breast tumors is an essential prerequisite for a reliable diagnosis. However, analyzing mammograms by radiologists is time-consuming and prone to errors. Therefore, the development of computer-aided diagnostic (CAD) systems to automate the mass segmentation procedure is greatly expected. Accurate breast mass segmentation in mammograms remains challenging in CAD systems due to the low contrast, various shapes, and fuzzy boundaries of masses. In this paper, we propose a fully automatic and effective mass segmentation model based on deep learning for improving segmentation performance. We propose an effective transformer-based encoder-decoder model (TrEnD). Firstly, we introduce a lightweight method for adaptive patch embedding (APE) of the transformer, which utilizes superpixels to adaptively adjust the size and position of each patch. Secondly, we introduce a hierarchical transformer-encoder and attention-gated-decoder structure, which is beneficial for progressively suppressing interference feature activations in irrelevant background areas. Thirdly, a dual-branch design is employed to extract and fuse globally coarse and locally fine features in parallel, which could capture the global contextual information and ensure the relevance and integrity of local information. The model is evaluated on two public datasets CBIS-DDSM and INbreast. To further demonstrate the robustness of TrEnD, different cropping strategies are applied to these datasets, termed tight, loose, maximal, and mix-frame. Finally, ablation analysis is performed to assess the individual contribution of each module to the model performance. The proposed segmentation model provides a high Dice coefficient and Intersection over Union (IoU) of 92.20% and 85.81% on the mix-frame CBIS-DDSM, while 91.83% and 85.29% for the mix-frame INbreast, respectively. The segmentation performance outperforms the current state-of-the-art approaches. By adding the APE and attention-gated module, the Dice and IoU have improved by 6.54% and 10.07%. According to extensive qualitative and quantitative assessments, the proposed network is effective for automatic breast mass segmentation, and has adequate potential to offer technical assistance for subsequent clinical diagnoses.