Semantic Segmentation Task Research Articles

Mask-free Iterative Refinement Network for weakly-supervised Few-shot Semantic Segmentation

The dependence on densely labeled samples in Few-shot Semantic Segmentation (FSS) poses challenges in terms of sample annotation. This paper introduces a more challenging few-shot semantic segmentation task that only utilizes intra-class images as weak supervision. To address this task, we propose a Mask-free Iterative Refinement Network consisting of a mask generation module (MGM) and an iterative refinement module (IRM), respectively addressing the inherent two challenges of locating segmented objects and deriving class-specific features in the absence of support mask and semantic labels. MGM generates pseudo-masks for the support image without requiring any training, which provides a rough estimation of the object locations, serving as an initial guidance for segmentation. IRM is designed to capture the co-occurrence class-specific information between the support and query images for segmentation and progressively refine the predicted mask in a bootstrap manner with the guidance of the class-specific information. Experimental results on three FSS benchmarks, i.e., FSS-1000, PASCAL-5i, and COCO-20i demonstrate that our proposed method achieves comparable or decent performance compared to existing zero-shot and supervised segmentation methods.

Semantic Segmentation-Driven Integration of Point Clouds from Mobile Scanning Platforms in Urban Environments

Precise and complete 3D representations of architectural structures or industrial sites are essential for various applications, including structural monitoring or cadastre. However, acquiring these datasets can be time-consuming, particularly for large objects. Mobile scanning systems offer a solution for such cases. In the case of complex scenes, multiple scanning systems are required to obtain point clouds that can be merged into a comprehensive representation of the object. Merging individual point clouds obtained from different sensors or at different times can be difficult due to discrepancies caused by moving objects or changes in the scene over time, such as seasonal variations in vegetation. In this study, we present the integration of point clouds obtained from two mobile scanning platforms within a built-up area. We utilized a combination of a quadruped robot and an unmanned aerial vehicle (UAV). The PointNet++ network was employed to conduct a semantic segmentation task, enabling the detection of non-ground objects. The experimental tests used the Toronto 3D dataset and DALES for network training. Based on the performance, the model trained on DALES was chosen for further research. The proposed integration algorithm involved semantic segmentation of both point clouds, dividing them into square subregions, and performing subregion selection by checking the emptiness or when both subregions contained points. Parameters such as local density, centroids, coverage, and Euclidean distance were evaluated. Point cloud merging and augmentation enhanced with semantic segmentation and clustering resulted in the exclusion of points associated with these movable objects from the point clouds. The comparative analysis of the method and simple merging was performed based on file size, number of points, mean roughness, and noise estimation. The proposed method provided adequate results with the improvement of point cloud quality indicators.

Structural tensor and frequency guided semi-supervised segmentation for medical images.

The method of semi-supervised semantic segmentation entails training with a limited number of labeled samples alongside many unlabeled samples, aiming to reduce dependence on pixel-level annotations. Most semi-supervised semantic segmentation methods primarily focus on sample augmentation in spatial dimensions to reduce the shortage of labeled samples. These methods tend to ignore the structural information of objects. In addition, frequency-domain information also supplies another perspective to evaluate information from images, which includes different properties compared to the spatialdomain. In this study, we attempt to answer these two questions: (1) is it helpful to provide structural information of objects in semi-supervised semantic segmentation tasks for medical images? (2) is it more effective to evaluate the segmentation performance in the frequency domain compared to the spatial domain for semi-supervised medical image segmentation? Therefore, we seek to introduce structural and frequency information to improve the performance of semi-supervised semantic segmentation for medicalimages. We present a novel structural tensor loss (STL) to guide feature learning on the spatial domain for semi-supervised semantic segmentation. Specifically, STL utilizes the structural information encoded in the tensors to enforce the consistency of objects across spatial regions, thereby promoting more robust and accurate feature extraction. Additionally, we proposed a frequency-domain alignment loss (FAL) to enable the neural networks to learn frequency-domain information across different augmented samples. It leverages the inherent patterns present in frequency-domain representations to guide the network in capturing and aligning features across diverse augmentation variations, thereby enhancing the model's robustness for the inputting variations. We conduct our experiments on three benchmark datasets, which include MRI (ACDC) for cardiac, CT (Synapse) for abdomen organs, and ultrasound image (BUSI) for breast lesion segmentation. The experimental results demonstrate that our method outperforms state-of-the-art semi-supervised approaches regarding the Dice similaritycoefficient. We find the proposed approach could improve the final performance of the semi-supervised medical image segmentation task. It will help reduce the need for medical image labels. Our code will are available at https://github.com/apple1986/STLFAL.

Tissue segmentation for traumatic brain injury based on multimodal MRI image fusion-semantic segmentation

Accurate segmentation of traumatic brain injury (TBI) has great significance for physicians to diagnose and assess a patient’s condition. The utilization of multimodal information plays a critical role in TBI segmentation. However, most of the existing methods mainly focus on direct extraction and selection of deep semantic features, whereas in this paper, we use image fusion as an auxiliary task for feature learning based on multimodal feature extraction to achieve more sufficient fusion of multimodal features. Therefore, we design a multimodal image fusion-semantic segmentation based framework. The proposed approach mainly consists of a semantic encoder module, a semantic segmentation module and an image fusion module. The semantic encoder compresses the input image into a smaller feature space to extract semantic features. The semantic segmentation module utilizes both the detailed information extracted by the encoder and the semantic information of high-level features extracted from the semantic segmentation module to generate the segmentation results. The image fusion module fuses semantic feature information from different modalities as an auxiliary task to semantic segmentation. Furthermore, to enhance the model’s performance even further, an uncertainty-based approach is employed, which dynamically adjusts the loss weights for the image fusion task and the semantic segmentation task during the model training process. The proposed method is evaluated on a private dataset, and compared with other widely recognized methods. It demonstrates outstanding performance in both Dice score and Recall metrics.

Nighttime scene understanding with label transfer scene parser

Semantic segmentation plays a crucial role in traffic scene understanding, especially in nighttime conditions. This paper tackles the task of semantic segmentation in nighttime scenes. The largest challenge of this task is the lack of annotated nighttime images to train a deep learning-based scene parser. The existing annotated datasets are abundant in daytime conditions but scarce in nighttime due to the high cost. Thus, we propose a novel Label Transfer Scene Parser (LTSP) framework for nighttime scene semantic segmentation by leveraging daytime annotation transfer. Our framework performs segmentation in the dark without training on real nighttime annotated data. In particular, we propose translating daytime images to nighttime conditions to obtain more data with annotation in an efficient way. In addition, we utilize the pseudo-labels inferred from unlabeled nighttime scenes to further train the scene parser. The novelty of our work is the ability to perform nighttime segmentation via daytime annotated labels and nighttime synthetic versions of the same set of images. The extensive experiments demonstrate the improvement and efficiency of our scene parser over the state-of-the-art methods with a similar semi-supervised approach on the benchmark of Nighttime Driving Test dataset. Notably, our proposed method utilizes only one-tenth of the amount of labeled and unlabeled data in comparison with the previous methods. Code is available at https://github.com/danhntd/Label_Transfer_Scene_Parser.git.

Style Optimization Networks for real-time semantic segmentation of rainy and foggy weather

Semantic segmentation is an essential task in the field of computer vision. Existing semantic segmentation models can achieve good results under good weather and lighting conditions. However, when the external environment changes, the effectiveness of these models are seriously affected. Therefore, we focus on the task of semantic segmentation in rainy and foggy weather. Fog is a common phenomenon in rainy weather conditions and has a negative impact on image visibility. Besides, to make the algorithm satisfy the application requirements of mobile devices, the computational cost and the real-time requirement of the model have become one of the major points of our research. In this paper, we propose a novel Style Optimization Network (SONet) architecture, containing a Style Optimization Module (SOM) that can dynamically learn style information, and a Key information Extraction Module (KEM) that extracts important spatial and contextual information. This can improve the learning ability and robustness of the model for rainy and foggy conditions. Meanwhile, we achieve real-time performance by using lightweight modules and a backbone network with low computational complexity. To validate the effectiveness of our SONet, we synthesized CityScapes dataset for rainy and foggy weather and evaluated the accuracy and complexity of our model. Our model achieves a segmentation accuracy of 75.29% MIoU and 83.62% MPA on a NVIDIA TITAN Xp GPU. Several comparative experiments have shown that our SONet can achieve good performance in semantic segmentation tasks under rainy and foggy weather, and due to the lightweight design of the model we have a good advantage in both accuracy and model complexity.

Accurate and Efficient Event-Based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network.

Spiking neural networks (SNNs), known for their low-power, event-driven computation, and intrinsic temporal dynamics, are emerging as promising solutions for processing dynamic, asynchronous signals from event-based sensors. Despite their potential, SNNs face challenges in training and architectural design, resulting in limited performance in challenging event-based dense prediction tasks compared with artificial neural networks (ANNs). In this work, we develop an efficient spiking encoder-decoder network (SpikingEDN) for large-scale event-based semantic segmentation (EbSS) tasks. To enhance the learning efficiency from dynamic event streams, we harness the adaptive threshold which improves network accuracy, sparsity, and robustness in streaming inference. Moreover, we develop a dual-path spiking spatially adaptive modulation (SSAM) module, which is specifically tailored to enhance the representation of sparse events and multimodal inputs, thereby considerably improving network performance. Our SpikingEDN attains a mean intersection over union (MIoU) of 72.57% on the DDD17 dataset and 58.32% on the larger DSEC-Semantic dataset, showing competitive results to the state-of-the-art ANNs while requiring substantially fewer computational resources. Our results shed light on the untapped potential of SNNs in event-based vision applications. The source codes are publicly available at https://github.com/EMI-Group/spikingedn.

Graph Transformer for 3D point clouds classification and semantic segmentation

Recently, graph-based and Transformer-based deep learning have demonstrated excellent performances on various point cloud tasks. Most of the existing graph-based methods rely on static graph, which take a fixed input to establish graph relations. Moreover, many graph-based methods apply maximizing and averaging to aggregate neighboring features, so that only a single neighboring point affects the feature of centroid or different neighboring points own the same influence on the centroid’s feature, which ignoring the correlation and difference between points. Most Transformer-based approaches extract point cloud features based on global attention and lack the feature learning on local neighbors. To solve the above issues of graph-based and Transformer-based models, we propose a new feature extraction block named Graph Transformer and construct a 3D point cloud learning network called GTNet to learn features of point clouds on local and global patterns. Graph Transformer integrates the advantages of graph-based and Transformer-based methods, and consists of Local Transformer that use intra-domain cross-attention and Global Transformer that use global self-attention. Finally, we use GTNet for shape classification, part segmentation and semantic segmentation tasks in this paper. The experimental results show that our model achieves good learning and prediction ability on most tasks. The source code and pre-trained model of GTNet will be released on https://github.com/NWUzhouwei/GTNet.

RGBD-based method for segmenting apparent pores within bridge towers

Abstract As a crucial technology in computer vision, image semantic segmentation is extensively applied to tasks such as detecting apparent defects in concrete, identifying structural cracks, and interpreting facility scenes within infrastructure settings. Challenges such as uneven lighting inside bridge towers and the similarity in color and texture between concrete pore structures and their surrounding areas frequently result in lower segmentation accuracy. This article introduces a multimodal semantic segmentation model incorporating depth information to tackle these challenges. By integrating depth and RGB images as inputs, the model constructs an interactive space and utilizes a cross-attention mechanism along with global context for guided learning, thus enabling precise feature extraction and segmentation. The experimental results demonstrate that this segmentation network excels on a custom-made concrete pore dataset, with a precision rate of 90.88%, recall rate of 87.48%, intersection over union (IoU) of 80.42%, and F1 and Dice coefficient of 89.10% and 89.15%, respectively. These outcomes affirm the network’s effectiveness in segmenting apparent defects in concrete and offer technical support and solutions for image semantic segmentation tasks in settings like bridge towers.

Containment Control-Guided Boundary Information for Semantic Segmentation

Real-time semantic segmentation is a challenging task in computer vision, especially in complex scenes. In this study, a novel three-branch semantic segmentation model is designed, aiming to effectively use boundary information to improve the accuracy of semantic segmentation. The proposed model introduces the concept of containment control in a pioneering way, which treats image interior elements as well as image boundary elements as followers and leaders in containment control, respectively. Based on this, we utilize two learnable feature fusion matrices in the high-level semantic information stage of the model to quantify the fusion process of internal and boundary features. Further, we design a dedicated loss function to update the parameters of the feature fusion matrices based on the criterion of containment control, which enables fine-grained communication between target features. In addition, our model incorporates a Feature Enhancement Unit (FEU) to tackle the challenge of maximizing the utility of multi-scale features essential for semantic segmentation tasks through the meticulous reconstruction of these features. The proposed model proves effective on the publicly available Cityscapes and CamVid datasets, achieving a trade-off between effectiveness and speed.

Bilinear Distance Feature Network for Semantic Segmentation in PowerLine Corridor Point Clouds.

Semantic segmentation of target objects in power transmission line corridor point cloud scenes is a crucial step in powerline tree barrier detection. The massive quantity, disordered distribution, and non-uniformity of point clouds in power transmission line corridor scenes pose significant challenges for feature extraction. Previous studies have often overlooked the core utilization of spatial information, limiting the network's ability to understand complex geometric shapes. To overcome this limitation, this paper focuses on enhancing the deep expression of spatial geometric information in segmentation networks and proposes a method called BDF-Net to improve RandLA-Net. For each input 3D point cloud data, BDF-Net first encodes the relative coordinates and relative distance information into spatial geometric feature representations through the Spatial Information Encoding block to capture the local spatial structure of the point cloud data. Subsequently, the Bilinear Pooling block effectively combines the feature information of the point cloud with the spatial geometric representation by leveraging its bilinear interaction capability thus learning more discriminative local feature descriptors. The Global Feature Extraction block captures the global structure information in the point cloud data by using the ratio between the point position and the relative position, so as to enhance the semantic understanding ability of the network. In order to verify the performance of BDF-Net, this paper constructs a dataset, PPCD, for the point cloud scenario of transmission line corridors and conducts detailed experiments on it. The experimental results show that BDF-Net achieves significant performance improvements in various evaluation metrics, specifically achieving an OA of 97.16%, a mIoU of 77.48%, and a mAcc of 87.6%, which are 3.03%, 16.23%, and 18.44% higher than RandLA-Net, respectively. Moreover, comparisons with other state-of-the-art methods also verify the superiority of BDF-Net in point cloud semantic segmentation tasks.

DSC-Net: learning discriminative spatial contextual features for semantic segmentation of large-scale ancient architecture point clouds

Semantic segmentation of point cloud data of architectural cultural heritage is of significant importance for HBIM modeling, disease extraction and analysis, and heritage restoration research fields. In the semantic segmentation task of architectural point cloud data, especially for the protection and analysis of architectural cultural heritage, the previous deep learning methods have poor segmentation effects due to the complexity and unevenness of the data, the high geometric feature similarity between different components, and the large scale changes. To this end, this paper proposes a novel encoder-decoder architecture called DSC-Net. It consists of an encoder-decoder structure based on point random sampling and several fully connected layers for semantic segmentation. To overcome the loss of key features caused by random downsampling, DSC-Net has developed two new feature aggregation schemes: the enhanced dual attention pooling module and the global context feature module, to learn discriminative features for the challenging scenes mentioned above. The former fully considers the topology and semantic similarity of neighboring points, generating attention features that can distinguish categories with similar structures. The latter uses spatial location and neighboring volume ratio to provide an overall view of different types of architectural scenes, helping the network understand the spatial relationships and hierarchical structures between different architectural elements. The proposed modules can be easily embedded into various network architectures for point cloud semantic segmentation. We conducted experiments on multiple datasets, including the ancient architecture dataset, the ArCH architectural cultural heritage dataset, and the publicly available architectural segmentation dataset S3DIS. The results show that the mIoU reached 63.56%, 55.84%, and 71.03% respectively. The experimental results prove that our method has the best segmentation effect in dealing with challenging architectural cultural heritage data and also demonstrates its practicality in a wider range of architectural point cloud segmentation applications.

Affordance-informed Robotic Manipulation via Intelligent Action Library

Abstract In the realm of conventional affordance detection, the primary objective is to provide insights into the potential uses of objects. However, a significant limitation remains as these conventional methods merely treat affordance detection as a semantic segmentation task, disregarding the crucial aspect of interpreting affordances for actions that can be performed by manipulator. To address this critical gap, we present a novel pipeline incorporating the Intelligent Action Library (IAL) concept. This framework enables affordance interpretation for various manipulation tasks, allowing robots to be taught and guided on how to execute specific actions based on the detected affordances and human-robot interaction. Through real-world experiments, we have demonstrated the ingenuity and dependability of our pipeline, effectively bridging the gap between affordance detection and manipulation task planning and execution. The integration of IAL facilitates a seamless connection between understanding affordances and empowering robots to perform tasks with precision and efficiency. The demo link is available to the public: https://youtu.be/_oBAer2Vl8k

Building extraction from remote sensing imagery: advanced squeeze-and-excitation residual network based methodology

Extracting buildings from remote sensing imagery (RSI) is an essential task in a wide range of applications, such as urban and monitoring. Deep learning has emerged as a powerful tool for this purpose, and in this research, we propose an advanced building extraction method based on SE-ResNet18 and SE-ResNet34 architectures. These models were selected through a rigorous comparative analysis of various deep learning models, including variations of residual networks (ResNet), squeeze-and-excitation residual networks (SE-ResNet), and visual geometry group (VGG), for their high performance in all metrics and their computational efficiency. Our proposed methodology outperformed all other models under consideration by a significant margin, demonstrating its robustness and efficiency. It achieved superior results with less computational effort and time, a testament to its potential as a powerful tool for semantic segmentation tasks in remote sensing applications. An extensive comparative evaluation involving a wide range of state-of-the-art works further validated our method’s effectiveness. Our method achieved an unparalleled intersection over union (IoU) score of 88.51%, indicative of its exceptional accuracy in identifying and segmenting buildings within the Wuhan University (WHU) building dataset. The overall performance of our method, which offers an excellent balance between high performance and computational efficiency, makes it a compelling choice for researchers and practitioners in the field.

High resolution remote sensing recognition of elm sparse forest via deep-learning-based semantic segmentation

Elm (Ulmus pumila L.) sparse forest plays an vital role in maintaining local ecological stability and security in the Otingdag Sandy Land area. Prior studies on elm canopy extraction have predominantly relied on manual parameter configuration, resulting in unsatisfactory levels of generalization. To meet the needs of high-precision and rapid recognition of elm sparse forests in large areas, this study proposed a recognition method for elm sparse forest that orients to high spatial resolution remote sensing imageries, using deep-learning-based semantic segmentation techniques. It can automatically learn features that are conducive to segmenting the canopy of elm trees, and retains good generalization ability on the Gaofen-2 imageries obtained in different regions. First, we constructed a dataset specialized for elm canopy semantic segmentation task, and annotated over 130,000 elm canopies based on Gaofen-2 imageries. In addition, we trained 7 deep-learning semantic segmentation model candidates. Among them, MANet showed the best performance, with its F1-score reaching 81.44%. Lastly, we applied edge detection to the elm canopy coverage area, and automatically extract the elm canopy. The proposed method can provide technical support for the investigation and monitoring of elm sparse forests, while facilitates local desertification prevention efforts in the entire Otingdag Sandy Region.

GRA-Net: Group response attention for deep learning

Activation function, one of the most critical components in deep learning, enables the artificial neural networks to learn complex patterns through nonlinearity. Currently, element-wise activation functions such as ReLU are widely utilized because of their simplicity and efficiency. However, these methods do not consider the interactions among neurons, which may produce redundant features and affect performance. Inspired by neuroscience, we propose Group Response Attention (GRA). It establishes connections among neurons, which means that the state of each member is determined by the entire group. Accordingly, GRA makes the features more diverse and reduces redundancy. In addition, we propose a pure attention network without any conventional activation function, termed as GRA-Net. Experiments conducted on typical computer vision tasks show that GRA-Net achieves state-of-the-art results. For the ImageNet-1K classification task, GRA-Net-T achieves 82.9% Top-1 accuracy. With comparable parameters and FLOPs, its performance improves by 1.6% and 0.8% over Swin-T and ConvNeXt-T. On the COCO detection task, as the backbone of Mask R-CNN, GRA-Net-T outperforms Swin-T and ConvNeXt-T by 2.1% and 2.0% on box AP. Meanwhile, the mIoU of GRA-Net-T on the ADE20K semantic segmentation task is higher than Swin-T and ConvNeXt-T by 1.9% and 1.0%.

Ionograms Trace Extraction Method Based on Multiscale Transformer Network

The echo traces in the ionograms contain key information about the ionosphere. Therefore, the accurate extraction of these traces is crucial for the subsequent work. This paper transforms the original signal processing problem into a semantic segmentation task, combines it with the currently popular deep learning techniques, and proposes a multiscale Transformer network to achieve pixel-level trace extraction. To train the proposed model, we built a dataset by discretizing the original echo data, labeling, and other preprocessing work. A series of advanced semantic segmentation networks are utilized for comparative experiments. The analysis of the results indicates that the proposed network excels in performance, achieving the highest scores on key semantic segmentation evaluation metrics, including mIoU, Kappa, Dice, and AUC-ROC. In addition, this paper also designs a series of ablation experiments to observe the changes in network performance and to evaluate the rationality of the network design. The experimental results demonstrate the effectiveness of the network in the trace extraction task, which plays a positive role in the subsequent electron density reversal work.

Semantic segmentation algorithm for video from UAV based on adaptive keyframe scheduling via similarity measurement

Abstract Unmanned aerial vehicle (UAV) videos exhibit complex object motion features and significant differences between frame features. To solve the problems of feature information loss and drastic accuracy decline in applying the video semantic segmentation method based on the fixed-period update strategy of keyframes, a keyframe recognition method based on similarity measurement is proposed, forming a video semantic segmentation algorithm based on adaptive keyframe scheduling. one keyframe recognition method based on pixel similarity measurement is established by modeling the similarity relationship between low-level pixels in adjacent frames. Meanwhile, the other keyframe recognition method based on feature similarity measurement is established by constructing a shallow Siamese network to measure the similarity relationship between features of frames. Then a discriminative network is constructed based on the obtained similarity of inter frames, and the segmentation process is accelerated by reusing features of keyframes through the optical flow network. Thereby a video semantic segmentation method for UAV based on adaptive keyframe strategy is established. The effectiveness of the proposed video semantic segmentation algorithm is verified on UAVid dataset. The results demonstrated that the speed of the proposed algorithm reaches 53.2 FPS and 54.5 FPS on the premise that the mean intersection over union is higher than 40% (this value is compared with the segmentation accuracy in the balanced mode when the similarity threshold is 0.76 and 0.88, respectively), which is 18.5 FPS and 19.8 FPS higher than the PSPNet image semantic segmentation algorithm. In addition, analysis of experiment results shows that the pixel-similarity-based keyframe recognition is suitable for high-precision video semantic segmentation scenes that need to improve segmentation efficiency, and the feature-similarity-based keyframe recognition is more suitable for high real-time video semantic segmentation tasks that require a small decrease in overall algorithm accuracy. 
In a word, the proposed video semantic segmentation algorithm based on adaptive keyframe scheduling via similarity measurement can improve segmentation speed while ensuring segmentation accuracy and stability.

DMRC-Net: Dual mechanism residual connection network for point cloud analysis

The purpose of point cloud analysis is to extract effective high-dimensional feature representations from point coordinates. However, existing methods only rely solely on global or local features. In this paper, we introduce the Dual Mechanism Residual Connection Network (DMRC-Net), a new end-to-end network architecture for more comprehensive representations. First, local features are encoded using graph convolution and fine-grained attention mechanisms, whereas global features are encoded using a multi-layer perceptron. Second, adaptive residual learning is designed to integrate two encoding results effectively. Furthermore, the structural information in local space is strengthened by collecting low-level geometric correlations. With these designs, DMRC-Net is able to capture both global and local details comprehensively. Experimental results show that our method performs well in point cloud classification, part segmentation, and semantic segmentation tasks.

Enhancing left ventricular segmentation in echocardiography with a modified mixed attention mechanism in SegFormer architecture

Echocardiography is a key tool for the diagnosis of cardiac diseases, and accurate left ventricular (LV) segmentation in echocardiographic videos is crucial for the assessment of cardiac function. However, since semantic segmentation of video needs to take into account the temporal correlation between frames, this makes the task very challenging. This article introduces an innovative method that incorporates a modified mixed attention mechanism into the SegFormer architecture, enabling it to effectively grasp the temporal correlation present in video data. The proposed method processes each time series by encoding the image input into the encoder to obtain the current time feature map. This map, along with the historical time feature map, is then fed into a time-sensitive mixed attention mechanism type of convolution block attention module (TCBAM). Its output can serve as the historical time feature map for the subsequent sequence, and a combination of the current time feature map and historical time feature map for the current sequence. The processed feature map is then input into the Multilayer Perceptron (MLP) and subsequent networks to generate the final segmented image. Through extensive experiments conducted on two different datasets: Hamad Medical Corporation, Tampere University, and Qatar University (HMC-QU), Cardiac Acquisitions for Multi-structure Ultrasound Segmentation (CAMUS) and Sunnybrook Cardiac Data (SCD), achieving a Dice coefficient of 97.92 % on the SCD dataset and an F1 score of 0.9263 on the CAMUS dataset, outperforming all other models. This research provides a promising solution to the temporal modeling challenge in video semantic segmentation tasks using transformer-based models and points out a promising direction for future research in this field.