Decoder Module Research Articles

MID: A Novel Mountainous Remote Sensing Imagery Registration Dataset Assessed by a Coarse-to-Fine Unsupervised Cascading Network

The geometric registration of mountainous remote sensing images is always a challenging project, as terrain fluctuations increase the complexity. Deep learning, with its superior computing power and data-driven nature, promises to solve this problem. However, the lack of an appropriate dataset limits the development of deep learning technology for mountainous remote sensing image registration, which it still an unsolved problem in photogrammetry and remote sensing. To remedy this problem, this paper presents a manually constructed imagery dataset of mountainous regions, called the MID (mountainous imagery dataset). To create the MID, we use 38 images from the Gaofen-2 satellite developed by China and generated 4093 pairs of reference and sensed image patches, making this the first real mountainous dataset to our knowledge. Simultaneously, we propose a fully unsupervised, convolutional-network-based iterative registration scheme for the MID. First, the large and global deformation of the reference and sensed images is reduced using an affine registration module, generating the coarse alignment. Then, the local and varied distortions are learned and eliminated progressively using a hybrid dilated convolution (HDC)-based encoder–decoder module with multistep iterations, achieving fine registration results. The HDC aims to increase the receptive field without blocking the artifacts, allowing for the continuous characteristics of the mountainous images of a local region to be represented. We provide a performance analysis of some typical registration algorithms and the developed approach for the MID. The proposed scheme gives the highest registration precision, achieving the subpixel alignment of mountainous remote sensing images. Additionally, the experimental results demonstrate the usability of the MID, which can lay a foundation for the development of deep learning technology in large mountainous remote sensing image registration tasks.

TransText: Improving scene text detection via transformer

Arbitrary shape scene text detection is still a challenging task in the computer vision community. Compared with anchor-based methods, segmentation-based methods have advantages in some respects, which detect text instances by generating accurate text components. In this paper, we propose a novel text components extraction network for arbitrary shape scene text detection. It can detect different text components through two parallel branches. In the first branch, the feature redistribution module (FRM) is proposed to extract text-related features and filter non-text information. Then, these text-related features are aggregated to generate text boundary maps. These text-related features are also fed to the second branch to progressively generate the text kernel maps. The second branch consists of a modified Transformer decoder with a multi-level progressive supervision strategy. It aims to capture spatial details and establish the dependencies between different text regions. In this way, our method can generate accurate text components without additional geometric calculations. Thanks to the powerful Transformer decoder and efficient differentiable binarization module, our method not only achieves advanced detection accuracy but also has a competitive inference speed. Specifically, with the ResNet-18 backbone, our method can run at 43.5 FPS and achieve an F-measure of 83.3% on the Total-Text dataset, 1.43 times faster than the latest state-of-the-art method. With the ResNet-50 backbone, our method achieves an F-measure of 85.2% and outperforms the latest state-of-the-art method by 0.2% on the Total-Text dataset. Code is available at: https://github.com/As-David/TransText.

AlgaeMask: An Instance Segmentation Network for Floating Algae Detection

Video surveillance on the offshore booster station and around the coast is a effective way to monitor floating macroalgae. Previous studies on floating algae detection are mainly based on traditional image segmentation methods. However, these algorithms cannot effectively solve the problem of extracting Ulva prolifra and Sargassum at different sizes and views. Recently, instance segmentation methods have achieved great success in computer vision applications. In this paper, based on the CenterMask network, a novel instance segmentation architecture named AlgaeMask is proposed for floating algae detection from the surveillance videos. To address the feature extraction ability of the network in the inter-dependencies for position and channel, we introduce a new OSA-V3 module with the dual-attention block, which consists of a position attention mechanism and channel attention mechanism. Meanwhile, scale-equalizing pyramid convolution is introduced to solve the problem of scale difference. Finally, we introduce the feature decoder module based on FCOS head and segmentation head to obtain the segmentation area of floating algae in each bounding box. The extensive experiment results show that the average precision of our AlgaeMask in the tasks of mask segmentation and box detection can reach 44.22% and 48.13%, respectively, which has 15.09% and 8.24% improvement over CenterMask. In addition, the AlgaeMask can meet the real-time requirements of floating algae detection.

PCUNet: A Context-Aware Deep Network for Coarse-to-Fine Point Cloud Completion

Point cloud completion aims at predicting a complete 3D shape from an incomplete input. It has important applications in the fields of intelligent manufacturing, augmented reality, virtual reality, self-driving cars, and intelligent robotics. Although deep learning-based point cloud completion technology has developed rapidly in recent years, there are still unsolved problems. Previous approaches predict each point independently and ignore contextual information. And, they usually predict a complete 3D shape based on a global feature vector extracted from an incomplete input, which leads to missing of some fine-grained details. In this paper, motivated by the transposed convolution and the “UNet” structure in neural networks for image processing, we propose a context-aware deep network termed as PCUNet for coarse-to-fine point cloud completion. It adopts an encoder-decoder structure, in which the encoder follows the design of the relation-shape convolutional neural network (RS-CNN), and the decoder consists of fully-connected layers and two stacked decoder modules for predicting complete point clouds. The contributions are twofold. First, we design the decoder module as a coordinate-guided context-aware upsampling module, in which contextual information can be taken into full account by neighbor aggregation. Second, to preserve fine-grained details in the input, we propose attention-enhanced skip connections for effective information propagation from the encoder to the decoder. Experiments are conducted on the widely used PCN and KITTI datasets. The results show that our proposed approach achieves competitive performance compared to the existing state-of-the-art approaches in terms of the Chamfer distance and the computational complexity metrics.

Parameter Identification in Power Transmission Systems Based on Graph Convolution Network

Parameter Identification plays an important role in electric power transmission systems. Existing approaches for parameter identification tasks typically have two limitations: (1) They generally ignored development trend of historical data, and did not mine characteristics of corresponding power grid branches. (2) They did not consider the constraints of power grid topology, and treated different branches independently. Therefore, they could not characterize correlations between the center node and its neighborhoods. To overcome these limitations, this work proposes a multi-task graph convolutional neural network (MT-GCN) which utilizes the graph convolutional network (GCN) and the fully convolutional network (FCN) as building blocks for parameter identification. Specially, GCN can extract the structure information to enhance local feature extraction. FCN is a decoding module following GCN module, and it is used to identify the parameters of each branch according to its characteristics. Compared with previous methods, the proposed method is significantly improved in accuracy. Besides, this method is robust to measurement noise and errors, and can cope with multiple conditions in real power transmission systems.

Semantic segmentation of bone structures in chest X-rays including unhealthy radiographs: A robust and accurate approach

The chest X-ray is a widely used medical imaging technique for the diagnosis of several lung diseases. Some nodules or other pathologies present in the lungs are difficult to visualize on chest X-rays because they are obscured byoverlying boneshadows. Segmentation of bone structures and suppressing them assist medical professionals in reliable diagnosis and organ morphometry. But segmentation of bone structures is challenging due to fuzzy boundaries of organs and inconsistent shape and size of organs due to health issues, age, and gender. The existing bone segmentation methods do not report their performance on abnormal chest X-rays, where it is even more critical to segment the bones. This work presents a robust encoder–decoder network for semantic segmentation of bone structures on normal as well as abnormal chest X-rays. The novelty here lies in combining techniques from two existing networks (Deeplabv3+ and U-net) to achieve robust and superior performance. The fully connected layers of the pre-trained ResNet50 network have been replaced by an Atrous spatial pyramid pooling block for improving the quality of the embedding in the encoder module. The decoder module includes four times upsampling blocks to connect both low-level and high-level features information enabling us to retain both the edges and detail information of the objects. At each level, the up-sampled decoder features are concatenated with the encoder features at a similar level and further fine-tuned to refine the segmentation output. We construct a diverse chest X-ray dataset with ground truth binary masks of anterior ribs, posterior ribs, and clavicle bone for experimentation. The dataset includes 100 samples of chest X-rays belonging to healthy and confirmed patients of lung diseases to maintain the diversity and test the robustness of our method. We test our method using multiple standard metrics and experimental results indicate an excellent performance on both normal and abnormal chest X-rays.

Progressive multi-branch embedding fusion network for underwater image enhancement

The underwater image enhancement techniques are essential for ocean research and engineering applications. In this paper, we propose a progressive multi-branch embedding fusion network (PMEFN) to improve image quality. Specifically, a multi-branch embedding fusion module (MEFM) is designed. The distorted images and its sharpened versions are used as the input, which are fused to learn the contextualized features based on a two-branch hybrid encoder–decoder module (HEDM2) combined with the triple attention module to focus on the noise region. Afterwards, we use the multi-stage refining framework to decompose the image enhancement or marine snow removal tasks into multiple stages and progressively learn the nonlinear functions from the distorted inputs. Additionally, the outputs generated at each stage are further refined and enhanced based on a three-branch hybrid encoder–decoder module (HEDM3). We perform experiments using real underwater datasets, including EUVP, UFO-120, UIEB, and synthetic dataset MSRB. The experimental results show that the proposed method has a superior performance as compared to other methods in terms of quantitative performance and visual quality. In addition, the effectiveness of each component is further validated by performing ablation experiments.

Transformer-Based Global Zenith Tropospheric Delay Forecasting Model

Zenith tropospheric delay (ZTD) plays an important role in high-precision global navigation satellite system (GNSS) positioning and meteorology. At present, commonly used ZTD forecasting models comprise empirical, meteorological parameter, and neural network models. The empirical model can only fit approximate periodic variations, and its accuracy is relatively low. The accuracy of the meteorological parameter model depends heavily on the accuracy of the meteorological parameters. The recurrent neural network (RNN) is suitable for short-term series data prediction, but for long-term series, the ZTD prediction accuracy is clearly reduced. Long short-term memory (LSTM) has superior forecasting accuracy for long-term ZTD series; however, the LSTM model is complex, cannot be parallelized, and is time-consuming. In this study, we propose a novel ZTD time-series forecasting utilizing transformer-based machine-learning methods that are popular in natural language processing (NLP) and forecasting global ZTD, the training parameters provided by the global geodetic observing system (GGOS). The proposed transformer model leverages self-attention mechanisms by encoder and decoder modules to learn complex patterns and dynamics from long ZTD time series. The numeric results showed that the root mean square error (RMSE) of the forecasting ZTD results were 1.8 cm and mean bias, STD, MAE, and R 0.0, 1.7, 1.3, and 0.95, respectively, which is superior to that of the LSTM, RNN, convolutional neural network (CNN), and GPT3 series models. We investigated the global distribution of these accuracy indicators, and the results demonstrated that the accuracy in continents was superior to maritime space transformer ZTD forecasting model accuracy at high latitudes superior to that at low latitude. In addition to the overall accuracy improvement, the proposed transformer ZTD forecast model also mitigates the accuracy variations in space and time, thereby guaranteeing high accuracy globally. This study provides a novel method to estimate the ZTD, which could potentially contribute to precise GNSS positioning and meteorology.

TransMF: Transformer-Based Multi-Scale Fusion Model for Crack Detection

Cracks are widespread in infrastructure that are closely related to human activity. It is very popular to use artificial intelligence to detect cracks intelligently, which is known as crack detection. The noise in the background of crack images, discontinuity of cracks and other problems make the crack detection task a huge challenge. Although many approaches have been proposed, there are still two challenges: (1) cracks are long and complex in shape, making it difficult to capture long-range continuity; (2) most of the images in the crack dataset have noise, and it is difficult to detect only the cracks and ignore the noise. In this paper, we propose a novel method called Transformer-based Multi-scale Fusion Model (TransMF) for crack detection, including an Encoder Module (EM), Decoder Module (DM) and Fusion Module (FM). The Encoder Module uses a hybrid of convolution blocks and Swin Transformer block to model the long-range dependencies of different parts in a crack image from a local and global perspective. The Decoder Module is designed with symmetrical structure to the Encoder Module. In the Fusion Module, the output in each layer with unique scales of Encoder Module and Decoder Module are fused in the form of convolution, which can release the effect of background noise and strengthen the correlations between relevant context in order to enhance the crack detection. Finally, the output of each layer of the Fusion Module is concatenated to achieve the purpose of crack detection. Extensive experiments on three benchmark datasets (CrackLS315, CRKWH100 and DeepCrack) demonstrate that the proposed TransMF in this paper exceeds the best performance of present baselines.

Relation constraint self-attention for image captioning

Self-attention based Transformer has been successfully introduced in the encoder-decoder framework of image captioning, which is superior in modeling the inner relations of inputs, i.e., image regions or semantic words. However, relations in self-attention are usually too dense to be fully optimized, which may result in noisy relations and attentions. Meanwhile, the prior relations, e.g., visual relation and semantic relation between objects, which are essential for understanding and describing an image, are ignored by current self-attention. Thus, the relation learning of self-attention in image captioning is biased, which leads to a dilution of the concentration of attentions. In this paper, we propose a Relation Constraint Self-Attention (RCSA) model to enhance the relation learning of self-attention in image captioning by constraining self-attention with prior relations. RCSA exploits the prior visual and semantic relation information from scene graph as constraint factors. And then it builds constraints for self-attention through two sub-modules: an RCSA-E encoder module and an RCSA-D decoder module. RCSA-E introduces the visual relation information to self-attention in encoder, which helps generate a sparse attention map by omitting the attention weights of irrelevant regions to highlight relevant visual features. RCSA-D extends the keys and values of self-attention in decoder with the semantic relation information to constrain the learning of semantic relation, and improve the accuracy of generated semantic words. Intuitively, RCSA-E endows model with an ability to figure out which region to omit and which region to focus by visual relation information; RCSA-D then strengthens the relation learning of the focused regions and improves the sentence generation with semantic relation information. Experiments on the MSCOCO dataset demonstrate the effectiveness of our proposed RCSA.

HRNet Encoder and Dual-Branch Decoder Framework-Based Scene Text Recognition Model

Scene text recognition (STR) is designed to automatically recognize the text content in natural scenes. Different from regular document text, text in natural scenes has the characteristics of irregular shapes, complex background, and distorted and blurred contents, which makes STR challenging. To solve the problems of STR for distorted, blurred, and low-resolution texts in natural scenes, this paper proposes a HRNet encoder and dual-branch decoder framework-based STR model. The model mainly consists of an encoder module and a dual-branch decoder module composed of a super-resolution branch and a recognition branch in parallel. In the encoder module, the HRNet is adopted to realize the cross-parallel aggregation representation with multiple resolutions during feature extraction and then outputs four kinds of feature maps with different resolutions. Moreover, the supervised attention module is used to strengthen the learning of the important feature information. In the decoder module, the dual-branch structure is adopted, in which the super-resolution branch takes the feature maps with the highest resolution obtained in the encoder module as input and restores images by upsampling through transposed convolution. The four kinds of feature maps with different resolutions are fused through independent transposed convolution layers for multiscale fusion in the recognition branch and then inputted into the attention-based decoder for text recognition. To improve the accuracy of text recognition, the feature extraction effect of the encoder module is together supervised by the super-resolution branch loss and the recognition branch loss. In addition, the super-resolution branch is only used for training and is abandoned during testing to reduce the complexity of the model. The proposed model is trained on Synth90K and SynthText datasets and tested on seven natural scene datasets. Compared with classical models such as ASTER, TextSR, and SCGAN, the recognition accuracy of the proposed model is improved and better recognition results can be achieved on irregular and blurred datasets such as IC15, SVTP, and CUTE80.

Pixel-Reasoning-Based Robotics Fine Grasping for Novel Objects with Deep EDINet Structure

Robotics grasp detection has mostly used the extraction of candidate grasping rectangles; those discrete sampling methods are time-consuming and may ignore the potential best grasp synthesis. This paper proposes a new pixel-level grasping detection method on RGB-D images. Firstly, a fine grasping representation is introduced to generate the gripper configurations of parallel-jaw, which can effectively resolve the gripper approaching conflicts and improve the applicability to unknown objects in cluttered scenarios. Besides, the adaptive grasping width is used to adaptively represent the grasping attribute, which is fine for objects. Then, the encoder–decoder–inception convolution neural network (EDINet) is proposed to predict the fine grasping configuration. In our findings, EDINet uses encoder, decoder, and inception modules to improve the speed and robustness of pixel-level grasping detection. The proposed EDINet structure was evaluated on the Cornell and Jacquard dataset; our method achieves 98.9% and 96.1% test accuracy, respectively. Finally, we carried out the grasping experiment on the unknown objects, and the results show that the average success rate of our network model is 97.2% in a single object scene and 93.7% in a cluttered scene, which out-performs the state-of-the-art algorithms. In addition, EDINet completes a grasp detection pipeline within only 25 ms.

Robot Path Planning via Neural-Network-Driven Prediction

This article presents a novel heuristic method named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">neural-network-driven prediction (NEED)</i> for robot path planning problems. Different from classical heuristic methods, NEED is not designed for some specific environments, which can be applied to various environments. NEED has an encoder–decoder structure, which consists of three modules: convolutional neural network backbone, spatial pooling module, and decoder module. By learning from a number of successful path planning cases, NEED can learn to analyze the environment structure and predict the promising search region for the new path planning problem. This predicted region serves as a heuristic to guide the search direction of path planning algorithms. A series of simulation experiments are conducted to demonstrate that NEED significantly improves the algorithm performance on solving new path planning problems. Meanwhile, NEED can also be applied to highly dynamic environments since it can output the prediction results at a speed of over 100 Hz on a laptop.

Solar Filament Detection Based on Improved DeepLab V3+

A novel solar filament detection method based on an improved DeepLab V3+ is proposed to address the low detection accuracy of small solar filaments in Hα full-disk solar images. First, the Xception structure of the backbone network is fine-tuned, and the low-level feature information of the filaments is added to the decoder module of the network to improve the utilization of the solar filament features. Second, the receptive field of dilated convolution is expanded, and the information utilization rate is increased via cascaded dilated convolution to improve the detection accuracy of the small solar filaments. In the decoder module, two depthwise separable convolutions are used instead of ordinary convolutions to reduce incomplete detections. Finally, a dense conditional random field is added to optimize the edge of the detection results. Experiments on a public data set comprising full-disk Hα images show that compared with the original Deeplab V3+ algorithm, the proposed method improves the mean pixel accuracy, mean intersection over union, and F1-score by 1.86%, 1.95%, and 2.18%, respectively, which also demonstrates its superiority over other existing solar filament detection algorithms.

Temporal information-guided dynamic dual-tracer PET signal separation network.

The difficulty of dynamic dual-tracer positron emission tomography (PET) technology is to separate the complete single-tracer information from mixed dual-tracer. Traditional methods cannot separate single-injection single-scan dynamic dual-tracer PET images. In this paper, we propose a deep learning framework based on gated recurrent unit (GRU) network and evaluate its performance with simulation experiments and realistic monkeydata. The proposed single-scan dynamic dual-tracer PET image separation network consists of three parts, including encoder, separation, and decoder module. Encoder part is to map the mixed time activity curves (TACs) from the low-dimensional space to the high-dimensional space to get mixed weight vector matrix. Separation part is to capture the temporal information of mixed weight vector matrix using bi-directional GRU (bi-GRU) layer to obtain the single-tracer masks, and the decoding part remaps the high-dimensional single-tracer weight vector matrix to the low-dimensional space to obtain two separated single tracers. In the simulation experiments under different tracers, phantoms, noise levels, arterial input function (AIF), and k-parameter with Gaussian random, compared to the stacked auto encoder network and traditional background subtraction method, GRU-based network has better performance with low bias and mean squared error. In the realistic study, the image results of GRU network have higher mean structural similarity and peak signal to noiseratio. This study demonstrates the feasibility of temporal information-guided neural network in single-injection single-scan dynamic dual-tracer PET images separation. The GRU-based network uses TAC temporal information without AIFs to make the separation results more robust and accurate, which significantly outperforms state-of-the-art method qualitatively andquantitatively.

Multiple Traffic Target Tracking with Spatial-Temporal Affinity Network.

Traffic target tracking is a core task in intelligent transportation system because it is useful for scene understanding and vehicle autonomous driving. Most state-of-the-art (SOTA) multiple object tracking (MOT) methods adopt a two-step procedure: object detection followed by data association. The object detection has made great progress with the development of deep learning. However, the data association still heavily depends on hand crafted constraints, such as appearance, shape, and motion, which need to be elaborately trained for a special object. In this study, a spatial-temporal encoder-decoder affinity network is proposed for multiple traffic targets tracking, aiming to utilize the power of deep learning to learn a robust spatial-temporal affinity feature of the detections and tracklets for data association. The proposed spatial-temporal affinity network contains a two-stage transformer encoder module to encode the features of the detections and the tracked targets at the image level and the tracklet level, aiming to capture the spatial correlation and temporal history information. Then, a spatial transformer decoder module is designed to compute the association affinity, where the results from the two-stage transformer encoder module are fed back to fully capture and encode the spatial and temporal information from the detections and the tracklets of the tracked targets. Thus, efficient affinity computation can be applied to perform data association in online tracking. To validate the effectiveness of the proposed method, three popular multiple traffic target tracking datasets, KITTI, UA-DETRAC, and VisDrone, are used for evaluation. On the KITTI dataset, the proposed method is compared with 15 SOTA methods and achieves 86.9% multiple object tracking accuracy (MOTA) and 85.71% multiple object tracking precision (MOTP). On the UA-DETRAC dataset, 12 SOTA methods are used to compare with the proposed method, and the proposed method achieves 20.82% MOTA and 35.65% MOTP, respectively. On the VisDrone dataset, the proposed method is compared with 10 SOTA trackers and achieves 40.5% MOTA and 74.1% MOTP, respectively. All those experimental results show that the proposed method is competitive to the state-of-the-art methods by obtaining superior tracking performance.

Hard Negative Samples Contrastive Learning for Remaining Useful-Life Prediction of Bearings

In recent years, deep learning has become prevalent in Remaining Useful-Life (RUL) prediction of bearings. The current deep-learning-based RUL methods tend to extract high dimensional features from the original vibration data to construct the Health Indicators (HIs), and then use the HIs to predict the remaining life of the bearings. These approaches ignore the sequential relationship of the original vibration data and seriously affect the prediction accuracy. In order to tackle this problem, we propose a hard negative sample contrastive learning prediction model (HNCPM) with encoder module, GRU regression module and decoder module, used for feature embedding, regression RUL prediction and vibration data reconstruction, respectively. We introduce self-supervised contrast learning by constructing positive and negative samples of vibration data rather than constructing any health indicators. Furthermore, to avoid the subtle variability of vibration data in the health stage to aggravate the degradation features learning of the model, we propose the hard negative samples by cosine similarity, which are most similar to the positive sample. Meanwhile, a novel infoNCE and MSE-based loss function is derived and applied to the HNCPM to simultaneously optimize a lower bound on mutual information of the positive and negative sample over life cycle, as well as the discrepancy between true and predicted values of the vibration data, such that the model can learn the fine-grained degradation representations by predicting the future without any HIs as labels. The HNCPM is validated on the IEEE PHM Challenge 2012 dataset. The results demonstrate that the prediction performance of our model is superior to the state-of-the-art methods.

Trailer hopper automatic detection method for silage harvesting based improved U-Net

At present, the automatic silage filling of self-propelled silage harvester is mainly realized by computer vision; the accuracy of trailer hopper detection is one of the main factors affecting the control reliability of the throwing arm. However, due to the limitations of the complicated harvest environment, changeable illumination, and various trailer types, the traditional trailer hopper detection methods cannot meet the operational requirements of the silage harvester with high efficiency and low loss. In this paper, we propose a new semantic segmentation network called RSHC U-Net, to achieve a better segmentation effect of trailer hopper in silage harvest image. The network is developed based on U-Net, which effectively integrates residual dissolution (RC) module, squeeze-and-excitation attention mechanism (SE) module, Hybrid exhausted dissolution (HDC) module, and cross-level gathering decoder (CLGD) module. The RC module solves the problem of the gradient decreasing with the increase of network depth; the SE module highlights the hopper edge features; the HDC module compensates for the detail information loss in the encoder; the CLGD module narrows the semantic gap between encoder and decoder. The experiment results show that the mean MIoU, Acc, Pre, and ST of the RSHC U-Net network are 90.38%, 95.17%, 90.22%, and 300.4 ms, respectively. The interior-point proportion threshold for the trailer hopper fitting algorithm is set to 95%. The elapsed time, fitting accuracy, and mean absolute error of the algorithm are 91.43 ms, 94.75%, and 9.71, respectively. Finally, the corner point spatial coordinates of the trailer hopper are obtained by ZED API to realize the spatial positioning of the hopper, which provides a basis for the subsequent silage filling path planning and the motion control of the throwing arm. This project has a certain technical reference value for fast, accurate, and automatic filling of silage.

MEA-Net: multilayer edge attention network for medical image segmentation

Medical image segmentation is a fundamental step in medical analysis and diagnosis. In recent years, deep learning networks have been used for precise segmentation. Numerous improved encoder–decoder structures have been proposed for various segmentation tasks. However, high-level features have gained more research attention than the abundant low-level features in the early stages of segmentation. Consequently, the learning of edge feature maps has been limited, which can lead to ambiguous boundaries of the predicted results. Inspired by the encoder–decoder network and attention mechanism, this study investigates a novel multilayer edge attention network (MEA-Net) to fully utilize the edge information in the encoding stages. MEA-Net comprises three major components: a feature encoder module, a feature decoder module, and an edge module. An edge feature extraction module in the edge module is designed to produce edge feature maps by a sequence of convolution operations so as to integrate the inconsistent edge information from different encoding stages. A multilayer attention guidance module is designed to use each attention feature map to filter edge information and select important and useful features. Through experiments, MEA-Net is evaluated on four medical image datasets, including tongue images, retinal vessel images, lung images, and clinical images. The evaluation values of the Accuracy of four medical image datasets are 0.9957, 0.9736, 0.9942, and 0.9993, respectively. The values of the Dice coefficient are 0.9902, 0.8377, 0.9885, and 0.9704, respectively. Experimental results demonstrate that the network being studied outperforms current state-of-the-art methods in terms of the five commonly used evaluation metrics. The proposed MEA-Net can be used for the early diagnosis of relevant diseases. In addition, clinicians can obtain more accurate clinical information from segmented medical images.

A Context Feature Enhancement Network for Building Extraction from High-Resolution Remote Sensing Imagery

The complexity and diversity of buildings make it challenging to extract low-level and high-level features with strong feature representation by using deep neural networks in building extraction tasks. Meanwhile, deep neural network-based methods have many network parameters, which take up a lot of memory and time in training and testing. We propose a novel fully convolutional neural network called the Context Feature Enhancement Network (CFENet) to address these issues. CFENet comprises three modules: the spatial fusion module, the focus enhancement module, and the feature decoder module. First, the spatial fusion module aggregates the spatial information of low-level features to obtain buildings’ outline and edge information. Secondly, the focus enhancement module fully aggregates the semantic information of high-level features to filter the information of building-related attribute categories. Finally, the feature decoder module decodes the output of the above two modules to segment the buildings more accurately. In a series of experiments on the WHU Building Dataset and the Massachusetts Building Dataset, our CFENet balances efficiency and accuracy compared to the other four methods we compared, and achieves optimality on all five evaluation metrics: PA, PC, F1, IoU, and FWIoU. This indicates that CFENet can effectively enhance and fuse buildings’ low-level and high-level features, improving building extraction accuracy.