Layered Decoding Research Articles

De-Aliasing and Accelerated Sparse Magnetic Resonance Image Reconstruction Using Fully Dense CNN with Attention Gates.

When sparsely sampled data are used to accelerate magnetic resonance imaging (MRI), conventional reconstruction approaches produce significant artifacts that obscure the content of the image. To remove aliasing artifacts, we propose an advanced convolutional neural network (CNN) called fully dense attention CNN (FDA-CNN). We updated the Unet model with the fully dense connectivity and attention mechanism for MRI reconstruction. The main benefit of FDA-CNN is that an attention gate in each decoder layer increases the learning process by focusing on the relevant image features and provides a better generalization of the network by reducing irrelevant activations. Moreover, densely interconnected convolutional layers reuse the feature maps and prevent the vanishing gradient problem. Additionally, we also implement a new, proficient under-sampling pattern in the phase direction that takes low and high frequencies from the k-space both randomly and non-randomly. The performance of FDA-CNN was evaluated quantitatively and qualitatively with three different sub-sampling masks and datasets. Compared with five current deep learning-based and two compressed sensing MRI reconstruction techniques, the proposed method performed better as it reconstructed smoother and brighter images. Furthermore, FDA-CNN improved the mean PSNR by 2 dB, SSIM by 0.35, and VIFP by 0.37 compared with Unet for the acceleration factor of 5.

U2CrackNet: a deeper architecture with two-level nested U-structure for pavement crack detection

Crack detection is important for evaluating pavement conditions. In this paper, we propose an automatic pavement crack detection method called as U2CrackNet. The architecture of U2CrackNet is a two-level nested U-structure which is an encoding and decoding architecture. Firstly, the crack features are extracted by the encoding layer so that the preliminary effective feature layer is obtained. Subsequently, the encoder and decoder are connected by an atrous spatial pyramid pooling (ASPP) model. The atrous convolution with different expansion rates can be used to capture the multi-scale crack information. Besides, in order to make the network pay more attention to the features of cracks, the crack feature map channels are given different weights through the effective channel attention mechanism after upsampling in the decoding layer. Finally, the crack saliency probability maps generated by each layer of the network are fused into the final crack saliency map through cascade operation. The proposed method has been evaluated on an expanded pavement crack dataset containing 8700 images. The experimental results demonstrate that the proposed U2CrackNet can obtain more clear and continuous cracks. Specifically, the precision, accuracy, F1-score, and Mean Intersection over Union (MIoU) are 89.51%, 98.95%, 81.45%, and 69.19%, respectively.

MF2ResU-Net: a multi-feature fusion deep learning architecture for retinal blood vessel segmentation

ObjectiveFor computer-aided Chinese medical diagnosis and aiming at the problem of insufficient segmentation, a novel multi-level method based on the multi-scale fusion residual neural network (MF2ResU-Net) model is proposed. MethodsTo obtain refined features of retinal blood vessels, three cascade connected U-Net networks are employed. To deal with the problem of difference between the parts of encoder and decoder, in MF2ResU-Net, shortcut connections are used to combine the encoder and decoder layers in the blocks. To refine the feature of segmentation, atrous spatial pyramid pooling (ASPP) is embedded to achieve multi-scale features for the final segmentation networks. ResultsThe MF2ResU-Net was superior to the existing methods on the criteria of sensitivity (Sen), specificity (Spe), accuracy (ACC), and area under curve (AUC), the values of which are 0.8013 and 0.8102, 0.9842 and 0.9809, 0.9700 and 0.9776, and 0.9797 and 0.9837, respectively for DRIVE and CHASE DB1. The results of experiments demonstrated the effectiveness and robustness of the model in the segmentation of complex curvature and small blood vessels. ConclusionBased on residual connections and multi-feature fusion, the proposed method can obtain accurate segmentation of retinal blood vessels by refining the segmentation features, which can provide another diagnosis method for computer-aided Chinese medical diagnosis.

Character-level arabic text generation from sign language video using encoder–decoder model

Video to text conversion is a vital activity in the field of computer vision. In recent years, deep learning algorithms have dominated automatic text generation in English, but there are a few research works available for other languages. In this paper, we propose a novel encoding–decoding system that generates character-level Arabic sentences from isolated RGB videos of Moroccan sign language. The video sequence was encoded by a spatiotemporal feature extraction using pose estimation models, while the label text of the video is transmitted to a sequence of representative vectors. Both the features and the label vector are joined and treated by a decoder layer to derive a final prediction. We trained the proposed system on an isolated Moroccan Sign Language dataset (MoSLD), composed of RGB videos from 125 MoSL signs. The experimental results reveal that the proposed model attains the best performance under several evaluation metrics.

A Novel Named Entity Recognition Algorithm for Hot Strip Rolling Based on BERT-Imseq2seq-CRF Model

Named entity recognition is not only the first step of text information extraction, but also the key process of constructing domain knowledge graphs. In view of the large amount of text data, complex process flow and urgent application needs in the hot strip rolling process, a novel named entity recognition algorithm based on BERT-Imseq2seq-CRF model is proposed in this paper. Firstly, the algorithm uses the BERT preprocessing language model to mine the dependencies in the domain text and obtain the corresponding representation vector. Then, the representation vector is sent to the encoder layer, and the output vector is input to the decoder at the same time, on the premise that the original model only considers the semantic vector. The Teacher-Forcing mechanism is integrated into the decoder layer to randomly modify the labeling results, and error accumulation is avoided to guarantee the sequence recognition effect. Finally, the validity of the labeling results is checked according to the conditional random field constraints, and the overall labeling quality of the algorithm is improved. The experimental results show that this model can efficiently and accurately predict the physical label of hot strip rolling, and the model performance index is better than other models, with the F1-Score reaching 91.47%. This model further provides technical support for information extraction and domain knowledge graph construction of hot strip rolling.

Intelligent defect detection for Power Patrol Inspection

In this paper, the insulator string images captured by Using unmanned aerial vehicles (UAVs) in the power inspection are taken as the research object. The image processing and deep learning methods are used to label the images, and the self-explosion fault of the identified and segmented insulator strings is identified and located.In the process of semantic segmentation of insulator string images, the given image data set is enhanced, and images are exchanged with image processing algorithms. Based on SegNet, each pixel in the picture is classified, and then the self-encoding is used for reference, and the image is first encoded and then decoded. The coding layer before the model uses the VGG-16 network to extract smaller pictures, so as to meet the requirements of semantic segmentation, and the decoding layer after the model obtains the classification probability value of each point and classifies each point. The experimental results show that the SegNet algorithm can realize the recognition and segmentation of insulator string images under complex conditions with high accuracy.In order to realize the identification and location of the self-exploding fault position of the insulator string, the training model can be used to detect the position of the self-exploding insulator in the image. First, the data of target detection is preprocessed, and then the data set transformation method of rotation transformation and image mirroring is used; then the SSD model is selected to identify and locate the self-explosion position of the insulator. The SSD model combines the advantages of the Faster R-CNN model and the YOLO model. It runs faster and has higher detection accuracy, which meets practical applications. The experimental results show that the SSD algorithm is more targeted to the insulator data set, and can improve the speed and accuracy of insulator identification and positioning.

Block-Based Scheduling Algorithm for Layered Decoding of Block LDPC Codes

Block-Based Scheduling Algorithm for Layered Decoding of Block LDPC Codes

Predicting the potential human lncRNA-miRNA interactions based on graph convolution network with conditional random field.

Long non-coding RNA (lncRNA) and microRNA (miRNA) are two typical types of non-coding RNAs (ncRNAs), their interaction plays an important regulatory role in many biological processes. Exploring the interactions between unknown lncRNA and miRNA can help us better understand the functional expression between lncRNA and miRNA. At present, the interactions between lncRNA and miRNA are mainly obtained through biological experiments, but such experiments are often time-consuming and labor-intensive, it is necessary to design a computational method that can predict the interactions between lncRNA and miRNA. In this paper, we propose a method based on graph convolutional neural (GCN) network and conditional random field (CRF) for predicting human lncRNA-miRNA interactions, named GCNCRF. First, we construct a heterogeneous network using the known interactions of lncRNA and miRNA in the LncRNASNP2 database, the lncRNA/miRNA integration similarity network, and the lncRNA/miRNA feature matrix. Second, the initial embedding of nodes is obtained using a GCN network. A CRF set in the GCN hidden layer can update the obtained preliminary embeddings so that similar nodes have similar embeddings. At the same time, an attention mechanism is added to the CRF layer to reassign weights to nodes to better grasp the feature information of important nodes and ignore some nodes with less influence. Finally, the final embedding is decoded and scored through the decoding layer. Through a 5-fold cross-validation experiment, GCNCRF has an area under the receiver operating characteristic curve value of 0.947 on the main dataset, which has higher prediction accuracy than the other six state-of-the-art methods.

SFA-Net: Scale and Feature Aggregate Network for Retinal Vessel Segmentation.

A U-Net-based network has achieved competitive performance in retinal vessel segmentation. Previous work has focused on using multilevel high-level features to improve segmentation accuracy but has ignored the importance of shallow-level features. In addition, multiple upsampling and convolution operations may destroy the semantic feature information contained in the decoder layer. To address these problems, we propose a scale and feature aggregate network (SFA-Net), which can make full use of multiscale high-level feature information and shallow features. In this paper, a residual atrous spatial feature aggregate block (RASF) is embedded at the end of the encoder to learn multiscale information. Furthermore, an attentional feature module (AFF) is proposed to enhance the effective fusion between shallow and high-level features. In addition, we designed the multi-path feature fusion (MPF) block to fuse high-level features of different decoder layers, which aims to learn the relationship between the high-level features of different paths and alleviate the information loss. We apply the network to the three benchmark datasets (DRIVE, STARE, and CHASE_DB1) and compare them with the other current state-of-the-art methods. The experimental results demonstrated that the proposed SFA-Net performs effectively, indicating that the network is suitable for processing some complex medical images.

A Bayesian Nonlinear Reduced Order Modeling Using Variational AutoEncoders

This paper presents a new nonlinear projection based model reduction using convolutional Variational AutoEncoders (VAEs). This framework is applied on transient incompressible flows. The accuracy is obtained thanks to the expression of the velocity and pressure fields in a nonlinear manifold maximising the likelihood on pre-computed data in the offline stage. A confidence interval is obtained for each time instant thanks to the definition of the reduced dynamic coefficients as independent random variables for which the posterior probability given the offline data is known. The parameters of the nonlinear manifold are optimized as the ones of the decoder layers of an autoencoder. The parameters of the conditional posterior probability of the reduced coefficients are the ones of the encoder layers of the same autoencoder. The optimization of both sets of the encoder and the decoder parameters is obtained thanks to the application of a variational Bayesian method, leading to variational autoencoders. This Reduced Order Model (ROM) is not a regression model over the offline pre-computed data. The numerical resolution of the ROM is based on the Chorin projection method. We apply this new nonlinear projection-based Reduced Order Modeling (ROM) for a 2D Karman Vortex street flow and a 3D incompressible and unsteady flow in an aeronautical injection system.

A novel transformer model for surface damage detection and cognition of concrete bridges

Bridges play an important role in modern transportation systems and road networks, and hence it is essential to use various models based on visual inspection to detect and prevent the damages on the surface of bridge structure. However, due to the limitation of traditional models or lack of modelling data, bridge damages are often difficult to be accurately detected. This paper proposed a novel deep learning model called Bridge Detection Transformers (BR-DETR) based on Detection Transformers (DETR). Through analysis of existing bridge damage instances, we used a copy-paste data augmentation method to create new samples and significantly increased the sample size. Convolution was replaced by Deformable Conv2D, which introduces two-dimensional offsets to the regular grid sampling positions of standard convolution. Convolutional Project Attention was also added after the self-attention layer, which enabled additional modeling of local spatial context. In each encoder and decoder layer, Locally-enhanced Feed-Forward (LeFF) was used to replace the Feedforward to promote the correlation between adjacent tokens in the spatial dimension. The BR-DETR model outperformed the DETR model in detection performance with increased mAP and recall on the augmented highway bridge damage dataset and on the augmented Shandong bridge damage dataset.

A Generative Image Inpainting Model Based on Edge and Feature Self-Arrangement Constraints.

At present, the image inpainting method based on deep learning has achieved a better inpainting effect than traditional methods, but the inpainting results still have problems such as local structure disorder and blurred texture when the images involving a large defect area are processed. This paper proposes a second-order generative image inpainting model based on edge and feature self-arrangement constraints. The model consists of two parts: edge repair network and image repair network. Based on the self-encoder, the edge repair network generates the edges in the defect area according to the known information of the image and improves the edge repair effect by minimizing the adversarial loss and feature matching loss. The image inpainting network fills the defect area with the edge repair result as a priori condition. On the basis of U-Net, the feature self-arrangement module (FSM) is proposed to reconstruct the coding features of a specific scale, and the reconstructed feature skips to connect the decoding layer of the same scale, and it is fused with the upper layer underlying features for decoding. Meanwhile, guide loss, adversarial loss, and reconstruction loss are introduced to narrow the difference between the repaired image and the original image. The experimental results show that the inpainting results of the proposed model have stronger structural connectivity and clearer textures and the performance of PSNR, SSIM, and mean L1 loss in the Celeba, Facade, and Places2 is better than other inpainting methods, indicating that the algorithm can produce an inpainting effect with highly connected structure, reasonable semantics, and fine details.

Fast Crack Segmentation with Depth-to-Space Operator for Pavement Maintenance

The quality of a city's infrastructure drives socioeconomic development. Specifically, it is important to streamline pavement quality monitoring to improve transportation. However, crack segmentation is a computational challenging problem that requires a fast response. In this paper, we propose a Fully Convolutional Network (FCN) for pavement crack segmentation using depth-to-space operation in the decoder and direct connections between the encoder and decoder layers to improve segmentation performance. This approach reduces the number of layers in the decoder. Consequently, training and inference computational costs are reduced. We tested our model on public datasets for comparison with fast state-of-the-art methods. Our model yielded better performance with lower computational costs, reaching real-time segmentation at the rate of 11 frames-per-second. Besides, we introduce a new dataset called CrackIPN as a benchmark that has four times more images and greater image diversity than commonly used datasets.

CSM-Net: Automatic joint segmentation of intima-media complex and lumen in carotid artery ultrasound images

The intima-media thickness (IMT) is an effective biomarker for atherosclerosis, which is commonly measured by ultrasound technique. However, the intima-media complex (IMC) segmentation for the IMT is challenging due to confused IMC boundaries and various noises. In this paper, we propose a flexible method CSM-Net for the joint segmentation of IMC and Lumen in carotid ultrasound images. Firstly, the cascaded dilated convolutions combined with the squeeze-excitation module are introduced for exploiting more contextual features on the highest-level layer of the encoder. Furthermore, a triple spatial attention module is utilized for emphasizing serviceable features on each decoder layer. Besides, a multi-scale weighted hybrid loss function is employed to resolve the class-imbalance issues. The experiments are conducted on a private dataset of 100 images for IMC and Lumen segmentation, as well as on two public datasets of 1600 images for IMC segmentation. For the private dataset, our method obtain the IMC Dice, Lumen Dice, Precision, Recall, and F1 score of 0.814 ± 0.061, 0.941 ± 0.024, 0.911 ± 0.044, 0.916 ± 0.039, and 0.913 ± 0.027, respectively. For the public datasets, we obtain the IMC Dice, Precision, Recall, and F1 score of 0.885 ± 0.067, 0.885 ± 0.070, 0.894 ± 0.089, and 0.885 ± 0.067, respectively. The results demonstrate that the proposed method precedes some cutting-edge methods, and the ablation experiments show the validity of each module. The proposed method may be useful for the IMC segmentation of carotid ultrasound images in the clinic. Our code is publicly available at https://github.com/yuanyc798/US-IMC-code.

Heterogeneous deep graph convolutional network with citation relational BERT for COVID-19 inline citation recommendation

The outbreak of COVID-19 brings almost the biggest explosions of scientific literature ever. Facing such volume literature, it is hard for researches to find desired citation when carrying out COVID-19 related research, especially for junior researchers. This paper presents a novel neural network based method, called citation relational BERT with heterogeneous deep graph convolutional network (CRB-HDGCN), for COVID-19 inline citation recommendation task. The CRB-HDGCN contains two main stages. The first stage is to enhance the representation learning of BERT model for COVID-19 inline citation recommendation task through CRB. To achieve the above goal, an augmented citation sentence corpus, which replaces the citation placeholder with the title of the cited papers, is used to lightly retrain BERT model. In addition, we extract three types of sentence pair according citation relation, and establish sentence prediction tasks to further fine-tune the BERT model. The second stage is to learn effective dense vector of nodes among COVID-19 bibliographic graph through HDGCN. The HDGCN contains four layers which are essentially all sub neural networks. The first layer is initial embedding layer which generates initial input vectors with fixed size through CRB and a multilayer perceptron. The second layer is a heterogeneous graph convolutional layer. In this layer, we expand traditional homogeneous graph convolutional network into heterogeneous by subtly adding heterogeneous nodes and relations. The third layer is a deep attention layer. This layer uses trainable project vectors to reweight the node importance simultaneously according to both node types and convolution layers, which further promotes the performance of learnt node vectors. The last decoder layer recovers the graph structure and let the whole network trainable. The recommendation is finally achieved by integrating the high performance heterogeneous vectors learnt from CRB-HDGCN with the query vectors. We conduct experiments on the CORD-19 and LitCovid datasets. The results show that compared with the second best method CO-Search, CRB-HDGCN improves MAP, MRR, P@100 and R@100 with 21.8%, 22.7%, 37.6% and 21.2% on CORD-19, and 29.1%, 25.9%, 15.3% and 11.3% on LitCovid, respectively.

Ghostformer: A GhostNet-Based Two-Stage Transformer for Small Object Detection.

In this paper, we propose a novel two-stage transformer with GhostNet, which improves the performance of the small object detection task. Specifically, based on the original Deformable Transformers for End-to-End Object Detection (deformable DETR), we chose GhostNet as the backbone to extract features, since it is better suited for an efficient feature extraction. Furthermore, at the target detection stage, we selected the 300 best bounding box results as regional proposals, which were subsequently set as primary object queries of the decoder layer. Finally, in the decoder layer, we optimized and modified the queries to increase the target accuracy. In order to validate the performance of the proposed model, we adopted a widely used COCO 2017 dataset. Extensive experiments demonstrated that the proposed scheme yielded a higher average precision (AP) score in detecting small objects than the existing deformable DETR model.

Power Line Scene Recognition Based on Convolutional Capsule Network with Image Enhancement

With the popularization of unmanned aerial vehicle (UAV) applications and the continuous development of the power grid network, identifying power line scenarios in advance is very important for the safety of low-altitude flight. The power line scene recognition (PLSR) under complex background environments is particularly important. The complex background environment of power lines is usually mixed by forests, rivers, mountains, buildings, and so on. In these environments, the detection of slender power lines is particularly difficult. In this paper, a PLSR method of complex backgrounds based on the convolutional capsule network with image enhancement is proposed. The enhancement edge features of power line scenes based on the guided filter are fused with the convolutional capsule network framework. First, the guided filter is used to enhance the power line features in order to improve the recognition of the power line in the complex background. Second, the convolutional capsule network is used to extract the depth hierarchical features of the scene image of power lines. Finally, the output layer of the convolutional capsule network identifies the power line and non-power line scenes, and through the decoding layer, the power lines are reconstructed in the power line scene. Experimental results show that the accuracy of the proposed method obtains 97.43% on the public dataset. Robustness and generalization test results show that it has a good application prospect. Furthermore, the power lines can be accurately extracted from the complex backgrounds based on the reconstructed module.

Study of Multiscale Fused Extraction of Cropland Plots in Remote Sensing Images Based on Attention Mechanism.

Cropland extraction from remote sensing images is an essential part of precise digital agriculture services. This paper proposed an SSGNet network of multiscale fused extraction of cropland based on the attention mechanism to address issues with complex cropland feature types in remote sensing images that resulted in blurred boundaries and low accuracy in plot partitioning. The proposed network contains different modules, such as spatial gradient guidance and dilated semantic fusion. It employs the image gradient attention guidance module to fully extract cropland plot features. This causes the feature to be transferred from the encoding layer to the decoding layer, creating layers full of key features within the cropland and making the extracted cropland information more accurate. In addition, this study also solves the problem caused by a large amount of spatial feature information, which losses easily during the downsampling process of continuous convolution in the coding layer. Aiming to solve this issue, we put forward a model for consensus fusion of multiscale spatial features to fuse each-layer feature of the coding layer through dilated convolution with different dilated ratios. This approach was proposed to make the segmentation results more comprehensive and complete. The lab findings showed that the Precision, Recall, MIoU, and F1 score of the multiscale fusion segmentation SSGNet network based on the attention mechanism had achieved 93.46%, 90.91%, 85.54%, and 92.73%, respectively. Its segmentation effect on cropland was better than other semantic segmentation networks and can effectively promote cropland semantic extraction.

Task-Oriented Multi-User Semantic Communications

While semantic communications have shown the potential in the case of single-modal single-users, its applications to the multi-user scenario remain limited. In this paper, we investigate deep learning (DL) based multi-user semantic communication systems for transmitting single-modal data and multimodal data, respectively. We adopt three intelligent tasks, including, image retrieval, machine translation, and visual question answering (VQA) as the transmission goal of semantic communication systems. We propose a Transformer based framework to unify the structure of transmitters for different tasks. For the single-modal multi-user system, we propose two Transformer based models, named, DeepSC-IR and DeepSC-MT, to perform image retrieval and machine translation, respectively. In this case, DeepSC-IR is trained to optimize the distance in embedding space between images and DeepSC-MT is trained to minimize the semantic errors by recovering the semantic meaning of sentences. For the multimodal multi-user system, we develop a Transformer enabled model, named, DeepSC-VQA, for the VQA task by extracting text-image information at the transmitters and fusing it at the receiver. In particular, a novel layer-wise Transformer is designed to help fuse multimodal data by adding connection between each of the encoder and decoder layers. Numerical results show that the proposed models are superior to traditional communications in terms of the robustness to channels, computational complexity, transmission delay, and the task-execution performance at various task-specific metrics.

Sparse point‐voxel aggregation network for efficient point cloud semantic segmentation

Effective and efficient semantic segmentation of 3D point cloud data is important for many tasks. Many methods for point cloud semantic segmentation rely on computationally expensive sampling and grouping layers to process irregular points, while others convert irregular points into regular volumetric grids and process them with a 3D U-Net-based semantic segmentation network. However, most of these methods suffer from high computational costs and cannot be applied to the real-time processing of large-scale point clouds. To address these issues, we propose a computationally efficient point-voxel-based network architecture named Sparse Point-Voxel Aggregation Network (SPVAN) for point cloud semantic segmentation. It consists of an encoding layer that consists of sparse convolution and MLP layers and a new decoding layer called Point Feature Aggregation Layer (PFAL) that is only composed of feature interpolation and MLP layers. Compared with recent popular point-voxel-based methods with the U-Net-based network, our method does not need 3D convolution networks in the decoding layer and thus achieves a higher speed. Experimental results on the large-scale SemanticKITTI dataset show that our method gets a good balance between the efficiency and the performance. Moreover, our method achieves on-par or better performance than previous methods for semantic segmentation on the challenging S3DIS dataset.