Layered Decoding Research Articles

Continuous conditional random field convolution for point cloud segmentation

Point cloud segmentation is the foundation of 3D environmental perception for modern intelligent systems. To solve this problem and image segmentation, conditional random fields (CRFs) are usually formulated as discrete models in label space to encourage label consistency, which is actually a kind of postprocessing. In this paper, we reconsider the CRF in feature space for point cloud segmentation because it can capture the structure of features well to improve the representation ability of features rather than simply smoothing. Therefore, we first model the point cloud features with a continuous quadratic energy model and formulate its solution process as a message-passing graph convolution, by which it can be easily integrated into a deep network. We theoretically demonstrate that the message passing in the graph convolution is equivalent to the mean-field approximation of a continuous CRF model. Furthermore, we build an encoder-decoder network based on the proposed continuous CRF graph convolution (CRFConv), in which the CRFConv embedded in the decoding layers can restore the details of high-level features that were lost in the encoding stage to enhance the location ability of the network, thereby benefiting segmentation. Analogous to the CRFConv, we show that the classical discrete CRF can also work collaboratively with the proposed network via another graph convolution to further improve the segmentation results. Experiments on various point cloud benchmarks demonstrate the effectiveness and robustness of the proposed method. Compared with the state-of-the-art methods, the proposed method can also achieve competitive segmentation performance.

Modeling feedback representations in ventral visual cortex using a generative adversarial autoencoder

In less than the blink of an eye, the human brain processes visual sensory input, interprets the visual scene, identifies faces, and recognizes objects. Decades of neurophysiological studies have demonstrated that the brain accomplishes these complicated tasks through a dense network of feedforward and feedback neural processes in the ventral visual cortex. So far, these visual processes are primarily modeled with feedforward hierarchical neural networks, and the computational role of feedback processes is poorly understood. In this study, we developed a generative autoencoder neural network model and adversarially trained it on a large categorically diverse data set of images (Objects, scenes, faces, and animates). We hypothesized that the feedback processes in the ventral visual pathway can be represented by reconstruction of the visual information performed by the generative model. To test the hypothesis, we compared representational similarity of the activity patterns in the internal layers of the proposed model with magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) data acquired while participants (N=15) viewed a set of 156 images organized in four categories of objects, scenes, faces, and animates. Our proposed model identified two segregated neural dynamics in the ventral visual pathway. The representational comparison with MEG data revealed a temporal hierarchy of processes transforming low level visual information into high level semantics in the feedforward sweep, and a temporally subsequent dynamics of inverse processes reconstructing low level visual information from a high level latent representation in the feedback sweep. Further, representational comparison of model encoder and decoder layers with two fMRI regions of interests, namely early visual cortex (EVC) and inferior temporal area (IT), revealed a growing categorical representation (similar to IT) along the encoder layers (feedforward sweep) and a progression in detail visual representations (akin to EVC) along the decoder layers (feedback sweep).

GCRFLDA: scoring lncRNA-disease associations using graph convolution matrix completion with conditional random field.

Long noncoding RNAs (lncRNAs) play important roles in various biological regulatory processes, and are closely related to the occurrence and development of diseases. Identifying lncRNA-disease associations is valuable for revealing the molecular mechanism of diseases and exploring treatment strategies. Thus, it is necessary to computationally predict lncRNA-disease associations as a complementary method for biological experiments. In this study, we proposed a novel prediction method GCRFLDA based on the graph convolutional matrix completion. GCRFLDA first constructed a graph using the available lncRNA-disease association information. Then, it constructed an encoder consisting of conditional random field and attention mechanism to learn efficient embeddings of nodes, and a decoder layer to score lncRNA-disease associations. In GCRFLDA, the Gaussian interaction profile kernels similarity and cosine similarity were fused as side information of lncRNA and disease nodes. Experimental results on four benchmark datasets show that GCRFLDA is superior to other existing methods. Moreover, we conducted case studies on four diseases and observed that 70 of 80 predicted associated lncRNAs were confirmed by the literature.

Attention Res-UNet with Guided Decoder for semantic segmentation of brain tumors

The automatic segmentation of brain tumors in Magnetic Resonance Imaging (MRI) plays a major role in accurate diagnosis and treatment planning. The present study proposes a new deep learning generator architecture called Attention Res-UNet with Guided Decoder (ARU-GD) for the segmentation of brain tumors. The proposed generator architecture have the capability to explicitly guide the learning process of each decoder layer. The individual loss function to each decoder layer helps to supervise the learning process of each layer in the decoder and thereby enables them to generate better feature maps. The attention gates in the generator focuses on the activation of relevant information instead of allowing all information to pass through the skip connections in the Res-UNet. Our model performed well in comparison to the baseline models i.e. UNet, Res-UNet, and Res-UNet with attention gates. The proposed ARU-GD is compared with popular deep learning models VGG-Net, MobileNet, QuickNAT, DenseNet and XceptionNet, and BraTS 2019 leaderboard models. The proposed ARU-GD has achieved Dice Scores of 0.911, 0.876 and 0.801 and mean IoU of 0.838, 0.781 and 0.668 on the whole tumor, tumor core and enhancing tumor respectively on unseen High-Grade Glioma test data. The implementation code is available on the following Github link.

LDPC Decoder Design Using Compensation Scheme of Group Comparison for 5G Communication Systems

This paper presents a dual-mode low-density parity-check (LDPC) decoding architecture that has excellent error-correcting capability and a high parallelism design for fifth-generation (5G) new-radio (NR) applications. We adopted a high parallelism design using a layered decoding schedule to meet the high throughput requirement of 5G NR systems. Although the increase in parallelism can efficiently enhance the throughput, the hardware implementation required to support high parallelism is a significant hardware burden. To efficiently reduce the hardware burden, we used a grouping search rather than a sorter, which was used in the minimum finder with decoding performance loss. Additionally, we proposed a compensation scheme to improve the decoding performance loss by revising the probabilistic second minimum of a grouping search. The post-layout implementation of the proposed dual-mode LDPC decoder is based on the Taiwan Semiconductor Manufacturing Company (TSMC) 40 nm complementary metal-oxide-semiconductor (CMOS) technology, using a compensation scheme of grouping comparison for 5G communication systems with a working frequency of 294.1 MHz. The decoding throughput achieved was at least 10.86 Gb/s without evaluating early termination, and the decoding power consumption was 313.3 mW.

TSU-net: Two-stage multi-scale cascade and multi-field fusion U-net for right ventricular segmentation

Accurate segmentation of the right ventricle from cardiac magnetic resonance images (MRI) is a critical step in cardiac function analysis and disease diagnosis. It is still an open problem due to some difficulties, such as a large variety of object sizes and ill-defined borders. In this paper, we present a TSU-net network that grips deeper features and captures targets of different sizes with multi-scale cascade and multi-field fusion in the right ventricle. TSU-net mainly contains two major components: Dilated-Convolution Block (DB) and Multi-Layer-Pool Block (MB). DB extracts and aggregates multi-scale features for the right ventricle. MB mainly relies on multiple effective field-of-views to detect objects at different sizes and fill boundary features. Different from previous networks, we used DB and MB to replace the convolution layer in the encoding layer, thus, we can gather multi-scale information of right ventricle, detect different size targets and fill boundary information in each encoding layer. In addition, in the decoding layer, we used DB to replace the convolution layer, so that we can aggregate the multi-scale features of the right ventricle in each decoding layer. Furthermore, the two-stage U-net structure is used to further improve the utilization of DB and MB through a two-layer encoding/decoding layer. Our method is validated on the RVSC, a public right ventricular data set. The results demonstrated that TSU-net achieved an average Dice coefficient of 0.86 on endocardium and 0.90 on the epicardium, thereby outperforming other models. It effectively assists doctors to diagnose the disease and promotes the development of medical images. In addition, we also provide an intuitive explanation of our network, which fully explain MB and TSU-net's ability to detect targets of different sizes and fill in boundary features.

Global-Attention-Based Neural Networks for Vision Language Intelligence

In this paper, we develop a novel global-attention-based neural network (GANN) for vision language intelligence, specifically, image captioning (language description of a given image). As many previous works, the encoder-decoder framework is adopted in our proposed model, in which the encoder is responsible for encoding the region proposal features and extracting global caption feature based on a specially designed module of predicting the caption objects, and the decoder generates captions by taking the obtained global caption feature along with the encoded visual features as inputs for each attention head of the decoder layer. The global caption feature is introduced for the purpose of exploring the latent contributions of region proposals for image captioning, and further helping the decoder better focus on the most relevant proposals so as to extract more accurate visual feature in each time step of caption generation. Our GANN is implemented by incorporating the global caption feature into the attention weight calculation phase in the word predication process in each head of the decoder layer. In our experiments, we qualitatively analyzed the proposed model, and quantitatively evaluated several state-of-the-art schemes with GANN on the MS-COCO dataset. Experimental results demonstrate the effectiveness of the proposed global attention mechanism for image captioning.

Double attention U-Net for brain tumor MR image segmentation

PurposeAutomatic segmentation of brain tumor from medical images is a challenging task because of tumor's uneven and irregular shapes. In this paper, the authors propose an attention-based nested segmentation network, named DAU-Net. In total, two types of attention mechanisms are introduced to make the U-Net network focus on the key feature regions. The proposed network has a deep supervised encoder–decoder architecture and a redesigned dense skip connection. DAU-Net introduces an attention mechanism between convolutional blocks so that the features extracted at different levels can be merged with a task-related selection.Design/methodology/approachIn the coding layer, the authors designed a channel attention module. It marks the importance of each feature graph in the segmentation task. In the decoding layer, the authors designed a spatial attention module. It marks the importance of different regional features. And by fusing features at different scales in the same coding layer, the network can fully extract the detailed information of the original image and learn more tumor boundary information.FindingsTo verify the effectiveness of the DAU-Net, experiments were carried out on the BRATS 2018 brain tumor magnetic resonance imaging (MRI) database. The segmentation results show that the proposed method has a high accuracy, with a Dice similarity coefficient (DSC) of 89% in the complete tumor, which is an improvement of 8.04 and 4.02%, compared with fully convolutional network (FCN) and U-Net, respectively.Originality/valueThe experimental results show that the proposed method has good performance in the segmentation of brain tumors. The proposed method has potential clinical applicability.

ONE-MINIUM-ONLY BASIC-SET TRELLIS MIN-MAX DECODER ARCHITECTURE FOR NONBINARY LDPC CODE

Nonbinary low-density-parity-check (NB-LDPC) code outperforms their binary counterpart in terms of error-correcting performance and error-floor property when the code length is moderate. However, the drawback of NB-LDPC decoders is high complexity and the complexity increases considerably when increasing the Galois-field order. In this paper, an One-Minimum-Only basic-set trellis min-max (OMO-BS-TMM) algorithm and the corresponding decoder architecture are proposed for NBLDPC codes to greatly reduce the complexity of the check node unit (CNU) as well as the whole decoder. In the proposed OMO-BS-TMM algorithm, only the first minimum values are used for generating the check node messages instead of using both the first and second minimum values, and the number of messages exchanged between the check node and the variable node is reduced in comparison with the previous works. Layered decoder architectures based on the proposed algorithm were implemented for the (837, 726) NB-LDPC code over GF(32) using 90-nm CMOS technology. The implementation results showed that the OMO-BS-TMM algorithm achieves the almost similar error-correcting performance, and a reduction of the complexity by 31.8% and 20.5% for the whole decoder, compared to previous works. Moreover, the proposed decoder achieves a higher throughput at 1.4 Gbps, compared with the other state-of-the-art NBLDPC decoders.

Polar Coding for Channels With Deletions

Deletion errors are notoriously difficult to correct. The capacity of the binary deletion channel is not yet fully known, and there is no completely satisfactory solution even to the specific problem of correcting only two deletion errors. In this paper, we show that polarization theory, along with the powerful methods of polar coding and successive-cancellation decoding, can be extended to channels with deletion errors. Our results further generalize to channels corrupted by insertion errors and/or combinations of insertions and deletions. It was recently proposed to correct deletion errors using polar codes designed for the BEC, precoded with a sufficiently powerful CRC code. In this approach, in order to correct ${d}$ deletions in a codeword of length ${n}$ , successive-cancellation decoding is run separately for each of the $\binom {n}{d}$ possible deletion patterns, while treating the deleted symbols as erasures. The resulting complexity of decoding is ${O}\bigl ({n}^{d+1}\log {n}\bigr)$ . In contrast, we develop herein a natural generalization of successive-cancellation decoding to channels with deletion errors. Our algorithm is based on the recursive structure of polar codes, processing the channel output in ${m} = \log _{2} {n}$ decoding layers, just like the conventional successive-cancellation decoder. Every node in each decoding layer propagates its uncertainty about the deletion pattern to the nodes in the next layer and, eventually, the correct deletion pattern becomes visible at the decoder output, with high probability. This makes it possible to consider at most $\binom {d+2}{2}$ scenarios at each node, and the overall decoding complexity is only ${O}\bigl ({d}^{3} {n} \log {n}\bigr)$ , which scales polynomially rather than exponentially with the number of deletions. Based on the proposed algorithm, we also investigate the channel polarization phenomenon. We are able to prove weak polarization where the number of deletions ${d}$ is small, i.e. ${d}={o}({n})$ . We also conjecture the strong polarization for when ${d}={O}(1)$ and provide some evidence to support this conjecture. The strong polarization, if proven, implies that our coding scheme achieves the capacity of any binary-input symmetric channel which furthermore introduces a constant number of deletions.

A novel convolutional neural network architecture of multispectral remote sensing images for automatic material classification

For real-world simulation, terrain models must combine various types of information on material and texture in terrain reconstruction for the three-dimensional numerical simulation of terrain. However, the construction of such models using the conventional method often involves high costs in both manpower and time. Therefore, this study used a convolutional neural network (CNN) architecture to classify material in multispectral remote sensing images to simplify the construction of future models. Visible light (i.e., RGB), near infrared (NIR), normalized difference vegetation index (NDVI), and digital surface model (DSM) images were examined.This paper proposes the use of the robust U-Net (RUNet) model, which integrates multiple CNN architectures, for material classification. This model, which is based on an improved U-Net architecture combined with the shortcut connections in the ResNet model, preserves the features of shallow network extraction. The architecture is divided into an encoding layer and a decoding layer. The encoding layer comprises 10 convolutional layers and 4 pooling layers. The decoding layer contains four upsampling layers, eight convolutional layers, and one classification convolutional layer. The material classification process in this study involved the training and testing of the RUNet model. Because of the large size of remote sensing images, the training process randomly cuts subimages of the same size from the training set and then inputs them into the RUNet model for training. To consider the spatial information of the material, the test process cuts multiple test subimages from the test set through mirror padding and overlapping cropping; RUNet then classifies the subimages. Finally, it merges the subimage classification results back into the original test image.The aerial image labeling dataset of the National Institute for Research in Digital Science and Technology (Inria, abbreviated from the French Institut national de recherche en sciences et technologies du numérique) was used as well as its configured dataset (called Inria-2) and a dataset from the International Society for Photogrammetry and Remote Sensing (ISPRS). Material classification was performed with RUNet. Moreover, the effects of the mirror padding and overlapping cropping were analyzed, as were the impacts of subimage size on classification performance. The Inria dataset achieved the optimal results; after the morphological optimization of RUNet, the overall intersection over union (IoU) and classification accuracy reached 70.82% and 95.66%, respectively. Regarding the Inria-2 dataset, the IoU and accuracy were 75.5% and 95.71%, respectively, after classification refinement. Although the overall IoU and accuracy were 0.46% and 0.04% lower than those of the improved fully convolutional network, the training time of the RUNet model was approximately 10.6 h shorter. In the ISPRS dataset experiment, the overall accuracy of the combined multispectral, NDVI, and DSM images reached 89.71%, surpassing that of the RGB images. NIR and DSM provide more information on material features, reducing the likelihood of misclassification caused by similar features (e.g., in color, shape, or texture) in RGB images. Overall, RUNet outperformed the other models in the material classification of remote sensing images. The present findings indicate that it has potential for application in land use monitoring and disaster assessment as well as in model construction for simulation systems.

A generative image inpainting network based on the attention transfer network across layer mechanism

To solve the problems of blur, artifacts, and semantic inaccuracy of existing deep learning-based image inpainting algorithm when the large-area irregular defect area images are repaired, combining the U-NET architecture and the idea of a generative adversarial network, a generative image inpainting network based on the attention transfer cross layer mechanism is proposed. The network is divided into two parts: generator and discriminator; the generator proposes a attention transfer network cross layer based on the U-NET architecture; after the input image is encoded, the attention transfer network cross layer is used to reconstruct the encoding feature map of each layer. The decoding is performed by fusing the skip connection with the corresponding potential feature in the multi-scale decoder, solving the information loss between the codec network layers. Moreover, each decoding layer of the multi-scale decoder is prompted to generate consistent content, and finally the repaired image is obtained. After the decoding is completed, the decoding feature map of each layer is converted into a multi-scale RGB image, and the reconstruction L1 loss is applied on each scale; by minimizing the adversarial loss, the discriminator can determine the authenticity of the repaired image; the perceptual loss and style loss are introduced to judge the output image, so as to restrict the generator to generate more realistic and reasonable content. A qualitative and quantitative comparative analysis is performed on the Celeba, Façade, and Places2 with the current mainstream image inpainting algorithms. The results show that the repair results in this paper are better than others in terms of mean L1 loss, PSNR, and SSIM, and more in line with human visual characteristics. The attention transfer network cross layer proposed in this paper can effectively reconstruct the more detailed coding feature map, and it can play an active guiding role in the coding process by using skip connection; the image inpainting network can generate content that is highly consistent with the structure and semantics of the real image when repairing large-area irregular defect areas, which is more in line with the human visual experience.

Automated Generation of Novel Fragments Using Screening Data, a Dual SMILES Autoencoder, Transfer Learning and Syntax Correction

Fragment-based hit identification (FBHI) allows proportionately greater coverage of chemical space using fewer molecules than traditional high-throughput screening approaches. However, effectively exploiting this advantage is highly dependent on the library design. Solubility, stability, chemical complexity, chemical/shape diversity, and synthetic tractability for fragment elaboration are all critical aspects, and molecule design remains a time-consuming task for computational and medicinal chemists. Artificial neural networks have attracted considerable attention in automated de novo design applications and could also prove useful for fragment library design. Chemical autoencoders are neural networks consisting of encoder and decoder parts, which respectively compress and decompress molecular representations. The decoder is applied to samples drawn from the space of compressed representations to generate novel molecules that can be scored for properties of interest. Here, we report an autoencoder model using a recurrent neural network architecture, which was trained using 486,565 fragments curated from commercial sources, to simultaneously reconstruct both SMILES and chemical fingerprints. To explore its utility in fragment design, we applied transfer learning to the fingerprint decoder layers to train a classifier using 66 frequent hitter fragments identified from our screening campaigns. Using a particle swarm optimization sampling approach, we compare the performance of this "dual" model to an architecture encoding SMILES only. The dual model produced valid SMILES with improved features, considering a range of properties including aromatic ring counts, heavy atom count, synthetic accessibility, and a new fragment complexity score we term Feature Complexity (FeCo). Additionally, we demonstrate that generative performance is further enhanced by use of a simple syntax-correction procedure during training, in which invalid and undesirable SMILES are spiked into the training set. Finally, we used the syntax-corrected model to generate a library of novel candidate privileged fragments.

Polarization image fusion with self-learned fusion strategy

Polarization image fusion aims to integrate intensity and degree of linear polarization images into one with more details, which is beneficial to improve the ability of targets detection under complex background. The fusion strategies in conventional methods are designed in a hand-crafted way and not robust to different fusion tasks. In this paper, we propose a novel and deep network to address the polarization image fusion issue with self-learned strategy. The network consists of Encoder, Fusion, and Decoder layers. Feature maps extracted by Encoder are fused, then fed into Decoder to generate fused images. Besides, a novel loss function is adopted to train the network in an unsupervised way, without ground truth of fused images. To verify the advantage, the network trained on polarization images is also used to infrared and visible images fusion, and multi-focus image fusion. Experimental results showed that our method outperforms several state-of-the-art methods in terms of visual quality and quantitative measurement. The proposed fused method can be applied in the military and civilian fields such as camouflage and hidden targets detection, medical diagnosis, and environmental monitoring.

An Efficient Transformer Decoder with Compressed Sub-layers

The large attention-based encoder-decoder network (Transformer) has become prevailing recently due to its effectiveness. But the high computation complexity of its decoder raises the inefficiency issue. By examining the mathematic formulation of the decoder, we show that under some mild conditions, the architecture could be simplified by compressing its sub-layers, the basic building block of Transformer, and achieves a higher parallelism. We thereby propose Compressed Attention Network, whose decoder layer consists of only one sub-layer instead of three. Extensive experiments on 14 WMT machine translation tasks show that our model is 1.42x faster with performance on par with a strong baseline. This strong baseline is already 2x faster than the widely used standard baseline without loss in performance.

Dermo-DOCTOR: A framework for concurrent skin lesion detection and recognition using a deep convolutional neural network with end-to-end dual encoders

Background and objectiveAutomated skin lesion analysis for simultaneous detection and recognition is still challenging for inter-class homogeneity and intra-class heterogeneity, leading to low generic capability of a single convolutional neural network (CNN) with limited datasets. MethodsThis article proposes an end-to-end deep CNN-based framework for simultaneous detection and recognition of the skin lesions, named Dermo-DOCTOR, consisting of two encoders. The feature maps from two encoders are fused channel-wise, called Fused Feature Map (FFM). The FFM is utilized for decoding in the detection sub-network, concatenating each stage of two encoders’ outputs with corresponding decoder layers to retrieve the lost spatial information due to pooling in the encoders. For the recognition sub-network, the outputs of three fully connected layers, utilizing feature maps of two encoders and FFM, are aggregated to obtain a final lesion class. We train and evaluate the proposed Dermo-Doctor utilizing two publicly available benchmark datasets, such as ISIC-2016 and ISIC-2017. ResultsThe achieved segmentation results exhibit mean intersection over unions of 85.0% and 80.0% respectively for ISIC-2016 and ISIC-2017 test datasets. The proposed Dermo-DOCTOR also demonstrates praiseworthy success in lesion recognition, providing the areas under the receiver operating characteristic curves of 0.98 and 0.91 respectively for those two datasets. The experimental results show that the proposed Dermo-DOCTOR outperforms the alternative methods mentioned in the literature, designed for skin lesion detection and recognition. ConclusionAs the Dermo-DOCTOR provides better results on two different test datasets, even with limited training data, it can be an auspicious computer-aided assistive tool for dermatologists.

Echo state network with a global reversible autoencoder for time series classification

• In order to reestablish the random initialization input weights of ESN, a single layer ESN with a global reversible AE (GRAE) algorithm is proposed to ensure that the outputs of GRAE are remarkably correlated with the input data. • The reservoir layer with a reversible function is calculated by pulling the ESN output back and injecting it into the reservoir layer. Thus, the feature learning is enriched by additional information, which results in a better performance. • The GRAE which is based on ridge regression to solve a least squares problem is entirely different from existing back-propagation (BP) based AEs. So, the training speed of GRAE is many times faster than that of the BP based AEs. • In existing ESN autoencoder methods, all weights in the reservoir layer are generated randomly. The simple cycle connection is utilized in the reservoir layer of ESN to make sure the reservoir weights can be generated deterministically. An echo state network (z) can provide an efficient dynamic solution for predicting time series problems. However, in most cases, ESN models are applied for predictions rather than classifications. The applications of ESN in time series classification (TSC) problems have yet to be fully studied. Moreover, the conventional randomly generated ESN is unlikely to be optimal because of the randomly generated input and reservoir weights, which are not always guaranteed to be optimal. Randomly generating all layer weights is improper, because a purely random layer might destroy the useful features. To overcome this disadvantage, this study provides a new input weight establishment framework of ESN based on autoencoder (AE) theory for TSC tasks. A global reversible AE (GRAE) algorithm is proposed to reestablish the random initialization input weights of the ESN. In existing ESN-AEs, the output weights obtained in the encoding process are directly reused as the initial input weights. By contrast, in GRAE, the reservoir layer with a reversible activation function is calculated by pulling the decoding layer output back and injecting it into the reservoir layer. Thus, feature learning is enriched by additional information, which results in improved performance. The current weights of the encoding layer are iteratively replaced by the decoding layer to ensure that the outputs of the GRAE are remarkably correlated with the input data. Visualization analyses and experiments of the input weights on a massive set of UCR time series datasets indicate that the proposed GRAE method can considerably improve the original two-layer ESN-based classifiers and the proposed GRAE-ESN classifier yields better performance compared with traditional state-of-the-art TSC classifiers. Furthermore, the proposed method can provide comparable performance and considerably faster training speed compared with three deep learning classifiers.

Enhanced U-Net: A Feature Enhancement Network for Polyp Segmentation.

Colonoscopy is a procedure to detect colorectal polyps which are the primary cause for developing colorectal cancer. However, polyp segmentation is a challenging task due to the diverse shape, size, color, and texture of polyps, shuttle difference between polyp and its background, as well as low contrast of the colonoscopic images. To address these challenges, we propose a feature enhancement network for accurate polyp segmentation in colonoscopy images. Specifically, the proposed network enhances the semantic information using the novel Semantic Feature Enhance Module (SFEM). Furthermore, instead of directly adding encoder features to the respective decoder layer, we introduce an Adaptive Global Context Module (AGCM), which focuses only on the encoder's significant and hard fine-grained features. The integration of these two modules improves the quality of features layer by layer, which in turn enhances the final feature representation. The proposed approach is evaluated on five colonoscopy datasets and demonstrates superior performance compared to other state-of-the-art models.

A novel weight initialization with adaptive hyper-parameters for deep semantic segmentation

The semantic segmentation process divides an image into its constituent objects and background by assigning a corresponding class label to each pixel in the image. Semantic segmentation is an important area in computer vision with wide practical applications. The contemporary semantic segmentation approaches are primarily based on two types of deep neural networks architectures i.e., symmetric and asymmetric networks. Both types of networks consist of several layers of neurons which are arranged in two sections called encoder and decoder. The encoder section receives the input image and the decoder section outputs the segmented image. However, both sections in symmetric networks have the same number of layers and the number of neurons in an encoder layer is the same as that of the corresponding layer in the decoder section but asymmetric networks do not strictly follow such one-one correspondence between encoder and decoder layers. At the moment, SegNet and ESNet are the two leading state-of-the-art symmetric encoder-decoder deep neural network architectures. However, both architectures require extensive training for good generalization and need several hundred epochs for convergence. This paper aims to improve the convergence and enhance network generalization by introducing two novelties into the network training process. The first novelty is a weight initialization method and the second contribution is an adaptive mechanism for dynamic layer learning rate adjustment in training loop. The proposed initialization technique uses transfer learning to initialize the encoder section of the network, but for initialization of decoder section, the weights of the encoder section layers are copied to the corresponding layers of the decoder section. The second contribution of the paper is an adaptive layer learning rate method, wherein the learning rates of the encoder layers are updated based on a metric representing the difference between the probability distributions of the input images and encoder weights. Likewise, the learning rates of the decoder layers are updated based on the difference between the probability distributions of the output labels and decoder weights. Intensive empirical validation of the proposed approach shows significant improvement in terms of faster convergence and generalization.

Jazz Bass Transcription Using a U-Net Architecture

In this paper, we adapt a recently proposed U-net deep neural network architecture from melody to bass transcription. We investigate pitch shifting and random equalization as data augmentation techniques. In a parameter importance study, we study the influence of the skip connection strategy between the encoder and decoder layers, the data augmentation strategy, as well as of the overall model capacity on the system’s performance. Using a training set that covers various music genres and a validation set that includes jazz ensemble recordings, we obtain the best transcription performance for a downscaled version of the reference algorithm combined with skip connections that transfer intermediate activations between the encoder and decoder. The U-net based method outperforms previous knowledge-driven and data-driven bass transcription algorithms by around five percentage points in overall accuracy. In addition to a pitch estimation improvement, the voicing estimation performance is clearly enhanced.