Sequences Of Different Lengths Research Articles

Raman Analysis of Polyvinyl Chloride Degraded under Different Temperature Conditions

In this work, the Raman spectroscopy method was used to study changes in the composition of polyvinyl chloride (PVC) during its degradation. Traditionally, such changes are analyzed optically due to the fact that during degradation, polyene sequences of different lengths are formed in the PVC structure, which change many properties of the material, including optical ones. The Raman spectroscopy method chosen in this work is characterized by high accuracy and speed of analysis, as well as the ability to diagnose the presence of polyenes in extremely low concentrations. Undegraded PVC films and films degraded in various temperature regimes are used as study samples. It is shown that the Raman spectra of the samples can be used to analyze both the length of the formed polyenes and their length distribution. It is found that the length distribution of polyenes depends on the conditions of thermal degradation of the samples.

Normalising sequence lengths using the relative duration of episodes: an application to doctoral trajectories in Germany.

To address significant variation of sequence lengths of doctoral trajectories, we propose sequence normalisation using the relative duration of episodes. We employ episode data from a panel study of doctorate holders in Germany where doctoral trajectories are measured in single months and differ in length up to several years. Utilising normalised sequences instead of absolute sequences, we are better able to identify typical trajectories. The graphical presentation of the cluster solutions more accurately depicts the underlying processes. Furthermore, it offers the possibility to define reference sequences without a fixed length. Normalising sequences instead of distances thus proves an easily implementable method to compare sequences of different lengths when the identification of patterns is a priority.

Deep Learning Regression with Sequences of Different Length: An Application for State of Health Trajectory Prediction and Remaining Useful Life Estimation in Lithium-Ion Batteries

This study presents methods to handle deep learning regressions with input and output sequences of different lengths. We discuss the Autoregressive one-step prediction framework and introduce an innovative one-time multi-step (OTMS) prediction approach, based on a custom loss function, that predicts all future steps in a single shot. The presented methodologies are then applied to simultaneously predict the State of Health (SoH) trajectory and estimate the Remaining Useful Life (RUL) of lithium-ion battery cells. Accurate estimates of SoH trajectory and RUL are essential for Battery Management Systems (BMSs), electronic systems that guarantee safety while maximizing performance and extending battery lifespan. In this context, the studied methodologies were compared using a rigorous cross-validation approach. The OTMS model showed better predictions in early cycles, while the Autoregressive model performed better in later cycles, suggesting a hybrid approach between these two methodologies as an optimal solution.

Design and evaluation of antisense sequence length for modified mouse U7 small nuclear RNA to induce efficient pre-messenger RNA splicing modulation in vitro.

Pre-messenger RNA (pre-mRNA) splicing modulation is an attractive approach for investigating the mechanisms of genetic disorders caused by mis-splicing. Previous reports have indicated that a modified U7 small nuclear RNA (U7 snRNA) is a prospective tool for modulating splicing both in vitro and in vivo. To date, very few studies have investigated the role of antisense sequence length in modified U7 snRNA. In this study, we designed a series of antisense sequences with various lengths and evaluated their efficiency in inducing splicing modulation. To express modified U7 snRNAs, we constructed a series of plasmid DNA sequences which codes cytomegalovirus (CMV) enhancer, human U1 promoter, and modified mouse U7 snRNAs with antisense sequences of different lengths. We evaluated in vitro splicing modulation efficiency using a luciferase reporter system for simple and precise evaluation as well as reverse transcription-polymerase chain reaction to monitor splicing patterns. Our in vitro assay findings suggest that antisense sequences of modified mouse U7 snRNAs have an optimal length for efficient splicing modulation, which depends on the target exon. In addition, antisense sequences that were either too long or too short decreased splicing modulation efficiency. To confirm reproducibility, we performed an in vitro assay using two target genes, mouse Fas and mouse Dmd. Together, our data suggests that the antisense sequence length should be optimized for modified mouse U7 snRNAs to induce efficient splicing modulation.

A novel reconstruction method for displacement missing data of arch dam via hierarchical clustering and deep learning

The absence of displacement monitoring data presents a challenge to real-time dam safety monitoring. This article introduces a novel method for reconstructing missing displacements, comprehensively taking into account the spatiotemporal correlation and causal effect mechanism of displacement. Firstly, adaptive weighted derivative dynamic time warping (AWDDTW) and adaptive weighted dynamic time warping (AWDTW), in conjunction with clustering algorithms, are proposed to extract spatiotemporal correlation among dam displacements. Secondly, by stacking residual blocks within the residual network and densifying the shortcut connections between residual blocks, the deep stacked residual network (DS-ResNet) is established to effectively capture the intricate mappings between displacements and various factors. Finally, using two arch dams, as examples, we simulated scenarios of continuous long-term and multiple short-term missing data to validate the new method proposed in this study. The results indicate that the proposed clustering algorithm can accurately compute the similarity between displacement sequences of different lengths and monitoring frequencies, thereby providing more precise displacement point partitioning results. Compared to models such as random forests, relevance vector machines, multilayer perceptron, ResNet and Transformer, the DS-ResNet demonstrates superior capability in identifying and extracting effective information from displacement sequences under both continuous long-term and multiple short-term missing data scenarios. Compared to methods that solely consider spatiotemporal correlation or causal effect mechanism of displacement, the proposed method can more comprehensively reflect the operating patterns of arch dams and more accurately reconstruct missing displacement values. The research presented herein offers new technical means and solution approaches for dam safety monitoring and operational management.

Multi-tailed vision transformer for efficient inference

Recently, Vision Transformer (ViT) has achieved promising performance in image recognition and gradually serves as a powerful backbone in various vision tasks. To satisfy the sequential input of Transformer, the tail of ViT first splits each image into a sequence of visual tokens with a fixed length. Then, the following self-attention layers construct the global relationship between tokens to produce useful representation for the downstream tasks. Empirically, representing the image with more tokens leads to better performance, yet the quadratic computational complexity of self-attention layer to the number of tokens could seriously influence the efficiency of ViT’s inference. For computational reduction, a few pruning methods progressively prune uninformative tokens in the Transformer encoder, while leaving the number of tokens before the Transformer untouched. In fact, fewer tokens as the input for the Transformer encoder can directly reduce the following computational cost. In this spirit, we propose a Multi-Tailed Vision Transformer (MT-ViT) in the paper. MT-ViT adopts multiple tails to produce visual sequences of different lengths for the following Transformer encoder. A tail predictor is introduced to decide which tail is the most efficient for the image to produce accurate prediction. Both modules are optimized in an end-to-end fashion, with the Gumbel-Softmax trick. Experiments on ImageNet-1K demonstrate that MT-ViT can achieve a significant reduction on FLOPs with no degradation of the accuracy and outperform compared methods in both accuracy and FLOPs.

Learning about trial sequences disrupts the partial reinforcement extinction effect in classical conditioning.

The partial reinforcement extinction effect (PREE) refers to the phenomenon that conditioned responding extinguishes more slowly if subjects had been inconsistently ("partially") reinforced than if they had been reinforced on every trial ("continuously" reinforced). One largely successful account of the PREE, known as sequential theory (Capaldi, 1966), suggests that, when subjects are partially reinforced, they learn that memories of sequences of nonreinforced trials are associated with subsequent reinforcement. This association helps to maintain responding (i.e., delay extinction) when the subjects experience nonreinforced trials during extinction. Sequential theory's explanation of the PREE hinges on subjects learning sequences of nonreinforced trials during acquisition. However, direct evidence for such sequential learning is not available in previous studies of the PREE where animals are trained with multiple sequences of different lengths that are randomly intermixed and, therefore, cannot anticipate whether a given trial will be reinforced during acquisition. The current study conducted two experiments that trained rats with a single fixed trial sequence to provide evidence of sequential learning during conditioning, and then observe its effect on the PREE. Under one condition the rats did learn about the fixed sequence but did not subsequently show a PREE, whereas other rats that did show a PREE had not learned the trial sequences during conditioning. Therefore, contrary to sequential theory's prediction, our result suggests that learning about the trial sequence is neither necessary nor sufficient for the PREE. We suggest that the PREE may instead depend on uncertainty about whether the conditioned stimulus will be reinforced. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Antibody-directed evolution reveals a mechanism for enhanced neutralization at the HIV-1 fusion peptide site

The HIV-1 fusion peptide (FP) represents a promising vaccine target, but global FP sequence diversity among circulating strains has limited anti-FP antibodies to ~60% neutralization breadth. Here we evolve the FP-targeting antibody VRC34.01 in vitro to enhance FP-neutralization using site saturation mutagenesis and yeast display. Successive rounds of directed evolution by iterative selection of antibodies for binding to resistant HIV-1 strains establish a variant, VRC34.01_mm28, as a best-in-class antibody with 10-fold enhanced potency compared to the template antibody and ~80% breadth on a cross-clade 208-strain neutralization panel. Structural analyses demonstrate that the improved paratope expands the FP binding groove to accommodate diverse FP sequences of different lengths while also recognizing the HIV-1 Env backbone. These data reveal critical antibody features for enhanced neutralization breadth and potency against the FP site of vulnerability and accelerate clinical development of broad HIV-1 FP-targeting vaccines and therapeutics.

New Construction of Asynchronous Channel Hopping Sequences in Cognitive Radio Networks.

The channel-hopping-based rendezvous is essential to alleviate the problem of under-utilization and scarcity of the spectrum in cognitive radio networks. It dynamically allows unlicensed secondary users to schedule rendezvous channels using the assigned hopping sequence to guarantee the self-organization property in a limited time. In this paper, we use the interleaving technique to cleverly construct a set of asynchronous channel-hopping sequences consisting of d sequences of period xN2 with flexible parameters, which can generate sequences of different lengths. By this advantage, the new designed CHSs can be used to adapt to the demands of various communication scenarios. Furthermore, we focus on the improved maximum-time-to-rendezvous and maximum-first-time-to-rendezvous performance of the new construction compared to the prior research at the same sequence length. The new channel-hopping sequences ensure that rendezvous occurs between any two sequences and the rendezvous times are random and unpredictable when using licensed channels under asynchronous access, although the full degree-of-rendezvous is not satisfied. Our simulation results show that the new construction is more balanced and unpredictable between the maximum-time-to-rendezvous and the mean and variance of time-to-rendezvous.

Searching for protein variants with desired properties using deep generative models

BackgroundProtein engineering aims to improve the functional properties of existing proteins to meet people’s needs. Current deep learning-based models have captured evolutionary, functional, and biochemical features contained in amino acid sequences. However, the existing generative models need to be improved when capturing the relationship between amino acid sites on longer sequences. At the same time, the distribution of protein sequences in the homologous family has a specific positional relationship in the latent space. We want to use this relationship to search for new variants directly from the vicinity of better-performing varieties.ResultsTo improve the representation learning ability of the model for longer sequences and the similarity between the generated sequences and the original sequences, we propose a temporal variational autoencoder (T-VAE) model. T-VAE consists of an encoder and a decoder. The encoder expands the receptive field of neurons in the network structure by dilated causal convolution, thereby improving the encoding representation ability of longer sequences. The decoder decodes the sampled data into variants closely resembling the original sequence.ConclusionCompared to other models, the person correlation coefficient between the predicted values of protein fitness obtained by T-VAE and the truth values was higher, and the mean absolute deviation was lower. In addition, the T-VAE model has a better representation learning ability for longer sequences when comparing the encoding of protein sequences of different lengths. These results show that our model has more advantages in representation learning for longer sequences. To verify the model’s generative effect, we also calculate the sequence identity between the generated data and the input data. The sequence identity obtained by T-VAE improved by 12.9% compared to the baseline model.

Chunk-Level Speech Emotion Recognition: A General Framework of Sequence-to-One Dynamic Temporal Modeling

A critical issue of current speech-based sequence-to-one learning tasks, such as speech emotion recognition(SER), is the dynamic temporal modeling for speech sentences with different durations. The goal is to extract an informative representation vector of the sentence from acoustic feature sequences with varied length. Traditional methods rely on static descriptions such as statistical functions or a universal background model (UBM), which are not capable of characterizing dynamic temporal changes. Recent advances in deep learning architectures provide promising results, directly extracting sentence-level representations from frame-level features. However, conventional cropping and padding techniques that deal with varied length sequences are not optimal, since they truncate or artificially add sentence-level information. Therefore, we propose a novel dynamic chunking approach, which maps the original sequences of different lengths into a fixed number of chunks that have the same duration by adjusting their overlap. This simple chunking procedure creates a flexible framework that can incorporate different feature extractions and sentence-level temporal aggregation approaches to cope, in a principled way, with different sequence-to-one tasks. Our experimental results based on three databases demonstrate that the proposed framework provides: 1) improvement in recognition accuracy, 2) robustness toward different temporal length predictions, and 3) high model computational efficiency advantages.

MCWS-Transformers: Towards an Efficient Modeling of Protein Sequences via Multi Context-Window Based Scaled Self-Attention.

This paper advances the self-attention mechanism in the standard transformer network specific to the modeling of the protein sequences. We introduce a novel context-window based scaled self-attention mechanism for processing protein sequences that is based on the notion of (i) local context and (ii) large contextual pattern. Both notions are essential to building a good representation for protein sequences. The proposed context-window based scaled self-attention mechanism is further used to build the multi context-window based scaled (MCWS) transformer network for the protein function prediction task at the protein sub-sequence level. Overall, the proposed MCWS transformer network produced improved predictive performances, outperforming existing state-of-the-art approaches by substantial margins. With respect to the standard transformer network, the proposed network produced improvements in F1-score of +2.30% and +2.08% on the biological process (BP) and molecular function (MF) datasets, respectively. The corresponding improvements over the state-of-the-art ProtVecGen-Plus+ProtVecGen-Ensemble approach are +3.38% (BP) and +2.86% (MF). Equally important, robust performances were obtained across protein sequences of different lengths.

Designing multi-epitope vaccine against important colorectal cancer (CRC) associated pathogens based on immunoinformatics approach

BackgroundIt seems that several members of intestinal gut microbiota like Streptococcus bovis, Bacteroides fragilis, Helicobacter pylori, Fusobacterium nucleatum, Enterococcus faecalis, Escherichia coli, Peptostreptococcus anaerobius may be considered as the causative agents of Colorectal Cancer (CRC). The present study used bioinformatics and immunoinformatics approaches to design a potential epitope-based multi-epitope vaccine to prevent CRC with optimal population coverage.MethodsIn this study, ten amino acid sequences of CRC-related pathogens were retrieved from the NCBI database. Three ABCpred, BCPREDS and LBtope online servers were considered for B cells prediction and the IEDB server for T cells (CD4+ and CD8+) prediction. Then, validation, allergenicity, toxicity and physicochemical analysis of all sequences were performed using web servers. A total of three linkers, AAY, GPGPG, and KK were used to bind CTL, HTL and BCL epitopes, respectively. In addition, the final construct was subjected to disulfide engineering, molecular docking, immune simulation and codon adaptation to design an effective vaccine production strategy.ResultsA total of 19 sequences of different lengths for linear B-cell epitopes, 19 and 18 sequences were considered as epitopes of CD4+ T and CD8+ cells, respectively. The predicted epitopes were joined by appropriate linkers because they play an important role in producing an extended conformation and protein folding. The final multi-epitope construct and Toll-like receptor 4 (TLR4) were evaluated by molecular docking, which revealed stable and strong binding interactions. Immunity simulation of the vaccine showed significantly high levels of immunoglobulins, helper T cells, cytotoxic T cells and INF-γ.ConclusionFinally, the results showed that the designed multi-epitope vaccine could serve as an excellent prophylactic candidate against CRC-associated pathogens, but in vitro and animal studies are needed to justify our findings for its use as a possible preventive measure.

Short designed peptide unfolding human telomeric G-quadruplex: mimicking the helicase function

Human telomeric DNA can fold into G-quadruplex structures involving the interaction of four guanine bases in a square planar arrangement. The highly distinctive nature of quadruplex topologies suggests that they can act as novel therapeutic targets. In this study, we provide the evidence of human telomeric G4 destabilization in dilute and cell-mimicking molecular crowing conditions upon peptide binding. We have used three human telomeric sequences of different lengths. CD data showed that these sequences folded into anti-parallel G-quadruplex and CD intensity decreased significantly on increasing the peptide concentration. UV-thermal melting results showed significant decrease in hypochromicity due to formation of G4-peptide complex at 295 nm. Fluorescence data showed the quenching on titrating the peptide with human telomere G4. Electrophoretic mobility shift assay confirmed the unfolding of G4 structure. Cell viability was significantly reduced in the presence of QW5 peptide with IC50 values as 8.78 μM and 7.72 μM after 72 and 96 hours of incubation respectively. These results confirmed that QW5 peptide has an ability to bind and unfold to human telomeric G-quadruplex and hence might be the key modulator for targeting diseases having over-representation of G4 motifs and their destabilization will be helpful in increasing the efficiency of DNA replication, transcription or duplex reannealing. Communicated by Ramaswamy H. Sarma

Insertion/deletion hotspots in the Nsp2, Nsp3, S1, and ORF8 genes of SARS-related coronaviruses

The genome of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) contains many insertions/deletions (indels) from the genomes of other SARS-related coronaviruses. Some of the identified indels have recently reported to involve relatively long segments of 10–300 consecutive bases and with diverse RNA sequences around gaps between virus species, both of which are different characteristics from the classical shorter in-frame indels. These non-classical complex indels have been identified in non-structural protein 3 (Nsp3), the S1 domain of the spike (S), and open reading frame 8 (ORF8). To determine whether the occurrence of these non-classical indels in specific genomic regions is ubiquitous among broad species of SARS-related coronaviruses in different animal hosts, the present study compared SARS-related coronaviruses from humans (SARS-CoV and SARS-CoV-2), bats (RaTG13 and Rc-o319), and pangolins (GX-P4L), by performing multiple sequence alignment. As a result, indel hotspots with diverse RNA sequences of different lengths between the viruses were confirmed in the Nsp2 gene (approximately 2500–2600 base positions in the overall 29,900 bases), Nsp3 gene (approximately 3000–3300 and 3800–3900 base positions), N-terminal domain of the spike protein (21,500–22,500 base positions), and ORF8 gene (27,800–28,200 base positions). Abnormally high rate of point mutations and complex indels in these regions suggest that the occurrence of mutations in these hotspots may be selectively neutral or even benefit the survival of the viruses. The presence of such indel hotspots has not been reported in different human SARS-CoV-2 strains in the last 2 years, suggesting a lower rate of indels in human SARS-CoV-2. Future studies to elucidate the mechanisms enabling the frequent development of long and complex indels in specific genomic regions of SARS-related coronaviruses would offer deeper insights into the process of viral evolution.

Micromotives of Vote Switchers and Macrotransitions: The Case of the Immigration Issue in a Regional Earthquake Election in Germany 2018

Which issue-related motives underlie voters’ decision to switch parties at the polls? Do switchers stick to the newly chosen party, or do they oscillate in a short-term way at intermediate elections? Relying on the behavioral theory of elections, we assumed aspiration-based voting of boundedly rational voters. We elicited issue-related switch and stay motives in an open-ended survey question format to identify the individual dominant aspirational frame. We traced the respondents’ voting trajectories over three consecutive elections, including two state (2013 and 2018) elections in Bavaria (Germany) and one German federal election (2017). We focused on one of the most polarizing and salient issues in these elections, namely immigration. The case of reference is the 2018 Bavarian state election. Here, the incumbent majoritarian center-right party Christian Social Union tried to deter the entry of the right-wing populist party Alternative for Germany by adapting to it on the immigration issue in tone and position. The selected case allows assessment of the impact of issue-based adaptive behavior of the incumbent party at the level of the voters’ switch or stay choices. We estimated the direction and number of voter flows for two interelection sequences of different lengths between different types of polls (federal and state). Our transition estimates are based on the hybrid multinomial Dirichlet model, a new technique integrating individual-level survey data and official aggregate data. Our estimates uncover substantial behavioral differences in the immigration issue public.

Sequence addition to the N- or C-terminus of the major capsid protein VP1 of norovirus affects its cleavage and assembly into virus-like particles

Norovirus (NoV) infection is a leading cause of non-bacterial gastroenteritis worldwide and there are currently no effective therapeutics available to target the virus. The norovirus major capsid protein VP1 is a potential candidate for the development of vaccines due to the similar morphology and immunogenicity of its assembled virus-like particles (VLPs) compared to native virions. In this study, we explored the effects of N- and C-terminal sequence additions to the VP1 of a GII.4 NoV during its assembly into VLPs. A series of sequences of different lengths derived from the minor capsid protein VP2 of the GII.4 NoV were added to the N- and C-terminus of VP1. The fusion proteins were expressed using a recombinant baculovirus expression system and the assembly of the expressed fusion proteins was subsequently observed under electron microscopy (EM). Our results indicated that all constructed fusion proteins were successfully expressed with different degrees of enzyme cleavage at the N-terminus. Electron microscopy revealed the successful assembly of VLPs of different sizes for all fusion proteins. An in vitro binding assay for VLP-histo-blood group antigens (HBGAs) indicated that all fusion proteins exhibited similar binding patterns compared with their wild-type VP1. Our results demonstrate that (Xi et al., 1990) [1] NoV VP1 can tolerate foreign sequences at its N- or C-terminus without affecting its ability to assemble into VLPs, and (Jiang et al., 1992) [2] that the cleavage pattern and effects of foreign sequences on the sizes of assembled VLPs observed in this study might represent important experimental data that can be used to elucidate VP1 self-assembly.

Gene silencing of VP1 gene of coxsackievirus B3 neurotropic strain Nancy by dsRNAs and siRNAs

Coxsackieviruses are distributed worldwide and can cause serious diseases threatening human health and even human life. Genetic silencing is considered a new opportunity to treat viral infections. In this study, we produced in vitro dsRNAs and siRNAs specific for two sequences of different length from the VP1 genetic regions of CVB3, and introduced them into HEp-2 cells with Oligofectamine as the transfection agent. We used the neurotropic strain Nancy. The 100% inhibitory concentrations of the applied dsRNAs and siRNAs were 0.3 μmol/L and fell in the range of non-cytotoxic concentration values. The produced VP1-specific dsRNAs and siRNAs were effective at low concentrations, suggesting that the produced RNAs are safe to use and possess excellent potential to treat CVB3 infection.

Composition, hydrogen bonding and viscoelastic properties correlation for ethylene/α,ω-alkenol copolymers

The influence of composition of ethylene/ α,ω -alkenol ( α,ω -alkenol = 10-undecen-1-ol or 9-decen-1-ol) copolymers on comonomer distribution and dynamic rheological properties were investigated. Thermal fractionation revealed formation of multiple peaks corresponding to the presence of methylene sequences of different length in copolymers. The share of fraction with shorter methylene sequence increased with increasing comonomer content. It turned out that viscoelastic behavior of ethylene/ α,ω -alkenol copolymers was dependent on comonomer content incorporated into the polymer chain. The relaxation time of reptation increased with increasing comonomer content what confirmed that comonomer units acted as physically crosslinking agent. FT-IR investigation proved the appearance of inter- and intramolecular hydrogen bonds in ethylene copolymers with α,ω -alkenols. Higher relative content of inter- and intramolecular hydrogen bonds was observed in α,ω -alkenol with a longer hydrocarbon chain. In addition, activation energy of flow increased with increasing of the comonomer content. Moreover, incorporation of α,ω -alkenols into the polymer chain resulted in a decrease of the packing length of copolymers with increasing comonomer content, what proved that the inter- and intramolecular hydrogen bond played a significant role in viscoelastic properties. • Amount of α,ω -alkenol incorporated in copolymers affected comonomer distribution. • The content of α,ω -alkenol played a key role in rheological behavior of copolymers. • The FT-IR study showed the presence of hydrogen bonds in E/ α,ω -alkenol copolymers. • The effect of hydrogen bonding is dependent on structure of α,ω -alkenol comonomer.

LAS-Transformer: An Enhanced Transformer Based on the Local Attention Mechanism for Speech Recognition

Recently, Transformer-based models have shown promising results in automatic speech recognition (ASR), outperforming models based on recurrent neural networks (RNNs) and convolutional neural networks (CNNs). However, directly applying a Transformer to the ASR task does not exploit the correlation among speech frames effectively, leaving the model trapped in a sub-optimal solution. To this end, we propose a local attention Transformer model for speech recognition that combines the high correlation among speech frames. Specifically, we use relative positional embedding, rather than absolute positional embedding, to improve the generalization of the Transformer for speech sequences of different lengths. Secondly, we add local attention based on parametric positional relations to the self-attentive module and explicitly incorporate prior knowledge into the self-attentive module to make the training process insensitive to hyperparameters, thus improving the performance. Experiments carried out on the LibriSpeech dataset show that our proposed approach achieves a word error rate of 2.3/5.5% by language model fusion without any external data and reduces the word error rate by 17.8/9.8% compared to the baseline. The results are also close to, or better than, other state-of-the-art end-to-end models.