Sequential Decoding Research Articles

On Sequential Decoding Metric Function of Polarization-Adjusted Convolutional (PAC) Codes

In this paper, we present a sequential decoding metric function, which leads to significantly improved computational complexity while maintaining the superiority of polarization-adjusted convolutional (PAC) codes’ error-correction performance. With the proposed metric function, the PAC codes’ decoding computational complexity is comparable to the computational complexity of sequential decoding of conventional convolutional codes (CCs). Moreover, simulation results show an improvement in the error-correction performance of low rate PAC codes when using the proposed metric function. Simulation results also show that using the proposed metric, the upper bound on the PAC codes’ computational complexity has a Pareto distribution. To reduce the worst-case latency of PAC sequential decoder, we limit the number of searches performed by sequential decoder. The results show that for PAC codes of length 128, search-limited sequential decoding can achieve an error-correction performance close to the error-correction performance of polar codes with successive cancellation list decoding with list size 64 and CRC length 11 with considerably less computational complexity.

Sequential Decoding of Short Length Binary Codes: Performance Versus Complexity

Sequential decoding of short length binary codes for the additive white Gaussian noise channel is considered. A variant of the variable-bias term (VBT) metric is introduced, producing useful trade-offs between performance and computational complexity. Comparisons are made with tail-biting convolutional codes decoded with a wrap-around Viterbi algorithm (WAVA) and with polar codes under successive-cancellation list (SCL) decoding. It is found that sequential decoding with the improved VBT metric has a better performance–complexity tradeoff than tail-biting codes under WAVA decoding (except at low complexities) but a worse performance–complexity tradeoff than polar codes under SCL decoding (except at high complexities).

A Bayesian Receiver With Improved Complexity-Reliability Trade-Off in Massive MIMO Systems

The stringent requirements on reliability and processing delay in the fifth-generation ($5$G) cellular networks introduce considerable challenges in the design of massive multiple-input-multiple-output (M-MIMO) receivers. The two main components of an M-MIMO receiver are a detector and a decoder. To improve the trade-off between reliability and complexity, a Bayesian concept has been considered as a promising approach that enhances classical detectors, e.g. minimum-mean-square-error detector. This work proposes an iterative M-MIMO detector based on a Bayesian framework, a parallel interference cancellation scheme, and a decision statistics combining concept. We then develop a high performance M-MIMO receiver, integrating the proposed detector with a low complexity sequential decoding for polar codes. Simulation results of the proposed detector show a significant performance gain compared to other low complexity detectors. Furthermore, the proposed M-MIMO receiver with sequential decoding ensures one order magnitude lower complexity compared to a receiver with stack successive cancellation decoding for polar codes from the 5G New Radio standard.

Dual attentive graph neural network for metro passenger flow prediction

Metro system has been increasingly recognized as a backbone of urban transportation system in many cities around the world. To improve the demand management and operation efficiency, it is crucial to have accurate prediction of real-time metro passenger flow. However, the forecast performance is often subject to the complex spatial and temporal distributions of the metro passenger flow data. To this end, we developed a novel dual attentive graph neural network that can effectively predict the distribution of metro traffic flow considering the spatial and temporal influences. Specifically, two directed complete metro graphs (i.e., inbound and outbound graphs) and the weighted matrix of them are proposed to characterize the inbound (entering the system) and outbound (leaving the system) passenger flow, respectively. The weighted matrix of inbound graph is estimated based on the historical origin-destination demand and that of the outbound graph is estimated based on the similarity metrics between every two stations. Moreover, to capture the dependencies between inbound and outbound flows, multi-layer graph spatial attention networks that incorporate the spatial context are applied to exploit the dynamic inter-station correlations. Then, the acquired dependency features integrated with external factors, such as weather conditions, are filtered by temporal attention and fed into a sequence decoder to produce short-term and long-term passenger flow predictions. Finally, a series experiments are conducted based on a comprehensive empirical dataset. Findings indicated that the proposed model does not only well predict the metro passenger flow, but also effectively detect the emergencies and incidents of metro system.

Construction of signature codes for multiple light sources-based visible light positioning system

A received signal strength-based visible light positioning system with multiple light sources is investigated. In this system, each light source is assigned a unique signature code, which is constructed based on the vertex coloring of graph theory. The signature code is separated into three parts for different purposes, i.e., discriminating the beginning of the received signals, distinguishing different light sources, and correcting errors when they occur. The length of the code is determined by the numbers of the light sources. A sequential decoding scheme is devised to identify different light sources stepwisely. The light source with the highest received light intensity is discerned first by utilizing the characteristic of the signature codes. Then it is reduced from the received signals, and the remaining light sources are discriminated one by one using this scheme. With the knowledge of the coordination of these light sources, the location of the receiver can be acquired based on trilateration positioning algorithm. The simulation and the experiment results verify that the proposed signature codes can eliminate the interference from multiple sources and achieve good positioning performance.

List Decoding of Arıkan's PAC Codes.

Polar coding gives rise to the first explicit family of codes that provably achieve capacity with efficient encoding and decoding for a wide range of channels. However, its performance at short blocklengths under standard successive cancellation decoding is far from optimal. A well-known way to improve the performance of polar codes at short blocklengths is CRC precoding followed by successive-cancellation list decoding. This approach, along with various refinements thereof, has largely remained the state of the art in polar coding since it was introduced in 2011. Recently, Arıkan presented a new polar coding scheme, which he called polarization-adjusted convolutional (PAC) codes. At short blocklengths, such codes offer a dramatic improvement in performance as compared to CRC-aided list decoding of conventional polar codes. PAC codes are based primarily upon the following main ideas: replacing CRC codes with convolutional precoding (under appropriate rate profiling) and replacing list decoding by sequential decoding. One of our primary goals in this paper is to answer the following question: is sequential decoding essential for the superior performance of PAC codes? We show that similar performance can be achieved using list decoding when the list size L is moderately large (say, ). List decoding has distinct advantages over sequential decoding in certain scenarios, such as low-SNR regimes or situations where the worst-case complexity/latency is the primary constraint. Another objective is to provide some insights into the remarkable performance of PAC codes. We first observe that both sequential decoding and list decoding of PAC codes closely match ML decoding thereof. We then estimate the number of low weight codewords in PAC codes, and use these estimates to approximate the union bound on their performance. These results indicate that PAC codes are superior to both polar codes and Reed–Muller codes. We also consider random time-varying convolutional precoding for PAC codes, and observe that this scheme achieves the same superior performance with constraint length as low as .

Belief Propagation Decoding of Short Graph-Based Channel Codes via Reinforcement Learning

In this work, we consider the decoding of short sparse graph-based channel codes via reinforcement learning (RL). Specifically, we focus on low-density parity-check (LDPC) codes, which for example have been standardized in the context of 5G cellular communication systems due to their excellent error correcting performance. LDPC codes are typically decoded via belief propagation on the corresponding bipartite (Tanner) graph of the code via flooding, i.e., all check and variable nodes in the Tanner graph are updated at once. We model the node-wise sequential LDPC scheduling scheme as a Markov decision process (MDP), and obtain optimized check node (CN) scheduling policies via RL to improve sequential decoding performance as compared to flooding. In each RL step, an agent decides which CN to schedule next by observing a reward associated with each choice. Repeated scheduling enables the agent to discover the optimized CN scheduling policy which is later incorporated in our RL-based sequential LDPC decoder. In order to reduce RL complexity, we propose a novel graph-induced CN clustering approach to partition the state space of the MDP in such a way that dependencies between clusters are minimized. Compared to standard decoding approaches from the literature, some of our proposed RL schemes not only improve the decoding performance, but also reduce the decoding complexity dramatically once the scheduling policy is learned. By concatenating an outer Hamming code with an inner LDPC code which is decoded based on our learned policy, we demonstrate significant improvements in the decoding performance compared to other LDPC decoding policies.

Short-Term Load Forecasting Using an Attended Sequential Encoder-Stacked Decoder Model with Online Training

The paper presents a new approach for the prediction of load active power 24 h ahead using an attended sequential encoder and stacked decoder model with Long Short-Term Memory cells. The load data are owned by the New York Independent System Operator (NYISO) and is dated from the years 2014–2017. Due to dynamics in the load patterns, multiple short pieces of training on pre-filtered data are executed in combination with the transfer learning concept. The evaluation is done by direct comparison with the results of the NYISO forecast and additionally under consideration of several benchmark methods. The results in terms of the Mean Absolute Percentage Error range from 1.5% for the highly loaded New York City zone to 3% for the Mohawk Valley zone with rather small load consumption. The execution time of a day ahead forecast including the training on a personal computer without GPU accounts to 10 s on average.

DASZL: Dynamic Action Signatures for Zero-shot Learning

There are many realistic applications of activity recognition where the set of potential activity descriptions is combinatorially large. This makes end-to-end supervised training of a recognition system impractical as no training set is practically able to encompass the entire label set. In this paper, we present an approach to fine-grained recognition that models activities as compositions of dynamic action signatures. This compositional approach allows us to reframe fine-grained recognition as zero-shot activity recognition, where a detector is composed "on the fly" from simple first-principles state machines supported by deep-learned components. We evaluate our method on the Olympic Sports and UCF101 datasets, where our model establishes a new state of the art under multiple experimental paradigms. We also extend this method to form a unique framework for zero-shot joint segmentation and classification of activities in video and demonstrate the first results in zero- shot decoding of complex action sequences on a widely-used surgical dataset. Lastly, we show that we can use off-the-shelf object detectors to recognize activities in completely de-novo settings with no additional training.

Time Series Anomaly Detection with Multiresolution Ensemble Decoding

Recurrent autoencoder is a popular model for time series anomaly detection, in which outliers or abnormal segments are identified by their high reconstruction errors. However, existing recurrent autoencoders can easily suffer from overfitting and error accumulation due to sequential decoding. In this paper, we propose a simple yet efficient recurrent network ensemble called Recurrent Autoencoder with Multiresolution Ensemble Decoding (RAMED). By using decoders with different decoding lengths and a new coarse-to-fine fusion mechanism, lower-resolution information can help long-range decoding for decoders with higher-resolution outputs. A multiresolution shape-forcing loss is further introduced to encourage decoders' outputs at multiple resolutions to match the input's global temporal shape. Finally, the output from the decoder with the highest resolution is used to obtain an anomaly score at each time step. Extensive empirical studies on real-world benchmark data sets demonstrate that the proposed RAMED model outperforms recent strong baselines on time series anomaly detection.

MANGO: A Mask Attention Guided One-Stage Scene Text Spotter

Recently end-to-end scene text spotting has become a popular research topic due to its advantages of global optimization and high maintainability in real applications. Most methods attempt to develop various region of interest (RoI) operations to concatenate the detection part and the sequence recognition part into a two-stage text spotting framework. However, in such framework, the recognition part is highly sensitive to the detected results (e.g., the compactness of text contours). To address this problem, in this paper, we propose a novel Mask AttentioN Guided One-stage text spotting framework named MANGO, in which character sequences can be directly recognized without RoI operation. Concretely, a position-aware mask attention module is developed to generate attention weights on each text instance and its characters. It allows different text instances in an image to be allocated on different feature map channels which are further grouped as a batch of instance features. Finally, a lightweight sequence decoder is applied to generate the character sequences. It is worth noting that MANGO inherently adapts to arbitrary-shaped text spotting and can be trained end-to-end with only coarse position information (e.g., rectangular bounding box) and text annotations. Experimental results show that the proposed method achieves competitive and even new state-of-the-art performance on both regular and irregular text spotting benchmarks, i.e., ICDAR 2013, ICDAR 2015, Total-Text, and SCUT-CTW1500.

Уточнение видовой принадлежности некоторых видов щелкунов (Coleoptera: Elateridae) на основе молекулярно-генетического анализа

The article is devoted to data on the approbation of modern methods and the refinement of species affiliation of soil-inhabiting larvae of click beetles (wireworm) based on the molecular genetic analysis (DNAbarcoding). For the first time, data obtained on the complete identity of certain DNA sequences of a number of species of click beetles living in the Kostanay Region (Kazakhstan) — dangerous pests of agricultural crops. At the same time, the World genetic bank (GenBank) did not find identical DNA sequences of decoded DNA nucleotide sequences for a number of studied specimens. The basis for this study was the materials collected in 2018 in the subzone of ordinary black earth on sandy loam soils (Mendykarinsky district). The selection of larvae was carried out by the method of standard soil-zoological samples. The fixation and storage of the selected click beetles larvae was carried out according to the method of preparing samples for molecular genetic analysis with fixation in 96 % alcohol. After the classic identification of the taxonomic position of the collected specimens, the species were identified by genetic analysis on the nucleotide sequence of the cytochrome C oxidase I subunit gene (COI). The assembly and decoding of the DNA nucleotide sequences of the studied samples were carried out using the programs «Codon Code Aligner» and «MEGA-X». As a result of the work carried out in the DNA laboratory of the Museum of Natural History (University of Oslo, Norway), it was possible to identify the complete identity of the DNA sequences of several mass species of click beetles, while a number of decoded DNA sequences of model specimens were absent in the genetic bank, which requires replenishment in it with new data.

Interactive Sequential Basket Recommendation by Learning Basket Couplings and Positive/Negative Feedback

Sequential recommendation , such as next-basket recommender systems (NBRS), which model users’ sequential behaviors and the relevant context/session, has recently attracted much attention from the research community. Existing session-based NBRS involve session representation and inter-basket relations but ignore their hybrid couplings with the intra-basket items, often producing irrelevant or similar items in the next basket. In addition, they do not predict next-baskets (more than one next basket recommended). Interactive recommendation further involves user feedback on the recommended basket. The existing work on next-item recommendation involves positive feedback on selected items but ignores negative feedback on unselected ones. Here, we introduce a new setting— interactive sequential basket recommendation , which iteratively predicts next baskets by learning the intra-/inter-basket couplings between items and both positive and negative user feedback on recommended baskets. A hierarchical attentive encoder-decoder model (HAEM) continuously recommends next baskets one after another during sequential interactions with users after analyzing the item relations both within a basket and between adjacent sequential baskets (i.e., intra-/inter-basket couplings) and incorporating the user selection and unselection (i.e., positive/negative) feedback on the recommended baskets to refine NBRS. HAEM comprises a basket encoder and a sequence decoder to model intra-/inter-basket couplings and a prediction decoder to sequentially predict next-baskets by interactive feedback-based refinement. Empirical analysis shows that HAEM significantly outperforms the state-of-the-art baselines for NBRS and session-based recommenders for accurate and novel recommendation. We also show the effect of continuously refining sequential basket recommendation by including unselection feedback during interactive recommendation.

Relaxed Decoding of Polar Codes With Large Kernels

A simplified decoding method for polar codes with large kernels is presented. The proposed approach consists in decoder-side substitution of some submatrices of the polarizing transformation with matrices, which admit simpler evaluation of log-likelihood ratios. This approach enables complexity reduction for the successive cancellation, list and sequential decoding algorithms.

WITHDRAWN: Performance evaluation of the sequential neural network decoder with improved bit error rate

Abstract Due to the dimensionality of the neural network channel code decoder's training complexity increases exponentially with the length of the code term. While the computational capacity has advanced considerably, it is still difficult to manage the word code for long blocks. In this work, we proposed a decoder based on a neural network called the Sequential Neural Decoder (SNND). The SNND consists of several Sub-Models, and transfers the final state of the present Sub-Model as initial state to the following model. The output of the SNND's bit error rate (BER) continues unchanged as submodels increase, and the performance of Viterbi soft decisions in the white additive Gaussian noise (AWGN) channel is near. The output of the SNND, however has decreased along with the increased order of modulation.

Polarization-Adjusted Convolutional (PAC) Codes: Sequential Decoding vs List Decoding

In the Shannon lecture at the 2019 International Symposium on Information Theory (ISIT), Arikan proposed to employ a one-to-one convolutional transform as a pre-coding step before the polar transform. The resulting codes of this concatenation are called polarization-adjusted convolutional (PAC) codes . In this scheme, a pair of polar mapper and demapper as pre- and post- processing devices are deployed around a memoryless channel, which provides polarized information to an outer decoder leading to improved error correction performance of the outer code. In this paper, the list decoding and sequential decoding (including Fano decoding and stack decoding) are first adapted for use to decode PAC codes. Then, to reduce the complexity of sequential decoding of PAC/polar codes, we propose (i) an adaptive heuristic metric, (ii) tree search constraints for backtracking to avoid exploration of unlikely sub-paths, and (iii) tree search strategies consistent with the pattern of error occurrence in polar codes. These contribute to the reduction of the average decoding time complexity from 50% to 80%, trading with 0.05 to 0.3 dB degradation in error correction performance within FER = $10^{-3}$ range, respectively, relative to not applying the corresponding search strategies. Additionally, as an important ingredient in Fano decoding of PAC/polar codes, an efficient computation method for the intermediate LLRs and partial sums is provided. This method is effective in backtracking and avoids storing the intermediate information or restarting the decoding process. Eventually, all three decoding algorithms are compared in terms of performance, complexity, and resource requirements.

Learning boosts the decoding of sound sequences in rat auditory cortex.

Continuous acoustic streams, such as speech signals, can be chunked into segments containing reoccurring patterns (e.g., words). Noninvasive recordings of neural activity in humans suggest that chunking is underpinned by low-frequency cortical entrainment to the segment presentation rate, and modulated by prior segment experience (e.g., words belonging to a familiar language). Interestingly, previous studies suggest that also primates and rodents may be able to chunk acoustic streams. Here, we test whether neural activity in the rat auditory cortex is modulated by previous segment experience. We recorded subdural responses using electrocorticography (ECoG) from the auditory cortex of 11 anesthetized rats. Prior to recording, four rats were trained to detect familiar triplets of acoustic stimuli (artificial syllables), three were passively exposed to the triplets, while another four rats had no training experience. While low-frequency neural activity peaks were observed at the syllable level, no triplet-rate peaks were observed. Notably, in trained rats (but not in passively exposed and naïve rats), familiar triplets could be decoded more accurately than unfamiliar triplets based on neural activity in the auditory cortex. These results suggest that rats process acoustic sequences, and that their cortical activity is modulated by the training experience even under subsequent anesthesia.

Priority queue architecture for sequential decoder of polar codes for implementation in hardware

Priority queue architecture for sequential decoder of polar codes for implementation in hardware

Adaptive Adapters: An Efficient Way to Incorporate BERT Into Neural Machine Translation

Large-scale pre-trained language models (e.g., BERT) have attracted great attention in recent years. It is straightforward to fine-tune them on natural language understanding tasks such as text classification, however, effectively and efficiently incorporating them into natural language generation tasks such as neural machine translation remains a challenging problem. In this paper, we integrate two pre-trained BERT models from the source and target language domains into a sequence-to-sequence model by introducing light-weight adapter modules. The adapters are inserted between BERT layers and tuned on downstream tasks, while the parameters of BERT models are fixed during fine-tuning. As pre-trained language models are usually very deep, inserting adapters into all layers will result in a considerable scale of new parameters. To deal with this problem, we introduce latent variables to decide whether using adapters or not in each layer, which are learned during fine-tuning. In this way, the model is able to automatically determine which adapters to use, therefore hugely promoting the parameter efficiency and decoding speed. We evaluate the proposed framework on various neural machine translation tasks. Equipped with parallel sequence decoding, our model consistently outperforms autoregressive baselines while reducing the inference latency by half. With automatic adapter selection, the proposed model further achieves 20% speedup while still outperforming autoregressive baselines. When applied to autoregressive decoding, the proposed model can also achieve comparable performance with the state-of-the-art baseline models.

Frame analysis of stylistic devices in literary text interpretation

The purpose of the article is to substantiate the thesis that the use of cognitive methodsof analyzing stylistic devices, along with traditional ones, is the most effective approach inthe process of interpreting literary texts. Until recently, the study of stylistic techniques wasconducted in accordance with the traditional style, which not always led to the coverage of all theirfunctions and aspects. The processes of understanding a text cannot be limited only to the study ofthe surface structures of language units and require the involvement of cognitive operations andactivation of deep knowledge structures. The article presents a number of examples of analyzingmetaphors from literary texts that combine the methods of component, contextual analysis withcognitive frame analysis. Frame analysis defines the knowledge structures embedded in stylistictechniques, the sequential decoding of which significantly expands the integral conceptualmeaning of a literary work and more accurately reveals the author’s intent. This approach allowsus to get into the deep semantics of stylistic units and reveal their conceptual meanings morewidely. Consequently, by engaging in cognitive actions, the highest degree of understanding ofthe text is achieved in the process of its interpretation.