Symbol Nodes Research Articles

The Syndrome Bit Flipping Algorithm for LDPC Codes

Performance of LDPC decoders at high SNR is dominated by trapping sets that induce an error floor in the performance curve. We propose a new algorithm that resolves trapping sets and lowers the error floor. The new algorithm, called Syndrome Bit Flipping (SBF), computes the sum of adjacent parity violations at each symbol node. Bits are flipped by comparing the syndrome sum against a time-varying threshold called the decoding key. SBF is compared to other bit-flipping decoders on the Binary Symmetric Channel (BSC), and is demonstrated as a post-processing step for a Noisy Gradient Descent Bit-Flipping (NGDBF) hardware decoder. We demonstrate the post-processing method for an LDPC code defined in the 802.3an standard, and find that the frame error rate is improved by at least two orders of magnitude, even as the required iterations are reduced by 33%.

Knowledge-Based Visual Question Answering Using Multi-Modal Semantic Graph

The field of visual question answering (VQA) has seen a growing trend of integrating external knowledge sources to improve performance. However, owing to the potential incompleteness of external knowledge sources and the inherent mismatch between different forms of data, current knowledge-based visual question answering (KBVQA) techniques are still confronted with the challenge of effectively integrating and utilizing multiple heterogeneous data. To address this issue, a novel approach centered on a multi-modal semantic graph (MSG) is proposed. The MSG serves as a mechanism for effectively unifying the representation of heterogeneous data and diverse types of knowledge. Additionally, a multi-modal semantic graph knowledge reasoning model (MSG-KRM) is introduced to perform reasoning and deep fusion of image–text information and external knowledge sources. The development of the semantic graph involves extracting keywords from the image object detection information, question text, and external knowledge texts, which are then represented as symbol nodes. Three types of semantic graphs are then constructed based on the knowledge graph, including vision, question, and the external knowledge text, with non-symbol nodes added to connect these three independent graphs and marked with respective node and edge types. During the inference stage, the multi-modal semantic graph and image–text information are embedded into the feature semantic graph through three embedding methods, and a type-aware graph attention module is employed for deep reasoning. The final answer prediction is a blend of the output from the pre-trained model, graph pooling results, and the characteristics of non-symbolic nodes. The experimental results on the OK-VQA dataset show that the MSG-KRM model is superior to existing methods in terms of overall accuracy score, achieving a score of 43.58, and with improved accuracy for most subclass questions, proving the effectiveness of the proposed method.

SRD: A Tree Structure Based Decoder for Online Handwritten Mathematical Expression Recognition

Recently, recognition of online handwritten mathe- matical expression has been greatly improved by employing encoder-decoder based methods. Existing encoder-decoder models use string decoders to generate LaTeX strings for mathematical expression recognition. However, in this paper, we importantly argue that string representations might not be the most natural for mathematical expressions – mathematical expressions are inherently tree structures other than flat strings. For this purpose, we propose a novel sequential relation decoder (SRD) that aims to decode expressions into tree structures for online handwritten mathematical expression recognition. At each step of tree construction, a sub-tree structure composed of a relation node and two symbol nodes is computed based on previous sub-tree structures. This is the first work that builds a tree structure based decoder for encoder-decoder based mathematical expression recognition. Compared with string decoders, a decoder that better understands tree structures is crucial for mathematical expression recognition as it brings a more reasonable learning objective and improves overall generalization ability. We demonstrate how the proposed SRD outperforms state-of-the-art string decoders through a set of experiments on CROHME database, which is currently the largest benchmark for online handwritten mathematical expression recognition.

Layered Decoding Algorithm and Two-Level Quasi-Cyclic Matrix Construction for Rate Compatible Modulation

Rate-compatible modulation (RCM) achieves seamless rate adaptation under time-varying channels by varying the number of transmitted symbols at a small granularity. The implementation of effective decoders based on the belief propagation (BP) algorithm suffers from the varying number of symbols and highly complex symbol nodes. Considering these RCM-specific characteristics, we present a top–down design methodology, including decoding algorithm, mapping matrix construction, and hardware implementation. The layered BP algorithm provides the potential to implement the decoders that fine-tune the decoding throughput according to the number of symbols. To mitigate the impact of inter-layer data dependency on decoding throughput and facilitate the implementation of low parallelism decoders, a construction and optimization algorithm is proposed to obtain the two-level quasi-cyclic matrices with small sub-matrix size. The proposed algorithm and matrix are applied to a partially parallel RCM decoder for a ${5544 \times 5544}$ matrix on a field-programmable gate-array (FPGA) device. A memory access structure that requires only one barrel shifter is used to improve area efficiency. The post-route results show that the decoder achieves a throughput of 377 M ~ 1100 Mbit/s at six iterations with a clock frequency of 218 MHz.

Approximate Message Passing Detector for Index Modulation With Multiple Active Resources

In this correspondence paper, we study the application of approximate message passing (AMP) algorithm in the signal detection of index modulation (IM) with multiple active resources (antennas or subcarriers). First, for uplink generalized spatial modulation (GSM), AMP detectors in both spatially uncorrelated and correlated channels are proposed. Specifically, an effective method to utilize the activation pattern constraint of GSM signal vector per user in the posterior probability computation of scalar symbol node is derived. Furthermore, a simple while effective damping strategy is introduced in AMP detector to guarantee convergence in correlated channels. Second, for orthogonal frequency division multiplexing with IM in rapidly time-varying channels, the damped AMP detector is further studied. In addition, the convergence of proposed detectors is analyzed by state evolution. Finally, the effectiveness of proposed AMP detectors is verified by simulations.

Low Complexity Decoding Algorithms for Rate Compatible Modulation

Rate compatible modulation (RCM) has high spectrum efficiency and achieves seamless and blind rate adaptation in wide range of channel conditions. However, due to many convolution operations at symbol nodes, the belief propagation decoding algorithm of RCM has a high level of computational complexity. In this paper, we investigate the low complexity algorithms for fast decoding of RCM. Instead of computing the outgoing messages at symbol nodes via multi-level convolutions, we first design a novel two-level computing structure (2L-RCM) for symbol nodes in the probability-domain, each level is composed of one set of multiplications followed by one set of additions. Based on 2L-RCM, we derive Log-2L-RCM decoding algorithm in the log-domain, which converts the multiplications and additions of 2L-RCM into additions and Jacobian logarithms, respectively. Furthermore, we propose some approximate algorithms to reduce the complexity of Jacobian logarithms. In particular, the improved Max-Log-RCM (IMax-Log-RCM) algorithm obtains good performance-complexity trade-off. The simulation results and the numerical analyses show that IMax-Log-RCM achieves only 0.3-dB worse decoding performance than the original decoding algorithm with much fewer additions.

A Parallel Belief Propagation Decoding Algorithm for Rate Compatible Modulation

This letter proposes a parallel belief propagation algorithm for fast iterative decoding of rate compatible modulation. In this proposed algorithm, the processing at symbol node is partitioned into several low complexity sub-processors. All sub-processors have similar structures and work in parallel without cross connections among them. Numerical and simulation results show that proper partition results in significant reduction of the computational complexity without decoding performance degradation.

Graphics processing unit-accelerated joint-bitplane belief propagation algorithm in DSC

The M-ary source with nonstationary correlation can be encoded with a single binary low-density parity-check (LDPC) code and decoded together in distributed source coding. The joint-bitplane belief propagation (JBBP) is a useful decoding algorithm for multiple bitplanes of an M-ary source. However, it suffers from the drawbacks of low computational efficiency and long execution time. Motivated by the evolution of the Graphics Processing Unit (GPU) and the inherent parallel characteristic of the JBBP, we propose a novel approach for the computationally intensive processing of the JBBP algorithm on GPU using the compute unified device architecture programming model. Two different parallel modes are utilized for the belief passing between different nodes of the JBBP. It is found that the bottlenecks of the JBBP lie in computing the overall probability mass functions (pmfs) of symbol nodes and the overall beliefs of bit nodes. Thus, a data partitioning method is leveraged to split a large array of pmfs into small pieces which can be loaded into L1 cache instead of global memory. The optimal block size is selected which not only assigns as large L1 cache as possible for individual thread, but also guarantees multiple active warps in each stream multiprocessor. Experimental results show that when the length-6336 (length-50,688, resp.) LDPC accumulate (LDPCA) code is used to compress the source, the JBBP decoder can achieve about 20$$\times $$× (41$$\times $$×, resp.) speedup on GPU compared with the original C code on CPU. Better performance would be further obtained with longer LDPCA codes. Moreover, the parallel JBBP is also applied in hyperspectral image compression and video coding and it shows good speedup performance.

Decoding LDPC Codes With Locally Maximum-Likelihood Binary Messages

A new low-complexity message passing algorithm is described for decoding low-density parity-check (LDPC) codes by exchanging binary messages. The algorithm computes the local maximum-likelihood binary message (LMLBM) at each symbol node, given the combination of local channel information and partial syndrome components from adjacent parity check nodes. When channel information is quantized, the locally ML messages are pre-computed and stored in a dynamic global lookup table. The proposed algorithm uses memoryless extrinsic messages so that density evolution thresholds can be directly computed. Thresholds are obtained for regular ensembles, predicting good performance on quantized binary-input additive white Gaussian noise (biAWGN) channels.

Bit-Clusters Trellis Search Based Iterative MIMO Detection Algorithm

A bit-clusters trellis-search (BCTS) based soft-input soft-output detection algorithm is proposed for iterative Multiple-Input Multiple-Output (MIMO) receivers. Compared to the symbol nodes trellis-search (SNTS) based MIMO detection algorithm, which significantly reduces the search complexity by limiting the number of candidates in each trellis stage compared to sphere decoding algorithm, the BCTS algorithm proposed in this article further reduces the number of candidates in the trellis. In the BCTS algorithm, we replace the symbol node in SNTS algorithm with bit-cluster that represents several relative symbols whose corresponding bit is 0 or 1 while the other bits are not concerned. Because the bit-cluster number is fewer than the symbol node number, the search complexity of proposed BCTS algorithm is much less than that of the SNTS algorithm. The simulation results show, by transforming symbol nodes to bit-clusters in trellis-search detection, the performance is improved nearly 0.5 dB when the bit-error-ratio reaches $$10^{-4}$$10-4; the complexity analysis shows the number of partial-Euclidian-distance (PED) comparisons and PED computations are approximately three and one times fewer than that of the SNTS algorithm respectively when constellation size is beyond 16. And this tendency will be more obvious with the transmit antenna number and constellation size increasing.

Efficient Progressive Edge-Growth Algorithm Based on Chinese Remainder Theorem

Progressive edge-growth (PEG) algorithm construction builds a Tanner graph, or equivalently a parity-check matrix, for an LDPC code by establishing edges between the symbol nodes and the check nodes in an edge-by-edge manner and maximizing the girth in a greedy fashion. This approach is simple but the complexity of the PEG algorithm scale is O(nm), where n is the number of symbol nodes and m is the number of check nodes. We deal with this problem by construct a base matrix H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sub> of size m <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sub> × n <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sub> with the PEG algorithm and simultaneously expand this base matrix into a parity-check matrix H of size mx n via the the Chinese remainder theorem (CRT), where m ≫ m <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sub> and n ≥ n <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sub> . The size of the base matrix is expanded without decreasing the girth. For convenience, the PEG and CRT combined algorithm is referred to as the PEG-CRT algorithm in this paper. Since a smaller matrix is constructed with the PEG algorithm and the complexity of the CRT computation is negligible compared to the PEG algorithm, the complexity of the whole code construction process is reduced. Furthermore, the proposed algorithm has a potential advantage of saving storage space by storing a smaller matrix H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sub> and expanding it to H "on-the-fly" in hardware. The expanded matrix H preserves the important properties of base matrix such as large girth, flexible code rate and low density. The complexity analysis shows that the complexity of the PEG-CRT algorithm does not grow with the code length n. Simulation results show that compared with the PEG LDPC codes of length n <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sub> , the expanded PEG-CRT LDPC codes have better bit error rate (BER) performance with the iterative decoding. It is also shown that compared with PEG LDPC codes of length n, which constructed with higher complexities, the PEG-CRT codes have similar BER performance.

Improved progressive edge-growth algorithm for fast encodable LDPC codes

The progressive edge-growth (PEG) algorithm is known to construct low-density parity-check (LDPC) codes at finite code lengths with large girths by establishing edges between symbol and check nodes in an edge-by-edge manner. The linear-encoding PEG (LPEG) algorithm, a simple variation of the PEG algorithm, can be applied to generate linear time encodable LDPC codes whose m parity bits p1, p2, ..., p m are computed recursively in m steps. In this article, we propose modifications of the LPEG algorithm to construct LDPC codes whose number of encoding steps is independent of the code length. The maximum degree of the symbol nodes in the Tanner graph is denoted by d s max ; The m parity bits of the proposed LDPC codes are divided into d s max subgroups and can be computed in only d s max steps. Since d s max ≪ m , the number of encoding steps can be significantly reduced. It has also been proved that the PEG codes and the codes proposed in this article have similar lower bound on girth. Simulation results showed that the proposed codes perform very well over the AWGN channel with an iterative decoding.

Increasing the local girth of irregular low-density parity-check codes based on degree-spectrum analysis

A low-density parity-check (LDPC) code can be described by a Tanner graph composed of symbol nodes and check nodes. Short cycles in the Tanner graph should be avoided because such cycles will degrade the performance of the decoder and hence increase the error rate. In most of the circumstances, cycles of length 4 can be totally eliminated in the Tanner graph when the LDPC code has been designed properly. Locally, for each variable node in the Tanner graph, the girth may take the value of 6 or 8, or even higher. In this study, the authors attempt to increase the local girths of a Tanner graph with an aim to improving the error performance of the code. The authors propose and analyse the ‘degree spectrum’ of the symbol nodes connecting to each of the check nodes. Based on the analysis, the authors can determine if the local girths can be further increased. The authors also propose simple ways to construct codes with higher local girths. Finally, the authors show the characteristics and the error performance of the codes constructed with the proposed methods.

Improved Construction of Irregular Progressive Edge-Growth Tanner Graphs

The progressive edge-growth algorithm is a well-known procedure to construct regular and irregular low-density parity-check codes. In this paper, we propose a modification of the original algorithm that improves the performance of these codes in the waterfall region when constructing codes complying with both, check and symbol node degree distributions. The proposed algorithm is thus interesting if a family of irregular codes with a complex check node degree distribution is used.

A Method for Grouping Symbol Nodes of Group Shuffled BP Decoding Algorithm

In this paper, we propose a method for enhancing performance of a sequential version of the belief-propagation (BP) decoding algorithm, the group shuffled BP decoding algorithm for low-density parity-check (LDPC) codes. An improved BP decoding algorithm, called the shuffled BP decoding algorithm, decodes each symbol node in serial at each iteration. To reduce the decoding delay of the shuffled BP decoding algorithm, the group shuffled BP decoding algorithm divides all symbol nodes into several groups. In contrast to the original group shuffled BP, which automatically generates groups according to symbol positions, in this paper we propose a method for grouping symbol nodes which generates groups according to the structure of a Tanner graph of the codes. The proposed method can accelerate the convergence of the group shuffled BP algorithm and obtain a lower error rate in a small number of iterations. We show by simulation results that the decoding performance of the proposed method is improved compared with those of the shuffled BP decoding algorithm and the group shuffled BP decoding algorithm.

Augmented Belief Propagation Decoding of Low-Density Parity Check Codes

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> We propose an augmented belief propagation (BP) decoder for low-density parity check (LDPC) codes which can be utilized on memoryless or intersymbol interference channels. The proposed method is a heuristic algorithm that eliminates a large number of pseudocodewords that can cause nonconvergence in the BP decoder. The augmented decoder is a multistage iterative decoder, where, at each stage, the original channel messages on select symbol nodes are replaced by saturated messages. The key element of the proposed method is the symbol selection process, which is based on the appropriately defined subgraphs of the code graph and/or the reliability of the information received from the channel. We demonstrate by examples that this decoder can be implemented to achieve substantial gains (compared to the standard locally-operating BP decoder) for short LDPC codes decoded on both memoryless and intersymbol interference Gaussian channels. Using the Margulis code example, we also show that the augmented decoder reduces the error floors. Finally, we discuss types of BP decoding errors and relate them to the augmented BP decoder. </para>

A Measure of the Girth Histogram for Improved Girth-Conditioning Construction of LDPC Codes

In this letter, we present a general quantitative measure of the girth histogram of low-density parity-check (LDPC) codes with joint consideration of the degrees and local girths of symbol nodes. We then propose a further girth-conditioning algorithm to optimize such a measure during the construction of LDPC codes. We show by simulation results that the algorithm improves not only the performance of irregular codes at high signal-to-noise ratios (SNR) considerably, but also the rates of regular codes

Augmented Belief-Propagation Decoding of Low-Density Parity-Check Codes

We propose an augmented belief-propagation (BP) decoder for low-density parity-check (LDPC) codes which can be used on memoryless or intersymbol-interference (ISI) channels. The proposed method is a heuristic algorithm that eliminates a large number of pseudocodewords that can cause nonconvergence in the BP decoder. The augmented decoder is a multistage iterative decoder, where at each stage, the original channel messages on select symbol nodes are replaced by saturated messages. The key element of the proposed method is the symbol-selection process, which is based on the appropriately defined subgraphs of the code graph and/or the reliability of the information received from the channel. We demonstrate by examples that this decoder can be implemented to achieve substantial gains (compared with the standard locally operating BP decoder) for short LDPC codes decoded on both memoryless and ISI Gaussian channels. Using the Margulis code example, we also show that the augmented decoder reduces the error floors. Finally, we discuss types of BP decoding errors and relate them to the augmented BP decoder.

Regular and irregular progressive edge-growth tanner graphs

We propose a general method for constructing Tanner graphs having a large girth by establishing edges or connections between symbol and check nodes in an edge-by-edge manner, called progressive edge-growth (PEG) algorithm. Lower bounds on the girth of PEG Tanner graphs and on the minimum distance of the resulting low-density parity-check (LDPC) codes are derived in terms of parameters of the graphs. Simple variations of the PEG algorithm can also be applied to generate linear-time encodeable LDPC codes. Regular and irregular LDPC codes using PEG Tanner graphs and allowing symbol nodes to take values over GF(q) (q>2) are investigated. Simulation results show that the PEG algorithm is a powerful algorithm to generate good short-block-length LDPC codes.

Performance Limits of Compound Codes with Symbol-Based Iterative Decoding

Abstract We study the performance limit of concatenated codes based on binary constituent codes, under symbol-based iterative decoding. Although the results exposed herein are valid for any compound code built from both block and convolutional codes, we mainly focus on parallel turbo codes based on binary convolutional codes. The performance limit is found by evaluating the probability density function of the APP decoder output via a general propagation formula governing the density law through iterations. We propose also a second method to estimate the signal-to-noise ratio limit where the a priori density is assumed to be gaussian distributed. The results show that a gain in performance is achieved when bit nodes are combined into quaternary symbol nodes. For example, with a rate 1/2 Turbo code and a rate 2/3 binary RSC constituent, the minimal attainable signal-to-noise ratio per bit is 0.31 dB which is 0.13 dB away from the capacity of the AWGN channel with BPSK modulation.