Parallel Decoding Research Articles

Theory and design of parallel decoding for a ternary optical computer.

A hardware-based parallel decoding scheme is proposed to address the problems of correctness and efficiency of software decoding for ternary optical computers. Based on the minimal primitive structure of the ternary optical computer, a hardware decoding voltage divider circuit and single-pixel transcoding of operation results are designed. A parallel decoding scheme is designed for the SJ-MSD unconventional adder based on Shen's theorem and the TW-MSD conventional adder under the degraded design theory, and a corresponding addressing scheme is proposed for the access of decoding results. After comprehensive consideration, the decoding scheme is finally selected as the time-sharing combination. The experiments show that the parallel decoding scheme of the ternary optical computer is practical and feasible.

Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning.

3D dense captioning requires a model to translate its understanding of an input 3D scene into several captions associated with different object regions. Existing methods adopt a sophisticated "detect-then-describe" pipeline, which builds explicit relation modules upon a 3D detector with numerous hand-crafted components. While these methods have achieved initial success, the cascade pipeline tends to accumulate errors because of duplicated and inaccurate box estimations and messy 3D scenes. In this paper, we first propose Vote2Cap-DETR, a simple-yet-effective transformer framework that decouples the decoding process of caption generation and object localization through parallel decoding. Moreover, we argue that object localization and description generation require different levels of scene understanding, which could be challenging for a shared set of queries to capture. To this end, we propose an advanced version, Vote2Cap-DETR++, which decouples the queries into localization and caption queries to capture task-specific features. Additionally, we introduce the iterative spatial refinement strategy to vote queries for faster convergence and better localization performance. We also insert additional spatial information to the caption head for more accurate descriptions. Without bells and whistles, extensive experiments on two commonly used datasets, ScanRefer and Nr3D, demonstrate Vote2Cap-DETR and Vote2Cap-DETR++ surpass conventional "detect-then-describe" methods by a large margin.

Expectation Propagation-Based Parallel Iterative Detection and Decoding for Massive MIMO

Expectation Propagation-Based Parallel Iterative Detection and Decoding for Massive MIMO

Transformer with a Parallel Decoder for Image Captioning

In this paper, a parallel decoder and a word group prediction module are proposed to speed up decoding and improve the effect of captions. The features of the image extracted by the encoder are linearly projected to different word groups, and then a unique relaxed mask matrix is designed to improve the decoding speed and the caption effect. First, since image captioning is composed of many words, sentences can also be broken down into word groups or words according to their syntactic structure, and we achieve this function through constituency parsing. Second, we make full use of the extracted features to predict the size of word groups. Then, a new embedding representing the information of the word is proposed based on word embedding. Finally, with the help of word groups, we design a mask matrix to modify the decoding process so that each step of the model can produce one or more words in parallel. Experiments on public datasets demonstrate that our method can reduce the time complexity while maintaining competitive performance.

Fast and High-Performance Learned Image Compression With Improved Checkerboard Context Model, Deformable Residual Module, and Knowledge Distillation.

Deep learning-based image compression has made great progresses recently. However, some leading schemes use serial context-adaptive entropy model to improve the rate-distortion (R-D) performance, which is very slow. In addition, the complexities of the encoding and decoding networks are quite high and not suitable for many practical applications. In this paper, we propose four techniques to balance the trade-off between the complexity and performance. We first introduce the deformable residual module to remove more redundancies in the input image, thereby enhancing compression performance. Second, we design an improved checkerboard context model with two separate distribution parameter estimation networks and different probability models, which enables parallel decoding without sacrificing the performance compared to the sequential context-adaptive model. Third, we develop a three-pass knowledge distillation scheme to retrain the decoder and entropy coding, and reduce the complexity of the core decoder network, which transfers both the final and intermediate results of the teacher network to the student network to improve its performance. Fourth, we introduce L1 regularization to make the numerical values of the latent representation more sparse, and we only encode non-zero channels in the encoding and decoding process to reduce the bit rate. This also reduces the encoding and decoding time. Experiments show that compared to the state-of-the-art learned image coding scheme, our method can be about 20 times faster in encoding and 70-90 times faster in decoding, and our R-D performance is also 2.3% higher. Our method achieves better rate-distortion performance than classical image codecs including H.266/VVC-intra (4:4:4) and some recent learned methods, as measured by both PSNR and MS-SSIM metrics on the Kodak and Tecnick-40 datasets.

Spatio-Temporal Convolutional Neural Network for Enhanced Inter Prediction in Video Coding.

This paper presents a convolutional neural network (CNN)-based enhancement to inter prediction in Versatile Video Coding (VVC). Our approach aims at improving the prediction signal of inter blocks with a residual CNN that incorporates spatial and temporal reference samples. It is motivated by the theoretical consideration that neural network-based methods have a higher degree of signal adaptivity than conventional signal processing methods and that spatially neighboring reference samples have the potential to improve the prediction signal by adapting it to the reconstructed signal in its immediate vicinity. We show that adding a polyphase decomposition stage to the CNN results in a significantly better trade-off between computational complexity and coding performance. Incorporating spatial reference samples in the inter prediction process is challenging: The fact that the input of the CNN for one block may depend on the output of the CNN for preceding blocks prohibits parallel processing. We solve this by introducing a novel signal plane that contains specifically constrained reference samples, enabling parallel decoding while maintaining a high compression efficiency. Overall, experimental results show average bit rate savings of 4.07% and 3.47% for the random access (RA) and low-delay B (LB) configurations of the JVET common test conditions, respectively.

EM-Trans: Edge-Aware Multimodal Transformer for RGB-D Salient Object Detection.

RGB-D salient object detection (SOD) has gained tremendous attention in recent years. In particular, transformer has been employed and shown great potential. However, existing transformer models usually overlook the vital edge information, which is a major issue restricting the further improvement of SOD accuracy. To this end, we propose a novel edge-aware RGB-D SOD transformer, called, which explicitly models the edge information in a dual-band decomposition framework. Specifically, we employ two parallel decoder networks to learn the high-frequency edge and low-frequency body features from the low-and high-level features extracted from a two-steam multimodal backbone network, respectively. Next, we propose a cross-attention complementarity exploration module to enrich the edge/body features by exploiting the multimodal complementarity information. The refined features are then fed into our proposed color-hint guided fusion module for enhancing the depth feature and fusing the multimodal features. Finally, the resulting features are fused using our deeply supervised progressive fusion module, which progressively integrates edge and body features for predicting saliency maps. Our model explicitly considers the edge information for accurate RGB-D SOD, overcoming the limitations of existing methods and effectively improving the performance. Extensive experiments on benchmark datasets demonstrate that is an effective RGB-D SOD framework that outperforms the current state-of-the-art models, both quantitatively and qualitatively. A further extension to RGB-T SOD demonstrates the promising potential of our model in various kinds of multimodal SOD tasks.

A tree-based model with branch parallel decoding for handwritten mathematical expression recognition

Handwritten mathematical expression recognition (HMER) is a challenging task in the field of computer vision due to the complex two-dimensional spatial structure and diverse handwriting styles of mathematical expressions (MEs). Recent mainstream approach treats MEs as objects with tree structures, modeled by sequence decoders or tree decoders. These decoders recognize the symbols and relationships between symbols in MEs in depth-first order, resulting in long decoding steps that can harm their performance, particularly for MEs with complex structures. In this paper, we propose a novel tree-based model with branch parallel decoding for HMER, which parses the structures of ME trees by explicitly predicting the relationships between symbols. In addition, a query constructing module is proposed to assist the decoder in decoding the branches of ME trees in parallel, thus reducing the number of decoding time steps and alleviating the problem of long sequence attention decoding. As a result, our model outperforms existing models on three widely-used benchmarks and demonstrates significant improvements in HMER performance.

Learned image compression via neighborhood-based attention optimization and context modeling with multi-scale guiding

In recent years, learned image compression has witnessed significant advancements. However, many existing learned image compression models predominantly rely on Convolutional Neural Networks (CNNs) and predicting the distribution of latent features to use ever more expensive entropy models, which cannot fully capture detail redundancy and lead to prohibitively slow decoding. To address this challenge, this paper introduces novel image compression models, MGIC and RMGIC, which consist of the Residual Neighborhood-based Attention Module (RNAM), Multi-Scale Guide Entropy Model (MGEM) and Refined Multi-Scale Guide Entropy Model (RMGEM). Specifically, Neighborhood Attention (NA) is combined with global feature learning to construct the RNAM, which can accurately capture the correlation of spatial neighborhoods and is more suitable for image compression. The proposed RNAM is flexible and could work as a plug-and-play component to enhance CNN-based models. Additionally, MGEM employs two mask convolutions with two kernel sizes to leverage multi-scale information for context modeling and eliminating the effect of parallel decoding on model performance. Building upon MGEM, RMGEM incorporates a refined two-stage spatial context modeling technique to enhance the utilization of spatial neighborhoods and achieve more precise entropy prediction. Furthermore, MGIC and RMGIC are derived by integrating RNAM, MGEM, and RMGEM into the variational autoencoder framework. The proposed methods can save more bandwidth and improve the quality of information storage. Extensive experimental evaluations have shown that the proposed method is effective and achieves state-of-the-art performance.

An efficient reconfigurable code rate cooperative low-density parity check codes for gigabits wide code encoder/decoder operations

In recent days, extensive digital communication process has been performed. Due to this phenomenon, a proper maintenance of authentication, communication without any overhead such as signal attenuation code rate fluctuations during digital communication process can be minimized and optimized by adopting parallel encoder and decoder operations. To overcome the above-mentioned drawbacks by using proposed reconfigurable code rate cooperative (RCRC) and low-density parity check (LDPC) method. The proposed RCRC-LDPC is capable to operate over gigabits/sec data and it effectively performs linear encoding, dual diagonal form, widens the range of code rate and optimal degree distribution of LDPC mother code. The proposed method optimize the transmission rate and it is capable to operate on 0.98 code rate. It is the highest upper bounded code rate as compared to the existing methods. The proposed method optimizes the transmission rate and is capable to operate on a 0.98 code rate. It is the highest upper bounded code rate as compared to the existing methods. the proposed method's implementation has been carried out using MATLAB and as per the simulation result, the proposed method is capable of reaching a throughput efficiency greater than 8.2 (1.9) gigabits per second with a clock frequency of 160 MHz.

Low-Complexity Parallel Min-Sum Medium-Density Parity-Check Decoder for McEliece Cryptosystem

Low-Complexity Parallel Min-Sum Medium-Density Parity-Check Decoder for McEliece Cryptosystem

Parallel window decoding enables scalable fault tolerant quantum computation

Large-scale quantum computers have the potential to hold computational capabilities beyond conventional computers. However, the physical qubits are prone to noise which must be corrected in order to perform fault-tolerant quantum computations. Quantum Error Correction (QEC) provides the path for realizing such computations. QEC generates a continuous stream of data that decoders must process at the rate it is received, which can be as fast as 1 μs per QEC round in superconducting quantum computers. If the decoder infrastructure cannot keep up, a data backlog problem is encountered and the computation runs exponentially slower. Today’s leading approaches to quantum error correction are not scalable as existing decoders typically run slower as the problem size is increased, inevitably hitting the backlog problem. Here, we show how to parallelize decoding to achieve almost arbitrary speed, removing this roadblock to scalability. Our parallelization requires some classical feed forward decisions to be delayed, slowing-down the logical clock speed. However, the slow-down is now only polynomial in the size of the QEC code, averting the exponential slowdown. We numerically demonstrate our parallel decoder for the surface code, showing no noticeable reduction in logical fidelity compared to previous decoders and demonstrating the predicted speedup.

ASIC design of LZ77 compressor for computational storage drives

AbstractThe advent of cloud computing, 5G, and artificial intelligence technology has spurred the exponential growth of big data and the need for data compression. To alleviate the CPU burden, computational engines for data compression integrated into network cards and storage devices, giving rise to computational storage. However, computational storage faces several challenges, such as limited memory resources for large hash tables, data dependency for parallel encoding and decoding, and stringent requirements for bandwidth, latency, and power efficiency. Here, an application‐specific integrated circuit‐based LZ77 solution that aims to address these challenges is proposed. Our solution reduces the hash table resource consumption and the bandwidth fluctuations, while achieving high compression ratios. The solution is implemented using Taiwan Semiconductor Manufacturing Company Limited 12 nm technology and demonstrate that a single LZ77 engine can achieve a 4 GB/s throughput at 1 GHz clock frequency. By using four engines, a computational storage drive to achieve 16 GB/s compression bandwidth is enabled. The solution achieves a comparable compression ratio as the standard LZ77 algorithm with a negligible compression ratio loss of 5%, balancing efficiency and effectiveness.

Ultrasound image segmentation based on Transformer and U-Net with joint loss

Background Ultrasound image segmentation is challenging due to the low signal-to-noise ratio and poor quality of ultrasound images. With deep learning advancements, convolutional neural networks (CNNs) have been widely used for ultrasound image segmentation. However, due to the intrinsic locality of convolutional operations and the varying shapes of segmentation objects, segmentation methods based on CNNs still face challenges with accuracy and generalization. In addition, Transformer is a network architecture with self-attention mechanisms that performs well in the field of computer vision. Based on the characteristics of Transformer and CNNs, we propose a hybrid architecture based on Transformer and U-Net with joint loss for ultrasound image segmentation, referred to as TU-Net. Methods TU-Net is based on the encoder-decoder architecture and includes encoder, parallel attention mechanism and decoder modules. The encoder module is responsible for reducing dimensions and capturing different levels of feature information from ultrasound images; the parallel attention mechanism is responsible for capturing global and multiscale local feature information; and the decoder module is responsible for gradually recovering dimensions and delineating the boundaries of the segmentation target. Additionally, we adopt joint loss to optimize learning and improve segmentation accuracy. We use experiments on datasets of two types of ultrasound images to verify the proposed architecture. We use the Dice scores, precision, recall, Hausdorff distance (HD) and average symmetric surface distance (ASD) as evaluation metrics for segmentation performance. Results For the brachia plexus and fetal head ultrasound image datasets, TU-Net achieves mean Dice scores of 79.59% and 97.94%; precisions of 81.25% and 98.18%; recalls of 80.19% and 97.72%; HDs (mm) of 12.44 and 6.93; and ASDs (mm) of 4.29 and 2.97, respectively. Compared with those of the other six segmentation algorithms, the mean values of TU-Net increased by approximately 3.41%, 2.62%, 3.74%, 36.40% and 31.96% for the Dice score, precision, recall, HD and ASD, respectively.

BEA-Net: Body and Edge Aware Network with Multi-Scale Short-Term Concatenation for Medical Image Segmentation.

Medical image segmentation is indispensable for diagnosis and prognosis of many diseases. To improve the segmentation performance, this study proposes a new 2D body and edge aware network with multi-scale short-term concatenation for medical image segmentation. Multi-scale short-term concatenation modules which concatenate successive convolution layers with different receptive fields, are proposed for capturing multi-scale representations with fewer parameters. Body generation modules with feature adjustment based on weight map computing via enlarging the receptive fields, and edge generation modules with multi-scale convolutions using Sobel kernels for edge detection, are proposed to separately learn body and edge features from convolutional features in decoders, making the proposed network be body and edge aware. Based on the body and edge modules, we design parallel body and edge decoders whose outputs are fused to achieve the final segmentation. Besides, deep supervision from the body and edge decoders is applied to ensure the effectiveness of the generated body and edge features and further improve the final segmentation. The proposed method is trained and evaluated on six public medical image segmentation datasets to show its effectiveness and generality. Experimental results show that the proposed method achieves better average Dice similarity coefficient and 95% Hausdorff distance than several benchmarks on all used datasets. Ablation studies validate the effectiveness of the proposed multi-scale representation learning modules, body and edge generation modules and deep supervision. The code is available at https://github.com/hulinkuang/BEA-Net.

DDHCN: Dual decoder Hyperformer convolutional network for Downstream-Adaptable user representation learning on app usage

In mobile scenarios, there is a need for general user representations to solve multiple target tasks. However, there are some challenges in the related research (e.g., difficulty in learning a representation that satisfies both great generalization and performance). To address these problems, we proposed a network for downstream-adaptable mobile user modeling, which employed a novel fine-tuning strategy for optimizing the performance of several downstream tasks. Additionally, we designed a time-difference module to eliminate the impact of low-frequency and non-uniform app usage behavior over time. A parallel decoder structure was developed to incorporate multi-type features by minimizing information loss. We evaluated our method on a real-world dataset of 100,000 mobile users and three downstream tasks (i.e., age prediction, gender prediction, and app recommendation). The experimental results showed that our method could outperform existing methods significantly. It achieved 96.5% ACC on gender prediction, 68.1% ACC on age prediction, and 64.2% Recall@5 on app recommendation. These results imply that our method performs well on both generalization and performance. It could be anticipated promising to the unseen tasks inference.

Design and Implementation of a Highly Efficient Quasi-Cyclic Low-Density Parity-Check Transceiving System Using an Overlapping Decoder.

The traditional LDPC encoding and decoding system is characterized by low throughput and high resource consumption, making it unsuitable for use in cost-efficient, energy-saving sensor networks. Aiming to optimize coding complexity and throughput, this paper proposes a combined design of a novel LDPC code structure and the corresponding overlapping decoding strategies. With regard to structure of LDPC code, a CCSDS-like quasi-cyclic parity check matrix (PCM) with uniform distribution of submatrices is constructed to maximize overlap depth and adapt the parallel decoding. In terms of reception decoding strategies, we use a modified 2-bit Min-Sum algorithm (MSA) that achieves a coding gain of 5 dB at a bit error rate of 10-6 compared to an uncoded BPSK, further mitigating resource consumption, and which only incurs a slight loss compared to the standard MSA. Moreover, a shift-register-based memory scheduling strategy is presented to fully utilize the quasi-cyclic characteristic and shorten the read/write latency. With proper overlap scheduling, the time consumption can be reduced by one third per iteration compared to the non-overlap algorithm. Simulation and implementation results demonstrate that our decoder can achieve a throughput up to 7.76 Gbps at a frequency of 156.25 MHz operating eight iterations, with a two-thirds resource consumption saving.

Si-MSPDNet: A multiscale Siamese network with parallel partial decoders for the 3-D measurement of spines in 3D ultrasonic images.

Early screening and frequent monitoring effectively decrease the risk of severe scoliosis, but radiation exposure is a consequence of traditional radiograph examinations. Additionally, traditional X-ray images on the coronal or sagittal plane have difficulty providing three-dimensional (3-D) information on spinal deformities. The Scolioscan system provides an innovative 3-D spine imaging approach via ultrasonic scanning, and its feasibility has been demonstrated in numerous studies. In this paper, to further examine the potential of spinal ultrasonic data for describing 3-D spinal deformities, we propose a novel deep-learning tracker named Si-MSPDNet for extracting widely employed landmarks (spinous process (SP)) in ultrasonic images of spines and establish a 3-D spinal profile to measure 3-D spinal deformities. Si-MSPDNet has a Siamese architecture. First, we employ two efficient two-stage encoders to extract features from the uncropped ultrasonic image and the patch centered on the SP cut from the image. Then, a fusion block is designed to strengthen the communication between encoded features and further refine them from channel and spatial perspectives. The SP is a very small target in ultrasonic images, so its representation is weak in the highest-level feature maps. To overcome this, we ignore the highest-level feature maps and introduce parallel partial decoders to localize the SP. The correlation evaluation in the traditional Siamese network is also expanded to multiple scales to enhance cooperation. Furthermore, we propose a binary guided mask based on vertebral anatomical prior knowledge, which can further improve the performance of our tracker by highlighting the potential region with SP. The binary-guided mask is also utilized for fully automatic initialization in tracking. We collected spinal ultrasonic data and corresponding radiographs on the coronal and sagittal planes from 150 patients to evaluate the tracking precision of Si-MSPDNet and the performance of the generated 3-D spinal profile. Experimental results revealed that our tracker achieved a tracking success rate of 100% and a mean IoU of 0.882, outperforming some commonly used tracking and real-time detection models. Furthermore, a high correlation existed on both the coronal and sagittal planes between our projected spinal curve and that extracted from the spinal annotation in X-ray images. The correlation between the tracking results of the SP and their ground truths on other projected planes was also satisfactory. More importantly, the difference in mean curvatures was slight on all projected planes between tracking results and ground truths. Thus, this study effectively demonstrates the promising potential of our 3-D spinal profile extraction method for the 3-D measurement of spinal deformities using 3-D ultrasound data.

Design and implementation of virtual-single-length turbo decoder for multi-user parallel decoding

Dealing with multiple users in cellular basestation with multiple codewords of variable length is a challenging task in parallel decoding. Thus, a novel turbo decoder with variable codeword-based parallel decoding is proposed for the 3GPP LTE (4G) system. To make multi-user decoding more efficient, all users are aggregated into a single block with zero insertions where all users can be decoded in the same cycle. Different users are separated by additional zeros to decode in parallel and improve the hardware utilization. Moreover, a stationary boundary enhancement is utilized to exchange and update the decoded messages between different sub-blocks of codeword for the next decoding interaction. This scheme can balance the performance loss and decoding latency of variable codeword-based parallel decoding. The proposed turbo decoder has a maximum throughput of 280.5 Mbps (VC707) and 212 Mbps (ZC706), where the design has been integrated and tested with OpenAirInterface (OAI) 4G eNB in campus trials.

Multi-information interaction graph neural network for joint entity and relation extraction

Overlap situation where different triplets share entities or relations is a common challenge in joint entity and relation extraction task. On the one hand, there is strong correlation between overlapping triplets. On the other hand, most of the existing large-scale training data come from distant supervision, which introduces incomplete annotations. These practical problems make the information interaction between triplets particularly important. However, there are two problems with the existing methods: (i) the neglect of information interaction between different triplets; (ii) the limited information utilization caused by the specific decoding order. To solve the above problems, we decompose decoding and information interaction. Specifically, entity and relation proposals are obtained by a proposal generator, then a multi-information interaction graph neural network with parallel decoder is proposed to complete the joint extraction task. In this way, the inherent decoding order is broken to achieve the purpose of fully exploiting multi-information interaction across triplets and within triplets. Experimental results show that our proposed model outperforms previous work, especially in the case of incomplete annotations.