Quantization Parameter Research Articles

Rate-Distortion Modeling for Bit Rate Constrained Point Cloud Compression

As being one of the main representation formats of 3D real world and well-suited for virtual reality and augmented reality applications, point clouds have gained a lot of popularity. In order to reduce the huge amount of data, a considerable amount of research on point cloud compression has been done. However, given a target bit rate, how to properly choose the color and geometry quantization parameters for compressing point clouds is still an open issue. In this paper, we propose a rate-distortion model based quantization parameter selection scheme for bit rate constrained point cloud compression. Firstly, to overcome the measurement uncertainty in evaluating the distortion of the point clouds, we propose a unified model to combine the geometry distortion and color distortion. In this model, we take into account the correlation between geometry and color variables of point clouds and derive a dimensionless quantity to represent the overall quality degradation. Then, we derive the relationships of overall distortion and bit rate with the quantization parameters. Finally, we formulate the bit rate constrained point cloud compression as a constrained minimization problem using the derived polynomial models and deduce the solution via an iterative numerical method. Experimental results show that the proposed algorithm can achieve optimal decoded point cloud quality at various target bit rates, and substantially outperform the video-rate-distortion model based point cloud compression scheme.

A Quality-of-Experience-Aware Framework for Versatile Video Coding-Based Video Transmission

Multimedia applications have expanded and resulted in huge demands for high resolution videos in order to enhance the end user Quality of Experience (QoE). Therefore, QoE becomes a critical factor that is often addressed in design considerations of Consumer Electronics (CE) devices. Furthermore, error prone transmission conditions can influence the QoE, potentially degrading the end-user multimedia experience. Although video coding standards such as Versatile Video Coding (VVC) take potential mitigation measures to improve the bandwidth reductions, it fails to address other significant network Quality of Service (QoS) parameters that impact transmission based QoE such as transmission delay and Packet Loss Rate (PLR) in the encoding processes. In response, this paper first derives a relationship between QoE that addresses the network QoS parameters and VVC dependent parameters. Subsequently, the derived QoE model is used with the VVC encoding process to adaptively optimize Lagrange parameters as well as the quantization parameter. Extensive experiments report significant improvements to VVC Test Model (VTM) 7.0, achieving superior QoE gains over the state-of-the-art research and up to 3.41% over the anchor implementations.

Visual Perception Based Intra Coding Algorithm for H.266/VVC

The latest international video coding standard, H.266/Versatile Video Coding (VVC), supports high-definition videos, with resolutions from 4 K to 8 K or even larger. It offers a higher compression ratio than its predecessor, H.265/High Efficiency Video Coding (HEVC). In addition to the quadtree partition structure of H.265/HEVC, the nested multi-type tree (MTT) structure of H.266/VVC provides more diverse splits through binary and ternary trees. It also includes many new coding tools, which tremendously increases the encoding complexity. This paper proposes a fast intra coding algorithm for H.266/VVC based on visual perception analysis. The algorithm applies the factor of average background luminance for just-noticeable-distortion to identify the visually distinguishable (VD) pixels within a coding unit (CU). We propose calculating the variances of the numbers of VD pixels in various MTT splits of a CU. Intra sub-partitions and matrix weighted intra prediction are turned off conditionally based on the variance of the four variances for MTT splits and a thresholding criterion. The fast horizontal/vertical splitting decisions for binary and ternary trees are proposed by utilizing random forest classifiers of machine learning techniques, which use the information of VD pixels and the quantization parameter. Experimental results show that the proposed algorithm achieves around 47.26% encoding time reduction with a Bjøntegaard Delta Bitrate (BDBR) of 1.535% on average under the All Intra configuration. Overall, this algorithm can significantly speed up H.266/VVC intra coding and outperform previous studies.

The Hardware Impact of Quantization and Pruning for Weights in Spiking Neural Networks

Energy efficient implementations and deployments of Spiking neural networks (SNNs) have been of great interest due to the possibility of developing artificial systems that can achieve the computational powers and energy efficiency of the biological brain. Efficient implementations of SNNs on modern digital hardware are also inspired by advances in machine learning and deep neural networks (DNNs). Two techniques widely employed in the efficient deployment of DNNs – the quantization and pruning of parameters, can both compress the model size, reduce memory footprints, and facilitate low-latency execution. The interaction between quantization and pruning and how they might impact model performance on SNN accelerators is currently unknown. We study various combinations of pruning and quantization in isolation, cumulatively, and simultaneously (jointly) to a state-of-the-art SNN targeting gesture recognition for dynamic vision sensor cameras (DVS). We show that this state-of-the-art model is amenable to aggressive parameter quantization, not suffering from any loss in accuracy down to ternary weights. However, pruning only maintains iso-accuracy up to 80% sparsity, which results in 45% more energy than the best quantization on our architectural model. Applying both pruning and quantization can result in an accuracy loss to offer a favourable trade-off on the energy-accuracy Pareto-frontier for the given hardware configuration.1

Fault-Tolerant Control for Unmanned Marine Vehicles via Quantized Integral Sliding Mode Output Feedback Technique

This paper presents a fault-tolerant control method based on the integral sliding mode (ISM) output feedback technique for the dynamic positioning control of an unmanned marine vehicle (UMV) subject to signal quantization and thruster faults. By using output information and estimates of thruster effectiveness factors, a novel integral sliding manifold is first constructed to maintain that the sliding function of system output equals the one of quantized output, which removes the region restriction that exists in the literature. Then based on the quantized signals of system output, a compensator is designed to introduce some degrees of design flexibility. To conquer the quantization effects, we incorporate a dynamic quantization parameter adjustment strategy and the fault-tolerant technique into the ISM output feedback control algorithm. The relation between the attraction area and quantization range has been revealed for the first time. Finally, the closed-loop stability can be guaranteed from every beginning time in despite of the quantization and thruster faults. Simulation through a typical floating production ship has verified the effectiveness of the proposed method.

A Highly Pipelined and Highly Parallel VLSI Architecture of CABAC Encoder for UHDTV Applications.

Recently, specifically designed video codecs have been preferred due to the expansion of video data in Internet of Things (IoT) devices. Context Adaptive Binary Arithmetic Coding (CABAC) is the entropy coding module widely used in recent video coding standards such as HEVC/H.265 and VVC/H.266. CABAC is a well known throughput bottleneck due to its strong data dependencies. Because the required context model of the current bin often depends on the results of the previous bin, the context model cannot be prefetched early enough and then results in pipeline stalls. To solve this problem, we propose a prediction-based context model prefetching strategy, effectively eliminating the clock consumption of the contextual model for accessing data in memory. Moreover, we offer multi-result context model update (MCMU) to reduce the critical path delay of context model updates in multi-bin/clock architecture. Furthermore, we apply pre-range update and pre-renormalize techniques to reduce the multiplex BAE's route delay due to the incomplete reliance on the encoding process. Moreover, to further speed up the processing, we propose to process four regular and several bypass bins in parallel with a variable bypass bin incorporation (VBBI) technique. Finally, a quad-loop cache is developed to improve the compatibility of data interactions between the entropy encoder and other video encoder modules. As a result, the pipeline architecture based on the context model prefetching strategy can remove up to 45.66% of the coding time due to stalls of the regular bin, and the parallel architecture can also save 29.25% of the coding time due to model update on average under the condition that the Quantization Parameter (QP) is equal to 22. At the same time, the throughput of our proposed parallel architecture can reach 2191 Mbin/s, which is sufficient to meet the requirements of 8 K Ultra High Definition Television (UHDTV). Additionally, the hardware efficiency (Mbins/s per k gates) of the proposed architecture is higher than that of existing advanced pipeline and parallel architectures.

Dynamic quantized control for switched fuzzy singularly perturbation systems with event-triggered protocol

This paper is concerned with the dynamic quantized control for switched fuzzy systems with singular perturbation and an improved event-triggered protocol. Essentially apart from the transition probabilities, the nonhomogeneous sojourn probabilities are employed to characterize the dynamic behavior of switched fuzzy singularly perturbed systems based on a deterministic switching signal. Benefiting from the dynamic quantization parameter, the quantization-based event-triggered protocol is presented, thereby decreasing the communication load. Based on the hidden Markov model, a novel event-triggered asynchronous control law is built. Finally, two examples are shown to clarify the practicality of the obtained results.

Quantiser-based Hands-off control for robust [𝒦, 𝒦ℒ] sector

This article discusses a quantiser-based stabilisation problem for uncertain nonlinear systems with the aid of robust [ K , KL ] sector. We confirmed that the quantised-feedback control steers the state trajectories of the uncertain nonlinear systems within the robust [ K , KL ] sector in finite time by adjusting the quantisation parameters. Furthermore, control-Lyapunov functions are helpful to obtain sufficient conditions for ensuring the closed-loop system's asymptotic stability and the reachability of the robust [ K , KL ] sector. Finally, a nonlinear electronic circuit is provided to test the effectiveness of the designed control law.

Pruning- and Quantization-Based Compression Algorithm for Number of Mixed Signals Identification Network

Source number estimation plays an important role in successful blind signal separation. At present, the application of machine learning allows the processing of signals without the time-consuming and complex work of manual feature extraction. However, the convolutional neural network (CNN) for processing complex signals has some problems, such as incomplete feature extraction and high resource consumption. In this paper, a lightweight source number estimation network (LSNEN), which can achieve a robust estimation of the number of mixed complex signals at low SNR (signal-to-noise ratio), is studied. Compared with other estimation methods, which require manual feature extraction, our network can realize the extraction of the depth feature of the original signal data. The convolutional neural network realizes complex mapping of modulated signals through the cascade of multiple three-dimensional convolutional modules. By using a three-dimensional convolution module, the mapping of complex signal convolution is realized. In order to deploy the network in the mobile terminal with limited resources, we further propose a compression method for the network. Firstly, the sparse structure network is obtained by the weight pruning method to accelerate the speed of network reasoning. Then, the weights and activation values of the network are quantified at a fixed point with the method of parameter quantization. Finally, a lightweight network for source number estimation was obtained, which was compressed from 12.92 MB to 3.78 MB with a compression rate of 70.74%, while achieving an accuracy of 94.4%. Compared with other estimation methods, the lightweight source number estimation network method proposed in this paper has higher accuracy, less model space occupation, and can realize the deployment of the mobile terminal.

Topological Recursion and Uncoupled BPS Structures II: Voros Symbols and the tau -Function

We continue our study of the correspondence between BPS structures and topological recursion in the uncoupled case, this time from the viewpoint of quantum curves. For spectral curves of hypergeometric type, we show the Borel-resummed Voros symbols of the corresponding quantum curves solve Bridgeland’s “BPS Riemann–Hilbert problem”. In particular, they satisfy the required jump property in agreement with the generalized definition of BPS indices Omega in our previous work. Furthermore, we observe the Voros coefficients define a closed one-form on the parameter space, and show that (log of) Bridgeland’s tau -function encoding the solution is none other than the corresponding potential, up to a constant. When the quantization parameter is set to a special value, this agrees with the Borel sum of the topological recursion partition function Z_textrm{TR}, up to a simple factor.

Pre-encoding based temporal dependent rate–distortion optimization for HEVC

Rate–distortion optimization (RDO) is a crucial technique in block-based hybrid video encoders, which determines the coding option for a certain coding unit to achieve optimal rate–distortion (R–D) performance. However, the current RDO method, which is applied in High Efficiency Video Coding (HEVC), is far from being globally optimal as it ignores the interaction among coding options. In fact, the current coding option not only determines the R–D point of the coded unit but also influences the achievable R–D​ performance of the subsequent coding units due to the spatial–temporal dependencies among the block/frame optimizations. In this paper, we first investigate the R–D characteristics, including operational R–D curves and R–D dependencies, in HEVC through several simulation experiments. It is shown that under the hierarchical fixed QP setting in the HEVC encoder, the temporal R–D dependence dominates the coding process, and the distortion of the reference frame has a strong impact on that of coded frames while less influence on rate. We then introduce distortion effect factors to quantitatively measure the temporal R–D​ dependency. The distortion effect factor at frame level reflects the correlation degree between the current coding frame and the previous coded frames, and the one at block level reflects the influence degree that the distortion of a block to that of subsequent blocks which refer to it in temporal domain. Finally, a pre-encoding based temporal dependent RDO is proposed, which adaptively adjusts quantization parameters and Lagrange multipliers according to the distortion effect factors. Experimental results show that under configurations of low-delay B and P frames, the proposed method can achieve 5.4% and 5.5% bit rate savings on average, respectively, with around 7% encoding complexity increment, compared with the original HEVC encoder.

Model Compression for Deep Neural Networks: A Survey

Currently, with the rapid development of deep learning, deep neural networks (DNNs) have been widely applied in various computer vision tasks. However, in the pursuit of performance, advanced DNN models have become more complex, which has led to a large memory footprint and high computation demands. As a result, the models are difficult to apply in real time. To address these issues, model compression has become a focus of research. Furthermore, model compression techniques play an important role in deploying models on edge devices. This study analyzed various model compression methods to assist researchers in reducing device storage space, speeding up model inference, reducing model complexity and training costs, and improving model deployment. Hence, this paper summarized the state-of-the-art techniques for model compression, including model pruning, parameter quantization, low-rank decomposition, knowledge distillation, and lightweight model design. In addition, this paper discusses research challenges and directions for future work.

Precise Encoding Complexity Control for Versatile Video Coding

Complexity reduction is a commonly used method to deal with complicated video coding standards, such as High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC). But the unstable performance under different video contents and Quantization Parameters (QPs) makes it difficult to precisely specify the target encoding time of every single sequence, which limits the practical use of the encoder. Inspired by rate control, in this paper, encoding time budget is regarded as a resource. We incorporate the hierarchical Group of Picture (GOP) encoding structure, the non-accelerated proportion, and block content variation within the frame, and design a top-down allocation and bottom-up feedback scheme, to achieve precise control of encoding complexity by controlling the maximum depth of QuadTree with nested Multi-type Tree (QTMT). In the scheme, Temporal ID (Tid)-based, Sum of Absolute Transformed Difference (SATD)-based methods and Linear (L) Model are designed to facilitate weighted allocation and feedback. The relationship between Planar Cost and encoding time is exploited as a Time-Cost (T-C) model, which guides the selection of Largest Coding Unit (LCU) encoding strategies in I-frames. For B-frames, we investigate a switching suppression assisted status-based method, to efficiently decide the encoding strategy of each LCU. Through the collaboration of the proposed technologies in the scheme, given any encoding time target achievable, encoding strategies will be automatically switched to help the actual encoding time gradually approach the target. To the best of our knowledge, this work is the first work on VVC-based encoding complexity control. The proposed scheme supports directly specifying the target encoding time or target Frame Per Second (FPS), and accurately realizing it within one pass. According to experimental results, under the target encoding time ratio of 80%, 60% and 40%, the average encoding time error is kept under 0.24%, 0.03% and 0.02% by our method, outperforming state-of-the-art works base on HEVC. These results prove the advantage of the proposed scheme.

Bayesian asymmetric quantized neural networks

This paper develops a robust model compression for neural networks via parameter quantization. Traditionally, quantized neural networks (QNN) were constructed by binary or ternary weights where the weights were deterministic. This paper generalizes QNN in two directions. First, M-ary QNN is developed to adjust the balance between memory storage and model capacity. The representation values and the quantization partitions in M-ary quantization are mutually estimated to enhance the resolution of gradients in neural network training. A flexible quantization with asymmetric partitions is formulated. Second, the variational inference is incorporated to implement the Bayesian asymmetric QNN. The uncertainty of weights is faithfully represented to enhance the robustness of the trained model in presence of heterogeneous data. Importantly, the multiple spike-and-slab prior is proposed to represent the quantization levels in Bayesian asymmetric learning. M-ary quantization is then optimized by maximizing the evidence lower bound of classification network. An adaptive parameter space is built to implement Bayesian quantization and neural representation. The experiments on various image recognition tasks show that M-ary QNN achieves similar performance as the full-precision neural network (FPNN), but the memory cost and the test time are significantly reduced relative to FPNN. The merit of Bayesian M-ary QNN using multiple spike-and-slab prior is investigated.

Anti-disturbance attitude quantized control of flexible spacecraft via a triggering and quantization joint dynamic event-triggered mechanism

This article investigates the problem of dynamic event-triggered anti-disturbance attitude quantized control based on the Takagi-Sugeno fuzzy method for flexible spacecraft, where multiple disturbances and parameter uncertainties are simultaneously considered. First, by employing the Takagi-Sugeno fuzzy model to represent the nonlinear disturbances with additional disturbance, a disturbance observer based on the Takagi-Sugeno fuzzy model is constructed under the unmeasurable premise variables to estimate the unknown nonlinear disturbances. Furthermore, a flexible vibration observer is formatted. Subsequently, a dynamic event-triggered mechanism with the quantization parameter is introduced, which can obtain a trade-off between quantization density and event-triggered frequency on the condition that the system stability is ensured. And an event-triggered anti-disturbance quantized controller is constructed, which can make up for Takagi-Sugeno fuzzy modeling disturbances and reject parameter uncertainties and other bounded disturbances. To guarantee the bound of the closed-loop signals, some sufficient conditions are obtained. At last, the efficiency of the suggested control strategy is illustrated via the simulation results.

Rate Control in Versatile Video Coding with Cosh Rate–Distortion Model

In this paper, we propose a rate control (RC) scheme for versatile video coding (VVC) to enhance coding performance. First, we propose a rate–distortion (R–D) model at the coding tree unit (CTU) level to describe the R–D relationship. Second, we adjust the quantization parameter (QP) and Lagrange multiplier at the coding unit (CU) level according to visual features to improve the rationality of local coding results. Finally, we propose a model parameter updating strategy to guarantee bitrate accuracy. The experimental results demonstrate that our RC method has better R–D performance with similar bitrate accuracy and better visual quality compared to the default RC strategy in VVC reference software VTM 10.2.

Deep Learning-Based Intra Mode Derivation for Versatile Video Coding

In intra coding, Rate Distortion Optimization (RDO) is performed to achieve the optimal intra mode from a pre-defined candidate list. The optimal intra mode is also required to be encoded and transmitted to the decoder side besides the residual signal, where lots of coding bits are consumed. To further improve the performance of intra coding in Versatile Video Coding (VVC) , an intelligent intra mode derivation method is proposed in this paper, termed as Deep Learning based Intra Mode Derivation (DLIMD) . In specific, the process of intra mode derivation is formulated as a multi-class classification task, which aims to skip the module of intra mode signaling for coding bits reduction. The architecture of DLIMD is developed to adapt to different quantization parameter settings and variable coding blocks including non-square ones, where only one single trained model is required. Different from the existing deep learning based classification problems, the hand-crafted features are also fed into intra mode derivation network besides the learned features from feature learning network. To compete with traditional methods, one additional binary flag is utilized in the video codec to indicate the selected scheme with RDO. Extensive experimental results reveal that the proposed method can achieve 2.28%, 1.74%, and 2.18% bit rate reduction on average for Y, U, and V components on the platform of VVC test model, which outperforms the state-of-the-art works.

Quantized Sliding Mode Fault-Tolerant Control for Unmanned Marine Vehicles with Thruster Saturation

In this paper, the sliding mode technique is used to study the quantized fault-tolerant control of unmanned marine vehicles with thruster saturation. Firstly, the sliding mode surface is constructed according to the full rank decomposition of input matrix, and the stability of sliding mode is guaranteed by linear matrix inequalities. An improved dynamic adjustment scheme of quantization parameter is proposed. Compared with the original adjustment scheme, the relationship between quantization parameter and desired targets is increased, so that the adjustment range of quantization parameters is more comprehensive. The sliding mode controller is combined with quantization parameter adjustment strategy to ensure the asymptotic stability of unmanned marine vehicles system. In addition, compared with the existing research results of quantitative fault tolerance problem without considering saturation, this paper gives a result of the domain of attraction affected by the fault of the thruster. Finally, the superiority of the proposed method is verified by simulation comparison.

Multi-Modality Deep Restoration of Extremely Compressed Face Videos.

Arguably the most common and salient object in daily video communications is the talking head, as encountered in social media, virtual classrooms, teleconferences, news broadcasting, talk shows, etc. When communication bandwidth is limited by network congestions or cost effectiveness, compression artifacts in talking head videos are inevitable. The resulting video quality degradation is highly visible and objectionable due to high acuity of human visual system to faces. To solve this problem, we develop a multi-modality deep convolutional neural network method for restoring face videos that are aggressively compressed. The main innovation is a new DCNN architecture that incorporates known priors of multiple modalities: the video-synchronized speech signal and semantic elements of the compression code stream, including motion vectors, code partition map and quantization parameters. These priors strongly correlate with the latent video and hence they are able to enhance the capability of deep learning to remove compression artifacts. Ample empirical evidences are presented to validate the superior performance of the proposed DCNN method on face videos over the existing state-of-the-art methods.

Smart Traffic Shaping Based on Distributed Reinforcement Learning for Multimedia Streaming over 5G-VANET Communication Technology

Vehicles serve as mobile nodes in a high-mobility MANET technique known as the vehicular ad hoc network (VANET), which is used in urban and rural areas as well as on highways. The VANET, based on 5G (5G-VANET), provides advanced facilities to the driving of vehicles such as reliable communication, less end-to-end latency, a higher data rate transmission, reasonable cost, and assured quality of experience (QoE) for delivered services. However, the crucial challenge with these recent technologies is to design a real-time multimedia traffic shaping that maintains smooth connectivity under the unpredictable change of channel capacity and data rate due to handover for rapid vehicle mobility among roadside units. This research proposes a smart real-time multimedia traffic shaping to control the amount and the rate of the traffic sent to the 5G-VANET based on distributed reinforcement learning (RMDRL). The proposed mechanism selects the accurate decisions of coding parameters such as quantization parameters, group of pictures, and frame rate that are used to manipulate the required traffic shaping of the multimedia stream on the 5G-VANET. Furthermore, the impact of the aforementioned three coding parameters has been comprehensively studied using five video clips to achieve the optimal traffic rate value for real-time multimedia streaming on 5G communication. The proposed algorithm outperforms the baseline traffic shaping in terms of peak-signal-to-noise-ratio (PSNR) and end-to-end frame delay. This research will open new comfortable facilities for vehicle manufacturing to enhance the data communication system on the 5G-VANET.