Block Partitioning Research Articles

Fast Multi-Type Tree Partitioning for Versatile Video Coding Using a Lightweight Neural Network

In this paper, we propose a fast decision scheme using a lightweight neural network (LNN) to avoid redundant block partitioning in versatile video coding (VVC). A more versatile block structure, named the multi-type tree (MTT) structure, which includes binary trees (BTs) and ternary trees (TTs), is adopted by VCC, in addition to the traditional quadtree structure. The MTT improved the coding efficiency compared with previous video coding standards. However, the new tree structures, mainly TT, significantly increased the complexity of the VVC encoder. Although widespread application of VVC has been inhibited, this problem has not yet been investigated thoroughly in the literature. In this study, we first determine the statistical characteristics of coded parameters that exhibit correlation with the TT and develop two useful types of features <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">—explicit VVC features</i> (EVFs) and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">derived VVC features</i> (DVFs) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">—</i> to facilitate the intra coding of VVC. These features can be obtained efficiently during the intra prediction before the determination of the best block partitioning during rate-distortion optimization in VVC encoding. Our LNN model decides whether to terminate the nested TT block structures subsequent to a quadtree based on the features. The experimental results confirm that the proposed method substantially decreases the encoding complexity of VVC with a slight coding loss under the All Intra configuration. Our code, models, and dataset are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/foriamweak/MTTPartitioning_LNN</uri> .

Geometric Partitioning Mode in Versatile Video Coding: Algorithm Review and Analysis

This paper presents an overview of the geometric partitioning mode (GPM) algorithm that is a part of the most recent Versatile Video Coding (VVC) standard. The GPM algorithm aims to increase the partitioning precision of moving objects using non-rectangular and asymmetric rectangular partitions on top of the conventional rectangular block partitioning structure of VVC. Novel features of GPM contributing to the increase in coding efficiency and the reduction in encoder and decoder complexity are detailed and analyzed in this paper. Evaluated with VVC test model version 8.0 under the joint video experts team common test conditions, experimental results show that the presented GPM algorithm provides luma Bjøntegaard Delta rate reduction of 0.70% for random access and of 1.55% for low delay with B slices configurations, with roughly 3% to 5% additional encoding time and negligible decoder runtime change. Furthermore, as GPM provides more precise partitions for the boundaries of the moving objects, an improvement of visual quality is seen in GPM coded sequences.

Kernelized dual regression incorporating local information for image set classification

Abstract In image set classification, dual linear regression classification (DLRC) has shown the excellent performance on face image data without the interference of the complex background. However, DLRC could not well identify the data set with the complex background. The complex background means that the background is cluttered and the viewpoint is unusual or the object is partially occluded. This paper proposes a new model, kernelized dual regression (KDR), based on DLRC and the kernel trick which is a useful technique in image classification. Different from DLRC, KDR adopts a block partitioning strategy to extract the local information, which is able to conquer the shortcoming of DLRC. To capture the nonlinear relationship between the training set and test set, KDR tactfully maps these image sets into a high-dimensional feature space by adopting the nonlinear mapping associated with the Gaussian kernel function. In the reproducing kernel Hilbert space (RKHS), KDR can find the joint coefficients by minimizing the distance between training set and test set, and has a closed-form solution. Extensive experiments on four datasets show that KDR could achieve better classification performance than that of DLRC and other existing methods.

Recursive Quad-Tree Block Partitioning for Data Embedding in Images

In this paper, we propose to embed messages in raw images using a recursive block partitioning technique adopted from HEVC video coding technology. This work introduces a quad-tree partitioning solution in which square blocks of pixels are partitioned recursively according to a homogeneity criterion. As such, large block sizes are allocated for homogenous areas of the image and smaller block sizes are used elsewhere. For this to work, the variable block sizes should cover the whole image without gaps. Additionally, the criterion used for determining the block partitioning should work for both message embedding and message extraction. As such, the extractor process can reproduce the same variable block sizes. Matrix encoding is used for message embedding and extraction, as such, a maximum of one-pixel value is changed per block. The proposed solution is assessed in terms of percentage of changed pixels, PSNR, SSIM, histogram changes and blind steganalysis. Comparison with existing work reveals that the proposed solution reduces both the image distortions and pixel change rates. With embedding rates ranging from 2.5 to 38%, the average PSNR of the cover images ranged from 72 to 60 dB, and the average normalized histogram differences ranged from 3 to 12 pixels. It is also shown that the proposed solution is less detectible when tested with blind steganalysis in comparison to existing solutions where the average detection accuracy is 49.8%.

Scene Segmentation of Multi-Band ISAR Fusion Imaging Based on MB-PCSBL

We consider the problem of achieving multi-band inverse synthetic aperture radar (ISAR) fusion imaging of block structure targets with unknown block partition and develop a block-sparse recovering method based on matrix block pattern-coupled sparse Bayesian learning algorithm. Based on the sparse representation of multi-band ISAR fusion imaging model, a pattern-coupled hierarchical Gaussian prior is proposed to characterize the pattern relevance of scattering coefficients. The sparsity of each coefficient is controlled not only by its own hyperparameter, but also by the hyperparameters corresponding to its eight neighboring coefficients in the data matrix. The correlations between the coefficients in rows and columns are determined by different parameters, respectively. The proposed prior model can increase the model flexibility and promote the generation of block structures. Moreover, the whole observation scene is segmented into multiple sub-scenes to reduce the memory storage space and the computational complexity. Parameters and the fusion image result of each sub-scene are derived by the expectation-maximization method. The multi-band ISAR fusion image result of the whole scene is obtained through the stitching of the sub-scenes imaging results. Experimental results demonstrate the effectiveness and superiority of the proposed algorithm.

A low-overhead asynchronous consensus framework for distributed bundle adjustment

Generally, the distributed bundle adjustment (DBA) method uses multiple worker nodes to solve the bundle adjustment problems and overcomes the computation and memory storage limitations of a single computer. However, the performance considerably degrades owing to the overhead introduced by the additional block partitioning step and synchronous waiting. Therefore, we propose a low-overhead consensus framework. A partial barrier based asynchronous method is proposed to early achieve consensus with respect to the faster worker nodes to avoid waiting for the slower ones. A scene summarization procedure is designed and integrated into the block partitioning step to ensure that clustering can be performed on the small summarized scene. Experiments conducted on public datasets show that our method can improve the worker node utilization rate and reduce the block partitioning time. Also, sample applications are demonstrated using our large-scale culture heritage datasets.

A fast method for variable-order Caputo fractional derivative with applications to time-fractional diffusion equations

In this paper, we propose a fast algorithm for the variable-order (VO) Caputo fractional derivative based on a shifted binary block partition and uniform polynomial approximations of degree r. Compared with the general direct method, the proposed algorithm can reduce the memory requirement from O(n) to O(rlogn) storage and the complexity from O(n2) to O(rnlogn) operations, where n is the number of time steps. As an application, we develop a fast finite difference method for solving a class of VO time-fractional diffusion equations. The computational workload is of O(rmnlogn) and the active memory requirement is of O(rmlogn), where m denotes the size of spatial grids. Theoretically, the unconditional stability and error analysis for the proposed fast finite difference method are given. Numerical results of one and two dimensional problems are presented to demonstrate the well performance of the proposed method.

Fast intra-coding unit partition decision in H.266/FVC based on deep learning

In the recent Future Video Coding (FVC) standard developed by the Joint Video Exploration Team (JVET), the quad-tree binary-tree (QTBT) block partition module makes use of rectangular block forms and additional square block sizes compared to quad-tree (QT) block partitioning module proposed in the predecessor High-Efficiency Video Coding (HEVC) standard. This block flexibility, induced with the QTBT module, significantly improves compression performance while it dramatically increases coding complexity due to the brute force search for Rate Distortion Optimization (RDO). To cope with this issue, it is necessary to consider the unique characteristics of QTBT in FVC. In this paper, we propose a fast QT partitioning algorithm based on a deep convolutional neural network (CNN) model to predict coding unit (CU) partition instead of RDO which enhances considerably QTBT performance for intra-mode coding. Based on a suitable diversified CU partition patterns database, the optimization process is set up with three levels CNN structure developed to learn the split or non-split decision from the established database. Experimental results reveal that the proposed algorithm can accelerate the QTBT block partition structure by reducing the intra-mode encoding time by an average of 35% with a bit rate increase of 1.7%, allowing its application in practical scenarios.

Fine-grain complexity control of HEVC intra prediction in battery-powered video codecs

The high-efficiency video coding (HEVC) standard improves the coding efficiency at the cost of a significantly more complex encoding process. This is an issue for a large number of video-capable devices that operate on batteries, with limited and varying processing power. A complexity controller enables an encoder to provide the best possible quality at any power quota. This paper proposes a complexity control method for HEVC intra coding, based on a Pareto-efficient rate–distortion–complexity (R–D–C) analysis. The proposed method limits the intra prediction for each block (as opposed to existing methods which limit the block partitioning), on a frame-level basis. This method consists of three steps, namely rate-complexity modeling, complexity allocation, and configuration selection. In the first step, a rate-complexity model is presented which estimates the encoding complexity according to the compression intensity. Then, according to the estimated complexity and target complexity, a complexity budget is allocated to each frame. Finally, an encoding configuration from a set of Pareto-efficient configurations is selected according to the allocated complexity and the video content, which offers the best compression performance. Experimental results indicate that the proposed method can adjust the complexity from 100 to 50%, with a mean error rate of less than 0.1%. The proposed method outperforms many state-of-the-art approaches, in terms of both control accuracy and compression efficiency. The encoding performance loss in terms of BD-rate varies from 0.06 to 3.69%, on average, for 90–60% computational complexity, respectively. The method can also be used for lower than 50% complexity if need be, with a higher BD-rate.

Adaptive Block Coordinate Descent for Distortion Optimization

AbstractWe present a new algorithm for optimizing geometric energies and computing positively oriented simplicial mappings. Our major improvements over the state‐of‐the‐art are: (i) introduction of new energies for repairing inverted and collapsed simplices; (ii) adaptive partitioning of vertices into coordinate blocks with the blended local‐global strategy for more efficient optimization and (iii) introduction of the displacement norm for improving convergence criteria and for controlling block partitioning. Together these improvements form the basis for the Adaptive Block Coordinate Descent (ABCD) algorithm aimed at robust geometric optimization. ABCD achieves state‐of‐the‐art results in distortion minimization, even under hard positional constraints and highly distorted invalid initializations that contain thousands of collapsed and inverted elements. Starting with an invalid non‐injective initial map, ABCD behaves as a modified block coordinate descent up to the point where the current mapping is cleared of invalid simplices. Then, the algorithm converges rapidly into the chosen iterative solver. Our method is very general, fast‐converging and easily parallelizable. We show over a wide range of 2D and 3D problems that our algorithm is more robust than existing techniques for locally injective mapping.

The computer desktop image compression based on clustering algorithm

SummaryWith the rapid development of computer software and hardware technology and network technology, aiming at the limitations of the traditional image compression standards and the deficiencies of the existing computer desktop image compression methods. By analyzing the characteristics of computer desktop image, according to the characteristics of desktop compression, a compression scheme based on high efficiency video coding (HEVC) and color clustering is proposed, which divides blocks into text/graphics blocks, natural image blocks and mixed blocks based on the features of histogram information and texture information of blocks. In block division and classification, an adaptive dynamic block classification algorithm is proposed which is different from the traditional block partitioning. Compared with the traditional method, the new block classification algorithm can save the code stream and improve the classification accuracy.

Ensemble Learning-Based Rate-Distortion Optimization for End-to-End Image Compression

End-to-end image compression using trained deep networks as encoding/decoding models has been developed substantially in the recent years. Previous work is limited in using a single encoding/decoding model, whereas we explore the usage of multiple encoding/decoding models as an ensemble. We propose several methods to obtain multiple models. First, we adopt the boosting strategy to train multiple networks with diversity as an ensemble. Second, we train an ensemble of multiple probability distribution models to reduce the distribution gap for efficient entropy coding. Third, we present a geometric transform-based self-ensemble method. The multiple models can be regarded as the multiple coding modes, similar to those in non-deep video coding schemes. We further adopt block-level model/mode selection at the encoder side to pursue rate-distortion optimization, where we use hierarchical block partitioning to improve the adaptation ability. Compared with single-model end-to-end compression, our proposed method improves the compression efficiency significantly, leading to 21% BD-rate reduction on the Kodak dataset, without increasing the decoding complexity. On the other hand, when keeping the same compression efficiency, our method can use much simplified decoding models, where the floating-point operations are reduced by 70%.

Parallel finite-difference algorithms for three-dimensional space-fractional diffusion equation with $$\psi $$-Caputo derivatives

The paper deals with the issues of parallel computations’ organization while solving three-dimensional space-fractional diffusion equation with the $$\psi $$ -Caputo derivatives using finite difference schemes. For an implicit scheme and locally one-dimensional splitting scheme, we present parallel algorithms for distributed memory systems that use one-dimensional block and red–black data partitioning. To reduce the order of algorithms’ computational complexity, we use an approach based on the expansion of integral operator’s kernel into series. We present the theoretical estimates of parallel algorithms’ performance and the results of computational experiments conducted on a testing problem that has an analytical solution for the case of the Caputo–Katugampola derivative. The results of the experiments show close-to-linear parallelization efficiency of one-dimensional splitting scheme with block partitioning and inefficiency of red–black partitioning in this case. For the implicit scheme, the scalability of parallel algorithms is weak and the use of red–black partitioning is more efficient than the use of block partitioning when running on a small number of computational resources.

Theory of Coupled Characteristic Modes

A theory of coupled characteristic modes is developed for computing the eigenmodes of a coupled array using the modes of the isolated elements and the corresponding mutual interaction matrix. The systematic blending of the matrix block partitioning method and the orthogonal properties of the eigenvectors provide a generalized formulation considering the interaction of all the isolated modes. Later, closed-form solutions have been derived for the dominant coupled modes with the assumption of one-to-one mode coupling of the isolated elements. The idea has been further extended for two-to-two mode interaction modeling which conforms into a fourth-order generalized characteristic equation. The higher-order extension can be solved in a similar numerical process. Different examples have been considered to validate the proposed technique.

Using ZDDs in the Mapping of Quantum Circuits

A critical step in quantum compilation is the transformation of a technology-independent quantum circuit into a technology-dependent form for a targeted device. In addition to mapping quantum gates into the supported gate set, it is necessary to map pseudo qubits in the technology-independent circuit into physical qubits of the technology-dependent circuit such that coupling constraints among qubits acting in multiple-qubit gates are satisfied. It is usually not possible to find such a mapping without adding SWAP gates into the circuit. To cope with the technical limitations of NISQ-era quantum devices, it is advantageous to find a mapping that requires as few additional gates as possible. The large search space of possible mappings makes this task a difficult combinatorial optimization problem. In this work, we demonstrate how zero-suppressed decision diagrams (ZDDs) can be used for typical implementation tasks in quantum mapping algorithms. We show how to maximally partition a quantum circuit into blocks of adjacent gates, and if adjacent gates within a circuit do not share common mapping permutations, we attempt to combine them using parallelized SWAP operations represented in a ZDD. Boundaries for the partitions are formed where adjacent gates are unable to be combined. Within each partition block, ZDDs represent all possible mappings of pseudo qubits to physical qubits.

A Novel Rate and Distortion Estimation Method Using Particle Filtering Based Prediction for Intra-Predictive Coding of Deep Block Partitioning Structures

In this paper, we propose a new R/D estimation method for intra-predictive coding with deep block partitioning structures. In our proposed R/D prediction, we adopt a particle filtering based prediction (PFP) to precisely predict intermediate R/D estimates for the next frame in a stochastic manner, which helps increasing the prediction accuracy of fast changing R/D values. Then, based on the intermediate R/D estimates by PFP, we infer an optimal model parameter of the TC’s probability density function (pdf) via convex optimization. We found that the proposed method brings about more stable R/D estimation performance thanks to both the improved prediction accuracy using the PFP for abrupt changes in true R/D values and the precise estimation of the optimal model parameter. Experimental results show that our method significantly reduces the normalized root mean square error from average 3.17 to 0.79 (74.90% reduction) for rate and from average 2.32 to 0.82 (64.61% reduction) for distortion, compared to the state-of-the art method.

Text tendency analysis based on multi-granularity emotional chunks and integrated learning

Internet is a new type of information exchange tool that has developed with the times. Now, it has been integrated into all aspects of our study and life. At the same time, in the era of the rise of social media, there are more and more platforms on the Internet. A variety of critical texts have also exploded in these platforms. In these opinions and comments, the subjective opinions of the presenters are included, and the emotional tendencies of the commenters are expressed. Nowadays, subjective text information resources are huge. One of the major problems to be solved by management objects facing information management is how to manage them effectively so that users can quickly and accurately find the required information. Therefore, classifying these text tendencies and mining the potential value in the text has broad application prospects. Based on the text granularity and processing efficiency, this paper conducts multi-granularity emotional block partitioning on network texts and compares the sentiment analysis under different granularities. Furthermore, a random subspace integrated learning text sentiment classification method based on BPSO (binary particle swarm optimization) is proposed to analyze text orientation. By simulating news site comments and e-commerce website reviews such as Taobao, the convergence analysis of BPSO in the optimization of the number of base classifiers shows that the BPSO algorithm can be applied to the random subspace method well, and the accuracy of the classification results is high.

Three-way active learning through clustering selection

In clustering-based active learning, the performance of the learner relies heavily on the quality of clustering results. Empirical studies have shown that different clustering techniques are applicable to different data. In this paper, we propose the three-way active learning through clustering selection (TACS) algorithm to dynamically select the appropriate techniques during the learning process. The algorithm follows the coarse-to-fine scheme of granular computing coupled with three-way instance processing. For label query, we select both representative instances with density peaks, and informative instances with the maximal total distance. For block partition, we revise six popular clustering techniques to speed up learning and accommodate binary splitting. For clustering evaluation, we define weighted entropy with 1-nearest-neighbor. For insufficient labels, we design tree pruning techniques with the use of a block queue. Experiments are undertaken on twelve UCI datasets. The results show that TACS is superior to single clustering technique based algorithms and other state-of-the-art active learning algorithms.

Lightweight Key Encapsulation Using LDPC Codes on FPGAs

In this paper, we present a lightweight hardware design for a recently proposed quantum-safe key encapsulation mechanism based on QC-LDPC codes called LEDAkem, which has been admitted as a round-2 candidate to the NIST post-quantum standardization project. Existing implementations focus on high speed while few of them take into account area or power efficiency, which are particularly decisive for low-cost or power constrained IoT applications. The solution we propose aims at maximizing the metric of area efficiency by rotating the QC-LDPC code representations amongst the block RAMs in digit level. Moreover, optimized parallelized computing techniques, lazy accumulation and block partition are exploited to improve key decapsulation in terms of area and timing efficiency. We show for instance that our area-optimized implementation for 128-bit security requires $6.82\times 10^5$ 6 . 82 × 10 5 cycles and $2.26\times 10^6$ 2 . 26 × 10 6 cycles to encapsulate and decapsulate a shared secret, respectively. The area-optimized design uses only 39 slices (3 percent of the available logic) and 809 slices (39 percent of the available logic) for key encapsulation and key decapsulation respectively, on a small-size low-end Xilinx Spartan-6 FPGA.

Motion density-based unequal error protection technique for video bitstream compressed by HEVC standard

Implementing a reliable unequal error protection (UEP) technique based on motion activity of video frames is considered as an effective method to improve performance of compressed video bitstream transmitted over the wireless network. Most of the existing works are relevant to motion activity of H.264/advanced video coding frames. These mainly determine importance of frames based on either size of partitions having motion information or motion activity of their blocks. However, in the latest coding standard High Efficiency Video Coding (HEVC), there exists an improved block partitioning method, which demands a different representation of motion activity, especially for frames comprising a large number of units. This paper proposes motion density-based scheme of unequal error protection (UEP) for HEVC-compressed video bitstream. In comparison with previously motion activity-based UEP techniques, motion density has better capability to extract importance of the small-size units and consequently determines importance of frames. This is achieved by number of simulations conducted for the inter-prediction mode of the video bitstream.