Fast Partitioning Research Articles

An adaptive cross-class transfer learning framework with two-level alignment

Because of insufficient data, cross-class transfer learning has potential prospects in motor imagery electroencephalogram (MI-EEG) based rehabilitation engineering, and it has not been effectively addressed to reduce cross-class variability between source and target domains and find the best class-to-class transitive correspondence (CCTC). In this paper, we propose an adaptive cross-class transfer learning (ACTL) framework with two-level alignment (TLA) for MI decoding. By using the fast partitioning around medoids method, the central samples of each class are acquired from the source domain and target domain respectively. Then, they are applied to construct the central alignment matrix to realize the 1st-level domain alignment for any possible CCTC case. The 2nd-level subject alignment is performed by Euclidean alignment for each class of aligned source domain. Furthermore, the central samples of the target domain together with the two-level aligned source domain are mapped into tangent space, the resulting features are fed to a teacher convolutional neural network (TCNN), it and the transferred student CNN (SCNN) are trained successively, and their parameters of distillation loss are optimized automatically by a scaling-based grid search method. Experiments are conducted on a public MI-EEG dataset with multiple subjects and MI-tasks, the proposed framework can find the optimal SCNN associated with the best CCTC, which achieves a statistically significant average classification accuracy of 86.37%. The results suggest that TLA is helpful for increasing the distribution similarity between different domains, and the knowledge distillation embedded in ACTL framework greatly simplifies the SCNN and outperforms the complicated TCNN.

An early CU partition mode decision algorithm in VVC based on variogram for virtual reality 360 degree videos

360-degree videos have become increasingly popular with the application of virtual reality (VR) technology. To encode such kind of videos with ultra-high resolution, an efficient and real-time video encoder becomes a key requirement. The Versatile Video Coding (VVC) standard has good coding performance. However, it has pretty high computational complexity which increasing the application cost of 360-degree videos. Among them, the decision of the quadtree with nested multi-type tree (QTMT) partitioning structure is one of the time-consuming procedures. In this paper, based on the characteristics of 360-degree video with Equirectangular projection (ERP) format, the empirical variogram combined with Mahalanobis distance is introduced to measure the difference between the horizontal and vertical directions of the CU, and a fast partition algorithm is proposed. The experimental results show that the algorithm saves 32.13% of the coding time with only an increase of 0.66% in BDBR.

A Method to Reduce the Intra-Frame Prediction Complexity of HEVC Based on D-CNN

Among a series of video coding standards jointly developed by ITU-T, VCEG, and MPEG, high-efficiency video coding (HEVC) is one of the most widely used video coding standards today. Therefore, it is still necessary to further reduce the coding complexity of HEVC. In the HEVC standard, a flexible partitioning procedure entitled “quad-tree partition” is proposed to significantly improve the coding efficiency, which, however, leads to high coding complexity. To reduce the coding complexity of the intra-frame prediction, this paper proposes a scheme based on a densely connected convolution neural network (D-CNN) to predict the partition of coding units (CUs). Firstly, a densely connected block was designed to improve the efficiency of the CU partition by fully extracting the pixel features of CTU. Then, efficient channel attention (ECA) and adaptive convolution kernel size were applied to a fast CU partition for the first time to capture the information of the D-CNN convolution channels. Finally, a threshold optimization strategy was formulated to select the best threshold for each depth to further balance the computation complexity of video coding and the performance of RD. The experimental results show that the proposed method reduces the encoding time of HEVC by 60.14%, with a negligible reduction in RD performance, which is better than the existing fast partitioning methods.

Accelerating QTMT-based CU partition and intra mode decision for versatile video coding

The H.266/VVC achieves about 50% bitrate saving compared to its predecessor H.265/HEVC at the expense of exponentially increased computational complexity. The most efficient but complex technique for H.266/VVC intra frame coding is the QuadTree with a nested Multi-type Tree encoding structure (QTMT), which usually requires traversing the Rate-Distortion (R-D) cost of each partition and each mode for the best option. To alleviate such computational burden while preserving the coding efficiency as much as possible, this paper develops a multi-feature guided Fast CU Partition (FCP) and Laplacian guided Fast Mode Selection (FMS) to accelerate the intra QTMT decision together. For FCP, we regard the CU partition as a classification problem and adopt the Support Vector Machine (SVM) for its low-complexity implementation; after evaluating the contribution of a set of features, three representative features of video textures are selected and used to train the SVM model. Additionally, an advanced technique is applied by adopting a soft decision in SVM for a more flexible trade-off between the complexity and R-D performance. For FMS, we utilize the Laplace operator to determine the most probable directions of the current CU and skip half of the candidate modes for runtime saving. Experimental results demonstrate that the proposed FCP reduces the encoding time of H.266/VVC by 51.03% with 1.65% Bjøntegaard Delta Bit-Rate (BDBR) increase; the proposed FMS reduces the encoding time by 12.68% with 0.09% BDBR loss. Their direct combination and advanced combination finally lead to 54.84% encoding time reduction with 1.74% BDBR increase and 40.39% encoding time reduction with 1.33% BDBR increase, respectively, outperforming state-of-the-art approaches significantly.

A fast full partitioning algorithm for HEVC-to-VVC video transcoding using Bayesian classifiers

The Versatile Video Coding (VVC) standard was released in 2020 to replace the High Efficiency Video Coding (HEVC) standard, making it necessary to convert HEVC encoded content to VCC to exploit its compression performance, which was achieved by using a larger block size of 128 × 128 pixels, among other new coding tools. However, 80.93% of the encoding time is spent on finding a suitable block partitioning. To reduce this time, this proposal presents an HEVC-to-VVC transcoding algorithm focused on accelerating the CTU partitioning decisions. The transcoder takes different information from the input bitstream of HEVC, and feeds it to two Bayes-based models. Experimental results show a time saving in the transcoding process of 45.40%, compared with the traditional cascade transcoder. This time gain has been obtained on average for all test sequences in the Random Access scenario, at the expense of only 1.50% BD-rate.

Machine Learning Based Fast QTMTT Partitioning Strategy for VVenC Encoder in Intra Coding

The newest video compression standard, Versatile Video Coding (VVC), was finalized in July 2020 by the Joint Video Experts Team (JVET). Its main goal is to reduce the bitrate by 50% over its predecessor video coding standard, the High Efficiency Video Coding (HEVC). Due to the new advanced tools and features included in VVC, it actually provides high coding performances—for instance, the Quad Tree with nested Multi-Type Tree (QTMTT) involved in the partitioning block. Furthermore, VVC introduces various techniques that allow for superior performance compared to HEVC, but with an increase in the computational complexity. To tackle this complexity, a fast Coding Unit partition algorithm based on machine learning for the intra configuration in VVC is proposed in this work. The proposed algorithm is formed by five binary Light Gradient Boosting Machine (LightGBM) classifiers, which can directly predict the most probable split mode for each coding unit without passing through the exhaustive process known as Rate Distortion Optimization (RDO). These LightGBM classifiers were offline trained on a large dataset; then, they were embedded on the optimized implementation of VVC known as VVenC. The results of our experiment show that our proposed approach has good trade-offs in terms of time-saving and coding efficiency. Depending on the preset chosen, our approach achieves an average time savings of 30.21% to 82.46% compared to the VVenC encoder anchor, and a Bjøntegaard Delta Bitrate (BDBR) increase of 0.67% to 3.01%, respectively.

New binary and ternary quasi-cyclic codes with good properties

One of the most important and challenging problems in coding theory is to construct codes with best possible parameters and properties. The class of quasi-cyclic (QC) codes is known to be fertile to produce such codes. Focusing on QC codes over the binary field, we have found 113 binary QC codes that are new among the class of QC codes using an implementation of a fast cyclic partitioning algorithm and the highly effective ASR algorithm. Moreover, these codes have the following additional properties: a) they have the same parameters as best known linear codes, and b) many of the have additional desired properties such as being reversible, LCD, self-orthogonal or dual-containing. Additionally, we present an algorithm for the generation of new codes from QC codes using ConstructionX, and introduce 33 new record breaking linear codes over GF(2), GF(3) and GF(5) produced from this method.

Two-step isolation of the two major paraconic acids of Cetraria islandica.

The lichen Cetraria islandica is traditionally used as a demulcent for the symptomatic treatment of irritations of the mouth and throat and associated dry cough, as well as for the treatment of temporary loss of appetite. In addition to depsides and depsidones, thalli contain paraconic acids, a group of secondary metabolites commonly found in lichens and fungi. Among those, protolichesterinic acid has shown promising pharmacological activities. However, the efficient isolation of paraconic acids is quite complex due to their very similar chemical structures and their weak ultraviolet absorption. In the present work, a two-step isolation protocol of protolichesterinic acid and lichesterinic acid from a complex paraconic acid mixture is described using Sephadex LH20 column chromatography and fast centrifugal partition chromatography. Final purities higher than 95% and recoveries above 50% are achieved. Additionally, reliable qualitative techniques for detecting and differentiating paraconic acids are described. Finally, some data on compound stability and enantiomeric purity are shown.

Discovery of a novel, highly potent and orally bioavailable pyrrolidinone indole series of irreversible Myeloperoxidase (MPO) inhibitors

Myeloperoxidase (MPO) is a heme-containing peroxidase from phagocytic cells, which plays an important role in the innate immune response. The primary anti-microbial function of MPO is achieved by catalyzing the oxidation of halides by hydrogen peroxide (H2O2). Upon activation of phagocytes, MPO activity is detectable in both phagosomes and extracellularly, where it can remain or transcytose into interstitial compartments. Activated MPO leads to oxidative stress and tissue damage in many inflammatory states, including cardiovascular disease. Starting from a low molecular weight (LMW) high throughput screening (HTS) hit, here we report the discovery of a novel pyrrolidinone indole (IN-4) as a highly potent MPO inhibitor. This compound displays similar in vitro potency across peroxidation, plasma and NETosis assays. In a dilution/dialysis study, <5% of the original MPO activity was detected post-incubation of MPO with IN-4, suggesting irreversible enzyme inhibition. A fast MPO inactivation rate (kinact/Ki) and low partition ratio (k3/k4) make IN-4 kinetic properties attractive for an MPO inhibitor. This compound also displays significant selectivity over the closely related thyroid peroxidase (TPO), and is selective for extracellular MPO over intracellular (neutrophil) MPO. Moreover, IN-4 shows good exposure, low clearance and high oral bioavailability in mice, rats and dogs. The high in vitro MPO activity and high oral exposure observed with IN-4 result in a dose-dependent inhibition of MPO activity in three mouse models of inflammation. In conclusion, IN-4 is a novel, potent, mechanism-based and selective MPO inhibitor, which may be used as superior therapeutic agent to treat multiple inflammatory conditions, including cardiovascular disease.

Partition Map Prediction for Fast Block Partitioning in VVC Intra-Frame Coding

The Versatile Video Coding (VVC) standard introduces a block partitioning structure known as quadtree plus nested multi-type tree (QTMTT), which allows more flexible block partitioning compared to its predecessors, like High Efficiency Video Coding (HEVC). Meanwhile, the partition search (PS) process, which is to find out the best partitioning structure for optimizing the rate-distortion cost, becomes far more complicated for VVC than for HEVC. Also, the PS process in VVC reference software (VTM) is not friendly to hardware implementation. We propose a partition map prediction method for fast block partitioning in VVC intra-frame encoding. The proposed method may replace PS totally or be combined with PS partially, thereby achieving adjustable acceleration of the VTM intra-frame encoding. Different from the previous methods for fast block partitioning, we propose to represent a QTMTT-based block partitioning structure by a partition map, which consists of a quadtree (QT) depth map, several multi-type tree (MTT) depth maps, and several MTT direction maps. We then propose to predict the optimal partition map from the pixels through a convolutional neural network (CNN). We propose a CNN structure, known as Down-Up-CNN, for the partition map prediction, where the CNN structure emulates the recursive nature of the PS process. Moreover, we design a post-processing algorithm to adjust the network output partition map, so as to obtain a standard-compliant block partitioning structure. The post-processing algorithm may produce a partial partition tree as well; then based on the partial partition tree, the PS process is performed to obtain the full tree. Experimental results show that the proposed method achieves 1.61× to 8.64× encoding acceleration for the VTM-10.0 intra-frame encoder, with the ratio depending on how much PS is performed. Especially, when achieving 3.89× encoding acceleration, the compression efficiency loss is 2.77% in BD-rate, which is a better tradeoff than the previous methods.

Fast CU Decision Algorithm Based on Texture Complexity and CNN for VVC

Versatile Video Coding (VVC) introduces many advanced video coding techniques. These advanced video coding techniques not only improve video compression efficiency and video quality, but also greatly increase coding time and computational complexity. Therefore, this paper uses a fast CU partitioning decision algorithm based on texture complexity and convolutional neural networks (CNN). First, we performed statistics and analysis of CU segmentation patterns for videos in the standard training set, then processed large blocks of CU based on texture complexity. To realize the purpose of fast partitioning, we design different CNN models for CU with different segmentation modes. In the design of CNN model, we use the symmetric convolutional kernel and asymmetric convolutional kernel to extract features in different directions effectively. In the loss function, we used the cross-entropy function to train the CNN model to improve the accuracy of the model. Finally, a double threshold is set in the candidate list to achieve a compromise between coding performance and coding complexity. Experimental results show that, compared to the VTM10.0 anchoring algorithm, our fast scheme, in terms of encoding time, decreases by 55.90% and BDBR increases by 1.79%; our moderate scheme, in terms of encoding time, decreases by 47.90%. BDBR increases by only 1.29%.

Fast CU Partitioning Algorithm for VVC Based on Multi-Stage Framework and Binary Subnets

VVC is the latest video compression technology available, and while the coding efficiency has improved significantly over the previous generation of standards, it has also led to a dramatic increase in coding complexity. As VVC uses a QTMT division structure, the more flexible division structure also allows for a significant increase in coding time. We have built a multi-stage network framework to solve the above problem by dividing the CU into different stages according to the size of the blocks. The desired features are extracted by dynamically adjusting to the size of the input CU. Secondly, we construct a binary classification subnet to perform the classification task at each stage and can determine the QT and MT division decisions. Finally, the resulting experimental results can demonstrate that our novel two-threshold decision scheme can achieve a balance between RD performance and TS. Our method succeeds in reducing the coding time by 49.08% to 52.56%, while the complexity of the negligible BD-BR increases by only 1.10% to 1.42%.

A Fast CU Partition Algorithm Based on Gradient Structural Similarity and Texture Features

The H.266/Versatile Video Coding (VVC) standard poses a great challenge for encoder design due to its high computational complexity and long encoding time. In this paper, the fast partitioning decision of coding blocks is investigated to reduce the computational complexity and save the coding time of VVC intra-frame predictive coding. A fast partitioning algorithm of VVC intra-frame coding blocks based on gradient structure similarity and directional features is proposed. First, the average gradient structure similarity of four sub-coding blocks under the current coding block is calculated, and two thresholds are set to determine whether the current coding block terminates the partitioning early or performs quadtree partitioning. Then, for the coding blocks that do not satisfy the above thresholds, the standard deviation of the vertical and horizontal directions of the current coding block is calculated to determine the texture direction and skip unnecessary partitioning to reduce computational complexity. Based on the VTM10.0 platform, this paper evaluates the performance of the designed fast algorithm for partitioning within the VVC coding unit. Compared with VTM10.0, the encoding rate is improved by 1.38% on average, and the encoder execution time is reduced by 49.32%. The overall algorithm achieves a better optimization of the existing VVC intra-frame coding technique.

Texture-based fast QTMT partition algorithm in VVC intra coding

Texture-based fast QTMT partition algorithm in VVC intra coding

Fast CU Partition Decision Algorithm for VVC Intra Coding Using an MET-CNN

The newest video coding standard, the versatile video coding standard (VVC/H.266), came into effect in November 2020. Different from the previous generation standard—high-efficiency video coding (HEVC/H.265)—VVC adopts a more flexible block division structure, the quad-tree with nested multi-type tree (QTMT) structure, which improves its coding performance by 24%. However, it also causes a substantial increase in computational complexity. Therefore, this paper first proposes the concept of a stage grid map, which divides the overall division of a 32 × 32 coding unit (CU) into four stages and represents it as a structured output. Second, a multi-stage early termination convolutional neural network (MET-CNN) model is devised to predict the full partition information of a CU with a size of 32 × 32. Finally, a fast CU partition decision algorithm for VVC intra coding based on an MET-CNN is proposed. The algorithm can predict all partition information of a CU with a size of 32 × 32 and its sub-CUs in one run, completely replacing the complex rate-distortion optimization (RDO) process. It also has an early exit mechanism, thereby greatly reducing the encoding time. The experimental results illustrate that the scheme proposed in this paper reduces the encoding time by 49.24% on average, while the Bjøntegaard Delta Bit Rate (BDBR) only increases by 0.97%.

A GCN-based fast CU partition method of intra-mode VVC

In this paper, a global convolutional network (GCN)-based fast coding unit (CU) partition method of intra-mode VVC is proposed. By using the GCN module with large kernel size convolutions, the proposed method can capture global information in CUs, leading to an accurate partition mode prediction in the quad-tree plus multi-type tree (QTMT) structure. Ranked according to predicted probabilities, the partition modes with lower probabilities are discarded, which reduces the computational complexity of VVC. Additionally, tradeoffs between performance and complexity can be achieved with different strategies. Experimental results demonstrated that the proposed method can reduce encoding time by 51.06%∼61.15% while increasing Bjøntegaard delta bit-rate (BD-BR) by 0.84%∼1.52% when implemented in VTM 10.0, outperforming the state-of-the-art methods, and that the proposed method can be used to accelerate VVenC 1.0 at the preset slower, achieving higher performance and lower complexity compared with the original VVenC 1.0 at the presets slow and medium.

HG-FCN: Hierarchical Grid Fully Convolutional Network for Fast VVC Intra Coding

As one of the key technologies of Versatile Video Coding (VVC), a flexible quad-tree with a nested multi-type tree (QTMT) partition structure significantly improves the rate-distortion (RD) performance. However, this structure brings additional complexity due to the recursive search for the best partition type. Traditional fast partition methods in previous encoders, cannot adapt to this new complex structure, because it’s too complicated to predict each block size from one layer to another layer. Some indirect bottom-up designed methods are simple enough, but cannot predict specific split structures, making the acceleration capacity limited. Therefore, in this paper, we propose a learning-based approach to effectively predict the QTMT structure without having to heuristically explore the partitions of each layer. Firstly, we propose a hierarchy grid fully convolutional network (HG-FCN) framework, which concisely requires inference only once to obtain the entire partition information of the current CU and sub-CUs, and the inference is highly parallel. Secondly, we design a representation of complicated QTMT of CU partition in the form of hierarchy grid map (HGM), which can directly and effectively predict the specific hierarchical split structure. Lastly, a dual-threshold decision scheme is adopted to automatically control the trade-off between coding performance and complexity. Extensive experiments demonstrate the effectiveness of HG-FCN, which can reduce 51.15% <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sim ~65.53$ </tex-math></inline-formula> % complexity of VVC intra coding with negligible 1.17% <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sim ~2.19$ </tex-math></inline-formula> % BD-BR increase, superior to other state-of-the-art methods.

SVM-Based Fast CU Partition Decision Algorithm for VVC Intra Coding

As a new coding standard, Versatile Video Coding (VVC) introduces the quad-tree plus multi-type tree (QTMT) partition structure, which significantly improves coding efficiency compared to High-Efficiency Video Coding (HEVC). The QTMT partition structure further enhances the flexibility of coding unit (CU) partitioning and improves the efficiency of VVC encoding high-resolution video, but introduces an unacceptable coding complexity at the same time. This paper proposes an SVM-based fast CU partition decision algorithm to reduce the coding complexity for VVC. First, the proportion of split modes with different CU sizes is analyzed to explore a method to effectively reduce coding complexity. Then, more reliable correlation features are selected based on the maximum ratio of the standard deviation (SD) and the edge point ratio (EPR) in sub-CUs. Finally, two SVM models are designed and trained using the selected features to provide guidance for deciding whether to divide and the direction of partition. The simulation results indicate that the proposed algorithm can save 54.05% coding time on average with 1.54% BDBR increase compared with VTM7.0.

Towards scalable and efficient Deep-RL in edge computing: A game-based partition approach

Currently, most edge-based Deep Reinforcement Learning (Deep-RL) applications have been deployed in the edge network, however, their mainstream studies are still short of adequate considerations on its limited compute and bandwidth resources. In this paper, we investigate the near on-policy of actions taking in distributed Deep-RL architecture, and propose a “hybrid near on-policy” Deep-RL framework, called Coknight, by leveraging a game-theory based DNN partition approach. We first formulate the partition problem into a variant of knapsack problem in device-edge setting, and then transform it into a potential game with a formal proof. Finally, we show the problem is NP-complete whereby an efficient distributed algorithm based on the potential game theory is developed from device perspective to achieve fast and dynamic partitioning. Coknight not only significantly improves the resource efficiency of the Deep-RL but also allows the inference to enforce the scalability of the actor policy. We prototype the framework with extensive experiments to validate our findings. The experimental results show that with the premise of a rapid convergence guarantee, Coknight, compared with Seed-RL, can reduce GPU utilization by 30% while providing large-scale scalability.

Configurable Fast Block Partitioning for VVC Intra Coding Using Light Gradient Boosting Machine

This article presents a configurable fast block partitioning decision for Versatile Video Coding (VVC) intra-frame prediction using Light Gradient Boosting Machine (LGBM). VVC further improves the coding efficiency by introducing a Quadtree with nested Multi-Type Tree (QTMT), enabling five split types allowing square and rectangular Coding Unit (CU) sizes. However, this improvement in the coding efficiency comes at the cost of a high computational burden since several combinations of block sizes and prediction modes are evaluated through the costly Rate-Distortion Optimization (RDO) process. In this article, we propose a partitioning decision using LGBM classifiers to avoid the exhaustive RDO process and skip the evaluation of split types that are unlikely to be chosen as the best one. For this purpose, five classifiers (one for each split type) were offline trained with an efficient training process and using effective features of texture, coding, and context information. The proposed solution is highly configurable and can provide several operation points with different tradeoffs between timesaving and coding efficiency, according to the application requirements. Considering five operation points, the configurable solution can reduce the encoding time from 35.22% to 61.34%, with coding efficiency losses from 0.46% to 2.43%. Compared to the state-of-the-art, our solution is able to outperform the related works in terms of combined rate-distortion and timesaving.