Coding Unit Research Articles

Low-complexity intra coding scheme based on Bayesian and L-BFGS for VVC

H.266/Versatile Video Coding (VVC) is the latest generation of video coding standards. Compared with H.265/High Efficiency Video Coding (HEVC), the number of intra modes expanded from 35 to 67, and the Quad Tree with nested Multi type Tree (QTMT) split structure is adopted. These encoding tools endow VVC with better compression performance at the cost of significantly increased computational complexity. To reduce computational complexity, a Low-complexity intra coding scheme is proposed in this paper, which includes two parts: fast Coding Unit (CU) split decision and fast intra mode decision. Firstly, a Bayesian-based fast CU split decision method is proposed for skipping vertical split and terminating split. This method improves the accuracy of probability estimation by combining parameter eigenvectors and non-parametric eigenvectors of multiple features. Secondly, a fast intra mode decision method based on Limit-Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) is proposed. This method reduces the number of Rough Mode Decision (RMD) and Rate Distortion Optimization (RDO) by searching for angle mode in a certain direction. Experimental results showed that the proposed scheme reduces the encoding time by 59.52%, while the BDBR increases by 1.13%.

RDNet: Rate–Distortion-Based Coding Unit Partition Network for Intra-Prediction

High efficiency video coding (HEVC) has been finalized as the most widely utilized video coding standard, jointly developed by ITU-T, VCEG, and MPEG. In HEVC, the quad-tree structure of the coding unit partition is one of the most substantial modules and provides significant coding gains following huge coding time. In this paper, a rate–distortion-based coding unit partition network (RDNet) is proposed to make partition decisions based on the statistical features. RDNet is composed of a prediction sub-network and a target sub-network, where the prediction sub-network is used to predict the CU partition modes of the intra-prediction and the target sub-network is designed to optimize the network parameters by evaluating the rate–distortion cost, respectively. To balance the prediction accuracy and the rate–distortion loss, a parameter-exchanging strategy is applied to control the parameters’ sharing between two networks. Experimental results prove that our model can reduce the encoding time of HEVC by 55.83~71.72% with an efficient BD-BR of 2.876~3.347%, and the ablation study evaluates the ability of our strategy on balancing the trade-off between coding accuracy and inference speed.

VR‐Based improvement of aerobics teaching quality

Currently, most of students do not pay much attention to aerobics. Affected by various factors, students' participation in aerobics learning is low and lack of interest. However, regular aerobics exercise can enhance physical fitness and improve physical and mental health. Compared with other sports, aerobics is not so boring, and the exercise effect is excellent. The introduction of virtual reality (VR) technology in aerobics teaching not only makes up for the deficiency of traditional aerobics teaching, but also brings students a great sense of novelty, and teachers' teaching efficiency and students' learning efficiency will be significantly improved. The empirical variogram‐based 360° aerobics teaching video coding unit (CU) partition algorithm is proposed to improve coding efficiency and provide real visual experience. Experiments show that the proposed algorithm performs well in BD‐Rate and Quality of Experience (QoE), so that students can adjust movements according to the standard aerobics movements, which will greatly improve the efficiency of aerobics teaching.

Fast coding unit size decision based on deep reinforcement learning for versatile video coding

Fast coding unit size decision based on deep reinforcement learning for versatile video coding

Fast Depth Intra Coding Based on Depth Edge Classification Network in 3D-HEVC

As the extension of high efficiency video coding (HEVC) standard, three dimensional-HEVC (3D-HEVC) is the latest 3D video coding standard. 3D-HEVC adopts many complicated coding algorithms to generate additional intermediate views for 3D video representation, which result in extremely high coding complexity. Therefore, this paper proposed a fast depth intra coding approach to reduce the 3D-HEVC complexity, which is based on convolutional neural network (CNN). First, we established a database based on the independent view of the depth map, which includes coding unit (CU) partition data of the depth map. Second, we constructed a depth edge classification CNN (DEC-CNN) framework to classify the edges for the depth map and embedded the network into a 3D-HEVC test platform. Finally, we utilized the pixel value of the binarized depth image to correct the above classification results. The experimental results demonstrated that our approach can reduce the intra coding time by 72.5% on average under negligible degradation of coding performance. This result outperforms the other state-of-the-art methods to reduce the coding complexity of 3D-HEVC.

Tunable early CU size decision for depth map intra coding in 3D-HEVC using unsupervised learning

To further improve coding gain for depth map in 3D extension of High Efficiency Video Coding (3D-HEVC), many new coding techniques are introduced, which drastically increase the coding complexity. In this work, an early intra coding unit (CU) size decision scheme is proposed for 3D-HEVC intra depth map coding based on unsupervised learning approach. First, we treat the early CU size decision as a clustering problem. Then, three clustering models are developed for those CUs with the sizes of 64×64, 32×32 and 16×16 to early determine whether they be further split or not. Among them, the center of the clustering model is obtained by using intra learning method. Finally, in order to meet the user's specific application preference, the similarity distance is introduced into the early CU size decision. By adjusting the similarity distance, a tunable early CU size decision is achieved to obtain different levels of coding complexity reduction. Experimental results show that the proposed scheme achieves encoding time reduction ranging from 54.0% to 68.4% on average for depth map intra coding, while the Bjontegaard delta bit rate (BDBR) of synthesized views only increased by 0.01% to 0.73%, it outperforms the state-of-the-art works in term of coding complexity reduction and BDBR increase.

The impact of hardware specifications on reaching quantum advantage in the fault tolerant regime

We investigate how hardware specifications can impact the final run time and the required number of physical qubits to achieve a quantum advantage in the fault tolerant regime. Within a particular time frame, both the code cycle time and the number of achievable physical qubits may vary by orders of magnitude between different quantum hardware designs. We start with logical resource requirements corresponding to a quantum advantage for a particular chemistry application, simulating the FeMo-co molecule, and explore to what extent slower code cycle times can be mitigated by using additional qubits. We show that in certain situations, architectures with considerably slower code cycle times will still be able to reach desirable run times, provided enough physical qubits are available. We utilize various space and time optimization strategies that have been previously considered within the field of error-correcting surface codes. In particular, we compare two distinct methods of parallelization: Game of Surface Code's Units and AutoCCZ factories. Finally, we calculate the number of physical qubits required to break the 256-bit elliptic curve encryption of keys in the Bitcoin network within the small available time frame in which it would actually pose a threat to do so. It would require 317 × 106 physical qubits to break the encryption within one hour using the surface code, a code cycle time of 1 μs, a reaction time of 10 μs, and a physical gate error of 10−3. To instead break the encryption within one day, it would require 13 × 106 physical qubits.

A Comprehensive Investigation on Emotional Detection in Deep Learning

Emotion recognition is a substantial problem in the field of Big Data. In a wide range of applications, reliable categorization, analysis, and interpretation of emotional content is greatly desired. In this paper, we look at how Deep Learning models perform on an emotion perception test. Facial Expression Recognition (FER) plays an important role in machine learning tasks. Deep Learning models do well in FER tasks, but they lack explanation for their conclusions. Based on the notion that facial expression is a mixture of facial muscle movements, we discover a link between Facial Action Coding Units (AUs) and Emotion label in the CK+ Dataset. In this study, we offer a model that uses AUs to explain the classification outcomes of a Convolutional Neural Network (CNN) model. The CNN model is trained using the CK+ Dataset and identifies emotions using extracted characteristics. The CNN model's retrieved features and emotion classes are used by the explanation model to classify multiple AUs. Explanation model creates AUs quite effectively with only characteristics and emotion classes acquired from the CNN model, according to our trial. Experimental research was constructed, and several deep learning approaches were tested using publically available datasets. The findings are highly intriguing, highlighting the benefits of each strategy and training style studied.

Efficient coding unit classifier for HEVC screen content coding based on machine learning

Efficient coding unit classifier for HEVC screen content coding based on machine learning

Early Termination of CU Partition Based on Boosting Neural Network for 3D-HEVC Inter-Coding

As an extension of the High Efficiency Video Coding (HEVC) standard, the 3D-HEVC needs to encode multiple texture videos and depth maps components. In the 3D-HEVC inter-coding test model, a large variety of Coding Unit (CU) sizes are adopted to select the one with the lowest Rate-Distortion (RD) cost as the best CU size. This technique provides the highest achievable coding efficiency, but it brings a huge computational complexity which limits 3D-HEVC from practical applications. In this paper, early termination of CU encoding is proposed to reduce the complexity caused by the CU size splitting process. The proposed algorithm is based on CU homogeneity and a boosting neural network clustering algorithm. The algorithm contained three main steps. The first step is for the extraction of various features from the original encoder. Then, the selection of the features, which had a high correlation with CU partition using a machine learning algorithm. In the second step, a boosting neural network model is used for training the selected features to derive the threshold values for our proposed algorithm. In the final step, an efficient early termination of CU splitting is released for texture videos and depth maps based on the extracted thresholds from the training model. The experimental results show that the proposed algorithm reduces a significant encoding time, while the loss in coding efficiency is negligible.

High Efficiency Intra CU Partition and Mode Decision Method for VVC

Versatile video coding (VVC/H.266) is the newest video compression standard, which is developed by the Joint Video Experts Team. Compared with previous encoding schemes, VVC achieves higher compression efficiency by introducing a new partition structure and additional intra prediction modes but results in high computational complexity. To efficiently solve this problem of redundant processing in quad-tree with nested multi-type tree structures and intra mode prediction, we propose a texture analysis–based ternary tree (TT) and binary tree (BT) partition strategy, and a gradient-based intra mode decision method to accelerate TT and BT partition and intra mode prediction, separately. The texture complexity and prediction direction of coding unit (CU) is calculated by texture detection method. A texture analysis–based TT and BT partition strategy is established by using the regression method based on analyzing the texture complexity of the CU. Then, a texture analysis–based TT and BT partition strategy is applied to reduce the redundant partition for each CU. By using the prediction direction of CU, a gradient-based intra mode decision method is established for skipping the impossible modes for each CU. Experimental results revealed that the proposed method could save 49.49% in encoding time and increase the Bjontegaard delta bit rate (BDBR) by only 0.56%. It confirms that the proposed method achieved high efficiency and a good balance between the BDBR and time saving.

Notice of Violation of IEEE Publication Principles: VVC/H.266 Intra Mode QTMT Based CU Partition Using CNN

Notice of Violation of IEEE Publication Principles <br><br>“VVC/H. 266 Intra Mode QTMT Based CU Partition Using CNN” <br>by Sameena Javaid, Safdar Rizvi, Muhammad Talha Ubaid, Abdullah Tariq <br>in IEEE Access, April 2022 <br><br>After careful and considered review of the content and authorship of this paper by a duly constituted expert committee, this paper has been found to be in violation of IEEE’s Publication Principles. <br><br>This paper copied content from the article below. The original text was copied without attribution (including appropriate references to the original author(s) and/or paper title) and without permission. <br><br> “DeepQTMT: A Deep Learning Approach for Fast QTMT-Based CU Partition of Intra-Mode VVC” <br> by Tianyi Li, Mai Xu, Runzhi Tang, Ying Chen, Qunliang Xing <br>in IEEE Transactions on Image Processing, Vol 30, 2021 <br><br> <br/> The latest standard for video coding is versatile video coding (VVC) / H.266 which is developed by the joint video exploration team (JVET). Its coding structure is a multi-type tree (MTT) structure, which consists of two types of trees that are Ternary Tree (TT) and Binary Tree (BT). Due to the use of brute force quest for residual rate-distortion the quad-tree and multi-type tree (QTMT) structure of the coding unit (CU) split and contributes over 98% of the encoding time. This structure is efficient in coding, however, increases computational complexity. The current paper proposes a deep learning technique to predict the QTMT based CU split rather than just the brute-force QTMT method to substantially speed up the time of the encoding process for VVC/H.266 intra mode. In the first phase, we developed an extensive database containing ample CU splitting patterns and various streaming videos. In the second phase, we suggest a multi-level exit CNN (MLE-CNN) model with a redundancy removal mechanism at different levels to determine the CU partition. In the third phase, for the training of MLECNN model we have established the adaptive loss function and analyzing the both unknown number of partition modes and the focus on RD cost minimization. Finally, a variable threshold decision system is established to achieve the targeted low complexity and RD performance. Ultimately experimental findings show that VVC/H.266 encoding time has reduced to 69.11% from 47.91% with insignificant bjontegaard delta bit rate (BDBR) to 2.919% from 1.023% which performs better than the existing futuristic and modern approaches.

HEVC Video Encryption With High Capacity Message Embedding by Altering Picture Reference Indices and Motion Vectors

A high capacity message embedding in encrypted HEVC video is proposed in this paper. The challenges addressed in this paper include keeping the encrypted video compliant with standardized decoders, correctly decrypting the video and finally, correctly extracting the message bits. The message embedding is achieved by altering the values of reference picture indices and motion vectors which results in scrambled video. Sixteen picture references are used in this work and therefore, combined with alteration of motion vectors, a maximum of six message bits can be embedded per coding unit. Motion vectors are altered by swapping their <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$x$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$y$ </tex-math></inline-formula> components and/or changing their signs. This is achieved with full compliance with the HEVC video syntax. To extract message bits, an authorized decoder builds a classification model per video sequence and uses it for predicting the true values of the reference indices and motion vectors. As such, message bits are extracted and the video is correctly reconstructed to its unscrambled state. Coding units that result in misclassification are identified at the encoder and excluded from message embedding. This results in slightly lower embedding rates but ensures accurate video reconstruction. Using nine video sequences of various resolutions that are compressed using four different quantization parameters, the experimental results revealed that the true average message embedding rate is 2.7 bits per coding unit or 173 kbit/s. This is achieved with accurate video reconstruction at the expense of increasing the bitrate of the encoder by 3%. Comparison with existing work shows that the proposed solution is superior in terms of embedding capacity whilst reducing the excessive bitrate of the encoder.

Teacher-led versus student-led seminar blended with portfolio for "assessment of learning": An interventional study.

Increase in competitive demands has led to the promotion of seminar presentation by the students to increase active learning and for the assessment of learning. Portfolios are an important tool for assessment for learning. The objectives of the study were to compare the gain in knowledge among the conventional and the blended seminar groups (intervention group), analyze the working portfolios to assess for learning, and to gather the perception of students. This was an interventional study (2016) with convenience sampling that included 27 students each in the conventional and interventional groups. It was conducted in a medical college in central India. The conventional group was teacher led and the blended seminar group was student led with interaction with the students regarding the contents of the portfolio. Student's unpaired and paired t tests were used for statistical analysis. P < 0.05 was considered as the level of significance. Class average normalized gain (g) was used as a measure of effectiveness of the intervention. Quantitative questions were analyzed using percentages and qualitative data using categorization. Significant difference was found between the conventional and intervention groups (P < 0.05) with gain "g" being 0.52 for the intervention group. On evaluation of the feedback, students commented regarding its interactive nature and progress during the learning process. The reflections were coded as text as the unit of coding and student as the unit of coding. It was also found that the students who were critical reflectors were the ones who scored > 50% in the posttest scores. The present study showed that seminar when blended with portfolio yielded positive results in the process of learning, and hence was effective in assessment for learning.

Enhanced Intra String Copy for Screen Content Coding in AVS3

Driven by growing applications that use computer screens as interfaces for daily remote interactions, almost all current video coding standards have included screen content coding (SCC) tools. Recently, an efficient SCC tool called intra string copy (ISC) has been adopted in the third-generation of audio video coding standard in China (AVS3). ISC has two coding unit (CU) level sub-modes: fully-matching-string and partially-matching-string based string prediction (FPSP) sub-mode and equal-value-string, unit-basis-vector-string, and unmatched-pixel-string based string prediction (EUSP) sub-mode. To further improve the coding efficiency of SCC, this paper proposes four enhancement techniques of ISC (EISC), including CU partition improvements, point vector (PV) relocation and reactivation, line-based overlapping string prediction, and an optimized coding method for string length in the EUSP sub-mode. Compared with the latest AVS3 reference software HPM with EISC disabled, using AVS3 SCC common test condition and YUV test sequences in text and graphics with motion and mixed content categories, the proposed technique achieves an average Y BD-rate reduction of 2.39% and 1.49% for all intra (AI) and low-delay B (LDB) configurations, respectively, with low additional encoding complexity and almost no additional decoding complexity. All proposed ISC enhancement techniques have been adopted in AVS3.

Software Defect Prediction Harnessing on Multi 1-Dimensional Convolutional Neural Network Structure

Developing successful software with no defects is one of the main goals of software projects. In order to provide a software project with the anticipated software quality, the prediction of software defects plays a vital role. Machine learning, and particularly deep learning, have been advocated for predicting software defects, however both suffer from inadequate accuracy, overfitting, and complicated structure. In this paper, we aim to address such issues in predicting software defects. We propose a novel structure of 1-Dimensional Convolutional Neural Network (1D-CNN), a deep learning architecture to extract useful knowledge, identifying and modelling the knowledge in the data sequence, reduce overfitting, and finally, predict whether the units of code are defects prone. We design large-scale empirical studies to reveal the proposed model's effectiveness by comparing four established traditional machine learning baseline models and four state-of-the-art baselines in software defect prediction based on the NASA datasets. The experimental results demonstrate that in terms of f-measure, an optimal and modest 1D-CNN with a dropout layer outperforms baseline and state-of-the-art models by 66.79% and 23.88%, respectively, in ways that minimize overfitting and improving prediction performance for software defects. According to the results, 1D-CNN seems to be successful in predicting software defects and may be applied and adopted for a practical problem in software engineering. This, in turn, could lead to saving software development resources and producing more reliable software.

Algorithms for building and operation modeling of large electrical circuits with memristor-diode crossbars in a biomorphic neuroprocessor

The biomorphic neuroprocessor is the hardware implementation of the impulse neural network in which incoming information from a set of numbers is converted into impulses, and outgoing information, on the contrary, from impulses into binary code. For the automatic building of electrical circuits of the input coding and output decoding units in neuroprocessor using ultra-large logic matrices based on a memristor-diode crossbar, appropriate algorithms have been developed. For the subsequent imitation modeling of information processing in these units, as well as in the memory matrix of the neuroprocessor, the algorithm for calculating large electrical circuits containing memristor-diode crossbars has been created. This simulation algorithm is based on the well-known algorithm of Simulation Program with Integrated Circuit Emphasis and includes original mathematical models of the memristor and the selective element of the Zener diode, as well as the algorithm for modeling the resistive switching of the memristor. The results of imitation modeling using the developed algorithms and corresponding programs showed the operability of the constructed electrical circuits of the input unit in the mode of encoding a binary number into a impulse frequency by a population of three neurons and the output unit of a neuroprocessor that decodes the impulses coming from neurons into binary format as well as the operability of the memory matrix under weighting and summing impulses. The created algorithms and programs package based on them can be used to effectively solve the engineering and technical problem of manufacturing a biomorphic neuroprocessor that requires modeling of information processing in individual neuroprocessor units based on large memristor-diode arrays in order to optimize their parameters.

Test Case Prioritization Using Partial Attention

Test case prioritization (TCP) aims to reorder the regression test suite with a goal of increasing the fault detection rate. Various TCP techniques have been proposed based on different prioritization strategies. Among them, the greedy-based techniques are the most widely-used TCP techniques. However, existing greedy-based techniques usually reorder all candidate test cases in prioritization iterations, resulting in both efficiency and effectiveness problems. In this paper, we propose a generic partial attention mechanism, which adopts the previous priority values (i.e., the number of additionally-covered code units) to avoid considering all candidate test cases. Incorporating the mechanism with the additional-greedy strategy, we implement a novel coverage-based TCP technique based on partition ordering (OCP). OCP first groups the candidate test cases into different partitions and updates the partitions on the descending order. We conduct a comprehensive experiment on 19 versions of Java programs and 30 versions of C programs to compare the effectiveness and efficiency of OCP with six state-of-the-art TCP techniques: total-greedy, additional-greedy, lexicographical-greedy, unify-greedy, art-based, and search-based. The experimental results show that OCP achieves a better fault detection rate than the state-of-the-arts. Moreover, the time costs of OCP are found to achieve 85%-99% improvement than most state-of-the-arts.

A Complexity Reduction Method for VVC Intra Prediction Based on Statistical Analysis and SAE-CNN

Compared with High Efficiency Video Coding (HEVC), the latest video coding standard Versatile Video Coding Standard (VVC), due to the introduction of many novel technologies and the introduction of the Quad-tree with nested Multi-type Tree (QTMT) division scheme in the block division method, the coding quality has been greatly improved. Due to the introduction of the QTMT scheme, the encoder needs to perform rate–distortion optimization for each division mode during Coding Unit (CU) division, so as to select the best division mode, which also leads to an increase in coding time and coding complexity. Therefore, we propose a VVC intra prediction complexity reduction algorithm based on statistical theory and the Size-adaptive Convolutional Neural Network (SAE-CNN). The algorithm combines the establishment of a pre-decision dictionary based on statistical theory and a Convolutional Neural Network (CNN) model based on adaptively adjusting the size of the pooling layer to form an adaptive CU size division decision process. The algorithm can make a decision on whether to divide CUs of different sizes, thereby avoiding unnecessary Rate–distortion Optimization (RDO) and reducing coding time. Experimental results show that compared with the original algorithm, our suggested algorithm can save 35.60% of the coding time and only increases the Bjøntegaard Delta Bit Rate (BD-BR) by 0.91%.

Fast CU Size Decision Method Based on Just Noticeable Distortion and Deep Learning

With the development of broadband networks and high-definition displays, people have higher expectations for the quality of video images, which also brings new requirements and challenges to video coding technology. Compared with H.265/High Efficiency Video Coding (HEVC), the latest video coding standard, Versatile Video Coding (VVC), can save 50%-bit rate while maintaining the same subjective quality, but it leads to extremely high encoding complexity. To decrease the complexity, a fast coding unit (CU) size decision method based on Just Noticeable Distortion (JND) and deep learning is proposed in this paper. Specifically, the hybrid JND threshold model is first designed to distinguish smooth, normal, or complex region. Then, if CU belongs to complex area, the Ultra-Spherical SVM (US-SVM) classifiers are trained for forecasting the best splitting mode. Experimental results illustrate that the proposed method can save about 52.35% coding runtime, which can realize a trade-off between the reduction of computational burden and coding efficiency compared with the latest methods.