Intra Prediction Research Articles

MetaIP: Meta-Network-Based Intra Prediction With Customized Parameters for Video Coding

MetaIP: Meta-Network-Based Intra Prediction With Customized Parameters for Video Coding

Complexity Reduction in Intra Prediction of HEVC Using a Modified Convolutional Neural Network Model Incorporating Depth Map and RGB Texture

Complexity Reduction in Intra Prediction of HEVC Using a Modified Convolutional Neural Network Model Incorporating Depth Map and RGB Texture

Learning-Based Fast Splitting and Directional Mode Decision for VVC Intra Prediction

Learning-Based Fast Splitting and Directional Mode Decision for VVC Intra Prediction

A multi-embedding domain video steganography algorithm based on TU partitioning and intra prediction mode

With High Efficiency Video Coding (HEVC) becoming a popular video coding standard in the world, combing the HEVC with data hiding methods is a complex task. In this paper, a multi-embedded video steganography algorithm based on Intra Prediction Mode (IPM) and Transform Unit (TU) partition is proposed. By analyzing the coding characteristics of the TU partition and the IPM, we select embedded regions that are relatively independent as much as possible. Then, suitable distortion functions are designed for different embedding domains. Using the minimum distortion framework of the STC model to embed messages. The experimental results indicate that the proposed method achieves a maximum embedding capacity 1.5 times larger than that of traditional single domain embedding methods, while maintaining good visual quality.

Acceleration of an Optimized Kvazaar All Intra Prediction on Embedded Systems Based on the Directional Texture Complexity

Acceleration of an Optimized Kvazaar All Intra Prediction on Embedded Systems Based on the Directional Texture Complexity

Fast Mode and CU Splitting Decision for Intra Prediction in VVC SCC

Fast Mode and CU Splitting Decision for Intra Prediction in VVC SCC

Chroma Intra Prediction With Lightweight Attention-Based Neural Networks

Neural networks can be successfully used for cross-component prediction in video coding. In particular, attention-based architectures are suitable for chroma intra prediction using luma information because of their capability to model relations between different channels. However, the complexity of such methods is still very high and should be further reduced, especially for decoding. In this paper, a cost-effective attention-based neural network is designed for chroma intra prediction. Moreover, with the goal of further improving coding performance, a novel approach is introduced to utilize more boundary information effectively. In addition to improving prediction, a simplification methodology is also proposed to reduce inference complexity by simplifying convolutions. The proposed schemes are integrated into H.266/Versatile Video Coding (VVC) pipeline, and only one additional binary block-level syntax flag is introduced to indicate whether a given block makes use of the proposed method. Experimental results demonstrate that the proposed scheme achieves up to -0.46%/-2.29%/-2.17% BD-rate reduction on Y/Cb/Cr components, respectively, compared with H.266/VVC anchor. Reductions in the encoding and decoding complexity of up to 22% and 61%, respectively, are achieved by the proposed scheme with respect to the previous attention-based chroma intra prediction method while maintaining coding performance.

Aligned Intra Prediction and Hyper Scale Decoder Under Multistage Context Model for JPEG AI

Aligned Intra Prediction and Hyper Scale Decoder Under Multistage Context Model for JPEG AI

Enhancing HEVC Compression: A Novel Algorithm for Swift and Efficient Intra Prediction

High-Efficiency Video Coding (HEVC) represents the cutting-edge in video compression standards for high-resolution visual data. The prowess of HEVC lies in its utilization of prediction, capitalizing on signal redundancies to attain optimal compression efficiency. A key feature, the Intra-Picture prediction block, anticipates a block within the present frame by referencing information from adjacent blocks within that same frame. Nonetheless, this innovation introduces a significant surge in encoding complexity. Addressing this surge and reducing the computational demands of the coder becomes imperative. The present research introduces an innovative algorithm targeting mode decisions grounded on intra prediction. This algorithm assesses the texture intricacy of the coding unit and establishes an optimal threshold, enabling swift coding unit size selection. Subsequently, an enhanced search procedure minimizes the candidate modes of prediction. Culminating this approach, early termination gets implemented for rate-distortion optimized quantization (RDOQ) within Intra prediction. Findings indicate that the novel algorithm facilitates a reduction of approximately 40.9% in encoding duration.

High Performance VLSI Architecture for DC and Planar Modes of Intra Prediction in HEVC

Abstract: High Efficiency Video Coding (HEVC) achieves improved compression efficiency, but it introduces higher computational complexity due to intricate partitioning and increased angular modes in intra prediction. In this study, we propose a hardware architecture for handling the direct current (DC) and planar modes of intra prediction in the HEVC standard. To evaluate our proposed architecture, we performed synthesis using both TSMC 180nm and TSMC 90nm technologies. We later conducted physical implementation using Physical Cell 90nm technology. The results show significant improvements when moving from TSMC 180nm to TSMC 90nm technology, with the chip area reducing by approximately three times and power consumption decreasing by over six times. After completing the physical layout phase, we obtained a chip area of 112.19 mm2 and a power consumption of 3.88 mW. Comparing the synthesis results with the physical design phase, we observed a slight increase in chip area by around 1.5 times, while the power consumption decreased by approximately 0.4 times. The proposed architecture achieves a throughput of 27 pixels per clock cycle and supports a block size of 16x16, with the potential for further extension

A Fragile Watermarking Method for Content-Authentication of H.264-AVC Video

This work proposes a blind fragile GOP-based watermarking technique that ensures the H.264-AVC video's content authenticity. In the H.264 video, the self-authentication code is created by securely hashing a number of intra prediction modes of I-frames in addition to motion vectors of P-frames and B-frames. The resulting authentication code is subsequently injected into the IPCM-Least block's Significant Bits (LSB) of some I-frames' luma and chroma pixel samples. To manage and regulate authentication data and watermark processes (embedding, detection, and verification), a key based on video footage and a secret key has been created. Intra prediction modes and motion vectors from the bitstream can be partially decoded to detect and confirm the hidden watermark data without the need for the original movie or complete video decoding. Several experiments were conducted to assess the sensitivity of the proposed method against signal processing, rate control and transcoding, conventional GOP-based and Frame-based attacks. Experimental simulations demonstrate that the implanted fragile watermark is sensitive to several low to harmful and content-preserving attacks. With only a modest bit-rate increase and hardly any perceptual quality deterioration, a large payload capacity is achieved. Experimental results show that the proposed method is suburb in term of imperceptibility, and lower effect on bit-rate. Applications like tamper-proofing, content-authentication, and covert communication are ideally suited for this technique. Based on achieved results, the devised method is capable of detecting any form of spatial and/or temporal manipulations which make it ideal for real-time applications.

Deep Intra Prediction by Jointly Exploiting Local and Non-Local Similarities

Intra prediction, which aims to remove the redundancies within a frame, has shown promising performance by simply projecting and interpolating samples along multiple angular directions. Recently, with numerous approaches devoted to learning nonlinear predictors with deep neural networks (DNN) based on local correlations, much less work has been dedicated to exploring non-local self-similarities in intra prediction. In this paper, we propose a unified prediction model that exploits both local and non-local correlations for intra prediction. The proposed model not only supports the nonlinear prediction using local reference samples as input, but also aggregates useful non-local information from a large reconstructed region with a Patch-level Non-local Attention Network (PNA-Net). More specifically, PNA-Net incorporates template matching with attention mechanism in feature domain to obtain the responses of all non-local features to the content to be predicted, leading to the prediction produced with weighted non-local patches. Finally, the predictions in the local and non-local manners are blended adaptively with a trainable network, ensuring the capability to handle a variety of contents. Experimental results on Versatile Video Coding (VVC) software VTM-11.0 show that the proposed model achieves on average 4.69% bit rate savings for natural scene sequences, and 4.24% bit rate savings for screen content sequences under the all intra configuration.

Local Geometry-Based Intra Prediction for Octree-Structured Geometry Coding of Point Clouds

Point cloud compression (PCC) is crucial for efficient and flexible storage as well as feasible transmission of point clouds in practice. For geometry compression, one popular approach is the octree-based solution. The intra prediction mechanism utilizes the spatial correlation in the static point cloud to predict the occupancy bit of the octree node for entropy coding, reducing the spatial redundancy. In this study, two local geometry-based prediction methods are proposed following statistical and theoretical analyses: binary prediction, which outputs the binary state (i.e., occupied or unoccupied), and ternary prediction, which provides a third option other than occupied or unoccupied (i.e., not predicted). In comparison to the state-of-the-art, the proposed binary prediction offers the Bjontegaard delta rate (BD-rate) of −0.8% for lossy compression and the bits per input point (bpip) of 100.09% for lossless compression in average, respectively. The binary prediction reduces the computational complexity in terms of more than 20% decrease in decoding time. In particular, it also provides noticeable reduction of the memory usage during entropy coding. The proposed ternary prediction provides −1.2% BD-rate for lossy compression and 97.19% bpip for lossless compression in average, respectively, in comparison to the state-of-the-art. While achieving performance gain, it is considerably more computational efficient by saving about 18% decoding time. Due to these advantages, part of the proposed ternary prediction has been adopted by the ongoing MPEG standard of geometry-based point cloud compression (G-PCC).

Frequency-Based Adaptive Interpolation Filter in Intra Prediction

This paper proposes a method to improve the fractional interpolation of reference samples in the Versatile Video Coding (VVC) intra prediction. The proposed method uses additional interpolation filters which use more integer-positioned reference samples for prediction according to the frequency information of the reference samples. In VVC, a 4-tap Discrete Cosine Transform-based interpolation filter (DCT-IF) and 4-tap Smoothing interpolation filter (SIF) are alternatively performed on the block size and block directional prediction mode for reference sample interpolation. This paper uses four alternative interpolation filters such as 8-tap/4-tap DCT-IFs, and 4-tap/8-tap SIFs and an interpolation filter selection method using a high-frequency ratio calculated from one-dimensional (1D) transform of the reference samples are proposed. The proposed frequency-based Adaptive Filter allows to achieve the overall Bjøntegaard Delta (BD) rate gains of −0.16%, −0.13%, and −0.09% for Y, Cb, and Cr components, respectively, compared with VVC.

Tree-Structured Data Clustering-Driven Neural Network for Intra Prediction in Video Coding.

Intra prediction is a crucial part of video compression, which utilizes local information in images to eliminate spatial redundancy. As the state-of-the-art video coding standard, Versatile Video Coding (H.266/VVC) employs multiple directional prediction modes in intra prediction to find the texture trend of local areas. Then the prediction is made based on reference samples in the selected direction. Recently, neural network-based intra prediction has achieved great success. Deep network models are trained and applied to assist the HEVC and VVC intra modes. In this paper, we propose a novel tree-structured data clustering-driven neural network (dubbed TreeNet) for intra prediction, which builds the networks and clusters the training data in a tree-structured manner. Specifically, in each network split and training process of TreeNet, every parent network on a leaf node is split into two child networks by adding or subtracting Gaussian random noise. Then data clustering-driven training is applied to train the two derived child networks using the clustered training data of their parent. On the one hand, the networks at the same level in TreeNet are trained with non-overlapping clustered datasets, and thus they can learn different prediction abilities. On the other hand, the networks at different levels are trained with hierarchically clustered datasets, and thus they will have different generalization abilities. TreeNet is integrated into VVC to assist or replace intra prediction modes to test its performance. In addition, a fast termination strategy is proposed to accelerate the search of TreeNet. The experimental results demonstrate that when TreeNet is used to assist the VVC Intra modes, TreeNet with depth = 3 can bring an average of 3.78% bitrate saving (up to 8.12%) over VTM-17.0. If TreeNet with the same depth replaces all VVC intra modes, an average of 1.59% bitrate saving can be reached.

Multi-Channel HEVC Steganography by Minimizing IPM Steganographic Distortions

Calibration is a common method for steganalysis, and Intra Prediction Mode (IPM) shift is a typical phenomenon used in calibration to detect video steganography. The current HEVC steganography lacks resistance to steganalysis based on this phenomenon because the new technology of HEVC introduces steganographic distortion in addition to providing more potential steganographic space. In this paper, an HEVC steganographic algorithm that resists IPM shift is proposed. First, we introduce the IPM shift in HEVC, and the previous H.264 steganalytic IPM shift feature is modeled and improved. By analyzing the HEVC encoding process, we found that modifying large-size blocks has a more significant impact on compression efficiency, while small ones are more sensitive to IPM optimality. Therefore, we perform the embedding channel division based on the block size and design the distortion function separately. In addition, we discover a unique IPM transition probability distribution in HEVC. According to our analysis, this unique distribution arises due to HEVC's MPM rules and the regularity of IPM direction. Modifying IPM in HEVC will change such distribution, thus, a mapping rule is designed based on this distribution to achieve a better embedding effect. Experimental results show that the channel division and proposed distortion function can effectively improve the overall performance. The proposed steganography outperforms the state-of-the-art steganography in resisting steganalysis, bitrate controlling, and visual quality.

Modelling of Depth Prediction Algorithm for Intra Prediction Complexity Reduction

Video compression gained its relevance with the boon of the internet, mobile phones, variable resolution acquisition device etc. The redundant information is explored in initial stages of compression that’s is prediction. Inter prediction that is prediction within the frame generates high computational complexity when working with traditional signal processing procedures. The paper proposes the design of a deep convolutional neural network model to perform inter prediction by crossing out the flaws in the traditional method. It briefs the modeling of network, mathematics behind each stage and evaluation of the proposed model with sample dataset. The video frame’s coding tree unit (CTU) of 64x64 is the input, the model converts and store it as a 16-element vector with the goodness of CNN network. It gives an overview of deep depth decision algorithm. The evaluation process shows that the model performs better for compression with less computational complexity.

Intra Prediction Method for Depth Video Coding by Block Clustering through Deep Learning

In this paper, we propose an intra-picture prediction method for depth video by a block clustering through a neural network. The proposed method solves a problem that the block that has two or more clusters drops the prediction performance of the intra prediction for depth video. The proposed neural network consists of both a spatial feature prediction network and a clustering network. The spatial feature prediction network utilizes spatial features in vertical and horizontal directions. The network contains a 1D CNN layer and a fully connected layer. The 1D CNN layer extracts the spatial features for a vertical direction and a horizontal direction from a top block and a left block of the reference pixels, respectively. 1D CNN is designed to handle time-series data, but it can also be applied to find the spatial features by regarding a pixel order in a certain direction as a timestamp. The fully connected layer predicts the spatial features of the block to be coded through the extracted features. The clustering network finds clusters from the spatial features which are the outputs of the spatial feature prediction network. The network consists of 4 CNN layers. The first 3 CNN layers combine two spatial features in the vertical and horizontal directions. The last layer outputs the probabilities that pixels belong to the clusters. The pixels of the block are predicted by the representative values of the clusters that are the average of the reference pixels belonging to the clusters. For the intra prediction for various block sizes, the block is scaled to the size of the network input. The prediction result through the proposed network is scaled back to the original size. In network training, the mean square error is used as a loss function between the original block and the predicted block. A penalty for output values far from both ends is introduced to the loss function for clear network clustering. In the simulation results, the bit rate is saved by up to 12.45% under the same distortion condition compared with the latest video coding standard.

Geometric Partitioning Mode with Inter and Intra Prediction for Beyond Versatile Video Coding

Geometric Partitioning Mode with Inter and Intra Prediction for Beyond Versatile Video Coding

Fusion-Based Versatile Video Coding Intra Prediction Algorithm with Template Matching and Linear Prediction.

The new generation video coding standard Versatile Video Coding (VVC) has adopted many novel technologies to improve compression performance, and consequently, remarkable results have been achieved. In practical applications, less data, in terms of bitrate, would reduce the burden of the sensors and improve their performance. Hence, to further enhance the intra compression performance of VVC, we propose a fusion-based intra prediction algorithm in this paper. Specifically, to better predict areas with similar texture information, we propose a fusion-based adaptive template matching method, which directly takes the error between reference and objective templates into account. Furthermore, to better utilize the correlation between reference pixels and the pixels to be predicted, we propose a fusion-based linear prediction method, which can compensate for the deficiency of single linear prediction. We implemented our algorithm on top of the VVC Test Model (VTM) 9.1. When compared with the VVC, our proposed fusion-based algorithm saves a bitrate of 0.89%, 0.84%, and 0.90% on average for the Y, Cb, and Cr components, respectively. In addition, when compared with some other existing works, our algorithm showed superior performance in bitrate savings.