Cross-component Prediction Research Articles

Cross-Component Prediction Boosted With Local and Non-Local Information in Video Coding.

In the versatile video coding (VVC) standard, cross-component prediction (CCP) is introduced to utilize the correlation between luma and chroma components. To further exploit the cross-component redundancy, a bundle of novel CCP methods with more sophisticated models are studied in enhanced compression model (ECM), which is a platform targeting at the next generation video coding standard, developed by JVET. This paper presents two methods, known as local-boosting CCP (LB-CCP) and non-local CCP (NL-CCP) to improve CCP with local and non-local information. With LB-CCP, prediction samples of CCP can be filtered with neighbouring samples. Besides, neighbouring template costs are calculated to determine the range of training samples, as well as the cross-component prediction method used in the chroma fusion mode. With NL-CCP, a CCP model can be derived with samples non-adjacent to the current block. As an equivalent but simpler implementation, the CCP model can be inherited from a CCP-coded neighbouring block as a spatial CCP candidate. Moreover, the CCP model of a previous CCP-coded block can be stored in a history-based table, which can be fetched and used by the current block as a history-based CCP candidate. A NL-CCP candidate list is built with the two kinds of candidates. The encoder can select the optimal candidate and send an index to the decoder. Experimental results show that LB-CCP together with NL-CCP can provide an average Bjontegaard delta rate (BD-rate) reduction of 0.27%, 2.31%, 2.44% on Y, Cb, Cr components, respectively, with a negligible change in the running time, compared with ECM-8.0 in all intra configurations. Both LB-CCP and NL-CCP have been adopted into ECM.

A Discrete-Mapping-Based Cross-Component Prediction Paradigm for Screen Content Coding.

Cross-component prediction is an important intra-prediction tool in the modern video coders. Existing prediction methods to exploit cross-component correlation include cross-component linear model and its extension of multi-model linear model. These models are designed for camera captured content. For screen content coding, where videos exhibit different signal characteristics, a cross-component prediction model tailored to their characteristics is desirable. As a pioneering work, we propose a discrete-mapping based cross-component prediction model for screen content coding. Our model relies on the core observation that, screen content videos typically comprise of regions with a few distinct colors and luma value (almost always) uniquely conveys chroma value. Based on this, the proposed method learns a discrete-mapping function from available reconstructed luma-chroma pairs and uses this function to derive chroma prediction from the co-located luma samples. To achieve higher accuracy, a multi-filter approach is employed to derive co-located luma values. The proposed method achieves 2.61%, 3.51% and 3.92% Y, U and V bit-rate savings respectively over Enhanced Compression Model (ECM) 4.0, with negligible complexity, for text and graphics media under all-intra configuration.

Chroma Intra Prediction With Lightweight Attention-Based Neural Networks

Neural networks can be successfully used for cross-component prediction in video coding. In particular, attention-based architectures are suitable for chroma intra prediction using luma information because of their capability to model relations between different channels. However, the complexity of such methods is still very high and should be further reduced, especially for decoding. In this paper, a cost-effective attention-based neural network is designed for chroma intra prediction. Moreover, with the goal of further improving coding performance, a novel approach is introduced to utilize more boundary information effectively. In addition to improving prediction, a simplification methodology is also proposed to reduce inference complexity by simplifying convolutions. The proposed schemes are integrated into H.266/Versatile Video Coding (VVC) pipeline, and only one additional binary block-level syntax flag is introduced to indicate whether a given block makes use of the proposed method. Experimental results demonstrate that the proposed scheme achieves up to -0.46%/-2.29%/-2.17% BD-rate reduction on Y/Cb/Cr components, respectively, compared with H.266/VVC anchor. Reductions in the encoding and decoding complexity of up to 22% and 61%, respectively, are achieved by the proposed scheme with respect to the previous attention-based chroma intra prediction method while maintaining coding performance.

Sub-Sampled Cross-Component Prediction for Emerging Video Coding Standards.

Cross-component linear model (CCLM) prediction has been repeatedly proven to be effective in reducing the inter-channel redundancies in video compression. Essentially speaking, the linear model is identically trained by employing accessible luma and chroma reference samples at both encoder and decoder, elevating the level of operational complexity due to the least square regression or max-min based model parameter derivation. In this paper, we investigate the capability of the linear model in the context of sub-sampled based cross-component correlation mining, as a means of significantly releasing the operation burden and facilitating the hardware and software design for both encoder and decoder. In particular, the sub-sampling ratios and positions are elaborately designed by exploiting the spatial correlation and the inter-channel correlation. Extensive experiments verify that the proposed method is characterized by its simplicity in operation and robustness in terms of rate-distortion performance, leading to the adoption by Versatile Video Coding (VVC) standard and the third generation of Audio Video Coding Standard (AVS3).

Prediction With Multicross Component for Future Video Coding

There has been a consensus of cross-component prediction in video compression that the interchannel redundancies can be effectively removed through a linear prediction model. To fully explore the similarities existing in different color components, we propose prediction with multicross component (PMC) as a new chroma intracoding mode for the emerging video coding standard AVS3. The proposed PMC enjoys extremely low operational complexity and high flexibility, which advocates to reuse and reform the linear prediction model of conventional two-step cross-component prediction mode. Moreover, dual-prediction patterns of PMC are exploited to enhance the prediction efficiency. Experimental results on AVS3 test model HPM-6.0 show that PMC brings 0.32%, 1.60%, and 2.39% BD-rate savings for Y, Cb, and Cr components, respectively, under all intraconfiguration with natural-scene videos. Moreover, 0.91%, 2.24%, and 2.82% BD-rate gains can be observed on screen-content videos. Subset modes of PMC have been adopted by AVS3.

Intra-Frame Coding Using a Conditional Autoencoder

Exploiting spatial redundancy in images is responsible for a large gain in the performance of image and video compression. The main tool to achieve this is called intra-frame prediction. In most state-of-the-art video coders, intra prediction is applied in a block-wise fashion. Up to now angular prediction was dominant, providing a low-complexity method covering a large variety of content. With deep learning, however, it is possible to create prediction methods covering a wider range of content, being able to predict structures which traditional modes can not predict accurately. Using the conditional autoencoder structure, we are able to train a single artificial neural network which is able to perform multi-mode prediction. In this paper, we derive the approach from the general formulation of the intra-prediction problem and introduce two extensions for spatial mode prediction and for chroma prediction support. Moreover, we propose a novel latent-space-based cross component prediction. We show the power of our prediction scheme with visual examples and report average gains of 1.13% in Bjøntegaard delta rate in the luma component and 1.21% in the chroma component compared to VTM using only traditional modes.

Cross-Component Prediction in HEVC

Video coding in the YCbCr color space has been widely used, since it is efficient for compression, but it can result in color distortion due to conversion error. Meanwhile, coding in the RGB color space maintains high color fidelity, having the drawback of a substantial bitrate increase with respect to YCbCr coding. Cross-component prediction (CCP) efficiently compresses video content by decorrelating color components while keeping high color fidelity. In this scheme, the chroma residual signal is predicted from the luma residual signal inside the coding loop. This paper gives a description of the CCP scheme from several points of view, from theoretical background to practical implementation. The proposed CCP scheme has been evaluated in standardization communities and adopted into H.265/High Efficiency Video Coding (HEVC) Range Extensions. The experimental results show significant coding performance improvements for both natural and screen content video, while the quality of all color components is maintained. The average coding gains for natural video are 17% and 5% bitrate reduction in the case of intra coding and 11% and 4% in the case of inter coding for RGB and YCbCr coding, respectively, while the average increment of encoding and decoding times in the HEVC reference software implementation are 10% and 4%, respectively.

Sustainable Aspects Regarding a Multi-Criteria & Cross-Component Prediction of Property Change Potentials Within the Pre-Development Phase of Technical Product Systems

A required faster generating of concepts due to a significantly shorter pre-development time in the automotive industry without physical prototypes is a key factor to reach a higher product maturity more readily and quickly, especially in terms of reducing development costs but also environmental impacts. Therefore, and with regard to a more agile development process, tailored product property changes have to be considered right at the beginning.Concerning this matter, the contribution presents a multi-criteria and cross-component systemic and systematic matrix-based analysis process of property change potentials within the pre-development phase of technical products. Herein, and based on the development of weight-optimized systems, a continuous monitoring and tracking of certain modifications of diverse sustainability factors (social and environmental) can be easily made and partly be directly influenced by a corresponding redesign of relevant and hierarchically ordered components.

색 공간 내 중복 정보 감소를 위한 HEVC 스크린 콘텐츠 부호화 기법 성능 분석

최근 동영상 압축에 관한 국제 표준화 기구인 JCT-VC (Joint Collaborative Team on Video Coding) 에서는 High Efficiency Video Coding (HEVC)의 확장 표준으로 HEVC/Range Extension (HEVC/RExt)의 개발을 완료하고 스크린 콘텐츠 동영상의 부호화기술을 위한 표준인 HEVC/Screen Content Coding (HEVC/SCC)을 제정 중이다. 기존 동영상 압축 과정에서는 이미지 센서로부터 취득한 RGB 영상을 변환하여 부호화를 수행하는 반면에 애니매이션, 그래픽스 등 컴퓨터로 합성한 영상을 일컫는 스크린 콘텐츠의 경우는 색 공간의 변환이 주관적 화질을 심각하게 열화 시킬 수 있으므로 기존 RGB 색 공간을 유지하며 효율적으로 색 공간 내 정보의 중복성을 줄이기 위한 부호화 기법이 필요하다. 본 논문에서는 HEVC/RExt.와 HEVC/SCC에서 개발한 스크린 콘텐츠 동영상 압축을 위한 색 요소 예측 기법과 루프 내 색 공간 변환 기술의 성능을 분석한다. 실험 결과에 의하면 색 요소 예측 기법은 평균 약 11.7% BD-rate 감소, 색 공간 예측 기법은 평균 약 16.4% BD-rate 감소를 보인다. 그러나 두 기법이 동시에 적용되는 경우 약 18.2%의 BD-rate 감소를 보여 두 기법의 부호화 효율이 약 9.9% 중첩된다. 본 결과를 응용하여 두 기법이 배타적으로 선택이 되게끔 부호기 고속화를 수행하는 경우 약 0.3%의 부호화 손실로 93%의 부호화 측정 시간을 제공한다. JCT-VC (Joint Collaborative Team on Video Coding) continues developing HEVC/Screen Content Coding (HEVC/SCC) as an extension for efficiently coding screen content videos, including computer animations, graphics, and electrical documents, based on HEVC/Range extension (HEVC/RExt.). Color space conversion from RGB space being captured from CCD sensors is popular in natural video coding. However, the conversion is often undesired for screen contents because of a significant loss in perceptual quality. Therefore, several coding tools including cross-component prediction (CCP) and in-loop adaptive color space transform (ACST) have been developed for an efficient screen content video coding in order to reduce the redundancies between color spaces while maintaining the original color space. In this paper, we review the two coding tools, i.e., CCP and ACST exploiting the correlation in the RGB color space and conduct the performance analysis of the coding tools. In our simulation results, CCP and ACST provide 11.7% BD-rate saving and 16.4% BD-rate saving, respectively, while the two coding tools provide 18.2% BD-rate saving in total. Following this idea, if the two coding tools are exclusively selected, we provide 93% encoding measurement time with a 0.3% coding loss.