Screen Content Video Coding Research Articles

A CTU-Level Screen Content Rate Control for Low-Delay Versatile Video Coding

In this paper, a rate control scheme for screen content video coding is proposed for the Versatile Video Coding (VVC) standard. In view of the critical challenges arising from the spatial and temporal unnaturalness of screen content sequences, the proposed method relies on the specifically designed pre-analysis such that the content information regarding the scene complexity can be obtained. As such, the estimated residual complexity is then incorporated into the proposed complexity-aware rate models and distortion models, leading to the optimal bit allocations for each frame and coding tree unit (CTU). In particular, the optimization problem can be analytically solved with the proposed models, and the coding parameters such as Lagrangian multiplier λ and quantization parameter of each frame and CTU could be delicately derived according to the allocated bits through the proposed analytical models. Extensive experiments have been conducted to evaluate the effectiveness of the proposed method. Compared to the default hierarchical λ-domain rate control and other screen content rate control algorithms, the proposed method could achieve obvious RD performance gain, and the bit-rate accuracy could be improved.

A fast algorithm based on gray level co-occurrence matrix and Gabor feature for HEVC screen content coding

To reduce the computational complexity of screen content video coding (SCC), a fast algorithm based on gray level co-occurrence matrix and Gabor feature model for HEVC-SCC, denoted as GGM, is proposed in this paper. By studying the correlation of non-zero number in gray level co-occurrence matrix with different partitioning depth, the coding unit (CU) size of intra coding can be prejudged, which selectively skips the intra prediction process of CU in other depth. With Gabor filter, the edge information reflecting the features of screen content images to the human visual system (HVS) are extracted. According to Gabor feature, CUs are classified into natural content CUs (NCCUs), smooth screen content CUs (SSCUs) and complex screen content CUs (CSCUs), with which, the calculation and judgment of unnecessary intra prediction modes are skipped. Under all-intra (AI) configuration, experimental results show that the proposed algorithm GGM can achieve encoding time saving by 42.13% compared with SCM-8.3, and with only 1.85% bit-rate increasement.

Rate Control for Predictive Transform Screen Content Video Coding Based on RANSAC

In predictive transform video coding, optimal bit allocation and quantization parameter (QP) estimation are important to control the bit rate of blocks, frames and the whole sequence. Common solutions to this problem rely on trained models to approximate the rate-distortion (R-D) characteristics of the video content during coding. Moreover, these solutions are mainly targeted for natural content sequences, whose characteristics differ greatly from those of screen content (SC) sequences. In this article, we depart from such trained R-D models and propose a low-complexity RC method for SC sequences that leverages the availability of information about the R-D characteristics of previously coded blocks within a frame. Namely, our method first allocates bits at the frame- and block-levels based on their motion and texture characteristics. It then approximates the R-D and R-QP curves of each block by a set control points and random sample consensus (RANSAC). Finally, it computes the appropriate block-level QP values to attain a target bit rate with the minimum distortion possible. The proposed RC method is embedded into a standard High-Efficiency Video Coding (H.265/HEVC) encoder and evaluated on several SC sequences. Our results show that our method not only attains better R-D performance than that of H.265/HEVC and other methods designed for SC sequences but also attains a more constant and higher reconstruction quality on all frames.

A content-based rate control algorithm for screen content video coding

The popularity of 3D screen content videos like 3D animations calls for better coding performance of screen content videos. The exiting R-λ rate control model is designed to enable the coding performance of conventional nature videos, which can not accurately describe the Rate-Distortion (R-D) relationships of the screen content videos, especially for videos containing limited colors and sharper edges. This paper proposes a new rate control (RC) scheme which takes the divergent characteristics of text contents, screen images and nature images in screen content videos into consideration. In particular, the content-based rate control algorithm is developed at Coding Tree Unit (CTU) level, where CTUs are divided into three categories including Text-CTUs (T-CTUs), Screen Image-CTUs (SI-CTUs) and Nature Image-CTUs (NI-CTUs). Three R-λ models reflecting different R-D relationships of different CTUs are established, and the model parameters are updated on the basis of their own models. Furthermore, in view of the fact that continuity and mutability simultaneously exist in screen content videos, two bit allocation schemes for regions containing only one content are determined in this paper. Experimental results show that the proposed RC scheme achieves 0.749 dB BDPSNR increase under the Low Delay configuration and 0.637 dB BDPSNR increase under the Random Access configuration on average, compared to the default R-λ model in HEVC Screen Content Coding extension (HEVC-SCC) reference software.

Adaptive bit-plane selection-based low complexity motion estimation for screen content coding

Screen content video coding has become an emerging research topic with the spread of applications such as cloud gaming, screen/desktop virtualization, and mobile or external display interfacing. Screen content videos have different features compared to conventional camcorder-captured scenes. In this work, a novel low bit-depth representation-based motion estimation approach is proposed to exploit screen content specific features to improve coding efficiency. The proposed approach is based on an adaptive selection of Gray-coded bit-planes in order to generate low bit-depth representation of original screen content frames. The experimental results show that the motion estimation performance of the proposed approach is significantly better compared to the methods in the same category for screen content videos.

Enhanced Coding Tools and Algorithms for Screen Content Video : A Review

Screen content video is usually generated by a computer program and is displayed on a screen, containing a lot of discontinuous textures and edges that seldom happen in natural video, with highly correlated patches in many regions of the picture. Based on these differences from camera-captured video, screen content video coding was developed as an extension of High Efficiency Video Coding. Discussed in this work are some recently developed, enhanced coding tools and algorithms.

Overview of Screen Content Video Coding: Technologies, Standards, and Beyond

This paper presents recent advances in screen content video coding, with an emphasis on two state-of-the-art standards: HEVC/H.265 Screen Content Coding Extensions (HEVC-SCC) by ISO/IEC Moving Picture Experts Group and ITU-T Video Coding Experts Group, and Display Stream Compression (DSC) by Video Electronics Standards Association. The HEVC-SCC enhances the capabilities of HEVC in coding screen content, while DSC provides lightweight compression for display links. Although targeting different application domains, they share some design principles and are expected to become the leading formats in the marketplace in the coming years. This paper provides a brief account of their background, key elements, performance, and complexity characteristics, according to their final specifications. As we survey these standards, we also summarize prior arts in the last decade and explore future research opportunities and standards developments in order to give a comprehensive overview of this field.

Guest Editorial Screen Content Video Coding and Applications

This special issue aims to present recent technical advances in screen content video coding and applications. Screen content video has recently evolved from a niche to the mainstream due to the rapid advances in mobile and cloud technologies. Real-time, low-latency transport of screen visuals between devices in the form of screen content video is becoming prevalent in many applications, e.g., wireless displays, screen mirroring, display interfaces, screen/desktop virtualization and cloud-based mobile virtual reality. Today’s commonly-used video coding methods, however, have been developed primarily with camera-captured content in mind. These new applications create an urgent need for efficient coding of screen content video, especially as the support of 4k or even 8k resolution begins to achieve mass market appeal.

Efficient Residual DPCM Using an <inline-formula> <tex-math notation="LaTeX">$l_1$</tex-math> </inline-formula> Robust Linear Prediction in Screen Content Video Coding

In this paper, a residual differential pulse code modulation (RDPCM) coding technique using a weighted linear combination of neighboring residual samples is proposed to provide coding efficiency in the screen content video coding. The RDPCM performs the sample-based prediction of residue to reduce spatial redundancies. The proposed method uses the $l_1$ optimization in the weight derivation by considering the statistical characteristics of graphical components in videos in an intracoding. Specifically we use the least absolute shrinkage and selection operator to derive the weights because the solution is accurate in high variance residue. Furthermore, we enhance parallelism in a line processing by restricting the support to the row-wise prediction to above samples or the column-wise prediction to the left samples. The proposed method uses an explicit RDPCM scheme, so a coding mode determined by rate-distortion optimization is transmitted to a decoder. For coding the overhead, we develop a context design in CABAC based on correlation between an intraprediction direction and an RDPCM prediction mode. It is demonstrated with the experimental results that the proposed method provides a significant coding gain over the state-of-the-art reference codec for screen content video coding.

Structured sparse representation of residue in screen content video coding

A novel transform method providing a regionally structured sparse model of residue for a video coding is proposed. Conventional learning-based video coding methods consider only the number of transformed coefficients in a dictionary construction but disregard any structural form of residue. A structural form of coefficients is used for another factor, specified by a set of local block patterns, in a transform design so as to yield the regionally compact distribution of the coefficients. A transform coefficient coding method is also developed because the distribution of the coefficients is different with that of conventional transforms. It is demonstrated with experiments that the proposed method outperforms the coding efficiency and visual quality over high efficiency video coding/screen content coding reference software.

스크린 콘텐츠 비디오의 압축을 위한 인접 화소의 가중 합을 이용한 적응적 Residual DPCM 기법

lx'`yz$ -ze`{('IJ|l$g. p01G}=> !~€E ‚3.1% o' k%g. AbstractIn this paper, we propose a novel residual differential pulse-code modulation (RDPCM) coding technique to improve coding efficiency of screen content videos. The proposed method uses a weighted combination of adjacent residues to provide an accurate estimate in RDPCM. The weights are trained in previously coded samples by using an L1 optimization problem with the least absolute shrinkage and selection operation (LASSO). The proposed method achieves BD-rate saving about 3.1% in all-intra coding. Keyword : HEVC, screen content video coding, RDPCM

Sample selective filter in HEVC intra‐prediction for screen content video coding

An efficient screen content video coding method to adaptively turn on/off filters used for intra-prediction is proposed. A block is classified into two categories based on a statistical property of a block and a transform coding mode in the compressed domain. Specifically, the Mahalanobis distance (MD) between the estimated probability density distribution and prediction samples in the left and top boundaries of the block to be coded and its transform coding mode is used. The set of the filters are disabled depending on the condition. Experimental results demonstrated that the proposed method outperforms the state-of-the-art video coding standard, i.e. the high-efficiency video coding/range extension (HEVC/RExt), in terms of rate-distortion (R-D) performance.