Video Compression Technology Research Articles

Understanding MPEG-I Coding Standardization in Immersive VR/AR Applications

After decennia of developing leading-edge 2D video compression technologies, the Moving Picture Expert Group (MPEG) is currently working on the new era of coding for immersive applications, referred to as MPEG-I, where “I” refers to the “Immersive” aspects. It ranges from 360° video with head-mounted displays to free navigation in 3D space with head-mounted and 3D light field displays. Two families of coding approaches, covering typical industrial workflows, are currently considered for standardization—MultiView + Depth (MVD) Video Coding and Point Cloud Coding—both supporting high-quality rendering at bitrates of up to a couple of hundreds of megabits per second. This paper provides a technical/historical overview of the acquisition, coding, and rendering technologies considered in the MPEG-I standardization activities.

Analysis of Emerging Video Codecs: Coding Tools, Compression Efficiency

The video compression community, from technologists to customers, is affected by the competition between two lines of development for future video compression technology beyond high-efficiency video coding (HEVC): AV1, as developed by the industry-driven Alliance for Open Media (AOM), and Versatile Video Coding (VVC), as developed and studied by the Joint Video Experts Team (JVET) on Future Video Coding of ITU-T Video Coding Expert Group (VCEG) and International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) Motion Picture Experts Group (MPEG). Both groups have asserted that their respective codecs, AV1 and VVC, outperform the state-of-the-art HEVC. However, in many cases, related studies are evaluating video codec performances based on different test conditions, scenarios, or codec implementations, which makes comparisons sometimes unfair and at least difficult to interpret. This paper provides a neutral and well-balanced evaluation of compression and runtime performance for both AV1 and VVC codecs, using their respective reference models, in comparison to two HEVC codec implementations: HEVC Test Model (HM) as the HEVC standard reference model and x265 as a well-known open-source HEVC codec implementation. Comparison of differentiating features of AV1 and VVC over HEVC is also reported. Our main findings show that, based on an objective analysis, AV1 allows bitrate reduction of 10% in comparison to HM for the same peak signal-to-noise ratio (PSNR) and Structural SIMilarity (SSIM) quality scores while increasing encoding runtime by 55.5x. However, we show that this objective advantage in coding efficiency does not necessarily translate into gain in terms of subjective quality assessment. Overall, objective results show VVC to be the best candidate for compression technology beyond HEVC.

Novel algorithms for protective digital privacy

Video is the recording, reproducing, or broadcasting of moving visual images. Visual multimedia source that combines a sequence of images to form a moving picture. The video transmits a signal to a screen and processes the order in which the screen captures should be shown. Videos usually have audio components that correspond with the pictures being shown on the screen. Video compression technologies are about reducing and removing redundant video data so that a digital video file can be effectively sent over a network and stored on computer disks. With efficient compression techniques, a significant reduction in file size can be achieved with little or no adverse effect on the visual quality. The video quality, however, can be affected if the file size is further lowered by raising the compression level for a given compression technique. Security is about the protection of assets. Security, in information technology <a href="http://searchdatacenter.techtarget.com/definition/IT">(IT), </a>is the defense of digital information and IT assets against internal and external, malicious and accidental threats. This defense includes detection, prevention and response to threats through the use of <a href="http://searchsecurity.techtarget.com/definition/security-policy">security policies, </a>software tools and IT services. Security refers to protective digital privacy measures that are applied to prevent unauthorized access to computers, databases and websites. Cryptography is closely related to the disciplines of <a href="http://searchsecurity.techtarget.com/definition/cryptology">cryptology </a>and <a href="http://searchsecurity.techtarget.com/definition/cryptanalysis">cryptanalysis. </a>Cryptography includes techniques such as microdots, merging words with images, and other ways to hide information in storage or transit. However, in today's computer-centric world, cryptography is most often associated with scrambling <a href="http://searchsecurity.techtarget.com/definition/plaintext">plaintext </a>(ordinary text, sometimes referred to as clear text into <a href="http://searchcio-midmarket.techtarget.com/definition/ciphertext">cipher text </a>(a process called <a href="http://searchsecurity.techtarget.com/definition/encryption">encryption), </a>then back again (known as decryption). Cryptography is evergreen and developments. Cryptography protects users by providing functionality for the encryption of data and authentication of other users. Compression is the process of reducing the number of bits or bytes needed to represent a given set of data. It allows saving more data. The project aims to implement security algorithm for data security. The data will be first encrypted using security techniques and that are done at the same time then it takes less processing time and more speed compression techniques will applied. If encryption and compression are done at the same time then it takes less processing time and more speed.

Parallel Full Search Algorithm for Motion Estimation on Graphic Processing Unit

Background: Advances in video compression technology have been driven by everincreasing processing power available in software and hardware. Methods: The emerging High-Efficiency Video Coding (HEVC) standard aims to provide a doubling in coding efficiency with respect to the H.264/AVC high profile, delivering the same video quality at half the bit rate. Results: Thus, the results show high computational complexity. In both standards, the motion estimation block presents a significant challenge in clock latency since it consumes more than 40% of the total encoding time. For these reasons, we proposed an optimized implementation of this algorithm on a low-cost NVIDIA GPU developed with CUDA language. Conclusion: This optimized implementation can provide high-performance video encoder where the speed reaches about 85.

Multimedia Technology Aids College Art Teaching Research

Social development has gradually entered the digital age and the advanced results of modern information technology have been widely applied in people’s daily life. In computer multimedia technology, key technologies include: streaming media technology, virtual reality technology, video compression technology, multimedia database technology, audio technology and video on demand technology. In the current art teaching work of colleges and universities, the role of digital technology such as computer multimedia has become increasingly prominent. Multimedia-assisted art teaching in colleges and universities has become one of the important directions to expand the art teaching resources of colleges and universities. This paper discusses computer multimedia technology, analyzes the key technologies of computer multimedia and discusses the practical application of computer multimedia technology. Aimed at the influence of computer multimedia technology products on college art teaching, how to use computer multimedia technology products to help colleges and universities improve the art teaching effect. The system design principle and module function of computer multimedia assisted art teaching in colleges and universities are studied. The new art teaching and learning mode involved in this system are conducive to enhancing the learning ability and artistic quality of college students.

Reversible Data Hiding Using Inter-Component Prediction in Multiview Video Plus Depth

With the advent of 3D video compression and Internet technology, 3D videos have been deployed worldwide. Data hiding is a part of watermarking technologies and has many capabilities. In this paper, we use 3D video as a cover medium for secret communication using a reversible data hiding (RDH) technology. RDH is advantageous, because the cover image can be completely recovered after extraction of the hidden data. Recently, Chung et al. introduced RDH for depth map using prediction-error expansion (PEE) and rhombus prediction for marking of 3D videos. The performance of Chung et al.’s method is efficient, but they did not find the way for developing pixel resources to maximize data capacity. In this paper, we will improve the performance of embedding capacity using PEE, inter-component prediction, and allowable pixel ranges. Inter-component prediction utilizes a strong correlation between the texture image and the depth map in MVD. Moreover, our proposed scheme provides an ability to control the quality of depth map by a simple formula. Experimental results demonstrate that the proposed method is more efficient than the existing RDH methods in terms of capacity.

Performance Analysis of OLSR Routing Protocol for Video Streaming Over Mobile Ad Hoc Networks.

In view of the increasing use of mobile devices in addition to the high demand for the applications provided by these devices including video streaming, most companies have tended to pay attention to the mobile ad hoc networks and search for solutions to the problems and obstacles that hindered the process of sending video in this type of networks. One of the solutions discussed: Solutions at the level of video compression technology where many of the standards that were used in the process of video encoding and that provide good video quality by using a few bits in the coding process as provided us with acceptable bandwidth for the user and are flexible in handling with errors. These standards are: "H. 264/ MPEG-4 part 10". There was also a solution at the level of routing during the transmission of video in real time over the mobile ad hoc networks so in this paper has been studied OLSR routing protocol that support the transmission of video on the basis of delay, network load and throughput and evaluated the performance by changing the model where the node is located (Node Placement Model) in a large network and in a small network as well as for different video resolutions.

Objective Video Quality Assessment Combining Transfer Learning With CNN.

Nowadays, video quality assessment (VQA) is essential to video compression technology applied to video transmission and storage. However, small-scale video quality databases with imbalanced samples and low-level feature representations for distorted videos impede the development of VQA methods. In this paper, we propose a full-reference (FR) VQA metric integrating transfer learning with a convolutional neural network (CNN). First, we imitate the feature-based transfer learning framework to transfer the distorted images as the related domain, which enriches the distorted samples. Second, to extract high-level spatiotemporal features of the distorted videos, a six-layer CNN with the acknowledged learning ability is pretrained and finetuned by the common features of the distorted image blocks (IBs) and video blocks (VBs), respectively. Notably, the labels of the distorted IBs and VBs are predicted by the classic FR metrics. Finally, based on saliency maps and the entropy function, we conduct a pooling stage to obtain the quality scores of the distorted videos by weighting the block-level scores predicted by the trained CNN. In particular, we introduce a preprocessing and a postprocessing to reduce the impact of inaccurate labels predicted by the FR-VQA metric. Due to feature learning in the proposed framework, two kinds of experimental schemes including train-test iterative procedures on one database and tests on one database with training other databases are carried out. The experimental results demonstrate that the proposed method has high expansibility and is on a par with some state-of-the-art VQA metrics on two widely used VQA databases with various compression distortions.

HDR video quality assessment: Perceptual evaluation of compressed HDR video

Compared with standard dynamic range (SDR) video, the high dynamic range (HDR) video can provide us significantly enhanced viewing experience. In particular, compared to SDR video, the HDR video has better contrast and preserves more details for the same scene. With the rapid development of HDR video compression technology, there is a lack of trusted quality measure of HDR video compression. In order to facilitate the future development of objective HDR quality assessment, we build a HDR video quality assessment database, in which the bitstream is created by compressing a series of HDR video sequences. In the compression, the quantization parameters (QP) are set to 12 levels according to the configuration of the codec. The subjective quality of each bitstream is rated by 22 viewers. It is revealed that the subject viewers have arrived at a reasonable agreement on the subjective quality of different QP levels. This paper presents the results of subjective quality assessment of HDR compressed video, which also exhibits that there is significant room to further improve the objective HDR video quality assessment algorithms.

MPEG Internet Video Coding Standard and Its Performance Evaluation

MPEG has produced standards that have provided the industry with the best video compression technologies. To address diverse Internet needs, MPEG issued a Call for Proposals (CfP) for Internet video coding (IVC) in July, 2011. The anticipation is that any patent declaration associated with the baseline profile of this standard will indicate that the patent owner is prepared to grant a free of charge license to an unrestricted number of applicants worldwide. Three codecs have responded to the CfP: Web video coding (WVC), video coding for browsers (VCB), and IVC. WVC is in fact the AVC baseline, and VCB uses the same coding tools as VP8. IVC has been developed in MPEG from scratch by combining well-known existing technology elements and new coding tools with royalty-free declarations. In June 2015, the IVC project was approved as ISO/IEC 14496–33 (MPEG-4 IVC). This standard can be highly beneficial for video services in the Internet domain. This paper describes the main coding tools used in IVC, and evaluates its objective and subjective performances compared with WVC, VCB, and AVC high profile (AVC HP). The experimental results show that IVC’s compression performance is approximately equal to that of the AVC HP for typical operational settings, both for streaming and low-delay applications, and is superior to WVC and VCB.

Video big data in smart city: Background construction and optimization for surveillance video processing

Transforming infrastructures, buildings and services with the sensed data from the Internet of Things (IoT) technique has drawn wide attention. Enormous video data from city surveillance cameras poses huge challenges of transmission, storage and analysis, which necessitates new video compression technologies. The fusion of video data generated from smart city could be used to support city management and urban policy. Based on the specific characteristics of surveillance video, which are successive pictures have very strong correlations and each picture can be divided into background and foreground, this work proposes a block-level background modeling (BBM) algorithm to support long-term reference structure for efficient surveillance video coding. A rate–distortion optimization for surveillance source (SRDO) algorithm is also developed to improve the coding performance. Experimental results show that the proposed BBM and SRDO can significantly improve the compression performance, which can effectively support diverse video applications in smart city.

Hybrid Motion Estimation Algorithm Based on PSO with Dynamic Threshold on Static Block Detection

Motion Estimation(ME) technology is a platform in Video compression technology and so it is most complex process in video encoder. The challenge in the ME is to detect the best Motion Vector (MV) in less number of search points in the search plane. Pattern oriented search provides significant reduction on search points, especially the Hexagonal Search(HS) is much better, but last two decades the video codec’s demands acceleration on ME process to lighter encoder to adapt for even small scale hard cores. So there is necessary of optimization on search process. The proposed work optimize the ME process based on Particle Swarm Optimization PSO, and to improve performance of PSO its hybrid with HS. Also in this paper a method is proposed to detect the static blocks in image sequence by dynamic threshold. The PSNR and average search points per block are two metrics to measure the performance and its compared with popular search pattern techniques. The hybrid technique reduce the search points to 70% with the PSNR loss of 0.8dB. This method could be adapted to any video codec, to speed its ME process.

Context-aware block-based motion estimation algorithm for multimedia internet of things (IoT) platform

Shaping video data into fast-responding transmission and high resolution output video using cost-effective video processing is desirable in many applications including Internet of Things (IoT) applications. In association with rapid development of IoT smart sensor applications, real-time processing of huge-amount of data for a video signal has become necessary leading to video compression technology. Motion estimation (ME) is necessary for improving the quality, but it has high computational complexity in video compression system. The present article, therefore, proposes a context-aware adaptive pattern-based ME algorithm for multimedia IoT platform to improve video compression. In the proposed algorithm, the motions are classified into large or small based on distortion value. Accordingly, the search pattern is chosen either small diamond search pattern (SDSP) or large diamond search pattern (LDSP) in each and every step of ME; allowing adaptive processing of large and small abstract information. Compared to conventional fast algorithms, the experimental results demonstrate up to 40 and 36% reduction in encoding time for low-delay main (LB-main) and random access main (RA-main) profiles respectively in HEVC test model 16.10 encoder with bit-rate loss of 0.071 and 0.246% for both the profiles, ensuring quality video and searching precision.

Quality of experience measurement of compressed multi-view video

Quality of Experience (QoE) measurement in stereoscopic 3D is a challenging topic and existing standards such as ITU-R BT.500 may no longer be adequate for reliability and efficiency. Yet, objective measurement of QoE is not well researched for 3D images and videos. We present a new subjective assessment methodology providing better time-efficiency and more accurate scores at the same or even less number of assessors, using new objective metrics for QoE measurement of compressed multi-view video. Our research focus is twofold. First, the novel subjective testing methodology is developed, and QoE of different multi-view video compression technologies is evaluated. A time-efficient solution is proposed as a multiple stimuli plus simultaneous presentation extension of the conventional Double Stimulus Continuous Quality Scale (DSCQS) method. Second, objective quality metrics are proposed based on structural similarity and depth maps. Experiments are conducted on an auto-stereoscopic display while H.264/MPEG-4 Advanced Video Coding (AVC)- and High Efficiency Video Coding (HEVC)-based encoders are used for compression. The results demonstrate that the new methodology is superior to the DSCQS method in terms of both evaluation time and accuracy. Furthermore, high correlation is achieved with the new objective quality metrics.

Deep Learning Can Be Crucial for Smart Consumer Electronics [Notes from the Editor

I would like to introduce the current issue with the announcement of a couple of items of good news. First, IEEE Consumer Electronics Magazine is now indexed in DBLP. I think this will enhance this publication's visibility. In effect, it may attract a wider variety of authors from the computer science and computer engineering fields. Second, three members of the IEEE Consumer Electronics (CE) Society were recently awarded the title of IEEE Fellow. The three members and their contributions to CE are as follows: Petronel Bigioi for leadership in advancing digital-imaging technology for handheld consumer devices Michael Isnardi for contributions to compliance testing and vision-based video-compression technologies Shaomin Peng for contributions to scalable video-decoding methods in the MPEG2 standard. Please join me in congratulating them for their noteworthy honor and achievements.

Fast and effective CU size decision based on spatial and temporal homogeneity detection

High-Efficiency Video Coding (HEVC) is the latest video coding standard of the Joint Collaborative Team on Video Coding (JCT-VC). HEVC noticeably improves compression performance when compared with previous standards such as H264, and represents a major step forward in video compression technology. However, this improvement is achieved by increasing the complexity of the encoding process. HEVC employs a novel flexible quad-tree coding block partitioning structure that enables the use of large and multi-sized coding, prediction, and transform blocks. This system is more efficient but also more computationally demanding. In this article an optimized CU size decision algorithm is proposed to reduce the computational cost of quad-tree partitioning by means of spatial and temporal homogeneity analysis and classification, which are directly applied to the input image. If a CU is classified as spatially or temporally homogeneous the quad-tree recursive process is stopped. Furthermore, this image pre-analysis is performed using logic units and embedded hardware on a GPU, thus avoiding unnecessary waiting states, so the computational cost associated with this process is zero for the processor in charge of the encoding process. In comparison with the reference HM16.2 test model, the encoding time is reduced by up to 32.69%, with negligible quality loss and a maximum BD-Rate increase of 1.2% for low-delay P configuration.

Medical Ultrasound Video Coding with H.265/HEVC Based on ROI Extraction.

High-efficiency video compression technology is of primary importance to the storage and transmission of digital medical video in modern medical communication systems. To further improve the compression performance of medical ultrasound video, two innovative technologies based on diagnostic region-of-interest (ROI) extraction using the high efficiency video coding (H.265/HEVC) standard are presented in this paper. First, an effective ROI extraction algorithm based on image textural features is proposed to strengthen the applicability of ROI detection results in the H.265/HEVC quad-tree coding structure. Second, a hierarchical coding method based on transform coefficient adjustment and a quantization parameter (QP) selection process is designed to implement the otherness encoding for ROIs and non-ROIs. Experimental results demonstrate that the proposed optimization strategy significantly improves the coding performance by achieving a BD-BR reduction of 13.52% and a BD-PSNR gain of 1.16 dB on average compared to H.265/HEVC (HM15.0). The proposed medical video coding algorithm is expected to satisfy low bit-rate compression requirements for modern medical communication systems.

Bit Rate Requirements for High Dynamic Range Video

High dynamic range (HDR) video can offer consumers a much improved viewing experience compared to current broadcast video. The dynamic range of current television images, referred to as standard dynamic range (SDR), is governed by cathode ray tube physics first documented about eighty years ago. The standards include the Electro-Optical Transfer Function (EOTF) and the Opto-Electrical Transfer Function (OETF), as defined in International Telecommunications Union Radiocommunication (ITU-R) Recommendations BT.1886 and ITU-R BT.709, respectively. Alternative transfer functions have been defined to support the transmission and rendering of HDR video signals. These transfer functions aim to provide perceptually uniform mapping of video signals to the higher luminance range of HDR displays while maintaining video signal bit depths used across current broadcast infrastructures. In consequence, these new transfer functions exhibit much higher non-linearity compared to the transfer functions used in today's SDR systems. This could lead to several implications, such as an increase in bit rates required to transmit HDR services; changes to the existing broadcast infrastructure, including graphics equipment and vision mixers; and the compatibility of HDR services with existing SDR displays. This paper studies the impact of such transfer functions on the efficiency of the video compression used for delivery to the final user. Results, in terms of compression efficiency and subjective picture quality, using single-layer High Efficiency Video Coding (HEVC, also known as H.265 and MPEG-H Part 2) video compression algorithms are presented. This seeks to answer the question of what bit rates will be required to provide HDR services using existing video compression technology.

CUQI: cardiac ultrasound video quality index.

Medical images and videos are now increasingly part of modern telecommunication applications, including telemedicinal applications, favored by advancements in video compression and communication technologies. Medical video quality evaluation is essential for modern applications since compression and transmission processes often compromise the video quality. Several state-of-the-art video quality metrics used for quality evaluation assess the perceptual quality of the video. For a medical video, assessing quality in terms of "diagnostic" value rather than "perceptual" quality is more important. We present a diagnostic-quality-oriented video quality metric for quality evaluation of cardiac ultrasound videos. Cardiac ultrasound videos are characterized by rapid repetitive cardiac motions and distinct structural information characteristics that are explored by the proposed metric. Cardiac ultrasound video quality index, the proposed metric, is a full reference metric and uses the motion and edge information of the cardiac ultrasound video to evaluate the video quality. The metric was evaluated for its performance in approximating the quality of cardiac ultrasound videos by testing its correlation with the subjective scores of medical experts. The results of our tests showed that the metric has high correlation with medical expert opinions and in several cases outperforms the state-of-the-art video quality metrics considered in our tests.

Efficient Bluetooth Video Transmission with Resource Manager

A scheme of wireless real video transmission with optimized Quality-of-service is proposed in this paper. It combines H.264 video compression technology with Bluetooth technology to reduce demand of bandwidth and power consumption. Resource manager (RM) is used to smooth video output traffic and to partially eliminate peak of video stream transmitted in Bluetooth channel. The RM monitors status of video transmitters, manipulates and coordinates the being encoded video, and adopts traffic credits to support real-time video services. Measured results show that a maximal speed of transmission can be achieved with a best suitable ACL packets type, rather than the maximum payload packet, according to SNR to achieve. The system can support real-time video transmission with good quality. And more, it can be realized easily in mobile application for its good transportability and robustness.