Improved Compression Performance Research Articles

Learned Video Compression with Adaptive Temporal Prior and Decoded Motion-aided Quality Enhancement

Learned video compression has drawn great attention and shown promising compression performance recently. In this article, we focus on the two components in the learned video compression framework, the conditional entropy model and quality enhancement module, to improve compression performance. Specifically, we propose an adaptive spatial-temporal entropy model for image, motion, and residual compression, which introduces a temporal prior to reduce temporal redundancy of latents and an additional modulated mask to evaluate the similarity and perform refinement. In addition, a quality enhancement module is proposed for predicted frame and reconstructed frame to improve frame quality and reduce the bitrate cost of residual coding. The module reuses decoded optical flow as a motion prior and utilizes deformable convolution to mine high-quality information from the reference frame in a bit-free manner. The two proposed coding tools are integrated into a pixel-domain residual coding–based compression framework to evaluate their effectiveness. Experimental results demonstrate that our framework achieves competitive compression performance in the low-delay scenario compared with recent learning-based methods and traditional H.265/HEVC in terms of Peak Signal-to-Noise Ratio (PSNR) and Multi-Scale Structural Similarity Index (MS-SSIM). The code is available at OpenLVC.

Preparation of tomato peel pomace powder/polylactic acid foams under supercritical CO2 conditions: Improvements in cell structure and foaming behavior

Polylactic acid (PLA) is an eco-friendly material that can help address the problems of petroleum depletion and pollution. Blending renewable biomass materials with PLA to create composite foams with a tunable pore structure, superior performance, and low cost is a green technique for improving the pore structure and mechanical characteristics of single PLA foams. PLA/TP composites were created using melted tomato peel pomace powder (TP), which has a lamellar structure, as a reinforcing agent. Then, the relationship between the vesicle structure, morphology, and properties of the PLA/TP composite foams produced through supercritical CO2 intermittent foaming were investigated. The findings revealed that TP considerably enhanced the rheological characteristics and crystalline behavior of PLA. The PLA/TP composite foam had a better cell structure, compression characteristics, and wettability than pure PLA. The expansion ratio of the PLA/TP composite could reach 18.8, and its thermal conductivity decreased from 174.2 mW/m·K at 100 °C to 57.8 mW/m·K at 120 °C. Furthermore, annealing before foaming decreased the average composite foam blister size from 110.09 to 66.53 μm, and the annealing process also improved compression performance. This study contributes to solving environmental difficulties and creating PLA foams with controlled bubble structures, uniform bubble sizes, and outstanding overall performance.

End-to-End Light Field Image Compression with Multi-Domain Feature Learning

Recently, end-to-end light field image compression methods have been explored to improve compression efficiency. However, these methods have difficulty in efficiently utilizing multi-domain features and their correlation, resulting in limited improvement in compression performance. To address this problem, a novel multi-domain feature learning-based light field image compression network (MFLFIC-Net) is proposed to improve compression efficiency. Specifically, an EPI-based angle completion module (E-ACM) is developed to obtain a complete angle feature by fully exploring the angle information with a large disparity contained in the epipolar plane image (EPI) domain. Furthermore, in order to effectively reduce redundant information in the light field image, a spatial-angle joint transform module (SAJTM) is proposed to reduce redundancy by modeling the intrinsic correlation between spatial and angle features. Experimental results demonstrate that MFLFIC-Net achieves superior performance on MS-SSIM and PSNR metrics compared to public state-of-the-art methods.

LSSVC: A Learned Spatially Scalable Video Coding Scheme.

Traditional block-based spatially scalable video coding has been studied for over twenty years. While significant advancements have been made, the scope for further improvement in compression performance is limited. Inspired by the success of learned video coding, we propose an end-to-end learned spatially scalable video coding scheme, LSSVC, which provides a new solution for scalable video coding. In LSSVC, we propose to use the motion, texture, and latent information of the base layer (BL) as interlayer information for compressing the enhancement layer (EL). To reduce interlayer redundancy, we design three modules to leverage the upsampled interlayer information. Firstly, we design a contextual motion vector (MV) encoder-decoder, which utilizes the upsampled BL motion information to help compress high-resolution MV. Secondly, we design a hybrid temporal-layer context mining module to learn more accurate contexts from the EL temporal features and the upsampled BL texture information. Thirdly, we use the upsampled BL latent information as an interlayer prior for the entropy model to estimate more accurate probability distribution parameters for the high-resolution latents. Experimental results show that our scheme surpasses H.265/SHVC reference software by a large margin. Our code is available at https://github.com/EsakaK/LSSVC.

Perceptual Video Coding for Machines via Satisfied Machine Ratio Modeling.

Video Coding for Machines (VCM) aims to compress visual signals for machine analysis. However, existing methods only consider a few machines, neglecting the majority. Moreover, the machine's perceptual characteristics are not leveraged effectively, resulting in suboptimal compression efficiency. To overcome these limitations, this paper introduces Satisfied Machine Ratio (SMR), a metric that statistically evaluates the perceptual quality of compressed images and videos for machines by aggregating satisfaction scores from them. Each score is derived from machine perceptual differences between original and compressed images. Targeting image classification and object detection tasks, we build two representative machine libraries for SMR annotation and create a large-scale SMR dataset to facilitate SMR studies. We then propose an SMR prediction model based on the correlation between deep feature differences and SMR. Furthermore, we introduce an auxiliary task to increase the prediction accuracy by predicting the SMR difference between two images in different quality. Extensive experiments demonstrate that SMR models significantly improve compression performance for machines and exhibit robust generalizability on unseen machines, codecs, datasets, and frame types. Code is available at https://github.com/ywwynm/SMR.

Using automated software evaluation to improve the performance of breast radiographers in tomosynthesis screening

ObjectiveTo improve breast radiographers’ individual performance by using automated software to assess the correctness of breast positioning and compression in tomosynthesis screening.Materials and methodsIn this retrospective longitudinal analysis of prospective cohorts, six breast radiographers with varying experience in the field were asked to use automated software to improve their performance in breast compression and positioning. The software tool automatically analyzes craniocaudal (CC) and mediolateral oblique (MLO) views for their positioning quality by scoring them according to PGMI classifications (perfect, good, moderate, inadequate) and checking whether the compression pressure is within the target range. The positioning and compression data from the studies acquired before the start of the project were used as individual baselines, while the data obtained after the training were used to test whether conscious use of the software could help the radiographers improve their performance. The percentage of views rated perfect or good and the percentage of views in target compression were used as overall metrics to assess changes in performance.ResultsFollowing the use of the software, all radiographers significantly increased the percentage of images rated as perfect or good in both CCs and MLOs. Individual improvements ranged from 7 to 14% for CC and 10 to 16% for MLO views. Moreover, most radiographers exhibited improved compression performance in CCs, with improvements up to 16%.ConclusionActive use of a software tool to automatically assess the correctness of breast compression and positioning in breast cancer screening can improve the performance of radiographers.Clinical relevance statementThis study suggests that the use of a software tool for automatically evaluating correctness of breast compression and positioning in breast cancer screening can improve the performance of radiographers on these metrics, which may ultimately lead to improved screening outcomes.Key Points• Proper breast positioning and compression are critical in breast cancer screening to ensure accurate diagnosis.• Active use of the software increased the quality of craniocaudal and mediolateral oblique views acquired by all radiographers.• Improved performance of radiographers is expected to improve screening outcomes.

Understanding the static performance of composite helical springs with braided nested structures

Application of composite helical springs (CHSs) is constrained by their poor static compression performance. In this study, a novel composite helical spring with a braided nested structure (BNCHS) is proposed. The fiber volume fraction (Vf) of BNCHS with braided angle of 15° and 30° (BNCHS15° and BNCHS30°) only increases by 0.9% and 1.8% respectively comparing with that of unidirectional composite helical spring with Vf of 55% (UCHS55%). The compression experimental results show that the spring constant of BNCHS15° and BNCHS30° can reach 105.4% and 171.4% higher than that of UCHS55% respectively. The internal mechanism of significantly improving compression performance of BNCHS is revealed by using a meso model. Numerical result shows that the mises stress of BNCHS15° and BNCHS30° can be 2.43 and 3.14 times higher than that of UCHS55% respectively. Finally, the resilience and specific spring stiffness of BNCHS and steel are compared, highlighting the obvious advantage of static performance of BNCHS.

Reversible Data Hiding in Encrypted Images Based on Time-Varying Huffman Coding Table

Image privacy protection and management face many challenges, such as privacy disclosure, copyright dispute, and traceability difficulties, with the development of big data. Reversible data hiding in encrypted images (RDHEI) has been widely considered as an effective means to tackle these challenges. In this paper, a RDHEI based on time-varying Huffman coding table (TV-HCT) method is proposed to improve the security, embedding rate (ER) and efficiency. First, the initial HCT is generated according to the prediction errors of an image, which can improve compression performance. And then, the TV-HCT is obtained by scrambling equal-length codewords in the initial HCT using timestamps. This realizes the time variability of compression coding stream (CCS) of an image in that the image TV-HCT has large change space. Analysis shows that the average change space of TV-HCT in UCID is 3.97×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">327</sup> , and the average ER of three databases is more than 0.44 bpp higher than the existing algorithms. Finally, the CCS is encrypted using the designed index class scrambling method to balance complexity and security. The proposed method not only strengthens the security against brute force attack and differential attack, but also improves ER and efficiency of the RDHEI technique. Experimental results and performance analysis demonstrate that the proposed algorithm outperforms the state-of-the-art RDHEI algorithms in terms of the security, ER and complexity.

Mechanically strong wood-based composite aerogels as oil adsorbents and sensors

This work reported a top-down strategy to develop wood-based composites by combining wood skeleton (WS) aerogel with Polydimethylsiloxane (PDMS). WS aerogel was extracted by extensively removing lignin and hemicellulose from natural wood followed with freeze-drying. The hypothesis is that the introduction of hydrophobic PDMS into WS aerogel is of great importance for improving its mechanical performance and expanding the potential applications. To this end, WS/PDMS composites were developed by forming PDMS-infiltrated-wood skeleton via immersion and polymerization after curing treatment. We further adjusted the infiltrated PDMS concentration from 1 wt% to 30 wt% to explore the mechanical properties of the developed wood-based composite aerogels. As a result, both WS/PDMS-1 (with infiltration of 1 wt% PDMS) and WS/PDMS-30 (with infiltration of 30 wt% PDMS) performed significantly improved compression performance and shape stability compared with WS aerogel. Besides, with the assistance of surface-heating treatment on WS (HWS), the obtained HWS/PDMS-1 were used as adsorbing agents and showed fast adsorption rate with a capacity of 12 g/g (methyl silicone oil). Alternatively, by increasing the PDMS content, the obtained WS/PDMS-30 exhibited excellent compression strength and cycling stability, which could be assembled into sensor devices. We believe that this strategy may facilitate the development of strong wood-based composites with interesting features.

QTMT‐LNN: A fast intra CU partition using lightweight neural network for 360‐degree video coding on VVC

In the newest generation of video coding standard, Versatile Video Coding (VVC), a new technique called Quad Tree with nested Multi-type Tree (QTMT) structure is introduced. QTMT significantly improves the coding efficiency, but the improvement in compression performance comes at the cost of drastically increased complexity. This paper proposes a fast intra partition algorithm using Lightweight Neural Network (LNN) to skip QTMT partition steps which are unlikely to be chosen as the best split modes. Specifically, five LNNs (for five QTMT split modes) are trained, using features that consider the characteristic of 360-degree videos. The experimental results demonstrate that the proposed QTMT-LNN can reduce the encoding time from 52.28% to 72.17% on average, with coding efficiency losses ranging from 0.71% to 1.78%. Compared to other fast algorithms for intra coding unit (CU) partition, the method outperforms the related works in terms of Bjontegaard delta bit-rate (BDBR) and encoding time reduction.

Fusion-Based Versatile Video Coding Intra Prediction Algorithm with Template Matching and Linear Prediction.

The new generation video coding standard Versatile Video Coding (VVC) has adopted many novel technologies to improve compression performance, and consequently, remarkable results have been achieved. In practical applications, less data, in terms of bitrate, would reduce the burden of the sensors and improve their performance. Hence, to further enhance the intra compression performance of VVC, we propose a fusion-based intra prediction algorithm in this paper. Specifically, to better predict areas with similar texture information, we propose a fusion-based adaptive template matching method, which directly takes the error between reference and objective templates into account. Furthermore, to better utilize the correlation between reference pixels and the pixels to be predicted, we propose a fusion-based linear prediction method, which can compensate for the deficiency of single linear prediction. We implemented our algorithm on top of the VVC Test Model (VTM) 9.1. When compared with the VVC, our proposed fusion-based algorithm saves a bitrate of 0.89%, 0.84%, and 0.90% on average for the Y, Cb, and Cr components, respectively. In addition, when compared with some other existing works, our algorithm showed superior performance in bitrate savings.

A Hierarchical modified AV1 codec for compression cartesian form of holograms in holo and object planes

Three-dimensional (3D) image reconstruction is emerging as a leading challenge for 3D media on the Internet and virtual reality. In this regard, compression performance in 3D technology is one of the most important issues. Various codecs such as HEVC and AV1 have been suggested to improve compression performance in 3D technology. In this study, a hybrid method based on AV1 codec combined with mathematical methods is proposed for improving the quality of this codec. In the proposed method, two AV1 compression steps are used to estimate the AV1 codec error using a linear relationship and added to the basic codec output to improve the compression performance. The proposed method shows better quality in the Cartesian form (real and imaginary parts) of the hologram in Holo and Object plane when compared to new codecs that have been proposed in the field of 3D compression. In addition, the proposed method can be used as a general compression method for 2D images. Based on simulation results, the proposed method improved the quality of reconstructed hologram by up to 63% and 5 dB in terms of BDRate and BDPSNR, respectively.

EPQuant: A Graph Neural Network compression approach based on product quantization

Graph Neural Networks (GNNs) have been widely used in graph analysis due to their strong performance on a wide variety of tasks. Unfortunately, as the size of graphs keeps growing, large graphs can easily consume Terabytes, and training on such graphs may take days. The high memory footprint limits the usage of GNNs on resource-constrained devices like smartphones and IoT devices. Hence, reducing the storage cost, training time, and inference latency is highly desirable. In this work, we apply Product Quantization (PQ) on GNNs for the first time to achieve superior memory capacity reduction. To alleviate the processing burden caused by PQ and improve compression performance, we propose Enhanced Product Quantization (EPQ). It reduces the input graph data, which tends to dominate memory consumption, and accelerates clustering in PQ. Moreover, an efficient quantization framework11Github repository:https://github.com/Lyun-Huang/EPQuant for GNNs is proposed, which combines EPQ with Scalar Quantization (SQ), to achieve improved compression performance and computation acceleration on off-the-shelf hardware, enabling the deployment of GNNs on resource-constrained devices. In addition, the proposed quantization framework can be applied to the existing GNNs without too much porting effort. Extensive experimental results show that the proposed quantization scheme can achieve 321.26× and 184.03× memory capacity compression for input graph data and overall storage, respectively. This impressive memory reduction can come with an accuracy loss of less than 1%.

BVI-CC: A Dataset for Research on Video Compression and Quality Assessment

The video technology scenery has been very vivid over the past years, with novel video coding technologies introduced that promise improved compression performance over state-of-the-art technologies. Despite the fact that a lot of video datasets are available, representative content of the wide parameter space along with subjective evaluations of variations of encoded content from an unpartial end is required. In response to this requirement, this paper features a dataset, the BVI-CC. Three video codecs were deployed to create the variations of the encoded sequences: High Efficiency Video Coding Test Model (HM), AOMedia Video 1 (AV1), and Versatile Video Coding Test Model (VTM). Nine source video sequences were carefully selected to offer both diversity and representativeness in the spatio-temporal domain. Different spatial resolution versions of the sequences were created and encoded by all three codecs at pre-defined target bit rates. The compression efficiency of the codecs was evaluated with commonly used objective quality metrics, and the subjective quality of their reconstructed content was also evaluated through psychophysical experiments. Additionally, an adaptive bit rate (convex hull rate-distortion optimization across spatial resolutions) test case was assessed using both objective and subjective evaluations. Finally, the computational complexities of the tested codecs were examined. All data have been made publicly available as part of the dataset, which can be used for coding performance evaluation and video quality metric development.

Perceptual Quality Assessment of HEVC and VVC Standards for 8K Video

With the growing data consumption of emerging video applications and users&#x2019; requirement for higher resolutions, up to 8K, a huge effort has been made in video compression technologies. Recently, versatile video coding (VVC) has been standardized by the moving picture expert group (MPEG), providing a significant improvement in compression performance over its predecessor high efficiency video coding (HEVC). In this paper, we provide a comparative subjective quality evaluation between VVC and HEVC standards for 8K resolution videos. In addition, we evaluate the perceived quality improvement offered by 8K over UHD 4K resolution. The compression performance of both VVC and HEVC standards has been conducted in random access (RA) coding configuration, using their respective reference software, VVC test model (VTM-11) and HEVC test model (HM-16.20). Objective measurements, using PSNR, MS-SSIM and VMAF metrics have shown that the bitrate gains offered by VVC over HEVC for 8K video content are around 31&#x0025;, 26&#x0025; and 35&#x0025;, respectively. Subjectively, VVC offers an average of around 41&#x0025; of bitrate reduction over HEVC for the same visual quality. A compression gain of 50&#x0025; has been reached for some tested video sequences regarding a Student&#x2019;s t-test analysis. In addition, for most tested scenes, a significant visual difference between uncompressed 4K and 8K has been noticed.

A Method Based on Curvature and Hierarchical Strategy for Dynamic Point Cloud Compression in Augmented and Virtual Reality System.

As a kind of information-intensive 3D representation, point cloud rapidly develops in immersive applications, which has also sparked new attention in point cloud compression. The most popular dynamic methods ignore the characteristics of point clouds and use an exhaustive neighborhood search, which seriously impacts the encoder’s runtime. Therefore, we propose an improved compression means for dynamic point cloud based on curvature estimation and hierarchical strategy to meet the demands in real-world scenarios. This method includes initial segmentation derived from the similarity between normals, curvature-based hierarchical refining process for iterating, and image generation and video compression technology based on de-redundancy without performance loss. The curvature-based hierarchical refining module divides the voxel point cloud into high-curvature points and low-curvature points and optimizes the initial clusters hierarchically. The experimental results show that our method achieved improved compression performance and faster runtime than traditional video-based dynamic point cloud compression.

Towards coding for VoD application: An enhanced video compression system with a content-fitted recursive restoration network

The video on demand (VoD) applications require real-time decoding for customers while pursuing a better compression performance. Recent work has demonstrated the great potential of deep learning techniques in video compression. Building a generic model to accommodate various videos and meet the real-time decoding is an arduous task currently. However, the ground truth, i.e. raw video herein, is available during the video encoding process in practice, which means generalization is unnecessary. Inspired by this, we propose an enhanced video compression system (EVCS) using a content-fitted recursive restoration network (CRRN) for VoD. On the encoder side, a lightweight CRRN adopting the recursive restoration strategy is trained for a group of consecutive frames to be well-fitted to this group and achieve a strong restoration ability. After that, the learned parameters of CRRN are transmitted to the decoder with the encoded bitstream. On the decoder side, CRRN can perform the same robust restoration operation on the decoded frames. The proposed framework can be compatible with all of the existing video coding systems. Experimental results show that the compression performance can be significantly improved in terms of 5.938% - 24.413% BD-BR reduction by integrating CRRN with VTM, HM, X265, X264, NVENC, and two end-to-end DNN codecs DVC, M-LVC. This allows the H.264 and H.265 codec IPs developed with a lot of effort to be reused. Compared with other DNN based restoration works over HM, this work achieves the best compression performance improvement with fewer FLOPs for DNN computation. EVCS can satisfy the real-time decoding for 1080p 30fps videos on the AI edge platform.

Image Compression Software Design for Zhurong Mars Exploration Rover

Because of the limited number of information transmission channels and high bit error rate in Mars image transmission, a method of image compression software design and realization of Mars rover was proposed. The characteristics of rover image data management were analyzed, and the three capabilities that the rover image compression software needed to have were concluded: uniform file management capability, high-efficiency compression capability, and high fault tolerance capability. For uniform file management capability, a file system for uniform management of multiple types of image data was proposed. To meet the high efficiency and high fault tolerance requirements of the compression algorithm, based on run-length coding algorithm of first-1 bit, the three important processes of multi-load image unification processing, color image preprocessing, and segmented bit-plane image coding were expanded, to improve compression performance and fault tolerance. Then a complete set of image compression software design schemes was formed, using FPGA+DSP, and the complete set of software was realized. Finally a simulation verification system was used to verify the compression effect. Verification results show the effectiveness of the algorithm. This software has been verified by China’s Zhurong Mars rover and can effectively meet the needs of Mars exploration missions. The software design can be used as a reference for image compression in subsequent ultra-long-distance deep space explorations.

Adaptive color space model based on dominant colors for image and video compression performance improvement

This paper describes the use of some color spaces in JPEG image compression algorithm and their impact in terms of image quality and compression ratio, and then proposes adaptive color space models (ACSM) to improve the performance of lossy image compression algorithm. The proposed ACSM consists of, dominant color analysis algorithm and YCoCg color space family. The YCoCg color space family is composed of three color spaces, which are YCcCr, YCpCg and YCyCb. The dominant colors analysis algorithm is developed which enables to automatically select one of the three color space models based on the suitability of the dominant colors contained in an image. The experimental results using sixty test images, which have varying colors, shapes and textures, show that the proposed adaptive color space model provides improved performance of 3 % to 10 % better than YCbCr, YDbDr, YCoCg and YCgCo-R color spaces family. In addition, the YCoCg color space family is a discrete transformation so its digital electronic implementation requires only two adders and two subtractors, both for forward and inverse conversions.

RETRACTED: Optimization of Motion Estimation Algorithm Based on FPGA Hardware System and Video Tracking

Motion Estimation (ME) is a computationally expensive step in video tracking. A thorough search technique produces the best execution time and the best accuracy. Computational burden, many fast search algorithms limit and report the number of places to search. Motion estimation, analysis of image sequences, computer vision, tracking a target, and is a critical process in a wide range of fields and applications such as video coding. Precision motion estimation lowers significantly bit rate, but can require a complexity of higher computational. This is different reference frames, sub-pixel estimation, and video compression standards, including techniques such as variable block size, MPEG, especially true for a new generation. Only the motion estimation in H.261 is limited to an integer pixel accuracy, rather than the pixel grid, not in the object is often located, move to move between pixels. The system proposed in the motion vector optimization motion estimation (OME) pixel accuracy. By estimating the amount of displacement at a finer resolution, improve prediction, it can expect better performance than motion estimation using an integer pixel precision. Optimization motion estimation (OME) and the compensation method is very similar to that employed in other standards. The main is that block-based motion estimation and compensation is suitable for optimization structures in MPEG-4. Motion estimation (ME) is one of the most computationally-intensive companies in the video compression technology. Video compression algorithm, MPEG1, MPEG4, such as H.261 and H.264, a lot of the standard used. Improved compression performance. Energy effective motion estimation technique is reduced to the rest of the frame involved in motion compensation.