Objective Quality Assessment Metrics Research Articles

PCQD‐AR: Subjective quality assessment of compressed point clouds with head‐mounted augmented reality

AbstractThis letter fully studies the coloured point cloud quality assessment in augmented reality (AR) environment through subjective test. Firstly, a point cloud dataset, named point cloud quality dataset‐AR, including ten reference point clouds and their 90 distorted versions is presented, which were generated using the reference software of video‐based point cloud compression across various combinations of geometry and texture quantization parameters. Then, the impact of geometry and texture distortions on perceived quality of point clouds in AR environment was discussed in detail. Moreover, the performance of existing objective point cloud quality assessment metrics on the proposed dataset is evaluated. The subjective dataset including the values of mean opinion score were released to public.

Crafting the MPEG metrics for objective and perceptual quality assessment of volumetric videos.

Efficient objective and perceptual metrics are valuable tools to evaluate the visual impact of compression artifacts on the visual quality of volumetric videos (VVs). In this paper, we present some of the MPEG group efforts to create, benchmark and calibrate objective quality assessment metrics for volumetric videos represented as textured meshes. We created a challenging dataset of 176 volumetric videos impaired with various distortions and conducted a subjective experiment to gather human opinions (more than 5896 subjective scores were collected). We adapted two state-of-the-art model-based metrics for point cloud evaluation to our context of textured mesh evaluation by selecting efficient sampling methods. We also present a new image-based metric for the evaluation of such VVs whose purpose is to reduce the cumbersome computation times inherent to the point-based metrics due to their use of multiple kd-tree searches. Each metric presented above is calibrated (i.e., selection of best values for parameters such as the number of views or grid sampling density) and evaluated on our new ground-truth subjective dataset. For each metric, the optimal selection and combination of features is determined by logistic regression through cross-validation. This performance analysis, combined with MPEG experts' requirements, lead to the validation of two selected metrics and recommendations on the features of most importance through learned feature weights.

A Crypto-Steganography Approach for Hiding Ransomware within HEVC Streams in Android IoT Devices.

Steganography is a vital security approach that hides any secret content within ordinary data, such as multimedia. This hiding aims to achieve the confidentiality of the IoT secret data; whether it is benign or malicious (e.g., ransomware) and for defensive or offensive purposes. This paper introduces a hybrid crypto-steganography approach for ransomware hiding within high-resolution video frames. This proposed approach is based on hybridizing an AES (advanced encryption standard) algorithm and LSB (least significant bit) steganography process. Initially, AES encrypts the secret Android ransomware data, and then LSB embeds it based on random selection criteria for the cover video pixels. This research examined broad objective and subjective quality assessment metrics to evaluate the performance of the proposed hybrid approach. We used different sizes of ransomware samples and different resolutions of HEVC (high-efficiency video coding) frames to conduct simulation experiments and comparison studies. The assessment results prove the superior efficiency of the introduced hybrid crypto-steganography approach compared to other existing steganography approaches in terms of (a) achieving the integrity of the secret ransomware data, (b) ensuring higher imperceptibility of stego video frames, (3) introducing a multi-level security approach using the AES encryption in addition to the LSB steganography, (4) performing randomness embedding based on RPS (random pixel selection) for concealing secret ransomware bits, (5) succeeding in fully extracting the ransomware data at the receiver side, (6) obtaining strong subjective and objective qualities for all tested evaluation metrics, (7) embedding different sizes of secret data at the same time within the video frame, and finally (8) passing the security scanning tests of 70 antivirus engines without detecting the existence of the embedded ransomware.

Objective Quality Assessment Metrics for Light Field Image Based on Textural Features

Light Field (LF) imaging is a plenoptic data collection method enabling a wide variety of image post-processing such as 3D extraction, viewpoint change and digital refocusing. Moreover, LF provides the capability to capture rich information about a scene, e.g., texture, geometric information, etc. Therefore, a quality assessment model for LF images is needed and poses significant challenges. Many LF Image Quality Assessment (LF-IQA) metrics have been recently presented based on the unique characteristics of LF images. The state-of-the-art objective assessment metrics have taken into account the image content and human visual system such as SSIM and IW-SSIM. However, most of these metrics are designed for images and video with natural content. Additionally, other models based on the LF characteristics (e.g., depth information, angle information) trade high performance for high computational complexity, along with them possessing difficulties of implementation for LF applications due to the immense data requirements of LF images. Hence, this paper presents a novel content-adaptive LF-IQA metric to improve the conventional LF-IQA performance that is also low in computational complexity. The experimental results clearly show improved performance compared to conventional objective IQA metrics, and we also identify metrics that are well-suited for LF image assessment. In addition, we present a comprehensive content-based feature analysis to determine the most appropriate feature that influences human visual perception among the widely used conventional objective IQA metrics. Finally, a rich LF dataset is selected from the EPFL dataset, allowing for the study of light field quality by qualitative factors such as depth (wide and narrow), focus (background or foreground) and complexity (simple and complex).

A Method of Codec Comparison and Selection for Good Quality Video Transmission Over Limited-Bandwidth Networks.

Finding a proper balance between video quality and the required bandwidth is an important issue, especially in networks of limited capacity. The problem of comparing the efficiency of video codecs and choosing the most suitable one in a specific situation has become very important. This paper proposes a method of comparing video codecs while also taking into account objective quality assessment metrics. The author shows the process of preparing video footage, assessing its quality, determining the rate–distortion curves, and calculating the bitrate saving for pairs of examined codecs. Thanks to the use of the spline interpolation method, the obtained results are better than those previously presented in the literature, and more resistant to the quality metric used.

Analysis of the Perceptual Quality Performance of Different HEVC Coding Tools

Each new video encoding standard includes encoding techniques that aim to improve the performance and quality of the previous standards. During the development of these techniques, PSNR was used as the main distortion metric. However, the PSNR metric does not consider the subjectivity of the human visual system, so that the performance of some coding tools is questionable from the perceptual point of view. To further explore this point, we have developed a detailed study about the perceptual sensibility of different HEVC video coding tools. In order to perform this study, we used some popular objective quality assessment metrics to measure the perceptual response of every single coding tool. The conclusion of this work will help to determine the set of HEVC coding tools that provides, in general, the best perceptual response.

Assessment of the Segmentation of RGB Remote Sensing Images: A Subjective Approach

The evaluation of remote sensing imagery segmentation results plays an important role in the further image analysis and decision-making. The search for the optimal segmentation method for a particular data set and the suitability of segmentation results for the use in satellite image classification are examples where the proper image segmentation quality assessment can affect the quality of the final result. There is no extensive research related to the assessment of the segmentation effectiveness of the images. The designed objective quality assessment metrics that can be used to assess the quality of the obtained segmentation results usually take into account the subjective features of the human visual system (HVS). A novel approach is used in the article to estimate the effectiveness of satellite image segmentation by relating and determining the correlation between subjective and objective segmentation quality metrics. Pearson’s and Spearman’s correlation was used for satellite images after applying a k-means++ clustering algorithm based on colour information. Simultaneously, the dataset of the satellite images with ground truth (GT) based on the “DeepGlobe Land Cover Classification Challenge” dataset was constructed for testing three classes of quality metrics for satellite image segmentation.

Point Cloud Rendering After Coding: Impacts on Subjective and Objective Quality

Recently, point clouds have shown to be a promising way to represent 3D visual data for a wide range of immersive applications, from augmented reality to autonomous cars. Emerging imaging sensors have made easier to perform richer and denser point cloud acquisition, notably with millions of points, thus raising the need for efficient point cloud coding solutions. In such a scenario, it is important to evaluate the impact and performance of several processing steps in a point cloud communication system, notably the quality degradations associated to point cloud coding solutions. Moreover, since point clouds are not directly visualized but rather processed with a rendering algorithm before shown on any display, the perceived quality of point cloud data highly depends on the rendering solution. In this context, the main objective of this paper is to study the impact of several coding and rendering solutions on the perceived user quality and in the performance of available objective quality assessment metrics. Another contribution regards the assessment of recent MPEG point cloud coding solutions for several popular rendering methods which were never presented before. The conclusions regard the visibility of three types of coding artifacts for the three considered rendering approaches as well as the strengths and weakness of objective quality metrics when point clouds are rendered after coding.

Overview of full-reference video quality metrics and their performance evaluations for videoconferencing application

With the development of video services available on various devices (mobile, PC, tablet, television, etc.), the end user experiences and requirements have changed. Objective video quality assessment tools and metrics are increasingly complex and adapted to different types of service context. Compared with other reviews of quality metrics, the full-reference (FR) video quality metrics reviewed (particularly, ViS3, SSIMplus, video multimethod assessment fusion, and open perceptual video quality metric) are more recent and have not been compared with each other yet. We compare and evaluate the performance of 10 FR metrics with respect to their accuracy in predicting the subjective perceived quality for the videoconferencing application. Emphasis is given to the metrics evaluating the visual quality (from human perception). A detailed statistical study is carried out on four subjective databases: Ecole polytechnique federale de Lausanne database, live mobile database, and two Orange Lab databases, with a large sample of distortion types (transmission errors, encoding bit rates, etc.).

Efficient video quality assessment for on-demand video transcoding using intensity variation analysis

Due to the wide spread usage of smart devices, adopting video contents service to the diverse end user’s service environment is an essential process. The heterogeneity of end users’ devices, usually referred as the device fragmentation, requires video transcoding which is a lossy process. Accordingly, the subsequent video quality degrading is inevitable. In such circumstances, minimizing perceptible quality loss of video is a key issue for the video contents service provider. However, the video quality loss caused in the process of transcoding is very difficult to measure. Because the “video quality” is a subjective term, it is almost impossible to estimate before video contents are delivered and actually serviced. To address this issue, many research efforts have been pursued for estimating subjective quality evaluation score using objective quality assessment metric. Structural Similarity (SSIM) is a well-known objective quality assessment method. Based on previous studies, this method has been used as a very effective quality assessment tool in video coding system. In this paper, we propose new video quality assessment metric using intensity variation analysis. The intensity metric-based video quality assessment has a high correlation with the SSIM regardless of the category of video contents, resolutions and even bitrate setting. The proposed method that measures inter-frame intensity variation (IV) is more efficient than SSIM in VBR transcoding system. Our experimental results show that the proposed video quality assessment shows up to 22 times faster than SSIM in the execution time. Ultimately, to take its advantage of the short latency and low execution overhead, IV-based video assessment is applicable to real on-demand transcoding and streaming environments while minimizing video quality degradation of transcoding.

No-reference image quality assessment using local binary pattern in the wavelet domain

Objective quality assessment metrics that are consistent with human judgments of image quality, play an important role in many image processing applications. No-reference image quality assessment (NR-IQA) algorithms evaluate the quality of distorted images without any information about the reference images. In this paper, we propose an accurate and simple method for NR-IQA. In this method local binary pattern (LBP) operator is first applied to subbands of wavelet decomposed image separately; then histogram of LBP codes is calculated and concatenated to form quality aware features. Finally, support vector regression (SVR) is utilized to map the feature vectors to subjective quality scores. The experimental results for LIVE and TID2008 databases show that our proposed method is highly correlated to the subjective scores, and competitive to state-of-the-art NR-IQA methods.

A New Image Quality Assessment Algorithm based on SSIM and Multiple Regressions

Image quality assessment (IQA) is crucial in image processing algorithms. In the state-of-the-art IQA index, the structural similarity (SSIM) index has been proved to be better objective quality assessment metric. However, the accuracy of SSIM is relatively lacking when used to access blurred images. And the component weights of structural similarity (SSIM) index are fixed in some past environments. So an improved assessment algorithm incorporating multiple linear regressions and SSIM index was proposed in this paper. We use regression analysis to adjust the component weight of SSIM index. So the improved algorithm is more accuracy on different distortion types’ quality assessment. Experimental results show that the improved SSIM algorithm is better than traditional methods in nonlinear regression correlation coefficient, Spearman correlation coefficient and out ratio.

Hoda Roodaki

With the growing demand for 3D video, efforts are underway to incorporate it in the next generation of broadcast and streaming applications and standards. Scalability is one possible solution to reduce the amount of data in multi-view/3D video in heterogeneous environments. But using Scalable Multi-view Video Coding (SMVC) for multi-view/3D video still has many unresolved challenges. In this thesis, we propose an adaptive framework to use SMVC in various 3D video applications effectively. For this issue, first, the proper scalable modality should be selected according to the application at hand, its related features and requirements. To the best of our knowledge, no work has systematically defined new and proper scalable modalities specifically for multi-view 3D video, so far. Hence, at the first step of the proposed framework, we will suggest a methodology to extract the proper scalable modalities for multi-view/3D video. In addition, while SMVC can help support heterogeneous receivers, the question becomes: how to scale the 3D video content in a given type of scalability and a specified application in order to achieve the highest performance and satisfy the receivers? constraint as much as possible? In other words, the proper mechanism to assign SMVC data to various layers should be clearly determined. This issue is considered as the second step of our proposed framework. This method uses the inter-layer and intralayer disparity concepts. Note that specific features of any given scalable modality should be used to define these concepts in that specific scalable modality. Simulation results indicate that the proposed method achieves relatively better compression rate for each layer, with much less overhead. At the next step of our proposed framework, we propose an analytical view-level rate model for multiview video coding. Our rate model takes into account both previous theoretical results as well as new results specifically obtained for multi-view video and confirmed by comprehensive practical experiments. Simulation results show that our model can predict the rate of each view with relatively high precision and a low estimation error of 12% on average for tested sequences. In addition, the evaluation of the overall visual quality of scalable multi-view video requires a new objective perceptual quality measure specifically designed for scalable multi-view/3D video. Although several subjective and objective quality assessment methods have been proposed for multi-view/3D sequences, no comparable attempt has been made for quality assessment of scalable multi-view/3D video so far. Hence, in this framework, we propose a new methodology to build suitable objective quality assessment metrics for different scalable modalities in multi-view/3D video. Our proposed methodology considers the importance of each layer and its content as a quality of experience factor in the overall quality. Furthermore, in addition to the quality of each layer, the concept of inter-layer and intra-layer disparity is considered as an effective feature to evaluate overall perceived quality more accurately. Our simulation results indicate that the correlation coefficient between our extracted objective quality evaluation metric and subjective quality assessment is 0.8 on average for tested video sequences. At the last step of our proposed framework, we present a novel method for rate-distortion optimization in scalable multi-view video that tries to minimize the perceptual distortion of decoded video under the conditions that the sum of bits generated from different views is constrained within a given bit budget. Since the constraint-based optimization problem is usually computational intensive, our proposed approach considers the concept of intra-layer and inter-layer disparity to reduce this computational complexity. Experimental results show that the proposed approach uses on average 24% and 42% less bitrate than the H.264/AVC rate-distortion optimization for base and base plus enhancement layers, respectively. Although the thesis is in Farsi (Persian), the following English papers capture most of its essence: H. Roodaki, M.R. Hashemi, and S. Shirmohammadi, ?A New Methodology to Derive Objective Quality Assessment Metrics for Scalable Multi-view 3D Video Coding?, ACM Transactions on Multimedia Computing, Communications, and Applications, Vol. 8, No. 3S, Article 44, September 2012, 25 pages. DOI: 10.1145/2348816.2348823 H. Roodaki, M.R. Hashemi, and S. Shirmohammadi, ?Rate-Distortion Optimization for Scalable Multi-View Video Coding?, Proc. IEEE International Conference on Multimedia and Expo, Chengdu, China, July 14-18 2014, 6 pages. DOI: 10.1109/ICME.2014.6890275 H. Roodaki, Z. Iravani, M.R. Hashemi, S. Shirmohammadi, and M. Gabbouj, ?A New Rate Distortion Model for Multi-View/3D Video Coding?, Proc. IEEE International Workshop on Hot Topics in 3D, in Proc. IEEE International Conference on Multimedia and Expo, July 15-19 2013, San Jose, USA, 6 pages. DOI: 10.1109/ICMEW.2013.6618338 H. Roodaki, M.R. Hashemi, and S. Shirmohammadi, ?New Scalable Modalities in Multi-view 3D Video?, Proc. ACM Workshop on Mobile Video, Oslo, Norway, February 27 2013, pp. 25-30. DOI: 10.1145/2457413.2457420

Information-theoretical image quality metric in digital image compression

Existing objective and subjective TV image quality assessment metrics are considered. New metrics of digital image quality test are founded. Local entropy approach to form objective TV image quality assessment metrics is demonstrated experimentally.

On the performance of video quality assessment metrics under different compression and packet loss scenarios.

When comparing the performance of video coding approaches, evaluating different commercial video encoders, or measuring the perceived video quality in a wireless environment, Rate/distortion analysis is commonly used, where distortion is usually measured in terms of PSNR values. However, PSNR does not always capture the distortion perceived by a human being. As a consequence, significant efforts have focused on defining an objective video quality metric that is able to assess quality in the same way as a human does. We perform a study of some available objective quality assessment metrics in order to evaluate their behavior in two different scenarios. First, we deal with video sequences compressed by different encoders at different bitrates in order to properly measure the video quality degradation associated with the encoding system. In addition, we evaluate the behavior of the quality metrics when measuring video distortions produced by packet losses in mobile ad hoc network scenarios with variable degrees of network congestion and node mobility. Our purpose is to determine if the analyzed metrics can replace the PSNR while comparing, designing, and evaluating video codec proposals, and, in particular, under video delivery scenarios characterized by bursty and frequent packet losses, such as wireless multihop environments.

Visually weighted reconstruction of compressive sensing MRI

Compressive sensing (CS) enables the reconstruction of a magnetic resonance (MR) image from undersampled data in k-space with relatively low-quality distortion when compared to the original image. In addition, CS allows the scan time to be significantly reduced. Along with a reduction in the computational overhead, we investigate an effective way to improve visual quality through the use of a weighted optimization algorithm for reconstruction after variable density random undersampling in the phase encoding direction over k-space. In contrast to conventional magnetic resonance imaging (MRI) reconstruction methods, the visual weight, in particular, the region of interest (ROI), is investigated here for quality improvement. In addition, we employ a wavelet transform to analyze the reconstructed image in the space domain and fully utilize data sparsity over the spatial and frequency domains. The visual weight is constructed by reflecting the perceptual characteristics of the human visual system (HVS), and then applied to ℓ1 norm minimization, which gives priority to each coefficient during the reconstruction process. Using objective quality assessment metrics, it was found that an image reconstructed using the visual weight has higher local and global quality than those processed by conventional methods.

DCT-based objective quality assessment metric of 2D/3D image

With the increasing growth of multimedia applications over the networking in recent years, users have put forward much higher requirements for multimedia quality of experience (QoE) than before. One of the representative requirements is the image quality. Therefore, the image quality assessment ranging from two-dimension (2D) image to three-dimension (3D) image has been getting much attention. In this paper, an efficient objective image quality assessment metric in block-based discrete cosine transform (DCT) coding is proposed. The metric incorporates properties of human visual system (HVS) to improve its validity and reliability in evaluating the quality of stereoscopic image. This is fulfilled by calculating the local pixel-based distortions in frequency domain, combining the simplified models of local visibility properties embodied in frequency domain, which consist of region of interest (ROI) mechanism (visual sensitivity), contrast sensitivity function (CSF) and contrast masking effect. The performance of the proposed metric is compared with other currently state-of-the-art objective image quality assessment metrics. The experimental results have demonstrated that the proposed metric is highly consistent with the subjective test scores. Moreover, the performance of the metric is also confirmed with the popular IRCCyN/IVC database. Therefore, the proposed metric is promising in term of the practical efficiency and reliability for real-life multimedia applications.

A new methodology to derive objective quality assessment metrics for scalable multiview 3D video coding

With the growing demand for 3D video, efforts are underway to incorporate it in the next generation of broadcast and streaming applications and standards. 3D video is currently available in games, entertainment, education, security, and surveillance applications. A typical scenario for multiview 3D consists of several 3D video sequences captured simultaneously from the same scene with the help of multiple cameras from different positions and through different angles. Multiview video coding provides a compact representation of these multiple views by exploiting the large amount of inter-view statistical dependencies. One of the major challenges in this field is how to transmit the large amount of data of a multiview sequence over error prone channels to heterogeneous mobile devices with different bandwidth, resolution, and processing/battery power, while maintaining a high visual quality. Scalable Multiview 3D Video Coding (SMVC) is one of the methods to address this challenge; however, the evaluation of the overall visual quality of the resulting scaled-down video requires a new objective perceptual quality measure specifically designed for scalable multiview 3D video. Although several subjective and objective quality assessment methods have been proposed for multiview 3D sequences, no comparable attempt has been made for quality assessment of scalable multiview 3D video. In this article, we propose a new methodology to build suitable objective quality assessment metrics for different scalable modalities in multiview 3D video. Our proposed methodology considers the importance of each layer and its content as a quality of experience factor in the overall quality. Furthermore, in addition to the quality of each layer, the concept of disparity between layers (inter-layer disparity) and disparity between the units of each layer (intra-layer disparity) is considered as an effective feature to evaluate overall perceived quality more accurately. Simulation results indicate that by using this methodology, more efficient objective quality assessment metrics can be introduced for each multiview 3D video scalable modalities.

Assessing Visual Quality of 3-D Polygonal Models

Recent advances in evaluating and measuring the perceived visual quality of three-dimensional (3-D) polygonal models are presented in this article, which analyzes the general process of objective quality assessment metrics and subjective user evaluation methods and presents a taxonomy of existing solutions. Simple geometric error computed directly on the 3-D models does not necessarily reflect the perceived visual quality; therefore, integrating perceptual issues for 3-D quality assessment is of great significance. This article discusses existing metrics, including perceptually based ones, computed either on 3-D data or on two-dimensional (2-D) projections, and evaluates their performance for their correlation with existing subjective studies.

Quality Evaluation of Color Plus Depth Map-Based Stereoscopic Video

In the near future, many conventional video applications are likely to be replaced by immersive video to provide a sense of ldquobeing there.rdquo This transition is facilitated by the recent advancement of 3D capture, coding, transmission, and display technologies. Stereoscopic video is the simplest form of 3D video available in the literature. ldquoColor plus depth maprdquo based stereoscopic video has attracted significant attention, as it can reduce storage and bandwidth requirements for the transmission of stereoscopic content over communication channels. However, quality assessment of coded video sequences can currently only be performed reliably using expensive and inconvenient subjective tests. To enable researchers to optimize 3D video systems in a timely fashion, it is essential that reliable objective measures are found. This paper investigates the correlation between subjective and objective evaluation of color plus depth video. The investigation is conducted for different compression ratios, and different video sequences. Transmission over Internet protocol (IP) is also investigated. Subjective tests are performed to determine the image quality and depth perception of a range of differently coded video sequences, with packet loss rates ranging from 0% to 20%. The subjective results are used to determine more accurate objective quality assessment metrics for 3D color plus depth video.