Perceived Quality Degradation Research Articles

Perceptual Quality Driven Frame-Rate Selection (PQD-FRS) for High-Frame-Rate Video

Video of higher frame rates (HFR) reduces the visual artifact in large screen display at the cost of a higher coding bit rate (or transmission bandwidth). In this work, we propose a perceptual quality driven frame rate selection (PQD-FRS) method that assigns a time-varying frame rate to a sequence so as to reduce its transmission cost. The objective of the PQD-FRS method is to offer perceptually indistinguishable experience for a certain percentage of viewers. We first conduct a subjective test to characterize the relationship between human perceived quality and video contents, and build a frame-rate-dependent video quality assessment dataset to serve as the ground truth. Then, we use a machine learning approach for the design of the key module of the PQD-FRS method, called the “satisfied user ratio (SUR) prediction.” The SUR prediction module predicts the percentage of satisfied viewers, who cannot differentiate video quality of a lower and HFR, using the support vector regression. It is confirmed by experimental results that the proposed SUR module can offer a so highly accurate prediction that the PQD-FRS system can dynamically assign a proper frame rate to video without any perceptual quality degradation for a majority of viewers.

Multi-layer authentication scheme for HEVC video based on embedded statistics

A multi-layer authentication scheme for HEVC compressed video is proposed. The combination of CU sizes, which is unique to HEVC and sensitive to video manipulation, is considered along with other elements in the HEVC coding standard to generate the authentication tag. Temporal dependency was enforced, where the authentication tag generated in one slice is embedded into its subsequent slice. By design, the authentication tag is repeatedly but selectively embedded into various elements in a HEVC video, including nonzero DCT coefficients, QP parameter values, and prediction modes, depending on the bit segment in the generated tag. Our scheme offers three layers of authentication to detect and localize the tampered regions in a HEVC video, as well as verifying the source/sender of the video using a shared secret key. Video sequences from various classes (resolutions) are considered to verify the performance of the proposed multi-layer authentication scheme. Results show that, at the expense of slight degradation in perceptual quality, the proposed scheme is robust against several common attacks. A functional comparison is performed between the proposed multi-layer authentication scheme and the conventional schemes.

A robust watermarking framework for High Efficiency Video Coding (HEVC) – Encoded video with blind extraction process

Digital watermarking is an efficient and promising means for copyright protection of multimedia objects. Digital videos are stored and transmitted in a compressed format, which has drawn a great deal of attention in compressed domain watermarking for video. The embedding and extraction of watermark bits in compressed domain, therefore does not require complete decoding and re-encoding of the compressed video. There have been several compression standards for video. High Efficiency Video Coding (HEVC), a successor to H.264 Advanced Video Coding (AVC), is the latest standard for video compression with high compression efficiency. In this paper, we propose a robust watermarking framework with a blind extraction process for HEVC encoded video. A readable watermark is embedded invisibly in 4×4 intra predicted blocks of the HEVC encoded video. Our watermarking framework enforces security by exploring the spatio-temporal characteristics of the compressed video and a random key for selection of embedding regions. We also analyze the strengths of different compressed domain features of HEVC encoded video for implementing the embedding algorithm. Experimental results demonstrate that the proposed work restrict the increase in video bit rate and degradation of perceptual quality. The proposed framework can also survive filtering, compressions, and noise additions maintaining good quality and robustness.

3D-MAD: A Full Reference Stereoscopic Image Quality Estimator Based on Binocular Lightness and Contrast Perception.

Algorithms for a stereoscopic image quality assessment (IQA) aim to estimate the qualities of 3D images in a manner that agrees with human judgments. The modern stereoscopic IQA algorithms often apply 2D IQA algorithms on stereoscopic views, disparity maps, and/or cyclopean images, to yield an overall quality estimate based on the properties of the human visual system. This paper presents an extension of our previous 2D most apparent distortion (MAD) algorithm to a 3D version (3D-MAD) to evaluate 3D image quality. The 3D-MAD operates via two main stages, which estimate perceived quality degradation due to 1) distortion of the monocular views and 2) distortion of the cyclopean view. In the first stage, the conventional MAD algorithm is applied on the two monocular views, and then the combined binocular quality is estimated via a weighted sum of the two estimates, where the weights are determined based on a block-based contrast measure. In the second stage, intermediate maps corresponding to the lightness distance and the pixel-based contrast are generated based on a multipathway contrast gain-control model. Then, the cyclopean view quality is estimated by measuring the statistical-difference-based features obtained from the reference stereopair and the distorted stereopair, respectively. Finally, the estimates obtained from the two stages are combined to yield an overall quality score of the stereoscopic image. Tests on various 3D image quality databases demonstrate that our algorithm significantly improves upon many other state-of-the-art 2D/3D IQA algorithms.

Learning discriminant DCT coefficients driven block descriptor for digital dropout detection system in degraded archived media

Digitization of old archived media is of great importance to preserve the originality of medium in terms of historical record as well as the means to quality improvement for reproduction purposes. However, digitization increases the exposure of the media to digital dropout error, thus presenting a significant degradation in perceptual quality of the converted video sequences. A numbers of mechanisms were investigated in the past to make these converted media more robust against digital dropout errors. Nevertheless, these techniques achieved little success, forcing manual quality check to assure standard quality. This paper presents an automatic solution to this problem based on discriminant DCT coefficients. Here, the idea is to build a block classification model by learning discriminant DCT coefficients first and utilize these coefficients along with an weighted neighborhood sampling strategy to formulate discriminant block descriptor so that within-class difference of the block features is minimized and between-class difference is maximized. This spatial detection is free from motion computation; thus performs accurately in presence of pathological motion (PM) and fast moving objects. Finally, the proposed method is compared against the existing methods to demonstrate improved detection accuracy using real degraded video archives.

혼합 해상도 양안식 영상에 대한 주관적 화질평가를 통한 고정 및 이동 융합형 3DTV 서비스의 품질 검증

양안식 3D 비디오의 효율적인 압축을 위해 다양한 기술들이 연구되었으며, 그 중 좌우영상으로 상이한 해상도의 영상을 사용하는 방식은 인간의 시각 시스템이 혼합 해상도의 양안식 영상을 높은 쪽 영상의 품질에 가깝게 인지하는 특성을 이용한 대표적인 비트레이트 절감 방식이다. 그러나 혼합 해상도 양안식 영상에서 좌우안 영상 해상도 차이가 일정 범위 이상일 경우 3D 영상의 품질 저하가 발생한다. 이에 다양한 연구에서 혼합 해상도 양안식 영상의 좌우해상도 차이에 따른 화질 저하 정도를 측정하려는 시도가 이루어졌으나, 기존 연구에서는 시청거리를 고려하지 않고 단순 좌우 영상의 크기만을 고려하여 실험하였으며 이에 각 연구별로 상이한 실험 결과가 도출되었다. 본 연구에서는 인간의 시각 시스템을 고려하여 최적 시청거리를 계산하고, 이에 기반하여 좌우영상 비율 별 주관적 화질평가를 수행하였다. 또한 실험 결과 분석을 통해 혼합 해상도 영상 기반의 고정 및 이동 융합형 3DTV 기술이 고품질 3D 서비스를 제공할 수 있음을 보인다. Various techniques have been developed for efficient compression of stereoscopic 3D videos. Mixed-resolution based approach is one representative bit-rate saving method based on the characteristics of human visual system that the mixed-resolution stereoscopic videos are perceived close to the higher resolution. However, when the difference between the left and right image resolutions is bigger than a certain threshold level, it causes the perceived quality degradation of the 3D images. Subsequently, several researches tried to find the correlation between the difference in resolution and the level of the perceived quality degradation, but they conducted the experiments just considering the difference in resolution without considering the viewing distances, so thereby different results were retrieved from test to test. In this work, we calculated the optimal viewing distance based on the human visual system, and conducted the subjective tests with the calculated viewing distance. With the results, we demonstrate that the fixed and mobile hybrid 3DTV, which is based on mixed-resolution stereoscopic images, can provide the high quality 3D services.

ViS3: an algorithm for video quality assessment via analysis of spatial and spatiotemporal slices

Algorithms for video quality assessment (VQA) aim to estimate the qualities of videos in a manner that agrees with human judgments of quality. Modern VQA algorithms often estimate video quality by comparing localized space-time regions or groups of frames from the reference and distorted videos, using comparisons based on visual features, statistics, and/or perceptual models. We present a VQA algorithm that estimates quality via separate estimates of perceived degradation due to (1) spatial distortion and (2) joint spatial and temporal distortion. The first stage of the algorithm estimates perceived quality degradation due to spatial distortion; this stage operates by adaptively applying to groups of spatial video frames the two strategies from the most apparent distortion algorithm with an extension to account for temporal masking. The second stage of the algorithm estimates perceived quality degradation due to joint spatial and temporal distortion; this stage operates by measuring the dissimilarity between the reference and distorted videos represented in terms of two-dimensional spatiotemporal slices. Finally, the estimates obtained from the two stages are combined to yield an overall estimate of perceived quality degradation. Testing on various video-quality databases demonstrates that our algorithm performs well in predicting video quality and is competitive with current state-of-the-art VQA algorithms.

A Scalable Reversible Data Embedding Method with progressive quality degradation functionality

This paper proposes a novel reversible information hiding method aiming to achieve scalable carrier capacity while progressively distorting the image quality. Unlike the conventional methods, the proposed method HAM (Histogram Association Mapping) purposely degrades the perceptual quality of the input image through data embedding. To the best of our knowledge, there is no method that attempts to significantly increase the carrier capacity while introducing (tolerating) intentional perceptual degradation for avoiding unauthorized viewing. HAM eliminates the expensive pre-processing step(s) required by the conventional histogram shifting data embedding approach and improves its carrier capacity. In particular, the host image is divided into non-overlapping blocks and each block is classified into two classes. Each class undergoes different HAM process to embed the external data while distorting quality of the image to the desired level. Experiments were conducted to measure the performances of the proposed method by using standard test images and CalTech 101 dataset. In the best case scenario, an average of ~2.88 bits per pixel is achieved as the effective carrier capacity for the CalTech 101 dataset. The proposed method is also compared with the conventional methods in terms of carrier capacity and scalability in perceptual quality degradation.

On the identification of relevant degradation indicators in super wideband listening quality assessment models

Recently, new objective speech quality evaluation methods, designed and adapted to new high voice quality contexts, have been developed. One interest of these methods is that they integrate voice quality perceptual dimensions reflecting the effects of frequency–response distortions, discontinuities, noise and/or speech level deviations respectively. This makes it possible to use these methods also to provide diagnostic information about specific aspects of the transmission systems’ quality, as perceived by end-users. In this paper, we present and analyze in depth two of these approaches namely POLQA (Perceived Objective Listening Quality Assessment) and DIAL (Diagnostic Instrumental Assessment of Listening quality), in terms of quality degradation indicators related to the perceptual dimensions these models could embed. The main goal of our work is to find and propose the most robust quality degradation indicators to reliably characterize the impact of degradations relative to the perceptual dimensions described above and to identify the underlying technical causes in super wideband telephone communications [50, 14000]Hz. To do so, the first step of our study was to identify in both models the correspondence between perceptual dimensions and quality degradation indicators. Such indicators could be either present in the model itself or derived from our own investigation of the model. In a second step, we analyzed the performance and robustness of the identified quality degradation indicators on speech samples only impaired by one degradation (representative of one perceptual dimension) at a time. This study highlighted the reliability of some of the quality degradation indicators embedded in the two models under study and stood for a first step in the evaluation of performance of these indicators to quantify the degradation for which they were designed.

Network convergence and QoS for future multimedia services in the VISION project

The emerging use of real-time 3D-based multimedia applications imposes strict quality of service (QoS) requirements on both access and core networks. These requirements and their impact to provide end-to-end 3D videoconferencing services have been studied within the Spanish-funded VISION project, where different scenarios were implemented showing an agile stereoscopic video call that might be offered to the general public in the near future. In view of the requirements, we designed an integrated access and core converged network architecture which provides the requested QoS to end-to-end IP sessions. Novel functional blocks are proposed to control core optical networks, the functionality of the standard ones is redefined, and the signaling improved to better meet the requirements of future multimedia services. An experimental test-bed to assess the feasibility of the solution was also deployed. In such test-bed, set-up and release of end-to-end sessions meeting specific QoS requirements are shown and the impact of QoS degradation in terms of the user perceived quality degradation is quantified. In addition, scalability results show that the proposed signaling architecture is able to cope with large number of requests introducing almost negligible delay.

Perceptually Scalable Extension of H.264

We propose a novel visual scalable video coding (VSVC) framework, named VSVC H.264/AVC. In this approach, the non-uniform sampling characteristic of the human eye is used to modify scalable video coding (SVC) H.264/AVC. We exploit the visibility of video content and the scalability of the video codec to achieve optimal subjective visual quality given limited system resources. To achieve the largest coding gain with controlled perceptual quality degradation, a perceptual weighting scheme is deployed wherein the compressed video is weighted as a function of visual saliency and of the non-uniform distribution of retinal photoreceptors. We develop a resource allocation algorithm emphasizing both efficiency and fairness by controlling the size of the salient region in each quality layer. Efficiency is emphasized on the low quality layer of the SVC. The bits saved by eliminating perceptual redundancy in regions of low interest are allocated to lower block-level distortions in salient regions. Fairness is enforced on the higher quality layers by enlarging the size of the salient regions. The simulation results show that the proposed VSVC framework significantly improves the subjective visual quality of compressed videos.

Perceptually Guided Fast Compression of 3-D Motion Capture Data

A time efficient compression technique, incorporating attention stimulating factors, for motion capture data is proposed. Compression ratios of 25:1 to 30:1 can be achieved with very little noticeable degradation in perceptual quality of animation. Experimental analysis shows that the proposed algorithm is much faster than comparable approaches using wavelets, thereby making our approach feasible for motion capture, transmission, and real-time synthesis on mobile devices, where processing power and memory capacity are limited.

No-Reference Quality Assessment for Networked Video via Primary Analysis of Bit Stream

A no-reference (NR) quality measure for networked video is introduced using information extracted from the compressed bit stream without resorting to complete video decoding. This NR video quality assessment measure accounts for three key factors which affect the overall perceived picture quality of networked video, namely, picture distortion caused by quantization, quality degradation due to packet loss and error propagation, and temporal effects of the human visual system. First, the picture quality in the spatial domain is measured, for each frame, relative to quantization under an error-free transmission condition. Second, picture quality is evaluated with respect to packet loss and the subsequent error propagation. The video frame quality in the spatial domain is, therefore, jointly determined by coding distortion and packet loss. Third, a pooling scheme is devised as the last step of the proposed quality measure to capture the perceived quality degradation in the temporal domain. The results obtained by performance evaluations using MPEG-4 coded video streams have demonstrated the effectiveness of the proposed NR video quality metric.

Single-ended parametric voicing-aware models for live assessment of packetized VoIP conversations

The perceptual quality of VoIP conversations depends tightly on the pattern of packet losses, i.e., the distribution and duration of packet loss runs. The wider (resp. smaller) the inter-loss gap (resp. loss gap) duration, the lower is the quality degradation. Moreover, a set of speech sequences impaired using an identical packet loss pattern results in a different degree of perceptual quality degradation because dropped voice packets have unequal impact on the perceived quality. Therefore, we consider the voicing feature of speech wave included in lost packets in addition to packet loss pattern to estimate speech quality scores. We distinguish between voiced, unvoiced, and silence packets. This enables to achieve better correlation and accuracy between human-based subjective and machine-calculated objective scores.This paper proposes novel no-reference parametric speech quality estimate models which account for the voicing feature of signal wave included in missing packets. Precisely, we develop separate speech quality estimate models, which capture the perceptual effect of removed voiced or unvoiced packets, using elaborated simple and multiple regression analyses. A new speech quality estimate model, which mixes voiced and unvoiced quality scores to compute the overall speech quality score at the end of an assessment interval, is developed following a rigorous multiple linear regression analysis. The input parameters of proposed voicing-aware speech quality estimate models, namely Packet Loss Ratio (PLR) and Effective Burstiness Probability (EBP), are extracted based on a novel Markov model of voicing-aware packet loss which captures properly the feature of packet loss process as well as the voicing property of speech wave included in lost packets. The conceived voicing-aware packet loss model is calibrated at run time using an efficient packet loss event driven algorithm. The performance evaluation study shows that our voicing-aware speech quality estimate models outperform voicing-unaware speech quality estimate models, especially in terms of accuracy over a wide range of conditions. Moreover, it validates the accuracy of the developed parametric no-reference speech quality models. In fact, we found that predicted scores using our speech quality models achieve an excellent correlation with measured scores (>0.95) and a small mean absolute deviation (<0.25) for ITU-T G.729 and G.711 speech CODECs.

A Support Vector Machine Approach for Detection and Localization of Transmission Errors Within Standard H.263++ Decoders

Wireless multimedia services are increasingly becoming popular boosting the need for better quality-of-experience (QoE) with minimal costs. The standard codecs employed by these systems remove spatio-temporal redundancies to minimize the bandwidth required. However, this increases the exposure of the system to transmission errors, thus presenting a significant degradation in perceptual quality of the reconstructed video sequences. A number of mechanisms were investigated in the past to make these codecs more robust against transmission errors. Nevertheless, these techniques achieved little success, forcing the transmission to be held at lower bit-error rates (BERs) to guarantee acceptable quality. This paper presents a novel solution to this problem based on the error detection capabilities of the transport protocols to identify potentially corrupted group-of-blocks (GOBs). The algorithm uses a support vector machine (SVM) at its core to localize visually impaired macroblocks (MBs) that require concealment within these GOBs. Hence, this method drastically reduces the region to be concealed compared to state-of-the-art error resilient strategies which assume a packet loss scenario. Testing on a standard H.263++ codec confirms that a significant gain in quality is achieved with error detection rates of 97.8% and peak signal-to-noise ratio (PSNR) gains of up to 5.33 dB. Moreover, most of the undetected errors provide minimal visual artifacts and are thus of little influence to the perceived quality of the reconstructed sequences.

A framework for end-to-end video quality prediction of MPEG video

This paper proposes, describes and evaluates a novel framework for video quality prediction of MPEG-based video services, considering the perceptual degradation that is introduced by the encoding process and the provision of the encoded signal over an error-prone wireless or wire-line network. The concept of video quality prediction is considered in this work, according to which the encoding parameters of the video service and the network QoS conditions are used for performing an estimation/prediction of the video quality level at the user side, without further processing of the actual encoded and transmitted video content. The proposed prediction framework consists of two discrete models: (i) a model for predicting the video quality of an encoded signal at a pre-encoding stage by correlating the spatiotemporal content dynamics to the bit rate that satisfies a specific level of user satisfaction; and (ii) a model that predicts primarily the undecodable frames (and subsequently the perceived quality degradation caused by them) based on the monitored averaged packet loss ratio of the network. The proposed framework is experimentally tested and validated with video signals encoded according to MPEG-4 standard.

Genetic fingerprinting for copyright protection of multicast media

Generally speaking, comparing to the unicast or broadcast methods, it is more efficient to transmit multimedia data via the multicast method to massive users. However, the ease of delivery of multimedia data may cause the copyright of such multimedia to be easily infringed upon, and the fingerprinting scheme is one of effective means for conquering this problem. Fingerprint embedding process often generates the multimedia contents into many different versions, which have to be transmitted via the unicast method. In this paper, we propose a new genetic fingerprinting scheme for copyright protection of multicast video. In this method, the encryption and decryption keys, which aim at scrambling and descrambling multimedia contents, are first produced with genetic algorithms. Next, multimedia data are then encrypted and multicast to all users. At the same time, a secure channel is employed to unicast a designated decryption key to each user. When a user deploys the designated key to decrypt the received data, a corresponding fingerprint would be embedded into the contents. Once upon the reception of the fingerprinted multimedia content, the embedded fingerprint can be extracted shortly, and the copyright can be confirmed and assured. Experimental results demonstrate that the proposed method can transmit multimedia data to clients effectively and cause only a slight degradation in perceptual quality.

Objective Quality Evaluation Method for Noise-Reduced Speech

We present a method for the objective quality evaluation of noise-reduced speech in wideband speech communication services, which utilize speech with a wider bandwidth (e. g., 7kHz) than the usual telephone bandwidth. Experiments indicate that the amount of residual noise and the distortion of speech and noise, which are quality factors, influence the perceived quality degradation of noise-reduced speech. From the results, we observe the principal relationships between these quality factors and perceived speech quality. On the basis of these relationships, we propose a method that quantifies each quality factor in noise-reduced speech by analyzing signals that can be measured and assesses the over-all perceived quality of noise-reduced speech using values of these quality factors. To verify the validity of the method, we perform a subjective listening test and compare subjective quality of noise-reduced speech with its estimation. In the test, we use various types of background noise and noise-reduction algorithms. The verification results indicate that the correlation between subjective quality and its objective estimation is sufficiently high regardless of the type of background noise and noise-reduction algorithm.

Visual sensitivity guided bit allocation for video coding

A video bit allocation technique adopting a visual distortion sensitivity model for better rate-visual distortion coding control is proposed in this paper. Instead of applying complicated semantic understanding, the proposed automatic distortion sensitivity analysis process analyzes both the motion and the texture structures in the video sequences in order to achieve better bit allocation for rate-constrained video coding. The proposed technique evaluates the perceptual distortion sensitivity on a macroblock basis, and allocates fewer bits to regions permitting large perceptual distortions for rate reduction. The proposed algorithm can be incorporated into existing video coding rate control schemes to achieve same visual quality at reduced bitrate. Experiments based on H.264 JM7.6 show that this technique achieves bit-rate saving of up to 40.61%. However, the conducted subjective viewing experiments show that there is no perceptual quality degradation. EDICS-1-CPRS, 3-QUAL.

Spatial error concealment with edge related perceptual considerations

Video transmission over error-prone networks can suffer from packet erasures which can greatly reduce the quality of the received video. Error concealment methods reduce the perceived quality degradation at the receiving end by masking the effects of such errors. They accomplish this by exploiting temporal and spatial correlations that exist in image sequences. Spatial error concealment approaches conceal errors by making use of spatial information only which is necessary in cases where motion information is not available or reliable. The performance of such methods can be greatly increased if perceptual considerations are taken into account as, e.g., the preservation of edge information. This paper proposes a spatial error concealment method that uses edge-related information in order not only to preserve existing edges but also to avoid introducing new strong ones by switching to a smooth approximation of missing information where necessary. A novel switching algorithm which uses the directional entropy of neighbouring edges chooses between two interpolation methods, a directional along detected edges or a bilinear using the nearest neighbouring pixels. Results show that the performance of the proposed method is better compared to both ‘single interpolation’ and to edge strength-based switching methods.