Objective Quality Assessment Method Research Articles

Review of Image Quality Assessment Methods for Compressed Images.

The compression of images for efficient storage and transmission is crucial in handling large data volumes. Lossy image compression reduces storage needs but introduces perceptible distortions affected by content, compression levels, and display environments. Each compression method generates specific visual anomalies like blocking, blurring, or color shifts. Standardizing efficient lossy compression necessitates evaluating perceptual quality. Objective measurements offer speed and cost efficiency, while subjective assessments, despite their cost and time implications, remain the gold standard. This paper delves into essential research queries to achieve visually lossless images. The paper describes the influence of compression on image quality, appropriate objective image quality metrics (IQMs), and the effectiveness of subjective assessment methods. It also provides an overview of the existing literature, surveys, and subjective and objective image quality assessment (IQA) methods. Our aim is to offer insights, identify challenges in existing methodologies, and assist researchers in selecting the most effective assessment approach for their needs.

Objective Video Quality Assessment Method for Object Recognition Tasks

In the field of video quality assessment for object recognition tasks, accurately predicting the impact of different quality factors on recognition algorithms remains a significant challenge. Our study introduces a novel evaluation framework designed to address this gap by focussing on machine vision rather than human perceptual quality metrics. We used advanced machine learning models and custom Video Quality Indicators to enhance the predictive accuracy of object recognition performance under various conditions. Our results indicate a model performance, achieving a mean square error (MSE) of 672.4 and a correlation coefficient of 0.77, which underscores the effectiveness of our approach in real-world scenarios. These findings highlight not only the robustness of our methodology but also its potential applicability in critical areas such as surveillance and telemedicine.

Non-Intrusive Speech Quality Assessment Based on Deep Neural Networks for Speech Communication.

Traditionally, speech quality evaluation relies on subjective assessments or intrusive methods that require reference signals or additional equipment. However, over recent years, non-intrusive speech quality assessment has emerged as a promising alternative, capturing much attention from researchers and industry professionals. This article presents a deep learning-based method that exploits large-scale intrusive simulated data to improve the accuracy and generalization of non-intrusive methods. The major contributions of this article are as follows. First, it presents a data simulation method, which generates degraded speech signals and labels their speech quality with the perceptual objective listening quality assessment (POLQA). The generated data is proven to be useful for pretraining the deep learning models. Second, it proposes to apply an adversarial speaker classifier to reduce the impact of speaker-dependent information on speech quality evaluation. Third, an autoencoder-based deep learning scheme is proposed following the principle of representation learning and adversarial training (AT) methods, which is able to transfer the knowledge learned from a large amount of simulated speech data labeled by POLQA. With the help of discriminative representations extracted from the autoencoder, the prediction model can be trained well on a relatively small amount of speech data labeled through subjective listening tests. Fourth, an end-to-end speech quality evaluation neural network is developed, which takes magnitude and phase spectral features as its inputs. This phase-aware model is more accurate than the model using only the magnitude spectral features. A large number of experiments are carried out with three datasets: one simulated with labels obtained using POLQA and two recorded with labels obtained using subjective listening tests. The results show that the presented phase-aware method improves the performance of the baseline model and the proposed model with latent representations extracted from the adversarial autoencoder (AAE) outperforms the state-of-the-art objective quality assessment methods, reducing the root mean square error (RMSE) by 10.5% and 12.2% on the Beijing Institute of Technology (BIT) dataset and Tencent Corpus, respectively. The code and supplementary materials are available at https://github.com/liushenme/AAE-SQA.

Panoramic radiograph quality assessment: Database and algorithm

High-quality panoramic radiographs are crucial for providing accurate diagnosis and appropriate treatment to the patients, which have been widely employed in dentist clinic. Unfortunately, panoramic radiographs can be bothered with various distortions during the capture and imaging process, which can further negatively affect the judgment of dentists. Therefore, to deal with the challenge of panoramic radiographs quality assessment, we first carry out a large scale panoramic radiograph quality assessment database, which contains 2009 images labeled by experienced dentists. Then we propose a novel objective quality assessment method based on the convolutional neural network (CNN). Specifically, the multi-scale features from the different stages of the CNN are employed to make full use of both low-level and high-level information. Moreover, weight-sharing auxiliary networks are utilized to improve the quality understanding of the proposed method. The experimental results show that the proposed method outperforms all the compared methods for panoramic radiographs quality assessment tasks and is more capable of handling the practical medical diagnosis situations. The proposed method can also be used as preliminary screening tool and provide useful guidelines for the dentists.

A survey on visual quality assessment methods for light fields

Unlike traditional cameras, Light Field (LF) cameras are able to capture both the intensity and direction of light rays from the scene. This rich information demands a certain amount of memory and bandwidth for storage and transmission, and to alleviate this requirement, LF content is processed and compressed. These operations often add degradations to the LF content that can affect its visual quality, requiring the use of methods that can measure visual quality as perceived by the end consumer. For this reason, the area of visual quality of immersive media has been gaining a lot of attention. This paper provides a comprehensive survey of state-of-the-art visual quality assessment methods for LF contents. First, we discuss the LF quality datasets that have been introduced, which contain subjective quality scores. Second, we discuss subjective quality assessment methods followed by objective quality assessment methods. Then we present a statistical comparison of a set of objective visual quality assessment methods. Finally, we summarize this survey and present future research demands for LF contents.

Objective Video Quality Assessment Method for Face Recognition Tasks

Nowadays, there are many metrics for overall Quality of Experience (QoE), both those with Full Reference (FR), such as Peak Signal-to-Noise Ratio (PSNR) or Structural Similarity (SSIM), and those with No Reference (NR), such as Video Quality Indicators (VQI), which are successfully used in video processing systems to evaluate videos whose quality is degraded by different processing scenarios. However, they are not suitable for video sequences used for recognition tasks (Target Recognition Videos, TRV). Therefore, correctly estimating the performance of the video processing pipeline in both manual and Computer Vision (CV) recognition tasks is still a major research challenge. There is a need for objective methods to evaluate video quality for recognition tasks. In response to this need, we show in this paper that it is possible to develop the new concept of an objective model for evaluating video quality for face recognition tasks. The model is trained, tested and validated on a representative set of image sequences. The set of degradation scenarios is based on the model of a digital camera and how the luminous flux reflected from the scene eventually becomes a digital image. The resulting degraded images are evaluated using a CV library for face recognition as well as VQI. The measured accuracy of a model, expressed as the value of the F-measure parameter, is 0.87.

Point cloud quality assessment based on geometry-aware texture descriptors

In recent years, 3D imaging technologies have advanced tremendously, allowing more faithful representations of the physical world. More specifically, imaging technology advancements have driven the production of plenoptic devices that can capture and display visual contents. These devices represent the visual data using an approximation of the plenoptic illumination function, which can describe visible objects in any position and from any point-of-view of the 3D space. Depending on the capturing device, this approximation can correspond to holograms, light fields, or Point Clouds (PCs) imaging formats. Among these formats, PCs have become very popular for a wide range of applications, such as immersive virtual reality scenarios. As a consequence, in the last couple of years, there has been a great effort to develop novel acquisition, representation, compression, and transmission solutions for PC contents in the research community. In particular, the development of objective quality assessment methods that are able to predict the perceptual quality of PCs has attracted a lot of attention. In this paper, we present an effective framework for assessing the quality of PCs, which is based on descriptors that extract geometry-aware texture information of PC contents. In this framework, the statistics of the extracted information are used to model the PC visual quality. We also present the research and experiments carried to evaluate the most appropriate distance metrics and regression methods to be used together with the proposed descriptors. Experimental results show that the proposed framework exhibit good and robust performance when compared with several state-of-the-art Point Cloud Quality Assessment (PCQA) methods. A C++ implementation of the metrics described in this paper can be found at https://gitlab.com/gpds-unb/pc_metric.

Rich Structural Index for Stereoscopic Image Quality Assessment.

The human visual system (HVS), affected by viewing distance when perceiving the stereo image information, is of great significance to study of stereoscopic image quality assessment. Many methods of stereoscopic image quality assessment do not have comprehensive consideration for human visual perception characteristics. In accordance with this, we propose a Rich Structural Index (RSI) for Stereoscopic Image objective Quality Assessment (SIQA) method based on multi-scale perception characteristics. To begin with, we put the stereo pair into the image pyramid based on Contrast Sensitivity Function (CSF) to obtain sensitive images of different resolution. Then, we obtain local Luminance and Structural Index (LSI) in a locally adaptive manner on gradient maps which consider the luminance masking and contrast masking. At the same time we use Singular Value Decomposition (SVD) to obtain the Sharpness and Intrinsic Structural Index (SISI) to effectively capture the changes introduced in the image (due to distortion). Meanwhile, considering the disparity edge structures, we use gradient cross-mapping algorithm to obtain Depth Texture Structural Index (DTSI). After that, we apply the standard deviation method for the above results to obtain contrast index of reference and distortion components. Finally, for the loss caused by the randomness of the parameters, we use Support Vector Machine Regression based on Genetic Algorithm (GA-SVR) training to obtain the final quality score. We conducted a comprehensive evaluation with state-of-the-art methods on four open databases. The experimental results show that the proposed method has stable performance and strong competitive advantage.

Video Quality Assessment of Danmaku-Based Video Saliency Regions

At present, objective quality assessment methods have significant advantages in video quality assessment (VQA), among which the objective assessment based on deep learning is superior in processing speed and accuracy. However, the objective assessment method has the limitation of poor interpretability of experimental results and lacks the participation of human thinking and experience. In this paper, we present a quality assessment method based on the danmaku visual saliency region. First, to obtain the sentiment trend of the danmaku, we conduct a sentiment analysis of the content of the danmaku. Then, to speed up the video quality assessment, we select the keyframes of the video according to the sentiment analysis. Finally, we innovatively use human visual characteristics to extract salient regions from video keyframes to make the results more subjective. The experiments show that the Pearson correlation coefficient (PCC) and root mean square error (RMSE) between the results of the proposed method and the subjective evaluation are improved, which conforms to the audience's subjective feelings about video viewing, and the assessment results are more accurate.

Perceptual Quality of Light Field Images and Impact of Visualization Techniques

Light Field imaging provides a wide range of interactive features, such as view point changing and refocusing, and it has been developing as a solution for six degree of freedom applications. Many ongoing efforts, specifically related to pre/post processing, compression, and rendering, have been devoted to the development of this technology. To benchmark processing algorithms, the knowledge of degraded or improved quality level is important. However, the definition of validated subjective and objective quality assessment methods is still an open and challenging issue. In this scenario, this contribution deals with: the impact of visualization techniques on Light Filed image Quality of Experience, a visualization technique recommended for subjective quality assessment, perceptual visual impact of compression and noise artifacts, performance analysis of 2D image quality measures applied to Light Field images, and a Light Field image quality dataset.

Combining Retargeting Quality and Depth Perception Measures for Quality Evaluation of Retargeted Stereopairs

Stereoscopic Image Retargeting (SIR) aims to adapt stereoscopic images and videos to 3D display devices with various aspect ratios by emphasizing the important content while retaining surrounding context with minimal visual distortion. To address the issue of SIR evaluation, this paper presents a new objective quality assessment method for retargeted stereopairs by combining image quality and depth perception measures. Specifically, the image quality measure is conducted between the source and retargeted intermediate views generated by the view synthesis method to characterize the geometric distortion and content loss of the retargeted stereopair, while several depth-aware features are extracted to measure the visual comfort/discomfort and depth sensation when human views a 3D scene. Then, the extracted features are integrated into an overall perceptual quality prediction. Experiment results on NBU SIRQA and SIRD databases verify the superiority of our method.

On the performance of objective quality metrics for lightfields

Lightfield (LF) technology has attained significant attention in recent years due to its capability to capture much richer textural and geometric information in the scene compared to the classical 2D representation. The resampling and compression operations on LFs often lead to visual quality degradation, thus, sophisticated visual quality assessment methods play a crucial role to ensure a pleasant viewing experience. To this end, it is necessary to examine the performance of quality assessment methods for LF contents. The paper provides a comprehensive study on the reliability of various objective algorithms for LF quality prediction. Three subjectively-annotated LF data sets were selected and an extensive quality estimation analysis has been conducted using several objective quality assessment methods. In total, 250 LFs (more than 48000 perspective images) were evaluated. The results were compared against human opinion scores using various correlation indices and their statistical significance. Next, a decision-making strategy was adopted to choose the most reliable quality metrics for evaluation of LFs and finally, a metric fusion framework was proposed to further improve the quality prediction accuracy. To best of our knowledge, the benchmark and the analytical methodologies used in this paper are the most comprehensive study on the objective quality assessment methods for LF application.

A novel point cloud quality assessment metric based on perceptual color distance patterns

In recent years, PCs have become very popular for a wide range of applications, such as immersive virtual reality scenarios. As a consequence, in the last couple of years, there has been a great effort to develop novel acquisition, representation, compression, and transmission solutions for PC contents in the research community. In particular, the development of objective quality assessment methods that are able to predict the perceptual quality of PCs. In this paper, we present an effective novel method for assessing the quality of PCs, which is based on descriptors that extract perceptual color distance-based texture information of PC contents, called Perceptual Color Distance Patterns (PCDP). In this framework, the statistics of the extracted information are used to model the PC visual quality. Experimental results show that the proposed framework exhibit good and robust performance when compared with several state-of-the-art point cloud quality assessment (PCQA) methods.

An analytic hierarchy process multi-criteria quantitative evaluation model of cinnamon decoction pieces

Background: The dried bark of Cinnamomum zeylanicum is a well-known medicinal constituent owing to its latent health benefits. The establishment of a wide-ranging and objective quality assessment method is a tricky to be solved instantly by cinnamon and other medicinal and edible herbs. Objectives: The objective was to introduce the analytic hierarchy process (AHP) to estimate widely and compare the quality of cinnamon decoction pieces. Materials and Methods: The chemical, morphological, and biological indexes of different batches of cinnamon decoction pieces are determined, and then a multi-criteria quantitative assessment model for the quality estimation of cinnamon decoction pieces is documented by adopting AHP. Results: The complete appraisal outcomes of AHP model are dependable with objective evaluation, and the results are unbiased and steady. Conclusion: The AHP multi-criteria quantitative evaluation model can be agreeably adapted to the quality evaluation of multi-component Chinese herbal decoction pieces, providing a new evidence for the quality evaluation of traditional Chinese medicine.

A generalized quality assessment method for natural and screen content images

A generalized objective quality assessment method is proposed for natural images andscreen content images. Since natural images and screen content images have different sta-tistical properties, the modelling of a generalized quality assessment method that worksfor both types of images is complicated because some properties of natural imag es andscreen content images are conflicting to one another. The proposed method assesses theperceptual quality of an image based on edge magnitude and direction. In this method, animage is first separated into regions with high and low gradients. Gradient is used due tothe small perceptual span of the human visual system for textual content. For high gradientregions, small kernel size of Prewitt operators is used to obtain the gradient magnitude anddirection. Correspondingly, bigger kernel size of Prewitt operators is utilized for low gra-dient regions. Visual quality indices are computed from both regions and pooled to obtainthe final quality index. From the performance comparison, it is shown that the proposedmethod could assess the perceived quality of natural images and screen content imageswith high accuracy.

IJNM Special Issue—International Journal of Network Management: QoE‐centric analysis and management of communication networks

IJNM Special Issue—International Journal of Network Management: QoE‐centric analysis and management of communication networks

A Novel Objective Quality Assessment Method for Transcoded Videos From H.264/AVC to H.265/HEVC Utilizing Probability Theory

The latest video coding standard H.265/HEVC is developed to succeed the previous coding standard H.264/AVC. However, a large amount of legacy content was coded with H.246/AVC. Therefore, transcoding from H.264/AVC to H.265/HEVC format is required. During the process of transcoding, we can easily calculate the distortion of the transcoded video with respect to the H.264/AVC-decoded video. However, since the original video is usually unavailable, the distortion between the original video and the transcoded video is unknown, which makes it difficult to control the coding quality of the transcoded video compared to the original video. In this paper, we propose a novel and accurate quality estimation method for transcoded videos utilizing probability theory. Experimental results demonstrate that the predicted quality of transcoded videos approximate the true value, with an average error of 0.28 dB, 0.41 dB, and 0.46 dB for Y, Cb, and Cr components, respectively.

Stochastic QoE-aware optimization of multisource multimedia content delivery for mobile cloud

The increasing popularity of mobile video streaming in wireless networks has stimulated growing demands for efficient video streaming services. However, due to the time-varying throughput and user mobility, it is still difficult to provide high quality video services for mobile users. Our proposed optimization method considers key factors such as video quality, bitrate level, and quality variations to enhance quality of experience over wireless networks. The mobile network and device parameters are estimated in order to deliver the best quality video for the mobile user. We develop a rate adaptation algorithm using Lyapunov optimization for multi-source multimedia content delivery to minimize the video rate switches and provide higher video quality. The multi-source manager algorithm is developed to select the best stream based on the path quality for each path. The node joining and cluster head election mechanism update the node information. As the proposed approach selects the optimal path, it also achieves fairness and stability among clients. The quality of experience feature metrics like bitrate level, rebuffering events, and bitrate switch frequency are employed to assess video quality. We also employ objective video quality assessment methods like VQM, MS-SSIM, and SSIMplus for video quality measurement closer to human visual assessment. Numerical results show the effectiveness of the proposed method as compared to the existing state-of-the-art methods in providing quality of experience and bandwidth utilization.

Ranking-Preserving Cross-Source Learning for Image Retargeting Quality Assessment.

Image retargeting techniques adjust images into different sizes and have attracted much attention recently. Objective quality assessment (OQA) of image retargeting results is often desired to automatically select the best results. Existing OQA methods train a model using some benchmarks (e.g., RetargetMe), in which subjective scores evaluated by users are provided. Observing that it is challenging even for human subjects to give consistent scores for retargeting results of different source images (diff-source-results), in this paper we propose a learning-based OQA method that trains a General Regression Neural Network (GRNN) model based on relative scores-which preserve the ranking-of retargeting results of the same source image (same-source-results). In particular, we develop a novel training scheme with provable convergence that learns a common base scalar for same-source-results. With this source specific offset, our computed scores not only preserve the ranking of subjective scores for same-source-results, but also provide a reference to compare the diff-source-results. We train and evaluate our GRNN model using human preference data collected in RetargetMe. We further introduce a subjective benchmark to evaluate the generalizability of different OQA methods. Experimental results demonstrate that our method outperforms ten representative OQA methods in ranking prediction and has better generalizability to different datasets.

A novel objective quality assessment method for video conferencing coding

Most recently, due to the demand of immersive communication, region-of-interest-based (ROI) high efficiency video coding (HEVC) approaches in conferencing scenarios have become increasingly important. However, there exists no objective metric, specially developed for efficiently evaluating the perceived visual quality of video conferencing coding. Therefore, this paper proposes a novel objective quality assessment method, namely Gaussian mixture model based peak signal-to-noise ratio (GMM-PSNR), for the perceptual video conferencing coding. First, eye tracking experiments, together with a real-time technique of face and facial feature extraction, are introduced. In the experiments, importance of background, face, and facial feature regions is identified, and it is then quantified based on eye fixation points over test videos. Next, assuming that the distribution of the eye fixation points obeys Gaussian mixture model, we utilize expectation-maximization (EM) algorithm to generate an importance weight map for each frame of video conferencing coding, in light of a new term eye fixation points/pixel (efp/p). According to the generated weight map, GMM-PSNR is developed for quality assessment by assigning different weights to the distortion of each pixel in the video frame. Finally, we utilize some experiments to investigate the correlation of the proposed GMM-PSNR and other conventional objective metrics with subjective quality metrics. The experimental results show the effectiveness of GMM-PSNR.