No-reference Metrics Research Articles

PFF-RVM: A new no reference image quality measure

In this paper, a new no-reference image quality metric is proposed and developed. A set of ten perceptual features are extracted from each distorted image, then Relevance Vector Machine (RVM) is employed to learn the mapping between the combined features and human opinion scores. Validation tests and simulations are conducted on the LIVE II and MDID 2013 databases. The predictive performances of this new metric (we called PFF-RVM for perceptual features fusion using relevance vector machine based metric) are compared to the most recent no-reference metrics in terms of correlation and monotonicity. Results show that the proposed metric has satisfactory and comparable performances to the most sophisticated and commonly used no-reference quality metrics in the state-of-the-art.

HLS Based Approach to Develop an Implementable HDR Algorithm

Hardware suitability of an algorithm can only be verified when the algorithm is actually implemented in the hardware. By hardware, we indicate system on chip (SoC) where both processor and field-programmable gate array (FPGA) are available. Our goal is to develop a simple algorithm that can be implemented on hardware where high-level synthesis (HLS) will reduce the tiresome work of manual hardware description language (HDL) optimization. We propose an algorithm to achieve high dynamic range (HDR) image from a single low dynamic range (LDR) image. We use highlight removal technique for this purpose. Our target is to develop parameter free simple algorithm that can be easily implemented on hardware. For this purpose, we use statistical information of the image. While software development is verified with state of the art, the HLS approach confirms that the proposed algorithm is implementable to hardware. The performance of the algorithm is measured using four no-reference metrics. According to the measurement of the structural similarity (SSIM) index metric and peak signal-to-noise ratio (PSNR), hardware simulated output is at least 98.87 percent and 39.90 dB similar to the software simulated output. Our approach is novel and effective in the development of hardware implementable HDR algorithm from a single LDR image using the HLS tool.

Combining Local and Global Measures for DIBR-Synthesized Image Quality Evaluation.

Depth-Image-Based-Rendering (DIBR) techniques are significant for three-dimensional (3D) video applications, e.g., 3D television and free viewpoint video (FVV). Unfortunately, the DIBR-synthesized image suffers from various distortions, which induce an annoying viewing experience for the entire FVV. Proposing a quality evaluator for DIBR-synthesized images is fundamental for the design of perceptual friendly FVV systems. Since the associated reference image is usually not accessible, full-reference (FR) methods cannot be directly applied for quality evaluation of the synthesized image. In addition, most traditional no-reference (NR) methods fail to effectively measure the specifically DIBR-related distortions. In this paper, we propose a novel NR quality evaluation method accounting for two categories of DIBR-related distortions, i.e., geometric distortions and sharpness. First, the disoccluded regions, as one of the most obvious geometric distortions, are captured by analyzing local similarity. Then, another typical geometric distortion (i.e., stretching) is detected and measured by calculating the similarity between it and its equal-size adjacent region. Second, considering the property of scale invariance, the global sharpness is measured as the distance between the distorted image and its downsampled version. Finally, the perceptual quality is estimated by linearly pooling the scores of two geometric distortions and sharpness together. Experimental results verify the superiority of the proposed method over the prevailing FR and NR metrics. More specifically, it is superior to all competing methods except APT in terms of effectiveness, but greatly outmatches APT in terms of implementation time.

A Novel No-Reference Metric for Estimating the Impact of Frame Freezing Artifacts on Perceptual Quality of Streamed Videos

Online monitoring of multimedia networks is required to ensure seamless and ubiquitous delivery of services to the end users. Quality of multimedia content, such as video streams, often gets degraded due to network losses such as packet loss. Frame freezing artifacts are introduced in a video stream when packet loss or packet delay takes place. Estimating the perceptual impact of these artifacts on quality of experience of end users helps service providers to maintain quality of service. In this paper, we have presented a novel no-reference video quality metric, which measures the impact of frame freezing due to packet loss and delay in video streaming networks. The proposed metric is based on several features that directly impact the quality of experience of end users. These features, including motion characteristics of videos, are calculated using the temporal information between video frames and then combined mathematically to form a video quality metric. Different weights are assigned to different features for better performance of the proposed metric. With detailed experiments, we have shown that our method outperforms other contemporary methods in terms of high accuracy and low computation time in frame freeze detection, low root mean square values, high coefficient of determination, and high correlation between subjective and objective measurements. We have used five video databases for our model's evaluation and validation. Furthermore, we have shown that our method is statistically superior to the other models in comparison.

Pansharpening for Multiband Images With Adaptive Spectral–Intensity Modulation

The pansharpening algorithm often faces an imbalance between spatial sharpness and spectral preservation, resulting in spectral and intensity inhomogeneities in the fused image. In this paper, to overcome this problem, we present a robust pansharpening method for multiband images with adaptive spectral–intensity modulation. In this method, we propose an adaptive spectral modulation coefficient (ASMC) and an adaptive intensity modulation coefficient (AIMC) to modulate the spectral and spatial information in the fused image, respectively. Among these coefficients, the ASMC is constructed based on two aspects: first, the details extracted from the panchromatic (PAN) and multispectral (MS) images; and second, the spectral relationship between each MS band. The AIMC is calculated by assessing the correlation and standard deviation between the PAN image and each MS band. Finally, we propose a mathematically linear model to combine ASMC and AIMC to achieve the fused image. Various remote-sensing satellite images were used in the evaluations. Experimental results indicate that the proposed method achieves outstanding performance in balancing spatial and spectral information and outperforms several state-of-the-art fusion methods in terms of both full-reference and no-reference metrics, and on visual inspection.

Benchmarking Single Image Dehazing and Beyond.

In this paper, we present a comprehensive study and evaluation of existing single image dehazing algorithms, using a new large-scale benchmark consisting of both synthetic and real-world hazy images, called REalistic Single Image DEhazing (RESIDE). RESIDE highlights diverse data sources and image contents, and is divided into five subsets, each serving different training or evaluation purposes. We further provide a rich variety of criteria for dehazing algorithm evaluation, ranging from full-reference metrics, to no-reference metrics, to subjective evaluation and the novel task-driven evaluation. Experiments on RESIDE shed light on the comparisons and limitations of stateof- the-art dehazing algorithms, and suggest promising future directions.

Perceptual quality evaluation for motion deblurring

Motion deblurring has been widely studied. However, the relevant quality evaluation of motion deblurred images remains an open problem. The motion deblurred images are usually contaminated by noise, ringing and residual blur (NRRB) simultaneously. Unfortunately, most of the existing quality metrics are not designed for multiply distorted images, so they are limited in predicting the quality of motion deblurred images. In this study, the authors propose a new quality metric for motion deblurred images by measuring NRRB. For a motion deblurred image, the noise level is first estimated. Then the ringing effect is measured by incorporating visual saliency model to adapt to the characteristic of the human visual system. A reblurring‐based method is proposed to extract similarity features between a motion deblurred image and its re‐blurred version for evaluating the residual blur. Finally, the overall quality score of a motion deblurred image is obtained by pooling the scores of noise, ringing and blur. Experimental results conducted on a motion deblurring database demonstrate that the proposed metric significantly outperforms the existing quality metrics. In addition, the proposed NRRB metric is used for improving the existing general‐purpose no‐reference metrics, and very encouraging results are achieved.

Efficient No-Reference Quality Assessment and Classification Model for Contrast Distorted Images

In this paper, an efficient Minkowski Distance based Metric (MDM) for no-reference (NR) quality assessment of contrast distorted images is proposed. It is shown that higher orders of Minkowski distance and entropy provide accurate quality prediction for the contrast distorted images. The proposed metric performs predictions by extracting only three features from the distorted images followed by a regression analysis. Furthermore, the proposed features are able to classify type of the contrast distorted images with a high accuracy. Experimental results on four datasets CSIQ, TID2013, CCID2014, and SIQAD show that the proposed metric with a very low complexity provides better quality predictions than the state-of-the-art NR metrics. The MATLAB source code of the proposed metric is available to public at http://www.synchromedia.ca/system/files/MDM.zip.

A Multi-Scale Learning Local Phase and Amplitude Blind Image Quality Assessment for Multiply Distorted Images

In the real application, digital images may undergo the process of acquisition, compression, and transmission, which causes the excess blurring, quantization, and noise. However, the metrics of image quality assessment for multiply distorted images are very limited. In this paper, we propose a new multi-scale learning quality-aware features blind image quality assessment algorithm for multiply distorted images by using both local phase and local amplitude. In the new model, a distorted image is decomposed into three scales by Gabor transform, and its phase congruency image (PCI), phase congruency covariance maximum image (PCCmax), and phase congruency covariance minimum image (PCCmin) are produced. Then, we calculate contrast sensitivity function and gray level-gradient co-occurrence matrix features from distorted image and its PCI, PCCmax, and PCCmin, and mean value of intensity of PCI, PCCmax, PCCmin, and overlapping blocked local amplitude of distorted image. At last, SVR is used to build the approximating function between these features and subjective mean opinion scores. Both local phase and local amplitude features are extracted at multi-scale images, which supply more flexibility than the previous single-scale methods in incorporating the variations of viewing scene. Comparative experiments between our proposed metric and the state-of-the-art full-reference and no-reference metrics are conducted on two newly released multiply distorted image databases (LIVEMD and MDID2013) that demonstrate the effectiveness of our proposed method.

A novel reversible data hiding method with image contrast enhancement

Recently, several image contrast enhancement methods have been proposed such that the original image can be recovered from its contrast-enhanced version. Hence a flexibility in changing image contrast can be provided when needed. However, the artificial distortions may be introduced into the image content after adopting these methods. Meanwhile, there is lack of using the adequate image quality metrics for performance evaluation. In this paper, a novel reversible data hiding method is proposed for image contrast enhancement. To better preserve image quality, it is restricted that only the adjacent bins in the original image histogram may be merged in the pre-processing. The proposed method has been applied to two image sets and compared with the previous methods. For image quality assessment, the PSNR, SSIM and three no-reference metrics have been adopted in performance evaluation. The experimental results have clearly shown that better visual quality can be achieved with the proposed method. Besides recovering the original images, extra data can be hidden into the contrast-enhanced images and correctly extracted.

NO-REFERENCE QUANTITATIVE ASSESSMENT OF GENERALIZED CONTRAST FOR COMPLEX IMAGES

Context. Nowadays the task of automatically measuring of image quality in real time is extremely relevant for the vast majority of practical applications. No-reference quantitative assessment of image quality is one of the most pressing and difficult problems of image processing. Generalized contrast is the most important quantitative characteristic which determines the objective quality of the image. Currently, the development of new effective methods of no-reference measuring of generalized contrast for complex image in automatic mode which have the level of computing costs, which are acceptable to implement the processing in real time, is one of the most urgent tasks of image pre-processing.Objective. Development of new histogram-based method for no-reference measurement of generalized contrast of complex images on the basis of the mean value for all contrast values of all pairs of image elements (objects and background) for various definitions of contrast kernel.Method. Analysis of known approaches to measurement of a local contrast of the image elements, of known methods of the quantitative assessment of generalized contrast of complex images as well of the results of experimental research for a series of complex real and test images allowed to reveal inherent patterns (accordance to basic requirements to the definition of contrast, the nature and the dynamic of contrast changes at the linear transformations of the brightness scale), which are manifested depending on the use of the different definitions of the contrast kernels and the metrics of generalized contrast of images.Results. New histogram-based method for no-reference measurement of generalized contrast for complex images is proposed. No-reference contrast metrics for the histogram-based measuring of generalized contrast of complex images on the basis of the average contrast of image elements for different definitions of contrast kernel is proposed.Conclusions. Proposed no-reference metrics on the basis of the average contrast of image elements for proposed contrast kernels allow providing accurate quantitative assessment of generalized contrast of the real complex images and enable to evaluate (predict) with reasonable accuracy the perceived image quality at carrying out of subjective (qualitative) expert estimates.

No-Reference Image Quality Assessment for Image Auto-Denoising

This paper proposes two new non-reference image quality metrics that can be adopted by the state-of-the-art image/video denoising algorithms for auto-denoising. The first metric is proposed based on the assumption that the noise should be independent of the original image. A direct measurement of this dependence is, however, impractical due to the relatively low accuracy of existing denoising method. The proposed metric thus tackles the homogeneous regions and highly-structured regions separately. Nevertheless, this metric is only stable when the noise level is relatively low. Most denoising algorithms reduce noise by (weighted) averaging repeated noisy measurements. As a result, another metric is proposed for high-level noise based on the fact that more noisy measurements will be required when the noise level increases. The number of measurements before converging is thus related to the quality of noisy images. Our patch-matching based metric proposes to iteratively find and add noisy image measurements for averaging until there is no visible difference between two successively averaged images. Both metrics are evaluated on LIVE2 (Sheikh et al. in LIVE image quality assessment database release 2: 2013) and TID2013 (Ponomarenko et al. in Color image database tid2013: Peculiarities and preliminary results: 2005) data sets using standard Spearman and Kendall rank-order correlation coefficients (ROCC), showing that they subjectively outperforms current state-of-the-art no-reference metrics. Quantitative evaluation w.r.t. different level of synthetic noisy images also demonstrates consistently higher performance over state-of-the-art non-reference metrics when used for image denoising.

Metrics and methods of video quality assessment: a brief review

With rapid development of video acquisition devices and wide applications of video data, more and more requirements are established to use video data query. video quality assessment and improvement become popular and important research issues which attract lots of researchers. The video quality can influence technique application, user experience, and application results. This paper firstly reviews research work on video query based applications. Then, various metrics of video quality assessment are reviewed according to the requirement of reference video, including full-reference metrics, no-reference metrics and reduced-reference metrics. In addition, methods of video quality assessment are reviewed by the features, which include visual features, bitstream-based or packet-based features, data features. Finally, a number of video quality improvement methods are briefly introduced.

HISTOGRAM-BASED METHOD FOR NO-REFERENCE MEASUREMENT OF GENERALIZED CONTRAST OF COMPLEX IMAGES

Background. Nowadays the task of automatically measuring of image quality in real time is extremely relevant for the vast majority of practical applications. No-reference quantitative assessment of image quality is one of the most pressing and difficult problems of image processing. Generalized contrast is the most important quantitative characteristic which determines the objective quality of the image. Currently, the development of new effective methods of no-reference measuring of generalized contrast for complex image in automatic mode with the level of computing costs, which are acceptable to implement the processing in real time, is one of the most urgent tasks of image preprocessing. Objective. Development of new histogram-based method for no-reference measurement of generalized contrast of complex (multi-element) images based on the average contrast of image elements (objects and background) for different definitions of contrast kernel. Methods. Analysis of known approaches to measurement of a local contrast of the image elements, of known methods of the quantitative assessment of generalized contrast of complex images as well as of the experimental research results for a series of complex real and test images allowed revealing inherent patterns (accordance to basic requirements to the definition of contrast, the nature and the dynamic of contrast changes at the linear transformations of the brightness scale), which are manifested depending on the use of the different definitions of the contrast kernels and the metrics of generalized contrast of images. Results. No-reference contrast metrics for the histogram-based measuring of generalized contrast of complex images based on the average contrast of image elements for different definitions of contrast kernel is proposed. Conclusions. Proposed no-reference metrics based on the average contrast of image elements for proposed contrast kernels allow providing accurate quantitative assessment (measurement) of generalized contrast of the real complex images and enable to evaluate (predict) with reasonable accuracy the perceived image quality at carrying out of subjective (qualitative) expert estimates.

Predictive no-reference assessment of video quality

Among the various means to evaluate the quality of video streams, light-weight No-Reference (NR) methods have low computation and may be executed on thin clients. Thus, these methods would be perfect candidates in cases of real-time quality assessment, automated quality control and in adaptive mobile streaming. Yet, existing real-time, NR approaches are not typically designed to tackle network distorted streams, thus performing poorly when compared to Full-Reference (FR) algorithms. In this work, we present a generic NR method whereby machine learning (ML) may be used to construct a quality metric trained on simplistic NR metrics. Testing our method on nine, representative ML algorithms allows us to show the generality of our approach, whilst finding the best-performing algorithms. We use an extensive video dataset (960 video samples), generated under a variety of lossy network conditions, thus verifying that our NR metric remains accurate under realistic streaming scenarios. In this way, we achieve a quality index that is comparably as computationally efficient as typical NR metrics and as accurate as the FR algorithm Video Quality Metric (97% correlation).

Hyperspectral face recognition using improved inter-channel alignment based on qualitative prediction models.

A fundamental limitation of hyperspectral imaging is the inter-band misalignment correlated with subject motion during data acquisition. One way of resolving this problem is to assess the alignment quality of hyperspectral image cubes derived from the state-of-the-art alignment methods. In this paper, we present an automatic selection framework for the optimal alignment method to improve the performance of face recognition. Specifically, we develop two qualitative prediction models based on: 1) a principal curvature map for evaluating the similarity index between sequential target bands and a reference band in the hyperspectral image cube as a full-reference metric; and 2) the cumulative probability of target colors in the HSV color space for evaluating the alignment index of a single sRGB image rendered using all of the bands of the hyperspectral image cube as a no-reference metric. We verify the efficacy of the proposed metrics on a new large-scale database, demonstrating a higher prediction accuracy in determining improved alignment compared to two full-reference and five no-reference image quality metrics. We also validate the ability of the proposed framework to improve hyperspectral face recognition.

Blind Quality Metric for Multidistortion Images Based on Cartoon and Texture Decomposition

In this letter, a no-reference (NR) hybrid image quality assessment (IQA) metric based on cartoon-texture decomposition (CTD) is presented. Focusing on images distorted by both blur and noise, the method takes properties of CTD to separate image into cartoon part with salient edges and texture part with noises. Then, the blur degree and noise level can be estimated separately from different image parts, combine with a joint effect prediction between blur and noise distortions, we present a cartoon-texture decomposition-based blind metric (CTDBBM). Comparative studies with classical full-reference IQA metrics and state-of-the-art NR metrics are conducted on multidistortion image database: LIVEMD. Experimental results show that the CTDBBM performs well and has a high consistency with the human opinions given in the database.

Pansharpening by Convolutional Neural Networks

A new pansharpening method is proposed, based on convolutional neural networks. We adapt a simple and effective three-layer architecture recently proposed for super-resolution to the pansharpening problem. Moreover, to improve performance without increasing complexity, we augment the input by including several maps of nonlinear radiometric indices typical of remote sensing. Experiments on three representative datasets show the proposed method to provide very promising results, largely competitive with the current state of the art in terms of both full-reference and no-reference metrics, and also at a visual inspection.

An experimental survey of no-reference video quality assessment methods

Purpose The Video Quality Metric (VQM) is one of the most used objective methods to assess video quality, because of its high correlation with the human visual system (HVS). VQM is, however, not viable in real-time deployments such as mobile streaming, not only due to its high computational demands but also because, as a Full Reference (FR) metric, it requires both the original video and its impaired counterpart. In contrast, No Reference (NR) objective algorithms operate directly on the impaired video and are considerably faster but loose out in accuracy. The purpose of this paper is to study how differently NR metrics perform in the presence of network impairments. Design/methodology/approach The authors assess eight NR metrics, alongside a lightweight FR metric, using VQM as benchmark in a self-developed network-impaired video data set. This paper covers a range of methods, a diverse set of video types and encoding conditions and a variety of network impairment test-cases. Findings The authors show the extent by which packet loss affects different video types, correlating the accuracy of NR metrics to the FR benchmark. This paper helps identifying the conditions under which simple metrics may be used effectively and indicates an avenue to control the quality of streaming systems. Originality/value Most studies in literature have focused on assessing streams that are either unaffected by the network (e.g. looking at the effects of video compression algorithms) or are affected by synthetic network impairments (i.e. via simulated network conditions). The authors show that when streams are affected by real network conditions, assessing Quality of Experience becomes even harder, as the existing metrics perform poorly.

QoE rating performance evaluation of ITU-T recommended video quality metrics in the context of video freezes

In real-time video streaming, video quality can be degraded due to network performance issues. Among other artefacts, video freezing and video jumping are factors that influence user experience. Service providers, operators and manufacturers are interested in evaluating the quality of experience (QoE) objectively because subjective assessment of QoE is expensive and, in many user cases, subjective assessment is not possible to perform. Different algorithms have been proposed and implemented in this regard. Some of them are in the recommendation list of the ITU Telecommunication Standardization Sector (ITU-T). In this paper, we study the effect of the freezing artefact on user experience and compare the mean opinion score of these videos with the results of two algorithms, the perceptual evaluation of video quality (PEVQ) and temporal quality metric (TQM). Both metrics are part of the ITU-T Recommendation J.247 Annex B and C. PEVQ is a full-reference video quality metric, whereas TQM is a no-reference quality metric. Another contribution of this paper is the study of the impact of different resolutions and frame rates on user experience and how accurately PEVQ and TQM measure varying frame rates.