No-reference Quality Metric Research Articles

Norm constraints pyramid for image dehazing

Dark channel prior-based methods have achieved remarkable performance for image dehazing. However, previous studies are mostly focused on the accuracy of the assumptions used in the target scenes, which incurs color distortion and brightness reduction when the models are used for real-world hazy images. We propose a norm constraints pyramid framework to improve the generalization performance of dehazing. First, a local color adaptive correction approach is devised to ascertain whether there is any color bias and thereafter rectify it automatically. Furthermore, multiple norm constraint methods are developed to improve the transmission and accomplish the first image removal. Finally, a non-linear enhancement method is created via this restriction that precisely modifies the brightness of an image. Through extensive experiments, we demonstrate that our framework establishes the new state-of- the-art performance for real-world dehazing, in terms of visual quality assessed by no-reference quality metrics as well as subjective evaluation and downstream task performance indicator.

Towards adversarial robustness verification of no-reference image- and video-quality metrics

In this paper, we propose a new method of analysing the stability of modern deep image- and video-quality metrics to different adversarial attacks. The stability analysis of quality metrics is becoming important because nowadays the majority of metrics employ neural networks. Unlike traditional quality metrics based on nature scene statistics or other hand-crafter features, learning-based methods are more vulnerable to adversarial attacks. The usage of such unstable metrics in benchmarks may lead to being exploited by the developers of image and video processing algorithms to achieve higher positions in leaderboards. The majority of known adversarial attacks on images designed for computer vision tasks are not fast enough to be used within real-time video processing algorithms. We propose four fast attacks on metrics suitable for real-life scenarios. The proposed methods are based on creating perturbations that increase metrics scores and can be applied frame-by-frame to attack videos. We analyse the stability of seven widely used no-reference image- and video-quality metrics to proposed attacks. The results showed that only three metrics are stable against our real-life attacks. This research yields insights to further aid in designing stable neural-network-based no-reference quality metrics. Proposed attacks can serve as an additional verification of metrics’ reliability.

No-Reference Quality Metrics for Image Decolorization

No-Reference Quality Metrics for Image Decolorization

A Deep Learning-Based No-Reference Quality Metric for High-Definition Images Compressed With HEVC

An accurate no-reference image quality assessment metric for compression artifacts is essential for the broadcasting and streaming industries. Although we have witnessed impressive advances in the capturing, delivery and display technologies, we have not managed to match them with an accurate and perceptual based no-reference image quality metric. In this paper, we propose a unique perceptual based no-reference quality metric for compressed HD frames/images that is based on the DenseNet network architecture. We focus on the effect HEVC (High Efficiency Video Coding) compression artifacts have on the visual quality of a broadcasted and streamed video, as this is a requirement of immense importance for these industries. We chose the Video Multi-Method Assessment Fusion (VMAF) metric as our base measure to map visual quality of HEVC compression artifacts to five visual quality levels. The original VMAF classification was changed to reflect High Definition (HD) resolution images. We trained a DenseNet network to classify compressed images into five visual categories using the dataset generated by the modified VMAF. DenseNet was chosen for its ability to process HD images. Our evaluations have shown that our no-reference metric achieves an impressive average accuracy of 94.13% in classifying the visual quality of compressed images.

No-Reference Light Field Image Quality Assessment Exploiting Saliency

In the near future, the broadcasting scenario will be characterized by immersive content. One of the systems for capturing the 3D content of a scene is the Light Field imaging. The huge amount of data and the specific transmission scenario impose strong constraints on services and applications. Among others, the evaluation of the quality of the received media cannot rely on the original signal but should be based only on the received data. In this direction, we propose a no-reference quality metric for light field images which is based on spatial and angular characteristics. In more details, the estimated saliency and cyclopean maps of light field images are exploited to extract the spatial features. The angular consistency features are, instead, measured with the use of the Global Luminance Distribution knowledge and the Weighted Local Binary Patterns operator on Epipolar Plane Images. The effectiveness of the proposed metric is assessed by comparing its performance with state-of-the-art quality metrics using 4 datasets: SMART, Win5-LID, VALID 10-bit, and VALID 8-bit. Furthermore, the performance is analyzed in cross-datasets, with different distortions, and for different saliency maps. The achieved results show that the performance of the proposed model outperforms state-of-the-art approaches and perform well for different distortion types and with various saliency models.

Toward a No-Reference Quality Metric for Camera-Captured Images.

Existing no-reference (NR) image quality assessment (IQA) metrics are still not convincing for evaluating the quality of the camera-captured images. Toward tackling this issue, we, in this article, establish a novel NR quality metric for quantifying the quality of the camera-captured images reliably. Since the image quality is hierarchically perceived from the low-level preliminary visual perception to the high-level semantic comprehension in the human brain, in our proposed metric, we characterize the image quality by exploiting both the low-level image properties and the high-level semantics of the image. Specifically, we extract a series of low-level features to characterize the fundamental image properties, including the brightness, saturation, contrast, noiseness, sharpness, and naturalness, which are highly indicative of the camera-captured image quality. Correspondingly, the high-level features are designed to characterize the semantics of the image. The low-level and high-level perceptual features play complementary roles in measuring the image quality. To infer the image quality, we employ the support vector regression (SVR) to map all the informative features to a single quality score. Thorough tests conducted on two standard camera-captured image databases demonstrate the effectiveness of the proposed quality metric in assessing the image quality and its superiority over the state-of-the-art NR quality metrics. The source code of the proposed metric for camera-captured images is released at https://github.com/YT2015?tab=repositories.

Reduced-Reference Quality Assessment of Point Clouds via Content-Oriented Saliency Projection

Many dense 3D point clouds have been exploited to represent visual objects instead of traditional images or videos. To evaluate the perceptual quality of various point clouds, in this letter, we propose a novel and efficient Reduced-Reference quality metric for point clouds, which is based on Content-oriented sAliency Projection (RR-CAP). Specifically, we make the first attempt to simplify reference and distorted point clouds into projected saliency maps with a downsampling operation. Through this process, we tackle the issue of transmitting large-volume original point clouds to user-ends for quality assessment. Then, motivated by the characteristics of the human visual system (HVS), the objective quality scores of distorted point clouds are produced by combining content-oriented similarity and statistical correlation measurements. Finally, extensive experiments are conducted on SJTU-PCQA and WPC databases. The experimental results demonstrate that our proposed algorithm outperforms existing reduced-reference and no-reference quality metrics, and significantly reduces the performance gap between state-of-the-art full-reference quality assessment methods. In addition, we show the performance variation of each proposed technical component by ablation tests.

An underwater-imaging-model-inspired no-reference quality metric for images in multi-colored environments

The color of an underwater target is different under different lighting conditions and underwater environments. To date, no quality metric has been proposed for images of underwater targets in multi-colored environments. In this paper, we proposed a no-reference quality metric for images of underwater targets in multi-colored environments (QMICE). This metric is a weighted combination of colorfulness index, contrast index and sharpness index. The colorfulness index is used to measure the color loss caused by absorption. The contrast index and sharpness index are used to measure the blurring caused by scattering. The weighted coefficients of the three indexes are calculated by multiple linear regression (MLR). For the contrast index and sharpness index, we proposed a grayscale conversion method that can adaptively adjust the coefficients of red, green, and blue (RGB) values to enhance their generalization ability under multi-colored environments. During the calculation of weighted coefficients, the quality scores based on turbidity are regarded as the true quality scores. It is more reliable than subjective assessment scores. The experimental results show that compared with the leading underwater image quality metrics available in the literature, the proposed metric has the best correlation between the metric predictions and the true quality scores. More importantly, QMICE can also be used to process underwater images in real time and evaluate the performance of underwater image restoration algorithms.

A practical residual block-based no-reference quality metric for neutron radiographic images

Objective image quality metric is a challenging problem for neutron radiography due to the lack of reference images. The existing no-reference image quality metric (NR-IQM) method designed for neutron radiographic images (NRIs) still exists two main shortcomings. Firstly, the limited L/D of the collimator would inevitably lead to geometric blur in the NRIs. However, the special distortion type of mixed Poisson–Gaussian noise is merely considered but ignores the blur distortion. Secondly, the distortion levels of Poisson noise added to an image cannot be adjusted. In order to address these two issues, a more practical objective NR-IQM method is proposed to predict the perceptual quality of NRIs in this paper. Experimental results demonstrate that the proposed method shows good consistency with the quality of authentic NRIs and also performs well in reflecting the tiny quality difference of enhanced NRIs, thus demonstrating its application potential in neutron radiography.

Subjective and no-reference quality metric of domain independent images and videos

There has been an unprecedented explosion in creation and consumption of high definition videos and images. This has led to the development and adoption of several systems which manipulate graphic content. Most of these systems require a robust, no-reference, domain independent, and subjective quality metric. The state-of-the-art algorithms aimed at no-reference quality assessment of images and videos employ neural networks that are content domain dependent and require application specific training. Further, present systems are trained to estimate quality metric in accordance to objective metrics such as Peak Signal to Noise Ratio or Mean Squared Error. We propose a convolutional neural network architecture that leverages a variable length long short term memory. The model will also make use of the Laplacian as a fourth layer in the input to ensure content domain independence in quality estimation. The system has been trained on the images of KonIQ-10K dataset and tested on KonViD-1k video dataset and LIVE video dataset. It has also been tested on 1000 frames and videos from the VIRAT dataset and images from BSD300 and Set5 datasets with artificial blur and distortion applied. The quality estimations were compared to the mean opinion square given by five human viewers and resulted in a 11% root mean squared error. Thus the proposed model provides a subjective quality estimation of graphic content without any dependence on the domain of the content.

Analyzing the effect of adding temporal features to an autoencoder-based video quality model

According to Cisco, most Internet traffic is currently comprised of videos. Therefore, developing a quality assessment method for assuring that those videos are received and displayed with quality at the user side is an important and challenging task. As a consequence, over the last decades, several no-reference video quality metrics have been proposed with the goal of blindly predicting (with no access to the original signal) the quality of videos in streaming applications. One of such metrics is NAVE, whose architecture includes an auto-encoder module that produces a compact set of visual features with a higher descriptive capacity. Nevertheless, the visual features in NAVE do not include descriptive temporal features that are sensitive to temporal degradation. In this work, we analyze the effect on accuracy performance of using a new type of temporal features, based on natural scene statistics. This approach has the goal of making the tested video quality metric more generic, i.e. sensitive to both spatial and temporal distortions and therefore adequate for video streaming applications.

Low contrast detection factor based contrast enhancement and restoration for underwater images

Marine ecosystem is the largest of earth’s aquatic ecosystems. It includes salt marshes, coral reefs, deep sea, sea floor, etc. To learn deep about the activities taking place inside, underwater imaging is a tool. But these images lack in contrast and brightness leading to the lack of information in the ocean activities. To enhance such low contrast underwater images, Low Contrast Detection Factor (LCDF) is proposed in this study. It uses the value, saturation and hue to enhance the low contrast regions and to restore the color. Quality assessment is done to substantiate the proposed algorithm. It is found that the entropy gives an average of 7.3. No-reference Quality Metrics such as Natural Image Quality Evaluator and Blind/ Referenceless Image Spatial Quality Evaluator shows an average value of 3.6 and 22.5, respectively. The blur metrics shows a value of 0.21. The quality metrics indicates that the naturalness of the underwater image is maintained while the contrast of the underwater image has increased.

Re-Visiting Discriminator for Blind Free-Viewpoint Image Quality Assessment

Accurate measurement of perceptual quality is important for various immersive multimedia, which demand real-time quality control or quality-based bench-marking for relevant algorithms. For instance, virtual views rendering in Free-Viewpoint (FV) navigation scenarios is a typical case that introduces challenging distortions, particularly the ones around dis-occluded regions. Existing quality metrics, most of which are targeting for impairments caused by compression or network condition, fail to quantify such non-uniform structure-related distortions. Moreover, the lack of quality databases for such distortions makes it even more challenging to develop robust quality metrics. In this work, a Generative Adversarial Networks based No-Reference (NR) quality Metric, namely GANs-NRM, is proposed. We first present an approach to create masks mimicking dis-occlusions/textureless regions, which is applicable on large-scale 2D image databases publicly available in the computer vision domain. Using these synthetic data, we then train a GANs-based context renderer with the capability of rendering those masked regions. Since the naturalness of the rendered dis-occluded regions strongly relates to the perceptual quality, we assume that the discriminator of the trained GANs has an intrinsic ability for quality assessment. We thus use the features extracted from the discriminator to learn a Bag-of-Distortion-Word (BDW) codebook. We show that a quality predictor can be then well trained using only a small amount of subjective quality data for the FV views rendering. Moreover, in the proposed framework, the discriminator is also adapted as a distortion-detector to locate possible distorted regions. According to the experimental results, the proposed model outperforms significantly the state-of-the-art quality metrics. The corresponding context renderer also shows appealing visualized results over other rendering algorithms.

Embedding in medical images with contrast enhancement and tamper detection capability

This paper presents an efficient data hiding technique capable of providing improved visual quality of watermarked images, besides having the ability to detect the tamper, if any. It is a spatial domain approach in which major emphasis is on improving the visual quality rather than increasing the PSNR or the embedding capacity. The medical images have been divided into Region of Interest (ROI) and Non-Region of Interest (NROI). Bringing out details that lie within the low dynamic range is very important in medical images for effective diagnosis. ROI being diagnostically critical region is enhanced using contrast stretching and subsequently, data is reversibly embedded into the peak bins of ROI. Only those peak bins are employed for reversible data embedding that have an adjacent empty bin to overcome the problem of overflow and underflow. In NROI, the uniform intensity and redundant information region, Least Significant Bit (LSB) embedding is employed for increasing the payload. For tamper detection, a fragile watermark has been embedded in the ROI. To evaluate the scheme various parameters like peak signal to noise ratio (PSNR), No-Reference Quality Metric for contrast-distorted images (NR-CDIQA) and Structural Similarity Index Matrix (SSIM) have been calculated. The experimental results show a remarkable increase in visual quality compared to state-of-art.

Novel Deep Level Image State Ensemble Enhancement Method for M87 Imaging

Standard spatial domain filters fail to adequately denoise and enhance the contrast of an image. These filters have drawbacks like oversmoothing, diminished texture, and lack of generative capabilities. This paper proposes a new method of image reconstruction, Image State Ensemble Enhancement (ISEE), based on our previous work, Image State Ensemble Decomposition (ISED). Deep level ISEE and ISED have been developed to produce a class of filters that can address these issues. Full-reference and no-reference quality metrics are used to assess the image, and the full reference metrics showed a marked improvement, while the no-reference metrics were often better than the test image. The test image was taken from the Spitzer Space Telescope (SST), and ISEE reconstruction yielded improved structural detail over that of ISED and the original test image. Glare and noise were reduced in a narrow bandwidth, which led to the discovery of a vortex-shaped structure and an outburst in M87′s dusty infrared core. The vortex is located over M87′s visible core and black hole. This is verified with an SST and Hubble Space Telescope (HST) overlay, ISEE processed image. A counter-jet channel was also discovered, and it appears to be the path of the unobservable superluminal counter-jet.

No-reference quality metric for contrast-distorted image based on gradient domain and HSV space

Image quality assessment (IQA) plays an important role in digital image forensics. Due to the occurrence of contrast distortion during image acquisition and manipulation, IQA for contrast is a major issue. And it is vital for benchmarking and optimizing the image tampering detection and contrast-enhancement algorithms. In this paper, a new no-reference/blind image quality assessment (IQA) metric is proposed for evaluating image contrast. This research seeks for the inter-relationship between contrast distortion and visual perception quality. The comprehensive quality metric is obtained by combining local binary pattern (LBP) descriptor on gradient domain with color moment on HSV color space. And a prediction model is trained with support vector regression (SVR). Extensive analysis and cross validation are performed on four contrast relevant image databases, which validates the superiority of our proposed blind technique over state-of-the-art no-reference IQA methods.

Novel Image State Ensemble Decomposition Method for M87 Imaging

This paper proposes a new method of image decomposition with a filtering capability. The image state ensemble decomposition (ISED) method has generative capabilities that work by removing a discrete ensemble of quanta from an image to provide a range of filters and images for a single red, green, and blue (RGB) input image. This method provides an image enhancement because ISED is a spatial domain filter that transforms or eliminates image regions that may have detrimental effects, such as noise, glare, and image artifacts, and it also improves the aesthetics of the image. ISED was used to generate 126 images from two tagged image file (TIF) images of M87 taken by the Spitzer Space Telescope. Analysis of the images used various full and no-reference quality metrics as well as histograms and color clouds. In most instances, the no-reference quality metrics of the generated images were shown to be superior to those of the two original images. Select ISED images yielded previously unknown galactic structures, reduced glare, and enhanced contrast, with good overall performance.

Fast Blind Quality Assessment of DIBR-Synthesized Video Based on High-High Wavelet Subband.

Free-viewpoint video, as the development direction of the next-generation video technologies, uses the depth-image-based rendering (DIBR) technique for the synthesis of video sequences at viewpoints, where real captured videos are missing. As reference videos at multiple viewpoints are not available, a blind reliable real-time quality metric of the synthesized video is needed. Although no-reference quality metrics dedicated to synthesized views successfully evaluate synthesized images, they are not that effective when evaluating synthesized video due to additional temporal flicker distortion typical only for video. In this paper, a new fast no-reference quality metric of synthesized video with synthesis distortions is proposed. It is guided by the fact that the DIBR-synthesized images are characterized by increased high frequency content. The metric is designed under the assumption that the perceived quality of DIBR-synthesized video can be estimated by quantifying the selected areas in the high-high wavelet subband. The threshold is used to select the most important distortion sensitive regions. The proposed No-Reference Morphological Wavelet with Threshold (NR_MWT) metric is computationally extremely efficient, comparable to PSNR, as the morphological wavelet transformation uses very short filters and only integer arithmetic. It is completely blind, without using machine learning techniques. Tested on the publicly available dataset of synthesized video sequences characterized by synthesis distortions, the metric achieves better performances and higher computational efficiency than the state-of-the-art metrics dedicated to DIBR-synthesized images and videos.

No-reference quality assessment for contrast-distorted images based on multifaceted statistical representation of structure

In many real-world applications, images are prone to be degraded by contrast distortions during image acquisition. Quality assessment for contrast-distorted images is vital for benchmarking and optimizing the contrast-enhancement algorithms. To this end, this paper proposes a no-reference quality metric for contrast-distorted images based on Multifaceted Statistical representation of Structure (MSS). The “Multifaceted” has two meanings, namely (1) not only the luminance information, but also the chromatic information is used for structure representation. This is inspired by the fact that the chromatic information on the one hand affects the perception of image quality as well, and on the other hand it changes along with the contrast distortions. Therefore, the chromatic information should be integrated with the luminance information for quality assessment of contrast-distorted images, a fact most existing quality metrics overlook; (2) regarding structure representation, three aspects, i.e. spatial intensity, spatial distribution, and orientation of structures are calculated, which is enlightened by the fact that the human visual system (HVS) is sensitive to the three aspects of structures. Specifically, the image is first transformed from RGB to the S-CIELAB color space to obtain a representation that is more consistent with the characteristics of the HVS, as well as to separate the chromatic information from the luminance. Then the statistical structural features are extracted from both luminance and chromatic channels. Finally, the back propagation (BP) neural network is adopted to train a quality prediction model. Experimental results conducted on four public contrast-distorted image databases demonstrate the superiority of the proposed method to the relevant state-of-the-arts.

No-reference quality metric for high dynamic range imaging system based on curvature analysis in tensor domain

No-reference quality metric for high dynamic range imaging system based on curvature analysis in tensor domain