No-reference Quality Assessment Research Articles

Wave-PCT: Wavelet point cloud transformer for point cloud quality assessment

Point cloud representation for real-world objects has seen a surge in interest recently, finding widespread applications in augmented reality, virtual reality, and autonomous driving. However, the process of acquiring, compressing, transmitting, and processing point cloud data often introduces unwanted distortions, highlighting the need for effective no-reference quality assessment methods for distorted point clouds. In this paper, we propose a Wavelet Point Cloud Transformer (Wave-PCT) method to assess the quality of point clouds without reference data. We start by performing patch sampling on a source point cloud, generating an extensive set of patches from the point cloud. These point cloud patches and their 2D projection images are processed using wavelets to produce multiscale local spectral features; moreover, we also extract global features of these patches using Pointnet++. Finally, a self-attention module is performed to fuse these local and global features, resulting in a distortion-sensitive quality score metric. We evaluate the Wave-PCT method on several point cloud quality assessment datasets using various metrics, and the experimental results indicate that the proposed framework exhibits outstanding and reliable performance when compared to various point cloud quality assessment methods.

Blind Quality Assessment of Dense 3D Point Clouds with Structure Guided Resampling

Objective quality assessment of three-dimensional (3D) point clouds is essential for the development of immersive multimedia systems in real-world applications. Despite the success of perceptual quality evaluation for 2D images and videos, blind/no-reference metrics are still scarce for 3D point clouds with large-scale irregularly distributed 3D points. Therefore, in this article, we propose an objective point cloud quality index with Structure Guided Resampling (SGR) to automatically evaluate the perceptually visual quality of dense 3D point clouds. The proposed SGR is a general-purpose blind quality assessment method without the assistance of any reference information. Specifically, considering that the human visual system is highly sensitive to structure information, we first exploit the unique normal vectors of point clouds to execute regional pre-processing that consists of keypoint resampling and local region construction. Then, we extract three groups of quality-related features, including (1) geometry density features, (2) color naturalness features, and (3) angular consistency features. Both the cognitive peculiarities of the human brain and naturalness regularity are involved in the designed quality-aware features that can capture the most vital aspects of distorted 3D point clouds. Extensive experiments on several publicly available subjective point cloud quality databases validate that our proposed SGR can compete with state-of-the-art full-reference, reduced-reference, and no-reference quality assessment algorithms.

NeRF-NQA: No-Reference Quality Assessment for Scenes Generated by NeRF and Neural View Synthesis Methods.

Neural View Synthesis (NVS) has demonstrated efficacy in generating high-fidelity dense viewpoint videos using a image set with sparse views. However, existing quality assessment methods like PSNR, SSIM, and LPIPS are not tailored for the scenes with dense viewpoints synthesized by NVS and NeRF variants, thus, they often fall short in capturing the perceptual quality, including spatial and angular aspects of NVS-synthesized scenes. Furthermore, the lack of dense ground truth views makes the full reference quality assessment on NVS-synthesized scenes challenging. For instance, datasets such as LLFF provide only sparse images, insufficient for complete full-reference assessments. To address the issues above, we propose NeRF-NQA, the first no-reference quality assessment method for densely-observed scenes synthesized from the NVS and NeRF variants. NeRF-NQA employs a joint quality assessment strategy, integrating both viewwise and pointwise approaches, to evaluate the quality of NVS-generated scenes. The viewwise approach assesses the spatial quality of each individual synthesized view and the overall inter-views consistency, while the pointwise approach focuses on the angular qualities of scene surface points and their compound inter-point quality. Extensive evaluations are conducted to compare NeRF-NQA with 23 mainstream visual quality assessment methods (from fields of image, video, and light-field assessment). The results demonstrate NeRF-NQA outperforms the existing assessment methods significantly and it shows substantial superiority on assessing NVS-synthesized scenes without references. An implementation of this paper are available at https://github.com/VincentQQu/NeRF-NQA.

A No-Reference Quality Assessment Method for Hyperspectral Sharpened Images via Benford’s Law

In recent years, hyperspectral (HS) sharpening technology has received high attention and HS sharpened images have been widely applied. However, the quality assessment of HS sharpened images has not been well addressed and is still limited to the use of full-reference quality evaluation. In this paper, a novel no-reference quality assessment method based on Benford’s law for HS sharpened images is proposed. Without a reference image, the proposed method detects fusion distortion by performing first digit distribution on three quality perception features in HS sharpened images, using the standard Benford’s law as a benchmark. The experiment evaluates 10 HS fusion methods on three HS datasets and selects four full-reference metrics and four no-reference metrics to compare with the proposed method. The experimental results demonstrate the superior performance of the proposed method.

MFE-Net: A Multi-Layer Feature Extraction Network for No-Reference Quality Assessment of 3-D Point Clouds

MFE-Net: A Multi-Layer Feature Extraction Network for No-Reference Quality Assessment of 3-D Point Clouds

Eye Scanpath Prediction-Based No-Reference Quality Assessment of Omnidirectional Images

Eye Scanpath Prediction-Based No-Reference Quality Assessment of Omnidirectional Images

Feature rectification and enhancement for no-reference image quality assessment

The assessment of image is a critical task in computer vision, particularly in a no-referenced context, which presents numerous challenges. Though Deep neural networks have exhibited excellent performance on no-reference quality assessment (NR-IQA), the distortion-aware NR-IQA methods are limited to specific distortion type and struggle to handle realistic distortion scenarios. In this work, we propose a feature rectification and enhancement convolutional neural network namely FREIQA for NR-IQA. Our approach targets to fully utilize the multi-stage semantic features to produce a well fused and rectified score vector for effective image quality regression. Specifically, the multi-stage semantic features extracted by a pre-trained backbone would be rectified with a multi-level channel attention module before integrating to a global score vector. The fused score vector then will be gradually optimized through the perception feature provided by the multi-stage semantic feature, and finally sent to quality regression network to obtain the image quality score. Experimental results on public benchmarks including LIVE, TID2013, CSIQ, LIVEC, KADID-10k, KonIQ-10k and Waterloo exploration database show that our FREIQA achieve the state-of-the-art performance.

No-Reference Quality Assessment of Extended Target Adaptive Optics Images Using Deep Neural Network.

This paper proposes a supervised deep neural network model for accomplishing highly efficient image quality assessment (IQA) for adaptive optics (AO) images. The AO imaging systems based on ground-based telescopes suffer from residual atmospheric turbulence, tracking error, and photoelectric noise, which can lead to varying degrees of image degradation, making image processing challenging. Currently, assessing the quality and selecting frames of AO images depend on either traditional IQA methods or manual evaluation by experienced researchers, neither of which is entirely reliable. The proposed network is trained by leveraging the similarity between the point spread function (PSF) of the degraded image and the Airy spot as its supervised training instead of relying on the features of the degraded image itself as a quality label. This approach is reflective of the relationship between the degradation factors of the AO imaging process and the image quality and does not require the analysis of the image's specific feature or degradation model. The simulation test data show a Spearman's rank correlation coefficient (SRCC) of 0.97, and our method was also validated using actual acquired AO images. The experimental results indicate that our method is more accurate in evaluating AO image quality compared to traditional IQA methods.

No-reference quality assessment of underwater image enhancement

Due to the attenuation and scattering of light in the water medium, real-world underwater images usually suffer from diverse quality defects, such as color casts, low contrast, and reduced visibility, etc. These quality defects accordingly cause adverse effects on underwater images in practical applications. To tackle the problem, many underwater image enhancement (UIE) techniques have been proposed for improving the quality of raw underwater images, showing heterogeneous performances regarding the enhanced results. Therefore, designing an objective quality metric that can effectively predict the visual quality of enhanced underwater images is desirable. In this paper, we propose a highly efficient yet accurate no-reference quality assessment method to evaluate different UIE results by analyzing the statistics of underwater images. Specifically, we first extract quality-aware features in terms of three key aspects: (1) colorfulness; (2) contrast; (3) visibility. Then, a quality regression model is trained to map the extracted features to subjective scores for enhanced underwater images. Given a testing underwater image, we also first extract its corresponding quality-aware feature vector and feed it into the trained quality regression model for quality prediction. We conduct extensive experiments on two databases to demonstrate the superiority of our proposed approach. The code is available at https://github.com/yia-yuese/NR-UIQA.

No-reference quality assessment for depth image based rendering-synthesized images based on multi-feature fusion

No-reference quality assessment for depth image based rendering-synthesized images based on multi-feature fusion

A no-reference quality assessment metric for dynamic 3D digital human

Dynamic Digital Humans (DDHs) refer to 3D digital models that are animated using predefined motions. However, these models are susceptible to noise and shift distortions during the generation process, as well as compression distortions during transmission and communication. It is crucial to evaluate the perceptual quality of DDHs in order to address these challenges. Typically, DDHs are presented as 2D rendered animation videos, making it reasonable to adapt video quality assessment (VQA) methods for DDH quality assessment (DDH-QA) tasks. Nevertheless, VQA methods heavily rely on viewpoints and exhibit limited sensitivity to geometry-based distortions. Consequently, this paper introduces a novel no-reference (NR) quality assessment metric specifically tailored for the DDH-QA challenge. In this proposed method, the geometry characteristics of DDHs are characterized by computing statistical parameters estimated from the distributions of their geometry attributes. Additionally, spatial and temporal features are extracted from the rendered videos. Subsequently, all relevant features are integrated and regressed into quality values. Through comprehensive experiments conducted on the DDH-QA database, the results demonstrate that the proposed method achieves state-of-the-art performance in assessing the quality of DDHs.

A Novel No-Reference Quality Assessment Metric for Stereoscopic Images with Consideration of Comprehensive 3D Quality Information.

Recently, stereoscopic image quality assessment has attracted a lot attention. However, compared with 2D image quality assessment, it is much more difficult to assess the quality of stereoscopic images due to the lack of understanding of 3D visual perception. This paper proposes a novel no-reference quality assessment metric for stereoscopic images using natural scene statistics with consideration of both the quality of the cyclopean image and 3D visual perceptual information (binocular fusion and binocular rivalry). In the proposed method, not only is the quality of the cyclopean image considered, but binocular rivalry and other 3D visual intrinsic properties are also exploited. Specifically, in order to improve the objective quality of the cyclopean image, features of the cyclopean images in both the spatial domain and transformed domain are extracted based on the natural scene statistics (NSS) model. Furthermore, to better comprehend intrinsic properties of the stereoscopic image, in our method, the binocular rivalry effect and other 3D visual properties are also considered in the process of feature extraction. Following adaptive feature pruning using principle component analysis, improved metric accuracy can be found in our proposed method. The experimental results show that the proposed metric can achieve a good and consistent alignment with subjective assessment of stereoscopic images in comparison with existing methods, with the highest SROCC (0.952) and PLCC (0.962) scores being acquired on the LIVE 3D database Phase I.

LFACon: Introducing Anglewise Attention to No-Reference Quality Assessment in Light Field Space.

Light field imaging can capture both the intensity information and the direction information of light rays. It naturally enables a six-degrees-of-freedom viewing experience and deep user engagement in virtual reality. Compared to 2D image assessment, light field image quality assessment (LFIQA) needs to consider not only the image quality in the spatial domain but also the quality consistency in the angular domain. However, there is a lack of metrics to effectively reflect the angular consistency and thus the angular quality of a light field image (LFI). Furthermore, the existing LFIQA metrics suffer from high computational costs due to the excessive data volume of LFIs. In this paper, we propose a novel concept of "anglewise attention" by introducing a multihead self-attention mechanism to the angular domain of an LFI. This mechanism better reflects the LFI quality. In particular, we propose three new attention kernels, including anglewise self-attention, anglewise grid attention, and anglewise central attention. These attention kernels can realize angular self-attention, extract multiangled features globally or selectively, and reduce the computational cost of feature extraction. By effectively incorporating the proposed kernels, we further propose our light field attentional convolutional neural network (LFACon) as an LFIQA metric. Our experimental results show that the proposed LFACon metric significantly outperforms the state-of-the-art LFIQA metrics. For the majority of distortion types, LFACon attains the best performance with lower complexity and less computational time.

Variance-based no-reference quality assessment of AWGN images

Variance-based no-reference quality assessment of AWGN images

No-reference quality assessment for low-light image enhancement: Subjective and objective methods

No-Reference (NR) quality assessment is a crucial approach for evaluating the quality of low-light enhanced images, as it is often difficult to acquire high-quality reference images in applications such as night-time automatic driving. However, current NR evaluation methods for low-light enhanced images often lack consideration of important characteristics such as color, structure, and naturalness. This paper proposes a novel NR quality assessment method for NR low-light enhanced images from both subjective and objective aspects. On the subjective side, we construct the Low-light Enhanced Images Subjective Dataset (LEISD) containing 2040 images with 255 different image contents. Each image was evaluated based on the Single Stimulus (SS) method by 20 subjects. On the objective side, we propose Multi-Features Reconciliation-based Quality Assessment (MFRQA) methods for low-light enhanced images by observing the low-light enhanced images. The MFRQA summarized four key feature perspectives: brightness, color, structure and naturalness, and employed the traditional machine learning model to reconcile the multi-features. Experimental results on the LEISD dataset demonstrate competitive performance and low complexity of our method compared to the representative quality metrics.

Parameter optimization for point clouds denoising based on no-reference quality assessment

Almost all point clouds denoising methods contain various parameters, which need to be set carefully to acquire desired results. In this paper, we introduce an evolutionary optimization algorithm based framework to obtain the parameter configuration of point clouds denoising methods automatically. New no-reference quality assessment metrics are proposed as objective functions to quantitatively evaluate the point clouds during the optimization process. The proposed metrics infer the quality of point clouds in terms of both smoothness and density. The ideas of manifold dimension and holes detection are combined to get the smoothness evaluation results. Simplified local outlier factor is further exploited for the density evaluation. Using public dataset and real-world scanned data, experimental results prove that the automatic tuning parameters provide a significant boost in performance compared with the manual tuning parameters. Furthermore, the results acquired by the proposed metrics achieve better or equivalent performance than the state-of-the-art metrics.

No-reference Quality Assessment for Contrast-distorted Images Based on Gray and Color-gray-difference Space

No-reference image quality assessment is a basic and challenging problem in the field of image processing. Among them, contrast distortion has a great impact on the perception of image quality. However, there are relatively few studies on no-reference quality assessment of contrast-distorted images. This article proposes a no-reference quality assessment algorithm for contrast-distorted images based on gray and color-gray-difference (CGD) space. In terms of gray space, we consider the local and global aspects, and use the distribution characteristics of the grayscale histogram to represent global features, while local features are described by the fusion of Local Binary Pattern (LBP) operator and gradient. In terms of CGD space, we first randomly extract patches from the entire image and then extract appropriate quality perception features in the patch’s CGD histogram. Finally, the AdaBoosting back propagation (BP) neural network is used to train the prediction model to predict the quality of the contrast-distorted image. Extensive analysis and cross-validation are carried out on five contrast-related image databases, and the experimental results have proved the superiority of this method compared with recent related algorithms.

(Retracted) Simulating human visual perception behaviors for video quality assessment

Multimedia video quality evaluation system plays an important role in modern Internet applications, such as WebRTC-based video conference. However, video quality may be affected during transmission because of noise or network congestion. Constructing an end-to-end intelligent video transmission monitoring system is of great significance. We design a system based on no-reference quality assessment, which can be used to evaluate the video-based object quality evaluation. More specifically, we first leverage Gabor filter for video segmentation. Then we conduct image smoothness estimation to measure the correlation between two adjacent frames. Subsequently, we consider two states of video distortion: compression and video transmission. We model these two states, respectively. We compare our VQA algorithm with a set of well-known NR-IQA. The reported VQA accuracies have clearly shown the superiority of our proposed video quality predictor.

Dynamically attentive viewport sequence for no-reference quality assessment of omnidirectional images.

Omnidirectional images (ODIs) have drawn great attention in virtual reality (VR) due to the capability of providing an immersive experience to users. However, ODIs are usually subject to various quality degradations during different processing stages. Thus, the quality assessment of ODIs is of critical importance to the community of VR. The quality assessment of ODIs is quite different from that of traditional 2D images. Existing IQA methods focus on extracting features from spherical scenes while ignoring the characteristics of actual viewing behavior of humans in continuously browsing an ODI through HMD and failing to characterize the temporal dynamics of the browsing process in terms of the temporal order of viewports. In this article, we resort to the law of gravity to detect the dynamically attentive regions of humans when viewing ODIs. In this article, we propose a novel no-reference (NR) ODI quality evaluation method by making efforts on two aspects including the construction of Dynamically Attentive Viewport Sequence (DAVS) from ODIs and the extraction of Quality-Aware Features (QAFs) from DAVS. The construction of DAVS aims to build a sequence of viewports that are likely to be explored by viewers based on the prediction of visual scanpath when viewers are freely exploring the ODI within the exploration time via HMD. A DAVS that contains only global motion can then be obtained by sampling a series of viewports from the ODI along the predicted visual scanpath. The subsequent quality evaluation of ODIs is performed merely based on the DAVS. The extraction of QAFs aims to obtain effective feature representations that are highly discriminative in terms of perceived distortion and visual quality. Finally, we can adopt a regression model to map the extracted QAFs to a single predicted quality score. Experimental results on two datasets demonstrate that the proposed method is able to deliver state-of-the-art performance.

No-Reference Quality Assessment for 3D Colored Point Cloud and Mesh Models

To improve the viewer's Quality of Experience (QoE) and optimize computer graphics applications, 3D model quality assessment (3D-QA) has become an important task in the multimedia area. Point cloud and mesh are the two most widely used digital representation formats of 3D models, the visual quality of which is quite sensitive to lossy operations like simplification and compression. Therefore, many related studies such as point cloud quality assessment (PCQA) and mesh quality assessment (MQA) have been carried out to measure the visual quality degradations of 3D models. However, a large part of previous studies utilize full-reference (FR) metrics, which indicates they can not predict the quality level with the absence of the reference 3D model. Furthermore, few 3D-QA metrics consider color information, which significantly restricts their effectiveness and scope of application. In this paper, we propose a no-reference (NR) quality assessment metric for colored 3D models represented by both point cloud and mesh. First, we project the 3D models from 3D space into quality-related geometry and color feature domains. Then, the 3D natural scene statistics (3D-NSS) and entropy are utilized to extract quality-aware features. Finally, machine learning is employed to regress the quality-aware features into visual quality scores. Our method is validated on the colored point cloud quality assessment database (SJTU-PCQA), the Waterloo point cloud assessment database (WPC), and the colored mesh quality assessment database (CMDM). The experimental results show that the proposed method outperforms most compared NR 3D-QA metrics with competitive computational resources and greatly reduces the performance gap with the state-of-the-art FR 3D-QA metrics. The code of the proposed model is publicly available now to facilitate further research.